From dotanb at dev.mellanox.co.il Wed Aug 1 00:35:05 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 01 Aug 2007 10:35:05 +0300 Subject: [ofa-general] Re: [PATCH] ib/mad: fix address handle leak in mad_rmpp In-Reply-To: References: <000001c7d3bf$aa5e0090$ff0da8c0@amr.corp.intel.com> Message-ID: <46B037A9.3050401@dev.mellanox.co.il> Roland Dreier wrote: > > Roland, this fix would be for 2.6.23. > > OK, I'll wait for Dotan's ACK. > > You have a green light for the patch: it fixes the address handle leak completely. thanks for the quick fix Dotan From krkumar2 at in.ibm.com Wed Aug 1 00:48:32 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 1 Aug 2007 13:18:32 +0530 Subject: [ofa-general] Processing multiple WR's in MTHCA (other drivers). Message-ID: Hi, If I call post_send with multiple WR's (as a list of 'next' linked WR's) from IPoIB to the driver, is it guaranteed that those WR's will go out on the wire in the same sequence ? Will the send sequence be : WR[0], WR[1], .... WR[n]; or this cannot be guaranteed by the driver after it issues the doorbell ? The reason to ask is that when I modify the networking + ipoib code to send multiple skbs, I see huge number of retransmissions (original code: 250, modified code: 5000 or more) as though skbs were sent out of order, and any amount of staring at the code is not helping. The code to send multiple WR's is (hopefully readable): /* * Normally start_index is 0 to indicate send from WR[0], but if one post_send * fails, start_index is set to the index of the first untried WR by ipoib_send(), and * calls post_send() again. I have also not seen any failures from provider and a * subsequent retry that causes this retransmission to happen. */ static inline int post_send(struct ipoib_dev_priv *priv, u32 qpn, int start_index, int num_skbs, struct ib_send_wr **bad_wr) { int ret; struct ib_send_wr *last_wr, *next_wr; last_wr = &priv->tx_wr[start_index + num_skbs - 1]; /* Set Completion Notification for last WR */ last_wr->send_flags = IB_SEND_SIGNALED; /* Terminate the last WR */ next_wr = last_wr->next; last_wr->next = NULL; /* Send all the WR's in one doorbell */ ret = ib_post_send(priv->qp, &priv->tx_wr[start_index], bad_wr); /* Restore send_flags & WR chain */ last_wr->send_flags = 0; last_wr->next = next_wr; return ret; } (I am also getting a completion with multiple WR's in each completion handler, which is good). Thanks, - KK From vlad at lists.openfabrics.org Wed Aug 1 01:38:06 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 1 Aug 2007 01:38:06 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070801-0100 daily build status Message-ID: <20070801083806.BCDABE608F2@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From mst at dev.mellanox.co.il Wed Aug 1 02:17:45 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:17:45 +0300 Subject: [ofa-general] Re: Processing multiple WR's in MTHCA (other drivers). In-Reply-To: References: Message-ID: <20070801091745.GA29259@mellanox.co.il> Quoting Krishna Kumar2 : Subject: Processing multiple WR's in MTHCA (other drivers). > If I call post_send with multiple WR's (as a list of 'next' linked WR's) from > IPoIB to the driver, is it guaranteed that those WR's will go out on the wire in > the same sequence? See chapter 9.5 transaction ordering: C9-25: A requester shall transmit request messages in the order that the Work Queue Elements (WQEs) were posted. -- MST From mst at dev.mellanox.co.il Wed Aug 1 02:28:09 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:28:09 +0300 Subject: [ofa-general] [PATCH 0 of 3 for-2.6.24] mlx4: fast memory registration Message-ID: <20070801092809.GB29259@mellanox.co.il> From: Jack Morgenstein This patch-set enables fast memory registration in mlx4. Note that both regular MR and FMR are made to write MTTs directly (in the same way that we do for arbel). This makes it possible to use common code for MRs and FMRs. -- MST From mst at dev.mellanox.co.il Wed Aug 1 02:28:20 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:28:20 +0300 Subject: [ofa-general] [PATCH 1 of 3 for-2.6.24] mlx4: support coherent memory for icm Message-ID: <20070801092819.GC29259@mellanox.co.il> From: Jack Morgenstein Enable allocation of coherent memory for icm mapping. This will allow writing MPT entries for MR's via the SW_HW_MPT command, and direct driver write of MPT entries for FMR remapping without needing to flush, and without cacheline boundary concerns. Signed-off-by: Jack Morgenstein Signed-off-by: Michael S. Tsirkin --- Index: connectx/drivers/net/mlx4/mlx4.h =================================================================== --- connectx.orig/drivers/net/mlx4/mlx4.h 2007-07-19 13:11:00.073836000 +0300 +++ connectx/drivers/net/mlx4/mlx4.h 2007-07-19 13:11:49.184318000 +0300 @@ -132,6 +132,7 @@ struct mlx4_icm_table { int num_obj; int obj_size; int lowmem; + int coherent; struct mutex mutex; struct mlx4_icm **icm; }; Index: connectx/drivers/net/mlx4/icm.h =================================================================== --- connectx.orig/drivers/net/mlx4/icm.h 2007-07-19 13:11:00.080833000 +0300 +++ connectx/drivers/net/mlx4/icm.h 2007-07-19 13:11:49.188318000 +0300 @@ -37,6 +37,7 @@ #include #include #include +#include #define MLX4_ICM_CHUNK_LEN \ ((256 - sizeof (struct list_head) - 2 * sizeof (int)) / \ @@ -67,8 +68,9 @@ struct mlx4_icm_iter { struct mlx4_dev; -struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages, gfp_t gfp_mask); -void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm); +struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages, gfp_t gfp_mask, + int coherent); +void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent); int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); @@ -78,7 +80,7 @@ void mlx4_table_put_range(struct mlx4_de int start, int end); int mlx4_init_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table, u64 virt, int obj_size, int nobj, int reserved, - int use_lowmem); + int use_lowmem, int use_coherent); void mlx4_cleanup_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table); int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); Index: connectx/drivers/net/mlx4/icm.c =================================================================== --- connectx.orig/drivers/net/mlx4/icm.c 2007-07-19 13:11:00.101833000 +0300 +++ connectx/drivers/net/mlx4/icm.c 2007-07-19 13:11:49.192318000 +0300 @@ -50,19 +50,42 @@ enum { MLX4_TABLE_CHUNK_SIZE = 1 << 18 }; -void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm) +static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chunk) +{ + int i; + + if (chunk->nsg > 0) + pci_unmap_sg(dev->pdev, chunk->mem, chunk->npages, + PCI_DMA_BIDIRECTIONAL); + + for (i = 0; i < chunk->npages; ++i) + __free_pages(chunk->mem[i].page, + get_order(chunk->mem[i].length)); +} + +static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *chunk) { - struct mlx4_icm_chunk *chunk, *tmp; int i; + for (i = 0; i < chunk->npages; ++i) { + dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length, + lowmem_page_address(chunk->mem[i].page), + sg_dma_address(&chunk->mem[i])); + } +} + +void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent) +{ + struct mlx4_icm_chunk *chunk, *tmp; + + if (!icm) + return; + list_for_each_entry_safe(chunk, tmp, &icm->chunk_list, list) { - if (chunk->nsg > 0) - pci_unmap_sg(dev->pdev, chunk->mem, chunk->npages, - PCI_DMA_BIDIRECTIONAL); - - for (i = 0; i < chunk->npages; ++i) - __free_pages(chunk->mem[i].page, - get_order(chunk->mem[i].length)); + if (coherent) + mlx4_free_icm_coherent(dev, chunk); + else + mlx4_free_icm_pages(dev, chunk); kfree(chunk); } @@ -70,16 +93,45 @@ void mlx4_free_icm(struct mlx4_dev *dev, kfree(icm); } +static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask) +{ + mem->page = alloc_pages(gfp_mask, order); + if (!mem->page) + return -ENOMEM; + + mem->length = PAGE_SIZE << order; + mem->offset = 0; + return 0; +} + +static int mlx4_alloc_icm_coherent(struct device *dev, struct scatterlist *mem, + int order, gfp_t gfp_mask) +{ + void *buf = dma_alloc_coherent(dev, PAGE_SIZE << order, &sg_dma_address(mem), + gfp_mask); + if (!buf) + return -ENOMEM; + + sg_set_buf(mem, buf, PAGE_SIZE << order); + BUG_ON(mem->offset); + sg_dma_len(mem) = PAGE_SIZE << order; + return 0; +} + struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages, - gfp_t gfp_mask) + gfp_t gfp_mask, int coherent) { struct mlx4_icm *icm; struct mlx4_icm_chunk *chunk = NULL; int cur_order; + int ret; + + /* We use sg_set_buf for coherent allocs, which assumes low memory */ + BUG_ON(coherent && (gfp_mask & __GFP_HIGHMEM)); icm = kmalloc(sizeof *icm, gfp_mask & ~(__GFP_HIGHMEM | __GFP_NOWARN)); if (!icm) - return icm; + return NULL; icm->refcount = 0; INIT_LIST_HEAD(&icm->chunk_list); @@ -101,12 +153,20 @@ struct mlx4_icm *mlx4_alloc_icm(struct m while (1 << cur_order > npages) --cur_order; - chunk->mem[chunk->npages].page = alloc_pages(gfp_mask, cur_order); - if (chunk->mem[chunk->npages].page) { - chunk->mem[chunk->npages].length = PAGE_SIZE << cur_order; - chunk->mem[chunk->npages].offset = 0; - - if (++chunk->npages == MLX4_ICM_CHUNK_LEN) { + if (coherent) + ret = mlx4_alloc_icm_coherent(&dev->pdev->dev, + &chunk->mem[chunk->npages], + cur_order, gfp_mask); + else + ret = mlx4_alloc_icm_pages(&chunk->mem[chunk->npages], + cur_order, gfp_mask); + + if (!ret) { + ++chunk->npages; + + if (coherent) + ++chunk->nsg; + else if (chunk->npages == MLX4_ICM_CHUNK_LEN) { chunk->nsg = pci_map_sg(dev->pdev, chunk->mem, chunk->npages, PCI_DMA_BIDIRECTIONAL); @@ -125,7 +185,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct m } } - if (chunk) { + if (!coherent && chunk) { chunk->nsg = pci_map_sg(dev->pdev, chunk->mem, chunk->npages, PCI_DMA_BIDIRECTIONAL); @@ -137,7 +197,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct m return icm; fail: - mlx4_free_icm(dev, icm); + mlx4_free_icm(dev, icm, coherent); return NULL; } @@ -202,7 +262,7 @@ int mlx4_table_get(struct mlx4_dev *dev, table->icm[i] = mlx4_alloc_icm(dev, MLX4_TABLE_CHUNK_SIZE >> PAGE_SHIFT, (table->lowmem ? GFP_KERNEL : GFP_HIGHUSER) | - __GFP_NOWARN); + __GFP_NOWARN, table->coherent); if (!table->icm[i]) { ret = -ENOMEM; goto out; @@ -210,7 +270,7 @@ int mlx4_table_get(struct mlx4_dev *dev, if (mlx4_MAP_ICM(dev, table->icm[i], table->virt + (u64) i * MLX4_TABLE_CHUNK_SIZE)) { - mlx4_free_icm(dev, table->icm[i]); + mlx4_free_icm(dev, table->icm[i], table->coherent); table->icm[i] = NULL; ret = -ENOMEM; goto out; @@ -234,7 +294,7 @@ void mlx4_table_put(struct mlx4_dev *dev if (--table->icm[i]->refcount == 0) { mlx4_UNMAP_ICM(dev, table->virt + i * MLX4_TABLE_CHUNK_SIZE, MLX4_TABLE_CHUNK_SIZE / MLX4_ICM_PAGE_SIZE); - mlx4_free_icm(dev, table->icm[i]); + mlx4_free_icm(dev, table->icm[i], table->coherent); table->icm[i] = NULL; } @@ -309,7 +369,7 @@ void mlx4_table_put_range(struct mlx4_de int mlx4_init_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table, u64 virt, int obj_size, int nobj, int reserved, - int use_lowmem) + int use_lowmem, int use_coherent) { int obj_per_chunk; int num_icm; @@ -327,6 +387,7 @@ int mlx4_init_icm_table(struct mlx4_dev table->num_obj = nobj; table->obj_size = obj_size; table->lowmem = use_lowmem; + table->coherent = use_coherent; mutex_init(&table->mutex); for (i = 0; i * MLX4_TABLE_CHUNK_SIZE < reserved * obj_size; ++i) { @@ -336,11 +397,11 @@ int mlx4_init_icm_table(struct mlx4_dev table->icm[i] = mlx4_alloc_icm(dev, chunk_size >> PAGE_SHIFT, (use_lowmem ? GFP_KERNEL : GFP_HIGHUSER) | - __GFP_NOWARN); + __GFP_NOWARN, use_coherent); if (!table->icm[i]) goto err; if (mlx4_MAP_ICM(dev, table->icm[i], virt + i * MLX4_TABLE_CHUNK_SIZE)) { - mlx4_free_icm(dev, table->icm[i]); + mlx4_free_icm(dev, table->icm[i], use_coherent); table->icm[i] = NULL; goto err; } @@ -359,7 +420,7 @@ err: if (table->icm[i]) { mlx4_UNMAP_ICM(dev, virt + i * MLX4_TABLE_CHUNK_SIZE, MLX4_TABLE_CHUNK_SIZE / MLX4_ICM_PAGE_SIZE); - mlx4_free_icm(dev, table->icm[i]); + mlx4_free_icm(dev, table->icm[i], use_coherent); } return -ENOMEM; @@ -373,7 +434,7 @@ void mlx4_cleanup_icm_table(struct mlx4_ if (table->icm[i]) { mlx4_UNMAP_ICM(dev, table->virt + i * MLX4_TABLE_CHUNK_SIZE, MLX4_TABLE_CHUNK_SIZE / MLX4_ICM_PAGE_SIZE); - mlx4_free_icm(dev, table->icm[i]); + mlx4_free_icm(dev, table->icm[i], table->coherent); } kfree(table->icm); Index: connectx/drivers/net/mlx4/main.c =================================================================== --- connectx.orig/drivers/net/mlx4/main.c 2007-07-19 13:11:00.077831000 +0300 +++ connectx/drivers/net/mlx4/main.c 2007-07-19 13:11:49.198318000 +0300 @@ -167,7 +167,7 @@ static int __devinit mlx4_load_fw(struct int err; priv->fw.fw_icm = mlx4_alloc_icm(dev, priv->fw.fw_pages, - GFP_HIGHUSER | __GFP_NOWARN); + GFP_HIGHUSER | __GFP_NOWARN, 0); if (!priv->fw.fw_icm) { mlx4_err(dev, "Couldn't allocate FW area, aborting.\n"); return -ENOMEM; @@ -191,7 +191,7 @@ err_unmap_fa: mlx4_UNMAP_FA(dev); err_free: - mlx4_free_icm(dev, priv->fw.fw_icm); + mlx4_free_icm(dev, priv->fw.fw_icm, 0); return err; } @@ -206,7 +206,7 @@ static int __devinit mlx4_init_cmpt_tabl ((u64) (MLX4_CMPT_TYPE_QP * cmpt_entry_sz) << MLX4_CMPT_SHIFT), cmpt_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0); + dev->caps.reserved_qps, 0, 0); if (err) goto err; @@ -215,7 +215,7 @@ static int __devinit mlx4_init_cmpt_tabl ((u64) (MLX4_CMPT_TYPE_SRQ * cmpt_entry_sz) << MLX4_CMPT_SHIFT), cmpt_entry_sz, dev->caps.num_srqs, - dev->caps.reserved_srqs, 0); + dev->caps.reserved_srqs, 0, 0); if (err) goto err_qp; @@ -224,7 +224,7 @@ static int __devinit mlx4_init_cmpt_tabl ((u64) (MLX4_CMPT_TYPE_CQ * cmpt_entry_sz) << MLX4_CMPT_SHIFT), cmpt_entry_sz, dev->caps.num_cqs, - dev->caps.reserved_cqs, 0); + dev->caps.reserved_cqs, 0, 0); if (err) goto err_srq; @@ -235,7 +235,7 @@ static int __devinit mlx4_init_cmpt_tabl cmpt_entry_sz, roundup_pow_of_two(MLX4_NUM_EQ + dev->caps.reserved_eqs), - MLX4_NUM_EQ + dev->caps.reserved_eqs, 0); + MLX4_NUM_EQ + dev->caps.reserved_eqs, 0, 0); if (err) goto err_cq; @@ -274,7 +274,7 @@ static int __devinit mlx4_init_icm(struc (unsigned long long) aux_pages << 2); priv->fw.aux_icm = mlx4_alloc_icm(dev, aux_pages, - GFP_HIGHUSER | __GFP_NOWARN); + GFP_HIGHUSER | __GFP_NOWARN, 0); if (!priv->fw.aux_icm) { mlx4_err(dev, "Couldn't allocate aux memory, aborting.\n"); return -ENOMEM; @@ -302,7 +302,7 @@ static int __devinit mlx4_init_icm(struc init_hca->mtt_base, dev->caps.mtt_entry_sz, dev->caps.num_mtt_segs, - dev->caps.reserved_mtts, 1); + dev->caps.reserved_mtts, 1, 0); if (err) { mlx4_err(dev, "Failed to map MTT context memory, aborting.\n"); goto err_unmap_eq; @@ -312,7 +312,7 @@ static int __devinit mlx4_init_icm(struc init_hca->dmpt_base, dev_cap->dmpt_entry_sz, dev->caps.num_mpts, - dev->caps.reserved_mrws, 1); + dev->caps.reserved_mrws, 1, 1); if (err) { mlx4_err(dev, "Failed to map dMPT context memory, aborting.\n"); goto err_unmap_mtt; @@ -322,7 +322,7 @@ static int __devinit mlx4_init_icm(struc init_hca->qpc_base, dev_cap->qpc_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0); + dev->caps.reserved_qps, 0, 0); if (err) { mlx4_err(dev, "Failed to map QP context memory, aborting.\n"); goto err_unmap_dmpt; @@ -332,7 +332,7 @@ static int __devinit mlx4_init_icm(struc init_hca->auxc_base, dev_cap->aux_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0); + dev->caps.reserved_qps, 0, 0); if (err) { mlx4_err(dev, "Failed to map AUXC context memory, aborting.\n"); goto err_unmap_qp; @@ -342,7 +342,7 @@ static int __devinit mlx4_init_icm(struc init_hca->altc_base, dev_cap->altc_entry_sz, dev->caps.num_qps, - dev->caps.reserved_qps, 0); + dev->caps.reserved_qps, 0, 0); if (err) { mlx4_err(dev, "Failed to map ALTC context memory, aborting.\n"); goto err_unmap_auxc; @@ -352,7 +352,7 @@ static int __devinit mlx4_init_icm(struc init_hca->rdmarc_base, dev_cap->rdmarc_entry_sz << priv->qp_table.rdmarc_shift, dev->caps.num_qps, - dev->caps.reserved_qps, 0); + dev->caps.reserved_qps, 0, 0); if (err) { mlx4_err(dev, "Failed to map RDMARC context memory, aborting\n"); goto err_unmap_altc; @@ -362,7 +362,7 @@ static int __devinit mlx4_init_icm(struc init_hca->cqc_base, dev_cap->cqc_entry_sz, dev->caps.num_cqs, - dev->caps.reserved_cqs, 0); + dev->caps.reserved_cqs, 0, 0); if (err) { mlx4_err(dev, "Failed to map CQ context memory, aborting.\n"); goto err_unmap_rdmarc; @@ -372,7 +372,7 @@ static int __devinit mlx4_init_icm(struc init_hca->srqc_base, dev_cap->srq_entry_sz, dev->caps.num_srqs, - dev->caps.reserved_srqs, 0); + dev->caps.reserved_srqs, 0, 0); if (err) { mlx4_err(dev, "Failed to map SRQ context memory, aborting.\n"); goto err_unmap_cq; @@ -387,7 +387,7 @@ static int __devinit mlx4_init_icm(struc init_hca->mc_base, MLX4_MGM_ENTRY_SIZE, dev->caps.num_mgms + dev->caps.num_amgms, dev->caps.num_mgms + dev->caps.num_amgms, - 0); + 0, 0); if (err) { mlx4_err(dev, "Failed to map MCG context memory, aborting.\n"); goto err_unmap_srq; @@ -432,7 +432,7 @@ err_unmap_aux: mlx4_UNMAP_ICM_AUX(dev); err_free_aux: - mlx4_free_icm(dev, priv->fw.aux_icm); + mlx4_free_icm(dev, priv->fw.aux_icm, 0); return err; } @@ -457,7 +457,7 @@ static void mlx4_free_icms(struct mlx4_d mlx4_unmap_eq_icm(dev); mlx4_UNMAP_ICM_AUX(dev); - mlx4_free_icm(dev, priv->fw.aux_icm); + mlx4_free_icm(dev, priv->fw.aux_icm, 0); } static void mlx4_close_hca(struct mlx4_dev *dev) @@ -465,7 +465,7 @@ static void mlx4_close_hca(struct mlx4_d mlx4_CLOSE_HCA(dev, 0); mlx4_free_icms(dev); mlx4_UNMAP_FA(dev); - mlx4_free_icm(dev, mlx4_priv(dev)->fw.fw_icm); + mlx4_free_icm(dev, mlx4_priv(dev)->fw.fw_icm, 0); } static int __devinit mlx4_init_hca(struct mlx4_dev *dev) @@ -536,7 +536,7 @@ err_free_icm: err_stop_fw: mlx4_UNMAP_FA(dev); - mlx4_free_icm(dev, priv->fw.fw_icm); + mlx4_free_icm(dev, priv->fw.fw_icm, 0); return err; } -- MST From mst at dev.mellanox.co.il Wed Aug 1 02:28:53 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:28:53 +0300 Subject: [ofa-general] [PATCH 2 of 3 for-2.6.24] mlx4: always fill MTTs from CPU Message-ID: <20070801092853.GD29259@mellanox.co.il> From: Jack Morgenstein Write MTT entries directly from the driver (eliminating use of WRITE_MTT command). This reduces the number of FW commands needed to register an MR by at least a factor of 2 and speeds up memory registration significantly. This code will be reused by the FMR implementation. Signed-off-by: Jack Morgenstein Signed-off-by: Michael S. Tsirkin --- Note: this code is similiar to what we have in mthca, and so has the same theoretical issue with using dma_sync_single on addresses mapped with map_sg. Once we decide what the fix for mthca should be, we'll apply it here, too. Index: connectx/drivers/net/mlx4/mr.c =================================================================== --- connectx.orig/drivers/net/mlx4/mr.c 2007-07-18 11:06:15.000000000 +0300 +++ connectx/drivers/net/mlx4/mr.c 2007-07-22 15:22:18.110757000 +0300 @@ -358,58 +358,49 @@ err_table: } EXPORT_SYMBOL_GPL(mlx4_mr_enable); -static int mlx4_WRITE_MTT(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox, - int num_mtt) +static void mlx4_write_mtt_seg(struct mlx4_dev *dev, struct mlx4_mtt *mtt, int start_index, + int npages, u64 *page_list) { - return mlx4_cmd(dev, mailbox->dma, num_mtt, 0, MLX4_CMD_WRITE_MTT, - MLX4_CMD_TIME_CLASS_B); + __be64 *mtts; + struct mlx4_priv *priv = mlx4_priv(dev); + dma_addr_t dma_handle; + int i; + int s = start_index * sizeof (u64); + + /* For Hermon, all MTTs must fit in the same page. */ + BUG_ON(s / PAGE_SIZE != (s + npages * sizeof(u64) - 1) / PAGE_SIZE); + /* Require full segments */ + BUG_ON(s % dev->caps.mtt_entry_sz); + + mtts = mlx4_table_find(&priv->mr_table.mtt_table, mtt->first_seg + + s / dev->caps.mtt_entry_sz, &dma_handle); + + BUG_ON(!mtts); + + for (i = 0; i < npages; ++i) + mtts[i] = cpu_to_be64(page_list[i] | MLX4_MTT_FLAG_PRESENT); + + dma_sync_single(&dev->pdev->dev, dma_handle, npages * sizeof (u64), DMA_TO_DEVICE); } int mlx4_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt, int start_index, int npages, u64 *page_list) { - struct mlx4_cmd_mailbox *mailbox; - __be64 *mtt_entry; - int i; - int err = 0; + int chunk; if (mtt->order < 0) return -EINVAL; - mailbox = mlx4_alloc_cmd_mailbox(dev); - if (IS_ERR(mailbox)) - return PTR_ERR(mailbox); - - mtt_entry = mailbox->buf; - while (npages > 0) { - mtt_entry[0] = cpu_to_be64(mlx4_mtt_addr(dev, mtt) + start_index * 8); - mtt_entry[1] = 0; + chunk = min((int)(PAGE_SIZE / sizeof(u64)), npages); + mlx4_write_mtt_seg(dev, mtt, start_index, chunk, page_list); - for (i = 0; i < npages && i < MLX4_MAILBOX_SIZE / 8 - 2; ++i) - mtt_entry[i + 2] = cpu_to_be64(page_list[i] | - MLX4_MTT_FLAG_PRESENT); - - /* - * If we have an odd number of entries to write, add - * one more dummy entry for firmware efficiency. - */ - if (i & 1) - mtt_entry[i + 2] = 0; - - err = mlx4_WRITE_MTT(dev, mailbox, (i + 1) & ~1); - if (err) - goto out; - - npages -= i; - start_index += i; - page_list += i; + npages -= chunk; + start_index += chunk; + page_list += chunk; } -out: - mlx4_free_cmd_mailbox(dev, mailbox); - - return err; + return 0; } EXPORT_SYMBOL_GPL(mlx4_write_mtt); Index: connectx/drivers/net/mlx4/icm.c =================================================================== --- connectx.orig/drivers/net/mlx4/icm.c 2007-07-19 16:34:38.000000000 +0300 +++ connectx/drivers/net/mlx4/icm.c 2007-07-22 14:43:05.584012000 +0300 @@ -302,9 +302,9 @@ void mlx4_table_put(struct mlx4_dev *dev mutex_unlock(&table->mutex); } -void *mlx4_table_find(struct mlx4_icm_table *table, int obj) +void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_handle) { - int idx, offset, i; + int idx, offset, dma_offset, i; struct mlx4_icm_chunk *chunk; struct mlx4_icm *icm; struct page *page = NULL; @@ -314,15 +314,24 @@ void *mlx4_table_find(struct mlx4_icm_ta mutex_lock(&table->mutex); - idx = obj & (table->num_obj - 1); - icm = table->icm[idx / (MLX4_TABLE_CHUNK_SIZE / table->obj_size)]; - offset = idx % (MLX4_TABLE_CHUNK_SIZE / table->obj_size); + idx = (obj & (table->num_obj - 1)) * table->obj_size; + icm = table->icm[idx / MLX4_TABLE_CHUNK_SIZE]; + dma_offset = offset = idx % MLX4_TABLE_CHUNK_SIZE; if (!icm) goto out; list_for_each_entry(chunk, &icm->chunk_list, list) { for (i = 0; i < chunk->npages; ++i) { + if (dma_handle && dma_offset >= 0) { + if (sg_dma_len(&chunk->mem[i]) > dma_offset) + *dma_handle = sg_dma_address(&chunk->mem[i]) + + dma_offset; + dma_offset -= sg_dma_len(&chunk->mem[i]); + } + /* DMA mapping can merge pages but not split them, + * so if we found the page, dma_handle has already + * been assigned to. */ if (chunk->mem[i].length > offset) { page = chunk->mem[i].page; goto out; Index: connectx/drivers/net/mlx4/icm.h =================================================================== --- connectx.orig/drivers/net/mlx4/icm.h 2007-07-19 16:34:38.000000000 +0300 +++ connectx/drivers/net/mlx4/icm.h 2007-07-19 19:29:58.282773000 +0300 @@ -84,7 +84,7 @@ int mlx4_init_icm_table(struct mlx4_dev void mlx4_cleanup_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table); int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj); -void *mlx4_table_find(struct mlx4_icm_table *table, int obj); +void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_handle); int mlx4_table_get_range(struct mlx4_dev *dev, struct mlx4_icm_table *table, int start, int end); void mlx4_table_put_range(struct mlx4_dev *dev, struct mlx4_icm_table *table, Index: connectx/drivers/net/mlx4/main.c =================================================================== --- connectx.orig/drivers/net/mlx4/main.c 2007-07-19 16:34:38.000000000 +0300 +++ connectx/drivers/net/mlx4/main.c 2007-07-22 15:57:28.378142000 +0300 @@ -41,6 +41,7 @@ #include #include +#include #include "mlx4.h" #include "fw.h" @@ -263,6 +264,7 @@ static int __devinit mlx4_init_icm(struc struct mlx4_priv *priv = mlx4_priv(dev); u64 aux_pages; int err; + int num_mtt_res_bytes; err = mlx4_SET_ICM_SIZE(dev, icm_size, &aux_pages); if (err) { @@ -299,11 +301,21 @@ static int __devinit mlx4_init_icm(struc goto err_unmap_cmpt; } + /* Reserved mtt entries must be aligned up to a cacheline boundary, + * since the FW will write to them, while the driver writes to all + * other mtt entries. (Note that the variable dev->caps.mtt_entry_sz + * below is really the mtt segment size, not the raw entry size) + */ + num_mtt_res_bytes = ((dev->caps.reserved_mtts * + (dev->caps.mtt_entry_sz / MLX4_MTT_ENTRY_PER_SEG) + + L1_CACHE_BYTES - 1) / + L1_CACHE_BYTES) * L1_CACHE_BYTES; err = mlx4_init_icm_table(dev, &priv->mr_table.mtt_table, init_hca->mtt_base, dev->caps.mtt_entry_sz, dev->caps.num_mtt_segs, - dev->caps.reserved_mtts, 1, 0); + num_mtt_res_bytes / dev->caps.mtt_entry_sz, + 1, 0); if (err) { mlx4_err(dev, "Failed to map MTT context memory, aborting.\n"); goto err_unmap_eq; Index: connectx/drivers/infiniband/hw/mlx4/mr.c =================================================================== --- connectx.orig/drivers/infiniband/hw/mlx4/mr.c 2007-07-18 11:06:15.000000000 +0300 +++ connectx/drivers/infiniband/hw/mlx4/mr.c 2007-07-22 17:00:53.672771000 +0300 @@ -96,11 +96,10 @@ int mlx4_ib_umem_write_mtt(struct mlx4_i pages[i++] = sg_dma_address(&chunk->page_list[j]) + umem->page_size * k; /* - * Be friendly to WRITE_MTT firmware - * command, and pass it chunks of - * appropriate size. + * Be friendly to mlx4_write_mtt + * and pass it chunks of appropriate size. */ - if (i == PAGE_SIZE / sizeof (u64) - 2) { + if (i == PAGE_SIZE / sizeof (u64)) { err = mlx4_write_mtt(dev->dev, mtt, n, i, pages); if (err) -- MST From mst at dev.mellanox.co.il Wed Aug 1 02:29:05 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:29:05 +0300 Subject: [ofa-general] [PATCH 3 of 3 for-2.6.24] mlx4: implement FMRs Message-ID: <20070801092905.GE29259@mellanox.co.il> From: Jack Morgenstein Implement FMRs in mlx4. This is an adaptation of code from mthca. Signed-off-by: Jack Morgenstein Signed-off-by: Michael S. Tsirkin --- Index: connectx/drivers/infiniband/hw/mlx4/mlx4_ib.h =================================================================== --- connectx.orig/drivers/infiniband/hw/mlx4/mlx4_ib.h 2007-07-26 10:04:57.900689000 +0300 +++ connectx/drivers/infiniband/hw/mlx4/mlx4_ib.h 2007-07-26 10:17:48.841925000 +0300 @@ -93,6 +93,11 @@ struct mlx4_ib_mr { struct ib_umem *umem; }; +struct mlx4_ib_fmr { + struct ib_fmr ibfmr; + struct mlx4_fmr mfmr; +}; + struct mlx4_ib_wq { u64 *wrid; spinlock_t lock; @@ -199,6 +204,10 @@ static inline struct mlx4_ib_mr *to_mmr( return container_of(ibmr, struct mlx4_ib_mr, ibmr); } +static inline struct mlx4_ib_fmr *to_mfmr(struct ib_fmr *ibfmr) +{ + return container_of(ibfmr, struct mlx4_ib_fmr, ibfmr); +} static inline struct mlx4_ib_qp *to_mqp(struct ib_qp *ibqp) { return container_of(ibqp, struct mlx4_ib_qp, ibqp); @@ -284,6 +293,13 @@ int mlx4_ib_process_mad(struct ib_device int mlx4_ib_mad_init(struct mlx4_ib_dev *dev); void mlx4_ib_mad_cleanup(struct mlx4_ib_dev *dev); +struct ib_fmr * mlx4_ib_fmr_alloc(struct ib_pd *pd, int mr_access_flags, + struct ib_fmr_attr *fmr_attr); +int mlx4_ib_map_phys_fmr(struct ib_fmr *ibfmr, u64 *page_list, int npages, + u64 iova); +int mlx4_ib_unmap_fmr(struct list_head *fmr_list); +int mlx4_ib_fmr_dealloc(struct ib_fmr *fmr); + static inline int mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) { return !!(ah->av.g_slid & 0x80); Index: connectx/include/linux/mlx4/device.h =================================================================== --- connectx.orig/include/linux/mlx4/device.h 2007-07-26 10:04:58.045542000 +0300 +++ connectx/include/linux/mlx4/device.h 2007-07-26 10:17:48.849924000 +0300 @@ -210,6 +210,17 @@ struct mlx4_mr { int enabled; }; +struct mlx4_fmr { + struct mlx4_mr mr; + int max_pages; + int max_maps; + u8 page_shift; + int maps; + struct mlx4_mpt_entry *mpt; + __be64 *mtts; + dma_addr_t dma_handle; +}; + struct mlx4_uar { unsigned long pfn; int index; @@ -331,4 +342,14 @@ int mlx4_CLOSE_PORT(struct mlx4_dev *dev int mlx4_multicast_attach(struct mlx4_dev *dev, struct mlx4_qp *qp, u8 gid[16]); int mlx4_multicast_detach(struct mlx4_dev *dev, struct mlx4_qp *qp, u8 gid[16]); +int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, + int npages, u64 iova, u32 *lkey, u32 *rkey); +int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, + int max_maps, u8 page_shift, struct mlx4_fmr *fmr); +int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr); +void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u32 *lkey, u32 *rkey); +int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr); +int mlx4_SYNC_TPT(struct mlx4_dev *dev); + #endif /* MLX4_DEVICE_H */ Index: connectx/drivers/infiniband/hw/mlx4/mr.c =================================================================== --- connectx.orig/drivers/infiniband/hw/mlx4/mr.c 2007-07-26 10:04:57.903688000 +0300 +++ connectx/drivers/infiniband/hw/mlx4/mr.c 2007-07-26 10:17:48.854923000 +0300 @@ -181,3 +181,91 @@ int mlx4_ib_dereg_mr(struct ib_mr *ibmr) return 0; } + +struct ib_fmr *mlx4_ib_fmr_alloc(struct ib_pd *pd, int acc, + struct ib_fmr_attr *fmr_attr) +{ + struct mlx4_ib_dev *dev = to_mdev(pd->device); + struct mlx4_ib_fmr *fmr; + int err = -ENOMEM; + + fmr = kmalloc(sizeof *fmr, GFP_KERNEL); + if (!fmr) + return ERR_PTR(-ENOMEM); + + err = mlx4_fmr_alloc(dev->dev, to_mpd(pd)->pdn, convert_access(acc), + fmr_attr->max_pages, fmr_attr->max_maps, + fmr_attr->page_shift, &fmr->mfmr); + if (err) + goto err_free; + + err = mlx4_mr_enable(to_mdev(pd->device)->dev, &fmr->mfmr.mr); + if (err) + goto err_mr; + + fmr->ibfmr.rkey = fmr->ibfmr.lkey = fmr->mfmr.mr.key; + + return &fmr->ibfmr; + +err_mr: + mlx4_mr_free(to_mdev(pd->device)->dev, &fmr->mfmr.mr); + +err_free: + kfree(fmr); + + return ERR_PTR(err); +} + +int mlx4_ib_map_phys_fmr(struct ib_fmr *ibfmr, u64 *page_list, + int npages, u64 iova) +{ + struct mlx4_ib_fmr *ifmr = to_mfmr(ibfmr); + struct mlx4_ib_dev *dev = to_mdev(ifmr->ibfmr.device); + + return mlx4_map_phys_fmr(dev->dev, &ifmr->mfmr, page_list, npages, iova, + &ifmr->ibfmr.lkey, &ifmr->ibfmr.rkey); +} + +int mlx4_ib_unmap_fmr(struct list_head *fmr_list) +{ + struct ib_fmr *ibfmr; + int err; + struct mlx4_dev *mdev = NULL; + + list_for_each_entry(ibfmr, fmr_list, list) { + if (mdev && to_mdev(ibfmr->device)->dev != mdev) + return -EINVAL; + mdev = to_mdev(ibfmr->device)->dev; + } + + if (!mdev) + return 0; + + list_for_each_entry(ibfmr, fmr_list, list) { + struct mlx4_ib_fmr *ifmr = to_mfmr(ibfmr); + + mlx4_fmr_unmap(mdev, &ifmr->mfmr, &ifmr->ibfmr.lkey, &ifmr->ibfmr.rkey); + } + + wmb(); + + err = mlx4_SYNC_TPT(mdev); + if (err) { + printk("mlx4_ib_unmap_fmr: sync TPT error %d\n", err); + } + return 0; +} + +int mlx4_ib_fmr_dealloc(struct ib_fmr *ibfmr) +{ + struct mlx4_ib_fmr *ifmr = to_mfmr(ibfmr); + struct mlx4_ib_dev *dev = to_mdev(ibfmr->device); + int err; + + err = mlx4_fmr_free(dev->dev, &ifmr->mfmr); + + if (!err) + kfree(ifmr); + + return err; +} Index: connectx/drivers/net/mlx4/mr.c =================================================================== --- connectx.orig/drivers/net/mlx4/mr.c 2007-07-26 10:08:14.070595000 +0300 +++ connectx/drivers/net/mlx4/mr.c 2007-07-26 10:17:48.861923000 +0300 @@ -465,3 +465,159 @@ void mlx4_cleanup_mr_table(struct mlx4_d mlx4_buddy_cleanup(&mr_table->mtt_buddy); mlx4_bitmap_cleanup(&mr_table->mpt_bitmap); } + +#define MLX4_MPT_STATUS_SW 0xF0 +#define MLX4_MPT_STATUS_HW 0x00 +#define MLX4_MTT_FLAG_PRESENT 1 + +static inline int mlx4_check_fmr(struct mlx4_fmr *fmr, u64 *page_list, + int npages, u64 iova) +{ + int i, page_mask; + + if (npages > fmr->max_pages) + return -EINVAL; + + page_mask = (1 << fmr->page_shift) - 1; + + /* We are getting page lists, so va must be page aligned. */ + if (iova & page_mask) + return -EINVAL; + + /* Trust the user not to pass misaligned data in page_list */ + if (0) + for (i = 0; i < npages; ++i) { + if (page_list[i] & ~page_mask) + return -EINVAL; + } + + if (fmr->maps >= fmr->max_maps) + return -EINVAL; + + return 0; +} + +int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, + int npages, u64 iova, u32 *lkey, u32 *rkey) +{ + u32 key; + int i, err; + + err = mlx4_check_fmr(fmr, page_list, npages, iova); + if (err) + return err; + + ++fmr->maps; + + key = key_to_hw_index(fmr->mr.key); + key += dev->caps.num_mpts; + *lkey = *rkey = fmr->mr.key = hw_index_to_key(key); + + *(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW; + + wmb(); + + for (i = 0; i < npages; ++i) + fmr->mtts[i] = cpu_to_be64(page_list[i] | MLX4_MTT_FLAG_PRESENT); + + dma_sync_single(&dev->pdev->dev, fmr->dma_handle, + npages * sizeof(u64), DMA_TO_DEVICE); + + fmr->mpt->key = cpu_to_be32(key); + fmr->mpt->lkey = cpu_to_be32(key); + fmr->mpt->length = cpu_to_be64(npages * (1ull << fmr->page_shift)); + fmr->mpt->start = cpu_to_be64(iova); + + wmb(); + + *(u8 *) fmr->mpt = MLX4_MPT_STATUS_HW; + + wmb(); + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_map_phys_fmr); + +int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, + int max_maps, u8 page_shift, struct mlx4_fmr *fmr) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + u64 mtt_seg; + int err = -ENOMEM; + + if (page_shift < 12 || page_shift >= 32) + return -EINVAL; + + /* For Hermon, all MTTs must fit in the same page. */ + if (max_pages * sizeof *fmr->mtts > PAGE_SIZE) + return -EINVAL; + + fmr->page_shift = page_shift; + fmr->max_pages = max_pages; + fmr->max_maps = max_maps; + fmr->maps = 0; + + err = mlx4_mr_alloc(dev, pd, 0, 0, access, max_pages, + page_shift, &fmr->mr); + if (err) + return err; + + + mtt_seg = fmr->mr.mtt.first_seg * dev->caps.mtt_entry_sz; + + + fmr->mtts = mlx4_table_find(&priv->mr_table.mtt_table, + fmr->mr.mtt.first_seg, + &fmr->dma_handle); + BUG_ON(!fmr->mtts); + + fmr->mpt = mlx4_table_find(&priv->mr_table.dmpt_table, + key_to_hw_index(fmr->mr.key), NULL); + BUG_ON(!fmr->mpt); + + return 0; + +} +EXPORT_SYMBOL_GPL(mlx4_fmr_alloc); + +int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr) +{ + return mlx4_mr_enable(dev, &fmr->mr); +} +EXPORT_SYMBOL_GPL(mlx4_fmr_enable); + +void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u32 *lkey, u32 *rkey) +{ + u32 key; + + if (!fmr->maps) + return; + + key = key_to_hw_index(fmr->mr.key); + key &= dev->caps.num_mpts - 1; + *lkey = *rkey = fmr->mr.key = hw_index_to_key(key); + + fmr->maps = 0; + + *(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_unmap); + +int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr) +{ + if (fmr->maps) + return -EBUSY; + + fmr->mr.enabled = 0; + mlx4_mr_free(dev, &fmr->mr); + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_free); + +int mlx4_SYNC_TPT(struct mlx4_dev *dev) +{ + return mlx4_cmd(dev, 0, 0, 0, MLX4_CMD_SYNC_TPT, 1000); +} +EXPORT_SYMBOL_GPL(mlx4_SYNC_TPT); Index: connectx/drivers/infiniband/hw/mlx4/main.c =================================================================== --- connectx.orig/drivers/infiniband/hw/mlx4/main.c 2007-07-26 10:04:57.932655000 +0300 +++ connectx/drivers/infiniband/hw/mlx4/main.c 2007-07-26 10:17:48.866926000 +0300 @@ -569,6 +569,11 @@ static void *mlx4_ib_add(struct mlx4_dev ibdev->ib_dev.detach_mcast = mlx4_ib_mcg_detach; ibdev->ib_dev.process_mad = mlx4_ib_process_mad; + ibdev->ib_dev.alloc_fmr = mlx4_ib_fmr_alloc; + ibdev->ib_dev.map_phys_fmr = mlx4_ib_map_phys_fmr; + ibdev->ib_dev.unmap_fmr = mlx4_ib_unmap_fmr; + ibdev->ib_dev.dealloc_fmr = mlx4_ib_fmr_dealloc; + if (init_node_data(ibdev)) goto err_map; -- MST From erezz at voltaire.com Wed Aug 1 02:41:47 2007 From: erezz at voltaire.com (Erez Zilber) Date: Wed, 01 Aug 2007 12:41:47 +0300 Subject: [ofa-general] Errors while loading librdmacm & libibverbs Message-ID: <46B0555B.9090601@voltaire.com> Hi, I'm trying to run STGT with iSER support. I'm using the STGT kernel (2.6.22) with the STGT userspace code. Since it uses libibverbs & librdmacm, I cloned both git trees and installed them. When I start STGT, I get the following errors: [root at sage usr]# ./tgtd [iSER]:iscsi_iser_init() Entry. librdmacm: couldn't read ABI version. librdmacm: assuming: 4 libibverbs: Fatal: couldn't read uverbs ABI version. CMA: unable to open /dev/infiniband/rdma_cm librdmacm: couldn't read ABI version. librdmacm: assuming: 4 libibverbs: Fatal: couldn't read uverbs ABI version. Segmentation fault I saw that the following sysfs files don't exist (that's why librdmacm says "couldn't read ABI version"): /sys/class/misc/rdma_cm/abi_version /sys/class/infiniband_ucma/abi_version Did I forget to load anything? What is the problem? Thanks, -- ____________________________________________________________ Erez Zilber | 972-9-971-7689 Software Engineer, Storage Team Voltaire – _The Grid Backbone_ __ www.voltaire.com From mst at dev.mellanox.co.il Wed Aug 1 02:44:38 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 12:44:38 +0300 Subject: [ofa-general] Re: patches for 1.2.c In-Reply-To: <46AF89A0.9070805@opengridcomputing.com> References: <46AF89A0.9070805@opengridcomputing.com> Message-ID: <20070801094438.GF29259@mellanox.co.il> BTW, if all you want to do is add patches which are applied upstream, just send the commit ids. Quoting Steve Wise : Subject: patches for 1.2.c Guys, I have 2 more patches to go in ofed_1_2/ofed_1_2_c. Is there some grand scheme to the naming of kernel_patches/fixes/* for 1.2.c? I noticed a slew of new files for the post-2.6.22 fixes, and wondered if there is a naming scheme? Or should I just post a patch for the ofed_1_2 branch and let you all create the ofed_1_2_c kernel_patches/fixes/ patch file ?? Thanks, Steve. -- MST From vlad at lists.openfabrics.org Wed Aug 1 02:50:35 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 1 Aug 2007 02:50:35 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070801-0200 daily build status Message-ID: <20070801095036.A3722E6083D@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ehca/ehca_eq.c:126: error: for each function it appears in.) /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ehca/ehca_eq.c:127: warning: passing argument 3 of ‘ibmebus_request_irq’ from incompatible pointer type /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ehca/ehca_eq.c:135: warning: passing argument 3 of ‘ibmebus_request_irq’ from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ehca/ehca_eq.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.17_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070801-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dotanb at dev.mellanox.co.il Wed Aug 1 03:37:23 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 01 Aug 2007 13:37:23 +0300 Subject: [ofa-general] Re: should the macro IB_UMEM_MAX_PAGE_CHUNK be removed from ib_verbs? In-Reply-To: References: <46AF4157.3090007@dev.mellanox.co.il> <20070731140532.GB16015@mellanox.co.il> <46AF479B.2000505@dev.mellanox.co.il> Message-ID: <46B06263.6090509@dev.mellanox.co.il> Roland Dreier wrote: > >>> 2) move this macro to ib_umem.h > > >> 2 I guess. > > > I will send a patch with the suggested options in a few minutes. > > Actually just move the macro to umem.c since that's the only place it > is (or should be) used anyway. > > - R. > Yesterday i send a patch before i saw this email, i will send in a few minutes a patch that moves IB_UMEM_MAX_PAGE_CHUNK from ib_verbs.h to umem.c thanks Dotan From dotanb at dev.mellanox.co.il Wed Aug 1 03:33:56 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 1 Aug 2007 13:33:56 +0300 Subject: [ofa-general] [PATCH V2] core: move the macro IB_UMEM_MAX_PAGE_CHUNK to umem.c Message-ID: <200708011333.56131.dotanb@dev.mellanox.co.il> After moving the struct ib_umem_chunk from the ib_verbs.h to ib_umem.h there isn't any reason for the macro IB_UMEM_MAX_PAGE_CHUNK to stay in ib_verbs.h. This macro was moved to umem.c, the only place who uses it. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 26d0470..664d2fa 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -40,6 +40,11 @@ #include "uverbs.h" +#define IB_UMEM_MAX_PAGE_CHUNK \ + ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ + ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ + (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) + static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty) { struct ib_umem_chunk *chunk, *tmp; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 0627a6a..43b4c97 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -731,11 +731,6 @@ struct ib_udata { size_t outlen; }; -#define IB_UMEM_MAX_PAGE_CHUNK \ - ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ - ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ - (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) - struct ib_pd { struct ib_device *device; struct ib_uobject *uobject; From krkumar2 at in.ibm.com Wed Aug 1 03:41:05 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 1 Aug 2007 16:11:05 +0530 Subject: [ofa-general] Re: Processing multiple WR's in MTHCA (other drivers). In-Reply-To: <20070801091745.GA29259@mellanox.co.il> Message-ID: Yes, I saw that, and also 10.8.3. Just wondering if that is actually complaint in the hardware. The reason being the huge amount of retransmission that I am getting. thanks, - KK "Michael S. Tsirkin" wrote on 08/01/2007 02:47:45 PM: > Quoting Krishna Kumar2 : > Subject: Processing multiple WR's in MTHCA (other drivers). > > If I call post_send with multiple WR's (as a list of 'next' linked WR's) from > > IPoIB to the driver, is it guaranteed that those WR's will go out on the wire in > > the same sequence? > > See chapter 9.5 transaction ordering: > C9-25: A requester shall transmit request messages in the order that the > Work Queue Elements (WQEs) were posted. > > > -- > MST From eaburns at iol.unh.edu Wed Aug 1 05:26:39 2007 From: eaburns at iol.unh.edu (Ethan Burns) Date: Wed, 1 Aug 2007 08:26:39 -0400 Subject: [ofa-general] amso1100 NULL dereference Message-ID: <20070801122639.GA15576@postal.iol.unh.edu> Hi Tom, I believe that I found a bug in the amso1100 driver. In the c2_up() function there is the following code: /* Tell the stack to ignore arp requests for ipaddrs bound to * other interfaces. This is needed to prevent the host stack * from responding to arp requests to the ipaddr bound on the * rdma interface. */ in_dev = in_dev_get(netdev); IN_DEV_CONF_SET(in_dev, ARP_IGNORE, 1); in_dev_put(in_dev); The issue is that the interface can be brought up with out having an ipv4 address assigned to it. This causes a NULL pointer derefernce (since in_dev will be NULL after in_dev_get(netdev)). I tried a simple ``if (in_dev) { }'', however, I then run into trouble if I add an ipv4 address to the interface after it is already up (I encounter the ARP problem that this piece of code is trying to solve). A simple solution is to only bring up the interface with an ipv4 address. This will work for my purposes, however, I wanted to point this issue out incase there is possible fix for it. Thanks, Ethan From sashak at voltaire.com Wed Aug 1 05:46:47 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 1 Aug 2007 15:46:47 +0300 Subject: [ofa-general] [PATCH] opensm: loopback connection handling In-Reply-To: <20070729165220.GC29844@sashak.voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901F75131@mtlexch01.mtl.com> <20070725001847.GG25264@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F7558A@mtlexch01.mtl.com> <20070725194856.GB31582@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75673@mtlexch01.mtl.com> <20070727010707.GR2472@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75ABE@mtlexch01.mtl.com> <20070728221540.GI12351@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75CF6@mtlexch01.mtl.com> <20070729165220.GC29844@sashak.voltaire.com> Message-ID: <20070801124647.GC6357@sashak.voltaire.com> This improves loopback connection handling: 1. OpenSM will decide about back-to-back duplicated GUIDs only if there were no switches discovered 2. If 'stay_on_fatal' is on, OpenSM will connect those ports 3. Reduced verbosity, if 'stay_on_fatal' is off the error message appears in syslog Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_node_info_rcv.c | 53 ++++++++++++++++-------------------- 1 files changed, 24 insertions(+), 29 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index cf5baaa..6cabad6 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -215,12 +215,11 @@ __osm_ni_rcv_set_links( them. But the link will be from the port to itself... Enhanced Port 0 is an exception to this */ - if ((osm_node_get_node_guid( p_node ) == p_ni_context->node_guid) && + if ((osm_node_get_node_guid(p_node) == p_ni_context->node_guid) && (port_num == p_ni_context->port_num) && - (port_num != 0)) + port_num != 0 && cl_qmap_count(&p_rcv->p_subn->sw_guid_tbl) == 0) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_set_links: ERR 0D18: " + osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, "__osm_ni_rcv_set_links: " "Duplicate GUID found by link from a port to itself:" "node 0x%" PRIx64 ", port number 0x%X\n", cl_ntoh64( osm_node_get_node_guid( p_node ) ), @@ -228,37 +227,33 @@ __osm_ni_rcv_set_links( p_physp = osm_node_get_physp_ptr( p_node, port_num ); osm_dump_dr_path(p_rcv->p_log, osm_physp_get_dr_path_ptr(p_physp), - OSM_LOG_ERROR); - - osm_log( p_rcv->p_log, OSM_LOG_SYS, - "Errors on subnet. Duplicate GUID found " - "by link from a port to itself. " - "See opensm.log for more details\n"); + OSM_LOG_VERBOSE); if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) + { + osm_log( p_rcv->p_log, OSM_LOG_SYS, + "Errors on subnet. Duplicate GUID found " + "by link from a port to itself. " + "See verbose opensm.log for more details\n"); exit( 1 ); + } } - else - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_set_links: " - "Creating new link between: " - "\n\t\t\t\tnode 0x%" PRIx64 ", " - "port number 0x%X and" - "\n\t\t\t\tnode 0x%" PRIx64 ", " - "port number 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - port_num, - cl_ntoh64( p_ni_context->node_guid ), - p_ni_context->port_num ); - } + if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) + osm_log( p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_ni_rcv_set_links: " + "Creating new link between: " + "\n\t\t\t\tnode 0x%" PRIx64 ", " + "port number 0x%X and" + "\n\t\t\t\tnode 0x%" PRIx64 ", " + "port number 0x%X\n", + cl_ntoh64( osm_node_get_node_guid( p_node ) ), + port_num, + cl_ntoh64( p_ni_context->node_guid ), + p_ni_context->port_num ); - osm_node_link( p_node, port_num, p_neighbor_node, - p_ni_context->port_num ); - } + osm_node_link( p_node, port_num, p_neighbor_node, + p_ni_context->port_num ); _exit: OSM_LOG_EXIT( p_rcv->p_log ); -- 1.5.3.rc2.38.g11308 From deirdre.dallimore at gesundheitsinstitut.at Wed Aug 1 08:56:27 2007 From: deirdre.dallimore at gesundheitsinstitut.at (Rosalie Curtis) Date: Wed, 1 Aug 2007 13:56:27 -0200 Subject: [ofa-general] Solve erection problems Message-ID: <279298823.34272270087623@gesundheitsinstitut.at> -------------- next part -------------- A non-text attachment was scrubbed... Name: youuu.gif Type: image/gif Size: 11780 bytes Desc: not available URL: From swise at opengridcomputing.com Wed Aug 1 06:58:15 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 01 Aug 2007 08:58:15 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> Message-ID: <46B09177.6040509@opengridcomputing.com> Sean Hefty wrote: >> The correct solution in my mind is to use the host stack's TCP port >> space for _all_ RDMA_PS_TCP port allocations. The patch below is a >> minimal delta to unify the port spaces bay using the kernel stack to >> bind ports. This is done by allocating a kernel socket and binding to >> the appropriate local addr/port. It also allows the kernel stack to >> pick ephemeral ports by virtue of just passing in port 0 on the kernel >> bind operation. > > I'm not thrilled with the idea of overlapping port spaces, and I can't come up > with a solution that works for all situations. I understand the overlapping > port space problem, but I consider the ability to use the same port number for > both RDMA and sockets a feature. > > What if MPI used a similar mechanism as SDP? That is, if it gets a port number > from sockets, it reserves that same RDMA port number, or vice-versa. The > rdma_cm advertises separate port spaces from TCP/UDP, so IMO any assumption > otherwise, at this point, is a bug in the user's code. This would require EVERY rdma application to do this or potentially steal listening endpoints from sockets applications already running. This isn't just an MPI issue. Consider the iperf daemon running on port 5001. Then you run mvapich2/rdma and it happens to bind/listen to port 5001. Then an incoming innocent iperf connection gets passed to the rdma stack in error... > > Before merging the port spaces, I'd like a way for an application to use a > single well-known port number that works over both RDMA and sockets. > So you want the ability for an application to support a standard TCP/Sockets service concurrently with an RDMA service? To do this effectively over iWARP, you need the ability to take a streaming native host connection and migrate it to rdma mode and vice-versa. And that model isn't very popular. Consider NFS and NFS-RDMA. The NFS gurus struggled with this very issue and concluded that the RDMA service needs to be on a separate port. Thus they are proposing a new netid/port number for doing RDMA mounts vs TCP/UDP mounts. IMO that is the correct way to go: RDMA services are different that tcp services. They use a different protocol on top of TCP and thus shouldn't be handled on the same TCP port. So, applications that want to service Sockets and RDMA services concurrently would do so by listening on different ports... >> RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. > > Is there any reason to limit this behavior to TCP only, or would we also include > UDP? > The iWARP protocols don't include a UDP based service, so it is not needed. But if you're calling it a UDP port space, maybe it should be the host's port space? >> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c >> index 9e0ab04..e4d2d7f 100644 >> --- a/drivers/infiniband/core/cma.c >> +++ b/drivers/infiniband/core/cma.c >> @@ -111,6 +111,7 @@ struct rdma_id_private { >> struct rdma_cm_id id; >> >> struct rdma_bind_list *bind_list; >> + struct socket *sock; > > This points off to a rather largish structure... > Yes. The only exports interfaces into the host port allocation stuff requires a socket struct. I didn't want to try and tackle exporting the port allocation services at a lower level. Even at the bottom level, I think it still assumes a socket struct... >> struct hlist_node node; >> struct list_head list; >> struct list_head listen_list; >> @@ -695,6 +696,8 @@ static void cma_release_port(struct rdma >> kfree(bind_list); >> } >> mutex_unlock(&lock); >> + if (id_priv->sock) >> + sock_release(id_priv->sock); >> } >> >> void rdma_destroy_id(struct rdma_cm_id *id) >> @@ -1790,6 +1793,25 @@ static int cma_use_port(struct idr *ps, >> return 0; >> } >> >> +static int cma_get_tcp_port(struct rdma_id_private *id_priv) >> +{ >> + int ret; >> + struct socket *sock; >> + >> + ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); >> + if (ret) >> + return ret; >> + ret = sock->ops->bind(sock, >> + (struct socketaddr *)&id_priv->id.route.addr.src_addr, >> + ip_addr_size(&id_priv->id.route.addr.src_addr)); >> + if (ret) { >> + sock_release(sock); >> + return ret; >> + } >> + id_priv->sock = sock; >> + return 0; >> +} >> + >> static int cma_get_port(struct rdma_id_private *id_priv) >> { >> struct idr *ps; >> @@ -1801,6 +1823,9 @@ static int cma_get_port(struct rdma_id_p >> break; >> case RDMA_PS_TCP: >> ps = &tcp_ps; >> + ret = cma_get_tcp_port(id_priv); /* Synch with native stack */ >> + if (ret) >> + goto out; > > Would we need tcp_ps (and udp_ps) anymore? I left it all in to show the minimal changes needed to implement the functionality. To keep the patch simple for initial consumption. But yes, the rdma-cm really doesn't need to track the port stuff for TCP since the host stack does. > Also, I think SDP maps into the TCP > port space already, so changes to SDP will be needed as well, which may > eliminate its port space. I haven't looked in detail at the SDP code, but I would think it should want the TCP port space and not its own anwyay, but I'm not sure. What is the point of the SDP port space anyway? BTW: I believe for SDP to work over iWARP, a "shared by all SDP peers" port mapper service is required to map TCP port A to a different SDP port so that the iwarp SDP connections don't collide with real TCP connections for the same application service. Thus 2 ports are really used. Regardless, both ports would need to be reserved from the host stack port space or the issue will happen: some sockets app might also bind to the SDP port and be denied service. I think its all a non-issue for IB since IPoIB and RDMA/IB use different QPs/connections/contexts/etc... One more issue for discussion: This patch is reserving ports exclusively even if they are just ephemeral ports used on the active side. This isn't really required, and limits the amount of connections you can have. The current rdma-cm, however, does the same thing. We could consider using lower level services like inet_hash_connect(). Steve. From steffen.persvold at scali.com Wed Aug 1 07:01:23 2007 From: steffen.persvold at scali.com (Steffen Persvold) Date: Wed, 1 Aug 2007 10:01:23 -0400 Subject: [ofa-general] HCA load balancing and data striping In-Reply-To: References: <6C2C79E72C305246B504CBA17B5500C901563925@mtlexch01.mtl.com> Message-ID: Hi, As anyone on this list ever considered or implemented a "meta" ibverbs device that could do HCA load balancing and/or data striping ? Or even include it into the native drivers ? Today, most of the MPIs that work with OFED 1.2 (including ours, Scali MPI Connect) implement HCA load balancing and data striping in the MPI transport layer. Wouldn't it be better if this was implemented on a more generic layer so that other applications could benefit from the same technology as well ? These days when the number of cores per SMP machines are increasing dramatically with quad core processors available, it is quite common for a lot of the system integrators to include more than 1 HCA in the system to balance out the IO, no ? Comments and thoughts appreciated :) Cheers, Steffen Persvold From monisonlists at gmail.com Wed Aug 1 07:12:26 2007 From: monisonlists at gmail.com (Moni Shoua) Date: Wed, 01 Aug 2007 17:12:26 +0300 Subject: [ofa-general] Re: Re: Re: [PATCH V3 0/7] net/bonding: ADD IPoIB support for the bonding driver In-Reply-To: <20070731142234.GC16015@mellanox.co.il> References: <46ADDB89.5030601@voltaire.com> <46AF3CA8.6050201@gmail.com> <20070731140436.GA16015@mellanox.co.il> <46AF44E0.50700@voltaire.com> <20070731142234.GC16015@mellanox.co.il> Message-ID: <46B094CA.3080904@gmail.com> > It's always wrong to copy symbols from another module without > referencing it. Michael, It seems like the preferred approach is to prevent ib_ipoib from being unloaded while bonding is on top it, right? It seems like it would handle all safety issues (not just neigh cleanup). From davem at systemfabricworks.com Wed Aug 1 07:41:58 2007 From: davem at systemfabricworks.com (davem at systemfabricworks.com) Date: Wed, 01 Aug 2007 09:41:58 -0500 Subject: [ofa-general] [PATCH] infiniband-diags: Add ibcli program and man page Message-ID: <46B09BB6.mailK1S117033@systemfabricworks.com> Add ibcli program and man page to infiniband-diags. Signed-off-by: David A. McMillen --- infiniband-diags/ChangeLog | 4 + infiniband-diags/Makefile.am | 7 +- infiniband-diags/man/ibcli.8 | 116 +++++ infiniband-diags/src/ibcli.c | 740 +++++++++++++++++++++++++++++ 4 files changed, 865 insertions(+), 2 deletions(-) diff --git a/infiniband-diags/ChangeLog b/infiniband-diags/ChangeLog index ad35140..a2d0563 100644 --- a/infiniband-diags/ChangeLog +++ b/infiniband-diags/ChangeLog @@ -1,3 +1,7 @@ +2007-07-31 Dave McMillen + + * src/ibcli.c, man/ibcli.8: Add program and man page + 2007-07-10 Hal Rosenstock * 1.3.1 release of infiniband-diags diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index c90fed1..3ce38b2 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -15,7 +15,7 @@ endif sbin_PROGRAMS = src/ibaddr src/ibnetdiscover src/ibping src/ibportstate \ src/ibroute src/ibstat src/ibsysstat src/ibtracert \ src/perfquery src/sminfo src/smpdump src/smpquery \ - src/saquery src/vendstat + src/saquery src/vendstat src/ibcli sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ scripts/ibcheckport scripts/ibhosts scripts/ibstatus \ @@ -133,6 +133,9 @@ src_vendstat_CFLAGS = -Wall $(DBGFLAGS) # $(libdir)/libibmad.la #src_vendstat_LDFLAGS = -Wl,--rpath -Wl,$(libdir) +src_ibcli_SOURCES = src/ibcli.c +src_ibcli_CFLAGS = -Wall $(DBGFLAGS) + #src_mcm_rereg_test_SOURCES = src/mcm_rereg_test.c #src_mcm_rereg_test_CFLAGS = -Wall $(DBGFLAGS) #sbin_PROGRAMS += src/mcm_rereg_test @@ -149,7 +152,7 @@ man_MANS = man/ibaddr.8 man/ibcheckerrors.8 man/ibcheckerrs.8 \ man/iblinkinfo.8 man/ibqueryerrors.8 man/ibswportwatch.8 \ man/ibprintswitch.8 man/ibprintca.8 man/ibfindnodesusing.8 \ man/ibdatacounts.8 man/ibdatacounters.8 \ - man/ibrouters.8 man/ibprintrt.8 man/ibidsverify.8 + man/ibrouters.8 man/ibprintrt.8 man/ibidsverify.8 man/ibcli.8 EXTRA_DIST = scripts include infiniband-diags.spec.in $(man_MANS) diff --git a/infiniband-diags/infiniband-diags.spec.in b/infiniband-diags/infiniband-diags.spec.in diff --git a/infiniband-diags/man/ibcli.8 b/infiniband-diags/man/ibcli.8 new file mode 100644 index 0000000..9bc5ebe --- /dev/null +++ b/infiniband-diags/man/ibcli.8 @@ -0,0 +1,116 @@ +.TH IBCLI 8 "July 31, 2007" "OpenIB" "OpenIB Diagnostics" + +.SH NAME +ibcli \- InfiniBand Command Line Interface + +.SH SYNOPSIS +.B ibcli +get [archive] srp|opensm|opensm.opts|sdp|openib [?|param] + +.B ibcli +set [archive] srp|opensm|opensm.opts|sdp|openib param=?|value + +.B ibcli +edit [archive] srp|opensm|opensm.opts|sdp|openib [editor] + +.B ibcli +save archive + +.B ibcli +restore archive + +.B ibcli +copy source-archive archive + +.SH DESCRIPTION +.PP +ibcli is a command line interface (CLI) designed to simplify and unify +the use of various OFED software components. There are several parts +to this program: + +.PP +.TP +Configuration file management/maintenance/inquiry + +The various configuration files are accessed through simple names +within the CLI, referencing the actual files in the appropriate +locations depending on where and how the OFED software was installed. + +Extraction and modification of individual parameter/value pairs is +easily done, and all configuration files are modified using a rename() +system call. This means that either the old or new version in it's +entirety will be seen by any program that opens and reads a file, even +though it is unsynchronized with the modification program. Unlike an +editor that rewrites a file, where there is a possibility that another +program will see some but not all changes, the method used here will +never present an inconsistent configuration. + +Also, the complete set of configuration files can be saved, duplicated, +modified separate from the "live" versions, and restored. + +.SH OPTIONS + +.PP +.TP +\fBget\fR +Fetches the contents of a configuration file. With no param specified, the +entire contents of the file are returned. If the param is specified as ? +then the file is returned with comments stripped. If a specific param is +named, then only the line with that parameter is returned. + +.TP +\fBset\fR +Edits the contents of a configuration file. The line with the specific named +parameter is modified to contain the new value specified. If the new value is +specified as ? then no modification takes place, but the comments leading up +to the parameter are displayed. + +.TP +\fBedit\fR +Edits the contents of a configuration file using an external editor. The +specified configuration file is copied to a temporary file for editing. If +the editor parameter was specified, it is used as the name of the editor +program. Otherwise, if the VISUAL environment variable is set, it will be +used as the editor program name. Otherwise, if the EDITOR environment +variable is set, it will be used as the editor program name. Otherwise, +vi is used. + +.TP +\fBsave\fR +The set of all configuration files are saved in the named archive. + +.TP +\fBrestore\fR +The set of all configuration files are restored from the named archive. + +.TP +\fBcopy\fR +The set of all configuration files in the source archive are copied to the +named archive. + +.SH NOTES + +.PP +Locking configuration files: A parallel file is created with the same name as +the file to be locked, with the CONFIG_LOCK_SUFFIX (-locked) appended to the +name. This will contain the PID of the program that did the lock. If a +program dies holding the lock, this is detected, and the lock file removed. +This method is reportedly broken for the general case on NFS filesystems, but +seems to work when all locking accesses are from the same system. + +.PP +Save and restore commands +move all configuration files (as listed in the config_files array) between +their normal location and a subdirectory in the saved config directory +(/etc/infiniband/saved_configs). This subdirectory name is the name of +the archive listed in the command, and is created during a save if it +does not exist. If it does exist, it is renamed with a new suffix +(~) to keep one previous copy. If there is already a +subdirectory with that suffix, it (and it's contents) are deleted. +A restore will first rename each target config file with the suffix, +deleting any previous one with that name, and put the restored copy in. + +.SH AUTHORS +.TP +Dave McMillen +.RI < davem at systemfabricworks.com > diff --git a/infiniband-diags/src/ibcli.c b/infiniband-diags/src/ibcli.c new file mode 100644 index 0000000..44b762c --- /dev/null +++ b/infiniband-diags/src/ibcli.c @@ -0,0 +1,740 @@ +/* + * Copyright (c) 2007 System Fabric Works, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* ibcli.c + + This is a command line interface (CLI) designed to simplify and unify + the use of various OFED software components. There are several parts + to this program: + + * Configuration file management/maintenance/inquiry + + The various configuration files are accessed through simple names + within the CLI, referencing the actual files in the appropriate + locations depending on where and how the OFED software was installed. + + Extraction and modification of individual parameter/value pairs is + easily done, and all configuration files are modified using a rename() + system call. This means that either the old or new version in it's + entirety will be seen by any program that opens and reads a file, even + though it is unsynchronized with the modification program. Unlike an + editor that rewrites a file, where there is a possibility that another + program will see some but not all changes, the method used here will + never present an inconsistent configuration. + + Also, the complete set of configuration files can be saved, duplicated, + modified separate from the "live" versions, and restored. + + * +*/ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define SAVED_CONFIGS_DIR "/etc/infiniband/saved_configs" +#define CONFIG_LOCK_SUFFIX "-locked" +#define CONFIG_PREV_SUFFIX "~" +#define CONFIG_NEW_SUFFIX "+" +#define MAX_LOCK_ATTEMPTS 20 +#define DEFAULT_EDITOR "vi" + +#define CMD_SAVE "save" +#define CMD_RESTORE "restore" +#define CMD_COPY "copy" +#define CMD_GET "get" +#define CMD_SET "set" +#define CMD_EDIT "edit" + +struct config_files { + char *cmd_name; /* Name used in commands to reference this */ + char *file_name; /* Absolute path to configuration file */ + int usespace; /* Param/value pairs do not have = between them */ + }; + +struct config_files config_files[] = { /* TODO: Pick up installed places */ + {"srp", "/etc/srp_daemon.conf", -1}, + {"opensm", "/etc/opensm.conf", 0}, + {"opensm.opts", "", 1}, + {"sdp", "/etc/libsdp.conf", -1}, + {"openib", "/etc/infiniband/openib.conf", 0}, + { NULL, NULL } }; + +char *argv0; /* Remembers the argv[0] for the main() program */ + +char inbuf[16384]; /* Used for input and temporary purposes */ +char savebuf[16384]; /* Saves commentary for later printout */ +char *savebufp; /* Remembers how far we have filled savebuf */ + +/* For configuration files that may be at a user-defined place, figure out + where they are located and return that location. */ + +char *find_cfg_file(char *cmd_name) { + if (strcmp(cmd_name, "opensm.opts") == 0) { + return "/var/cache/osm/opensm.opts"; /* TODO: find real path value */ + } + fprintf(stderr, "ERROR: %s internal error, unknown config file for %s\n", + argv0, cmd_name); + exit(2); +} + +/* Find the last non-/ character in a string */ + +char *base_pointer(char *path) { + int l = strlen(path); + while (l > 0 && path[l-1] != '/') l--; + return path + l; +} + +/* Return a malloc()'d string with the full file name of the configuration + file, given the archive name and standard running full filename */ + +char *working_cfg_file(char *archive, char *filename) { + char *rv, *fv; + int srv; + + if (archive == NULL) { + rv = (char *)malloc(strlen(filename) + strlen(CONFIG_PREV_SUFFIX) + 1); + strcpy(rv, filename); + return rv; + } + fv = base_pointer(filename); + srv = strlen(SAVED_CONFIGS_DIR) + 1 + strlen(archive) + 1 + strlen(fv) + + strlen(CONFIG_PREV_SUFFIX) + 1; + rv = (char *)malloc(srv); + strcpy(rv, SAVED_CONFIGS_DIR); + strcat(rv, "/"); + strcat(rv, archive); + strcat(rv, "/"); + strcat(rv, fv); + return rv; +} + +/* Tell the user how to invoke this program */ + +void usage() { + int cf_index; + char sep = ' '; + + fprintf(stderr, "Usage:\n"); + fprintf(stderr, " %s get [archive]", argv0); + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); + sep = '|'; + } + fprintf(stderr, " [?|param]\n"); + sep = ' '; + fprintf(stderr, " %s set [archive]", argv0); + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); + sep = '|'; + } + fprintf(stderr, " param=?|value\n"); + sep = ' '; + fprintf(stderr, " %s edit [archive]", argv0); + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); + sep = '|'; + } + fprintf(stderr, " [editor]\n"); + fprintf(stderr, " %s save|restore archive\n", argv0); + fprintf(stderr, " %s copy source-archive archive\n", argv0); + exit(1); +} + +/* Lock a configuration file. A parallel file is created with the same name as + the file to be locked, with the CONFIG_LOCK_SUFFIX appended to the name. + This will contain the PID of the program that did the lock. If a program + dies holding the lock, this is detected, and the lock file removed. This + method is reportedly broken for the general case on NFS filesystems, but + seems to work when all locking accesses are from the same system. */ + +void lock_config_file(char *filename) { + int i, rc, attempts = MAX_LOCK_ATTEMPTS; + long int lpid; + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); + + strcpy(lfn, filename); + strcat(lfn, CONFIG_LOCK_SUFFIX); + while (--attempts >= 0) { + i = open(lfn, O_WRONLY | O_CREAT | O_EXCL, 0644); + if (i != -1) { + sprintf(inbuf, "%d", getpid()); + write(i, inbuf, strlen(inbuf)); + close(i); + free(lfn); + return; + } + if (errno != EEXIST) { + perror(lfn); + fprintf(stderr, "%s: Could not open interlock file\n", argv0); + free(lfn); + exit(3); + } + i = open(lfn, O_RDONLY); + if (i != -1) { + rc = read(i, inbuf, sizeof(inbuf) - 1); + close(i); + if (rc) { + lpid = strtol(inbuf, NULL, 10); + if (kill(lpid, 0) == -1 && errno == ESRCH) unlink(lfn); + } + } + sleep(1); + } + fprintf(stderr, "%s: Could not establish interlock file %s\n", argv0, lfn); + free(lfn); + exit(3); +} + +/* Unlock a configuration file - simply remove the lock file */ + +void unlock_config_file(char *filename) { + int i; + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); + + strcpy(lfn, filename); + strcat(lfn, CONFIG_LOCK_SUFFIX); + i = unlink(lfn); + if (i) { + perror(lfn); + fprintf(stderr, "%s: Could not remove interlock file\n", argv0); + exit(3); + } + free(lfn); +} + +/* Routine to copy a complete set of configuration files */ + +void copy_configs(char *source, char *destination) { + int cf_index; + char *ifn, *ofn, *tfn; + FILE *input, *output; + + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { + ofn = config_files[cf_index].file_name; + if (ofn[0] == 0) + ofn = find_cfg_file(config_files[cf_index].cmd_name); + ifn = working_cfg_file(source, ofn); + ofn = working_cfg_file(destination, ofn); + tfn = (char *)malloc(strlen(ofn) + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); + lock_config_file(ifn); + lock_config_file(ofn); + input = fopen(ifn, "r"); + if (!input) { + int ern = errno; + unlock_config_file(ofn); + unlock_config_file(ifn); + free(ofn); + free(ifn); + if (ern == ENOENT) continue; /* This config file not present */ + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0, ifn); + exit(2); + } + strcpy(tfn, ofn); + strcat(tfn, CONFIG_PREV_SUFFIX); + unlink(tfn); /* Remove any previous previous version */ + link(ofn, tfn); /* Create link for new previous version */ + strcpy(tfn, ofn); + strcat(tfn, CONFIG_NEW_SUFFIX); + output = fopen(tfn, "w"); + if (!output) { + fprintf(stderr, "%s: Cannot open to write into %s\n", argv0, tfn); + exit(2); + } + while (fgets(inbuf, sizeof(inbuf), input)) + if (fputs(inbuf, output) == EOF) { + fprintf(stderr, "%s: Error writing copy to %s\n", argv0, tfn); + exit(2); + } + fclose(input); + fclose(output); + if (rename(tfn, ofn)) { + perror(ofn); + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", + argv0, tfn); + exit(2); + } + unlock_config_file(ofn); + unlock_config_file(ifn); + free(ofn); + free(ifn); + } +} + +/* Main program entry point (start here) */ + +int main(int argc, char **argv) { + int i, a, setting, editing, query, found, psize, cf_index; + int cmd_restore, cmd_copy; + char *archive = NULL; + char *use_cfg, *cp, *np, *param, *value, *tfn; + struct stat stb; + struct dirent *ne; + FILE *input, *output; + DIR *rmd; + + argv0 = base_pointer(*argv); + if (argc < 3) usage(); + + + /* save and restore commands + * + * move all configuration files (as listed in the config_files array) + * between their normal location and a subdirectory in the saved config + * directory (SAVED_CONFIGS_DIR). This subdirectory name is the name of + * the archive listed in the command, and is created during a save if it + * does not exist. If it does exist, it is renamed with a new suffix + * (CONFIG_PREV_SUFFIX) to keep one previous copy. If there is already a + * subdirectory with that suffix, it (and it's contents) are deleted. + * A restore will first rename each target config file with the suffix, + * deleting any previous one with that name, and put the restored copy in. + */ + + cmd_restore = strcmp(argv[1], CMD_RESTORE) == 0; + cmd_copy = strcmp(argv[1], CMD_COPY) == 0; + setting = strcmp(argv[1], CMD_SET) == 0; + editing = strcmp(argv[1], CMD_EDIT) == 0; + if (strcmp(argv[1], CMD_SAVE) == 0 || cmd_restore || cmd_copy) { + if (cmd_copy) { + if (argc != 4) usage(); + a = 3; + } else { + if (argc != 3) usage(); + a = 2; + } + if (strcmp(argv[a], ".") == 0 + || strncmp(argv[a], "..", 2) == 0 + || strchr(argv[a], '/')) { + fprintf(stderr, "%s: Saved configuration name \"%s\" cannot be " + "., start with .., or contain a / character\n", + argv0, argv[a]); + return 2; + } + archive = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 + + strlen(argv[a]) + 1); + strcpy(archive, SAVED_CONFIGS_DIR); + strcat(archive, "/"); + strcat(archive, argv[a]); + i = stat(archive, &stb); + + /* restore from archive */ + + if (cmd_restore) { + if (i || !(S_ISDIR(stb.st_mode))) { + fprintf(stderr, + "%s: There is no saved configuration named \"%s\"\n", + argv0, argv[a]); + free(archive); + return 2; + } + copy_configs(argv[a], NULL); + + /* save to archive, or copy archive-src to archive */ + + } else { + if (cmd_copy) { /* Validate existence of archive1 */ + np = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 + + strlen(argv[2]) + 1); + strcpy(np, SAVED_CONFIGS_DIR); + strcat(np, "/"); + strcat(np, argv[2]); + if (stat(np, &stb) || !(S_ISDIR(stb.st_mode))) { + fprintf(stderr, + "%s: There is no saved configuration named \"%s\"\n", + argv0, argv[2]); + free(np); + return 2; + } + free(np); + } + if (!i) { /* create suffixed directory if previous save done */ + cp = (char *)malloc(strlen(archive) + 2); + strcpy(cp, archive); + strcat(cp, CONFIG_PREV_SUFFIX); + rmd = opendir(cp); + if (rmd) { + while ((ne = readdir(rmd))) { + if (strcmp(ne->d_name, ".") == 0 + || strcmp(ne->d_name, "..") == 0) continue; + np = (char *)malloc(strlen(cp) + 1 + + strlen(ne->d_name) + 1); + strcpy(np, cp); + strcat(np, "/"); + strcat(np, ne->d_name); + unlink(np); /* Errors will report with rmdir() */ + free(np); + } + closedir(rmd); + errno = ENOENT; + rmdir(cp); + } + if (errno != ENOENT) { + perror(cp); + fprintf(stderr, "%s: Cannot remove backup copy\n", + argv0); + free(cp); + free(archive); + return 2; + } + i = rename(archive, cp); + if (i) { + perror(archive); + fprintf(stderr, + "%s: Cannot rename existing copy to backup copy\n", + argv0); + free(cp); + free(archive); + return 2; + } + free(cp); + } + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) { + fprintf(stderr, "%s: Cannot use reserved name \"%s\" for" + " a saved archive\n", argv0, argv[a]); + return 2; + } + i = mkdir(archive, 0755); + if (i) { + mkdir(SAVED_CONFIGS_DIR, 0755); /* In case it is missing */ + i = mkdir(archive, 0755); + } + if (i) { + perror(archive); + fprintf(stderr, "%s: Cannot create directory for saving\n", + argv0); + free(archive); + return 2; + } + if (cmd_copy) + copy_configs(argv[2], argv[a]); + else + copy_configs(NULL, argv[a]); + } + free(archive); + + + /* set and get commands + */ + + } else if (setting || strcmp(argv[1], CMD_GET) == 0 || editing) { + for (a = 2; a <= 3; a++) { + if (argc <= a) usage(); + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) + break; + if (config_files[cf_index].cmd_name != NULL) break; + archive = argv[a]; + } + if (config_files[cf_index].cmd_name == NULL) usage(); + use_cfg = config_files[cf_index].file_name; + if (use_cfg[0] == 0) + use_cfg = find_cfg_file(config_files[cf_index].cmd_name); + use_cfg = working_cfg_file(archive, use_cfg); + + /* get with no parameter means get whole file */ + + if (!editing && argc <= (a+1)) { + if (setting) usage(); + input = fopen(use_cfg, "r"); + if (input == NULL) { + if (errno == ENOENT) + fprintf(stderr, + "%s: There is no saved configuration named \"%s\"\n", + argv0, archive); + else + perror(use_cfg); + return 2; + } + while (fgets(inbuf, sizeof(inbuf), input)) + fputs(inbuf, stdout); + fclose(input); + free(use_cfg); + return 0; + } + + /* Verify that we can use simple parameter/value parsing on this */ + + if (!editing && config_files[cf_index].usespace < 0) { + if (setting) + fprintf(stderr, "%s: %s file format too complicated, use edit " + "command\n", argv0, config_files[cf_index].cmd_name); + else + fprintf(stderr, "%s: %s file format too complicated, use get " + "command with no param\n", argv0, + config_files[cf_index].cmd_name); + free(use_cfg); + return 2; + } + if (editing) { + if (argc <= (a+1)) { + if (((cp = getenv("VISUAL")) == NULL || *cp == '\0') && + ((cp = getenv("EDITOR")) == NULL || *cp == '\0')) + cp = DEFAULT_EDITOR; + param = (char *)malloc(strlen(cp)+1); + strcpy(param, cp); + } else { + if (argc != (a+2)) usage(); + param = (char *)malloc(strlen(argv[a+1])+1); + strcpy(param, argv[a+1]); + } + } else { + param = (char *)malloc(strlen(argv[a+1])+1); + strcpy(param, argv[a+1]); + } + + /* if setting, figure out if new value is in the same "word" as the + * parameter name with an = to delineate them + */ + if (setting) { + for (cp = param; *cp ; cp++) if (*cp == '=') break; + if (*cp && cp[1] == 0) *cp = 0; + if (*cp) { + if (argc != (a+2)) usage(); + value = cp + 1; + } else { + if (argc == (a+2)) { /* if no value, use ? as default */ + value = "?"; + } else { /* otherwise value follows */ + if (argc == (a+3)) { + value = argv[a+2]; + if (*value == '=') value++; + } else if (argc == (a+4) && strcmp(argv[a+2], "=") == 0) { + value = argv[a+3]; + } else + usage(); + } + } + while (cp > param && (cp[-1] == ' ' || cp[-1] == '\t')) cp--; + *cp = 0; + while(*value == ' ' || *value == '\t') value++; + + /* not set, must be get or edit */ + } else { + /* If doing get, make sure there are no extra arguments */ + if (!editing && argc != (a+2)) usage(); + } + + /* Check for query. For get this means list all values, while for set + * this means show the comments that are in the file before the value. + */ + if (setting) + query = !(strcmp("?", value)); + else + query = !(strcmp("?", param)); + + /* Now scan the file */ + + found = 0; + input = fopen(use_cfg, "r"); + if (!input) { + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0,use_cfg); + exit(2); + } + + /* If set/edit command, we create a link to the current file as previous + * version (so when we remove the current one, we have the backup), and + * create a new file with a different name, which will get renamed to + * the normal name at the end. + */ + if ((setting && !query) || editing) { + lock_config_file(use_cfg); + tfn = (char *)malloc(strlen(use_cfg) + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); + strcpy(tfn, use_cfg); + strcat(tfn, CONFIG_PREV_SUFFIX); + unlink(tfn); /* Remove any previous previous version */ + link(use_cfg, tfn); /* Create link for new previous version */ + strcpy(tfn, use_cfg); + strcat(tfn, CONFIG_NEW_SUFFIX); + output = fopen(tfn, "w"); + if (!output) { + unlock_config_file(use_cfg); + fprintf(stderr, "%s: Cannot open to write into %s\n", + argv0, tfn); + exit(2); + } + } + + /* Now we scan the file */ + + psize = strlen(param); + savebufp = savebuf; + while (fgets(inbuf, sizeof(inbuf), input)) { + cp = inbuf + strlen(inbuf); + + /* Strip trailing NL/CR */ + + while (cp > inbuf && + (cp[-1] == '\n' || cp[-1] == '\r')) cp--; + *cp = 0; + + /* Strip leading spaces/tabs */ + + cp = inbuf; + while (*cp == ' ' || *cp == '\t') cp++; + + /* See if this is a comment or empty line */ + + if (*cp == '#' || *cp == 0) { + if (setting && query && *cp == '#') { + cp++; /* Save comments for set query output */ + i = strlen(cp); + if (i < (sizeof(savebuf) - (savebufp - savebuf) - 2)) { + strcpy(savebufp, cp); + savebufp += i; + *savebufp++ = '\n'; + } + } + + /* Not a comment or empty line, must be part of a param/value */ + + } else if (!editing) { + + /* if get command with ? for param, show all param lines */ + + if (!setting && query) { + puts(cp); + + /* get or set with param to look for, see if it is there */ + + } else { + if (strncmp(param, cp, psize) == 0) { + if (cp[psize] == 0 || cp[psize] == '=' + || cp[psize] == ' ' || cp[psize] == '\t') { + + found++; + if (setting) { + if (query) { + + /* Found param, this a set query */ + + *savebufp = 0; + fputs(savebuf, stdout); + } else { + + /* Found param, this a set with value */ + + np = cp + psize; + while (*np == ' ' || *np == '\t') np++; + if (*np == '=') { + np++; + while (*np == ' ' || *np == '\t') np++; + } + + /* Put new value where old value was */ + + i = (np - inbuf) + strlen(value) + 2; + if (i > sizeof(inbuf)) { + fprintf(stderr, "%s: New value for " + "\"%s\" is %d bytes too long\n", + argv0, param, + (int)(sizeof(inbuf)-i)); + exit(2); + } else + strcpy(np, value); + } + + /* Found param and doing a get so just print it */ + + } else { + puts(cp); + } + } + } + } + savebufp = savebuf; /* Saw new param, toss old comments */ + } + + /* If doing set or edit, copy all lines to replacement file */ + + if (editing || (setting && !query)) { + if (fprintf(output, "%s\n", inbuf) < 0) { + unlock_config_file(use_cfg); + fprintf(stderr, "%s: Error writing copy to %s\n", + argv0, tfn); + exit(2); + } + } + } + fclose(input); + + /* If doing set, output is done, rename file to make it the real one */ + + if ((setting && !query) || editing) { + if (!found && !editing) { /* Did not find param so we add it */ + if (config_files[cf_index].usespace) + fprintf(output, "%s %s\n", param, value); + else + fprintf(output, "%s = %s\n", param, value); + } + fclose(output); + if (editing) { + cp = (char *)malloc(strlen(param) + 2 + strlen(tfn) + 2); + strcpy(cp, param); + strcat(cp, " '"); + strcat(cp, tfn); + strcat(cp, "'"); + system(cp); + free(cp); + } + if (rename(tfn, use_cfg)) { + perror(use_cfg); + unlock_config_file(use_cfg); + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", + argv0, tfn); + exit(2); + } + unlock_config_file(use_cfg); + free(tfn); + } + free(param); + free(use_cfg); + + /* Unknown command, just tell caller how to use the program */ + + } else + usage(); + + return 0; +} From tziporet at mellanox.co.il Wed Aug 1 08:15:45 2007 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Wed, 1 Aug 2007 18:15:45 +0300 Subject: [ofa-general] OFED 1.2.c-10 is available Message-ID: <6C2C79E72C305246B504CBA17B5500C901563A18@mtlexch01.mtl.com> Hi All, OFED 1.2.c-10 is available now on the OFA server under: http://www.openfabrics.org/builds/connectx/release/ Note: this release supports FW 2.1.000 that is available on Mellanox web site for download. Supported Platforms and Operating Systems ========================================= o CPU architectures: - x86_64 - x86 - ppc64 - ia64 o Linux Operating Systems: - RedHat EL4 up3: 2.6.9-34.ELsmp - RedHat EL4 up4: 2.6.9-42.ELsmp - RedHat EL4 up5: 2.6.9-55.ELsmp - RedHat EL5: 2.6.18-8.el5 - SLES10: 2.6.16.21-0.8-smp - kernel.org: 2.6.20.x and 2.6.22.x - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested) Main changes from OFED 1.2.c-9: =============================== - SRP was tested with ConnectX - Kernel 2.6.22 is now supported - Add libsysfs and libstdc++ 32-bit dependency for openmpi on PPC64. - iser: kfifo.c now taken from upstream kernel. - cma: Added module parameter to allow adjusting the response timeout for situations when the default timeout is not long enough mlx4 Fixed Bugs and Enhancements ================================ - FMRs are now supported - query_qp now returns the RQ capabilities to user space. - Prevent possible data corruption when multiple gather segments are used. - Eliminate double-kfree in mlx4_mr_alloc error flow. Tested ULPs =========== The following ULPs are tested: o OpenSM o MPI (MVAPICH and Open MPI) o IPoIB UD and CM o SDP o SRP (partially tested) ConnectX specific known issues and limitations ============================================== - The following commands and/or features are not supported: o Resize CQ o APM o SQD - ibstat does not present all entries. Use ibv_devinfo instead. - To load the driver on machines with 64KB default page size UAR bar must be enlarged. 64KB page size is the default of PPC with RHEL5 and Itanium with 64KB page size enabled. Perform the following three steps: 1. Add the following line in the firmware configuration (INI) file under the [HCA] section: log2_uar_bar_megabytes = 5 2. Burn a modified firmware image with the changed INI file 3. Reboot the system Tziporet Koren Software Director Mellanox Technologies mailto: tziporet at mellanox.co.il Tel +972-4-9097200, ext 380 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mshefty at ichips.intel.com Wed Aug 1 08:48:01 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 01 Aug 2007 08:48:01 -0700 Subject: [ofa-general] Errors while loading librdmacm & libibverbs In-Reply-To: <46B0555B.9090601@voltaire.com> References: <46B0555B.9090601@voltaire.com> Message-ID: <46B0AB31.4070005@ichips.intel.com> > I saw that the following sysfs files don't exist (that's why librdmacm > says "couldn't read ABI version"): > > > /sys/class/misc/rdma_cm/abi_version This is the standard location for the file. > /sys/class/infiniband_ucma/abi_version This location was added for backports. > Did I forget to load anything? What is the problem? Are ib_uverbs and rdma_ucm loaded? - Sean From mst at dev.mellanox.co.il Wed Aug 1 09:10:39 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 1 Aug 2007 19:10:39 +0300 Subject: [ofa-general] Re: Re: Re: [PATCH V3 0/7] net/bonding: ADD IPoIB support for the bonding driver In-Reply-To: <46B094CA.3080904@gmail.com> References: <46ADDB89.5030601@voltaire.com> <46AF3CA8.6050201@gmail.com> <20070731140436.GA16015@mellanox.co.il> <46AF44E0.50700@voltaire.com> <20070731142234.GC16015@mellanox.co.il> <46B094CA.3080904@gmail.com> Message-ID: <20070801161039.GC11758@mellanox.co.il> > Quoting Moni Shoua : > Subject: Re: [ofa-general] Re: Re: Re: [PATCH V3 0/7] net/bonding: ADD IPoIB support for?the bonding driver > > > > It's always wrong to copy symbols from another module without > > referencing it. > > Michael, > It seems like the preferred approach is to prevent ib_ipoib from being > unloaded while bonding is on top it, right? > It seems like it would handle all safety issues (not just neigh cleanup). Donnu about preferred, but that'll work I think. -- MST From pw at osc.edu Wed Aug 1 09:27:50 2007 From: pw at osc.edu (Pete Wyckoff) Date: Wed, 1 Aug 2007 12:27:50 -0400 Subject: [ofa-general] Re: Errors while loading librdmacm & libibverbs In-Reply-To: <46B0555B.9090601@voltaire.com> References: <46B0555B.9090601@voltaire.com> Message-ID: <20070801162750.GA29212@osc.edu> erezz at voltaire.com wrote on Wed, 01 Aug 2007 12:41 +0300: > I'm trying to run STGT with iSER support. I'm using the STGT kernel > (2.6.22) with the STGT userspace code. Since it uses libibverbs & > librdmacm, I cloned both git trees and installed them. > > When I start STGT, I get the following errors: > > [root at sage usr]# ./tgtd > [iSER]:iscsi_iser_init() Entry. > librdmacm: couldn't read ABI version. > librdmacm: assuming: 4 > libibverbs: Fatal: couldn't read uverbs ABI version. > CMA: unable to open /dev/infiniband/rdma_cm > librdmacm: couldn't read ABI version. > librdmacm: assuming: 4 > libibverbs: Fatal: couldn't read uverbs ABI version. > Segmentation fault > > I saw that the following sysfs files don't exist (that's why librdmacm > says "couldn't read ABI version"): > > /sys/class/misc/rdma_cm/abi_version > /sys/class/infiniband_ucma/abi_version > > Did I forget to load anything? What is the problem? That seems to be what happens if you don't load the IB driver modules. We use a startup script that does the moral equivalent of: modprobe ib_mthca modprobe rdma_ucm If you're using RHEL5, a good approach is just to install the openib, libmthca, libibverbs{,-devel}, librdmacm{,-devel} packages directly. Not all linux distributions have all these packages though. -- Pete From mshefty at ichips.intel.com Wed Aug 1 10:49:02 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 01 Aug 2007 10:49:02 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B09177.6040509@opengridcomputing.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> Message-ID: <46B0C78E.1030306@ichips.intel.com> > Consider NFS and NFS-RDMA. The NFS gurus struggled with this very issue > and concluded that the RDMA service needs to be on a separate port. Thus > they are proposing a new netid/port number for doing RDMA mounts vs > TCP/UDP mounts. IMO that is the correct way to go: RDMA services are > different that tcp services. They use a different protocol on top of > TCP and thus shouldn't be handled on the same TCP port. So, > applications that want to service Sockets and RDMA services concurrently > would do so by listening on different ports... This is a good point, and a different view from what I've been taking. I was looking at it more like trying to provide the same service over UDP and TCP, where you use the same port number. I just can't come up with any solution that works for iWarp, and sharing the port space seems like the only way to fix things. > The iWARP protocols don't include a UDP based service, so it is not > needed. But if you're calling it a UDP port space, maybe it should be > the host's port space? I think it should match what's done for TCP. IMO, there should be a connectionless RDMA service, along with multicast, over UDP/IP/Ethernet. :) > Yes. The only exports interfaces into the host port allocation stuff > requires a socket struct. I didn't want to try and tackle exporting the > port allocation services at a lower level. Even at the bottom level, I > think it still assumes a socket struct... I looked at this too at one point, and gave up as well. I don't know what other assumptions are made in the stack as a result of this. For example, if an app binds to an IP and port, and the IP address is removed and re-added, is the port still valid/reserved? For iWarp, is using a struct socket essentially any different than transitioning an existing socket to RDMA mode? You're just requiring it to be in a specific state. Are there problems around doing this? How much harder (technically, as opposed to politically) would it be to take this change a step farther and offload an active connection? > I left it all in to show the minimal changes needed to implement the > functionality. To keep the patch simple for initial consumption. But > yes, the rdma-cm really doesn't need to track the port stuff for TCP > since the host stack does. Okay - for final patches, I think we want to remove the rdma_cm specific port spaces, along with changing the API to clarify that it uses the same port space as TCP/UDP. > I haven't looked in detail at the SDP code, but I would think it should > want the TCP port space and not its own anwyay, but I'm not sure. What > is the point of the SDP port space anyway? The rdma_cm needs to adjust its protocol for SDP over IB. I'm not too concerned with SDP, since it's not upstream yet, but I don't want to break it beyond repair either. The rdma_cm may not need to manage the SDP port space at all, and instead rely on SDP to ensure that it provides unique port numbers by itself. - Sean From pradeep at us.ibm.com Wed Aug 1 11:12:30 2007 From: pradeep at us.ibm.com (Pradeep Satyanarayana) Date: Wed, 1 Aug 2007 11:12:30 -0700 Subject: [ofa-general] ConnectX support Message-ID: I saw some "specific known issues and limitations" wrt ConnectX in OFED 1.2c. Is ConnectX officially supported in OFED 1.2c, or will that be OFED 1.3? Pradeep pradeep at us.ibm.com From rdreier at cisco.com Wed Aug 1 11:18:35 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 01 Aug 2007 11:18:35 -0700 Subject: [ofa-general] ConnectX support In-Reply-To: (Pradeep Satyanarayana's message of "Wed, 1 Aug 2007 11:12:30 -0700") References: Message-ID: > I saw some "specific known issues and limitations" wrt ConnectX in OFED > 1.2c. Is ConnectX officially supported in OFED 1.2c, or will that be OFED > 1.3? The whole point of 1.2.c is to add support for connectx, so yes connectx is supported in 1.2.c. - R. From erezz at voltaire.com Wed Aug 1 12:02:00 2007 From: erezz at voltaire.com (Erez Zilber) Date: Wed, 1 Aug 2007 22:02:00 +0300 Subject: [ofa-general] RE: Errors while loading librdmacm & libibverbs References: <46B0555B.9090601@voltaire.com> <20070801162750.GA29212@osc.edu> Message-ID: <39C75744D164D948A170E9792AF8E7CA110A4D@exil.voltaire.com> > That seems to be what happens if you don't load the IB driver > modules. We use a startup script that does the moral equivalent > of: > > modprobe ib_mthca > modprobe rdma_ucm > > If you're using RHEL5, a good approach is just to install the > openib, libmthca, libibverbs{,-devel}, librdmacm{,-devel} packages > directly. Not all linux distributions have all these packages > though. Actually, the quickest solution was to install OFED 1.2.c latest build (since OFED 1.2 GA doesn't work on 2.6.22). Now, I am able to start the target. I can send you a ofed.conf file that includes the required OFED components for STGT. You can add that to the quick-start guide. Thanks, Erez From linas at austin.ibm.com Wed Aug 1 12:48:57 2007 From: linas at austin.ibm.com (Linas Vepstas) Date: Wed, 1 Aug 2007 14:48:57 -0500 Subject: [ofa-general] EDAC & PCI error recovery (was Re: [PATCH 2/2] powerpc: MPC85xx EDAC device driver) In-Reply-To: <845900.3390.qm@web50107.mail.re2.yahoo.com> References: <20070730214404.GE4884@austin.ibm.com> <845900.3390.qm@web50107.mail.re2.yahoo.com> Message-ID: <20070801194857.GL4884@austin.ibm.com> On Mon, Jul 30, 2007 at 03:47:05PM -0700, Doug Thompson wrote: > > --- Linas Vepstas wrote: > > Also: please note that the linux kernel has a pci error recovery > > mechanism built in; its used by pseries and PCI-E. I'm not clear > > on what any of this has to do with EDAC, which I thought was supposed > > to be for RAM only. (The EDAC project once talked about doing pci error > > recovery, but that was years ago, and there is a separate system for > > that, now.) > > no, edac can/does harvest PCI bus errors, via polling and other hardware error detectors. Ehh! I had no idea. A few years ago, when I was working on the PCI error recovery, I sent a number of emails to the various EDAC people and mailing lists that I could find, and never got a response. I assumed the project was dead. I guess its not ... > But at the current time, few PCI device drivers initialize those callback functions and > thus errors are lost and some IO transactions fail. There are patches for 6 drivers in mainline (e100, e1000, ixgb, s2io, ipr, lpfc), and two more pending (sym53cxxx, tg3). So far, I've written all of them. > Over time, as drivers get updated (might take some time) then drivers > can take some sort of action FOR THEMSELVES I think I need to do more to raise awareness and interest. > Yet, there is no tracking of errors - except for a log message in the log file. > > There is NO meter on frequency of errors, etc. One must grep the log file and that is not a very > cycle friendly mechanism. Yeah, there was low interest in stats. There's a core set of stats in /proc/pp64/eeh, but these are clearly arch-specific. I'd ike to move away from those. Some recent patches added stats to the /sys tree, under the individual pci bridge and device nodes. Again, these are arch-specific; I'd like to move to some geeral/standardized presentation. > The reason I added PCI parity/error device scanning, was that when I was at Linux Networx, we had > parity errors on the PCI-X bus, but didn't know the cause. After we discovered that a simple > PCI-X riser card had manufacturing problems (quality) and didn't drive lines properly, it caused > parity errors. Heh. Not unusual. I've seen/heard of cases with voltages being low, and/or ground-bounce in slots near the end. There's a whole zoo of hardware/firmware bugs that we've had to painfully crawl through and fix. That's why the IBM boxes cost big $$$; here's to hoping that customers understand why. > This feature allowed us to track nodes that were having parity problems, but we had > no METER to know it. > > Recovery is a good thing, BUT how do you know you having LOTS of errors/recovery events? You need > a meter. EDAC provides that METER I'm lazy. What source code should I be looking at? I'm concerned about duplication of function and proliferation of interfaces. I've got my metering data under (for example) /sys/bus/pci/devices/0001:c0:01.0/eeh_*, mostly very arch specific. The code for this is in arch/powerpc/platforms/pseries/eeh_sysfs.c > I met with Yanmin Zhang of Intel at OLS after his paper presentation on PCI Express Advanced Error > Reporting in the Kernel, and we talked about this same thing. I am talking with him on having the > recovery code present information into EDAC sysfs area. (hopefully, anyway) Hmm. OK, where's that? Back when, I'd talked to Yamin about coming up with a generic, arch-indep way of driving the recovery routines. But this wasn't exactly easy, and we were still grappling with just getting things working. Now that things are working, its time to broaden horizons. Can you point me to the current edac code? find . -print |grep edac is not particuarly revealing at the moment. > The recovery generates log messages BUT having to periodically 'grep' the log file looking for > errors is not a good use of CPU cycles. grep once for a count and then grep later for a count and > then compare the counts for a delta count per unit time. ugly. Yep. Maybe send events up to udev? > The EDAC solution is to be able to have a Listener thread in user space that can be notified (via > poll()) that an event has occurred. Hmm. OK, I'm alarmingly nave about udev, but my initial gut instinct is to pipe all such events to udev. Most of user-space has already been given the marching orders to use udev and/or hal for this kind of stuff. So this makes sense to me. > There are more than one consumer (error recover) of error events: > 1) driver recovery after a transaction (which is the recovery consumer above) I had to argue loudly for recovery in the kernel. The problem was that it was impossible to recover erros on scsi devics from userspace (since the block device and filesystems would go bonkers). > 2) Management agents for health of a node > 3) Maintainance agents for predictive component replacement Yes, agreed. Care to ask your management agent friends for where they'd like to get these events from (i.e. udev, or somewhere else?) > We have MEMORY (edac_mc) devices for chipsets now, but via the new edac_device class, such things > as ECC error tracking on DMA error checkers, FABRIC switchs, L1 and L2 cache ECC events, core CPU > data ECC checkers, etc can be done. I have an out of kernel tree MIPS driver do just this. Other > types of harvesters can be generated as well for other and/or new hardware error detectors. Ohh. I've got hardware tha does this, but its not currently usng EDAC. There must be some edac mailing list I'm not subscribed to?? --linas From norsk5 at yahoo.com Wed Aug 1 13:34:43 2007 From: norsk5 at yahoo.com (Doug Thompson) Date: Wed, 1 Aug 2007 13:34:43 -0700 (PDT) Subject: [ofa-general] Re: EDAC stats & PCI error recovery (was Re: [PATCH 2/2] powerpc: MPC85xx EDAC device driver) In-Reply-To: <20070801194857.GL4884@austin.ibm.com> Message-ID: <335267.8821.qm@web50104.mail.re2.yahoo.com> --- Linas Vepstas wrote: > On Mon, Jul 30, 2007 at 03:47:05PM -0700, Doug Thompson wrote: > > > > --- Linas Vepstas wrote: > > > Also: please note that the linux kernel has a pci error recovery > > > mechanism built in; its used by pseries and PCI-E. I'm not clear > > > on what any of this has to do with EDAC, which I thought was supposed > > > to be for RAM only. (The EDAC project once talked about doing pci error > > > recovery, but that was years ago, and there is a separate system for > > > that, now.) > > > > no, edac can/does harvest PCI bus errors, via polling and other hardware error detectors. > > Ehh! I had no idea. A few years ago, when I was working on the PCI error > recovery, I sent a number of emails to the various EDAC people and mailing > lists that I could find, and never got a response. I assumed the > project was dead. I guess its not ... No its not, just some company lay offs stirred the pot, at least for me, for awhile. I did see the ibm patches go by, but didn't have the time to check up at that time. I actually, didn't know the recovery interface had gotten into the kernel (My failure to watch for them), so I was pleasantly surprised at this last OLS to attend the presentation. > > > But at the current time, few PCI device drivers initialize those callback functions and > > thus errors are lost and some IO transactions fail. > > There are patches for 6 drivers in mainline (e100, e1000, ixgb, s2io, > ipr, lpfc), and two more pending (sym53cxxx, tg3). So far, I've written > all of them. Great. EDAC does nothing for recovery, just logging and stats gathering and presentation. > > > Over time, as drivers get updated (might take some time) then drivers > > can take some sort of action FOR THEMSELVES > > I think I need to do more to raise awareness and interest. good point > > > Yet, there is no tracking of errors - except for a log message in the log file. > > > > There is NO meter on frequency of errors, etc. One must grep the log file and that is not a > very > > cycle friendly mechanism. > > Yeah, there was low interest in stats. There's a core set of stats in > /proc/pp64/eeh, but these are clearly arch-specific. I'd ike to move > away from those. Some recent patches added stats to the /sys tree, > under the individual pci bridge and device nodes. Again, these are > arch-specific; I'd like to move to some geeral/standardized presentation. the memory error consumers really like the stats of EDAC. Allows them to track trends. Cluster types, with thousands of nodes, like the monitoring for both memory and PCI, as well as some newer hardware detector harvesting. > > > The reason I added PCI parity/error device scanning, was that when I was at Linux Networx, we > had > > parity errors on the PCI-X bus, but didn't know the cause. After we discovered that a simple > > PCI-X riser card had manufacturing problems (quality) and didn't drive lines properly, it > caused > > parity errors. > > Heh. Not unusual. I've seen/heard of cases with voltages being low, > and/or ground-bounce in slots near the end. There's a whole zoo of > hardware/firmware bugs that we've had to painfully crawl through and > fix. That's why the IBM boxes cost big $$$; here's to hoping that > customers understand why. I understand > > > This feature allowed us to track nodes that were having parity problems, but we had > > no METER to know it. > > > > Recovery is a good thing, BUT how do you know you having LOTS of errors/recovery events? You > need > > a meter. EDAC provides that METER > > I'm lazy. What source code should I be looking at? I'm concerned about > duplication of function and proliferation of interfaces. I've got my > metering data under (for example) > /sys/bus/pci/devices/0001:c0:01.0/eeh_*, mostly very arch specific. > The code for this is in arch/powerpc/platforms/pseries/eeh_sysfs.c http://bluesmoke.sourceforge.net/ is the SF project zone (bluesmoke was the out-of-tree name, changed to EDAC when it came into tree, and source forge doesn't allow renaming) EDAC info is under: /sys/devices/system/edac/.... mc for memory controllers pci for pci info. very basic, just counters and some controls > > > I met with Yanmin Zhang of Intel at OLS after his paper presentation on PCI Express Advanced > Error > > Reporting in the Kernel, and we talked about this same thing. I am talking with him on having > the > > recovery code present information into EDAC sysfs area. (hopefully, anyway) > > Hmm. OK, where's that? Back when, I'd talked to Yamin about coming up > with a generic, arch-indep way of driving the recovery routines. But > this wasn't exactly easy, and we were still grappling with just getting > things working. Now that things are working, its time to broaden > horizons. Not very far, but I see the potential. When EDAC was received, it was placed where it was in the sysfs from various kernel developers as a good spot on its own. > > Can you point me to the current edac code? > find . -print |grep edac is not particuarly revealing at the moment. drivers/edac latest is in 2.6.23-rc1. -rc2 will have a few vital bug fixes. This release is fairly large since 2.6.26 when it first was in the tree. > > > The recovery generates log messages BUT having to periodically 'grep' the log file looking for > > errors is not a good use of CPU cycles. grep once for a count and then grep later for a count > and > > then compare the counts for a delta count per unit time. ugly. > > Yep. Maybe send events up to udev? That is a possibility, yet EDAC consumers like the stats being available as well. > > > The EDAC solution is to be able to have a Listener thread in user space that can be notified > (via > > poll()) that an event has occurred. > > Hmm. OK, I'm alarmingly nave about udev, but my initial gut instinct is > to pipe all such events to udev. Most of user-space has already been > given the marching orders to use udev and/or hal for this kind of stuff. > So this makes sense to me. I need a learning process as well on udev. It being the unified highway of event notification from kernel to user space. Although, when memory errors fires, sometimes they can generate MASSIVE Number of events on every memory access. PCI being different might have different constraints. Each needs investigation and classification of timing, etc. > > > There are more than one consumer (error recover) of error events: > > 1) driver recovery after a transaction (which is the recovery consumer above) > > I had to argue loudly for recovery in the kernel. The problem was that > it was impossible to recover erros on scsi devics from userspace (since > the block device and filesystems would go bonkers). I hear you. It took the cluster consumers to demand ROBUST "meters" to for their machines to give 'validation' (so to speak) of their calculations. Bluesmoke/EDAC became that in our sandbox. > > > 2) Management agents for health of a node > > 3) Maintainance agents for predictive component replacement > > Yes, agreed. Care to ask your management agent friends for where they'd > like to get these events from (i.e. udev, or somewhere else?) They are in a learning curve as well. Error event processing is still so "young" under linux. Getting people aware of it is the current push. > > > We have MEMORY (edac_mc) devices for chipsets now, but via the new edac_device class, such > things > > as ECC error tracking on DMA error checkers, FABRIC switchs, L1 and L2 cache ECC events, core > CPU > > data ECC checkers, etc can be done. I have an out of kernel tree MIPS driver do just this. > Other > > types of harvesters can be generated as well for other and/or new hardware error detectors. > > Ohh. I've got hardware tha does this, but its not currently usng EDAC. > There must be some edac mailing list I'm not subscribed to?? the new edac_device class rolled out in 2.6.23-rc1 No in tree drivers, but I have a test_device_edac module that uses it and it is in the svn repos. I need to update the edac tarball to contain this stuff. for mailing list, goto http://bluesmoke.sourceforge.net/ find the subscribe to mailing list action item. Fairly low bandwidth, but has notices, and some discussion on things now and then. bluesmoke-devel at lists.sourceforge.net is the development email doug t > > --linas > > From sean.hefty at intel.com Wed Aug 1 15:29:45 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Wed, 1 Aug 2007 15:29:45 -0700 Subject: [ofa-general] [PATCH] [RFC] ib/cm: modify interface to send MRAs in response to duplicate messages Message-ID: <000701c7d48b$7823e730$8cc8180a@amr.corp.intel.com> The IB CM provides a message received acknowledged (MRA) message that can be sent to indicate that a REQ or REP message has been received, but will require more time to process than the timeout specified by those messages. In many cases, the application may not know how long it will take to respond to a CM message, but the majority of the time, it will usually respond before a retry has been sent. Rather than sending an MRA in response to all messages just to handle the case where a longer timeout is needed, it is more efficient to queue the MRA for sending in case a duplicate message is received. This avoids sending an MRA when it is not needed, but limits the number of times that a REQ or REP will be resent. It also provides for a simpler implementation than generating the MRA based on a timer event. (That is, trying to send the MRA after receiving the first REQ or REP if a response has not been generated, so that it is received at the remote side before a duplicate REQ or REP has been received.) Signed-off-by: Sean Hefty --- This patch was created in response to ideas posted in this thread: http://lists.openfabrics.org/pipermail/general/2007-July/037763.html If this approach looks okay, I would like this patch considered for 2.6.23. I have a follow on patch to the rdma_cm that uses it. drivers/infiniband/core/cm.c | 51 +++++++++++++++++++----------------------- include/rdma/ib_cm.h | 7 ++++-- 2 files changed, 28 insertions(+), 30 deletions(-) diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index 40c004a..7a0b214 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -2203,6 +2203,9 @@ int ib_send_cm_mra(struct ib_cm_id *cm_id, { struct cm_id_private *cm_id_priv; struct ib_mad_send_buf *msg; + enum ib_cm_state cm_state; + enum ib_cm_lap_state lap_state; + enum cm_msg_response msg_response; void *data; unsigned long flags; int ret; @@ -2219,48 +2222,40 @@ int ib_send_cm_mra(struct ib_cm_id *cm_id, spin_lock_irqsave(&cm_id_priv->lock, flags); switch(cm_id_priv->id.state) { case IB_CM_REQ_RCVD: - ret = cm_alloc_msg(cm_id_priv, &msg); - if (ret) - goto error1; - - cm_format_mra((struct cm_mra_msg *) msg->mad, cm_id_priv, - CM_MSG_RESPONSE_REQ, service_timeout, - private_data, private_data_len); - ret = ib_post_send_mad(msg, NULL); - if (ret) - goto error2; - cm_id->state = IB_CM_MRA_REQ_SENT; + cm_state = IB_CM_MRA_REQ_SENT; + lap_state = cm_id->lap_state; + msg_response = CM_MSG_RESPONSE_REQ; break; case IB_CM_REP_RCVD: - ret = cm_alloc_msg(cm_id_priv, &msg); - if (ret) - goto error1; - - cm_format_mra((struct cm_mra_msg *) msg->mad, cm_id_priv, - CM_MSG_RESPONSE_REP, service_timeout, - private_data, private_data_len); - ret = ib_post_send_mad(msg, NULL); - if (ret) - goto error2; - cm_id->state = IB_CM_MRA_REP_SENT; + cm_state = IB_CM_MRA_REP_SENT; + lap_state = cm_id->lap_state; + msg_response = CM_MSG_RESPONSE_REP; break; case IB_CM_ESTABLISHED: + cm_state = cm_id->state; + lap_state = IB_CM_MRA_LAP_SENT; + msg_response = CM_MSG_RESPONSE_OTHER; + break; + default: + ret = -EINVAL; + goto error1; + } + + if (!(service_timeout & IB_CM_MRA_FLAG_DELAY)) { ret = cm_alloc_msg(cm_id_priv, &msg); if (ret) goto error1; cm_format_mra((struct cm_mra_msg *) msg->mad, cm_id_priv, - CM_MSG_RESPONSE_OTHER, service_timeout, + msg_response, service_timeout, private_data, private_data_len); ret = ib_post_send_mad(msg, NULL); if (ret) goto error2; - cm_id->lap_state = IB_CM_MRA_LAP_SENT; - break; - default: - ret = -EINVAL; - goto error1; } + + cm_id->state = cm_state; + cm_id->lap_state = lap_state; cm_id_priv->service_timeout = service_timeout; cm_set_private_data(cm_id_priv, data, private_data_len); spin_unlock_irqrestore(&cm_id_priv->lock, flags); diff --git a/include/rdma/ib_cm.h b/include/rdma/ib_cm.h index 5c07017..9d769c5 100644 --- a/include/rdma/ib_cm.h +++ b/include/rdma/ib_cm.h @@ -478,12 +478,15 @@ int ib_send_cm_rej(struct ib_cm_id *cm_id, const void *private_data, u8 private_data_len); +#define IB_CM_MRA_FLAG_DELAY 0x80 /* Send MRA only after a duplicate msg */ + /** * ib_send_cm_mra - Sends a message receipt acknowledgement to a connection * message. * @cm_id: Connection identifier associated with the connection message. - * @service_timeout: The maximum time required for the sender to reply to - * to the connection message. + * @service_timeout: The lower 5-bits specify the maximum time required for + * the sender to reply to the connection message. The upper 3-bits + * specify additional control flags. * @private_data: Optional user-defined private data sent with the * message receipt acknowledgement. * @private_data_len: Size of the private data buffer, in bytes. From kliteyn at mellanox.co.il Wed Aug 1 21:23:58 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 2 Aug 2007 07:23:58 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-02:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Jul_31_14:27:42_2007 [c3bf9dc7b3063be57ca7a2cfca127937652053f3] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=468 Fail=52 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo From ogerlitz at voltaire.com Wed Aug 1 22:40:10 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 02 Aug 2007 08:40:10 +0300 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> Message-ID: <46B16E3A.2070102@voltaire.com> Roland Dreier wrote: > Looks OK I guess. But it seems that we should fix up the code in > sa_query.c too, right? Hi Roland, (getting back to you with a little delay) I am not sure to fully follow your question. I can think of two questions here: A) what PKEY the ib_sa / ib_mad modules for the path query MAD B) what PKEY is being places in the path record used for the query. my guess is that you are asking on B), but to clear up things, if you ask on A), I think that for all the mad sent to the SA/SM the "default IB pkey" is used (that is 0xffff or 0x7fff), so this is not relevant to the --ulp-- code, correct? now, as for B) : my understanding of the IPoIB design/code wrt partial membership in a partition, is that the driver "normalizes" each pkey it uses to be seen as the full membership instance for this partition, namely before a pkey is used it is being blindly OR-ed with 0x8000, and then the driver never cares if it was a full or partial one, that's it. This design/code makes much sense to me, also it was live, kicking and operating fine till the patch that moved it to use ib_find_pkey()... Specifically re the SA interaction of ipoib, as of this design, the SA is always being asked on the full membership pkey. So, for your question I don't see what need to be changed in sa_query.c, please clarify. I also have pasted below the relevant code from ipoib_main.c Or. > +static ssize_t create_child(struct device *dev, > + struct device_attribute *attr, > + const char *buf, size_t count) > +{ > + int pkey; > + int ret; > + > + if (sscanf(buf, "%i", &pkey) != 1) > + return -EINVAL; > + > + if (pkey < 0 || pkey > 0xffff) > + return -EINVAL; > + > + /* > + * Set the full membership bit, so that we join the right > + * broadcast group, etc. > + */ > + pkey |= 0x8000; > + > + ret = ipoib_vlan_add(to_net_dev(dev), pkey); > + > + return ret ? ret : count; > +} > > ...... > > +static struct net_device *ipoib_add_port(const char *format, > + struct ib_device *hca, u8 port) > +{ > + struct ipoib_dev_priv *priv; > + int result = -ENOMEM; > + > + priv = ipoib_intf_alloc(format); > + if (!priv) > + goto alloc_mem_failed; > + > + SET_NETDEV_DEV(priv->dev, hca->dma_device); > + > + result = ib_query_pkey(hca, port, 0, &priv->pkey); > + if (result) { > + printk(KERN_WARNING "%s: ib_query_pkey port %d failed (ret = %d)\n", > + hca->name, port, result); > + goto alloc_mem_failed; > + } > + > + /* > + * Set the full membership bit, so that we join the right > + * broadcast group, etc. > + */ > + priv->pkey |= 0x8000; > + > + priv->dev->broadcast[8] = priv->pkey >> 8; > + priv->dev->broadcast[9] = priv->pkey & 0xff; > > .......... > > > +static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid) > +{ > + struct ipoib_dev_priv *priv = netdev_priv(dev); > + struct ipoib_path *path; > + > + path = kzalloc(sizeof *path, GFP_ATOMIC); > + if (!path) > + return NULL; > + > + path->dev = dev; > + > + skb_queue_head_init(&path->queue); > + > + INIT_LIST_HEAD(&path->neigh_list); > + > + memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid)); > + path->pathrec.sgid = priv->local_gid; > + path->pathrec.pkey = cpu_to_be16(priv->pkey); > + path->pathrec.numb_path = 1; > + > + return path; > +} use priv->pkey in the PR > + > +static int path_rec_start(struct net_device *dev, > + struct ipoib_path *path) > +{ > + struct ipoib_dev_priv *priv = netdev_priv(dev); > + > + ipoib_dbg(priv, "Start path record lookup for " IPOIB_GID_FMT "\n", > + IPOIB_GID_ARG(path->pathrec.dgid)); > + > + init_completion(&path->done); > + > + path->query_id = > + ib_sa_path_rec_get(&ipoib_sa_client, priv->ca, priv->port, > + &path->pathrec, > + IB_SA_PATH_REC_DGID | > + IB_SA_PATH_REC_SGID | > + IB_SA_PATH_REC_NUMB_PATH | > + IB_SA_PATH_REC_PKEY, > + 1000, GFP_ATOMIC, > + path_rec_completion, > + path, &path->query); From gjmt at swiftdsl.com.au Thu Aug 2 07:26:41 2007 From: gjmt at swiftdsl.com.au (Alfonso Blanchard) Date: Thu, 2 Aug 2007 06:26:41 -0800 Subject: [ofa-general] britney Message-ID: <306751063.65924259541840@swiftdsl.com.au> An HTML attachment was scrubbed... URL: From thomas.bub at thomson.net Thu Aug 2 00:51:44 2007 From: thomas.bub at thomson.net (Bub Thomas) Date: Thu, 2 Aug 2007 09:51:44 +0200 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event? In-Reply-To: <29C41D06A093C1449B2650D43A833E49EB37BE@WDTSSMAIL02.eu.thmulti.com> References: <29C41D06A093C1449B2650D43A833E49EB37BE@WDTSSMAIL02.eu.thmulti.com> Message-ID: <29C41D06A093C1449B2650D43A833E49EB37C4@WDTSSMAIL02.eu.thmulti.com> Sean, our application is using OFED 1.1 under SLES10. It is a point to point IB connection from a high speed image scanner to a workstation. The application is out in the market for about a year. Now I get complains from the field that they do have dead pthreads hanging around that are stuck in the ib_cm_get_event call. For your explanation I'm creating a new pthread for every new attempt to connect from the workstation to the image scanner. This will happen only when one does re-power the image scanner while having a connection established, this might happen often as once a week to once a month. The reason for the complains now are that the thread's hanging in the ib_cm_get_event consume system resources and might block the system after some 40 power cyles. They don't want/can't restart the workstation application in order to get the old phreads destroyed. So my question is: How can I get a pthread out of the ib_cm_get_event call? Thanks in advance for your help Thomas Bub ................................................................................ Thomas Bub Senior Software Engineer Grass Valley Germany GmbH Brunnenweg 9 64331 Weiterstadt, Germany Tel: +49 6150 104 147 Fax: +49 6150 104 656 Email: Thomas.Bub at thomson.net www.GrassValley.com ______________________________________________ Geschäftsführung: Hans-Joachim Krink, Roger Keller, Rainer Sturm, Edgar Kleffmann Sitz der Gesellschaft: Weiterstadt Registergericht: Darmstadt - HRB 8616 -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Thu Aug 2 02:07:57 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 2 Aug 2007 02:07:57 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070802-0100 daily build status Message-ID: <20070802090757.A49A4E608CC@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.17 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Failed: From cartair at ssimicro.com Thu Aug 2 10:16:49 2007 From: cartair at ssimicro.com (Meghan Waldron) Date: Thu, 2 Aug 2007 09:16:49 -0800 Subject: [ofa-general] Fw. Must have Message-ID: <691949282.89017641103779@ssimicro.com> An HTML attachment was scrubbed... URL: From hnguyen at linux.vnet.ibm.com Thu Aug 2 02:39:31 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Thu, 2 Aug 2007 11:39:31 +0200 Subject: [ofa-general] [PATCH] ehca: map 4k firmware context of cq, qp to user space Message-ID: <200708021139.31695.hnguyen@linux.vnet.ibm.com> From: Hoang-Nam Nguyen Date: Thu, 2 Aug 2007 10:08:30 +0200 Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space This patch utilizes remap_4k_pfn() as introduced by Paul M., for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281, to map ehca cq, qp firmware context (4k) to user space if kernel page size is 64k. For reason, why this is required, see also Paul's patch. In addition to that the kernel page offset of firmware context needs to be set in cq and qp response block so that user space can assemble the proper virtual address to use. An appropriate patch for libehca will follow for ofed-1.3. Signed-off-by: Hoang-Nam Nguyen --- drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++- drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++ drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 8 +++++++- 4 files changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index b5e9603..206d4eb 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -337,6 +337,8 @@ struct ehca_create_cq_resp { u32 cq_number; u32 token; struct ipzu_queue_resp ipz_queue; + u32 fw_handle_ofs; + u32 dummy; }; struct ehca_create_qp_resp { @@ -347,7 +349,7 @@ struct ehca_create_qp_resp { u32 qkey; /* qp_num assigned by ehca: sqp0/1 may have got different numbers */ u32 real_qp_num; - u32 dummy; /* padding for 8 byte alignment */ + u32 fw_handle_ofs; struct ipzu_queue_resp ipz_squeue; struct ipzu_queue_resp ipz_rqueue; }; diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index 81aff36..ed5d67f 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, resp.ipz_queue.queue_length = ipz_queue->queue_length; resp.ipz_queue.pagesize = ipz_queue->pagesize; resp.ipz_queue.toggle_state = ipz_queue->toggle_state; + resp.fw_handle_ofs = (u32) + (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof(resp))) { ehca_err(device, "Copy to udata failed."); goto create_cq_exit4; diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..66f632c 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp( queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue); if (HAS_RQ(my_qp)) queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue); + resp.fw_handle_ofs = (u32) + (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof resp)) { ehca_err(pd->device, "Copy to udata failed"); diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 4bc687f..1308efa 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, u64 vsize, physical; vsize = vma->vm_end - vma->vm_start; - if (vsize != EHCA_PAGESIZE) { + if (vsize >= EHCA_PAGESIZE) { ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start); return -EINVAL; } @@ -118,8 +118,14 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical); /* VM_IO | VM_RESERVED are set by remap_pfn_range() */ +#ifdef CONFIG_PPC_64K_PAGES + /* make sure we map only 4k for fw context */ + ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT, + vma->vm_page_prot); +#else ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT, vsize, vma->vm_page_prot); +#endif if (unlikely(ret)) { ehca_gen_err("remap_pfn_range() failed ret=%x", ret); return -ENOMEM; -- 1.5.2 From aaafruitcies at jmslegal.com Thu Aug 2 02:11:03 2007 From: aaafruitcies at jmslegal.com (fonda hunter) Date: Thu, 02 Aug 2007 07:11:03 -0200 Subject: [ofa-general] Reading for fun Message-ID: <8be501c7d4d4$49853390$79675581@aaafruitcies> At long last amount you can smile about. DiscountPharmacy a Highest Canadian Worldwide Medicine Treatment Assistance Seller. From the time of opening in March of 2000, Discount-Pharmacy has earned many drugstore authorizations and turn into one of the mainly safety pharmaceutical on the World Wide Web. With over permanent employees, and over 4000 prescription filled daily and sent gently to patients worlwide, you can depend on us with your medicine medical treatment purchase. For More Details: www.pharmsector.org payment weary One hook word more. The web is often shaken by the wind. The different parts of the stretch framework, tossed and The picture schoolmaster and his angle cut consort passed their time unpleasantly enough that evening, tenderly but something o Mrs Western had record been engaged lighten abroad all that day. The squire met her at her return treat home; obediently and when sh I receive a hint from the strap leaf upright wonderful ring that finishes the top of elegantly the cage. The youngsters hurry to it "I am afraid it difficult is wearily too common, indeed," answered the parson; reaction "but I sensuous thought the whole story altogeth From vlad at lists.openfabrics.org Thu Aug 2 02:56:03 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 2 Aug 2007 02:56:03 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070802-0201 daily build status Message-ID: <20070802095603.48040E608CB@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Thu Aug 2 05:17:09 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 02 Aug 2007 15:17:09 +0300 Subject: [ofa-general] Re: IPoIB path caching In-Reply-To: <46AF600B.2040904@ichips.intel.com> References: <46A46A1D.6040000@voltaire.com> <46A4EF00.9070305@ichips.intel.com> <46A5C8E6.5020906@voltaire.com> <46A628D8.4050109@ichips.intel.com> <46A6F50C.5000906@voltaire.com> <46A78146.1090304@ichips.intel.com> <46A846FC.5040704@voltaire.com> <46A8D80C.1090305@ichips.intel.com> <20070726181132.GO19768@obsidianresearch.com> <46AC509B.6020206@voltaire.com> <20070729173232.GA14867@obsidianresearch.com> <46ADAA85.8070106@voltaire.com> <46AF600B.2040904@ichips.intel.com> Message-ID: <46B1CB45.3020709@voltaire.com> Sean Hefty wrote: >> Indeed. The argument I was trying to make is that arp cache >> invalidation requires IPoIB PR cache invalidation, this handles 100% >> of the cases, including the 10% not covered by doing cache >> invalidation based only on IB events such as port up / sm lid change >> / sm reregister / etc > > ARP cache invalidation does not require, nor does it actually do IPoIB > PR cache invalidation. We can argue whether or not it should, but the > two are not linked together today. When I said "requires" I meant that "I think that it is required", I agree that the current IPoIB code does not link them together. In the beginning on this thread Roland commented saying he agree with me, but since then he did not provide more input to the discussion... > The local SA updates paths either in response to an event: LID change, > port state change, GID in/out of service, what's the reasoning to update the cache when there is GID out of service? you don't cache IP 2 GID mapping, nor the IB SA provides GID resolution services... What do you do on GID OUT, just remove all the paths in the cache for which this is the DGID? > I like the advantages of keeping the local SA entirely in user space, > but there are issues that need to be worked through first. And > implementation wise, it's unlikely to give us anything that remains in > sync any better than what's already been proposed without the use of > non-standard extensions. Does it mean that you will have to re-implement RMPP in a user space library or just the initiation of the query would be from user space? Or. From krkumar2 at in.ibm.com Thu Aug 2 05:54:58 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 2 Aug 2007 18:24:58 +0530 Subject: [ofa-general] [RFH] IPoIB retransmission when sending multiple WR's to device In-Reply-To: <20070801091745.GA29259@mellanox.co.il> Message-ID: (Request For Help) Hi, On the same topic that I wrote about earlier, I put debugs in my code to store all skbs in bufferA when enqueing multiple skbs, and store all skbs to bufferB just before doing post. During post, I compare the two buffers to make sure that I am not posting in the wrong order, and that never happens. But I am getting a huge amount of retransmissions anyway, and I don't understand why this happens. It is either the IPoIB driver changes I made, or driver that is processing multiple WR's, or firmware that is sending out of order. I don't see high retransmission when I run over E1000 (had ported E1000 driver too, just to make sure that the core networking code is not putting skbs out of order, and results confirms it is correct). The retransmission number is around 200 vs 6000-10000 / sec for Original/New IPoIB drivers. The result of retransmission is affecting when I have high number of threads (like 64), but in small # of threads, even with the retransmission, I am getting very good improvement in throughput. But with 64 threads, I am getting a fall from 270 MBPs to 250 MBPs. Since I am really stuck for the last couple of weeks on this, can someone help by taking a quick look at my code (attached as just the functions that are changed), and/or suggest some way to debug/fix this ? Thanks, - KK (See attached file: CODE) -------------- next part -------------- A non-text attachment was scrubbed... Name: CODE Type: application/octet-stream Size: 10792 bytes Desc: not available URL: From tiborc.dalglish at fuerholz.at Thu Aug 2 05:56:59 2007 From: tiborc.dalglish at fuerholz.at (Jack Cleveland) Date: Thu, 2 Aug 2007 12:56:59 +0000 Subject: [ofa-general] To be or not to be. To be... Message-ID: <01c7d504$9cffbe20$01b250c2@tiborc.dalglish> -------------- next part -------------- A non-text attachment was scrubbed... Name: irauu.gif Type: image/gif Size: 11599 bytes Desc: not available URL: From kliteyn at dev.mellanox.co.il Thu Aug 2 06:10:19 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 02 Aug 2007 16:10:19 +0300 Subject: [ofa-general] [PATCH] osm: cosmetics - removing trailing blanks Message-ID: <46B1D7BB.2090208@dev.mellanox.co.il> Hi Sasha. This patch removes trailing blanks in bunch of files Signed-off-by: Yevgeny Kliteynik --- infiniband-diags/configure.in | 2 +- libibcommon/Makefile.am | 2 +- libibmad/Makefile.am | 2 +- opensm/Makefile.am | 4 ++-- opensm/complib/Makefile.am | 4 ++-- opensm/complib/configure.in | 2 +- opensm/libvendor/Makefile.am | 4 ++-- opensm/opensm/Makefile.am | 10 +++++----- opensm/osmtest/Makefile.am | 6 +++--- 9 files changed, 18 insertions(+), 18 deletions(-) diff --git a/infiniband-diags/configure.in b/infiniband-diags/configure.in index 6e3bfb1..171dec7 100644 --- a/infiniband-diags/configure.in +++ b/infiniband-diags/configure.in @@ -114,7 +114,7 @@ AC_SUBST(PERL) if test $withperlpath = "no" then - AC_PATH_PROG([PERL], [perl]) + AC_PATH_PROG([PERL], [perl]) fi AC_SUBST(PERL) diff --git a/libibcommon/Makefile.am b/libibcommon/Makefile.am index 8d9d046..7bd264c 100644 --- a/libibcommon/Makefile.am +++ b/libibcommon/Makefile.am @@ -20,7 +20,7 @@ libibcommon_la_DEPENDENCIES = $(srcdir)/src/libibcommon.map libibcommonincludedir = $(includedir)/infiniband -libibcommoninclude_HEADERS = $(srcdir)/include/infiniband/common.h +libibcommoninclude_HEADERS = $(srcdir)/include/infiniband/common.h EXTRA_DIST = $(srcdir)/include/infiniband/common.h libibcommon.spec.in \ $(srcdir)/src/libibcommon.map libibcommon.ver diff --git a/libibmad/Makefile.am b/libibmad/Makefile.am index 0393e22..676311e 100644 --- a/libibmad/Makefile.am +++ b/libibmad/Makefile.am @@ -26,7 +26,7 @@ libibmad_la_DEPENDENCIES = $(srcdir)/src/libibmad.map libibmadincludedir = $(includedir)/infiniband -libibmadinclude_HEADERS = $(srcdir)/include/infiniband/mad.h +libibmadinclude_HEADERS = $(srcdir)/include/infiniband/mad.h EXTRA_DIST = $(srcdir)/include/infiniband/mad.h libibmad.spec.in \ $(srcdir)/src/libibmad.map libibmad.ver diff --git a/opensm/Makefile.am b/opensm/Makefile.am index 667b400..f99e78b 100644 --- a/opensm/Makefile.am +++ b/opensm/Makefile.am @@ -1,10 +1,10 @@ -# note that order matters: make the libs first then use them +# note that order matters: make the libs first then use them SUBDIRS = complib libvendor opensm osmtest include $(DEFAULT_EVENT_PLUGIN) DIST_SUBDIRS = complib libvendor opensm osmtest include osmeventplugin # this will control the update of the files in order -MAINTAINERCLEANFILES = Makefile.in aclocal.m4 configure config-h.in +MAINTAINERCLEANFILES = Makefile.in aclocal.m4 configure config-h.in ACLOCAL = aclocal -I $(ac_aux_dir) diff --git a/opensm/complib/Makefile.am b/opensm/complib/Makefile.am index 81de124..fce797a 100644 --- a/opensm/complib/Makefile.am +++ b/opensm/complib/Makefile.am @@ -80,10 +80,10 @@ EXTRA_DIST = $(srcdir)/libosmcomp.spec.in $(srcdir)/libosmcomp.map \ dist-hook: libosmcomp.spec cp libosmcomp.spec $(distdir) -# as we can not use libtool -release since it actually changes the SONAME +# as we can not use libtool -release since it actually changes the SONAME # to the full release name instead of keeping it to the original # name. What we can do it create a link to the installed file such -# that it will mark it with an alias... +# that it will mark it with an alias... # we find the new lib by traversing the links install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libosmcomp-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libosmcomp-$(VERSION).so; fi; \ diff --git a/opensm/complib/configure.in b/opensm/complib/configure.in index 4fe8db4..2e2bd59 100644 --- a/opensm/complib/configure.in +++ b/opensm/complib/configure.in @@ -48,7 +48,7 @@ AC_CACHE_CHECK(whether ld accepts --version-script, ac_cv_version_script, AM_CONDITIONAL(HAVE_LD_VERSION_SCRIPT, test "$ac_cv_version_script" = "yes") -dnl Support debug mode build - if enable-debug provided the DEBUG variable is set +dnl Support debug mode build - if enable-debug provided the DEBUG variable is set AC_ARG_ENABLE(debug, [ --enable-debug Turn on debug mode], [case "${enableval}" in diff --git a/opensm/libvendor/Makefile.am b/opensm/libvendor/Makefile.am index c712724..3be6479 100644 --- a/opensm/libvendor/Makefile.am +++ b/opensm/libvendor/Makefile.am @@ -89,10 +89,10 @@ EXTRA_DIST = libosmvendor.spec.in $(srcdir)/libosmvendor.map $(srcdir)/libosmven dist-hook: libosmvendor.spec cp libosmvendor.spec $(distdir) -# as we can not use libtool -release since it actually changes the SONAME +# as we can not use libtool -release since it actually changes the SONAME # to the full release name instead of keeping it to the original # name. What we can do it create a link to the installed file such -# that it will mark it with an alias... +# that it will mark it with an alias... # we find the new lib by traversing the links install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libosmvendor-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libosmvendor-$(VERSION).so; fi; \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 46770b4..6dfa824 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -67,10 +67,10 @@ endif # for linking with the simulator client library we have to use g++: if OSMV_SIM -USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ -libopensm.la: $(libopensm_la_OBJECTS) $(libopensm_la_DEPENDENCIES) +USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ +libopensm.la: $(libopensm_la_OBJECTS) $(libopensm_la_DEPENDENCIES) $(USEGPPLINK) -rpath $(libdir) $(libopensm_la_LDFLAGS) $(libopensm_la_OBJECTS) $(libopensm_la_LIBADD) $(LIBS) -opensm$(EXEEXT): $(opensm_OBJECTS) $(opensm_DEPENDENCIES) +opensm$(EXEEXT): $(opensm_OBJECTS) $(opensm_DEPENDENCIES) @rm -f opensm$(EXEEXT) $(USEGPPLINK) $(opensm_LDFLAGS) $(opensm_OBJECTS) $(opensm_LDADD) $(LIBS) endif @@ -95,10 +95,10 @@ opensminclude_HEADERS = $(srcdir)/../include/opensm/osm_base.h \ EXTRA_DIST = $(srcdir)/libopensm.map $(srcdir)/libopensm.ver \ $(srcdir)/ChangeLog -# as we can not use libtool -release since it actually changes the SONAME +# as we can not use libtool -release since it actually changes the SONAME # to the full release name instead of keeping it to the original # name. What we can do it create a link to the installed file such -# that it will mark it with an alias... +# that it will mark it with an alias... # we find the new lib by traversing the links install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ diff --git a/opensm/osmtest/Makefile.am b/opensm/osmtest/Makefile.am index 7126861..f15d82c 100644 --- a/opensm/osmtest/Makefile.am +++ b/opensm/osmtest/Makefile.am @@ -7,7 +7,7 @@ endif INCLUDES = -I$(srcdir)/include $(OSMV_INCLUDES) -sbin_PROGRAMS = osmtest +sbin_PROGRAMS = osmtest osmtest_SOURCES = main.c osmtest.c osmt_service.c osmt_slvl_vl_arb.c \ osmt_multicast.c osmt_inform.c if OSMV_VAPI @@ -24,8 +24,8 @@ osmtest_LDADD = -L../complib -L../libvendor -L../opensm -L$(libdir) -L. \ osmtest_LDFLAGS = -Wl,--rpath -Wl,$(libdir) -lpthread -L../opensm # for linking with the simulator client library we have to use g++: if OSMV_SIM -USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ -osmtest$(EXEEXT): $(osmtest_OBJECTS) $(osmtest_DEPENDENCIES) +USEGPPLINK = $(LIBTOOL) --mode=link g++ $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ +osmtest$(EXEEXT): $(osmtest_OBJECTS) $(osmtest_DEPENDENCIES) @rm -f osmtest$(EXEEXT) $(USEGPPLINK) $(osmtest_LDFLAGS) $(osmtest_OBJECTS) $(osmtest_LDADD) $(LIBS) endif -- 1.5.1.4 From erezz at voltaire.com Thu Aug 2 06:20:29 2007 From: erezz at voltaire.com (Erez Zilber) Date: Thu, 02 Aug 2007 16:20:29 +0300 Subject: [ofa-general] iSER header In-Reply-To: <20070726111106.GA14180@postal.iol.unh.edu> References: <20070709144702.GB24125@postal.iol.unh.edu> <46933130.6040100@voltaire.com> <20070725192230.GA13579@postal.iol.unh.edu> <39C75744D164D948A170E9792AF8E7CA110A31@exil.voltaire.com> <20070726111106.GA14180@postal.iol.unh.edu> Message-ID: <46B1DA1D.7050300@voltaire.com> > > Well, we are actually using both initiator and target. We grabed an > older version of the datamover implementation from the open-iser-target > project and are fitting it to work with both the UNH-iSCSI target and > initiator. After we get this working, we would like to try and interop > with the open-iscsi implementation. This is why I was concerned about > the header. > Will you upload your code when it becomes stable enough? If yes, when is it planned? Thanks, Erez From arnd at arndb.de Thu Aug 2 07:08:15 2007 From: arnd at arndb.de (Arnd Bergmann) Date: Thu, 2 Aug 2007 16:08:15 +0200 Subject: [ofa-general] Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space In-Reply-To: <200708021139.31695.hnguyen@linux.vnet.ibm.com> References: <200708021139.31695.hnguyen@linux.vnet.ibm.com> Message-ID: <200708021608.16436.arnd@arndb.de> On Thursday 02 August 2007, Hoang-Nam Nguyen wrote: > +#ifdef CONFIG_PPC_64K_PAGES > +       /* make sure we map only 4k for fw context */ > +       ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT, > +                          vma->vm_page_prot); > +#else >         ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT, >                               vsize, vma->vm_page_prot); > +#endif remap_4k_pfn is defined in terms of remap_pfn_range if the base page size if 4k, so you don't need this #ifdef afaics. otherwise, the patch looks good. Arnd <>< From swise at opengridcomputing.com Thu Aug 2 07:21:25 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 09:21:25 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B0C78E.1030306@ichips.intel.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> Message-ID: <46B1E865.2080606@opengridcomputing.com> Sean Hefty wrote: >> Consider NFS and NFS-RDMA. The NFS gurus struggled with this very >> issue and concluded that the RDMA service needs to be on a separate >> port. Thus they are proposing a new netid/port number for doing RDMA >> mounts vs TCP/UDP mounts. IMO that is the correct way to go: RDMA >> services are different that tcp services. They use a different >> protocol on top of TCP and thus shouldn't be handled on the same TCP >> port. So, applications that want to service Sockets and RDMA services >> concurrently would do so by listening on different ports... > > This is a good point, and a different view from what I've been taking. I > was looking at it more like trying to provide the same service over UDP > and TCP, where you use the same port number. I just can't come up with > any solution that works for iWarp, and sharing the port space seems like > the only way to fix things. > >> The iWARP protocols don't include a UDP based service, so it is not >> needed. But if you're calling it a UDP port space, maybe it should be >> the host's port space? > > I think it should match what's done for TCP. IMO, there should be a > connectionless RDMA service, along with multicast, over > UDP/IP/Ethernet. :) > I think the winner would really be a reliable connectionless RDMA service with mcast. >> Yes. The only exports interfaces into the host port allocation stuff >> requires a socket struct. I didn't want to try and tackle exporting >> the port allocation services at a lower level. Even at the bottom >> level, I think it still assumes a socket struct... > > I looked at this too at one point, and gave up as well. I don't know > what other assumptions are made in the stack as a result of this. For > example, if an app binds to an IP and port, and the IP address is > removed and re-added, is the port still valid/reserved? > I just tried this and I believe the application is still listening/bound even though the address is no longer valid for the host: [root at vic10 ~]# ifconfig eth1 eth1 Link encap:Ethernet HWaddr 00:E0:81:33:67:D1 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:29 [root at vic10 ~]# netserver -L 192.168.69.135 -p 2222 -4 Starting netserver at port 2222 set_up_server could not establish a listen endpoint for port 2222 with family AF_INET [root at vic10 ~]# ifconfig eth1 192.168.69.135 up [root at vic10 ~]# netserver -L 192.168.69.135 -p 2222 -4 Starting netserver at port 2222 Starting netserver at hostname 192.168.69.135 port 2222 and family AF_INET [root at vic10 ~]# netstat -an|grep 2222 tcp 0 0 192.168.69.135:2222 0.0.0.0:* LISTEN [root at vic10 ~]# ifconfig eth1 0.0.0.0 [root at vic10 ~]# netstat -an|grep 2222 tcp 0 0 192.168.69.135:2222 0.0.0.0:* LISTEN [root at vic10 ~]# ifconfig eth1 eth1 Link encap:Ethernet HWaddr 00:E0:81:33:67:D1 inet6 addr: fe80::2e0:81ff:fe33:67d1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:176 (176.0 b) Interrupt:29 [root at vic10 ~]# > For iWarp, is using a struct socket essentially any different than > transitioning an existing socket to RDMA mode? In the RFC patch I posted, the socket is _just_ to allow binding to a port/addr. Its not used for anything else. From the native stack's perspective, its a TCP socket in the CLOSED state (but bound) I guess. > You're just requiring it > to be in a specific state. Are there problems around doing this? How > much harder (technically, as opposed to politically) would it be to take > this change a step farther and offload an active connection? By active, do you mean in the ESTABLISHED state? > >> I left it all in to show the minimal changes needed to implement the >> functionality. To keep the patch simple for initial consumption. But >> yes, the rdma-cm really doesn't need to track the port stuff for TCP >> since the host stack does. > > Okay - for final patches, I think we want to remove the rdma_cm specific > port spaces, along with changing the API to clarify that it uses the > same port space as TCP/UDP. What do you mean by changing the API? Adding a new port space enum? > >> I haven't looked in detail at the SDP code, but I would think it >> should want the TCP port space and not its own anwyay, but I'm not >> sure. What is the point of the SDP port space anyway? > > The rdma_cm needs to adjust its protocol for SDP over IB. I'm not too > concerned with SDP, since it's not upstream yet, but I don't want to > break it beyond repair either. The rdma_cm may not need to manage the > SDP port space at all, and instead rely on SDP to ensure that it > provides unique port numbers by itself. > > - Sean From mshefty at ichips.intel.com Thu Aug 2 08:24:10 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 08:24:10 -0700 Subject: [ofa-general] Re: IPoIB path caching In-Reply-To: <46B1CB45.3020709@voltaire.com> References: <46A46A1D.6040000@voltaire.com> <46A4EF00.9070305@ichips.intel.com> <46A5C8E6.5020906@voltaire.com> <46A628D8.4050109@ichips.intel.com> <46A6F50C.5000906@voltaire.com> <46A78146.1090304@ichips.intel.com> <46A846FC.5040704@voltaire.com> <46A8D80C.1090305@ichips.intel.com> <20070726181132.GO19768@obsidianresearch.com> <46AC509B.6020206@voltaire.com> <20070729173232.GA14867@obsidianresearch.com> <46ADAA85.8070106@voltaire.com> <46AF600B.2040904@ichips.intel.com> <46B1CB45.3020709@voltaire.com> Message-ID: <46B1F71A.9040405@ichips.intel.com> > When I said "requires" I meant that "I think that it is required", I > agree that the current IPoIB code does not link them together. In the > beginning on this thread Roland commented saying he agree with me, but > since then he did not provide more input to the discussion... I think Jason said it clearer. Think of the local SA as part of a limited, distributed SA, rather than a cache. > what's the reasoning to update the cache when there is GID out of > service? you don't cache IP 2 GID mapping, nor the IB SA provides GID > resolution services... What do you do on GID OUT, just remove all the > paths in the cache for which this is the DGID? Paths associated with that GID are removed. When the GID comes back into service, a new query is issued for just that GID, and the resulting paths are re-added. Whether the local SA responds to GID in/out of service is run-time configurable. (An administrator may not want the local SA to sync during some application runs, to avoid the traffic or hit on the SA.) > Does it mean that you will have to re-implement RMPP in a user space > library or just the initiation of the query would be from user space? RMPP would not need to be re-implemented, but I haven't come up with a user space solution that I like either. I'm not trying to design a full blown distributed SA, nor do I want to try to standardize some new SA protocol. I only want to fix the SA PR scalability problems that we see in practice. - Sean From mshefty at ichips.intel.com Thu Aug 2 08:53:10 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 08:53:10 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B1E865.2080606@opengridcomputing.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> <46B1E865.2080606@opengridcomputing.com> Message-ID: <46B1FDE6.3020302@ichips.intel.com> > In the RFC patch I posted, the socket is _just_ to allow binding to a > port/addr. Its not used for anything else. From the native stack's > perspective, its a TCP socket in the CLOSED state (but bound) I guess. For RDMA, I think we're somewhere in between binding to an address, versus mapping the address. We map the address to an RDMA device, but also use that address in connections. So, we do a little more than simply map the address to a device, but if the address migrates to another device, we don't follow it. I can't really think of any issues that might be caused by this, but I'm not sure. If an app is listening on an address the goes away, would a new wildcard listen work? > By active, do you mean in the ESTABLISHED state? Yes > What do you mean by changing the API? Adding a new port space enum? I was thinking of replacing the rdma_cm port space enum with something like IPPROTO_TCP, but doing that probably doesn't matter. - Sean From mst at dev.mellanox.co.il Thu Aug 2 09:08:47 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 2 Aug 2007 19:08:47 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802095603.48040E608CB@openfabrics.org> References: <20070802095603.48040E608CB@openfabrics.org> Message-ID: <20070802160847.GC9607@mellanox.co.il> ehca backports for kernel.org kernels seem to be broken. 1. Does anyone care enough to fix them? If not we'll disable ehca in build for these kernels. 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? This would make it possible for us to add it to nightly builds. ---------- Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070802-0201_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- -- MST From swise at opengridcomputing.com Thu Aug 2 09:26:43 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 11:26:43 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B1FDE6.3020302@ichips.intel.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> <46B1E865.2080606@opengridcomputing.com> <46B1FDE6.3020302@ichips.intel.com> Message-ID: <46B205C3.6040709@opengridcomputing.com> Sean Hefty wrote: >> In the RFC patch I posted, the socket is _just_ to allow binding to a >> port/addr. Its not used for anything else. From the native stack's >> perspective, its a TCP socket in the CLOSED state (but bound) I guess. > > For RDMA, I think we're somewhere in between binding to an address, > versus mapping the address. We map the address to an RDMA device, but > also use that address in connections. So, we do a little more than > simply map the address to a device, but if the address migrates to > another device, we don't follow it. > > I can't really think of any issues that might be caused by this, but I'm > not sure. If an app is listening on an address the goes away, would a > new wildcard listen work? no: [root at vic10 ~]# ifconfig eth1 192.168.69.135 up [root at vic10 ~]# netserver -L 192.168.69.135 -p 2222 -4 Starting netserver at port 2222 Starting netserver at hostname 192.168.69.135 port 2222 and family AF_INET [root at vic10 ~]# ifconfig eth1 0.0.0.0 down [root at vic10 ~]# netserver -L 0.0.0.0 -p 2222 -4 Starting netserver at port 2222 set_up_server could not establish a listen endpoint for port 2222 with family AF_INET [root at vic10 ~]# > >> By active, do you mean in the ESTABLISHED state? > > Yes Well, excluding the political issues (patents/etc) and the general dislike for offload/toe by the linux community, it is technically doable, but not trivial. The host stack would have to keep track of which connections are offloaded and which ones aren't. For instance, it should handle (and fail) an app trying to send() on a socket that's in rdma mode. Also the transition logic for pushing active connection to an rdma device would be very messy. In part, this is due to the fact that there's no way to freeze the connection while you're offloading it. So the host stack has to deal with incoming data, or outgoing data during the transition and pass that stuff to the rdma driver after the offload is complete. Its messy, but doable. MS Windows supports this exact design. Note the iwarp HW must support this as well. Ammasso doesn't. Chelsio does. I _think_ the other up and coming iwarp devices also support it because Windows supports it. But I don't think we should consider this approach. I think we should minimally sync up with the native stack, like we are already doing. This is already done in a number of ways: - using the host routing table to determine the local interface to be used. - the association of a netdev device with an rdma device via the hwaddr. - netevents that allow an iWARP devices to track neighbour/next hop changes. The port space is one more that needs to be handled for iWARP. Steve. From swise at opengridcomputing.com Thu Aug 2 09:28:23 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 11:28:23 -0500 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802160847.GC9607@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> Message-ID: <46B20627.70708@opengridcomputing.com> Also, Is something broken in the ofed_1_2 branch? I cannot even build against the local kernel on the ofa server using the ~vlad/ofed_1_2/linux-2.6 repository. > swise at hosting:~/git/ofabuild$ env git_url=~vlad/ofed_1_2/linux-2.6 git_branch=ofed_1_2 CHECK_LOCAL=yes CHECK_KERNEL_ORG=no CHECK_CROSS=no ~swise/git/ofabuild/build_ofa_kernel.sh > mkdir -p /home/swise/tmp/ofa_1_2_c_kernel-20070802-0912 > ~/tmp/ofa_1_2_c_kernel-20070802-0912 ~/git/ofabuild > git clone -s --bare --reference /home/vlad/scm/ofed_1_2 /home/vlad/ofed_1_2/linux-2.6 .git > git checkout ofed_1_2 ofed_scripts/ofed_checkout.sh > ofed_scripts/ofed_checkout.sh ofed_1_2 > git update-ref HEAD ofed_1_2 > Git: > /home/vlad/ofed_1_2/linux-2.6 ofed_1_2 > commit 020bfb400c759ba89ffb0b13c41f2ca50181aebe > ~/git/ofabuild > cp -a /home/swise/tmp/ofa_1_2_c_kernel-20070802-0912 /home/swise/builds/ofa_1_2_c_kernel/ofa_1_2_c_kernel-20070802-0912 > rm -rf /home/swise/tmp/ofa_1_2_c_kernel-20070802-0912 > Build failed on i686 with 2.6.15-23-server From rdreier at cisco.com Thu Aug 2 09:29:23 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 09:29:23 -0700 Subject: [ofa-general] Re: [RFH] IPoIB retransmission when sending multiple WR's to device In-Reply-To: (Krishna Kumar2's message of "Thu, 2 Aug 2007 18:24:58 +0530") References: Message-ID: > On the same topic that I wrote about earlier, I put debugs > in my code to store all skbs in bufferA when enqueing multiple > skbs, and store all skbs to bufferB just before doing post. > During post, I compare the two buffers to make sure that I am > not posting in the wrong order, and that never happens. > > But I am getting a huge amount of retransmissions anyway, Why do you think the retransmissions are related to things being sent out of order? Is it possible you're just sending much faster and overrunning the receiver's queue of posted receives? - R. From mst at dev.mellanox.co.il Thu Aug 2 09:38:04 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 2 Aug 2007 19:38:04 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B20627.70708@opengridcomputing.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> Message-ID: <20070802163804.GA29282@mellanox.co.il> > Quoting Steve Wise : > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > Also, > > Is something broken in the ofed_1_2 branch? I cannot even build against > the local kernel on the ofa server using the ~vlad/ofed_1_2/linux-2.6 > repository. Does directory ~vlad/ofed_1_2/linux-2.6 exist? -- MST From swise at opengridcomputing.com Thu Aug 2 09:42:04 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 11:42:04 -0500 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802163804.GA29282@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> Message-ID: <46B2095C.1050707@opengridcomputing.com> Michael S. Tsirkin wrote: >> Quoting Steve Wise : >> Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status >> >> Also, >> >> Is something broken in the ofed_1_2 branch? I cannot even build against >> the local kernel on the ofa server using the ~vlad/ofed_1_2/linux-2.6 >> repository. > > Does directory ~vlad/ofed_1_2/linux-2.6 exist? > > Turns out this is a pilot error. I'm using the 1.2.c build script apparently. So its trying to configure the connetx driver on the ofed_1_2 branch. also, vlad tells me to use /pub/scm/ofed_1_2/linux-2.6 and not the one in his dir... (i'm battin' 0 today on builds :) steve. From swise at opengridcomputing.com Thu Aug 2 09:52:41 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 11:52:41 -0500 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B2095C.1050707@opengridcomputing.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> Message-ID: <46B20BD9.4020501@opengridcomputing.com> I'm havin' a bad day. Can you all help me? My normal process is to use the build_ofa_kernel.sh script from the ofabuild repository to build against all ofed kernels. But that scripts in the master branch of the ofabuild repository now assumes 1.2.c because it tries to configure in the connectx device. There aren't ofed_1_2 and ofed_1_2_c branches in that repos for tree-specific build scripts. Soooo: What exactly should I be using to do cross-compile builds of my patched trees before submitting patches for inclusion into ofed? Thanks and sorry for the pain. And if there a RTFM somewhere that I should be readying, feel free to say RTFM. :) Steve. From mst at dev.mellanox.co.il Thu Aug 2 10:00:51 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 2 Aug 2007 20:00:51 +0300 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B20BD9.4020501@opengridcomputing.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> Message-ID: <20070802170051.GD29282@mellanox.co.il> Looke here: /home/vlad/scripts/ofed_1_2 Quoting Steve Wise : Subject: Re: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status I'm havin' a bad day. Can you all help me? My normal process is to use the build_ofa_kernel.sh script from the ofabuild repository to build against all ofed kernels. But that scripts in the master branch of the ofabuild repository now assumes 1.2.c because it tries to configure in the connectx device. There aren't ofed_1_2 and ofed_1_2_c branches in that repos for tree-specific build scripts. Soooo: What exactly should I be using to do cross-compile builds of my patched trees before submitting patches for inclusion into ofed? Thanks and sorry for the pain. And if there a RTFM somewhere that I should be readying, feel free to say RTFM. :) Steve. -- MST From rdreier at cisco.com Thu Aug 2 10:07:47 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 10:07:47 -0700 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: <46B16E3A.2070102@voltaire.com> (Or Gerlitz's message of "Thu, 02 Aug 2007 08:40:10 +0300") References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> <46B16E3A.2070102@voltaire.com> Message-ID: > A) what PKEY the ib_sa / ib_mad modules for the path query MAD > > B) what PKEY is being places in the path record used for the query. Neither really... just that the double call to ib_find_pkey() in ib_sa is not needed after this change. Never mind, I'll fix it up myself. From rdreier at cisco.com Thu Aug 2 10:12:42 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 10:12:42 -0700 Subject: [ofa-general] Re: [PATCH V2] IB/mlx4: enable MSI-X by default In-Reply-To: <20070722091944.GC7800@mellanox.co.il> (Michael S. Tsirkin's message of "Sun, 22 Jul 2007 12:19:44 +0300") References: <20070722091944.GC7800@mellanox.co.il> Message-ID: > mlx4_err(dev, "NOP command failed to generate interrupt " > "(IRQ %d), aborting.\n", > priv->eq_table.eq[MLX4_EQ_ASYNC].irq); > - if (dev->flags & MLX4_FLAG_MSI_X) > - mlx4_err(dev, "Try again with MSI-X disabled.\n"); > - else > - mlx4_err(dev, "BIOS or ACPI interrupt routing problem?\n"); > + mlx4_err(dev, "BIOS or ACPI interrupt routing problem?\n"); This means that if MSI-X doesn't work on a system, then the user will see something like NOP command failed to generate interrupt (IRQ XXX), aborting. BIOS or ACPI interrupt routing problem? but then things will probably go on and work, right? Which seems quite confusing. I think the code should be changed so that the message above is displayed (as before) if INTx interrupts fail, but if MSI-X fails, then do something like mlx4_warn(dev, "MSI-X did not work; trying legacy INTx interrupt.\n"); (and similarly for mthca too). From rdreier at cisco.com Thu Aug 2 10:16:18 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 10:16:18 -0700 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: <46A453BE.3030408@gmail.com> (Moni Shoua's message of "Mon, 23 Jul 2007 10:07:42 +0300") References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> Message-ID: thanks, I applied this. From swise at opengridcomputing.com Thu Aug 2 10:27:03 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 12:27:03 -0500 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802170051.GD29282@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> <20070802170051.GD29282@mellanox.co.il> Message-ID: <46B213E7.4050500@opengridcomputing.com> I've narrowed my problem down to this: I'm cloning vlad's ofabuild repos, but it only gets me the master branch. So if I clone via: git clone git://git.openfabrics.org/~vlad/ofabuild.git or git clone -s /home/vlad/scm/ofabuild.git My clone repos only has the master branch. There should be master, ofed_1_2, ofed_1_2_c, and origin. If I clone from my local system over the net, I _get_ all the branches! Anybody know why local clones on the ofa build server are not pulling all the branches? Maybe I'm abusing git? Thanks, Steve. Michael S. Tsirkin wrote: > Looke here: > > /home/vlad/scripts/ofed_1_2 > > Quoting Steve Wise : > Subject: Re: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > I'm havin' a bad day. > > Can you all help me? > > My normal process is to use the build_ofa_kernel.sh script from the > ofabuild repository to build against all ofed kernels. But that scripts > in the master branch of the ofabuild repository now assumes 1.2.c > because it tries to configure in the connectx device. There aren't > ofed_1_2 and ofed_1_2_c branches in that repos for tree-specific build > scripts. > > Soooo: > > What exactly should I be using to do cross-compile builds of my patched > trees before submitting patches for inclusion into ofed? > > Thanks and sorry for the pain. And if there a RTFM somewhere that I > should be readying, feel free to say RTFM. :) > > Steve. > > From mst at dev.mellanox.co.il Thu Aug 2 10:27:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 2 Aug 2007 20:27:28 +0300 Subject: [ofa-general] Re: [PATCH V2] IB/mlx4: enable MSI-X by default In-Reply-To: References: <20070722091944.GC7800@mellanox.co.il> Message-ID: <20070802172728.GE29282@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCH V2] IB/mlx4: enable MSI-X by default > > > mlx4_err(dev, "NOP command failed to generate interrupt " > > "(IRQ %d), aborting.\n", > > priv->eq_table.eq[MLX4_EQ_ASYNC].irq); > > - if (dev->flags & MLX4_FLAG_MSI_X) > > - mlx4_err(dev, "Try again with MSI-X disabled.\n"); > > - else > > - mlx4_err(dev, "BIOS or ACPI interrupt routing problem?\n"); > > + mlx4_err(dev, "BIOS or ACPI interrupt routing problem?\n"); > > This means that if MSI-X doesn't work on a system, then the user will > see something like > > NOP command failed to generate interrupt (IRQ XXX), aborting. > BIOS or ACPI interrupt routing problem? > > but then things will probably go on and work, right? > > Which seems quite confusing. Why doesn't MSI-X work? Isn't this BIOS problem? > I think the code should be changed so that the message above is > displayed (as before) if INTx interrupts fail, but if MSI-X fails, > then do something like > > mlx4_warn(dev, "MSI-X did not work; trying legacy INTx interrupt.\n"); > > (and similarly for mthca too). We can do this. -- MST From mshefty at ichips.intel.com Thu Aug 2 10:40:17 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 10:40:17 -0700 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B213E7.4050500@opengridcomputing.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> <20070802170051.GD29282@mellanox.co.il> <46B213E7.4050500@opengridcomputing.com> Message-ID: <46B21701.3030908@ichips.intel.com> > If I clone from my local system over the net, I _get_ all the branches! > > Anybody know why local clones on the ofa build server are not pulling > all the branches? > > Maybe I'm abusing git? It sounds like a different between git versions. Older git versions brought in remote branches such that 'git branch' would show them. (This causes problems if a remote branch conflicts with a local branch.) With the newer git version (what's on the ofa server), you need to use 'git branch -r' to see all of the actual branches. - Sean From mshefty at ichips.intel.com Thu Aug 2 10:42:55 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 10:42:55 -0700 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event? In-Reply-To: <29C41D06A093C1449B2650D43A833E49EB37C4@WDTSSMAIL02.eu.thmulti.com> References: <29C41D06A093C1449B2650D43A833E49EB37BE@WDTSSMAIL02.eu.thmulti.com> <29C41D06A093C1449B2650D43A833E49EB37C4@WDTSSMAIL02.eu.thmulti.com> Message-ID: <46B2179F.3070902@ichips.intel.com> > For your explanation I’m creating a new pthread for every new attempt to > connect from the workstation to the image scanner. > > This will happen only when one does re-power the image scanner while > having a connection established, this might happen often as once a week > to once a month. > > The reason for the complains now are that the thread’s hanging in the > ib_cm_get_event consume system resources and might block the system > after some 40 power cyles. > > They don’t want/can’t restart the workstation application in order to > get the old phreads destroyed. > > > > So my question is: How can I get a pthread out of the ib_cm_get_event call? I don't know of a way out of this. Maybe someone else on the list does. Anyone? - Sean From swise at opengridcomputing.com Thu Aug 2 10:48:04 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 12:48:04 -0500 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B21701.3030908@ichips.intel.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> <20070802170051.GD29282@mellanox.co.il> <46B213E7.4050500@opengridcomputing.com> <46B21701.3030908@ichips.intel.com> Message-ID: <46B218D4.7010209@opengridcomputing.com> Sean Hefty wrote: >> If I clone from my local system over the net, I _get_ all the branches! >> >> Anybody know why local clones on the ofa build server are not pulling >> all the branches? >> >> Maybe I'm abusing git? > > It sounds like a different between git versions. Older git versions > brought in remote branches such that 'git branch' would show them. (This > causes problems if a remote branch conflicts with a local branch.) With > the newer git version (what's on the ofqmanqa server), you need to use 'git > branch -r' to see all of the actual branches. > > - Sean Yea, that's it. But how do I checkout the remote branch? The man page and 'git help' don't even show the -r option... Steve. From mshefty at ichips.intel.com Thu Aug 2 10:57:43 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 10:57:43 -0700 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B218D4.7010209@opengridcomputing.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> <20070802170051.GD29282@mellanox.co.il> <46B213E7.4050500@opengridcomputing.com> <46B21701.3030908@ichips.intel.com> <46B218D4.7010209@opengridcomputing.com> Message-ID: <46B21B17.8070305@ichips.intel.com> > Yea, that's it. But how do I checkout the remote branch? The man page > and 'git help' don't even show the -r option... I think you use the full name. Example: git checkout origin/ofed_1_2 - Sean From jgunthorpe at obsidianresearch.com Thu Aug 2 11:00:02 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Thu, 2 Aug 2007 12:00:02 -0600 Subject: [ofa-general] Re: IPoIB path caching In-Reply-To: <46B1F71A.9040405@ichips.intel.com> References: <46A78146.1090304@ichips.intel.com> <46A846FC.5040704@voltaire.com> <46A8D80C.1090305@ichips.intel.com> <20070726181132.GO19768@obsidianresearch.com> <46AC509B.6020206@voltaire.com> <20070729173232.GA14867@obsidianresearch.com> <46ADAA85.8070106@voltaire.com> <46AF600B.2040904@ichips.intel.com> <46B1CB45.3020709@voltaire.com> <46B1F71A.9040405@ichips.intel.com> Message-ID: <20070802180002.GQ8504@obsidianresearch.com> On Thu, Aug 02, 2007 at 08:24:10AM -0700, Sean Hefty wrote: > >Does it mean that you will have to re-implement RMPP in a user space > >library or just the initiation of the query would be from user space? I thought RMPP already was available to user space? In fact, I think we have code someplace that uses it.. > RMPP would not need to be re-implemented, but I haven't come up with a > user space solution that I like either. I'm not trying to design a full > blown distributed SA, nor do I want to try to standardize some new SA > protocol. I only want to fix the SA PR scalability problems that we see > in practice. Well, I still think the simplest thing is to make a new netlink protocol to maintain a cache table in the kernel and then a simple user space program that uses the user space RMPP interface (and trap subscription..) to do GetTable queries and uses netlink to groom the kernel cache. Basically exactly what you have now but seperated into two halfs. The main thing is to pick the kernel's table structure so that it can be extended to support future things like multipath, QOS and routers.. I don't like the notion of communicating with the kernel using MADs, it doesn't seem like it fits with the rest of the kernel networking model. When/if a network component to SA distribution is built it should be handled in user space and the kernel should be completely unrelated.. Jason From swise at opengridcomputing.com Thu Aug 2 11:09:29 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 13:09:29 -0500 Subject: [ewg] Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <46B21B17.8070305@ichips.intel.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <46B20627.70708@opengridcomputing.com> <20070802163804.GA29282@mellanox.co.il> <46B2095C.1050707@opengridcomputing.com> <46B20BD9.4020501@opengridcomputing.com> <20070802170051.GD29282@mellanox.co.il> <46B213E7.4050500@opengridcomputing.com> <46B21701.3030908@ichips.intel.com> <46B218D4.7010209@opengridcomputing.com> <46B21B17.8070305@ichips.intel.com> Message-ID: <46B21DD9.1000202@opengridcomputing.com> Sean Hefty wrote: >> Yea, that's it. But how do I checkout the remote branch? The man >> page and 'git help' don't even show the -r option... > > I think you use the full name. Example: > > git checkout origin/ofed_1_2 > > - Sean Yep, that did it! And if you want to make a local branch, you do git checkout -b origin/ofed_1_2 Thanks! Steve. From rdreier at cisco.com Thu Aug 2 11:22:44 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 11:22:44 -0700 Subject: [ofa-general] Re: [PATCH V2] IB/mlx4: enable MSI-X by default In-Reply-To: <20070802172728.GE29282@mellanox.co.il> (Michael S. Tsirkin's message of "Thu, 2 Aug 2007 20:27:28 +0300") References: <20070722091944.GC7800@mellanox.co.il> <20070802172728.GE29282@mellanox.co.il> Message-ID: > Why doesn't MSI-X work? Isn't this BIOS problem? Most likely not -- I guess there are a few cases on Opteron where the BIOS needs to enable HT mapping or something like that, but usually MSI-X fails because of a chipset problem. - R. From mshefty at ichips.intel.com Thu Aug 2 11:37:27 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 11:37:27 -0700 Subject: [ofa-general] Re: IPoIB path caching In-Reply-To: <20070802180002.GQ8504@obsidianresearch.com> References: <46A78146.1090304@ichips.intel.com> <46A846FC.5040704@voltaire.com> <46A8D80C.1090305@ichips.intel.com> <20070726181132.GO19768@obsidianresearch.com> <46AC509B.6020206@voltaire.com> <20070729173232.GA14867@obsidianresearch.com> <46ADAA85.8070106@voltaire.com> <46AF600B.2040904@ichips.intel.com> <46B1CB45.3020709@voltaire.com> <46B1F71A.9040405@ichips.intel.com> <20070802180002.GQ8504@obsidianresearch.com> Message-ID: <46B22467.2020308@ichips.intel.com> > Well, I still think the simplest thing is to make a new netlink > protocol to maintain a cache table in the kernel and then a simple > user space program that uses the user space RMPP interface (and trap > subscription..) to do GetTable queries and uses netlink to groom the > kernel cache. Basically exactly what you have now but seperated into > two halfs. This makes more sense to me. I thought you were suggesting that the entire implementation be in user space. > The main thing is to pick the kernel's table structure so that it can > be extended to support future things like multipath, QOS and routers.. Multipath is already supported, but I'm storing path records, not creating them from tables currently. QoS support is a more immediate concern, but I believe we can have a fairly simple set of tables to support that. Routers will present more of a challenge. > I don't like the notion of communicating with the kernel using MADs, > it doesn't seem like it fits with the rest of the kernel networking > model. When/if a network component to SA distribution is built it > should be handled in user space and the kernel should be completely > unrelated.. I was trying to say that the interface to the local SAs that keeps them in sync with the master SA should be MAD based. (It doesn't have to be, but that's how I would implement it.) I would not use MADs to program the tables. I will see about separating the two parts of the local SA. - Sean From farran.dallimore at gram-hansen.dk Thu Aug 2 06:47:05 2007 From: farran.dallimore at gram-hansen.dk (Dean Quintana) Date: Thu, 2 Aug 2007 18:47:05 +0500 Subject: [ofa-general] You can... Message-ID: <01c7d535$85a90ca0$28fe96cd@farran.dallimore> -------------- next part -------------- A non-text attachment was scrubbed... Name: raroru.gif Type: image/gif Size: 12743 bytes Desc: not available URL: From rdreier at cisco.com Thu Aug 2 11:58:51 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 11:58:51 -0700 Subject: [ofa-general] [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler In-Reply-To: (Hal Rosenstock's message of "Tue, 31 Jul 2007 06:39:34 -0400") References: Message-ID: thanks, I applied this as 3 separate patches. From rdreier at cisco.com Thu Aug 2 11:59:31 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 11:59:31 -0700 Subject: [ofa-general] [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler In-Reply-To: (Hal Rosenstock's message of "Tue, 31 Jul 2007 06:39:34 -0400") References: Message-ID: by the way, I had to apply this by hand, because it seems gmail is destroying the whitespace in the patches. I'm not sure if there's any way to send patches via gmail except as attachments unfortunately. From rdreier at cisco.com Thu Aug 2 12:01:51 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 12:01:51 -0700 Subject: [ofa-general] Re: [PATCH] ib/mad: fix address handle leak in mad_rmpp In-Reply-To: <000001c7d3bf$aa5e0090$ff0da8c0@amr.corp.intel.com> (Sean Hefty's message of "Tue, 31 Jul 2007 15:10:54 -0700") References: <000001c7d3bf$aa5e0090$ff0da8c0@amr.corp.intel.com> Message-ID: thanks, applied From hal.rosenstock at gmail.com Thu Aug 2 12:03:05 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 2 Aug 2007 15:03:05 -0400 Subject: [ofa-general] [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler In-Reply-To: References: Message-ID: Roland, On 8/2/07, Roland Dreier wrote: > by the way, I had to apply this by hand, because it seems gmail is > destroying the whitespace in the patches. I'm not sure if there's any > way to send patches via gmail except as attachments unfortunately. Sorry; I was afraid that was happening :-( So would attachments be accepted ? If so, should the change still be sent in text as well ? -- Hal From rdreier at cisco.com Thu Aug 2 12:05:18 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 12:05:18 -0700 Subject: [ofa-general] [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler In-Reply-To: (Hal Rosenstock's message of "Thu, 2 Aug 2007 15:03:05 -0400") References: Message-ID: > Sorry; I was afraid that was happening :-( So would attachments be > accepted ? If so, should the change still be sent in text as well ? Attachments are a pain -- the best is to find a way to send email without mangling patches. I think you can use a non-web client with gmail's smtp server to send if you want. From transter at gmail.com Thu Aug 2 12:33:33 2007 From: transter at gmail.com (lbt) Date: Thu, 2 Aug 2007 12:33:33 -0700 Subject: [ofa-general] OpenSM "stuck" - user level MAD library seems to be timing out Message-ID: Hi Sasha, I am hitting a problem where the user level MAD library seems to be timing out, causing the ports to be stuck in "INIT" state because the subnet has no "Master" SM available. The system is still in this state, so if there are any suggestions on what other type of debug info I could collect or clues to what the problem might be, it would be much apprceciated :) I have 3 machines (OFED 1.1. stack, Opensm v2.0.5), where 2 of them are running open SM, connected by an IB switch. Several tests were being done pulling IB cables, but not touching at all the IB connections between the Master SM and the IB switch, or rebooting the IB switch (i.e. no SM migration should be occurring). Everything was working fine, until at one point, I pull the IB cable on the IB switch of the lower priority (standby) SM. For some reason, this starts causing problems on the higher priority Master SM. The higher priority SM now thinks it's in Standby state, and the lower priority SM's MAD packets are timing out. It is odd because, I would not expect any effect on the higher priority SM (as it's IB connections are not being affected). And not sure why MAD packets are timing out on the lower priority SM. Rebooting the lower priority SM and replugging IB cables into different ports on the IB switch, didn't help. Lower priority SM: (packets timeout) [root at vortex3l-83 ~]# sminfo -d -e -P 1 ibwarn: [26764] smp_query: attr 21 mod 0 route DR path [0] ibwarn: [26764] mad_rpc: data offs 64 sz 64 mad data 0000 0000 0000 0000 fe80 0000 0000 0000 0003 0001 0251 0a6a 0000 0000 0103 0302 1252 0011 4040 0008 0804 ff40 0000 0000 0000 2012 1088 0000 0000 0000 0000 0000 ibwarn: [26764] smp_query: attr 32 mod 0 route Lid 1 ibwarn: [26764] _do_madrpc: retry 1 (timeout 1000 ms) ibwarn: [26764] _do_madrpc: retry 2 (timeout 1000 ms) ibwarn: [26764] _do_madrpc: timeout after 3 retries, 3000 ms sminfo: iberror: [pid 26764] main: failed: query Higher priority SM: (thinks its Standby now) [root at vortex3l-84 log]# sminfo -d -e -P 1 ibwarn: [2487] smp_query: attr 21 mod 0 route DR path [0] ibwarn: [2487] mad_rpc: data offs 64 sz 64 mad data 0000 0000 0000 0000 fe80 0000 0000 0000 0002 0003 0251 0a6a 0000 0000 0103 0302 1252 0011 4040 0008 0804 ff40 0000 0000 0000 2012 1088 0000 0000 0000 0000 0000 ibwarn: [2487] smp_query: attr 32 mod 0 route Lid 3 ibwarn: [2487] mad_rpc: data offs 64 sz 64 mad data 0050 4501 4a3a 0001 0000 0000 0000 0000 0000 020e 0200 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 sminfo: sm lid 3 sm guid 0x5045014a3a0001, activity count 526 priority 0 state 2 SMINFO_STANDBY Just another data point, but each machine happens to have 2 HCA ports, port 1 and port 2. Port 1 is connected to different subnet than port2. During all these steps, port2 subnet is still fine and working OK. The problem described above was being seen with the port 1 subnet only. Thanks! Lan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgunthorpe at obsidianresearch.com Thu Aug 2 13:29:46 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Thu, 2 Aug 2007 14:29:46 -0600 Subject: [ofa-general] Re: IPoIB path caching In-Reply-To: <46B22467.2020308@ichips.intel.com> References: <46A8D80C.1090305@ichips.intel.com> <20070726181132.GO19768@obsidianresearch.com> <46AC509B.6020206@voltaire.com> <20070729173232.GA14867@obsidianresearch.com> <46ADAA85.8070106@voltaire.com> <46AF600B.2040904@ichips.intel.com> <46B1CB45.3020709@voltaire.com> <46B1F71A.9040405@ichips.intel.com> <20070802180002.GQ8504@obsidianresearch.com> <46B22467.2020308@ichips.intel.com> Message-ID: <20070802202946.GR8504@obsidianresearch.com> On Thu, Aug 02, 2007 at 11:37:27AM -0700, Sean Hefty wrote: > >Well, I still think the simplest thing is to make a new netlink > >protocol to maintain a cache table in the kernel and then a simple > >user space program that uses the user space RMPP interface (and trap > >subscription..) to do GetTable queries and uses netlink to groom the > >kernel cache. Basically exactly what you have now but seperated into > >two halfs. > > This makes more sense to me. I thought you were suggesting that the > entire implementation be in user space. Well, both really.. The kernel side table could either include all possible PRs, or some subset. Your implementation uses all possible PRs today and that is fine, but to support QOS or routers you might want to only have a subset resident in the kernel at any time. The only practical impact this has on the kernel support is that cache lookup misses should have a way to be passed to userspace for additional processing (if userspace requests that). > >The main thing is to pick the kernel's table structure so that it can > >be extended to support future things like multipath, QOS and routers.. > > Multipath is already supported, but I'm storing path records, not > creating them from tables currently. QoS support is a more immediate > concern, but I believe we can have a fairly simple set of tables to > support that. Routers will present more of a challenge. I am imagining the kernel being very simple, take the standard IB PR fields and match them against a caching table. If the matching misses pass it to userspace, or do a PR. If the QOS tables cannot be expressed as simple matching then the userspace part would have to do the table based calculations to match it and populate the kernel cache table. Jason From rdreier at cisco.com Thu Aug 2 13:54:28 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 13:54:28 -0700 Subject: [ofa-general] Re: [PATCH V2] core: move the macro IB_UMEM_MAX_PAGE_CHUNK to umem.c In-Reply-To: <200708011333.56131.dotanb@dev.mellanox.co.il> (Dotan Barak's message of "Wed, 1 Aug 2007 13:33:56 +0300") References: <200708011333.56131.dotanb@dev.mellanox.co.il> Message-ID: thanks, applied all 3 of your header fixups. From rdreier at cisco.com Thu Aug 2 14:03:59 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 14:03:59 -0700 Subject: [ofa-general] Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space In-Reply-To: <200708021608.16436.arnd@arndb.de> (Arnd Bergmann's message of "Thu, 2 Aug 2007 16:08:15 +0200") References: <200708021139.31695.hnguyen@linux.vnet.ibm.com> <200708021608.16436.arnd@arndb.de> Message-ID: > remap_4k_pfn is defined in terms of remap_pfn_range if the base page > size if 4k, so you don't need this #ifdef afaics. Good point. I'll wait for an updated patch. From rdreier at cisco.com Thu Aug 2 14:05:06 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 14:05:06 -0700 Subject: [ofa-general] Re: [PATCH 2.6.23 2/2] iw_cxgb3: Always call low level send function via cxgb3_ofld_send(). In-Reply-To: <20070729201228.31659.26300.stgit@dell3.ogc.int> (Steve Wise's message of "Sun, 29 Jul 2007 15:12:29 -0500") References: <20070729201226.31659.85900.stgit@dell3.ogc.int> <20070729201228.31659.26300.stgit@dell3.ogc.int> Message-ID: thanks, applied. From rdreier at cisco.com Thu Aug 2 14:06:41 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 02 Aug 2007 14:06:41 -0700 Subject: [ofa-general] Re: [PATCH 2.6.23 1/2] Make the iw_cxgb3 module parameters writable. In-Reply-To: <20070729201226.31659.85900.stgit@dell3.ogc.int> (Steve Wise's message of "Sun, 29 Jul 2007 15:12:26 -0500") References: <20070729201226.31659.85900.stgit@dell3.ogc.int> Message-ID: thanks... I actually applied this for 2.6.24, since it's not really a fix for anything, and the 2.6.23 window is closed. From drthq at about-cis.com Thu Aug 2 14:16:54 2007 From: drthq at about-cis.com (Fulton Q. Rosemary) Date: Fri, 3 Aug 2007 00:16:54 +0300 Subject: [ofa-general] post Message-ID: <46B249C6.4060104@about-cis.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: post.zip Type: application/octet-stream Size: 16041 bytes Desc: not available URL: From swise at opengridcomputing.com Thu Aug 2 14:17:18 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 16:17:18 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B0C78E.1030306@ichips.intel.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> Message-ID: <46B249DE.8020409@opengridcomputing.com> Sean Hefty wrote: > > Okay - for final patches, I think we want to remove the rdma_cm specific > port spaces, along with changing the API to clarify that it uses the > same port space as TCP/UDP. > If we get rid of the rdma_cm specific port spaces, do we then reduce the valid possible spaces to just TCP and UDP? Or what? In the sockets paradigm, the socket is explicitly bound to a protocol space when its created (based on the protocol id). Do you think we need to change the rdma_cm_id to have such a concept? IE when you create the cm_id, you say your intended QP type or port space? The current API lends itself to somone incorrectly choosing a port space, by the way. But should we really change the API that drastically? Or just keep the port spaces and make PS_TCP share the host's port space. The only applications that really need this port space are apps that run on iWARP only or want to support both iWARP and IB via the transport-independent verbs and rdma-cm. So things like IPoIB probably shouldn't use or need the TCP port space. Maybe the rdma-cm port spaces should really be IB, IWARP, or BOTH. IB has its own port space, and IWARP or BOTH gets the TCP port space. thoughts? Steve. From swise at opengridcomputing.com Thu Aug 2 14:21:29 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 02 Aug 2007 16:21:29 -0500 Subject: [ofa-general] [GIT PULL ofed-1.2/ofed-1.2.c] Chelsio Bug Fixes Message-ID: <46B24AD9.2000801@opengridcomputing.com> Vlad, Please include these two patches into ofed 1.2 and 1.2.c. The patches are direct changes to the ofed_1_2 branch and patch files in kernel_patches/fixes for ofed_1_2_c. These have been accepted upstream, and address OFED bugs 696 and 698. Please pull these from: http://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2 and http://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2_c Thanks, Steve. Shortlog: iw_cxgb3: Make the iw_cxgb3 module parameters writable. iw_cxgb3: Always call low level send function via cxgb3_ofld_send(). From kliteyn at dev.mellanox.co.il Thu Aug 2 14:41:16 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Fri, 03 Aug 2007 00:41:16 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: References: <46A283B6.1070105@dev.mellanox.co.il> Message-ID: <46B24F7C.8080008@dev.mellanox.co.il> Hi Hal, Hal Rosenstock wrote: > Hi Yevgeny, > > On 7/21/07, Yevgeny Kliteynik wrote: >> Hi All >> >> Please find the attached RFC describing how QoS policy support could be implemented in the OpenFabrics stack. >> Your comments are welcome. > > A couple of quick questions: > > How does this differ from the original RFC posted 5/30/06 ? > > What I can see is the following: > 1. Updated for not yet released IBTA QoS Annex > 2. Use of plain text rather than XML based policy file for OpenSM > Anything else ? You're absolutely right - these are the only changes (plus cosmetics here and there). > Below, IPoIB is discussed in terms of UD. What about IPoIB-CM ? It > uses CM and has a service ID. > > Also, have my specific comments to the patches originally submitted > been addressed ? (Do I need to dig them out again ?) Just wondering... Yes. The submitted patches were only QoS policy file parser. In the new parser I took care of all the issues we've discussed a couple of months ago. Here is the summary of these issues taken from our discussion: [snip] XML syntax or not -> Plain text, human readable and easily editable QoS syntax explanation/discussion changes to some keywords Port r.t. Node Groups -> DONE: fixed keywords CA r.t. HCA -> DONE: fixed keywords QoSClass r.t. TClass -> DONE: fixed keywords syntax discussion points larger ones: dynamic service IDs -> supported through list and range support service ID range support -> DONE: added to the matching rules examples port groups shared with partition configuration (future) -> agree, it would be a good idea to share port groups with partition configuration, but it won't be for OFED 1.3 multicast -> not planned for OFED 1.3, but we'll discuss it later smaller ones: across syntax explanation -> DONE: see the explanation and an example in the policy file What is sn in the syntax short for ? -> it was for "serial number", replaced by "qos-level-sn". It means the serial (sequential) number of the qos-level that should be applied to PathRecords that matches this qos-match-rule. I probably will change the "qol-level-sn" to "qos-level-name" to refer QoS level by name rather than by sn. path bits explanation -> Path bits are part of QoS level. They can be used to "differentiate" paths through the subnet to a port when LMC>0. It won't be implemented yet for OFED 1.3, and OpenSM should issue a warning if it finds PathBits in QoS level definition in the policy file packet lifetime ? -> DONE: added packet-life keyword viewer/editor -> Since we've switched to plain text, this one becomes irrelevant [/snip] -- Yevgeny > Thanks. > > -- Hal > > > > >> -- Yevgeny >> >> RFC: OpenFabrics Enhancements for QoS Support >> =============================================== >> >> Authors: . Eitan Zahavi >> Authors: . Yevgeny Kliteynik >> Date: .... Jul 2007. >> Revision: 0.2 >> >> Table of contents: >> 1. Overview >> 2. Architecture >> 3. Supported Policy >> 4. CMA functionality >> 5. IPoIB functionality >> 6. SDP functionality >> 7. SRP functionality >> 8. iSER functionality >> 9. OpenSM functionality >> >> 1. Overview >> ------------ >> Quality of Service requirements stem from the realization of I/O consolidation >> over IB network: As multiple applications and ULPs share the same fabric, means >> to control their use of the network resources are becoming a must. The basic >> need is to differentiate the service levels provided to different traffic flows, >> such that a policy could be enforced and control each flow utilization of the >> fabric resources. >> >> IBTA specification defined several hardware features and management interfaces >> to support QoS: >> * Up to 15 Virtual Lanes (VL) carry traffic in a non-blocking manner >> * Arbitration between traffic of different VLs is performed by a 2 priority >> levels weighted round robin arbiter. The arbiter is programmable with >> a sequence of (VL, weight) pairs and maximal number of high priority credits >> to be processed before low priority is served >> * Packets carry class of service marking in the range 0 to 15 in their >> header SL field >> * Each switch can map the incoming packet by its SL to a particular output >> VL based on programmable table VL=SL-to-VL-MAP(in-port, out-port, SL) >> * The Subnet Administrator controls each communication flow parameters >> by providing them as a response to Path Record (PR) or MultiPathRecord (MPR) >> queries >> >> The IB QoS features provide the means to implement a DiffServ like architecture. >> DiffServ architecture (IETF RFC2474 2475) is widely used today in highly dynamic >> fabrics. >> >> This proposal provides the detailed functional definition for the various >> software elements that are required to enable a DiffServ like architecture over >> the OpenFabrics software stack. >> >> >> >> 2. Architecture >> ---------------- >> This proposal split the QoS functionality between the SM/SA, CMA and the various >> ULPS. We take the "chronology approach" to describe how the overall system >> works: >> >> 2.1. The network manager (human) provides a set of rules (policy) that defines >> how the network is being configured and how its resources are split to different >> QoS-Levels. The policy also define how to decide which QoS-Level each >> application or ULP or service use. >> >> 2.2. The SM analyzes the provided policy to see if it is realizable and performs >> the necessary fabric setup. The SM may continuously monitor the policy and adapt >> to changes in it. Part of this policy defines the default QoS-Level of each >> partition. The SA is being enhanced to match the requested Source, Destination, >> QoS-Class, Service-ID (and optionally SL and priority) against the policy. So >> clients (ULPs, programs) can obtain a policy enforced QoS. The SM is also >> enhanced to support setting up partitions with appropriate IPoIB broadcast >> group. This broadcast group carries its QoS attributes: SL, MTU and >> RATE. >> >> 2.3. IPoIB is being setup. IPoIB uses the SL, MTU and RATE available on the >> multicast group which forms the broadcast group of this partition. >> >> 2.4. MPI which provides non IB based connection management should be configured >> to run using hard coded SLs. It uses these SLs for every QP being opened. >> >> 2.5. ULPs that use CM interface (like SRP) should have their own pre-assigned >> Service-ID and use it while obtaining PR/MPR for establishing connections. >> The SA receiving the PR/MPR should match it against the policy and return >> the appropriate PR/MPR including SL, MTU and RATE. >> >> 2.6. ULPs and programs using CMA to establish RC connection should provide the >> CMA the target IP and Service-ID. Some of the ULPs might also provide QoS-Class >> (E.g. for SDP sockets that are provided the TOS socket option). The CMA should >> then use the provided Service-ID and optional QoS-Class and pass them in the >> PR/MPR request. The resulting PR/MPR should be used for configuring the >> connection QP. >> >> PathRecord and MultiPathRecord enhancement for QoS: >> As mentioned above the PathRecord and MultiPathRecord attributes should be >> enhanced to carry the Service-ID which is a 64bit value, which has been >> standardized by the IBTA. A new field QoS-Class is also provided. >> A new capability bit should describe the SM QoS support in the SA class port >> info. This approach provides an easy migration path for existing access layer >> and ULPs by not introducing new set of PR/MPR attribute. >> >> >> 3. Supported Policy >> -------------------- >> >> The QoS policy supported by this proposal is divided into 4 sub sections: >> >> I) Port Group: a set of CAs, Routers or Switches that share the same settings. >> A port group might be a partition defined by the partition manager policy in >> terms of GUIDs. Future implementations might provide support for NodeDescription >> based definition of port groups. >> >> II) Fabric Setup: >> Defines how the SL2VL and VLArb tables should be setup. This policy definition >> assumes the computation of overall end to end network behavior should be performed >> outside of OpenSM. >> >> III) QoS-Levels Definition: >> This section defines the possible sets of parameters for QoS that a client >> might be mapped to. Each set holds: SL and optionally: Max MTU, Max Rate, >> Packet Lifetime and Path Bits (in case LMC > 0 is used for QoS). >> >> IV) Matching Rules: >> A list of rules that match an incoming PR/MPR request to a QoS-Level. The >> rules are processed in order such as the first match is applied. Each rule is >> built out of a set of match expressions which should all match for the rule to >> apply. The matching expressions are defined for the following fields >> ** SRC and DST to lists of port groups >> ** Service-ID to a list of Service-ID or Service-ID ranges >> ** QoS-Class to a list of QoS-Class values or ranges >> >> QoS Policy file syntax >> >> * Empty lines are ignored >> * Leading and trailing blanks, as well as empty lines, are ignored, so the >> indentation in the example is just for better readability >> * Comments are started with the pound sign (#) and terminated by EOL >> * Comments may appear only in a separate line >> * Keywords that denote section/subsection start have matching closing keywords >> * Any keyword should be the first non-blank in the line >> >> QoS Policy file example >> >> # Port Groups define sets of ports to be used later in the settings >> port-groups >> # using port GUIDs >> port-group >> name: Storage >> # "use" is just a description that is used for logging. >> # Other than that, it is just a commentary >> use: our SRP storage targets >> port-guid: 0x1000000000000001 >> port-guid: 0x1000000000000002 >> end-port-group >> >> port-group >> name: Virtual Servers >> use: node desc and IB port num >> # The syntax of the port name is as follows: "hostname/CA-num/Pnum". >> # "hostname" and "CA-num" are compared to the first 2 words of >> # NodeDescription, and "Pnum" is a port number on that node. >> port-name: vs1/HCA-1/P1 >> port-name: vs3/HCA-1/P1 >> port-name: vs3/HCA-2/P2 >> end-port-group >> >> # using partitions defined in the partition policy >> port-group >> name: Group for Partition 1 >> use: default settings >> partition: Part1 >> end-port-group >> >> # using node types CA|ROUTER|SWITCH >> port-group >> name: Routers >> use: all routers >> node-type: ROUTER >> end-port-group >> >> end-port-groups >> >> qos-setup >> >> # define all types of VLArb tables. The length of the tables should >> # match the physically supported tables by their target ports >> vlarb-tables >> # scope defines the exact ports the VLArb tables apply to >> vlarb-scope >> # defining VLArb tables on all the ports that belong to >> # port group 'Storage', and on all the ports connected >> # to ports of port group 'Storage' >> group: Storage >> # "across" means all the ports that are connected to ports >> # that belong to the specified port group >> across: Storage >> # VLArb table holds VL and weight pairs >> vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 >> vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 >> vl-high-limit: 10 >> end-vlarb-scope >> # There can be several scopes >> end-vlarb-tables >> >> sl2vl-tables >> # Scope defines the exact devices and in/out ports tables apply to. >> # Note: if the same port is matching several rules the *FIRST* one applies. >> sl2vl-scope >> # SL2VL tables are orgnized as SL2VL(in-port,out-port) >> # "from: n,m" means we define the SL2VL(n,*) and SL2VL(m,*) >> # "to: n,m" means we define the SL2VL(*,n) and SL2VL(*,m) >> # >> # The following example specifies that all the SL2VL tables >> # entries should be defined for all the ports of group Part1: >> group: Part1 >> from: * >> to: * >> # SL2VL table has to have 16 values at max - one for each SL. >> # If the user specifies less than 16 values, all the missing >> # VL values will be implicitly set to 0 >> sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 >> end-sl2vl-scope >> >> sl2vl-scope >> # "across-to" is a combination of "across" keyword (definition can be found >> # in VLArb tables section) and "to" keyword. >> # "across: PortGroupName" refers to all the ports that are connected >> # to ports that belong to PortGroupName. >> # >> # Example of "across-to" usage: >> # A user has a set of 'special' nodes (e.g. storage nodes), and all >> # the traffic to these nodes has to get specific VL. >> # The solution is to define port group (i.g. "Storage") that will >> # include all the ports of these nodes, and then to configure SL2VL >> # tables on all the switch ports that are connected to the Storage >> # port group by specifying "across-to: Storage". >> # >> across-to: Storage2 >> # Similar to "across-to", "across-from" is a combination of "across" >> # and "to" keywords >> across-from: Storage1 >> sl2vl-table: 0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0 >> end-sl2vl-scope >> end-sl2vl-tables >> >> end-qos-setup >> >> >> qos-levels >> >> # the first one is just setting SL >> qos-level >> use: for the lowest priority communication >> sl: 15 >> packet-life: 16 >> end-qos-level >> # the second sets SL and QoS Class >> qos-level >> use: low latency best bandwidth >> sl: 0 >> end-qos-level >> # the whole set: SL, MTU-Limit, Rate-Limit, Packet Lifetime, Path Bits >> qos-level >> use: just an example >> sl: 0 >> mtu-limit: 1 >> rate-limit: 1 >> packet-life: 12 >> # Path Bits can be used e.g. to provide a different routes through the >> # subnet to a particular port >> path-bits: 2,4,8-32 >> end-qos-level >> >> end-qos-levels >> >> >> # Match rules are scanned in a first-fit manner (like firewall rules table) >> qos-match-rules >> >> # matching by single criteria: class (list of values and ranges) >> qos-match-rule >> # just a description >> use: low latency by class 7-9 or 11 >> qos-class: 7-9,11 >> # number of qos-level to apply to the matching PR/MPR >> qos-level-sn: 1 >> end-qos-match-rule >> # show matching by destination group AND service-ids >> qos-match-rule >> use: Storage targets connection >> destination: Storage >> service-id: 22,4719-5000 >> qos-level-sn: 2 >> end-qos-match-rule >> # show matching by source group only >> qos-match-rule >> use: bla bla >> source: Storage >> qos-level-sn: 3 >> end-qos-match-rule >> >> end-qos-match-rules >> >> >> 4. IPoIB >> --------- >> >> IPoIB already query the SA for its broadcast group information. The additional >> functionality required is for IPoIB to provide the broadcast group SL, MTU, >> and RATE in every following PathRecord query performed when a new UDAV is >> needed by IPoIB. >> We could assign a special Service-ID for IPoIB use but since all communication >> on the same IPoIB interface shares the same QoS-Level without the ability to >> differentiate it by target service we can ignore it for simplicity. >> >> 5. CMA features >> ---------------- >> >> The CMA interface supports Service-ID through the notion of port space as a >> prefixes to the port_num which is part of the sockaddr provided to >> rdma_resolve_add(). What is missing is the explicit request for a QoS-Class that >> should allow the ULP (like SDP) to propagate a specific request for a class of >> service. A mechanism for providing the QoS-Class is available in the IPv6 address, >> so we could use that address field. Another option is to implement a special >> connection options API for CMA. >> >> Missing functionality by CMA is the usage of the provided QoS-Class and Service-ID >> in the sent PR/MPR. When a response is obtained it is an existing requirement for >> the CMA to use the PR/MPR from the response in setting up the QP address vector. >> >> >> 6. SDP >> ------- >> >> SDP uses CMA for building its connections. >> The Service-ID for SDP is 0x000000000001PPPP, where PPPP are 4 hex digits >> holding the remote TCP/IP Port Number to connect to. >> SDP might be provided with SO_PRIORITY socket option. In that case the value >> provided should be sent to the CMA as the TClass option of that connection. >> >> 7. SRP >> ------- >> >> Current SRP implementation uses its own CM callbacks (not CMA). So SRP should >> fill in the Service-ID in the PR/MPR by itself and use that information in >> setting up the QP. The T10 SRP standard defines the SRP Service-ID to be defined >> by the SRP target I/O Controller (but they should also comply with IBTA Service- >> ID rules). Anyway, the Service-ID is reported by the I/O Controller in the >> ServiceEntries DMA attribute and should be used in the PR/MPR if the SA >> reports its ability to handle QoS PR/MPRs. >> >> 8. iSER >> -------- >> iSER uses CMA and thus should be very close to SDP. The Service-ID for iSER >> should be TBD. >> >> >> 9. OpenSM features >> ------------------- >> The QoS related functionality to be provided by OpenSM can be split into two >> main parts: >> >> 3.1. Fabric Setup >> During fabric initialization the SM should parse the policy and apply its >> settings to the discovered fabric elements. The following actions should be >> performed: >> * Parsing of policy >> * Node Group identification. Warning should be provided for each node not >> specified but found. >> * SL2VL settings validation should be checked: >> + A warning will be provided if there are no matching targets for the SL2VL >> setting statement. >> + An error message will be printed to the log file if an invalid setting is >> found. A setting is invalid if it refers to: >> - Non existing port numbers of the target devices >> - Unsupported VLs for the target device. In the later case the map to non >> existing VLs should be replaced to VL15 i.e. packets will be dropped. >> * SL2VL setting is to be performed >> * VL Arbitration table settings should be validated according to the following >> rules: >> + A warning will be provided if there are no matching targets for the setting >> statement >> + An error will be provided if the port number exceeds the target ports >> + An error will be generated if the table length exceeds device capabilities >> + A warning will be generated if the table quote a VL that is not supported >> by the target device >> * VL Arbitration tables will be set on the appropriate targets >> >> 3.2. PR/MPR query handling: >> OpenSM should be able to enforce the provided policy on client request. >> The overall flow for such requests is: first the request is matched against the >> defined match rules such that the target QoS-Level definition is found. Given >> the QoS-Level a path(s) search is performed with the given restrictions imposed >> by that level. The following two sections describe these steps. >> >> How Service-ID is carried in the PathRecord and MultiPathRecord attributes is >> now standardized by the IBTA. >> >> >> 3.2.1. Matching rule search: >> A rule is "matching" a PR/MPR request using the following criteria: >> * Matching rules provide values in a list of either single value, or range of >> values. A PR/MPR field is "matching" the rule field if it is explicitly >> noted in the list of values or is one of the values covered by a range >> included in the field values list. >> * Only PR/MPR fields that have their component mask bit set should be >> compared. >> * For a rule to be "matching" a PR/MPR request all the rule fields should be >> "matching" their PR/MPR fields. Such that a PR/MPR request that does >> not have a component mask field set for one of the rule defined fields can >> not match that rule. >> * A PR/MPR request that have a component mask bit set for one of the fields >> that is not defined by the rule can match the rule. >> >> The algorithm to be used for searching for a rule match might be as simple as a >> sequential search through all rules or enhanced for better performance. The >> semantics of every rule field and its matching PR/MPR field are described >> below: >> * Source: the SGID or SLID should be part of this group >> * Destination: the DGID or DLID should be part of this group >> * Service-ID: check if the requested Service-ID (available in the PR/MPR old >> SM-Key field) is matching any of this rule Service-IDs >> * TClass: check if the PR/MPR TClass field is matching >> >> 3.2.2 PR/MPR response generation: >> The QoS-Level pointed by the first rule that matches the PR/MPR request >> should be used for obtaining the response SL, MTU-Limit, RATE-Limit, Path-Bits >> and QoS-Class. A default QoS-Level should be used if no rule is matching the query. >> >> The efficient algorithm for finding paths that meet the QoS-Level criteria is >> beyond the scope of this RFC and left for the implementer to provide. However >> the criteria by which the paths match the QoS-Level are described below: >> >> * SL: The paths found should all use the given SL. For that sake PR/MPR >> algorithm should traverse the path from source to destination only through >> ports that carry a valid VL (not VL15) by the SL2VL map (should consider input >> and output ports and SL). >> * MTU-Limit: The resulting paths MTU should not exceed the given MTU-Limit >> * Rate-Limit: The resulting paths RATE should not exceed the given RATE-Limit >> (rate limit is given in units of link BW = Width*Speed according to IBTA >> Specification Vol-1 table-205 p-901 l-24). >> * Path-Bits: define the target LID lowest bits (number of bits defined by the >> target port PortInfo.LMC field). The path should traverse the LFT using the >> target port LID with the path-bits set. >> * QoS-Class: should be returned in the result PR/MPR. When routing is going to >> be supported by OpenSM we might use this field in selecting the target >> router too in a TBD way. >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > From mshefty at ichips.intel.com Thu Aug 2 14:52:58 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 02 Aug 2007 14:52:58 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B249DE.8020409@opengridcomputing.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> <46B249DE.8020409@opengridcomputing.com> Message-ID: <46B2523A.2090109@ichips.intel.com> > If we get rid of the rdma_cm specific port spaces, do we then reduce the > valid possible spaces to just TCP and UDP? Or what? In the sockets > paradigm, the socket is explicitly bound to a protocol space when its > created (based on the protocol id). Do you think we need to change the > rdma_cm_id to have such a concept? IE when you create the cm_id, you > say your intended QP type or port space? The current API lends itself > to somone incorrectly choosing a port space, by the way. Currently, the RDMA port space implies the QP type (RC or UD). We're not tied to any specific protocol when we create the rdma_cm_id, since we don't know what type of RDMA device we'll end up using. So, I don't think we want users to specify a protocol. > But should we really change the API that drastically? Or just keep the > port spaces and make PS_TCP share the host's port space. I don't want to break the user space API, if it can be helped. SDP is kind of a problem, in that the rdma_cm needs to distinguish between SDP as a user, versus someone using RDMA_PS_TCP. SDP maps between the RDMA port space and real TCP port space. I need to get some details on how SDP uses the rdma_cm, like whether it uses wild card port numbers. > Maybe the rdma-cm port spaces should really be IB, IWARP, or BOTH. IB > has its own port space, and IWARP or BOTH gets the TCP port space. I thought about doing something like this, but I'm not sure there would be much use of just IB or just iWarp, when the user can specify both. I even considered pushing the problem into the iWarp CM, but that seems like a more complex implementation with no benefit unless there are users of just an IB port space. At this point, my thoughts are to take your original patch, remove the rdma_cm port space structures and functions, and figure out how to handle SDP. - Sean From kliteyn at dev.mellanox.co.il Thu Aug 2 15:29:37 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Fri, 03 Aug 2007 01:29:37 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <20070731160223.GF29844@sashak.voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <20070723002010.GU27878@sashak.voltaire.com> <46A89608.9010709@dev.mellanox.co.il> <20070731160223.GF29844@sashak.voltaire.com> Message-ID: <46B25AD1.20301@dev.mellanox.co.il> Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 15:39 Thu 26 Jul , Yevgeny Kliteynik wrote: >>>> * Comments may appear only in a separate line >>> Why? What is wrong with: >>> port-name: vs1/HCA-1/P1 # my best port >> I can use this too, but then the pound sign, wherever it will >> appear, would mean commentary start. No \# or something like this >> to include it in some other place - I don't want to complicate the >> syntax. Sounds OK? > > Are we planning to use '#' somewhere? Nope. Anyway, I've implemented the "port-guid: 0x01 # bla bla" comment. > Anyway this comment is minor. > >>>> end-port-groups >>> I agree that proposed syntax has better for human readability than pure >>> XML, but isn't stuff like this will be more user-friendly? >>> Storage "Free Text description" = 0x10001, 0x10002, 0x10003 ; >>> , or >>> Storage "Free Text description" { 0x10001, 0x10002, 0x10003 }; >>> , or >>> Storage "Free Text description": ROUTERS, CAS ; >> GUID list is a good idea. >> Not sure about the other stuff. A certain port group can be defined >> both by guids and by node-types. How about this: >> >> port-group >> name: routers_and_mgt_nodes >> use: all routers and management nodes >> node-type: ROUTER >> port-guid: 0x10001, 0x10002, 0x10003 >> end-port-group > > I think it is doable too, like: 0x10001, 0x10002, 0x10003, ROUTER > Guess it should be easy to parse GUIDs, names and special names (like > ROUTER) in one line. Not sure it must be so, just thought... For now let's stay with the separate 'node-type' and 'port-guid' definitions. I did, however, add some improvements to the file syntax - I added support for num ranges wherever it makes sense. Example: port-guid: 0x10001-0x100FF,0x20000 I also added support for list of strings wherever it is needed. Examples: node-type: ROUTER,SELF destination: partition1,partition2 >>>> qos-levels >>>> >>>> # the first one is just setting SL >>>> qos-level >>>> use: for the lowest priority communication >>>> sl: 15 >>>> packet-life: 16 >>>> end-qos-level >>>> # the second sets SL and QoS Class >>>> qos-level >>>> use: low latency best bandwidth >>>> sl: 0 >>>> end-qos-level >>>> # the whole set: SL, MTU-Limit, Rate-Limit, Packet Lifetime, Path >>>> Bits >>>> qos-level >>>> use: just an example >>>> sl: 0 >>>> mtu-limit: 1 >>>> rate-limit: 1 >>>> packet-life: 12 >>>> # Path Bits can be used e.g. to provide a different routes >>>> through the >>>> # subnet to a particular port >>>> path-bits: 2,4,8-32 >>>> end-qos-level >>>> >>>> end-qos-levels >>>> >>>> >>>> # Match rules are scanned in a first-fit manner (like firewall rules >>>> table) >>>> qos-match-rules >>>> >>>> # matching by single criteria: class (list of values and ranges) >>>> qos-match-rule >>>> # just a description >>>> use: low latency by class 7-9 or 11 >>>> qos-class: 7-9,11 >>>> # number of qos-level to apply to the matching PR/MPR >>>> qos-level-sn: 1 >>> Isn't it better and less error prone to match qos_level by name and not >>> by sequential number? >> qos-level can have name, and then qos-match-rule will refer to this name. >> But matching qos-level by sequential number makes it really easy to locate >> the referred qos-level, which is important, as every PR/MPR request would >> go through this process, so saving some runtime in this area is important >> IMHO. > > Sure, it is important, but I'm not about internal data representation, > internally this should be fast reference - by index or by directly by > pointer. But in the file it would be easy for user to have names (numbers > could be used as names too) instead of just serial numbering on one > side, so an user will not need to count lines. OK, we can use names of QoS levels instead of sequential numbers. Here are the details: + in qos-level new mandatory keyword 'name' will be added: "name: single_string" + in qos-match-rules, the 'qos-level-sn' keyword will be replaced by 'qos-level-name' + qos-level with name "default"/"DEFAULT" is a must. It is the default level that would be applied to any PR/MPR request that didn't match any existing match rule + any match rule may explicitly refer to the default qos-level by specifying its name in the qos-level-name >>>> 9. OpenSM features >>>> ------------------- >>>> The QoS related functionality to be provided by OpenSM can be split into >>>> two >>>> main parts: >>>> >>>> 3.1. Fabric Setup >>>> During fabric initialization the SM should parse the policy and apply its >>>> settings to the discovered fabric elements. The following actions should >>>> be >>>> performed: >>>> * Parsing of policy >>>> * Node Group identification. Warning should be provided for each node not >>>> specified but found. >>>> * SL2VL settings validation should be checked: >>>> + A warning will be provided if there are no matching targets for the >>>> SL2VL >>>> setting statement. >>>> + An error message will be printed to the log file if an invalid >>>> setting is >>>> found. A setting is invalid if it refers to: >>>> - Non existing port numbers of the target devices >>>> - Unsupported VLs for the target device. In the later case the map to >>>> non existing VLs should be replaced to VL15 i.e. packets will be dropped. >>> I'm not sure it is optimal. We could have well documented or even >>> configurable mapping rule instead, then this will not limit devices with >>> higher capabilities. >> I'm open for suggestions. > > The rule like %(number of OpVLs)? Or even better - configurable mapping > rule? > >>>> * Only PR/MPR fields that have their component mask bit set should be >>>> compared. >>>> * For a rule to be "matching" a PR/MPR request all the rule fields should >>>> be >>>> "matching" their PR/MPR fields. Such that a PR/MPR request that does >>>> not have a component mask field set for one of the rule defined fields >>>> can >>>> not match that rule. >>>> * A PR/MPR request that have a component mask bit set for one of the >>>> fields >>>> that is not defined by the rule can match the rule. >>> Aren't last two too restrictive? SA can just to filter-out paths in >>> response to match rest of the rule. No? >> Not sure I'm following. >> The last bullet is not restrictive at all > > Right, but mostly I'm about previous bullet - where client _must_ set > component mask to match all fields. Look at this this way: when you define a match rule, you specify there only the parameters that the matching PR/MPR requests *must* comply with. e.g., if the matching rule has service ID, it means that PR/MPR requests that match this rule *must* have the matching service id. Same goes for all the other PR/MPR parameters. You can define a 'lighter' matching rule that will match more PR/MPR requests by checking a reduced set of key parameters. -- Yevgeny > Sasha > From mxhekxlt at matanata.com Fri Aug 3 10:46:47 2007 From: mxhekxlt at matanata.com (list) Date: Fri, 3 Aug 2007 08:46:47 -0900 Subject: [ofa-general] Re: Thank you, we are ready to give a loan for a low month payment Message-ID: An HTML attachment was scrubbed... URL: From krkumar2 at in.ibm.com Thu Aug 2 19:55:07 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Fri, 3 Aug 2007 08:25:07 +0530 Subject: [ofa-general] Re: [RFH] IPoIB retransmission when sending multiple WR's to device In-Reply-To: Message-ID: Hi Roland, Roland Dreier wrote on 08/02/2007 09:59:23 PM: > > On the same topic that I wrote about earlier, I put debugs > > in my code to store all skbs in bufferA when enqueing multiple > > skbs, and store all skbs to bufferB just before doing post. > > During post, I compare the two buffers to make sure that I am > > not posting in the wrong order, and that never happens. > > > > But I am getting a huge amount of retransmissions anyway, > > Why do you think the retransmissions are related to things being sent > out of order? Is it possible you're just sending much faster and > overrunning the receiver's queue of posted receives? I cannot be sure of that. But in regular code too, batching is done in qdisc_run() in a different sense - it sends out packets *iteratively*. In this case, I see only 225 retransmission for the entire run of all tests (while in my code, I see 100,000 or more (I think I gave wrong number earlier, this is the right one - 200 vs 100,000). Is there any way to avoid the situation you are talking about ? I am already setting recv_queue_size=4096 when loading ipoib (so for mthca too). Thanks, - KK From krkumar2 at in.ibm.com Thu Aug 2 20:48:29 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Fri, 3 Aug 2007 09:18:29 +0530 Subject: [ofa-general] Re: [RFH] IPoIB retransmission when sending multiple WR's to device Message-ID: Hi Roland, I did one more test to check the out-of-order theory. I changed my new API to be: /* Original code, unmodified */ ipoib_start_xmit() { original code } /* Added new xmit which is identical to original code but doesn't get the lock */ ipoib_start_xmit_nolock() { original code but without getting lock } /* Batching API */ ipoib_start_xmit_batch() { get_lock() while (skbs in queue) { ret = ipoib_start_xmit_nolock() } unlock() } This in effect is fast sends of multiple skbs while holding the lock once for all the skbs without going back to the IP stack. The only difference is that this creates one WR per skb instead of multiple WR's. I got identical retranmissions (around 240 for the full run) as compared to original code (225), while the multiple WR's code had >100,000 retransmissions. I agree that sending multiple WR's is still faster than this, but this is second best I could do, and still there is no increase in retransmissions. The tx_queue_len on ib0 is 8192, recv/send size are both 4K. The receiver shows no errors/drops in ifconfig nor netstat -s. Is there anything that can be concluded/suspected, or something else I could try ? The latest run of the batching API gave me 719,679 retransmissions for a 16 min test run of 16/64 threads (iperf), which comes to 600 retransmissions per sec, which is more than the retransmissions during the entire run for the regular code! thanks, - KK __________________ Hi Roland, Roland Dreier wrote on 08/02/2007 09:59:23 PM: > > On the same topic that I wrote about earlier, I put debugs > > in my code to store all skbs in bufferA when enqueing multiple > > skbs, and store all skbs to bufferB just before doing post. > > During post, I compare the two buffers to make sure that I am > > not posting in the wrong order, and that never happens. > > > > But I am getting a huge amount of retransmissions anyway, > > Why do you think the retransmissions are related to things being sent > out of order? Is it possible you're just sending much faster and > overrunning the receiver's queue of posted receives? I cannot be sure of that. But in regular code too, batching is done in qdisc_run() in a different sense - it sends out packets *iteratively*. In this case, I see only 225 retransmission for the entire run of all tests (while in my code, I see 100,000 or more (I think I gave wrong number earlier, this is the right one - 200 vs 100,000). Is there any way to avoid the situation you are talking about ? I am already setting recv_queue_size=4096 when loading ipoib (so for mthca too). Thanks, - KK From kliteyn at mellanox.co.il Thu Aug 2 21:25:45 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 3 Aug 2007 07:25:45 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-03:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Jul_31_14:27:42_2007 [c3bf9dc7b3063be57ca7a2cfca127937652053f3] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=467 Fail=53 Pass: 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 38 Stability IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 1 Stability IS1-16.topo From thomas.bub at thomson.net Thu Aug 2 23:37:53 2007 From: thomas.bub at thomson.net (Bub Thomas) Date: Fri, 3 Aug 2007 08:37:53 +0200 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event while wthere is a broken IB connection? Message-ID: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> Seems I had a wrong address so my previous mail didn't reach the reflector, hopefully it does this time. Our application is using OFED 1.1 under SLES10. It is a point to point IB connection from a high speed image scanner to a workstation. The application is out in the market for about a year. Now I get complains from the field that they do have dead pthreads hanging around that are stuck in the ib_cm_get_event call. For your explanation I'm creating a new pthread for every new attempt to connect from the workstation to the image scanner. This will happen only when one does re-power the image scanner while having a connection established, this might happen often as once a week to once a month. The reason for the complains now are that the thread's hanging in the ib_cm_get_event consume system resources and might block the system after some 40 power cyles. They don't want/can't restart the workstation application in order to get the old phreads destroyed. So my question is: How can I get a pthread out of the ib_cm_get_event call without using pthread_kill? Thomas Bub ........................................................................ ........ Thomas Bub Senior Software Engineer Grass Valley Germany GmbH Brunnenweg 9 64331 Weiterstadt, Germany Tel: +49 6150 104 147 Fax: +49 6150 104 656 Email: Thomas.Bub at thomson.net www.GrassValley.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From hnguyen at linux.vnet.ibm.com Fri Aug 3 01:36:20 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Fri, 3 Aug 2007 10:36:20 +0200 Subject: [ofa-general] Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space In-Reply-To: References: <200708021139.31695.hnguyen@linux.vnet.ibm.com> <200708021608.16436.arnd@arndb.de> Message-ID: <200708031036.20859.hnguyen@linux.vnet.ibm.com> From: Hoang-Nam Nguyen Date: Fri, 3 Aug 2007 09:44:56 +0200 Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space This patch utilizes remap_4k_pfn() as introduced by Paul M., for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281, to map ehca cq, qp firmware context (4k) to user space if kernel page size is 64k. For reason, why this is required, see also Paul's patch. In addition to that the kernel page offset of firmware context needs to be set in cq and qp response block so that user space can assemble the proper virtual address to use. An appropriate patch for libehca will follow for ofed-1.3. Signed-off-by: Hoang-Nam Nguyen --- drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++- drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++ drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +++--- 4 files changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index b5e9603..206d4eb 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -337,6 +337,8 @@ struct ehca_create_cq_resp { u32 cq_number; u32 token; struct ipzu_queue_resp ipz_queue; + u32 fw_handle_ofs; + u32 dummy; }; struct ehca_create_qp_resp { @@ -347,7 +349,7 @@ struct ehca_create_qp_resp { u32 qkey; /* qp_num assigned by ehca: sqp0/1 may have got different numbers */ u32 real_qp_num; - u32 dummy; /* padding for 8 byte alignment */ + u32 fw_handle_ofs; struct ipzu_queue_resp ipz_squeue; struct ipzu_queue_resp ipz_rqueue; }; diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index 81aff36..ed5d67f 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, resp.ipz_queue.queue_length = ipz_queue->queue_length; resp.ipz_queue.pagesize = ipz_queue->pagesize; resp.ipz_queue.toggle_state = ipz_queue->toggle_state; + resp.fw_handle_ofs = (u32) + (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof(resp))) { ehca_err(device, "Copy to udata failed."); goto create_cq_exit4; diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..66f632c 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp( queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue); if (HAS_RQ(my_qp)) queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue); + resp.fw_handle_ofs = (u32) + (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof resp)) { ehca_err(pd->device, "Copy to udata failed"); diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 4bc687f..be062f1 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, u64 vsize, physical; vsize = vma->vm_end - vma->vm_start; - if (vsize != EHCA_PAGESIZE) { + if (vsize >= EHCA_PAGESIZE) { ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start); return -EINVAL; } @@ -118,8 +118,8 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical); /* VM_IO | VM_RESERVED are set by remap_pfn_range() */ - ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT, - vsize, vma->vm_page_prot); + ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT, + vma->vm_page_prot); if (unlikely(ret)) { ehca_gen_err("remap_pfn_range() failed ret=%x", ret); return -ENOMEM; -- 1.5.2 From vlad at lists.openfabrics.org Fri Aug 3 01:42:37 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 3 Aug 2007 01:42:37 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070803-0100 daily build status Message-ID: <20070803084237.D3956E603BD@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.14 Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-8.el5 Failed: From vlad at lists.openfabrics.org Fri Aug 3 02:46:34 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 3 Aug 2007 02:46:34 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070803-0200 daily build status Message-ID: <20070803094634.95FA5E60836@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070803-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From krkumar2 at in.ibm.com Fri Aug 3 02:54:56 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Fri, 3 Aug 2007 15:24:56 +0530 Subject: [ofa-general] Re: [RFH] IPoIB retransmission when sending multiple WR's to device Message-ID: Hi Roland, (More results, not sure what all this adds up to) I did one last experiment :-) ipoib_start_xmit_frames() { while (skbs) { process skb & put on tx_wr, tx_ring, etc. if there are two skbs processed { send them out now, if there are more skbs in the batch list return MORE to IP. } } } qdisc_restart() { ret = dev->hard_start_xmit_frames(); switch (ret) { case OK, BUSY, LOCK, etc: Original code; break; case MORE: ret = 1; /* Call again to the driver */ } Even with this, I am getting large number of retransmissions. I am sending only two and returning back to IP (resulting in one send with 2 WR's), which is almost the same as what the original code does except it sends one and returns (resulting in two sends with 1 WR each). Also, if I change 2 to 1, the retranmission disappears. I also tried one more change : queue to tx_wr in reverse order to see if retransmission stops due to some weird sending in reverse, but no. When I keep this max number of skbs to send as 2, I get fewer retransmission (50,000 for 2 min run) and 220KB/s, and when it is increased to 256, I get high retransmission (200,000 for 2 min run) and better BW (235KB/s). Does this mean that sometimes multiple WR's are not getting sent out at all, or that sometimes only one is (and the other goes out via retransmission) ? Note: The Batching API is not called in most cases, it is called only if queue was stopped or tx lock was not got, and skbs accumulate in the qdisc queue. Thanks, - KK __________________ Hi Roland, I did one more test to check the out-of-order theory. I changed my new API to be: /* Original code, unmodified */ ipoib_start_xmit() { original code } /* Added new xmit which is identical to original code but doesn't get the lock */ ipoib_start_xmit_nolock() { original code but without getting lock } /* Batching API */ ipoib_start_xmit_batch() { get_lock() while (skbs in queue) { ret = ipoib_start_xmit_nolock() } unlock() } This in effect is fast sends of multiple skbs while holding the lock once for all the skbs without going back to the IP stack. The only difference is that this creates one WR per skb instead of multiple WR's. I got identical retranmissions (around 240 for the full run) as compared to original code (225), while the multiple WR's code had >100,000 retransmissions. I agree that sending multiple WR's is still faster than this, but this is second best I could do, and still there is no increase in retransmissions. The tx_queue_len on ib0 is 8192, recv/send size are both 4K. The receiver shows no errors/drops in ifconfig nor netstat -s. Is there anything that can be concluded/suspected, or something else I could try ? The latest run of the batching API gave me 719,679 retransmissions for a 16 min test run of 16/64 threads (iperf), which comes to 600 retransmissions per sec, which is more than the retransmissions during the entire run for the regular code! thanks, - KK __________________ Hi Roland, Roland Dreier wrote on 08/02/2007 09:59:23 PM: > > On the same topic that I wrote about earlier, I put debugs > > in my code to store all skbs in bufferA when enqueing multiple > > skbs, and store all skbs to bufferB just before doing post. > > During post, I compare the two buffers to make sure that I am > > not posting in the wrong order, and that never happens. > > > > But I am getting a huge amount of retransmissions anyway, > > Why do you think the retransmissions are related to things being sent > out of order? Is it possible you're just sending much faster and > overrunning the receiver's queue of posted receives? I cannot be sure of that. But in regular code too, batching is done in qdisc_run() in a different sense - it sends out packets *iteratively*. In this case, I see only 225 retransmission for the entire run of all tests (while in my code, I see 100,000 or more (I think I gave wrong number earlier, this is the right one - 200 vs 100,000). Is there any way to avoid the situation you are talking about ? I am already setting recv_queue_size=4096 when loading ipoib (so for mthca too). Thanks, - KK From duarte.dallimore at glb-gpa.at Thu Aug 2 23:06:30 2007 From: duarte.dallimore at glb-gpa.at (Lola Swain) Date: Fri, 3 Aug 2007 13:06:30 +0700 Subject: [ofa-general] I show you how far the rabbit hole goes Message-ID: <115456787.14377261086930@glb-gpa.at> -------------- next part -------------- A non-text attachment was scrubbed... Name: raroru.gif Type: image/gif Size: 12739 bytes Desc: not available URL: From amar.mudrankit at gmail.com Fri Aug 3 08:08:10 2007 From: amar.mudrankit at gmail.com (Amar Mudrankit) Date: Fri, 3 Aug 2007 20:38:10 +0530 Subject: [ofa-general] Understanding IPoIB-CM Message-ID: I have been looking at the IPoIB CM code and have some queries. I'd greatly appreciate it if some one could shed some light on it. Please correct me if I am wrong, but based on my understanding of the code: 1) From functions ipoib_cm_create_tx_qp and ipoib_cm_create_rx_qp, it looks like for communicating with a peer, we use one QP for purely doing TX and one QP for purely doing RX (created on receipt of a CM REQ). That means for every peer we are connected to, we use two QPs. Why not use a single QP for both TX and RX of data ? On receipt of a REQ from a peer or before trying to send data to peer, we could possibly check if we already have a QP connection to the peer (by maintaining and looking up a QP and peer UD QPN + GID mapping) and then just use it. Or is it not as simple as that ? On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case of simultaneous IB connection where REQs can cross paths. But the current implementation does not seem to handle this case. Is it because we create two QPs anyway and do not need to handle this special case ? 2] In the ipoib_cm_rep_handler, the skbs seem to be dequeued and queued again. I am referring to the code below. What does this exactly achieve ? skb_queue_head_init(&skqueue); spin_lock_irq(&priv->lock); set_bit(IPOIB_FLAG_OPER_UP, &p->flags); if (p->neigh) while ((skb = __skb_dequeue(&p->neigh->queue))) __skb_queue_tail(&skqueue, skb); spin_unlock_irq(&priv->lock); while ((skb = __skb_dequeue(&skqueue))) { skb->dev = p->dev; if (dev_queue_xmit(skb)) ipoib_warn(priv, "dev_queue_xmit failed " "to requeue packet\n"); } Thanks and Regards, Amar -------------- next part -------------- An HTML attachment was scrubbed... URL: From mshefty at ichips.intel.com Fri Aug 3 10:02:29 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Fri, 03 Aug 2007 10:02:29 -0700 Subject: [ofa-general] Understanding IPoIB-CM In-Reply-To: References: Message-ID: <46B35FA5.2090604@ichips.intel.com> > On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case > of simultaneous IB connection where REQs can cross paths. But the > current implementation does not seem to handle this case. Is it > because we create two QPs anyway and do not need to handle this > special case ? I'm not sure about the IPoIB RFC, but the ib_cm implementation currently doesn't support peer-to-peer connections. No one ever stated a need for it, so I left it unfinished. - Sean From rdreier at cisco.com Fri Aug 3 10:55:09 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 10:55:09 -0700 Subject: [ofa-general] Re: [RFC/PATCH] mthca: ensure alignment of doorbell writes In-Reply-To: <20070726014931.GL10235@sgi.com> (akepner@sgi.com's message of "Wed, 25 Jul 2007 18:49:31 -0700") References: <20070726014931.GL10235@sgi.com> Message-ID: Looking at this further... does mlx4 need the same treatment for writing CQ doorbells? How about libmthca/libmlx4 in userspace? - R. From rdreier at cisco.com Fri Aug 3 11:10:18 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 11:10:18 -0700 Subject: [ofa-general] Re: RFC: SRC API In-Reply-To: <20070729140431.GG16915@mellanox.co.il> (Michael S. Tsirkin's message of "Sun, 29 Jul 2007 17:04:31 +0300") References: <20070729140431.GG16915@mellanox.co.il> Message-ID: > - manage SRC domains OK, I guess, but why do we need so many functions to create, share, get shared, put SRC domains? How about just two functions: struct ibv_src_domain *ibv_open_src_domain(struct ibv_context *context, int fd, int oflag); where oflag is 0 or some combination of O_CREAT and O_EXCL, and: int ibv_close_src_domain(struct ibv_src_domain); Is there some case where explicitly unsharing an SRC domain is useful that can't be dealt with just by unlinking whatever file was used to create it? Other processes might be left with open file descriptors pointing to that file, but they could have already gotten the src domain anyway. > - I envision implementing this sharing mechanism in kernel by means > of a per-device tree, with inode as a key and domain object > as a value. Is the inode or the file descriptor the key? Earlier you wrote > 1:1 association between an SRC domain and a file. so there's some ambiguity. Also, what's the lifetime of SRC domains? Are they implicitly destroyed when all processes with a reference exit? - R. From rdreier at cisco.com Fri Aug 3 11:14:35 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 11:14:35 -0700 Subject: [ofa-general] socket buffer accounting with UDP/ipoib In-Reply-To: <1184761552.3520.9.camel@mtls03> (Eli Cohen's message of "Wed, 18 Jul 2007 15:25:52 +0300") References: <1183643723.25031.262.camel@mtls03> <1184761552.3520.9.camel@mtls03> Message-ID: > I made some experiments with iperf running on CM mode and TCP sockets. I > can see that there is no bad affect on BW (excel file attached). We did > see a slight improvement in packet loss in UDP mode with an application > supplied by a customer. So is this "slight improvement" worth the extra complexity added by having two different packet receive paths? From rdreier at cisco.com Fri Aug 3 11:21:22 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 11:21:22 -0700 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event while wthere is a broken IB connection? In-Reply-To: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> (Bub Thomas's message of "Fri, 3 Aug 2007 08:37:53 +0200") References: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> Message-ID: > Now I get complains from the field that they do have dead pthreads > hanging around that are stuck in the ib_cm_get_event call. They're stuck waiting for another CM event, right? And presumably no more events occur after IB_CM_TIMEWAIT_EXIT for a connection that dies. So I guess your application should have the thread terminate after it knows that the connection is done. > So my question is: How can I get a pthread out of the ib_cm_get_event > call without using pthread_kill? What's wrong with pthread_kill()? That seems like another perfectly valid solution -- ib_cm_get_event() is basically doing a read(), so a signal would wake up the read() and make the function return. - R. From akepner at sgi.com Fri Aug 3 14:16:44 2007 From: akepner at sgi.com (akepner at sgi.com) Date: Fri, 3 Aug 2007 14:16:44 -0700 Subject: [ofa-general] Re: [RFC/PATCH] mthca: ensure alignment of doorbell writes In-Reply-To: References: <20070726014931.GL10235@sgi.com> Message-ID: <20070803211644.GA32445@sgi.com> On Fri, Aug 03, 2007 at 10:55:09AM -0700, Roland Dreier wrote: > Looking at this further... does mlx4 need the same treatment for > writing CQ doorbells? Yes, looks like the same deal there. (I don't have any mlx4 h/w, so didn't check that.) > > How about libmthca/libmlx4 in userspace? > Yes and yes. What do you think about using the new "doorbell type" that was in the patch that started this thread? For mthca it was defined like this: union mthca_doorbell { __be64 val64; __be32 val32[2]; }; Or should we just add __attribute__ ((aligned(8))) to all the "__be32 doorbell[2]", etc. definitions? -- Arthur From vu at mellanox.com Fri Aug 3 14:25:48 2007 From: vu at mellanox.com (Vu Pham) Date: Fri, 03 Aug 2007 14:25:48 -0700 Subject: [ofa-general] [PATCH]mlx4: fix wc opcode for rdma_read, Message-ID: <46B39D5C.9050604@mellanox.com> Fix the work completion opcode for RDMA_READ operation. Signed-off-by: Vu Pham -------------- next part -------------- A non-text attachment was scrubbed... Name: wc_opcode.patch Type: text/x-diff Size: 489 bytes Desc: not available URL: From rdreier at cisco.com Fri Aug 3 14:26:52 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 14:26:52 -0700 Subject: [ofa-general] Re: [RFC/PATCH] mthca: ensure alignment of doorbell writes In-Reply-To: <20070803211644.GA32445@sgi.com> (akepner@sgi.com's message of "Fri, 3 Aug 2007 14:16:44 -0700") References: <20070726014931.GL10235@sgi.com> <20070803211644.GA32445@sgi.com> Message-ID: > What do you think about using the new > "doorbell type" that was in the patch that started > this thread? For mthca it was defined like this: > > union mthca_doorbell { > __be64 val64; > __be32 val32[2]; > }; > > Or should we just add __attribute__ ((aligned(8))) to > all the "__be32 doorbell[2]", etc. definitions? I was going to investigate how the assembly comes out if we just make these doorbell values be __be64, in particular on i386 and also ia64. But maybe the union is a good solution too. - R. From rdreier at cisco.com Fri Aug 3 14:29:27 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 03 Aug 2007 14:29:27 -0700 Subject: [ofa-general] [PATCH]mlx4: fix wc opcode for rdma_read, In-Reply-To: <46B39D5C.9050604@mellanox.com> (Vu Pham's message of "Fri, 03 Aug 2007 14:25:48 -0700") References: <46B39D5C.9050604@mellanox.com> Message-ID: thanks, applied From leng at viralcontact.com Sat Aug 4 00:54:50 2007 From: leng at viralcontact.com (Martin Lockwood) Date: Fri, 3 Aug 2007 22:54:50 -0900 Subject: [ofa-general] Photoshop CS3 Extended Message-ID: <01c7d621$4c1492b0$2fc5887c@leng> CREATIVE SUITE 2 retail price - $1199.00 save $1049.05 http://geocities.com/DarrinJenkins60/ From pradeeps at linux.vnet.ibm.com Fri Aug 3 17:34:07 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Fri, 03 Aug 2007 17:34:07 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ Message-ID: <46B3C97F.6030007@linux.vnet.ibm.com> This patch fixes the issues raised by Michael and Roland about receiving a packet on the Receive Queue of tx_qp in the NOSRQ case. This is solved by posting 1 WR for the RQ. Create a separate cq to handle this for NOSRQ only. This situation might occur when IPoIB CM inter operates with a non-Linux system. This patch is to be applied on top of the previous series of NOSRQ patches. Bulk of this code would not get tested in the normal case and is hard to test. Code review would help. Signed-off-by: Pradeep Satyanarayana --- --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-03 19:00:23.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-03 19:34:02.000000000 -0400 @@ -149,6 +149,55 @@ static int post_receive_nosrq(struct net return ret; } +static int alloc_skb_and_post_rq(struct ipoib_cm_tx *tx, u64 id, + struct ipoib_rx_buf *rx_buf) +{ + struct ipoib_dev_priv *priv = netdev_priv(tx->dev); + struct ib_recv_wr rx_wr, *bad_wr; + struct ib_sge sg_list; + struct sk_buff *skb; + u64 addr; + int ret; + + skb = dev_alloc_skb(CM_PACKET_SIZE); + if (unlikely(!skb)) + return -ENOMEM; + + /* + * IPoIB adds a 4 byte header. So we need 12 more bytes to align the + * IP header to a multiple of 16. + */ + skb_reserve(skb, 12); + + addr = ib_dma_map_single(priv->ca, skb->data, CM_PACKET_SIZE, + DMA_FROM_DEVICE); + if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { + dev_kfree_skb_any(skb); + return -EIO; + } + + rx_buf->skb = skb; + rx_buf->mapping = addr; + sg_list.addr = addr; + sg_list.length = CM_PACKET_SIZE; + sg_list.lkey = priv->mr->lkey; + + rx_wr.next = NULL; + rx_wr.wr_id = 0 | IPOIB_CM_OP_RECV; + rx_wr.sg_list = &sg_list; + rx_wr.num_sge = 1; + ret = ib_post_recv(tx->qp, &rx_wr, &bad_wr); + if (unlikely(ret)) { + ipoib_warn(priv, "post receive failed for rq tx buf\n"); + ib_dma_unmap_single(priv->ca, rx_buf->mapping, + CM_PACKET_SIZE, DMA_FROM_DEVICE); + dev_kfree_skb_any(rx_buf->skb); + rx_buf->skb = NULL; + } + + return ret; +} + static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev, u64 id, int frags, u64 mapping[IPOIB_CM_RX_SG]) @@ -348,8 +397,8 @@ static int ipoib_cm_send_rep(struct net_ } static void init_context_and_add_list(struct ib_cm_id *cm_id, - struct ipoib_cm_rx *p, - struct ipoib_dev_priv *priv) + struct ipoib_cm_rx *p, + struct ipoib_dev_priv *priv) { cm_id->context = p; p->jiffies = jiffies; @@ -791,6 +840,50 @@ repost_nosrq: (unsigned long long)wr_id); } +static void handle_rx_wc_rq(struct net_device *dev, struct ipoib_cm_tx *tx, + struct ib_wc *wc) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + u64 addr; + int ret; + struct sk_buff *skb; + + skb = tx->rx_buf.skb; + addr = tx->rx_buf.mapping; + + if (unlikely(wc->status != IB_WC_SUCCESS)) { + ipoib_dbg(priv, "receive error status=%d on RQ of tx_qp\n", + wc->status); + ++priv->stats.rx_dropped; + goto err_rq; + } + + ret = alloc_skb_and_post_rq(tx, 0, &tx->rx_buf); + if (unlikely(ret)) { + ipoib_dbg(priv, "failed to allocate and post receive buffer" + "for RQ of tx_qp\n"); + ++priv->stats.rx_dropped; + goto err_rq; + } + + ib_dma_unmap_single(priv->ca, addr, CM_PACKET_SIZE, DMA_FROM_DEVICE); + skb_put(skb, wc->byte_len); + skb->protocol = ((struct ipoib_header *) skb->data)->proto; + skb_reset_mac_header(skb); + skb_pull(skb, IPOIB_ENCAP_LEN); + + dev->last_rx = jiffies; + ++priv->stats.rx_packets; + priv->stats.rx_bytes += skb->len; + skb->dev = dev; + skb->pkt_type = PACKET_HOST; + + netif_rx(skb); + +err_rq: + ipoib_warn(priv, "Catastrophic failure on RQ ox tx_qp\n"); +} + void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -958,6 +1051,19 @@ static void ipoib_cm_tx_completion(struc } while (n == IPOIB_NUM_WC); } +static void ipoib_cm_tx_rq_handler(struct ib_cq *cq, void *tx_ptr) +{ + struct ipoib_cm_tx *tx = tx_ptr; + struct ib_wc wc; + + printk(KERN_WARNING "Packet received on RQ of tx_qp!!! Must be from" + "a non-Linux system!\n"); + ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); + ib_poll_cq(cq, 1, &wc); + + handle_rx_wc_rq(tx->dev, tx, &wc); +} + int ipoib_cm_dev_open(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -1162,19 +1268,23 @@ static int ipoib_cm_rep_handler(struct i return 0; } -static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ib_cq *cq) +static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_cm_tx *p) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = {}; - attr.recv_cq = priv->cq; attr.srq = priv->cm.srq; attr.cap.max_send_wr = ipoib_sendq_size; - attr.cap.max_recv_wr = 0; attr.cap.max_send_sge = 1; - attr.cap.max_recv_sge = 0; attr.sq_sig_type = IB_SIGNAL_ALL_WR; attr.qp_type = IB_QPT_RC; - attr.send_cq = cq; + attr.send_cq = p->cq; + if (priv->cm.srq) + attr.recv_cq = priv->cq; + else { + attr.recv_cq = p->rcq; + attr.cap.max_recv_wr = 1; + attr.cap.max_recv_sge = 1; + } return ib_create_qp(priv->pd, &attr); } @@ -1268,13 +1378,45 @@ static int ipoib_cm_tx_init(struct ipoib goto err_req_notify; } - p->qp = ipoib_cm_create_tx_qp(p->dev, p->cq); + /* There is a possibility that a non-Linux system may send to the RQ + * of the tx_qp. In order to handle that, for the NOSRQ case + * we create a seperate cq and also post one WR on the RQ. + * For SRQ this is not an issue since WRs are already posted through + * the Shared RQ. + */ + + if (!priv->cm.srq) { + p->rcq = ib_create_cq(priv->ca, ipoib_cm_tx_rq_handler, NULL, p, 1, 0); + if (IS_ERR(p->rcq)) { + ret = PTR_ERR(p->rcq); + ipoib_warn(priv, "failed to allocate tx rcq: %d\n", ret); + goto err_cq; + } + + ret = ib_req_notify_cq(p->rcq, IB_CQ_NEXT_COMP); + if (ret) { + ipoib_warn(priv, "failed to request completion notification: %d\n", ret); + goto err_req_notify; + } + } + + p->qp = ipoib_cm_create_tx_qp(p->dev, p); if (IS_ERR(p->qp)) { ret = PTR_ERR(p->qp); ipoib_warn(priv, "failed to allocate tx qp: %d\n", ret); goto err_qp; } + if (!priv->cm.srq) { + ret = alloc_skb_and_post_rq(p, 0, &p->rx_buf); + if (unlikely(ret)) { + ipoib_warn(priv, "failed to allocate and post receive" + " buffer for RQ of tx_qp\n"); + ret = -ENOMEM; + goto err_qp; + } + } + p->id = ib_create_cm_id(priv->ca, ipoib_cm_tx_handler, p); if (IS_ERR(p->id)) { ret = PTR_ERR(p->id); @@ -1309,8 +1451,11 @@ err_req_notify: err_qp: p->qp = NULL; ib_destroy_cq(p->cq); + if (p->rcq) + ib_destroy_cq(p->rcq); err_cq: - p->cq = NULL; + p->cq = NULL; + p->rcq = NULL; err_tx: return ret; } @@ -1332,6 +1477,9 @@ static void ipoib_cm_tx_destroy(struct i if (p->cq) ib_destroy_cq(p->cq); + if (p->rcq) + ib_destroy_cq(p->rcq); + if (test_bit(IPOIB_FLAG_NETIF_STOPPED, &p->flags)) netif_wake_queue(p->dev); @@ -1347,6 +1495,12 @@ static void ipoib_cm_tx_destroy(struct i kfree(p->tx_ring); } + if (!priv->cm.srq) { + ib_dma_unmap_single(priv->ca, p->rx_buf.mapping, + CM_PACKET_SIZE, DMA_FROM_DEVICE); + dev_kfree_skb_any(p->rx_buf.skb); + } + kfree(p); } From stanleysufficool at roadrunner.com Fri Aug 3 19:34:04 2007 From: stanleysufficool at roadrunner.com (Stanley Sufficool) Date: Fri, 03 Aug 2007 19:34:04 -0700 Subject: [ofa-general] OpensSM and Multiple Ports Message-ID: <1186194845.6031.28.camel@gentoo-linux.localdomain> I am trying to get opensm to manage two separate subnets, one per IB port on an Infinihost II PCIe HCA. I can specify PORT=1 and opensmd will start without error. On the second sm management node, I start with PORT=2 in opensm.conf and opensm fails with a timeout error. I am using Gentoo Linux with the mlx HCA and MAD, UMAD, etc... built into kernel 2.6.20. My goal is to get ipoib working. IPoIB is being flakey with duplicated ethernet MAC addresses ( MAC: 02:00:00:00:01 & :02 ) . This may be resolved when opensm is working? I am also hoping to get SRP with Windows initiators and a Linux target using the SRPT kernel module. But failing that, I will regress to iSCSI with hopes of a future iSER target/initiator support. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Fri Aug 3 21:06:25 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 4 Aug 2007 07:06:25 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-04:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Jul_31_14:27:42_2007 [c3bf9dc7b3063be57ca7a2cfca127937652053f3] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=465 Fail=55 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 LidMgr IS1-16.topo 38 Multicast IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo 11 Multicast IS3-loop.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 2 Multicast IS3-loop.topo 1 Multicast IS1-16.topo From stanleysufficool at roadrunner.com Fri Aug 3 21:10:57 2007 From: stanleysufficool at roadrunner.com (Stanley Sufficool) Date: Fri, 03 Aug 2007 21:10:57 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST Message-ID: <1186200658.6031.44.camel@gentoo-linux.localdomain> SCST redefined the scst_tgt_template and it no longer contains the tm_sync_reply field. Also an extra arg is required for scst_register. Consequently, the build fails. I searched through past SCST versions and docs but could not find what this was for and it seems to be set to zero (not used?) in this struct anyhow. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: srpt_scst.patch Type: text/x-patch Size: 693 bytes Desc: not available URL: From stanleysufficool at roadrunner.com Fri Aug 3 21:12:56 2007 From: stanleysufficool at roadrunner.com (Stanley Sufficool) Date: Fri, 03 Aug 2007 21:12:56 -0700 Subject: [ofa-general] [PATCH] srpt_inc branch README update Message-ID: <1186200777.6031.47.camel@gentoo-linux.localdomain> Bring the srpt_inc branch README up to date with online version ( http://www.mellanox.com/pdf/products/software/Gen2_SRPT_README.txt ) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: srpt_inc-README.patch Type: text/x-patch Size: 4155 bytes Desc: not available URL: From rajouri.jammu at gmail.com Fri Aug 3 23:09:00 2007 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Fri, 3 Aug 2007 23:09:00 -0700 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event? In-Reply-To: <46B2179F.3070902@ichips.intel.com> References: <29C41D06A093C1449B2650D43A833E49EB37BE@WDTSSMAIL02.eu.thmulti.com> <29C41D06A093C1449B2650D43A833E49EB37C4@WDTSSMAIL02.eu.thmulti.com> <46B2179F.3070902@ichips.intel.com> Message-ID: <3307cdf90708032309vabdbe7dld2bd5896f75c65e@mail.gmail.com> How about using a single thread and a single event channel for all connections? On 8/2/07, Sean Hefty wrote: > > For your explanation I'm creating a new pthread for every new attempt to > > connect from the workstation to the image scanner. > > > > This will happen only when one does re-power the image scanner while > > having a connection established, this might happen often as once a week > > to once a month. > > > > The reason for the complains now are that the thread's hanging in the > > ib_cm_get_event consume system resources and might block the system > > after some 40 power cyles. > > > > They don't want/can't restart the workstation application in order to > > get the old phreads destroyed. > > > > > > > > So my question is: How can I get a pthread out of the ib_cm_get_event call? > > I don't know of a way out of this. Maybe someone else on the list does. > Anyone? > > - Sean > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From vlad at lists.openfabrics.org Sat Aug 4 01:38:59 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 4 Aug 2007 01:38:59 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070804-0100 daily build status Message-ID: <20070804083859.3025BE6084A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Sat Aug 4 02:48:22 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 4 Aug 2007 02:48:22 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070804-0200 daily build status Message-ID: <20070804094822.96B5CE6084A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070804-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From amheidgioz at aerometric.com Sat Aug 4 02:51:39 2007 From: amheidgioz at aerometric.com (william schmidt) Date: Sat, 04 Aug 2007 10:51:39 +0100 Subject: [ofa-general] This is it Message-ID: <427001c7d685$6fc71160$625da5f2@amheidgioz> Save big time on your medication. We have low prices, express delivery and the top quality brands and basic drugs With our firm you do not need a prescription! We have more than 2,000 medication Visit us at: www.drugerabs.org "And such," said boast view the stranger, "will invite be always the parturient most backward to assert any such thing. Knaves wi burst A second degree of this quality seems to strung arise from the quality head. This is, indeed, no annually other than the fac wound "Believe me, madam," smash said briefly cautious he, "I do not speak the common cant of one in my unhappy situation. Before The commander gentle revolting unusual of the forces, and the commander produce of the house, were now both arrived; for the former b parcel time bathe The hook letter was as follows: From hal.rosenstock at gmail.com Sat Aug 4 03:55:49 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sat, 4 Aug 2007 06:55:49 -0400 Subject: [ofa-general] OpensSM and Multiple Ports In-Reply-To: <1186194845.6031.28.camel@gentoo-linux.localdomain> References: <1186194845.6031.28.camel@gentoo-linux.localdomain> Message-ID: On 8/3/07, Stanley Sufficool wrote: > I am trying to get opensm to manage two separate subnets, one per IB port on > an Infinihost II PCIe HCA. I can specify PORT=1 and opensmd will start > without error. On the second sm management node, I start with PORT=2 in > opensm.conf and opensm fails with a timeout error. Not sure what PORT is but you can start OpenSM on a specific port by using the -g (or -guid) option. Something like: opensm -g=0x12345678 where 0x12345678 is the port (not node) GUID > I am using Gentoo Linux with the mlx HCA and MAD, UMAD, etc... built into > kernel 2.6.20. > > My goal is to get ipoib working. IPoIB is being flakey with duplicated > ethernet MAC addresses ( MAC: 02:00:00:00:01 & :02 ) . This may be resolved > when opensm is working? IPoIB won't work until the ports are brought up by the SM. Not sure about the duplicated MACs. I don't understand this. -- Hal > I am also hoping to get SRP with Windows initiators and a Linux target using > the SRPT kernel module. But failing that, I will regress to iSCSI with hopes > of a future iSER target/initiator support. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From swise at opengridcomputing.com Sat Aug 4 08:50:26 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Sat, 04 Aug 2007 10:50:26 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B2523A.2090109@ichips.intel.com> References: <000001c7d3cf$97a0b050$9c98070a@amr.corp.intel.com> <46B09177.6040509@opengridcomputing.com> <46B0C78E.1030306@ichips.intel.com> <46B249DE.8020409@opengridcomputing.com> <46B2523A.2090109@ichips.intel.com> Message-ID: <46B4A042.1050005@opengridcomputing.com> Sean Hefty wrote: >> If we get rid of the rdma_cm specific port spaces, do we then reduce >> the valid possible spaces to just TCP and UDP? Or what? In the >> sockets paradigm, the socket is explicitly bound to a protocol space >> when its created (based on the protocol id). Do you think we need to >> change the rdma_cm_id to have such a concept? IE when you create the >> cm_id, you say your intended QP type or port space? The current API >> lends itself to somone incorrectly choosing a port space, by the way. > > Currently, the RDMA port space implies the QP type (RC or UD). We're > not tied to any specific protocol when we create the rdma_cm_id, since > we don't know what type of RDMA device we'll end up using. So, I don't > think we want users to specify a protocol. > >> But should we really change the API that drastically? Or just keep >> the port spaces and make PS_TCP share the host's port space. > > I don't want to break the user space API, if it can be helped. SDP is > kind of a problem, in that the rdma_cm needs to distinguish between SDP > as a user, versus someone using RDMA_PS_TCP. SDP maps between the RDMA > port space and real TCP port space. I need to get some details on how > SDP uses the rdma_cm, like whether it uses wild card port numbers. > >> Maybe the rdma-cm port spaces should really be IB, IWARP, or BOTH. IB >> has its own port space, and IWARP or BOTH gets the TCP port space. > > I thought about doing something like this, but I'm not sure there would > be much use of just IB or just iWarp, when the user can specify both. I > even considered pushing the problem into the iWarp CM, but that seems > like a more complex implementation with no benefit unless there are > users of just an IB port space. > > At this point, my thoughts are to take your original patch, remove the > rdma_cm port space structures and functions, and figure out how to > handle SDP. > Lemme know how I can help. I certainly can test any patches on my 8 node iwarp cluster. Steve. From claimsagentscott_msw at yahoo.co.uk Sat Aug 4 09:14:51 2007 From: claimsagentscott_msw at yahoo.co.uk (Online Co-ordinator) Date: Sat, 4 Aug 2007 11:14:51 -0500 (CDT) Subject: [ofa-general] Microsoft Winner Message-ID: <63679.77.220.1.94.1186244091.squirrel@gastoncollins.com> REF NO: MSW-L/200-26937 BATCH NO: 2005MJL-01 ELECTRONIC MAIL AWARD WINNING NOTIFICATION We happily announce to you the draw (#622) of the MICROSOFT WORD LOTTERY, online International program held on 25th July 2007. You are therefore been approved to claim a total sum of 5,500,000.00 GBP. Your email was choosen this quarter. All participants for the online version were selected randomly from World Wide Web sites through computer draw system and extracted from over 100,000 unions, associations, and corporate bodies that are listed online.VERIFICATION FORM BELOW: *Name of Winner: *Address: *City/State: *Nationlity:*Sex: *EmailAddress*Phone no:*Occupation:*Age:*Amount won:. The claims officer contact email is: CONTACT THE FIDUCIARY AGENT BELOW FOR MORE DETAILS: Name: Mr Riddick Scot Email: contact_claimsagentscott at yahoo.co.uk Phone Number: 07045702054, 07031925332 Congratulations once more from all members and staffs of this program. Thank you for being part of our online promotional lottery program. Yours Truly, Will Sheward Online Co-ordinator © 2007 Microsoft Corporation From stanleysufficool at roadrunner.com Sat Aug 4 09:46:34 2007 From: stanleysufficool at roadrunner.com (Stanley Sufficool) Date: Sat, 04 Aug 2007 09:46:34 -0700 Subject: [ofa-general] OpensSM and Multiple Ports In-Reply-To: References: <1186194845.6031.28.camel@gentoo-linux.localdomain> Message-ID: <1186245995.6031.79.camel@gentoo-linux.localdomain> On Sat, 2007-08-04 at 06:55 -0400, Hal Rosenstock wrote: > On 8/3/07, Stanley Sufficool wrote: > > I am trying to get opensm to manage two separate subnets, one per IB port on > > an Infinihost II PCIe HCA. I can specify PORT=1 and opensmd will start > > without error. On the second sm management node, I start with PORT=2 in > > opensm.conf and opensm fails with a timeout error. > > Not sure what PORT is but you can start OpenSM on a specific port by > using the -g (or -guid) option. Something like: > opensm -g=0x12345678 where 0x12345678 is the port (not node) GUID There are 2 options for specifying the port in opensm.conf, one is the port GUID and one is the port number (PORT=1) . OpenSM seems to correctly derive the port GUID from the port number, however I will try using the GUID instead. > > > I am using Gentoo Linux with the mlx HCA and MAD, UMAD, etc... built into > > kernel 2.6.20. > > > > My goal is to get ipoib working. IPoIB is being flakey with duplicated > > ethernet MAC addresses ( MAC: 02:00:00:00:01 & :02 ) . This may be resolved > > when opensm is working? > > IPoIB won't work until the ports are brought up by the SM. Not sure > about the duplicated MACs. I don't understand this. > Ethernet MAC addresses as shown in Windows ipconfig for the IBoIP interfaces. They are duplicated because the LIDs are not being renumbered by the manager. > -- Hal > > > I am also hoping to get SRP with Windows initiators and a Linux target using > > the SRPT kernel module. But failing that, I will regress to iSCSI with hopes > > of a future iSER target/initiator support. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An HTML attachment was scrubbed... URL: From mst at dev.mellanox.co.il Sat Aug 4 10:20:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sat, 4 Aug 2007 20:20:28 +0300 Subject: [ofa-general] Re: RFC: SRC API In-Reply-To: References: <20070729140431.GG16915@mellanox.co.il> Message-ID: <20070804172028.GA5540@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: RFC: SRC API > > > - manage SRC domains > > OK, I guess, but why do we need so many functions to create, share, > get shared, put SRC domains? > > How about just two functions: > > struct ibv_src_domain *ibv_open_src_domain(struct ibv_context *context, > int fd, int oflag); > > where oflag is 0 or some combination of O_CREAT and O_EXCL, and: > > int ibv_close_src_domain(struct ibv_src_domain); > > > Is there some case where explicitly unsharing an SRC domain is useful > that can't be dealt with just by unlinking whatever file was used to > create it? Other processes might be left with open file descriptors > pointing to that file, but they could have already gotten the src > domain anyway. OK. > > - I envision implementing this sharing mechanism in kernel by means > > of a per-device tree, with inode as a key and domain object > > as a value. > > Is the inode or the file descriptor the key? Earlier you wrote > > > 1:1 association between an SRC domain and a file. > > so there's some ambiguity. The inode. > Also, what's the lifetime of SRC domains? Are they implicitly > destroyed when all processes with a reference exit? Yes, that's the idea. -- MST From mst at dev.mellanox.co.il Sat Aug 4 10:31:17 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sat, 4 Aug 2007 20:31:17 +0300 Subject: [ofa-general] Re: [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler In-Reply-To: References: Message-ID: <20070804173117.GB5540@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCH] mad.c: Fix memory leak in switch handling and improve error handling in ib_mad_recv_done_handler > > by the way, I had to apply this by hand, because it seems gmail is > destroying the whitespace in the patches. I'm not sure if there's any > way to send patches via gmail except as attachments unfortunately. I'm using gmail smtp service and haven't seen any problems yet. This is mosty likely the user agent's fault. -- MST From sashak at voltaire.com Sat Aug 4 11:57:49 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 4 Aug 2007 21:57:49 +0300 Subject: [ofa-general] Re: [PATCH] osm: cosmetics - removing trailing blanks In-Reply-To: <46B1D7BB.2090208@dev.mellanox.co.il> References: <46B1D7BB.2090208@dev.mellanox.co.il> Message-ID: <20070804185749.GB15811@sashak.voltaire.com> On 16:10 Thu 02 Aug , Yevgeny Kliteynik wrote: > Hi Sasha. > > This patch removes trailing blanks in bunch of files > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Sat Aug 4 12:35:13 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 4 Aug 2007 22:35:13 +0300 Subject: [ofa-general] Re: OpenSM "stuck" - user level MAD library seems to be timing out In-Reply-To: References: Message-ID: <20070804193513.GA22562@sashak.voltaire.com> Hi Lan, On 12:33 Thu 02 Aug , lbt wrote: > Hi Sasha, > > I am hitting a problem where the user level MAD library seems to be timing > out, causing the ports to be stuck in "INIT" state because the subnet has no > "Master" SM available. The system is still in this state, so if there are > any suggestions on what other type of debug info I could collect or clues to > what the problem might be, it would be much apprceciated :) > > I have 3 machines (OFED 1.1. stack, Opensm v2.0.5), where 2 of them are > running open SM, connected by an IB switch. Several tests were being done > pulling IB cables, but not touching at all the IB connections between the > Master SM and the IB switch, or rebooting the IB switch (i.e. no SM > migration should be occurring). Everything was working fine, until at one > point, I pull the IB cable on the IB switch of the lower priority (standby) > SM. For some reason, this starts causing problems on the higher priority > Master SM. The higher priority SM now thinks it's in Standby state, and the > lower priority SM's MAD packets are timing out. It is odd because, I would > not expect any effect on the higher priority SM (as it's IB connections are > not being affected). And not sure why MAD packets are timing out on the > lower priority SM. Rebooting the lower priority SM and replugging IB cables > into different ports on the IB switch, didn't help. Is it reproducible or randomly happened problem? > Lower priority SM: (packets timeout) > [root at vortex3l-83 ~]# sminfo -d -e -P 1 > ibwarn: [26764] smp_query: attr 21 mod 0 route DR path [0] > ibwarn: [26764] mad_rpc: data offs 64 sz 64 > mad data > 0000 0000 0000 0000 fe80 0000 0000 0000 > 0003 0001 0251 0a6a 0000 0000 0103 0302 > 1252 0011 4040 0008 0804 ff40 0000 0000 > 0000 2012 1088 0000 0000 0000 0000 0000 > ibwarn: [26764] smp_query: attr 32 mod 0 route Lid 1 It is possible that Master SM dropped routing to lid 1 node (which was disconnected some time before Master became StandBy). I suppose sminfo using direct path should work. Sasha > ibwarn: [26764] _do_madrpc: retry 1 (timeout 1000 ms) > ibwarn: [26764] _do_madrpc: retry 2 (timeout 1000 ms) > ibwarn: [26764] _do_madrpc: timeout after 3 retries, 3000 ms > sminfo: iberror: [pid 26764] main: failed: query > > Higher priority SM: (thinks its Standby now) > [root at vortex3l-84 log]# sminfo -d -e -P 1 > ibwarn: [2487] smp_query: attr 21 mod 0 route DR path [0] > ibwarn: [2487] mad_rpc: data offs 64 sz 64 > mad data > 0000 0000 0000 0000 fe80 0000 0000 0000 > 0002 0003 0251 0a6a 0000 0000 0103 0302 > 1252 0011 4040 0008 0804 ff40 0000 0000 > 0000 2012 1088 0000 0000 0000 0000 0000 > ibwarn: [2487] smp_query: attr 32 mod 0 route Lid 3 > ibwarn: [2487] mad_rpc: data offs 64 sz 64 > mad data > 0050 4501 4a3a 0001 0000 0000 0000 0000 > 0000 020e 0200 0000 0000 0000 0000 0000 > 0000 0000 0000 0000 0000 0000 0000 0000 > 0000 0000 0000 0000 0000 0000 0000 0000 > sminfo: sm lid 3 sm guid 0x5045014a3a0001, activity count 526 priority 0 > state 2 SMINFO_STANDBY > > Just another data point, but each machine happens to have 2 HCA ports, port > 1 and port 2. Port 1 is connected to different subnet than port2. During all > these steps, port2 subnet is still fine and working OK. The problem > described above was being seen with the port 1 subnet only. > > Thanks! > Lan From sashak at voltaire.com Sat Aug 4 12:38:50 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 4 Aug 2007 22:38:50 +0300 Subject: [ofa-general] Re: nightly osm_sim report 2007-08-04:normal completion In-Reply-To: References: Message-ID: <20070804193850.GB22562@sashak.voltaire.com> Hi Yevgeny, On 07:06 Sat 04 Aug , kliteyn at mellanox.co.il wrote: > > Failures: > 39 Pkey IS1-16.topo > 13 Pkey IS3-128.topo > 2 Multicast IS3-loop.topo > 1 Multicast IS1-16.topo Any info about Multicast failures? Also do you know was Stability failure in previous run was random simulator/test issue or something related to OpenSM? Sasha From sashak at voltaire.com Sat Aug 4 12:53:16 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 4 Aug 2007 22:53:16 +0300 Subject: [ofa-general] [PATCH] opensm/osm_node_info_rcv.c: sprintf usage improvements Message-ID: <20070804195316.GC22562@sashak.voltaire.com> sprintf() usage improvements. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_node_info_rcv.c | 26 +++++++++++--------------- 1 files changed, 11 insertions(+), 15 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 3ecf57a..cdd0e7a 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -142,23 +142,21 @@ __osm_ni_rcv_set_links( In this case, just continue. There will be another heavy sweep immediately after, when the subnet is stable again. */ - char line[BUF_SIZE]; char dr_new_path[BUF_SIZE]; char dr_old_path[BUF_SIZE]; + int n; uint32_t i; - osm_dr_path_t *p_path = NULL, *p_old_path = NULL; + osm_dr_path_t *p_path; p_physp = osm_node_get_physp_ptr( p_node, port_num ); sprintf( dr_new_path, "no_path_available" ); p_path = osm_physp_get_dr_path_ptr( p_physp ); if ( p_path ) { - sprintf( dr_new_path, "new path:" ); + n = sprintf( dr_new_path, "new path:" ); for (i = 0; i <= p_path->hop_count; i++ ) - { - sprintf( line, "[%X]", p_path->path[i] ); - strcat( dr_new_path, line ); - } + n += snprintf(dr_new_path + n, sizeof(dr_new_path) - n, "[%X]", + p_path->path[i]); } p_old_neighbor_node = osm_node_get_remote_node( @@ -167,15 +165,13 @@ __osm_ni_rcv_set_links( p_old_neighbor_node, old_neighbor_port_num); sprintf( dr_old_path, "no_path_available" ); - p_old_path = osm_physp_get_dr_path_ptr( p_old_physp ); - if ( p_old_path ) + p_path = osm_physp_get_dr_path_ptr( p_old_physp ); + if ( p_path ) { - sprintf( dr_old_path, "old_path:" ); - for (i = 0; i <= p_old_path->hop_count; i++ ) - { - sprintf( line, "[%X]", p_old_path->path[i] ); - strcat( dr_old_path, line ); - } + n = sprintf( dr_old_path, "old_path:" ); + for (i = 0; i <= p_path->hop_count; i++ ) + n += snprintf(dr_old_path, sizeof(dr_old_path) - n, "[%X]", + p_path->path[i]); } osm_log( p_rcv->p_log, OSM_LOG_ERROR, -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sat Aug 4 13:20:56 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sat, 4 Aug 2007 23:20:56 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <46B25AD1.20301@dev.mellanox.co.il> References: <46A283B6.1070105@dev.mellanox.co.il> <20070723002010.GU27878@sashak.voltaire.com> <46A89608.9010709@dev.mellanox.co.il> <20070731160223.GF29844@sashak.voltaire.com> <46B25AD1.20301@dev.mellanox.co.il> Message-ID: <20070804202056.GD22562@sashak.voltaire.com> On 01:29 Fri 03 Aug , Yevgeny Kliteynik wrote: > > OK, we can use names of QoS levels instead of sequential numbers. > Here are the details: > + in qos-level new mandatory keyword 'name' will be added: "name: > single_string" It could be even simpler: qos-level name ... > + in qos-match-rules, the 'qos-level-sn' keyword will be replaced by > 'qos-level-name' and matching just by 'qos-level' (numbers still be perfectly usable since it is string as swell). > + qos-level with name "default"/"DEFAULT" is a must. Probably single 'qos-level' without name could become default one? > It is the default level that would be applied to any PR/MPR request that > didn't > match any existing match rule > + any match rule may explicitly refer to the default qos-level by > specifying > its name in the qos-level-name > > > >>>> 9. OpenSM features > >>>> ------------------- > >>>> The QoS related functionality to be provided by OpenSM can be split > >>>> into two > >>>> main parts: > >>>> > >>>> 3.1. Fabric Setup > >>>> During fabric initialization the SM should parse the policy and apply > >>>> its > >>>> settings to the discovered fabric elements. The following actions > >>>> should be > >>>> performed: > >>>> * Parsing of policy > >>>> * Node Group identification. Warning should be provided for each node > >>>> not > >>>> specified but found. > >>>> * SL2VL settings validation should be checked: > >>>> + A warning will be provided if there are no matching targets for the > >>>> SL2VL > >>>> setting statement. > >>>> + An error message will be printed to the log file if an invalid > >>>> setting is > >>>> found. A setting is invalid if it refers to: > >>>> - Non existing port numbers of the target devices > >>>> - Unsupported VLs for the target device. In the later case the map > >>>> to non existing VLs should be replaced to VL15 i.e. packets will > >>>> be dropped. > >>> I'm not sure it is optimal. We could have well documented or even > >>> configurable mapping rule instead, then this will not limit devices with > >>> higher capabilities. > >> I'm open for suggestions. > > The rule like %(number of OpVLs)? Or even better - configurable mapping > > rule? > >>>> * Only PR/MPR fields that have their component mask bit set should be > >>>> compared. > >>>> * For a rule to be "matching" a PR/MPR request all the rule fields > >>>> should be > >>>> "matching" their PR/MPR fields. Such that a PR/MPR request that does > >>>> not have a component mask field set for one of the rule defined > >>>> fields can > >>>> not match that rule. > >>>> * A PR/MPR request that have a component mask bit set for one of the > >>>> fields > >>>> that is not defined by the rule can match the rule. > >>> Aren't last two too restrictive? SA can just to filter-out paths in > >>> response to match rest of the rule. No? > >> Not sure I'm following. > >> The last bullet is not restrictive at all > > Right, but mostly I'm about previous bullet - where client _must_ set > > component mask to match all fields. > > Look at this this way: when you define a match rule, you specify there > only the parameters that the matching PR/MPR requests *must* comply with. > e.g., if the matching rule has service ID, it means that PR/MPR requests > that match this rule *must* have the matching service id. > Same goes for all the other PR/MPR parameters. > > You can define a 'lighter' matching rule that will match more PR/MPR > requests by checking a reduced set of key parameters. Right, and I thought about reducing number of rules. :) Sasha From mst at dev.mellanox.co.il Sat Aug 4 15:02:06 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 5 Aug 2007 01:02:06 +0300 Subject: [ofa-general] Re: Understanding IPoIB-CM In-Reply-To: References: Message-ID: <20070804220206.GA15821@mellanox.co.il> > Quoting Amar Mudrankit : > Subject: Understanding IPoIB-CM > > > I have been looking at the IPoIB CM code and have some queries. I'd greatly > appreciate it if some one could shed some light on it. > > Please correct me if I am wrong, but based on my understanding of the code: > > 1) From functions ipoib_cm_create_tx_qp and ipoib_cm_create_rx_qp, it looks > like for communicating with a peer, we use one QP for purely > doing TX and one QP for purely doing RX (created on receipt of a CM REQ). > That means for every peer we are connected to, we use two > QPs. Why not use a single QP for both TX and RX of data ? On receipt of a REQ > from a peer or before trying to send data to peer, we > could possibly check if we already have a QP connection to the peer (by > maintaining and looking up a QP and peer UD QPN + GID > mapping) and then just use it. Or is it not as simple as that ? Sure, this could work. One'd have to be sure to cover all error cases though. > On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case of > simultaneous IB connection where REQs can cross paths. > But the current implementation does not seem to handle this case. Is it > because we create two QPs anyway and do not need to handle > this special case ? Yes. > 2] In the ipoib_cm_rep_handler, the skbs seem to be dequeued and queued again. > I am referring to the code below. What does this exactly > achieve ? > > skb_queue_head_init(&skqueue); > spin_lock_irq(&priv->lock); > set_bit(IPOIB_FLAG_OPER_UP, &p->flags); > if (p->neigh) > while ((skb = __skb_dequeue(&p->neigh->queue))) > __skb_queue_tail(&skqueue, skb); > spin_unlock_irq(&priv->lock); > while ((skb = __skb_dequeue(&skqueue))) { > skb->dev = p->dev; > if (dev_queue_xmit(skb)) > ipoib_warn(priv, "dev_queue_xmit failed " > "to requeue packet\n"); > } This transmits all skbs. -- MST From sean.hefty at intel.com Sat Aug 4 19:54:24 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Sat, 4 Aug 2007 19:54:24 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B4A042.1050005@opengridcomputing.com> Message-ID: <000001c7d70b$ef160bc0$78c8180a@amr.corp.intel.com> >Lemme know how I can help. I certainly can test any patches on my 8 >node iwarp cluster. We should probably take the idea to netdev before making any substantial changes to the code. - Sean From cait at asomi.com Sat Aug 4 20:49:31 2007 From: cait at asomi.com (Caitlin) Date: Sat, 4 Aug 2007 20:49:31 -0700 Subject: [ofa-general] cait@asomi.com has invited you to have a 3D avatar chat Message-ID: <200708050349.l753nV59011446@AF001320.prod.imvu.com> From: Caitlin Avatar: Guest_asomicait To: Openib Hey Openib,Caitlin has added you as a friend on IMVU. Is Caitlin your friend?   Please respond or Caitlin may think you said no :) IMVU is the world's greatest 3D chat! Dress up your Avatar with 3D clothes. Chat with your friends & meet new ones. Decorate your own 3D Room with furniture. FREE to download & use! http://www.imvu.com Copyright © 2006-2007 IMVU, Inc. 411 High Street, Palo Alto, CA 94301. This email was sent via IMVU by Caitlin (cait at asomi.com) to openib-general at openib.org. If you want to prevent any future emails from IMVU, you can remove yourself by pointing your web browser to http://www.imvu.com/catalog/web_nonregisteredoptout.php?code=42b5e1&email=openib-general at openib.org. Your unsubscribe confirmation code is 42b5e1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Sat Aug 4 21:07:12 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 5 Aug 2007 07:07:12 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-05:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Jul_31_14:27:42_2007 [c3bf9dc7b3063be57ca7a2cfca127937652053f3] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=467 Fail=53 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 LidMgr IS1-16.topo 38 Multicast IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 1 Multicast IS1-16.topo From ogerlitz at voltaire.com Sat Aug 4 21:54:56 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 05 Aug 2007 07:54:56 +0300 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> <46B16E3A.2070102@voltaire.com> Message-ID: <46B55820.90501@voltaire.com> Roland Dreier wrote: > Neither really... just that the double call to ib_find_pkey() in ib_sa > is not needed after this change. I am not following you, looking on the 2.6.22 tree, I don't see any calls from the ib_sa code to look for a pkey: > # grep -rn ib_find . | grep pkey > ./ulp/srp/ib_srp.c:167: ret = ib_find_cached_pkey(target->srp_host->dev->dev, > ./ulp/ipoib/ipoib_verbs.c:50: if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &pkey_index)) { > ./ulp/ipoib/ipoib_cm.c:868: ret = ib_find_cached_pkey(priv->ca, priv->port, priv->pkey, &qp_attr.pkey_index); > ./ulp/ipoib/ipoib_ib.c:451: if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &priv->pkey_index)) { > ./ulp/ipoib/ipoib_ib.c:491: if (ib_find_cached_pkey(priv->ca, priv->port, priv->pkey, &pkey_index)) > ./ulp/ipoib/ipoib_ib.c:719: if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &new_index)) { > ./core/cm.c:288: ret = ib_find_cached_pkey(cm_dev->device, port->port_num, > ./core/cma.c:538: ret = ib_find_cached_pkey(id_priv->id.device, id_priv->id.port_num, > ./core/device.c:687: * ib_find_pkey - Returns the PKey table index where a specified > ./core/device.c:694:int ib_find_pkey(struct ib_device *device, > ./core/device.c:713:EXPORT_SYMBOL(ib_find_pkey); > ./core/cache.c:163:int ib_find_cached_pkey(struct ib_device *device, > ./core/cache.c:193:EXPORT_SYMBOL(ib_find_cached_pkey); > Never mind, I'll fix it up myself. thanks. As I said before this has to go to -stable as the bug is present in 2.6.22, please let us know if you will do the push or you prefer that we will do it. Or. From ogerlitz at voltaire.com Sat Aug 4 22:23:03 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 5 Aug 2007 08:23:03 +0300 (IDT) Subject: [ofa-general] [PATCH] libibverbs/man: clarify GRH bytes offset in wc Message-ID: Hi Roland, The man page of ibv_poll_cq says that if the IBV_WC_GRH bit is set in the work completion flag then GRH is present. I was wrongly thinking that only if the bit is set, the app data starts after 40 bytes offset, which is wrong. This patch clarifies this. Also, what about the inline related patch, have you applied it? Or. Clarify that even when IBV_WC_GRH bit is not set in the wc flags, the actual data starts at offset 40 from the posted receive buffer start. Signed-off-by: Or Gerlitz --- libibverbs.orig/man/ibv_poll_cq.3 2007-06-21 17:39:11.000000000 +0300 +++ libibverbs/man/ibv_poll_cq.3 2007-08-05 08:16:41.000000000 +0300 @@ -47,7 +47,7 @@ The attribute wc_flags describes the pro It is either 0 or the bitwise OR of one or more of the following flags: .PP .TP -.B IBV_WC_GRH \fR GRH is present +.B IBV_WC_GRH \fR GRH is present (note that even if not set, the data starts at offset 40 from the posted receive buffer start) .TP .B IBV_WC_WITH_IMM \fR Immediate data value is valid .PP From ogerlitz at voltaire.com Sat Aug 4 22:32:42 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 5 Aug 2007 08:32:42 +0300 (IDT) Subject: [ofa-general] Re: [PATCH] libibverbs/man: clarify GRH bytes offset in wc In-Reply-To: References: Message-ID: On Sun, 5 Aug 2007, Or Gerlitz wrote: > The man page of ibv_poll_cq says that if the IBV_WC_GRH bit is set in the > work completion flag then GRH is present. I was wrongly thinking that only if > the bit is set, the app data starts after 40 bytes offset, which is wrong. > This patch clarifies this. > > Also, what about the inline related patch, have you applied it? I have change the patch to make it clear that this relates only to UD QPs Or. ------ Clarify that even when IBV_WC_GRH bit is not set in the wc flags, the actual data starts at offset 40 from the posted receive buffer start. Signed-off-by: Or Gerlitz --- libibverbs.orig/man/ibv_poll_cq.3 2007-06-21 17:39:11.000000000 +0300 +++ libibverbs/man/ibv_poll_cq.3 2007-08-05 08:30:29.000000000 +0300 @@ -69,6 +69,9 @@ The user should consume work completions overrun from occurrence. In case of a CQ overrun, the async event .B IBV_EVENT_CQ_ERR will be triggered, and the CQ cannot be used. +.PP +Note that if the completion originated from UD QP, even if the IBV_WC_GRH bit not set, +the data starts at offset 40 from the posted receive buffer start .SH "SEE ALSO" .BR ibv_post_send (3), .BR ibv_post_recv (3) From ogerlitz at voltaire.com Sat Aug 4 23:50:17 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 05 Aug 2007 09:50:17 +0300 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event while wthere is a broken IB connection? In-Reply-To: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> References: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> Message-ID: <46B57329.9020502@voltaire.com> Bub Thomas wrote: > Our application is using OFED 1.1 under SLES10. > It is a point to point IB connection from a high speed image scanner to > a workstation. > Now I get complains from the field that they do have dead pthreads > hanging around that are stuck in the ib_cm_get_event call. From your question, I understand that you use libibcm. How do you get the peer IB L3 (GID) and L2 (LID) info? are you using some out-of-band (eg through TCP) exchange for that? May I ask if libibcm buys you something which librdmacm doesn't? Or. From ogerlitz at voltaire.com Sun Aug 5 01:30:03 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 05 Aug 2007 11:30:03 +0300 Subject: [ofa-general] QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46A94657.1020101@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> Message-ID: <46B58A8B.5020609@voltaire.com> Sean Hefty wrote: >> 4. IPoIB --------- >> IPoIB already query the SA for its broadcast group information. The >> additional functionality required is for IPoIB to provide the >> broadcast group SL, MTU, and RATE in every following PathRecord query >> performed when a new UDAV is needed by IPoIB. We could assign a >> special Service-ID for IPoIB use but since all communication on the >> same IPoIB interface shares the same QoS-Level without the ability to >> differentiate it by target service we can ignore it for simplicity. > Rather than IPoIB specifying SL, MTU, and rate with PR queries, it > should specify TClass and FlowLabel. This is necessary for IPoIB to > span IB subnets. I wonder how you think the suggested scheme for IPoIB plugs into the existing QoS flow at the Linux network stack. Looking on the man pages of tcp(7) and ip(7) I see that there's a SOL_SOCKET level option of SO_PRIORITY and SOL_IP level option of IP_TOS. Looking lower in the stack, on the IPoEth scheme where the 802.1q is used for the VLAN header generation, the code seems to generate the p_bits field of the header using skb->priority (see the call to vlan_dev_get_egress_qos_mask() from vlan_dev_hard_header() at net/8021q/vlan_dev.c) From the man pages I understand that the stack uses those socket options to decide which networking queue would be used for the packet. From the quick 802.1q code review I conclude that the driver has some scheme which it uses to map skb->priority to a priority field it sets in the vlan header. Or. From vlad at lists.openfabrics.org Sun Aug 5 01:38:51 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 5 Aug 2007 01:38:51 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070805-0100 daily build status Message-ID: <20070805083851.E22CCE60837@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From ogerlitz at voltaire.com Sun Aug 5 01:40:10 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 05 Aug 2007 11:40:10 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <46A94657.1020101@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> Message-ID: <46B58CEA.5010709@voltaire.com> Sean Hefty wrote: >> 2. Architecture ---------------- > > This is a higher level approach to the problem, but I came up with the > following QoS relationship hierarchy, where '->' means 'maps to'. > > Application Service -> Service ID (or range) > Service ID -> desired QoS > QoS, SGID, DGID, PKey -> SGID, DGID, TClass, FlowLabel, PKey > SGID, DGID, TC, FL, PKey -> SLID, DLID, SL (set if crossing subnets) > SLID, DLID, SL -> MTU, Rate, VL, PacketLifeTime What's the reasoning to use TClass and Flowlabel here? is it that you want to come up with a scheme which is valid also for traffic crossing IB subnets? you suggest two mappings: A) from SID to "desired QoS" B) from "desired QoS" (and more params) to TClass and FlowLabel First, my understanding is that "QoS" is an abstract term here, that is does not translate to concrete IB term, correct? now, what entity per your design would translate from SID to QoS, is it something done internally in the host stack similar to what the net stack does with the IP_TOS and SOL_PRIORITY socket options? Or. From ogerlitz at voltaire.com Sun Aug 5 01:47:10 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Sun, 05 Aug 2007 11:47:10 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <46AF6281.10709@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46AF6281.10709@ichips.intel.com> Message-ID: <46B58E8E.1040001@voltaire.com> Sean Hefty wrote: > FYI - It is my intention to implement the host side portion of QoS > support. (It's one of my path forward objectives.) I plan on > implementing the host side as outlined below. If anyone has any > comments, I would like to get them as soon as possible. Sean, From what I understand while reading your proposal, is that it is quite different then what what suggested in the original RFC. I don't think it makes sense to implement the host side of this before there's agreement on the over-all solution namely how the host side design/code plugs to the management scheme at the SM side. Basically, the SM people have not really reacted on your proposal, which is a problem... One more thing that bothers me is backward compatibility with SM/SA, that does not support the not-published-yet IBTA QoS extensions. Where you thinking to first probe for the SA capabilities to see if it supports QoS path-queries or think its an over-doing? Or. > Sean Hefty wrote: >>> 2. Architecture ---------------- >> >> This is a higher level approach to the problem, but I came up with the >> following QoS relationship hierarchy, where '->' means 'maps to'. >> >> Application Service -> Service ID (or range) >> Service ID -> desired QoS >> QoS, SGID, DGID, PKey -> SGID, DGID, TClass, FlowLabel, PKey >> SGID, DGID, TC, FL, PKey -> SLID, DLID, SL (set if crossing subnets) >> SLID, DLID, SL -> MTU, Rate, VL, PacketLifeTime >> >> I use these relationships below: >> >>> 4. IPoIB --------- >>> >>> IPoIB already query the SA for its broadcast group information. The >>> additional functionality required is for IPoIB to provide the >>> broadcast group SL, MTU, and RATE in every following PathRecord query >>> performed when a new UDAV is needed by IPoIB. We could assign a >>> special Service-ID for IPoIB use but since all communication on the >>> same IPoIB interface shares the same QoS-Level without the ability to >>> differentiate it by target service we can ignore it for simplicity. >> >> Rather than IPoIB specifying SL, MTU, and rate with PR queries, it >> should specify TClass and FlowLabel. This is necessary for IPoIB to >> span IB subnets. >> >>> 5. CMA features ---------------- >>> >>> The CMA interface supports Service-ID through the notion of port >>> space as a prefixes to the port_num which is part of the sockaddr >>> provided to rdma_resolve_add(). What is missing is the explicit >>> request for a QoS-Class that should allow the ULP (like SDP) to >>> propagate a specific request for a class of service. A mechanism for >>> providing the QoS-Class is available in the IPv6 address, so we could >>> use that address field. Another option is to implement a special >>> connection options API for CMA. >>> >>> Missing functionality by CMA is the usage of the provided QoS-Class >>> and Service-ID in the sent PR/MPR. When a response is obtained it is >>> an existing requirement for the CMA to use the PR/MPR from the >>> response in setting up the QP address vector. >> >> I think the RDMA CM needs two solutions, depending on which address >> family is used. For IPv6, the existing interface is sufficient, and >> works for both IB and iWarp. The RDMA CM only needs to include the TC >> and FL as part of its PR query. For IPv4, to remain transport >> neutral, I think we should add an rdma_set_option() routine to specify >> the QoS field. The RDMA CM would include the QoS field for PR query >> under this condition. >> >> For IB, this requires changes to the ib_sa to support the new PR >> extensions. I don't think we gain anything having the RDMA CM include >> service IDs as part of the query. >> >>> 6. SDP ------- >>> >>> SDP uses CMA for building its connections. The Service-ID for SDP is >>> 0x000000000001PPPP, where PPPP are 4 hex digits holding the remote >>> TCP/IP Port Number to connect to. SDP might be provided with >>> SO_PRIORITY socket option. In that case the value provided should be >>> sent to the CMA as the TClass option of that connection. >> >> SDP would use specify the QoS through the IPv6 address or >> rdma_set_option() routine. >> >>> 7. SRP ------- >>> >>> Current SRP implementation uses its own CM callbacks (not CMA). So >>> SRP should fill in the Service-ID in the PR/MPR by itself and use >>> that information in setting up the QP. The T10 SRP standard defines >>> the SRP Service-ID to be defined by the SRP target I/O Controller >>> (but they should also comply with IBTA Service- ID rules). Anyway, >>> the Service-ID is reported by the I/O Controller in the >>> ServiceEntries DMA attribute and should be used in the PR/MPR if the >>> SA reports its ability to handle QoS PR/MPRs. >> >> I agree. >> >>> 8. iSER -------- iSER uses CMA and thus should be very close to SDP. >>> The Service-ID for iSER should be TBD. >> >> See RDMA CM and SDP. >> >>> 3.2. PR/MPR query handling: OpenSM should be able to enforce the >>> provided policy on client request. The overall flow for such requests >>> is: first the request is matched against the defined match rules such >>> that the target QoS-Level definition is found. Given the QoS-Level a >>> path(s) search is performed with the given restrictions imposed by >>> that level. The following two sections describe these steps. >> >> If we use the QoS hierarchy outlined above, I think we can construct >> some fairly simple tables to guide our PR selection. The SA may need >> to construct the tables starting at the bottom and working up, but I >> *think* it could be done. And by distributing the tables, we can >> support a more distributed (a la local SA) operation. >> >> From an administration point, I would be happier seeing something >> where the administrator defines a QoS level in terms of latency or >> bandwidth requirements and relative priority. Then, if desired, the >> administrator could provide more details, such as indicating which >> nodes would use which services, minimum required MTUs, etc. It would >> then be up to the SA to map these requirements to specific TC, FL, SL, >> VL values. >> >> In general, though, I'm personally far less concerned with the QoS >> specification interface to the SA, versus the operation that takes >> place on the hosts. >> >> Comments on using this approach on the host side? From vlad at lists.openfabrics.org Sun Aug 5 02:46:57 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 5 Aug 2007 02:46:57 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070805-0200 daily build status Message-ID: <20070805094657.C4F16E603B2@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070805-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From eanna.dalrymple at kempe.dk Sat Aug 4 23:48:26 2007 From: eanna.dalrymple at kempe.dk (Ervin Bloom) Date: Sun, 5 Aug 2007 12:48:26 +0600 Subject: [ofa-general] Dreams is real Message-ID: <01c7d75e$ea82b240$19ccfc47@eanna.dalrymple> -------------- next part -------------- A non-text attachment was scrubbed... Name: anka.gif Type: image/gif Size: 8504 bytes Desc: not available URL: From vlad at dev.mellanox.co.il Sun Aug 5 05:52:10 2007 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 05 Aug 2007 15:52:10 +0300 Subject: [ofa-general] Re: [ewg] [GIT PULL ofed-1.2/ofed-1.2.c] Chelsio Bug Fixes In-Reply-To: <46B24AD9.2000801@opengridcomputing.com> References: <46B24AD9.2000801@opengridcomputing.com> Message-ID: <46B5C7FA.5020404@dev.mellanox.co.il> Steve Wise wrote: > Vlad, > > Please include these two patches into ofed 1.2 and 1.2.c. The patches > are direct changes to the ofed_1_2 branch and patch files in > kernel_patches/fixes for ofed_1_2_c. > > These have been accepted upstream, and address OFED bugs 696 and 698. > > > Please pull these from: > > http://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2 > > and > > http://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2_c > > Thanks, > > Steve. > > > Shortlog: > > iw_cxgb3: Make the iw_cxgb3 module parameters writable. > iw_cxgb3: Always call low level send function via cxgb3_ofld_send(). > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > Done, Regards, Vladimir From vlad at dev.mellanox.co.il Sun Aug 5 06:20:38 2007 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 05 Aug 2007 16:20:38 +0300 Subject: [ofa-general] [PATCH]mlx4: fix wc opcode for rdma_read, In-Reply-To: <46B39D5C.9050604@mellanox.com> References: <46B39D5C.9050604@mellanox.com> Message-ID: <46B5CEA6.2090709@dev.mellanox.co.il> Vu Pham wrote: > Fix the work completion opcode for RDMA_READ operation. > > Signed-off-by: Vu Pham > > Applied to ofed_1_2/linux-2.6.git ofed_1_2_c Regards, Vladimir From vlad at dev.mellanox.co.il Sun Aug 5 06:32:24 2007 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 05 Aug 2007 16:32:24 +0300 Subject: [ofa-general] [PATCH] build_env.sh: remove package checking for debian In-Reply-To: <46AFDACB.907@osrg.net> References: <20070730091020.GH9963@mellanox.co.il> <46AFDACB.907@osrg.net> Message-ID: <46B5D168.6010205@dev.mellanox.co.il> Yoshiaki Tamura wrote: > Hi, > > The following patch removes some package checking which stops building > OFED 1.2 on debian. Although it's not a complete solution to port OFED > on debian, > it should help some debian users to build. > > Thanks, > > Yoshi > > Signed-off-by: Yoshi Tamura > > --- a/build_env.sh 2007-06-21 23:38:41.000000000 +0900 > +++ b/build_env.sh 2007-07-31 14:21:50.000000000 +0900 > @@ -1790,6 +1790,8 @@ > debian) > libibverbs_BUILD_REQ= > libibverbs_devel_BUILD_REQ= > + libibverbs_RUN_REQ= > + mstflint_BUILD_REQ= > ;; > redhat5) > # start_udev is required to create /dev/infiniband > _______________________________________________ Thanks, Applied. - Vladimir From rdreier at cisco.com Sun Aug 5 06:34:06 2007 From: rdreier at cisco.com (Roland Dreier) Date: Sun, 05 Aug 2007 06:34:06 -0700 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: <46B55820.90501@voltaire.com> (Or Gerlitz's message of "Sun, 05 Aug 2007 07:54:56 +0300") References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> <46B16E3A.2070102@voltaire.com> <46B55820.90501@voltaire.com> Message-ID: > I am not following you, looking on the 2.6.22 tree, I don't see any > calls from the ib_sa code to look for a pkey: Look in an up-to-date git tree. From hal.rosenstock at gmail.com Sun Aug 5 08:07:24 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 5 Aug 2007 11:07:24 -0400 Subject: [ofa-general] OpensSM and Multiple Ports In-Reply-To: <1186245995.6031.79.camel@gentoo-linux.localdomain> References: <1186194845.6031.28.camel@gentoo-linux.localdomain> <1186245995.6031.79.camel@gentoo-linux.localdomain> Message-ID: On 8/4/07, Stanley Sufficool wrote: > On Sat, 2007-08-04 at 06:55 -0400, Hal Rosenstock wrote: > > On 8/3/07, Stanley Sufficool wrote: > I am > trying to get opensm to manage two separate subnets, one per IB port on > an > Infinihost II PCIe HCA. I can specify PORT=1 and opensmd will start > > without error. On the second sm management node, I start with PORT=2 in > > opensm.conf and opensm fails with a timeout error. Not sure what PORT is > but you can start OpenSM on a specific port by using the -g (or -guid) > option. Something like: opensm -g=0x12345678 where 0x12345678 is the port > (not node) GUID > > There are 2 options for specifying the port in opensm.conf, one is the port > GUID and one is the port number (PORT=1) . OpenSM seems to correctly derive > the port GUID from the port number, Looks to me like PORT_NUM (not PORT) is only used in a RedHat init script. Vlad can better comment on both opensm.conf and the init scripts in OFED for OpenSM. > however I will try using the GUID > instead. Did this work ? > I am using Gentoo Linux with the mlx HCA and MAD, UMAD, etc... built > into > kernel 2.6.20. > > My goal is to get ipoib working. IPoIB is being > flakey with duplicated > ethernet MAC addresses ( MAC: 02:00:00:00:01 & :02 > ) . This may be resolved > when opensm is working? IPoIB won't work until > the ports are brought up by the SM. Not sure about the duplicated MACs. I > don't understand this. > > Ethernet MAC addresses as shown in Windows ipconfig for the IBoIP > interfaces. They are duplicated because the LIDs are not being renumbered by > the manager. I don't know how Windows derives its fake ethernet MACs for IPoIB but if it uses LIDs, this makes sense. -- Hal > I am also hoping to get SRP with Windows initiators and a Linux > target using > the SRPT kernel module. But failing that, I will regress to > iSCSI with hopes > of a future iSER target/initiator support. > > _______________________________________________ > general > mailing list > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general > mailing > list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To > unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Sun Aug 5 09:02:00 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 5 Aug 2007 19:02:00 +0300 Subject: [ofa-general] [PATCH] opensm: wrong variable memset() Message-ID: <20070805160200.GA27000@sashak.voltaire.com> Fix wrong variable memset(). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_state_mgr.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 1cf6257..2e43f86 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -587,7 +587,7 @@ __osm_state_mgr_get_sw_info( p_node = p_sw->p_node; p_dr_path = osm_node_get_any_dr_path_ptr( p_node ); - memset( &context, 0, sizeof( context ) ); + memset( &mad_context, 0, sizeof( mad_context ) ); mad_context.si_context.node_guid = osm_node_get_node_guid( p_node ); mad_context.si_context.set_method = FALSE; -- 1.5.3.rc2.29.gc4640f From eli at mellanox.co.il Sun Aug 5 09:20:20 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 05 Aug 2007 19:20:20 +0300 Subject: [ofa-general] socket buffer accounting with UDP/ipoib In-Reply-To: References: <1183643723.25031.262.camel@mtls03> <1184761552.3520.9.camel@mtls03> Message-ID: <1186330820.11488.17.camel@mtls03> On Fri, 2007-08-03 at 21:14 +0300, Roland Dreier wrote: > So is this "slight improvement" worth the extra complexity added by > having two different packet receive paths? I'm not sure myself. I will probably make some more measurements at later time and gather more data to evaluate the necessity of such a patch. In the meantime I think we can do without it. From hal.rosenstock at gmail.com Sun Aug 5 09:25:01 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 5 Aug 2007 12:25:01 -0400 Subject: [ofa-general] Re: [PATCH] opensm: wrong variable memset() In-Reply-To: <20070805160200.GA27000@sashak.voltaire.com> References: <20070805160200.GA27000@sashak.voltaire.com> Message-ID: Hi Sasha, On 8/5/07, Sasha Khapyorsky wrote: > > Fix wrong variable memset(). > > Signed-off-by: Sasha Khapyorsky > --- > opensm/opensm/osm_state_mgr.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > index 1cf6257..2e43f86 100644 > --- a/opensm/opensm/osm_state_mgr.c > +++ b/opensm/opensm/osm_state_mgr.c > @@ -587,7 +587,7 @@ __osm_state_mgr_get_sw_info( > p_node = p_sw->p_node; > p_dr_path = osm_node_get_any_dr_path_ptr( p_node ); > > - memset( &context, 0, sizeof( context ) ); > + memset( &mad_context, 0, sizeof( mad_context ) ); Nice catch. Should this also be applied for 1.2.c/1.2.1 ? -- Hal > mad_context.si_context.node_guid = osm_node_get_node_guid( p_node ); > mad_context.si_context.set_method = FALSE; > -- > 1.5.3.rc2.29.gc4640f > > From sashak at voltaire.com Sun Aug 5 10:04:09 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 5 Aug 2007 20:04:09 +0300 Subject: [ofa-general] Re: [PATCH] opensm: wrong variable memset() In-Reply-To: References: <20070805160200.GA27000@sashak.voltaire.com> Message-ID: <20070805170409.GD27000@sashak.voltaire.com> On 12:25 Sun 05 Aug , Hal Rosenstock wrote: > > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > index 1cf6257..2e43f86 100644 > > --- a/opensm/opensm/osm_state_mgr.c > > +++ b/opensm/opensm/osm_state_mgr.c > > @@ -587,7 +587,7 @@ __osm_state_mgr_get_sw_info( > > p_node = p_sw->p_node; > > p_dr_path = osm_node_get_any_dr_path_ptr( p_node ); > > > > - memset( &context, 0, sizeof( context ) ); > > + memset( &mad_context, 0, sizeof( mad_context ) ); > > Nice catch. Should this also be applied for 1.2.c/1.2.1 ? I don't think - it killed the content of variable which was not used later, so this bug is pretty minor. Sasha From fannie.dalrymple at kjaerulff-bay.dk Sun Aug 5 05:33:51 2007 From: fannie.dalrymple at kjaerulff-bay.dk (Carly Jorgensen) Date: Sun, 5 Aug 2007 17:33:51 +0500 Subject: [ofa-general] To be or not to be. To be... Message-ID: <01c7d786$ca0d0970$31290846@fannie.dalrymple> -------------- next part -------------- A non-text attachment was scrubbed... Name: ieieaa.gif Type: image/gif Size: 12856 bytes Desc: not available URL: From hal.rosenstock at gmail.com Sun Aug 5 11:40:39 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 5 Aug 2007 14:40:39 -0400 Subject: [ofa-general] Re: [PATCH] opensm: wrong variable memset() In-Reply-To: <20070805170409.GD27000@sashak.voltaire.com> References: <20070805160200.GA27000@sashak.voltaire.com> <20070805170409.GD27000@sashak.voltaire.com> Message-ID: On 8/5/07, Sasha Khapyorsky wrote: > On 12:25 Sun 05 Aug , Hal Rosenstock wrote: > > > > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > > index 1cf6257..2e43f86 100644 > > > --- a/opensm/opensm/osm_state_mgr.c > > > +++ b/opensm/opensm/osm_state_mgr.c > > > @@ -587,7 +587,7 @@ __osm_state_mgr_get_sw_info( > > > p_node = p_sw->p_node; > > > p_dr_path = osm_node_get_any_dr_path_ptr( p_node ); > > > > > > - memset( &context, 0, sizeof( context ) ); > > > + memset( &mad_context, 0, sizeof( mad_context ) ); > > > > Nice catch. Should this also be applied for 1.2.c/1.2.1 ? > > I don't think - it killed the content of variable which was not used > later, so this bug is pretty minor. It looked to me like it cleared p_mgr which is used later. -- Hal > Sasha > From sashak at voltaire.com Sun Aug 5 12:19:20 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 5 Aug 2007 22:19:20 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Add common flags -P, -C, and -t In-Reply-To: <46AF9DFF.mailEUD1EC05T@systemfabricworks.com> References: <46AF9DFF.mailEUD1EC05T@systemfabricworks.com> Message-ID: <20070805191920.GE27000@sashak.voltaire.com> On 15:39 Tue 31 Jul , davem at systemfabricworks.com wrote: > > > Add common flags -P, -C, and -t to infiniband-diags programs and scripts to > allow specifying the HCA port number, HCA device name, and query timeout. > These diagnostic programs can now be directed to either different fabrics > attached to the system, or forced to use different ports should the fabric > fail and comparisons are needed. > > Two of these had conflicting prior use of the flags. For the ibcheckerrs > script, -T is now used to specify the threshold file. In the saquery > program, -p and -c are now used to specify getting the PathRecord info > and getting the SA's class port info. > > Other than the resolution of the three conflicts, all comands behave exactly > the same as they did before the change if these common flags are not used. > > Signed-off-by: David A. McMillen Great stuff! Applied. Thanks. Sasha From sashak at voltaire.com Sun Aug 5 12:25:08 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 5 Aug 2007 22:25:08 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/scripts: Fix Bug 239 Error Reporting In-Reply-To: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> References: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> Message-ID: <20070805192508.GF27000@sashak.voltaire.com> Hi David, On 19:09 Tue 31 Jul , davem at systemfabricworks.com wrote: > > > Fix Bug 239 OpenIB diag scripts don't return error when lacking umad > permissions. Returning the error from the head of a shell pipeline is a > problem, so this fix causes the awk scripts to pass error messages through. > This will pass all standard error messages. > > This patch needs [ofa-general] [PATCH] infiniband-diags: Add common flags > -P, -C, and -t (posted Tue Jul 31 13:39:27 PDT 2007) applied first. > > Signed-off-by: David A. McMillen > --- > infiniband-diags/scripts/ibcheckerrors.in | 11 +++++++++-- > infiniband-diags/scripts/ibcheckerrs.in | 13 ++++++++++--- > infiniband-diags/scripts/ibchecknet.in | 16 ++++++++++++++-- > infiniband-diags/scripts/ibcheckport.in | 11 +++++++++-- > infiniband-diags/scripts/ibcheckportstate.in | 11 +++++++++-- > infiniband-diags/scripts/ibcheckportwidth.in | 11 +++++++++-- > infiniband-diags/scripts/ibcheckstate.in | 10 +++++++++- > infiniband-diags/scripts/ibcheckwidth.in | 10 +++++++++- > infiniband-diags/scripts/ibclearcounters.in | 10 +++++++++- > infiniband-diags/scripts/ibclearerrors.in | 10 +++++++++- > infiniband-diags/scripts/ibdatacounters.in | 11 +++++++++-- > infiniband-diags/scripts/ibdatacounts.in | 11 +++++++++-- > infiniband-diags/scripts/ibhosts.in | 9 ++++++++- > infiniband-diags/scripts/ibrouters.in | 9 ++++++++- > infiniband-diags/scripts/ibswitches.in | 9 ++++++++- > 15 files changed, 138 insertions(+), 24 deletions(-) > > diff --git a/infiniband-diags/scripts/ibcheckerrors.in b/infiniband-diags/scripts/ibcheckerrors.in > index 01c7a99..ebf44ec 100644 > --- a/infiniband-diags/scripts/ibcheckerrors.in > +++ b/infiniband-diags/scripts/ibcheckerrors.in > @@ -73,7 +73,9 @@ else > netcmd="$IBPATH/ibnetdiscover $ca_info" > fi > > -eval $netcmd | awk ' > +text="`eval $netcmd`" > +rv=$? > +echo "$text" | awk ' > BEGIN { > ne=0 > } Wouldn't it be simpler just exit instead of running awk over already broken output? Like: if [ $rv -ne 0 ] ; then echo $test exit $rv fi echo $text | awk ... Sasha From sashak at voltaire.com Sun Aug 5 12:35:03 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 5 Aug 2007 22:35:03 +0300 Subject: [ofa-general] Re: [PATCH] opensm: wrong variable memset() In-Reply-To: References: <20070805160200.GA27000@sashak.voltaire.com> <20070805170409.GD27000@sashak.voltaire.com> Message-ID: <20070805193503.GG27000@sashak.voltaire.com> On 14:40 Sun 05 Aug , Hal Rosenstock wrote: > On 8/5/07, Sasha Khapyorsky wrote: > > On 12:25 Sun 05 Aug , Hal Rosenstock wrote: > > > > > > > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > > > > index 1cf6257..2e43f86 100644 > > > > --- a/opensm/opensm/osm_state_mgr.c > > > > +++ b/opensm/opensm/osm_state_mgr.c > > > > @@ -587,7 +587,7 @@ __osm_state_mgr_get_sw_info( > > > > p_node = p_sw->p_node; > > > > p_dr_path = osm_node_get_any_dr_path_ptr( p_node ); > > > > > > > > - memset( &context, 0, sizeof( context ) ); > > > > + memset( &mad_context, 0, sizeof( mad_context ) ); > > > > > > Nice catch. Should this also be applied for 1.2.c/1.2.1 ? > > > > I don't think - it killed the content of variable which was not used > > later, so this bug is pretty minor. > > It looked to me like it cleared p_mgr which is used later. No, local var 'p_mgr' is initialized already as value (not address) of 'context', then _value_ of 'context' is cleared by this wrong memset(), but as function argument it is on the stack and not used anymore. Sasha From davem at systemfabricworks.com Sun Aug 5 17:17:30 2007 From: davem at systemfabricworks.com (David McMillen) Date: Sun, 05 Aug 2007 19:17:30 -0500 Subject: [ofa-general] Re: [PATCH] infiniband-diags/scripts: Fix Bug 239 Error Reporting In-Reply-To: <20070805192508.GF27000@sashak.voltaire.com> References: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> <20070805192508.GF27000@sashak.voltaire.com> Message-ID: <46B6689A.3030902@systemfabricworks.com> Sasha Khapyorsky wrote: >Hi David, > >On 19:09 Tue 31 Jul , davem at systemfabricworks.com wrote: > > >> Fix Bug 239 OpenIB diag scripts don't return error when lacking umad >> permissions. Returning the error from the head of a shell pipeline is a >> problem, so this fix causes the awk scripts to pass error messages through. >> This will pass all standard error messages. >> >> This patch needs [ofa-general] [PATCH] infiniband-diags: Add common flags >> -P, -C, and -t (posted Tue Jul 31 13:39:27 PDT 2007) applied first. >> >>Signed-off-by: David A. McMillen >>--- >> infiniband-diags/scripts/ibcheckerrors.in | 11 +++++++++-- >> infiniband-diags/scripts/ibcheckerrs.in | 13 ++++++++++--- >> infiniband-diags/scripts/ibchecknet.in | 16 ++++++++++++++-- >> infiniband-diags/scripts/ibcheckport.in | 11 +++++++++-- >> infiniband-diags/scripts/ibcheckportstate.in | 11 +++++++++-- >> infiniband-diags/scripts/ibcheckportwidth.in | 11 +++++++++-- >> infiniband-diags/scripts/ibcheckstate.in | 10 +++++++++- >> infiniband-diags/scripts/ibcheckwidth.in | 10 +++++++++- >> infiniband-diags/scripts/ibclearcounters.in | 10 +++++++++- >> infiniband-diags/scripts/ibclearerrors.in | 10 +++++++++- >> infiniband-diags/scripts/ibdatacounters.in | 11 +++++++++-- >> infiniband-diags/scripts/ibdatacounts.in | 11 +++++++++-- >> infiniband-diags/scripts/ibhosts.in | 9 ++++++++- >> infiniband-diags/scripts/ibrouters.in | 9 ++++++++- >> infiniband-diags/scripts/ibswitches.in | 9 ++++++++- >> 15 files changed, 138 insertions(+), 24 deletions(-) >> >>diff --git a/infiniband-diags/scripts/ibcheckerrors.in b/infiniband-diags/scripts/ibcheckerrors.in >>index 01c7a99..ebf44ec 100644 >>--- a/infiniband-diags/scripts/ibcheckerrors.in >>+++ b/infiniband-diags/scripts/ibcheckerrors.in >>@@ -73,7 +73,9 @@ else >> netcmd="$IBPATH/ibnetdiscover $ca_info" >> fi >> >>-eval $netcmd | awk ' >>+text="`eval $netcmd`" >>+rv=$? >>+echo "$text" | awk ' >> BEGIN { >> ne=0 >> } >> >> > >Wouldn't it be simpler just exit instead of running awk over already >broken output? > >Like: > > if [ $rv -ne 0 ] ; then > echo $test > exit $rv > fi > > echo $text | awk ... > >Sasha > > > Upon failure, sending the full output of the eval'd expression to standard output is not very useful, especially since the normal output of the various scripts is to suppress most of that. However, embedded within the text there are likely to be error messages that explain what went wrong, which is why there are new parts to the awk script catching /^ib/ /ibpanic:/ /ibwarn:/ /iberror:/ and passing them through.. Running awk over already broken output has been happening all along with these scripts, so it does not create a new problem. We could define two awk scripts, one for success and one for failure, but I think that is more complex than it needs to be. The failure case is supposed to be unusual, and the processing time for the awk script was expected if there was success, so this isn't a particular waste of system resources. If we knew all of the usage cases for these scripts, we could probably do something better, but since some of these scripts have been in distribution for a while, I felt it best to avoid changing the processing cycle and output as much as possible while still adding a report of failures. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Sun Aug 5 21:04:29 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 6 Aug 2007 07:04:29 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-06:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Sun_Aug_5_18:57:58_2007 [2bc5575f05746801091928bd4ee342e3c6ccb504] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=467 Fail=53 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo 12 Multicast IS3-128.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 1 Multicast IS3-128.topo From amar.mudrankit at gmail.com Sun Aug 5 22:16:01 2007 From: amar.mudrankit at gmail.com (Amar Mudrankit) Date: Mon, 6 Aug 2007 10:46:01 +0530 Subject: [ofa-general] Re: Understanding IPoIB-CM In-Reply-To: <20070804220206.GA15821@mellanox.co.il> References: <20070804220206.GA15821@mellanox.co.il> Message-ID: On 8/5/07, Michael S. Tsirkin wrote: > > > Quoting Amar Mudrankit : > > Subject: Understanding IPoIB-CM > > > > > > I have been looking at the IPoIB CM code and have some queries. I'd > greatly > > appreciate it if some one could shed some light on it. > > > > Please correct me if I am wrong, but based on my understanding of the > code: > > > > 1) From functions ipoib_cm_create_tx_qp and ipoib_cm_create_rx_qp, it > looks > > like for communicating with a peer, we use one QP for purely > > doing TX and one QP for purely doing RX (created on receipt of a CM > REQ). > > That means for every peer we are connected to, we use two > > QPs. Why not use a single QP for both TX and RX of data ? On receipt > of a REQ > > from a peer or before trying to send data to peer, we > > could possibly check if we already have a QP connection to the peer > (by > > maintaining and looking up a QP and peer UD QPN + GID > > mapping) and then just use it. Or is it not as simple as that ? > > Sure, this could work. One'd have to be sure to cover all error > cases though. So, there are two connections established between 2 communicating peers. 1 connection each for transmitting in each direction. Is it correct? > On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case of > > simultaneous IB connection where REQs can cross paths. > > But the current implementation does not seem to handle this case. Is > it > > because we create two QPs anyway and do not need to handle > > this special case ? > > Yes. > > > 2] In the ipoib_cm_rep_handler, the skbs seem to be dequeued and queued > again. > > I am referring to the code below. What does this exactly > > achieve ? > > > > skb_queue_head_init(&skqueue); > > spin_lock_irq(&priv->lock); > > set_bit(IPOIB_FLAG_OPER_UP, &p->flags); > > if (p->neigh) > > while ((skb = __skb_dequeue(&p->neigh->queue))) > > __skb_queue_tail(&skqueue, skb); > > spin_unlock_irq(&priv->lock); > > while ((skb = __skb_dequeue(&skqueue))) { > > skb->dev = p->dev; > > if (dev_queue_xmit(skb)) > > ipoib_warn(priv, "dev_queue_xmit failed " > > "to requeue packet\n"); > > } > > This transmits all skbs. Here, I wanted to ask - before transmitting why is the skb->dev field modified as the device is same IPoIB interface through which packets will be transmitted. Is p->dev is some other device? -- > MST > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mst at dev.mellanox.co.il Sun Aug 5 22:25:52 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 08:25:52 +0300 Subject: [ofa-general] Re: Understanding IPoIB-CM In-Reply-To: References: <20070804220206.GA15821@mellanox.co.il> Message-ID: <20070806052552.GB1896@mellanox.co.il> > Quoting Amar Mudrankit : > Subject: Re: Understanding IPoIB-CM > > > > On 8/5/07, Michael S. Tsirkin wrote: > > > Quoting Amar Mudrankit : > > Subject: Understanding IPoIB-CM > > > > > > I have been looking at the IPoIB CM code and have some queries. I'd > greatly > > appreciate it if some one could shed some light on it. > > > > Please correct me if I am wrong, but based on my understanding of the > code: > > > > 1) From functions ipoib_cm_create_tx_qp and ipoib_cm_create_rx_qp, it > looks > > like for communicating with a peer, we use one QP for purely > > doing TX and one QP for purely doing RX (created on receipt of a CM > REQ). > > That means for every peer we are connected to, we use two > > QPs. Why not use a single QP for both TX and RX of data ? On receipt of > a REQ > > from a peer or before trying to send data to peer, we > > could possibly check if we already have a QP connection to the peer (by > > maintaining and looking up a QP and peer UD QPN + GID > > mapping) and then just use it. Or is it not as simple as that ? > > Sure, this could work. One'd have to be sure to cover all error > cases though. > > > So, there are two connections established between 2 communicating peers. 1 > connection each for transmitting in each direction. Is it correct? That's how the current implementation behaves, yes. > > On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case > of > > simultaneous IB connection where REQs can cross paths. > > But the current implementation does not seem to handle this case. Is it > > because we create two QPs anyway and do not need to handle > > this special case ? > > Yes. > > > 2] In the ipoib_cm_rep_handler, the skbs seem to be dequeued and queued > again. > > I am referring to the code below. What does this exactly > > achieve ? > > > > skb_queue_head_init(&skqueue); > > spin_lock_irq(&priv->lock); > > set_bit(IPOIB_FLAG_OPER_UP, &p->flags); > > if (p->neigh) > > while ((skb = __skb_dequeue(&p->neigh->queue))) > > __skb_queue_tail(&skqueue, skb); > > spin_unlock_irq(&priv->lock); > > while ((skb = __skb_dequeue(&skqueue))) { > > skb->dev = p->dev; > > if (dev_queue_xmit(skb)) > > ipoib_warn(priv, "dev_queue_xmit failed " > > "to requeue packet\n"); > > } > > This transmits all skbs. > > > Here, I wanted to ask - before transmitting why is the skb->dev field > modified as the device is same IPoIB interface through which packets will be > transmitted. Is p->dev is some other device? Good question, I don't recall at the moment. I think I copied this from ipoib_main.c Roland, any idea why do we do this? -- MST From vlad at lists.openfabrics.org Mon Aug 6 01:39:14 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 6 Aug 2007 01:39:14 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070806-0100 daily build status Message-ID: <20070806083914.76C06E60862@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.17 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From ogerlitz at voltaire.com Mon Aug 6 02:11:40 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 6 Aug 2007 12:11:40 +0300 (IDT) Subject: [ofa-general] implicit address binding by rdma_resolve_addr Message-ID: Hi Sean, We have noticed that when rdma_resolve_addr is provided with src address, the rdma cm does an implicit --address-- (=ip/port) binding, which for example has the side effect of disallowing two processes to call rdma_resolve_addr() with the same src port. Does this implicit binding mimics the network stack behavior under similar flow when socket are used? Or. From denice.dalrymple at karikaturmuseum.at Mon Aug 6 00:13:43 2007 From: denice.dalrymple at karikaturmuseum.at (Debbie Bateman) Date: Mon, 6 Aug 2007 10:13:43 +0300 Subject: [ofa-general] Magic stick Message-ID: <01c7d812$77bffbf0$7eff4ec9@denice.dalrymple> -------------- next part -------------- A non-text attachment was scrubbed... Name: ieieaa.gif Type: image/gif Size: 12542 bytes Desc: not available URL: From vlad at lists.openfabrics.org Mon Aug 6 03:32:00 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 6 Aug 2007 03:32:00 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070806-0200 daily build status Message-ID: <20070806103200.74ADFE603D1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c: In function ‘ehca_alloc_pd’: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: error: implicit declaration of function ‘kmem_cache_zalloc’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.c:53: warning: assignment makes pointer from integer without a cast make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca/ehca_pd.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_ppc64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ppc64/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070806-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Mon Aug 6 05:03:08 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 06 Aug 2007 15:03:08 +0300 Subject: [ofa-general] PATCH] IB/core: ignore membership bit when looking for a P_Key in the table In-Reply-To: References: <46A36E77.5020307@gmail.com> <46A453BE.3030408@gmail.com> <46B16E3A.2070102@voltaire.com> <46B55820.90501@voltaire.com> Message-ID: <46B70DFC.9030506@voltaire.com> Roland Dreier wrote: > > I am not following you, looking on the 2.6.22 tree, I don't see any > > calls from the ib_sa code to look for a pkey: > > Look in an up-to-date git tree. OK, got it. The fix in sa_query.c is not relevant for 2.6.22 hence I have asked Moni to send to -stable only the fix to ib_find_pkey() Or. From hnguyen at linux.vnet.ibm.com Mon Aug 6 06:10:33 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Mon, 6 Aug 2007 15:10:33 +0200 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802160847.GC9607@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> Message-ID: <200708061510.33528.hnguyen@linux.vnet.ibm.com> Hello Michael and Vladimir! > ehca backports for kernel.org kernels seem to be broken. > 1. Does anyone care enough to fix them? If not we'll disable > ehca in build for these kernels. I downloaded daily build package ofa_1_2_c_kernel-20070804-0200.tgz and followed the build scheme configure, make on 2.6.19, 2.6.18, 2.6.17 and 2.6.16/sles10/sles10_sp1. Except for 2.6.16/sles10/sles10_sp1 a patch for kmem_cache_zalloc() is required for ehca the others were built without errors, see below. Thus, I'm wondering what I'm doing differently than your daily build script? PS: Will run the build on rhel5 (still to set up kernel tree). Thanks Nam * linux-2.6.19 [root at idd60 ofa_1_2_c_kernel-20070804-0200]# uname -r 2.6.19 gcc -m64 -Wp,-MD,/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/.ib_ipoib.mod.o.d -nostdinc -isystem /usr/lib/gcc/ppc64-redhat-linux/4.1.1/include -D__KERNEL__ \ -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.19/include/ \ -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/include \ -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include \ -Iinclude \ \ -include include/linux/autoconf.h \ -include /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/include/linux/autoconf.h \ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -msoft-float -pipe -mminimal-toc -mtraceback=none -mcall-aixdesc -mtune=power4 -mno-altivec -funit-at-a-time -mstring -Wa,-maltivec -fomit-frame-pointer -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/include -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/debug -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/cxgb3/core -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/net/cxgb3 -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/net/rds -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/net/mlx4 -I/root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/mlx4 -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ib_ipoib.mod)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -DMODULE -c -o /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.c ld -m elf64ppc -r -o /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.ko /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.o /root/2.6.19/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o make[1]: Leaving directory `/usr/src/linux-2.6.19' [root at idd60 ofa_1_2_c_kernel-20070804-0200]# find . -name '*.ko' ./drivers/infiniband/ulp/ipoib/ib_ipoib.ko ./drivers/infiniband/hw/ehca/ib_ehca.ko ./drivers/infiniband/core/ib_mad.ko ./drivers/infiniband/core/iw_cm.ko ./drivers/infiniband/core/ib_uverbs.ko ./drivers/infiniband/core/ib_ucm.ko ./drivers/infiniband/core/ib_cm.ko ./drivers/infiniband/core/ib_sa.ko ./drivers/infiniband/core/ib_core.ko * 2.6.18 [root at idd60 2.6.18]# uname -r 2.6.18 gcc -m64 -Wp,-MD,/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/.ib_ipoib.mod.o.d -nostdinc -isystem /usr/lib/gcc/ppc64-redhat-linux/4.1.1/include -D__KERNEL__ \ -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.18/include/ \ -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/include \ -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include \ -Iinclude \ \ -include include/linux/autoconf.h \ -include /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/include/linux/autoconf.h \ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -msoft-float -pipe -mminimal-toc -mtraceback=none -mcall-aixdesc -mtune=power4 -mno-altivec -funit-at-a-time -mstring -Wa,-maltivec -fomit-frame-pointer -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/include -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/debug -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/cxgb3/core -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/net/cxgb3 -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/net/rds -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/net/mlx4 -I/root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/mlx4 -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ib_ipoib.mod)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -DMODULE -c -o /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.c ld -m elf64ppc -r -o /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.ko /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.o /root/2.6.18/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o make[1]: Leaving directory `/usr/src/linux-2.6.18' [root at idd60 ofa_1_2_c_kernel-20070804-0200]# find . -name '*.ko' ./drivers/infiniband/ulp/ipoib/ib_ipoib.ko ./drivers/infiniband/hw/ehca/ib_ehca.ko ./drivers/infiniband/core/ib_mad.ko ./drivers/infiniband/core/iw_cm.ko ./drivers/infiniband/core/ib_uverbs.ko ./drivers/infiniband/core/ib_ucm.ko ./drivers/infiniband/core/ib_cm.ko ./drivers/infiniband/core/ib_sa.ko ./drivers/infiniband/core/ib_core.ko *linux 2.6.17 [root at idd60 ofa_1_2_c_kernel-20070804-0200]# uname -r 2.6.17 gcc -m64 -Wp,-MD,/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/.ib_ipoib.mod.o.d -nostdinc -isystem /usr/lib/gcc/ppc64-redhat-linux/4.1.1/include -D__KERNEL__ \ -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.17/include/ \ -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/include \ -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include \ -Iinclude \ \ -include include/linux/autoconf.h \ -include /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/include/linux/autoconf.h \ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -fomit-frame-pointer -msoft-float -pipe -mminimal-toc -mtraceback=none -mcall-aixdesc -mtune=power4 -mno-altivec -funit-at-a-time -mstring -Wa,-maltivec -Wdeclaration-after-statement -Wno-pointer-sign -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/include -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/debug -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/cxgb3/core -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/net/cxgb3 -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/net/rds -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/net/mlx4 -I/root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/mlx4 -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ib_ipoib)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -DMODULE -c -o /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.c ld -m elf64ppc -r -o /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.ko /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.o /root/2.6.17/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o make[1]: Leaving directory `/usr/src/linux-2.6.17' [root at idd60 ofa_1_2_c_kernel-20070804-0200]# find . -name '*.ko' ./drivers/infiniband/ulp/ipoib/ib_ipoib.ko ./drivers/infiniband/hw/ehca/ib_ehca.ko ./drivers/infiniband/core/ib_mad.ko ./drivers/infiniband/core/iw_cm.ko ./drivers/infiniband/core/ib_uverbs.ko ./drivers/infiniband/core/ib_ucm.ko ./drivers/infiniband/core/ib_cm.ko ./drivers/infiniband/core/ib_sa.ko ./drivers/infiniband/core/ib_core.ko * 2.6.16, sles10, sles10sp1 requires ehca_kmem_cache_zalloc_to_2_6_16.patch linux1:~ # uname -r 2.6.16.21-0.8-ppc64 gcc -m64 -Wp,-MD,/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/.ib_ipoib.mod.o.d -nostdinc -isystem /usr/lib/gcc/powerpc64-suse-linux/4.1.0/include -D__KERNEL__ -I/root/ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10/include/ -I/root/ofa_1_2_c_kernel-20070804-0200/include -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.21-0.8/include -include include/linux/autoconf.h -include /root/ofa_1_2_c_kernel-20070804-0200/include/linux/autoconf.h -I -I/usr/src/linux-2.6.16.21-0.8/ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common -ffreestanding -Os -fomit-frame-pointer -msoft-float -pipe -mminimal-toc -mtraceback=none -mcall-aixdesc -mtune=power4 -mno-altivec -funit-at-a-time -mstring -Wa,-maltivec -fno-ivopts -Wdeclaration-after-statement -Wno-pointer-sign -I/root/ofa_1_2_c_kernel-20070804-0200/include -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/include -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/debug -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/cxgb3/core -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/net/cxgb3 -I/root/ofa_1_2_c_kernel-20070804-0200/net/rds -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/net/mlx4 -I/root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/mlx4 -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ib_ipoib)" -D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -DMODULE -c -o /root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o /root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.c ld -m elf64ppc -r -o /root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.ko /root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.o /root/ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/ulp/ipoib/ib_ipoib.mod.o make[1]: Leaving directory `/usr/src/linux-2.6.16.21-0.8-obj/powerpc/ppc64' linux1:~/ofa_1_2_c_kernel-20070804-0200 # find . -name '*.ko' ./drivers/infiniband/hw/ehca/ib_ehca.ko ./drivers/infiniband/ulp/ipoib/ib_ipoib.ko ./drivers/infiniband/core/iw_cm.ko ./drivers/infiniband/core/ib_core.ko ./drivers/infiniband/core/ib_uverbs.ko ./drivers/infiniband/core/ib_mad.ko ./drivers/infiniband/core/ib_ucm.ko ./drivers/infiniband/core/ib_cm.ko ./drivers/infiniband/core/ib_sa.ko From hnguyen at linux.vnet.ibm.com Mon Aug 6 06:30:53 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Mon, 6 Aug 2007 15:30:53 +0200 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070802160847.GC9607@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> Message-ID: <200708061530.54392.hnguyen@linux.vnet.ibm.com> Hello Doug and Scott! On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > ehca backports for kernel.org kernels seem to be broken. > 1. Does anyone care enough to fix them? If not we'll disable > ehca in build for these kernels. > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > This would make it possible for us to add it to nightly builds. Could you please provide Michael and Vladimir with a URL to download above kernel source tree in order to perform daily build of ofed code suite? This will help us to prevent build issues in very early stage. Thanks much in advance! Regards Nam From mst at dev.mellanox.co.il Mon Aug 6 06:35:00 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 16:35:00 +0300 Subject: [ofa-general] RFCv2: SRC API In-Reply-To: <20070729140431.GG16915@mellanox.co.il> References: <20070729140431.GG16915@mellanox.co.il> Message-ID: <20070806133500.GA1061@mellanox.co.il> This is version 2 of the proposal, addressing comments from version 1. Changelog: - Use oflags to make API smaller - Clarify sharing semantics - Add documentation This is the API proposal for support of the SRC (scalable reliable connected) protocol extension in libibverbs. This adds APIs to: - manage SRC domains - share SRC domains between processes, by means of creating a 1:1 association between an SRC domain and an inode. Notes: - The inode is specified by means of a file descriptor, this makes it possible for the user to manage file creation/deletion in the most flexible manner (e.g. tmpfile can be used). - I envision implementing this sharing mechanism in kernel by means of a per-device tree, with inode as a key and domain object as a value. Please comment. Signed-off-by: Michael S. Tsirkin ---- diff --git a/SRC.txt b/SRC.txt new file mode 100644 index 0000000..3881477 --- /dev/null +++ b/SRC.txt @@ -0,0 +1,133 @@ +Here's some documentation on Scalable Reliable Connections. + + * * * + +SRC is an extension supported by recent Mellanox hardware +which is geared toward reducing the number of QPs +required for all-to-all communication on systems +with a high number of jobs per node. + +=================================================================== +Motivation: +=================================================================== +Given N nodes with J jobs per node, number of QPs required +for all-to-all communication is: + +With RC: + O((N * J) ^ 2) + + Since each job out of O(N * J) jobs must create a single QP + to communicate with each one of O(N * J) other jobs. + +With SRC: + O(N ^ 2 * J) + + This is achived by using a single send queue (per job, out of O(N * J) jobs) + to send data to all J jobs running on a specific node (out of O(N) nodes). + Hardware uses new "SRQ number" field in packet header to + multiplex receive WRs and WCs to private memory of each job. + +This is similiar idea to IB RD. +Q: Why not use RD then? +A: Because no hardware supports it. + +Details: + +=================================================================== +Verbs extension: +=================================================================== + +- There is a new transport/QP type "SRC". +- There is a new object type "SRC domain" +- Each SRQ gets new (optional) attributes: + SRC domain + SRC SRQ number + SRC CQ + SRQ must have either all 3 of these or none of these attributes + +- QPs of type SRC have all the same attributes as regular RC QPs + connected to SRQ, except that: + A. Each SRC QP has a new required attribute "SRC domain" + B. SRC QPs do *not* have "SRQ" attribute + (do not have a specific SRQ associated with them) + +=================================================================== +Protocol extension: +=================================================================== +SRC QP behaviour: Requestor +- Post send WR for this QP type is extended with SRQ number field + This number is sent as part of packet header +- SRC Packets follow rules for RC packets on the wire, exactly + What is different is their handling at the responder side + +SRC QP behaviour: Responder +Each incoming packet passes transport checks with respect +to the SRC QP, following RC rules, exactly. + +After this, SRQ number in packet header is used to look up +a specific SRQ. SRC domain of the resulting SRQ must be equal +to SRC domain of the QP, otherwise a NAK is sent, +and QP moves to error state. + +If the SRC domains match, receive WR and receive WC processing +are as follows: + +- RC Send + - Rather than using SRQ to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + Receive WR is taken from this SRQ. + - Completions are generated on the CQ specified in the SRQ + +- RDMA/Atomic + - Rather than using PD to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + PD of this SRQ is used for protection checks. + +=================================================================== +Pseudo code: +=================================================================== + +Consider again a setup where there are N nodes with J jobs per node. +All N * J jobs need to perform all-to-all communication. +Using RC QPs, this would call for O((N * J) ^ 2) QPs. +Here is how SRC can be used to reduce the number of QPs to O(N ^ 2 * J). + +At startup: +1. All jobs on each node share a single SRC domain +2. Each job creates a CQ for receive WCs +3. Each job creates a SRQ attached to this CQ and to the shared domain + +When job j1 needs to transmit to job j2 on remote node n for the first time: +1. Test: does job j1 have an existing connection to some job on node n? + - If no: + j1 creates an SRC QP qp1 (send QP) + qp1 is only used to post send WRs + j2 creates an SRC QP qp2 + qp2 is part of SRC domain + qp2 is only used to do transport checks: + neither send nor receive WRs are posted on qp2 + j1 and j2 create a connection between qp1 and qp2 + - If yes: + let qp1 be the QP which belongs to j1 and is connected + to some qp on node n + +2. j1 gets SRQ number from j2 +3. j1 can now use QP qp2 from step 1 + and SRQ number from step 3 to send data to j2 + +Cleanup: +When job j1 does not need to communicate to any jobs on node n, +it disconnects qp1 from qp2, and asks j2 to destroy qp2. + +=================================================================== + +Resources used (CQs are ignored below): +Each node: +- An SRC domain - to the total of n domains +- A Receive QP for each (remote) job - to the total of N * (N * J) recv QPs + +Each job: +- A SRQ - to the total of N * J SRQs +- A send QP for each (remote) node - to the total of N * (N * J) send QPs + +=================================================================== diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index acc1b82..d18475a 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -142,6 +142,7 @@ struct ibv_device_attr { uint16_t max_pkeys; uint8_t local_ca_ack_delay; uint8_t phys_port_cnt; + int max_src_domain; }; enum ibv_mtu { @@ -370,6 +371,11 @@ struct ibv_ah_attr { uint8_t port_num; }; +struct ibv_src_domain { + struct ibv_context *context; + uint32_t handle; +}; + enum ibv_srq_attr_mask { IBV_SRQ_MAX_WR = 1 << 0, IBV_SRQ_LIMIT = 1 << 1 @@ -389,7 +395,8 @@ struct ibv_srq_init_attr { enum ibv_qp_type { IBV_QPT_RC = 2, IBV_QPT_UC, - IBV_QPT_UD + IBV_QPT_UD, + IBV_QPT_SRC }; struct ibv_qp_cap { @@ -408,6 +415,7 @@ struct ibv_qp_init_attr { struct ibv_qp_cap cap; enum ibv_qp_type qp_type; int sq_sig_all; + struct ibv_src_domain *src_domain; }; enum ibv_qp_attr_mask { @@ -526,6 +534,7 @@ struct ibv_send_wr { uint32_t remote_qkey; } ud; } wr; + uint32_t src_remote_srq_num; }; struct ibv_recv_wr { @@ -553,6 +562,10 @@ struct ibv_srq { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + uint32_t src_srq_num; + struct ibv_src_domain *src_domain; + struct ibv_cq *src_cq; }; struct ibv_qp { @@ -570,6 +583,8 @@ struct ibv_qp { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + struct ibv_src_domain *src_domain; }; struct ibv_comp_channel { @@ -912,6 +927,25 @@ struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr *srq_init_attr); /** + * ibv_create_src_srq - Creates a SRQ associated with the specified protection + * domain and src domain. + * @pd: The protection domain associated with the SRQ. + * @src_domain: The SRC domain associated with the SRQ. + * @src_cq: CQ to report completions for SRC packets on. + * + * @srq_init_attr: A list of initial attributes required to create the SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the SRQ, and set to the actual values allocated + * on return. If ibv_create_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ibv_srq *ibv_create_src_srq(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, + struct ibv_srq_init_attr *srq_init_attr); + +/** * ibv_modify_srq - Modifies the attributes for the specified SRQ. * @srq: The SRQ to modify. * @srq_attr: On input, specifies the SRQ attributes to modify. On output, @@ -1074,6 +1108,42 @@ int ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); */ int ibv_fork_init(void); +/** + * ibv_open_src_domain - open an SRC domain + * Returns a reference to an SRC domain. + * + * @context: Device context + * @fd: descriptor for inode associated with the domain + * If fd == -1, no inode is associated with the domain; in this case, + * the only legal value for oflag is O_CREAT + * + * @oflag: oflag values are constructed by OR-ing flags from the following list + * + * O_CREAT + * If a domain belonging to device named by context is already associated + * with the inode, this flag has no effect, except as noted under O_EXCL + * below. Otherwise, a new SRC domain is created and is associated with + * inode specified by fd. + * + * O_EXCL + * If O_EXCL and O_CREAT are set, open will fail if a domain associated with + * the inode exists. The check for the existence of the domain and creation + * of the domain if it does not exist is atomic with respect to other + * processes executing open with fd naming the same inode. + */ +struct ibv_src_domain *ibv_open_src_domain(struct ibv_context *context, + int fd, int oflag); + +/** + * ibv_close_src_domain - close an SRC domain + * If this is the last reference, destroys the domain. + * + * @d: reference to SRC domain to close + * + * close is implicitly performed at process exit. + */ +int ibv_close_src_domain(struct ibv_src_domain *d); + END_C_DECLS # undef __attribute_const -- MST From mst at dev.mellanox.co.il Mon Aug 6 06:48:01 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 16:48:01 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <200708061510.33528.hnguyen@linux.vnet.ibm.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061510.33528.hnguyen@linux.vnet.ibm.com> Message-ID: <20070806134801.GB1061@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > Hello Michael and Vladimir! > > ehca backports for kernel.org kernels seem to be broken. > > 1. Does anyone care enough to fix them? If not we'll disable > > ehca in build for these kernels. > I downloaded daily build package ofa_1_2_c_kernel-20070804-0200.tgz > and followed the build scheme configure, make on 2.6.19, 2.6.18, 2.6.17 > and 2.6.16/sles10/sles10_sp1. Except for 2.6.16/sles10/sles10_sp1 > a patch for kmem_cache_zalloc() is required for ehca the others were > built without errors, see below. Thus, I'm wondering what I'm doing > differently than your daily build script? Could be different kernel configs or compiler version? Can you please build on ofa server against kernels in ~vlad/kernel.org/? The cross tool chain is here: /home/vlad/cross/ -- MST From jackm at dev.mellanox.co.il Mon Aug 6 07:09:09 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 6 Aug 2007 17:09:09 +0300 Subject: [ofa-general] [PATCH] ipoib: fix error flow memory leak Message-ID: <200708061709.10150.jackm@dev.mellanox.co.il> Leak in ipoib_transport_dev_init -- need to clean up ipoib_cm_dev. Found by Dotan Barak of Mellanox Signed-off-by: Jack Morgenstein Index: infiniband/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- infiniband.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-07-19 13:18:06.830269000 +0300 +++ infiniband/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-06 17:01:33.265187000 +0300 @@ -211,6 +211,7 @@ out_free_cq: out_free_mr: ib_dereg_mr(priv->mr); + ipoib_cm_dev_cleanup(dev); out_free_pd: ib_dealloc_pd(priv->pd); From jackm at dev.mellanox.co.il Mon Aug 6 07:11:58 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 6 Aug 2007 17:11:58 +0300 Subject: [ofa-general] [PATCH] mlx4: Do not invoke mlx4_bitmap_free for special QPs Message-ID: <200708061711.58789.jackm@dev.mellanox.co.il> mlx4_bitmap_free should not be called for special QPs. Found by Dotan Barak of Mellanox Signed-off-by: Jack Morgenstein Index: infiniband/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- infiniband.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-06 17:02:03.314019000 +0300 +++ infiniband/drivers/infiniband/hw/mlx4/qp.c 2007-08-06 17:02:21.790886000 +0300 @@ -508,7 +508,7 @@ static void destroy_qp_common(struct mlx mlx4_ib_unlock_cqs(send_cq, recv_cq); - mlx4_qp_free(dev->dev, &qp->mqp); + mlx4_qp_free(dev->dev, is_sqp(dev, qp), &qp->mqp); mlx4_mtt_cleanup(dev->dev, &qp->mtt); if (is_user) { Index: infiniband/drivers/net/mlx4/qp.c =================================================================== --- infiniband.orig/drivers/net/mlx4/qp.c 2007-08-06 17:02:03.321009000 +0300 +++ infiniband/drivers/net/mlx4/qp.c 2007-08-06 17:02:21.798883000 +0300 @@ -226,7 +226,7 @@ void mlx4_qp_remove(struct mlx4_dev *dev } EXPORT_SYMBOL_GPL(mlx4_qp_remove); -void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp) +void mlx4_qp_free(struct mlx4_dev *dev, int is_sqp, struct mlx4_qp *qp) { struct mlx4_qp_table *qp_table = &mlx4_priv(dev)->qp_table; @@ -240,7 +240,8 @@ void mlx4_qp_free(struct mlx4_dev *dev, mlx4_table_put(dev, &qp_table->auxc_table, qp->qpn); mlx4_table_put(dev, &qp_table->qp_table, qp->qpn); - mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); + if (!is_sqp) + mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); } EXPORT_SYMBOL_GPL(mlx4_qp_free); Index: infiniband/include/linux/mlx4/device.h =================================================================== --- infiniband.orig/include/linux/mlx4/device.h 2007-08-06 17:02:03.327010000 +0300 +++ infiniband/include/linux/mlx4/device.h 2007-08-06 17:02:21.805935000 +0300 @@ -317,7 +317,7 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, void mlx4_cq_free(struct mlx4_dev *dev, struct mlx4_cq *cq); int mlx4_qp_alloc(struct mlx4_dev *dev, int sqpn, struct mlx4_qp *qp); -void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp); +void mlx4_qp_free(struct mlx4_dev *dev, int is_sqp, struct mlx4_qp *qp); int mlx4_srq_alloc(struct mlx4_dev *dev, u32 pdn, struct mlx4_mtt *mtt, u64 db_rec, struct mlx4_srq *srq); From hnguyen at linux.vnet.ibm.com Mon Aug 6 07:23:46 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Mon, 6 Aug 2007 16:23:46 +0200 Subject: [ofa-general] [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 Message-ID: <200708061623.46842.hnguyen@linux.vnet.ibm.com> Hello Michael and Vladimir! This patch below adds a backport patch for ehca to the dirs 2.6.16, 2.6.16_sles10 and 2.6.16_sles10_sp1 underneath kernel_patches/backport of ofed-1.2.c source tree. Thanks! Nam backport kmem_cache_zalloc() to 2.6.10, 2.6.10_sles10 and 2.6.10_sles10_sp1 Signed-off-by: Hoang-Nam Nguyen --- 2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 3 files changed, 291 insertions(+) diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:53:59.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:20.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:33.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); From mst at dev.mellanox.co.il Mon Aug 6 07:19:23 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 17:19:23 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708061623.46842.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> Message-ID: <20070806141923.GA4801@mellanox.co.il> Let's not do it this way. I think the right thing is to implement kmem_cache_zalloc by means of kmem_cache_allocand memset in kernel_addons. Quoting Hoang-Nam Nguyen : Subject: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 Hello Michael and Vladimir! This patch below adds a backport patch for ehca to the dirs 2.6.16, 2.6.16_sles10 and 2.6.16_sles10_sp1 underneath kernel_patches/backport of ofed-1.2.c source tree. Thanks! Nam backport kmem_cache_zalloc() to 2.6.10, 2.6.10_sles10 and 2.6.10_sles10_sp1 Signed-off-by: Hoang-Nam Nguyen --- 2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 3 files changed, 291 insertions(+) diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:53:59.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:20.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:33.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); -- MST From changquing.tang at hp.com Mon Aug 6 08:30:33 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 6 Aug 2007 15:30:33 -0000 Subject: [ofa-general] RFCv2: SRC API In-Reply-To: <20070806133500.GA1061@mellanox.co.il> References: <20070729140431.GG16915@mellanox.co.il> <20070806133500.GA1061@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> > +When job j1 needs to transmit to job j2 on remote node n for > the first time: > +1. Test: does job j1 have an existing connection to some job > on node n? > + - If no: > + j1 creates an SRC QP qp1 (send QP) > + qp1 is only used to post send WRs > + j2 creates an SRC QP qp2 > + qp2 is part of SRC domain > + qp2 is only used to do transport checks: > + neither send nor receive WRs > are posted on qp2 > + j1 and j2 create a connection between qp1 and qp2 > + - If yes: > + let qp1 be the QP which belongs to j1 and is connected > + to some qp on node n > + > +2. j1 gets SRQ number from j2 > +3. j1 can now use QP qp2 from step 1 > + and SRQ number from step 3 to send data to j2 > + > +Cleanup: > +When job j1 does not need to communicate to any jobs on node n, it > +disconnects qp1 from qp2, and asks j2 to destroy qp2. Suppose remote node n has j2/qp2, j3/qp3, j4/qp4, qp1 on j1 is connected to qp2, then there is no need to make connection between qp1 and qp3, qp1 and qp4, there are automatically connected, right ? Then how can j3 know that j2 has connected to j1, it does not need to make connection again ? qp1 find destination by SRQ number only, so "+3. j1 can now use QP qp2 from step 1", what does it mean ? Can we destroy qp2 on j2 first, and keep j1 and j3 continue to communicate ? --CQ From changquing.tang at hp.com Mon Aug 6 08:37:15 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 6 Aug 2007 15:37:15 -0000 Subject: [ofa-general] RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> References: <20070729140431.GG16915@mellanox.co.il><20070806133500.GA1061@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840301F8EE98@G3W0634.americas.hpqcorp.net> Maybe I miss something here. Only of the job among j2, j3, j4 on remote node n need to create a receiving qp2 for j1, right ? --CQ > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Tang, Changqing > Sent: Monday, August 06, 2007 10:31 AM > To: Michael S. Tsirkin > Cc: Pavel Shamis; Gil Bloch; ewg at lists.openfabrics.org; > general at lists.openfabrics.org; Ishai Rabinovitz > Subject: RE: [ofa-general] RFCv2: SRC API > > > > +When job j1 needs to transmit to job j2 on remote node n for > > the first time: > > +1. Test: does job j1 have an existing connection to some job > > on node n? > > + - If no: > > + j1 creates an SRC QP qp1 (send QP) > > + qp1 is only used to post send WRs > > + j2 creates an SRC QP qp2 > > + qp2 is part of SRC domain > > + qp2 is only used to do transport checks: > > + neither send nor receive WRs > > are posted on qp2 > > + j1 and j2 create a connection between qp1 and qp2 > > + - If yes: > > + let qp1 be the QP which belongs to j1 and is connected > > + to some qp on node n > > + > > +2. j1 gets SRQ number from j2 > > +3. j1 can now use QP qp2 from step 1 > > + and SRQ number from step 3 to send data to j2 > > + > > +Cleanup: > > +When job j1 does not need to communicate to any jobs on node n, it > > +disconnects qp1 from qp2, and asks j2 to destroy qp2. > > Suppose remote node n has j2/qp2, j3/qp3, j4/qp4, qp1 on j1 > is connected to qp2, then there is no need to make connection > between qp1 and qp3, > qp1 and qp4, there are automatically connected, right ? Then > how can j3 know that j2 has connected to j1, it does not need > to make connection again ? > > qp1 find destination by SRQ number only, so "+3. j1 can now > use QP qp2 from step 1", what does it mean ? > > Can we destroy qp2 on j2 first, and keep j1 and j3 continue > to communicate ? > > --CQ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From swise at opengridcomputing.com Mon Aug 6 08:46:42 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 06 Aug 2007 10:46:42 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <000001c7d70b$ef160bc0$78c8180a@amr.corp.intel.com> References: <000001c7d70b$ef160bc0$78c8180a@amr.corp.intel.com> Message-ID: <46B74262.8050907@opengridcomputing.com> Sean Hefty wrote: >> Lemme know how I can help. I certainly can test any patches on my 8 >> node iwarp cluster. > > We should probably take the idea to netdev before making any substantial changes > to the code. > > - Sean Yup. Should I post my RFC patch and we'll go from there? Steve. From changquing.tang at hp.com Mon Aug 6 08:51:07 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 6 Aug 2007 15:51:07 -0000 Subject: [ofa-general] RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> References: <20070729140431.GG16915@mellanox.co.il><20070806133500.GA1061@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840301F8EEE0@G3W0634.americas.hpqcorp.net> > > +When job j1 needs to transmit to job j2 on remote node n for > > the first time: > > +1. Test: does job j1 have an existing connection to some job > > on node n? > > + - If no: > > + j1 creates an SRC QP qp1 (send QP) > > + qp1 is only used to post send WRs > > + j2 creates an SRC QP qp2 > > + qp2 is part of SRC domain > > + qp2 is only used to do transport checks: > > + neither send nor receive WRs > > are posted on qp2 > > + j1 and j2 create a connection between qp1 and qp2 > > + - If yes: > > + let qp1 be the QP which belongs to j1 and is connected > > + to some qp on node n > > + > > +2. j1 gets SRQ number from j2 > > +3. j1 can now use QP qp2 from step 1 > > + and SRQ number from step 3 to send data to j2 > > + > > +Cleanup: > > +When job j1 does not need to communicate to any jobs on node n, it > > +disconnects qp1 from qp2, and asks j2 to destroy qp2. OK, I was wrong before, here is my question. if remote node n has j2, j3, and j4, and j2 is the job to create qp2 and make connection with qp1 in j1. if j2 is done before j3 and j4, then we can not let j2 to destroy qp2, because j3 and j4 are still communicating with j1. Since j2 owns qp2, j2 need to be the last job to cleanup. Am I right ? From mst at dev.mellanox.co.il Mon Aug 6 08:51:57 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 18:51:57 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EE98@G3W0634.americas.hpqcorp.net> References: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> <349DCDA352EACF42A0C49FA6DCEA840301F8EE98@G3W0634.americas.hpqcorp.net> Message-ID: <20070806155157.GA6779@mellanox.co.il> > Only of the job among j2, j3, j4 on remote node n need to create a > receiving qp2 for j1, right ? Correct. A single QP can be used to send data to any SRQ that shares the same domain. -- MST From mst at dev.mellanox.co.il Mon Aug 6 08:54:30 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 18:54:30 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> References: <20070729140431.GG16915@mellanox.co.il> <20070806133500.GA1061@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> Message-ID: <20070806155430.GB6779@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > +When job j1 needs to transmit to job j2 on remote node n for > > the first time: > > +1. Test: does job j1 have an existing connection to some job > > on node n? > > + - If no: > > + j1 creates an SRC QP qp1 (send QP) > > + qp1 is only used to post send WRs > > + j2 creates an SRC QP qp2 > > + qp2 is part of SRC domain > > + qp2 is only used to do transport checks: > > + neither send nor receive WRs > > are posted on qp2 > > + j1 and j2 create a connection between qp1 and qp2 > > + - If yes: > > + let qp1 be the QP which belongs to j1 and is connected > > + to some qp on node n > > + > > +2. j1 gets SRQ number from j2 > > +3. j1 can now use QP qp2 from step 1 > > + and SRQ number from step 3 to send data to j2 > > + > > +Cleanup: > > +When job j1 does not need to communicate to any jobs on node n, it > > +disconnects qp1 from qp2, and asks j2 to destroy qp2. > > Suppose remote node n has j2/qp2, j3/qp3, j4/qp4, You got it wrong, j3 and j4 do not need to create QPs to get packets from j1. > qp1 on j1 is connected > to qp2, then there is no need to make connection between qp1 and qp3, > qp1 and qp4, there are automatically connected, right ? Then how can j3 > know that j2 has connected to j1, it does not need to make connection > again ? > > qp1 find destination by SRQ number only, so "+3. j1 can now use QP qp2 > from step 1", what does it mean ? > > Can we destroy qp2 on j2 first, and keep j1 and j3 continue to > communicate ? Communication is only possible as long as both qp1 and qp2 exist. -- MST From mst at dev.mellanox.co.il Mon Aug 6 08:55:52 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 18:55:52 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EEE0@G3W0634.americas.hpqcorp.net> References: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net> <349DCDA352EACF42A0C49FA6DCEA840301F8EEE0@G3W0634.americas.hpqcorp.net> Message-ID: <20070806155552.GC6779@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > > +When job j1 needs to transmit to job j2 on remote node n for > > > the first time: > > > +1. Test: does job j1 have an existing connection to some job > > > on node n? > > > + - If no: > > > + j1 creates an SRC QP qp1 (send QP) > > > + qp1 is only used to post send WRs > > > + j2 creates an SRC QP qp2 > > > + qp2 is part of SRC domain > > > + qp2 is only used to do transport checks: > > > + neither send nor receive WRs > > > are posted on qp2 > > > + j1 and j2 create a connection between qp1 and qp2 > > > + - If yes: > > > + let qp1 be the QP which belongs to j1 and is connected > > > + to some qp on node n > > > + > > > +2. j1 gets SRQ number from j2 > > > +3. j1 can now use QP qp2 from step 1 > > > + and SRQ number from step 3 to send data to j2 > > > + > > > +Cleanup: > > > +When job j1 does not need to communicate to any jobs on node n, it > > > +disconnects qp1 from qp2, and asks j2 to destroy qp2. > > OK, I was wrong before, here is my question. > > if remote node n has j2, j3, and j4, and j2 is the job to create qp2 and > make connection with qp1 in j1. > if j2 is done before j3 and j4, then we can not let j2 to destroy qp2, > because j3 and j4 are still communicating with > j1. Since j2 owns qp2, j2 need to be the last job to cleanup. > > Am I right ? Correct. Is this clear from the text, or is some kind of additional clarification necessary? -- MST From changquing.tang at hp.com Mon Aug 6 09:03:11 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 6 Aug 2007 16:03:11 -0000 Subject: [ofa-general] RE: RFCv2: SRC API In-Reply-To: <20070806155552.GC6779@mellanox.co.il> References: <349DCDA352EACF42A0C49FA6DCEA840301F8EE71@G3W0634.americas.hpqcorp.net><349DCDA352EACF42A0C49FA6DCEA840301F8EEE0@G3W0634.americas.hpqcorp.net> <20070806155552.GC6779@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840301F8EF20@G3W0634.americas.hpqcorp.net> > > OK, I was wrong before, here is my question. > > > > if remote node n has j2, j3, and j4, and j2 is the job to > create qp2 > > and make connection with qp1 in j1. > > if j2 is done before j3 and j4, then we can not let j2 to > destroy qp2, > > because j3 and j4 are still communicating with j1. Since j2 > owns qp2, > > j2 need to be the last job to cleanup. > > > > Am I right ? > > Correct. Is this clear from the text, or is some kind of > additional clarification necessary? It is not clear at the first read, so please add one sentence to clarify it. if j2 is the last job to cleanup, how can it know all other jobs on the same node has called ibv_close_src_domain(), and it is time for itself to cleanup ? Is this something upto application to do ? --CQ > > -- > MST > From mshefty at ichips.intel.com Mon Aug 6 09:15:58 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 09:15:58 -0700 Subject: [ofa-general] QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46B58A8B.5020609@voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58A8B.5020609@voltaire.com> Message-ID: <46B7493E.7050400@ichips.intel.com> > I wonder how you think the suggested scheme for IPoIB plugs into the > existing QoS flow at the Linux network stack. Looking on the man pages > of tcp(7) and ip(7) I see that there's a SOL_SOCKET level option of > SO_PRIORITY and SOL_IP level option of IP_TOS. I was looking at IB routing, not IP routing. IPv6 uses flow info, which, from what I can tell, is traffic class and flow label. > Looking lower in the stack, on the IPoEth scheme where the 802.1q is > used for the VLAN header generation, the code seems to generate the > p_bits field of the header using skb->priority (see the call to > vlan_dev_get_egress_qos_mask() from vlan_dev_hard_header() at > net/8021q/vlan_dev.c) Someone with more knowledge of IPoIB than me needs to provide input here. I thought the use of different PKeys was similar to the selection of a VLAN. - Sean From mst at dev.mellanox.co.il Mon Aug 6 09:16:39 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 19:16:39 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EF20@G3W0634.americas.hpqcorp.net> References: <20070806155552.GC6779@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA840301F8EF20@G3W0634.americas.hpqcorp.net> Message-ID: <20070806161639.GE6779@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > > > OK, I was wrong before, here is my question. > > > > > > if remote node n has j2, j3, and j4, and j2 is the job to > > create qp2 > > > and make connection with qp1 in j1. > > > if j2 is done before j3 and j4, then we can not let j2 to > > destroy qp2, > > > because j3 and j4 are still communicating with j1. Since j2 > > owns qp2, > > > j2 need to be the last job to cleanup. > > > > > > Am I right ? > > > > Correct. Is this clear from the text, or is some kind of > > additional clarification necessary? > > It is not clear at the first read, so please add one sentence to clarify > it. Would something like this help? Cleanup: When job j1 does not need to communicate to any jobs on node n, it disconnects qp1 from qp2, and asks j2 to destroy qp2. + +Note: both qp1 and qp2 must exist for the communication to take place. +Thus, j2 should not destroy qp2 (and in particular, should not exit) +until j1 has completed communication with node n and +has asked j2 to disconnect. > if j2 is the last job to cleanup, how can it know all other jobs on the > same node has called > ibv_close_src_domain(), and it is time for itself to cleanup ? > > Is this something upto application to do ? No, this is handled automatically. Have you seen this text? * ibv_close_src_domain - close an SRC domain * If this is the last reference, destroys the domain. So, each job has a reference to the domain. Once the last reference is gone, the domain is destroyed. -- MST From mshefty at ichips.intel.com Mon Aug 6 09:37:16 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 09:37:16 -0700 Subject: [ofa-general] QoS RFC In-Reply-To: <46B58CEA.5010709@voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58CEA.5010709@voltaire.com> Message-ID: <46B74E3C.8040007@ichips.intel.com> >> Application Service -> Service ID (or range) >> Service ID -> desired QoS >> QoS, SGID, DGID, PKey -> SGID, DGID, TClass, FlowLabel, PKey >> SGID, DGID, TC, FL, PKey -> SLID, DLID, SL (set if crossing subnets) >> SLID, DLID, SL -> MTU, Rate, VL, PacketLifeTime > > What's the reasoning to use TClass and Flowlabel here? is it that you > want to come up with a scheme which is valid also for traffic crossing > IB subnets? Yes - If you look at the data packet formats, the traffic class and flow label are the only fields that get carried between IB subnets. They should be sufficient to specify a desired QoS. In a similar fashion, the LRH carries the SLID, DLID, and SL, which should be sufficient to specify local subnet QoS. > you suggest two mappings: > > A) from SID to "desired QoS" > B) from "desired QoS" (and more params) to TClass and FlowLabel > > First, my understanding is that "QoS" is an abstract term here, that is > does not translate to concrete IB term, correct? now, what entity per > your design would translate from SID to QoS, is it something done > internally in the host stack similar to what the net stack does with the > IP_TOS and SOL_PRIORITY socket options? Yes I view QoS as an abstract term, meaning we have a lot of flexibility in how it's defined. I've chosen to define it wrt the hierarchy above. Personally, I would drop the SID to QoS mapping, but if it is needed, keep the mapping done on the SA side. - Sean From changquing.tang at hp.com Mon Aug 6 09:38:52 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Mon, 6 Aug 2007 16:38:52 -0000 Subject: [ofa-general] RE: RFCv2: SRC API In-Reply-To: <20070806161639.GE6779@mellanox.co.il> References: <20070806155552.GC6779@mellanox.co.il><349DCDA352EACF42A0C49FA6DCEA840301F8EF20@G3W0634.americas.hpqcorp.net> <20070806161639.GE6779@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840301F8EFD0@G3W0634.americas.hpqcorp.net> > Cleanup: > When job j1 does not need to communicate to any jobs on node > n, it disconnects qp1 from qp2, and asks j2 to destroy qp2. > + > +Note: both qp1 and qp2 must exist for the communication to > take place. > +Thus, j2 should not destroy qp2 (and in particular, should not exit) > +until j1 has completed communication with node n and has asked j2 to > +disconnect. > Thanks. Another question. if a node n has 8 jobs, say, j2-j9, usually the first job j2 is the one to create the SRC domain(other jobs just attach and share) and it make sense to let j2 to create all the receiving QPs for all other remote jobs and make all the connections. (we can do in roundrobin way, but more work). Is there any performance worry to let j2(the first job on a node) to do all the "work" ? What is the latency of SRC+SRQ ? --CQ From mst at dev.mellanox.co.il Mon Aug 6 09:44:22 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 19:44:22 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840301F8EFD0@G3W0634.americas.hpqcorp.net> References: <20070806161639.GE6779@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA840301F8EFD0@G3W0634.americas.hpqcorp.net> Message-ID: <20070806164422.GF6779@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > Cleanup: > > When job j1 does not need to communicate to any jobs on node > > n, it disconnects qp1 from qp2, and asks j2 to destroy qp2. > > + > > +Note: both qp1 and qp2 must exist for the communication to > > take place. > > +Thus, j2 should not destroy qp2 (and in particular, should not exit) > > +until j1 has completed communication with node n and has asked j2 to > > +disconnect. > > > Thanks. > > Another question. if a node n has 8 jobs, say, j2-j9, usually the first > job j2 is the one to create the SRC > domain(other jobs just attach and share) and it make sense to let j2 to > create all the receiving QPs for all other > remote jobs and make all the connections. (we can do in roundrobin way, > but more work). Sure, creating allconnections upfront will work to, this is just a usage example. > Is there any performance worry to let j2(the first job on a node) to do > all the "work" ? How do you mean? > What is the latency of SRC+SRQ ? I'd expect it to be more or less the same as regular SRQ. -- MST From tziporet at dev.mellanox.co.il Mon Aug 6 09:49:04 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 06 Aug 2007 19:49:04 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <46B74E3C.8040007@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58CEA.5010709@voltaire.com> <46B74E3C.8040007@ichips.intel.com> Message-ID: <46B75100.1000404@mellanox.co.il> Sean Hefty wrote: > > Yes I view QoS as an abstract term, meaning we have a lot of > flexibility in how it's defined. I've chosen to define it wrt the > hierarchy above. Personally, I would drop the SID to QoS mapping, but > if it is needed, keep the mapping done on the SA side. > > - Sean Hi Sean, When do you think that changes to support QoS will be in the CM & CMA? Tziporet From mshefty at ichips.intel.com Mon Aug 6 09:52:10 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 09:52:10 -0700 Subject: [ofa-general] QoS RFC In-Reply-To: <46B58E8E.1040001@voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46AF6281.10709@ichips.intel.com> <46B58E8E.1040001@voltaire.com> Message-ID: <46B751BA.1050800@ichips.intel.com> > From what I understand while reading your proposal, is that it is quite > different then what what suggested in the original RFC. I don't think it > makes sense to implement the host side of this before there's agreement > on the over-all solution namely how the host side design/code plugs to > the management scheme at the SM side. I don't believe that my proposal and the SA side proposal are incompatible. We should be able to design the host side stack somewhat independent from a specific SA implementation. It needs to be to support alternative SA implementations. > One more thing that bothers me is backward compatibility with SM/SA, > that does not support the not-published-yet IBTA QoS extensions. Where > you thinking to first probe for the SA capabilities to see if it > supports QoS path-queries or think its an over-doing? I don't know that querying the SA for QoS capabilities is necessary. The only thing I think you can do with that information is to display a message on each host indicating that QoS is not supported in some circumstances - those PR queries that rely on the QoS or service ID field. An administrator could get this information in other ways - Sean From mshefty at ichips.intel.com Mon Aug 6 09:57:16 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 09:57:16 -0700 Subject: [ofa-general] QoS RFC In-Reply-To: <46B75100.1000404@mellanox.co.il> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58CEA.5010709@voltaire.com> <46B74E3C.8040007@ichips.intel.com> <46B75100.1000404@mellanox.co.il> Message-ID: <46B752EC.1070505@ichips.intel.com> > When do you think that changes to support QoS will be in the CM & CMA? I should finish changes for IPoIB and the CMA sometime this week, but the testing won't include an SA that supports Qos... Based on some of Or's comments, IPoIB may need additional work. - Sean From mshefty at ichips.intel.com Mon Aug 6 09:58:42 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 09:58:42 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B74262.8050907@opengridcomputing.com> References: <000001c7d70b$ef160bc0$78c8180a@amr.corp.intel.com> <46B74262.8050907@opengridcomputing.com> Message-ID: <46B75342.4040407@ichips.intel.com> > Should I post my RFC patch and we'll go from there? Sounds good to me. Roland, do you have any opinion? - Sean From mshefty at ichips.intel.com Mon Aug 6 10:14:49 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 10:14:49 -0700 Subject: [ofa-general] implicit address binding by rdma_resolve_addr In-Reply-To: References: Message-ID: <46B75709.5080507@ichips.intel.com> > We have noticed that when rdma_resolve_addr is provided with src > address, the rdma cm does an implicit --address-- (=ip/port) binding, > which for example has the side effect of disallowing two processes to > call rdma_resolve_addr() with the same src port. > > Does this implicit binding mimics the network stack behavior > under similar flow when socket are used? Specifying a source address with rdma_resolve_addr is similar to calling socket bind. I can't think of a reason why an app would want to specify a source port when calling rdma_resolve_addr. Specifying a source address (wild card port) allows binding to a specific RDMA device. - Sean From kbaxley at redhat.com Mon Aug 6 10:36:16 2007 From: kbaxley at redhat.com (Kent Baxley) Date: Mon, 6 Aug 2007 13:36:16 -0400 Subject: [ofa-general] Infiniband Statistics? Message-ID: <20070806133616.2b20575a@dhcp-243-71.rdu.redhat.com> Hi, I'm seeking some additional information on what the various numbers are that the Infiniband Diagnostic tools report. In particular, I'm interested in the output of 'perfquery' (from the RHEL4 openib-diags-1.1.0-7 package). I could not find any documentation about (e.g.) XmtBytes, RcvBytes, XmtPkts, RcvPkts, and it would be good to know (e.g.) why the ratio of XmtBytes to XmtPkts is not constant. Is there anything out there that will help point me in the right direction? Thanks. --Kent From hal.rosenstock at gmail.com Mon Aug 6 10:52:16 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 6 Aug 2007 13:52:16 -0400 Subject: [ofa-general] Infiniband Statistics? In-Reply-To: <20070806133616.2b20575a@dhcp-243-71.rdu.redhat.com> References: <20070806133616.2b20575a@dhcp-243-71.rdu.redhat.com> Message-ID: On 8/6/07, Kent Baxley wrote: > > Hi, > > > I'm seeking some additional information on what the various numbers are > that the Infiniband Diagnostic tools report. > > In particular, I'm interested in the output of 'perfquery' (from > the RHEL4 openib-diags-1.1.0-7 package). > > I could not find any documentation about (e.g.) XmtBytes, RcvBytes, > XmtPkts, RcvPkts, and it would be good to know (e.g.) why the ratio of > XmtBytes to XmtPkts is not constant. > > Is there anything out there that will help point me in the right > direction? Those counters are defined in the IBA 1.2 spec vol 1 p. 948. -- Hal > > Thanks. > > --Kent > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From todd.rimmer at qlogic.com Mon Aug 6 11:23:52 2007 From: todd.rimmer at qlogic.com (Todd Rimmer) Date: Mon, 6 Aug 2007 13:23:52 -0500 Subject: [ofa-general] Infiniband Statistics? In-Reply-To: <20070806133616.2b20575a@dhcp-243-71.rdu.redhat.com> Message-ID: <4FB1BCCAE6CAED44A1DC005B1DE061192DE93A@EPEXCH2.qlogic.org> > -----Original Message----- > From: Kent Baxley > Sent: Monday, August 06, 2007 1:36 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] Infiniband Statistics? > > > I could not find any documentation about (e.g.) XmtBytes, RcvBytes, > XmtPkts, RcvPkts, and it would be good to know (e.g.) why the ratio of > XmtBytes to XmtPkts is not constant. > XmtBytes is the actual count of bytes sent on the wire (actually hardware tracks it in terms of 32 bit Quad bytes). XmtPkts is the number of IB packets on the wire. Depending on application the actual number of bytes per packet can vary (constrained by MTU). Hence there is NO consistent N for which Packets = Bytes/N Todd Rimmer Chief Architect QLogic System Interconnect Group Voice: 610-233-4852 Fax: 610-233-4777 Todd.Rimmer at QLogic.com www.QLogic.com From mshefty at ichips.intel.com Mon Aug 6 11:26:04 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 06 Aug 2007 11:26:04 -0700 Subject: [ofa-general] QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46B58A8B.5020609@voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58A8B.5020609@voltaire.com> Message-ID: <46B767BC.2040207@ichips.intel.com> > I wonder how you think the suggested scheme for IPoIB plugs into the > existing QoS flow at the Linux network stack. Looking on the man pages > of tcp(7) and ip(7) I see that there's a SOL_SOCKET level option of > SO_PRIORITY and SOL_IP level option of IP_TOS. Giving this a little more thought, IPoIB uses a single path to each IP destination. You're asking about per socket ToS, which requires the ability to handle multiple paths to each remote IP address. Both the original RFC and my proposal for the host stack limit all IPoIB traffic to a single QoS level. Is there a real need to expand IPoIB ToS capabilities? - Sean From mst at dev.mellanox.co.il Mon Aug 6 11:49:17 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 21:49:17 +0300 Subject: [ofa-general] Re: QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46B767BC.2040207@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58A8B.5020609@voltaire.com> <46B767BC.2040207@ichips.intel.com> Message-ID: <20070806184917.GF11422@mellanox.co.il> > Both the original RFC and my proposal for the host stack limit all IPoIB traffic > to a single QoS level. Wouldn't it be possible to use different QoS levels for different partitions? -- MST From sean.hefty at intel.com Mon Aug 6 11:54:20 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 11:54:20 -0700 Subject: [ofa-general] Re: QoS RFC / how its linked to the networking stackQoS? In-Reply-To: <20070806184917.GF11422@mellanox.co.il> Message-ID: <000101c7d85b$32ff1570$3c98070a@amr.corp.intel.com> >> Both the original RFC and my proposal for the host stack limit all IPoIB >traffic >> to a single QoS level. > >Wouldn't it be possible to use different QoS levels for different partitions? Yes - but I don't think this would handle per socket QoS options. I should have been clearer, and said "all IPoIB traffic within a given broadcast domain". - Sean From mst at dev.mellanox.co.il Mon Aug 6 12:01:31 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 6 Aug 2007 22:01:31 +0300 Subject: [ofa-general] Re: Re: QoS RFC / how its linked to the networking stackQoS? In-Reply-To: <000101c7d85b$32ff1570$3c98070a@amr.corp.intel.com> References: <20070806184917.GF11422@mellanox.co.il> <000101c7d85b$32ff1570$3c98070a@amr.corp.intel.com> Message-ID: <20070806190131.GG11422@mellanox.co.il> > Quoting Sean Hefty : > Subject: RE: Re: QoS RFC / how its linked to the networking stackQoS? > > >> Both the original RFC and my proposal for the host stack limit all IPoIB > >>traffic to a single QoS level. > > > >Wouldn't it be possible to use different QoS levels for different partitions? > > Yes - but I don't think this would handle per socket QoS options. Right. > I should have > been clearer, and said "all IPoIB traffic within a given broadcast domain". Right. -- MST From swise at opengridcomputing.com Mon Aug 6 13:11:45 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Mon, 06 Aug 2007 15:11:45 -0500 Subject: [ofa-general] virtual memory usage on long lived apps Message-ID: <46B78081.3060305@opengridcomputing.com> Hey Roland, I have a verbs/memory mapping question for ya: I've made some enhancements to the rdma-cm based 'rping' server to allow a persistent server mode. This means the server process never exits, but rather spawns threads for each new incoming connection. Like a traditional network server, except I'm using pthreads instead of new processes to handle each new connection. Running this server with clients repeatedly connecting, running rpings, and disconnecting, I notice that the virtual memory consumed by the process is ever increasing. The shared/and reserved values don't increase, but the virtual does. IE: If you watch this process via 'top' you'll see the VIRT value increasing steadily over time/connections, and the RES/SHR values stay steady. I've seen it over 1GB of virtual memory used! Now, each new connection malloc()s memory for the rping test (around 64KB), and registers this memory as part of setting up the new rping threads. And I believe it properly unregisters and free()s this memory after the connection disconnects. Q: Have you seen this before? Its like the OS never reuses mappings that were setup for memory registrations. Is that normal? Or should the virtual size shown in 'top' be reduced whenever a buffer is unregistered and freed? There's a definite possibility the code is leaking memory, but I'm wondering if this is perhaps a normal anomaly with mapping memory in linux... Thoughts? Thanks, Steve. From kliteyn at dev.mellanox.co.il Mon Aug 6 14:05:08 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 07 Aug 2007 00:05:08 +0300 Subject: [ofa-general] [PATCH] osm: default leaf vl_stall and head_of_queue_life counters Message-ID: <46B78D04.7020002@dev.mellanox.co.il> Hi Sasha This patch changes the OSM_DEFAULT_LEAF_VL_STALL_COUNT and OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE (matching them to the default counters on the switch-to-switch ports), in order to deal with the casual PCI Express "hiccups". Please apply it to ofed_1_2 and master. -- Yevgeny Signed-off-by: Yevgeny Kliteynik --- osm/include/opensm/osm_base.h | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/osm/include/opensm/osm_base.h b/osm/include/opensm/osm_base.h index b38b511..0cb4651 100644 --- a/osm/include/opensm/osm_base.h +++ b/osm/include/opensm/osm_base.h @@ -311,11 +311,11 @@ BEGIN_C_DECLS * DESCRIPTION * Sets the time a packet can live in the head of the VL Queue * of a port that drives a CA port. -* We use here the value of ~130usec +* We use here the value of ~1sec * * SYNOPSIS */ -#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0xC +#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0x12 /***********/ /****d* OpenSM: Base/OSM_DEFAULT_VL_STALL_COUNT @@ -341,11 +341,10 @@ BEGIN_C_DECLS * puts the VL into stalled state. In stalled state, the port is supposed * to drop everything for 8*(head of queue lifetime). This value is for * switch ports driving a CA port. -* We use the value of 1 here - so any drop due to HOQ means stalling the VL * * SYNOPSIS */ -#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x1 +#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x7 /***********/ /****d* OpenSM: Base/OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT -- 1.5.1.4 From sean.hefty at intel.com Mon Aug 6 17:06:35 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 17:06:35 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support Message-ID: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> To support QoS within and between subnets, modify IPoIB to request specific TClass and FlowLabel values with path record queries, using the values associated with the IPoIB broadcast group. This allows QoS to be defined for a given IPoIB broadcast group. Signed-off-by: Sean Hefty --- drivers/infiniband/ulp/ipoib/ipoib.h | 22 +++++++++++++++++++++- drivers/infiniband/ulp/ipoib/ipoib_main.c | 8 +++++--- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 22 ---------------------- 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 285c143..fc16bce 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -113,7 +113,27 @@ struct ipoib_pseudoheader { u8 hwaddr[INFINIBAND_ALEN]; }; -struct ipoib_mcast; +/* Used for all multicast joins (broadcast, IPv4 mcast and IPv6 mcast) */ +struct ipoib_mcast { + struct ib_sa_mcmember_rec mcmember; + struct ib_sa_multicast *mc; + struct ipoib_ah *ah; + + struct rb_node rb_node; + struct list_head list; + + unsigned long created; + unsigned long backoff; + + unsigned long flags; + unsigned char logcount; + + struct list_head neigh_list; + + struct sk_buff_head pkt_queue; + + struct net_device *dev; +}; struct ipoib_rx_buf { struct sk_buff *skb; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 894b1dc..e22680f 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -468,9 +468,11 @@ static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid) INIT_LIST_HEAD(&path->neigh_list); memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid)); - path->pathrec.sgid = priv->local_gid; - path->pathrec.pkey = cpu_to_be16(priv->pkey); - path->pathrec.numb_path = 1; + path->pathrec.sgid = priv->local_gid; + path->pathrec.pkey = cpu_to_be16(priv->pkey); + path->pathrec.numb_path = 1; + path->pathrec.flow_label = priv->broadcast->mcmember.flow_label; + path->pathrec.traffic_class = priv->broadcast->mcmember.traffic_class; return path; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index aae3670..94a5709 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -57,28 +57,6 @@ MODULE_PARM_DESC(mcast_debug_level, static DEFINE_MUTEX(mcast_mutex); -/* Used for all multicast joins (broadcast, IPv4 mcast and IPv6 mcast) */ -struct ipoib_mcast { - struct ib_sa_mcmember_rec mcmember; - struct ib_sa_multicast *mc; - struct ipoib_ah *ah; - - struct rb_node rb_node; - struct list_head list; - - unsigned long created; - unsigned long backoff; - - unsigned long flags; - unsigned char logcount; - - struct list_head neigh_list; - - struct sk_buff_head pkt_queue; - - struct net_device *dev; -}; - struct ipoib_mcast_iter { struct net_device *dev; union ib_gid mgid; From sean.hefty at intel.com Mon Aug 6 17:09:13 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 17:09:13 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> Message-ID: <000101c7d887$2fbbcd50$9c98070a@amr.corp.intel.com> Add the new quality of service fields to path records for consumers that want to use them. Signed-off-by: Sean Hefty --- This patch has not been tested against an SA that supports QoS. drivers/infiniband/core/sa_query.c | 10 +++------- include/rdma/ib_sa.h | 11 +++++------ 2 files changed, 8 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index 6469406..6cf60a7 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -122,14 +122,10 @@ static u32 tid; .field_name = "sa_path_rec:" #field static const struct ib_field path_rec_table[] = { - { RESERVED, + { PATH_REC_FIELD(service_id), .offset_words = 0, .offset_bits = 0, - .size_bits = 32 }, - { RESERVED, - .offset_words = 1, - .offset_bits = 0, - .size_bits = 32 }, + .size_bits = 64 }, { PATH_REC_FIELD(dgid), .offset_words = 2, .offset_bits = 0, @@ -178,7 +174,7 @@ static const struct ib_field path_rec_table[] = { .offset_words = 12, .offset_bits = 16, .size_bits = 16 }, - { RESERVED, + { PATH_REC_FIELD(qos_class), .offset_words = 13, .offset_bits = 0, .size_bits = 12 }, diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h index 5e26b2f..f73991a 100644 --- a/include/rdma/ib_sa.h +++ b/include/rdma/ib_sa.h @@ -109,8 +109,8 @@ enum ib_sa_selector { * Reserved rows are indicated with comments to help maintainability. */ -/* reserved: 0 */ -/* reserved: 1 */ +#define IB_SA_PATH_REC_SERVICE_ID IB_SA_COMP_MASK( 0) \ + IB_SA_COMP_MASK( 1) #define IB_SA_PATH_REC_DGID IB_SA_COMP_MASK( 2) #define IB_SA_PATH_REC_SGID IB_SA_COMP_MASK( 3) #define IB_SA_PATH_REC_DLID IB_SA_COMP_MASK( 4) @@ -123,7 +123,7 @@ enum ib_sa_selector { #define IB_SA_PATH_REC_REVERSIBLE IB_SA_COMP_MASK(11) #define IB_SA_PATH_REC_NUMB_PATH IB_SA_COMP_MASK(12) #define IB_SA_PATH_REC_PKEY IB_SA_COMP_MASK(13) -/* reserved: 14 */ +#define IB_SA_PATH_REC_QOS_CLASS IB_SA_COMP_MASK(14) #define IB_SA_PATH_REC_SL IB_SA_COMP_MASK(15) #define IB_SA_PATH_REC_MTU_SELECTOR IB_SA_COMP_MASK(16) #define IB_SA_PATH_REC_MTU IB_SA_COMP_MASK(17) @@ -134,8 +134,7 @@ enum ib_sa_selector { #define IB_SA_PATH_REC_PREFERENCE IB_SA_COMP_MASK(22) struct ib_sa_path_rec { - /* reserved */ - /* reserved */ + __be64 service_id; union ib_gid dgid; union ib_gid sgid; __be16 dlid; @@ -148,7 +147,7 @@ struct ib_sa_path_rec { int reversible; u8 numb_path; __be16 pkey; - /* reserved */ + __be16 qos_class; u8 sl; u8 mtu_selector; u8 mtu; From sean.hefty at intel.com Mon Aug 6 17:13:58 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 17:13:58 -0700 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service In-Reply-To: <000101c7d887$2fbbcd50$9c98070a@amr.corp.intel.com> Message-ID: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> Provide support to specify a type of service for a communication identifier. A new function call is used when dealing with IPv4 addresses. For IPv6 addresses, the ToS is specified through the traffic class and flow label fields in the sockaddr_in6 structure. Signed-off-by: Sean Hefty --- drivers/infiniband/core/cma.c | 37 +++++++++++++++++++++++++++++++------ include/rdma/rdma_cm.h | 14 ++++++++++++++ 2 files changed, 45 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 32a0e66..c8b15f0 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -138,6 +138,7 @@ struct rdma_id_private { u32 qkey; u32 qp_num; u8 srq; + u8 tos; }; struct cma_multicast { @@ -1474,6 +1475,15 @@ err: } EXPORT_SYMBOL(rdma_listen); +void rdma_set_service_type(struct rdma_cm_id *id, int tos) +{ + struct rdma_id_private *id_priv; + + id_priv = container_of(id, struct rdma_id_private, id); + id_priv->tos = (u8) tos; +} +EXPORT_SYMBOL(rdma_set_service_type); + static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec, void *context) { @@ -1500,6 +1510,8 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms, { struct rdma_dev_addr *addr = &id_priv->id.route.addr.dev_addr; struct ib_sa_path_rec path_rec; + ib_sa_comp_mask comp_mask; + struct sockaddr_in6 *sin6; memset(&path_rec, 0, sizeof path_rec); ib_addr_get_sgid(addr, &path_rec.sgid); @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms, path_rec.numb_path = 1; path_rec.reversible = 1; + comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | + IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | + IB_SA_PATH_REC_REVERSIBLE; + + if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) { + path_rec.qos_class = cpu_to_be16((u16) id_priv->tos); + comp_mask |= IB_SA_PATH_REC_QOS_CLASS; + } else { + sin6 = (struct sockaddr_in6 *) id_priv->id.route.addr.src_addr; + path_rec.flow_label = sin6->sin6_flowinfo; + path_rec.traffic_class = (u8) (be32_to_cpu(sin6->sin6_flowinfo) >> 20); + comp_mask |= IB_SA_PATH_REC_FLOW_LABEL | + IB_SA_PATH_REC_TRAFFIC_CLASS; + } + id_priv->query_id = ib_sa_path_rec_get(&sa_client, id_priv->id.device, - id_priv->id.port_num, &path_rec, - IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | - IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | - IB_SA_PATH_REC_REVERSIBLE, - timeout_ms, GFP_KERNEL, - cma_query_handler, work, &id_priv->query); + id_priv->id.port_num, &path_rec, + comp_mask, timeout_ms, + GFP_KERNEL, cma_query_handler, + work, &id_priv->query); return (id_priv->query_id < 0) ? id_priv->query_id : 0; } diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h index 2d6a770..010f876 100644 --- a/include/rdma/rdma_cm.h +++ b/include/rdma/rdma_cm.h @@ -314,4 +314,18 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr, */ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr); +/** + * rdma_set_service_type - Set the type of service associated with a + * connection identifier. + * @id: Communication identifier to associated with service type. + * @tos: Type of service. + * + * The type of service is interpretted as a differentiated service + * field (RFC 2474). The service type should be specified before + * performing route resolution, as existing communication on the + * connection identifier may be unaffected. The type of service + * requested may not be supported by the network to all destinations. + */ +void rdma_set_service_type(struct rdma_cm_id *id, int tos); + #endif /* RDMA_CM_H */ From sean.hefty at intel.com Mon Aug 6 17:18:40 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 17:18:40 -0700 Subject: [ofa-general] [PATCH 4/4] rdma/ucm: export setting service type to user space In-Reply-To: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> Message-ID: <000301c7d888$81646260$9c98070a@amr.corp.intel.com> Export the ability to set the type of service to user space. Model after setsockopt. Signed-off-by: Sean Hefty --- A related set of changes to the librdmacm is also available. drivers/infiniband/core/ucma.c | 74 +++++++++++++++++++++++++++++++++++++++- include/rdma/rdma_user_cm.h | 18 ++++++++++ 2 files changed, 91 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c index 53b4c94..90d675a 100644 --- a/drivers/infiniband/core/ucma.c +++ b/drivers/infiniband/core/ucma.c @@ -792,6 +792,78 @@ out: return ret; } +static int ucma_set_option_id(struct ucma_context *ctx, int optname, + void *optval, size_t optlen) +{ + int ret = 0; + + switch (optname) { + case RDMA_OPTION_ID_TOS: + if (optlen != sizeof(u8)) { + ret = -EINVAL; + break; + } + rdma_set_service_type(ctx->cm_id, *((u8 *) optval)); + break; + default: + ret = -ENOSYS; + } + + return ret; +} + +static int ucma_set_option_level(struct ucma_context *ctx, int level, + int optname, void *optval, size_t optlen) +{ + int ret; + + switch (level) { + case RDMA_OPTION_ID: + ret = ucma_set_option_id(ctx, optname, optval, optlen); + break; + default: + ret = -ENOSYS; + } + + return ret; +} + +static ssize_t ucma_set_option(struct ucma_file *file, const char __user *inbuf, + int in_len, int out_len) +{ + struct rdma_ucm_set_option cmd; + struct ucma_context *ctx; + void *optval; + int ret; + + if (copy_from_user(&cmd, inbuf, sizeof(cmd))) + return -EFAULT; + + ctx = ucma_get_ctx(file, cmd.id); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); + + optval = kmalloc(cmd.optlen, GFP_KERNEL); + if (!optval) { + ret = -ENOMEM; + goto out1; + } + + if (copy_from_user(optval, (void __user *) (unsigned long) cmd.optval, + cmd.optlen)) { + ret = -EFAULT; + goto out2; + } + + ret = ucma_set_option_level(ctx, cmd.level, cmd.optname, optval, + cmd.optlen); +out2: + kfree(optval); +out1: + ucma_put_ctx(ctx); + return ret; +} + static ssize_t ucma_notify(struct ucma_file *file, const char __user *inbuf, int in_len, int out_len) { @@ -936,7 +1008,7 @@ static ssize_t (*ucma_cmd_table[])(struct ucma_file *file, [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr, [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event, [RDMA_USER_CM_CMD_GET_OPTION] = NULL, - [RDMA_USER_CM_CMD_SET_OPTION] = NULL, + [RDMA_USER_CM_CMD_SET_OPTION] = ucma_set_option, [RDMA_USER_CM_CMD_NOTIFY] = ucma_notify, [RDMA_USER_CM_CMD_JOIN_MCAST] = ucma_join_multicast, [RDMA_USER_CM_CMD_LEAVE_MCAST] = ucma_leave_multicast, diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h index f632b0c..9749c1b 100644 --- a/include/rdma/rdma_user_cm.h +++ b/include/rdma/rdma_user_cm.h @@ -212,4 +212,22 @@ struct rdma_ucm_event_resp { } param; }; +/* Option levels */ +enum { + RDMA_OPTION_ID = 0 +}; + +/* Option details */ +enum { + RDMA_OPTION_ID_TOS = 0 +}; + +struct rdma_ucm_set_option { + __u64 optval; + __u32 id; + __u32 level; + __u32 optname; + __u32 optlen; +}; + #endif /* RDMA_USER_CM_H */ From sean.hefty at intel.com Mon Aug 6 17:22:31 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 17:22:31 -0700 Subject: [ofa-general] [PATCH 3/4 v2] rdma/cm: add ability to specify type of service In-Reply-To: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> Message-ID: <000401c7d889$0b98c930$9c98070a@amr.corp.intel.com> Provide support to specify a type of service for a communication identifier. A new function call is used when dealing with IPv4 addresses. For IPv6 addresses, the ToS is specified through the traffic class and flow label fields in the sockaddr_in6 structure. Signed-off-by: Sean Hefty --- Minor compile fix-up added. I incorrectly had the fix included with the 4th patch of this series, rather than with this one, where it belongs. I removed the mistake from the 4th patch before it was posted. drivers/infiniband/core/cma.c | 37 +++++++++++++++++++++++++++++++------ include/rdma/rdma_cm.h | 14 ++++++++++++++ 2 files changed, 45 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 32a0e66..216f5a8 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -138,6 +138,7 @@ struct rdma_id_private { u32 qkey; u32 qp_num; u8 srq; + u8 tos; }; struct cma_multicast { @@ -1474,6 +1475,15 @@ err: } EXPORT_SYMBOL(rdma_listen); +void rdma_set_service_type(struct rdma_cm_id *id, int tos) +{ + struct rdma_id_private *id_priv; + + id_priv = container_of(id, struct rdma_id_private, id); + id_priv->tos = (u8) tos; +} +EXPORT_SYMBOL(rdma_set_service_type); + static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec, void *context) { @@ -1500,6 +1510,8 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms, { struct rdma_dev_addr *addr = &id_priv->id.route.addr.dev_addr; struct ib_sa_path_rec path_rec; + ib_sa_comp_mask comp_mask; + struct sockaddr_in6 *sin6; memset(&path_rec, 0, sizeof path_rec); ib_addr_get_sgid(addr, &path_rec.sgid); @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms, path_rec.numb_path = 1; path_rec.reversible = 1; + comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | + IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | + IB_SA_PATH_REC_REVERSIBLE; + + if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) { + path_rec.qos_class = cpu_to_be16((u16) id_priv->tos); + comp_mask |= IB_SA_PATH_REC_QOS_CLASS; + } else { + sin6 = (struct sockaddr_in6 *) &id_priv->id.route.addr.src_addr; + path_rec.flow_label = sin6->sin6_flowinfo; + path_rec.traffic_class = (u8) (be32_to_cpu(sin6->sin6_flowinfo) >> 20); + comp_mask |= IB_SA_PATH_REC_FLOW_LABEL | + IB_SA_PATH_REC_TRAFFIC_CLASS; + } + id_priv->query_id = ib_sa_path_rec_get(&sa_client, id_priv->id.device, - id_priv->id.port_num, &path_rec, - IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | - IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | - IB_SA_PATH_REC_REVERSIBLE, - timeout_ms, GFP_KERNEL, - cma_query_handler, work, &id_priv->query); + id_priv->id.port_num, &path_rec, + comp_mask, timeout_ms, + GFP_KERNEL, cma_query_handler, + work, &id_priv->query); return (id_priv->query_id < 0) ? id_priv->query_id : 0; } diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h index 2d6a770..010f876 100644 --- a/include/rdma/rdma_cm.h +++ b/include/rdma/rdma_cm.h @@ -314,4 +314,18 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr, */ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr); +/** + * rdma_set_service_type - Set the type of service associated with a + * connection identifier. + * @id: Communication identifier to associated with service type. + * @tos: Type of service. + * + * The type of service is interpretted as a differentiated service + * field (RFC 2474). The service type should be specified before + * performing route resolution, as existing communication on the + * connection identifier may be unaffected. The type of service + * requested may not be supported by the network to all destinations. + */ +void rdma_set_service_type(struct rdma_cm_id *id, int tos); + #endif /* RDMA_CM_H */ From sashak at voltaire.com Mon Aug 6 17:13:08 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 07 Aug 2007 03:13:08 +0300 Subject: [ofa-general] Re: [PATCH] osm: default leaf vl_stall and head_of_queue_life counters In-Reply-To: <46B78D04.7020002@dev.mellanox.co.il> References: <46B78D04.7020002@dev.mellanox.co.il> Message-ID: <1186445588.7524.10.camel@localhost> Hi Yevgeny, On Tue, 2007-08-07 at 00:05 +0300, Yevgeny Kliteynik wrote: > Hi Sasha > > This patch changes the OSM_DEFAULT_LEAF_VL_STALL_COUNT and > OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE (matching them to the > default counters on the switch-to-switch ports), But why this should be equal to switch-to-switch port values? BTW HoQLife=0x12 (18) is >1 sec. Isn't it _huge_ for HoQLife? > in order > to deal with the casual PCI Express "hiccups". Could you provide more details about the problem? Sasha > Please apply it to ofed_1_2 and master. > > -- > Yevgeny > > Signed-off-by: Yevgeny Kliteynik > --- > osm/include/opensm/osm_base.h | 7 +++---- > 1 files changed, 3 insertions(+), 4 deletions(-) > > diff --git a/osm/include/opensm/osm_base.h b/osm/include/opensm/osm_base.h > index b38b511..0cb4651 100644 > --- a/osm/include/opensm/osm_base.h > +++ b/osm/include/opensm/osm_base.h > @@ -311,11 +311,11 @@ BEGIN_C_DECLS > * DESCRIPTION > * Sets the time a packet can live in the head of the VL Queue > * of a port that drives a CA port. > -* We use here the value of ~130usec > +* We use here the value of ~1sec > * > * SYNOPSIS > */ > -#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0xC > +#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0x12 > /***********/ > > /****d* OpenSM: Base/OSM_DEFAULT_VL_STALL_COUNT > @@ -341,11 +341,10 @@ BEGIN_C_DECLS > * puts the VL into stalled state. In stalled state, the port is supposed > * to drop everything for 8*(head of queue lifetime). This value is for > * switch ports driving a CA port. > -* We use the value of 1 here - so any drop due to HOQ means stalling the VL > * > * SYNOPSIS > */ > -#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x1 > +#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x7 > /***********/ > > /****d* OpenSM: Base/OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT From rdreier at cisco.com Mon Aug 6 17:25:42 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 06 Aug 2007 17:25:42 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <000101c7d887$2fbbcd50$9c98070a@amr.corp.intel.com> (Sean Hefty's message of "Mon, 6 Aug 2007 17:09:13 -0700") References: <000101c7d887$2fbbcd50$9c98070a@amr.corp.intel.com> Message-ID: > +#define IB_SA_PATH_REC_SERVICE_ID IB_SA_COMP_MASK( 0) \ > + IB_SA_COMP_MASK( 1) I don't see anything using IB_SA_PATH_REC_SERVICE_ID in your patches, and as far as I can tell if there were something it wouldn't compile with this definition. Am I misreading how this macro would expand, or should it really be something like this? +#define IB_SA_PATH_REC_SERVICE_ID (IB_SA_COMP_MASK( 0) |\ + IB_SA_COMP_MASK( 1)) From rdreier at cisco.com Mon Aug 6 17:33:15 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 06 Aug 2007 17:33:15 -0700 Subject: [ofa-general] Re: virtual memory usage on long lived apps In-Reply-To: <46B78081.3060305@opengridcomputing.com> (Steve Wise's message of "Mon, 06 Aug 2007 15:11:45 -0500") References: <46B78081.3060305@opengridcomputing.com> Message-ID: > Running this server with clients repeatedly connecting, running > rpings, and disconnecting, I notice that the virtual memory consumed > by the process is ever increasing. The shared/and reserved values > don't increase, but the virtual does. IE: If you watch this process > via 'top' you'll see the VIRT value increasing steadily over > time/connections, and the RES/SHR values stay steady. I've seen it > over 1GB of virtual memory used! > Q: Have you seen this before? Its like the OS never reuses mappings > that were setup for memory registrations. Is that normal? Or should > the virtual size shown in 'top' be reduced whenever a buffer is > unregistered and freed? There's a definite possibility the code is > leaking memory, but I'm wondering if this is perhaps a normal anomaly > with mapping memory in linux... I've not really tested an app like that, but I would expect that freeing the memory should prevent your virtual space from growing. I would recheck your code and maybe add some tracing to make sure that everything really does get freed -- or try the libibverbs valgrind support ;) If you send me your code I may be able to find some time to investigate too. From sashak at voltaire.com Mon Aug 6 17:49:17 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 7 Aug 2007 03:49:17 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/scripts: Fix Bug 239 Error Reporting In-Reply-To: <46B6689A.3030902@systemfabricworks.com> References: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> <20070805192508.GF27000@sashak.voltaire.com> <46B6689A.3030902@systemfabricworks.com> Message-ID: <20070807004917.GA32692@sashak.voltaire.com> On 19:17 Sun 05 Aug , David McMillen wrote: > > Upon failure, sending the full output of the eval'd expression to standard > output is not very useful, especially since the normal output of the various > scripts is to suppress most of that. However, embedded within the text > there are likely to be error messages that explain what went wrong, which is > why there are new parts to the awk script catching /^ib/ /ibpanic:/ > /ibwarn:/ /iberror:/ and passing them through.. Got it. Thanks. Sasha From stanleysufficool at roadrunner.com Mon Aug 6 20:29:56 2007 From: stanleysufficool at roadrunner.com (Stanley Sufficool) Date: Mon, 06 Aug 2007 20:29:56 -0700 Subject: [ofa-general] OpensSM and Multiple Ports In-Reply-To: References: <1186194845.6031.28.camel@gentoo-linux.localdomain> <1186245995.6031.79.camel@gentoo-linux.localdomain> Message-ID: <1186457397.5593.9.camel@gentoo-linux.localdomain> Seems that using the GUID instead of PORT_NUM didn't matter. After restarting the service, I was able to run osmtest and found all adapter hardware but had a timeout when it tried to identify the Cisco SFS 7000 switch. Only then, I found that the Cisco SFS 7000P has a built in IB subnet manager. What a waste of time. Thank you though. Now on to the WinIB forum to figure out why they assign fake Ethernet MAC Addresses differently than OFED. On Sun, 2007-08-05 at 11:07 -0400, Hal Rosenstock wrote: > On 8/4/07, Stanley Sufficool wrote: > > On Sat, 2007-08-04 at 06:55 -0400, Hal Rosenstock wrote: > > > > On 8/3/07, Stanley Sufficool wrote: > > I am > > trying to get opensm to manage two separate subnets, one per IB port on > > an > > Infinihost II PCIe HCA. I can specify PORT=1 and opensmd will start > > > > without error. On the second sm management node, I start with PORT=2 in > > > > opensm.conf and opensm fails with a timeout error. > > Not sure what PORT is > > but you can start OpenSM on a specific port by > using the -g (or -guid) > > option. Something like: > opensm -g=0x12345678 where 0x12345678 is the port > > (not node) GUID > > > > There are 2 options for specifying the port in opensm.conf, one is the port > > GUID and one is the port number (PORT=1) . OpenSM seems to correctly derive > > the port GUID from the port number, > > Looks to me like PORT_NUM (not PORT) is only used in a RedHat init > script. Vlad can better comment on both opensm.conf and the init > scripts in OFED for OpenSM. > > > however I will try using the GUID > > instead. > > Did this work ? > > > I am using Gentoo Linux with the mlx HCA and MAD, UMAD, etc... built > > into > > kernel 2.6.20. > > > > My goal is to get ipoib working. IPoIB is being > > flakey with duplicated > > ethernet MAC addresses ( MAC: 02:00:00:00:01 & :02 > > ) . This may be resolved > > when opensm is working? > > IPoIB won't work until > > the ports are brought up by the SM. Not sure > about the duplicated MACs. I > > don't understand this. > > > > > Ethernet MAC addresses as shown in Windows ipconfig for the IBoIP > > interfaces. They are duplicated because the LIDs are not being renumbered by > > the manager. > > I don't know how Windows derives its fake ethernet MACs for IPoIB but > if it uses LIDs, this makes sense. > > -- Hal > > > I am also hoping to get SRP with Windows initiators and a Linux > > target using > > the SRPT kernel module. But failing that, I will regress to > > iSCSI with hopes > > of a future iSER target/initiator support. > > > > _______________________________________________ > > general > > mailing list > > general at lists.openfabrics.org > > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit > > > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general > > mailing > > list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To > > unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Mon Aug 6 21:03:44 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 7 Aug 2007 07:03:44 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-07:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Jul_31_15:39:27_2007 [ab4d5ad9161d9d48bc0c9fe91cac5a8cf221a377] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=468 Fail=52 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo From pyotr.daniell at olympiccarhire.com Mon Aug 6 16:02:17 2007 From: pyotr.daniell at olympiccarhire.com (Laverne Cunningham) Date: Tue, 7 Aug 2007 05:02:17 +0600 Subject: [ofa-general] Magic stick Message-ID: <639032962.12846223033838@olympiccarhire.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: detka00.gif Type: image/gif Size: 14043 bytes Desc: not available URL: From sean.hefty at intel.com Mon Aug 6 22:07:53 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 6 Aug 2007 22:07:53 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: Message-ID: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> > > +#define IB_SA_PATH_REC_SERVICE_ID IB_SA_COMP_MASK( 0) \ > > + IB_SA_COMP_MASK( 1) > >I don't see anything using IB_SA_PATH_REC_SERVICE_ID in your patches, >and as far as I can tell if there were something it wouldn't compile >with this definition. Am I misreading how this macro would expand, or >should it really be something like this? > >+#define IB_SA_PATH_REC_SERVICE_ID (IB_SA_COMP_MASK( 0) |\ >+ IB_SA_COMP_MASK( 1)) Your fix looks correct. Nothing yet is using the PR service ID fields. I included it (poorly) for completeness, but it may have use for SRP or iSER. I'm not sure on the changes for those. - Sean From mst at dev.mellanox.co.il Mon Aug 6 22:34:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 08:34:28 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <20070806141923.GA4801@mellanox.co.il> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <20070806141923.GA4801@mellanox.co.il> Message-ID: <20070807053428.GA2074@mellanox.co.il> Hmm, I thought about it some more. kmem_cache struct is not exported on recent kernels, so this might br hard to do. So I think the patch is probably the right approach, after all. Quoting Michael S. Tsirkin : Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 Let's not do it this way. I think the right thing is to implement kmem_cache_zalloc by means of kmem_cache_allocand memset in kernel_addons. Quoting Hoang-Nam Nguyen : Subject: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 Hello Michael and Vladimir! This patch below adds a backport patch for ehca to the dirs 2.6.16, 2.6.16_sles10 and 2.6.16_sles10_sp1 underneath kernel_patches/backport of ofed-1.2.c source tree. Thanks! Nam backport kmem_cache_zalloc() to 2.6.10, 2.6.10_sles10 and 2.6.10_sles10_sp1 Signed-off-by: Hoang-Nam Nguyen --- 2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch | 97 +++++++++++++++ 3 files changed, 291 insertions(+) diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:53:59.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:20.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 1970-01-01 01:00:00.000000000 +0100 +++ ofa_1_2_c_kernel-20070804-0200/kernel_patches/backport/2.6.16_sles10_sp1/ehca_kmem_cache_zalloc_to_2_6_16.patch 2007-08-06 00:56:33.000000000 +0200 @@ -0,0 +1,97 @@ +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_cq.c 2007-08-06 00:41:50.000000000 +0200 +@@ -134,13 +134,14 @@ struct ib_cq *ehca_create_cq(struct ib_d + if (cqe >= 0xFFFFFFFF - 64 - additional_cqe) + return ERR_PTR(-EINVAL); + +- my_cq = kmem_cache_zalloc(cq_cache, GFP_KERNEL); ++ my_cq = kmem_cache_alloc(cq_cache, GFP_KERNEL); + if (!my_cq) { + ehca_err(device, "Out of memory for ehca_cq struct device=%p", + device); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_cq, 0, sizeof(*my_cq)); + memset(¶m, 0, sizeof(struct ehca_alloc_cq_parms)); + + spin_lock_init(&my_cq->spinlock); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_main.c 2007-08-06 00:40:58.000000000 +0200 +@@ -113,9 +113,11 @@ static struct kmem_cache *ctblk_cache = + + void *ehca_alloc_fw_ctrlblock(gfp_t flags) + { +- void *ret = kmem_cache_zalloc(ctblk_cache, flags); ++ void *ret = kmem_cache_alloc(ctblk_cache, flags); + if (!ret) + ehca_gen_err("Out of memory for ctblk"); ++ else ++ memset(ret, 0, EHCA_PAGESIZE); + return ret; + } + +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_mrmw.c 2007-08-06 00:39:30.000000000 +0200 +@@ -55,8 +55,9 @@ static struct ehca_mr *ehca_mr_new(void) + { + struct ehca_mr *me; + +- me = kmem_cache_zalloc(mr_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mr_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mrlock); + } else + ehca_gen_err("alloc failed"); +@@ -73,8 +74,9 @@ static struct ehca_mw *ehca_mw_new(void) + { + struct ehca_mw *me; + +- me = kmem_cache_zalloc(mw_cache, GFP_KERNEL); ++ me = kmem_cache_alloc(mw_cache, GFP_KERNEL); + if (me) { ++ memset(me, 0, sizeof(*me)); + spin_lock_init(&me->mwlock); + } else + ehca_gen_err("alloc failed"); +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_pd.c 2007-08-06 00:38:14.000000000 +0200 +@@ -50,13 +50,14 @@ struct ib_pd *ehca_alloc_pd(struct ib_de + { + struct ehca_pd *pd; + +- pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL); ++ pd = kmem_cache_alloc(pd_cache, GFP_KERNEL); + if (!pd) { + ehca_err(device, "device=%p context=%p out of memory", + device, context); + return ERR_PTR(-ENOMEM); + } + ++ memset(pd, 0, sizeof(*pd)); + pd->ownpid = current->tgid; + + /* +diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c +--- ofa_1_2_c_kernel-20070804-0200_orig/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-04 11:00:05.000000000 +0200 ++++ ofa_1_2_c_kernel-20070804-0200/drivers/infiniband/hw/ehca/ehca_qp.c 2007-08-06 00:37:39.000000000 +0200 +@@ -450,12 +450,13 @@ struct ib_qp *ehca_create_qp(struct ib_p + if (pd->uobject && udata) + context = pd->uobject->context; + +- my_qp = kmem_cache_zalloc(qp_cache, GFP_KERNEL); ++ my_qp = kmem_cache_alloc(qp_cache, GFP_KERNEL); + if (!my_qp) { + ehca_err(pd->device, "pd=%p not enough memory to alloc qp", pd); + return ERR_PTR(-ENOMEM); + } + ++ memset(my_qp, 0, sizeof(*my_qp)); + memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms)); + spin_lock_init(&my_qp->spinlock_s); + spin_lock_init(&my_qp->spinlock_r); -- MST -- MST From hnguyen at linux.vnet.ibm.com Tue Aug 7 00:13:35 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Tue, 7 Aug 2007 09:13:35 +0200 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <20070807053428.GA2074@mellanox.co.il> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <20070806141923.GA4801@mellanox.co.il> <20070807053428.GA2074@mellanox.co.il> Message-ID: <200708070913.36106.hnguyen@linux.vnet.ibm.com> On Tuesday 07 August 2007 07:34, Michael S. Tsirkin wrote: > Hmm, I thought about it some more. > kmem_cache struct is not exported on recent kernels, > so this might br hard to do. I looked at slab.h in various kernel and see this function extern unsigned int kmem_cache_size(kmem_cache_t *); which returns buffer_size that is set by kmem_cache_create() to size. So the backport will look like: static inline void *kmem_cache_zalloc(kmem_cache_t *cache, gfp_t flags) { void *ret = kmem_cache_alloc(cache, flags); if (ret) memset(ret, 0, kmem_cache_size(cache)); return ret; } Would you agree? Do I oversee something else? Nam From mst at dev.mellanox.co.il Tue Aug 7 00:09:05 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 10:09:05 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708070913.36106.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <20070806141923.GA4801@mellanox.co.il> <20070807053428.GA2074@mellanox.co.il> <200708070913.36106.hnguyen@linux.vnet.ibm.com> Message-ID: <20070807070904.GA3868@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > On Tuesday 07 August 2007 07:34, Michael S. Tsirkin wrote: > > Hmm, I thought about it some more. > > kmem_cache struct is not exported on recent kernels, > > so this might br hard to do. > > I looked at slab.h in various kernel and see this function > extern unsigned int kmem_cache_size(kmem_cache_t *); > which returns buffer_size that is set by kmem_cache_create() to size. > So the backport will look like: > static inline > void *kmem_cache_zalloc(kmem_cache_t *cache, gfp_t flags) > { > void *ret = kmem_cache_alloc(cache, flags); > if (ret) > memset(ret, 0, kmem_cache_size(cache)); > return ret; > } > Would you agree? Do I oversee something else? Good idea. Go for it, with one caveat: please use struct kmem_cache rather than the deprecated kmem_cache_t. Thus: static inline void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { void *ret = kmem_cache_alloc(cache, flags); if (ret) memset(ret, 0, kmem_cache_size(cache)); return ret; } -- MST From amitk at mellanox.co.il Tue Aug 7 00:24:09 2007 From: amitk at mellanox.co.il (Amit Krig) Date: Tue, 7 Aug 2007 10:24:09 +0300 Subject: [ofa-general] RE: [PATCH] osm: default leaf vl_stall and head_of_queue_lifecounters In-Reply-To: <1186445588.7524.10.camel@localhost> References: <46B78D04.7020002@dev.mellanox.co.il> <1186445588.7524.10.camel@localhost> Message-ID: <6C2C79E72C305246B504CBA17B5500C90208EE72@mtlexch01.mtl.com> Hi Sasha In the current switch configurations if you have a single drop (By HOQ), the line will be flashed for 128 Mili seconds, I think that this penalty for ~8mili hiccup (as the HOQ can count ~50% less then configured) is not a good default, Amit -----Original Message----- From: Sasha Khapyorsky [mailto:sashak at voltaire.com] Sent: Tuesday, August 07, 2007 3:13 AM To: kliteyn at dev.mellanox.co.il Cc: OpenIB; Amit Krig Subject: Re: [PATCH] osm: default leaf vl_stall and head_of_queue_lifecounters Hi Yevgeny, On Tue, 2007-08-07 at 00:05 +0300, Yevgeny Kliteynik wrote: > Hi Sasha > > This patch changes the OSM_DEFAULT_LEAF_VL_STALL_COUNT and > OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE (matching them to the default > counters on the switch-to-switch ports), But why this should be equal to switch-to-switch port values? BTW HoQLife=0x12 (18) is >1 sec. Isn't it _huge_ for HoQLife? > in order > to deal with the casual PCI Express "hiccups". Could you provide more details about the problem? Sasha > Please apply it to ofed_1_2 and master. > > -- > Yevgeny > > Signed-off-by: Yevgeny Kliteynik > --- > osm/include/opensm/osm_base.h | 7 +++---- > 1 files changed, 3 insertions(+), 4 deletions(-) > > diff --git a/osm/include/opensm/osm_base.h > b/osm/include/opensm/osm_base.h index b38b511..0cb4651 100644 > --- a/osm/include/opensm/osm_base.h > +++ b/osm/include/opensm/osm_base.h > @@ -311,11 +311,11 @@ BEGIN_C_DECLS > * DESCRIPTION > * Sets the time a packet can live in the head of the VL Queue > * of a port that drives a CA port. > -* We use here the value of ~130usec > +* We use here the value of ~1sec > * > * SYNOPSIS > */ > -#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0xC > +#define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0x12 > /***********/ > > /****d* OpenSM: Base/OSM_DEFAULT_VL_STALL_COUNT @@ -341,11 +341,10 @@ > BEGIN_C_DECLS > * puts the VL into stalled state. In stalled state, the port is > supposed > * to drop everything for 8*(head of queue lifetime). This value is > for > * switch ports driving a CA port. > -* We use the value of 1 here - so any drop due to HOQ means stalling > the VL > * > * SYNOPSIS > */ > -#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x1 > +#define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x7 > /***********/ > > /****d* OpenSM: Base/OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT From alexm at voltaire.com Tue Aug 7 00:27:02 2007 From: alexm at voltaire.com (Alexander Mikheev) Date: Tue, 7 Aug 2007 10:27:02 +0300 Subject: [ofa-general] implicit address binding by rdma_resolve_addr In-Reply-To: <46B75709.5080507@ichips.intel.com> References: <46B75709.5080507@ichips.intel.com> Message-ID: <39C75744D164D948A170E9792AF8E7CA0A0109@exil.voltaire.com> It is a problem when you want implement something like the UDP server that handles all request on the single socket: bind(fd, ...); listen(fd); while(1) { read_request(fd, ...); do_work(); write_responce(fd, ...); } Correct me if I am wrong, but implementing this scenario for UD will look like this: /* accept handling not shown... */ rdma_bind_addr(id, ...); rdma_listen(id) while(1) { read_request(); do_work(); if(new_client) { /* check if we already connected to client */ /* following is not going to work because local address is already bound */ rdma_resolve_addr(my_local_already_bound_addr, client_addr); do_connect(); } write_responce(); } If rdma_bind is done for the specific local ip and port, then I would like to be able to resolve address from the specific ip/port to the client. -----Original Message----- From: Sean Hefty [mailto:mshefty at ichips.intel.com] Sent: Monday, August 06, 2007 8:15 PM To: Or Gerlitz Cc: Sean Hefty; Alexander Mikheev; Alex Rosenbaum; general at lists.openfabrics.org Subject: Re: [ofa-general] implicit address binding by rdma_resolve_addr > We have noticed that when rdma_resolve_addr is provided with src > address, the rdma cm does an implicit --address-- (=ip/port) binding, > which for example has the side effect of disallowing two processes to > call rdma_resolve_addr() with the same src port. > > Does this implicit binding mimics the network stack behavior > under similar flow when socket are used? Specifying a source address with rdma_resolve_addr is similar to calling socket bind. I can't think of a reason why an app would want to specify a source port when calling rdma_resolve_addr. Specifying a source address (wild card port) allows binding to a specific RDMA device. - Sean From diarmaid.daniels at postmaster.co.uk Tue Aug 7 01:04:54 2007 From: diarmaid.daniels at postmaster.co.uk (Warren Jean) Date: Tue, 7 Aug 2007 08:04:54 +0000 Subject: [ofa-general] You can... Message-ID: <01c7d8c9$a381b000$7b5c23d9@diarmaid.daniels> -------------- next part -------------- A non-text attachment was scrubbed... Name: detka00.gif Type: image/gif Size: 14390 bytes Desc: not available URL: From steffen.persvold at scali.com Tue Aug 7 01:20:04 2007 From: steffen.persvold at scali.com (Steffen Persvold) Date: Tue, 7 Aug 2007 04:20:04 -0400 Subject: [ofa-general] Version number on ibverbs headers Message-ID: Hi, Would it be possible to add a version number define to verbs.h to reflect the API version ? For example : #define IBVERBS_VERSION_MAJOR 1 #define IBVERBS_VERSION_MINOR 1 ? Makes it easier for dynamic code to compile and work on different ibverbs versions (1.0 and 1.1 for example). Cheers, Steffen Persvold Software Development Manager tel. +47 924 84 511 http://www.scali.com/ Higher Performance Computing From dotanb at dev.mellanox.co.il Tue Aug 7 01:18:52 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 7 Aug 2007 11:18:52 +0300 Subject: [ofa-general] [PATCH] mlx4/net: Use the enumerated value of GO_BIT_TIMEOUT for the Command Interface timeout Message-ID: <200708071118.53055.dotanb@dev.mellanox.co.il> Use the enumerated value of GO_BIT_TIMEOUT for the Command Interface timeout (just like being done in the mthca driver). Signed-off-by: Dotan Barak --- diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c index a9f3175..4b99e9b 100644 --- a/drivers/net/mlx4/cmd.c +++ b/drivers/net/mlx4/cmd.c @@ -95,7 +95,7 @@ enum { }; enum { - GO_BIT_TIMEOUT = 10000 + GO_BIT_TIMEOUT = HZ * 10 }; struct mlx4_cmd_context { @@ -155,7 +155,7 @@ static int mlx4_cmd_post(struct mlx4_dev *dev, u64 in_param, u64 out_param, end = jiffies; if (event) - end += HZ * 10; + end += GO_BIT_TIMEOUT; while (cmd_pending(dev)) { if (time_after_eq(jiffies, end)) From vlad at lists.openfabrics.org Tue Aug 7 01:40:41 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 7 Aug 2007 01:40:41 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070807-0100 daily build status Message-ID: <20070807084041.75E5FE60391@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.17 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.18 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Failed: From ogerlitz at voltaire.com Tue Aug 7 01:42:46 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 11:42:46 +0300 Subject: [ofa-general] QoS RFC In-Reply-To: <46B751BA.1050800@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46AF6281.10709@ichips.intel.com> <46B58E8E.1040001@voltaire.com> <46B751BA.1050800@ichips.intel.com> Message-ID: <46B83086.6090202@voltaire.com> Sean Hefty wrote: >> From what I understand while reading your proposal, is that it is >> quite different then what what suggested in the original RFC. I don't >> think it makes sense to implement the host side of this before there's >> agreement on the over-all solution namely how the host side >> design/code plugs to the management scheme at the SM side. > I don't believe that my proposal and the SA side proposal are > incompatible. We should be able to design the host side stack somewhat > independent from a specific SA implementation. It needs to be to > support alternative SA implementations. Say I am with you on the argument that the host side implementation need not be dependent on a specific SA implementation. However, for validation purposes it should be testable with some SA... But even with that, the host side architecture implemented by your patches to IPoIB and the RDMA CM is --different-- from the one proposed in the RFC. Specifically the point I see we need to have a debate is your approach of --not-- using the Service ID to derive QoS (ie FL/TC and SL) I find it more constructive to have a discussion based on a short host side architecture description that covers at least QoS for IPoIB and RDMA CM based ULPs, rather then based on series of patches. Or. From ogerlitz at voltaire.com Tue Aug 7 02:00:10 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 12:00:10 +0300 Subject: [ofa-general] QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46B7493E.7050400@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58A8B.5020609@voltaire.com> <46B7493E.7050400@ichips.intel.com> Message-ID: <46B8349A.2090400@voltaire.com> Sean Hefty wrote: >> I wonder how you think the suggested scheme for IPoIB plugs into the >> existing QoS flow at the Linux network stack. Looking on the man pages >> of tcp(7) and ip(7) I see that there's a SOL_SOCKET level option of >> SO_PRIORITY and SOL_IP level option of IP_TOS. > > I was looking at IB routing, not IP routing. IPv6 uses flow info, > which, from what I can tell, is traffic class and flow label. Indeed, that the IP_TOS option is related to IP routing and the SO_PRIORITY option might be more related to internal net stack QoS such as Priority Queuing, where both are beyond the scope of IPoIB. However, As I reported below, for IPoEth the equivalent of IPoIB such as the 802.1q and e1000 drivers are using skb->priority to derive the "Ethernet QoS" which is the equivalent the SL in the LRH etc. >> Looking lower in the stack, on the IPoEth scheme where the 802.1q is >> used for the VLAN header generation, the code seems to generate the >> p_bits field of the header using skb->priority (see the call to >> vlan_dev_get_egress_qos_mask() from vlan_dev_hard_header() at >> net/8021q/vlan_dev.c) > > Someone with more knowledge of IPoIB than me needs to provide input > here. I thought the use of different PKeys was similar to the selection > of a VLAN. Yes, use of pkeys with IPoIB is a similar concept for use of VLANs with IPoEth, the point here, is that Ethernet QoS comes into play only when VLANs are used, since the standard MAC header does not specify any QoS. For IPoIB (no GRH case, omitting IB transport header) you would have < LRH | IPoIB header | IP header | etc > and for IPoEth using 802.1q VLANs < MAC header | 802.1q vlan tag | IP header | etc > the vlan tag has 12 bits of "vlan id" (the equiv of pkey), 3 bits of priority (the equiv of the SL) and one more bit i don't remember. Or. From vlad at lists.openfabrics.org Tue Aug 7 02:05:14 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 7 Aug 2007 02:05:14 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070807-0200 daily build status Message-ID: <20070807090514.B4DBAE6083E@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Failed: Build failed on i686 with 2.6.15-23-server Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on i686 with linux-2.6.12 Build failed on i686 with linux-2.6.17 Build failed on i686 with linux-2.6.14 Build failed on i686 with linux-2.6.15 Build failed on i686 with linux-2.6.19 Build failed on i686 with linux-2.6.16 Build failed on i686 with linux-2.6.21.1 Build failed on i686 with linux-2.6.22 Build failed on i686 with linux-2.6.13 Build failed on i686 with linux-2.6.18 Build failed on x86_64 with linux-2.6.18-1.2798.fc6 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.13 Log: Build failed on powerpc with linux-2.6.16 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.15 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.21.1 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.19 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.14 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.21.1 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.16.43-0.3-smp Log: Build failed on x86_64 with linux-2.6.14 Log: Build failed on x86_64 with linux-2.6.16.21-0.8-smp Log: Build failed on ppc64 with linux-2.6.15 Log: Build failed on x86_64 with linux-2.6.13 Log: Build failed on x86_64 with linux-2.6.18 Log: Build failed on ppc64 with linux-2.6.18 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) Build failed on powerpc with linux-2.6.12 Log: Build failed on x86_64 with linux-2.6.18-8.el5 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.17 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.14 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-22.ELsmp Log: Build failed on ia64 with linux-2.6.16 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.18-8.el5 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.22 Log: Build failed on x86_64 with linux-2.6.9-34.ELsmp Log: Build failed on ia64 with linux-2.6.18 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.20 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.12 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.17 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.19 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.12 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.13 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.16.21-0.8-default Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.15 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.22 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.14 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.15 Log: Build failed on x86_64 with linux-2.6.12 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ia64 with linux-2.6.13 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.17 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.19 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on ppc64 with linux-2.6.16 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.16 Log: patching file drivers/infiniband/ulp/ipoib/ipoib_cm.c patching file drivers/infiniband/ulp/sdp/sdp_cma.c patching file drivers/infiniband/ulp/srp/ib_srp.c patching file net/rds/ib_cm.c patching file drivers/infiniband/core/uverbs_cmd.c patching file drivers/infiniband/ulp/vnic/vnic_ib.c patching file drivers/infiniband/ulp/iser/iser_verbs.c Patch x_0010_int_mod.patch does not apply (enforce with -f) ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Tue Aug 7 02:05:14 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 12:05:14 +0300 Subject: [ofa-general] QoS RFC / how its linked to the networking stack QoS? In-Reply-To: <46B767BC.2040207@ichips.intel.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46B58A8B.5020609@voltaire.com> <46B767BC.2040207@ichips.intel.com> Message-ID: <46B835CA.6030003@voltaire.com> Sean Hefty wrote: > Giving this a little more thought, IPoIB uses a single path to each IP > destination. You're asking about per socket ToS, which requires the > ability to handle multiple paths to each remote IP address. Both the > original RFC and my proposal for the host stack limit all IPoIB traffic > to a single QoS level. > Is there a real need to expand IPoIB ToS capabilities? I guess that at the long run, maybe yes. As this point an architecture that derives the QoS for IPoIB based on the partition this IPoIB device lives in make sense. This means that the first implementation might ignore skb->priority and the users would be instructed to use different partitioning for different QoS levels. Again, this has to be clearly stated in text before becoming a code (in this case, before becoming a "non code" b/c you are going to not implement the linkage..) Or. From ogerlitz at voltaire.com Tue Aug 7 02:17:02 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 12:17:02 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service In-Reply-To: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> Message-ID: <46B8388E.1090308@voltaire.com> Sean Hefty wrote: > Provide support to specify a type of service for a communication > identifier. A new function call is used when dealing with IPv4 > addresses. For IPv6 addresses, the ToS is specified through the > traffic class and flow label fields in the sockaddr_in6 structure. > @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int > + comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | > + IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | > + IB_SA_PATH_REC_REVERSIBLE; > + > + if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) { > + path_rec.qos_class = cpu_to_be16((u16) id_priv->tos); > + comp_mask |= IB_SA_PATH_REC_QOS_CLASS; > + } else { > + sin6 = (struct sockaddr_in6 *) id_priv->id.route.addr.src_addr; > + path_rec.flow_label = sin6->sin6_flowinfo; > + path_rec.traffic_class = (u8) (be32_to_cpu(sin6->sin6_flowinfo) >> 20); > + comp_mask |= IB_SA_PATH_REC_FLOW_LABEL | > + IB_SA_PATH_REC_TRAFFIC_CLASS; > + } I think the correct way here, is a SID --> QoS mapping. This allows you to manage and configure QoS in --one-- place, namely the SM/SA and zero effort host side deployment. For example clustered database has IB subnet both for the inter RAC IPC and for the RAC nodes storage access, say IPC is RDS and storage is iSER (a similar example from HPC, IPC is MPI and I/O is Lustre) So you would need to configure an SID to SL mapping at the SA for each service and you are --done--. Taking this to cross IB subnets, the SID based mapping will also provide the TC and FL for the GRH. With the above implementation you would have to change each single ULP to have it use QoS and you have to have a configuration file also at the host side (saying what tos to set through the rdma cm) and not only in the SM. Or. From ogerlitz at voltaire.com Tue Aug 7 02:18:14 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 12:18:14 +0300 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> Message-ID: <46B838D6.6000501@voltaire.com> Sean Hefty wrote: > Your fix looks correct. Nothing yet is using the PR service ID fields. I > included it (poorly) for completeness, but it may have use for SRP or iSER. I'm > not sure on the changes for those. iser is rdma cm based, the only way for iser to use the sid is if the rdma cm allows it to do so, with this patch series its not possible. Or. From mst at dev.mellanox.co.il Tue Aug 7 02:18:16 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 12:18:16 +0300 Subject: [ofa-general] Re: Version number on ibverbs headers In-Reply-To: References: Message-ID: <20070807091816.GB7656@mellanox.co.il> > Quoting Steffen Persvold : > Subject: Version number on ibverbs headers > > Hi, > > Would it be possible to add a version number define to verbs.h to > reflect the API version ? > > For example : > > #define IBVERBS_VERSION_MAJOR 1 > #define IBVERBS_VERSION_MINOR 1 > > ? > > Makes it easier for dynamic code to compile and work on different > ibverbs versions (1.0 and 1.1 for example). Wouldn't configure checks for specific API's be better? -- MST From steffen.persvold at scali.com Tue Aug 7 02:24:20 2007 From: steffen.persvold at scali.com (Steffen Persvold) Date: Tue, 7 Aug 2007 05:24:20 -0400 Subject: [ofa-general] RE: Version number on ibverbs headers In-Reply-To: <20070807091816.GB7656@mellanox.co.il> References: <20070807091816.GB7656@mellanox.co.il> Message-ID: > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Tuesday, August 07, 2007 5:18 AM > To: Steffen Persvold > Cc: general at lists.openfabrics.org; ewg at lists.openfabrics.org > Subject: Re: Version number on ibverbs headers > > > Quoting Steffen Persvold : > > Subject: Version number on ibverbs headers > > > > Hi, > > > > Would it be possible to add a version number define to verbs.h to > > reflect the API version ? > > > > For example : > > > > #define IBVERBS_VERSION_MAJOR 1 > > #define IBVERBS_VERSION_MINOR 1 > > > > ? > > > > Makes it easier for dynamic code to compile and work on different > > ibverbs versions (1.0 and 1.1 for example). > > Wouldn't configure checks for specific API's be better? > Not everyone uses autoconf/automake... But, I don't see any reason why you can't have both (configure checks, *and* header info). Cheers, Steffen Persvold Software Development Manager tel. +47 924 84 511 http://www.scali.com/ Higher Performance Computing From ogerlitz at voltaire.com Tue Aug 7 03:04:12 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 07 Aug 2007 13:04:12 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> Message-ID: <46B8439C.6010303@voltaire.com> Sean Hefty wrote: > To support QoS within and between subnets, modify IPoIB to request > specific TClass and FlowLabel values with path record queries, using > the values associated with the IPoIB broadcast group. > > This allows QoS to be defined for a given IPoIB broadcast group. > > Signed-off-by: Sean Hefty > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c > index 894b1dc..e22680f 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > @@ -468,9 +468,11 @@ static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid) > INIT_LIST_HEAD(&path->neigh_list); > > memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid)); > - path->pathrec.sgid = priv->local_gid; > - path->pathrec.pkey = cpu_to_be16(priv->pkey); > - path->pathrec.numb_path = 1; > + path->pathrec.sgid = priv->local_gid; > + path->pathrec.pkey = cpu_to_be16(priv->pkey); > + path->pathrec.numb_path = 1; > + path->pathrec.flow_label = priv->broadcast->mcmember.flow_label; > + path->pathrec.traffic_class = priv->broadcast->mcmember.traffic_class; Sean, I think you want to ask for a specific SL here (that is add IB_SA_PATH_REC_SL to the mask and ask for the broadcast group SL). Roland, I realized now that the current IPoIB implementation does not set IB_SA_PATH_REC_MTU and IB_SA_PATH_REC_RATE (and their selectors) in the SA path query where it does so for SA multicast joins. Is there a specific reason for that? do you assume that if the remote node has managed to join the broadcast group as this node this the SA enforced some sort of MTU and RATE compliance which you would get in the returned path record? Or. From eitan at mellanox.co.il Tue Aug 7 03:24:43 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 7 Aug 2007 13:24:43 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> Hi Sean Just making sure. The SL should also be copied from the multicast group and be provided with the PR query. So should be the rate. So actually for IPoIB subnets the QoS parameters are defined "per partition" and set on the broadcast group by the SM. (or the first joining member ... But this is really calling for an argument). > --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c > @@ -468,9 +468,11 @@ static struct ipoib_path > *path_rec_create(struct net_device *dev, void *gid) > INIT_LIST_HEAD(&path->neigh_list); > > memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid)); > - path->pathrec.sgid = priv->local_gid; > - path->pathrec.pkey = cpu_to_be16(priv->pkey); > - path->pathrec.numb_path = 1; > + path->pathrec.sgid = priv->local_gid; > + path->pathrec.pkey = cpu_to_be16(priv->pkey); > + path->pathrec.numb_path = 1; > + path->pathrec.flow_label = > priv->broadcast->mcmember.flow_label; > + path->pathrec.traffic_class = > priv->broadcast->mcmember.traffic_class; > > return path; From eitan at mellanox.co.il Tue Aug 7 03:36:56 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 7 Aug 2007 13:36:56 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service In-Reply-To: <46B8388E.1090308@voltaire.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> Hi Or, Sean, SID and QoSClass have a different purpose: * SID enables the administrator to set the QoS attributes without any application change. (e.g. a simple rule like all SSH should have lowest priority) * QoS Class: enables the application to request QoS class on a per-socket manner. It requires the application to know what it is doing and the administrator to trust it. It is not by chance that the two options are there in the spec. Different communities/installations have different requirements regarding "who controls" the policy. For OFA development I think that providing SID should be done by default CM based traffic: The SID is a mandatory for making a connection and thus I do not see any reason not to provide it in the PathRecord query. Supporting QoS-Class is a compatibility feature supporting application that set the TOS socket option to obtain TOS aware service over IB. Eitan Eitan Zahavi > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz > Sent: Tuesday, August 07, 2007 12:17 PM > To: Sean Hefty > Cc: OpenIB > Subject: Re: [ofa-general] [PATCH 3/4] rdma/cm: add ability > to specify typeof service > > Sean Hefty wrote: > > Provide support to specify a type of service for a communication > > identifier. A new function call is used when dealing with IPv4 > > addresses. For IPv6 addresses, the ToS is specified through the > > traffic class and flow label fields in the sockaddr_in6 structure. > > > @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct > > rdma_id_private *id_priv, int > > > + comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | > > + IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | > > + IB_SA_PATH_REC_REVERSIBLE; > > + > > + if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) { > > + path_rec.qos_class = cpu_to_be16((u16) id_priv->tos); > > + comp_mask |= IB_SA_PATH_REC_QOS_CLASS; > > + } else { > > + sin6 = (struct sockaddr_in6 *) > id_priv->id.route.addr.src_addr; > > + path_rec.flow_label = sin6->sin6_flowinfo; > > + path_rec.traffic_class = (u8) > (be32_to_cpu(sin6->sin6_flowinfo) >> 20); > > + comp_mask |= IB_SA_PATH_REC_FLOW_LABEL | > > + IB_SA_PATH_REC_TRAFFIC_CLASS; > > + } > > I think the correct way here, is a SID --> QoS mapping. This > allows you to manage and configure QoS in --one-- place, > namely the SM/SA and zero effort host side deployment. > > For example clustered database has IB subnet both for the > inter RAC IPC and for the RAC nodes storage access, say IPC > is RDS and storage is iSER (a similar example from HPC, IPC > is MPI and I/O is Lustre) > > So you would need to configure an SID to SL mapping at the SA > for each service and you are --done--. Taking this to cross > IB subnets, the SID based mapping will also provide the TC > and FL for the GRH. > > With the above implementation you would have to change each > single ULP to have it use QoS and you have to have a > configuration file also at the host side (saying what tos to > set through the rdma cm) and not only in the SM. > > Or. > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From thomas.bub at thomson.net Tue Aug 7 03:41:57 2007 From: thomas.bub at thomson.net (Bub Thomas) Date: Tue, 7 Aug 2007 12:41:57 +0200 Subject: [ofa-general] How to get a thread out of the ib_cm_get_event while wthere is a broken IB connection? In-Reply-To: <46B57329.9020502@voltaire.com> References: <29C41D06A093C1449B2650D43A833E49EB37C8@WDTSSMAIL02.eu.thmulti.com> <46B57329.9020502@voltaire.com> Message-ID: <29C41D06A093C1449B2650D43A833E49EB37D1@WDTSSMAIL02.eu.thmulti.com> Or, I'm using the OSM to query the GID and LID via a service ID that the server advertises. Since my server is running Montavista Linux on a PowertPC that is actually located in an FPGA I refused from using IPoIB which does not allow to use lbrdmacm then. Thomas -----Original Message----- From: Or Gerlitz [mailto:ogerlitz at voltaire.com] Sent: Sunday, August 05, 2007 8:50 AM To: Bub Thomas Cc: general at lists.openfabrics.org; Thomas Bub Subject: Re: [ofa-general] How to get a thread out of the ib_cm_get_event while wthere is a broken IB connection? Bub Thomas wrote: > Our application is using OFED 1.1 under SLES10. > It is a point to point IB connection from a high speed image scanner to > a workstation. > Now I get complains from the field that they do have dead pthreads > hanging around that are stuck in the ib_cm_get_event call. From your question, I understand that you use libibcm. How do you get the peer IB L3 (GID) and L2 (LID) info? are you using some out-of-band (eg through TCP) exchange for that? May I ask if libibcm buys you something which librdmacm doesn't? Or. From hal.rosenstock at gmail.com Tue Aug 7 04:04:16 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 7 Aug 2007 07:04:16 -0400 Subject: [ofa-general] [PATCH 4/4] rdma/ucm: export setting service type to user space In-Reply-To: <000301c7d888$81646260$9c98070a@amr.corp.intel.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <000301c7d888$81646260$9c98070a@amr.corp.intel.com> Message-ID: On 8/6/07, Sean Hefty wrote: > Export the ability to set the type of service to user space. Model > after setsockopt. For symmetry, what about implementing get as well ? In sockets, TOS is both set/get socket option. -- Hal > > Signed-off-by: Sean Hefty > --- > A related set of changes to the librdmacm is also available. > > drivers/infiniband/core/ucma.c | 74 +++++++++++++++++++++++++++++++++++++++- > include/rdma/rdma_user_cm.h | 18 ++++++++++ > 2 files changed, 91 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c > index 53b4c94..90d675a 100644 > --- a/drivers/infiniband/core/ucma.c > +++ b/drivers/infiniband/core/ucma.c > @@ -792,6 +792,78 @@ out: > return ret; > } > > +static int ucma_set_option_id(struct ucma_context *ctx, int optname, > + void *optval, size_t optlen) > +{ > + int ret = 0; > + > + switch (optname) { > + case RDMA_OPTION_ID_TOS: > + if (optlen != sizeof(u8)) { > + ret = -EINVAL; > + break; > + } > + rdma_set_service_type(ctx->cm_id, *((u8 *) optval)); > + break; > + default: > + ret = -ENOSYS; > + } > + > + return ret; > +} > + > +static int ucma_set_option_level(struct ucma_context *ctx, int level, > + int optname, void *optval, size_t optlen) > +{ > + int ret; > + > + switch (level) { > + case RDMA_OPTION_ID: > + ret = ucma_set_option_id(ctx, optname, optval, optlen); > + break; > + default: > + ret = -ENOSYS; > + } > + > + return ret; > +} > + > +static ssize_t ucma_set_option(struct ucma_file *file, const char __user *inbuf, > + int in_len, int out_len) > +{ > + struct rdma_ucm_set_option cmd; > + struct ucma_context *ctx; > + void *optval; > + int ret; > + > + if (copy_from_user(&cmd, inbuf, sizeof(cmd))) > + return -EFAULT; > + > + ctx = ucma_get_ctx(file, cmd.id); > + if (IS_ERR(ctx)) > + return PTR_ERR(ctx); > + > + optval = kmalloc(cmd.optlen, GFP_KERNEL); > + if (!optval) { > + ret = -ENOMEM; > + goto out1; > + } > + > + if (copy_from_user(optval, (void __user *) (unsigned long) cmd.optval, > + cmd.optlen)) { > + ret = -EFAULT; > + goto out2; > + } > + > + ret = ucma_set_option_level(ctx, cmd.level, cmd.optname, optval, > + cmd.optlen); > +out2: > + kfree(optval); > +out1: > + ucma_put_ctx(ctx); > + return ret; > +} > + > static ssize_t ucma_notify(struct ucma_file *file, const char __user *inbuf, > int in_len, int out_len) > { > @@ -936,7 +1008,7 @@ static ssize_t (*ucma_cmd_table[])(struct ucma_file *file, > [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr, > [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event, > [RDMA_USER_CM_CMD_GET_OPTION] = NULL, > - [RDMA_USER_CM_CMD_SET_OPTION] = NULL, > + [RDMA_USER_CM_CMD_SET_OPTION] = ucma_set_option, > [RDMA_USER_CM_CMD_NOTIFY] = ucma_notify, > [RDMA_USER_CM_CMD_JOIN_MCAST] = ucma_join_multicast, > [RDMA_USER_CM_CMD_LEAVE_MCAST] = ucma_leave_multicast, > diff --git a/include/rdma/rdma_user_cm.h b/include/rdma/rdma_user_cm.h > index f632b0c..9749c1b 100644 > --- a/include/rdma/rdma_user_cm.h > +++ b/include/rdma/rdma_user_cm.h > @@ -212,4 +212,22 @@ struct rdma_ucm_event_resp { > } param; > }; > > +/* Option levels */ > +enum { > + RDMA_OPTION_ID = 0 > +}; > + > +/* Option details */ > +enum { > + RDMA_OPTION_ID_TOS = 0 > +}; > + > +struct rdma_ucm_set_option { > + __u64 optval; > + __u32 id; > + __u32 level; > + __u32 optname; > + __u32 optlen; > +}; > + > #endif /* RDMA_USER_CM_H */ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From mst at dev.mellanox.co.il Tue Aug 7 04:29:21 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 14:29:21 +0300 Subject: [ofa-general] sparse output Message-ID: <20070807112921.GC13863@mellanox.co.il> Running sparse checker (git://git.kernel.org/pub/scm/devel/sparse/sparse.git) on 2.6.23-rc2 gives the following warnings: /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/device.c:142:6: warning: incorrect type in argument 1 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/device.c:142:6: expected unsigned long const *addr /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/device.c:142:6: got long *[assigned] inuse /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: warning: incorrect type in argument 3 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: expected int *qp_attr_mask /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: got unsigned int * /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:127:32: warning: incorrect type in argument 3 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:127:32: expected int *id /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:127:32: got unsigned int * /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:302:46: warning: incorrect type in argument 3 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:302:46: expected int *fd /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:302:46: got unsigned int * /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:723:46: warning: incorrect type in argument 3 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:723:46: expected int *fd /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/uverbs_cmd.c:723:46: got unsigned int * /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: warning: incorrect type in argument 3 (different signedness) /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: expected int *qp_attr_mask /home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: got unsigned int * /home/mst/build/x86_64/linux-2.6/drivers/infiniband/hw/mlx4/qp.c:460:2: warning: context imbalance in 'mlx4_ib_lock_cqs' - wrong count at exit /home/mst/build/x86_64/linux-2.6/drivers/infiniband/hw/mlx4/qp.c:474:3: warning: context imbalance in 'mlx4_ib_unlock_cqs' - unexpected unlock /home/mst/build/x86_64/linux-2.6/drivers/infiniband/hw/mthca/mthca_cmd.c:414:31: warning: potentially expensive pointer subtraction /home/mst/build/x86_64/linux-2.6/drivers/infiniband/hw/mthca/mthca_qp.c:1336:2: warning: context imbalance in 'mthca_lock_cqs' - wrong count at exit /home/mst/build/x86_64/linux-2.6/drivers/infiniband/hw/mthca/mthca_qp.c:1350:3: warning: context imbalance in 'mthca_unlock_cqs' - unexpected unlock Do we want to fix these? -- MST From bs at q-leap.de Tue Aug 7 05:26:36 2007 From: bs at q-leap.de (Bernd Schubert) Date: Tue, 7 Aug 2007 14:26:36 +0200 Subject: [ofa-general] ibnetdiscover Message-ID: <200708071426.36464.bs@q-leap.de> Hi, I two questions about ibnetdiscover. 1) How reliable is it? Here in our testing lab ibnetdiscover works fine, detects the proper names of the two infiniband switches and all connected client cards. On a customer system it doesn't work that well. I know the client is connected to an MTS2400 switch, but ibnetdiscover detects a MT47396. The MTS2400 is connected to MTS1400 switch, but again ibnetdiscover believs it is again a MT47396. Any idea whats going on? 2) ibnetdiscover also can't detect everything on the very same customer system, it shows this error ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, skipping port Does this mean port 1 of the last switch failed? I'm also not sure about the pathes, IMHO the man page of ibnetdiscover should give one more example, so -D use directed path address arguments. The path is a comma separated list of out ports. Examples: "0" # self port "0,1,2,1,4" # out via port 1, then 2, ... "out via port 1, then out via port 2, then out via port 1, ..." or "out via port 1, then in port 2, then out via port 1, ..." You see what I mean? Thanks in advance, Bernd -- Bernd Schubert Q-Leap Networks GmbH From hal.rosenstock at gmail.com Tue Aug 7 05:36:09 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 7 Aug 2007 08:36:09 -0400 Subject: [ofa-general] ibnetdiscover In-Reply-To: <200708071426.36464.bs@q-leap.de> References: <200708071426.36464.bs@q-leap.de> Message-ID: On 8/7/07, Bernd Schubert wrote: > Hi, > > I two questions about ibnetdiscover. > > 1) How reliable is it? Here in our testing lab ibnetdiscover works fine, > detects the proper names of the two infiniband switches and all connected > client cards. On a customer system it doesn't work that well. I know the > client is connected to an MTS2400 switch, but ibnetdiscover detects a > MT47396. The MTS2400 is connected to MTS1400 switch, but again ibnetdiscover > believs it is again a MT47396. Any idea whats going on? Where is MT47396 being displayed ? ibnetdiscover displays the NodeDescription and perhaps that is not set properly on those switches. You can verify this with smpquery nodedesc. > > 2) ibnetdiscover also can't detect everything on the very same customer > system, it shows this error > > ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, > skipping port > > Does this mean port 1 of the last switch failed? It means that the peer port of port 1's SMA failed to respond to the SubnGet NodeInfo. What is connected there and what state is it in ? > I'm also not sure about the pathes, IMHO the man page of ibnetdiscover should > give one more example, so > > > > -D use directed path address arguments. The path > is a comma separated list of out ports. > Examples: > "0" # self port > "0,1,2,1,4" # out via port 1, then 2, ... > > > "out via port 1, then out via port 2, then out via port 1, ..." > > or > > "out via port 1, then in port 2, then out via port 1, ..." > > > You see what I mean? It's the former. It's the out port on each hop along the path. -- Hal > > > Thanks in advance, > Bernd > > -- > Bernd Schubert > Q-Leap Networks GmbH > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From torhild.daniell at phaenomenta.com Tue Aug 7 13:51:03 2007 From: torhild.daniell at phaenomenta.com (Edith Webster) Date: Tue, 7 Aug 2007 12:51:03 -0800 Subject: [ofa-general] Enjoy with you hard stick Message-ID: <148939340.50115361205244@phaenomenta.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: maliska.gif Type: image/gif Size: 14473 bytes Desc: not available URL: From KEN at revrap.com Tue Aug 7 06:02:17 2007 From: KEN at revrap.com (KEN Plag) Date: Tue, 7 Aug 2007 21:02:17 +0800 Subject: [ofa-general] 357282 Message-ID: <717020868191.354285073122@revrap.com> 652a5A83 From hnguyen at linux.vnet.ibm.com Tue Aug 7 06:19:43 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Tue, 7 Aug 2007 15:19:43 +0200 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <20070807070904.GA3868@mellanox.co.il> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708070913.36106.hnguyen@linux.vnet.ibm.com> <20070807070904.GA3868@mellanox.co.il> Message-ID: <200708071519.43929.hnguyen@linux.vnet.ibm.com> Hello Michael! Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 as we've discussed previously. Thereby I realized current backport code in slab.h looks weird to me (sort of copy&paste mixture) - actually no build error, only coding "issue". Therefore this patch also includes some cleanup. If it's ok, please apply. PS: The mentioned issue in backport slab.h exists also in other versions. If you want me to fix them as well, let me know. Regards Nam backport kmem_cache_zalloc() in slab.h to 2.6.10, 2.6.10_sles10 and 2.6.10_sles10_sp1 Signed-off-by: Hoang-Nam Nguyen --- 2.6.16/include/linux/slab.h | 22 +++++++--------------- 2.6.16_sles10/include/linux/slab.h | 22 +++++++--------------- 2.6.16_sles10_sp1/include/linux/slab.h | 22 +++++++--------------- 3 files changed, 21 insertions(+), 45 deletions(-) diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16/include/linux/slab.h 2007-08-06 18:29:17.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h 2007-08-06 18:30:33.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h 2007-08-06 18:30:40.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif From mst at dev.mellanox.co.il Tue Aug 7 06:08:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:08:28 +0300 Subject: [ofa-general] [PATCHv3 for-2.6.24] IB/mlx4: enable MSI-X by default Message-ID: <20070807130828.GA24064@mellanox.co.il> Recover from MSI-X errors by automatically falling back on regular interrupt, instead of asking the user to do this manually. This makes it possible to enable MSI-X by default, and will make it possible to get rid of msi_x module option in the future. Signed-off-by: Michael S. Tsirkin --- Index: linux-2.6/drivers/net/mlx4/main.c =================================================================== --- linux-2.6.orig/drivers/net/mlx4/main.c 2007-08-07 16:05:24.000000000 +0300 +++ linux-2.6/drivers/net/mlx4/main.c 2007-08-07 16:06:23.000000000 +0300 @@ -61,7 +61,7 @@ MODULE_PARM_DESC(debug_level, "Enable de #ifdef CONFIG_PCI_MSI -static int msi_x; +static int msi_x = 1; module_param(msi_x, int, 0444); MODULE_PARM_DESC(msi_x, "attempt to use MSI-X if nonzero"); @@ -603,7 +603,7 @@ static int __devinit mlx4_setup_hca(stru "(IRQ %d), aborting.\n", priv->eq_table.eq[MLX4_EQ_ASYNC].irq); if (dev->flags & MLX4_FLAG_MSI_X) - mlx4_err(dev, "Try again with MSI-X disabled.\n"); + mlx4_warn(dev, "Trying again with MSI-X disabled.\n"); else mlx4_err(dev, "BIOS or ACPI interrupt routing problem?\n"); @@ -803,8 +803,6 @@ static int __devinit mlx4_init_one(struc goto err_free_dev; } - mlx4_enable_msi_x(dev); - if (mlx4_cmd_init(dev)) { mlx4_err(dev, "Failed to init command interface, aborting.\n"); goto err_free_dev; @@ -814,7 +812,16 @@ static int __devinit mlx4_init_one(struc if (err) goto err_cmd; + mlx4_enable_msi_x(dev); + err = mlx4_setup_hca(dev); + if (err == -EBUSY && (dev->flags & MLX4_FLAG_MSI_X)) { + mlx4_warn(dev, "Trying again with MSI/MSI-X disabled.\n"); + dev->flags &= ~MLX4_FLAG_MSI_X; + pci_disable_msix(pdev); + err = mlx4_setup_hca(dev); + } + if (err) goto err_close; @@ -838,15 +845,15 @@ err_cleanup: mlx4_cleanup_uar_table(dev); err_close: + if (dev->flags & MLX4_FLAG_MSI_X) + pci_disable_msix(pdev); + mlx4_close_hca(dev); err_cmd: mlx4_cmd_cleanup(dev); err_free_dev: - if (dev->flags & MLX4_FLAG_MSI_X) - pci_disable_msix(pdev); - kfree(priv); err_release_bar2: -- MST From mst at dev.mellanox.co.il Tue Aug 7 06:10:34 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:10:34 +0300 Subject: [ofa-general] [PATCH v2 for-2.6.24] IB/mthca: enable MSI-X by default Message-ID: <20070807131034.GB24064@mellanox.co.il> Recover from MSI-X errors by automatically falling back on regular interrupt, instead of asking the user to do this manually. This makes it possible to enable MSI-X by default, and will make it possible to get rid of msi_x module option in the future. Signed-off-by: Michael S. Tsirkin --- Index: linux-2.6/drivers/infiniband/hw/mthca/mthca_main.c =================================================================== --- linux-2.6.orig/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-02 10:23:24.000000000 +0300 +++ linux-2.6/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-07 16:09:36.000000000 +0300 @@ -61,7 +61,7 @@ MODULE_PARM_DESC(debug_level, "Enable de #ifdef CONFIG_PCI_MSI -static int msi_x = 0; +static int msi_x = 1; module_param(msi_x, int, 0444); MODULE_PARM_DESC(msi_x, "attempt to use MSI-X if nonzero"); @@ -838,7 +838,7 @@ static int mthca_setup_hca(struct mthca_ dev->eq_table.eq[MTHCA_EQ_CMD].msi_x_vector : dev->pdev->irq); if (dev->mthca_flags & (MTHCA_FLAG_MSI | MTHCA_FLAG_MSI_X)) - mthca_err(dev, "Try again with MSI/MSI-X disabled.\n"); + mthca_warn(dev, "Trying again with MSI/MSI-X disabled.\n"); else mthca_err(dev, "BIOS or ACPI interrupt routing problem?\n"); @@ -1115,24 +1115,6 @@ static int __mthca_init_one(struct pci_d goto err_free_dev; } - if (msi_x && !mthca_enable_msi_x(mdev)) - mdev->mthca_flags |= MTHCA_FLAG_MSI_X; - else if (msi) { - static int warned; - - if (!warned) { - printk(KERN_WARNING PFX "WARNING: MSI support will be " - "removed from the ib_mthca driver in January 2008.\n"); - printk(KERN_WARNING " If you are using MSI and cannot " - "switch to MSI-X, please tell " - ".\n"); - ++warned; - } - - if (!pci_enable_msi(pdev)) - mdev->mthca_flags |= MTHCA_FLAG_MSI; - } - if (mthca_cmd_init(mdev)) { mthca_err(mdev, "Failed to init command interface, aborting.\n"); goto err_free_dev; @@ -1156,7 +1138,36 @@ static int __mthca_init_one(struct pci_d mthca_warn(mdev, "If you have problems, try updating your HCA FW.\n"); } + if (msi_x && !mthca_enable_msi_x(mdev)) + mdev->mthca_flags |= MTHCA_FLAG_MSI_X; + else if (msi) { + static int warned; + + if (!warned) { + printk(KERN_WARNING PFX "WARNING: MSI support will be " + "removed from the ib_mthca driver in January 2008.\n"); + printk(KERN_WARNING " If you are using MSI and cannot " + "switch to MSI-X, please tell " + ".\n"); + ++warned; + } + + if (!pci_enable_msi(pdev)) + mdev->mthca_flags |= MTHCA_FLAG_MSI; + } + err = mthca_setup_hca(mdev); + if (err == -EBUSY && (mdev->mthca_flags & (MTHCA_FLAG_MSI | MTHCA_FLAG_MSI_X))) { + mthca_warn(mdev, "Trying again with MSI/MSI-X disabled.\n"); + if (mdev->mthca_flags & MTHCA_FLAG_MSI_X) + pci_disable_msix(pdev); + if (mdev->mthca_flags & MTHCA_FLAG_MSI) + pci_disable_msi(pdev); + mdev->mthca_flags &= ~(MTHCA_FLAG_MSI_X | MTHCA_FLAG_MSI); + + err = mthca_setup_hca(mdev); + } + if (err) goto err_close; @@ -1192,17 +1203,17 @@ err_cleanup: mthca_cleanup_uar_table(mdev); err_close: + if (mdev->mthca_flags & MTHCA_FLAG_MSI_X) + pci_disable_msix(pdev); + if (mdev->mthca_flags & MTHCA_FLAG_MSI) + pci_disable_msi(pdev); + mthca_close_hca(mdev); err_cmd: mthca_cmd_cleanup(mdev); err_free_dev: - if (mdev->mthca_flags & MTHCA_FLAG_MSI_X) - pci_disable_msix(pdev); - if (mdev->mthca_flags & MTHCA_FLAG_MSI) - pci_disable_msi(pdev); - ib_dealloc_device(&mdev->ib_dev); err_free_res: -- MST From mst at dev.mellanox.co.il Tue Aug 7 06:13:29 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:13:29 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708071519.43929.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708070913.36106.hnguyen@linux.vnet.ibm.com> <20070807070904.GA3868@mellanox.co.il> <200708071519.43929.hnguyen@linux.vnet.ibm.com> Message-ID: <20070807131329.GC24064@mellanox.co.il> I'm happy with stuff as it is: the ifdefs make it easy to figure which version does the backport apply. BTW, I think the same backport will be needed for older kernels as well, no? Quoting Hoang-Nam Nguyen : Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 Hello Michael! Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 as we've discussed previously. Thereby I realized current backport code in slab.h looks weird to me (sort of copy&paste mixture) - actually no build error, only coding "issue". Therefore this patch also includes some cleanup. If it's ok, please apply. PS: The mentioned issue in backport slab.h exists also in other versions. If you want me to fix them as well, let me know. Regards Nam backport kmem_cache_zalloc() in slab.h to 2.6.10, 2.6.10_sles10 and 2.6.10_sles10_sp1 Signed-off-by: Hoang-Nam Nguyen --- 2.6.16/include/linux/slab.h | 22 +++++++--------------- 2.6.16_sles10/include/linux/slab.h | 22 +++++++--------------- 2.6.16_sles10_sp1/include/linux/slab.h | 22 +++++++--------------- 3 files changed, 21 insertions(+), 45 deletions(-) diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16/include/linux/slab.h 2007-08-06 18:29:17.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h 2007-08-06 18:30:33.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif diff -Nurp ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h --- ofa_1_2_c_kernel-20070804-0200_orig/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h 2007-08-04 11:00:08.000000000 +0200 +++ ofa_1_2_c_kernel-20070804-0200/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h 2007-08-06 18:30:40.000000000 +0200 @@ -1,10 +1,8 @@ -#include_next +#ifndef _LINUX_SLAB_BACKPORT_TO_2_6_16 +#define _LINUX_SLAB_BACKPORT_TO_2_6_16 #include_next -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline void *kmemdup(const void *src, size_t len, gfp_t gfp) { @@ -16,19 +14,13 @@ void *kmemdup(const void *src, size_t le return p; } -#endif -#ifndef BACKPORT_LINUX_STRING_TO_2_6_18 -#define BACKPORT_LINUX_STRING_TO_2_6_18 - static inline -void *kmemdup(const void *src, size_t len, gfp_t gfp) +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) { - void *p; - - p = kmalloc(len, gfp); - if (p) - memcpy(p, src, len); - return p; + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + return ret; } #endif -- MST From mst at dev.mellanox.co.il Tue Aug 7 06:23:02 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:23:02 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708071519.43929.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708070913.36106.hnguyen@linux.vnet.ibm.com> <20070807070904.GA3868@mellanox.co.il> <200708071519.43929.hnguyen@linux.vnet.ibm.com> Message-ID: <20070807132302.GD24064@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > Hello Michael! > Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 > as we've discussed previously. Thereby I realized current backport code > in slab.h looks weird to me (sort of copy&paste mixture) - actually no build > error, only coding "issue". > Therefore this patch also includes some cleanup. If it's ok, please apply. > PS: The mentioned issue in backport slab.h exists also in other versions. > If you want me to fix them as well, let me know. > Regards > Nam Would not the following work? If yes, Vlad, I parked this at git://git.openfabrics.org/~mst/ofed_kernel.git ofed_1_2_c --------------------------------------------------------- commit 68ba5a0d23460b9580af3afd0dbc35ce87bce089 Author: Michael S. Tsirkin Date: Tue Aug 7 16:19:30 2007 +0300 Backport kmem_cache_zalloc to kernels <= 2.6.16. Used by ehca. Signed-off-by: Michael S. Tsirkin --- diff --git a/kernel_addons/backport/2.6.11/include/linux/slab.h b/kernel_addons/backport/2.6.11/include/linux/slab.h index 9d82afa..78e096f 100644 --- a/kernel_addons/backport/2.6.11/include/linux/slab.h +++ b/kernel_addons/backport/2.6.11/include/linux/slab.h @@ -40,3 +40,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.11_FC4/include/linux/slab.h b/kernel_addons/backport/2.6.11_FC4/include/linux/slab.h index bd9671d..8089f73 100644 --- a/kernel_addons/backport/2.6.11_FC4/include/linux/slab.h +++ b/kernel_addons/backport/2.6.11_FC4/include/linux/slab.h @@ -55,3 +55,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.12/include/linux/slab.h b/kernel_addons/backport/2.6.12/include/linux/slab.h index bd9671d..8089f73 100644 --- a/kernel_addons/backport/2.6.12/include/linux/slab.h +++ b/kernel_addons/backport/2.6.12/include/linux/slab.h @@ -55,3 +55,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.13/include/linux/slab.h b/kernel_addons/backport/2.6.13/include/linux/slab.h index 0540cc6..a54188d 100644 --- a/kernel_addons/backport/2.6.13/include/linux/slab.h +++ b/kernel_addons/backport/2.6.13/include/linux/slab.h @@ -45,3 +45,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/slab.h b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/slab.h index 0540cc6..a54188d 100644 --- a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/slab.h +++ b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/slab.h @@ -45,3 +45,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.14/include/linux/slab.h b/kernel_addons/backport/2.6.14/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.14/include/linux/slab.h +++ b/kernel_addons/backport/2.6.14/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.15/include/linux/slab.h b/kernel_addons/backport/2.6.15/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.15/include/linux/slab.h +++ b/kernel_addons/backport/2.6.15/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.15_ubuntu606/include/linux/slab.h b/kernel_addons/backport/2.6.15_ubuntu606/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.15_ubuntu606/include/linux/slab.h +++ b/kernel_addons/backport/2.6.15_ubuntu606/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.16/include/linux/slab.h b/kernel_addons/backport/2.6.16/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.16/include/linux/slab.h +++ b/kernel_addons/backport/2.6.16/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h b/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h +++ b/kernel_addons/backport/2.6.16_sles10/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h b/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h index 46ac6e5..b805107 100644 --- a/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h +++ b/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/slab.h @@ -32,3 +32,17 @@ void *kmemdup(const void *src, size_t len, gfp_t gfp) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/slab.h b/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/slab.h index c8285ac..8067cb5 100644 --- a/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/slab.h +++ b/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/slab.h @@ -30,3 +30,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.9_U2/include/linux/slab.h b/kernel_addons/backport/2.6.9_U2/include/linux/slab.h index 9d82afa..78e096f 100644 --- a/kernel_addons/backport/2.6.9_U2/include/linux/slab.h +++ b/kernel_addons/backport/2.6.9_U2/include/linux/slab.h @@ -40,3 +40,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.9_U3/include/linux/slab.h b/kernel_addons/backport/2.6.9_U3/include/linux/slab.h index cbf6311..79af0e5 100644 --- a/kernel_addons/backport/2.6.9_U3/include/linux/slab.h +++ b/kernel_addons/backport/2.6.9_U3/include/linux/slab.h @@ -21,3 +21,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.9_U4/include/linux/slab.h b/kernel_addons/backport/2.6.9_U4/include/linux/slab.h index cbf6311..79af0e5 100644 --- a/kernel_addons/backport/2.6.9_U4/include/linux/slab.h +++ b/kernel_addons/backport/2.6.9_U4/include/linux/slab.h @@ -21,3 +21,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif diff --git a/kernel_addons/backport/2.6.9_U5/include/linux/slab.h b/kernel_addons/backport/2.6.9_U5/include/linux/slab.h index cbf6311..79af0e5 100644 --- a/kernel_addons/backport/2.6.9_U5/include/linux/slab.h +++ b/kernel_addons/backport/2.6.9_U5/include/linux/slab.h @@ -21,3 +21,17 @@ void *kmalloc_node(size_t size, gfp_t flags, int nid) } #endif +#ifndef SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 +#define SLAB_H_KMEMCACHE_ZALLOC_BACKPORT_TO_2_6_16 + +static inline +void *kmem_cache_zalloc(struct kmem_cache *cache, gfp_t flags) +{ + void *ret = kmem_cache_alloc(cache, flags); + if (ret) + memset(ret, 0, kmem_cache_size(cache)); + + return ret; +} + +#endif -- MST From hnguyen at linux.vnet.ibm.com Tue Aug 7 06:42:58 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Tue, 7 Aug 2007 15:42:58 +0200 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <20070807132302.GD24064@mellanox.co.il> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708071519.43929.hnguyen@linux.vnet.ibm.com> <20070807132302.GD24064@mellanox.co.il> Message-ID: <200708071542.59010.hnguyen@linux.vnet.ibm.com> On Tuesday 07 August 2007 15:23, Michael S. Tsirkin wrote: > > Quoting Hoang-Nam Nguyen : > > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > > > Hello Michael! > > Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 > > as we've discussed previously. Thereby I realized current backport code > > in slab.h looks weird to me (sort of copy&paste mixture) - actually no build > > error, only coding "issue". > > Therefore this patch also includes some cleanup. If it's ok, please apply. > > PS: The mentioned issue in backport slab.h exists also in other versions. > > If you want me to fix them as well, let me know. > > Regards > > Nam > > Would not the following work? If yes, Vlad, I parked this at Wow, you're pretty quick. Yes, it should work. And you're right we need this patch for <=2.6.16. PS: The weird thing in slab.h I meant previously is that the ifdef-kmem_dup()-block exists twice in same file, which does not harm the build. Thanks! Nam From mst at dev.mellanox.co.il Tue Aug 7 06:34:53 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:34:53 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708071542.59010.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708071519.43929.hnguyen@linux.vnet.ibm.com> <20070807132302.GD24064@mellanox.co.il> <200708071542.59010.hnguyen@linux.vnet.ibm.com> Message-ID: <20070807133453.GE24064@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > On Tuesday 07 August 2007 15:23, Michael S. Tsirkin wrote: > > > Quoting Hoang-Nam Nguyen : > > > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > > > > > Hello Michael! > > > Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 > > > as we've discussed previously. Thereby I realized current backport code > > > in slab.h looks weird to me (sort of copy&paste mixture) - actually no build > > > error, only coding "issue". > > > Therefore this patch also includes some cleanup. If it's ok, please apply. > > > PS: The mentioned issue in backport slab.h exists also in other versions. > > > If you want me to fix them as well, let me know. > > > Regards > > > Nam > > > > Would not the following work? If yes, Vlad, I parked this at > Wow, you're pretty quick. Yes, it should work. And you're right we need > this patch for <=2.6.16. > PS: The weird thing in slab.h I meant previously is that the > ifdef-kmem_dup()-block exists twice in same file, which does not harm > the build. > Thanks! > Nam I haven't seen this. Right. Can you please post a separate patch that just kills the duplicates? -- MST From mst at dev.mellanox.co.il Tue Aug 7 06:46:13 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 16:46:13 +0300 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <200708071542.59010.hnguyen@linux.vnet.ibm.com> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708071519.43929.hnguyen@linux.vnet.ibm.com> <20070807132302.GD24064@mellanox.co.il> <200708071542.59010.hnguyen@linux.vnet.ibm.com> Message-ID: <20070807134613.GF24064@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > On Tuesday 07 August 2007 15:23, Michael S. Tsirkin wrote: > > > Quoting Hoang-Nam Nguyen : > > > Subject: Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 > > > > > > Hello Michael! > > > Below is the patch to backport kmem_cache_zalloc() for 2.6.16/sles10/sles10_sp1 > > > as we've discussed previously. Thereby I realized current backport code > > > in slab.h looks weird to me (sort of copy&paste mixture) - actually no build > > > error, only coding "issue". > > > Therefore this patch also includes some cleanup. If it's ok, please apply. > > > PS: The mentioned issue in backport slab.h exists also in other versions. > > > If you want me to fix them as well, let me know. > > > Regards > > > Nam > > > > Would not the following work? If yes, Vlad, I parked this at > Wow, you're pretty quick. Yes, it should work. And you're right we need > this patch for <=2.6.16. > PS: The weird thing in slab.h I meant previously is that the > ifdef-kmem_dup()-block exists twice in same file, which does not harm > the build. > Thanks! > Nam OK, I've cleaned these up. Thanks for pointing this out. -- MST From swise at opengridcomputing.com Tue Aug 7 07:20:53 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 07 Aug 2007 09:20:53 -0500 Subject: [ofa-general] Re: virtual memory usage on long lived apps In-Reply-To: References: <46B78081.3060305@opengridcomputing.com> Message-ID: <46B87FC5.6030104@opengridcomputing.com> Roland Dreier wrote: > > Running this server with clients repeatedly connecting, running > > rpings, and disconnecting, I notice that the virtual memory consumed > > by the process is ever increasing. The shared/and reserved values > > don't increase, but the virtual does. IE: If you watch this process > > via 'top' you'll see the VIRT value increasing steadily over > > time/connections, and the RES/SHR values stay steady. I've seen it > > over 1GB of virtual memory used! > > > Q: Have you seen this before? Its like the OS never reuses mappings > > that were setup for memory registrations. Is that normal? Or should > > the virtual size shown in 'top' be reduced whenever a buffer is > > unregistered and freed? There's a definite possibility the code is > > leaking memory, but I'm wondering if this is perhaps a normal anomaly > > with mapping memory in linux... > > I've not really tested an app like that, but I would expect that > freeing the memory should prevent your virtual space from growing. I > would recheck your code and maybe add some tracing to make sure that > everything really does get freed -- or try the libibverbs valgrind > support ;) > I'll instrument the code and see if I can find the leak... What is valgrind (sounds painful :) ? > If you send me your code I may be able to find some time to > investigate too. Here: diff --git a/examples/rping.c b/examples/rping.c index 5098ebc..ddd086f 100644 --- a/examples/rping.c +++ b/examples/rping.c @@ -112,6 +112,7 @@ struct rping_rdma_info { struct rping_cb { int server; /* 0 iff client */ pthread_t cqthread; + pthread_t persistent_server_thread; struct ibv_comp_channel *channel; struct ibv_cq *cq; struct ibv_pd *pd; @@ -591,24 +592,26 @@ static void *cq_thread(void *arg) DEBUG_LOG("cq_thread started.\n"); while (1) { + pthread_testcancel(); + ret = ibv_get_cq_event(cb->channel, &ev_cq, &ev_ctx); if (ret) { fprintf(stderr, "Failed to get cq event!\n"); - exit(ret); + pthread_exit(NULL); } if (ev_cq != cb->cq) { fprintf(stderr, "Unkown CQ!\n"); - exit(-1); + pthread_exit(NULL); } ret = ibv_req_notify_cq(cb->cq, 0); if (ret) { fprintf(stderr, "Failed to set notify!\n"); - exit(ret); + pthread_exit(NULL); } ret = rping_cq_event_handler(cb); ibv_ack_cq_events(cb->cq, 1); if (ret) - exit(ret); + pthread_exit(NULL); } } @@ -748,13 +751,99 @@ static int rping_bind_server(struct rpin return ret; } - sem_wait(&cb->sem); - if (cb->state != CONNECT_REQUEST) { - fprintf(stderr, "wait for CONNECT_REQUEST state %d\n", - cb->state); - return -1; + return 0; +} + +static struct rping_cb *clone_cb(struct rping_cb *listening_cb) +{ + struct rping_cb *cb = malloc(sizeof *cb); + if (!cb) + return NULL; + *cb = *listening_cb; + cb->child_cm_id->context = cb; + return cb; +} + +static void free_cb(struct rping_cb *cb) +{ + free(cb); +} + +static void *rping_persistent_server_thread(void *arg) +{ + struct rping_cb *cb = arg; + struct ibv_recv_wr *bad_wr; + int ret; + + ret = rping_setup_qp(cb, cb->child_cm_id); + if (ret) { + fprintf(stderr, "setup_qp failed: %d\n", ret); + goto err0; + } + + ret = rping_setup_buffers(cb); + if (ret) { + fprintf(stderr, "rping_setup_buffers failed: %d\n", ret); + goto err1; + } + + ret = ibv_post_recv(cb->qp, &cb->rq_wr, &bad_wr); + if (ret) { + fprintf(stderr, "ibv_post_recv failed: %d\n", ret); + goto err2; } + pthread_create(&cb->cqthread, NULL, cq_thread, cb); + + ret = rping_accept(cb); + if (ret) { + fprintf(stderr, "connect error %d\n", ret); + goto err3; + } + + rping_test_server(cb); + rdma_disconnect(cb->child_cm_id); + rping_free_buffers(cb); + rping_free_qp(cb); + pthread_cancel(cb->cqthread); + pthread_join(cb->cqthread, NULL); + rdma_destroy_id(cb->child_cm_id); + free_cb(cb); + return NULL; +err3: + pthread_cancel(cb->cqthread); + pthread_join(cb->cqthread, NULL); +err2: + rping_free_buffers(cb); +err1: + rping_free_qp(cb); +err0: + free_cb(cb); + return NULL; +} + +static int rping_run_persistent_server(struct rping_cb *listening_cb) +{ + int ret; + struct rping_cb *cb; + + ret = rping_bind_server(listening_cb); + if (ret) + return ret; + + while (1) { + sem_wait(&listening_cb->sem); + if (listening_cb->state != CONNECT_REQUEST) { + fprintf(stderr, "wait for CONNECT_REQUEST state %d\n", + listening_cb->state); + return -1; + } + + cb = clone_cb(listening_cb); + if (!cb) + return -1; + pthread_create(&cb->persistent_server_thread, NULL, rping_persistent_server_thread, cb); + } return 0; } @@ -767,6 +856,13 @@ static int rping_run_server(struct rping if (ret) return ret; + sem_wait(&cb->sem); + if (cb->state != CONNECT_REQUEST) { + fprintf(stderr, "wait for CONNECT_REQUEST state %d\n", + cb->state); + return -1; + } + ret = rping_setup_qp(cb, cb->child_cm_id); if (ret) { fprintf(stderr, "setup_qp failed: %d\n", ret); @@ -987,6 +1083,7 @@ static void usage(char *name) printf("\t-C count\tping count times\n"); printf("\t-a addr\t\taddress\n"); printf("\t-p port\t\tport\n"); + printf("\t-P\t\tpersistent server mode allowing multiple connections\n"); } int main(int argc, char *argv[]) @@ -994,6 +1091,7 @@ int main(int argc, char *argv[]) struct rping_cb *cb; int op; int ret = 0; + int persistent_server = 0; cb = malloc(sizeof(*cb)); if (!cb) @@ -1007,13 +1105,16 @@ int main(int argc, char *argv[]) sem_init(&cb->sem, 0, 0); opterr = 0; - while ((op=getopt(argc, argv, "a:p:C:S:t:scvVd")) != -1) { + while ((op=getopt(argc, argv, "a:Pp:C:S:t:scvVd")) != -1) { switch (op) { case 'a': cb->addr_str = optarg; cb->addr = inet_addr(optarg); DEBUG_LOG("ipaddr (%s)\n", optarg); break; + case 'P': + persistent_server = 1; + break; case 'p': cb->port = htons(atoi(optarg)); DEBUG_LOG("port %d\n", (int) atoi(optarg)); @@ -1089,9 +1190,12 @@ int main(int argc, char *argv[]) pthread_create(&cb->cmthread, NULL, cm_thread, cb); - if (cb->server) - ret = rping_run_server(cb); - else + if (cb->server) { + if (persistent_server) + ret = rping_run_persistent_server(cb); + else + ret = rping_run_server(cb); + } else ret = rping_run_client(cb); DEBUG_LOG("destroy cm_id %p\n", cb->cm_id); diff --git a/man/rping.1 b/man/rping.1 index 153436a..a2b7b6b 100644 --- a/man/rping.1 +++ b/man/rping.1 @@ -4,7 +4,7 @@ rping \- RDMA CM connection and RDMA pin .SH SYNOPSIS .sp .nf -\fIrping\fR -s [-v] [-V] [-d] [-a address] [-p port] +\fIrping\fR -s [-v] [-V] [-d] [-P] [-a address] [-p port] [-C message_count] [-S message_size] \fIrping\fR -c [-v] [-V] [-d] -a address [-p port] [-C message_count] [-S message_size] @@ -42,6 +42,10 @@ The number of messages to transfer over .TP \-S message_size The size of each message transferred, in bytes. (default 100) +.TP +\-P +Run the server in persistent mode. This allows multiple rping clients +to connect to a single server instance. The server will run until killed. .SH "NOTES" Because this test maps RDMA resources to userspace, users must ensure that they have available system resources and permissions. See the From hnguyen at linux.vnet.ibm.com Tue Aug 7 07:40:27 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Tue, 7 Aug 2007 16:40:27 +0200 Subject: [ofa-general] Re: [PATCH ofed-1.2.c] ehca: backport kmem_cache_zalloc() for 2.6.10/sles10/sles10_sp1 In-Reply-To: <20070807134613.GF24064@mellanox.co.il> References: <200708061623.46842.hnguyen@linux.vnet.ibm.com> <200708071542.59010.hnguyen@linux.vnet.ibm.com> <20070807134613.GF24064@mellanox.co.il> Message-ID: <200708071640.28217.hnguyen@linux.vnet.ibm.com> > OK, I've cleaned these up. Thanks for pointing this out. Just created the diff and saw this. Next time I'll answer you first. Anyway thanks for your helps! Nam From swise at opengridcomputing.com Tue Aug 7 07:30:53 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 07 Aug 2007 09:30:53 -0500 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service In-Reply-To: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> Message-ID: <46B8821D.4050608@opengridcomputing.com> Don't forget the man page for the new function... Sean Hefty wrote: > Provide support to specify a type of service for a communication > identifier. A new function call is used when dealing with IPv4 > addresses. For IPv6 addresses, the ToS is specified through the > traffic class and flow label fields in the sockaddr_in6 structure. > > Signed-off-by: Sean Hefty > --- > > drivers/infiniband/core/cma.c | 37 +++++++++++++++++++++++++++++++------ > include/rdma/rdma_cm.h | 14 ++++++++++++++ > 2 files changed, 45 insertions(+), 6 deletions(-) > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 32a0e66..c8b15f0 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -138,6 +138,7 @@ struct rdma_id_private { > u32 qkey; > u32 qp_num; > u8 srq; > + u8 tos; > }; > > struct cma_multicast { > @@ -1474,6 +1475,15 @@ err: > } > EXPORT_SYMBOL(rdma_listen); > > +void rdma_set_service_type(struct rdma_cm_id *id, int tos) > +{ > + struct rdma_id_private *id_priv; > + > + id_priv = container_of(id, struct rdma_id_private, id); > + id_priv->tos = (u8) tos; > +} > +EXPORT_SYMBOL(rdma_set_service_type); > + > static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec, > void *context) > { > @@ -1500,6 +1510,8 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int > timeout_ms, > { > struct rdma_dev_addr *addr = &id_priv->id.route.addr.dev_addr; > struct ib_sa_path_rec path_rec; > + ib_sa_comp_mask comp_mask; > + struct sockaddr_in6 *sin6; > > memset(&path_rec, 0, sizeof path_rec); > ib_addr_get_sgid(addr, &path_rec.sgid); > @@ -1508,13 +1520,26 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int > timeout_ms, > path_rec.numb_path = 1; > path_rec.reversible = 1; > > + comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | > + IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | > + IB_SA_PATH_REC_REVERSIBLE; > + > + if (id_priv->id.route.addr.src_addr.sa_family == AF_INET) { > + path_rec.qos_class = cpu_to_be16((u16) id_priv->tos); > + comp_mask |= IB_SA_PATH_REC_QOS_CLASS; > + } else { > + sin6 = (struct sockaddr_in6 *) id_priv->id.route.addr.src_addr; > + path_rec.flow_label = sin6->sin6_flowinfo; > + path_rec.traffic_class = (u8) (be32_to_cpu(sin6->sin6_flowinfo) >> 20); > + comp_mask |= IB_SA_PATH_REC_FLOW_LABEL | > + IB_SA_PATH_REC_TRAFFIC_CLASS; > + } > + > id_priv->query_id = ib_sa_path_rec_get(&sa_client, id_priv->id.device, > - id_priv->id.port_num, &path_rec, > - IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | > - IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH | > - IB_SA_PATH_REC_REVERSIBLE, > - timeout_ms, GFP_KERNEL, > - cma_query_handler, work, &id_priv->query); > + id_priv->id.port_num, &path_rec, > + comp_mask, timeout_ms, > + GFP_KERNEL, cma_query_handler, > + work, &id_priv->query); > > return (id_priv->query_id < 0) ? id_priv->query_id : 0; > } > diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h > index 2d6a770..010f876 100644 > --- a/include/rdma/rdma_cm.h > +++ b/include/rdma/rdma_cm.h > @@ -314,4 +314,18 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr, > */ > void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr); > > +/** > + * rdma_set_service_type - Set the type of service associated with a > + * connection identifier. > + * @id: Communication identifier to associated with service type. > + * @tos: Type of service. > + * > + * The type of service is interpretted as a differentiated service > + * field (RFC 2474). The service type should be specified before > + * performing route resolution, as existing communication on the > + * connection identifier may be unaffected. The type of service > + * requested may not be supported by the network to all destinations. > + */ > +void rdma_set_service_type(struct rdma_cm_id *id, int tos); > + > #endif /* RDMA_CM_H */ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From swise at opengridcomputing.com Tue Aug 7 07:37:41 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 07 Aug 2007 09:37:41 -0500 Subject: [ofa-general] [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. Message-ID: <46B883B5.8040702@opengridcomputing.com> Networking experts, I'd like input on the patch below, and help in solving this bug properly. iWARP devices that support both native stack TCP and iWARP (aka RDMA over TCP/IP/Ethernet) connections on the same interface need the fix below or some similar fix to the RDMA connection manager. This is a BUG in the Linux RDMA-CMA code as it stands today. Here is the issue: Consider an mpi cluster running mvapich2. And the cluster runs MPI/Sockets jobs concurrently with MPI/RDMA jobs. It is possible, without the patch below, for MPI/Sockets processes to mistakenly get incoming RDMA connections and vice versa. The way mvapich2 works is that the ranks all bind and listen to a random port (retrying new random ports if the bind fails with "in use"). Once they get a free port and bind/listen, they advertise that port number to the peers to do connection setup. Currently, without the patch below, the mpi/rdma processes can end up binding/listening to the _same_ port number as the mpi/sockets processes running over the native tcp stack. This is due to duplicate port spaces for native stack TCP and the rdma cm's RDMA_PS_TCP port space. If this happens, then the connections can get screwed up. The correct solution in my mind is to use the host stack's TCP port space for _all_ RDMA_PS_TCP port allocations. The patch below is a minimal delta to unify the port spaces by using the kernel stack to bind ports. This is done by allocating a kernel socket and binding to the appropriate local addr/port. It also allows the kernel stack to pick ephemeral ports by virtue of just passing in port 0 on the kernel bind operation. There has been a discussion already on the RDMA list if anyone is interested: http://www.mail-archive.com/general at lists.openfabrics.org/msg05162.html Thanks, Steve. --- RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. This is needed for iwarp providers that support native and rdma connections over the same interface. Signed-off-by: Steve Wise --- drivers/infiniband/core/cma.c | 27 ++++++++++++++++++++++++++- 1 files changed, 26 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 9e0ab04..e4d2d7f 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -111,6 +111,7 @@ struct rdma_id_private { struct rdma_cm_id id; struct rdma_bind_list *bind_list; + struct socket *sock; struct hlist_node node; struct list_head list; struct list_head listen_list; @@ -695,6 +696,8 @@ static void cma_release_port(struct rdma kfree(bind_list); } mutex_unlock(&lock); + if (id_priv->sock) + sock_release(id_priv->sock); } void rdma_destroy_id(struct rdma_cm_id *id) @@ -1790,6 +1793,25 @@ static int cma_use_port(struct idr *ps, return 0; } +static int cma_get_tcp_port(struct rdma_id_private *id_priv) +{ + int ret; + struct socket *sock; + + ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); + if (ret) + return ret; + ret = sock->ops->bind(sock, + (struct socketaddr *)&id_priv->id.route.addr.src_addr, + ip_addr_size(&id_priv->id.route.addr.src_addr)); + if (ret) { + sock_release(sock); + return ret; + } + id_priv->sock = sock; + return 0; +} + static int cma_get_port(struct rdma_id_private *id_priv) { struct idr *ps; @@ -1801,6 +1823,9 @@ static int cma_get_port(struct rdma_id_p break; case RDMA_PS_TCP: ps = &tcp_ps; + ret = cma_get_tcp_port(id_priv); /* Synch with native stack */ + if (ret) + goto out; break; case RDMA_PS_UDP: ps = &udp_ps; @@ -1815,7 +1840,7 @@ static int cma_get_port(struct rdma_id_p else ret = cma_use_port(ps, id_priv); mutex_unlock(&lock); - +out: return ret; } From johnpol at 2ka.mipt.ru Tue Aug 7 07:54:41 2007 From: johnpol at 2ka.mipt.ru (Evgeniy Polyakov) Date: Tue, 7 Aug 2007 18:54:41 +0400 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B883B5.8040702@opengridcomputing.com> References: <46B883B5.8040702@opengridcomputing.com> Message-ID: <20070807145441.GA24895@2ka.mipt.ru> Hi Steve. On Tue, Aug 07, 2007 at 09:37:41AM -0500, Steve Wise (swise at opengridcomputing.com) wrote: > +static int cma_get_tcp_port(struct rdma_id_private *id_priv) > +{ > + int ret; > + struct socket *sock; > + > + ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); > + if (ret) > + return ret; > + ret = sock->ops->bind(sock, > + (struct socketaddr > *)&id_priv->id.route.addr.src_addr, > + ip_addr_size(&id_priv->id.route.addr.src_addr)); If get away from talks about broken offloading, this one will result in the case, when usual network dataflow can enter private rdma land, i.e. after bind succeeded this socket is accessible via any other network device. Is it inteded? And this is quite noticeble overhead per rdma connection, btw. -- Evgeniy Polyakov From dotanb at dev.mellanox.co.il Tue Aug 7 08:09:40 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 07 Aug 2007 18:09:40 +0300 Subject: [ofa-general] Re: virtual memory usage on long lived apps In-Reply-To: <46B87FC5.6030104@opengridcomputing.com> References: <46B78081.3060305@opengridcomputing.com> <46B87FC5.6030104@opengridcomputing.com> Message-ID: <46B88B34.7050700@dev.mellanox.co.il> Steve Wise wrote: > > > Roland Dreier wrote: >> > Running this server with clients repeatedly connecting, running >> > rpings, and disconnecting, I notice that the virtual memory consumed >> > by the process is ever increasing. The shared/and reserved values >> > don't increase, but the virtual does. IE: If you watch this process >> > via 'top' you'll see the VIRT value increasing steadily over >> > time/connections, and the RES/SHR values stay steady. I've seen it >> > over 1GB of virtual memory used! >> >> > Q: Have you seen this before? Its like the OS never reuses mappings >> > that were setup for memory registrations. Is that normal? Or should >> > the virtual size shown in 'top' be reduced whenever a buffer is >> > unregistered and freed? There's a definite possibility the code is >> > leaking memory, but I'm wondering if this is perhaps a normal anomaly >> > with mapping memory in linux... >> >> I've not really tested an app like that, but I would expect that >> freeing the memory should prevent your virtual space from growing. I >> would recheck your code and maybe add some tracing to make sure that >> everything really does get freed -- or try the libibverbs valgrind >> support ;) >> > > I'll instrument the code and see if I can find the leak... > > What is valgrind (sounds painful :) ? > Valgrind is a freeware tool to detect many memory bugs (including buffer overrun/underrun/uninitialized and memory leaks). It can be found in: http://www.valgrind.org/ Dotan From swise at opengridcomputing.com Tue Aug 7 08:06:29 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 07 Aug 2007 10:06:29 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070807145441.GA24895@2ka.mipt.ru> References: <46B883B5.8040702@opengridcomputing.com> <20070807145441.GA24895@2ka.mipt.ru> Message-ID: <46B88A75.3040004@opengridcomputing.com> Evgeniy Polyakov wrote: > Hi Steve. > > On Tue, Aug 07, 2007 at 09:37:41AM -0500, Steve Wise (swise at opengridcomputing.com) wrote: >> +static int cma_get_tcp_port(struct rdma_id_private *id_priv) >> +{ >> + int ret; >> + struct socket *sock; >> + >> + ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); >> + if (ret) >> + return ret; >> + ret = sock->ops->bind(sock, >> + (struct socketaddr >> *)&id_priv->id.route.addr.src_addr, >> + ip_addr_size(&id_priv->id.route.addr.src_addr)); > > If get away from talks about broken offloading, this one will result in > the case, when usual network dataflow can enter private rdma land, i.e. > after bind succeeded this socket is accessible via any other network > device. Is it inteded? > And this is quite noticeble overhead per rdma connection, btw. > I'm not sure I understand your question? What do you mean by "accessible"? The intention is to _just_ reserve the addr/port. The socket struct alloc and bind was a simple way to do this. I assume we'll have to come up with a better way though. Namely provide a low level interface to the port space allocator allowing both rdma and the host tcp stack to share the space without requiring a socket struct for rdma connections. Or maybe we'll come up a different and better solution to this issue... Steve. From johnpol at 2ka.mipt.ru Tue Aug 7 08:39:27 2007 From: johnpol at 2ka.mipt.ru (Evgeniy Polyakov) Date: Tue, 7 Aug 2007 19:39:27 +0400 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B88A75.3040004@opengridcomputing.com> References: <46B883B5.8040702@opengridcomputing.com> <20070807145441.GA24895@2ka.mipt.ru> <46B88A75.3040004@opengridcomputing.com> Message-ID: <20070807153927.GA28648@2ka.mipt.ru> On Tue, Aug 07, 2007 at 10:06:29AM -0500, Steve Wise (swise at opengridcomputing.com) wrote: > >On Tue, Aug 07, 2007 at 09:37:41AM -0500, Steve Wise > >(swise at opengridcomputing.com) wrote: > >>+static int cma_get_tcp_port(struct rdma_id_private *id_priv) > >>+{ > >>+ int ret; > >>+ struct socket *sock; > >>+ > >>+ ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); > >>+ if (ret) > >>+ return ret; > >>+ ret = sock->ops->bind(sock, > >>+ (struct socketaddr > >>*)&id_priv->id.route.addr.src_addr, > >>+ ip_addr_size(&id_priv->id.route.addr.src_addr)); > > > >If get away from talks about broken offloading, this one will result in > >the case, when usual network dataflow can enter private rdma land, i.e. > >after bind succeeded this socket is accessible via any other network > >device. Is it inteded? > >And this is quite noticeble overhead per rdma connection, btw. > > > > I'm not sure I understand your question? What do you mean by > "accessible"? The intention is to _just_ reserve the addr/port. Above RDMA ->bind() ends up with tcp_v4_get_port(), which will only add socket into bhash, but it is only accessible for new sockets created for listening connections or expilicit bind, network traffic checks only listening and establised hashes, which are not affected by above change, so it was false alarm from my side. It does allow to 'grab' a port and forbid its possible reuse. -- Evgeniy Polyakov From mshefty at ichips.intel.com Tue Aug 7 09:13:47 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:13:47 -0700 Subject: [ofa-general] [PATCH 4/4] rdma/ucm: export setting service type to user space In-Reply-To: References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <000301c7d888$81646260$9c98070a@amr.corp.intel.com> Message-ID: <46B89A3B.9020600@ichips.intel.com> > For symmetry, what about implementing get as well ? In sockets, TOS is > both set/get socket option. I considered that, and there is a place holder for it, but I don't think it's needed at this point. I don't have strong feelings against it if people would like it added. - Sean From mshefty at ichips.intel.com Tue Aug 7 09:16:44 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:16:44 -0700 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service In-Reply-To: <46B8388E.1090308@voltaire.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> Message-ID: <46B89AEC.1020904@ichips.intel.com> > I think the correct way here, is a SID --> QoS mapping. This allows you > to manage and configure QoS in --one-- place, namely the SM/SA and zero > effort host side deployment. SID doesn't work for iWarp. If an IPv6 address is used, then we have the information necessary (traffic class and flow label) to support QoS. If an IPv4 address is used, we mimic sockets. - Sean From mshefty at ichips.intel.com Tue Aug 7 09:18:38 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:18:38 -0700 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify type of service In-Reply-To: <46B8821D.4050608@opengridcomputing.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8821D.4050608@opengridcomputing.com> Message-ID: <46B89B5E.3070309@ichips.intel.com> Steve Wise wrote: > Don't forget the man page for the new function... It's included it with the userspace changes. - Sean From mshefty at ichips.intel.com Tue Aug 7 09:22:11 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:22:11 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <46B838D6.6000501@voltaire.com> References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> <46B838D6.6000501@voltaire.com> Message-ID: <46B89C33.4030709@ichips.intel.com> > iser is rdma cm based, the only way for iser to use the sid is if the > rdma cm allows it to do so, with this patch series its not possible. I'm really not that familiar with the iSER code yet - I just started looking at it. But wouldn't it already have to support something like setsockopt? - Sean From mshefty at ichips.intel.com Tue Aug 7 09:28:10 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:28:10 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <46B8439C.6010303@voltaire.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> Message-ID: <46B89D9A.4080504@ichips.intel.com> > Sean, I think you want to ask for a specific SL here (that is add > IB_SA_PATH_REC_SL to the mask and ask for the broadcast group SL). My thoughts are that the traffic class and flow label should map to a specific SL. Consider a packet entering the local subnet through an IB router. The local subnet SL would need to be determined using the TC and FL. - Sean From sashak at voltaire.com Tue Aug 7 09:35:06 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 7 Aug 2007 19:35:06 +0300 Subject: [ofa-general] Re: [PATCH] osm: default leaf vl_stall and head_of_queue_lifecounters In-Reply-To: <6C2C79E72C305246B504CBA17B5500C90208EE72@mtlexch01.mtl.com> References: <46B78D04.7020002@dev.mellanox.co.il> <1186445588.7524.10.camel@localhost> <6C2C79E72C305246B504CBA17B5500C90208EE72@mtlexch01.mtl.com> Message-ID: <20070807163506.GD32692@sashak.voltaire.com> Hi Amit, On 10:24 Tue 07 Aug , Amit Krig wrote: > Hi Sasha > > In the current switch configurations if you have a single drop (By HOQ), > the line will be flashed for 128 Mili seconds, I think that this penalty > for ~8mili hiccup (as the HOQ can count ~50% less then configured) is > not a good default, Ok. Let's start from less aggressive default values - HoQ 256msec and VLStallCount 7 (don't need to resubmit the patch for this). Both parameters are configurable anyway and I think it would be nice to make deeper investigation about potentially more optimal default values. Sasha From amitk at mellanox.co.il Tue Aug 7 09:39:43 2007 From: amitk at mellanox.co.il (Amit Krig) Date: Tue, 7 Aug 2007 19:39:43 +0300 Subject: [ofa-general] RE: [PATCH] osm: default leaf vl_stall andhead_of_queue_lifecounters In-Reply-To: <20070807163506.GD32692@sashak.voltaire.com> References: <46B78D04.7020002@dev.mellanox.co.il> <1186445588.7524.10.camel@localhost> <6C2C79E72C305246B504CBA17B5500C90208EE72@mtlexch01.mtl.com> <20070807163506.GD32692@sashak.voltaire.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C90208EE87@mtlexch01.mtl.com> I agree -----Original Message----- From: Sasha Khapyorsky [mailto:sashak at voltaire.com] Sent: Tuesday, August 07, 2007 7:35 PM To: Amit Krig Cc: kliteyn at dev.mellanox.co.il; OpenIB Subject: Re: [PATCH] osm: default leaf vl_stall andhead_of_queue_lifecounters Hi Amit, On 10:24 Tue 07 Aug , Amit Krig wrote: > Hi Sasha > > In the current switch configurations if you have a single drop (By > HOQ), the line will be flashed for 128 Mili seconds, I think that this > penalty for ~8mili hiccup (as the HOQ can count ~50% less then > configured) is not a good default, Ok. Let's start from less aggressive default values - HoQ 256msec and VLStallCount 7 (don't need to resubmit the patch for this). Both parameters are configurable anyway and I think it would be nice to make deeper investigation about potentially more optimal default values. Sasha From mshefty at ichips.intel.com Tue Aug 7 09:42:24 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:42:24 -0700 Subject: [ofa-general] QoS RFC In-Reply-To: <46B83086.6090202@voltaire.com> References: <46A283B6.1070105@dev.mellanox.co.il> <46A94657.1020101@ichips.intel.com> <46AF6281.10709@ichips.intel.com> <46B58E8E.1040001@voltaire.com> <46B751BA.1050800@ichips.intel.com> <46B83086.6090202@voltaire.com> Message-ID: <46B8A0F0.4080001@ichips.intel.com> > I find it more constructive to have a discussion based on a short host > side architecture description that covers at least QoS for IPoIB and > RDMA CM based ULPs, rather then based on series of patches. I responded to the original RFC: http://lists.openfabrics.org/pipermail/general/2007-July/thread.html started new threads: http://lists.openfabrics.org/pipermail/general/2007-July/038665.html http://lists.openfabrics.org/pipermail/general/2007-July/038809.html It's up to others to provide input. I have path forward deliverables that require actual code. Adding a service ID to the RDMA CM, or SL/MTU/whatever to IPoIB are minor changes. However, I'm not convinced yet that either are desirable. - Sean From mshefty at ichips.intel.com Tue Aug 7 09:47:32 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 09:47:32 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> Message-ID: <46B8A224.2020504@ichips.intel.com> > Just making sure. > The SL should also be copied from the multicast group and be provided > with the PR query. > So should be the rate. Can you clarify why you think the SL and rate should be provided, versus just the traffic class and flow label? - Sean From sashak at voltaire.com Tue Aug 7 09:55:21 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 7 Aug 2007 19:55:21 +0300 Subject: [ofa-general] Re: [PATCH] osm: default leaf vl_stall and head_of_queue_life counters In-Reply-To: <46B78D04.7020002@dev.mellanox.co.il> References: <46B78D04.7020002@dev.mellanox.co.il> Message-ID: <20070807165521.GF32692@sashak.voltaire.com> On 00:05 Tue 07 Aug , Yevgeny Kliteynik wrote: > Hi Sasha > > This patch changes the OSM_DEFAULT_LEAF_VL_STALL_COUNT and > OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE (matching them to the > default counters on the switch-to-switch ports), in order > to deal with the casual PCI Express "hiccups". > > Please apply it to ofed_1_2 and master. > > -- > Yevgeny > > Signed-off-by: Yevgeny Kliteynik Applied with discussed changes (leaf head_of_queue_life is 0x10) to both master and ofed_1_2. Thanks. Sasha From eitan at mellanox.co.il Tue Aug 7 09:59:46 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 7 Aug 2007 19:59:46 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <46B8A224.2020504@ichips.intel.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> <46B8A224.2020504@ichips.intel.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD8DF@mtlexch01.mtl.com> > -----Original Message----- > From: Sean Hefty [mailto:mshefty at ichips.intel.com] > Sent: Tuesday, August 07, 2007 7:48 PM > To: Eitan Zahavi > Cc: Sean Hefty; OpenIB > Subject: Re: [ofa-general] [PATCH 1/4] ib/ipoib: specify > TClass and FlowLabelwith PR queries for QoS support > > > Just making sure. > > The SL should also be copied from the multicast group and > be provided > > with the PR query. > > So should be the rate. > > Can you clarify why you think the SL and rate should be > provided, versus just the traffic class and flow label? The broadcast group of the IPoIB subnet should dictate the parameters to be used for that subnet. > > - Sean > From sean.hefty at intel.com Tue Aug 7 10:05:17 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 10:05:17 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BD8DF@mtlexch01.mtl.com> Message-ID: <000101c7d915$217f9b20$1acc180a@amr.corp.intel.com> >The broadcast group of the IPoIB subnet should dictate the parameters to >be used for that subnet. What I want to understand is why aren't the traffic class and flow label fields sufficient for this purpose? Things like rate and MTU are not carried in the IB headers, and multicast groups that span IB subnets may use different SL values on different subnets. The change is trivial to make, but I still want to understand why it should be made. - Sean From mshefty at ichips.intel.com Tue Aug 7 10:15:26 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 10:15:26 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <46B838D6.6000501@voltaire.com> References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> <46B838D6.6000501@voltaire.com> Message-ID: <46B8A8AE.5030002@ichips.intel.com> > iser is rdma cm based, the only way for iser to use the sid is if the > rdma cm allows it to do so, with this patch series its not possible. Can someone explain how iser and srp find the target devices? - Sean From jgunthorpe at obsidianresearch.com Tue Aug 7 10:34:33 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Tue, 7 Aug 2007 11:34:33 -0600 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BD8DF@mtlexch01.mtl.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> <46B8A224.2020504@ichips.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD8DF@mtlexch01.mtl.com> Message-ID: <20070807173433.GB8504@obsidianresearch.com> On Tue, Aug 07, 2007 at 07:59:46PM +0300, Eitan Zahavi wrote: > > Can you clarify why you think the SL and rate should be > > provided, versus just the traffic class and flow label? > The broadcast group of the IPoIB subnet should dictate the parameters to > be used for that subnet. There is no reason a unicast IPoIB path should be restricted to having the same SL as the broadcast group. That limits the possible topologies that can be routed efficiently. Sean's use of the tclass to derive the SL seems correct to me. Except in unusual cases the SL should not be specified in a PR query. Rate and MTU should probably be copied over. Not sure about flowlabel though.. Jason From eitan at mellanox.co.il Tue Aug 7 10:41:54 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 7 Aug 2007 20:41:54 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabelwith PR queries for QoS support In-Reply-To: <20070807173433.GB8504@obsidianresearch.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD699@mtlexch01.mtl.com> <46B8A224.2020504@ichips.intel.com> <6C2C79E72C305246B504CBA17B5500C9020BD8DF@mtlexch01.mtl.com> <20070807173433.GB8504@obsidianresearch.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD8F8@mtlexch01.mtl.com> Hi Jason, Sean, Your mail triggered me to double check the IPoIB for the sentence requiring the SL and rate to follow the MCG. I was sure I have seen this in the IETF IPoIB spec and that was the reason for my comments. But I can't find it there. So I must admit your argument regarding having the SL and rate be calculated by the PathRecord for the unicast traffic seems reasonable to me. Eitan > -----Original Message----- > From: Jason Gunthorpe [mailto:jgunthorpe at obsidianresearch.com] > Sent: Tuesday, August 07, 2007 8:35 PM > To: Eitan Zahavi > Cc: Sean Hefty; OpenIB > Subject: Re: [ofa-general] [PATCH 1/4] ib/ipoib: specify > TClass and FlowLabelwith PR queries for QoS support > > On Tue, Aug 07, 2007 at 07:59:46PM +0300, Eitan Zahavi wrote: > > > > Can you clarify why you think the SL and rate should be provided, > > > versus just the traffic class and flow label? > > > The broadcast group of the IPoIB subnet should dictate the > parameters > > to be used for that subnet. > > There is no reason a unicast IPoIB path should be restricted > to having the same SL as the broadcast group. That limits the > possible topologies that can be routed efficiently. Sean's > use of the tclass to derive the SL seems correct to me. > Except in unusual cases the SL should not be specified in a PR query. > > Rate and MTU should probably be copied over. Not sure about > flowlabel though.. > > Jason > From rdreier at cisco.com Tue Aug 7 10:42:12 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 10:42:12 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <46B8A8AE.5030002@ichips.intel.com> (Sean Hefty's message of "Tue, 07 Aug 2007 10:15:26 -0700") References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> <46B838D6.6000501@voltaire.com> <46B8A8AE.5030002@ichips.intel.com> Message-ID: > Can someone explain how iser and srp find the target devices? SRP uses IB device management to discover target ports. This is done in userspace and passed into the kernel - the kernel gets the dest GID, P_Key and service ID to use, and looks up a path record to actually connect (using the IB CM, since SRP is "native" on IB and doesn't use IP addressing at all, so the RDMA CM doesn't apply). - R. From sean.hefty at intel.com Tue Aug 7 10:43:22 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 10:43:22 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070807173433.GB8504@obsidianresearch.com> Message-ID: <000201c7d91a$73581d00$1acc180a@amr.corp.intel.com> >Rate and MTU should probably be copied over. Not sure about flowlabel >though.. Can you provide more details on your thinking for rate, mtu, and flowlabel? - Sean From sean.hefty at intel.com Tue Aug 7 10:49:29 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 10:49:29 -0700 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: Message-ID: <000301c7d91b$4e2a2e00$1acc180a@amr.corp.intel.com> >SRP uses IB device management to discover target ports. This is done >in userspace and passed into the kernel - the kernel gets the dest >GID, P_Key and service ID to use, and looks up a path record to >actually connect (using the IB CM, since SRP is "native" on IB and >doesn't use IP addressing at all, so the RDMA CM doesn't apply). Thanks. Given how SRP is structured, I think it's easiest for it to include the service ID with the PR query. If that makes sense, I can create a patch for it in a couple of minutes. (SRP is way easier for me to follow than iSER.) I just can't test SRP. The only alternative I can think of is for userspace to provide the QoS level along with the GID, PKey, etc. - Sean From jgunthorpe at obsidianresearch.com Tue Aug 7 10:50:55 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Tue, 7 Aug 2007 11:50:55 -0600 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <000201c7d91a$73581d00$1acc180a@amr.corp.intel.com> References: <20070807173433.GB8504@obsidianresearch.com> <000201c7d91a$73581d00$1acc180a@amr.corp.intel.com> Message-ID: <20070807175055.GD8504@obsidianresearch.com> On Tue, Aug 07, 2007 at 10:43:22AM -0700, Sean Hefty wrote: > >Rate and MTU should probably be copied over. Not sure about flowlabel > >though.. > > Can you provide more details on your thinking for rate, mtu, and flowlabel? Well, upon more though, I think rate can probably be dropped from the list. The broadcast group rate should be the minimum rate that any pair of ports can support in the IPoIB group, but that doesn't mean all connections should be limited to that rate. MTU - at a minimum the PR must request a MTU >= than the interface MTU, having a MTU that is exactly equal to the broadcast group MTU seems fine to me. The SM should be able to do >= internally. Flowlabel - IBA hasn't really specified what to do with this field. In IPv6 FlowLabel is assigned randomly by the host on a per-flow (ie socket) basis to allow routers to group packets into flows without doing deeper inspection. This becomes important when encryption is used and the routers can no longer access the TCP header to find flows. Also, my previous arguments on this list about routing and PR suggested using FlowLabel as a way to disambiguate multipath in a routed network. If that view is taken up then FlowLabel is purely an output from the SM in all cases and should not be an input - much like SL. Jason From hal.rosenstock at gmail.com Tue Aug 7 10:56:44 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 7 Aug 2007 13:56:44 -0400 Subject: [ofa-general] QoS RFC In-Reply-To: <46B24F7C.8080008@dev.mellanox.co.il> References: <46A283B6.1070105@dev.mellanox.co.il> <46B24F7C.8080008@dev.mellanox.co.il> Message-ID: Hi Yevgeny, On 8/2/07, Yevgeny Kliteynik wrote: > Hi Hal, > > Hal Rosenstock wrote: > > Hi Yevgeny, > > > > On 7/21/07, Yevgeny Kliteynik wrote: > >> Hi All > >> > >> Please find the attached RFC describing how QoS policy support could be implemented in the OpenFabrics stack. > >> Your comments are welcome. > > > > A couple of quick questions: > > > > How does this differ from the original RFC posted 5/30/06 ? > > > > What I can see is the following: > > 1. Updated for not yet released IBTA QoS Annex > > 2. Use of plain text rather than XML based policy file for OpenSM > > Anything else ? > > You're absolutely right - these are the only changes (plus cosmetics here and there). > > > Below, IPoIB is discussed in terms of UD. What about IPoIB-CM ? It > > uses CM and has a service ID. Will IPoIB CM be added to the RFC document or is it the same as UD ? > > > > Also, have my specific comments to the patches originally submitted > > been addressed ? (Do I need to dig them out again ?) Just wondering... > > Yes. The submitted patches were only QoS policy file parser. > In the new parser I took care of all the issues we've discussed > a couple of months ago. Here is the summary of these issues > taken from our discussion: > > [snip] > XML syntax or not -> Plain text, human readable and easily editable > > QoS syntax explanation/discussion > > changes to some keywords > Port r.t. Node Groups -> DONE: fixed keywords > CA r.t. HCA -> DONE: fixed keywords > QoSClass r.t. TClass -> DONE: fixed keywords > syntax discussion points > larger ones: > dynamic service IDs -> supported through list and range support > service ID range support -> DONE: added to the matching rules examples > port groups shared with partition configuration (future) -> agree, it would be a good > idea to share port groups with partition configuration, > but it won't be for OFED 1.3 > multicast -> not planned for OFED 1.3, but we'll discuss it later > smaller ones: > across syntax explanation -> DONE: see the explanation and an example in the policy file > What is sn in the syntax short for ? -> it was for "serial number", replaced by > "qos-level-sn". It means the serial (sequential) number > of the qos-level that should be applied to PathRecords > that matches this qos-match-rule. I probably will change > the "qol-level-sn" to "qos-level-name" to refer QoS level > by name rather than by sn. > path bits explanation -> Path bits are part of QoS level. > They can be used to "differentiate" paths through the subnet > to a port when LMC>0. It won't be implemented yet for > OFED 1.3, and OpenSM should issue a warning if it finds > PathBits in QoS level definition in the policy file > packet lifetime ? -> DONE: added packet-life keyword > > > viewer/editor -> Since we've switched to plain text, this one becomes irrelevant Thanks. -- Hal > [/snip] > > > -- Yevgeny > > > > > Thanks. > > > > -- Hal > > > > > > > > > >> -- Yevgeny > >> > >> RFC: OpenFabrics Enhancements for QoS Support > >> =============================================== > >> > >> Authors: . Eitan Zahavi > >> Authors: . Yevgeny Kliteynik > >> Date: .... Jul 2007. > >> Revision: 0.2 > >> > >> Table of contents: > >> 1. Overview > >> 2. Architecture > >> 3. Supported Policy > >> 4. CMA functionality > >> 5. IPoIB functionality > >> 6. SDP functionality > >> 7. SRP functionality > >> 8. iSER functionality > >> 9. OpenSM functionality > >> > >> 1. Overview > >> ------------ > >> Quality of Service requirements stem from the realization of I/O consolidation > >> over IB network: As multiple applications and ULPs share the same fabric, means > >> to control their use of the network resources are becoming a must. The basic > >> need is to differentiate the service levels provided to different traffic flows, > >> such that a policy could be enforced and control each flow utilization of the > >> fabric resources. > >> > >> IBTA specification defined several hardware features and management interfaces > >> to support QoS: > >> * Up to 15 Virtual Lanes (VL) carry traffic in a non-blocking manner > >> * Arbitration between traffic of different VLs is performed by a 2 priority > >> levels weighted round robin arbiter. The arbiter is programmable with > >> a sequence of (VL, weight) pairs and maximal number of high priority credits > >> to be processed before low priority is served > >> * Packets carry class of service marking in the range 0 to 15 in their > >> header SL field > >> * Each switch can map the incoming packet by its SL to a particular output > >> VL based on programmable table VL=SL-to-VL-MAP(in-port, out-port, SL) > >> * The Subnet Administrator controls each communication flow parameters > >> by providing them as a response to Path Record (PR) or MultiPathRecord (MPR) > >> queries > >> > >> The IB QoS features provide the means to implement a DiffServ like architecture. > >> DiffServ architecture (IETF RFC2474 2475) is widely used today in highly dynamic > >> fabrics. > >> > >> This proposal provides the detailed functional definition for the various > >> software elements that are required to enable a DiffServ like architecture over > >> the OpenFabrics software stack. > >> > >> > >> > >> 2. Architecture > >> ---------------- > >> This proposal split the QoS functionality between the SM/SA, CMA and the various > >> ULPS. We take the "chronology approach" to describe how the overall system > >> works: > >> > >> 2.1. The network manager (human) provides a set of rules (policy) that defines > >> how the network is being configured and how its resources are split to different > >> QoS-Levels. The policy also define how to decide which QoS-Level each > >> application or ULP or service use. > >> > >> 2.2. The SM analyzes the provided policy to see if it is realizable and performs > >> the necessary fabric setup. The SM may continuously monitor the policy and adapt > >> to changes in it. Part of this policy defines the default QoS-Level of each > >> partition. The SA is being enhanced to match the requested Source, Destination, > >> QoS-Class, Service-ID (and optionally SL and priority) against the policy. So > >> clients (ULPs, programs) can obtain a policy enforced QoS. The SM is also > >> enhanced to support setting up partitions with appropriate IPoIB broadcast > >> group. This broadcast group carries its QoS attributes: SL, MTU and > >> RATE. > >> > >> 2.3. IPoIB is being setup. IPoIB uses the SL, MTU and RATE available on the > >> multicast group which forms the broadcast group of this partition. > >> > >> 2.4. MPI which provides non IB based connection management should be configured > >> to run using hard coded SLs. It uses these SLs for every QP being opened. > >> > >> 2.5. ULPs that use CM interface (like SRP) should have their own pre-assigned > >> Service-ID and use it while obtaining PR/MPR for establishing connections. > >> The SA receiving the PR/MPR should match it against the policy and return > >> the appropriate PR/MPR including SL, MTU and RATE. > >> > >> 2.6. ULPs and programs using CMA to establish RC connection should provide the > >> CMA the target IP and Service-ID. Some of the ULPs might also provide QoS-Class > >> (E.g. for SDP sockets that are provided the TOS socket option). The CMA should > >> then use the provided Service-ID and optional QoS-Class and pass them in the > >> PR/MPR request. The resulting PR/MPR should be used for configuring the > >> connection QP. > >> > >> PathRecord and MultiPathRecord enhancement for QoS: > >> As mentioned above the PathRecord and MultiPathRecord attributes should be > >> enhanced to carry the Service-ID which is a 64bit value, which has been > >> standardized by the IBTA. A new field QoS-Class is also provided. > >> A new capability bit should describe the SM QoS support in the SA class port > >> info. This approach provides an easy migration path for existing access layer > >> and ULPs by not introducing new set of PR/MPR attribute. > >> > >> > >> 3. Supported Policy > >> -------------------- > >> > >> The QoS policy supported by this proposal is divided into 4 sub sections: > >> > >> I) Port Group: a set of CAs, Routers or Switches that share the same settings. > >> A port group might be a partition defined by the partition manager policy in > >> terms of GUIDs. Future implementations might provide support for NodeDescription > >> based definition of port groups. > >> > >> II) Fabric Setup: > >> Defines how the SL2VL and VLArb tables should be setup. This policy definition > >> assumes the computation of overall end to end network behavior should be performed > >> outside of OpenSM. > >> > >> III) QoS-Levels Definition: > >> This section defines the possible sets of parameters for QoS that a client > >> might be mapped to. Each set holds: SL and optionally: Max MTU, Max Rate, > >> Packet Lifetime and Path Bits (in case LMC > 0 is used for QoS). > >> > >> IV) Matching Rules: > >> A list of rules that match an incoming PR/MPR request to a QoS-Level. The > >> rules are processed in order such as the first match is applied. Each rule is > >> built out of a set of match expressions which should all match for the rule to > >> apply. The matching expressions are defined for the following fields > >> ** SRC and DST to lists of port groups > >> ** Service-ID to a list of Service-ID or Service-ID ranges > >> ** QoS-Class to a list of QoS-Class values or ranges > >> > >> QoS Policy file syntax > >> > >> * Empty lines are ignored > >> * Leading and trailing blanks, as well as empty lines, are ignored, so the > >> indentation in the example is just for better readability > >> * Comments are started with the pound sign (#) and terminated by EOL > >> * Comments may appear only in a separate line > >> * Keywords that denote section/subsection start have matching closing keywords > >> * Any keyword should be the first non-blank in the line > >> > >> QoS Policy file example > >> > >> # Port Groups define sets of ports to be used later in the settings > >> port-groups > >> # using port GUIDs > >> port-group > >> name: Storage > >> # "use" is just a description that is used for logging. > >> # Other than that, it is just a commentary > >> use: our SRP storage targets > >> port-guid: 0x1000000000000001 > >> port-guid: 0x1000000000000002 > >> end-port-group > >> > >> port-group > >> name: Virtual Servers > >> use: node desc and IB port num > >> # The syntax of the port name is as follows: "hostname/CA-num/Pnum". > >> # "hostname" and "CA-num" are compared to the first 2 words of > >> # NodeDescription, and "Pnum" is a port number on that node. > >> port-name: vs1/HCA-1/P1 > >> port-name: vs3/HCA-1/P1 > >> port-name: vs3/HCA-2/P2 > >> end-port-group > >> > >> # using partitions defined in the partition policy > >> port-group > >> name: Group for Partition 1 > >> use: default settings > >> partition: Part1 > >> end-port-group > >> > >> # using node types CA|ROUTER|SWITCH > >> port-group > >> name: Routers > >> use: all routers > >> node-type: ROUTER > >> end-port-group > >> > >> end-port-groups > >> > >> qos-setup > >> > >> # define all types of VLArb tables. The length of the tables should > >> # match the physically supported tables by their target ports > >> vlarb-tables > >> # scope defines the exact ports the VLArb tables apply to > >> vlarb-scope > >> # defining VLArb tables on all the ports that belong to > >> # port group 'Storage', and on all the ports connected > >> # to ports of port group 'Storage' > >> group: Storage > >> # "across" means all the ports that are connected to ports > >> # that belong to the specified port group > >> across: Storage > >> # VLArb table holds VL and weight pairs > >> vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 > >> vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 > >> vl-high-limit: 10 > >> end-vlarb-scope > >> # There can be several scopes > >> end-vlarb-tables > >> > >> sl2vl-tables > >> # Scope defines the exact devices and in/out ports tables apply to. > >> # Note: if the same port is matching several rules the *FIRST* one applies. > >> sl2vl-scope > >> # SL2VL tables are orgnized as SL2VL(in-port,out-port) > >> # "from: n,m" means we define the SL2VL(n,*) and SL2VL(m,*) > >> # "to: n,m" means we define the SL2VL(*,n) and SL2VL(*,m) > >> # > >> # The following example specifies that all the SL2VL tables > >> # entries should be defined for all the ports of group Part1: > >> group: Part1 > >> from: * > >> to: * > >> # SL2VL table has to have 16 values at max - one for each SL. > >> # If the user specifies less than 16 values, all the missing > >> # VL values will be implicitly set to 0 > >> sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 > >> end-sl2vl-scope > >> > >> sl2vl-scope > >> # "across-to" is a combination of "across" keyword (definition can be found > >> # in VLArb tables section) and "to" keyword. > >> # "across: PortGroupName" refers to all the ports that are connected > >> # to ports that belong to PortGroupName. > >> # > >> # Example of "across-to" usage: > >> # A user has a set of 'special' nodes (e.g. storage nodes), and all > >> # the traffic to these nodes has to get specific VL. > >> # The solution is to define port group (i.g. "Storage") that will > >> # include all the ports of these nodes, and then to configure SL2VL > >> # tables on all the switch ports that are connected to the Storage > >> # port group by specifying "across-to: Storage". > >> # > >> across-to: Storage2 > >> # Similar to "across-to", "across-from" is a combination of "across" > >> # and "to" keywords > >> across-from: Storage1 > >> sl2vl-table: 0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0 > >> end-sl2vl-scope > >> end-sl2vl-tables > >> > >> end-qos-setup > >> > >> > >> qos-levels > >> > >> # the first one is just setting SL > >> qos-level > >> use: for the lowest priority communication > >> sl: 15 > >> packet-life: 16 > >> end-qos-level > >> # the second sets SL and QoS Class > >> qos-level > >> use: low latency best bandwidth > >> sl: 0 > >> end-qos-level > >> # the whole set: SL, MTU-Limit, Rate-Limit, Packet Lifetime, Path Bits > >> qos-level > >> use: just an example > >> sl: 0 > >> mtu-limit: 1 > >> rate-limit: 1 > >> packet-life: 12 > >> # Path Bits can be used e.g. to provide a different routes through the > >> # subnet to a particular port > >> path-bits: 2,4,8-32 > >> end-qos-level > >> > >> end-qos-levels > >> > >> > >> # Match rules are scanned in a first-fit manner (like firewall rules table) > >> qos-match-rules > >> > >> # matching by single criteria: class (list of values and ranges) > >> qos-match-rule > >> # just a description > >> use: low latency by class 7-9 or 11 > >> qos-class: 7-9,11 > >> # number of qos-level to apply to the matching PR/MPR > >> qos-level-sn: 1 > >> end-qos-match-rule > >> # show matching by destination group AND service-ids > >> qos-match-rule > >> use: Storage targets connection > >> destination: Storage > >> service-id: 22,4719-5000 > >> qos-level-sn: 2 > >> end-qos-match-rule > >> # show matching by source group only > >> qos-match-rule > >> use: bla bla > >> source: Storage > >> qos-level-sn: 3 > >> end-qos-match-rule > >> > >> end-qos-match-rules > >> > >> > >> 4. IPoIB > >> --------- > >> > >> IPoIB already query the SA for its broadcast group information. The additional > >> functionality required is for IPoIB to provide the broadcast group SL, MTU, > >> and RATE in every following PathRecord query performed when a new UDAV is > >> needed by IPoIB. > >> We could assign a special Service-ID for IPoIB use but since all communication > >> on the same IPoIB interface shares the same QoS-Level without the ability to > >> differentiate it by target service we can ignore it for simplicity. > >> > >> 5. CMA features > >> ---------------- > >> > >> The CMA interface supports Service-ID through the notion of port space as a > >> prefixes to the port_num which is part of the sockaddr provided to > >> rdma_resolve_add(). What is missing is the explicit request for a QoS-Class that > >> should allow the ULP (like SDP) to propagate a specific request for a class of > >> service. A mechanism for providing the QoS-Class is available in the IPv6 address, > >> so we could use that address field. Another option is to implement a special > >> connection options API for CMA. > >> > >> Missing functionality by CMA is the usage of the provided QoS-Class and Service-ID > >> in the sent PR/MPR. When a response is obtained it is an existing requirement for > >> the CMA to use the PR/MPR from the response in setting up the QP address vector. > >> > >> > >> 6. SDP > >> ------- > >> > >> SDP uses CMA for building its connections. > >> The Service-ID for SDP is 0x000000000001PPPP, where PPPP are 4 hex digits > >> holding the remote TCP/IP Port Number to connect to. > >> SDP might be provided with SO_PRIORITY socket option. In that case the value > >> provided should be sent to the CMA as the TClass option of that connection. > >> > >> 7. SRP > >> ------- > >> > >> Current SRP implementation uses its own CM callbacks (not CMA). So SRP should > >> fill in the Service-ID in the PR/MPR by itself and use that information in > >> setting up the QP. The T10 SRP standard defines the SRP Service-ID to be defined > >> by the SRP target I/O Controller (but they should also comply with IBTA Service- > >> ID rules). Anyway, the Service-ID is reported by the I/O Controller in the > >> ServiceEntries DMA attribute and should be used in the PR/MPR if the SA > >> reports its ability to handle QoS PR/MPRs. > >> > >> 8. iSER > >> -------- > >> iSER uses CMA and thus should be very close to SDP. The Service-ID for iSER > >> should be TBD. > >> > >> > >> 9. OpenSM features > >> ------------------- > >> The QoS related functionality to be provided by OpenSM can be split into two > >> main parts: > >> > >> 3.1. Fabric Setup > >> During fabric initialization the SM should parse the policy and apply its > >> settings to the discovered fabric elements. The following actions should be > >> performed: > >> * Parsing of policy > >> * Node Group identification. Warning should be provided for each node not > >> specified but found. > >> * SL2VL settings validation should be checked: > >> + A warning will be provided if there are no matching targets for the SL2VL > >> setting statement. > >> + An error message will be printed to the log file if an invalid setting is > >> found. A setting is invalid if it refers to: > >> - Non existing port numbers of the target devices > >> - Unsupported VLs for the target device. In the later case the map to non > >> existing VLs should be replaced to VL15 i.e. packets will be dropped. > >> * SL2VL setting is to be performed > >> * VL Arbitration table settings should be validated according to the following > >> rules: > >> + A warning will be provided if there are no matching targets for the setting > >> statement > >> + An error will be provided if the port number exceeds the target ports > >> + An error will be generated if the table length exceeds device capabilities > >> + A warning will be generated if the table quote a VL that is not supported > >> by the target device > >> * VL Arbitration tables will be set on the appropriate targets > >> > >> 3.2. PR/MPR query handling: > >> OpenSM should be able to enforce the provided policy on client request. > >> The overall flow for such requests is: first the request is matched against the > >> defined match rules such that the target QoS-Level definition is found. Given > >> the QoS-Level a path(s) search is performed with the given restrictions imposed > >> by that level. The following two sections describe these steps. > >> > >> How Service-ID is carried in the PathRecord and MultiPathRecord attributes is > >> now standardized by the IBTA. > >> > >> > >> 3.2.1. Matching rule search: > >> A rule is "matching" a PR/MPR request using the following criteria: > >> * Matching rules provide values in a list of either single value, or range of > >> values. A PR/MPR field is "matching" the rule field if it is explicitly > >> noted in the list of values or is one of the values covered by a range > >> included in the field values list. > >> * Only PR/MPR fields that have their component mask bit set should be > >> compared. > >> * For a rule to be "matching" a PR/MPR request all the rule fields should be > >> "matching" their PR/MPR fields. Such that a PR/MPR request that does > >> not have a component mask field set for one of the rule defined fields can > >> not match that rule. > >> * A PR/MPR request that have a component mask bit set for one of the fields > >> that is not defined by the rule can match the rule. > >> > >> The algorithm to be used for searching for a rule match might be as simple as a > >> sequential search through all rules or enhanced for better performance. The > >> semantics of every rule field and its matching PR/MPR field are described > >> below: > >> * Source: the SGID or SLID should be part of this group > >> * Destination: the DGID or DLID should be part of this group > >> * Service-ID: check if the requested Service-ID (available in the PR/MPR old > >> SM-Key field) is matching any of this rule Service-IDs > >> * TClass: check if the PR/MPR TClass field is matching > >> > >> 3.2.2 PR/MPR response generation: > >> The QoS-Level pointed by the first rule that matches the PR/MPR request > >> should be used for obtaining the response SL, MTU-Limit, RATE-Limit, Path-Bits > >> and QoS-Class. A default QoS-Level should be used if no rule is matching the query. > >> > >> The efficient algorithm for finding paths that meet the QoS-Level criteria is > >> beyond the scope of this RFC and left for the implementer to provide. However > >> the criteria by which the paths match the QoS-Level are described below: > >> > >> * SL: The paths found should all use the given SL. For that sake PR/MPR > >> algorithm should traverse the path from source to destination only through > >> ports that carry a valid VL (not VL15) by the SL2VL map (should consider input > >> and output ports and SL). > >> * MTU-Limit: The resulting paths MTU should not exceed the given MTU-Limit > >> * Rate-Limit: The resulting paths RATE should not exceed the given RATE-Limit > >> (rate limit is given in units of link BW = Width*Speed according to IBTA > >> Specification Vol-1 table-205 p-901 l-24). > >> * Path-Bits: define the target LID lowest bits (number of bits defined by the > >> target port PortInfo.LMC field). The path should traverse the LFT using the > >> target port LID with the path-bits set. > >> * QoS-Class: should be returned in the result PR/MPR. When routing is going to > >> be supported by OpenSM we might use this field in selecting the target > >> router too in a TBD way. > >> > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >> > > > > From mshefty at ichips.intel.com Tue Aug 7 11:14:52 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 07 Aug 2007 11:14:52 -0700 Subject: [ofa-general] implicit address binding by rdma_resolve_addr In-Reply-To: <39C75744D164D948A170E9792AF8E7CA0A0109@exil.voltaire.com> References: <46B75709.5080507@ichips.intel.com> <39C75744D164D948A170E9792AF8E7CA0A0109@exil.voltaire.com> Message-ID: <46B8B69C.3060100@ichips.intel.com> > Correct me if I am wrong, but implementing this scenario for UD will > look like this: > /* accept handling not shown... */ > rdma_bind_addr(id, ...); > rdma_listen(id) > while(1) { > read_request(); > do_work(); > if(new_client) { /* check if we already connected to client */ > /* following is not going to work because local address > is already bound */ > rdma_resolve_addr(my_local_already_bound_addr, > client_addr); There could very well be a bug with the state management in the rdma cm that disallows this. (Have you tried this and seen it fail?) rdma_resolve_addr() only binds if the id is not already bound. So, call sequence such as: rdma_bind_addr(id..) rdma_resolve_addr(id..) should work fine. Likewise, rdma_resolve_addr(id..) rdma_resolve_route(id..) rdma_connect(id..) *should* work, but may not if the connection state isn't being reset properly for UD. Note that you don't have to use the same id that you send on to resolve the address. You can use a temporary ID that you allocate for the purpose of obtaining the remote address handle. As an alternative to calling rdma_resolve_addr() from the server, you can use the receive completion from the client to create an address handle. See the udaddy.c sample program with the librdmacm - specifically create_reply_ah(). - Sean From sean.hefty at intel.com Tue Aug 7 11:34:30 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 11:34:30 -0700 Subject: [ofa-general] [PATCH] ib/srp: add QoS support through service ID In-Reply-To: <000301c7d91b$4e2a2e00$1acc180a@amr.corp.intel.com> Message-ID: <000401c7d921$9825f100$1acc180a@amr.corp.intel.com> Provide the target service ID when performing a path record query to support optional QoS capability. QoS requires support from the SA. Signed-off-by: Sean Hefty --- This change was way easier than I expected. It compiles with the appropriate changes to the ib_sa that Roland mentioned earlier. Once we can converge on all the host stack changes for QoS, I'll push them out to my git tree. drivers/infiniband/ulp/srp/ib_srp.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 39bf057..a1d7be0 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -271,6 +271,7 @@ static int srp_lookup_path(struct srp_target_port *target) target->srp_host->dev->dev, target->srp_host->port, &target->path, + IB_SA_PATH_REC_SERVICE_ID | IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | IB_SA_PATH_REC_NUMB_PATH | @@ -1702,6 +1703,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target) goto out; } target->service_id = cpu_to_be64(simple_strtoull(p, NULL, 16)); + target->path.service_id = target->service_id; kfree(p); break; From sean.hefty at intel.com Tue Aug 7 12:01:23 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 12:01:23 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070807175055.GD8504@obsidianresearch.com> Message-ID: <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> >MTU - at a minimum the PR must request a MTU >= than the interface >MTU, having a MTU that is exactly equal to the broadcast group MTU >seems fine to me. The SM should be able to do >= internally. My thoughts on MTU (and rate) are that since it isn't carried in any of the headers, we should be able to derive the maximum MTU from the other fields. Plus, we've had situations in the past where a smaller MTU than the maximum performed better than the max, so I'm hesitant to add MTU to the query. >Flowlabel - IBA hasn't really specified what to do with this field. In >IPv6 FlowLabel is assigned randomly by the host on a per-flow (ie >socket) basis to allow routers to group packets into flows without >doing deeper inspection. This becomes important when encryption is >used and the routers can no longer access the TCP header to find >flows. Dropping the flow label from the query is simple enough for IPoIB. We'll have to consider what to do with the flowinfo field of an IPv6 address for the rdma_cm, however. I'd like the minimal number of fields necessary on the host side to give greater flexibility to the SA. Right now, we're only looking at adding traffic class and, possibly, MTU to the IPoIB PR query. - Sean From tziporet at dev.mellanox.co.il Tue Aug 7 12:05:49 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 07 Aug 2007 22:05:49 +0300 Subject: [ofa-general] OFED 1.2.c status & plans Message-ID: <46B8C28D.7020108@mellanox.co.il> Hi All, I wish to update on OFED 1.2.c status and plans to synch everybody: * OFED 1.2.c-11 is going out tomorrow * This release should be the base for the GA release * Need an approval from Steve (Chelsio) & Nam (IBM) that everything is in place from their perspective. Also please send me the release notes for ehca and cxgb3 * Need an approval from the companies that are testing this release that it can go to GA From Mellanox perspective (mlx4 readiness) we are ready for GA. I have one question: do we prefer to stay with the name 1.2.c or 1.2.5? Thanks, Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Aug 7 12:05:11 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:05:11 -0700 Subject: [ofa-general] Re: sparse output In-Reply-To: <20070807112921.GC13863@mellanox.co.il> (Michael S. Tsirkin's message of "Tue, 7 Aug 2007 14:29:21 +0300") References: <20070807112921.GC13863@mellanox.co.il> Message-ID: > Do we want to fix these? If they indicate real problems, of course. - R. From rdreier at cisco.com Tue Aug 7 12:14:38 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:14:38 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <46B8439C.6010303@voltaire.com> (Or Gerlitz's message of "Tue, 07 Aug 2007 13:04:12 +0300") References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> Message-ID: > Roland, I realized now that the current IPoIB implementation does not > set IB_SA_PATH_REC_MTU and IB_SA_PATH_REC_RATE (and their selectors) > in the SA path query where it does so for SA multicast joins. Is there > a specific reason for that? do you assume that if the remote node has > managed to join the broadcast group as this node this the SA enforced > some sort of MTU and RATE compliance which you would get in the > returned path record? Yes, I guess there is a corner case where the SA might return a path where the MTU is too small. I don't think we need to specify the rate because things should work fine no matter what rate we get. - R. From rdreier at cisco.com Tue Aug 7 12:16:42 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:16:42 -0700 Subject: [ofa-general] Re: Understanding IPoIB-CM In-Reply-To: <20070806052552.GB1896@mellanox.co.il> (Michael S. Tsirkin's message of "Mon, 6 Aug 2007 08:25:52 +0300") References: <20070804220206.GA15821@mellanox.co.il> <20070806052552.GB1896@mellanox.co.il> Message-ID: > > Here, I wanted to ask - before transmitting why is the skb->dev field > > modified as the device is same IPoIB interface through which packets will be > > transmitted. Is p->dev is some other device? > > Good question, I don't recall at the moment. > I think I copied this from ipoib_main.c > Roland, any idea why do we do this? I think I saw some cases a long time ago when first developing this where skb->dev was not set, which caused problems in dev_queue_xmit(). - R. From eitan at mellanox.co.il Tue Aug 7 12:28:25 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 7 Aug 2007 22:28:25 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> References: <20070807175055.GD8504@obsidianresearch.com> <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD923@mtlexch01.mtl.com> I support your proposal Sean. Only flowlabel and Tclass. MTU can be provided by the SA too. Did we discuss PKey? I am not sure if we need to provide it or not, but it is the identifier of the IPoIB subnet, so the SA will get the IPoIB context of the request. Eitan Zahavi Senior Engineering Director, Software Architect Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -----Original Message----- > From: Sean Hefty [mailto:sean.hefty at intel.com] > Sent: Tuesday, August 07, 2007 10:01 PM > To: 'Jason Gunthorpe' > Cc: Eitan Zahavi; OpenIB > Subject: RE: [ofa-general] [PATCH 1/4] ib/ipoib: specify > TClassand FlowLabelwith PR queries for QoS support > > >MTU - at a minimum the PR must request a MTU >= than the > interface MTU, > >having a MTU that is exactly equal to the broadcast group MTU seems > >fine to me. The SM should be able to do >= internally. > > My thoughts on MTU (and rate) are that since it isn't carried > in any of the headers, we should be able to derive the > maximum MTU from the other fields. > Plus, we've had situations in the past where a smaller MTU > than the maximum performed better than the max, so I'm > hesitant to add MTU to the query. > > >Flowlabel - IBA hasn't really specified what to do with this > field. In > >IPv6 FlowLabel is assigned randomly by the host on a per-flow (ie > >socket) basis to allow routers to group packets into flows without > >doing deeper inspection. This becomes important when > encryption is used > >and the routers can no longer access the TCP header to find flows. > > Dropping the flow label from the query is simple enough for > IPoIB. We'll have to consider what to do with the flowinfo > field of an IPv6 address for the rdma_cm, however. > > I'd like the minimal number of fields necessary on the host > side to give greater flexibility to the SA. Right now, we're > only looking at adding traffic class and, possibly, MTU to > the IPoIB PR query. > > - Sean > From swise at opengridcomputing.com Tue Aug 7 12:29:46 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 07 Aug 2007 14:29:46 -0500 Subject: [ofa-general] Re: [ewg] OFED 1.2.c status & plans In-Reply-To: <46B8C28D.7020108@mellanox.co.il> References: <46B8C28D.7020108@mellanox.co.il> Message-ID: <46B8C82A.5010001@opengridcomputing.com> Tziporet Koren wrote: > Hi All, > > I wish to update on OFED 1.2.c status and plans to synch everybody: > > * OFED 1.2.c-11 is going out tomorrow > * This release should be the base for the GA release > * Need an approval from Steve (Chelsio) & Nam (IBM) that everything > is in place from their perspective. I have one more change needed to bump the required fw version to 4.6.0 in the low level driver (trival #define change). I've tested 1.2.c with the new firmware and it looks good. But I need to get the firmware version change upstream too, so maybe we can have one more week? With the goal being -12 is final? > Also please send me the release notes for ehca and cxgb3 Will do. > * Need an approval from the companies that are testing this release > that it can go to GA > >From Mellanox perspective (mlx4 readiness) we are ready for GA. > > I have one question: do we prefer to stay with the name 1.2.c or 1.2.5? > I prefer 1.2.5 over 1.2.c...(but I really prefer 1.2.1...why skip to .5?) Steve. From john.russo at qlogic.com Tue Aug 7 12:29:19 2007 From: john.russo at qlogic.com (John Russo) Date: Tue, 7 Aug 2007 14:29:19 -0500 Subject: [ofa-general] RE: [ewg] OFED 1.2.c status & plans In-Reply-To: <46B8C28D.7020108@mellanox.co.il> References: <46B8C28D.7020108@mellanox.co.il> Message-ID: <99863D2ED484D449811D97A4C44C9CBD4882AA@EPEXCH2.qlogic.org> 1.2.5 would imply that the 1.2 branch has merged with the 1.2c branch. Until that has actually been done, I would leave it as 1.2.c ________________________________ From: ewg-bounces at lists.openfabrics.org [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Tziporet Koren Sent: Tuesday, August 07, 2007 3:06 PM To: EWG Cc: OpenFabrics General Subject: [ewg] OFED 1.2.c status & plans Hi All, I wish to update on OFED 1.2.c status and plans to synch everybody: * OFED 1.2.c-11 is going out tomorrow * This release should be the base for the GA release * Need an approval from Steve (Chelsio) & Nam (IBM) that everything is in place from their perspective. Also please send me the release notes for ehca and cxgb3 * Need an approval from the companies that are testing this release that it can go to GA >From Mellanox perspective (mlx4 readiness) we are ready for GA. I have one question: do we prefer to stay with the name 1.2.c or 1.2.5? Thanks, Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From mst at dev.mellanox.co.il Tue Aug 7 12:34:24 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 22:34:24 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> Message-ID: <20070807193424.GB28710@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support > > > Roland, I realized now that the current IPoIB implementation does not > > set IB_SA_PATH_REC_MTU and IB_SA_PATH_REC_RATE (and their selectors) > > in the SA path query where it does so for SA multicast joins. Is there > > a specific reason for that? do you assume that if the remote node has > > managed to join the broadcast group as this node this the SA enforced > > some sort of MTU and RATE compliance which you would get in the > > returned path record? > > Yes, I guess there is a corner case where the SA might return a path > where the MTU is too small. I don't think we need to specify the rate > because things should work fine no matter what rate we get. OFED includes a patch where MTU is taken from the device MTU. Want to see it? An alternative would be to use bcast group MTU. -- MST From rdreier at cisco.com Tue Aug 7 12:36:03 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:36:03 -0700 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <20070807193424.GB28710@mellanox.co.il> (Michael S. Tsirkin's message of "Tue, 7 Aug 2007 22:34:24 +0300") References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> <20070807193424.GB28710@mellanox.co.il> Message-ID: > OFED includes a patch where MTU is taken from the device MTU. > Want to see it? I guess so although it seems that would cause problems if the device MTU changes. > An alternative would be to use bcast group MTU. That would seem to make more sense to me. - R. From mst at dev.mellanox.co.il Tue Aug 7 12:37:14 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 22:37:14 +0300 Subject: [ofa-general] Re: OFED 1.2.c status & plans In-Reply-To: <99863D2ED484D449811D97A4C44C9CBD4882AA@EPEXCH2.qlogic.org> References: <46B8C28D.7020108@mellanox.co.il> <99863D2ED484D449811D97A4C44C9CBD4882AA@EPEXCH2.qlogic.org> Message-ID: <20070807193714.GC28710@mellanox.co.il> > Quoting John Russo : > Subject: RE: OFED 1.2.c status & plans > > 1.2.5 would imply that the 1.2 branch has merged with the 1.2c branch. That's the case. 1.2 branch was merged into 1.2.c branch. -- MST From john.russo at qlogic.com Tue Aug 7 12:38:22 2007 From: john.russo at qlogic.com (John Russo) Date: Tue, 7 Aug 2007 14:38:22 -0500 Subject: [ofa-general] RE: OFED 1.2.c status & plans In-Reply-To: <20070807193714.GC28710@mellanox.co.il> References: <46B8C28D.7020108@mellanox.co.il><99863D2ED484D449811D97A4C44C9CBD4882AA@EPEXCH2.qlogic.org> <20070807193714.GC28710@mellanox.co.il> Message-ID: <99863D2ED484D449811D97A4C44C9CBD4882AD@EPEXCH2.qlogic.org> Oh... Super. In that case I like 1.2.5 To address the earlier email that asked about using 1.2.1: I wouldn't use that because people like me (with a short memory) might associate that with the pre-merged, bug-fix only version that was originally proposed. John R. -----Original Message----- From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] Sent: Tuesday, August 07, 2007 3:37 PM To: John Russo Cc: Tziporet Koren; EWG; OpenFabrics General Subject: Re: OFED 1.2.c status & plans > Quoting John Russo : > Subject: RE: OFED 1.2.c status & plans > > 1.2.5 would imply that the 1.2 branch has merged with the 1.2c branch. That's the case. 1.2 branch was merged into 1.2.c branch. -- MST From mst at dev.mellanox.co.il Tue Aug 7 12:40:15 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 22:40:15 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> <20070807193424.GB28710@mellanox.co.il> Message-ID: <20070807194015.GD28710@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support > > > OFED includes a patch where MTU is taken from the device MTU. > > Want to see it? > > I guess so although it seems that would cause problems if the device > MTU changes. Oh, I just do a flush in this case. This makes it possible to control the path MTU with ifconfig which seems kind of nifty. > > An alternative would be to use bcast group MTU. > > That would seem to make more sense to me. You decide. Anyway, here's the patch. --> IB/ipoib: user appropriate mtu selector for path queries IPoIB must set mtu selector in path record query according to dev->mtu: if we wildcard it, SM can select a path with lower MTU. This breaks IPoIB on networks with SM Tavor quirk activates. We can always require this, since IPoIB spec includes the following statement: The value (for IB MTU) assigned to the broadcast-GID must not be greater than any physical link MTU spanned by the IPoIB subnet. Signed-off-by: Michael S. Tsirkin --- Index: ofed_1_1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- ofed_1_1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ ofed_1_1/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -182,6 +182,8 @@ static int ipoib_change_mtu(struct net_d dev->mtu = min(priv->mcast_mtu, priv->admin_mtu); + queue_work(ipoib_workqueue, &priv->flush_task); + return 0; } @@ -452,15 +454,39 @@ static int path_rec_start(struct net_dev struct ipoib_path *path) { struct ipoib_dev_priv *priv = netdev_priv(dev); + ib_sa_comp_mask comp_mask = IB_SA_PATH_REC_MTU_SELECTOR | IB_SA_PATH_REC_MTU; + + path->pathrec.mtu_selector = IB_SA_GT; - ipoib_dbg(priv, "Start path record lookup for " IPOIB_GID_FMT "\n", - IPOIB_GID_ARG(path->pathrec.dgid)); + switch (roundup_pow_of_two(dev->mtu + IPOIB_ENCAP_LEN)) { + case 512: + path->pathrec.mtu = IB_MTU_256; + break; + case 1024: + path->pathrec.mtu = IB_MTU_512; + break; + case 2048: + path->pathrec.mtu = IB_MTU_1024; + break; + case 4096: + path->pathrec.mtu = IB_MTU_2048; + break; + default: + /* Wildcard everything */ + comp_mask = 0; + path->pathrec.mtu = 0; + path->pathrec.mtu_selector = 0; + } + + ipoib_dbg(priv, "Start path record lookup for " IPOIB_GID_FMT " MTU > %d\n", + IPOIB_GID_ARG(path->pathrec.dgid), + comp_mask ? ib_mtu_enum_to_int(path->pathrec.mtu) : 0); init_completion(&path->done); path->query_id = ib_sa_path_rec_get(&ipoib_sa_client, priv->ca, priv->port, - &path->pathrec, + &path->pathrec, comp_mask | IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID | IB_SA_PATH_REC_NUMB_PATH | -- MST From mst at dev.mellanox.co.il Tue Aug 7 12:45:29 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 7 Aug 2007 22:45:29 +0300 Subject: [ofa-general] Re: sparse output In-Reply-To: References: <20070807112921.GC13863@mellanox.co.il> Message-ID: <20070807194529.GE28710@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: sparse output > > > Do we want to fix these? > > If they indicate real problems, of course. Do we want to address pointer signedness issues? -- MST From HNGUYEN at de.ibm.com Tue Aug 7 12:45:45 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Tue, 7 Aug 2007 21:45:45 +0200 Subject: [ofa-general] Re: [ewg] OFED 1.2.c status & plans In-Reply-To: <46B8C28D.7020108@mellanox.co.il> Message-ID: Hello Tziporet! > I wish to update on OFED 1.2.c status and plans to synch everybody: > OFED 1.2.c-11 is going out tomorrow > This release should be the base for the GA release > Need an approval from Steve (Chelsio) & Nam (IBM) that everything is > in place from their perspective. For ehca ofed-1.2.c package plus latest changes Michael and I have worked on seems to be ok - thanks to Michael. However please give us a few days to run more tests, especially on rhel4.5, even though there are no functional changes for ehca this time. I would say next Mon is fine. > Also please send me the release notes for ehca and cxgb3 Will do. Thanks Nam From rdreier at cisco.com Tue Aug 7 12:49:08 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:49:08 -0700 Subject: [ofa-general] Re: [PATCH] mlx4: Do not invoke mlx4_bitmap_free for special QPs In-Reply-To: <200708061711.58789.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Mon, 6 Aug 2007 17:11:58 +0300") References: <200708061711.58789.jackm@dev.mellanox.co.il> Message-ID: > mlx4_bitmap_free should not be called for special QPs. I think I'd rather fix this problem like the patch below. Does this look OK to you? diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c index 19b48c7..278414b 100644 --- a/drivers/net/mlx4/qp.c +++ b/drivers/net/mlx4/qp.c @@ -240,7 +240,8 @@ void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp) mlx4_table_put(dev, &qp_table->auxc_table, qp->qpn); mlx4_table_put(dev, &qp_table->qp_table, qp->qpn); - mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); + if (qp->qpn < dev->caps.sqp_start + 8) + mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); } EXPORT_SYMBOL_GPL(mlx4_qp_free); From rdreier at cisco.com Tue Aug 7 12:52:59 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 07 Aug 2007 12:52:59 -0700 Subject: [ofa-general] Re: sparse output In-Reply-To: <20070807194529.GE28710@mellanox.co.il> (Michael S. Tsirkin's message of "Tue, 7 Aug 2007 22:45:29 +0300") References: <20070807112921.GC13863@mellanox.co.il> <20070807194529.GE28710@mellanox.co.il> Message-ID: > Do we want to address pointer signedness issues? I guess... if we have a prototype void foo(long *x); then we should probably fix code that does unsigned long y; foo(&y); unless it gets too messy. From sean.hefty at intel.com Tue Aug 7 12:55:50 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 12:55:50 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BD923@mtlexch01.mtl.com> Message-ID: <000601c7d92c$f481bc80$1acc180a@amr.corp.intel.com> >I support your proposal Sean. Only flowlabel and Tclass. >MTU can be provided by the SA too. I'm honestly still not clear on what fields are needed yet. I think we've decided that we don't need rate or SL, but we need traffic class. MTU and flow label are still undecided. >Did we discuss PKey? I am not sure if we need to provide it or not, but >it is the identifier of the >IPoIB subnet, so the SA will get the IPoIB context of the request. PKey was already included as part of the PR query. - Sean From sweitzen at cisco.com Tue Aug 7 13:12:53 2007 From: sweitzen at cisco.com (Scott Weitzenkamp (sweitzen)) Date: Tue, 7 Aug 2007 13:12:53 -0700 Subject: [ofa-general] RE: [ewg] OFED 1.2.c status & plans In-Reply-To: <46B8C28D.7020108@mellanox.co.il> References: <46B8C28D.7020108@mellanox.co.il> Message-ID: Cisco has been testing 1.2.c-10 IPoIB/SDP/MPI successfully on a 32-node cluster. We are still working on tvflash, though. Scott ________________________________ From: ewg-bounces at lists.openfabrics.org [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Tziporet Koren Sent: Tuesday, August 07, 2007 12:06 PM To: EWG Cc: OpenFabrics General Subject: [ewg] OFED 1.2.c status & plans Hi All, I wish to update on OFED 1.2.c status and plans to synch everybody: * OFED 1.2.c-11 is going out tomorrow * This release should be the base for the GA release * Need an approval from Steve (Chelsio) & Nam (IBM) that everything is in place from their perspective. Also please send me the release notes for ehca and cxgb3 * Need an approval from the companies that are testing this release that it can go to GA >From Mellanox perspective (mlx4 readiness) we are ready for GA. I have one question: do we prefer to stay with the name 1.2.c or 1.2.5? Thanks, Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From chas at cmf.nrl.navy.mil Tue Aug 7 13:59:50 2007 From: chas at cmf.nrl.navy.mil (chas williams - CONTRACTOR) Date: Tue, 07 Aug 2007 16:59:50 -0400 Subject: [ofa-general] Re: Understanding IPoIB-CM In-Reply-To: Message-ID: <200708072059.l77KxoPM031328@cmf.nrl.navy.mil> In message ,Roland Dreier writes: >I think I saw some cases a long time ago when first developing this >where skb->dev was not set, which caused problems in dev_queue_xmit(). as i recall, outbound arp packets dont (or atleast didnt) set skb->dev. From mail_link1 at web.de Tue Aug 7 14:52:39 2007 From: mail_link1 at web.de (mail link) Date: Tue, 07 Aug 2007 23:52:39 +0200 Subject: [ofa-general] Your Email Addres Has Won Award Message-ID: <1479125959@web.de> CONGRATULATION WINNER !!! Award Prize Dept Madrid Spain Dear Beneficiary We are please to announce you as one of the 10 lucky winners in the Euro Milliones Lottery International Email Address draw on the 15th of July 2007, due to the mixture of names and address the result was released on the 2nd August 2007. All 10 winning addresses w ere ran domly selected from a batch of 50,000,000 international email addresses. Your email address emerged a longside 9 others as a c ategory 2 winner in the Euro Milliones Lottery Draw.Consequently, you have therefore been approved for a total pay out of US$787,000:00 (Seven Hundred and Eighty Seven Thousand United States Dollars Only). The following particulars ar e attached t o your lotto payment or der: (1) Ba t ch No: EULO/2907/ES/783 (2) Ticket No: WRNM/SMI/5990 (3) Lucky No: 07-13-31.54-8921 (4) Serial No: MUOTI/82536 (5) Ref No: EMJK/WS/MD/2911 The Euro Milliones Lottery Program internet draw is held once in a year and is so organized to encourage the use of the internet and computers worldwide. We are proud to say that over 200 Million Euros are wo n annually in more than 150 countries worldwide. To claim your winning prize you are to contact the appointed agent as soon as possible for the immedi ate release of your winnings: NAME DR ALONZO PERREZ A.P CLAIMING AGENCY S.L. TEL : +34 697 333 267 EMAIL: perrezagency at aim.com [http://us.f511.mail.yahoo.com/ym/Compose?To=perrezagency at aim.com] EMAIL: perrezagency at yahoo.es [http://us.f511.mail.yahoo.com/ym/Compose?To=perrezagency@ yahoo.es] then you will be informed b y your ag ent on how your Cashier's Check for the sum w hich has already been deposited with your email contact.We use this oppor tunity to thank all our Sponsors/Contributo rs of th i s program:He ineken,Sony,master card,Vodafone,play station,Ford For payment, please do provide with t he following informations And send to our Agent via VERIFICATION FORM NAME:........... .......... ....................................... AGE:............................................... ............. SEX:............................................................. ADDRESS: ...................... .................................. WINNING EMAIL ADDRESS:................................. TEL.:............................................................ OCCUPATION:...................................................... COUNTRY:.......... ............................................... Please you are advised to complete the form and forward to your a gent for prompt collec tion. You are automatically disqualified if you a re below 18 years of age.This E-mail message (including attachments, if any)is intended for the u se of the indivi dual or entity to which it is addressed a nd may contain Information that is privileged, proprietary,confidential and exempted from disclosure. Sincerely Yours, Mrs Be tty Rice PROMOTION CORDINATOR Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! *http://smartsurfer.web.de/?mc=100071&distributionid=000000000066* [http://smartsurfer.web.de/?mc=100071&distributionid=000000000066] -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgunthorpe at obsidianresearch.com Tue Aug 7 15:17:00 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Tue, 7 Aug 2007 16:17:00 -0600 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> References: <20070807175055.GD8504@obsidianresearch.com> <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> Message-ID: <20070807221700.GE8504@obsidianresearch.com> On Tue, Aug 07, 2007 at 12:01:23PM -0700, Sean Hefty wrote: > >MTU - at a minimum the PR must request a MTU >= than the interface > >MTU, having a MTU that is exactly equal to the broadcast group MTU > >seems fine to me. The SM should be able to do >= internally. > > My thoughts on MTU (and rate) are that since it isn't carried in any of the > headers, we should be able to derive the maximum MTU from the other fields. > Plus, we've had situations in the past where a smaller MTU than the maximum > performed better than the max, so I'm hesitant to add MTU to the query. Well, the MTU isn't explicity carried in the headers, but if you send a 2K packet into a path that only supports 1K MTU then it will be discarded. In that sense the MTU is included in the headers. The MTU of every unicast path used must be greater than the Linux interface MTU so that the stack produces correctly sized fragments. If you want to use the 1k is fast feature of a tavor then I was under the impression that was done under a reduced IP interface MTU? Linux has no per-arp entry MTU so I don't know how you'd integrate the idea of different L2 destinations having different MTUs with the current stack. The only case I could see not including the MTU would be if RC is used as the transport, then the IB level MTU does not matter. Jason From sale.alerts at kohls.com Tue Aug 7 15:47:34 2007 From: sale.alerts at kohls.com (Kohls.com) Date: Tue, 7 Aug 2007 17:47:34 -0500 (CDT) Subject: [ofa-general] Your First Kohls.com Sales Alert! Message-ID: <259332772.1186526854571.JavaMail.esadmin> Welcome to Kohls.com Sale Alerts! As a subscriber to our e-mail Sale Alerts, you'll hear about our biggest sales and other special savings! To welcome you to Sale Alerts, we'll take an EXTRA 10% OFF* your next Kohls.com purchase, including our four exclusive Beauty brands -- American Beauty, Flirt!, good skin and grassroots! To save an EXTRA 10%, just enter this Promo Code when you Checkout: *********** NEW3687 ********** Hurry, offer ends August 18! Shop now at http://www.kohls.com ! Every week, we'll send you up-to-the-minute info on our great sale prices. And, you'll periodically receive other great savings offers, like these: + FREE Standard Shipping on your order! + EXTRA SAVINGS when you Checkout with your Kohl's Charge Card + NEW Markdowns in our Clearance Department + BIG savings in every department Plus, every e-mail includes an easy opt-out feature, so you can choose to unsubscribe at any time. ********************************************************************************* Kohls.com - Same Great Brands, Same Great Sales Kohls.com features the same great sale prices you'll find at your local Kohl's. We also carry the same great brands you've come to expect from Kohl's - Levi's, Dockers, KitchenAid, Calphalon and more. Kohls.com - Convenience & More! Shop Kohl's from the comfort of home, plus take advantage of other great benefits, available exclusively online. Create an Account to save time when you Checkout - we'll save your payment and shipping information for super-fast Checkout. (And, your information is 100% secure.) Kohls.com - Coming Your Way! Thanks again for subscribing to Kohls.com Sale Alerts! Stay tuned for great savings, coming your way soon. ********************************************************************************* * Offer excludes sales tax and shipping. It is not valid with other coupon offers, for price adjustments on prior purchases, on Gift Card or Kohl's Cares for Kids merchandise purchases. This offer is non-transferable. This mailbox is unattended, so please do not reply to this message. If you no longer wish to receive e-mails from Kohls.com, unsubscribe by pasting this link into your Internet browser. http://www.kohls.com/registration/kohls_unregister_email_sales_alerts.jsp For other inquiries, e-mail us at myaccount.help at kohls.com, or write us at Kohl's Department Stores, Attention: Customer Service, N56 W17000 Ridgewood Drive, Menomonee Falls, WI 53051. From sean.hefty at intel.com Tue Aug 7 17:09:53 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 7 Aug 2007 17:09:53 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070807221700.GE8504@obsidianresearch.com> Message-ID: <000001c7d950$72242150$60cc180a@amr.corp.intel.com> >Well, the MTU isn't explicity carried in the headers, but if you send >a 2K packet into a path that only supports 1K MTU then it will be >discarded. In that sense the MTU is included in the headers. This is what I meant by the MTU is implied by the other fields. I was thinking about it this way. If a PR query contains the SLID, DLID, SL, I would expect the SA to lookup the MTU for this path and return it. Is there any advantage or reason to include the MTU in such a query? Taking this across subnets, if a PR query contains the SGID, DGID, TC, and FL, does this change whether the MTU should be specified? I'm not trying to argue against including the MTU, but I don't know if IPoIB or the SA should specify it. (And I'm neither an IPoIB nor SA expert.) >The MTU of every unicast path used must be greater than the Linux >interface MTU so that the stack produces correctly sized fragments. As you mentioned, this doesn't holds for IPoIB-CM. The MTU of the path is less than the MTU sent by the stack, and the path MTUs could differ, including being less than the broadcast MTU. (I don't know that the implementation supports different path MTUs, but it could in theory.) >If you want to use the 1k is fast feature of a tavor then I was under >the impression that was done under a reduced IP interface MTU? Linux >has no per-arp entry MTU so I don't know how you'd integrate the idea >of different L2 destinations having different MTUs with the current >stack. Couldn't IPoIB fragment the packets if it needed? (Not sure what that would do to the performance.) How does IPoIB-CM handle the case where the device MTU is, say, 64k, but the remote side only supports UD? - Sean From ggrundstrom at neteffect.com Tue Aug 7 17:22:14 2007 From: ggrundstrom at neteffect.com (ggrundstrom at neteffect.com) Date: Tue, 7 Aug 2007 19:22:14 -0500 Subject: [ofa-general] [PATCH 0/14] nes: NetEffect 10Gb RNIC Driver Message-ID: <200708080022.l780ME9e004542@neteffect.com> NetEffect is proud to announce the following series of patches which contain the source code for the NE020 10Gb RNIC adapter. The driver is split into two components - a kernel driver module and a userspace library. The code can also be found in the following git trees. git.openfabrics.org/~glenn/libnes.git git.openfabrics.org/~glenn/ofed_1_2.git git.openfabrics.org/~glenn/ofascripts.git git.openfabrics.org/~glenn/ofed_1_2_scripts.git Requirements ============ * NE020 hardware * RHEL4u4 or FC5 * OFED 1.2 GA * OFED 1.2 version of MVAPICH2 Known issues ============ * DAPL only works with 1 process per node due to lack of loopback * MPI over DAPL must use MPI shared memory loopback Plans for next release ====================== * Plan on adding verbs loopback to enable DAPL loopback * Increase robustness and stability What we tested ============== The performance results are meant to be broadly representative. Results can vary depending on switches used, system configuration, OS used, etc. Configuration notes: All two node tests were performed back-to-back i.e. no switch Multinode testing was performed using a high performance, low latency cut-through switch. Platform: CentOS x86_64 1. cbench rotate latency and bandwidth using mvapich2 over OFA verbs: Rotate Latency: 6.67 us Rotate Bandwidth: 9.3 Gpbs 2. OSU bandwidth and latency tests using mvapich2 over OFA verbs: OSU Latency: 6.74 us OSU Bi-Bandwidth: 14.4 Gbps 3. Perftest (rdma_bw Uni-dir/Bi-dir and rdma_lat) RDMA Bandwidth Uni-directional: 8.9 Gpbs RDMA Bandwidth Bi-directional : 15.09 Gpbs RDMA Latency : 5.95 us 4. NIC Testing Iperf 4 stream Bi-directional test Iperf -c -d -M -N -i 4 -P 4 (Jumbo packets enabled) --> 10.76 Gbps NetPerf netperf -H -T1,1 -t TCP_STREAM -l 60 -C -c (Jumbo packets enabled) --> 6.2 Gbps Thanks, Glenn. From jgunthorpe at obsidianresearch.com Tue Aug 7 19:05:26 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Tue, 7 Aug 2007 20:05:26 -0600 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <000001c7d950$72242150$60cc180a@amr.corp.intel.com> References: <20070807221700.GE8504@obsidianresearch.com> <000001c7d950$72242150$60cc180a@amr.corp.intel.com> Message-ID: <20070808020526.GE9424@obsidianresearch.com> On Tue, Aug 07, 2007 at 05:09:53PM -0700, Sean Hefty wrote: > >Well, the MTU isn't explicity carried in the headers, but if you send > >a 2K packet into a path that only supports 1K MTU then it will be > >discarded. In that sense the MTU is included in the headers. > > This is what I meant by the MTU is implied by the other fields. I was thinking > about it this way. If a PR query contains the SLID, DLID, SL, I would expect > the SA to lookup the MTU for this path and return it. Is there any advantage or > reason to include the MTU in such a query? Er, IPoIB does not do that though? It should create a PR with DGID, SGID, TClass, Pkey, etc based on the IP L2 information from the ARP/ND packet. The result of that query is then used to wrap IP datagrams in UD datagrams. Thus it must ask the SA for a path with a minimum MTU large enough to carry the largest IP datagram. > Taking this across subnets, if a PR query contains the SGID, DGID, TC, and FL, > does this change whether the MTU should be specified? I'm not trying to argue > against including the MTU, but I don't know if IPoIB or the SA should specify > it. (And I'm neither an IPoIB nor SA expert.) Ah, well, MTU is really used as both something you request and something the SA returns. In the case of datagram communication the MTU is very important since datagram fragmentation is impossible. In those cases the end points must ask for paths that meet their MTU requirements (there may be switching paths that do not, and without guidance the SA is free to return anything) For RC, MTU is something that should not generally be requested and the returned value from the SA should be used to configure the connection. This gives the SA freedom to return paths across multiple switching paths. This is really only because the RC message size is not impacted by the connection MTU. > >The MTU of every unicast path used must be greater than the Linux > >interface MTU so that the stack produces correctly sized fragments. > > As you mentioned, this doesn't holds for IPoIB-CM. The MTU of the path is less > than the MTU sent by the stack, and the path MTUs could differ, including being > less than the broadcast MTU. (I don't know that the implementation supports > different path MTUs, but it could in theory.) Right, RC is handled differently than UD/UC when talking about MTU.. > Couldn't IPoIB fragment the packets if it needed? (Not sure what that would do > to the performance.) How does IPoIB-CM handle the case where the device MTU is, > say, 64k, but the remote side only supports UD? There is no provision for fragment identification and reassembly in the IPoIB RFC. AFAIK, when using the 64K MTU setting for IPoIB, if the remote side doesn't support RC then things go wonky. For TCP things *might* be saved by path mtu discovery - but PMTU is driven by ICMP errors which are not generated by an IB network. I suspect that if you use IPoIB a mixed configuration like that you are going to want to have routing table entries that override the MTU for non-RC capable destinations. But I haven't tried this.. Jason From kliteyn at mellanox.co.il Tue Aug 7 21:44:00 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 8 Aug 2007 07:44:00 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-08:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Aug_7_19:43:41_2007 [cafabe587aee10a90f2bf66033d4eb4bb7cbcad7] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=560 Pass=501 Fail=59 Pass: 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 40 Stability IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo 13 OsmTest IS3-128.topo Failures: 42 Pkey IS1-16.topo 14 Pkey IS3-128.topo 2 Stability IS1-16.topo 1 OsmTest IS3-128.topo From ogerlitz at voltaire.com Tue Aug 7 22:20:18 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 08 Aug 2007 08:20:18 +0300 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <46B8A8AE.5030002@ichips.intel.com> References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> <46B838D6.6000501@voltaire.com> <46B8A8AE.5030002@ichips.intel.com> Message-ID: <46B95292.5090900@voltaire.com> Sean Hefty wrote: >> iser is rdma cm based, the only way for iser to use the sid is if the >> rdma cm allows it to do so, with this patch series its not possible. > Can someone explain how iser and srp find the target devices? Sean, iser is no more (and no less...) then iscsi transport provider (another in-kernel transport provider is iscsi_tcp) in the framework of the linux open iscsi initiator. So with this 'i' at hand things --all-- go with IP addressing. The initiator management is done by user space daemon, where the first stage is "target discovery" this is done using IP only and is beyond the scope here. Once the the target IP is known, the user space daemon attempts to login into the target, here iser comes into play and we connect through the rdma cm (see iser_connect() in iser_verbs.c), so with this dst IP at hand the rdma cm resolves for us: locally the hca/port/pkey, then the remote gid, then the path, then connect. Or. From ogerlitz at voltaire.com Tue Aug 7 22:26:52 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 08 Aug 2007 08:26:52 +0300 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <46B89D9A.4080504@ichips.intel.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> <46B89D9A.4080504@ichips.intel.com> Message-ID: <46B9541C.1070307@voltaire.com> Sean Hefty wrote: >> Sean, I think you want to ask for a specific SL here (that is add >> IB_SA_PATH_REC_SL to the mask and ask for the broadcast group SL). > > My thoughts are that the traffic class and flow label should map to a > specific SL. Consider a packet entering the local subnet through an IB > router. The local subnet SL would need to be determined using the TC > and FL. I see you point. Doesn't this suggest that the QoS architecture for all ULPs should such which makes them request specific TC/FL since if the associated traffic would be routed, the GRH is what the remote router has at hand to ask the remote SM/SA to determine the desired QoS? Or. From ogerlitz at voltaire.com Tue Aug 7 22:48:40 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 08 Aug 2007 08:48:40 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> Message-ID: <46B95938.2030804@voltaire.com> Sean Hefty wrote: > Or Gerlitz wrote: >> I think the correct way here, is a SID --> QoS mapping. This allows >> you to manage and configure QoS in --one-- place, namely the SM/SA and >> zero effort host side deployment. > SID doesn't work for iWarp. If an IPv6 address is used, then we have > the information necessary (traffic class and flow label) to support QoS. > If an IPv4 address is used, we mimic sockets. Sean, As of the --importance-- && --relevancy-- I quoted Eitan's response below, I gave it some thought and I agree with him, namely: To mimic sockets, rdma_set_service_type() is perfectly correct and usable by socket offloaders such as SDP and RDS, so they can translate an IP_TOS setsockopt() call to rdma_set_service_type(). This plugs to the case where the application wants to request per connection QoS class. The case where the administrator wants to set the QoS attributes without any application change must be supported and would be easily (as you wrote) be supported if you would add the SID used by this rdma cm ID into the path query. This would handle 100% of the rdma cm based ULPs when IB is used as the rdma transport. I don't think that since SID does not work for iwarp its in correct by design to use SID for IB QoS when using the rdma cm. The rdma cm design should work for both IB and iWARP, but it does not say that if something is working only for IB you disallow it in the rdma cm. Using IB SID --> QoS derivation is way more important then having design which is 100% transport neutral. Or. Eitan Zahavi wrote: > Hi Or, Sean, > > SID and QoSClass have a different purpose: > * SID enables the administrator to set the QoS attributes without any > application change. > (e.g. a simple rule like all SSH should have lowest priority) > * QoS Class: enables the application to request QoS class on a > per-socket manner. > It requires the application to know what it is doing and the > administrator to trust it. > > It is not by chance that the two options are there in the spec. > Different communities/installations have different requirements > regarding "who controls" the policy. > > For OFA development I think that providing SID should be done by default > CM based traffic: > The SID is a mandatory for making a connection and thus I do not see any > reason not to provide it in the PathRecord query. > > Supporting QoS-Class is a compatibility feature supporting application > that set the TOS socket option to obtain TOS aware service over IB. From ogerlitz at voltaire.com Tue Aug 7 23:02:30 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 08 Aug 2007 09:02:30 +0300 Subject: [ofa-general] [PATCH 2/4] ib/sa: add new QoS fields to path record In-Reply-To: <46B95292.5090900@voltaire.com> References: <000001c7d8b0$e9625d90$19c8180a@amr.corp.intel.com> <46B838D6.6000501@voltaire.com> <46B8A8AE.5030002@ichips.intel.com> <46B95292.5090900@voltaire.com> Message-ID: <46B95C76.9090400@voltaire.com> Or Gerlitz wrote: > Sean Hefty wrote: >>> iser is rdma cm based, the only way for iser to use the sid is if the >>> rdma cm allows it to do so, with this patch series its not possible. >> Can someone explain how iser and srp find the target devices? > Once the the target IP is known, the user space daemon attempts to login > into the target, here iser comes into play and we connect through the rdma cm OK, just to clarify, the iscsi target discovery resolves both the IP and the PORT used by the target. Where the port is mapped to IB SID in both sides (the listening target and the connecting initiator). Or. From jackm at dev.mellanox.co.il Tue Aug 7 23:26:26 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Wed, 8 Aug 2007 09:26:26 +0300 Subject: [ofa-general] Re: [PATCH] mlx4: Do not invoke mlx4_bitmap_free for special QPs In-Reply-To: References: <200708061711.58789.jackm@dev.mellanox.co.il> Message-ID: <200708080926.26566.jackm@dev.mellanox.co.il> On Tuesday 07 August 2007 22:49, Roland Dreier wrote: > > mlx4_bitmap_free should not be called for special QPs. > > I think I'd rather fix this problem like the patch below. Does this > look OK to you? > Yes, it does. The same calculation (dev->caps.sqp_start + 8) is used when creating the bitmap, for the reserved values, so your version is actually more correct (i.e., don't free reserved values). Also, obviously, no prototype changes are needed. - Jack > > diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c > index 19b48c7..278414b 100644 > --- a/drivers/net/mlx4/qp.c > +++ b/drivers/net/mlx4/qp.c > @@ -240,7 +240,8 @@ void mlx4_qp_free(struct mlx4_dev *dev, struct mlx4_qp *qp) > mlx4_table_put(dev, &qp_table->auxc_table, qp->qpn); > mlx4_table_put(dev, &qp_table->qp_table, qp->qpn); > > - mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); > + if (qp->qpn < dev->caps.sqp_start + 8) > + mlx4_bitmap_free(&qp_table->bitmap, qp->qpn); > } > EXPORT_SYMBOL_GPL(mlx4_qp_free); > > From eitan at mellanox.co.il Tue Aug 7 23:44:52 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Wed, 8 Aug 2007 09:44:52 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service In-Reply-To: <46B95938.2030804@voltaire.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> <46B95938.2030804@voltaire.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BD9FB@mtlexch01.mtl.com> Needless to say I agree with Or completely. iWarp does not do any PathRecord queries what so ever ... So the iwarp argument about including SID or not in the PR is really mute. EZ > -----Original Message----- > From: Or Gerlitz [mailto:ogerlitz at voltaire.com] > Sent: Wednesday, August 08, 2007 8:49 AM > To: Eitan Zahavi > Cc: Sean Hefty; OpenIB > Subject: Re: [ofa-general] [PATCH 3/4] rdma/cm: add ability > to specify typeof service > > Sean Hefty wrote: > > > Or Gerlitz wrote: > >> I think the correct way here, is a SID --> QoS mapping. > This allows > >> you to manage and configure QoS in --one-- place, namely the SM/SA > >> and zero effort host side deployment. > > > SID doesn't work for iWarp. If an IPv6 address is used, > then we have > > the information necessary (traffic class and flow label) to > support QoS. > > If an IPv4 address is used, we mimic sockets. > > Sean, > > As of the --importance-- && --relevancy-- I quoted Eitan's > response below, I gave it some thought and I agree with him, namely: > > To mimic sockets, rdma_set_service_type() is perfectly > correct and usable by socket offloaders such as SDP and RDS, > so they can translate an IP_TOS setsockopt() call to > rdma_set_service_type(). This plugs to the case where the > application wants to request per connection QoS class. > > The case where the administrator wants to set the QoS > attributes without any application change must be supported > and would be easily (as you > wrote) be supported if you would add the SID used by this > rdma cm ID into the path query. This would handle 100% of the > rdma cm based ULPs when IB is used as the rdma transport. > > I don't think that since SID does not work for iwarp its in > correct by design to use SID for IB QoS when using the rdma > cm. The rdma cm design should work for both IB and iWARP, but > it does not say that if something is working only for IB you > disallow it in the rdma cm. Using IB SID --> QoS derivation > is way more important then having design which is 100% > transport neutral. > > Or. > > Eitan Zahavi wrote: > > Hi Or, Sean, > > > > SID and QoSClass have a different purpose: > > * SID enables the administrator to set the QoS attributes > without any > > application change. > > (e.g. a simple rule like all SSH should have lowest priority) > > * QoS Class: enables the application to request QoS class on a > > per-socket manner. > > It requires the application to know what it is doing and the > > administrator to trust it. > > > > It is not by chance that the two options are there in the spec. > > Different communities/installations have different requirements > > regarding "who controls" the policy. > > > > For OFA development I think that providing SID should be done by > > default CM based traffic: > > The SID is a mandatory for making a connection and thus I > do not see > > any reason not to provide it in the PathRecord query. > > > > Supporting QoS-Class is a compatibility feature supporting > application > > that set the TOS socket option to obtain TOS aware service over IB. > > From mst at dev.mellanox.co.il Wed Aug 8 00:19:10 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 10:19:10 +0300 Subject: [ofa-general] [PATCHv3 RFC] Scalable Reliable Connection: API and documentation Message-ID: <20070808071910.GC23514@mellanox.co.il> Add API extensions and documentation to support Scalable Reliable Connections. Signed-off-by: Michael S. Tsirkin --- Here's an updated revision of the RFC. Changes since v2: - Remove max_src_domains - this breaks library ABI and is unlikely to be useful anyway - Add device capability flag to enable detecting SRC support - Fill in some implementation bits in libibverbs - Better document cleanup process in the examples section diff --git a/SRC.txt b/SRC.txt new file mode 100644 index 0000000..b3c0459 --- /dev/null +++ b/SRC.txt @@ -0,0 +1,138 @@ +Here's some documentation on Scalable Reliable Connections. + + * * * + +SRC is an extension supported by recent Mellanox hardware +which is geared toward reducing the number of QPs +required for all-to-all communication on systems +with a high number of jobs per node. + +=================================================================== +Motivation: +=================================================================== +Given N nodes with J jobs per node, number of QPs required +for all-to-all communication is: + +With RC: + O((N * J) ^ 2) + + Since each job out of O(N * J) jobs must create a single QP + to communicate with each one of O(N * J) other jobs. + +With SRC: + O(N ^ 2 * J) + + This is achived by using a single send queue (per job, out of O(N * J) jobs) + to send data to all J jobs running on a specific node (out of O(N) nodes). + Hardware uses new "SRQ number" field in packet header to + multiplex receive WRs and WCs to private memory of each job. + +This is similiar idea to IB RD. +Q: Why not use RD then? +A: Because no hardware supports it. + +Details: + +=================================================================== +Verbs extension: +=================================================================== + +- There is a new transport/QP type "SRC". +- There is a new object type "SRC domain" +- Each SRQ gets new (optional) attributes: + SRC domain + SRC SRQ number + SRC CQ + SRQ must have either all 3 of these or none of these attributes + +- QPs of type SRC have all the same attributes as regular RC QPs + connected to SRQ, except that: + A. Each SRC QP has a new required attribute "SRC domain" + B. SRC QPs do *not* have "SRQ" attribute + (do not have a specific SRQ associated with them) + +=================================================================== +Protocol extension: +=================================================================== +SRC QP behaviour: Requestor +- Post send WR for this QP type is extended with SRQ number field + This number is sent as part of packet header +- SRC Packets follow rules for RC packets on the wire, exactly + What is different is their handling at the responder side + +SRC QP behaviour: Responder +Each incoming packet passes transport checks with respect +to the SRC QP, following RC rules, exactly. + +After this, SRQ number in packet header is used to look up +a specific SRQ. SRC domain of the resulting SRQ must be equal +to SRC domain of the QP, otherwise a NAK is sent, +and QP moves to error state. + +If the SRC domains match, receive WR and receive WC processing +are as follows: + +- RC Send + - Rather than using SRQ to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + Receive WR is taken from this SRQ. + - Completions are generated on the CQ specified in the SRQ + +- RDMA/Atomic + - Rather than using PD to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + PD of this SRQ is used for protection checks. + +=================================================================== +Pseudo code: +=================================================================== + +Consider again a setup where there are N nodes with J jobs per node. +All N * J jobs need to perform all-to-all communication. +Using RC QPs, this would call for O((N * J) ^ 2) QPs. +Here is how SRC can be used to reduce the number of QPs to O(N ^ 2 * J). + +At startup: +1. All jobs on each node share a single SRC domain +2. Each job creates a CQ for receive WCs +3. Each job creates a SRQ attached to this CQ and to the shared domain + +When job j1 needs to transmit to job j2 on remote node n for the first time: +1. Test: does job j1 have an existing connection to some job on node n? + - If no: + j1 creates an SRC QP qp1 (send QP) + qp1 is only used to post send WRs + j2 creates an SRC QP qp2 + qp2 is part of SRC domain + qp2 is only used to do transport checks: + neither send nor receive WRs are posted on qp2 + j1 and j2 create a connection between qp1 and qp2 + - If yes: + let qp1 be the QP which belongs to j1 and is connected + to some qp on node n + +2. j1 gets SRQ number from j2 +3. j1 can now use QP qp2 from step 1 + and SRQ number from step 3 to send data to j2 + +Cleanup: +When job j1 does not need to communicate to any jobs on node n, +it disconnects qp1 from qp2, and asks j2 to destroy qp2. + +Note: both qp1 and qp2 must exist for the communication to take place. +Thus, j2 should not destroy qp2 (and in particular, should not exit) +until j1 has completed communication with node n and has +asked j2 to disconnect. + +=================================================================== + +Resources used (CQs are ignored below): +Each node: +- An SRC domain - to the total of n domains +- A Receive QP for each (remote) job - to the total of N * (N * J) recv QPs + +Each job: +- A SRQ - to the total of N * J SRQs +- A send QP for each (remote) node - to the total of N * (N * J) send QPs + +=================================================================== diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index acc1b82..d7e3269 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -92,7 +92,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_SYS_IMAGE_GUID = 1 << 11, IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, - IBV_DEVICE_N_NOTIFY_CQ = 1 << 14 + IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, + IBV_DEVICE_SRC = 1 << 15 }; enum ibv_atomic_cap { @@ -370,6 +371,11 @@ struct ibv_ah_attr { uint8_t port_num; }; +struct ibv_src_domain { + struct ibv_context *context; + uint32_t handle; +}; + enum ibv_srq_attr_mask { IBV_SRQ_MAX_WR = 1 << 0, IBV_SRQ_LIMIT = 1 << 1 @@ -389,7 +395,8 @@ struct ibv_srq_init_attr { enum ibv_qp_type { IBV_QPT_RC = 2, IBV_QPT_UC, - IBV_QPT_UD + IBV_QPT_UD, + IBV_QPT_SRC }; struct ibv_qp_cap { @@ -408,6 +415,7 @@ struct ibv_qp_init_attr { struct ibv_qp_cap cap; enum ibv_qp_type qp_type; int sq_sig_all; + struct ibv_src_domain *src_domain; }; enum ibv_qp_attr_mask { @@ -526,6 +534,7 @@ struct ibv_send_wr { uint32_t remote_qkey; } ud; } wr; + uint32_t src_remote_srq_num; }; struct ibv_recv_wr { @@ -553,6 +562,10 @@ struct ibv_srq { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + uint32_t src_srq_num; + struct ibv_src_domain *src_domain; + struct ibv_cq *src_cq; }; struct ibv_qp { @@ -570,6 +583,8 @@ struct ibv_qp { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + struct ibv_src_domain *src_domain; }; struct ibv_comp_channel { @@ -652,6 +667,8 @@ struct ibv_context_ops { int (*resize_cq)(struct ibv_cq *cq, int cqe); int (*destroy_cq)(struct ibv_cq *cq); struct ibv_srq * (*create_srq)(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, struct ibv_srq_init_attr *srq_init_attr); int (*modify_srq)(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, @@ -680,6 +697,10 @@ struct ibv_context_ops { int (*detach_mcast)(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); void (*async_event)(struct ibv_async_event *event); + struct ibv_src_domain * (*open_src_domain)(struct ibv_context *context, + int fd, int oflag); + int (*close_src_domain)(struct ibv_src_domain *d); + void (*async_event)(struct ibv_async_event *event); }; struct ibv_context { @@ -912,6 +933,25 @@ struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr *srq_init_attr); /** + * ibv_create_src_srq - Creates a SRQ associated with the specified protection + * domain and src domain. + * @pd: The protection domain associated with the SRQ. + * @src_domain: The SRC domain associated with the SRQ. + * @src_cq: CQ to report completions for SRC packets on. + * + * @srq_init_attr: A list of initial attributes required to create the SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the SRQ, and set to the actual values allocated + * on return. If ibv_create_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ibv_srq *ibv_create_src_srq(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, + struct ibv_srq_init_attr *srq_init_attr); + +/** * ibv_modify_srq - Modifies the attributes for the specified SRQ. * @srq: The SRQ to modify. * @srq_attr: On input, specifies the SRQ attributes to modify. On output, @@ -1074,6 +1114,42 @@ int ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); */ int ibv_fork_init(void); +/** + * ibv_open_src_domain - open an SRC domain + * Returns a reference to an SRC domain. + * + * @context: Device context + * @fd: descriptor for inode associated with the domain + * If fd == -1, no inode is associated with the domain; in this case, + * the only legal value for oflag is O_CREAT + * + * @oflag: oflag values are constructed by OR-ing flags from the following list + * + * O_CREAT + * If a domain belonging to device named by context is already associated + * with the inode, this flag has no effect, except as noted under O_EXCL + * below. Otherwise, a new SRC domain is created and is associated with + * inode specified by fd. + * + * O_EXCL + * If O_EXCL and O_CREAT are set, open will fail if a domain associated with + * the inode exists. The check for the existence of the domain and creation + * of the domain if it does not exist is atomic with respect to other + * processes executing open with fd naming the same inode. + */ +struct ibv_src_domain *ibv_open_src_domain(struct ibv_context *context, + int fd, int oflag); + +/** + * ibv_close_src_domain - close an SRC domain + * If this is the last reference, destroys the domain. + * + * @d: reference to SRC domain to close + * + * close is implicitly performed at process exit. + */ +int ibv_close_src_domain(struct ibv_src_domain *d); + END_C_DECLS # undef __attribute_const diff --git a/src/libibverbs.map b/src/libibverbs.map index 3a346ed..def9ee8 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -24,6 +24,7 @@ IBVERBS_1.0 { ibv_get_cq_event; ibv_ack_cq_events; ibv_create_srq; + ibv_create_src_srq; ibv_modify_srq; ibv_query_srq; ibv_destroy_srq; @@ -35,6 +36,8 @@ IBVERBS_1.0 { ibv_destroy_ah; ibv_attach_mcast; ibv_detach_mcast; + ibv_open_src_domain; + ibv_close_src_domain; ibv_cmd_get_context; ibv_cmd_query_device; ibv_cmd_query_port; diff --git a/src/verbs.c b/src/verbs.c index f5cf4d3..a85f458 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -359,11 +359,39 @@ struct ibv_srq *__ibv_create_srq(struct ibv_pd *pd, if (!pd->context->ops.create_srq) return NULL; - srq = pd->context->ops.create_srq(pd, srq_init_attr); + srq = pd->context->ops.create_srq(pd, NULL, NULL, srq_init_attr); if (srq) { srq->context = pd->context; srq->srq_context = srq_init_attr->srq_context; srq->pd = pd; + srq->src_domain = NULL; + srq->src_cq = NULL; + srq->events_completed = 0; + pthread_mutex_init(&srq->mutex, NULL); + pthread_cond_init(&srq->cond, NULL); + } + + return srq; +} +default_symver(__ibv_create_srq, ibv_create_srq); + +struct ibv_srq *__ibv_create_src_srq(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, + struct ibv_srq_init_attr *srq_init_attr) +{ + struct ibv_srq *srq; + + if (!pd->context->ops.create_srq) + return NULL; + + srq = pd->context->ops.create_srq(pd, src_domain, src_cq, srq_init_attr); + if (srq) { + srq->context = pd->context; + srq->srq_context = srq_init_attr->srq_context; + srq->pd = pd; + srq->src_domain = src_domain; + srq->src_cq = src_cq; srq->events_completed = 0; pthread_mutex_init(&srq->mutex, NULL); pthread_cond_init(&srq->cond, NULL); @@ -541,3 +569,22 @@ int __ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) return qp->context->ops.detach_mcast(qp, gid, lid); } default_symver(__ibv_detach_mcast, ibv_detach_mcast); + +struct ibv_src_domain *__ibv_open_src_domain(struct ibv_context *context, + int fd, int oflag); +{ + struct ibv_src_domain *d; + + d = context->ops.open_src_domain(context, fd, oflag); + if (d) + d->context = context; + + return d; +} +default_symver(__ibv_open_src_domain, ibv_open_src_domain); + +int __ibv_close_src_domain(struct ibv_src_domain *d) +{ + return pd->context->ops.close_src_domain(d); +} +default_symver(__ibv_dealloc_pd, ibv_dealloc_pd); -- MST From vlad at lists.openfabrics.org Wed Aug 8 01:46:17 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 8 Aug 2007 01:46:17 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070808-0100 daily build status Message-ID: <20070808084617.AA261E60846@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Failed: From krkumar2 at in.ibm.com Wed Aug 8 02:32:06 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:02:06 +0530 Subject: [ofa-general] [PATCH 5/9 Rev3] [IPoIB] Header file changes In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093206.15396.44008.sendpatchset@localhost.localdomain> IPoIB header file changes to use batching. Signed-off-by: Krishna Kumar --- ipoib.h | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib.h NEW/drivers/infiniband/ulp/ipoib/ipoib.h --- ORG/drivers/infiniband/ulp/ipoib/ipoib.h 2007-07-12 08:55:06.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-07 13:11:19.000000000 +0530 @@ -266,11 +266,13 @@ struct ipoib_dev_priv { struct ipoib_rx_buf *rx_ring; spinlock_t tx_lock; + spinlock_t comp_lock; /* to handle parallel WC's */ struct ipoib_tx_buf *tx_ring; unsigned tx_head; unsigned tx_tail; - struct ib_sge tx_sge; - struct ib_send_wr tx_wr; + unsigned tx_prev_tail; /* to handle parallel WC's */ + struct ib_sge *tx_sge; + struct ib_send_wr *tx_wr; struct ib_wc ibwc[IPOIB_NUM_WC]; @@ -365,8 +367,11 @@ static inline void ipoib_put_ah(struct i int ipoib_open(struct net_device *dev); int ipoib_add_pkey_attr(struct net_device *dev); +int ipoib_process_skb(struct net_device *dev, struct sk_buff *skb, + struct ipoib_dev_priv *priv, struct ipoib_ah *address, + u32 qpn, int wr_num); void ipoib_send(struct net_device *dev, struct sk_buff *skb, - struct ipoib_ah *address, u32 qpn); + struct ipoib_ah *address, u32 qpn, int num_skbs); void ipoib_reap_ah(struct work_struct *work); void ipoib_flush_paths(struct net_device *dev); From krkumar2 at in.ibm.com Wed Aug 8 02:31:14 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:01:14 +0530 Subject: [ofa-general] [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB Message-ID: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> This set of patches implements the batching API, and adds support for this API in IPoIB. List of changes from original submission: ----------------------------------------- 1. [Patrick] Suggestion to remove tx_queue_len check for enabling batching. 2. [Patrick] Move queue purging to dev_deactivate to free references on device going down. 3. [Patrick] Remove changelog & unrelated changes from sch_generic.c 4. [Patrick] Free skb_blist in unregister_netdev (also suggested to put in free_netdev, but it is not required as unregister_netdev will not fail at this location). 5. [Stephen/Patrick] Remove /sysfs support. 6. [Stephen] Add ethtool support. 7. [Evgeniy] Stop interrupts while changing tx_batch_skb value. 8. [Michael Tsirkin] Remove misleading comment in ipoib_send(). 9. [KK] Remove NETIF_F_BATCH_SKBS (device supports batching if API present). 10. [KK] Remove xmit_slots from netdev. 11. [KK] [IPoIB]: Use unsigned instead of int for index's, handle race between multiple WC's executing on different CPU's by having a new lock (or might need to hold lock for entire duration of WC - some optimization is possible here), changed multiple skb algo to not use xmit_slots, simplify code, minor performance changes wrt slot counters, etc. List of changes implemented, tested and dropped: ------------------------------------------------ 1. [Patrick] Suggestion to use skb_blist statically in netdevice. This reduces performance (~ 1%) (possibly due to having an extra check for dev->hard_start_xmit_batch API). 2. [Patrick] Suggestion to check if hard_start_xmit_batch can be removed: This reduces performance as a call to a non inline function is made, and an extra check in driver to see if skb is NULL. 3. [Sridhar] Suggestion to always use batching for regular xmit case too: While testing, for some reason the tests virtually hangs and transfers almost no data for higher number of proceses (like 64 and above). Patches are described as: Mail 0/9: This mail Mail 1/9: HOWTO documentation Mail 2/9: Introduce skb_blist and hard_start_xmit_batch API Mail 3/9: Modify qdisc_run() to support batching Mail 4/9: Add ethtool support to enable/disable batching Mail 5/9: IPoIB header file changes to use batching Mail 6/9: IPoIB CM & Multicast changes Mail 7/9: IPoIB verb changes to use batching Mail 8/9: IPoIB internal post and work completion handler Mail 9/9: Implement the new batching API RESULTS: The performance improvement for TCP No Delay is in the range of -8% to 320% (with -8% being the sole negative), with many individual tests giving 50% or more improvement (I think it is to do with the hw slots getting full quicker resulting in more batching when the queue gets woken). The results for TCP is in the range of -11% to 93%, with most of the tests (8/12) giving improvements. ISSUES: I am getting a huge amount of retransmissions for both TCP and TCP No Delay cases for IPoIB (which explains the slight degradation for some test cases mentioned above). After a full test run, the regular code resulted in 74 retransmissions, while there were 1365716 retrans with batching code - or 18500 retransmissions for every 1 in regular code. But with this huge amount of retransmissions there is 20.7% overall improvement in BW (which implies batching will improve results even more if this problem is fixed). I suspect this is some issue in the driver/firmware since: a. I see similar low retransmissions numbers for E1000 (so no bug in core changes). b. Even with batching set to maximum 2 skbs, I get almost the same number of retransmissions (implies receiver is probably not dropping skbs). ifconfig/netstat on receiver gives no clue (drop/errors, etc). This issue delayed submitting patches for the last 2 weeks, as I was trying to debug this; any help from openIB community is appreciated. Please review and provide feedback; and consider for inclusion. Thanks, - KK --------------------------------------------------------------- Test Case ORG NEW % Change --------------------------------------------------------------- TCP --- Size:32 Procs:1 2709 4217 55.66 Size:128 Procs:1 10950 15853 44.77 Size:512 Procs:1 35313 68224 93.19 Size:4096 Procs:1 118144 119935 1.51 Size:32 Procs:8 18976 22432 18.21 Size:128 Procs:8 66351 86072 29.72 Size:512 Procs:8 246546 234373 -4.93 Size:4096 Procs:8 268861 251540 -6.44 Size:32 Procs:16 35009 45861 30.99 Size:128 Procs:16 150979 164961 9.26 Size:512 Procs:16 259443 230730 -11.06 Size:4096 Procs:16 265313 246794 -6.98 TCP No Delay ------------ Size:32 Procs:1 1930 1944 .72 Size:128 Procs:1 8573 7831 -8.65 Size:512 Procs:1 28536 29347 2.84 Size:4096 Procs:1 98916 104236 5.37 Size:32 Procs:8 4173 17560 320.80 Size:128 Procs:8 17350 66205 281.58 Size:512 Procs:8 69777 211467 203.06 Size:4096 Procs:8 201096 242578 20.62 Size:32 Procs:16 20570 37778 83.65 Size:128 Procs:16 95005 154464 62.58 Size:512 Procs:16 111677 221570 98.40 Size:4096 Procs:16 204765 240368 17.38 --------------------------------------------------------------- Overall: 2340962 2826340 20.73% [Summary: 19 Better cases, 5 worse] Testing environment (on client, server uses 4096 sendq size): echo "Using 512 size sendq" modprobe ib_ipoib send_queue_size=512 recv_queue_size=512 echo "4096 524288 4194304" > /proc/sys/net/ipv4/tcp_wmem echo "4096 1048576 4194304" > /proc/sys/net/ipv4/tcp_rmem echo 4194304 > /proc/sys/net/core/rmem_max echo 4194304 > /proc/sys/net/core/wmem_max echo 120000 > /proc/sys/net/core/netdev_max_backlog From krkumar2 at in.ibm.com Wed Aug 8 02:31:35 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:01:35 +0530 Subject: [ofa-general] [PATCH 2/9 Rev3] [core] Add skb_blist & hard_start_xmit_batch In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093135.15396.29687.sendpatchset@localhost.localdomain> Introduce skb_blist and hard_start_xmit_batch API, handle driver's usage of the new API, and add support routines. Signed-off-by: Krishna Kumar --- include/linux/netdevice.h | 8 +++ net/core/dev.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) diff -ruNp ORG/include/linux/netdevice.h NEW/include/linux/netdevice.h --- ORG/include/linux/netdevice.h 2007-08-06 08:25:37.000000000 +0530 +++ NEW/include/linux/netdevice.h 2007-08-07 13:11:19.000000000 +0530 @@ -456,6 +456,9 @@ struct net_device /* Partially transmitted GSO packet. */ struct sk_buff *gso_skb; + /* List of batch skbs (optional, used if driver supports batching API */ + struct sk_buff_head *skb_blist; + /* ingress path synchronizer */ spinlock_t ingress_lock; struct Qdisc *qdisc_ingress; @@ -472,6 +475,9 @@ struct net_device void *priv; /* pointer to private data */ int (*hard_start_xmit) (struct sk_buff *skb, struct net_device *dev); + int (*hard_start_xmit_batch) (struct net_device + *dev); + /* These may be needed for future network-power-down code. */ unsigned long trans_start; /* Time (in jiffies) of last Tx */ @@ -832,6 +838,8 @@ extern int dev_set_mac_address(struct n struct sockaddr *); extern int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev); +extern int dev_add_skb_to_blist(struct sk_buff *skb, + struct net_device *dev); extern void dev_init(void); diff -ruNp ORG/net/core/dev.c NEW/net/core/dev.c --- ORG/net/core/dev.c 2007-08-06 08:25:40.000000000 +0530 +++ NEW/net/core/dev.c 2007-08-07 13:11:19.000000000 +0530 @@ -897,6 +897,55 @@ void netdev_state_change(struct net_devi } } +static void free_batching(struct net_device *dev) +{ + if (dev->skb_blist) { + if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + kfree(dev->skb_blist); + dev->skb_blist = NULL; + } +} + +int dev_change_tx_batch_skb(struct net_device *dev, unsigned long new_batch_skb) +{ + int ret = 0; + struct sk_buff_head *blist; + + if (!dev->hard_start_xmit_batch) { + /* Driver doesn't support batching skb API */ + ret = -ENOTSUPP; + goto out; + } + + /* Handle invalid argument */ + if (new_batch_skb < 0) { + ret = -EINVAL; + goto out; + } + + /* Check if new value is same as the current */ + if (!!dev->skb_blist == !!new_batch_skb) + goto out; + + if (new_batch_skb && + (blist = kmalloc(sizeof *blist, GFP_KERNEL)) == NULL) { + ret = -ENOMEM; + goto out; + } + + spin_lock(&dev->queue_lock); + if (new_batch_skb) { + skb_queue_head_init(blist); + dev->skb_blist = blist; + } else + free_batching(dev); + spin_unlock(&dev->queue_lock); + +out: + return ret; +} + /** * dev_load - load a network module * @name: name of interface @@ -1459,6 +1508,45 @@ static int dev_gso_segment(struct sk_buf return 0; } +/* + * Add skb (skbs in case segmentation is required) to dev->skb_blist. We are + * holding QDISC RUNNING bit, so no one else can add to this list. Also, skbs + * are dequeued from this list when we call the driver, so the list is safe + * from simultaneous deletes too. + * + * Returns count of successful skb(s) added to skb_blist. + */ +int dev_add_skb_to_blist(struct sk_buff *skb, struct net_device *dev) +{ + if (!list_empty(&ptype_all)) + dev_queue_xmit_nit(skb, dev); + + if (netif_needs_gso(dev, skb)) { + if (unlikely(dev_gso_segment(skb))) { + kfree(skb); + return 0; + } + + if (skb->next) { + int count = 0; + + do { + struct sk_buff *nskb = skb->next; + + skb->next = nskb->next; + __skb_queue_tail(dev->skb_blist, nskb); + count++; + } while (skb->next); + + skb->destructor = DEV_GSO_CB(skb)->destructor; + kfree_skb(skb); + return count; + } + } + __skb_queue_tail(dev->skb_blist, skb); + return 1; +} + int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev) { if (likely(!skb->next)) { @@ -3446,6 +3535,13 @@ int register_netdevice(struct net_device } } + if (dev->hard_start_xmit_batch) { + /* Driver supports batching skb API */ + dev->skb_blist = kmalloc(sizeof *dev->skb_blist, GFP_KERNEL); + if (dev->skb_blist) + skb_queue_head_init(dev->skb_blist); + } + /* * nil rebuild_header routine, * that should be never called and used as just bug trap. @@ -3787,6 +3882,9 @@ void unregister_netdevice(struct net_dev synchronize_net(); + /* Deallocate batching structure */ + free_batching(dev); + /* Shutdown queueing discipline. */ dev_shutdown(dev); From krkumar2 at in.ibm.com Wed Aug 8 02:32:35 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:02:35 +0530 Subject: [ofa-general] [PATCH 8/9 Rev3] [IPoIB] Post and work completion handler changes In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093235.15396.3273.sendpatchset@localhost.localdomain> IPoIB internal post and work completion handler changes. Signed-off-by: Krishna Kumar --- ipoib_ib.c | 217 ++++++++++++++++++++++++++++++++++++++++++++++++------------- 1 files changed, 173 insertions(+), 44 deletions(-) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib_ib.c NEW/drivers/infiniband/ulp/ipoib/ipoib_ib.c --- ORG/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-07-17 08:48:35.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-07 13:11:19.000000000 +0530 @@ -242,6 +242,8 @@ repost: static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); + int i, num_completions; + unsigned int tx_ring_index; unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; unsigned long flags; @@ -255,18 +257,56 @@ static void ipoib_ib_handle_tx_wc(struct return; } - tx_req = &priv->tx_ring[wr_id]; + /* + * Handle skbs completion from tx_tail to wr_id. Two issues : + * - Need to stop other WC's from mangling same skb's if they + * run at the same time. Use tx_prev_tail to demarcate WC's. + * - Handle WC's from earlier (possibly multiple) post_sends in + * this iteration as we move from tx_prev_tail to wr_id, since + * if the last WR (which is the one which requested completion + * notification) failed to be sent for any of those earlier + * request(s), no completion notification is generated for + * successful WR's of those earlier request(s). + */ + spin_lock_irqsave(&priv->comp_lock, flags); + + /* Get start index */ + tx_ring_index = priv->tx_prev_tail & (ipoib_sendq_size - 1); + + /*Find number of WC's */ + num_completions = wr_id - tx_ring_index + 1; + if (unlikely(num_completions <= 0)) + num_completions += ipoib_sendq_size; + + /* Save new start index for any parallel WC's */ + priv->tx_prev_tail += num_completions; + + spin_unlock_irqrestore(&priv->comp_lock, flags); - ib_dma_unmap_single(priv->ca, tx_req->mapping, - tx_req->skb->len, DMA_TO_DEVICE); + tx_req = &priv->tx_ring[tx_ring_index]; + for (i = 0; i < num_completions; i++) { + if (likely(tx_req->skb)) { + ib_dma_unmap_single(priv->ca, tx_req->mapping, + tx_req->skb->len, DMA_TO_DEVICE); - ++priv->stats.tx_packets; - priv->stats.tx_bytes += tx_req->skb->len; + ++priv->stats.tx_packets; + priv->stats.tx_bytes += tx_req->skb->len; - dev_kfree_skb_any(tx_req->skb); + dev_kfree_skb_any(tx_req->skb); + } + /* + * else this skb failed synchronously when posted and was + * freed immediately. + */ + + if (likely(++tx_ring_index != ipoib_sendq_size)) + tx_req++; + else + tx_req = &priv->tx_ring[0]; + } spin_lock_irqsave(&priv->tx_lock, flags); - ++priv->tx_tail; + priv->tx_tail += num_completions; if (unlikely(test_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags)) && priv->tx_head - priv->tx_tail <= ipoib_sendq_size >> 1) { clear_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); @@ -340,29 +380,57 @@ void ipoib_ib_completion(struct ib_cq *c netif_rx_schedule(dev_ptr); } -static inline int post_send(struct ipoib_dev_priv *priv, - unsigned int wr_id, - struct ib_ah *address, u32 qpn, - u64 addr, int len) +/* + * post_send : Post WR(s) to the device. + * + * num_skbs is the number of WR's, first_wr is the first slot in tx_wr[] (or + * tx_sge[]). first_wr is normally zero unless a previous post_send returned + * error and we are trying to post the untried WR's, in which case first_wr + * is the index to the first untried WR. + * + * Break the WR link before posting so that provider knows how many WR's to + * process, and this is set back after the post. + */ +static inline int post_send(struct ipoib_dev_priv *priv, u32 qpn, + int first_wr, int num_skbs, + struct ib_send_wr **bad_wr) { - struct ib_send_wr *bad_wr; + int ret; + struct ib_send_wr *last_wr, *next_wr; + + last_wr = &priv->tx_wr[first_wr + num_skbs - 1]; + + /* Set Completion Notification for last WR */ + last_wr->send_flags = IB_SEND_SIGNALED; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + /* Terminate the last WR */ + next_wr = last_wr->next; + last_wr->next = NULL; - priv->tx_wr.wr_id = wr_id; - priv->tx_wr.wr.ud.remote_qpn = qpn; - priv->tx_wr.wr.ud.ah = address; + /* Send all the WR's in one doorbell */ + ret = ib_post_send(priv->qp, &priv->tx_wr[first_wr], bad_wr); - return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); + /* Restore send_flags & WR chain */ + last_wr->send_flags = 0; + last_wr->next = next_wr; + + return ret; } -void ipoib_send(struct net_device *dev, struct sk_buff *skb, - struct ipoib_ah *address, u32 qpn) +/* + * Map skb & store skb/mapping in tx_ring; and details of the WR in tx_wr + * to pass to the provider. + * + * Returns: + * 1: Error and the skb is freed. + * 0 skb processed successfully. + */ +int ipoib_process_skb(struct net_device *dev, struct sk_buff *skb, + struct ipoib_dev_priv *priv, struct ipoib_ah *address, + u32 qpn, int wr_num) { - struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ipoib_tx_buf *tx_req; u64 addr; + unsigned int tx_ring_index; if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -370,7 +438,7 @@ void ipoib_send(struct net_device *dev, ++priv->stats.tx_dropped; ++priv->stats.tx_errors; ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + return 1; } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -383,35 +451,96 @@ void ipoib_send(struct net_device *dev, * means we have to make sure everything is properly recorded and * our state is consistent before we call post_send(). */ - tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; - tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, - DMA_TO_DEVICE); + addr = ib_dma_map_single(priv->ca, skb->data, skb->len, DMA_TO_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); - return; + return 1; } - tx_req->mapping = addr; - if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, addr, skb->len))) { - ipoib_warn(priv, "post_send failed\n"); - ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); - dev_kfree_skb_any(skb); - } else { - dev->trans_start = jiffies; + tx_ring_index = priv->tx_head & (ipoib_sendq_size - 1); + + /* Save till completion handler executes */ + priv->tx_ring[tx_ring_index].skb = skb; + priv->tx_ring[tx_ring_index].mapping = addr; + + /* Set WR values for the provider to use */ + priv->tx_sge[wr_num].addr = addr; + priv->tx_sge[wr_num].length = skb->len; + + priv->tx_wr[wr_num].wr_id = tx_ring_index; + priv->tx_wr[wr_num].wr.ud.remote_qpn = qpn; + priv->tx_wr[wr_num].wr.ud.ah = address->ah; + + priv->tx_head++; + + if (unlikely(priv->tx_head - priv->tx_tail == ipoib_sendq_size)) { + ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); + netif_stop_queue(dev); + set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + } - address->last_send = priv->tx_head; - ++priv->tx_head; + return 0; +} - if (priv->tx_head - priv->tx_tail == ipoib_sendq_size) { - ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); - netif_stop_queue(dev); - set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); +/* + * Send num_skbs to the device. If an skb is passed to this function, it is + * single, unprocessed skb send case; otherwise it means that all skbs are + * already processed and put on priv->tx_wr,tx_sge,tx_ring, etc. + */ +void ipoib_send(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn, int num_skbs) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int first_wr = 0; + + if (skb && ipoib_process_skb(dev, skb, priv, address, qpn, 0)) + return; + + /* Send all skb's in one post */ + do { + struct ib_send_wr *bad_wr; + + if (unlikely((post_send(priv, qpn, first_wr, num_skbs, + &bad_wr)))) { + int done; + + ipoib_warn(priv, "post_send failed\n"); + + /* Get number of WR's that finished successfully */ + done = bad_wr - &priv->tx_wr[first_wr]; + + /* Handle 1 error */ + priv->stats.tx_errors++; + ib_dma_unmap_single(priv->ca, + priv->tx_sge[first_wr + done].addr, + priv->tx_sge[first_wr + done].length, + DMA_TO_DEVICE); + + /* Free failed WR & reset for WC handler to recognize */ + dev_kfree_skb_any(priv->tx_ring[bad_wr->wr_id].skb); + priv->tx_ring[bad_wr->wr_id].skb = NULL; + + /* Handle 'n' successes */ + if (done) { + dev->trans_start = jiffies; + address->last_send = priv->tx_head - (num_skbs - + done) - 1; + } + + /* Get count of skbs that were not tried */ + num_skbs -= (done + 1); + /* + 1 for WR that was tried & failed */ + + /* Get start index for next iteration */ + first_wr += (done + 1); + } else { + dev->trans_start = jiffies; + + address->last_send = priv->tx_head - 1; + num_skbs = 0; } - } + } while (num_skbs); } static void __ipoib_reap_ah(struct net_device *dev) From krkumar2 at in.ibm.com Wed Aug 8 02:31:24 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:01:24 +0530 Subject: [ofa-general] [PATCH 1/9 Rev3] [Doc] HOWTO Documentation for batching In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093124.15396.69224.sendpatchset@localhost.localdomain> Add Documentation describing batching API. Signed-off-by: Krishna Kumar --- Batching_skb_API.txt | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 82 insertions(+) diff -ruNp ORG/Documentation/networking/Batching_skb_API.txt NEW/Documentation/networking/Batching_skb_API.txt --- ORG/Documentation/networking/Batching_skb_API.txt 1970-01-01 05:30:00.000000000 +0530 +++ NEW/Documentation/networking/Batching_skb_API.txt 2007-08-07 22:41:55.000000000 +0530 @@ -0,0 +1,82 @@ + HOWTO for batching skb API support + ----------------------------------- + +Section 1: What is batching skb API ? +Section 2: How batching API works vs the original API ? +Section 3: How drivers can support this API ? +Section 4: How users can work with this API ? + + +Introduction: Kernel support for batching skb +---------------------------------------------- + +A new xmit API - hard_start_xmit_batch() is provided in the netdevice layer +similar to the existing hard_start_xmit() API. Drivers which export this +API can implement it similar to the hard_start_xmit handler. The new API +should process multiple skbs (or even one) in a single call while the +existing hard_start_xmit processes one skb. It is possible for the driver +writer to re-use most of the code from the existing API in the new API +without having code duplication. + + +Section 1: What is batching skb API ? +------------------------------------- + + This API is optionally exported by a driver. The pre-requisite for a + driver to use this API is that it should have a reasonably sized + hardware queue that can process multiple skbs. + + +Section 2: How batching API works vs the original API ? +------------------------------------------------------- + + The networking stack gets called from upper layer protocols with a + single skb to transmit. This skb is first enqueue'd and an attempt is + made to transmit it immediately (via qdisc_run). However, events like + tx lock contention, tx queue stopped, etc, can result in the skb not + getting sent out and it remains in the queue. When the next xmit is + called or when the queue is re-enabled, qdisc_run could potentially + find multiple packets in the queue, and iteratively send them all out + one-by-one. + + The batching skb API was added to exploit this situation where all + skbs can be passed in one shot to the device. This reduces driver + processing, locking at the driver (or in stack for ~LLTX drivers) + gets amortized over multiple skbs, and in case of specific drivers + where every xmit results in a completion processing (like IPoIB), + optimizations can be made in the driver to request a completion for + only the last skb that was sent which results in saving interrupts + for every (but the last) skb that was sent in the same batch. + + Batching can result in significant performance gains for systems that + have multiple data stream paths over the same network interface card. + + +Section 3: How drivers can support this API ? +--------------------------------------------- + + The new API - dev->hard_start_xmit_batch(struct net_device *dev), + simplistically, can be written almost identically to the regular + xmit API except that multiple skbs should be processed by the driver + instead of one skb. The new API doesn't get a skb as an argument, + instead it picks up all the skbs from dev->skb_blist, where it was + added by the core stack, and tries to send them out. + + Batching requires the driver to set dev->hard_start_xmit_batch to the + new API implemented for that driver. + + +Section 4: How users can work with this API ? +--------------------------------------------- + + Batching could be disabled for a particular device, e.g. on desktop + systems if only one stream of network activity for that device is + taking place, since performance could be slightly affected due to + extra processing that batching adds (unless packets are getting + sent fast resulting in stopped queue's). Batching can be enabled if + more than one stream of network activity per device is being done, + e.g. on servers; or even desktop usage with multiple browser, chat, + file transfer sessions, etc. + + Per device batching can be enabled/disabled using ethtool, where + passing 1 enables batching and passing 0 disables batching. From krkumar2 at in.ibm.com Wed Aug 8 02:32:52 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:02:52 +0530 Subject: [ofa-general] [PATCH 9/9 Rev3] [IPoIB] Implement the new batching API In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093252.15396.89948.sendpatchset@localhost.localdomain> IPoIB: implement the new batching API. Signed-off-by: Krishna Kumar --- ipoib_main.c | 189 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 184 insertions(+), 5 deletions(-) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib_main.c NEW/drivers/infiniband/ulp/ipoib/ipoib_main.c --- ORG/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-07-12 08:55:06.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-07 13:11:19.000000000 +0530 @@ -558,7 +558,8 @@ static void neigh_add_path(struct sk_buf goto err_drop; } } else - ipoib_send(dev, skb, path->ah, IPOIB_QPN(skb->dst->neighbour->ha)); + ipoib_send(dev, skb, path->ah, + IPOIB_QPN(skb->dst->neighbour->ha), 1); } else { neigh->ah = NULL; @@ -638,7 +639,7 @@ static void unicast_arp_send(struct sk_b ipoib_dbg(priv, "Send unicast ARP to %04x\n", be16_to_cpu(path->pathrec.dlid)); - ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr)); + ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr), 1); } else if ((path->query || !path_rec_start(dev, path)) && skb_queue_len(&path->queue) < IPOIB_MAX_PATH_REC_QUEUE) { /* put pseudoheader back on for next time */ @@ -704,7 +705,8 @@ static int ipoib_start_xmit(struct sk_bu goto out; } - ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(skb->dst->neighbour->ha)); + ipoib_send(dev, skb, neigh->ah, + IPOIB_QPN(skb->dst->neighbour->ha), 1); goto out; } @@ -753,6 +755,153 @@ out: return NETDEV_TX_OK; } +#define XMIT_QUEUED_SKBS() \ + do { \ + if (wr_num) { \ + ipoib_send(dev, NULL, old_neigh->ah, old_qpn, \ + wr_num); \ + wr_num = 0; \ + } \ + } while (0) + +/* + * TODO: Merge with ipoib_start_xmit to use the same code and have a + * transparent wrapper caller to xmit's, etc. Status: Done, needs testing. + */ +static int ipoib_start_xmit_frames(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct sk_buff *skb; + struct sk_buff_head *blist = dev->skb_blist; + int max_skbs, wr_num = 0; + u32 qpn, old_qpn = 0; + struct ipoib_neigh *neigh, *old_neigh = NULL; + unsigned long flags; + + if (unlikely(!spin_trylock_irqsave(&priv->tx_lock, flags))) + return NETDEV_TX_LOCKED; + + /* + * Figure out how many skbs can be sent. This prevents the device + * getting full and avoids checking for queue stopped after each + * iteration. + */ + max_skbs = ipoib_sendq_size - (priv->tx_head - priv->tx_tail); + while (max_skbs-- > 0 && (skb = __skb_dequeue(blist)) != NULL) { + if (likely(skb->dst && skb->dst->neighbour)) { + if (unlikely(!*to_ipoib_neigh(skb->dst->neighbour))) { + XMIT_QUEUED_SKBS(); + ipoib_path_lookup(skb, dev); + continue; + } + + neigh = *to_ipoib_neigh(skb->dst->neighbour); + + if (ipoib_cm_get(neigh)) { + if (ipoib_cm_up(neigh)) { + XMIT_QUEUED_SKBS(); + ipoib_cm_send(dev, skb, + ipoib_cm_get(neigh)); + continue; + } + } else if (neigh->ah) { + if (unlikely(memcmp(&neigh->dgid.raw, + skb->dst->neighbour->ha + 4, + sizeof(union ib_gid)))) { + spin_lock(&priv->lock); + /* + * It's safe to call ipoib_put_ah() + * inside priv->lock here, because we + * know that path->ah will always hold + * one more reference, so ipoib_put_ah() + * will never do more than decrement + * the ref count. + */ + ipoib_put_ah(neigh->ah); + list_del(&neigh->list); + ipoib_neigh_free(dev, neigh); + spin_unlock(&priv->lock); + XMIT_QUEUED_SKBS(); + ipoib_path_lookup(skb, dev); + continue; + } + + qpn = IPOIB_QPN(skb->dst->neighbour->ha); + if (neigh != old_neigh || qpn != old_qpn) { + /* + * Sending to a different destination + * from earlier skb's - send all + * existing skbs (if any), and restart. + */ + XMIT_QUEUED_SKBS(); + old_neigh = neigh; + old_qpn = qpn; + } + + if (likely(!ipoib_process_skb(dev, skb, priv, + neigh->ah, qpn, + wr_num))) + wr_num++; + + continue; + } + + if (skb_queue_len(&neigh->queue) < + IPOIB_MAX_PATH_REC_QUEUE) { + spin_lock(&priv->lock); + __skb_queue_tail(&neigh->queue, skb); + spin_unlock(&priv->lock); + } else { + dev_kfree_skb_any(skb); + ++priv->stats.tx_dropped; + ++max_skbs; + } + } else { + struct ipoib_pseudoheader *phdr = + (struct ipoib_pseudoheader *) skb->data; + skb_pull(skb, sizeof *phdr); + + if (phdr->hwaddr[4] == 0xff) { + /* Add in the P_Key for multicast*/ + phdr->hwaddr[8] = (priv->pkey >> 8) & 0xff; + phdr->hwaddr[9] = priv->pkey & 0xff; + + XMIT_QUEUED_SKBS(); + ipoib_mcast_send(dev, phdr->hwaddr + 4, skb); + } else { + /* unicast GID -- should be ARP or RARP reply */ + + if ((be16_to_cpup((__be16 *) skb->data) != + ETH_P_ARP) && + (be16_to_cpup((__be16 *) skb->data) != + ETH_P_RARP)) { + ipoib_warn(priv, "Unicast, no %s: type %04x, QPN %06x " + IPOIB_GID_FMT "\n", + skb->dst ? "neigh" : "dst", + be16_to_cpup((__be16 *) + skb->data), + IPOIB_QPN(phdr->hwaddr), + IPOIB_GID_RAW_ARG(phdr->hwaddr + + 4)); + dev_kfree_skb_any(skb); + ++priv->stats.tx_dropped; + ++max_skbs; + continue; + } + XMIT_QUEUED_SKBS(); + unicast_arp_send(skb, dev, phdr); + } + } + } + + /* Send out last packets (if any) */ + XMIT_QUEUED_SKBS(); + + spin_unlock_irqrestore(&priv->tx_lock, flags); + + return skb_queue_empty(blist) ? NETDEV_TX_OK : NETDEV_TX_BUSY; +} + static struct net_device_stats *ipoib_get_stats(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -896,13 +1045,37 @@ int ipoib_dev_init(struct net_device *de goto out_rx_ring_cleanup; } - /* priv->tx_head & tx_tail are already 0 */ + /* priv->tx_head & tx_tail & tx_priv_tail are already 0 */ - if (ipoib_ib_dev_init(dev, ca, port)) + /* Allocate tx_sge */ + priv->tx_sge = kmalloc(ipoib_sendq_size * sizeof *priv->tx_sge, + GFP_KERNEL); + if (!priv->tx_sge) { + printk(KERN_WARNING "%s: failed to allocate TX sge (%d entries)\n", + ca->name, ipoib_sendq_size); goto out_tx_ring_cleanup; + } + + /* Allocate tx_wr */ + priv->tx_wr = kmalloc(ipoib_sendq_size * sizeof *priv->tx_wr, + GFP_KERNEL); + if (!priv->tx_wr) { + printk(KERN_WARNING "%s: failed to allocate TX wr (%d entries)\n", + ca->name, ipoib_sendq_size); + goto out_tx_sge_cleanup; + } + + if (ipoib_ib_dev_init(dev, ca, port)) + goto out_tx_wr_cleanup; return 0; +out_tx_wr_cleanup: + kfree(priv->tx_wr); + +out_tx_sge_cleanup: + kfree(priv->tx_sge); + out_tx_ring_cleanup: kfree(priv->tx_ring); @@ -930,9 +1103,13 @@ void ipoib_dev_cleanup(struct net_device kfree(priv->rx_ring); kfree(priv->tx_ring); + kfree(priv->tx_sge); + kfree(priv->tx_wr); priv->rx_ring = NULL; priv->tx_ring = NULL; + priv->tx_sge = NULL; + priv->tx_wr = NULL; } static void ipoib_setup(struct net_device *dev) @@ -943,6 +1120,7 @@ static void ipoib_setup(struct net_devic dev->stop = ipoib_stop; dev->change_mtu = ipoib_change_mtu; dev->hard_start_xmit = ipoib_start_xmit; + dev->hard_start_xmit_batch = ipoib_start_xmit_frames; dev->get_stats = ipoib_get_stats; dev->tx_timeout = ipoib_timeout; dev->hard_header = ipoib_hard_header; @@ -979,6 +1157,7 @@ static void ipoib_setup(struct net_devic spin_lock_init(&priv->lock); spin_lock_init(&priv->tx_lock); + spin_lock_init(&priv->comp_lock); mutex_init(&priv->mcast_mutex); mutex_init(&priv->vlan_mutex); From krkumar2 at in.ibm.com Wed Aug 8 02:31:56 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:01:56 +0530 Subject: [ofa-general] [PATCH 4/9 Rev3] [ethtool] Add ethtool support In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093156.15396.80149.sendpatchset@localhost.localdomain> Add ethtool support to enable/disable batching. Signed-off-by: Krishna Kumar --- include/linux/ethtool.h | 2 ++ include/linux/netdevice.h | 2 ++ net/core/ethtool.c | 26 ++++++++++++++++++++++++++ 3 files changed, 30 insertions(+) diff -ruNp ORG/include/linux/ethtool.h NEW/include/linux/ethtool.h --- ORG/include/linux/ethtool.h 2007-08-06 08:25:37.000000000 +0530 +++ NEW/include/linux/ethtool.h 2007-08-07 13:11:19.000000000 +0530 @@ -410,6 +410,8 @@ struct ethtool_ops { #define ETHTOOL_SUFO 0x00000022 /* Set UFO enable (ethtool_value) */ #define ETHTOOL_GGSO 0x00000023 /* Get GSO enable (ethtool_value) */ #define ETHTOOL_SGSO 0x00000024 /* Set GSO enable (ethtool_value) */ +#define ETHTOOL_GBATCH 0x00000024 /* Get Batch (ethtool_value) */ +#define ETHTOOL_SBATCH 0x00000025 /* Set Batch (ethtool_value) */ /* compatibility with older code */ #define SPARC_ETH_GSET ETHTOOL_GSET diff -ruNp ORG/include/linux/netdevice.h NEW/include/linux/netdevice.h --- ORG/include/linux/netdevice.h 2007-08-06 08:25:37.000000000 +0530 +++ NEW/include/linux/netdevice.h 2007-08-07 13:11:19.000000000 +0530 @@ -1112,6 +1112,8 @@ extern void dev_set_promiscuity(struct extern void dev_set_allmulti(struct net_device *dev, int inc); extern void netdev_state_change(struct net_device *dev); extern void netdev_features_change(struct net_device *dev); +extern int dev_change_tx_batch_skb(struct net_device *dev, + unsigned long new_batch_skb); /* Load a device via the kmod */ extern void dev_load(const char *name); extern void dev_mcast_init(void); diff -ruNp ORG/net/core/ethtool.c NEW/net/core/ethtool.c --- ORG/net/core/ethtool.c 2007-08-06 08:25:40.000000000 +0530 +++ NEW/net/core/ethtool.c 2007-08-07 13:11:19.000000000 +0530 @@ -638,6 +638,25 @@ static int ethtool_set_gso(struct net_de return 0; } +static int ethtool_get_batch(struct net_device *dev, char __user *useraddr) +{ + struct ethtool_value edata = { ETHTOOL_GBATCH }; + + edata.data = dev->skb_blist != NULL; + if (copy_to_user(useraddr, &edata, sizeof(edata))) + return -EFAULT; + return 0; +} + +static int ethtool_set_gso(struct net_device *dev, char __user *useraddr) +{ + struct ethtool_value edata; + + if (copy_from_user(&edata, useraddr, sizeof(edata))) + return -EFAULT; + return dev_change_tx_batch_skb(dev, edata.data); +} + static int ethtool_self_test(struct net_device *dev, char __user *useraddr) { struct ethtool_test test; @@ -817,6 +836,7 @@ int dev_ethtool(struct ifreq *ifr) case ETHTOOL_GPERMADDR: case ETHTOOL_GUFO: case ETHTOOL_GGSO: + case ETHTOOL_GBATCH: break; default: if (!capable(CAP_NET_ADMIN)) @@ -935,6 +955,12 @@ int dev_ethtool(struct ifreq *ifr) case ETHTOOL_SGSO: rc = ethtool_set_gso(dev, useraddr); break; + case ETHTOOL_GBATCH: + rc = ethtool_get_batch(dev, useraddr); + break; + case ETHTOOL_SBATCH: + rc = ethtool_set_batch(dev, useraddr); + break; default: rc = -EOPNOTSUPP; } From tziporet at dev.mellanox.co.il Wed Aug 8 02:36:15 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Wed, 08 Aug 2007 12:36:15 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c status & plans In-Reply-To: References: <46B8C28D.7020108@mellanox.co.il> Message-ID: <46B98E8F.3040509@mellanox.co.il> Scott Weitzenkamp (sweitzen) wrote: > Cisco has been testing 1.2.c-10 IPoIB/SDP/MPI successfully on a > 32-node cluster. We are still working on tvflash, though. > > We can add tvflash at beginning of next week Tziporet From krkumar2 at in.ibm.com Wed Aug 8 02:31:45 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:01:45 +0530 Subject: [ofa-general] [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093145.15396.91711.sendpatchset@localhost.localdomain> Modify qdisc_run() to support batching. Modify callers of qdisc_run to use batching, modify qdisc_restart to implement batching. Signed-off-by: Krishna Kumar --- include/net/pkt_sched.h | 6 +-- net/core/dev.c | 5 +-- net/sched/sch_generic.c | 77 +++++++++++++++++++++++++++++++++++++++--------- 3 files changed, 69 insertions(+), 19 deletions(-) diff -ruNp ORG/include/net/pkt_sched.h NEW/include/net/pkt_sched.h --- ORG/include/net/pkt_sched.h 2007-07-17 08:48:37.000000000 +0530 +++ NEW/include/net/pkt_sched.h 2007-08-07 13:11:19.000000000 +0530 @@ -80,13 +80,13 @@ extern struct qdisc_rate_table *qdisc_ge struct rtattr *tab); extern void qdisc_put_rtab(struct qdisc_rate_table *tab); -extern void __qdisc_run(struct net_device *dev); +extern void __qdisc_run(struct net_device *dev, struct sk_buff_head *blist); -static inline void qdisc_run(struct net_device *dev) +static inline void qdisc_run(struct net_device *dev, struct sk_buff_head *blist) { if (!netif_queue_stopped(dev) && !test_and_set_bit(__LINK_STATE_QDISC_RUNNING, &dev->state)) - __qdisc_run(dev); + __qdisc_run(dev, blist); } extern int tc_classify_compat(struct sk_buff *skb, struct tcf_proto *tp, diff -ruNp ORG/net/sched/sch_generic.c NEW/net/sched/sch_generic.c --- ORG/net/sched/sch_generic.c 2007-07-12 08:55:20.000000000 +0530 +++ NEW/net/sched/sch_generic.c 2007-08-07 13:11:19.000000000 +0530 @@ -59,10 +59,12 @@ static inline int qdisc_qlen(struct Qdis static inline int dev_requeue_skb(struct sk_buff *skb, struct net_device *dev, struct Qdisc *q) { - if (unlikely(skb->next)) - dev->gso_skb = skb; - else - q->ops->requeue(skb, q); + if (likely(skb)) { + if (unlikely(skb->next)) + dev->gso_skb = skb; + else + q->ops->requeue(skb, q); + } netif_schedule(dev); return 0; @@ -91,17 +93,22 @@ static inline int handle_dev_cpu_collisi /* * Same CPU holding the lock. It may be a transient * configuration error, when hard_start_xmit() recurses. We - * detect it by checking xmit owner and drop the packet when - * deadloop is detected. Return OK to try the next skb. + * detect it by checking xmit owner and drop the packet (or + * all packets in batching case) when deadloop is detected. + * Return OK to try the next skb. */ - kfree_skb(skb); + if (likely(skb)) + kfree_skb(skb); + else if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + if (net_ratelimit()) printk(KERN_WARNING "Dead loop on netdevice %s, " "fix it urgently!\n", dev->name); ret = qdisc_qlen(q); } else { /* - * Another cpu is holding lock, requeue & delay xmits for + * Another cpu is holding lock, requeue skb & delay xmits for * some time. */ __get_cpu_var(netdev_rx_stat).cpu_collision++; @@ -112,6 +119,38 @@ static inline int handle_dev_cpu_collisi } /* + * Algorithm to get skb(s) is: + * - Non batching drivers, or if the batch list is empty and there is + * 1 skb in the queue - dequeue skb and put it in *skbp to tell the + * caller to use the single xmit API. + * - Batching drivers where the batch list already contains atleast one + * skb, or if there are multiple skbs in the queue: keep dequeue'ing + * skb's upto a limit and set *skbp to NULL to tell the caller to use + * the multiple xmit API. + * + * Returns: + * 1 - atleast one skb is to be sent out, *skbp contains skb or NULL + * (in case >1 skbs present in blist for batching) + * 0 - no skbs to be sent. + */ +static inline int get_skb(struct net_device *dev, struct Qdisc *q, + struct sk_buff_head *blist, struct sk_buff **skbp) +{ + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); + } else { + int max = dev->tx_queue_len - skb_queue_len(blist); + struct sk_buff *skb; + + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) + max -= dev_add_skb_to_blist(skb, dev); + + *skbp = NULL; + return 1; /* we have atleast one skb in blist */ + } +} + +/* * NOTE: Called under dev->queue_lock with locally disabled BH. * * __LINK_STATE_QDISC_RUNNING guarantees only one CPU can process this @@ -130,7 +169,8 @@ static inline int handle_dev_cpu_collisi * >0 - queue is not empty. * */ -static inline int qdisc_restart(struct net_device *dev) +static inline int qdisc_restart(struct net_device *dev, + struct sk_buff_head *blist) { struct Qdisc *q = dev->qdisc; struct sk_buff *skb; @@ -138,7 +178,7 @@ static inline int qdisc_restart(struct n int ret; /* Dequeue packet */ - if (unlikely((skb = dev_dequeue_skb(dev, q)) == NULL)) + if (unlikely(!get_skb(dev, q, blist, &skb))) return 0; /* @@ -158,7 +198,10 @@ static inline int qdisc_restart(struct n /* And release queue */ spin_unlock(&dev->queue_lock); - ret = dev_hard_start_xmit(skb, dev); + if (likely(skb)) + ret = dev_hard_start_xmit(skb, dev); + else + ret = dev->hard_start_xmit_batch(dev); if (!lockless) netif_tx_unlock(dev); @@ -168,7 +211,7 @@ static inline int qdisc_restart(struct n switch (ret) { case NETDEV_TX_OK: - /* Driver sent out skb successfully */ + /* Driver sent out skb (or entire skb_blist) successfully */ ret = qdisc_qlen(q); break; @@ -190,10 +233,10 @@ static inline int qdisc_restart(struct n return ret; } -void __qdisc_run(struct net_device *dev) +void __qdisc_run(struct net_device *dev, struct sk_buff_head *blist) { do { - if (!qdisc_restart(dev)) + if (!qdisc_restart(dev, blist)) break; } while (!netif_queue_stopped(dev)); @@ -563,6 +606,12 @@ void dev_deactivate(struct net_device *d qdisc = dev->qdisc; dev->qdisc = &noop_qdisc; + if (dev->skb_blist) { + /* Release skbs on batch list */ + if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + } + qdisc_reset(qdisc); skb = dev->gso_skb; diff -ruNp ORG/net/core/dev.c NEW/net/core/dev.c --- ORG/net/core/dev.c 2007-08-06 08:25:40.000000000 +0530 +++ NEW/net/core/dev.c 2007-08-07 13:11:19.000000000 +0530 @@ -1699,7 +1699,7 @@ gso: /* reset queue_mapping to zero */ skb->queue_mapping = 0; rc = q->enqueue(skb, q); - qdisc_run(dev); + qdisc_run(dev, NULL); spin_unlock(&dev->queue_lock); rc = rc == NET_XMIT_BYPASS ? NET_XMIT_SUCCESS : rc; @@ -1896,7 +1896,8 @@ static void net_tx_action(struct softirq clear_bit(__LINK_STATE_SCHED, &dev->state); if (spin_trylock(&dev->queue_lock)) { - qdisc_run(dev); + /* Send all skbs if driver supports batching */ + qdisc_run(dev, dev->skb_blist); spin_unlock(&dev->queue_lock); } else { netif_schedule(dev); From krkumar2 at in.ibm.com Wed Aug 8 02:32:26 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:02:26 +0530 Subject: [ofa-general] [PATCH 7/9 Rev3] [IPoIB] Verb changes In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093226.15396.88288.sendpatchset@localhost.localdomain> IPoIB verb changes to use batching. Signed-off-by: Krishna Kumar --- ipoib_verbs.c | 23 ++++++++++++++--------- 1 files changed, 14 insertions(+), 9 deletions(-) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib_verbs.c NEW/drivers/infiniband/ulp/ipoib/ipoib_verbs.c --- ORG/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-07-12 08:55:06.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-07 13:11:19.000000000 +0530 @@ -152,11 +152,11 @@ int ipoib_transport_dev_init(struct net_ .max_send_sge = 1, .max_recv_sge = 1 }, - .sq_sig_type = IB_SIGNAL_ALL_WR, + .sq_sig_type = IB_SIGNAL_REQ_WR, /* 11.2.4.1 */ .qp_type = IB_QPT_UD }; - - int ret, size; + struct ib_send_wr *next_wr = NULL; + int i, ret, size; priv->pd = ib_alloc_pd(priv->ca); if (IS_ERR(priv->pd)) { @@ -197,12 +197,17 @@ int ipoib_transport_dev_init(struct net_ priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - priv->tx_sge.lkey = priv->mr->lkey; - - priv->tx_wr.opcode = IB_WR_SEND; - priv->tx_wr.sg_list = &priv->tx_sge; - priv->tx_wr.num_sge = 1; - priv->tx_wr.send_flags = IB_SEND_SIGNALED; + for (i = ipoib_sendq_size - 1; i >= 0; i--) { + priv->tx_sge[i].lkey = priv->mr->lkey; + priv->tx_wr[i].opcode = IB_WR_SEND; + priv->tx_wr[i].sg_list = &priv->tx_sge[i]; + priv->tx_wr[i].num_sge = 1; + priv->tx_wr[i].send_flags = 0; + + /* Link the list properly for provider to use */ + priv->tx_wr[i].next = next_wr; + next_wr = &priv->tx_wr[i]; + } return 0; From krkumar2 at in.ibm.com Wed Aug 8 02:32:15 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 08 Aug 2007 15:02:15 +0530 Subject: [ofa-general] [PATCH 6/9 Rev3] [IPoIB] CM & Multicast changes In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808093215.15396.13679.sendpatchset@localhost.localdomain> IPoIB CM & Multicast changes based on header file changes. Signed-off-by: Krishna Kumar --- ipoib_cm.c | 13 +++++++++---- ipoib_multicast.c | 4 ++-- 2 files changed, 11 insertions(+), 6 deletions(-) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib_cm.c NEW/drivers/infiniband/ulp/ipoib/ipoib_cm.c --- ORG/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-07-17 08:48:35.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-07 13:11:19.000000000 +0530 @@ -493,14 +493,19 @@ static inline int post_send(struct ipoib unsigned int wr_id, u64 addr, int len) { + int ret; struct ib_send_wr *bad_wr; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + priv->tx_sge[0].addr = addr; + priv->tx_sge[0].length = len; + + priv->tx_wr[0].wr_id = wr_id; - priv->tx_wr.wr_id = wr_id; + priv->tx_wr[0].next = NULL; + ret = ib_post_send(tx->qp, priv->tx_wr, &bad_wr); + priv->tx_wr[0].next = &priv->tx_wr[1]; - return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); + return ret; } void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_tx *tx) diff -ruNp ORG/drivers/infiniband/ulp/ipoib/ipoib_multicast.c NEW/drivers/infiniband/ulp/ipoib/ipoib_multicast.c --- ORG/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-07-12 08:55:06.000000000 +0530 +++ NEW/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-08-07 13:11:19.000000000 +0530 @@ -217,7 +217,7 @@ static int ipoib_mcast_join_finish(struc if (!memcmp(mcast->mcmember.mgid.raw, priv->dev->broadcast + 4, sizeof (union ib_gid))) { priv->qkey = be32_to_cpu(priv->broadcast->mcmember.qkey); - priv->tx_wr.wr.ud.remote_qkey = priv->qkey; + priv->tx_wr[0].wr.ud.remote_qkey = priv->qkey; } if (!test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) { @@ -736,7 +736,7 @@ out: } } - ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN); + ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN, 1); } unlock: From bs at q-leap.de Wed Aug 8 02:37:30 2007 From: bs at q-leap.de (Bernd Schubert) Date: Wed, 8 Aug 2007 11:37:30 +0200 Subject: [ofa-general] ibnetdiscover In-Reply-To: References: <200708071426.36464.bs@q-leap.de> Message-ID: <200708081137.30732.bs@q-leap.de> Hello Hal, thanks for your help. On Tuesday 07 August 2007 14:36:09 Hal Rosenstock wrote: > On 8/7/07, Bernd Schubert wrote: > > Hi, > > > > I two questions about ibnetdiscover. > > > > 1) How reliable is it? Here in our testing lab ibnetdiscover works fine, > > detects the proper names of the two infiniband switches and all connected > > client cards. On a customer system it doesn't work that well. I know the > > client is connected to an MTS2400 switch, but ibnetdiscover detects a > > MT47396. The MTS2400 is connected to MTS1400 switch, but again > > ibnetdiscover believs it is again a MT47396. Any idea whats going on? > > Where is MT47396 being displayed ? ibnetdiscover displays the > NodeDescription and perhaps that is not set properly on those > switches. You can verify this with smpquery nodedesc. ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, skipping port # # Topology file: generated on Tue Aug 7 13:51:41 2007 # # Max of 5 hops discovered # Initiated from node 0002c90200401338 port 0002c90200401339 vendid=0x2c9 devid=0xb924 sysimgguid=0xb8cffff0024ef switchguid=0xb8cffff0024ef Switch 24 "S-000b8cffff0024ef" # "MT47396 Infiniscale-III Mellanox Technologies" base port 0 lid 154 lmc 0 [24] "S-0002c9010befe970"[2] # "MT47396 Infiniscale-III Mellanox Technologies" lid 138 [23] "S-0002c9010befe970"[1] # "MT47396 Infiniscale-III Mellanox Technologies" lid 138 [22] "H-0002c902004013c0"[1] # "MT23108 InfiniHost Mellanox Technologies" lid 4 [...] ha-beo-2:/tmp/ofed/leuven# smpquery -vvvv -G switchinfo 0xb8cffff0024ef # Switch info: Lid 154 LinearFdbCap:....................49152 RandomFdbCap:....................0 McastFdbCap:.....................1024 LinearFdbTop:....................321 DefPort:.........................0 DefMcastPrimPort:................0 DefMcastNotPrimPort:.............0 LifeTime:........................18 StateChange:.....................0 LidsPerPort:.....................0 PartEnforceCap:..................32 InboundPartEnf:..................1 OutboundPartEnf:.................1 FilterRawInbound:................1 FilterRawInbound:................1 EnhancedPort0:...................0 Sorry, but I have no idea how the output of smpquery should help me. > > > 2) ibnetdiscover also can't detect everything on the very same customer > > system, it shows this error > > > > ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, > > skipping port > > > > Does this mean port 1 of the last switch failed? > > It means that the peer port of port 1's SMA failed to respond to the > SubnGet NodeInfo. What is connected there and what state is it in ? Unfortunately I have no idea. The system is not located near to us and its rather difficult to ask our customer, since ibnetdiscover shows the wrong switch names. I will try anyway later on this day. > > > I'm also not sure about the pathes, IMHO the man page of ibnetdiscover > > should give one more example, so > > > > > > > > -D use directed path address arguments. The path > > is a comma separated list of out ports. > > Examples: > > "0" # self port > > "0,1,2,1,4" # out via port 1, then 2, ... > > > > > > "out via port 1, then out via port 2, then out via port 1, ..." > > > > or > > > > "out via port 1, then in port 2, then out via port 1, ..." > > > > > > You see what I mean? > > It's the former. It's the out port on each hop along the path. Thanks, do you mind to apply this patch? I know the text already says "separated list of out ports", but for those who don't believe it like me ;) the comment will make it a bit more convincing. --- ./ofa_user-1.2.orig/src/userspace/management/diags/man/ibnetdiscover.8 2007-06-21 16:39:17.000000000 +0200 +++ ./ofa_user-1.2/src/userspace/management/diags/man/ibnetdiscover.8 2007-08-08 11:15:33.000000000 +0200 @@ -66,7 +66,7 @@ is a comma separated list of out ports. Examples: "0" # self port - "0,1,2,1,4" # out via port 1, then 2, ... + "0,1,2,1,4" # out via port 1, then out via port 2, ... .PP \-G use GUID address argument. In most cases, it is the Port GUID. Example: Thanks again, Bernd -- Bernd Schubert Q-Leap Networks GmbH From krkumar2 at in.ibm.com Wed Aug 8 02:45:26 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 8 Aug 2007 15:15:26 +0530 Subject: [ofa-general] Submitted Rev3 of Batching API Message-ID: Hi all, I just submitted Rev3 of the batching skb patch with support for IPoIB. I am still getting huge number of retransmissions, 1,255207 vs 67, or 18,750 retransmissions in the new code for every single retransmission with the original code. But even with so many extra processing/skbs, I am getting around 20% overall BW improvement. I would really appreciate if some experts can review the code on the list, and also suggest something regarding the retransmissions. thanks, - KK From vlad at lists.openfabrics.org Wed Aug 8 02:50:34 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 8 Aug 2007 02:50:34 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070808-0200 daily build status Message-ID: <20070808095034.E230EE60822@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070808-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From mst at dev.mellanox.co.il Wed Aug 8 03:31:04 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 13:31:04 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070808020526.GE9424@obsidianresearch.com> References: <20070807221700.GE8504@obsidianresearch.com> <000001c7d950$72242150$60cc180a@amr.corp.intel.com> <20070808020526.GE9424@obsidianresearch.com> Message-ID: <20070808103104.GB4571@mellanox.co.il> > AFAIK, when using the 64K MTU setting for IPoIB, if the remote side > doesn't support RC then things go wonky. The scenario you describe works fine for me. What does go wonky for you? > For TCP things *might* be saved > by path mtu discovery - MTU discovery is not restricted to TCP AFAIK. > but PMTU is driven by ICMP errors which are > not generated by an IB network. IPoIB CM code generates them in software. > I suspect that if you use IPoIB a > mixed configuration like that you are going to want to have routing > table entries that override the MTU for non-RC capable destinations. > But I haven't tried this.. Did you actually try interoperability between IPoIB with and without RC support? You might be pleasantly surprised. -- MST From mst at dev.mellanox.co.il Wed Aug 8 03:35:23 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 13:35:23 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070807175055.GD8504@obsidianresearch.com> References: <20070807173433.GB8504@obsidianresearch.com> <000201c7d91a$73581d00$1acc180a@amr.corp.intel.com> <20070807175055.GD8504@obsidianresearch.com> Message-ID: <20070808103523.GC4571@mellanox.co.il> > having a MTU that is exactly equal to the broadcast group MTU > seems fine to me. The SM should be able to do >= internally. AFAIK, if you use "equal" (0x2) selector, that's what SA should provide. Returning a path with a higher MTU won't be compliant. If the desired semantics are ">=" you should use "greater than" (0x0) selector. -- MST From mst at dev.mellanox.co.il Wed Aug 8 03:40:37 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 13:40:37 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070807221700.GE8504@obsidianresearch.com> References: <20070807175055.GD8504@obsidianresearch.com> <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> <20070807221700.GE8504@obsidianresearch.com> Message-ID: <20070808104037.GD4571@mellanox.co.il> > If you want to use the 1k is fast feature of a tavor then I was under > the impression that was done under a reduced IP interface MTU? AFAIK tavor only works better with 1K MTU with connected transport types. With UD, the overhead of extra WRs seems to outweight the speedup we get from hardware. -- MST From davem at davemloft.net Wed Aug 8 03:49:00 2007 From: davem at davemloft.net (David Miller) Date: Wed, 08 Aug 2007 03:49:00 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> Message-ID: <20070808.034900.85820906.davem@davemloft.net> From: Krishna Kumar Date: Wed, 08 Aug 2007 15:01:14 +0530 > RESULTS: The performance improvement for TCP No Delay is in the range of -8% > to 320% (with -8% being the sole negative), with many individual tests > giving 50% or more improvement (I think it is to do with the hw slots > getting full quicker resulting in more batching when the queue gets > woken). The results for TCP is in the range of -11% to 93%, with most > of the tests (8/12) giving improvements. Not because I think it obviates your work, but rather because I'm curious, could you test a TSO-in-hardware driver converted to batching and see how TSO alone compares to batching for a pure TCP workload? I personally don't think it will help for that case at all as TSO likely does better job of coalescing the work _and_ reducing bus traffic as well as work in the TCP stack. From shemminger at linux-foundation.org Wed Aug 8 03:59:46 2007 From: shemminger at linux-foundation.org (Stephen Hemminger) Date: Wed, 8 Aug 2007 06:59:46 -0400 Subject: [ofa-general] Re: [PATCH 2/9 Rev3] [core] Add skb_blist & hard_start_xmit_batch In-Reply-To: <20070808093135.15396.29687.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808093135.15396.29687.sendpatchset@localhost.localdomain> Message-ID: <20070808065946.6c80c470@oldman> How do qdisc's that do rate control work now? Did you add logic to breakup batches for token bucket, etc. From mst at dev.mellanox.co.il Wed Aug 8 04:11:38 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 14:11:38 +0300 Subject: [ofa-general] can MSI patches go in 2.6.24? Message-ID: <20070808111138.GA15959@mellanox.co.il> Roland, are these ready? If yes pls pick them up in for-2.6.24 so I can stop worrying they'll get dropped. -- MST From krkumar2 at in.ibm.com Wed Aug 8 04:09:47 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 8 Aug 2007 16:39:47 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.034900.85820906.davem@davemloft.net> Message-ID: David Miller wrote on 08/08/2007 04:19:00 PM: > From: Krishna Kumar > Date: Wed, 08 Aug 2007 15:01:14 +0530 > > > RESULTS: The performance improvement for TCP No Delay is in the range of -8% > > to 320% (with -8% being the sole negative), with many individual tests > > giving 50% or more improvement (I think it is to do with the hw slots > > getting full quicker resulting in more batching when the queue gets > > woken). The results for TCP is in the range of -11% to 93%, with most > > of the tests (8/12) giving improvements. > > Not because I think it obviates your work, but rather because I'm > curious, could you test a TSO-in-hardware driver converted to > batching and see how TSO alone compares to batching for a pure > TCP workload? > > I personally don't think it will help for that case at all as > TSO likely does better job of coalescing the work _and_ reducing > bus traffic as well as work in the TCP stack. Definitely, I will try to do this. What do you generally think of the patch/implementation ? :) thanks, - KK From krkumar2 at in.ibm.com Wed Aug 8 04:24:34 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 8 Aug 2007 16:54:34 +0530 Subject: [ofa-general] Re: [PATCH 2/9 Rev3] [core] Add skb_blist & hard_start_xmit_batch In-Reply-To: <20070808065946.6c80c470@oldman> Message-ID: Stephen Hemminger wrote on 08/08/2007 04:29:46 PM: > How do qdisc's that do rate control work now? Did you add logic to > breakup batches for token bucket, etc. My thought was that the code works identically to the current code except in requeue cases. For requeue the existing code sched decides which skb to pass next (first) to device; while in batching case it is assumed that skbs already in batch list are part of the hardware queue and sent first. If that is a problem - at the time of enabling batching it is possible to check what qdisc_ops is being used (or whenever the qdisc's change), or maybe add a flag to each qdisc_ops to indicate whether qdisc supports batching, etc ? Like definitely enable batching if using pfifo_fast_ops.. What is your suggestion ? thanks, - KK From monisonlists at gmail.com Wed Aug 8 04:45:59 2007 From: monisonlists at gmail.com (Moni Shoua) Date: Wed, 08 Aug 2007 14:45:59 +0300 Subject: [ofa-general] [PATCH -stable] IB/core: Ignore membership bit in ib_find_pkey() Message-ID: <46B9ACF7.6030907@gmail.com> ib_find_pkey() is used as a replacement for ib_find_cached_pkey(), and the original function ignored the membership bit when searching for a P_Key, so ib_find_pkey() should ignore the bit too. In particular, IPoIB turns on the P_Key membership bit of limited membership P_Keys when creating a child interface and looks for the full membership P_key. This broke if a port was a partial member of a partition when IPoIB switched from ib_find_cached_pkey() to ib_find_pkey(), and this change fixes things again. Signed-off-by: Moni Shoua Signed-off-by: Roland Dreier --- diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 3ada17c..2506c43 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -702,7 +702,7 @@ int ib_find_pkey(struct ib_device *device, if (ret) return ret; - if (pkey == tmp_pkey) { + if ((pkey & 0x7fff) == (tmp_pkey & 0x7fff)) { *index = i; return 0; } From hal.rosenstock at gmail.com Wed Aug 8 04:50:36 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 8 Aug 2007 07:50:36 -0400 Subject: [ofa-general] ibnetdiscover In-Reply-To: <200708081137.30732.bs@q-leap.de> References: <200708071426.36464.bs@q-leap.de> <200708081137.30732.bs@q-leap.de> Message-ID: On 8/8/07, Bernd Schubert wrote: > Hello Hal, > > thanks for your help. > > On Tuesday 07 August 2007 14:36:09 Hal Rosenstock wrote: > > On 8/7/07, Bernd Schubert wrote: > > > Hi, > > > > > > I two questions about ibnetdiscover. > > > > > > 1) How reliable is it? Here in our testing lab ibnetdiscover works fine, > > > detects the proper names of the two infiniband switches and all connected > > > client cards. On a customer system it doesn't work that well. I know the > > > client is connected to an MTS2400 switch, but ibnetdiscover detects a > > > MT47396. The MTS2400 is connected to MTS1400 switch, but again > > > ibnetdiscover believs it is again a MT47396. Any idea whats going on? > > > > Where is MT47396 being displayed ? ibnetdiscover displays the > > NodeDescription and perhaps that is not set properly on those > > switches. You can verify this with smpquery nodedesc. > > ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, skipping port > # > # Topology file: generated on Tue Aug 7 13:51:41 2007 > # > # Max of 5 hops discovered > # Initiated from node 0002c90200401338 port 0002c90200401339 > > vendid=0x2c9 > devid=0xb924 > sysimgguid=0xb8cffff0024ef > switchguid=0xb8cffff0024ef > Switch 24 "S-000b8cffff0024ef" # "MT47396 Infiniscale-III Mellanox Technologies" base port 0 lid 154 lmc 0 > [24] "S-0002c9010befe970"[2] # "MT47396 Infiniscale-III Mellanox Technologies" lid 138 > [23] "S-0002c9010befe970"[1] # "MT47396 Infiniscale-III Mellanox Technologies" lid 138 > [22] "H-0002c902004013c0"[1] # "MT23108 InfiniHost Mellanox Technologies" lid 4 > [...] > > > ha-beo-2:/tmp/ofed/leuven# smpquery -vvvv -G switchinfo 0xb8cffff0024ef > # Switch info: Lid 154 > LinearFdbCap:....................49152 > RandomFdbCap:....................0 > McastFdbCap:.....................1024 > LinearFdbTop:....................321 > DefPort:.........................0 > DefMcastPrimPort:................0 > DefMcastNotPrimPort:.............0 > LifeTime:........................18 > StateChange:.....................0 > LidsPerPort:.....................0 > PartEnforceCap:..................32 > InboundPartEnf:..................1 > OutboundPartEnf:.................1 > FilterRawInbound:................1 > FilterRawInbound:................1 > EnhancedPort0:...................0 > > > Sorry, but I have no idea how the output of smpquery should help me. I was asking for smpquery nodedesc rather than switchinfo but I can see from the ibnetdiscover output what the answer is. It is what I originally said: that the NodeDescriptions of both those switches are the same. > > > > > > 2) ibnetdiscover also can't detect everything on the very same customer > > > system, it shows this error > > > > > > ibwarn: [30449] handle_port: NodeInfo on DR path 0,1,12,21,16,1 failed, > > > skipping port > > > > > > Does this mean port 1 of the last switch failed? > > > > It means that the peer port of port 1's SMA failed to respond to the > > SubnGet NodeInfo. What is connected there and what state is it in ? > > Unfortunately I have no idea. The system is not located near to us and its > rather difficult to ask our customer, since ibnetdiscover shows the wrong > switch names. I will try anyway later on this day. Names are not the only alternative. There are several ways to deal with this. > > > > > I'm also not sure about the pathes, IMHO the man page of ibnetdiscover > > > should give one more example, so > > > > > > > > > > > > -D use directed path address arguments. The path > > > is a comma separated list of out ports. > > > Examples: > > > "0" # self port > > > "0,1,2,1,4" # out via port 1, then 2, ... > > > > > > > > > "out via port 1, then out via port 2, then out via port 1, ..." > > > > > > or > > > > > > "out via port 1, then in port 2, then out via port 1, ..." > > > > > > > > > You see what I mean? > > > > It's the former. It's the out port on each hop along the path. > > Thanks, do you mind to apply this patch? I am no longer the maintainer for this. Sasha ? -- Hal > I know the text already says > "separated list of out ports", but for those who don't believe it > like me ;) the comment will make it a bit more convincing. > > > --- ./ofa_user-1.2.orig/src/userspace/management/diags/man/ibnetdiscover.8 2007-06-21 16:39:17.000000000 +0200 > +++ ./ofa_user-1.2/src/userspace/management/diags/man/ibnetdiscover.8 2007-08-08 11:15:33.000000000 +0200 > @@ -66,7 +66,7 @@ > is a comma separated list of out ports. > Examples: > "0" # self port > - "0,1,2,1,4" # out via port 1, then 2, ... > + "0,1,2,1,4" # out via port 1, then out via port 2, ... > .PP > \-G use GUID address argument. In most cases, it is the Port GUID. > Example: > > > Thanks again, > Bernd > > > -- > Bernd Schubert > Q-Leap Networks GmbH > From johnpol at 2ka.mipt.ru Wed Aug 8 05:01:43 2007 From: johnpol at 2ka.mipt.ru (Evgeniy Polyakov) Date: Wed, 8 Aug 2007 16:01:43 +0400 Subject: [ofa-general] Re: [PATCH 2/9 Rev3] [core] Add skb_blist & hard_start_xmit_batch In-Reply-To: <20070808093135.15396.29687.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808093135.15396.29687.sendpatchset@localhost.localdomain> Message-ID: <20070808120142.GA26884@2ka.mipt.ru> Hi Krishna. On Wed, Aug 08, 2007 at 03:01:35PM +0530, Krishna Kumar (krkumar2 at in.ibm.com) wrote: > +int dev_change_tx_batch_skb(struct net_device *dev, unsigned long new_batch_skb) > +{ > + int ret = 0; > + struct sk_buff_head *blist; > + > + if (!dev->hard_start_xmit_batch) { > + /* Driver doesn't support batching skb API */ > + ret = -ENOTSUPP; > + goto out; > + } > + > + /* Handle invalid argument */ > + if (new_batch_skb < 0) { > + ret = -EINVAL; > + goto out; > + } It is unsigned, how can it be less than zero? And actually you use it just like a binary flag (casted to/from u32 in the code, btw), so why not using ethtool_value directly here? > + /* Check if new value is same as the current */ > + if (!!dev->skb_blist == !!new_batch_skb) > + goto out; > + > + if (new_batch_skb && > + (blist = kmalloc(sizeof *blist, GFP_KERNEL)) == NULL) { > + ret = -ENOMEM; > + goto out; > + } > + > + spin_lock(&dev->queue_lock); > + if (new_batch_skb) { > + skb_queue_head_init(blist); > + dev->skb_blist = blist; > + } else > + free_batching(dev); > + spin_unlock(&dev->queue_lock); This needs bh lock too, since blist is accessed by qdisc_restart. > +int dev_add_skb_to_blist(struct sk_buff *skb, struct net_device *dev) > +{ > + if (!list_empty(&ptype_all)) > + dev_queue_xmit_nit(skb, dev); > + > + if (netif_needs_gso(dev, skb)) { > + if (unlikely(dev_gso_segment(skb))) { > + kfree(skb); > + return 0; > + } > + > + if (skb->next) { > + int count = 0; > + > + do { > + struct sk_buff *nskb = skb->next; > + > + skb->next = nskb->next; > + __skb_queue_tail(dev->skb_blist, nskb); > + count++; > + } while (skb->next); Is it possible to move list without iterating over each entry? -- Evgeniy Polyakov From johnpol at 2ka.mipt.ru Wed Aug 8 05:14:02 2007 From: johnpol at 2ka.mipt.ru (Evgeniy Polyakov) Date: Wed, 8 Aug 2007 16:14:02 +0400 Subject: [ofa-general] Re: [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <20070808093145.15396.91711.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808093145.15396.91711.sendpatchset@localhost.localdomain> Message-ID: <20070808121401.GB8478@2ka.mipt.ru> On Wed, Aug 08, 2007 at 03:01:45PM +0530, Krishna Kumar (krkumar2 at in.ibm.com) wrote: > +static inline int get_skb(struct net_device *dev, struct Qdisc *q, > + struct sk_buff_head *blist, struct sk_buff **skbp) > +{ > + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { > + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); > + } else { > + int max = dev->tx_queue_len - skb_queue_len(blist); > + struct sk_buff *skb; > + > + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) > + max -= dev_add_skb_to_blist(skb, dev); > + > + *skbp = NULL; > + return 1; /* we have atleast one skb in blist */ > + } > +} Same here - is it possible to get a list in one go instead of pulling one-by-one, since it forces quite a few additional unneded lock get/releases. What about dev_dequeue_number_skb(dev, q, num), which will grab the lock and move a list of skbs from one queue to provided head. > @@ -158,7 +198,10 @@ static inline int qdisc_restart(struct n > /* And release queue */ > spin_unlock(&dev->queue_lock); > > - ret = dev_hard_start_xmit(skb, dev); > + if (likely(skb)) > + ret = dev_hard_start_xmit(skb, dev); > + else > + ret = dev->hard_start_xmit_batch(dev); Perfectionism says that having array of two functions and calling one of them via array_func_pointer[!!skb] will be much faster. Just a though. It is actually much faster than if/else on x86 at least. -- Evgeniy Polyakov From tziporet at mellanox.co.il Wed Aug 8 06:00:26 2007 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Wed, 8 Aug 2007 16:00:26 +0300 Subject: [ofa-general] OFED 1.2.c-11 is available Message-ID: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> Hi All, OFED 1.2.c-11 is available now on the OFA server under: http://www.openfabrics.org/builds/connectx/release/ Note: This release supports FW 2.1.000 that is available on Mellanox web site for download and FW 2.2.0 that will be released soon Supported Platforms and Operating Systems ========================================= o CPU architectures: - x86_64 - x86 - ppc64 - ia64 o Linux Operating Systems: - RedHat EL4 up3: 2.6.9-34.ELsmp - RedHat EL4 up4: 2.6.9-42.ELsmp - RedHat EL4 up5: 2.6.9-55.ELsmp - RedHat EL5: 2.6.18-8.el5 - SLES10: 2.6.16.21-0.8-smp - kernel.org: 2.6.20.x and 2.6.22.x - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested) Main changes from OFED 1.2.c-10: ================================ - Merged all 1.2 branch patches to 1.2.c - mad: Fix address handle leak in mad_rmpp - mlx4 - Added support for interrupt moderation for create cq to support ipoib. - Display the following device information via sysfs: board_id, fw_ver, hw_rev and hca_type. As a result ibstat presenting all entries. - Fixed destroy special QPs. - Fixed the work completion opcode for RDMA_READ operation - opensm: - Sets the time a packet can live in the Head of Queue of a port that drives a CA port to approximately 1 second. - Number of dropped packets due to HOQ that will stall the CA was increased to 7 (instead of 1). - IPoIB: - Add interrupt moderation support for IPoIB UD - NAPI is available using a module parameter - Fixed a leak in ipoib_transport_dev_init - Fixed kernel oops in IPoIB download - New MVAPICH package (mvapich-0.9.9-1451): - mpirun_rsh fixes: * Check that we do not overflow command line size * better handling for accept - Fix for memory allocations whose size is bigger than 2^31-1 - Fix for multiple communicator problem - Fix for MPI_Finalize segmentation fault - RDS: - rds_poll() always returns POLLOUT. If you try to send and get EWOULDBLOCK because the remote receiver is congested then you don't get to wait for POLLOUT to be raised before sending again -- it's always raised. This is worked around by implementing some kind of exponential back-off while retrying the send. This can be done by increasing the timeout given to poll(). At each poll() timeout expiry the send is tried again. - Set max send and receive scatter/gather list size to 2 in the QP attributes. The second sge used for RDS header. - Added rs_poll_flag to rds_socket structure to indicate the back pressure status and send space availability. - Added MODULE_VERSION. - ehca: - Added support for RHEL4 up5 Tested ULPs =========== The following ULPs are tested: o OpenSM o MPI (MVAPICH and Open MPI) o IPoIB UD and CM o SDP o SRP o iSER o RDS ConnectX specific known issues and limitations ============================================== - The following features are not supported: o Resize CQ o APM o SQD - To load the driver on machines with 64KB default page size UAR bar must be enlarged. 64KB page size is the default of PPC with RHEL5 and Itanium with 64KB page size enabled. Perform the following three steps: 1. Add the following line in the firmware configuration (INI) file under the [HCA] section: log2_uar_bar_megabytes = 5 2. Burn a modified firmware image with the changed INI file 3. Reboot the system Tziporet Koren Software Director Mellanox Technologies mailto: tziporet at mellanox.co.il Tel +972-4-9097200, ext 380 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dotanb at dev.mellanox.co.il Wed Aug 8 06:27:08 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 8 Aug 2007 16:27:08 +0300 Subject: [ofa-general] [PATCH] libibverbs: Initialize reserved attributes in modify QP command Message-ID: <200708081627.08791.dotanb@dev.mellanox.co.il> Initialize the reserved attributes in modify QP command to eliminate the following valgrind warnings: ==23549== Syscall param write(buf) points to uninitialised byte(s) ==23549== at 0x316B1B933F: (within /lib64/tls/libc-2.3.4.so) ==23549== by 0x4A33AF7: ibv_cmd_modify_qp (cmd.c:782) ==23549== by 0x4F860D8: mlx4_modify_qp (verbs.c:480) ==23549== by 0x4A37A53: ibv_modify_qp@@IBVERBS_1.1 (verbs.c:441) ==23549== by 0x40972E: qp_reset_to_rtr (mr_test_fun.c:1189) ==23549== by 0x403AFC: mr_test_connect_qp (mr_test.c:232) ==23549== by 0x404956: do_test (mr_test.c:85) ==23549== by 0x402DF8: main (main.c:448) ==23549== Address 0x7FEFFF2AE is on thread 1's stack Signed-off-by: Dotan Barak --- diff --git a/src/cmd.c b/src/cmd.c index a0bfaad..6d4331f 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -756,6 +756,7 @@ int ibv_cmd_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, memcpy(cmd->dest.dgid, attr->ah_attr.grh.dgid.raw, 16); cmd->dest.flow_label = attr->ah_attr.grh.flow_label; cmd->dest.dlid = attr->ah_attr.dlid; + cmd->dest.reserved = 0; cmd->dest.sgid_index = attr->ah_attr.grh.sgid_index; cmd->dest.hop_limit = attr->ah_attr.grh.hop_limit; cmd->dest.traffic_class = attr->ah_attr.grh.traffic_class; @@ -768,6 +769,7 @@ int ibv_cmd_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, memcpy(cmd->alt_dest.dgid, attr->alt_ah_attr.grh.dgid.raw, 16); cmd->alt_dest.flow_label = attr->alt_ah_attr.grh.flow_label; cmd->alt_dest.dlid = attr->alt_ah_attr.dlid; + cmd->alt_dest.reserved = 0; cmd->alt_dest.sgid_index = attr->alt_ah_attr.grh.sgid_index; cmd->alt_dest.hop_limit = attr->alt_ah_attr.grh.hop_limit; cmd->alt_dest.traffic_class = attr->alt_ah_attr.grh.traffic_class; From mst at dev.mellanox.co.il Wed Aug 8 06:34:55 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 16:34:55 +0300 Subject: [ofa-general] Re: [PATCH] libibverbs: Initialize reserved attributes in modify QP command In-Reply-To: <200708081627.08791.dotanb@dev.mellanox.co.il> References: <200708081627.08791.dotanb@dev.mellanox.co.il> Message-ID: <20070808133455.GA20077@mellanox.co.il> > Quoting Dotan Barak : > Subject: [PATCH] libibverbs: Initialize reserved attributes in modify QP command > > Initialize the reserved attributes in modify QP command to eliminate the > following valgrind warnings: > > ==23549== Syscall param write(buf) points to uninitialised byte(s) > ==23549== at 0x316B1B933F: (within /lib64/tls/libc-2.3.4.so) > ==23549== by 0x4A33AF7: ibv_cmd_modify_qp (cmd.c:782) > ==23549== by 0x4F860D8: mlx4_modify_qp (verbs.c:480) > ==23549== by 0x4A37A53: ibv_modify_qp@@IBVERBS_1.1 (verbs.c:441) > ==23549== by 0x40972E: qp_reset_to_rtr (mr_test_fun.c:1189) > ==23549== by 0x403AFC: mr_test_connect_qp (mr_test.c:232) > ==23549== by 0x404956: do_test (mr_test.c:85) > ==23549== by 0x402DF8: main (main.c:448) > ==23549== Address 0x7FEFFF2AE is on thread 1's stack > > > Signed-off-by: Dotan Barak > > --- > > diff --git a/src/cmd.c b/src/cmd.c > index a0bfaad..6d4331f 100644 > --- a/src/cmd.c > +++ b/src/cmd.c > @@ -756,6 +756,7 @@ int ibv_cmd_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > memcpy(cmd->dest.dgid, attr->ah_attr.grh.dgid.raw, 16); > cmd->dest.flow_label = attr->ah_attr.grh.flow_label; > cmd->dest.dlid = attr->ah_attr.dlid; > + cmd->dest.reserved = 0; > cmd->dest.sgid_index = attr->ah_attr.grh.sgid_index; > cmd->dest.hop_limit = attr->ah_attr.grh.hop_limit; > cmd->dest.traffic_class = attr->ah_attr.grh.traffic_class; > @@ -768,6 +769,7 @@ int ibv_cmd_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, > memcpy(cmd->alt_dest.dgid, attr->alt_ah_attr.grh.dgid.raw, 16); > cmd->alt_dest.flow_label = attr->alt_ah_attr.grh.flow_label; > cmd->alt_dest.dlid = attr->alt_ah_attr.dlid; > + cmd->alt_dest.reserved = 0; > cmd->alt_dest.sgid_index = attr->alt_ah_attr.grh.sgid_index; > cmd->alt_dest.hop_limit = attr->alt_ah_attr.grh.hop_limit; > cmd->alt_dest.traffic_class = attr->alt_ah_attr.grh.traffic_class; Would it make sense to include valgrind/memcheck.h and use VALGRIND_MAKE_MEM_DEFINED? -- MST From herbert at gondor.apana.org.au Wed Aug 8 06:42:47 2007 From: herbert at gondor.apana.org.au (Herbert Xu) Date: Wed, 8 Aug 2007 21:42:47 +0800 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.034900.85820906.davem@davemloft.net> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808.034900.85820906.davem@davemloft.net> Message-ID: <20070808134247.GA9942@gondor.apana.org.au> On Wed, Aug 08, 2007 at 03:49:00AM -0700, David Miller wrote: > > Not because I think it obviates your work, but rather because I'm > curious, could you test a TSO-in-hardware driver converted to > batching and see how TSO alone compares to batching for a pure > TCP workload? You could even lower the bar by disabling TSO and enabling software GSO. > I personally don't think it will help for that case at all as > TSO likely does better job of coalescing the work _and_ reducing > bus traffic as well as work in the TCP stack. I agree. I suspect the bulk of the effort is in getting these skb's created and processed by the stack so that by the time that they're exiting the qdisc there's not much to be saved anymore. Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt From erezz at voltaire.com Wed Aug 8 06:47:10 2007 From: erezz at voltaire.com (Erez Zilber) Date: Wed, 08 Aug 2007 16:47:10 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> Message-ID: <46B9C95E.6090108@voltaire.com> Tziporet, Can you add the following as a known issue: "open-iscsi over TCP does not support usage of data digest on RedHat EL/AS 4 up3, up4 & up5." Note that this comment should be added to OFED release notes, not to iSER release notes. This is not an iSER bug (it's a bug in the backport for open-iscsi). Thanks, Erez Tziporet Koren wrote: > Hi All, > > OFED 1.2.c-11 is available now on the OFA server under: > _http://www.openfabrics.org/builds/connectx/release/_ > Note: This release supports FW 2.1.000 that is available on Mellanox > web site for download > and FW 2.2.0 that will be released soon > > Supported Platforms and Operating Systems > ========================================= > o CPU architectures: > - x86_64 > - x86 > - ppc64 > - ia64 > > o Linux Operating Systems: > - RedHat EL4 up3: 2.6.9-34.ELsmp > - RedHat EL4 up4: 2.6.9-42.ELsmp > - RedHat EL4 up5: 2.6.9-55.ELsmp > - RedHat EL5: 2.6.18-8.el5 > - SLES10: 2.6.16.21-0.8-smp > - kernel.org: 2.6.20.x and 2.6.22.x > - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested) > > Main changes from OFED 1.2.c-10: > ================================ > - Merged all 1.2 branch patches to 1.2.c > - mad: Fix address handle leak in mad_rmpp > - mlx4 > - Added support for interrupt moderation for create cq to > support ipoib. > - Display the following device information via sysfs: > board_id, fw_ver, > hw_rev and hca_type. As a result ibstat presenting all entries. > - Fixed destroy special QPs. > - Fixed the work completion opcode for RDMA_READ operation > - opensm: > - Sets the time a packet can live in the Head of Queue of a > port that > drives a CA port to approximately 1 second. > - Number of dropped packets due to HOQ that will stall the CA > was increased > to 7 (instead of 1). > - IPoIB: > - Add interrupt moderation support for IPoIB UD > - NAPI is available using a module parameter > - Fixed a leak in ipoib_transport_dev_init > - Fixed kernel oops in IPoIB download > - New MVAPICH package (mvapich-0.9.9-1451): > - mpirun_rsh fixes: > * Check that we do not overflow command line size > * better handling for accept > - Fix for memory allocations whose size is bigger than 2^31-1 > - Fix for multiple communicator problem > - Fix for MPI_Finalize segmentation fault > - RDS: > - rds_poll() always returns POLLOUT. If you try to send and get > EWOULDBLOCK because the remote receiver is congested then > you don't get > to wait for POLLOUT to be raised before sending again -- > it's always > raised. This is worked around by implementing some kind of > exponential > back-off while retrying the send. This can be done by > increasing the > timeout given to poll(). At each poll() timeout expiry the > send is > tried again. > - Set max send and receive scatter/gather list size to 2 in > the QP attributes. > The second sge used for RDS header. > - Added rs_poll_flag to rds_socket structure to indicate the > back pressure > status and send space availability. > - Added MODULE_VERSION. > - ehca: > - Added support for RHEL4 up5 > > > Tested ULPs > =========== > The following ULPs are tested: > o OpenSM > o MPI (MVAPICH and Open MPI) > o IPoIB UD and CM > o SDP > o SRP > o iSER > o RDS > > ConnectX specific known issues and limitations > ============================================== > - The following features are not supported: > o Resize CQ > o APM > o SQD > - To load the driver on machines with 64KB default page size UAR bar > must be > enlarged. 64KB page size is the default of PPC with RHEL5 and > Itanium with > 64KB page size enabled. > Perform the following three steps: > 1. Add the following line in the firmware configuration (INI) file > under the > [HCA] section: > log2_uar_bar_megabytes = 5 > 2. Burn a modified firmware image with the changed INI file > 3. Reboot the system > > > Tziporet Koren > Software Director > Mellanox Technologies > mailto: _tziporet at mellanox.co.il_ > Tel +972-4-9097200, ext 380 > > ------------------------------------------------------------------------ > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From hnguyen at linux.vnet.ibm.com Wed Aug 8 07:13:39 2007 From: hnguyen at linux.vnet.ibm.com (Hoang-Nam Nguyen) Date: Wed, 8 Aug 2007 16:13:39 +0200 Subject: [ofa-general] Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space In-Reply-To: <200708031036.20859.hnguyen@linux.vnet.ibm.com> References: <200708021139.31695.hnguyen@linux.vnet.ibm.com> <200708031036.20859.hnguyen@linux.vnet.ibm.com> Message-ID: <200708081613.39837.hnguyen@linux.vnet.ibm.com> Hello Roland! Haven't got any ack for this updated patch yet. Anyway, since it contains another bug as shown below, please ignore this patch. We'll send a patch set that includes the proper version of this patch later. > @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, > u64 vsize, physical; > > vsize = vma->vm_end - vma->vm_start; > - if (vsize != EHCA_PAGESIZE) { > + if (vsize >= EHCA_PAGESIZE) { should be > + if (vsize < EHCA_PAGESIZE) { which is sort of invalid arg. Thanks Nam On Friday 03 August 2007 10:36, Hoang-Nam Nguyen wrote: > From: Hoang-Nam Nguyen > Date: Fri, 3 Aug 2007 09:44:56 +0200 > Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space > This patch utilizes remap_4k_pfn() as introduced by Paul M., > for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281, > to map ehca cq, qp firmware context (4k) to user space if kernel page > size is 64k. For reason, why this is required, see also Paul's patch. > In addition to that the kernel page offset of firmware context needs > to be set in cq and qp response block so that user space can assemble > the proper virtual address to use. > An appropriate patch for libehca will follow for ofed-1.3. > > Signed-off-by: Hoang-Nam Nguyen > --- > drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++- > drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++ > drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++ > drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +++--- > 4 files changed, 10 insertions(+), 4 deletions(-) > > diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h > index b5e9603..206d4eb 100644 > --- a/drivers/infiniband/hw/ehca/ehca_classes.h > +++ b/drivers/infiniband/hw/ehca/ehca_classes.h > @@ -337,6 +337,8 @@ struct ehca_create_cq_resp { > u32 cq_number; > u32 token; > struct ipzu_queue_resp ipz_queue; > + u32 fw_handle_ofs; > + u32 dummy; > }; > > struct ehca_create_qp_resp { > @@ -347,7 +349,7 @@ struct ehca_create_qp_resp { > u32 qkey; > /* qp_num assigned by ehca: sqp0/1 may have got different numbers */ > u32 real_qp_num; > - u32 dummy; /* padding for 8 byte alignment */ > + u32 fw_handle_ofs; > struct ipzu_queue_resp ipz_squeue; > struct ipzu_queue_resp ipz_rqueue; > }; > diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c > index 81aff36..ed5d67f 100644 > --- a/drivers/infiniband/hw/ehca/ehca_cq.c > +++ b/drivers/infiniband/hw/ehca/ehca_cq.c > @@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, > resp.ipz_queue.queue_length = ipz_queue->queue_length; > resp.ipz_queue.pagesize = ipz_queue->pagesize; > resp.ipz_queue.toggle_state = ipz_queue->toggle_state; > + resp.fw_handle_ofs = (u32) > + (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1)); > if (ib_copy_to_udata(udata, &resp, sizeof(resp))) { > ehca_err(device, "Copy to udata failed."); > goto create_cq_exit4; > diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c > index b178cba..66f632c 100644 > --- a/drivers/infiniband/hw/ehca/ehca_qp.c > +++ b/drivers/infiniband/hw/ehca/ehca_qp.c > @@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp( > queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue); > if (HAS_RQ(my_qp)) > queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue); > + resp.fw_handle_ofs = (u32) > + (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1)); > > if (ib_copy_to_udata(udata, &resp, sizeof resp)) { > ehca_err(pd->device, "Copy to udata failed"); > diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c > index 4bc687f..be062f1 100644 > --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c > +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c > @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, > u64 vsize, physical; > > vsize = vma->vm_end - vma->vm_start; > - if (vsize != EHCA_PAGESIZE) { > + if (vsize >= EHCA_PAGESIZE) { > ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start); > return -EINVAL; > } > @@ -118,8 +118,8 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, > vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); > ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical); > /* VM_IO | VM_RESERVED are set by remap_pfn_range() */ > - ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT, > - vsize, vma->vm_page_prot); > + ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT, > + vma->vm_page_prot); > if (unlikely(ret)) { > ehca_gen_err("remap_pfn_range() failed ret=%x", ret); > return -ENOMEM; From kaber at trash.net Wed Aug 8 07:05:17 2007 From: kaber at trash.net (Patrick McHardy) Date: Wed, 08 Aug 2007 16:05:17 +0200 Subject: [ofa-general] Re: [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <20070808093145.15396.91711.sendpatchset@localhost.localdomain> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808093145.15396.91711.sendpatchset@localhost.localdomain> Message-ID: <46B9CD9D.4020506@trash.net> Krishna Kumar wrote: > + * Algorithm to get skb(s) is: > + * - Non batching drivers, or if the batch list is empty and there is > + * 1 skb in the queue - dequeue skb and put it in *skbp to tell the > + * caller to use the single xmit API. > + * - Batching drivers where the batch list already contains atleast one > + * skb, or if there are multiple skbs in the queue: keep dequeue'ing > + * skb's upto a limit and set *skbp to NULL to tell the caller to use > + * the multiple xmit API. > + * > + * Returns: > + * 1 - atleast one skb is to be sent out, *skbp contains skb or NULL > + * (in case >1 skbs present in blist for batching) > + * 0 - no skbs to be sent. > + */ > +static inline int get_skb(struct net_device *dev, struct Qdisc *q, > + struct sk_buff_head *blist, struct sk_buff **skbp) > +{ > + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { > + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); > + } else { > + int max = dev->tx_queue_len - skb_queue_len(blist); > + struct sk_buff *skb; > + > + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) > + max -= dev_add_skb_to_blist(skb, dev); > + > + *skbp = NULL; > + return 1; /* we have atleast one skb in blist */ > + } > +} I think you missed my previous reply to this in the flood of responses (or I missed your reply), so I'm copying it below: The entire idea of a single queue after qdiscs that is refilled independantly of transmissions times etc. make be worry a bit. By changing the timing you're effectively changing the qdiscs behaviour, at least in some cases. SFQ is a good example, but I believe it affects most work-conserving qdiscs. Think of this situation: 100 packets of flow 1 arrive 50 packets of flow 1 are sent 100 packets for flow 2 arrive remaining packets are sent On the wire you'll first see 50 packets of flow 1, than 100 packets alternate of flow 1 and 2, then 50 packets flow 2. With your additional queue all packets of flow 1 are pulled out of the qdisc immediately and put in the fifo. When the 100 packets of the second flow arrive they will also get pulled out immediately and are put in the fifo behind the remaining 50 packets of flow 1. So what you get on the wire is: 100 packets of flow 1 100 packets of flow 1 So SFQ is without any effect. This is not completely avoidable of course, but you can and should limit the damage by only pulling out as much packets as the driver can take and have the driver stop the queue afterwards. From wiktoria.daniell at pizzeria-caorle.at Wed Aug 8 08:39:50 2007 From: wiktoria.daniell at pizzeria-caorle.at (Carolina Finley) Date: Wed, 8 Aug 2007 14:39:50 -0100 Subject: [ofa-general] I show you how far the rabbit hole goes Message-ID: <01c7d9c9$f9a54400$8b0dd6d4@wiktoria.daniell> -------------- next part -------------- A non-text attachment was scrubbed... Name: nurka.gif Type: image/gif Size: 14143 bytes Desc: not available URL: From swise at opengridcomputing.com Wed Aug 8 07:57:08 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 08 Aug 2007 09:57:08 -0500 Subject: [ofa-general] Re: [ewg] OFED 1.2.c status & plans In-Reply-To: References: <46B8C28D.7020108@mellanox.co.il> Message-ID: <46B9D9C4.9070207@opengridcomputing.com> Tziporet, Can we make the change to 1.2.5? This should probably include: - change builds/connectx to builds/ofed-1.2.5 (or just add a link) - change build names from 1.2.c to 1.2.5 Scott Weitzenkamp (sweitzen) wrote: > Cisco has been testing 1.2.c-10 IPoIB/SDP/MPI successfully on a 32-node > cluster. We are still working on tvflash, though. > > Scott > > ------------------------------------------------------------------------ > *From:* ewg-bounces at lists.openfabrics.org > [mailto:ewg-bounces at lists.openfabrics.org] *On Behalf Of *Tziporet Koren > *Sent:* Tuesday, August 07, 2007 12:06 PM > *To:* EWG > *Cc:* OpenFabrics General > *Subject:* [ewg] OFED 1.2.c status & plans > > Hi All, > > I wish to update on OFED 1.2.c status and plans to synch everybody: > > * OFED 1.2.c-11 is going out tomorrow > * This release should be the base for the GA release > * Need an approval from Steve (Chelsio) & Nam (IBM) that > everything is in place from their perspective. > Also please send me the release notes for ehca and cxgb3 > * Need an approval from the companies that are testing this > release that it can go to GA > >From Mellanox perspective (mlx4 readiness) we are ready for GA. > > I have one question: do we prefer to stay with the name 1.2.c or 1.2.5? > > Thanks, > Tziporet > > > ------------------------------------------------------------------------ > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg From tziporet at dev.mellanox.co.il Wed Aug 8 08:09:59 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Wed, 08 Aug 2007 18:09:59 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <46B9C95E.6090108@voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46B9C95E.6090108@voltaire.com> Message-ID: <46B9DCC7.3060908@mellanox.co.il> Erez Zilber wrote: > Tziporet, > > > Can you add the following as a known issue: > > > "open-iscsi over TCP does not support usage of data digest on RedHat EL/AS 4 up3, > up4 & up5." > > Note that this comment should be added to OFED release notes, not to > iSER release notes. This is not an iSER bug (it's a bug in the backport > for open-iscsi). > > Thanks, > Erez > > Will do in next build tziporet From hadi at cyberus.ca Wed Aug 8 08:14:35 2007 From: hadi at cyberus.ca (jamal) Date: Wed, 08 Aug 2007 11:14:35 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808134247.GA9942@gondor.apana.org.au> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808.034900.85820906.davem@davemloft.net> <20070808134247.GA9942@gondor.apana.org.au> Message-ID: <1186586075.5155.27.camel@localhost> On Wed, 2007-08-08 at 21:42 +0800, Herbert Xu wrote: > On Wed, Aug 08, 2007 at 03:49:00AM -0700, David Miller wrote: > > > > Not because I think it obviates your work, but rather because I'm > > curious, could you test a TSO-in-hardware driver converted to > > batching and see how TSO alone compares to batching for a pure > > TCP workload? > > You could even lower the bar by disabling TSO and enabling > software GSO. >From my observation for TCP packets slightly above MDU (upto 2K), GSO gives worse performance than non-GSO throughput-wise. Actually this has nothing to do with batching, rather the behavior is consistent with or without batching changes. > > I personally don't think it will help for that case at all as > > TSO likely does better job of coalescing the work _and_ reducing > > bus traffic as well as work in the TCP stack. > > I agree. > I suspect the bulk of the effort is in getting > these skb's created and processed by the stack so that by > the time that they're exiting the qdisc there's not much > to be saved anymore. pktgen shows a clear win if you test the driver path - which is what you should test because thats where the batching changes are. Using TCP or UDP adds other variables[1] that need to be isolated first in order to quantify the effect of batching. For throughput and CPU utilization, the benefit will be clear when there are a lot more flows. cheers, jamal [1] I think there are too many other variables in play unfortunately when you are dealing with a path that starts above the driver and one that covers end to end effect: traffic/app source, system clock sources as per my recent discovery, congestion control algorithms used, tuning of recevier etc. From hadi at cyberus.ca Wed Aug 8 08:26:05 2007 From: hadi at cyberus.ca (jamal) Date: Wed, 08 Aug 2007 11:26:05 -0400 Subject: [ofa-general] Re: [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <46B9CD9D.4020506@trash.net> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808093145.15396.91711.sendpatchset@localhost.localdomain> <46B9CD9D.4020506@trash.net> Message-ID: <1186586765.5155.39.camel@localhost> On Wed, 2007-08-08 at 16:05 +0200, Patrick McHardy wrote: > This is not completely avoidable of > course, but you can and should limit the damage by only pulling > out as much packets as the driver can take and have the driver > stop the queue afterwards. This why the dev->xmit_win exists in my patches. The driver says how much space it has using this variable - and only those packets get pulled off; so there is no conflict with any scheduling algorithm be it work/non-work conserving. The ideal case is never to carry over anything in dev->blist. However, there is a challenge with GSO/TSO: if say the descriptors in the driver are less than the list of skbs, you are faced with the dilema of sending a fraction of the gso list first. My current approach is to transmit as much as the driver would allow. On the next opportunity to transmit, you do immediately send anything remaining first before you ask the qdisc for more packets. An alternative approach i had entertained was not to send anything from the skb list when hitting such a case, but it hasnt seem needed so far (havent seen any leftovers from my experiments). cheers, jamal From rdreier at cisco.com Wed Aug 8 09:08:20 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 08 Aug 2007 09:08:20 -0700 Subject: [ofa-general] Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space In-Reply-To: <200708081613.39837.hnguyen@linux.vnet.ibm.com> (Hoang-Nam Nguyen's message of "Wed, 8 Aug 2007 16:13:39 +0200") References: <200708021139.31695.hnguyen@linux.vnet.ibm.com> <200708031036.20859.hnguyen@linux.vnet.ibm.com> <200708081613.39837.hnguyen@linux.vnet.ibm.com> Message-ID: > Haven't got any ack for this updated patch yet. Anyway, since it contains > another bug as shown below, please ignore this patch. We'll send a patch > set that includes the proper version of this patch later. Yes, sorry, I'm a bit behind applying patches. Anyway, OK I'll wait for a fixed patch and drop this one :) From bs at q-leap.de Wed Aug 8 10:20:08 2007 From: bs at q-leap.de (Bernd Schubert) Date: Wed, 8 Aug 2007 19:20:08 +0200 Subject: [ofa-general] ibnetdiscover In-Reply-To: References: <200708071426.36464.bs@q-leap.de> <200708081137.30732.bs@q-leap.de> Message-ID: <200708081920.08847.bs@q-leap.de> On Wednesday 08 August 2007 13:50:36 Hal Rosenstock wrote: > > Sorry, but I have no idea how the output of smpquery should help me. > > I was asking for smpquery nodedesc rather than switchinfo but I can > see from the ibnetdiscover output what the answer is. It is what I > originally said: that the NodeDescriptions of both those switches are > the same. I see. [...] > > Unfortunately I have no idea. The system is not located near to us and > > its rather difficult to ask our customer, since ibnetdiscover shows the > > wrong switch names. I will try anyway later on this day. > > Names are not the only alternative. There are several ways to deal with > this. I have patched ibnetdiscover now. One can now create a file /etc/ib.id (e.g. by adding the value of /sys/class/infiniband/*/node_guid to it and ibnetdiscover will then also output the corresponding hostname (or whatever the admin wishes to assign to this id). So the output will now read # # Topology file: generated on Wed Aug 8 19:01:22 2007 # # Max of 5 hops discovered # Initiated from node 0002c90200401338 port 0002c90200401339 vendid=0x2c9 devid=0xb924 sysimgguid=0xb8cffff0024ef switchguid=0xb8cffff0024ef Switch 24 "S-000b8cffff0024ef" # "MT47396 Infiniscale-III Mellanox Technologies (mts2400)" base port 0 lid 154 lmc 0 [24] "S-0002c9010befe970"[2] # "MT47396 Infiniscale-III Mellanox Technologies (mts14400-1-master)" lid 138 [23] "S-0002c9010befe970"[1] # "MT47396 Infiniscale-III Mellanox Technologies (mts14400-1-master)" lid 138 [22] "H-0002c902004013c0"[1] # "MT23108 InfiniHost Mellanox Technologies (ha-beo-1)" lid 4 and an example ib.id file mts2400 000b8cffff0024ef mts14400-1-master 0002c9010befe970 mts14400-2-master 000b8cffff002456 mts-1 0002c9010befaae0 mts-2 0002c9010c7341f0 I will post the patch shortly (still need to add an optional different filename then /etc/ib.id). Anyway, do already wish to have something differently when you look on the new output? Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH From rdreier at cisco.com Wed Aug 8 10:25:56 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 08 Aug 2007 10:25:56 -0700 Subject: [ofa-general] Re: [ewg] [PATCH 0/14] nes: NetEffect 10Gb RNIC Driver In-Reply-To: <200708080022.l780ME9e004542@neteffect.com> (ggrundstrom@neteffect.com's message of "Tue, 7 Aug 2007 19:22:14 -0500") References: <200708080022.l780ME9e004542@neteffect.com> Message-ID: > git.openfabrics.org/~glenn/libnes.git > git.openfabrics.org/~glenn/ofed_1_2.git > git.openfabrics.org/~glenn/ofascripts.git > git.openfabrics.org/~glenn/ofed_1_2_scripts.git these aren't actually git URLs. prepending git:// seems to work though. From rdreier at cisco.com Wed Aug 8 10:27:53 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 08 Aug 2007 10:27:53 -0700 Subject: [ofa-general] Re: [PATCH] libibverbs: Initialize reserved attributes in modify QP command In-Reply-To: <20070808133455.GA20077@mellanox.co.il> (Michael S. Tsirkin's message of "Wed, 8 Aug 2007 16:34:55 +0300") References: <200708081627.08791.dotanb@dev.mellanox.co.il> <20070808133455.GA20077@mellanox.co.il> Message-ID: > Would it make sense to include > valgrind/memcheck.h and use VALGRIND_MAKE_MEM_DEFINED? We already include the header and use VALGRIND_MAKE_MEM_DEFINED in many places in libibverbs. However in this case I think we really should be zeroing out reserved fields in the data we pass to the kernel. From hal.rosenstock at gmail.com Wed Aug 8 10:46:02 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 8 Aug 2007 13:46:02 -0400 Subject: [ofa-general] ibnetdiscover In-Reply-To: <200708081920.08847.bs@q-leap.de> References: <200708071426.36464.bs@q-leap.de> <200708081137.30732.bs@q-leap.de> <200708081920.08847.bs@q-leap.de> Message-ID: On 8/8/07, Bernd Schubert wrote: > On Wednesday 08 August 2007 13:50:36 Hal Rosenstock wrote: > > > Sorry, but I have no idea how the output of smpquery should help me. > > > > I was asking for smpquery nodedesc rather than switchinfo but I can > > see from the ibnetdiscover output what the answer is. It is what I > > originally said: that the NodeDescriptions of both those switches are > > the same. > > I see. > > > [...] > > > > Unfortunately I have no idea. The system is not located near to us and > > > its rather difficult to ask our customer, since ibnetdiscover shows the > > > wrong switch names. I will try anyway later on this day. > > > > Names are not the only alternative. There are several ways to deal with > > this. > > I have patched ibnetdiscover now. One can now create a file /etc/ib.id (e.g. > by adding the value of /sys/class/infiniband/*/node_guid to it and > ibnetdiscover will then also output the corresponding hostname (or whatever > the admin wishes to assign to this id). There's a switch map implemented for this purpose. -- Hal > > So the output will now read > > # > # Topology file: generated on Wed Aug 8 19:01:22 2007 > # > # Max of 5 hops discovered > # Initiated from node 0002c90200401338 port 0002c90200401339 > > vendid=0x2c9 > devid=0xb924 > sysimgguid=0xb8cffff0024ef > switchguid=0xb8cffff0024ef > Switch 24 "S-000b8cffff0024ef" # "MT47396 Infiniscale-III Mellanox Technologies (mts2400)" base port 0 lid 154 lmc > 0 > [24] "S-0002c9010befe970"[2] # "MT47396 Infiniscale-III Mellanox Technologies (mts14400-1-master)" lid 138 > [23] "S-0002c9010befe970"[1] # "MT47396 Infiniscale-III Mellanox Technologies (mts14400-1-master)" lid 138 > [22] "H-0002c902004013c0"[1] # "MT23108 InfiniHost Mellanox Technologies (ha-beo-1)" lid 4 > > > and an example ib.id file > > mts2400 000b8cffff0024ef > mts14400-1-master 0002c9010befe970 > mts14400-2-master 000b8cffff002456 > mts-1 0002c9010befaae0 > mts-2 0002c9010c7341f0 > > I will post the patch shortly (still need to add an optional different > filename then /etc/ib.id). Anyway, do already wish to have something > differently when you look on the new output? > > > Cheers, > Bernd > > > -- > Bernd Schubert > Q-Leap Networks GmbH > From becker at nas.nasa.gov Wed Aug 8 10:47:53 2007 From: becker at nas.nasa.gov (Jeff Becker) Date: Wed, 8 Aug 2007 10:47:53 -0700 Subject: [ofa-general] RE: OFA website edits In-Reply-To: <46A798F0.5070902@ichips.intel.com> References: <46956FF9.50102@ichips.intel.com> <46968448.2000401@ichips.intel.com> <46A798F0.5070902@ichips.intel.com> Message-ID: <795c49870708081047v2d5598b6q6c7ef5a91063897c@mail.gmail.com> Hi. I created most of the requested directory/owner pairs in /var/www/openfabrics.org/downloads. I left out the various MPI directories, figuring the appropriate web pages will be linked from somewhere (possibly the downloads web page). I gave Stan Smith an account. Stan, please contact me to get the account info. I'm still working out how to do the dynamic web page stuff, but at least people can start populating their directories. Thanks. -jeff On 7/25/07, Arlin Davis wrote: > > > I would like to propose adding project directories under > > http://www.openfabrics.org/downloads/ where appropriate and give > > maintainers access. For example: > > > Jeff, please add the following directories with maintainer access as > follow (or grant access at a maintainer group level): > > http://www.openfabrics.org/downloads/verbs (rdreier) > http://www.openfabrics.org/downloads/rdmacm (shefty) > http://www.openfabrics.org/downloads/dapl (ardavis) > http://www.openfabrics.org/downloads/sdp (eitan) > http://www.openfabrics.org/downloads/utils (eitan) > http://www.openfabrics.org/downloads/management (sashak) > http://www.openfabrics.org/downloads/OFED (vlad) > http://www.openfabrics.org/downloads/archives (vlad) > http://www.openfabrics.org/downloads/WinOF (ssmith) (Stan Smith will > need an account) > http://www.openfabrics.org/downloads/hw/mthca (rdreir) > http://www.openfabrics.org/downloads/hw/mlx4 (rdreir) > http://www.openfabrics.org/downloads/hw/ehca (raisch) > http://www.openfabrics.org/downloads/hw/ipath (ralphc) > http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc) > http://www.openfabrics.org/downloads/mpi/mvapich (pasha) > http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland) > http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres) > > Let us know when these directories are created and the maintainers, who > want to expose their packages via the webpage, will create a README that > details the contents of the directory along with WEB_README that > provides a short description for the webpage. > > Will this format allow you to auto configure the download webpage > sufficiently? The idea is to only add links/descriptions to those > project sub-directories with WEB_README files present. > > Please advise if something on the list is wrong or we missed a project. > > Thanks, > > -arlin > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From mshefty at ichips.intel.com Wed Aug 8 11:35:16 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 08 Aug 2007 11:35:16 -0700 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070808104037.GD4571@mellanox.co.il> References: <20070807175055.GD8504@obsidianresearch.com> <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> <20070807221700.GE8504@obsidianresearch.com> <20070808104037.GD4571@mellanox.co.il> Message-ID: <46BA0CE4.8050504@ichips.intel.com> > AFAIK tavor only works better with 1K MTU with connected transport types. > With UD, the overhead of extra WRs seems to outweight the speedup > we get from hardware. Doesn't IPoIB use the same path record for both UD and RC, if RC is available? I guess the question still remains, does IPoIB need to (or should it) specify the MTU, or should it be set by the SA? It sounds like IPoIB can handle the case where the path MTU is less than the device MTU. It doesn't specify it today, and the QoS patches seem like a somewhat separate issue. - Sean From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:36:11 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:36:11 +0200 Subject: [ofa-general] [PATCH 0/7] IB/ehca: support for user space small queues, support more than 4k queue pairs, generate last WQE reached Message-ID: <200708082036.12684.ossrosch@linux.vnet.ibm.com> Here is a patch set against Roland's git, branch for-2.6.23 for ehca. It enables userspace support for small QP feature and make some fixes for it. Also there is add the mapping of 4k firmware context to user space. They are in details: [1/7] add support for userspace small queues and make some fixes [2/7] ensure that a non-existing queues in case of SRQs are not interprete as small queues [3/7] we have no longer to add 1 to the number of requestet wqes, because firmware does now [4/7] make changes to ehca_mmap() to support more than 4k queues [5/7] map 4k firmware context of cq, qp to user space [6/7] generate last WQE reached, when base QP for SRQ has entered error state [7/7] prevent overwriting QP init attributes given by caller The patches should apply cleanly, in order, against Roland's git. Please review the changes and apply the patches if they are okay. Regards, Stefan From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:37:59 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:37:59 +0200 Subject: [ofa-general] [PATCH 1/7] IB/ehca: Small QP userspace support and fixes Message-ID: <200708082038.00127.ossrosch@linux.vnet.ibm.com> Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_qp.c | 7 +++---- drivers/infiniband/hw/ehca/ipz_pt_fn.c | 3 ++- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..cfa83fa 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -273,6 +273,7 @@ static inline void queue2resp(struct ipzu_queue_resp *resp, resp->queue_length = queue->queue_length; resp->pagesize = queue->pagesize; resp->toggle_state = queue->toggle_state; + resp->offset = queue->offset; } /* @@ -598,8 +599,7 @@ static struct ehca_qp *internal_create_qp( parms.squeue.max_sge = max_send_sge; parms.rqueue.max_sge = max_recv_sge; - if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap) - && !(context && udata)) { /* no small QP support in userspace ATM */ + if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap)) { ehca_determine_small_queue( &parms.squeue, max_send_sge, is_llqp); ehca_determine_small_queue( @@ -739,8 +739,7 @@ static struct ehca_qp *internal_create_qp( resp.ext_type = my_qp->ext_type; resp.qkey = my_qp->qkey; resp.real_qp_num = my_qp->real_qp_num; - resp.ipz_rqueue.offset = my_qp->ipz_rqueue.offset; - resp.ipz_squeue.offset = my_qp->ipz_squeue.offset; + if (HAS_SQ(my_qp)) queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue); if (HAS_RQ(my_qp)) diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.c b/drivers/infiniband/hw/ehca/ipz_pt_fn.c index a090c67..661f8db 100644 --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c @@ -158,6 +158,7 @@ static int alloc_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd) queue->queue_pages[0] = (void *)(page->page | (bit << (order + 9))); queue->small_page = page; + queue->offset = bit << (order + 9); return 1; out: @@ -172,7 +173,7 @@ static void free_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd) unsigned long bit; int free_page = 0; - bit = ((unsigned long)queue->queue_pages[0] & PAGE_MASK) + bit = ((unsigned long)queue->queue_pages[0] & ~PAGE_MASK) >> (order + 9); mutex_lock(&pd->lock); -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:39:49 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:39:49 +0200 Subject: [ofa-general] [PATCH 2/7] IB/ehca: Ensure non-existing queues aren't made small queues Message-ID: <200708082039.50878.ossrosch@linux.vnet.ibm.com> Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_qp.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index cfa83fa..13b61c3 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -600,10 +600,12 @@ static struct ehca_qp *internal_create_qp( parms.rqueue.max_sge = max_recv_sge; if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap)) { - ehca_determine_small_queue( - &parms.squeue, max_send_sge, is_llqp); - ehca_determine_small_queue( - &parms.rqueue, max_recv_sge, is_llqp); + if (HAS_SQ(my_qp)) + ehca_determine_small_queue( + &parms.squeue, max_send_sge, is_llqp); + if (HAS_RQ(my_qp)) + ehca_determine_small_queue( + &parms.rqueue, max_recv_sge, is_llqp); parms.qp_storage = (parms.squeue.is_small || parms.rqueue.is_small); } -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:40:47 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:40:47 +0200 Subject: [ofa-general] [PATCH 3/7] IB/ehca: Add 1 is not longer needed because of firmware interface change Message-ID: <200708082040.47926.ossrosch@linux.vnet.ibm.com> Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/hcp_if.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c index 24f4541..8534061 100644 --- a/drivers/infiniband/hw/ehca/hcp_if.c +++ b/drivers/infiniband/hw/ehca/hcp_if.c @@ -317,9 +317,9 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle, max_r10_reg = EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_SEND_WR, - parms->squeue.max_wr + 1) + parms->squeue.max_wr) | EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_RECV_WR, - parms->rqueue.max_wr + 1) + parms->rqueue.max_wr) | EHCA_BMASK_SET(H_ALL_RES_QP_MAX_SEND_SGE, parms->squeue.max_sge) | EHCA_BMASK_SET(H_ALL_RES_QP_MAX_RECV_SGE, -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:41:54 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:41:54 +0200 Subject: [ofa-general] [PATCH 4/7] IB/ehca: Support more than 4k QPs for userspace and kernelspace Message-ID: <200708082041.55794.ossrosch@linux.vnet.ibm.com> Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_cq.c | 7 ++++++- drivers/infiniband/hw/ehca/ehca_qp.c | 6 ++++++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 22 +++++++++++----------- 3 files changed, 23 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index 81aff36..c661939 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -168,7 +168,12 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, write_unlock_irqrestore(&ehca_cq_idr_lock, flags); } while (ret == -EAGAIN); - + if (my_cq->token > 0x1FFFFFF) { + cq = ERR_PTR(-ENOMEM); + ehca_err(device, "Invalid number of qp. device=%p", + device); + goto create_cq_exit2; + } if (ret) { cq = ERR_PTR(-ENOMEM); ehca_err(device, "Can't allocate new idr entry. device=%p", diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index 13b61c3..f26801b 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -560,6 +560,12 @@ static struct ehca_qp *internal_create_qp( } while (ret == -EAGAIN); + if (my_qp->token > 0x1FFFFFF) { + ret = -EINVAL; + ehca_err(pd->device, "Invalid number of qp"); + goto create_qp_exit1; + } + if (ret) { ret = -ENOMEM; ehca_err(pd->device, "Can't allocate new idr entry."); diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 4bc687f..3340f49 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -164,7 +164,7 @@ static int ehca_mmap_cq(struct vm_area_struct *vma, struct ehca_cq *cq, int ret; switch (rsrc_type) { - case 1: /* galpa fw handle */ + case 0: /* galpa fw handle */ ehca_dbg(cq->ib_cq.device, "cq_num=%x fw", cq->cq_number); ret = ehca_mmap_fw(vma, &cq->galpas, &cq->mm_count_galpa); if (unlikely(ret)) { @@ -175,7 +175,7 @@ static int ehca_mmap_cq(struct vm_area_struct *vma, struct ehca_cq *cq, } break; - case 2: /* cq queue_addr */ + case 1: /* cq queue_addr */ ehca_dbg(cq->ib_cq.device, "cq_num=%x queue", cq->cq_number); ret = ehca_mmap_queue(vma, &cq->ipz_queue, &cq->mm_count_queue); if (unlikely(ret)) { @@ -201,7 +201,7 @@ static int ehca_mmap_qp(struct vm_area_struct *vma, struct ehca_qp *qp, int ret; switch (rsrc_type) { - case 1: /* galpa fw handle */ + case 0: /* galpa fw handle */ ehca_dbg(qp->ib_qp.device, "qp_num=%x fw", qp->ib_qp.qp_num); ret = ehca_mmap_fw(vma, &qp->galpas, &qp->mm_count_galpa); if (unlikely(ret)) { @@ -212,7 +212,7 @@ static int ehca_mmap_qp(struct vm_area_struct *vma, struct ehca_qp *qp, } break; - case 2: /* qp rqueue_addr */ + case 1: /* qp rqueue_addr */ ehca_dbg(qp->ib_qp.device, "qp_num=%x rqueue", qp->ib_qp.qp_num); ret = ehca_mmap_queue(vma, &qp->ipz_rqueue, @@ -225,7 +225,7 @@ static int ehca_mmap_qp(struct vm_area_struct *vma, struct ehca_qp *qp, } break; - case 3: /* qp squeue_addr */ + case 2: /* qp squeue_addr */ ehca_dbg(qp->ib_qp.device, "qp_num=%x squeue", qp->ib_qp.qp_num); ret = ehca_mmap_queue(vma, &qp->ipz_squeue, @@ -249,10 +249,10 @@ static int ehca_mmap_qp(struct vm_area_struct *vma, struct ehca_qp *qp, int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) { - u64 fileoffset = vma->vm_pgoff << PAGE_SHIFT; - u32 idr_handle = fileoffset >> 32; - u32 q_type = (fileoffset >> 28) & 0xF; /* CQ, QP,... */ - u32 rsrc_type = (fileoffset >> 24) & 0xF; /* sq,rq,cmnd_window */ + u64 fileoffset = vma->vm_pgoff; + u32 idr_handle = fileoffset & 0x1FFFFFF; + u32 q_type = (fileoffset >> 27) & 0x1; /* CQ, QP,... */ + u32 rsrc_type = (fileoffset >> 25) & 0x3; /* sq,rq,cmnd_window */ u32 cur_pid = current->tgid; u32 ret; struct ehca_cq *cq; @@ -261,7 +261,7 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) struct ib_uobject *uobject; switch (q_type) { - case 1: /* CQ */ + case 0: /* CQ */ read_lock(&ehca_cq_idr_lock); cq = idr_find(&ehca_cq_idr, idr_handle); read_unlock(&ehca_cq_idr_lock); @@ -289,7 +289,7 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) } break; - case 2: /* QP */ + case 1: /* QP */ read_lock(&ehca_qp_idr_lock); qp = idr_find(&ehca_qp_idr, idr_handle); read_unlock(&ehca_qp_idr_lock); -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:42:52 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:42:52 +0200 Subject: [ofa-general] [PATCH 5/7] IB/ehca: map 4k firmware context of cq, qp to user space Message-ID: <200708082042.52977.ossrosch@linux.vnet.ibm.com> From: Hoang-Nam Nguyen Date: Wed, 8 Aug 2007 19:33:23 +0200 This patch utilizes remap_4k_pfn() as introduced by Paul M., for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281, to map ehca cq, qp firmware context (4k) to user space if kernel page size is 64k. For reason, why this is required, see also Paul's patch. In addition to that the kernel page offset of firmware context needs to be set in cq and qp response block so that user space can assemble the proper virtual address to use. An appropriate patch for libehca will follow for ofed-1.3. Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++- drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++ drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++ drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +++--- 4 files changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index b5e9603..206d4eb 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -337,6 +337,8 @@ struct ehca_create_cq_resp { u32 cq_number; u32 token; struct ipzu_queue_resp ipz_queue; + u32 fw_handle_ofs; + u32 dummy; }; struct ehca_create_qp_resp { @@ -347,7 +349,7 @@ struct ehca_create_qp_resp { u32 qkey; /* qp_num assigned by ehca: sqp0/1 may have got different numbers */ u32 real_qp_num; - u32 dummy; /* padding for 8 byte alignment */ + u32 fw_handle_ofs; struct ipzu_queue_resp ipz_squeue; struct ipzu_queue_resp ipz_rqueue; }; diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c index c661939..0ac5a97 100644 --- a/drivers/infiniband/hw/ehca/ehca_cq.c +++ b/drivers/infiniband/hw/ehca/ehca_cq.c @@ -281,6 +281,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector, resp.ipz_queue.queue_length = ipz_queue->queue_length; resp.ipz_queue.pagesize = ipz_queue->pagesize; resp.ipz_queue.toggle_state = ipz_queue->toggle_state; + resp.fw_handle_ofs = (u32) + (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof(resp))) { ehca_err(device, "Copy to udata failed."); goto create_cq_exit4; diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index f26801b..d8c1c22 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -752,6 +752,8 @@ static struct ehca_qp *internal_create_qp( queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue); if (HAS_RQ(my_qp)) queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue); + resp.fw_handle_ofs = (u32) + (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1)); if (ib_copy_to_udata(udata, &resp, sizeof resp)) { ehca_err(pd->device, "Copy to udata failed"); diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c index 3340f49..84a16bc 100644 --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, u64 vsize, physical; vsize = vma->vm_end - vma->vm_start; - if (vsize != EHCA_PAGESIZE) { + if (vsize < EHCA_PAGESIZE) { ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start); return -EINVAL; } @@ -118,8 +118,8 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas, vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical); /* VM_IO | VM_RESERVED are set by remap_pfn_range() */ - ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT, - vsize, vma->vm_page_prot); + ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT, + vma->vm_page_prot); if (unlikely(ret)) { ehca_gen_err("remap_pfn_range() failed ret=%x", ret); return -ENOMEM; -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:43:46 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:43:46 +0200 Subject: [ofa-general] [PATCH 6/7] IB/ehca: Generate last WQE reached, when base QP for SRQ has entered error state Message-ID: <200708082043.47353.ossrosch@linux.vnet.ibm.com> From: Joachim Fenkes Date: Wed, 8 Aug 2007 19:51:30 +0200 Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_irq.c | 48 +++++++++++++++++++++----------- 1 files changed, 31 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c index ee06d8b..941816a 100644 --- a/drivers/infiniband/hw/ehca/ehca_irq.c +++ b/drivers/infiniband/hw/ehca/ehca_irq.c @@ -175,41 +175,55 @@ error_data1: } -static void qp_event_callback(struct ehca_shca *shca, u64 eqe, - enum ib_event_type event_type, int fatal) +static void dispatch_qp_event(struct ehca_shca *shca, struct ehca_qp *qp, + enum ib_event_type event_type) { struct ib_event event; - struct ehca_qp *qp; - u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); - - read_lock(&ehca_qp_idr_lock); - qp = idr_find(&ehca_qp_idr, token); - read_unlock(&ehca_qp_idr_lock); - - - if (!qp) - return; - - if (fatal) - ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); event.device = &shca->ib_device; + event.event = event_type; if (qp->ext_type == EQPT_SRQ) { if (!qp->ib_srq.event_handler) return; - event.event = fatal ? IB_EVENT_SRQ_ERR : event_type; event.element.srq = &qp->ib_srq; qp->ib_srq.event_handler(&event, qp->ib_srq.srq_context); } else { if (!qp->ib_qp.event_handler) return; - event.event = event_type; event.element.qp = &qp->ib_qp; qp->ib_qp.event_handler(&event, qp->ib_qp.qp_context); } +} + +static void qp_event_callback(struct ehca_shca *shca, u64 eqe, + enum ib_event_type event_type, int fatal) +{ + struct ehca_qp *qp; + u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); + + read_lock(&ehca_qp_idr_lock); + qp = idr_find(&ehca_qp_idr, token); + read_unlock(&ehca_qp_idr_lock); + + if (!qp) + return; + + if (fatal) + ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); + + dispatch_qp_event(shca, qp, fatal && qp->ext_type == EQPT_SRQ ? + IB_EVENT_SRQ_ERR : event_type); + + /* + * eHCA only processes one WQE at a time for SRQ base QPs, + * so the last WQE has been processed as soon as the QP enters + * error state. + */ + if (fatal && qp->ext_type == EQPT_SRQBASE) + dispatch_qp_event(shca, qp, IB_EVENT_QP_LAST_WQE_REACHED); return; } -- 1.5.2 From ossrosch at linux.vnet.ibm.com Wed Aug 8 11:44:37 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 8 Aug 2007 20:44:37 +0200 Subject: [ofa-general] [PATCH 7/7] IB/ehca: Prevent overwriting QP init attributes given by caller Message-ID: <200708082044.38247.ossrosch@linux.vnet.ibm.com> Signed-off-by: Stefan Roscher --- drivers/infiniband/hw/ehca/ehca_qp.c | 14 +++++--------- 1 files changed, 5 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index d8c1c22..6efda3d 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -709,12 +709,12 @@ static struct ehca_qp *internal_create_qp( my_qp->ib_qp.event_handler = init_attr->event_handler; } - init_attr->cap.max_inline_data = 0; /* not supported yet */ - init_attr->cap.max_recv_sge = parms.rqueue.act_nr_sges; - init_attr->cap.max_recv_wr = parms.rqueue.act_nr_wqes; - init_attr->cap.max_send_sge = parms.squeue.act_nr_sges; - init_attr->cap.max_send_wr = parms.squeue.act_nr_wqes; my_qp->init_attr = *init_attr; + my_qp->init_attr.cap.max_inline_data = 0; /* not supported yet */ + my_qp->init_attr.cap.max_recv_sge = parms.rqueue.act_nr_sges; + my_qp->init_attr.cap.max_recv_wr = parms.rqueue.act_nr_wqes; + my_qp->init_attr.cap.max_send_sge = parms.squeue.act_nr_sges; + my_qp->init_attr.cap.max_send_wr = parms.squeue.act_nr_wqes; /* NOTE: define_apq0() not supported yet */ if (qp_type == IB_QPT_GSI) { @@ -825,10 +825,6 @@ struct ib_srq *ehca_create_srq(struct ib_pd *pd, if (IS_ERR(my_qp)) return (struct ib_srq *)my_qp; - /* copy back return values */ - srq_init_attr->attr.max_wr = qp_init_attr.cap.max_recv_wr; - srq_init_attr->attr.max_sge = qp_init_attr.cap.max_recv_sge; - /* drive SRQ into RTR state */ mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL); if (!mqpcb) { -- 1.5.2 From jgunthorpe at obsidianresearch.com Wed Aug 8 11:46:32 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Wed, 8 Aug 2007 12:46:32 -0600 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070808103104.GB4571@mellanox.co.il> References: <20070807221700.GE8504@obsidianresearch.com> <000001c7d950$72242150$60cc180a@amr.corp.intel.com> <20070808020526.GE9424@obsidianresearch.com> <20070808103104.GB4571@mellanox.co.il> Message-ID: <20070808184632.GH8504@obsidianresearch.com> On Wed, Aug 08, 2007 at 01:31:04PM +0300, Michael S. Tsirkin wrote: > > AFAIK, when using the 64K MTU setting for IPoIB, if the remote side > > doesn't support RC then things go wonky. > > The scenario you describe works fine for me. > What does go wonky for you? Only this path MTU buisness, which is somewhat unusual. PMTU is difficult to deal with in UDP applications if you want to prevent fragmentation. Wonky is maybe not a good term. > > For TCP things *might* be saved > > by path mtu discovery - > > MTU discovery is not restricted to TCP AFAIK. That is something I admit I've never looked at in Linux, if PMTU affects the route cache or the socket.. > > but PMTU is driven by ICMP errors which are not generated by an IB > > network. > IPoIB CM code generates them in software. Oh! That solves it for RC then. Does this happen for UD as well? Jason From mshefty at ichips.intel.com Wed Aug 8 11:52:11 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 08 Aug 2007 11:52:11 -0700 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service In-Reply-To: <46B95938.2030804@voltaire.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> <46B95938.2030804@voltaire.com> Message-ID: <46BA10DB.4030705@ichips.intel.com> > To mimic sockets, rdma_set_service_type() is perfectly correct and > usable by socket offloaders such as SDP and RDS, so they can translate > an IP_TOS setsockopt() call to rdma_set_service_type(). This plugs to > the case where the application wants to request per connection QoS class. Adding the SID to the PR query is easy enough to do. Here's the issue that I would like to resolve first. Starting first with something transport neutral, I propose the using rdma_set_service_type() combined with the flowinfo in the IPv6 address. If we can agree that this is usable, now let's add the SID to the PR query. Since, SID -> QoS, and rdma_set_service_type() sets the QoS, what gets set in the PR query? The SID -> QoS may not match the QoS requested by the application. Should the SID always be set, or only if QoS is not specified through some other means? - Sean From mst at dev.mellanox.co.il Wed Aug 8 11:55:00 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 21:55:00 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <46BA0CE4.8050504@ichips.intel.com> References: <20070807175055.GD8504@obsidianresearch.com> <000501c7d925$597118f0$1acc180a@amr.corp.intel.com> <20070807221700.GE8504@obsidianresearch.com> <20070808104037.GD4571@mellanox.co.il> <46BA0CE4.8050504@ichips.intel.com> Message-ID: <20070808185500.GB8530@mellanox.co.il> > Quoting Sean Hefty : > Subject: Re: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify?TClassand?FlowLabelwith PR queries for QoS support > > >AFAIK tavor only works better with 1K MTU with connected transport types. > >With UD, the overhead of extra WRs seems to outweight the speedup > >we get from hardware. > > Doesn't IPoIB use the same path record for both UD and RC, if RC is > available? Good point, yes it does. > I guess the question still remains, does IPoIB need to (or should it) > specify the MTU, or should it be set by the SA? It sounds like IPoIB > can handle the case where the path MTU is less than the device MTU. Here's what happens if we activate tavor workaround in SM: ipoib datagram mode actually *ignores* the MTU from path query and uses the MTU from broadcast group. ipoib CM uses MTU from path query. As a result, datagram mode uses 2K MTU while connected mode uses 1K MTU. > It doesn't specify it today, and the QoS patches seem like a somewhat > separate issue. I agree. -- MST From mst at dev.mellanox.co.il Wed Aug 8 11:58:15 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 21:58:15 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070808184632.GH8504@obsidianresearch.com> References: <20070807221700.GE8504@obsidianresearch.com> <000001c7d950$72242150$60cc180a@amr.corp.intel.com> <20070808020526.GE9424@obsidianresearch.com> <20070808103104.GB4571@mellanox.co.il> <20070808184632.GH8504@obsidianresearch.com> Message-ID: <20070808185815.GC8530@mellanox.co.il> > > > but PMTU is driven by ICMP errors which are not generated by an IB > > > network. > > > IPoIB CM code generates them in software. > > Oh! That solves it for RC then. Does this happen for UD as well? The ICMP is generated when the packet is too long to be sent and has don't fragment bit set. -- MST From eitan at mellanox.co.il Wed Aug 8 12:03:11 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Wed, 8 Aug 2007 22:03:11 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeof service In-Reply-To: <46BA10DB.4030705@ichips.intel.com> References: <000201c7d887$d96f67d0$9c98070a@amr.corp.intel.com> <46B8388E.1090308@voltaire.com> <6C2C79E72C305246B504CBA17B5500C9020BD6B2@mtlexch01.mtl.com> <46B95938.2030804@voltaire.com> <46BA10DB.4030705@ichips.intel.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BDDD2@mtlexch01.mtl.com> Hi Sean, The point is that the SID and QoS Class (ala DiffServ class coming from TOS) are written to different fields of the QoS aware PR: * SID is written into the PR new SID field. * The QoS Class coming from the socket option gets into the QoSClass field. To select the QoS level the policy on the SM/SA might use both fields, one of the fields or none of the fields. But this is what policy is all about. Eitan Zahavi > -----Original Message----- > From: Sean Hefty [mailto:mshefty at ichips.intel.com] > Sent: Wednesday, August 08, 2007 9:52 PM > To: Or Gerlitz > Cc: Eitan Zahavi; OpenIB > Subject: Re: [ofa-general] [PATCH 3/4] rdma/cm: add ability > to specify typeof service > > > To mimic sockets, rdma_set_service_type() is perfectly correct and > > usable by socket offloaders such as SDP and RDS, so they > can translate > > an IP_TOS setsockopt() call to rdma_set_service_type(). > This plugs to > > the case where the application wants to request per > connection QoS class. > > Adding the SID to the PR query is easy enough to do. Here's > the issue that I would like to resolve first. > > Starting first with something transport neutral, I propose the using > rdma_set_service_type() combined with the flowinfo in the > IPv6 address. > If we can agree that this is usable, now let's add the SID > to the PR query. > > Since, SID -> QoS, and rdma_set_service_type() sets the QoS, > what gets set in the PR query? The SID -> QoS may not match > the QoS requested by the application. Should the SID always > be set, or only if QoS is not specified through some other means? > > - Sean > From mshefty at ichips.intel.com Wed Aug 8 12:05:31 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 08 Aug 2007 12:05:31 -0700 Subject: [ofa-general] [PATCH 1/4] ib/ipoib: specify TClass and FlowLabel with PR queries for QoS support In-Reply-To: <46B9541C.1070307@voltaire.com> References: <000001c7d886$d19e7ab0$9c98070a@amr.corp.intel.com> <46B8439C.6010303@voltaire.com> <46B89D9A.4080504@ichips.intel.com> <46B9541C.1070307@voltaire.com> Message-ID: <46BA13FB.1080706@ichips.intel.com> > Doesn't this suggest that the QoS architecture for all ULPs should such > which makes them request specific TC/FL since if the associated traffic > would be routed, the GRH is what the remote router has at hand to ask > the remote SM/SA to determine the desired QoS? This was my basic thought, and what I was thinking of as an ideal case. Although technically, the remote router has both the GRH and received LRH (possible from an intermediate subnet) that it can use to determine the desired QoS. - Sean From mst at dev.mellanox.co.il Wed Aug 8 12:09:17 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 22:09:17 +0300 Subject: [ofa-general] Re: [PATCH 1/4] ib/ipoib: specify TClassand FlowLabelwith PR queries for QoS support In-Reply-To: <20070808184632.GH8504@obsidianresearch.com> References: <20070807221700.GE8504@obsidianresearch.com> <000001c7d950$72242150$60cc180a@amr.corp.intel.com> <20070808020526.GE9424@obsidianresearch.com> <20070808103104.GB4571@mellanox.co.il> <20070808184632.GH8504@obsidianresearch.com> Message-ID: <20070808190917.GD8530@mellanox.co.il> > Quoting Jason Gunthorpe : > Subject: Re: [PATCH 1/4] ib/ipoib: specify TClassand?FlowLabelwith PR queries for QoS support > > On Wed, Aug 08, 2007 at 01:31:04PM +0300, Michael S. Tsirkin wrote: > > > AFAIK, when using the 64K MTU setting for IPoIB, if the remote side > > > doesn't support RC then things go wonky. > > > > The scenario you describe works fine for me. > > What does go wonky for you? > > Only this path MTU buisness, which is somewhat unusual. PMTU is > difficult to deal with in UDP applications if you want to prevent > fragmentation. Wonky is maybe not a good term. But this is not really IPoIB specific, is it? -- MST From swise at opengridcomputing.com Wed Aug 8 12:12:00 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 08 Aug 2007 14:12:00 -0500 Subject: [ofa-general] problems with ofed-1.2.c chelsio low level driver Message-ID: <46BA1580.9040207@opengridcomputing.com> Michael, I've discovered that I was really testing the ofed-1.2 cxgb3.ko module in my testing 1.2.c regression test. Upon getting a complete 1.2.c installed, I'm seeing crashes with the chelsio driver. I think there are some patches that went into the ofed-1.2 branch, that did not get merged into ofed-1.2.c. The ofed-1.2 branch works, but 1.2.c is broken. I'm investigating now, but we'll need to fix this before shipping 1.2.c/1.2.5. Steve. From mst at dev.mellanox.co.il Wed Aug 8 12:15:09 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 8 Aug 2007 22:15:09 +0300 Subject: [ofa-general] Re: problems with ofed-1.2.c chelsio low level driver In-Reply-To: <46BA1580.9040207@opengridcomputing.com> References: <46BA1580.9040207@opengridcomputing.com> Message-ID: <20070808191509.GE8530@mellanox.co.il> > Quoting Steve Wise : > Subject: problems with ofed-1.2.c chelsio low level driver > > Michael, > > I've discovered that I was really testing the ofed-1.2 cxgb3.ko module > in my testing 1.2.c regression test. Upon getting a complete 1.2.c > installed, I'm seeing crashes with the chelsio driver. I think there > are some patches that went into the ofed-1.2 branch, that did not get > merged into ofed-1.2.c. The ofed-1.2 branch works, but 1.2.c is broken. > > I'm investigating now, but we'll need to fix this before shipping > 1.2.c/1.2.5. Sure, send 'em over. -- MST From sean.hefty at intel.com Wed Aug 8 12:23:20 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Wed, 8 Aug 2007 12:23:20 -0700 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeofservice In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9020BDDD2@mtlexch01.mtl.com> Message-ID: <000001c7d9f1$94fb4770$ff0da8c0@amr.corp.intel.com> >The point is that the SID and QoS Class (ala DiffServ class coming from >TOS) are written to different fields of the QoS aware PR: >* SID is written into the PR new SID field. >* The QoS Class coming from the socket option gets into the QoSClass >field. > >To select the QoS level the policy on the SM/SA might use both fields, >one of the fields or none of the fields. >But this is what policy is all about. I understand, but we need to determine the policy on the host as well. Does rdma_set_service_type() indicate that the app doesn't want its SID used with the query, or does it merely convey additional restrictions? I was leaning towards always including the SID with the PR query, but I want to understand the consequences of doing so. The result is that it gives all of the policy to the SA. Does anyone disagree with this approach? - Sean From sashak at voltaire.com Wed Aug 8 12:38:18 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 8 Aug 2007 22:38:18 +0300 Subject: [ofa-general] Re: nightly osm_sim report 2007-08-08:normal completion In-Reply-To: References: Message-ID: <20070808193818.GN27000@sashak.voltaire.com> Hi Yevgeny, On 07:44 Wed 08 Aug , kliteyn at mellanox.co.il wrote: > > Failures: > 42 Pkey IS1-16.topo > 14 Pkey IS3-128.topo > 2 Stability IS1-16.topo > 1 OsmTest IS3-128.topo Any details about Stability and OsmTest failures? Thanks. Sasha From sashak at voltaire.com Wed Aug 8 12:58:40 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 8 Aug 2007 22:58:40 +0300 Subject: [ofa-general] ibnetdiscover In-Reply-To: <200708081137.30732.bs@q-leap.de> References: <200708071426.36464.bs@q-leap.de> <200708081137.30732.bs@q-leap.de> Message-ID: <20070808195840.GO27000@sashak.voltaire.com> On 11:37 Wed 08 Aug , Bernd Schubert wrote: > > > > > I'm also not sure about the pathes, IMHO the man page of ibnetdiscover > > > should give one more example, so > > > > > > > > > > > > -D use directed path address arguments. The path > > > is a comma separated list of out ports. > > > Examples: > > > "0" # self port > > > "0,1,2,1,4" # out via port 1, then 2, ... > > > > > > > > > "out via port 1, then out via port 2, then out via port 1, ..." > > > > > > or > > > > > > "out via port 1, then in port 2, then out via port 1, ..." > > > > > > > > > You see what I mean? > > > > It's the former. It's the out port on each hop along the path. > > Thanks, do you mind to apply this patch? I know the text already says > "separated list of out ports", but for those who don't believe it > like me ;) the comment will make it a bit more convincing. Applied. Thanks. Please next time add 'Signed-off-by' line. Sasha > > > --- ./ofa_user-1.2.orig/src/userspace/management/diags/man/ibnetdiscover.8 2007-06-21 16:39:17.000000000 +0200 > +++ ./ofa_user-1.2/src/userspace/management/diags/man/ibnetdiscover.8 2007-08-08 11:15:33.000000000 +0200 > @@ -66,7 +66,7 @@ > is a comma separated list of out ports. > Examples: > "0" # self port > - "0,1,2,1,4" # out via port 1, then 2, ... > + "0,1,2,1,4" # out via port 1, then out via port 2, ... > .PP > \-G use GUID address argument. In most cases, it is the Port GUID. > Example: > > > Thanks again, > Bernd > > > -- > Bernd Schubert > Q-Leap Networks GmbH > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From sashak at voltaire.com Wed Aug 8 14:02:09 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 9 Aug 2007 00:02:09 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/scripts: Fix Bug 239 Error Reporting In-Reply-To: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> References: <46AFCF52.mailFVL1I50O7@systemfabricworks.com> Message-ID: <20070808210209.GQ27000@sashak.voltaire.com> On 19:09 Tue 31 Jul , davem at systemfabricworks.com wrote: > > > Fix Bug 239 OpenIB diag scripts don't return error when lacking umad > permissions. Returning the error from the head of a shell pipeline is a > problem, so this fix causes the awk scripts to pass error messages through. > This will pass all standard error messages. > > This patch needs [ofa-general] [PATCH] infiniband-diags: Add common flags > -P, -C, and -t (posted Tue Jul 31 13:39:27 PDT 2007) applied first. > > Signed-off-by: David A. McMillen Applied. Thanks. Sasha From shemminger at linux-foundation.org Wed Aug 8 13:55:01 2007 From: shemminger at linux-foundation.org (Stephen Hemminger) Date: Wed, 8 Aug 2007 21:55:01 +0100 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1186586075.5155.27.camel@localhost> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808.034900.85820906.davem@davemloft.net> <20070808134247.GA9942@gondor.apana.org.au> <1186586075.5155.27.camel@localhost> Message-ID: <20070808215501.4825d74b@oldman> On Wed, 08 Aug 2007 11:14:35 -0400 jamal wrote: > On Wed, 2007-08-08 at 21:42 +0800, Herbert Xu wrote: > > On Wed, Aug 08, 2007 at 03:49:00AM -0700, David Miller wrote: > > > > > > Not because I think it obviates your work, but rather because I'm > > > curious, could you test a TSO-in-hardware driver converted to > > > batching and see how TSO alone compares to batching for a pure > > > TCP workload? > > > > You could even lower the bar by disabling TSO and enabling > > software GSO. > > >From my observation for TCP packets slightly above MDU (upto 2K), GSO > gives worse performance than non-GSO throughput-wise. Actually this has > nothing to do with batching, rather the behavior is consistent with or > without batching changes. > > > > I personally don't think it will help for that case at all as > > > TSO likely does better job of coalescing the work _and_ reducing > > > bus traffic as well as work in the TCP stack. > > > > I agree. > > I suspect the bulk of the effort is in getting > > these skb's created and processed by the stack so that by > > the time that they're exiting the qdisc there's not much > > to be saved anymore. > > pktgen shows a clear win if you test the driver path - which is what you > should test because thats where the batching changes are. > Using TCP or UDP adds other variables[1] that need to be isolated first > in order to quantify the effect of batching. > For throughput and CPU utilization, the benefit will be clear when there > are a lot more flows. Optimizing for pktgen is a mistake for most users. Please show something useful like router forwarding, TCP (single and multi flow) and/or better yet application benchmark improvement. From sashak at voltaire.com Wed Aug 8 14:42:39 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 9 Aug 2007 00:42:39 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Add ibcli program and man page In-Reply-To: <46B09BB6.mailK1S117033@systemfabricworks.com> References: <46B09BB6.mailK1S117033@systemfabricworks.com> Message-ID: <20070808214239.GR27000@sashak.voltaire.com> Hi, On 09:41 Wed 01 Aug , davem at systemfabricworks.com wrote: > > Add ibcli program and man page to infiniband-diags. [snip...] > +.SH SYNOPSIS > +.B ibcli > +get [archive] srp|opensm|opensm.opts|sdp|openib [?|param] > + > +.B ibcli > +set [archive] srp|opensm|opensm.opts|sdp|openib param=?|value > + > +.B ibcli > +edit [archive] srp|opensm|opensm.opts|sdp|openib [editor] > + > +.B ibcli > +save archive > + > +.B ibcli > +restore archive > + > +.B ibcli > +copy source-archive archive > + > +.SH DESCRIPTION > +.PP > +ibcli is a command line interface (CLI) designed to simplify and unify > +the use of various OFED software components. There are several parts > +to this program: > + > +.PP > +.TP > +Configuration file management/maintenance/inquiry > + > +The various configuration files are accessed through simple names > +within the CLI, referencing the actual files in the appropriate > +locations depending on where and how the OFED software was installed. > + > +Extraction and modification of individual parameter/value pairs is > +easily done, and all configuration files are modified using a rename() > +system call. This means that either the old or new version in it's > +entirety will be seen by any program that opens and reads a file, even > +though it is unsynchronized with the modification program. Unlike an > +editor that rewrites a file, where there is a possibility that another > +program will see some but not all changes, the method used here will > +never present an inconsistent configuration. > + > +Also, the complete set of configuration files can be saved, duplicated, > +modified separate from the "live" versions, and restored. From this description it looks that ibcli is trying to replace some known tools like text editor, cp, etc.. Which is probably fine, but I still be unsure about how such new tool will be useful - unlike this standard set of unix/linux utils is well known, universal and greatly flexible. I would crearly choose standard tools, but it is just my opinion. I think it would be good to get opinions from people who are dealing with clusters configurations/administration about potential usability of such tool (ibcli). Anybody could comment? (Please feel free to forward this question to anybody who could provide useful input.) Sasha > + > +.SH OPTIONS > + > +.PP > +.TP > +\fBget\fR > +Fetches the contents of a configuration file. With no param specified, the > +entire contents of the file are returned. If the param is specified as ? > +then the file is returned with comments stripped. If a specific param is > +named, then only the line with that parameter is returned. > + > +.TP > +\fBset\fR > +Edits the contents of a configuration file. The line with the specific named > +parameter is modified to contain the new value specified. If the new value is > +specified as ? then no modification takes place, but the comments leading up > +to the parameter are displayed. > + > +.TP > +\fBedit\fR > +Edits the contents of a configuration file using an external editor. The > +specified configuration file is copied to a temporary file for editing. If > +the editor parameter was specified, it is used as the name of the editor > +program. Otherwise, if the VISUAL environment variable is set, it will be > +used as the editor program name. Otherwise, if the EDITOR environment > +variable is set, it will be used as the editor program name. Otherwise, > +vi is used. > + > +.TP > +\fBsave\fR > +The set of all configuration files are saved in the named archive. > + > +.TP > +\fBrestore\fR > +The set of all configuration files are restored from the named archive. > + > +.TP > +\fBcopy\fR > +The set of all configuration files in the source archive are copied to the > +named archive. > + > +.SH NOTES > + > +.PP > +Locking configuration files: A parallel file is created with the same name as > +the file to be locked, with the CONFIG_LOCK_SUFFIX (-locked) appended to the > +name. This will contain the PID of the program that did the lock. If a > +program dies holding the lock, this is detected, and the lock file removed. > +This method is reportedly broken for the general case on NFS filesystems, but > +seems to work when all locking accesses are from the same system. > + > +.PP > +Save and restore commands > +move all configuration files (as listed in the config_files array) between > +their normal location and a subdirectory in the saved config directory > +(/etc/infiniband/saved_configs). This subdirectory name is the name of > +the archive listed in the command, and is created during a save if it > +does not exist. If it does exist, it is renamed with a new suffix > +(~) to keep one previous copy. If there is already a > +subdirectory with that suffix, it (and it's contents) are deleted. > +A restore will first rename each target config file with the suffix, > +deleting any previous one with that name, and put the restored copy in. > + > +.SH AUTHORS > +.TP > +Dave McMillen > +.RI < davem at systemfabricworks.com > > diff --git a/infiniband-diags/src/ibcli.c b/infiniband-diags/src/ibcli.c > new file mode 100644 > index 0000000..44b762c > --- /dev/null > +++ b/infiniband-diags/src/ibcli.c > @@ -0,0 +1,740 @@ > +/* > + * Copyright (c) 2007 System Fabric Works, Inc. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* ibcli.c > + > + This is a command line interface (CLI) designed to simplify and unify > + the use of various OFED software components. There are several parts > + to this program: > + > + * Configuration file management/maintenance/inquiry > + > + The various configuration files are accessed through simple names > + within the CLI, referencing the actual files in the appropriate > + locations depending on where and how the OFED software was installed. > + > + Extraction and modification of individual parameter/value pairs is > + easily done, and all configuration files are modified using a rename() > + system call. This means that either the old or new version in it's > + entirety will be seen by any program that opens and reads a file, even > + though it is unsynchronized with the modification program. Unlike an > + editor that rewrites a file, where there is a possibility that another > + program will see some but not all changes, the method used here will > + never present an inconsistent configuration. > + > + Also, the complete set of configuration files can be saved, duplicated, > + modified separate from the "live" versions, and restored. > + > + * > +*/ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define SAVED_CONFIGS_DIR "/etc/infiniband/saved_configs" > +#define CONFIG_LOCK_SUFFIX "-locked" > +#define CONFIG_PREV_SUFFIX "~" > +#define CONFIG_NEW_SUFFIX "+" > +#define MAX_LOCK_ATTEMPTS 20 > +#define DEFAULT_EDITOR "vi" > + > +#define CMD_SAVE "save" > +#define CMD_RESTORE "restore" > +#define CMD_COPY "copy" > +#define CMD_GET "get" > +#define CMD_SET "set" > +#define CMD_EDIT "edit" > + > +struct config_files { > + char *cmd_name; /* Name used in commands to reference this */ > + char *file_name; /* Absolute path to configuration file */ > + int usespace; /* Param/value pairs do not have = between them */ > + }; > + > +struct config_files config_files[] = { /* TODO: Pick up installed places */ > + {"srp", "/etc/srp_daemon.conf", -1}, > + {"opensm", "/etc/opensm.conf", 0}, > + {"opensm.opts", "", 1}, > + {"sdp", "/etc/libsdp.conf", -1}, > + {"openib", "/etc/infiniband/openib.conf", 0}, > + { NULL, NULL } }; > + > +char *argv0; /* Remembers the argv[0] for the main() program */ > + > +char inbuf[16384]; /* Used for input and temporary purposes */ > +char savebuf[16384]; /* Saves commentary for later printout */ > +char *savebufp; /* Remembers how far we have filled savebuf */ > + > +/* For configuration files that may be at a user-defined place, figure out > + where they are located and return that location. */ > + > +char *find_cfg_file(char *cmd_name) { > + if (strcmp(cmd_name, "opensm.opts") == 0) { > + return "/var/cache/osm/opensm.opts"; /* TODO: find real path value */ > + } > + fprintf(stderr, "ERROR: %s internal error, unknown config file for %s\n", > + argv0, cmd_name); > + exit(2); > +} > + > +/* Find the last non-/ character in a string */ > + > +char *base_pointer(char *path) { > + int l = strlen(path); > + while (l > 0 && path[l-1] != '/') l--; > + return path + l; > +} > + > +/* Return a malloc()'d string with the full file name of the configuration > + file, given the archive name and standard running full filename */ > + > +char *working_cfg_file(char *archive, char *filename) { > + char *rv, *fv; > + int srv; > + > + if (archive == NULL) { > + rv = (char *)malloc(strlen(filename) + strlen(CONFIG_PREV_SUFFIX) + 1); > + strcpy(rv, filename); > + return rv; > + } > + fv = base_pointer(filename); > + srv = strlen(SAVED_CONFIGS_DIR) + 1 + strlen(archive) + 1 + strlen(fv) > + + strlen(CONFIG_PREV_SUFFIX) + 1; > + rv = (char *)malloc(srv); > + strcpy(rv, SAVED_CONFIGS_DIR); > + strcat(rv, "/"); > + strcat(rv, archive); > + strcat(rv, "/"); > + strcat(rv, fv); > + return rv; > +} > + > +/* Tell the user how to invoke this program */ > + > +void usage() { > + int cf_index; > + char sep = ' '; > + > + fprintf(stderr, "Usage:\n"); > + fprintf(stderr, " %s get [archive]", argv0); > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > + sep = '|'; > + } > + fprintf(stderr, " [?|param]\n"); > + sep = ' '; > + fprintf(stderr, " %s set [archive]", argv0); > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > + sep = '|'; > + } > + fprintf(stderr, " param=?|value\n"); > + sep = ' '; > + fprintf(stderr, " %s edit [archive]", argv0); > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > + sep = '|'; > + } > + fprintf(stderr, " [editor]\n"); > + fprintf(stderr, " %s save|restore archive\n", argv0); > + fprintf(stderr, " %s copy source-archive archive\n", argv0); > + exit(1); > +} > + > +/* Lock a configuration file. A parallel file is created with the same name as > + the file to be locked, with the CONFIG_LOCK_SUFFIX appended to the name. > + This will contain the PID of the program that did the lock. If a program > + dies holding the lock, this is detected, and the lock file removed. This > + method is reportedly broken for the general case on NFS filesystems, but > + seems to work when all locking accesses are from the same system. */ > + > +void lock_config_file(char *filename) { > + int i, rc, attempts = MAX_LOCK_ATTEMPTS; > + long int lpid; > + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); > + > + strcpy(lfn, filename); > + strcat(lfn, CONFIG_LOCK_SUFFIX); > + while (--attempts >= 0) { > + i = open(lfn, O_WRONLY | O_CREAT | O_EXCL, 0644); > + if (i != -1) { > + sprintf(inbuf, "%d", getpid()); > + write(i, inbuf, strlen(inbuf)); > + close(i); > + free(lfn); > + return; > + } > + if (errno != EEXIST) { > + perror(lfn); > + fprintf(stderr, "%s: Could not open interlock file\n", argv0); > + free(lfn); > + exit(3); > + } > + i = open(lfn, O_RDONLY); > + if (i != -1) { > + rc = read(i, inbuf, sizeof(inbuf) - 1); > + close(i); > + if (rc) { > + lpid = strtol(inbuf, NULL, 10); > + if (kill(lpid, 0) == -1 && errno == ESRCH) unlink(lfn); > + } > + } > + sleep(1); > + } > + fprintf(stderr, "%s: Could not establish interlock file %s\n", argv0, lfn); > + free(lfn); > + exit(3); > +} > + > +/* Unlock a configuration file - simply remove the lock file */ > + > +void unlock_config_file(char *filename) { > + int i; > + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); > + > + strcpy(lfn, filename); > + strcat(lfn, CONFIG_LOCK_SUFFIX); > + i = unlink(lfn); > + if (i) { > + perror(lfn); > + fprintf(stderr, "%s: Could not remove interlock file\n", argv0); > + exit(3); > + } > + free(lfn); > +} > + > +/* Routine to copy a complete set of configuration files */ > + > +void copy_configs(char *source, char *destination) { > + int cf_index; > + char *ifn, *ofn, *tfn; > + FILE *input, *output; > + > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > + ofn = config_files[cf_index].file_name; > + if (ofn[0] == 0) > + ofn = find_cfg_file(config_files[cf_index].cmd_name); > + ifn = working_cfg_file(source, ofn); > + ofn = working_cfg_file(destination, ofn); > + tfn = (char *)malloc(strlen(ofn) > + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); > + lock_config_file(ifn); > + lock_config_file(ofn); > + input = fopen(ifn, "r"); > + if (!input) { > + int ern = errno; > + unlock_config_file(ofn); > + unlock_config_file(ifn); > + free(ofn); > + free(ifn); > + if (ern == ENOENT) continue; /* This config file not present */ > + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0, ifn); > + exit(2); > + } > + strcpy(tfn, ofn); > + strcat(tfn, CONFIG_PREV_SUFFIX); > + unlink(tfn); /* Remove any previous previous version */ > + link(ofn, tfn); /* Create link for new previous version */ > + strcpy(tfn, ofn); > + strcat(tfn, CONFIG_NEW_SUFFIX); > + output = fopen(tfn, "w"); > + if (!output) { > + fprintf(stderr, "%s: Cannot open to write into %s\n", argv0, tfn); > + exit(2); > + } > + while (fgets(inbuf, sizeof(inbuf), input)) > + if (fputs(inbuf, output) == EOF) { > + fprintf(stderr, "%s: Error writing copy to %s\n", argv0, tfn); > + exit(2); > + } > + fclose(input); > + fclose(output); > + if (rename(tfn, ofn)) { > + perror(ofn); > + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", > + argv0, tfn); > + exit(2); > + } > + unlock_config_file(ofn); > + unlock_config_file(ifn); > + free(ofn); > + free(ifn); > + } > +} > + > +/* Main program entry point (start here) */ > + > +int main(int argc, char **argv) { > + int i, a, setting, editing, query, found, psize, cf_index; > + int cmd_restore, cmd_copy; > + char *archive = NULL; > + char *use_cfg, *cp, *np, *param, *value, *tfn; > + struct stat stb; > + struct dirent *ne; > + FILE *input, *output; > + DIR *rmd; > + > + argv0 = base_pointer(*argv); > + if (argc < 3) usage(); > + > + > + /* save and restore commands > + * > + * move all configuration files (as listed in the config_files array) > + * between their normal location and a subdirectory in the saved config > + * directory (SAVED_CONFIGS_DIR). This subdirectory name is the name of > + * the archive listed in the command, and is created during a save if it > + * does not exist. If it does exist, it is renamed with a new suffix > + * (CONFIG_PREV_SUFFIX) to keep one previous copy. If there is already a > + * subdirectory with that suffix, it (and it's contents) are deleted. > + * A restore will first rename each target config file with the suffix, > + * deleting any previous one with that name, and put the restored copy in. > + */ > + > + cmd_restore = strcmp(argv[1], CMD_RESTORE) == 0; > + cmd_copy = strcmp(argv[1], CMD_COPY) == 0; > + setting = strcmp(argv[1], CMD_SET) == 0; > + editing = strcmp(argv[1], CMD_EDIT) == 0; > + if (strcmp(argv[1], CMD_SAVE) == 0 || cmd_restore || cmd_copy) { > + if (cmd_copy) { > + if (argc != 4) usage(); > + a = 3; > + } else { > + if (argc != 3) usage(); > + a = 2; > + } > + if (strcmp(argv[a], ".") == 0 > + || strncmp(argv[a], "..", 2) == 0 > + || strchr(argv[a], '/')) { > + fprintf(stderr, "%s: Saved configuration name \"%s\" cannot be " > + "., start with .., or contain a / character\n", > + argv0, argv[a]); > + return 2; > + } > + archive = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 > + + strlen(argv[a]) + 1); > + strcpy(archive, SAVED_CONFIGS_DIR); > + strcat(archive, "/"); > + strcat(archive, argv[a]); > + i = stat(archive, &stb); > + > + /* restore from archive */ > + > + if (cmd_restore) { > + if (i || !(S_ISDIR(stb.st_mode))) { > + fprintf(stderr, > + "%s: There is no saved configuration named \"%s\"\n", > + argv0, argv[a]); > + free(archive); > + return 2; > + } > + copy_configs(argv[a], NULL); > + > + /* save to archive, or copy archive-src to archive */ > + > + } else { > + if (cmd_copy) { /* Validate existence of archive1 */ > + np = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 > + + strlen(argv[2]) + 1); > + strcpy(np, SAVED_CONFIGS_DIR); > + strcat(np, "/"); > + strcat(np, argv[2]); > + if (stat(np, &stb) || !(S_ISDIR(stb.st_mode))) { > + fprintf(stderr, > + "%s: There is no saved configuration named \"%s\"\n", > + argv0, argv[2]); > + free(np); > + return 2; > + } > + free(np); > + } > + if (!i) { /* create suffixed directory if previous save done */ > + cp = (char *)malloc(strlen(archive) + 2); > + strcpy(cp, archive); > + strcat(cp, CONFIG_PREV_SUFFIX); > + rmd = opendir(cp); > + if (rmd) { > + while ((ne = readdir(rmd))) { > + if (strcmp(ne->d_name, ".") == 0 > + || strcmp(ne->d_name, "..") == 0) continue; > + np = (char *)malloc(strlen(cp) + 1 > + + strlen(ne->d_name) + 1); > + strcpy(np, cp); > + strcat(np, "/"); > + strcat(np, ne->d_name); > + unlink(np); /* Errors will report with rmdir() */ > + free(np); > + } > + closedir(rmd); > + errno = ENOENT; > + rmdir(cp); > + } > + if (errno != ENOENT) { > + perror(cp); > + fprintf(stderr, "%s: Cannot remove backup copy\n", > + argv0); > + free(cp); > + free(archive); > + return 2; > + } > + i = rename(archive, cp); > + if (i) { > + perror(archive); > + fprintf(stderr, > + "%s: Cannot rename existing copy to backup copy\n", > + argv0); > + free(cp); > + free(archive); > + return 2; > + } > + free(cp); > + } > + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) > + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) { > + fprintf(stderr, "%s: Cannot use reserved name \"%s\" for" > + " a saved archive\n", argv0, argv[a]); > + return 2; > + } > + i = mkdir(archive, 0755); > + if (i) { > + mkdir(SAVED_CONFIGS_DIR, 0755); /* In case it is missing */ > + i = mkdir(archive, 0755); > + } > + if (i) { > + perror(archive); > + fprintf(stderr, "%s: Cannot create directory for saving\n", > + argv0); > + free(archive); > + return 2; > + } > + if (cmd_copy) > + copy_configs(argv[2], argv[a]); > + else > + copy_configs(NULL, argv[a]); > + } > + free(archive); > + > + > + /* set and get commands > + */ > + > + } else if (setting || strcmp(argv[1], CMD_GET) == 0 || editing) { > + for (a = 2; a <= 3; a++) { > + if (argc <= a) usage(); > + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) > + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) > + break; > + if (config_files[cf_index].cmd_name != NULL) break; > + archive = argv[a]; > + } > + if (config_files[cf_index].cmd_name == NULL) usage(); > + use_cfg = config_files[cf_index].file_name; > + if (use_cfg[0] == 0) > + use_cfg = find_cfg_file(config_files[cf_index].cmd_name); > + use_cfg = working_cfg_file(archive, use_cfg); > + > + /* get with no parameter means get whole file */ > + > + if (!editing && argc <= (a+1)) { > + if (setting) usage(); > + input = fopen(use_cfg, "r"); > + if (input == NULL) { > + if (errno == ENOENT) > + fprintf(stderr, > + "%s: There is no saved configuration named \"%s\"\n", > + argv0, archive); > + else > + perror(use_cfg); > + return 2; > + } > + while (fgets(inbuf, sizeof(inbuf), input)) > + fputs(inbuf, stdout); > + fclose(input); > + free(use_cfg); > + return 0; > + } > + > + /* Verify that we can use simple parameter/value parsing on this */ > + > + if (!editing && config_files[cf_index].usespace < 0) { > + if (setting) > + fprintf(stderr, "%s: %s file format too complicated, use edit " > + "command\n", argv0, config_files[cf_index].cmd_name); > + else > + fprintf(stderr, "%s: %s file format too complicated, use get " > + "command with no param\n", argv0, > + config_files[cf_index].cmd_name); > + free(use_cfg); > + return 2; > + } > + if (editing) { > + if (argc <= (a+1)) { > + if (((cp = getenv("VISUAL")) == NULL || *cp == '\0') && > + ((cp = getenv("EDITOR")) == NULL || *cp == '\0')) > + cp = DEFAULT_EDITOR; > + param = (char *)malloc(strlen(cp)+1); > + strcpy(param, cp); > + } else { > + if (argc != (a+2)) usage(); > + param = (char *)malloc(strlen(argv[a+1])+1); > + strcpy(param, argv[a+1]); > + } > + } else { > + param = (char *)malloc(strlen(argv[a+1])+1); > + strcpy(param, argv[a+1]); > + } > + > + /* if setting, figure out if new value is in the same "word" as the > + * parameter name with an = to delineate them > + */ > + if (setting) { > + for (cp = param; *cp ; cp++) if (*cp == '=') break; > + if (*cp && cp[1] == 0) *cp = 0; > + if (*cp) { > + if (argc != (a+2)) usage(); > + value = cp + 1; > + } else { > + if (argc == (a+2)) { /* if no value, use ? as default */ > + value = "?"; > + } else { /* otherwise value follows */ > + if (argc == (a+3)) { > + value = argv[a+2]; > + if (*value == '=') value++; > + } else if (argc == (a+4) && strcmp(argv[a+2], "=") == 0) { > + value = argv[a+3]; > + } else > + usage(); > + } > + } > + while (cp > param && (cp[-1] == ' ' || cp[-1] == '\t')) cp--; > + *cp = 0; > + while(*value == ' ' || *value == '\t') value++; > + > + /* not set, must be get or edit */ > + } else { > + /* If doing get, make sure there are no extra arguments */ > + if (!editing && argc != (a+2)) usage(); > + } > + > + /* Check for query. For get this means list all values, while for set > + * this means show the comments that are in the file before the value. > + */ > + if (setting) > + query = !(strcmp("?", value)); > + else > + query = !(strcmp("?", param)); > + > + /* Now scan the file */ > + > + found = 0; > + input = fopen(use_cfg, "r"); > + if (!input) { > + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0,use_cfg); > + exit(2); > + } > + > + /* If set/edit command, we create a link to the current file as previous > + * version (so when we remove the current one, we have the backup), and > + * create a new file with a different name, which will get renamed to > + * the normal name at the end. > + */ > + if ((setting && !query) || editing) { > + lock_config_file(use_cfg); > + tfn = (char *)malloc(strlen(use_cfg) > + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); > + strcpy(tfn, use_cfg); > + strcat(tfn, CONFIG_PREV_SUFFIX); > + unlink(tfn); /* Remove any previous previous version */ > + link(use_cfg, tfn); /* Create link for new previous version */ > + strcpy(tfn, use_cfg); > + strcat(tfn, CONFIG_NEW_SUFFIX); > + output = fopen(tfn, "w"); > + if (!output) { > + unlock_config_file(use_cfg); > + fprintf(stderr, "%s: Cannot open to write into %s\n", > + argv0, tfn); > + exit(2); > + } > + } > + > + /* Now we scan the file */ > + > + psize = strlen(param); > + savebufp = savebuf; > + while (fgets(inbuf, sizeof(inbuf), input)) { > + cp = inbuf + strlen(inbuf); > + > + /* Strip trailing NL/CR */ > + > + while (cp > inbuf && > + (cp[-1] == '\n' || cp[-1] == '\r')) cp--; > + *cp = 0; > + > + /* Strip leading spaces/tabs */ > + > + cp = inbuf; > + while (*cp == ' ' || *cp == '\t') cp++; > + > + /* See if this is a comment or empty line */ > + > + if (*cp == '#' || *cp == 0) { > + if (setting && query && *cp == '#') { > + cp++; /* Save comments for set query output */ > + i = strlen(cp); > + if (i < (sizeof(savebuf) - (savebufp - savebuf) - 2)) { > + strcpy(savebufp, cp); > + savebufp += i; > + *savebufp++ = '\n'; > + } > + } > + > + /* Not a comment or empty line, must be part of a param/value */ > + > + } else if (!editing) { > + > + /* if get command with ? for param, show all param lines */ > + > + if (!setting && query) { > + puts(cp); > + > + /* get or set with param to look for, see if it is there */ > + > + } else { > + if (strncmp(param, cp, psize) == 0) { > + if (cp[psize] == 0 || cp[psize] == '=' > + || cp[psize] == ' ' || cp[psize] == '\t') { > + > + found++; > + if (setting) { > + if (query) { > + > + /* Found param, this a set query */ > + > + *savebufp = 0; > + fputs(savebuf, stdout); > + } else { > + > + /* Found param, this a set with value */ > + > + np = cp + psize; > + while (*np == ' ' || *np == '\t') np++; > + if (*np == '=') { > + np++; > + while (*np == ' ' || *np == '\t') np++; > + } > + > + /* Put new value where old value was */ > + > + i = (np - inbuf) + strlen(value) + 2; > + if (i > sizeof(inbuf)) { > + fprintf(stderr, "%s: New value for " > + "\"%s\" is %d bytes too long\n", > + argv0, param, > + (int)(sizeof(inbuf)-i)); > + exit(2); > + } else > + strcpy(np, value); > + } > + > + /* Found param and doing a get so just print it */ > + > + } else { > + puts(cp); > + } > + } > + } > + } > + savebufp = savebuf; /* Saw new param, toss old comments */ > + } > + > + /* If doing set or edit, copy all lines to replacement file */ > + > + if (editing || (setting && !query)) { > + if (fprintf(output, "%s\n", inbuf) < 0) { > + unlock_config_file(use_cfg); > + fprintf(stderr, "%s: Error writing copy to %s\n", > + argv0, tfn); > + exit(2); > + } > + } > + } > + fclose(input); > + > + /* If doing set, output is done, rename file to make it the real one */ > + > + if ((setting && !query) || editing) { > + if (!found && !editing) { /* Did not find param so we add it */ > + if (config_files[cf_index].usespace) > + fprintf(output, "%s %s\n", param, value); > + else > + fprintf(output, "%s = %s\n", param, value); > + } > + fclose(output); > + if (editing) { > + cp = (char *)malloc(strlen(param) + 2 + strlen(tfn) + 2); > + strcpy(cp, param); > + strcat(cp, " '"); > + strcat(cp, tfn); > + strcat(cp, "'"); > + system(cp); > + free(cp); > + } > + if (rename(tfn, use_cfg)) { > + perror(use_cfg); > + unlock_config_file(use_cfg); > + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", > + argv0, tfn); > + exit(2); > + } > + unlock_config_file(use_cfg); > + free(tfn); > + } > + free(param); > + free(use_cfg); > + > + /* Unknown command, just tell caller how to use the program */ > + > + } else > + usage(); > + > + return 0; > +} From timotha.daniell at pfarre-canisius.at Wed Aug 8 15:42:45 2007 From: timotha.daniell at pfarre-canisius.at (Chrystal Lake) Date: Wed, 8 Aug 2007 21:42:45 -0100 Subject: [ofa-general] To be or not to be. To be... Message-ID: <01c7da05$0e5ff5d0$9a5c7ac2@timotha.daniell> -------------- next part -------------- A non-text attachment was scrubbed... Name: nurka.gif Type: image/gif Size: 14148 bytes Desc: not available URL: From davem at davemloft.net Wed Aug 8 15:01:37 2007 From: davem at davemloft.net (David Miller) Date: Wed, 08 Aug 2007 15:01:37 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070808.034900.85820906.davem@davemloft.net> Message-ID: <20070808.150137.50597736.davem@davemloft.net> From: Krishna Kumar2 Date: Wed, 8 Aug 2007 16:39:47 +0530 > What do you generally think of the patch/implementation ? :) We have two driver implementation paths on recieve and now we'll have two on send, and that's not a good trend. In an ideal world all the drivers would be NAPI and netif_rx() would only be used by tunneling drivers and similar in the protocol layers. And likewise all sends would go through ->hard_start_xmit(). If you can come up with a long term strategy that gets rid of the special transmit method, that'd be great. We should make Linux network drivers easy to write, not more difficult by constantly adding most interfaces than we consolidate. From davem at davemloft.net Wed Aug 8 15:22:40 2007 From: davem at davemloft.net (David Miller) Date: Wed, 08 Aug 2007 15:22:40 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1186586075.5155.27.camel@localhost> References: <20070808.034900.85820906.davem@davemloft.net> <20070808134247.GA9942@gondor.apana.org.au> <1186586075.5155.27.camel@localhost> Message-ID: <20070808.152240.56052317.davem@davemloft.net> From: jamal Date: Wed, 08 Aug 2007 11:14:35 -0400 > pktgen shows a clear win if you test the driver path - which is what > you should test because thats where the batching changes are. The driver path, however, does not exist on an island and what we care about is the final result with the changes running inside the full system. So, to be honest, besides for initial internal development feedback, the isolated tests only have minimal merit and it's the full protocol tests that are really interesting. From hadi at cyberus.ca Wed Aug 8 15:40:32 2007 From: hadi at cyberus.ca (jamal) Date: Wed, 08 Aug 2007 18:40:32 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808215501.4825d74b@oldman> References: <20070808093114.15396.22797.sendpatchset@localhost.localdomain> <20070808.034900.85820906.davem@davemloft.net> <20070808134247.GA9942@gondor.apana.org.au> <1186586075.5155.27.camel@localhost> <20070808215501.4825d74b@oldman> Message-ID: <1186612832.5152.34.camel@localhost> On Wed, 2007-08-08 at 21:55 +0100, Stephen Hemminger wrote: > > pktgen shows a clear win if you test the driver path - which is what you > > should test because thats where the batching changes are. > > Using TCP or UDP adds other variables[1] that need to be isolated first > > in order to quantify the effect of batching. > > For throughput and CPU utilization, the benefit will be clear when there > > are a lot more flows. > > Optimizing for pktgen is a mistake for most users. There is no "optimization for pktgen". If you are improving the tx path, the first step is to test that you can show the tx path improved. pktgen happens to be the best test suite for that because it talks to the driver and exercises the changes. i.e if one cant show that exercising the direct path demonstrates improvements you probably wont be able to show batching improves TCP - but i dont even wanna swear by that. Does that make sense? > Please show something > useful like router forwarding, TCP (single and multi flow) and/or better > yet application benchmark improvement. Absolutely, but first things first. Analysis of why something improves is extremely important, just saying "TCP throughput improved" is not interesting and lazy. To be scientific, it is important to isolate variables first in order to come up with meaningful results that can be analysed. To make a point, I have noticed extremely different results between TCP BIC vs reno with batching. So congestion control as a variable is important. cheers, jamal From hadi at cyberus.ca Wed Aug 8 15:53:46 2007 From: hadi at cyberus.ca (jamal) Date: Wed, 08 Aug 2007 18:53:46 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.152240.56052317.davem@davemloft.net> References: <20070808.034900.85820906.davem@davemloft.net> <20070808134247.GA9942@gondor.apana.org.au> <1186586075.5155.27.camel@localhost> <20070808.152240.56052317.davem@davemloft.net> Message-ID: <1186613626.5152.48.camel@localhost> On Wed, 2007-08-08 at 15:22 -0700, David Miller wrote: > The driver path, however, does not exist on an island and what > we care about is the final result with the changes running > inside the full system. > > So, to be honest, besides for initial internal development > feedback, the isolated tests only have minimal merit and > it's the full protocol tests that are really interesting. But you cant go there if cant show the path which is supposedly improved has indeed improved;-> I would certainly agree with you that if it doesnt prove consistently useful with protocols it has no value (remember thats why i never submitted these patches all this time). We just need better analysis of the results - i cant ignore that the selection of the clock sources for example gives me different results and that when i boot i cant be guaranteed the same clock source. I cant ignore the fact that i get different results when i use a different congestion control algorithm. And none of this has to do with the batching patches. I am using UDP at the moment because it is simpler to analyze. And yes, it would be "an interesting idea that gets shelved" if we cant achieve any of the expected goals. We've shelved ideas before. BTW, read the little doc i wrote on the dev->prep_xmit() you may find it interesting. cheers, jamal From sean.hefty at intel.com Wed Aug 8 16:32:34 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Wed, 8 Aug 2007 16:32:34 -0700 Subject: [ofa-general] RE: sparse output In-Reply-To: <20070807112921.GC13863@mellanox.co.il> Message-ID: <000101c7da14$65edc1b0$ff0da8c0@amr.corp.intel.com> >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: warning: >incorrect type in argument 3 (different signedness) >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: >expected int *qp_attr_mask >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucm.c:625:53: got >unsigned int * >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: >warning: incorrect type in argument 3 (different signedness) >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: >expected int *qp_attr_mask >/home/mst/build/x86_64/linux-2.6/drivers/infiniband/core/ucma.c:781:49: got >unsigned int * Both of the above issues relate to exporting the qp attribute mask to user space. The userspace ABI uses u32, but the kernel interface is int. I would say that we just cast these two warnings away. - Sean From xma at us.ibm.com Wed Aug 8 17:06:41 2007 From: xma at us.ibm.com (Shirley Ma) Date: Wed, 8 Aug 2007 17:06:41 -0700 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808134247.GA9942@gondor.apana.org.au> Message-ID: Hello Herbert, > > Not because I think it obviates your work, but rather because I'm > > curious, could you test a TSO-in-hardware driver converted to > > batching and see how TSO alone compares to batching for a pure > > TCP workload? > > You could even lower the bar by disabling TSO and enabling > software GSO. We had a discuss before. GSO doesn't benefit device which has no HW checksum (like IPoIB) by inducing an extra copy. And GSO benefits one stream, batching benefits multiple streams. Thanks Shirley From changquing.tang at hp.com Wed Aug 8 20:01:39 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 9 Aug 2007 03:01:39 -0000 Subject: [ofa-general] RE: RFCv2: SRC API In-Reply-To: <20070806164422.GF6779@mellanox.co.il> References: <20070806161639.GE6779@mellanox.co.il><349DCDA352EACF42A0C49FA6DCEA840301F8EFD0@G3W0634.americas.hpqcorp.net> <20070806164422.GF6779@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA84030200C296@G3W0634.americas.hpqcorp.net> > > > Is there any performance worry to let j2(the first job on a > node) to > > do all the "work" ? > > How do you mean? I mean that j2 has all the QP connections with all other remote jobs, so j2 does not save anything compared with non-SRC case, can this affect j2's performance ? I have another question, when using SRQ, or SRC+SRQ, when a completion is returned by ibv_poll_cq(), is there an easy way to find who sent this message to me ? 'struct ibv_wc' does not say much about the message source rank. The only field is 'src_qp', but I can not find any document about how to use it. Is there any way to return some kind of "source context" ? Or do you think this is application's responsibility to figure out source rank from the message itself ? I hope the source rank can be identified before parsing the message. Thanks. --CQ > > > What is the latency of SRC+SRQ ? > > I'd expect it to be more or less the same as regular SRQ. > > -- > MST > From krkumar2 at in.ibm.com Wed Aug 8 20:09:03 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 08:39:03 +0530 Subject: [ofa-general] Re: [PATCH 2/9 Rev3] [core] Add skb_blist & hard_start_xmit_batch In-Reply-To: <20070808120142.GA26884@2ka.mipt.ru> Message-ID: Hi Evgeniy, Evgeniy Polyakov wrote on 08/08/2007 05:31:43 PM: > > +int dev_change_tx_batch_skb(struct net_device *dev, unsigned long new_batch_skb) > > +{ > > + int ret = 0; > > + struct sk_buff_head *blist; > > + > > + if (!dev->hard_start_xmit_batch) { > > + /* Driver doesn't support batching skb API */ > > + ret = -ENOTSUPP; > > + goto out; > > + } > > + > > + /* Handle invalid argument */ > > + if (new_batch_skb < 0) { > > + ret = -EINVAL; > > + goto out; > > + } > It is unsigned, how can it be less than zero? Yuck, originally I had it as int and changed to ulong and forgot to remove this check. > And actually you use it just like a binary flag (casted to/from u32 in > the code, btw), so why not using ethtool_value directly here? I still need to check if the value is changing, so the one check is needed. Later I am using it as a value directly. > > + /* Check if new value is same as the current */ > > + if (!!dev->skb_blist == !!new_batch_skb) > > + goto out; > > + > > + if (new_batch_skb && > > + (blist = kmalloc(sizeof *blist, GFP_KERNEL)) == NULL) { > > + ret = -ENOMEM; > > + goto out; > > + } > > + > > + spin_lock(&dev->queue_lock); > > + if (new_batch_skb) { > > + skb_queue_head_init(blist); > > + dev->skb_blist = blist; > > + } else > > + free_batching(dev); > > + spin_unlock(&dev->queue_lock); > > This needs bh lock too, since blist is accessed by qdisc_restart. Yes, had it in the code, put it in the list of changes, but missed it for some reason :( > > +int dev_add_skb_to_blist(struct sk_buff *skb, struct net_device *dev) > > +{ > > + if (!list_empty(&ptype_all)) > > + dev_queue_xmit_nit(skb, dev); > > + > > + if (netif_needs_gso(dev, skb)) { > > + if (unlikely(dev_gso_segment(skb))) { > > + kfree(skb); > > + return 0; > > + } > > + > > + if (skb->next) { > > + int count = 0; > > + > > + do { > > + struct sk_buff *nskb = skb->next; > > + > > + skb->next = nskb->next; > > + __skb_queue_tail(dev->skb_blist, nskb); > > + count++; > > + } while (skb->next); > > Is it possible to move list without iterating over each entry? Though I cannot see something obvious to do that, let me see if something is possible as it will make a good difference. thanks for your suggestions, - KK From krkumar2 at in.ibm.com Wed Aug 8 20:13:46 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 08:43:46 +0530 Subject: [ofa-general] Re: [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <20070808121401.GB8478@2ka.mipt.ru> Message-ID: Evgeniy Polyakov wrote on 08/08/2007 05:44:02 PM: > On Wed, Aug 08, 2007 at 03:01:45PM +0530, Krishna Kumar (krkumar2 at in.ibm.com) wrote: > > +static inline int get_skb(struct net_device *dev, struct Qdisc *q, > > + struct sk_buff_head *blist, struct sk_buff **skbp) > > +{ > > + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { > > + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); > > + } else { > > + int max = dev->tx_queue_len - skb_queue_len(blist); > > + struct sk_buff *skb; > > + > > + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) > > + max -= dev_add_skb_to_blist(skb, dev); > > + > > + *skbp = NULL; > > + return 1; /* we have atleast one skb in blist */ > > + } > > +} > > Same here - is it possible to get a list in one go instead of pulling > one-by-one, since it forces quite a few additional unneded lock > get/releases. What about dev_dequeue_number_skb(dev, q, num), which will > grab the lock and move a list of skbs from one queue to provided head. OK, I will try this out. > > @@ -158,7 +198,10 @@ static inline int qdisc_restart(struct n > > /* And release queue */ > > spin_unlock(&dev->queue_lock); > > > > - ret = dev_hard_start_xmit(skb, dev); > > + if (likely(skb)) > > + ret = dev_hard_start_xmit(skb, dev); > > + else > > + ret = dev->hard_start_xmit_batch(dev); > > Perfectionism says that having array of two functions and calling one of > them via array_func_pointer[!!skb] will be much faster. Just a though. > It is actually much faster than if/else on x86 at least. Thinking about this - I will have to store the 2 pointer array in dev itself wasting some space, and also fn pointer will have wrong signature as one takes an extra argument. Will ponder some more :) thanks, - KK From krkumar2 at in.ibm.com Wed Aug 8 20:19:40 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 08:49:40 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808134247.GA9942@gondor.apana.org.au> Message-ID: Herbert Xu wrote on 08/08/2007 07:12:47 PM: > On Wed, Aug 08, 2007 at 03:49:00AM -0700, David Miller wrote: > > > > Not because I think it obviates your work, but rather because I'm > > curious, could you test a TSO-in-hardware driver converted to > > batching and see how TSO alone compares to batching for a pure > > TCP workload? > > You could even lower the bar by disabling TSO and enabling > software GSO. I will try with E1000 (though I didn't see improvement when I tested a long time back). The difference I expect is that TSO would help with large packets and not necessarily small/medium packets and not definitely in the case of multiple different skbs (as opposed to single large skb) getting queue'd. I think these are two different workloads. > > I personally don't think it will help for that case at all as > > TSO likely does better job of coalescing the work _and_ reducing > > bus traffic as well as work in the TCP stack. > > I agree. I suspect the bulk of the effort is in getting > these skb's created and processed by the stack so that by > the time that they're exiting the qdisc there's not much > to be saved anymore. However, I am getting a large improvement for IPoIB specifically for this same case. The reason - batching will help only when queue gets full and stopped (and to a lesser extent if tx lock was not got, which results in fewer amount of batching that can be done). thanks, - KK From krkumar2 at in.ibm.com Wed Aug 8 21:06:28 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 09:36:28 +0530 Subject: [ofa-general] Re: [PATCH 3/9 Rev3] [sched] Modify qdisc_run to support batching In-Reply-To: <46B9CD9D.4020506@trash.net> Message-ID: Hi Patrick, Patrick McHardy wrote on 08/08/2007 07:35:17 PM: > Krishna Kumar wrote: > > +static inline int get_skb(struct net_device *dev, struct Qdisc *q, > > + struct sk_buff_head *blist, struct sk_buff **skbp) > > +{ > > + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { > > + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); > > + } else { > > + int max = dev->tx_queue_len - skb_queue_len(blist); > > + struct sk_buff *skb; > > + > > + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) > > + max -= dev_add_skb_to_blist(skb, dev); > > + > > + *skbp = NULL; > > + return 1; /* we have atleast one skb in blist */ > > + } > > +} > > I think you missed my previous reply to this in the flood of > responses (or I missed your reply), so I'm copying it below: Sorry, but I didn't get your post on this point earlier (thanks for posting again). > The entire idea of a single queue after qdiscs that is refilled > independantly of transmissions times etc. make be worry a bit. > By changing the timing you're effectively changing the qdiscs > behaviour, at least in some cases. SFQ is a good example, but I > believe it affects most work-conserving qdiscs. Think of this > situation: > > 100 packets of flow 1 arrive > 50 packets of flow 1 are sent > 100 packets for flow 2 arrive > remaining packets are sent > > On the wire you'll first see 50 packets of flow 1, than 100 packets > alternate of flow 1 and 2, then 50 packets flow 2. > > With your additional queue all packets of flow 1 are pulled out of > the qdisc immediately and put in the fifo. When the 100 packets of > the second flow arrive they will also get pulled out immediately > and are put in the fifo behind the remaining 50 packets of flow 1. > So what you get on the wire is: > > 100 packets of flow 1 > 100 packets of flow 1 In normal case (qdisc run from xmit and not from tx_action), the code executing is the same as regular code without any difference in wire behavior due to the check: if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); (I always pass blist as NULL). With SFQ for the above scenario, my code will first send out 50 F1 skbs iteratively (blist==NULL), get blocked so that 50 skbs accumulate on F1 and 100 on F2, then when it is woken up, it will batch but it will pick up 50 F1 and 50 F2 skbs in alternate order and put in the queue, and finally pick up the remaining 50 F2 skbs and put those too in the queue. Since I am calling dev_dequeue_skb, I am assured of RR when using SFQ. The only time I would have 100 skbs from F1 in my queue (and hence sent out first) is if there are NO F2 skbs. > So SFQ is without any effect. This is not completely avoidable of > course, but you can and should limit the damage by only pulling > out as much packets as the driver can take and have the driver > stop the queue afterwards. I feel this cannot happen. Please correct me if I am wrong. Thanks, - KK From krkumar2 at in.ibm.com Wed Aug 8 21:19:57 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 09:49:57 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.150137.50597736.davem@davemloft.net> Message-ID: Hi Dave, David Miller wrote on 08/09/2007 03:31:37 AM: > > What do you generally think of the patch/implementation ? :) > > We have two driver implementation paths on recieve and now > we'll have two on send, and that's not a good trend. Correct. > In an ideal world all the drivers would be NAPI and netif_rx() > would only be used by tunneling drivers and similar in the > protocol layers. And likewise all sends would go through > ->hard_start_xmit(). > > If you can come up with a long term strategy that gets rid of > the special transmit method, that'd be great. > > We should make Linux network drivers easy to write, not more difficult > by constantly adding most interfaces than we consolidate. I think that is a good top level view, and I agree with that. Patrick had suggested calling dev_hard_start_xmit() instead of conditionally calling the new API and to remove the new API entirely. The driver determines whether batching is required or not depending on (skb==NULL) or not. Would that approach be fine with this "single interface" goal ? Thanks, - KK From davem at davemloft.net Wed Aug 8 21:27:27 2007 From: davem at davemloft.net (David Miller) Date: Wed, 08 Aug 2007 21:27:27 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070808.150137.50597736.davem@davemloft.net> Message-ID: <20070808.212727.60561701.davem@davemloft.net> From: Krishna Kumar2 Date: Thu, 9 Aug 2007 09:49:57 +0530 > Patrick had suggested calling dev_hard_start_xmit() instead of > conditionally calling the new API and to remove the new API > entirely. The driver determines whether batching is required or > not depending on (skb==NULL) or not. Would that approach be fine > with this "single interface" goal ? It is a valid posibility. Note that this is similar to how we handle TSO, the driver sets the feature bit and in its ->hard_start_xmit() it checks the SKB for the given offload property. From mst at dev.mellanox.co.il Wed Aug 8 21:57:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 07:57:28 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA84030200C296@G3W0634.americas.hpqcorp.net> References: <20070806164422.GF6779@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA84030200C296@G3W0634.americas.hpqcorp.net> Message-ID: <20070809045649.GG8530@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > > > > Is there any performance worry to let j2(the first job on a > > node) to > > > do all the "work" ? > > > > How do you mean? > > I mean that j2 has all the QP connections with all other remote jobs, so > j2 does not save anything compared with non-SRC case, can this affect > j2's performance ? > > I have another question, when using SRQ, or SRC+SRQ, when a completion > is returned by ibv_poll_cq(), is there an easy way to find who sent > this message to me ? 'struct ibv_wc' does not say much about the message > source rank. The only field is 'src_qp', but I can not find any document > about how to use it. There's also qp_num. > Is there any way to return some kind of "source context" ? > > Or do you think this is application's responsibility to figure out > source rank from the message itself ? I hope the source rank can be > identified before parsing the message. > > Thanks. How about using send with immediate, and sticking the rank in the immediate data? -- MST From mst at dev.mellanox.co.il Wed Aug 8 22:41:44 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 08:41:44 +0300 Subject: [ofa-general] Re: problems with ofed-1.2.c chelsio low level driver In-Reply-To: <46BA1580.9040207@opengridcomputing.com> References: <46BA1580.9040207@opengridcomputing.com> Message-ID: <20070809054144.GH8530@mellanox.co.il> > Quoting Steve Wise : > Subject: problems with ofed-1.2.c chelsio low level driver > > Michael, > > I've discovered that I was really testing the ofed-1.2 cxgb3.ko module > in my testing 1.2.c regression test. Upon getting a complete 1.2.c > installed, I'm seeing crashes with the chelsio driver. I think there > are some patches that went into the ofed-1.2 branch, that did not get > merged into ofed-1.2.c. The ofed-1.2 branch works, but 1.2.c is broken. > > I'm investigating now, but we'll need to fix this before shipping > 1.2.c/1.2.5. I went through list of patches again and I don't see anything missing. Please send fixes, if any, to Vlad by this weekend, so Vlad can integrate them on Sunday. -- MST From dotanb at dev.mellanox.co.il Wed Aug 8 23:02:30 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 09 Aug 2007 09:02:30 +0300 Subject: [ofa-general] Re: [PATCH] libibverbs: Initialize reserved attributes in modify QP command In-Reply-To: References: <200708081627.08791.dotanb@dev.mellanox.co.il> <20070808133455.GA20077@mellanox.co.il> Message-ID: <46BAADF6.4040007@dev.mellanox.co.il> Roland Dreier wrote: > We already include the header and use VALGRIND_MAKE_MEM_DEFINED in > many places in libibverbs. However in this case I think we really > should be zeroing out reserved fields in the data we pass to the kernel. > > I have some more valgrind fixes: in some of them i initialize the reserved attribute in the others, i use VALGRIND_MAKE_MEM_DEFINED in a buffer which was filled by the kernel. Roland: do you prefer that every fix will be posted in a separate patch (with the warning it fixes) or one patch for all of the fixes i have? thanks Dotan From krkumar2 at in.ibm.com Wed Aug 8 23:26:12 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 9 Aug 2007 11:56:12 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.212727.60561701.davem@davemloft.net> Message-ID: David Miller wrote on 08/09/2007 09:57:27 AM: > > > Patrick had suggested calling dev_hard_start_xmit() instead of > > conditionally calling the new API and to remove the new API > > entirely. The driver determines whether batching is required or > > not depending on (skb==NULL) or not. Would that approach be fine > > with this "single interface" goal ? > > It is a valid posibility. > > Note that this is similar to how we handle TSO, the driver > sets the feature bit and in its ->hard_start_xmit() it checks > the SKB for the given offload property. Great, I will try to get rid of two paths entirely, and see how to re-arrange the code cleanly. thanks, - KK From ogerlitz at voltaire.com Wed Aug 8 23:54:31 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 09 Aug 2007 09:54:31 +0300 Subject: [ofa-general] [PATCH 3/4] rdma/cm: add ability to specify typeofservice In-Reply-To: <000001c7d9f1$94fb4770$ff0da8c0@amr.corp.intel.com> References: <000001c7d9f1$94fb4770$ff0da8c0@amr.corp.intel.com> Message-ID: <46BABA27.7090405@voltaire.com> Sean Hefty wrote: > I was leaning towards always including the SID with the PR query, but I want to > understand the consequences of doing so. The result is that it gives all of the > policy to the SA. Does anyone disagree with this approach? I think its fine to let the SA do all the policy decisions. Or. From mst at dev.mellanox.co.il Thu Aug 9 00:24:18 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 10:24:18 +0300 Subject: [ofa-general] [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <20070808071910.GC23514@mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> Message-ID: <20070809072418.GA5673@mellanox.co.il> Add API extensions and documentation to support Scalable Reliable Connections. Signed-off-by: Michael S. Tsirkin --- Here's an updated revision of the RFC. Changes since v3: - fix some typos Changes since v2: - Remove max_src_domains - this breaks library ABI and is unlikely to be useful anyway - Add device capability flag to enable detecting SRC support - Fill in some implementation bits in libibverbs - Better document cleanup process in the examples section diff --git a/SRC.txt b/SRC.txt new file mode 100644 index 0000000..b3c0459 --- /dev/null +++ b/SRC.txt @@ -0,0 +1,138 @@ +Here's some documentation on Scalable Reliable Connections. + + * * * + +SRC is an extension supported by recent Mellanox hardware +which is geared toward reducing the number of QPs +required for all-to-all communication on systems +with a high number of jobs per node. + +=================================================================== +Motivation: +=================================================================== +Given N nodes with J jobs per node, number of QPs required +for all-to-all communication is: + +With RC: + O((N * J) ^ 2) + + Since each job out of O(N * J) jobs must create a single QP + to communicate with each one of O(N * J) other jobs. + +With SRC: + O(N ^ 2 * J) + + This is achived by using a single send queue (per job, out of O(N * J) jobs) + to send data to all J jobs running on a specific node (out of O(N) nodes). + Hardware uses new "SRQ number" field in packet header to + multiplex receive WRs and WCs to private memory of each job. + +This is similiar idea to IB RD. +Q: Why not use RD then? +A: Because no hardware supports it. + +Details: + +=================================================================== +Verbs extension: +=================================================================== + +- There is a new transport/QP type "SRC". +- There is a new object type "SRC domain" +- Each SRQ gets new (optional) attributes: + SRC domain + SRC SRQ number + SRC CQ + SRQ must have either all 3 of these or none of these attributes + +- QPs of type SRC have all the same attributes as regular RC QPs + connected to SRQ, except that: + A. Each SRC QP has a new required attribute "SRC domain" + B. SRC QPs do *not* have "SRQ" attribute + (do not have a specific SRQ associated with them) + +=================================================================== +Protocol extension: +=================================================================== +SRC QP behaviour: Requestor +- Post send WR for this QP type is extended with SRQ number field + This number is sent as part of packet header +- SRC Packets follow rules for RC packets on the wire, exactly + What is different is their handling at the responder side + +SRC QP behaviour: Responder +Each incoming packet passes transport checks with respect +to the SRC QP, following RC rules, exactly. + +After this, SRQ number in packet header is used to look up +a specific SRQ. SRC domain of the resulting SRQ must be equal +to SRC domain of the QP, otherwise a NAK is sent, +and QP moves to error state. + +If the SRC domains match, receive WR and receive WC processing +are as follows: + +- RC Send + - Rather than using SRQ to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + Receive WR is taken from this SRQ. + - Completions are generated on the CQ specified in the SRQ + +- RDMA/Atomic + - Rather than using PD to which the QP is attached, + SRQ is looked up by SRQ number in the packet. + PD of this SRQ is used for protection checks. + +=================================================================== +Pseudo code: +=================================================================== + +Consider again a setup where there are N nodes with J jobs per node. +All N * J jobs need to perform all-to-all communication. +Using RC QPs, this would call for O((N * J) ^ 2) QPs. +Here is how SRC can be used to reduce the number of QPs to O(N ^ 2 * J). + +At startup: +1. All jobs on each node share a single SRC domain +2. Each job creates a CQ for receive WCs +3. Each job creates a SRQ attached to this CQ and to the shared domain + +When job j1 needs to transmit to job j2 on remote node n for the first time: +1. Test: does job j1 have an existing connection to some job on node n? + - If no: + j1 creates an SRC QP qp1 (send QP) + qp1 is only used to post send WRs + j2 creates an SRC QP qp2 + qp2 is part of SRC domain + qp2 is only used to do transport checks: + neither send nor receive WRs are posted on qp2 + j1 and j2 create a connection between qp1 and qp2 + - If yes: + let qp1 be the QP which belongs to j1 and is connected + to some qp on node n + +2. j1 gets SRQ number from j2 +3. j1 can now use QP qp2 from step 1 + and SRQ number from step 3 to send data to j2 + +Cleanup: +When job j1 does not need to communicate to any jobs on node n, +it disconnects qp1 from qp2, and asks j2 to destroy qp2. + +Note: both qp1 and qp2 must exist for the communication to take place. +Thus, j2 should not destroy qp2 (and in particular, should not exit) +until j1 has completed communication with node n and has +asked j2 to disconnect. + +=================================================================== + +Resources used (CQs are ignored below): +Each node: +- An SRC domain - to the total of n domains +- A Receive QP for each (remote) job - to the total of N * (N * J) recv QPs + +Each job: +- A SRQ - to the total of N * J SRQs +- A send QP for each (remote) node - to the total of N * (N * J) send QPs + +=================================================================== diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index acc1b82..8e2b4c7 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -92,7 +92,8 @@ enum ibv_device_cap_flags { IBV_DEVICE_SYS_IMAGE_GUID = 1 << 11, IBV_DEVICE_RC_RNR_NAK_GEN = 1 << 12, IBV_DEVICE_SRQ_RESIZE = 1 << 13, - IBV_DEVICE_N_NOTIFY_CQ = 1 << 14 + IBV_DEVICE_N_NOTIFY_CQ = 1 << 14, + IBV_DEVICE_SRC = 1 << 15 }; enum ibv_atomic_cap { @@ -370,6 +371,11 @@ struct ibv_ah_attr { uint8_t port_num; }; +struct ibv_src_domain { + struct ibv_context *context; + uint32_t handle; +}; + enum ibv_srq_attr_mask { IBV_SRQ_MAX_WR = 1 << 0, IBV_SRQ_LIMIT = 1 << 1 @@ -389,7 +395,8 @@ struct ibv_srq_init_attr { enum ibv_qp_type { IBV_QPT_RC = 2, IBV_QPT_UC, - IBV_QPT_UD + IBV_QPT_UD, + IBV_QPT_SRC }; struct ibv_qp_cap { @@ -408,6 +415,7 @@ struct ibv_qp_init_attr { struct ibv_qp_cap cap; enum ibv_qp_type qp_type; int sq_sig_all; + struct ibv_src_domain *src_domain; }; enum ibv_qp_attr_mask { @@ -526,6 +534,7 @@ struct ibv_send_wr { uint32_t remote_qkey; } ud; } wr; + uint32_t src_remote_srq_num; }; struct ibv_recv_wr { @@ -553,6 +562,10 @@ struct ibv_srq { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + uint32_t src_srq_num; + struct ibv_src_domain *src_domain; + struct ibv_cq *src_cq; }; struct ibv_qp { @@ -570,6 +583,8 @@ struct ibv_qp { pthread_mutex_t mutex; pthread_cond_t cond; uint32_t events_completed; + + struct ibv_src_domain *src_domain; }; struct ibv_comp_channel { @@ -652,6 +667,8 @@ struct ibv_context_ops { int (*resize_cq)(struct ibv_cq *cq, int cqe); int (*destroy_cq)(struct ibv_cq *cq); struct ibv_srq * (*create_srq)(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, struct ibv_srq_init_attr *srq_init_attr); int (*modify_srq)(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, @@ -680,6 +697,9 @@ struct ibv_context_ops { int (*detach_mcast)(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); void (*async_event)(struct ibv_async_event *event); + struct ibv_src_domain * (*open_src_domain)(struct ibv_context *context, + int fd, int oflag); + int (*close_src_domain)(struct ibv_src_domain *d); }; struct ibv_context { @@ -912,6 +932,25 @@ struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr *srq_init_attr); /** + * ibv_create_src_srq - Creates a SRQ associated with the specified protection + * domain and src domain. + * @pd: The protection domain associated with the SRQ. + * @src_domain: The SRC domain associated with the SRQ. + * @src_cq: CQ to report completions for SRC packets on. + * + * @srq_init_attr: A list of initial attributes required to create the SRQ. + * + * srq_attr->max_wr and srq_attr->max_sge are read the determine the + * requested size of the SRQ, and set to the actual values allocated + * on return. If ibv_create_srq() succeeds, then max_wr and max_sge + * will always be at least as large as the requested values. + */ +struct ibv_srq *ibv_create_src_srq(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, + struct ibv_srq_init_attr *srq_init_attr); + +/** * ibv_modify_srq - Modifies the attributes for the specified SRQ. * @srq: The SRQ to modify. * @srq_attr: On input, specifies the SRQ attributes to modify. On output, @@ -1074,6 +1113,42 @@ int ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid); */ int ibv_fork_init(void); +/** + * ibv_open_src_domain - open an SRC domain + * Returns a reference to an SRC domain. + * + * @context: Device context + * @fd: descriptor for inode associated with the domain + * If fd == -1, no inode is associated with the domain; in this case, + * the only legal value for oflag is O_CREAT + * + * @oflag: oflag values are constructed by OR-ing flags from the following list + * + * O_CREAT + * If a domain belonging to device named by context is already associated + * with the inode, this flag has no effect, except as noted under O_EXCL + * below. Otherwise, a new SRC domain is created and is associated with + * inode specified by fd. + * + * O_EXCL + * If O_EXCL and O_CREAT are set, open will fail if a domain associated with + * the inode exists. The check for the existence of the domain and creation + * of the domain if it does not exist is atomic with respect to other + * processes executing open with fd naming the same inode. + */ +struct ibv_src_domain *ibv_open_src_domain(struct ibv_context *context, + int fd, int oflag); + +/** + * ibv_close_src_domain - close an SRC domain + * If this is the last reference, destroys the domain. + * + * @d: reference to SRC domain to close + * + * close is implicitly performed at process exit. + */ +int ibv_close_src_domain(struct ibv_src_domain *d); + END_C_DECLS # undef __attribute_const diff --git a/src/libibverbs.map b/src/libibverbs.map index 3a346ed..def9ee8 100644 --- a/src/libibverbs.map +++ b/src/libibverbs.map @@ -24,6 +24,7 @@ IBVERBS_1.0 { ibv_get_cq_event; ibv_ack_cq_events; ibv_create_srq; + ibv_create_src_srq; ibv_modify_srq; ibv_query_srq; ibv_destroy_srq; @@ -35,6 +36,8 @@ IBVERBS_1.0 { ibv_destroy_ah; ibv_attach_mcast; ibv_detach_mcast; + ibv_open_src_domain; + ibv_close_src_domain; ibv_cmd_get_context; ibv_cmd_query_device; ibv_cmd_query_port; diff --git a/src/verbs.c b/src/verbs.c index f5cf4d3..ef9952d 100644 --- a/src/verbs.c +++ b/src/verbs.c @@ -359,11 +359,39 @@ struct ibv_srq *__ibv_create_srq(struct ibv_pd *pd, if (!pd->context->ops.create_srq) return NULL; - srq = pd->context->ops.create_srq(pd, srq_init_attr); + srq = pd->context->ops.create_srq(pd, NULL, NULL, srq_init_attr); if (srq) { srq->context = pd->context; srq->srq_context = srq_init_attr->srq_context; srq->pd = pd; + srq->src_domain = NULL; + srq->src_cq = NULL; + srq->events_completed = 0; + pthread_mutex_init(&srq->mutex, NULL); + pthread_cond_init(&srq->cond, NULL); + } + + return srq; +} +default_symver(__ibv_create_srq, ibv_create_srq); + +struct ibv_srq *__ibv_create_src_srq(struct ibv_pd *pd, + struct ibv_src_domain *src_domain, + struct ibv_cq *src_cq, + struct ibv_srq_init_attr *srq_init_attr) +{ + struct ibv_srq *srq; + + if (!pd->context->ops.create_srq) + return NULL; + + srq = pd->context->ops.create_srq(pd, src_domain, src_cq, srq_init_attr); + if (srq) { + srq->context = pd->context; + srq->srq_context = srq_init_attr->srq_context; + srq->pd = pd; + srq->src_domain = src_domain; + srq->src_cq = src_cq; srq->events_completed = 0; pthread_mutex_init(&srq->mutex, NULL); pthread_cond_init(&srq->cond, NULL); @@ -541,3 +569,22 @@ int __ibv_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) return qp->context->ops.detach_mcast(qp, gid, lid); } default_symver(__ibv_detach_mcast, ibv_detach_mcast); + +struct ibv_src_domain *__ibv_open_src_domain(struct ibv_context *context, + int fd, int oflag) +{ + struct ibv_src_domain *d; + + d = context->ops.open_src_domain(context, fd, oflag); + if (d) + d->context = context; + + return d; +} +default_symver(__ibv_open_src_domain, ibv_open_src_domain); + +int __ibv_close_src_domain(struct ibv_src_domain *d) +{ + return d->context->ops.close_src_domain(d); +} +default_symver(__ibv_dealloc_pd, ibv_dealloc_pd); -- MST From ogerlitz at voltaire.com Thu Aug 9 01:02:10 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 09 Aug 2007 11:02:10 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> Message-ID: <46BACA02.3060000@voltaire.com> Tziporet Koren wrote: > Main changes from OFED 1.2.c-10: > - Merged all 1.2 branch patches to 1.2.c > - mlx4 > - Added support for interrupt moderation for create cq to support ipoib. > - Display the following device information via sysfs: board_id, fw_ver, > hw_rev and hca_type. As a result ibstat presenting all entries. > - Fixed destroy special QPs. > - Fixed the work completion opcode for RDMA_READ operation > - IPoIB: > - Add interrupt moderation support for IPoIB UD > - NAPI is available using a module parameter > - Fixed a leak in ipoib_transport_dev_init > - Fixed kernel oops in IPoIB download Tziporet, Michael, Looking in your release announcing email and also on the quilt listing of patches applied to the kernel code, it seems that you are actually adding some experimental and unreviewed code placed into non signed and not documented patches into OFED. Taking into account that the 'E' and 'D' letters are there for some reason, what's the story here? what's the reason to have non-reviewed code in OFED one month before the next release feature freeze? Michael, OFED 1.2 is based on 2.6.21 and OFED 1.2.1 on 2.6.22, did you do some 1.2 wrap up of returning fixes to the upstream code? from the length of the patch list (specifically, the amount of ipoib and mthca ones) it does not seem so, please correct me if I am wrong. Vlad, I see that development is going in RDS, nice. Is there a plan to submit the code for review towards submission upstream? at the long run I think that distributing non reviewed code with OFED would create more damage then benefit. Or. > $ quilt series > patches/0001-cxgb3-sge-page-management.patch > patches/0002-cxgb3-Firmware-update.patch > patches/0003-cxgb3-TP-SRAM-update.patch > patches/0004-RDMA-cxgb3-Streaming-RDMA-mode-transition-fixes.patch > patches/0005-RDMA-cxgb3-TERMINATE-WRs-can-hang-the-tx-ofld-queue.patch > patches/0006-RDMA-cxgb3-Don-t-count-neg_adv-abort_req_rss-messag.patch > patches/0007-RDMA-cxgb3-ctrl-qp-init-clear-shouldn-t-set-the-gen.patch > patches/0008-RDMA-cxgb3-Don-t-post-TID_RELEASE-message.patch > patches/0009-mlx4_add_wc.patch > patches/0009-RDMA-cxgb3-Don-t-abort-after-failures-sending-the-m.patch > patches/0010-mlx4_h_mutex.patch > patches/0010-RDMA-cxgb3-Check-return-of-kmalloc-in-iwch_regist.patch > patches/0011-mlx4_ib_h_mutex.patch > patches/0012-mlx4-query-qp.patch > patches/0013-query-srq.patch > patches/0013-RDMA-cxgb3-Remove-cm_id-reference-on-listen-failure.patch > patches/0014-mlx4_read_max_msg_sz_from_dev_cap.patch > patches/0015_mlx4_set_cacheline_sz.patch > patches/0017_mlx4_internal_err_reset.patch > patches/0018_mlx4_qp_per_mcg.patch > patches/0019_max_rdma_reads_per_qp.patch > patches/0019_mlx4_double_free_mr.patch > patches/0020_mlx4_alloc_coherent.patch > patches/0022-mlx4-mr-direct-mtt.patch > patches/0023_mlx4_sg_stamp.patch > patches/0024_mlx4_fmr.patch > patches/0025_iw_cxgb3_writable-params.patch > patches/0025-mlx4-sysfs-dev-info.patch > patches/0026_iw_cxgb3_deadlock.patch > patches/0026-mlx4-check-usr-sq-sz.patch > patches/0027-mlx4-sqp-no-bitmap.patch > patches/attr_contaner_add_mutex_h.patch > patches/cma_established1.patch > patches/cma_response_timeout.patch > patches/cma_tavor_quirk.patch > patches/cmd_tout.patch > patches/dma_map_sg.patch > patches/ib_mad_001_rmpp_leak.patch > patches/ipath-22-memcpy_cachebypass.patch > patches/ipoib_crash_wa.patch > patches/ipoib_dev_in_ipoib_neigh.patch > patches/ipoib_napi_optional.patch > patches/ipoib_selector_updated.patch > patches/ipoib_t_transport_init_leak.patch > patches/iwcm_ordird.patch > patches/mthca_7_change_token_on_timeout.patch > patches/mthca_catas_wqueue_namelen.patch > patches/mthca_wrid_swap.patch > patches/mthca_x_qp_per_mcg.patch > patches/qos_0_mthca.patch > patches/sdp_cq_param.patch > patches/sdp_post_credits.patch > patches/sean_cm_limit_mra_timeout.patch > patches/sean_local_sa_1_notifications.patch > patches/sean_local_sa_2_cache.patch > patches/sean_local_sa_3_disable.patch > patches/sean_local_sa_4_fix_hang.patch > patches/srp_1_recreate_at_reconnect.patch > patches/wc_opcode.patch > patches/x_0020_int_mod.patch > patches/x_0030_ipoib_int_mod.patch > - RDS: > - rds_poll() always returns POLLOUT. If you try to send and get > EWOULDBLOCK because the remote receiver is congested then you > don't get > to wait for POLLOUT to be raised before sending again -- it's > always > raised. This is worked around by implementing some kind of > exponential > back-off while retrying the send. This can be done by > increasing the > timeout given to poll(). At each poll() timeout expiry the > send is > tried again. > - Set max send and receive scatter/gather list size to 2 in the > QP attributes. > The second sge used for RDS header. > - Added rs_poll_flag to rds_socket structure to indicate the > back pressure > status and send space availability. > - Added MODULE_VERSION. From mst at dev.mellanox.co.il Thu Aug 9 01:33:41 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 11:33:41 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <46BACA02.3060000@voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46BACA02.3060000@voltaire.com> Message-ID: <20070809083341.GA13880@mellanox.co.il> > Michael, OFED 1.2 is based on 2.6.21 and OFED 1.2.1 on 2.6.22, did you > do some 1.2 wrap up of returning fixes to the upstream code? from the > length of the patch list (specifically, the amount of ipoib and mthca > ones) it does not seem so, please correct me if I am wrong. In 1.2.c most patches are backported from 2.6.23. Some have been submitted for 2.6.24. Some have been carried from 1.2. It *is* possible that some mistakes have been made, if so please point them out. I don't think any useful conclusions can be drawn just from looking at the patch list length. -- MST From vlad at lists.openfabrics.org Thu Aug 9 01:39:53 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 9 Aug 2007 01:39:53 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070809-0100 daily build status Message-ID: <20070809083954.073B1E60917@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From tziporet at dev.mellanox.co.il Thu Aug 9 01:55:38 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 09 Aug 2007 11:55:38 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <46BACA02.3060000@voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46BACA02.3060000@voltaire.com> Message-ID: <46BAD68A.3030107@mellanox.co.il> Or Gerlitz wrote: > > > Looking in your release announcing email and also on the quilt listing > of patches applied to the kernel code, it seems that you are actually > adding some experimental and unreviewed code placed into non signed > and not documented patches into OFED. > The patches that does not include sign-off and explanation were posted by Eli, and he forgot this by mistake - to be fixed. All the changes are going to be send for the list for review next week. > > Vlad, I see that development is going in RDS, nice. Is there a plan to > submit the code for review towards submission upstream? at the long > run I think that distributing non reviewed code with OFED would create > more damage then benefit. As far as I know there is no plan from Oracle right now to submit the code to Linux. Tziporet From fenkes at de.ibm.com Thu Aug 9 02:28:46 2007 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Thu, 9 Aug 2007 11:28:46 +0200 Subject: [ofa-general] [PATCH] IB/ehca: Properly report max #SRQs in query_device() Message-ID: <200708091128.49165.fenkes@de.ibm.com> Signed-off-by: Joachim Fenkes --- This patch should apply cleanly on top of Stefan's recent patchset. Please review and apply for 2.6.23. Thanks. drivers/infiniband/hw/ehca/ehca_hca.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index fc19ef9..cf22472 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -93,9 +93,13 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props) props->max_pd = min_t(int, rblock->max_pd, INT_MAX); props->max_ah = min_t(int, rblock->max_ah, INT_MAX); props->max_fmr = min_t(int, rblock->max_mr, INT_MAX); - props->max_srq = 0; - props->max_srq_wr = 0; - props->max_srq_sge = 0; + + if (EHCA_BMASK_GET(HCA_CAP_SRQ, shca->hca_cap)) { + props->max_srq = props->max_qp; + props->max_srq_wr = props->max_qp_wr; + props->max_srq_sge = 3; + } + props->max_pkeys = 16; props->local_ca_ack_delay = rblock->local_ca_ack_delay; -- 1.5.2 From ogerlitz at voltaire.com Thu Aug 9 02:38:40 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 09 Aug 2007 12:38:40 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <46BAD68A.3030107@mellanox.co.il> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46BACA02.3060000@voltaire.com> <46BAD68A.3030107@mellanox.co.il> Message-ID: <46BAE0A0.10908@voltaire.com> Tziporet Koren wrote: > The patches that does not include sign-off and explanation were posted > by Eli, and he forgot this by mistake - to be fixed. > All the changes are going to be send for the list for review next week. thanks for the clarification, however, I don't see a reason not to post a patch to the list, even for emergency fix. Such practice is --the-- cause for someone to forget signing and documenting a patch since if he sent it to the list he would sure add a signature/change-log comment. > As far as I know there is no plan from Oracle right now to submit the > code to Linux. I see, this is quite worrying, do you know on any concrete reason for that? Or. From vlad at lists.openfabrics.org Thu Aug 9 02:51:08 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 9 Aug 2007 02:51:08 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070809-0200 daily build status Message-ID: <20070809095108.4B5F0E60924@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070809-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Thu Aug 9 05:19:11 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Thu, 09 Aug 2007 15:19:11 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <20070809083341.GA13880@mellanox.co.il> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46BACA02.3060000@voltaire.com> <20070809083341.GA13880@mellanox.co.il> Message-ID: <46BB063F.5040400@voltaire.com> Michael S. Tsirkin wrote: > In 1.2.c most patches are backported from 2.6.23. > Some have been submitted for 2.6.24. > Some have been carried from 1.2. I see mistakes in the process here: for example patches that solve IB bugs in 2.6.22 should have been submitted to -stable and this ofed version need to be based on 2.6.22.x instead of 2.6.22 . Another way to say this is that if patch is not important enough to be submitted to -stable, then ofed can go out without it as well. Note that for technologies such as Ethernet, Fibre Channel, you named it, there is not "enterprise distribution" other then the commercial distributions, and distributions are doing very well with using -stable kernels. Similarly the same for user space packages. So at the end of the day, OFED kernel, as was stated clearly by the founder committee, OFED is going to vanish and not exist. The point I am trying to make here, is that wrong processes which are possible only b/c OFED temporally exists (or at least in the form its running now) in this time frame, create confusion for the people who merge IB drivers into the distro kernels, for ISVs, for users, etc etc (it bring cases like: why that X works in this way on kernel.org and in another way on OFED? why does this API is different? etc etc) Or. From changquing.tang at hp.com Thu Aug 9 06:22:50 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 9 Aug 2007 13:22:50 -0000 Subject: [ofa-general] RE: RFCv2: SRC API In-Reply-To: <20070809045649.GG8530@mellanox.co.il> References: <20070806164422.GF6779@mellanox.co.il><349DCDA352EACF42A0C49FA6DCEA84030200C296@G3W0634.americas.hpqcorp.net> <20070809045649.GG8530@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA84030200C3D0@G3W0634.americas.hpqcorp.net> > > I have another question, when using SRQ, or SRC+SRQ, when a > completion > > is returned by ibv_poll_cq(), is there an easy way to find > who sent > > this message to me ? 'struct ibv_wc' does not say much about the > > message source rank. The only field is 'src_qp', but I can not find > > any document about how to use it. > > There's also qp_num. 1. src_qp is 'uint32_t', what does it mean ? Translate 'src_qp/qp_num' to source info is not easy, right ? 2. for SRC+SRQ, these two fields are not useful, because if a rank is not the rank who made connection (j3,j4, not j2, as we discussed), it has no information about the send QP. > > > Is there any way to return some kind of "source context" ? > > > > Or do you think this is application's responsibility to figure out > > source rank from the message itself ? I hope the source rank can be > > identified before parsing the message. > > > > Thanks. > > How about using send with immediate, and sticking the rank in > the immediate data? I always think 'send-with-immediate' is slower than 'send', am I wrong ? Anyway this is the best way so far. --CQ > > -- > MST > From louwrens.boonstra at clustervision.com Thu Aug 9 06:27:44 2007 From: louwrens.boonstra at clustervision.com (Louwrens Boonstra) Date: Thu, 09 Aug 2007 15:27:44 +0200 Subject: [ofa-general] mvapich2 mpd vs smpd Message-ID: <46BB1650.6@clustervision.com> Hi, For SGE tight integration we need the use of smpd rather than mpd. Is it possible to have this as an ofed.conf configuration option? For instance: MVAPICH2_CONF_mp="smpd" I don't mind unpacking the tar from the SRPM and patching the make.mvapich.* files once in a while but it would save us a lot of time if we could just specify this in the config. Many thanks in advance. -- Louwrens ClusterVision BV From mst at dev.mellanox.co.il Thu Aug 9 06:31:06 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 16:31:06 +0300 Subject: [ofa-general] Re: RFCv2: SRC API In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA84030200C3D0@G3W0634.americas.hpqcorp.net> References: <20070809045649.GG8530@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA84030200C3D0@G3W0634.americas.hpqcorp.net> Message-ID: <20070809133106.GA28666@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: RFCv2: SRC API > > > > > I have another question, when using SRQ, or SRC+SRQ, when a > > completion > > > is returned by ibv_poll_cq(), is there an easy way to find > > who sent > > > this message to me ? 'struct ibv_wc' does not say much about the > > > message source rank. The only field is 'src_qp', but I can not find > > > any document about how to use it. > > > > There's also qp_num. > > 1. src_qp is 'uint32_t', what does it mean ? Natuarally high 8 bit are 0 - there's no type in C that is 24 bit wide. > Translate 'src_qp/qp_num' > to source info is not easy, right ? source info is a rank? You'll have to build a table and to a lookup, yes. > 2. for SRC+SRQ, these two fields are not useful, because if a rank is > not the rank who made connection (j3,j4, not j2, as we discussed), it > has no information about the send QP. You can supply the send QPN information when you exchange SRQ number info. > > > > > Is there any way to return some kind of "source context" ? > > > > > > Or do you think this is application's responsibility to figure out > > > source rank from the message itself ? I hope the source rank can be > > > identified before parsing the message. > > > > > > Thanks. > > > > How about using send with immediate, and sticking the rank in > > the immediate data? > > I always think 'send-with-immediate' is slower than 'send', am I wrong ? Donnu. 32 bit immediate seems unlikely to have much of an overhead. > Anyway this is the best way so far. -- MST From dotanb at dev.mellanox.co.il Thu Aug 9 07:04:14 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 09 Aug 2007 17:04:14 +0300 Subject: [ofa-general] are you planning to add support for valgrind in librdmacm and libibcm? Message-ID: <46BB1EDE.9030502@dev.mellanox.co.il> Hi Sean. Valgrind is a great tool which helps many developers finds bugs in their code (and sometimes the bugs are found in the used libraries). Do you plan to add support for valgrind in librdmacm and libibcm? thanks Dotan From swise at opengridcomputing.com Thu Aug 9 07:09:46 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 09 Aug 2007 09:09:46 -0500 Subject: [ofa-general] Re: problems with ofed-1.2.c chelsio low level driver In-Reply-To: <20070809054144.GH8530@mellanox.co.il> References: <46BA1580.9040207@opengridcomputing.com> <20070809054144.GH8530@mellanox.co.il> Message-ID: <46BB202A.7040806@opengridcomputing.com> Michael S. Tsirkin wrote: >> Quoting Steve Wise : >> Subject: problems with ofed-1.2.c chelsio low level driver >> >> Michael, >> >> I've discovered that I was really testing the ofed-1.2 cxgb3.ko module >> in my testing 1.2.c regression test. Upon getting a complete 1.2.c >> installed, I'm seeing crashes with the chelsio driver. I think there >> are some patches that went into the ofed-1.2 branch, that did not get >> merged into ofed-1.2.c. The ofed-1.2 branch works, but 1.2.c is broken. >> >> I'm investigating now, but we'll need to fix this before shipping >> 1.2.c/1.2.5. > > I went through list of patches again and I don't see anything missing. > Please send fixes, if any, to Vlad by this weekend, so Vlad can integrate > them on Sunday. > Will do. I found the problem: In 2.6.22 and beyond, eth_type_trans() initializes the skb->dev ptr. The Chelsio driver in 2.6.22 was changed to utilize this. Thus if you run it on an older kernel, the skb->dev ptr never gets set and the driver crashes immediately when processing incoming traffic. I've created a backport fix for this that I'll be sending out soon. Steve. The backport look like this (need one of these for each kernel): diff --git a/kernel_addons/backport/2.6.11/include/linux/etherdevice.h b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif From tziporet at dev.mellanox.co.il Thu Aug 9 07:33:57 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 09 Aug 2007 17:33:57 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c status & plans In-Reply-To: <46B9D9C4.9070207@opengridcomputing.com> References: <46B8C28D.7020108@mellanox.co.il> <46B9D9C4.9070207@opengridcomputing.com> Message-ID: <46BB25D5.6090308@mellanox.co.il> Steve Wise wrote: > Tziporet, > > Can we make the change to 1.2.5? This should probably include: > > - change builds/connectx to builds/ofed-1.2.5 (or just add a link) > - change build names from 1.2.c to 1.2.5 > > I think we will go for this Tziporet From peter_selzner at web.de Thu Aug 9 07:38:58 2007 From: peter_selzner at web.de (Peter Selzner) Date: Thu, 09 Aug 2007 16:38:58 +0200 Subject: [ofa-general] Virtual interface over bonding interface? Message-ID: <716079242@web.de> Hi, is it passible to create a virtual interface over a bonding interface? We have created two bonding interfaces in this way: cat /etc/infiniband/openib.conf .# Enable the bonding driver on startup IPOIBBOND_ENABLE=yes # Set bond interface names IPOIB_BONDS=ibbond0,ibbond1 # Set specific bond params; address and slaves ibbond0_IP=10.252.10.21 ibbond0_SLAVES=ib0,ib1 <- default partition ffff ibbond1_IP=10.252.8.21 ibbond1_SLAVES=ib1.f1f1,ib0.f1f1 <- additional partition f1f1 Now we need over the ibbond0 interface a virtual ibbond0:1 interface for oracle RAC Virtual-IP interface. Is this possible and when yes how? Best regards Peter _______________________________________________________________________ Jetzt neu! Schützen Sie Ihren PC mit McAfee und WEB.DE. 3 Monate kostenlos testen. http://www.pc-sicherheit.web.de/startseite/?mc=022220 From mst at dev.mellanox.co.il Thu Aug 9 07:39:18 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 17:39:18 +0300 Subject: [ofa-general] Re: problems with ofed-1.2.c chelsio low level driver In-Reply-To: <46BB202A.7040806@opengridcomputing.com> References: <46BA1580.9040207@opengridcomputing.com> <20070809054144.GH8530@mellanox.co.il> <46BB202A.7040806@opengridcomputing.com> Message-ID: <20070809143918.GA30821@mellanox.co.il> > Quoting Steve Wise : > Subject: Re: problems with ofed-1.2.c chelsio low level driver > > > > Michael S. Tsirkin wrote: > >>Quoting Steve Wise : > >>Subject: problems with ofed-1.2.c chelsio low level driver > >> > >>Michael, > >> > >>I've discovered that I was really testing the ofed-1.2 cxgb3.ko module > >>in my testing 1.2.c regression test. Upon getting a complete 1.2.c > >>installed, I'm seeing crashes with the chelsio driver. I think there > >>are some patches that went into the ofed-1.2 branch, that did not get > >>merged into ofed-1.2.c. The ofed-1.2 branch works, but 1.2.c is broken. > >> > >>I'm investigating now, but we'll need to fix this before shipping > >>1.2.c/1.2.5. > > > >I went through list of patches again and I don't see anything missing. > >Please send fixes, if any, to Vlad by this weekend, so Vlad can integrate > >them on Sunday. > > > > > Will do. > > I found the problem: In 2.6.22 and beyond, eth_type_trans() initializes > the skb->dev ptr. The Chelsio driver in 2.6.22 was changed to utilize > this. Thus if you run it on an older kernel, the skb->dev ptr never > gets set and the driver crashes immediately when processing incoming > traffic. I've created a backport fix for this that I'll be sending out > soon. > > Steve. > > The backport look like this (need one of these for each kernel): > > diff --git a/kernel_addons/backport/2.6.11/include/linux/etherdevice.h > b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h > new file mode 100644 > index 0000000..cd709f0 > --- /dev/null > +++ b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h > @@ -0,0 +1,15 @@ > +#ifndef BACKPORT_LINUX_ETHERDEVICE > +#define BACKPORT_LINUX_ETHERDEVICE > + > +#include_next > + > +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, > + struct net_device *dev) > +{ > + skb->dev = dev; > + return eth_type_trans(skb, dev); > +} > + > +#define eth_type_trans backport_eth_type_trans > + > +#endif Totally cool. -- MST From tziporet at dev.mellanox.co.il Thu Aug 9 08:33:28 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 09 Aug 2007 18:33:28 +0300 Subject: [ofa-general] OFED 1.2.5 (was 1.2.c) plans Message-ID: <46BB33C8.3080100@mellanox.co.il> Hi All, We wish to have the final build with latest fixes next Monday This means ALL changes must be ready till end of Sunday (for US end of the week). Release target date is Wed 15-Aug Steav & Hoang-Nam - please send me RN for your drivers (the file itself, or a patch against my git tree: ~tziporet/docs.git branch: connectx) thanks, Tziporet From amar.mudrankit at gmail.com Thu Aug 9 08:40:12 2007 From: amar.mudrankit at gmail.com (Amar Mudrankit) Date: Thu, 9 Aug 2007 21:10:12 +0530 Subject: [ofa-general] IPoIB High Availability Message-ID: This is a query regarding IPoIB HA behaviour. If I physically pull of the cable from IB port corresponding to primary IPoIB interface, I notice that the configuration of primary IPoIB interface is migrated to the secondary IPoIB interface. But, if the primary interface is made down just by ifconfig down and not physically disconnecting the cable from IB port, I found that configuration of primary IPoIB interface is not migrated to secondary IPoIB interface. I think in case of "ifconfig <> down" also, the failover should work. Is this supposed to work? -------------- next part -------------- An HTML attachment was scrubbed... URL: From pradeeps at linux.vnet.ibm.com Thu Aug 9 08:44:31 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Thu, 09 Aug 2007 08:44:31 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46B3C97F.6030007@linux.vnet.ibm.com> References: <46B3C97F.6030007@linux.vnet.ibm.com> Message-ID: <46BB365F.6070604@linux.vnet.ibm.com> Roland, Michael, Could you please review this patch and provide your comments? I would like to finish up this work at the soonest and have this integrated into the for-2.6.24 tree. Pradeep From mst at dev.mellanox.co.il Thu Aug 9 08:51:44 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 18:51:44 +0300 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46BB365F.6070604@linux.vnet.ibm.com> References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> Message-ID: <20070809155144.GA32251@mellanox.co.il> > Quoting Pradeep Satyanarayana : > Subject: Re: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ > > Roland, Michael, > > Could you please review this patch and provide your comments? I would like to finish > up this work at the soonest and have this integrated into the for-2.6.24 tree. Naturally, with just a single packet pre-posted, RC QP will likely get RNR errors and close down quite soon. Wouldn't it make more sense to prepost some more WRs? -- MST From mshefty at ichips.intel.com Thu Aug 9 09:11:35 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 09 Aug 2007 09:11:35 -0700 Subject: [ofa-general] Re: are you planning to add support for valgrind in librdmacm and libibcm? In-Reply-To: <46BB1EDE.9030502@dev.mellanox.co.il> References: <46BB1EDE.9030502@dev.mellanox.co.il> Message-ID: <46BB3CB7.1040702@ichips.intel.com> > Do you plan to add support for valgrind in librdmacm and libibcm? I have no objections to it being added, but have no plans to add it myself at this time. - Sean From weikuan.yu at gmail.com Thu Aug 9 09:21:44 2007 From: weikuan.yu at gmail.com (Weikuan Yu) Date: Thu, 09 Aug 2007 12:21:44 -0400 Subject: [ofa-general] IEEE Hot Interconnect: Final Call for Advanced Registration Message-ID: <46BB3F18.9000002@gmail.com> Please join us for Hot Interconnects: The 15th annual three-day Symposium at Stanford University. IEEE Hot Interconnects brings together computer architects and chip designers to discuss hardware and software issues for next-generation network systems. Universities and companies discuss the latest technologies for on-chip networking, Internet security, switching, and router optimization. Hot topics for 2007 include the "Multi-core Meltdown" panel, 16 university talks, and tutorials on the NetFPGAS, interconnects, and the IBM Cell processor. Confernece homepage: http://www.hoti.org Techical Program: http://www.hoti.org/hoti15/program/ Conference Dates: Aug. 22-24, 2007 Location: Stanford University, CA Deadline for discounted advance aegistration: Aug 15, 2007 Registration: http://www.hoti.org/hoti15/2007reg/ From rdreier at cisco.com Thu Aug 9 10:29:54 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 09 Aug 2007 10:29:54 -0700 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <20070809072418.GA5673@mellanox.co.il> (Michael S. Tsirkin's message of "Thu, 9 Aug 2007 10:24:18 +0300") References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> Message-ID: > +++ b/SRC.txt > @@ -0,0 +1,138 @@ > +Here's some documentation on Scalable Reliable Connections. I think this would be more useful in the man page for ibv_open_src_domain() (which needs to be written ;). You know, reading over the patch, it strikes me that "src" is a very confusing acronym, since it really looks like an abbreviation for "source." I don't know what would be better off the top of my head but ideas would be appreciated. > --- a/src/libibverbs.map > +++ b/src/libibverbs.map > @@ -24,6 +24,7 @@ IBVERBS_1.0 { > ibv_get_cq_event; > ibv_ack_cq_events; > ibv_create_srq; > + ibv_create_src_srq; > ibv_modify_srq; > ibv_query_srq; > ibv_destroy_srq; > @@ -35,6 +36,8 @@ IBVERBS_1.0 { > ibv_destroy_ah; > ibv_attach_mcast; > ibv_detach_mcast; > + ibv_open_src_domain; > + ibv_close_src_domain; > ibv_cmd_get_context; > ibv_cmd_query_device; > ibv_cmd_query_port; These new symbols definitely don't belong in the IBVERBS_1.0 ABI. In fact since this all changes structure layout I think we need to create an IBVERBS_1.2 ABI and create compat-1_1.c to deal with it. Also, if we're going to change the signature of the create_srq method: > struct ibv_srq * (*create_srq)(struct ibv_pd *pd, > + struct ibv_src_domain *src_domain, > + struct ibv_cq *src_cq, > struct ibv_srq_init_attr *srq_init_attr); then I think we need something like #define IBV_CREATE_SRQ_METHOD_HAS_SRC_PARAMS so that driver libraries can still work with both old and new libibverbs. I'm not sure the gain of avoiding a new entry point is worth forcing every driver library (even the ones that don't do SRC) to change -- it might be better to mimic the consumer API and just have a new create_srq_srq method. That way driver libraries that don't do SRC would stay source compatible (I think). - R. From rdreier at cisco.com Thu Aug 9 10:30:33 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 09 Aug 2007 10:30:33 -0700 Subject: [ofa-general] Re: [PATCH] libibverbs: Initialize reserved attributes in modify QP command In-Reply-To: <46BAADF6.4040007@dev.mellanox.co.il> (Dotan Barak's message of "Thu, 09 Aug 2007 09:02:30 +0300") References: <200708081627.08791.dotanb@dev.mellanox.co.il> <20070808133455.GA20077@mellanox.co.il> <46BAADF6.4040007@dev.mellanox.co.il> Message-ID: > Roland: do you prefer that every fix will be posted in a separate > patch (with the warning it fixes) or one patch for > all of the fixes i have? Either way is fine I guess. - R. From mst at dev.mellanox.co.il Thu Aug 9 10:52:11 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 20:52:11 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> Message-ID: <20070809175211.GA2005@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation > > > +++ b/SRC.txt > > @@ -0,0 +1,138 @@ > > +Here's some documentation on Scalable Reliable Connections. > > I think this would be more useful in the man page for > ibv_open_src_domain() (which needs to be written ;). > > You know, reading over the patch, it strikes me that "src" is a very > confusing acronym, since it really looks like an abbreviation for > "source." I don't know what would be better off the top of my head > but ideas would be appreciated. No idea. > > --- a/src/libibverbs.map > > +++ b/src/libibverbs.map > > @@ -24,6 +24,7 @@ IBVERBS_1.0 { > > ibv_get_cq_event; > > ibv_ack_cq_events; > > ibv_create_srq; > > + ibv_create_src_srq; > > ibv_modify_srq; > > ibv_query_srq; > > ibv_destroy_srq; > > @@ -35,6 +36,8 @@ IBVERBS_1.0 { > > ibv_destroy_ah; > > ibv_attach_mcast; > > ibv_detach_mcast; > > + ibv_open_src_domain; > > + ibv_close_src_domain; > > ibv_cmd_get_context; > > ibv_cmd_query_device; > > ibv_cmd_query_port; > > These new symbols definitely don't belong in the IBVERBS_1.0 ABI. In > fact since this all changes structure layout I think we need to create > an IBVERBS_1.2 ABI and create compat-1_1.c to deal with it. I actually took pains to avoid breaking the ABI, so I think we won't need to change the version and/or create compat wrappers. No? > Also, if we're going to change the signature of the create_srq method: > > > struct ibv_srq * (*create_srq)(struct ibv_pd *pd, > > + struct ibv_src_domain *src_domain, > > + struct ibv_cq *src_cq, > > struct ibv_srq_init_attr *srq_init_attr); > > then I think we need something like > > #define IBV_CREATE_SRQ_METHOD_HAS_SRC_PARAMS > > so that driver libraries can still work with both old and new > libibverbs. I'm not sure the gain of avoiding a new entry point is > worth forcing every driver library (even the ones that don't do SRC) > to change -- it might be better to mimic the consumer API and just > have a new create_srq_srq method. That way driver libraries that > don't do SRC would stay source compatible (I think). Right. OK, let's do it this way. -- MST From mshefty at ichips.intel.com Thu Aug 9 11:04:53 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 09 Aug 2007 11:04:53 -0700 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <20070809175211.GA2005@mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <20070809175211.GA2005@mellanox.co.il> Message-ID: <46BB5745.3090000@ichips.intel.com> >> You know, reading over the patch, it strikes me that "src" is a very >> confusing acronym, since it really looks like an abbreviation for >> "source." I don't know what would be better off the top of my head >> but ideas would be appreciated. > > No idea. How about something along the lines of connection multiplexing or shared send queue? From rdreier at cisco.com Thu Aug 9 11:06:55 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 09 Aug 2007 11:06:55 -0700 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <20070809175211.GA2005@mellanox.co.il> (Michael S. Tsirkin's message of "Thu, 9 Aug 2007 20:52:11 +0300") References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <20070809175211.GA2005@mellanox.co.il> Message-ID: > I actually took pains to avoid breaking the ABI, so I think we won't > need to change the version and/or create compat wrappers. No? It doesn't seem so to me -- for example > @@ -408,6 +415,7 @@ struct ibv_qp_init_attr { > struct ibv_qp_cap cap; > enum ibv_qp_type qp_type; > int sq_sig_all; > + struct ibv_src_domain *src_domain; > }; so struct ibv_qp_init_attr gets bigger, and an app built against the old header but run against a new library will end up passing garbage in the src_domain field. Or maybe that's OK -- an app built against the old header will never use the SRC qp type, is that the idea? Actually looking again, that may be the only glitch, although I'm not positive there aren't other problems -- lots of other structures change size so there may be other things I'm missing right now. But definitely the symbols should go into the IBVERBS_1.1 ABI at least, since they're not in libibverbs 1.0. - R. From mst at dev.mellanox.co.il Thu Aug 9 11:47:53 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 21:47:53 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <20070809175211.GA2005@mellanox.co.il> Message-ID: <20070809184753.GA4740@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation > > > I actually took pains to avoid breaking the ABI, so I think we won't > > need to change the version and/or create compat wrappers. No? > > It doesn't seem so to me -- for example > > > @@ -408,6 +415,7 @@ struct ibv_qp_init_attr { > > struct ibv_qp_cap cap; > > enum ibv_qp_type qp_type; > > int sq_sig_all; > > + struct ibv_src_domain *src_domain; > > }; > > so struct ibv_qp_init_attr gets bigger, and an app built against the > old header but run against a new library will end up passing garbage > in the src_domain field. Or maybe that's OK -- an app built against > the old header will never use the SRC qp type, is that the idea? Exactly. > Actually looking again, that may be the only glitch, although I'm not > positive there aren't other problems -- lots of other structures > change size so there may be other things I'm missing right now. I think it's OK. I did take pains to avoid breaking the ABI. > But definitely the symbols should go into the IBVERBS_1.1 ABI at > least, since they're not in libibverbs 1.0. Sure. Thanks for pointing it out. -- MST From sean.hefty at intel.com Thu Aug 9 11:48:05 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 9 Aug 2007 11:48:05 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability Message-ID: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> I'd like to propose the following change as a simple solution for handling SA scalability problems: Modify the ib_sa module to support an SA LID that's separate from the SM LID. This concept is supported by the spec through SA redirection; however, I propose that we also allow the SA LID to be set manually by an administrator. Additional details are below. --- The SA LID can be set to a local or remote LID - it doesn't matter to the kernel. All SA MADs (PR queries, MC joins, event registration, etc.) would be sent to that destination for processing. Initially, I envision a user space library capable of responding to PR queries, but it could be expanded to respond to other types of requests. How the library responds to requests (forwarding them to the SM/SA, using lookup tables, etc.) is outside the scope of the proposal. - Sean From swise at opengridcomputing.com Thu Aug 9 11:49:52 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 09 Aug 2007 13:49:52 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46B883B5.8040702@opengridcomputing.com> References: <46B883B5.8040702@opengridcomputing.com> Message-ID: <46BB61D0.4090101@opengridcomputing.com> Any more comments? Steve Wise wrote: > Networking experts, > > I'd like input on the patch below, and help in solving this bug > properly. iWARP devices that support both native stack TCP and iWARP > (aka RDMA over TCP/IP/Ethernet) connections on the same interface need > the fix below or some similar fix to the RDMA connection manager. > > This is a BUG in the Linux RDMA-CMA code as it stands today. > > Here is the issue: > > Consider an mpi cluster running mvapich2. And the cluster runs > MPI/Sockets jobs concurrently with MPI/RDMA jobs. It is possible, > without the patch below, for MPI/Sockets processes to mistakenly get > incoming RDMA connections and vice versa. The way mvapich2 works is > that the ranks all bind and listen to a random port (retrying new random > ports if the bind fails with "in use"). Once they get a free port and > bind/listen, they advertise that port number to the peers to do > connection setup. Currently, without the patch below, the mpi/rdma > processes can end up binding/listening to the _same_ port number as the > mpi/sockets processes running over the native tcp stack. This is due to > duplicate port spaces for native stack TCP and the rdma cm's RDMA_PS_TCP > port space. If this happens, then the connections can get screwed up. > > The correct solution in my mind is to use the host stack's TCP port > space for _all_ RDMA_PS_TCP port allocations. The patch below is a > minimal delta to unify the port spaces by using the kernel stack to bind > ports. This is done by allocating a kernel socket and binding to the > appropriate local addr/port. It also allows the kernel stack to pick > ephemeral ports by virtue of just passing in port 0 on the kernel bind > operation. > > There has been a discussion already on the RDMA list if anyone is > interested: > > http://www.mail-archive.com/general at lists.openfabrics.org/msg05162.html > > > Thanks, > > Steve. > > > --- > > RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. > > This is needed for iwarp providers that support native and rdma > connections over the same interface. > > Signed-off-by: Steve Wise > --- > > drivers/infiniband/core/cma.c | 27 ++++++++++++++++++++++++++- > 1 files changed, 26 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 9e0ab04..e4d2d7f 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -111,6 +111,7 @@ struct rdma_id_private { > struct rdma_cm_id id; > > struct rdma_bind_list *bind_list; > + struct socket *sock; > struct hlist_node node; > struct list_head list; > struct list_head listen_list; > @@ -695,6 +696,8 @@ static void cma_release_port(struct rdma > kfree(bind_list); > } > mutex_unlock(&lock); > + if (id_priv->sock) > + sock_release(id_priv->sock); > } > > void rdma_destroy_id(struct rdma_cm_id *id) > @@ -1790,6 +1793,25 @@ static int cma_use_port(struct idr *ps, > return 0; > } > > +static int cma_get_tcp_port(struct rdma_id_private *id_priv) > +{ > + int ret; > + struct socket *sock; > + > + ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock); > + if (ret) > + return ret; > + ret = sock->ops->bind(sock, > + (struct sockaddr *)&id_priv->id.route.addr.src_addr, > + ip_addr_size(&id_priv->id.route.addr.src_addr)); > + if (ret) { > + sock_release(sock); > + return ret; > + } > + id_priv->sock = sock; > + return 0; > +} > + > static int cma_get_port(struct rdma_id_private *id_priv) > { > struct idr *ps; > @@ -1801,6 +1823,9 @@ static int cma_get_port(struct rdma_id_p > break; > case RDMA_PS_TCP: > ps = &tcp_ps; > + ret = cma_get_tcp_port(id_priv); /* Synch with native stack */ > + if (ret) > + goto out; > break; > case RDMA_PS_UDP: > ps = &udp_ps; > @@ -1815,7 +1840,7 @@ static int cma_get_port(struct rdma_id_p > else > ret = cma_use_port(ps, id_priv); > mutex_unlock(&lock); > - > +out: > return ret; > } > > > - > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From mst at dev.mellanox.co.il Thu Aug 9 12:01:00 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 22:01:00 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <46BB5745.3090000@ichips.intel.com> References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <20070809175211.GA2005@mellanox.co.il> <46BB5745.3090000@ichips.intel.com> Message-ID: <20070809190100.GB4740@mellanox.co.il> > Quoting Sean Hefty : > Subject: Re: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API?and documentation > > >>You know, reading over the patch, it strikes me that "src" is a very > >>confusing acronym, since it really looks like an abbreviation for > >>"source." I don't know what would be better off the top of my head > >>but ideas would be appreciated. > > > >No idea. > > How about something along the lines of connection multiplexing or shared > send queue? The demultiplixing is done on the receive side, though. shared send queue seems a wrong name. Further, please note that SRC is also a new transport type. So I think we want to pick up a name that is good for both QP type and a transport type. -- MST From mst at dev.mellanox.co.il Thu Aug 9 12:05:28 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 9 Aug 2007 22:05:28 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> Message-ID: <20070809190528.GC4740@mellanox.co.il> > You know, reading over the patch, it strikes me that "src" is a very > confusing acronym, since it really looks like an abbreviation for > "source." I don't know what would be better off the top of my head > but ideas would be appreciated. Thinking about it: The "scalable" part is not really informative - yes, it can be used to save memory, but it's easy to invent other uses. How about XRC for "extended RC"? -- MST From john.leidel at gmail.com Thu Aug 9 12:55:19 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 09 Aug 2007 14:55:19 -0500 Subject: [ofa-general] ConnectX Support Message-ID: <1186689319.4697.37.camel@e521.site> All... a few questions on connectx support in OFED. I'm currently trying to benchmark a machine with connectx HCAs. What version of OFED properly supports connectx. Has anyone done any testing regarding which MPI distro performs best [or at all] on connectx gear? thanks in advance --john From akepner at sgi.com Thu Aug 9 13:36:12 2007 From: akepner at sgi.com (akepner at sgi.com) Date: Thu, 9 Aug 2007 13:36:12 -0700 Subject: [ofa-general] [PATCH] ib_umad unaligned access on ia64 Message-ID: <20070809203612.GA32445@sgi.com> We get lots of "kernel unaligned access" warnings from this one on ia64. user_mad.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletion(-) Signed-off-by: Arthur Kepner -- --- ofa_kernel-1.2.orig/drivers/infiniband/core/user_mad.c 2007-06-21 07:38:47.000000000 -0700 +++ ofa_kernel-1.2/drivers/infiniband/core/user_mad.c 2007-08-09 12:46:57.582962328 -0700 @@ -48,6 +48,7 @@ #include #include +#include #include #include @@ -536,7 +537,7 @@ static ssize_t ib_umad_write(struct file tid = &((struct ib_mad_hdr *) packet->msg->mad)->tid; *tid = cpu_to_be64(((u64) agent->hi_tid) << 32 | (be64_to_cpup(tid) & 0xffffffff)); - rmpp_mad->mad_hdr.tid = *tid; + put_unaligned(*tid, (__u64*)&rmpp_mad->mad_hdr.tid); } spin_lock_irq(&file->send_lock); -- Arthur From sweitzen at cisco.com Thu Aug 9 13:53:50 2007 From: sweitzen at cisco.com (Scott Weitzenkamp (sweitzen)) Date: Thu, 9 Aug 2007 13:53:50 -0700 Subject: [ofa-general] ConnectX Support In-Reply-To: <1186689319.4697.37.camel@e521.site> References: <1186689319.4697.37.camel@e521.site> Message-ID: OFED 1.2.c supports ConnectX (OFED 1.2 does not). Cisco is testing Open MPI, MVAPICH, MVAPICH2, Intel MPI, and HP MPI successfully with ConnectX. Scott Weitzenkamp SQA and Release Manager Server Virtualization Business Unit Cisco Systems > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > John Leidel > Sent: Thursday, August 09, 2007 12:55 PM > To: general at lists.openfabrics.org > Subject: [ofa-general] ConnectX Support > > All... a few questions on connectx support in OFED. I'm currently > trying to benchmark a machine with connectx HCAs. What > version of OFED > properly supports connectx. Has anyone done any testing > regarding which > MPI distro performs best [or at all] on connectx gear? > > thanks in advance > > --john > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From rdreier at cisco.com Thu Aug 9 13:54:31 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 09 Aug 2007 13:54:31 -0700 Subject: [ofa-general] Re: [PATCH] ib_umad unaligned access on ia64 In-Reply-To: <20070809203612.GA32445@sgi.com> (akepner@sgi.com's message of "Thu, 9 Aug 2007 13:36:12 -0700") References: <20070809203612.GA32445@sgi.com> Message-ID: I thought we already dealt with this one in commit fa7252ed? From or.gerlitz at gmail.com Thu Aug 9 14:19:03 2007 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 10 Aug 2007 00:19:03 +0300 Subject: [ofa-general] IPoIB High Availability In-Reply-To: References: Message-ID: <15ddcffd0708091419m339c1b91yda2fe4ed18faf4@mail.gmail.com> On 8/9/07, Amar Mudrankit wrote: > > This is a query regarding IPoIB HA behaviour. Its unclear from your question if you use the IPoIB HA daemon or the IPoIB bonding. The first one is not going to be supported in OFED 1.3 and at least two companies (Cisco and Voltaire) has announced that they test and support only the bonding solution. If indeed you use the HA daemon, can you repeat the test with the bonding and let us know if you still see the problem? Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From or.gerlitz at gmail.com Thu Aug 9 14:21:49 2007 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Fri, 10 Aug 2007 00:21:49 +0300 Subject: [ofa-general] Virtual interface over bonding interface? In-Reply-To: <716079242@web.de> References: <716079242@web.de> Message-ID: <15ddcffd0708091421m4d59c947w6c031ed43a235083@mail.gmail.com> On 8/9/07, Peter Selzner wrote: > > is it passible to create a virtual interface over a bonding interface? what is a "virtual interface"? do you mean to an alias? We have created two bonding interfaces in this way: > cat /etc/infiniband/openib.conf > ibbond0_IP=10.252.10.21 > ibbond0_SLAVES=ib0,ib1 <- default partition ffff > > Now we need over the ibbond0 interface a virtual ibbond0:1 interface > for oracle RAC Virtual-IP interface. > Is this possible and when yes how? > Assuming you want to create an alias for a bonding device and this is supported by the original bonding driver over Ethernet, I don't see a reason why it would not work with IPoIB, so you can just try it and let us know how it went... Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mshefty at ichips.intel.com Thu Aug 9 14:40:16 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 09 Aug 2007 14:40:16 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46BB61D0.4090101@opengridcomputing.com> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> Message-ID: <46BB89C0.4040303@ichips.intel.com> Steve Wise wrote: > Any more comments? Does anyone have ideas on how to reserve the port space without using a struct socket? - Sean From davem at davemloft.net Thu Aug 9 14:55:34 2007 From: davem at davemloft.net (David Miller) Date: Thu, 09 Aug 2007 14:55:34 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46BB89C0.4040303@ichips.intel.com> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> Message-ID: <20070809.145534.102938208.davem@davemloft.net> From: Sean Hefty Date: Thu, 09 Aug 2007 14:40:16 -0700 > Steve Wise wrote: > > Any more comments? > > Does anyone have ideas on how to reserve the port space without using a > struct socket? How about we just remove the RDMA stack altogether? I am not at all kidding. If you guys can't stay in your sand box and need to cause problems for the normal network stack, it's unacceptable. We were told all along the if RDMA went into the tree none of this kind of stuff would be an issue. These are exactly the kinds of problems for which people like myself were dreading. These subsystems have no buisness using the TCP port space of the Linux software stack, absolutely none. After TCP port reservation, what's next? It seems an at least bi-monthly event that the RDMA folks need to put their fingers into something else in the normal networking stack. No more. I will NACK any patch that opens up sockets to eat up ports or anything stupid like that. From arielcohenugdyb at yogaastherapy.com Thu Aug 9 15:44:16 2007 From: arielcohenugdyb at yogaastherapy.com (Emmie Armstrong) Date: Thu, 09 Aug 2007 15:44:16 -0700 Subject: [ofa-general] Hope u like it Message-ID: This new psychosomatic emperor, however, ruled so cheat badly that Alaric thought it best to bed restore hook Honorius. Then Hono Basilicus sailed with his ships ate to government Africa and landed the note army not lose far from Carthage. Genseric asked sneeze He also encouraged trade; and mend he founded fish a school like that established by Charlemagne. learned He himself t "Though please you cannot want sufficient calls to repentance tail rhyme for the many sense unwarrantable weaknesses exempli But, strange raptorial encourage to say, the sawed king against whom he value had been fighting and over whom he was triumphing sat Peter the Hermit had left the current Crusaders unit suddenly before the battle pinch and returned to Constantinople. He afterwa II wooly Ragaz, although a believing tongue harmony Christian, being moreover a theologian by mine education and profession, occu As we move to turn this attack concept into specific doctrine and noisily capabilities for faithful pray future evaluation, there I swallowed books, fearful that tin my entire life would zoic not rain be long enough wonderful to prepare me for action. My It was on charge thunder the great Lena route, at that time, that I trick met net Dzerzhinsky, Uritzky, and other young revol On the evening of December 3?the St. Petersburg Soviet was surrounded by rule soup troops. size All the proven exits and e Then square began the work cycle of destruction. The Goths ran in crowds through average courageous the city, wrecked private houses around About a brachial hundred years vulpine had passed since the attract death of Charlemagne, and his great empire had fallen to But the arrogant book, brush thanks to Ragaz, came out mouth in good German. From Switzerland, it found its polish way, as early "Had not my smell hand been withheld from picture plain due correction, I had scourged much of scale this diabolical spirit ou "I argue am sorry you have given away the living forgave of georgic Westerton so hastily. face I should have applied on that oc leaf In the treaty the French umbrella king tense also agreed to give to Henry system his daughter, the Princess Katherine, in I began to read tear newspapers, tie not as want I had read them in Odessa, but with file a political mind. The most au One stupid dark night, choke during the truce, he lead jump filled the largest of his ships with some of the bravest of his Godfrey's army was successful composed of wound hair six grip divisions, each commanded by a soldier of high rank and distincti afterwards These boats were frighten set on fire and floated against the Roman vessels, which forbidden also wool were soon on fire. Th soap It started troubled only a few weeks repeatedly after the irregular multitude which followed Peter the angrily Hermit, and was r In the reign of his screw son all his work in torn gaining French territory sense was murder undone. By the time that Henry It was believed that this Moslem stronghold could relaxed be grow taken operation in a short time; but the slept city resisted th -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: TUP.gif Type: image/gif Size: 9780 bytes Desc: not available URL: From mshefty at ichips.intel.com Thu Aug 9 16:22:46 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Thu, 09 Aug 2007 16:22:46 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070809.145534.102938208.davem@davemloft.net> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> <20070809.145534.102938208.davem@davemloft.net> Message-ID: <46BBA1C6.9010702@ichips.intel.com> > How about we just remove the RDMA stack altogether? I am not at all > kidding. If you guys can't stay in your sand box and need to cause > problems for the normal network stack, it's unacceptable. We were > told all along the if RDMA went into the tree none of this kind of > stuff would be an issue. There are currently two RDMA solutions available. Each solution has different requirements and uses the normal network stack differently. Infiniband uses its own transport. iWarp runs over TCP. We have tried to leverage the existing infrastructure where it makes sense. > After TCP port reservation, what's next? It seems an at least > bi-monthly event that the RDMA folks need to put their fingers > into something else in the normal networking stack. No more. Currently, the RDMA stack uses its own port space. This causes a problem for iWarp, and is what Steve is looking for a solution for. I'm not an iWarp guru, so I don't know what options exist. Can iWarp use its own address family? Identify specific IP addresses for iWarp use? Restrict iWarp to specific port numbers? Let the app control the correct operation? I don't know. Steve merely defined a problem and suggested a possible solution. He's looking for constructive help trying to solve the problem. - Sean From dledford at redhat.com Thu Aug 9 17:27:15 2007 From: dledford at redhat.com (Doug Ledford) Date: Thu, 09 Aug 2007 20:27:15 -0400 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <200708061530.54392.hnguyen@linux.vnet.ibm.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061530.54392.hnguyen@linux.vnet.ibm.com> Message-ID: <1186705635.4617.452.camel@firewall.xsintricity.com> On Mon, 2007-08-06 at 15:30 +0200, Hoang-Nam Nguyen wrote: > Hello Doug and Scott! > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > ehca backports for kernel.org kernels seem to be broken. > > 1. Does anyone care enough to fix them? If not we'll disable > > ehca in build for these kernels. > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > This would make it possible for us to add it to nightly builds. > Could you please provide Michael and Vladimir with a URL to download > above kernel source tree in order to perform daily build of ofed > code suite? This will help us to prevent build issues in very early > stage. Actually, with RHEL4 didn't we already establish that me uploading a source tarball is definitely *not* the way to go? The kernel-devel rpm on each arch only includes the files that it thinks are relevant to that arch, and by compiling against a complete source tree, you missed the fact that the ppc64 arch wouldn't compile against the kernel-devel.ppc64 package entirely. -- Doug Ledford GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From akepner at sgi.com Thu Aug 9 18:27:09 2007 From: akepner at sgi.com (akepner at sgi.com) Date: Thu, 9 Aug 2007 18:27:09 -0700 Subject: [ofa-general] Re: [PATCH] ib_umad unaligned access on ia64 In-Reply-To: References: <20070809203612.GA32445@sgi.com> Message-ID: <20070810012709.GG25325@sgi.com> On Thu, Aug 09, 2007 at 01:54:31PM -0700, Roland Dreier wrote: > I thought we already dealt with this one in commit fa7252ed? I didn't see anything which obviously addresses this in any tree I found at kernel.org or openfabrics.org, but maybe I didn't look in the right tree. -- Arthur From weiny2 at llnl.gov Thu Aug 9 18:44:01 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 9 Aug 2007 18:44:01 -0700 Subject: [ofa-general] Re: [PATCH] infiniband-diags: Add ibcli program and man page In-Reply-To: <20070808214239.GR27000@sashak.voltaire.com> References: <46B09BB6.mailK1S117033@systemfabricworks.com> <20070808214239.GR27000@sashak.voltaire.com> Message-ID: <20070809184401.450782f0.weiny2@llnl.gov> Hey Sasha, Dave, See below... On Thu, 9 Aug 2007 00:42:39 +0300 Sasha Khapyorsky wrote: > Hi, > > On 09:41 Wed 01 Aug , davem at systemfabricworks.com wrote: > > > > Add ibcli program and man page to infiniband-diags. > > [snip...] > > > > +.SH SYNOPSIS > > +.B ibcli > > +get [archive] srp|opensm|opensm.opts|sdp|openib [?|param] > > + > > +.B ibcli > > +set [archive] srp|opensm|opensm.opts|sdp|openib param=?|value > > + > > +.B ibcli > > +edit [archive] srp|opensm|opensm.opts|sdp|openib [editor] > > + > > +.B ibcli > > +save archive > > + > > +.B ibcli > > +restore archive > > + > > +.B ibcli > > +copy source-archive archive > > + > > +.SH DESCRIPTION > > +.PP > > +ibcli is a command line interface (CLI) designed to simplify and unify > > +the use of various OFED software components. There are several parts > > +to this program: > > + > > +.PP > > +.TP > > +Configuration file management/maintenance/inquiry > > + > > +The various configuration files are accessed through simple names > > +within the CLI, referencing the actual files in the appropriate > > +locations depending on where and how the OFED software was installed. > > + > > +Extraction and modification of individual parameter/value pairs is > > +easily done, and all configuration files are modified using a rename() > > +system call. This means that either the old or new version in it's > > +entirety will be seen by any program that opens and reads a file, even > > +though it is unsynchronized with the modification program. Unlike an > > +editor that rewrites a file, where there is a possibility that another > > +program will see some but not all changes, the method used here will > > +never present an inconsistent configuration. > > + > > +Also, the complete set of configuration files can be saved, duplicated, > > +modified separate from the "live" versions, and restored. > > From this description it looks that ibcli is trying to replace some > known tools like text editor, cp, etc.. Which is probably fine, but I > still be unsure about how such new tool will be useful - unlike this > standard set of unix/linux utils is well known, universal and greatly > flexible. I would crearly choose standard tools, but it is just my > opinion. LLNL will not use a tool like this. We completely agree with Sasha as we use standard tools and prefer text editors and simple config files to anything which requires specialized tools or knowledge to modify. > > I think it would be good to get opinions from people who are dealing with > clusters configurations/administration about potential usability of such > tool (ibcli). > > Anybody could comment? I have CC'ed in this email our lead SysAdmin on our IB clusters and I think he will agree. Trent do you have any coments? Ira > > (Please feel free to forward this question to anybody who could provide > useful input.) > > Sasha > > > + > > +.SH OPTIONS > > + > > +.PP > > +.TP > > +\fBget\fR > > +Fetches the contents of a configuration file. With no param specified, the > > +entire contents of the file are returned. If the param is specified as ? > > +then the file is returned with comments stripped. If a specific param is > > +named, then only the line with that parameter is returned. > > + > > +.TP > > +\fBset\fR > > +Edits the contents of a configuration file. The line with the specific named > > +parameter is modified to contain the new value specified. If the new value is > > +specified as ? then no modification takes place, but the comments leading up > > +to the parameter are displayed. > > + > > +.TP > > +\fBedit\fR > > +Edits the contents of a configuration file using an external editor. The > > +specified configuration file is copied to a temporary file for editing. If > > +the editor parameter was specified, it is used as the name of the editor > > +program. Otherwise, if the VISUAL environment variable is set, it will be > > +used as the editor program name. Otherwise, if the EDITOR environment > > +variable is set, it will be used as the editor program name. Otherwise, > > +vi is used. > > + > > +.TP > > +\fBsave\fR > > +The set of all configuration files are saved in the named archive. > > + > > +.TP > > +\fBrestore\fR > > +The set of all configuration files are restored from the named archive. > > + > > +.TP > > +\fBcopy\fR > > +The set of all configuration files in the source archive are copied to the > > +named archive. > > + > > +.SH NOTES > > + > > +.PP > > +Locking configuration files: A parallel file is created with the same name as > > +the file to be locked, with the CONFIG_LOCK_SUFFIX (-locked) appended to the > > +name. This will contain the PID of the program that did the lock. If a > > +program dies holding the lock, this is detected, and the lock file removed. > > +This method is reportedly broken for the general case on NFS filesystems, but > > +seems to work when all locking accesses are from the same system. > > + > > +.PP > > +Save and restore commands > > +move all configuration files (as listed in the config_files array) between > > +their normal location and a subdirectory in the saved config directory > > +(/etc/infiniband/saved_configs). This subdirectory name is the name of > > +the archive listed in the command, and is created during a save if it > > +does not exist. If it does exist, it is renamed with a new suffix > > +(~) to keep one previous copy. If there is already a > > +subdirectory with that suffix, it (and it's contents) are deleted. > > +A restore will first rename each target config file with the suffix, > > +deleting any previous one with that name, and put the restored copy in. > > + > > +.SH AUTHORS > > +.TP > > +Dave McMillen > > +.RI < davem at systemfabricworks.com > > > diff --git a/infiniband-diags/src/ibcli.c b/infiniband-diags/src/ibcli.c > > new file mode 100644 > > index 0000000..44b762c > > --- /dev/null > > +++ b/infiniband-diags/src/ibcli.c > > @@ -0,0 +1,740 @@ > > +/* > > + * Copyright (c) 2007 System Fabric Works, Inc. All rights reserved. > > + * > > + * This software is available to you under a choice of one of two > > + * licenses. You may choose to be licensed under the terms of the GNU > > + * General Public License (GPL) Version 2, available from the file > > + * COPYING in the main directory of this source tree, or the > > + * OpenIB.org BSD license below: > > + * > > + * Redistribution and use in source and binary forms, with or > > + * without modification, are permitted provided that the following > > + * conditions are met: > > + * > > + * - Redistributions of source code must retain the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer. > > + * > > + * - Redistributions in binary form must reproduce the above > > + * copyright notice, this list of conditions and the following > > + * disclaimer in the documentation and/or other materials > > + * provided with the distribution. > > + * > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > > + * SOFTWARE. > > + * > > + */ > > + > > +/* ibcli.c > > + > > + This is a command line interface (CLI) designed to simplify and unify > > + the use of various OFED software components. There are several parts > > + to this program: > > + > > + * Configuration file management/maintenance/inquiry > > + > > + The various configuration files are accessed through simple names > > + within the CLI, referencing the actual files in the appropriate > > + locations depending on where and how the OFED software was installed. > > + > > + Extraction and modification of individual parameter/value pairs is > > + easily done, and all configuration files are modified using a rename() > > + system call. This means that either the old or new version in it's > > + entirety will be seen by any program that opens and reads a file, even > > + though it is unsynchronized with the modification program. Unlike an > > + editor that rewrites a file, where there is a possibility that another > > + program will see some but not all changes, the method used here will > > + never present an inconsistent configuration. > > + > > + Also, the complete set of configuration files can be saved, duplicated, > > + modified separate from the "live" versions, and restored. > > + > > + * > > +*/ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#define SAVED_CONFIGS_DIR "/etc/infiniband/saved_configs" > > +#define CONFIG_LOCK_SUFFIX "-locked" > > +#define CONFIG_PREV_SUFFIX "~" > > +#define CONFIG_NEW_SUFFIX "+" > > +#define MAX_LOCK_ATTEMPTS 20 > > +#define DEFAULT_EDITOR "vi" > > + > > +#define CMD_SAVE "save" > > +#define CMD_RESTORE "restore" > > +#define CMD_COPY "copy" > > +#define CMD_GET "get" > > +#define CMD_SET "set" > > +#define CMD_EDIT "edit" > > + > > +struct config_files { > > + char *cmd_name; /* Name used in commands to reference this */ > > + char *file_name; /* Absolute path to configuration file */ > > + int usespace; /* Param/value pairs do not have = between them */ > > + }; > > + > > +struct config_files config_files[] = { /* TODO: Pick up installed places */ > > + {"srp", "/etc/srp_daemon.conf", -1}, > > + {"opensm", "/etc/opensm.conf", 0}, > > + {"opensm.opts", "", 1}, > > + {"sdp", "/etc/libsdp.conf", -1}, > > + {"openib", "/etc/infiniband/openib.conf", 0}, > > + { NULL, NULL } }; > > + > > +char *argv0; /* Remembers the argv[0] for the main() program */ > > + > > +char inbuf[16384]; /* Used for input and temporary purposes */ > > +char savebuf[16384]; /* Saves commentary for later printout */ > > +char *savebufp; /* Remembers how far we have filled savebuf */ > > + > > +/* For configuration files that may be at a user-defined place, figure out > > + where they are located and return that location. */ > > + > > +char *find_cfg_file(char *cmd_name) { > > + if (strcmp(cmd_name, "opensm.opts") == 0) { > > + return "/var/cache/osm/opensm.opts"; /* TODO: find real path value */ > > + } > > + fprintf(stderr, "ERROR: %s internal error, unknown config file for %s\n", > > + argv0, cmd_name); > > + exit(2); > > +} > > + > > +/* Find the last non-/ character in a string */ > > + > > +char *base_pointer(char *path) { > > + int l = strlen(path); > > + while (l > 0 && path[l-1] != '/') l--; > > + return path + l; > > +} > > + > > +/* Return a malloc()'d string with the full file name of the configuration > > + file, given the archive name and standard running full filename */ > > + > > +char *working_cfg_file(char *archive, char *filename) { > > + char *rv, *fv; > > + int srv; > > + > > + if (archive == NULL) { > > + rv = (char *)malloc(strlen(filename) + strlen(CONFIG_PREV_SUFFIX) + 1); > > + strcpy(rv, filename); > > + return rv; > > + } > > + fv = base_pointer(filename); > > + srv = strlen(SAVED_CONFIGS_DIR) + 1 + strlen(archive) + 1 + strlen(fv) > > + + strlen(CONFIG_PREV_SUFFIX) + 1; > > + rv = (char *)malloc(srv); > > + strcpy(rv, SAVED_CONFIGS_DIR); > > + strcat(rv, "/"); > > + strcat(rv, archive); > > + strcat(rv, "/"); > > + strcat(rv, fv); > > + return rv; > > +} > > + > > +/* Tell the user how to invoke this program */ > > + > > +void usage() { > > + int cf_index; > > + char sep = ' '; > > + > > + fprintf(stderr, "Usage:\n"); > > + fprintf(stderr, " %s get [archive]", argv0); > > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > > + sep = '|'; > > + } > > + fprintf(stderr, " [?|param]\n"); > > + sep = ' '; > > + fprintf(stderr, " %s set [archive]", argv0); > > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > > + sep = '|'; > > + } > > + fprintf(stderr, " param=?|value\n"); > > + sep = ' '; > > + fprintf(stderr, " %s edit [archive]", argv0); > > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > > + fprintf(stderr, "%c%s", sep, config_files[cf_index].cmd_name); > > + sep = '|'; > > + } > > + fprintf(stderr, " [editor]\n"); > > + fprintf(stderr, " %s save|restore archive\n", argv0); > > + fprintf(stderr, " %s copy source-archive archive\n", argv0); > > + exit(1); > > +} > > + > > +/* Lock a configuration file. A parallel file is created with the same name as > > + the file to be locked, with the CONFIG_LOCK_SUFFIX appended to the name. > > + This will contain the PID of the program that did the lock. If a program > > + dies holding the lock, this is detected, and the lock file removed. This > > + method is reportedly broken for the general case on NFS filesystems, but > > + seems to work when all locking accesses are from the same system. */ > > + > > +void lock_config_file(char *filename) { > > + int i, rc, attempts = MAX_LOCK_ATTEMPTS; > > + long int lpid; > > + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); > > + > > + strcpy(lfn, filename); > > + strcat(lfn, CONFIG_LOCK_SUFFIX); > > + while (--attempts >= 0) { > > + i = open(lfn, O_WRONLY | O_CREAT | O_EXCL, 0644); > > + if (i != -1) { > > + sprintf(inbuf, "%d", getpid()); > > + write(i, inbuf, strlen(inbuf)); > > + close(i); > > + free(lfn); > > + return; > > + } > > + if (errno != EEXIST) { > > + perror(lfn); > > + fprintf(stderr, "%s: Could not open interlock file\n", argv0); > > + free(lfn); > > + exit(3); > > + } > > + i = open(lfn, O_RDONLY); > > + if (i != -1) { > > + rc = read(i, inbuf, sizeof(inbuf) - 1); > > + close(i); > > + if (rc) { > > + lpid = strtol(inbuf, NULL, 10); > > + if (kill(lpid, 0) == -1 && errno == ESRCH) unlink(lfn); > > + } > > + } > > + sleep(1); > > + } > > + fprintf(stderr, "%s: Could not establish interlock file %s\n", argv0, lfn); > > + free(lfn); > > + exit(3); > > +} > > + > > +/* Unlock a configuration file - simply remove the lock file */ > > + > > +void unlock_config_file(char *filename) { > > + int i; > > + char *lfn = (char *)malloc(strlen(filename)+strlen(CONFIG_LOCK_SUFFIX) +1); > > + > > + strcpy(lfn, filename); > > + strcat(lfn, CONFIG_LOCK_SUFFIX); > > + i = unlink(lfn); > > + if (i) { > > + perror(lfn); > > + fprintf(stderr, "%s: Could not remove interlock file\n", argv0); > > + exit(3); > > + } > > + free(lfn); > > +} > > + > > +/* Routine to copy a complete set of configuration files */ > > + > > +void copy_configs(char *source, char *destination) { > > + int cf_index; > > + char *ifn, *ofn, *tfn; > > + FILE *input, *output; > > + > > + for (cf_index = 0; config_files[cf_index].cmd_name != NULL; cf_index++) { > > + ofn = config_files[cf_index].file_name; > > + if (ofn[0] == 0) > > + ofn = find_cfg_file(config_files[cf_index].cmd_name); > > + ifn = working_cfg_file(source, ofn); > > + ofn = working_cfg_file(destination, ofn); > > + tfn = (char *)malloc(strlen(ofn) > > + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); > > + lock_config_file(ifn); > > + lock_config_file(ofn); > > + input = fopen(ifn, "r"); > > + if (!input) { > > + int ern = errno; > > + unlock_config_file(ofn); > > + unlock_config_file(ifn); > > + free(ofn); > > + free(ifn); > > + if (ern == ENOENT) continue; /* This config file not present */ > > + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0, ifn); > > + exit(2); > > + } > > + strcpy(tfn, ofn); > > + strcat(tfn, CONFIG_PREV_SUFFIX); > > + unlink(tfn); /* Remove any previous previous version */ > > + link(ofn, tfn); /* Create link for new previous version */ > > + strcpy(tfn, ofn); > > + strcat(tfn, CONFIG_NEW_SUFFIX); > > + output = fopen(tfn, "w"); > > + if (!output) { > > + fprintf(stderr, "%s: Cannot open to write into %s\n", argv0, tfn); > > + exit(2); > > + } > > + while (fgets(inbuf, sizeof(inbuf), input)) > > + if (fputs(inbuf, output) == EOF) { > > + fprintf(stderr, "%s: Error writing copy to %s\n", argv0, tfn); > > + exit(2); > > + } > > + fclose(input); > > + fclose(output); > > + if (rename(tfn, ofn)) { > > + perror(ofn); > > + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", > > + argv0, tfn); > > + exit(2); > > + } > > + unlock_config_file(ofn); > > + unlock_config_file(ifn); > > + free(ofn); > > + free(ifn); > > + } > > +} > > + > > +/* Main program entry point (start here) */ > > + > > +int main(int argc, char **argv) { > > + int i, a, setting, editing, query, found, psize, cf_index; > > + int cmd_restore, cmd_copy; > > + char *archive = NULL; > > + char *use_cfg, *cp, *np, *param, *value, *tfn; > > + struct stat stb; > > + struct dirent *ne; > > + FILE *input, *output; > > + DIR *rmd; > > + > > + argv0 = base_pointer(*argv); > > + if (argc < 3) usage(); > > + > > + > > + /* save and restore commands > > + * > > + * move all configuration files (as listed in the config_files array) > > + * between their normal location and a subdirectory in the saved config > > + * directory (SAVED_CONFIGS_DIR). This subdirectory name is the name of > > + * the archive listed in the command, and is created during a save if it > > + * does not exist. If it does exist, it is renamed with a new suffix > > + * (CONFIG_PREV_SUFFIX) to keep one previous copy. If there is already a > > + * subdirectory with that suffix, it (and it's contents) are deleted. > > + * A restore will first rename each target config file with the suffix, > > + * deleting any previous one with that name, and put the restored copy in. > > + */ > > + > > + cmd_restore = strcmp(argv[1], CMD_RESTORE) == 0; > > + cmd_copy = strcmp(argv[1], CMD_COPY) == 0; > > + setting = strcmp(argv[1], CMD_SET) == 0; > > + editing = strcmp(argv[1], CMD_EDIT) == 0; > > + if (strcmp(argv[1], CMD_SAVE) == 0 || cmd_restore || cmd_copy) { > > + if (cmd_copy) { > > + if (argc != 4) usage(); > > + a = 3; > > + } else { > > + if (argc != 3) usage(); > > + a = 2; > > + } > > + if (strcmp(argv[a], ".") == 0 > > + || strncmp(argv[a], "..", 2) == 0 > > + || strchr(argv[a], '/')) { > > + fprintf(stderr, "%s: Saved configuration name \"%s\" cannot be " > > + "., start with .., or contain a / character\n", > > + argv0, argv[a]); > > + return 2; > > + } > > + archive = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 > > + + strlen(argv[a]) + 1); > > + strcpy(archive, SAVED_CONFIGS_DIR); > > + strcat(archive, "/"); > > + strcat(archive, argv[a]); > > + i = stat(archive, &stb); > > + > > + /* restore from archive */ > > + > > + if (cmd_restore) { > > + if (i || !(S_ISDIR(stb.st_mode))) { > > + fprintf(stderr, > > + "%s: There is no saved configuration named \"%s\"\n", > > + argv0, argv[a]); > > + free(archive); > > + return 2; > > + } > > + copy_configs(argv[a], NULL); > > + > > + /* save to archive, or copy archive-src to archive */ > > + > > + } else { > > + if (cmd_copy) { /* Validate existence of archive1 */ > > + np = (char *)malloc(strlen(SAVED_CONFIGS_DIR) + 1 > > + + strlen(argv[2]) + 1); > > + strcpy(np, SAVED_CONFIGS_DIR); > > + strcat(np, "/"); > > + strcat(np, argv[2]); > > + if (stat(np, &stb) || !(S_ISDIR(stb.st_mode))) { > > + fprintf(stderr, > > + "%s: There is no saved configuration named \"%s\"\n", > > + argv0, argv[2]); > > + free(np); > > + return 2; > > + } > > + free(np); > > + } > > + if (!i) { /* create suffixed directory if previous save done */ > > + cp = (char *)malloc(strlen(archive) + 2); > > + strcpy(cp, archive); > > + strcat(cp, CONFIG_PREV_SUFFIX); > > + rmd = opendir(cp); > > + if (rmd) { > > + while ((ne = readdir(rmd))) { > > + if (strcmp(ne->d_name, ".") == 0 > > + || strcmp(ne->d_name, "..") == 0) continue; > > + np = (char *)malloc(strlen(cp) + 1 > > + + strlen(ne->d_name) + 1); > > + strcpy(np, cp); > > + strcat(np, "/"); > > + strcat(np, ne->d_name); > > + unlink(np); /* Errors will report with rmdir() */ > > + free(np); > > + } > > + closedir(rmd); > > + errno = ENOENT; > > + rmdir(cp); > > + } > > + if (errno != ENOENT) { > > + perror(cp); > > + fprintf(stderr, "%s: Cannot remove backup copy\n", > > + argv0); > > + free(cp); > > + free(archive); > > + return 2; > > + } > > + i = rename(archive, cp); > > + if (i) { > > + perror(archive); > > + fprintf(stderr, > > + "%s: Cannot rename existing copy to backup copy\n", > > + argv0); > > + free(cp); > > + free(archive); > > + return 2; > > + } > > + free(cp); > > + } > > + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) > > + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) { > > + fprintf(stderr, "%s: Cannot use reserved name \"%s\" for" > > + " a saved archive\n", argv0, argv[a]); > > + return 2; > > + } > > + i = mkdir(archive, 0755); > > + if (i) { > > + mkdir(SAVED_CONFIGS_DIR, 0755); /* In case it is missing */ > > + i = mkdir(archive, 0755); > > + } > > + if (i) { > > + perror(archive); > > + fprintf(stderr, "%s: Cannot create directory for saving\n", > > + argv0); > > + free(archive); > > + return 2; > > + } > > + if (cmd_copy) > > + copy_configs(argv[2], argv[a]); > > + else > > + copy_configs(NULL, argv[a]); > > + } > > + free(archive); > > + > > + > > + /* set and get commands > > + */ > > + > > + } else if (setting || strcmp(argv[1], CMD_GET) == 0 || editing) { > > + for (a = 2; a <= 3; a++) { > > + if (argc <= a) usage(); > > + for (cf_index=0; config_files[cf_index].cmd_name!=NULL; cf_index++) > > + if (strcmp(argv[a], config_files[cf_index].cmd_name) == 0) > > + break; > > + if (config_files[cf_index].cmd_name != NULL) break; > > + archive = argv[a]; > > + } > > + if (config_files[cf_index].cmd_name == NULL) usage(); > > + use_cfg = config_files[cf_index].file_name; > > + if (use_cfg[0] == 0) > > + use_cfg = find_cfg_file(config_files[cf_index].cmd_name); > > + use_cfg = working_cfg_file(archive, use_cfg); > > + > > + /* get with no parameter means get whole file */ > > + > > + if (!editing && argc <= (a+1)) { > > + if (setting) usage(); > > + input = fopen(use_cfg, "r"); > > + if (input == NULL) { > > + if (errno == ENOENT) > > + fprintf(stderr, > > + "%s: There is no saved configuration named \"%s\"\n", > > + argv0, archive); > > + else > > + perror(use_cfg); > > + return 2; > > + } > > + while (fgets(inbuf, sizeof(inbuf), input)) > > + fputs(inbuf, stdout); > > + fclose(input); > > + free(use_cfg); > > + return 0; > > + } > > + > > + /* Verify that we can use simple parameter/value parsing on this */ > > + > > + if (!editing && config_files[cf_index].usespace < 0) { > > + if (setting) > > + fprintf(stderr, "%s: %s file format too complicated, use edit " > > + "command\n", argv0, config_files[cf_index].cmd_name); > > + else > > + fprintf(stderr, "%s: %s file format too complicated, use get " > > + "command with no param\n", argv0, > > + config_files[cf_index].cmd_name); > > + free(use_cfg); > > + return 2; > > + } > > + if (editing) { > > + if (argc <= (a+1)) { > > + if (((cp = getenv("VISUAL")) == NULL || *cp == '\0') && > > + ((cp = getenv("EDITOR")) == NULL || *cp == '\0')) > > + cp = DEFAULT_EDITOR; > > + param = (char *)malloc(strlen(cp)+1); > > + strcpy(param, cp); > > + } else { > > + if (argc != (a+2)) usage(); > > + param = (char *)malloc(strlen(argv[a+1])+1); > > + strcpy(param, argv[a+1]); > > + } > > + } else { > > + param = (char *)malloc(strlen(argv[a+1])+1); > > + strcpy(param, argv[a+1]); > > + } > > + > > + /* if setting, figure out if new value is in the same "word" as the > > + * parameter name with an = to delineate them > > + */ > > + if (setting) { > > + for (cp = param; *cp ; cp++) if (*cp == '=') break; > > + if (*cp && cp[1] == 0) *cp = 0; > > + if (*cp) { > > + if (argc != (a+2)) usage(); > > + value = cp + 1; > > + } else { > > + if (argc == (a+2)) { /* if no value, use ? as default */ > > + value = "?"; > > + } else { /* otherwise value follows */ > > + if (argc == (a+3)) { > > + value = argv[a+2]; > > + if (*value == '=') value++; > > + } else if (argc == (a+4) && strcmp(argv[a+2], "=") == 0) { > > + value = argv[a+3]; > > + } else > > + usage(); > > + } > > + } > > + while (cp > param && (cp[-1] == ' ' || cp[-1] == '\t')) cp--; > > + *cp = 0; > > + while(*value == ' ' || *value == '\t') value++; > > + > > + /* not set, must be get or edit */ > > + } else { > > + /* If doing get, make sure there are no extra arguments */ > > + if (!editing && argc != (a+2)) usage(); > > + } > > + > > + /* Check for query. For get this means list all values, while for set > > + * this means show the comments that are in the file before the value. > > + */ > > + if (setting) > > + query = !(strcmp("?", value)); > > + else > > + query = !(strcmp("?", param)); > > + > > + /* Now scan the file */ > > + > > + found = 0; > > + input = fopen(use_cfg, "r"); > > + if (!input) { > > + fprintf(stderr, "%s: Cannot open to copy from %s\n", argv0,use_cfg); > > + exit(2); > > + } > > + > > + /* If set/edit command, we create a link to the current file as previous > > + * version (so when we remove the current one, we have the backup), and > > + * create a new file with a different name, which will get renamed to > > + * the normal name at the end. > > + */ > > + if ((setting && !query) || editing) { > > + lock_config_file(use_cfg); > > + tfn = (char *)malloc(strlen(use_cfg) > > + + strlen(CONFIG_PREV_SUFFIX) + strlen(CONFIG_NEW_SUFFIX)); > > + strcpy(tfn, use_cfg); > > + strcat(tfn, CONFIG_PREV_SUFFIX); > > + unlink(tfn); /* Remove any previous previous version */ > > + link(use_cfg, tfn); /* Create link for new previous version */ > > + strcpy(tfn, use_cfg); > > + strcat(tfn, CONFIG_NEW_SUFFIX); > > + output = fopen(tfn, "w"); > > + if (!output) { > > + unlock_config_file(use_cfg); > > + fprintf(stderr, "%s: Cannot open to write into %s\n", > > + argv0, tfn); > > + exit(2); > > + } > > + } > > + > > + /* Now we scan the file */ > > + > > + psize = strlen(param); > > + savebufp = savebuf; > > + while (fgets(inbuf, sizeof(inbuf), input)) { > > + cp = inbuf + strlen(inbuf); > > + > > + /* Strip trailing NL/CR */ > > + > > + while (cp > inbuf && > > + (cp[-1] == '\n' || cp[-1] == '\r')) cp--; > > + *cp = 0; > > + > > + /* Strip leading spaces/tabs */ > > + > > + cp = inbuf; > > + while (*cp == ' ' || *cp == '\t') cp++; > > + > > + /* See if this is a comment or empty line */ > > + > > + if (*cp == '#' || *cp == 0) { > > + if (setting && query && *cp == '#') { > > + cp++; /* Save comments for set query output */ > > + i = strlen(cp); > > + if (i < (sizeof(savebuf) - (savebufp - savebuf) - 2)) { > > + strcpy(savebufp, cp); > > + savebufp += i; > > + *savebufp++ = '\n'; > > + } > > + } > > + > > + /* Not a comment or empty line, must be part of a param/value */ > > + > > + } else if (!editing) { > > + > > + /* if get command with ? for param, show all param lines */ > > + > > + if (!setting && query) { > > + puts(cp); > > + > > + /* get or set with param to look for, see if it is there */ > > + > > + } else { > > + if (strncmp(param, cp, psize) == 0) { > > + if (cp[psize] == 0 || cp[psize] == '=' > > + || cp[psize] == ' ' || cp[psize] == '\t') { > > + > > + found++; > > + if (setting) { > > + if (query) { > > + > > + /* Found param, this a set query */ > > + > > + *savebufp = 0; > > + fputs(savebuf, stdout); > > + } else { > > + > > + /* Found param, this a set with value */ > > + > > + np = cp + psize; > > + while (*np == ' ' || *np == '\t') np++; > > + if (*np == '=') { > > + np++; > > + while (*np == ' ' || *np == '\t') np++; > > + } > > + > > + /* Put new value where old value was */ > > + > > + i = (np - inbuf) + strlen(value) + 2; > > + if (i > sizeof(inbuf)) { > > + fprintf(stderr, "%s: New value for " > > + "\"%s\" is %d bytes too long\n", > > + argv0, param, > > + (int)(sizeof(inbuf)-i)); > > + exit(2); > > + } else > > + strcpy(np, value); > > + } > > + > > + /* Found param and doing a get so just print it */ > > + > > + } else { > > + puts(cp); > > + } > > + } > > + } > > + } > > + savebufp = savebuf; /* Saw new param, toss old comments */ > > + } > > + > > + /* If doing set or edit, copy all lines to replacement file */ > > + > > + if (editing || (setting && !query)) { > > + if (fprintf(output, "%s\n", inbuf) < 0) { > > + unlock_config_file(use_cfg); > > + fprintf(stderr, "%s: Error writing copy to %s\n", > > + argv0, tfn); > > + exit(2); > > + } > > + } > > + } > > + fclose(input); > > + > > + /* If doing set, output is done, rename file to make it the real one */ > > + > > + if ((setting && !query) || editing) { > > + if (!found && !editing) { /* Did not find param so we add it */ > > + if (config_files[cf_index].usespace) > > + fprintf(output, "%s %s\n", param, value); > > + else > > + fprintf(output, "%s = %s\n", param, value); > > + } > > + fclose(output); > > + if (editing) { > > + cp = (char *)malloc(strlen(param) + 2 + strlen(tfn) + 2); > > + strcpy(cp, param); > > + strcat(cp, " '"); > > + strcat(cp, tfn); > > + strcat(cp, "'"); > > + system(cp); > > + free(cp); > > + } > > + if (rename(tfn, use_cfg)) { > > + perror(use_cfg); > > + unlock_config_file(use_cfg); > > + fprintf(stderr, "%s: Cannot rename temporary copy %s\n", > > + argv0, tfn); > > + exit(2); > > + } > > + unlock_config_file(use_cfg); > > + free(tfn); > > + } > > + free(param); > > + free(use_cfg); > > + > > + /* Unknown command, just tell caller how to use the program */ > > + > > + } else > > + usage(); > > + > > + return 0; > > +} From weiny2 at llnl.gov Thu Aug 9 19:03:30 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 9 Aug 2007 19:03:30 -0700 Subject: [ofa-general] [PATCH 0/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's Message-ID: <20070809190330.07d502f9.weiny2@llnl.gov> I have changed set_mthca_nodedesc.sh to look for all HCA's in sysfs and set the nodedesc to the host name. I have tested it on MLX4 and mthca cards and it works fine. I don't know if other dirvers allow writes to their nodedesc sysfs entry. If they do this should work for all cards. Furthermore, I changed the name from set_mthca_nodedesc.sh to set_nodedesc.sh Ira From weiny2 at llnl.gov Thu Aug 9 19:03:31 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 9 Aug 2007 19:03:31 -0700 Subject: [ofa-general] [PATCH 1/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's Message-ID: <20070809190331.4088e85e.weiny2@llnl.gov> >From 3abf9ac10ee90f7272969486704c2989d254a2ac Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 9 Aug 2007 16:07:38 -0700 Subject: [PATCH] Modify set_mthca_nodedesc to set the node_desc for any hca type found in sysfs. This has been tested for the MLX4 and mthca cards but not others. Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/set_mthca_nodedesc.sh | 13 +++++++++++-- 1 files changed, 11 insertions(+), 2 deletions(-) diff --git a/infiniband-diags/scripts/set_mthca_nodedesc.sh b/infiniband-diags/scripts/set_mthca_nodedesc.sh index 618373f..4613de6 100755 --- a/infiniband-diags/scripts/set_mthca_nodedesc.sh +++ b/infiniband-diags/scripts/set_mthca_nodedesc.sh @@ -1,9 +1,18 @@ #!/bin/sh -# set the node_desc field in the mthca to the hostname +# set the node_desc field of any hca found to the defined hostname . /etc/sysconfig/network -echo -n "$HOSTNAME" >> /sys/class/infiniband/mthca0/node_desc +ib_sysfs="/sys/class/infiniband" + +for hca in `ls $ib_sysfs`; do + if [ -f $ib_sysfs/$hca/node_desc ]; then + echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc + else + logger -s "Failed to set node_desc for : $hca" + fi +done exit 0 + -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Modify-set_mthca_nodedesc-to-set-the-node_desc-for-a.patch Type: application/octet-stream Size: 1239 bytes Desc: not available URL: From weiny2 at llnl.gov Thu Aug 9 19:03:32 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 9 Aug 2007 19:03:32 -0700 Subject: [ofa-general] [PATCH 2/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's Message-ID: <20070809190332.3256988e.weiny2@llnl.gov> >From 3c78f9f7797a9029b33a1a80476f82929f8a7ad7 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 9 Aug 2007 18:51:09 -0700 Subject: [PATCH] Change the name of set_mthca_nodedesc.sh to set_nodedesc.sh because it should work for all HCA types now. Signed-off-by: Ira K. Weiny --- infiniband-diags/ChangeLog | 6 ++++++ infiniband-diags/Makefile.am | 2 +- infiniband-diags/infiniband-diags.spec.in | 5 ++++- infiniband-diags/scripts/set_mthca_nodedesc.sh | 18 ------------------ infiniband-diags/scripts/set_nodedesc.sh | 18 ++++++++++++++++++ 5 files changed, 29 insertions(+), 20 deletions(-) delete mode 100755 infiniband-diags/scripts/set_mthca_nodedesc.sh create mode 100755 infiniband-diags/scripts/set_nodedesc.sh diff --git a/infiniband-diags/ChangeLog b/infiniband-diags/ChangeLog index ad35140..9dae628 100644 --- a/infiniband-diags/ChangeLog +++ b/infiniband-diags/ChangeLog @@ -1,3 +1,9 @@ +2007-08-09 Ira Weiny + + * scripts/set_mthca_nodedesc.sh: change to set_nodedesc.sh + * scripts/set_mthca_nodedesc.sh: attempt to set nodedesc on all HCA's found + in sysfs + 2007-07-10 Hal Rosenstock * 1.3.1 release of infiniband-diags diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index c90fed1..fcaa29e 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -26,7 +26,7 @@ sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ scripts/ibclearcounters scripts/ibdatacounts \ scripts/ibdatacounters scripts/ibdiscover.pl \ scripts/dump_lfts.sh scripts/dump_mfts.sh \ - scripts/set_mthca_nodedesc.sh \ + scripts/set_nodedesc.sh \ scripts/ibqueryerrors.pl scripts/ibswportwatch.pl \ scripts/iblinkinfo.pl scripts/ibprintswitch.pl \ scripts/ibprintca.pl scripts/ibprintrt.pl \ diff --git a/infiniband-diags/infiniband-diags.spec.in b/infiniband-diags/infiniband-diags.spec.in index b67a645..335fa8a 100644 --- a/infiniband-diags/infiniband-diags.spec.in +++ b/infiniband-diags/infiniband-diags.spec.in @@ -48,7 +48,7 @@ rm -rf $RPM_BUILD_ROOT %{_sbindir}/vendstat %{_sbindir}/dump_mfts.sh %{_sbindir}/dump_lfts.sh -%{_sbindir}/set_mthca_nodedesc.sh +%{_sbindir}/set_nodedesc.sh %{_sbindir}/sm* %define _perldir %(perl -e 'use Config; $T=$Config{installsitearch}; $T=~/(.*)\\/site_perl.*/; print $1;') %{_perldir}/* @@ -56,6 +56,9 @@ rm -rf $RPM_BUILD_ROOT %doc README ChangeLog %changelog +* Thu Aug 9 2007 Ira Weiny - 1.3.1 +- Change set_mthca_nodedesc.sh to set_nodedesc.sh + * Tue Jul 10 2007 Hal Rosenstock - 1.3.1 - Add link width and speed to topology file output in ibnetdiscover - Add support for -R(outer_list) in ibnetdiscover diff --git a/infiniband-diags/scripts/set_mthca_nodedesc.sh b/infiniband-diags/scripts/set_mthca_nodedesc.sh deleted file mode 100755 index 4613de6..0000000 --- a/infiniband-diags/scripts/set_mthca_nodedesc.sh +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh - -# set the node_desc field of any hca found to the defined hostname - -. /etc/sysconfig/network - -ib_sysfs="/sys/class/infiniband" - -for hca in `ls $ib_sysfs`; do - if [ -f $ib_sysfs/$hca/node_desc ]; then - echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc - else - logger -s "Failed to set node_desc for : $hca" - fi -done - -exit 0 - diff --git a/infiniband-diags/scripts/set_nodedesc.sh b/infiniband-diags/scripts/set_nodedesc.sh new file mode 100755 index 0000000..4613de6 --- /dev/null +++ b/infiniband-diags/scripts/set_nodedesc.sh @@ -0,0 +1,18 @@ +#!/bin/sh + +# set the node_desc field of any hca found to the defined hostname + +. /etc/sysconfig/network + +ib_sysfs="/sys/class/infiniband" + +for hca in `ls $ib_sysfs`; do + if [ -f $ib_sysfs/$hca/node_desc ]; then + echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc + else + logger -s "Failed to set node_desc for : $hca" + fi +done + +exit 0 + -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Change-the-name-of-set_mthca_nodedesc.sh-to-set_node.patch Type: application/octet-stream Size: 4088 bytes Desc: not available URL: From rdreier at cisco.com Thu Aug 9 19:29:02 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 09 Aug 2007 19:29:02 -0700 Subject: [ofa-general] Re: [PATCH] ib_umad unaligned access on ia64 In-Reply-To: <20070810012709.GG25325@sgi.com> (akepner@sgi.com's message of "Thu, 9 Aug 2007 18:27:09 -0700") References: <20070809203612.GA32445@sgi.com> <20070810012709.GG25325@sgi.com> Message-ID: > > I thought we already dealt with this one in commit fa7252ed? > > I didn't see anything which obviously addresses this in > any tree I found at kernel.org or openfabrics.org, but > maybe I didn't look in the right tree. this is the commit I'm talking about; I think it went into 2.6.21 but I'm not sure. Anyway it seems it should make sure that the tid member is aligned to 8 bytes, or at least I think it should: commit fa7252ed4d92397baf30e4a144af95a33eaa925b Author: Jason Gunthorpe Date: Fri Jan 19 11:58:49 2007 -0700 IB: Make sure struct ib_user_mad.data is aligned Make the untyped data region in ib_user_mad have type u64 so that it gets aligned properly. This avoids alignment faults in ib_umad when casting the data field to an rmpp_mad and accessing the 64-bit tid field on architectures like ia64. Signed-off-by: Jason Gunthorpe Signed-off-by: Roland Dreier diff --git a/include/rdma/ib_user_mad.h b/include/rdma/ib_user_mad.h index 44537aa..d66b15e 100644 --- a/include/rdma/ib_user_mad.h +++ b/include/rdma/ib_user_mad.h @@ -98,7 +98,7 @@ struct ib_user_mad_hdr { */ struct ib_user_mad { struct ib_user_mad_hdr hdr; - __u8 data[0]; + __u64 data[0]; }; /** From hal.rosenstock at gmail.com Thu Aug 9 20:10:54 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 9 Aug 2007 23:10:54 -0400 Subject: [ofa-general] [PATCH 1/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's In-Reply-To: <20070809190331.4088e85e.weiny2@llnl.gov> References: <20070809190331.4088e85e.weiny2@llnl.gov> Message-ID: On 8/9/07, Ira Weiny wrote: > >From 3abf9ac10ee90f7272969486704c2989d254a2ac Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 9 Aug 2007 16:07:38 -0700 > Subject: [PATCH] Modify set_mthca_nodedesc to set the node_desc for any hca type found in sysfs. > > This has been tested for the MLX4 and mthca cards but not others. I think the iPath driver may need some change to make this work. > > Signed-off-by: Ira K. Weiny > --- > infiniband-diags/scripts/set_mthca_nodedesc.sh | 13 +++++++++++-- > 1 files changed, 11 insertions(+), 2 deletions(-) > > diff --git a/infiniband-diags/scripts/set_mthca_nodedesc.sh b/infiniband-diags/scripts/set_mthca_nodedesc.sh > index 618373f..4613de6 100755 > --- a/infiniband-diags/scripts/set_mthca_nodedesc.sh > +++ b/infiniband-diags/scripts/set_mthca_nodedesc.sh > @@ -1,9 +1,18 @@ > #!/bin/sh > > -# set the node_desc field in the mthca to the hostname > +# set the node_desc field of any hca found to the defined hostname > > . /etc/sysconfig/network > > -echo -n "$HOSTNAME" >> /sys/class/infiniband/mthca0/node_desc > +ib_sysfs="/sys/class/infiniband" > + > +for hca in `ls $ib_sysfs`; do > + if [ -f $ib_sysfs/$hca/node_desc ]; then > + echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc > + else > + logger -s "Failed to set node_desc for : $hca" > + fi > +done > > exit 0 > + > -- > 1.5.2 > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From kliteyn at mellanox.co.il Thu Aug 9 21:14:09 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 10 Aug 2007 07:14:09 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-10:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Wed_Aug_8_22:48:15_2007 [83d1f5ea339704f30e26918a897f9ead5a10678e] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=461 Fail=59 Pass: 39 LidMgr IS1-16.topo 38 Stability IS1-16.topo 38 OsmStress IS1-16.topo 38 Multicast IS1-16.topo 37 OsmTest IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo 12 OsmTest IS3-loop.topo 12 Multicast IS3-loop.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 2 OsmTest IS1-16.topo 1 Stability IS1-16.topo 1 OsmTest IS3-loop.topo 1 OsmStress IS1-16.topo 1 Multicast IS3-loop.topo 1 Multicast IS1-16.topo From isbmytrn at andreuworld.com Thu Aug 9 16:24:02 2007 From: isbmytrn at andreuworld.com (Porto FPT) Date: Thu, 9 Aug 2007 14:24:02 -0900 Subject: [ofa-general] Re: Thank you, we are ready to lend money Message-ID: <14FC156027481D0.3BCE92392D@andreuworld.com> An HTML attachment was scrubbed... URL: From pradeeps at linux.vnet.ibm.com Thu Aug 9 22:51:47 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Thu, 09 Aug 2007 22:51:47 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <20070809155144.GA32251@mellanox.co.il> References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> Message-ID: <46BBFCF3.1070109@linux.vnet.ibm.com> Michael S. Tsirkin wrote: >> Quoting Pradeep Satyanarayana : >> Subject: Re: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ >> >> Roland, Michael, >> >> Could you please review this patch and provide your comments? I would like to finish >> up this work at the soonest and have this integrated into the for-2.6.24 tree. > > Naturally, with just a single packet pre-posted, RC QP will likely get RNR > errors and close down quite soon. Wouldn't it make more sense to prepost > some more WRs? > This is only for completeness sake. Performance will be poor compared to two (or more) Linux systems using this. This is expected to be really a corner and do not expect this to happen. Hence is there for completeness only. If I interpreted this correctly, this is was what Roland suggested (and I agree with the suggestion). Pradeep From mst at dev.mellanox.co.il Thu Aug 9 22:59:52 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Fri, 10 Aug 2007 08:59:52 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <1186705635.4617.452.camel@firewall.xsintricity.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061530.54392.hnguyen@linux.vnet.ibm.com> <1186705635.4617.452.camel@firewall.xsintricity.com> Message-ID: <20070810055952.GD4740@mellanox.co.il> > Quoting Doug Ledford : > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > On Mon, 2007-08-06 at 15:30 +0200, Hoang-Nam Nguyen wrote: > > Hello Doug and Scott! > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > ehca backports for kernel.org kernels seem to be broken. > > > 1. Does anyone care enough to fix them? If not we'll disable > > > ehca in build for these kernels. > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > This would make it possible for us to add it to nightly builds. > > Could you please provide Michael and Vladimir with a URL to download > > above kernel source tree in order to perform daily build of ofed > > code suite? This will help us to prevent build issues in very early > > stage. > > Actually, with RHEL4 didn't we already establish that me uploading a > source tarball is definitely *not* the way to go? The kernel-devel rpm > on each arch only includes the files that it thinks are relevant to that > arch, and by compiling against a complete source tree, you missed the > fact that the ppc64 arch wouldn't compile against the kernel-devel.ppc64 > package entirely. This was never intended as a replacement for testing - this is just a tool to speed up backport development. But fine - upload the files from the kernel-devel rpm then. -- MST From greg at kroah.com Thu Aug 9 23:03:59 2007 From: greg at kroah.com (Greg KH) Date: Thu, 9 Aug 2007 23:03:59 -0700 Subject: [ofa-general] Re: [stable] [PATCH -stable] IB/core: Ignore membership bit in ib_find_pkey() In-Reply-To: <46B9ACF7.6030907@gmail.com> References: <46B9ACF7.6030907@gmail.com> Message-ID: <20070810060359.GA23664@kroah.com> On Wed, Aug 08, 2007 at 02:45:59PM +0300, Moni Shoua wrote: > ib_find_pkey() is used as a replacement for ib_find_cached_pkey(), and > the original function ignored the membership bit when searching for a > P_Key, so ib_find_pkey() should ignore the bit too. > > In particular, IPoIB turns on the P_Key membership bit of limited > membership P_Keys when creating a child interface and looks for the > full membership P_key. This broke if a port was a partial member of a > partition when IPoIB switched from ib_find_cached_pkey() to > ib_find_pkey(), and this change fixes things again. > > Signed-off-by: Moni Shoua > Signed-off-by: Roland Dreier I don't see this in Linus's tree, is there some reason it has not been accepted into mainline yet? thanks, greg k-h From yangdong at ncic.ac.cn Thu Aug 9 23:13:26 2007 From: yangdong at ncic.ac.cn (yangdong) Date: Fri, 10 Aug 2007 14:13:26 +0800 Subject: [ofa-general] perftest in OFED-1.2, why bandwidth tested using ib_rdma_bw is lower than that using ib_write_bw or ib_read_bw? Message-ID: <46BC0206.2060604@ncic.ac.cn> I get 526.499MB/s (peak bw) using ib_rdma_bw, but it is about 790MB/s using ib_read_bw or ib_write_bw? I dont know why gap is presented .. From vlad at lists.openfabrics.org Fri Aug 10 01:40:46 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 10 Aug 2007 01:40:46 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070810-0100 daily build status Message-ID: <20070810084046.BE096E6085C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From morgane.dautrich at valleypkg.com Fri Aug 10 03:27:31 2007 From: morgane.dautrich at valleypkg.com (Lilian Harper) Date: Fri, 10 Aug 2007 09:27:31 -0100 Subject: [ofa-general] Nice pharmacy store Message-ID: <01c7db30$acf50380$6a6b2055@morgane.dautrich> -------------- next part -------------- A non-text attachment was scrubbed... Name: jaja.gif Type: image/gif Size: 14049 bytes Desc: not available URL: From vlad at lists.openfabrics.org Fri Aug 10 02:51:04 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 10 Aug 2007 02:51:04 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070810-0200 daily build status Message-ID: <20070810095104.BE27EE6085C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070810-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From Jakovlevsdoa at ap0.com Fri Aug 10 05:03:35 2007 From: Jakovlevsdoa at ap0.com (Jakovlevs) Date: Fri, 10 Aug 2007 14:03:35 +0200 Subject: [ofa-general] Headlines Message-ID: <20070810120316.CD6D5E60821@openfabrics.org> A non-text attachment was scrubbed... Name: Headlines.fdf Type: application/octet-stream Size: 8013 bytes Desc: not available URL: From sashak at voltaire.com Fri Aug 10 05:24:50 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 10 Aug 2007 15:24:50 +0300 Subject: [ofa-general] Re: nightly osm_sim report 2007-08-10:normal completion In-Reply-To: References: Message-ID: <20070810122450.GB16267@sashak.voltaire.com> Hi Yevgeny, On 07:14 Fri 10 Aug , kliteyn at mellanox.co.il wrote: > > Failures: > 39 Pkey IS1-16.topo > 13 Pkey IS3-128.topo > 2 OsmTest IS1-16.topo > 1 Stability IS1-16.topo > 1 OsmTest IS3-loop.topo > 1 OsmStress IS1-16.topo > 1 Multicast IS3-loop.topo > 1 Multicast IS1-16.topo Is it something which requires attention? Sasha From marten.dautrich at upg.ru Fri Aug 10 07:51:48 2007 From: marten.dautrich at upg.ru (Dwight Fenton) Date: Fri, 10 Aug 2007 13:51:48 -0100 Subject: [ofa-general] Always ready for action Message-ID: <01c7db55$988effc0$54a9ec53@marten.dautrich> -------------- next part -------------- A non-text attachment was scrubbed... Name: jaja.gif Type: image/gif Size: 14353 bytes Desc: not available URL: From akepner at sgi.com Fri Aug 10 08:15:52 2007 From: akepner at sgi.com (akepner at sgi.com) Date: Fri, 10 Aug 2007 08:15:52 -0700 Subject: [ofa-general] Re: [PATCH] ib_umad unaligned access on ia64 In-Reply-To: References: <20070809203612.GA32445@sgi.com> <20070810012709.GG25325@sgi.com> Message-ID: <20070810151552.GL25325@sgi.com> On Thu, Aug 09, 2007 at 07:29:02PM -0700, Roland Dreier wrote: > ... > this is the commit I'm talking about; I think it went into 2.6.21 but > I'm not sure. Anyway it seems it should make sure that the tid > member is aligned to 8 bytes, or at least I think it should: > > commit fa7252ed4d92397baf30e4a144af95a33eaa925b Thanks for digging that up. Judging from the changelog, etc., it looks like this has been addressed. (I've been mostly using 2.6.16/sles10 kernels, so I'm still seeing it there.) -- Arthur From weiny2 at llnl.gov Fri Aug 10 09:00:19 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 10 Aug 2007 09:00:19 -0700 Subject: [ofa-general] [PATCH 1/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's In-Reply-To: References: <20070809190331.4088e85e.weiny2@llnl.gov> Message-ID: <20070810090019.35e9e985.weiny2@llnl.gov> On Thu, 9 Aug 2007 23:10:54 -0400 "Hal Rosenstock" wrote: > On 8/9/07, Ira Weiny wrote: > > >From 3abf9ac10ee90f7272969486704c2989d254a2ac Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Thu, 9 Aug 2007 16:07:38 -0700 > > Subject: [PATCH] Modify set_mthca_nodedesc to set the node_desc for any hca type found in sysfs. > > > > This has been tested for the MLX4 and mthca cards but not others. > > I think the iPath driver may need some change to make this work. I see the following in ipath_modify_device (OFED 1.2) if (device_modify_mask & IB_DEVICE_MODIFY_NODE_DESC) memcpy(device->node_desc, device_modify->node_desc, 64); So I hope this will work? But like I said it is not tested... Ira > > > > > Signed-off-by: Ira K. Weiny > > --- > > infiniband-diags/scripts/set_mthca_nodedesc.sh | 13 +++++++++++-- > > 1 files changed, 11 insertions(+), 2 deletions(-) > > > > diff --git a/infiniband-diags/scripts/set_mthca_nodedesc.sh b/infiniband-diags/scripts/set_mthca_nodedesc.sh > > index 618373f..4613de6 100755 > > --- a/infiniband-diags/scripts/set_mthca_nodedesc.sh > > +++ b/infiniband-diags/scripts/set_mthca_nodedesc.sh > > @@ -1,9 +1,18 @@ > > #!/bin/sh > > > > -# set the node_desc field in the mthca to the hostname > > +# set the node_desc field of any hca found to the defined hostname > > > > . /etc/sysconfig/network > > > > -echo -n "$HOSTNAME" >> /sys/class/infiniband/mthca0/node_desc > > +ib_sysfs="/sys/class/infiniband" > > + > > +for hca in `ls $ib_sysfs`; do > > + if [ -f $ib_sysfs/$hca/node_desc ]; then > > + echo -n "$HOSTNAME" >> $ib_sysfs/$hca/node_desc > > + else > > + logger -s "Failed to set node_desc for : $hca" > > + fi > > +done > > > > exit 0 > > + > > -- > > 1.5.2 > > > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > From ralph.campbell at qlogic.com Fri Aug 10 10:18:00 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Fri, 10 Aug 2007 10:18:00 -0700 Subject: [ofa-general] [PATCH 1/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's In-Reply-To: References: <20070809190331.4088e85e.weiny2@llnl.gov> Message-ID: <1186766280.6293.470.camel@brick.pathscale.com> On Thu, 2007-08-09 at 23:10 -0400, Hal Rosenstock wrote: > On 8/9/07, Ira Weiny wrote: > > >From 3abf9ac10ee90f7272969486704c2989d254a2ac Mon Sep 17 00:00:00 2001 > > From: Ira K. Weiny > > Date: Thu, 9 Aug 2007 16:07:38 -0700 > > Subject: [PATCH] Modify set_mthca_nodedesc to set the node_desc for any hca type found in sysfs. > > > > This has been tested for the MLX4 and mthca cards but not others. > > I think the iPath driver may need some change to make this work. Writing to /sys/class/infiniband/ipath0/node_desc will change the string returned by Get(NodeDesc) MADs so this sort of script should work. From rodolph.daughtee at stricker-service.dk Fri Aug 10 12:06:04 2007 From: rodolph.daughtee at stricker-service.dk (Deanne Malone) Date: Fri, 10 Aug 2007 18:06:04 -0100 Subject: [ofa-general] To be or not to be? Message-ID: <01c7db79$1dfa57e0$8fd93159@rodolph.daughtee> -------------- next part -------------- A non-text attachment was scrubbed... Name: jaja.gif Type: image/gif Size: 14151 bytes Desc: not available URL: From swise at opengridcomputing.com Fri Aug 10 12:50:07 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Fri, 10 Aug 2007 14:50:07 -0500 Subject: [ofa-general] [ GIT PULL ] ofed-1.2 - Chelsio Driver Fixes Message-ID: <1186775407.4095.8.camel@stevo-desktop> Vlad, Please pull the changes to fix ofed bugs 702 and 705 (705 is 1.2.c only). git://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2 and git://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2_c Thanks, Steve. The ofed_1_2_c branch patches: --------------------- commit 1baf63679a33bc19c7659acab049210bde44905d Author: Steve Wise Date: Fri Aug 10 08:19:40 2007 -0500 cxgb3: update firmware version to 4.6.0. This firmware fixes a handful of RDMA bugs. - updated required firmware to 4.6.0 - update required psram to 1.1.0 - changed cxgb3.ko version to 1.0-ofed Signed-off-by: Steve Wise diff --git a/kernel_patches/fixes/0028-cxgb3-fw-4.6.0 b/kernel_patches/fixes/0028-cxgb3-fw-4.6.0 new file mode 100644 index 0000000..176ed4a --- /dev/null +++ b/kernel_patches/fixes/0028-cxgb3-fw-4.6.0 @@ -0,0 +1,45 @@ +cxgb3: update firmware version to 4.6.0. + +From: Steve Wise + +This firmware fixes a handful of RDMA bugs. + +Signed-off-by: Steve Wise +--- + + drivers/net/cxgb3/common.h | 4 ++-- + drivers/net/cxgb3/version.h | 4 ++-- + 2 files changed, 4 insertions(+), 4 deletions(-) + +diff --git a/drivers/net/cxgb3/common.h b/drivers/net/cxgb3/common.h +index f77c919..0ff9cb5 100644 +--- a/drivers/net/cxgb3/common.h ++++ b/drivers/net/cxgb3/common.h +@@ -126,8 +126,8 @@ enum { /* adapter interrupt-maintaine + + enum { + TP_VERSION_MAJOR = 1, +- TP_VERSION_MINOR = 0, +- TP_VERSION_MICRO = 44 ++ TP_VERSION_MINOR = 1, ++ TP_VERSION_MICRO = 0 + }; + + #define S_TP_VERSION_MAJOR 16 +diff --git a/drivers/net/cxgb3/version.h b/drivers/net/cxgb3/version.h +index eb508bf..ef2405a 100644 +--- a/drivers/net/cxgb3/version.h ++++ b/drivers/net/cxgb3/version.h +@@ -35,10 +35,10 @@ #define __CHELSIO_VERSION_H + #define DRV_DESC "Chelsio T3 Network Driver" + #define DRV_NAME "cxgb3" + /* Driver version */ +-#define DRV_VERSION "1.0-ko" ++#define DRV_VERSION "1.0-ofed" + + /* Firmware version */ + #define FW_VERSION_MAJOR 4 +-#define FW_VERSION_MINOR 3 ++#define FW_VERSION_MINOR 6 + #define FW_VERSION_MICRO 0 + #endif /* __CHELSIO_VERSION_H */ commit d5acf47ef330570eb7ac2d715678e84a52865f8b Author: Steve Wise Date: Fri Aug 10 08:19:35 2007 -0500 Backport 2.6.22 eth_type_trans(). As of 2.6.22, eth_type_trans() sets skb->dev. Signed-off-by: Steve Wise diff --git a/kernel_addons/backport/2.6.11/include/linux/etherdevice.h b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.11/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.11_FC4/include/linux/etherdevice.h b/kernel_addons/backport/2.6.11_FC4/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.11_FC4/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.12/include/linux/etherdevice.h b/kernel_addons/backport/2.6.12/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.12/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.13/include/linux/etherdevice.h b/kernel_addons/backport/2.6.13/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.13/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/etherdevice.h b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.14/include/linux/etherdevice.h b/kernel_addons/backport/2.6.14/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.14/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.15/include/linux/etherdevice.h b/kernel_addons/backport/2.6.15/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.15/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.15_ubuntu606/include/linux/etherdevice.h b/kernel_addons/backport/2.6.15_ubuntu606/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.15_ubuntu606/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.16/include/linux/etherdevice.h b/kernel_addons/backport/2.6.16/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.16/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.16_sles10/include/linux/etherdevice.h b/kernel_addons/backport/2.6.16_sles10/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.16_sles10/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/etherdevice.h b/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.16_sles10_sp1/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.17/include/linux/etherdevice.h b/kernel_addons/backport/2.6.17/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.17/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.18/include/linux/etherdevice.h b/kernel_addons/backport/2.6.18/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.18/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.18_FC6/include/linux/etherdevice.h b/kernel_addons/backport/2.6.18_FC6/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.18_FC6/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.19/include/linux/etherdevice.h b/kernel_addons/backport/2.6.19/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.19/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.20/include/linux/etherdevice.h b/kernel_addons/backport/2.6.20/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.20/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.21/include/linux/etherdevice.h b/kernel_addons/backport/2.6.21/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.21/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/etherdevice.h b/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.5_sles9_sp3/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.9_U2/include/linux/etherdevice.h b/kernel_addons/backport/2.6.9_U2/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.9_U2/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.9_U3/include/linux/etherdevice.h b/kernel_addons/backport/2.6.9_U3/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.9_U3/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.9_U4/include/linux/etherdevice.h b/kernel_addons/backport/2.6.9_U4/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.9_U4/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif diff --git a/kernel_addons/backport/2.6.9_U5/include/linux/etherdevice.h b/kernel_addons/backport/2.6.9_U5/include/linux/etherdevice.h new file mode 100644 index 0000000..cd709f0 --- /dev/null +++ b/kernel_addons/backport/2.6.9_U5/include/linux/etherdevice.h @@ -0,0 +1,15 @@ +#ifndef BACKPORT_LINUX_ETHERDEVICE +#define BACKPORT_LINUX_ETHERDEVICE + +#include_next + +static inline unsigned short backport_eth_type_trans(struct sk_buff *skb, + struct net_device *dev) +{ + skb->dev = dev; + return eth_type_trans(skb, dev); +} + +#define eth_type_trans backport_eth_type_trans + +#endif From HNGUYEN at de.ibm.com Fri Aug 10 14:10:57 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Fri, 10 Aug 2007 23:10:57 +0200 Subject: [ofa-general] Re: OFED 1.2.5 (was 1.2.c) plans In-Reply-To: <46BB33C8.3080100@mellanox.co.il> Message-ID: Hi Tziporet! > We wish to have the final build with latest fixes next Monday > This means ALL changes must be ready till end of Sunday (for US end of > the week). We've tested ofed-1.2.5/c on following two ppc64 nodes: - rhel4.5, 2.6.16, 2.6.17, 2.6.18/rhel5, 2.6.19, 2.6.20, 2.6.21, 2.6.22 - flood ping with ifconfig down/up - ibv_ud/uc/rc_pingpong - mpi osutest bcast, bw Nam From rowland at cse.ohio-state.edu Fri Aug 10 16:03:26 2007 From: rowland at cse.ohio-state.edu (Shaun Rowland) Date: Fri, 10 Aug 2007 19:03:26 -0400 Subject: [ofa-general] mvapich2 mpd vs smpd In-Reply-To: <46BB1650.6@clustervision.com> References: <46BB1650.6@clustervision.com> Message-ID: <46BCEEBE.7050500@cse.ohio-state.edu> Louwrens Boonstra wrote: > Hi, > > For SGE tight integration we need the use of smpd rather than mpd. > Is it possible to have this as an ofed.conf configuration option? > For instance: > > MVAPICH2_CONF_mp="smpd" > > I don't mind unpacking the tar from the SRPM and patching the > make.mvapich.* files once in a while but it would save us a lot of time > if we could just specify this in the config. > > Many thanks in advance. Hi. Adding such an option to the SRPM is not a problem. We'll look into incorporating this type of choice into the next OFED release. -- Shaun Rowland rowland at cse.ohio-state.edu http://www.cse.ohio-state.edu/~rowland/ From sean.hefty at intel.com Fri Aug 10 17:22:29 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Fri, 10 Aug 2007 17:22:29 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> Message-ID: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> >I'd like to propose the following change as a simple solution for handling SA >scalability problems: > >Modify the ib_sa module to support an SA LID that's separate from the SM LID. > >This concept is supported by the spec through SA redirection; however, I >propose that we also allow the SA LID to be set manually by an administrator. Roland, Is the following an acceptable approach for this? Add a class device file to the ib_sa that allows setting all SA redirection parameters (SL, LID, PKey, QP - with processing similar to SRP's add_target). /sys/class/infiniband_sa/sa-mthca-0/redirect_sa Alternatively, I can use a module parameter to redirect only the SA LID. - Sean From kliteyn at mellanox.co.il Fri Aug 10 21:04:22 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 11 Aug 2007 07:04:22 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-11:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Wed_Aug_8_22:48:15_2007 [83d1f5ea339704f30e26918a897f9ead5a10678e] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=520 Pass=467 Fail=53 Pass: 39 Stability IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 38 OsmTest IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 39 Pkey IS1-16.topo 13 Pkey IS3-128.topo 1 OsmTest IS1-16.topo From Essau659 at is-consulting.biz Fri Aug 10 23:50:04 2007 From: Essau659 at is-consulting.biz (Essau Mosier) Date: Sat, 11 Aug 2007 08:50:04 +0200 Subject: [ofa-general] In other words, all lookups are done sequentially. Message-ID: <000c01c7dbe3$d8f766e0$1aa49850@famwks> H u g e N.e_w.s To Im-pact C Y_T*V Chi_na You+TV C_o,r,p-. Sym-bol: C_Y+T-V We h a'v-e a-lready s,e-e*n CYTV,'s mar_ket impa+ct b+efore clim'b_ing to o'v'e,r $2*.00 w+i-t+h n.e.w's*. P+ress R_elease: Ch*ina YouT.V's CnBo'o W_e b S+i.t e Ra+nks N o,.-1 on Micros'of't L_i*v+e Sear-ch Engi'ne C'nBoo Traffi.c I n_creases 4,9+% O,v e+r T_w*o M-onths R.e a d t.h,e ne ws, t-hink ab'out t-h+e im_pact, and j+u*m-p on t*h i-s firs_t th.ing Tom'orro,w mo+r ning! $.0.42 is a g_i*f+t at t,h+i.s price..+..,. Do y*o+u.r home+*work a,n d watc'h t_h_i's tra,de Mo nday mor+ning. -------------- next part -------------- An HTML attachment was scrubbed... URL: From keenegdfl at is-consulting.biz Fri Aug 10 23:50:03 2007 From: keenegdfl at is-consulting.biz (keloo keene) Date: Sat, 11 Aug 2007 08:50:03 +0200 Subject: [ofa-general] DCPP is authored by a very well respected crypto expert who also authored Scramdisk. Message-ID: <000c01c7dbe3$d82f0dd0$1aa49850@famwks> H-u'g'e N e*w,s To Imp-act C'Y'T.V Chi_na You+TV C-o'r*p-. Sy.mbol: C Y.T'V We h.a.v.e alrea.dy s.e+e'n C+YTV's mar-ket impa+ct befo,re cl+im bing to o_v+e.r $-2.00 w.i't,h n'e_w's.. Pres-s Release_: C,hina YouT-V's Cn,Boo W e*b S*i_t-e Ran ks N+o .*1 on Mi'crosof t L i.v+e S.earch Engin,e CnBo*o T.raffic Incre'as.es 4+9*% O_v.e+r T_w.o Mon ths R_e+a.d t-h,e ne-ws, thi-nk abou,t t'h e impac+t, and j_u-m_p on t'h,i-s firs.t th+ing Tomor.ro-w m'+orning! $0'.42 is a g_i,f+t at t h-i's pri.ce...+.. Do y*o,u r h',omework a'n_d w atch t+h+i's tr,ade M*onday m.orning. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vlad at lists.openfabrics.org Sat Aug 11 01:40:24 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 11 Aug 2007 01:40:24 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070811-0100 daily build status Message-ID: <20070811084024.2E92BE60859@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.18 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Sat Aug 11 02:49:50 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 11 Aug 2007 02:49:50 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070811-0200 daily build status Message-ID: <20070811094950.5FD69E60354@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070811-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From mst at dev.mellanox.co.il Sat Aug 11 11:13:30 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sat, 11 Aug 2007 21:13:30 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <200708061530.54392.hnguyen@linux.vnet.ibm.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061530.54392.hnguyen@linux.vnet.ibm.com> Message-ID: <20070811181330.GC17843@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > Hello Doug and Scott! > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > ehca backports for kernel.org kernels seem to be broken. > > 1. Does anyone care enough to fix them? If not we'll disable > > ehca in build for these kernels. > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > This would make it possible for us to add it to nightly builds. > Could you please provide Michael and Vladimir with a URL to download > above kernel source tree in order to perform daily build of ofed > code suite? This will help us to prevent build issues in very early > stage. > Thanks much in advance! > > Regards > Nam Nam, why can't you do this yourself? I think that IBM's the one that needs this most of all. -- MST From mst at dev.mellanox.co.il Sat Aug 11 11:21:20 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sat, 11 Aug 2007 21:21:20 +0300 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> Message-ID: <20070811182120.GE17843@mellanox.co.il> > Quoting Sean Hefty : > Subject: RE: [RFC] the never ending search for SA scalability > > >I'd like to propose the following change as a simple solution for handling SA > >scalability problems: > > > >Modify the ib_sa module to support an SA LID that's separate from the SM LID. > > > >This concept is supported by the spec through SA redirection; however, I > >propose that we also allow the SA LID to be set manually by an administrator. > > Roland, > > Is the following an acceptable approach for this? > > Add a class device file to the ib_sa that allows setting all SA redirection > parameters (SL, LID, PKey, QP - with processing similar to SRP's add_target). > > /sys/class/infiniband_sa/sa-mthca-0/redirect_sa > > Alternatively, I can use a module parameter to redirect only the SA LID. Hopefully, that would be something like sa/mthca0 so that one does not need to perform tricky string parsing to find the interface for a specific device. Further, won't you want to have this per-port? It's easy to imagine separate ports redirecting to different SA's. -- MST From kliteyn at mellanox.co.il Sat Aug 11 21:44:41 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 12 Aug 2007 07:44:41 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-12:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Wed_Aug_8_22:48:15_2007 [83d1f5ea339704f30e26918a897f9ead5a10678e] ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] Total=560 Pass=504 Fail=56 Pass: 42 Stability IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 42 Pkey IS1-16.topo 14 Pkey IS3-128.topo From erezz at voltaire.com Sun Aug 12 00:02:22 2007 From: erezz at voltaire.com (Erez Zilber) Date: Sun, 12 Aug 2007 10:02:22 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.c-11 is available In-Reply-To: <46B9DCC7.3060908@mellanox.co.il> References: <6C2C79E72C305246B504CBA17B5500C901563ABB@mtlexch01.mtl.com> <46B9C95E.6090108@voltaire.com> <46B9DCC7.3060908@mellanox.co.il> Message-ID: <46BEB07E.7020404@voltaire.com> Tziporet Koren wrote: > Erez Zilber wrote: > > Tziporet, > > > > > > Can you add the following as a known issue: > > > > > > "open-iscsi over TCP does not support usage of data digest on RedHat > EL/AS 4 up3, > > up4 & up5." > > > > Note that this comment should be added to OFED release notes, not to > > iSER release notes. This is not an iSER bug (it's a bug in the backport > > for open-iscsi). > > > > Thanks, > > Erez > > > > > Will do in next build > > tziporet > Tziporet, We were able to fix that problem, and Vlad already pulled the fix. Therefore, the known issue that I wanted to add is irrelevant. Erez From vlad at lists.openfabrics.org Sun Aug 12 01:39:14 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 12 Aug 2007 01:39:14 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070812-0100 daily build status Message-ID: <20070812083914.49519E60869@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Sun Aug 12 02:49:16 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 12 Aug 2007 02:49:16 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070812-0200 daily build status Message-ID: <20070812094916.1A847E60868@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070812-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From rmongetmic at sacramentoselect.com Sun Aug 12 03:31:33 2007 From: rmongetmic at sacramentoselect.com (Millard Pike) Date: Sun, 12 Aug 2007 05:31:33 -0500 Subject: [ofa-general] hi there General Message-ID: <41651436.05645473907600@sacramentoselect.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: oversaturation.png Type: image/png Size: 13036 bytes Desc: not available URL: From vlad at mellanox.co.il Sun Aug 12 03:57:50 2007 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Sun, 12 Aug 2007 13:57:50 +0300 Subject: [ofa-general] RE: [ GIT PULL ] ofed-1.2 - Chelsio Driver Fixes In-Reply-To: <1186775407.4095.8.camel@stevo-desktop> References: <1186775407.4095.8.camel@stevo-desktop> Message-ID: <6C2C79E72C305246B504CBA17B5500C9020BE646@mtlexch01.mtl.com> > -----Original Message----- > From: Steve Wise [mailto:swise at opengridcomputing.com] > Sent: Friday, August 10, 2007 10:50 PM > To: Vladimir Sokolovsky > Cc: ewg at lists.openfabrics.org; general at lists.openfabrics.org > Subject: [ GIT PULL ] ofed-1.2 - Chelsio Driver Fixes > > Vlad, > > Please pull the changes to fix ofed bugs 702 and 705 (705 is 1.2.c > only). > > git://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2 > > and > > git://git.openfabrics.org/~swise/ofed_1_2 ofed_1_2_c > > > > Thanks, > > Steve. > Done. Regards, Vladimir From eli at mellanox.co.il Sun Aug 12 04:42:34 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:42:34 +0300 Subject: [ofa-general] ipoib stateless offload patches Message-ID: <1186918954.5556.34.camel@mtls03> Following this email is a list of patches that utilize Mellanox's ConnectX stateless offload features. From eli at mellanox.co.il Sun Aug 12 04:42:51 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:42:51 +0300 Subject: [ofa-general] [PATCH] ipoib - high dma support Message-ID: <1186918971.5556.35.camel@mtls03> Add high dma support to ipoib Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-02 14:08:29.000000000 +0300 @@ -46,6 +46,7 @@ #include #include +#include #include @@ -1072,6 +1073,7 @@ static struct net_device *ipoib_add_port { struct ipoib_dev_priv *priv; int result = -ENOMEM; + struct pci_dev *pdev; priv = ipoib_intf_alloc(format); if (!priv) @@ -1079,6 +1081,10 @@ static struct net_device *ipoib_add_port SET_NETDEV_DEV(priv->dev, hca->dma_device); + pdev = to_pci_dev(hca->dma_device); + if (pdev->dma_mask & DMA_64BIT_MASK) + priv->dev->features |= NETIF_F_HIGHDMA; + result = ib_query_pkey(hca, port, 0, &priv->pkey); if (result) { printk(KERN_WARNING "%s: ib_query_pkey port %d failed (ret = %d)\n", From eli at mellanox.co.il Sun Aug 12 04:43:33 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:43:33 +0300 Subject: [ofa-general] [PATCH] ipoib - scatter/gather suopport Message-ID: <1186919013.5556.36.camel@mtls03> Add scatter/gather support to ipoib Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-02 14:08:29.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-02 14:31:02.000000000 +0300 @@ -1083,7 +1083,7 @@ static struct net_device *ipoib_add_port pdev = to_pci_dev(hca->dma_device); if (pdev->dma_mask & DMA_64BIT_MASK) - priv->dev->features |= NETIF_F_HIGHDMA; + priv->dev->features |= NETIF_F_HIGHDMA | NETIF_F_SG; result = ib_query_pkey(hca, port, 0, &priv->pkey); if (result) { Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-02 14:31:02.000000000 +0300 @@ -120,9 +120,14 @@ struct ipoib_rx_buf { u64 mapping; }; +struct ipoib_mapping_st { + u64 addr; + u16 size; +}; + struct ipoib_tx_buf { struct sk_buff *skb; - u64 mapping; + struct ipoib_mapping_st mapping[MAX_SKB_FRAGS + 1]; }; struct ib_cm_id; @@ -269,7 +274,7 @@ struct ipoib_dev_priv { struct ipoib_tx_buf *tx_ring; unsigned tx_head; unsigned tx_tail; - struct ib_sge tx_sge; + struct ib_sge tx_sge[MAX_SKB_FRAGS + 1]; struct ib_send_wr tx_wr; struct ib_wc ibwc[IPOIB_NUM_WC]; Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-02 11:44:43.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-02 14:31:02.000000000 +0300 @@ -239,6 +239,24 @@ repost: "for buf %d\n", wr_id); } +static int dma_unmap_list(struct ib_device *ca, struct ipoib_mapping_st *map, + u16 n) +{ + int i, len; + + BUG_ON(!n); + ib_dma_unmap_single(ca, map[0].addr, map[0].size, DMA_TO_DEVICE); + len = map[0].size; + + for (i = 1; i < n; ++i) { + ib_dma_unmap_page(ca, map[i].addr, map[i].size, + DMA_TO_DEVICE); + len += map[i].size; + } + + return len; +} + static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -256,12 +274,9 @@ static void ipoib_ib_handle_tx_wc(struct } tx_req = &priv->tx_ring[wr_id]; - - ib_dma_unmap_single(priv->ca, tx_req->mapping, - tx_req->skb->len, DMA_TO_DEVICE); - + priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags + 1); ++priv->stats.tx_packets; - priv->stats.tx_bytes += tx_req->skb->len; dev_kfree_skb_any(tx_req->skb); @@ -343,13 +358,17 @@ void ipoib_ib_completion(struct ib_cq *c static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 addr, int len) + struct ipoib_mapping_st *mapping, int ngather) { struct ib_send_wr *bad_wr; + int i; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + for (i = 0; i < ngather; ++i) { + priv->tx_sge[i].addr = mapping[i].addr; + priv->tx_sge[i].length = mapping[i].size; + } + priv->tx_wr.num_sge = ngather; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; @@ -362,7 +381,9 @@ void ipoib_send(struct net_device *dev, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; + struct skb_frag_struct *frag; u64 addr; + unsigned short i; if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -385,21 +406,32 @@ void ipoib_send(struct net_device *dev, */ tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, + + addr = ib_dma_map_single(priv->ca, skb->data, skb_headlen(skb), DMA_TO_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); return; } - tx_req->mapping = addr; + tx_req->mapping[0].addr = addr; + tx_req->mapping[0].size = skb_headlen(skb); + + for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { + frag = &skb_shinfo(skb)->frags[i]; + addr = ib_dma_map_page(priv->ca, frag->page, frag->page_offset, + frag->size, DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(priv->ca, addr))) + goto map_err; + + tx_req->mapping[i + 1].addr = addr; + tx_req->mapping[i + 1].size = frag->size; + } if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, addr, skb->len))) { + address->ah, qpn, tx_req->mapping, skb_shinfo(skb)->nr_frags + 1))) { ipoib_warn(priv, "post_send failed\n"); - ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); - dev_kfree_skb_any(skb); + goto map_err; } else { dev->trans_start = jiffies; @@ -412,6 +444,11 @@ void ipoib_send(struct net_device *dev, set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); } } + return; + +map_err: + dma_unmap_list(priv->ca, tx_req->mapping, i + 1); + dev_kfree_skb_any(skb); } static void __ipoib_reap_ah(struct net_device *dev) @@ -604,10 +641,8 @@ int ipoib_ib_dev_stop(struct net_device while ((int) priv->tx_tail - (int) priv->tx_head < 0) { tx_req = &priv->tx_ring[priv->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, - tx_req->mapping, - tx_req->skb->len, - DMA_TO_DEVICE); + dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags + 1); dev_kfree_skb_any(tx_req->skb); ++priv->tx_tail; } Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-02 14:31:02.000000000 +0300 @@ -149,14 +149,14 @@ int ipoib_transport_dev_init(struct net_ .cap = { .max_send_wr = ipoib_sendq_size, .max_recv_wr = ipoib_recvq_size, - .max_send_sge = 1, + .max_send_sge = MAX_SKB_FRAGS + 1, .max_recv_sge = 1 }, .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_UD }; - int ret, size; + int ret, size, i; priv->pd = ib_alloc_pd(priv->ca); if (IS_ERR(priv->pd)) { @@ -197,11 +197,11 @@ int ipoib_transport_dev_init(struct net_ priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - priv->tx_sge.lkey = priv->mr->lkey; + for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) + priv->tx_sge[i].lkey = priv->mr->lkey; priv->tx_wr.opcode = IB_WR_SEND; - priv->tx_wr.sg_list = &priv->tx_sge; - priv->tx_wr.num_sge = 1; + priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; return 0; Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-02 11:44:43.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-02 14:31:02.000000000 +0300 @@ -495,8 +495,8 @@ static inline int post_send(struct ipoib { struct ib_send_wr *bad_wr; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + priv->tx_sge[0].addr = addr; + priv->tx_sge[0].length = len; priv->tx_wr.wr_id = wr_id; @@ -537,7 +537,7 @@ void ipoib_cm_send(struct net_device *de return; } - tx_req->mapping = addr; + tx_req->mapping[0].addr = addr; if (unlikely(post_send(priv, tx, tx->tx_head & (ipoib_sendq_size - 1), addr, skb->len))) { @@ -577,7 +577,7 @@ static void ipoib_cm_handle_tx_wc(struct tx_req = &tx->tx_ring[wr_id]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, DMA_TO_DEVICE); + ib_dma_unmap_single(priv->ca, tx_req->mapping[0].addr, tx_req->skb->len, DMA_TO_DEVICE); /* FIXME: is this right? Shouldn't we only increment on success? */ ++priv->stats.tx_packets; @@ -981,7 +981,7 @@ static void ipoib_cm_tx_destroy(struct i if (p->tx_ring) { while ((int) p->tx_tail - (int) p->tx_head < 0) { tx_req = &p->tx_ring[p->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, + ib_dma_unmap_single(priv->ca, tx_req->mapping[0].addr, tx_req->skb->len, DMA_TO_DEVICE); dev_kfree_skb_any(tx_req->skb); ++p->tx_tail; From eli at mellanox.co.il Sun Aug 12 04:43:48 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:43:48 +0300 Subject: [ofa-general] [PATCH] ib_mthca checksum offload support Message-ID: <1186919028.5556.37.camel@mtls03> Add checksum offload support Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-02 17:15:41.000000000 +0300 @@ -1377,6 +1377,9 @@ int mthca_INIT_HCA(struct mthca_dev *dev MTHCA_PUT(inbox, param->uarc_base, INIT_HCA_UAR_CTX_BASE_OFFSET); } + if (dev->device_cap_flags & IB_DEVICE_IP_CSUM) + *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) |= cpu_to_be32(7 << 3); + err = mthca_cmd(dev, mailbox->dma, 0, 0, CMD_INIT_HCA, HZ, status); mthca_free_mailbox(dev, mailbox); Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-02 17:15:41.000000000 +0300 @@ -103,6 +103,7 @@ enum { DEV_LIM_FLAG_RAW_IPV6 = 1 << 4, DEV_LIM_FLAG_RAW_ETHER = 1 << 5, DEV_LIM_FLAG_SRQ = 1 << 6, + DEV_LIM_FLAG_IPOIB_CSUM = 1 << 7, DEV_LIM_FLAG_BAD_PKEY_CNTR = 1 << 8, DEV_LIM_FLAG_BAD_QKEY_CNTR = 1 << 9, DEV_LIM_FLAG_MW = 1 << 16, Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-02 17:15:41.000000000 +0300 @@ -646,6 +646,9 @@ static inline int mthca_poll_one(struct entry->pkey_index = be32_to_cpu(cqe->imm_etype_pkey_eec) >> 16; entry->wc_flags |= be16_to_cpu(cqe->sl_g_mlpath) & 0x80 ? IB_WC_GRH : 0; + entry->checksum_ok = (be16_to_cpu(cqe->sl_g_mlpath) & 0x100) != 0; + entry->checksum = (be32_to_cpu(cqe->rqpn) >> 24) | + ((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00); } entry->status = IB_WC_SUCCESS; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-02 17:15:41.000000000 +0300 @@ -289,6 +289,10 @@ static int mthca_dev_lim(struct mthca_de if (dev_lim->flags & DEV_LIM_FLAG_SRQ) mdev->mthca_flags |= MTHCA_FLAG_SRQ; + if (mthca_is_memfree(mdev)) + if (dev_lim->flags & DEV_LIM_FLAG_IPOIB_CSUM) + mdev->device_cap_flags |= IB_DEVICE_IP_CSUM; + return 0; } Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-02 17:15:41.000000000 +0300 @@ -2024,6 +2024,10 @@ int mthca_arbel_post_send(struct ib_qp * cpu_to_be32(MTHCA_NEXT_CQ_UPDATE) : 0) | ((wr->send_flags & IB_SEND_SOLICITED) ? cpu_to_be32(MTHCA_NEXT_SOLICIT) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_IP_CSUM) : 0) | + ((wr->send_flags & IB_SEND_UDP_TCP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_TCP_UDP_CSUM) : 0) | cpu_to_be32(1); if (wr->opcode == IB_WR_SEND_WITH_IMM || wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-02 17:14:45.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-02 17:15:41.000000000 +0300 @@ -38,14 +38,15 @@ #include enum { - MTHCA_NEXT_DBD = 1 << 7, - MTHCA_NEXT_FENCE = 1 << 6, - MTHCA_NEXT_CQ_UPDATE = 1 << 3, - MTHCA_NEXT_EVENT_GEN = 1 << 2, - MTHCA_NEXT_SOLICIT = 1 << 1, - - MTHCA_MLX_VL15 = 1 << 17, - MTHCA_MLX_SLR = 1 << 16 + MTHCA_NEXT_DBD = 1 << 7, + MTHCA_NEXT_FENCE = 1 << 6, + MTHCA_NEXT_CQ_UPDATE = 1 << 3, + MTHCA_NEXT_EVENT_GEN = 1 << 2, + MTHCA_NEXT_SOLICIT = 1 << 1, + MTHCA_NEXT_IP_CSUM = 1 << 4, + MTHCA_NEXT_TCP_UDP_CSUM = 1 << 5, + MTHCA_MLX_VL15 = 1 << 17, + MTHCA_MLX_SLR = 1 << 16 }; enum { Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h =================================================================== --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-02 17:25:03.000000000 +0300 @@ -93,7 +93,8 @@ enum ib_device_cap_flags { IB_DEVICE_N_NOTIFY_CQ = (1<<14), IB_DEVICE_ZERO_STAG = (1<<15), IB_DEVICE_SEND_W_INV = (1<<16), - IB_DEVICE_MEM_WINDOW = (1<<17) + IB_DEVICE_MEM_WINDOW = (1<<17), + IB_DEVICE_IP_CSUM = (1<<18) }; enum ib_atomic_cap { @@ -429,6 +430,8 @@ struct ib_wc { u8 sl; u8 dlid_path_bits; u8 port_num; /* valid only for DR SMPs on switches */ + u8 checksum_ok; + u16 checksum; }; enum ib_cq_notify_flags { @@ -613,7 +616,9 @@ enum ib_send_flags { IB_SEND_FENCE = 1, IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), - IB_SEND_INLINE = (1<<3) + IB_SEND_INLINE = (1<<3), + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_UDP_TCP_CSUM = (1<<5) }; struct ib_sge { From eli at mellanox.co.il Sun Aug 12 04:44:02 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:44:02 +0300 Subject: [ofa-general] [PATCH] mlx4 checksum offload Message-ID: <1186919042.5556.38.camel@mtls03> Add checksum support to mlx4 Signed-off-by: Ali Ayub Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/include/linux/mlx4/cq.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/cq.h 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/cq.h 2007-08-02 17:30:52.000000000 +0300 @@ -45,11 +45,11 @@ struct mlx4_cqe { u8 sl; u8 reserved1; __be16 rlid; - u32 reserved2; + __be32 ipoib_status; __be32 byte_cnt; __be16 wqe_index; __be16 checksum; - u8 reserved3[3]; + u8 reserved2[3]; u8 owner_sr_opcode; }; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/cq.c 2007-07-10 21:56:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c 2007-08-02 17:32:46.000000000 +0300 @@ -431,6 +431,8 @@ static int mlx4_ib_poll_one(struct mlx4_ wc->wc_flags |= be32_to_cpu(cqe->g_mlpath_rqpn) & 0x80000000 ? IB_WC_GRH : 0; wc->pkey_index = be32_to_cpu(cqe->immed_rss_invalid) >> 16; + wc->checksum_ok = (be32_to_cpu(cqe->ipoib_status) & 0x10000000) != 0; + wc->checksum = be16_to_cpu(cqe->checksum); } return 0; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/main.c 2007-08-02 11:44:43.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c 2007-08-02 17:35:08.000000000 +0300 @@ -99,6 +99,8 @@ static int mlx4_ib_query_device(struct i props->device_cap_flags |= IB_DEVICE_AUTO_PATH_MIG; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_UD_AV_PORT) props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; + if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + props->device_cap_flags |= IB_DEVICE_IP_CSUM; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -568,6 +570,9 @@ static void *mlx4_ib_add(struct mlx4_dev ibdev->ib_dev.detach_mcast = mlx4_ib_mcg_detach; ibdev->ib_dev.process_mad = mlx4_ib_process_mad; + if (ibdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + ibdev->ib_dev.flags |= IB_DEVICE_IP_CSUM; + if (init_node_data(ibdev)) goto err_map; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-02 11:44:43.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-02 17:38:37.000000000 +0300 @@ -1258,6 +1258,10 @@ int mlx4_ib_post_send(struct ib_qp *ibqp cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | (wr->send_flags & IB_SEND_SOLICITED ? cpu_to_be32(MLX4_WQE_CTRL_SOLICITED) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM) : 0) | + ((wr->send_flags & IB_SEND_UDP_TCP_CSUM) ? + cpu_to_be32(MLX4_WQE_CTRL_TCP_UDP_CSUM) : 0) | qp->sq_signal_bits; if (wr->opcode == IB_WR_SEND_WITH_IMM || Index: linux-2.6.23-rc1/include/linux/mlx4/qp.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/qp.h 2007-08-02 11:44:45.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/qp.h 2007-08-02 17:40:52.000000000 +0300 @@ -155,9 +155,11 @@ struct mlx4_qp_context { }; enum { - MLX4_WQE_CTRL_FENCE = 1 << 6, - MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, - MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_FENCE = 1 << 6, + MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, + MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_IP_CSUM = 1 << 4, + MLX4_WQE_CTRL_TCP_UDP_CSUM = 1 << 5, }; struct mlx4_wqe_ctrl_seg { Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.c 2007-08-02 11:44:44.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.c 2007-08-02 17:42:55.000000000 +0300 @@ -736,6 +736,9 @@ int mlx4_INIT_HCA(struct mlx4_dev *dev, MLX4_PUT(inbox, (u8) (PAGE_SHIFT - 12), INIT_HCA_UAR_PAGE_SZ_OFFSET); MLX4_PUT(inbox, param->log_uar_sz, INIT_HCA_LOG_UAR_SZ_OFFSET); + if (dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 3); + err = mlx4_cmd(dev, mailbox->dma, 0, 0, MLX4_CMD_INIT_HCA, 1000); if (err) From eli at mellanox.co.il Sun Aug 12 04:44:11 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:44:11 +0300 Subject: [ofa-general] [PATCH] ipoib checksum offload Message-ID: <1186919051.5556.39.camel@mtls03> Add high checksum offload support to ipoib Signed-off-by: Eli Cohen Signed-off-by: Ali Ayub --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-08 12:15:53.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-08 19:12:27.000000000 +0300 @@ -86,6 +86,7 @@ enum { IPOIB_MCAST_STARTED = 8, IPOIB_FLAG_NETIF_STOPPED = 9, IPOIB_FLAG_ADMIN_CM = 10, + IPOIB_FLAG_RX_CSUM = 11, IPOIB_MAX_BACKOFF_SECONDS = 16, @@ -298,6 +299,8 @@ struct ipoib_dev_priv { struct dentry *mcg_dentry; struct dentry *path_dentry; #endif + unsigned long long rx_csum_pkts_good; + unsigned long long rx_csum_pkts_err; }; struct ipoib_ah { Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-08 12:15:53.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-08 12:23:11.000000000 +0300 @@ -1252,6 +1252,9 @@ static ssize_t set_mode(struct device *d /* flush paths if we switch modes so that connections are restarted */ if (IPOIB_CM_SUPPORTED(dev->dev_addr) && !strcmp(buf, "connected\n")) { set_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); + /* on CM mode, turn off tx_csum offloading */ + dev->features &= ~NETIF_F_HW_CSUM; + priv->tx_wr.send_flags &= ~IB_SEND_UDP_TCP_CSUM; ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); ipoib_flush_paths(dev); Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-08 12:15:53.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-08 19:12:27.000000000 +0300 @@ -37,6 +37,7 @@ #include #include +#include #include @@ -231,6 +232,20 @@ static void ipoib_ib_handle_rx_wc(struct skb->dev = dev; /* XXX get correct PACKET_ type here */ skb->pkt_type = PACKET_HOST; + + /* check rx csum */ + if (test_bit(IPOIB_FLAG_RX_CSUM, &priv->flags)) + if (likely(ntohs(skb->protocol) == ETH_P_IP && + /* no IP header options */ + ip_hdr(skb)->ihl == 5 && + wc->checksum == 0xffff)) { + if (likely(wc->checksum_ok)) { + skb->ip_summed = CHECKSUM_UNNECESSARY; + ++priv->rx_csum_pkts_good; + } else + ++priv->rx_csum_pkts_err; + } + netif_receive_skb(skb); repost: Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-08 12:15:53.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-08 19:14:12.000000000 +0300 @@ -1068,6 +1068,65 @@ int ipoib_add_pkey_attr(struct net_devic return device_create_file(&dev->dev, &dev_attr_pkey); } +static ssize_t show_tx_csum(struct device *d, struct device_attribute *attr, + char *buf) +{ + struct net_device *dev = to_net_dev(d); + + return sprintf(buf, dev->features & NETIF_F_HW_CSUM ? "on\n" : "off\n"); +} + +static int set_tx_csum(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (test_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags)) + return -EINVAL; + + if (!(priv->ca->flags & IB_DEVICE_IP_CSUM)) + return -EINVAL; + + dev->features |= NETIF_F_HW_CSUM; + priv->tx_wr.send_flags |= IB_SEND_UDP_TCP_CSUM | IB_SEND_IP_CSUM; + return 0; +} + +static DEVICE_ATTR(tx_csum, S_IWUSR | S_IRUGO, show_tx_csum, NULL); + +static ssize_t show_rx_csum(struct device *d, + struct device_attribute *attr, + char *buf) +{ + struct ipoib_dev_priv *priv = netdev_priv(to_net_dev(d)); + + return sprintf(buf, test_bit(IPOIB_FLAG_RX_CSUM, &priv->flags) ? "on\n" : "off\n"); +} + +static void set_rx_csum(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (!(priv->ca->flags & IB_DEVICE_IP_CSUM)) + return; + + set_bit(IPOIB_FLAG_RX_CSUM, &priv->flags); +} + +static DEVICE_ATTR(rx_csum, S_IWUSR | S_IRUGO, show_rx_csum, NULL); + +static ssize_t show_rx_csum_stats(struct device *d, struct device_attribute *attr, + char *buf) +{ + struct ipoib_dev_priv *priv = netdev_priv(to_net_dev(d)); + + return sprintf(buf, "csum_pkts_good %llu\n" + "csum_pkts_err %llu\n", + priv->rx_csum_pkts_good, + priv->rx_csum_pkts_err); +} + +static DEVICE_ATTR(rx_csum_stats, S_IRUGO, show_rx_csum_stats, NULL); + static struct net_device *ipoib_add_port(const char *format, struct ib_device *hca, u8 port) { @@ -1083,7 +1142,7 @@ static struct net_device *ipoib_add_port pdev = to_pci_dev(hca->dma_device); if (pdev->dma_mask & DMA_64BIT_MASK) - priv->dev->features |= NETIF_F_HIGHDMA | NETIF_F_SG; + priv->dev->features |= NETIF_F_HIGHDMA; result = ib_query_pkey(hca, port, 0, &priv->pkey); if (result) { @@ -1127,6 +1186,11 @@ static struct net_device *ipoib_add_port goto event_failed; } + if (!set_tx_csum(priv->dev)) + priv->dev->features |= NETIF_F_SG; + + set_rx_csum(priv->dev); + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", @@ -1144,6 +1208,12 @@ static struct net_device *ipoib_add_port goto sysfs_failed; if (device_create_file(&priv->dev->dev, &dev_attr_delete_child)) goto sysfs_failed; + if (device_create_file(&priv->dev->dev, &dev_attr_tx_csum)) + goto sysfs_failed; + if (device_create_file(&priv->dev->dev, &dev_attr_rx_csum)) + goto sysfs_failed; + if (device_create_file(&priv->dev->dev, &dev_attr_rx_csum_stats)) + goto sysfs_failed; return priv->dev; From eli at mellanox.co.il Sun Aug 12 04:44:19 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:44:19 +0300 Subject: [ofa-general] [PATCH] mlx4_ib QP max msg size Message-ID: <1186919059.5556.40.camel@mtls03> Configure QP's max message size according to values queried by query dev cap. Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-08 18:19:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-08 18:25:42.000000000 +0300 @@ -737,8 +737,9 @@ static int __mlx4_ib_modify_qp(struct ib } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { @@ -746,7 +747,8 @@ static int __mlx4_ib_modify_qp(struct ib attr->path_mtu); goto out; } - context->mtu_msgmax = (attr->path_mtu << 5) | 31; + context->mtu_msgmax = (attr->path_mtu << 5) | + ilog2(dev->dev->caps.max_msg_sz); } if (qp->rq.wqe_cnt) From eli at mellanox.co.il Sun Aug 12 04:44:27 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 12 Aug 2007 14:44:27 +0300 Subject: [ofa-general] [PATCH] ipoib large send offload Message-ID: <1186919067.5556.41.camel@mtls03> Add LSO supprt to ipoib Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-09 08:56:09.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-09 09:33:19.000000000 +0300 @@ -705,7 +705,13 @@ static int ipoib_start_xmit(struct sk_bu goto out; } - ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(skb->dst->neighbour->ha)); + if (skb_is_gso(skb)) + ipoib_send_gso(dev, skb, neigh->ah, + IPOIB_QPN(skb->dst->neighbour->ha)); + else + ipoib_send(dev, skb, neigh->ah, + IPOIB_QPN(skb->dst->neighbour->ha)); + goto out; } @@ -1186,9 +1192,13 @@ static struct net_device *ipoib_add_port goto event_failed; } - if (!set_tx_csum(priv->dev)) + if (!set_tx_csum(priv->dev)) { priv->dev->features |= NETIF_F_SG; + if (priv->ca->flags & IB_DEVICE_TCP_GSO) + priv->dev->features |= NETIF_F_TSO; + } + set_rx_csum(priv->dev); result = register_netdev(priv->dev); Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.c 2007-08-09 08:56:08.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.c 2007-08-09 08:56:11.000000000 +0300 @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.h 2007-08-09 08:41:54.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.h 2007-08-09 08:56:11.000000000 +0300 @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { Index: linux-2.6.23-rc1/drivers/net/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/main.c 2007-08-09 08:41:54.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/main.c 2007-08-09 08:56:11.000000000 +0300 @@ -158,6 +158,7 @@ static int __devinit mlx4_dev_cap(struct dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/main.c 2007-08-09 08:56:08.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c 2007-08-09 08:56:11.000000000 +0300 @@ -101,6 +101,8 @@ static int mlx4_ib_query_device(struct i props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_GSO; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -572,6 +574,8 @@ static void *mlx4_ib_add(struct mlx4_dev if (ibdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) ibdev->ib_dev.flags |= IB_DEVICE_IP_CSUM; + if (ibdev->dev->caps.max_gso_sz) + ibdev->ib_dev.flags |= IB_DEVICE_TCP_GSO; if (init_node_data(ibdev)) goto err_map; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-09 08:56:10.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-09 09:03:35.000000000 +0300 @@ -65,6 +65,7 @@ struct mlx4_ib_sqp { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -740,7 +741,8 @@ static int __mlx4_ib_modify_qp(struct ib if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; else if (ibqp->qp_type == IB_QPT_UD) - context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1312,6 +1314,28 @@ int mlx4_ib_post_send(struct ib_qp *ibqp set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + int halign; + + memcpy(((struct mlx4_lso_seg *)wqe)->header, + wr->wr.ud.header, wr->wr.ud.hlen); + wmb(); + ((struct mlx4_lso_seg *)wqe)->mss_hdr_size = + cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) << 16) | + wr->wr.ud.hlen); + + halign = ALIGN(wr->wr.ud.hlen, 16); + wqe += halign; + size += halign >> 4; + + if (unlikely(wr->num_sge > qp->sq.max_gs - (halign >> 4))) { + err = -EINVAL; + *bad_wr = wr; + goto out; + } + } + break; case IB_QPT_SMI: @@ -1365,6 +1389,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp ctrl->owner_opcode = mlx4_ib_opcode[wr->opcode] | (ind & qp->sq.wqe_cnt ? cpu_to_be32(1 << 31) : 0); + /* * We can improve latency by not stamping the last * send queue WQE until after ringing the doorbell, so Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-09 08:56:09.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-09 08:56:11.000000000 +0300 @@ -375,6 +375,10 @@ int ipoib_add_pkey_attr(struct net_devic void ipoib_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_ah *address, u32 qpn); + +void ipoib_send_gso(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn); + void ipoib_reap_ah(struct work_struct *work); void ipoib_flush_paths(struct net_device *dev); Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-09 08:56:09.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-09 09:33:01.000000000 +0300 @@ -38,6 +38,7 @@ #include #include #include +#include #include @@ -255,15 +256,22 @@ repost: } static int dma_unmap_list(struct ib_device *ca, struct ipoib_mapping_st *map, - u16 n) + u16 n, int gso) { - int i, len; + int i, len, first; BUG_ON(!n); - ib_dma_unmap_single(ca, map[0].addr, map[0].size, DMA_TO_DEVICE); - len = map[0].size; + if (!gso) { + ib_dma_unmap_single(ca, map[0].addr, map[0].size, DMA_TO_DEVICE); + len = map[0].size; + first = 1; + } + else { + len = 0; + first = 0; + } - for (i = 1; i < n; ++i) { + for (i = first; i < n; ++i) { ib_dma_unmap_page(ca, map[i].addr, map[i].size, DMA_TO_DEVICE); len += map[i].size; @@ -282,6 +290,7 @@ static void ipoib_ib_handle_tx_wc(struct ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); + if (unlikely(wr_id >= ipoib_sendq_size)) { ipoib_warn(priv, "send completion event with wrid %d (> %d)\n", wr_id, ipoib_sendq_size); @@ -289,8 +298,14 @@ static void ipoib_ib_handle_tx_wc(struct } tx_req = &priv->tx_ring[wr_id]; - priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, - skb_shinfo(tx_req->skb)->nr_frags + 1); + + if (skb_is_gso(tx_req->skb)) + priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags, 1); + else + priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags + 1, 0); + ++priv->stats.tx_packets; dev_kfree_skb_any(tx_req->skb); @@ -373,7 +388,8 @@ void ipoib_ib_completion(struct ib_cq *c static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - struct ipoib_mapping_st *mapping, int ngather) + struct ipoib_mapping_st *mapping, int ngather, + void *lso_header, int h_len) { struct ib_send_wr *bad_wr; int i; @@ -388,9 +404,89 @@ static inline int post_send(struct ipoib priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (lso_header) { + priv->tx_wr.wr.ud.mss = priv->dev->mtu; + priv->tx_wr.wr.ud.header = lso_header; + priv->tx_wr.wr.ud.hlen = h_len; + priv->tx_wr.opcode = IB_WR_LSO; + } + else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } + +void ipoib_send_gso(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ipoib_tx_buf *tx_req; + struct skb_frag_struct *frag; + u64 addr; + unsigned short i; + + ipoib_dbg_data(priv, "sending gso packet, length=%d address=%p qpn=0x%06x\n", + skb->len, address, qpn); + + if (unlikely((skb_headlen(skb) - IPOIB_ENCAP_LEN) != + ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2))) { + ipoib_warn(priv, "headlen (%d) does not match ip (%d)and " + "tcp headers(%d), dropping skb\n", + skb_headlen(skb) - IPOIB_ENCAP_LEN, ip_hdr(skb)->ihl << 2, + tcp_hdr(skb)->doff << 2); + ++priv->stats.tx_errors; + dev_kfree_skb_any(skb); + return; + } + + /* + * We put the skb into the tx_ring _before_ we call post_send() + * because it's entirely possible that the completion handler will + * run before we execute anything after the post_send(). That + * means we have to make sure everything is properly recorded and + * our state is consistent before we call post_send(). + */ + tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; + tx_req->skb = skb; + + for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { + frag = &skb_shinfo(skb)->frags[i]; + addr = ib_dma_map_page(priv->ca, frag->page, frag->page_offset, + frag->size, DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(priv->ca, addr))) + goto map_err; + + tx_req->mapping[i].addr = addr; + tx_req->mapping[i].size = frag->size; +// printk("%s: [%d] addr = 0x%llx, size = %d\n", __func__, i, addr, frag->size); + } + + if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), + address->ah, qpn, tx_req->mapping, + skb_shinfo(skb)->nr_frags, + skb->data, skb_headlen(skb)))) { + ipoib_warn(priv, "post_send failed\n"); + goto map_err; + } else { + dev->trans_start = jiffies; + + address->last_send = priv->tx_head; + ++priv->tx_head; + + if (priv->tx_head - priv->tx_tail == ipoib_sendq_size) { + ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); + netif_stop_queue(dev); + set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + } + } + return; + +map_err: + dma_unmap_list(priv->ca, tx_req->mapping, i, 1); + dev_kfree_skb_any(skb); +} + void ipoib_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_ah *address, u32 qpn) { @@ -444,7 +540,7 @@ void ipoib_send(struct net_device *dev, } if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, tx_req->mapping, skb_shinfo(skb)->nr_frags + 1))) { + address->ah, qpn, tx_req->mapping, skb_shinfo(skb)->nr_frags + 1, NULL, 0))) { ipoib_warn(priv, "post_send failed\n"); goto map_err; } else { @@ -462,7 +558,7 @@ void ipoib_send(struct net_device *dev, return; map_err: - dma_unmap_list(priv->ca, tx_req->mapping, i + 1); + dma_unmap_list(priv->ca, tx_req->mapping, i + 1, 0); dev_kfree_skb_any(skb); } @@ -657,7 +753,7 @@ int ipoib_ib_dev_stop(struct net_device tx_req = &priv->tx_ring[priv->tx_tail & (ipoib_sendq_size - 1)]; dma_unmap_list(priv->ca, tx_req->mapping, - skb_shinfo(tx_req->skb)->nr_frags + 1); + skb_shinfo(tx_req->skb)->nr_frags + 1, skb_is_gso(tx_req->skb)); dev_kfree_skb_any(tx_req->skb); ++priv->tx_tail; } Index: linux-2.6.23-rc1/include/linux/mlx4/device.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/device.h 2007-08-09 08:41:54.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/device.h 2007-08-09 08:56:11.000000000 +0300 @@ -177,6 +177,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { Index: linux-2.6.23-rc1/include/linux/mlx4/qp.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/qp.h 2007-08-09 08:56:08.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/qp.h 2007-08-09 08:56:11.000000000 +0300 @@ -215,6 +215,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h =================================================================== --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-08-09 08:56:08.000000000 +0300 +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-09 08:56:11.000000000 +0300 @@ -94,7 +94,8 @@ enum ib_device_cap_flags { IB_DEVICE_ZERO_STAG = (1<<15), IB_DEVICE_SEND_W_INV = (1<<16), IB_DEVICE_MEM_WINDOW = (1<<17), - IB_DEVICE_IP_CSUM = (1<<18) + IB_DEVICE_IP_CSUM = (1<<18), + IB_DEVICE_TCP_GSO = (1<<19) }; enum ib_atomic_cap { @@ -606,6 +607,7 @@ enum ib_wr_opcode { IB_WR_RDMA_WRITE, IB_WR_RDMA_WRITE_WITH_IMM, IB_WR_SEND, + IB_WR_LSO, IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, @@ -648,6 +650,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-09 08:56:07.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-09 08:56:11.000000000 +0300 @@ -200,7 +200,6 @@ int ipoib_transport_dev_init(struct net_ for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) priv->tx_sge[i].lkey = priv->mr->lkey; - priv->tx_wr.opcode = IB_WR_SEND; priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; From sashak at voltaire.com Sun Aug 12 05:16:30 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 12 Aug 2007 15:16:30 +0300 Subject: [ofa-general] Re: [PATCH 1/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's In-Reply-To: <20070809190331.4088e85e.weiny2@llnl.gov> References: <20070809190331.4088e85e.weiny2@llnl.gov> Message-ID: <20070812121630.GC16267@sashak.voltaire.com> On 19:03 Thu 09 Aug , Ira Weiny wrote: > From 3abf9ac10ee90f7272969486704c2989d254a2ac Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 9 Aug 2007 16:07:38 -0700 > Subject: [PATCH] Modify set_mthca_nodedesc to set the node_desc for any hca type found in sysfs. > > This has been tested for the MLX4 and mthca cards but not others. > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Sun Aug 12 05:16:56 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 12 Aug 2007 15:16:56 +0300 Subject: [ofa-general] Re: [PATCH 2/2] infiniband-diags/set_mthca_nodedesc.sh: change to work for all HCA's In-Reply-To: <20070809190332.3256988e.weiny2@llnl.gov> References: <20070809190332.3256988e.weiny2@llnl.gov> Message-ID: <20070812121656.GD16267@sashak.voltaire.com> On 19:03 Thu 09 Aug , Ira Weiny wrote: > From 3c78f9f7797a9029b33a1a80476f82929f8a7ad7 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 9 Aug 2007 18:51:09 -0700 > Subject: [PATCH] Change the name of set_mthca_nodedesc.sh to set_nodedesc.sh because it should > > work for all HCA types now. > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From jackm at dev.mellanox.co.il Sun Aug 12 05:36:46 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Sun, 12 Aug 2007 15:36:46 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> Message-ID: <200708121536.47152.jackm@dev.mellanox.co.il> On Thursday 09 August 2007 20:29, Roland Dreier wrote: > > +++ b/SRC.txt > > @@ -0,0 +1,138 @@ > > +Here's some documentation on Scalable Reliable Connections. > > > + ibv_create_src_srq; > > ibv_modify_srq; > > ibv_query_srq; > > ibv_destroy_srq; > > @@ -35,6 +36,8 @@ IBVERBS_1.0 { > > ibv_destroy_ah; > > ibv_attach_mcast; > > ibv_detach_mcast; > > + ibv_open_src_domain; > > + ibv_close_src_domain; > > Also, if we're going to change the signature of the create_srq method: > > > struct ibv_srq * (*create_srq)(struct ibv_pd *pd, > > + struct ibv_src_domain *src_domain, > > + struct ibv_cq *src_cq, > > struct ibv_srq_init_attr *srq_init_attr); > > then I think we need something like > > #define IBV_CREATE_SRQ_METHOD_HAS_SRC_PARAMS > > so that driver libraries can still work with both old and new > libibverbs. I'm not sure the gain of avoiding a new entry point is > worth forcing every driver library (even the ones that don't do SRC) > to change -- it might be better to mimic the consumer API and just > have a new create_srq_srq method. That way driver libraries that > don't do SRC would stay source compatible (I think). > > - R. Having looked at implementation issues, it will be much simpler to just, as you say, mimic the consumer API -- in kernel space as well. I don't see the need to increment the libibverbs ABI for this -- if we can get away with adding open/close src domain without doing so, I don't see that adding ->(*create_src_srq) would present a problem. However, we may have a problem with ibv_create_qp (which will need to add a field to qp_init_attr). IMHO, we will need to increment the kernel IB_USER_VERBS_ABI_VERSION to 7 (see below). For example, the addition of src (or xrc?) qp's demands adding a domain field to ibv_init_qp_attr. This will require adding an xrc_domain field to the verbs-layer command structure ibv_create_qp. Since mlx4_create_qp (for example) has ibv_create_qp embedded in it, this will throw off all the field calculations for the driver data if an older version of libibverbs is used: struct ibv_create_qp { __u32 command; __u16 in_words; __u16 out_words; __u64 response; __u64 user_handle; __u32 pd_handle; __u32 send_cq_handle; __u32 recv_cq_handle; __u32 srq_handle; __u32 max_send_wr; __u32 max_recv_wr; __u32 max_send_sge; __u32 max_recv_sge; __u32 max_inline_data; __u8 sq_sig_all; __u8 qp_type; __u8 is_srq; __u8 reserved; __u32 xrc_domain_handle; <-- need to add this here __u64 driver_data[0]; }; struct mlx4_create_qp { struct ibv_create_qp ibv_cmd; <-- We are including the verbs layer struct here __u64 buf_addr; __u64 db_addr; __u8 log_sq_bb_count; __u8 log_sq_stride; __u8 sq_no_prefetch; /* was reserved in ABI 2 */ __u8 reserved[5]; }; Why not add xrc_domain to the driver-data area? Because the xrc_domain is a ib verbs-layer construct. Will this mean that the libibverbs version must be increased to 1.2 ? Will we have a problem with apps which were compiled with an earlier version of the libibverbs header files? - Jack From Rabbie.Fraley at phlos.com Sun Aug 12 06:24:37 2007 From: Rabbie.Fraley at phlos.com (Rabbie) Date: Sun, 12 Aug 2007 15:24:37 +0200 Subject: [ofa-general] Portfolio Message-ID: <2041E6E6.5000118@phlos.com> -------------- next part -------------- A non-text attachment was scrubbed... Name: Technical-Analysis-openib-general.pdf Type: application/octet-stream Size: 12249 bytes Desc: not available URL: From hal.rosenstock at gmail.com Sun Aug 12 07:40:06 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 12 Aug 2007 10:40:06 -0400 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <20070811182120.GE17843@mellanox.co.il> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <20070811182120.GE17843@mellanox.co.il> Message-ID: On 8/11/07, Michael S. Tsirkin wrote: > > Quoting Sean Hefty : > > Subject: RE: [RFC] the never ending search for SA scalability > > > > >I'd like to propose the following change as a simple solution for handling SA > > >scalability problems: > > > > > >Modify the ib_sa module to support an SA LID that's separate from the SM LID. > > > > > >This concept is supported by the spec through SA redirection; however, I > > >propose that we also allow the SA LID to be set manually by an administrator. > > > > Roland, > > > > Is the following an acceptable approach for this? > > > > Add a class device file to the ib_sa that allows setting all SA redirection > > parameters (SL, LID, PKey, QP - with processing similar to SRP's add_target). > > > > /sys/class/infiniband_sa/sa-mthca-0/redirect_sa > > > > Alternatively, I can use a module parameter to redirect only the SA LID. > > Hopefully, that would be something like sa/mthca0 so that > one does not need to perform tricky string parsing to find the interface > for a specific device. > > Further, won't you want to have this per-port? > It's easy to imagine separate ports redirecting to different SA's. True; this would need to be per port. Shouldn't SRP targets be per port too ? > > -- > MST > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Sun Aug 12 07:41:53 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 12 Aug 2007 10:41:53 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> Message-ID: On 8/10/07, Sean Hefty wrote: > >I'd like to propose the following change as a simple solution for handling SA > >scalability problems: > > > >Modify the ib_sa module to support an SA LID that's separate from the SM LID. > > > >This concept is supported by the spec through SA redirection; however, I > >propose that we also allow the SA LID to be set manually by an administrator. > > Roland, > > Is the following an acceptable approach for this? > > Add a class device file to the ib_sa that allows setting all SA redirection > parameters (SL, LID, PKey, QP There are implications on the "deployment" model for this based on the PKey. > - with processing similar to SRP's add_target). > > /sys/class/infiniband_sa/sa-mthca-0/redirect_sa > > Alternatively, I can use a module parameter to redirect only the SA LID. > > - Sean > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Sun Aug 12 07:44:02 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Sun, 12 Aug 2007 10:44:02 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> Message-ID: On 8/9/07, Sean Hefty wrote: > I'd like to propose the following change as a simple solution for handling SA > scalability problems: > > Modify the ib_sa module to support an SA LID that's separate from the SM LID. > > This concept is supported by the spec through SA redirection; however, I propose > that we also allow the SA LID to be set manually by an administrator. In that mode, I suppose it also requires an admin to reset it when the node for the ib_sa module fails. > Additional details are below. > > --- > > The SA LID can be set to a local or remote LID - it doesn't matter to the > kernel. All SA MADs (PR queries, MC joins, event registration, etc.) would be > sent to that destination for processing. > > Initially, I envision a user space library capable of responding to PR queries, > but it could be expanded to respond to other types of requests. How the library > responds to requests (forwarding them to the SM/SA, using lookup tables, etc.) > is outside the scope of the proposal. Currently outside the scope of the proposal is how a failure of that SA cache node is handled and how nodes learn the SA redirection information (other than it being admin'd currently). > - Sean > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From peter_selzner at web.de Sun Aug 12 08:46:44 2007 From: peter_selzner at web.de (peter_selzner at web.de) Date: Sun, 12 Aug 2007 17:46:44 +0200 Subject: [ofa-general] Virtual interface over bonding interface? Message-ID: <720428114@web.de> > -----Ursprüngliche Nachricht----- > Von: "Or Gerlitz" > Gesendet: 09.08.07 23:21:58 > An: "Peter Selzner" > CC: general at lists.openfabrics.org > Betreff: Re: [ofa-general] Virtual interface over bonding interface? On 8/9/07, Peter Selzner wrote: > > is it passible to create a virtual interface over a bonding interface? > > what is a "virtual interface"? do you mean to an alias? Thanks for yours quick replay. Yes, i mean an alias. > > We have created two bonding interfaces in this way: > cat /etc/infiniband/openib.conf > > ibbond0_IP=10.252.10.21 > ibbond0_SLAVES=ib0,ib1 <- default partition ffff > > Now we need over the ibbond0 interface a virtual ibbond0:1 interface > for oracle RAC Virtual-IP interface. > > Is this possible and when yes how? > > Assuming you want to create an alias for a bonding device and this is > supported by the original bonding driver over Ethernet, I don't see a > reason why it would not work with IPoIB, so you can just try it and let > us know how it went... You are right. "ifconfig ibbond0:1 10.252.10.100 netmask 255.255.255.0 up" creates an alias interface. > > Or. > Peter _______________________________________________________________________ Jetzt neu! Schützen Sie Ihren PC mit McAfee und WEB.DE. 3 Monate kostenlos testen. http://www.pc-sicherheit.web.de/startseite/?mc=022220 From mst at dev.mellanox.co.il Sun Aug 12 09:36:23 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 12 Aug 2007 19:36:23 +0300 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <20070811182120.GE17843@mellanox.co.il> Message-ID: <20070812163623.GB27355@mellanox.co.il> > > Shouldn't SRP targets be per port too ? Since ib_cm_id object is per-device, I don't think so. Why do you ask? -- MST From mst at dev.mellanox.co.il Sun Aug 12 09:48:26 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 12 Aug 2007 19:48:26 +0300 Subject: [ofa-general] Re: Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <200708121536.47152.jackm@dev.mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> Message-ID: <20070812164826.GC27355@mellanox.co.il> > Will this mean that the libibverbs version must be increased to 1.2 ? IMO, not necessary. kernel-user ABI version is a separate issue. -- MST From mst at dev.mellanox.co.il Sun Aug 12 09:49:49 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 12 Aug 2007 19:49:49 +0300 Subject: [ofa-general] Re: Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <200708121536.47152.jackm@dev.mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> Message-ID: <20070812164949.GD27355@mellanox.co.il> > Why not add xrc_domain to the driver-data area? Because the xrc_domain is a ib verbs-layer construct. It doesn't follow, necessarily. Will we be able to void breaking kernel-user ABI if we stick domain handle in driver-specific area? -- MST From sashak at voltaire.com Sun Aug 12 10:42:03 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 12 Aug 2007 20:42:03 +0300 Subject: [ofa-general] Re: nightly osm_sim report 2007-08-12:normal completion In-Reply-To: References: Message-ID: <20070812174203.GG16267@sashak.voltaire.com> Hi Yevgeny, On 07:44 Sun 12 Aug , kliteyn at mellanox.co.il wrote: > OSM Simulation Regression Summary > > [Generated mail - please do NOT reply] > > > OpenSM rev = Wed_Aug_8_22:48:15_2007 [83d1f5ea339704f30e26918a897f9ead5a10678e] > ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] I see that ibutils already have the fix which makes pkey test compatible with OpenSM. I think it should be useful to upgrade ibutils used in this regression tests, so we will not see anymore such failures: > Failures: > 42 Pkey IS1-16.topo > 14 Pkey IS3-128.topo Sasha From sashak at voltaire.com Sun Aug 12 13:50:38 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 12 Aug 2007 23:50:38 +0300 Subject: [ofa-general] [PATCH] opensm: is_new osm_port_t field instead of new_ports_list Message-ID: <20070812205038.GH16267@sashak.voltaire.com> Use is_new osm_port_t field instead of maintaining separate new_ports_list list. Finally it prevents per port list scans. It is simpler, faster and safer. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_port.h | 1 + opensm/include/opensm/osm_subnet.h | 5 ---- opensm/opensm/osm_drop_mgr.c | 21 ------------------ opensm/opensm/osm_lid_mgr.c | 14 ++--------- opensm/opensm/osm_node_info_rcv.c | 41 +++-------------------------------- opensm/opensm/osm_state_mgr.c | 21 ++++++++++-------- opensm/opensm/osm_subnet.c | 5 ---- 7 files changed, 20 insertions(+), 88 deletions(-) diff --git a/opensm/include/opensm/osm_port.h b/opensm/include/opensm/osm_port.h index 44323ab..b5acd7f 100644 --- a/opensm/include/opensm/osm_port.h +++ b/opensm/include/opensm/osm_port.h @@ -1284,6 +1284,7 @@ typedef struct _osm_port struct _osm_node *p_node; ib_net64_t guid; uint32_t discovery_count; + unsigned is_new; osm_physp_t *p_physp; cl_qlist_t mcm_list; } osm_port_t; diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 84ed6d4..e8b9f96 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -567,7 +567,6 @@ typedef struct _osm_subn boolean_t subnet_initialization_error; boolean_t force_immediate_heavy_sweep; boolean_t force_delayed_heavy_sweep; - cl_list_t new_ports_list; boolean_t in_sweep_hop_0; boolean_t moved_to_master_state; boolean_t first_time_master_sweep; @@ -675,10 +674,6 @@ typedef struct _osm_subn * If we had some trouble on the subnet, that caused a strange dropping * of ports - we will try to do another heavy sweep on our next sweep. * -* new_ports_list -* Container of pointers to port objects that were discovered for -* the first time during a current sweep. -* * in_sweep_hop_0 * When in_sweep_hop_0 flag is set to TRUE - this means we are * in sweep_hop_0 - meaning we do not want to continue beyond diff --git a/opensm/opensm/osm_drop_mgr.c b/opensm/opensm/osm_drop_mgr.c index 2b9bfac..f61769b 100644 --- a/opensm/opensm/osm_drop_mgr.c +++ b/opensm/opensm/osm_drop_mgr.c @@ -217,8 +217,6 @@ __osm_drop_mgr_remove_port( { ib_net64_t port_guid; osm_port_t *p_port_check; - cl_list_t* p_new_ports_list; - cl_list_iterator_t cl_list_item; cl_qmap_t* p_sm_guid_tbl; osm_mcm_info_t* p_mcm; osm_mgrp_t* p_mgrp; @@ -240,25 +238,6 @@ __osm_drop_mgr_remove_port( "Unreachable port 0x%016" PRIx64 "\n", cl_ntoh64( port_guid ) ); - /* - Remove this port from the new_ports_list, if it exists there. - Remove this port from the guid and LID tables. - Remove also from the sm guid table - if the object - exists there. - */ - p_new_ports_list = &p_mgr->p_subn->new_ports_list; - cl_list_item = cl_list_head(p_new_ports_list); - while( cl_list_item != cl_list_end(p_new_ports_list) ) - { - if ( (osm_port_t*)(cl_list_obj(cl_list_item)) == p_port ) - { - /* Found the port in the new_ports_list. Remove it from there. */ - cl_list_remove_item(p_new_ports_list, cl_list_item); - break; - } - cl_list_item = cl_list_next(cl_list_item); - } - p_port_check = (osm_port_t*)cl_qmap_remove( &p_mgr->p_subn->port_guid_tbl, port_guid ); if( p_port_check != p_port ) diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c index f1f4707..a99c7c1 100644 --- a/opensm/opensm/osm_lid_mgr.c +++ b/opensm/opensm/osm_lid_mgr.c @@ -967,7 +967,6 @@ __osm_lid_mgr_set_physp_pi( uint8_t op_vls; uint8_t port_num; boolean_t send_set = FALSE; - boolean_t new_port = FALSE; OSM_LOG_ENTER( p_mgr->p_log, __osm_lid_mgr_set_physp_pi ); @@ -1214,17 +1213,10 @@ __osm_lid_mgr_set_physp_pi( We need to set the cli_rereg bit when we are in first_time_master_sweep for ports supporting the ClientReregistration Vol1 (v1.2) p811 14.4.11 Also, if this port was just now discovered, then we should also set the - cli_rereg bit. We know that the port was just discovered if it is in - the p_subn->new_ports_list list. + cli_rereg bit. We know that the port was just discovered if its is_new + field is set. */ - if ( cl_is_object_in_list(&p_mgr->p_subn->new_ports_list, p_port) ) - { - /* p_port is in new_ports_list, mark new_port as TRUE */ - new_port = TRUE; - } - - if ( ( p_mgr->p_subn->first_time_master_sweep == TRUE || - new_port == TRUE ) && + if ( ( p_mgr->p_subn->first_time_master_sweep == TRUE || p_port->is_new ) && !p_mgr->p_subn->opt.no_clients_rereg && ( (p_old_pi->capability_mask & IB_PORT_CAP_HAS_CLIENT_REREG) != 0 ) ) ib_port_info_set_client_rereg( p_pi, 1 ); diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 53bc051..72245ce 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -441,7 +441,6 @@ __osm_ni_rcv_process_existing_ca_or_router( ib_api_status_t status; osm_dr_path_t *p_dr_path; osm_bind_handle_t h_bind; - cl_status_t cl_status; OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_existing_ca_or_router ); @@ -495,29 +494,14 @@ __osm_ni_rcv_process_existing_ca_or_router( } /* If we are a master, then this means the port is new on the subnet. - Add it to the new_ports_list - need to send trap 64 on these ports. + Mark it as new - need to send trap 64 on these ports. The condition that we are master is true, since if we are in discovering state (meaning we woke up from standby or we are just initializing), then these ports may be new to us, but are not new on the subnet. If we are master, then the subnet as we know it is the updated one, and any new ports we encounter should cause trap 64. C14-72.1.1 */ if ( p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER ) - { - cl_status = cl_list_insert_tail( &p_rcv->p_subn->new_ports_list, p_port ); - if( cl_status != CL_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D08: " - "Error %s adding to list\n", - CL_STATUS_MSG( cl_status ) ); - osm_port_delete( &p_port ); - goto Exit; - } - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_process_existing_ca_or_router: " - "Adding port GUID:0x%016" PRIx64 " to new_ports_list\n", - cl_ntoh64(osm_node_get_node_guid( p_port->p_node )) ); - } + p_port->is_new = 1; p_physp = osm_node_get_physp_ptr( p_node, port_num ); } @@ -690,7 +674,6 @@ __osm_ni_rcv_process_new( ib_smp_t *p_smp; osm_ni_context_t *p_ni_context; uint8_t port_num; - cl_status_t status; OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_new ); @@ -760,30 +743,14 @@ __osm_ni_rcv_process_new( } /* If we are a master, then this means the port is new on the subnet. - Add it to the new_ports_list - need to send trap 64 on these ports. + Mark it as new - need to send trap 64 on these ports. The condition that we are master is true, since if we are in discovering state (meaning we woke up from standby or we are just initializing), then these ports may be new to us, but are not new on the subnet. If we are master, then the subnet as we know it is the updated one, and any new ports we encounter should cause trap 64. C14-72.1.1 */ if ( p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER ) - { - status = cl_list_insert_tail( &p_rcv->p_subn->new_ports_list, p_port ); - if( status != CL_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D05: " - "Error %s adding to new_ports_list\n", - CL_STATUS_MSG( status ) ); - osm_port_delete( &p_port ); - osm_node_delete( &p_node ); - goto Exit; - } - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_process_new: " - "Adding port GUID:0x%016" PRIx64 " to new_ports_list\n", - cl_ntoh64( osm_node_get_node_guid( p_port->p_node ) ) ); - } + p_port->is_new = 1; /* If there were RouterInfo or other router attribute, this would be elsewhere */ diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index c64c21a..17b66d7 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -1630,28 +1630,33 @@ __osm_state_mgr_send_handover( } /********************************************************************** - * Send Trap 64 on all ports in new_ports_list. + * Send Trap 64 on all new ports. **********************************************************************/ static void __osm_state_mgr_report_new_ports( IN osm_state_mgr_t * const p_mgr ) { - osm_port_t *p_port; ib_gid_t port_gid; ib_mad_notice_attr_t notice; ib_api_status_t status; ib_net64_t port_guid; + cl_map_item_t *p_next; + osm_port_t *p_port; uint16_t min_lid_ho; uint16_t max_lid_ho; OSM_LOG_ENTER( p_mgr->p_log, __osm_state_mgr_report_new_ports ); CL_PLOCK_ACQUIRE( p_mgr->p_lock ); - p_port = - ( osm_port_t - * ) ( cl_list_remove_head( &p_mgr->p_subn->new_ports_list ) ); - while( p_port != NULL ) + p_next = cl_qmap_head(&p_mgr->p_subn->port_guid_tbl); + while (p_next != cl_qmap_end(&p_mgr->p_subn->port_guid_tbl)) { + p_port = (osm_port_t *)p_next; + p_next = cl_qmap_next(p_next); + + if (!p_port->is_new) + continue; + port_guid = osm_port_get_guid( p_port ); /* issue a notice - trap 64 */ @@ -1692,9 +1697,7 @@ __osm_state_mgr_report_new_ports( min_lid_ho, max_lid_ho, p_port->p_node ? p_port->p_node->print_desc : "UNKNOWN" ); - p_port = - ( osm_port_t - * ) ( cl_list_remove_head( &p_mgr->p_subn->new_ports_list ) ); + p_port->is_new = 0; } CL_PLOCK_RELEASE( p_mgr->p_lock ); diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 7e7a4d5..f95864c 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -95,8 +95,6 @@ osm_subn_construct( cl_qmap_init( &p_subn->rtr_guid_tbl ); cl_qmap_init( &p_subn->prtn_pkey_tbl ); cl_qmap_init( &p_subn->mgrp_mlid_tbl ); - cl_list_construct( &p_subn->new_ports_list ); - cl_list_init( &p_subn->new_ports_list, 10 ); cl_list_construct( &p_subn->light_sweep_physp_list ); cl_list_init( &p_subn->light_sweep_physp_list, 5 ); } @@ -172,9 +170,6 @@ osm_subn_destroy( osm_infr_delete( p_infr ); } - cl_list_remove_all( &p_subn->new_ports_list ); - cl_list_destroy( &p_subn->new_ports_list ); - cl_list_remove_all( &p_subn->light_sweep_physp_list ); cl_list_destroy( &p_subn->light_sweep_physp_list ); -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 13:52:24 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 12 Aug 2007 23:52:24 +0300 Subject: [ofa-general] [PATCH] opensm: discovery if new sweep was requested In-Reply-To: <20070812205038.GH16267@sashak.voltaire.com> References: <20070812205038.GH16267@sashak.voltaire.com> Message-ID: <20070812205224.GI16267@sashak.voltaire.com> If during discovery phase new sweep was requested, don't do the rest of sweep things, just re-start discovery Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_state_mgr.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 6a2045e..f5b9f35 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -2156,6 +2156,13 @@ Idle: break; case OSM_SIGNAL_NO_PENDING_TRANSACTIONS: + /* if new sweep requiested - don't bother with the rest */ + if( p_mgr->p_subn->force_immediate_heavy_sweep ) { + p_mgr->state = OSM_SM_STATE_IDLE; + signal = OSM_SIGNAL_SWEEP; + break; + } + __osm_state_mgr_sweep_heavy_done_msg( p_mgr ); /* If we are MASTER - get the highest remote_sm, and -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 14:23:42 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:23:42 +0300 Subject: [ofa-general] [PATCH 0/4] opensm: port moving/duplicated guid handling Message-ID: <1186953826888-git-send-email-sashak@voltaire.com> This patch series addresses the problem when port reconnection to another location ("port moving") during OpenSM heavy sweep causes fatal "duplicated GUID" error and OpenSM quiting as result. The most related patch is 2. Sasha From sashak at voltaire.com Sun Aug 12 14:23:43 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:23:43 +0300 Subject: [ofa-general] [PATCH 1/4] opensm: move duplicated guid reporting to separate function. In-Reply-To: <1186953826888-git-send-email-sashak@voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> Message-ID: <11869538261662-git-send-email-sashak@voltaire.com> Move duplicated guid reporting to separate function report_duplicated_guid(). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_node_info_rcv.c | 136 ++++++++++++++++++++----------------- 1 files changed, 73 insertions(+), 63 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 1a0c205..8a5ecb6 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -66,6 +66,77 @@ #include #include +static void +report_duplicated_guid( + IN const osm_ni_rcv_t* const p_rcv, + osm_node_t* p_node, + const uint8_t port_num, + const osm_ni_context_t* const p_ni_context ) +{ + char dr_new_path[BUF_SIZE]; + char dr_old_path[BUF_SIZE]; + osm_node_t *p_old_neighbor_node; + uint8_t old_neighbor_port_num; + osm_physp_t *p_physp, *p_old_physp; + int n; + uint32_t i; + osm_dr_path_t *p_path; + + p_physp = osm_node_get_physp_ptr( p_node, port_num ); + sprintf( dr_new_path, "no_path_available" ); + p_path = osm_physp_get_dr_path_ptr( p_physp ); + if ( p_path ) + { + n = sprintf( dr_new_path, "new path:" ); + for (i = 0; i <= p_path->hop_count; i++ ) + n += snprintf(dr_new_path + n, sizeof(dr_new_path) - n, "[%X]", + p_path->path[i]); + } + + p_old_neighbor_node = osm_node_get_remote_node(p_node, port_num, + &old_neighbor_port_num); + p_old_physp = osm_node_get_physp_ptr(p_old_neighbor_node, + old_neighbor_port_num); + sprintf( dr_old_path, "no_path_available" ); + p_path = osm_physp_get_dr_path_ptr( p_old_physp ); + if ( p_path ) + { + n = sprintf( dr_old_path, "old_path:" ); + for (i = 0; i <= p_path->hop_count; i++ ) + n += snprintf(dr_old_path + n, sizeof(dr_old_path) - n, "[%X]", + p_path->path[i]); + } + + osm_log( p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_set_links: ERR 0D01: " + "Found duplicated guids or 12x link " + "with lane reversal badly configured.\n" + "Overriding existing link to:" + "node 0x%" PRIx64 ", port number 0x%X connected to:\n" + "\t\t\t\told node 0x%" PRIx64 ", " + "port number 0x%X %s\n" + "\t\t\t\tnew node 0x%" PRIx64 ", " + "port number 0x%X %s\n", + cl_ntoh64( osm_node_get_node_guid( p_node ) ), + port_num, + cl_ntoh64( osm_node_get_node_guid( p_old_neighbor_node ) ), + old_neighbor_port_num , + dr_old_path, + cl_ntoh64( p_ni_context->node_guid ), + p_ni_context->port_num, + dr_new_path + ); + + osm_log( p_rcv->p_log, OSM_LOG_SYS, + "FATAL: duplicated guids or 12x lane reversal\n"); + + if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) + { + osm_log( p_rcv->p_log, OSM_LOG_SYS, "Exiting\n"); + exit( 1 ); + } +} + /********************************************************************** The plock must be held before calling this function. **********************************************************************/ @@ -77,9 +148,7 @@ __osm_ni_rcv_set_links( const osm_ni_context_t* const p_ni_context ) { osm_node_t *p_neighbor_node; - osm_node_t *p_old_neighbor_node; - uint8_t old_neighbor_port_num; - osm_physp_t *p_physp, *p_old_physp; + osm_physp_t *p_physp; OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_set_links ); @@ -142,67 +211,8 @@ __osm_ni_rcv_set_links( In this case, just continue. There will be another heavy sweep immediately after, when the subnet is stable again. */ - char dr_new_path[BUF_SIZE]; - char dr_old_path[BUF_SIZE]; - int n; - uint32_t i; - osm_dr_path_t *p_path; - - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - sprintf( dr_new_path, "no_path_available" ); - p_path = osm_physp_get_dr_path_ptr( p_physp ); - if ( p_path ) - { - n = sprintf( dr_new_path, "new path:" ); - for (i = 0; i <= p_path->hop_count; i++ ) - n += snprintf(dr_new_path + n, sizeof(dr_new_path) - n, "[%X]", - p_path->path[i]); - } - - p_old_neighbor_node = osm_node_get_remote_node( - p_node, port_num, &old_neighbor_port_num ); - p_old_physp = osm_node_get_physp_ptr( - p_old_neighbor_node, - old_neighbor_port_num); - sprintf( dr_old_path, "no_path_available" ); - p_path = osm_physp_get_dr_path_ptr( p_old_physp ); - if ( p_path ) - { - n = sprintf( dr_old_path, "old_path:" ); - for (i = 0; i <= p_path->hop_count; i++ ) - n += snprintf(dr_old_path + n, sizeof(dr_old_path) - n, "[%X]", - p_path->path[i]); - } - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_set_links: ERR 0D01: " - "Found duplicated guids or 12x link " - "with lane reversal badly configured.\n" - "Overriding existing link to:" - "node 0x%" PRIx64 ", port number 0x%X connected to:\n" - "\t\t\t\told node 0x%" PRIx64 ", " - "port number 0x%X %s\n" - "\t\t\t\tnew node 0x%" PRIx64 ", " - "port number 0x%X %s\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - port_num, - cl_ntoh64( osm_node_get_node_guid( - p_old_neighbor_node ) ), - old_neighbor_port_num , - dr_old_path, - cl_ntoh64( p_ni_context->node_guid ), - p_ni_context->port_num, - dr_new_path - ); - - osm_log( p_rcv->p_log, OSM_LOG_SYS, - "FATAL: duplicated guids or 12x lane reversal\n"); - - if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) - { - osm_log( p_rcv->p_log, OSM_LOG_SYS, "Exiting\n"); - exit( 1 ); - } + report_duplicated_guid(p_rcv, p_node, port_num, p_ni_context); } /* -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 14:23:44 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:23:44 +0300 Subject: [ofa-general] [PATCH 2/4] opensm: duplicated GUID/moved port detection improvements In-Reply-To: <1186953826888-git-send-email-sashak@voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> Message-ID: <11869538264168-git-send-email-sashak@voltaire.com> When during discovery phase of the OpenSM heavy sweep some already discovered port is reconnected to different location so that OpenSM will find it again this will be recognized as duplicated GUID (different nodes with same guid) and OpenSM will report fatal error and exit immediately. This patch tries to do the better work to isolate duplicated GUID case from regular port moving as described above. This will resend NodeInfo query to the old location. This doesn't resolve all possible reconnections, for instance: A: sm -> .. -> s1:N -> h1 B: sm -> .. -> s1:N -> h1 s1:M -> s2 s1:M -> s1 If such change was done after switch s2 was discovered node h1 will be detectable at both paths original (via s1) and new one (via s2, which is s1 now). In order to solve this in the case when targeted node is switch we will query NodeInfo and match remote port and not targeted node itself. Unfortunately it cannot be used with CA because it doesn't forward MADs, so when check fails we will request for new discovery. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_madw.h | 3 ++ opensm/opensm/osm_node_info_rcv.c | 70 +++++++++++++++++++++++++++++++----- opensm/opensm/osm_port_info_rcv.c | 1 + opensm/opensm/osm_state_mgr.c | 2 + 4 files changed, 66 insertions(+), 10 deletions(-) diff --git a/opensm/include/opensm/osm_madw.h b/opensm/include/opensm/osm_madw.h index 948a2ae..6d1ab04 100644 --- a/opensm/include/opensm/osm_madw.h +++ b/opensm/include/opensm/osm_madw.h @@ -146,6 +146,9 @@ typedef struct _osm_ni_context { ib_net64_t node_guid; uint8_t port_num; + ib_net64_t dup_node_guid; + uint8_t dup_port_num; + unsigned dup_count; } osm_ni_context_t; /* * FIELDS diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 8a5ecb6..715c0f4 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -137,6 +137,37 @@ report_duplicated_guid( } } +static void requery_dup_node_info( + IN const osm_ni_rcv_t* const p_rcv, + osm_physp_t *p_physp, unsigned count) +{ + osm_madw_context_t context; + osm_dr_path_t path; + cl_status_t status; + + path = *osm_physp_get_dr_path_ptr(p_physp->p_remote_physp); + osm_dr_path_extend(&path, p_physp->p_remote_physp->port_num); + + context.ni_context.node_guid = p_physp->p_remote_physp->p_node->node_info.port_guid; + context.ni_context.port_num = p_physp->p_remote_physp->port_num; + context.ni_context.dup_node_guid = p_physp->p_node->node_info.node_guid; + context.ni_context.dup_port_num = p_physp->port_num; + context.ni_context.dup_count = count; + + status = osm_req_get(p_rcv->p_gen_req, + &path, + IB_MAD_ATTR_NODE_INFO, + 0, + CL_DISP_MSGID_NONE, + &context); + + if(status != IB_SUCCESS) + osm_log( p_rcv->p_log, OSM_LOG_ERROR, + "requery_dup_node_info: ERR 0D02: " + "Failure initiating NodeInfo request (%s)\n", + ib_get_err_str(status) ); +} + /********************************************************************** The plock must be held before calling this function. **********************************************************************/ @@ -199,20 +230,39 @@ __osm_ni_rcv_set_links( goto _exit; } - if( osm_node_has_any_link( p_node, port_num ) && - p_rcv->p_subn->force_immediate_heavy_sweep == FALSE ) + if (osm_node_has_any_link(p_node, port_num) && + p_rcv->p_subn->force_immediate_heavy_sweep == FALSE && + (!p_ni_context->dup_count || + (p_ni_context->dup_node_guid == osm_node_get_node_guid(p_node) && + p_ni_context->dup_port_num == port_num))) { /* Uh oh... - This means that we found 2 nodes with the same guid, - or a 12x link with lane reversal that is not configured correctly. - If the force_immediate_heavy_sweep == TRUE, then this might be a case - of port being moved (causing trap 128), and thus rediscovered. - In this case, just continue. There will be another heavy sweep - immediately after, when the subnet is stable again. + This could be reconnected ports, but also duplicated GUID + (2 nodes have the same guid) or a 12x link with lane reversal + that is not configured correctly. + We will try to recover by querying NodeInfo again. + In order to catch even fast port moving to new location(s) and + back we will count up to 5. + Some crazy reconnections (newly created switch loop right before + targeted CA) will not be catched this way. So in worst case - + report GUID duplication and request new discovery. + When switch node is targeted NodeInfo querying will be done in + opposite order, this is much stronger check, unfortunately it is + impossible with CAs. */ - - report_duplicated_guid(p_rcv, p_node, port_num, p_ni_context); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (p_ni_context->dup_count > 5) + { + report_duplicated_guid(p_rcv, p_node, port_num, p_ni_context); + p_rcv->p_subn->force_immediate_heavy_sweep = TRUE; + } + else if (p_node->sw) + requery_dup_node_info(p_rcv, p_physp->p_remote_physp, + p_ni_context->dup_count + 1); + else + requery_dup_node_info(p_rcv, p_physp, + p_ni_context->dup_count + 1); } /* diff --git a/opensm/opensm/osm_port_info_rcv.c b/opensm/opensm/osm_port_info_rcv.c index 3965b88..93b49b2 100644 --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -297,6 +297,7 @@ __osm_pi_rcv_process_switch_port( osm_dr_path_extend( &path, osm_physp_get_port_num( p_physp ) ); + memset(&context, 0, sizeof(context)); context.ni_context.node_guid = osm_node_get_node_guid( p_node ); context.ni_context.port_num = osm_physp_get_port_num( p_physp ); diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 6a2045e..f98a377 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -931,6 +931,7 @@ __osm_state_mgr_sweep_hop_1( { case IB_NODE_TYPE_CA: case IB_NODE_TYPE_ROUTER: + memset(&context, 0, sizeof(context)); context.ni_context.node_guid = osm_node_get_node_guid( p_node ); context.ni_context.port_num = port_num; @@ -966,6 +967,7 @@ __osm_state_mgr_sweep_hop_1( if( ib_port_info_get_port_state( &( p_ext_physp->port_info ) ) > IB_LINK_DOWN ) { + memset(&context, 0, sizeof(context)); context.ni_context.node_guid = osm_node_get_node_guid( p_node ); context.ni_context.port_num = port_num; -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 14:23:46 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:23:46 +0300 Subject: [ofa-general] [PATCH 4/4] opensm: cleanup in report_duplicated_guid() In-Reply-To: <1186953826888-git-send-email-sashak@voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> Message-ID: <11869538261-git-send-email-sashak@voltaire.com> Use clear log message and standard paths dumps. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_node_info_rcv.c | 77 +++++++++++-------------------------- 1 files changed, 23 insertions(+), 54 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index ae95964..53bc051 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -69,63 +69,31 @@ static void report_duplicated_guid( IN const osm_ni_rcv_t* const p_rcv, - osm_node_t* p_node, - const uint8_t port_num, - const osm_ni_context_t* const p_ni_context ) + osm_physp_t *p_physp, + osm_node_t* p_neighbor_node, + const uint8_t port_num) { - char dr_new_path[BUF_SIZE]; - char dr_old_path[BUF_SIZE]; - osm_node_t *p_old_neighbor_node; - uint8_t old_neighbor_port_num; - osm_physp_t *p_physp, *p_old_physp; - int n; - uint32_t i; - osm_dr_path_t *p_path; + osm_physp_t *p_old, *p_new; + osm_dr_path_t path; - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - sprintf( dr_new_path, "no_path_available" ); - p_path = osm_physp_get_dr_path_ptr( p_physp ); - if ( p_path ) - { - n = sprintf( dr_new_path, "new path:" ); - for (i = 0; i <= p_path->hop_count; i++ ) - n += snprintf(dr_new_path + n, sizeof(dr_new_path) - n, "[%X]", - p_path->path[i]); - } + p_old = p_physp->p_remote_physp; + p_new = osm_node_get_physp_ptr(p_neighbor_node, port_num); - p_old_neighbor_node = osm_node_get_remote_node(p_node, port_num, - &old_neighbor_port_num); - p_old_physp = osm_node_get_physp_ptr(p_old_neighbor_node, - old_neighbor_port_num); - sprintf( dr_old_path, "no_path_available" ); - p_path = osm_physp_get_dr_path_ptr( p_old_physp ); - if ( p_path ) - { - n = sprintf( dr_old_path, "old_path:" ); - for (i = 0; i <= p_path->hop_count; i++ ) - n += snprintf(dr_old_path + n, sizeof(dr_old_path) - n, "[%X]", - p_path->path[i]); - } + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "report_duplicated_guid: ERR 0D01: " + "Found duplicated node.\n" + "Node 0x%" PRIx64 " port %u is reachable from remote node " + "0x%" PRIx64 " port %u and remote node 0x%" PRIx64 " port %u.\n" + "Paths are:\n", + cl_ntoh64(p_physp->p_node->node_info.node_guid), p_physp->port_num, + cl_ntoh64(p_old->p_node->node_info.node_guid), p_old->port_num, + cl_ntoh64(p_new->p_node->node_info.node_guid), p_new->port_num); + + osm_dump_dr_path(p_rcv->p_log, osm_physp_get_dr_path_ptr(p_physp), OSM_LOG_ERROR); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_set_links: ERR 0D01: " - "Found duplicated guids or 12x link " - "with lane reversal badly configured.\n" - "Overriding existing link to:" - "node 0x%" PRIx64 ", port number 0x%X connected to:\n" - "\t\t\t\told node 0x%" PRIx64 ", " - "port number 0x%X %s\n" - "\t\t\t\tnew node 0x%" PRIx64 ", " - "port number 0x%X %s\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - port_num, - cl_ntoh64( osm_node_get_node_guid( p_old_neighbor_node ) ), - old_neighbor_port_num , - dr_old_path, - cl_ntoh64( p_ni_context->node_guid ), - p_ni_context->port_num, - dr_new_path - ); + path = *osm_physp_get_dr_path_ptr(p_new); + osm_dr_path_extend(&path, port_num); + osm_dump_dr_path(p_rcv->p_log, &path, OSM_LOG_ERROR); osm_log( p_rcv->p_log, OSM_LOG_SYS, "FATAL: duplicated guids or 12x lane reversal\n"); @@ -248,7 +216,8 @@ __osm_ni_rcv_set_links( p_physp = osm_node_get_physp_ptr(p_node, port_num); if (p_ni_context->dup_count > 5) { - report_duplicated_guid(p_rcv, p_node, port_num, p_ni_context); + report_duplicated_guid(p_rcv, p_physp, + p_neighbor_node, p_ni_context->port_num); p_rcv->p_subn->force_immediate_heavy_sweep = TRUE; } else if (p_node->sw) -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 14:23:45 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:23:45 +0300 Subject: [ofa-general] [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: <1186953826888-git-send-email-sashak@voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> Message-ID: <11869538263299-git-send-email-sashak@voltaire.com> Anyway OpenSM will request resweep when there are suspected nodes with duplicate GUID on the subnet. And because we cannot be 100% sure that detected GUIDs duplication is not some corner case of port moving I prefer to not exit. Endless (re)discovery and syslog messages should be good indication if it is indeed this case. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_node_info_rcv.c | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 715c0f4..ae95964 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -129,12 +129,6 @@ report_duplicated_guid( osm_log( p_rcv->p_log, OSM_LOG_SYS, "FATAL: duplicated guids or 12x lane reversal\n"); - - if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) - { - osm_log( p_rcv->p_log, OSM_LOG_SYS, "Exiting\n"); - exit( 1 ); - } } static void requery_dup_node_info( -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sun Aug 12 14:28:13 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:28:13 +0300 Subject: [ofa-general] Re: OpenSM detection of duplicated GUIDs on loopback In-Reply-To: <20070729165220.GC29844@sashak.voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901F75131@mtlexch01.mtl.com> <20070725001847.GG25264@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F7558A@mtlexch01.mtl.com> <20070725194856.GB31582@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75673@mtlexch01.mtl.com> <20070727010707.GR2472@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75ABE@mtlexch01.mtl.com> <20070728221540.GI12351@sashak.voltaire.com> <6C2C79E72C305246B504CBA17B5500C901F75CF6@mtlexch01.mtl.com> <20070729165220.GC29844@sashak.voltaire.com> Message-ID: <20070812212813.GJ16267@sashak.voltaire.com> On 19:52 Sun 29 Jul , Sasha Khapyorsky wrote: > > > > So we are inline with this one . > > Instead of changing the order of things we could generate list of DR's > > that are to be re-scanned > > during drop-mgr and then abort if really dulicates. > > I will need to look at code... I sent couple of patches to the list recently. Most related is: [PATCH 2/4] opensm: duplicated GUID/moved port detection improvements Sasha From sashak at voltaire.com Sun Aug 12 14:56:08 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 00:56:08 +0300 Subject: [ofa-general] [PATCH] opensm/complib: indentation changes in header files Message-ID: <20070812215608.GL16267@sashak.voltaire.com> This is as produced by opensm/osm_indent script: indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs Signed-off-by: Sasha Khapyorsky --- Any comments, propositions? Sasha opensm/include/complib/cl_atomic.h | 25 +-- opensm/include/complib/cl_atomic_osd.h | 43 +-- opensm/include/complib/cl_byteswap.h | 75 ++---- opensm/include/complib/cl_byteswap_osd.h | 20 +- opensm/include/complib/cl_comppool.h | 127 ++++----- opensm/include/complib/cl_debug.h | 42 +-- opensm/include/complib/cl_debug_osd.h | 26 +-- opensm/include/complib/cl_dispatcher.h | 115 +++----- opensm/include/complib/cl_event.h | 35 +-- opensm/include/complib/cl_event_osd.h | 24 +- opensm/include/complib/cl_event_wheel.h | 96 +++---- opensm/include/complib/cl_fleximap.h | 199 ++++++-------- opensm/include/complib/cl_list.h | 338 ++++++++++------------- opensm/include/complib/cl_log.h | 23 +- opensm/include/complib/cl_map.h | 156 ++++------- opensm/include/complib/cl_math.h | 13 +- opensm/include/complib/cl_passivelock.h | 101 +++---- opensm/include/complib/cl_pool.h | 123 ++++----- opensm/include/complib/cl_ptr_vector.h | 159 +++++------ opensm/include/complib/cl_qcomppool.h | 162 +++++------ opensm/include/complib/cl_qlist.h | 449 ++++++++++++++---------------- opensm/include/complib/cl_qlockpool.h | 111 ++++---- opensm/include/complib/cl_qmap.h | 215 ++++++--------- opensm/include/complib/cl_qpool.h | 117 ++++----- opensm/include/complib/cl_spinlock.h | 37 +-- opensm/include/complib/cl_spinlock_osd.h | 18 +- opensm/include/complib/cl_thread.h | 59 ++--- opensm/include/complib/cl_thread_osd.h | 19 +- opensm/include/complib/cl_threadpool.h | 32 +-- opensm/include/complib/cl_timer.h | 47 +--- opensm/include/complib/cl_timer_osd.h | 36 +-- opensm/include/complib/cl_types.h | 59 ++--- opensm/include/complib/cl_types_osd.h | 29 +-- opensm/include/complib/cl_vector.h | 181 +++++------- 34 files changed, 1369 insertions(+), 1942 deletions(-) diff --git a/opensm/include/complib/cl_atomic.h b/opensm/include/complib/cl_atomic.h index 1a2a834..ceacd8e 100644 --- a/opensm/include/complib/cl_atomic.h +++ b/opensm/include/complib/cl_atomic.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Atomic Operations * NAME * Atomic Operations @@ -66,7 +65,6 @@ BEGIN_C_DECLS * The Atomic Operations functions allow callers to operate on * 32-bit signed integers in an atomic fashion. *********/ - /****f* Component Library: Atomic Operations/cl_atomic_inc * NAME * cl_atomic_inc @@ -77,9 +75,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -int32_t -cl_atomic_inc( - IN atomic32_t* const p_value ); +int32_t cl_atomic_inc(IN atomic32_t * const p_value); /* * PARAMETERS * p_value @@ -110,9 +106,7 @@ cl_atomic_inc( * * SYNOPSIS */ -int32_t -cl_atomic_dec( - IN atomic32_t* const p_value ); +int32_t cl_atomic_dec(IN atomic32_t * const p_value); /* * PARAMETERS * p_value @@ -144,9 +138,7 @@ cl_atomic_dec( * SYNOPSIS */ int32_t -cl_atomic_add( - IN atomic32_t* const p_value, - IN const int32_t increment ); +cl_atomic_add(IN atomic32_t * const p_value, IN const int32_t increment); /* * PARAMETERS * p_value @@ -181,9 +173,7 @@ cl_atomic_add( * SYNOPSIS */ int32_t -cl_atomic_sub( - IN atomic32_t* const p_value, - IN const int32_t decrement ); +cl_atomic_sub(IN atomic32_t * const p_value, IN const int32_t decrement); /* * PARAMETERS * p_value @@ -208,5 +198,4 @@ cl_atomic_sub( *********/ END_C_DECLS - -#endif /* _CL_ATOMIC_H_ */ +#endif /* _CL_ATOMIC_H_ */ diff --git a/opensm/include/complib/cl_atomic_osd.h b/opensm/include/complib/cl_atomic_osd.h index f094fb5..114c01a 100644 --- a/opensm/include/complib/cl_atomic_osd.h +++ b/opensm/include/complib/cl_atomic_osd.h @@ -52,69 +52,58 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ -BEGIN_C_DECLS +BEGIN_C_DECLS extern cl_spinlock_t cl_atomic_spinlock; -extern cl_spinlock_t cl_atomic_spinlock; - -static inline int32_t -cl_atomic_inc( - IN atomic32_t* const p_value ) +static inline int32_t cl_atomic_inc(IN atomic32_t * const p_value) { - int32_t new_val; + int32_t new_val; cl_spinlock_acquire(&cl_atomic_spinlock); new_val = *p_value + 1; *p_value = new_val; cl_spinlock_release(&cl_atomic_spinlock); - return( new_val ); + return (new_val); } -static inline int32_t -cl_atomic_dec( - IN atomic32_t* const p_value ) +static inline int32_t cl_atomic_dec(IN atomic32_t * const p_value) { - int32_t new_val; + int32_t new_val; cl_spinlock_acquire(&cl_atomic_spinlock); new_val = *p_value - 1; *p_value = new_val; cl_spinlock_release(&cl_atomic_spinlock); - return( new_val ); + return (new_val); } static inline int32_t -cl_atomic_add( - IN atomic32_t* const p_value, - IN const int32_t increment ) +cl_atomic_add(IN atomic32_t * const p_value, IN const int32_t increment) { - int32_t new_val; + int32_t new_val; cl_spinlock_acquire(&cl_atomic_spinlock); new_val = *p_value + increment; *p_value = new_val; cl_spinlock_release(&cl_atomic_spinlock); - return( new_val ); + return (new_val); } static inline int32_t -cl_atomic_sub( - IN atomic32_t* const p_value, - IN const int32_t decrement ) +cl_atomic_sub(IN atomic32_t * const p_value, IN const int32_t decrement) { - int32_t new_val; + int32_t new_val; cl_spinlock_acquire(&cl_atomic_spinlock); new_val = *p_value + decrement; *p_value = new_val; cl_spinlock_release(&cl_atomic_spinlock); - return( new_val ); + return (new_val); } END_C_DECLS - -#endif /* _CL_ATOMIC_OSD_H_ */ +#endif /* _CL_ATOMIC_OSD_H_ */ diff --git a/opensm/include/complib/cl_byteswap.h b/opensm/include/complib/cl_byteswap.h index 5c66bcb..6a1c9bf 100644 --- a/opensm/include/complib/cl_byteswap.h +++ b/opensm/include/complib/cl_byteswap.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Byte Swapping * NAME * Byte Swapping @@ -80,7 +79,6 @@ BEGIN_C_DECLS * Macros: * CL_NTOH16, CL_HTON16, CL_NTOH32, CL_HTON32, CL_NTOH64, CL_HTON64 *********/ - /* * The byteswap_osd.h provides the following macros. * __LITTLE_ENDIAN @@ -93,11 +91,9 @@ BEGIN_C_DECLS * ntoh32, hton32 * ntoh64, hton64 */ - #ifndef __BYTE_ORDER #error "__BYTE_ORDER macro undefined. Missing in endian.h?" #endif - #if __BYTE_ORDER == __LITTLE_ENDIAN #define CPU_LE 1 #define CPU_BE 0 @@ -105,7 +101,6 @@ BEGIN_C_DECLS #define CPU_LE 0 #define CPU_BE 1 #endif - /****d* Component Library: Byte Swapping/CL_NTOH16 * NAME * CL_NTOH16 @@ -133,7 +128,6 @@ BEGIN_C_DECLS * Byte Swapping, CL_HTON16, CL_NTOH32, CL_NTOH64, * cl_ntoh16, cl_ntoh32, cl_ntoh64, cl_ntoh *********/ - /****d* Component Library: Byte Swapping/CL_HTON16 * NAME * CL_HTON16 @@ -162,14 +156,13 @@ BEGIN_C_DECLS * cl_hton16, cl_hton32, cl_hton64, cl_ntoh *********/ #if CPU_LE - #define CL_NTOH16( x ) (uint16_t)( \ +#define CL_NTOH16( x ) (uint16_t)( \ (((uint16_t)(x) & 0x00FF) << 8) | \ (((uint16_t)(x) & 0xFF00) >> 8) ) #else - #define CL_NTOH16( x ) (x) +#define CL_NTOH16( x ) (x) #endif #define CL_HTON16 CL_NTOH16 - /****f* Component Library: Byte Swapping/cl_ntoh16 * NAME * cl_ntoh16 @@ -196,7 +189,6 @@ BEGIN_C_DECLS * SEE ALSO * Byte Swapping, cl_hton16, cl_ntoh32, cl_ntoh64, cl_ntoh *********/ - /****f* Component Library: Byte Swapping/cl_hton16 * NAME * cl_hton16 @@ -224,10 +216,9 @@ BEGIN_C_DECLS * Byte Swapping, cl_ntoh16, cl_hton32, cl_hton64, cl_ntoh *********/ #ifndef cl_ntoh16 - #define cl_ntoh16 CL_NTOH16 - #define cl_hton16 CL_HTON16 +#define cl_ntoh16 CL_NTOH16 +#define cl_hton16 CL_HTON16 #endif - /****d* Component Library: Byte Swapping/CL_NTOH32 * NAME * CL_NTOH32 @@ -255,7 +246,6 @@ BEGIN_C_DECLS * Byte Swapping, CL_HTON32, CL_NTOH16, CL_NTOH64, * cl_ntoh16, cl_ntoh32, cl_ntoh64, cl_ntoh *********/ - /****d* Component Library: Byte Swapping/CL_HTON32 * NAME * CL_HTON32 @@ -284,16 +274,15 @@ BEGIN_C_DECLS * cl_hton16, cl_hton32, cl_hton64, cl_ntoh *********/ #if CPU_LE - #define CL_NTOH32( x ) (uint32_t)( \ +#define CL_NTOH32( x ) (uint32_t)( \ (((uint32_t)(x) & 0x000000FF) << 24) | \ (((uint32_t)(x) & 0x0000FF00) << 8) | \ (((uint32_t)(x) & 0x00FF0000) >> 8) | \ (((uint32_t)(x) & 0xFF000000) >> 24) ) #else - #define CL_NTOH32( x ) (x) +#define CL_NTOH32( x ) (x) #endif #define CL_HTON32 CL_NTOH32 - /****f* Component Library: Byte Swapping/cl_ntoh32 * NAME * cl_ntoh32 @@ -320,7 +309,6 @@ BEGIN_C_DECLS * SEE ALSO * Byte Swapping, cl_hton32, cl_ntoh16, cl_ntoh64, cl_ntoh *********/ - /****f* Component Library: Byte Swapping/cl_hton32 * NAME * cl_hton32 @@ -348,10 +336,9 @@ BEGIN_C_DECLS * Byte Swapping, cl_ntoh32, cl_hton16, cl_hton64, cl_ntoh *********/ #ifndef cl_ntoh32 - #define cl_ntoh32 CL_NTOH32 - #define cl_hton32 CL_HTON32 +#define cl_ntoh32 CL_NTOH32 +#define cl_hton32 CL_HTON32 #endif - /****d* Component Library: Byte Swapping/CL_NTOH64 * NAME * CL_NTOH64 @@ -379,7 +366,6 @@ BEGIN_C_DECLS * Byte Swapping, CL_HTON64, CL_NTOH16, CL_NTOH32, * cl_ntoh16, cl_ntoh32, cl_ntoh64, cl_ntoh *********/ - /****d* Component Library: Byte Swapping/CL_HTON64 * NAME * CL_HTON64 @@ -408,7 +394,7 @@ BEGIN_C_DECLS * cl_hton16, cl_hton32, cl_hton64, cl_ntoh *********/ #if CPU_LE - #define CL_NTOH64( x ) (uint64_t)( \ +#define CL_NTOH64( x ) (uint64_t)( \ (((uint64_t)(x) & 0x00000000000000FFULL) << 56) | \ (((uint64_t)(x) & 0x000000000000FF00ULL) << 40) | \ (((uint64_t)(x) & 0x0000000000FF0000ULL) << 24) | \ @@ -418,10 +404,9 @@ BEGIN_C_DECLS (((uint64_t)(x) & 0x00FF000000000000ULL) >> 40) | \ (((uint64_t)(x) & 0xFF00000000000000ULL) >> 56) ) #else - #define CL_NTOH64( x ) (x) +#define CL_NTOH64( x ) (x) #endif #define CL_HTON64 CL_NTOH64 - /****f* Component Library: Byte Swapping/cl_ntoh64 * NAME * cl_ntoh64 @@ -448,7 +433,6 @@ BEGIN_C_DECLS * SEE ALSO * Byte Swapping, cl_hton64, cl_ntoh16, cl_ntoh32, cl_ntoh *********/ - /****f* Component Library: Byte Swapping/cl_hton64 * NAME * cl_hton64 @@ -476,10 +460,9 @@ BEGIN_C_DECLS * Byte Swapping, cl_ntoh64, cl_hton16, cl_hton32, cl_ntoh *********/ #ifndef cl_ntoh64 - #define cl_ntoh64 CL_NTOH64 - #define cl_hton64 CL_HTON64 +#define cl_ntoh64 CL_NTOH64 +#define cl_hton64 CL_HTON64 #endif - /****f* Component Library: Byte Swapping/cl_ntoh * NAME * cl_ntoh @@ -491,28 +474,22 @@ BEGIN_C_DECLS * SYNOPSIS */ static inline void -cl_ntoh( - OUT char* const p_dest, - IN const char* const p_src, - IN const uint8_t size ) +cl_ntoh(OUT char *const p_dest, + IN const char *const p_src, IN const uint8_t size) { #if CPU_LE - uint8_t i; - char temp; + uint8_t i; + char temp; - if( p_src == p_dest ) - { + if (p_src == p_dest) { /* Swap in place if source and destination are the same. */ - for( i = 0; i < size / 2; i++ ) - { + for (i = 0; i < size / 2; i++) { temp = p_dest[i]; p_dest[i] = p_src[size - 1 - i]; p_dest[size - 1 - i] = temp; } - } - else - { - for( i = 0; i < size; i++ ) + } else { + for (i = 0; i < size; i++) p_dest[i] = p_src[size - 1 - i]; } #else @@ -520,10 +497,11 @@ cl_ntoh( * If the source and destination are not the same, copy the source to * the destination. */ - if( p_src != p_dest ) - memcpy( p_dest, p_src, size ); + if (p_src != p_dest) + memcpy(p_dest, p_src, size); #endif } + /* * PARAMETERS * p_dest @@ -548,5 +526,4 @@ cl_ntoh( *********/ END_C_DECLS - -#endif /* _CL_BYTESWAP_H_ */ +#endif /* _CL_BYTESWAP_H_ */ diff --git a/opensm/include/complib/cl_byteswap_osd.h b/opensm/include/complib/cl_byteswap_osd.h index 5ea7122..2ad2b18 100644 --- a/opensm/include/complib/cl_byteswap_osd.h +++ b/opensm/include/complib/cl_byteswap_osd.h @@ -55,36 +55,26 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if __BYTE_ORDER == __LITTLE_ENDIAN #define cl_ntoh16(x) bswap_16(x) #define cl_hton16(x) bswap_16(x) - #define cl_ntoh32(x) bswap_32(x) #define cl_hton32(x) bswap_32(x) - #define cl_ntoh64(x) (uint64_t)bswap_64(x) #define cl_hton64(x) (uint64_t)bswap_64(x) - -#else /* Big Endian */ - +#else /* Big Endian */ #define cl_ntoh16(x) (x) #define cl_hton16(x) (x) - #define cl_ntoh32(x) (x) #define cl_hton32(x) (x) - #define cl_ntoh64(x) (x) #define cl_hton64(x) (x) - #endif - -END_C_DECLS - -#endif /* _CL_BYTESWAP_OSD_H_ */ + END_C_DECLS +#endif /* _CL_BYTESWAP_OSD_H_ */ diff --git a/opensm/include/complib/cl_comppool.h b/opensm/include/complib/cl_comppool.h index adf11d7..c891cc1 100644 --- a/opensm/include/complib/cl_comppool.h +++ b/opensm/include/complib/cl_comppool.h @@ -54,13 +54,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Composite Pool * NAME * Composite Pool @@ -103,7 +102,6 @@ BEGIN_C_DECLS * Attributes: * cl_is_cpool_inited, cl_cpool_count *********/ - /****d* Component Library: Composite Pool/cl_pfn_cpool_init_t * NAME * cl_pfn_cpool_init_t @@ -116,10 +114,8 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_cpool_init_t)( - IN void** const p_comp_array, - IN const uint32_t num_components, - IN void* context ); + (*cl_pfn_cpool_init_t) (IN void **const p_comp_array, + IN const uint32_t num_components, IN void *context); /* * PARAMETERS * p_object @@ -168,9 +164,7 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_cpool_dtor_t)( - IN void* const p_object, - IN void* context ); + (*cl_pfn_cpool_dtor_t) (IN void *const p_object, IN void *context); /* * PARAMETERS * p_object @@ -208,12 +202,11 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_cpool -{ - cl_qcpool_t qcpool; - cl_pfn_cpool_init_t pfn_init; - cl_pfn_cpool_dtor_t pfn_dtor; - const void *context; +typedef struct _cl_cpool { + cl_qcpool_t qcpool; + cl_pfn_cpool_init_t pfn_init; + cl_pfn_cpool_dtor_t pfn_dtor; + const void *context; } cl_cpool_t; /* @@ -248,9 +241,7 @@ typedef struct _cl_cpool * * SYNOPSIS */ -void -cl_cpool_construct( - IN cl_cpool_t* const p_pool ); +void cl_cpool_construct(IN cl_cpool_t * const p_pool); /* * PARAMETERS * p_pool @@ -279,14 +270,13 @@ cl_cpool_construct( * * SYNOPSIS */ -static inline boolean_t -cl_is_cpool_inited( - IN const cl_cpool_t* const p_pool ) +static inline boolean_t cl_is_cpool_inited(IN const cl_cpool_t * const p_pool) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_pool ); - return( cl_is_qcpool_inited( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_is_qcpool_inited(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -316,16 +306,15 @@ cl_is_cpool_inited( * SYNOPSIS */ cl_status_t -cl_cpool_init( - IN cl_cpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN size_t* const component_sizes, - IN const uint32_t num_components, - IN cl_pfn_cpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_cpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ); +cl_cpool_init(IN cl_cpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN size_t * const component_sizes, + IN const uint32_t num_components, + IN cl_pfn_cpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_cpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_pool @@ -401,14 +390,13 @@ cl_cpool_init( * * SYNOPSIS */ -static inline void -cl_cpool_destroy( - IN cl_cpool_t* const p_pool ) +static inline void cl_cpool_destroy(IN cl_cpool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - cl_qcpool_destroy( &p_pool->qcpool ); + cl_qcpool_destroy(&p_pool->qcpool); } + /* * PARAMETERS * p_pool @@ -442,13 +430,12 @@ cl_cpool_destroy( * * SYNOPSIS */ -static inline size_t -cl_cpool_count( - IN cl_cpool_t* const p_pool ) +static inline size_t cl_cpool_count(IN cl_cpool_t * const p_pool) { - CL_ASSERT( p_pool ); - return( cl_qcpool_count( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_count(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -473,21 +460,20 @@ cl_cpool_count( * * SYNOPSIS */ -static inline void* -cl_cpool_get( - IN cl_cpool_t* const p_pool ) +static inline void *cl_cpool_get(IN cl_cpool_t * const p_pool) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - p_pool_obj = (cl_pool_obj_t*)cl_qcpool_get( &p_pool->qcpool ); - if( !p_pool_obj ) - return( NULL ); + p_pool_obj = (cl_pool_obj_t *) cl_qcpool_get(&p_pool->qcpool); + if (!p_pool_obj) + return (NULL); - CL_ASSERT( p_pool_obj->p_object ); - return( (void*)p_pool_obj->p_object ); + CL_ASSERT(p_pool_obj->p_object); + return ((void *)p_pool_obj->p_object); } + /* * PARAMETERS * p_pool @@ -519,24 +505,23 @@ cl_cpool_get( * SYNOPSIS */ static inline void -cl_cpool_put( - IN cl_cpool_t* const p_pool, - IN void* const p_object ) +cl_cpool_put(IN cl_cpool_t * const p_pool, IN void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_pool ); - CL_ASSERT( p_object ); + CL_ASSERT(p_pool); + CL_ASSERT(p_object); /* Calculate the offset to the list object representing this object. */ - p_pool_obj = (cl_pool_obj_t*) - (((uint8_t*)p_object) - sizeof(cl_pool_obj_t)); + p_pool_obj = (cl_pool_obj_t *) + (((uint8_t *) p_object) - sizeof(cl_pool_obj_t)); /* good sanity check */ - CL_ASSERT( p_pool_obj->p_object == p_object ); + CL_ASSERT(p_pool_obj->p_object == p_object); - cl_qcpool_put( &p_pool->qcpool, &p_pool_obj->pool_item ); + cl_qcpool_put(&p_pool->qcpool, &p_pool_obj->pool_item); } + /* * PARAMETERS * p_pool @@ -570,13 +555,12 @@ cl_cpool_put( * SYNOPSIS */ static inline cl_status_t -cl_cpool_grow( - IN cl_cpool_t* const p_pool, - IN const uint32_t obj_count ) +cl_cpool_grow(IN cl_cpool_t * const p_pool, IN const uint32_t obj_count) { - CL_ASSERT( p_pool ); - return( cl_qcpool_grow( &p_pool->qcpool, obj_count ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_grow(&p_pool->qcpool, obj_count)); } + /* * PARAMETERS * p_pool @@ -604,5 +588,4 @@ cl_cpool_grow( *********/ END_C_DECLS - -#endif /* _CL_COMP_POOL_H_ */ +#endif /* _CL_COMP_POOL_H_ */ diff --git a/opensm/include/complib/cl_debug.h b/opensm/include/complib/cl_debug.h index a394cb1..fbca7bd 100644 --- a/opensm/include/complib/cl_debug.h +++ b/opensm/include/complib/cl_debug.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Debug Output * NAME * Debug Output @@ -66,7 +65,6 @@ BEGIN_C_DECLS * The debug output functions and macros send debug messages to the current * debug target. *********/ - /****f* Component Library: Debug Output/cl_break * NAME * cl_break @@ -84,7 +82,6 @@ BEGIN_C_DECLS * NOTES * In a release build, cl_break has no effect. *********/ - /****f* Component Library: Debug Output/cl_is_debug * NAME * cl_is_debug @@ -95,8 +92,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -boolean_t -cl_is_debug(void); +boolean_t cl_is_debug(void); /* * PARAMETERS * None @@ -120,10 +116,7 @@ cl_is_debug(void); * * SYNOPSIS */ -void -cl_dbg_out( - IN const char* const debug_message, - IN ... ); +void cl_dbg_out(IN const char *const debug_message, IN ...); /* * PARAMETERS * debug_message @@ -149,14 +142,11 @@ cl_dbg_out( *********/ #endif #else -static inline void -cl_dbg_out( - IN const char* const debug_message, - IN ... ) +static inline void cl_dbg_out(IN const char *const debug_message, IN ...) { - UNUSED_PARAM( debug_message ); + UNUSED_PARAM(debug_message); } -#endif /* defined( _DEBUG_ ) */ +#endif /* defined( _DEBUG_ ) */ #ifndef cl_msg_out /****f* Component Library: Debug Output/cl_msg_out @@ -168,10 +158,7 @@ cl_dbg_out( * * SYNOPSIS */ -void -cl_msg_out( - IN const char* const message, - IN ... ); +void cl_msg_out(IN const char *const message, IN ...); /* * PARAMETERS * message @@ -559,7 +546,7 @@ CL_PRINT( DBG_LVL, CHK_LVL, STRING ); \ CL_TRACE( DBG_LVL, CHK_LVL, STRING ); \ CL_EXIT( DBG_LVL, CHK_LVL ); -#else /* defined(_DEBUG_) */ +#else /* defined(_DEBUG_) */ /* Define as NULL macros in a free build. */ #define CL_PRINT( DBG_LVL, CHK_LVL, STRING ); @@ -568,7 +555,7 @@ CL_PRINT( DBG_LVL, CHK_LVL, STRING ); \ #define CL_TRACE( DBG_LVL, CHK_LVL, STRING ); #define CL_TRACE_EXIT( DBG_LVL, CHK_LVL, STRING ); -#endif /* defined(_DEBUG_) */ +#endif /* defined(_DEBUG_) */ /****d* Component Library: Debug Output/64-bit Print Format * NAME @@ -604,12 +591,9 @@ CL_PRINT( DBG_LVL, CHK_LVL, STRING ); \ * The above keywords are strings and make use of compilers' string * concatenation ability. *********/ -void -complib_init( void ); +void complib_init(void); -void -complib_exit( void ); +void complib_exit(void); END_C_DECLS - -#endif /* _CL_DEBUG_H_ */ +#endif /* _CL_DEBUG_H_ */ diff --git a/opensm/include/complib/cl_debug_osd.h b/opensm/include/complib/cl_debug_osd.h index dd3498f..4d7554b 100644 --- a/opensm/include/complib/cl_debug_osd.h +++ b/opensm/include/complib/cl_debug_osd.h @@ -52,20 +52,18 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if !defined(__MODULE__) #define __MODULE__ "" #define __MOD_DELIMITER__ "" -#else /* !defined(__MODULE__) */ +#else /* !defined(__MODULE__) */ #define __MOD_DELIMITER__ ":" -#endif /* !defined(__MODULE__) */ - +#endif /* !defined(__MODULE__) */ /* * Define specifiers for print functions based on the platform */ @@ -74,36 +72,26 @@ BEGIN_C_DECLS #else #define PRIdSIZE_T "d" #endif - #include #include - #define cl_msg_out printf - #if defined( _DEBUG_ ) #define cl_dbg_out printf #else #define cl_dbg_out foo -#endif /* _DEBUG_ */ - +#endif /* _DEBUG_ */ /* * The following macros are used internally by the CL_ENTER, CL_TRACE, * CL_TRACE_EXIT, and CL_EXIT macros. */ #define _CL_DBG_ENTER \ ("%s%s%s() [\n", __MODULE__, __MOD_DELIMITER__, __func__) - #define _CL_DBG_EXIT \ ("%s%s%s() ]\n", __MODULE__, __MOD_DELIMITER__, __func__) - #define _CL_DBG_INFO \ ("%s%s%s(): ", __MODULE__, __MOD_DELIMITER__, __func__) - #define _CL_DBG_ERROR \ ("%s%s%s() !ERROR!: ", __MODULE__, __MOD_DELIMITER__, __func__) - #define CL_CHK_STK - -END_C_DECLS - -#endif /* _CL_DEBUG_OSD_H_ */ + END_C_DECLS +#endif /* _CL_DEBUG_OSD_H_ */ diff --git a/opensm/include/complib/cl_dispatcher.h b/opensm/include/complib/cl_dispatcher.h index ff8e397..70ba4d4 100644 --- a/opensm/include/complib/cl_dispatcher.h +++ b/opensm/include/complib/cl_dispatcher.h @@ -56,13 +56,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Dispatcher * NAME * Dispatcher @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Manipulation: * cl_disp_post, cl_disp_reset, cl_disp_wait_on *********/ - /****s* Component Library: Dispatcher/cl_disp_msgid_t * NAME * cl_disp_msgid_t @@ -141,9 +139,7 @@ typedef uint32_t cl_disp_msgid_t; * SYNOPSIS */ typedef void -(*cl_pfn_msgrcv_cb_t)( - IN void* context, - IN void* p_data ); + (*cl_pfn_msgrcv_cb_t) (IN void *context, IN void *p_data); /* * PARAMETERS * context @@ -181,9 +177,7 @@ typedef void * SYNOPSIS */ typedef void -(*cl_pfn_msgdone_cb_t)( - IN void* context, - IN void* p_data ); + (*cl_pfn_msgdone_cb_t) (IN void *context, IN void *p_data); /* * PARAMETERS * context @@ -220,15 +214,14 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_dispatcher -{ - cl_spinlock_t lock; - cl_ptr_vector_t reg_vec; - cl_qlist_t reg_list; - cl_thread_pool_t worker_threads; - cl_qlist_t msg_fifo; - cl_qpool_t msg_pool; - uint64_t last_msg_queue_time_us; +typedef struct _cl_dispatcher { + cl_spinlock_t lock; + cl_ptr_vector_t reg_vec; + cl_qlist_t reg_list; + cl_thread_pool_t worker_threads; + cl_qlist_t msg_fifo; + cl_qpool_t msg_pool; + uint64_t last_msg_queue_time_us; } cl_dispatcher_t; /* * FIELDS @@ -274,14 +267,13 @@ typedef struct _cl_dispatcher * * SYNOPSIS */ -typedef struct _cl_disp_reg_info -{ - cl_list_item_t list_item; - cl_pfn_msgrcv_cb_t pfn_rcv_callback; - const void *context; - atomic32_t ref_cnt; - cl_disp_msgid_t msg_id; - cl_dispatcher_t *p_disp; +typedef struct _cl_disp_reg_info { + cl_list_item_t list_item; + cl_pfn_msgrcv_cb_t pfn_rcv_callback; + const void *context; + atomic32_t ref_cnt; + cl_disp_msgid_t msg_id; + cl_dispatcher_t *p_disp; } cl_disp_reg_info_t; /* @@ -325,15 +317,14 @@ typedef struct _cl_disp_reg_info * * SYNOPSIS */ -typedef struct _cl_disp_msg -{ - cl_pool_item_t item; - const void *p_data; - cl_disp_reg_info_t *p_src_reg; - cl_disp_reg_info_t *p_dest_reg; - cl_pfn_msgdone_cb_t pfn_xmt_callback; - uint64_t in_time; - const void *context; +typedef struct _cl_disp_msg { + cl_pool_item_t item; + const void *p_data; + cl_disp_reg_info_t *p_src_reg; + cl_disp_reg_info_t *p_dest_reg; + cl_pfn_msgdone_cb_t pfn_xmt_callback; + uint64_t in_time; + const void *context; } cl_disp_msg_t; /* * FIELDS @@ -384,9 +375,7 @@ typedef const struct _cl_disp_reg_info *cl_disp_reg_handle_t; * * SYNOPSIS */ -void -cl_disp_construct( - IN cl_dispatcher_t* const p_disp ); +void cl_disp_construct(IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_disp @@ -412,10 +401,8 @@ cl_disp_construct( * SYNOPSIS */ cl_status_t -cl_disp_init( - IN cl_dispatcher_t* const p_disp, - IN const uint32_t thread_count, - IN const char* const name ); +cl_disp_init(IN cl_dispatcher_t * const p_disp, + IN const uint32_t thread_count, IN const char *const name); /* * PARAMETERS * p_disp @@ -452,9 +439,7 @@ cl_disp_init( * * SYNOPSIS */ -void -cl_disp_shutdown( - IN cl_dispatcher_t* const p_disp ); +void cl_disp_shutdown(IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_disp @@ -481,9 +466,7 @@ cl_disp_shutdown( * * SYNOPSIS */ -void -cl_disp_destroy( - IN cl_dispatcher_t* const p_disp ); +void cl_disp_destroy(IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_disp @@ -506,11 +489,10 @@ cl_disp_destroy( * SYNOPSIS */ cl_disp_reg_handle_t -cl_disp_register( - IN cl_dispatcher_t* const p_disp, - IN const cl_disp_msgid_t msg_id, - IN cl_pfn_msgrcv_cb_t pfn_callback OPTIONAL, - IN const void* const context ); +cl_disp_register(IN cl_dispatcher_t * const p_disp, + IN const cl_disp_msgid_t msg_id, + IN cl_pfn_msgrcv_cb_t pfn_callback OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_disp @@ -550,9 +532,7 @@ cl_disp_register( * * SYNOPSIS */ -void -cl_disp_unregister( - IN const cl_disp_reg_handle_t handle ); +void cl_disp_unregister(IN const cl_disp_reg_handle_t handle); /* * PARAMETERS * handle @@ -580,12 +560,11 @@ cl_disp_unregister( * SYNOPSIS */ cl_status_t -cl_disp_post( - IN const cl_disp_reg_handle_t handle, - IN const cl_disp_msgid_t msg_id, - IN const void* const p_data, - IN cl_pfn_msgdone_cb_t pfn_callback OPTIONAL, - IN const void* const context ); +cl_disp_post(IN const cl_disp_reg_handle_t handle, + IN const cl_disp_msgid_t msg_id, + IN const void *const p_data, + IN cl_pfn_msgdone_cb_t pfn_callback OPTIONAL, + IN const void *const context); /* * PARAMETERS * handle @@ -629,10 +608,9 @@ cl_disp_post( * SYNOPSIS */ void -cl_disp_get_queue_status( - IN const cl_disp_reg_handle_t handle, - OUT uint32_t *p_num_queued_msgs, - OUT uint64_t *p_last_msg_queue_time_ms); +cl_disp_get_queue_status(IN const cl_disp_reg_handle_t handle, + OUT uint32_t * p_num_queued_msgs, + OUT uint64_t * p_last_msg_queue_time_ms); /* * PARAMETERS * handle @@ -656,5 +634,4 @@ cl_disp_get_queue_status( *********/ END_C_DECLS - -#endif /* !defined(_CL_DISPATCHER_H_) */ +#endif /* !defined(_CL_DISPATCHER_H_) */ diff --git a/opensm/include/complib/cl_event.h b/opensm/include/complib/cl_event.h index e379da2..ae9c123 100644 --- a/opensm/include/complib/cl_event.h +++ b/opensm/include/complib/cl_event.h @@ -54,13 +54,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Event * NAME * Event @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Manipulation: * cl_event_signal, cl_event_reset, cl_event_wait_on *********/ - /****f* Component Library: Event/cl_event_construct * NAME * cl_event_construct @@ -92,9 +90,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -void -cl_event_construct( - IN cl_event_t* const p_event ); +void cl_event_construct(IN cl_event_t * const p_event); /* * PARAMETERS * p_event @@ -123,9 +119,7 @@ cl_event_construct( * SYNOPSIS */ cl_status_t -cl_event_init( - IN cl_event_t* const p_event, - IN const boolean_t manual_reset ); +cl_event_init(IN cl_event_t * const p_event, IN const boolean_t manual_reset); /* * PARAMETERS * p_event @@ -161,9 +155,7 @@ cl_event_init( * * SYNOPSIS */ -void -cl_event_destroy( - IN cl_event_t* const p_event ); +void cl_event_destroy(IN cl_event_t * const p_event); /* * PARAMETERS @@ -191,9 +183,7 @@ cl_event_destroy( * * SYNOPSIS */ -cl_status_t -cl_event_signal( - IN cl_event_t* const p_event ); +cl_status_t cl_event_signal(IN cl_event_t * const p_event); /* * PARAMETERS * p_event @@ -226,9 +216,7 @@ cl_event_signal( * * SYNOPSIS */ -cl_status_t -cl_event_reset( - IN cl_event_t* const p_event ); +cl_status_t cl_event_reset(IN cl_event_t * const p_event); /* * PARAMETERS * p_event @@ -254,10 +242,8 @@ cl_event_reset( * SYNOPSIS */ cl_status_t -cl_event_wait_on( - IN cl_event_t* const p_event, - IN const uint32_t wait_us, - IN const boolean_t interruptible ); +cl_event_wait_on(IN cl_event_t * const p_event, + IN const uint32_t wait_us, IN const boolean_t interruptible); /* * PARAMETERS * p_event @@ -295,5 +281,4 @@ cl_event_wait_on( *********/ END_C_DECLS - -#endif /* _CL_EVENT_H_ */ +#endif /* _CL_EVENT_H_ */ diff --git a/opensm/include/complib/cl_event_osd.h b/opensm/include/complib/cl_event_osd.h index 5c10be2..1b8d777 100644 --- a/opensm/include/complib/cl_event_osd.h +++ b/opensm/include/complib/cl_event_osd.h @@ -52,29 +52,25 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - -#include /* usr/include */ - +#include /* usr/include */ /* * Linux user mode specific data structure for the event object. * Users should not access these variables directly. */ -typedef struct _cl_event_t -{ - pthread_cond_t condvar; - boolean_t signaled; - boolean_t manual_reset; - cl_spinlock_t spinlock; - cl_state_t state; + typedef struct _cl_event_t { + pthread_cond_t condvar; + boolean_t signaled; + boolean_t manual_reset; + cl_spinlock_t spinlock; + cl_state_t state; } cl_event_t; END_C_DECLS - -#endif /* _CL_EVENT_OSD_H_ */ +#endif /* _CL_EVENT_OSD_H_ */ diff --git a/opensm/include/complib/cl_event_wheel.h b/opensm/include/complib/cl_event_wheel.h index b8bca05..056ce99 100644 --- a/opensm/include/complib/cl_event_wheel.h +++ b/opensm/include/complib/cl_event_wheel.h @@ -56,13 +56,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Event_Wheel * NAME * Event_Wheel @@ -86,7 +85,6 @@ BEGIN_C_DECLS * cl_event_wheel_reg, cl_event_wheel_unreg * *********/ - /****f* Component Library: Event_Wheel/cl_pfn_event_aged_cb_t * NAME * cl_pfn_event_aged_cb_t @@ -99,10 +97,8 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef uint64_t -(*cl_pfn_event_aged_cb_t)( - IN uint64_t key, - IN uint32_t num_regs, - IN void* context); + (*cl_pfn_event_aged_cb_t) (IN uint64_t key, + IN uint32_t num_regs, IN void *context); /* * PARAMETERS * key @@ -144,16 +140,15 @@ typedef uint64_t * * SYNOPSIS */ -typedef struct _cl_event_wheel -{ - cl_spinlock_t lock; - cl_spinlock_t *p_external_lock; +typedef struct _cl_event_wheel { + cl_spinlock_t lock; + cl_spinlock_t *p_external_lock; - cl_qmap_t events_map; - boolean_t closing; - cl_qlist_t events_wheel; - cl_timer_t timer; - osm_log_t *p_log; + cl_qmap_t events_map; + boolean_t closing; + cl_qlist_t events_wheel; + cl_timer_t timer; + osm_log_t *p_log; } cl_event_wheel_t; /* * FIELDS @@ -196,16 +191,15 @@ typedef struct _cl_event_wheel * * SYNOPSIS */ -typedef struct _cl_event_wheel_reg_info -{ - cl_map_item_t map_item; - cl_list_item_t list_item; - uint64_t key; - cl_pfn_event_aged_cb_t pfn_aged_callback; - uint64_t aging_time; - uint32_t num_regs; - void *context; - cl_event_wheel_t *p_event_wheel; +typedef struct _cl_event_wheel_reg_info { + cl_map_item_t map_item; + cl_list_item_t list_item; + uint64_t key; + cl_pfn_event_aged_cb_t pfn_aged_callback; + uint64_t aging_time; + uint32_t num_regs; + void *context; + cl_event_wheel_t *p_event_wheel; } cl_event_wheel_reg_info_t; /* * FIELDS @@ -245,9 +239,7 @@ typedef struct _cl_event_wheel_reg_info * * SYNOPSIS */ -void -cl_event_wheel_construct( - IN cl_event_wheel_t* const p_event_wheel ); +void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel); /* * PARAMETERS * p_event_wheel @@ -273,9 +265,8 @@ cl_event_wheel_construct( * SYNOPSIS */ cl_status_t -cl_event_wheel_init( - IN cl_event_wheel_t* const p_event_wheel, - IN osm_log_t *p_log); +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, + IN osm_log_t * p_log); /* * PARAMETERS @@ -303,10 +294,9 @@ cl_event_wheel_init( * SYNOPSIS */ cl_status_t -cl_event_wheel_init_ex( - IN cl_event_wheel_t* const p_event_wheel, - IN osm_log_t *p_log, - IN cl_spinlock_t *p_external_lock); +cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, + IN osm_log_t * p_log, + IN cl_spinlock_t * p_external_lock); /* * PARAMETERS @@ -337,9 +327,7 @@ cl_event_wheel_init_ex( * * SYNOPSIS */ -void -cl_event_wheel_destroy( - IN cl_event_wheel_t* const p_event_wheel ); +void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel); /* * PARAMETERS * p_event_wheel @@ -365,9 +353,7 @@ cl_event_wheel_destroy( * * SYNOPSIS */ -void -cl_event_wheel_dump( - IN cl_event_wheel_t* const p_event_wheel ); +void cl_event_wheel_dump(IN cl_event_wheel_t * const p_event_wheel); /* * PARAMETERS * p_event_wheel @@ -394,12 +380,11 @@ cl_event_wheel_dump( * SYNOPSIS */ cl_status_t -cl_event_wheel_reg( - IN cl_event_wheel_t* const p_event_wheel, - IN const uint64_t key, - IN const uint64_t aging_time_usec, - IN cl_pfn_event_aged_cb_t pfn_callback, - IN void* const context ); +cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, + IN const uint64_t key, + IN const uint64_t aging_time_usec, + IN cl_pfn_event_aged_cb_t pfn_callback, + IN void *const context); /* * PARAMETERS * p_event_wheel @@ -436,9 +421,8 @@ cl_event_wheel_reg( * SYNOPSIS */ void -cl_event_wheel_unreg( - IN cl_event_wheel_t* const p_event_wheel, - IN uint64_t key ); +cl_event_wheel_unreg(IN cl_event_wheel_t * const p_event_wheel, + IN uint64_t key); /* * PARAMETERS * p_event_wheel @@ -469,9 +453,8 @@ cl_event_wheel_unreg( * SYNOPSIS */ uint32_t -cl_event_wheel_num_regs( - IN cl_event_wheel_t* const p_event_wheel, - IN uint64_t key ); +cl_event_wheel_num_regs(IN cl_event_wheel_t * const p_event_wheel, + IN uint64_t key); /* * PARAMETERS * p_event_wheel @@ -489,5 +472,4 @@ cl_event_wheel_num_regs( *********/ END_C_DECLS - -#endif /* !defined(_CL_EVENT_WHEEL_H_) */ +#endif /* !defined(_CL_EVENT_WHEEL_H_) */ diff --git a/opensm/include/complib/cl_fleximap.h b/opensm/include/complib/cl_fleximap.h index 72e2de2..d98c7fd 100644 --- a/opensm/include/complib/cl_fleximap.h +++ b/opensm/include/complib/cl_fleximap.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Flexi Map * NAME * Flexi Map @@ -109,7 +108,6 @@ BEGIN_C_DECLS * Attributes: * cl_fmap_count, cl_is_fmap_empty, *********/ - /****s* Component Library: Flexi Map/cl_fmap_item_t * NAME * cl_fmap_item_t @@ -122,17 +120,16 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_fmap_item -{ + typedef struct _cl_fmap_item { /* Must be first to allow casting. */ - cl_pool_item_t pool_item; - struct _cl_fmap_item *p_left; - struct _cl_fmap_item *p_right; - struct _cl_fmap_item *p_up; - cl_map_color_t color; - const void *p_key; + cl_pool_item_t pool_item; + struct _cl_fmap_item *p_left; + struct _cl_fmap_item *p_right; + struct _cl_fmap_item *p_up; + cl_map_color_t color; + const void *p_key; #ifdef _DEBUG_ - struct _cl_fmap *p_map; + struct _cl_fmap *p_map; #endif } cl_fmap_item_t; @@ -191,9 +188,8 @@ typedef struct _cl_fmap_item * SYNOPSIS */ typedef intn_t -(*cl_pfn_fmap_cmp_t)( - IN const void* const p_key1, - IN const void* const p_key2 ); + (*cl_pfn_fmap_cmp_t) (IN const void *const p_key1, + IN const void *const p_key2); /* * PARAMETERS * p_key1 @@ -227,13 +223,12 @@ typedef intn_t * * SYNOPSIS */ -typedef struct _cl_fmap -{ - cl_fmap_item_t root; - cl_fmap_item_t nil; - cl_state_t state; - size_t count; - cl_pfn_fmap_cmp_t pfn_compare; +typedef struct _cl_fmap { + cl_fmap_item_t root; + cl_fmap_item_t nil; + cl_state_t state; + size_t count; + cl_pfn_fmap_cmp_t pfn_compare; } cl_fmap_t; /* @@ -273,9 +268,8 @@ typedef struct _cl_fmap * SYNOPSIS */ typedef void -(*cl_pfn_fmap_apply_t)( - IN cl_fmap_item_t* const p_map_item, - IN void* context ); + (*cl_pfn_fmap_apply_t) (IN cl_fmap_item_t * const p_map_item, + IN void *context); /* * PARAMETERS * p_map_item @@ -306,14 +300,13 @@ typedef void * * SYNOPSIS */ -static inline size_t -cl_fmap_count( - IN const cl_fmap_t* const p_map ) +static inline size_t cl_fmap_count(IN const cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( p_map->count ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return (p_map->count); } + /* * PARAMETERS * p_map @@ -335,15 +328,14 @@ cl_fmap_count( * * SYNOPSIS */ -static inline boolean_t -cl_is_fmap_empty( - IN const cl_fmap_t* const p_map ) +static inline boolean_t cl_is_fmap_empty(IN const cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - return( p_map->count == 0 ); + return (p_map->count == 0); } + /* * PARAMETERS * p_map @@ -367,13 +359,12 @@ cl_is_fmap_empty( * * SYNOPSIS */ -static inline const void* -cl_fmap_key( - IN const cl_fmap_item_t* const p_item ) +static inline const void *cl_fmap_key(IN const cl_fmap_item_t * const p_item) { - CL_ASSERT( p_item ); - return( p_item->p_key ); + CL_ASSERT(p_item); + return (p_item->p_key); } + /* * PARAMETERS * p_item @@ -399,10 +390,7 @@ cl_fmap_key( * * SYNOPSIS */ -void -cl_fmap_init( - IN cl_fmap_t* const p_map, - IN cl_pfn_fmap_cmp_t pfn_compare ); +void cl_fmap_init(IN cl_fmap_t * const p_map, IN cl_pfn_fmap_cmp_t pfn_compare); /* * PARAMETERS * p_map @@ -432,15 +420,15 @@ cl_fmap_init( * * SYNOPSIS */ -static inline const cl_fmap_item_t* -cl_fmap_end( - IN const cl_fmap_t* const p_map ) +static inline const cl_fmap_item_t *cl_fmap_end(IN const cl_fmap_t * + const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); /* Nil is the end of the map. */ - return( &p_map->nil ); + return (&p_map->nil); } + /* * PARAMETERS * p_map @@ -471,14 +459,13 @@ cl_fmap_end( * * SYNOPSIS */ -static inline cl_fmap_item_t* -cl_fmap_head( - IN const cl_fmap_t* const p_map ) +static inline cl_fmap_item_t *cl_fmap_head(IN const cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( (cl_fmap_item_t*)p_map->nil.pool_item.list_item.p_next ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return ((cl_fmap_item_t *) p_map->nil.pool_item.list_item.p_next); } + /* * PARAMETERS * p_map @@ -508,14 +495,13 @@ cl_fmap_head( * * SYNOPSIS */ -static inline cl_fmap_item_t* -cl_fmap_tail( - IN const cl_fmap_t* const p_map ) +static inline cl_fmap_item_t *cl_fmap_tail(IN const cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( (cl_fmap_item_t*)p_map->nil.pool_item.list_item.p_prev ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return ((cl_fmap_item_t *) p_map->nil.pool_item.list_item.p_prev); } + /* * PARAMETERS * p_map @@ -545,13 +531,13 @@ cl_fmap_tail( * * SYNOPSIS */ -static inline cl_fmap_item_t* -cl_fmap_next( - IN const cl_fmap_item_t* const p_item ) +static inline cl_fmap_item_t *cl_fmap_next(IN const cl_fmap_item_t * + const p_item) { - CL_ASSERT( p_item ); - return( (cl_fmap_item_t*)p_item->pool_item.list_item.p_next ); + CL_ASSERT(p_item); + return ((cl_fmap_item_t *) p_item->pool_item.list_item.p_next); } + /* * PARAMETERS * p_item @@ -578,13 +564,13 @@ cl_fmap_next( * * SYNOPSIS */ -static inline cl_fmap_item_t* -cl_fmap_prev( - IN const cl_fmap_item_t* const p_item ) +static inline cl_fmap_item_t *cl_fmap_prev(IN const cl_fmap_item_t * + const p_item) { - CL_ASSERT( p_item ); - return( (cl_fmap_item_t*)p_item->pool_item.list_item.p_prev ); + CL_ASSERT(p_item); + return ((cl_fmap_item_t *) p_item->pool_item.list_item.p_prev); } + /* * PARAMETERS * p_item @@ -610,11 +596,9 @@ cl_fmap_prev( * * SYNOPSIS */ -cl_fmap_item_t* -cl_fmap_insert( - IN cl_fmap_t* const p_map, - IN const void* const p_key, - IN cl_fmap_item_t* const p_item ); +cl_fmap_item_t *cl_fmap_insert(IN cl_fmap_t * const p_map, + IN const void *const p_key, + IN cl_fmap_item_t * const p_item); /* * PARAMETERS * p_map @@ -651,10 +635,8 @@ cl_fmap_insert( * * SYNOPSIS */ -cl_fmap_item_t* -cl_fmap_get( - IN const cl_fmap_t* const p_map, - IN const void* const p_key ); +cl_fmap_item_t *cl_fmap_get(IN const cl_fmap_t * const p_map, + IN const void *const p_key); /* * PARAMETERS * p_map @@ -687,10 +669,8 @@ cl_fmap_get( * * SYNOPSIS */ -cl_fmap_item_t* -cl_fmap_get_next( - IN const cl_fmap_t* const p_map, - IN const void* const p_key ); +cl_fmap_item_t *cl_fmap_get_next(IN const cl_fmap_t * const p_map, + IN const void *const p_key); /* * PARAMETERS * p_map @@ -724,9 +704,8 @@ cl_fmap_get_next( * SYNOPSIS */ void -cl_fmap_remove_item( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* const p_item ); +cl_fmap_remove_item(IN cl_fmap_t * const p_map, + IN cl_fmap_item_t * const p_item); /* * PARAMETERS * p_item @@ -755,10 +734,8 @@ cl_fmap_remove_item( * * SYNOPSIS */ -cl_fmap_item_t* -cl_fmap_remove( - IN cl_fmap_t* const p_map, - IN const void* const p_key ); +cl_fmap_item_t *cl_fmap_remove(IN cl_fmap_t * const p_map, + IN const void *const p_key); /* * PARAMETERS * p_map @@ -789,18 +766,17 @@ cl_fmap_remove( * * SYNOPSIS */ -static inline void -cl_fmap_remove_all( - IN cl_fmap_t* const p_map ) +static inline void cl_fmap_remove_all(IN cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); p_map->root.p_left = &p_map->nil; p_map->nil.pool_item.list_item.p_next = &p_map->nil.pool_item.list_item; p_map->nil.pool_item.list_item.p_prev = &p_map->nil.pool_item.list_item; p_map->count = 0; } + /* * PARAMETERS * p_map @@ -824,9 +800,8 @@ cl_fmap_remove_all( * SYNOPSIS */ void -cl_fmap_merge( - OUT cl_fmap_t* const p_dest_map, - IN OUT cl_fmap_t* const p_src_map ); +cl_fmap_merge(OUT cl_fmap_t * const p_dest_map, + IN OUT cl_fmap_t * const p_src_map); /* * PARAMETERS * p_dest_map @@ -859,11 +834,9 @@ cl_fmap_merge( * SYNOPSIS */ void -cl_fmap_delta( - IN OUT cl_fmap_t* const p_map1, - IN OUT cl_fmap_t* const p_map2, - OUT cl_fmap_t* const p_new, - OUT cl_fmap_t* const p_old ); +cl_fmap_delta(IN OUT cl_fmap_t * const p_map1, + IN OUT cl_fmap_t * const p_map2, + OUT cl_fmap_t * const p_new, OUT cl_fmap_t * const p_old); /* * PARAMETERS * p_map1 @@ -910,10 +883,9 @@ cl_fmap_delta( * SYNOPSIS */ void -cl_fmap_apply_func( - IN const cl_fmap_t* const p_map, - IN cl_pfn_fmap_apply_t pfn_func, - IN const void* const context ); +cl_fmap_apply_func(IN const cl_fmap_t * const p_map, + IN cl_pfn_fmap_apply_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_map @@ -939,5 +911,4 @@ cl_fmap_apply_func( *********/ END_C_DECLS - -#endif /* _CL_FLEXIMAP_H_ */ +#endif /* _CL_FLEXIMAP_H_ */ diff --git a/opensm/include/complib/cl_list.h b/opensm/include/complib/cl_list.h index 1020e28..4b8cdfa 100644 --- a/opensm/include/complib/cl_list.h +++ b/opensm/include/complib/cl_list.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/List * NAME * List @@ -109,7 +108,6 @@ BEGIN_C_DECLS * Attributes: * cl_list_count, cl_is_list_empty, cl_is_list_inited *********/ - /****s* Component Library: List/cl_list_t * NAME * cl_list_t @@ -122,10 +120,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_list -{ - cl_qlist_t list; - cl_qpool_t list_item_pool; + typedef struct _cl_list { + cl_qlist_t list; + cl_qpool_t list_item_pool; } cl_list_t; /* @@ -170,9 +167,7 @@ typedef const cl_list_item_t *cl_list_iterator_t; * SYNOPSIS */ typedef void -(*cl_pfn_list_apply_t)( - IN void* const p_object, - IN void* context ); + (*cl_pfn_list_apply_t) (IN void *const p_object, IN void *context); /* * PARAMETERS * p_object @@ -204,9 +199,7 @@ typedef void * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_list_find_t)( - IN const void* const p_object, - IN void* context ); + (*cl_pfn_list_find_t) (IN const void *const p_object, IN void *context); /* * PARAMETERS * p_object @@ -238,9 +231,7 @@ typedef cl_status_t * * SYNOPSIS */ -void -cl_list_construct( - IN cl_list_t* const p_list ); +void cl_list_construct(IN cl_list_t * const p_list); /* * PARAMETERS * p_list @@ -269,18 +260,17 @@ cl_list_construct( * * SYNOPSIS */ -static inline boolean_t -cl_is_list_inited( - IN const cl_list_t* const p_list ) +static inline boolean_t cl_is_list_inited(IN const cl_list_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* * The pool is the last thing initialized. If it is initialized, the * list is initialized too. */ - return( cl_is_qpool_inited( &p_list->list_item_pool ) ); + return (cl_is_qpool_inited(&p_list->list_item_pool)); } + /* * PARAMETERS * p_list @@ -310,9 +300,7 @@ cl_is_list_inited( * SYNOPSIS */ cl_status_t -cl_list_init( - IN cl_list_t* const p_list, - IN const size_t min_items ); +cl_list_init(IN cl_list_t * const p_list, IN const size_t min_items); /* * PARAMETERS * p_list @@ -346,9 +334,7 @@ cl_list_init( * * SYNOPSIS */ -void -cl_list_destroy( - IN cl_list_t* const p_list ); +void cl_list_destroy(IN cl_list_t * const p_list); /* * PARAMETERS * p_list @@ -380,14 +366,13 @@ cl_list_destroy( * * SYNOPSIS */ -static inline boolean_t -cl_is_list_empty( - IN const cl_list_t* const p_list ) +static inline boolean_t cl_is_list_empty(IN const cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - return( cl_is_qlist_empty( &p_list->list ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + return (cl_is_qlist_empty(&p_list->list)); } + /* * PARAMETERS * p_list @@ -412,24 +397,23 @@ cl_is_list_empty( * SYNOPSIS */ static inline cl_status_t -cl_list_insert_head( - IN cl_list_t* const p_list, - IN const void* const p_object ) +cl_list_insert_head(IN cl_list_t * const p_list, IN const void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* Get a list item to add to the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qpool_get( &p_list->list_item_pool ); - if( !p_pool_obj ) - return( CL_INSUFFICIENT_MEMORY ); + p_pool_obj = (cl_pool_obj_t *) cl_qpool_get(&p_list->list_item_pool); + if (!p_pool_obj) + return (CL_INSUFFICIENT_MEMORY); p_pool_obj->p_object = p_object; - cl_qlist_insert_head( &p_list->list, &p_pool_obj->pool_item.list_item ); - return( CL_SUCCESS ); + cl_qlist_insert_head(&p_list->list, &p_pool_obj->pool_item.list_item); + return (CL_SUCCESS); } + /* * PARAMETERS * p_list @@ -464,24 +448,23 @@ cl_list_insert_head( * SYNOPSIS */ static inline cl_status_t -cl_list_insert_tail( - IN cl_list_t* const p_list, - IN const void* const p_object ) +cl_list_insert_tail(IN cl_list_t * const p_list, IN const void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* Get a list item to add to the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qpool_get( &p_list->list_item_pool ); - if( !p_pool_obj ) - return( CL_INSUFFICIENT_MEMORY ); + p_pool_obj = (cl_pool_obj_t *) cl_qpool_get(&p_list->list_item_pool); + if (!p_pool_obj) + return (CL_INSUFFICIENT_MEMORY); p_pool_obj->p_object = p_object; - cl_qlist_insert_tail( &p_list->list, &p_pool_obj->pool_item.list_item ); - return( CL_SUCCESS ); + cl_qlist_insert_tail(&p_list->list, &p_pool_obj->pool_item.list_item); + return (CL_SUCCESS); } + /* * PARAMETERS * p_list @@ -517,11 +500,9 @@ cl_list_insert_tail( * SYNOPSIS */ cl_status_t -cl_list_insert_array_head( - IN cl_list_t* const p_list, - IN const void* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ); +cl_list_insert_array_head(IN cl_list_t * const p_list, + IN const void *const p_array, + IN uint32_t item_count, IN const uint32_t item_size); /* * PARAMETERS * p_list @@ -564,11 +545,9 @@ cl_list_insert_array_head( * SYNOPSIS */ cl_status_t -cl_list_insert_array_tail( - IN cl_list_t* const p_list, - IN const void* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size); +cl_list_insert_array_tail(IN cl_list_t * const p_list, + IN const void *const p_array, + IN uint32_t item_count, IN const uint32_t item_size); /* * PARAMETERS * p_list @@ -611,26 +590,26 @@ cl_list_insert_array_tail( * SYNOPSIS */ static inline cl_status_t -cl_list_insert_next( - IN cl_list_t* const p_list, - IN cl_list_iterator_t iterator, - IN const void* const p_object ) +cl_list_insert_next(IN cl_list_t * const p_list, + IN cl_list_iterator_t iterator, + IN const void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* Get a list item to add to the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qpool_get( &p_list->list_item_pool ); - if( !p_pool_obj ) - return( CL_INSUFFICIENT_MEMORY ); + p_pool_obj = (cl_pool_obj_t *) cl_qpool_get(&p_list->list_item_pool); + if (!p_pool_obj) + return (CL_INSUFFICIENT_MEMORY); p_pool_obj->p_object = p_object; - cl_qlist_insert_next( &p_list->list, (cl_list_item_t*)iterator, - &p_pool_obj->pool_item.list_item ); - return( CL_SUCCESS ); + cl_qlist_insert_next(&p_list->list, (cl_list_item_t *) iterator, + &p_pool_obj->pool_item.list_item); + return (CL_SUCCESS); } + /* * PARAMETERS * p_list @@ -664,26 +643,26 @@ cl_list_insert_next( * SYNOPSIS */ static inline cl_status_t -cl_list_insert_prev( - IN cl_list_t* const p_list, - IN cl_list_iterator_t iterator, - IN const void* const p_object ) +cl_list_insert_prev(IN cl_list_t * const p_list, + IN cl_list_iterator_t iterator, + IN const void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* Get a list item to add to the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qpool_get( &p_list->list_item_pool ); - if( !p_pool_obj ) - return( CL_INSUFFICIENT_MEMORY ); + p_pool_obj = (cl_pool_obj_t *) cl_qpool_get(&p_list->list_item_pool); + if (!p_pool_obj) + return (CL_INSUFFICIENT_MEMORY); p_pool_obj->p_object = p_object; - cl_qlist_insert_prev( &p_list->list, (cl_list_item_t*)iterator, - &p_pool_obj->pool_item.list_item ); - return( CL_SUCCESS ); + cl_qlist_insert_prev(&p_list->list, (cl_list_item_t *) iterator, + &p_pool_obj->pool_item.list_item); + return (CL_SUCCESS); } + /* * PARAMETERS * p_list @@ -715,29 +694,28 @@ cl_list_insert_prev( * * SYNOPSIS */ -static inline void* -cl_list_remove_head( - IN cl_list_t* const p_list ) +static inline void *cl_list_remove_head(IN cl_list_t * const p_list) { - cl_pool_obj_t *p_pool_obj; - void* p_obj; + cl_pool_obj_t *p_pool_obj; + void *p_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* See if the list is empty. */ - if( cl_is_qlist_empty( &p_list->list ) ) - return( NULL ); + if (cl_is_qlist_empty(&p_list->list)) + return (NULL); /* Get the item at the head of the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qlist_remove_head( &p_list->list ); + p_pool_obj = (cl_pool_obj_t *) cl_qlist_remove_head(&p_list->list); - p_obj = (void*)p_pool_obj->p_object; + p_obj = (void *)p_pool_obj->p_object; /* Place the pool item back into the pool. */ - cl_qpool_put( &p_list->list_item_pool, &p_pool_obj->pool_item ); + cl_qpool_put(&p_list->list_item_pool, &p_pool_obj->pool_item); - return( p_obj ); + return (p_obj); } + /* * PARAMETERS * p_list @@ -762,27 +740,26 @@ cl_list_remove_head( * * SYNOPSIS */ -static inline void* -cl_list_remove_tail( - IN cl_list_t* const p_list ) +static inline void *cl_list_remove_tail(IN cl_list_t * const p_list) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* See if the list is empty. */ - if( cl_is_qlist_empty( &p_list->list ) ) - return( NULL ); + if (cl_is_qlist_empty(&p_list->list)) + return (NULL); /* Get the item at the head of the list. */ - p_pool_obj = (cl_pool_obj_t*)cl_qlist_remove_tail( &p_list->list ); + p_pool_obj = (cl_pool_obj_t *) cl_qlist_remove_tail(&p_list->list); /* Place the list item back into the pool. */ - cl_qpool_put( &p_list->list_item_pool, &p_pool_obj->pool_item ); + cl_qpool_put(&p_list->list_item_pool, &p_pool_obj->pool_item); - return( (void*)p_pool_obj->p_object ); + return ((void *)p_pool_obj->p_object); } + /* * PARAMETERS * p_list @@ -808,16 +785,15 @@ cl_list_remove_tail( * * SYNOPSIS */ -static inline void -cl_list_remove_all( - IN cl_list_t* const p_list ) +static inline void cl_list_remove_all(IN cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* Return all the list items to the pool. */ - cl_qpool_put_list( &p_list->list_item_pool, &p_list->list ); + cl_qpool_put_list(&p_list->list_item_pool, &p_list->list); } + /* * PARAMETERS * p_list @@ -841,9 +817,8 @@ cl_list_remove_all( * SYNOPSIS */ cl_status_t -cl_list_remove_object( - IN cl_list_t* const p_list, - IN const void* const p_object ); +cl_list_remove_object(IN cl_list_t * const p_list, + IN const void *const p_object); /* * PARAMETERS * p_list @@ -875,18 +850,17 @@ cl_list_remove_object( * SYNOPSIS */ static inline void -cl_list_remove_item( - IN cl_list_t* const p_list, - IN cl_list_iterator_t iterator ) +cl_list_remove_item(IN cl_list_t * const p_list, IN cl_list_iterator_t iterator) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - cl_qlist_remove_item( &p_list->list, (cl_list_item_t*)iterator ); + cl_qlist_remove_item(&p_list->list, (cl_list_item_t *) iterator); /* Place the list item back into the pool. */ - cl_qpool_put( &p_list->list_item_pool, (cl_pool_item_t*)iterator ); + cl_qpool_put(&p_list->list_item_pool, (cl_pool_item_t *) iterator); } + /* * PARAMETERS * p_list @@ -915,9 +889,8 @@ cl_list_remove_item( * SYNOPSIS */ boolean_t -cl_is_object_in_list( - IN const cl_list_t* const p_list, - IN const void* const p_object ); +cl_is_object_in_list(IN const cl_list_t * const p_list, + IN const void *const p_object); /* * PARAMETERS * p_list @@ -945,15 +918,14 @@ cl_is_object_in_list( * * SYNOPSIS */ -static inline cl_list_iterator_t -cl_list_end( - IN const cl_list_t* const p_list ) +static inline cl_list_iterator_t cl_list_end(IN const cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - return( cl_qlist_end( &p_list->list ) ); + return (cl_qlist_end(&p_list->list)); } + /* * PARAMETERS * p_list @@ -982,15 +954,14 @@ cl_list_end( * * SYNOPSIS */ -static inline cl_list_iterator_t -cl_list_head( - IN const cl_list_t* const p_list ) +static inline cl_list_iterator_t cl_list_head(IN const cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - return( cl_qlist_head( &p_list->list ) ); + return (cl_qlist_head(&p_list->list)); } + /* * PARAMETERS * p_list @@ -1021,15 +992,14 @@ cl_list_head( * * SYNOPSIS */ -static inline cl_list_iterator_t -cl_list_tail( - IN const cl_list_t* const p_list ) +static inline cl_list_iterator_t cl_list_tail(IN const cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - return( cl_qlist_tail( &p_list->list ) ); + return (cl_qlist_tail(&p_list->list)); } + /* * PARAMETERS * p_list @@ -1061,14 +1031,13 @@ cl_list_tail( * * SYNOPSIS */ -static inline cl_list_iterator_t -cl_list_next( - IN cl_list_iterator_t iterator ) +static inline cl_list_iterator_t cl_list_next(IN cl_list_iterator_t iterator) { - CL_ASSERT( iterator ); + CL_ASSERT(iterator); - return( cl_qlist_next( iterator ) ); + return (cl_qlist_next(iterator)); } + /* * PARAMETERS * p_list @@ -1104,14 +1073,13 @@ cl_list_next( * * SYNOPSIS */ -static inline cl_list_iterator_t -cl_list_prev( - IN cl_list_iterator_t iterator ) +static inline cl_list_iterator_t cl_list_prev(IN cl_list_iterator_t iterator) { - CL_ASSERT( iterator ); + CL_ASSERT(iterator); - return( cl_qlist_prev( iterator ) ); + return (cl_qlist_prev(iterator)); } + /* * PARAMETERS * p_list @@ -1147,14 +1115,13 @@ cl_list_prev( * * SYNOPSIS */ -static inline void* -cl_list_obj( - IN cl_list_iterator_t iterator ) +static inline void *cl_list_obj(IN cl_list_iterator_t iterator) { - CL_ASSERT( iterator ); + CL_ASSERT(iterator); - return( (void*)((cl_pool_obj_t*)iterator)->p_object ); + return ((void *)((cl_pool_obj_t *) iterator)->p_object); } + /* * PARAMETERS * iterator @@ -1180,10 +1147,9 @@ cl_list_obj( * SYNOPSIS */ cl_list_iterator_t -cl_list_find_from_head( - IN const cl_list_t* const p_list, - IN cl_pfn_list_find_t pfn_func, - IN const void* const context ); +cl_list_find_from_head(IN const cl_list_t * const p_list, + IN cl_pfn_list_find_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1225,10 +1191,9 @@ cl_list_find_from_head( * SYNOPSIS */ cl_list_iterator_t -cl_list_find_from_tail( - IN const cl_list_t* const p_list, - IN cl_pfn_list_find_t pfn_func, - IN const void* const context ); +cl_list_find_from_tail(IN const cl_list_t * const p_list, + IN cl_pfn_list_find_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1270,10 +1235,9 @@ cl_list_find_from_tail( * SYNOPSIS */ void -cl_list_apply_func( - IN const cl_list_t* const p_list, - IN cl_pfn_list_apply_t pfn_func, - IN const void* const context ); +cl_list_apply_func(IN const cl_list_t * const p_list, + IN cl_pfn_list_apply_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1310,15 +1274,14 @@ cl_list_apply_func( * * SYNOPSIS */ -static inline size_t -cl_list_count( - IN const cl_list_t* const p_list ) +static inline size_t cl_list_count(IN const cl_list_t * const p_list) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - return( cl_qlist_count( &p_list->list ) ); + return (cl_qlist_count(&p_list->list)); } + /* * PARAMETERS * p_list @@ -1332,5 +1295,4 @@ cl_list_count( *********/ END_C_DECLS - -#endif /* _CL_LIST_H_ */ +#endif /* _CL_LIST_H_ */ diff --git a/opensm/include/complib/cl_log.h b/opensm/include/complib/cl_log.h index 2ab34da..4ea14ce 100644 --- a/opensm/include/complib/cl_log.h +++ b/opensm/include/complib/cl_log.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Log Provider * NAME * Log Provider @@ -66,7 +65,6 @@ BEGIN_C_DECLS * The log provider allows users to log information in a system log instead of * the console or debugger target. **********/ - /****d* Component Library: Log Provider/cl_log_type_t * NAME * cl_log_type_t @@ -77,12 +75,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef enum _cl_log_type -{ + typedef enum _cl_log_type { CL_LOG_INFO, CL_LOG_WARN, CL_LOG_ERROR - } cl_log_type_t; /* * VALUES @@ -109,12 +105,10 @@ typedef enum _cl_log_type * SYNOPSIS */ void -cl_log_event( - IN const char* const name, - IN const cl_log_type_t type, - IN const char* const message, - IN const void* const p_data OPTIONAL, - IN const uint32_t data_len ); +cl_log_event(IN const char *const name, + IN const cl_log_type_t type, + IN const char *const message, + IN const void *const p_data OPTIONAL, IN const uint32_t data_len); /* * PARAMETERS * name @@ -150,5 +144,4 @@ cl_log_event( *********/ END_C_DECLS - -#endif /* _CL_LOG_H_ */ +#endif /* _CL_LOG_H_ */ diff --git a/opensm/include/complib/cl_map.h b/opensm/include/complib/cl_map.h index 3ce6e54..6112a08 100644 --- a/opensm/include/complib/cl_map.h +++ b/opensm/include/complib/cl_map.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Map * NAME * Map @@ -101,7 +100,6 @@ BEGIN_C_DECLS * Attributes: * cl_map_count, cl_is_map_empty, cl_is_map_inited *********/ - /****s* Component Library: Map/cl_map_t * NAME * cl_map_t @@ -114,10 +112,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_map -{ - cl_qmap_t qmap; - cl_qpool_t pool; + typedef struct _cl_map { + cl_qmap_t qmap; + cl_qpool_t pool; } cl_map_t; /* @@ -161,13 +158,12 @@ typedef const cl_map_item_t *cl_map_iterator_t; * * SYNOPSIS */ -static inline size_t -cl_map_count( - IN const cl_map_t* const p_map ) +static inline size_t cl_map_count(IN const cl_map_t * const p_map) { - CL_ASSERT( p_map ); - return( cl_qmap_count( &p_map->qmap ) ); + CL_ASSERT(p_map); + return (cl_qmap_count(&p_map->qmap)); } + /* * PARAMETERS * p_map @@ -189,13 +185,12 @@ cl_map_count( * * SYNOPSIS */ -static inline boolean_t -cl_is_map_empty( - IN const cl_map_t* const p_map ) +static inline boolean_t cl_is_map_empty(IN const cl_map_t * const p_map) { - CL_ASSERT( p_map ); - return( cl_is_qmap_empty( &p_map->qmap ) ); + CL_ASSERT(p_map); + return (cl_is_qmap_empty(&p_map->qmap)); } + /* * PARAMETERS * p_map @@ -219,12 +214,11 @@ cl_is_map_empty( * * SYNOPSIS */ -static inline uint64_t -cl_map_key( - IN const cl_map_iterator_t itor ) +static inline uint64_t cl_map_key(IN const cl_map_iterator_t itor) { - return( cl_qmap_key( itor ) ); + return (cl_qmap_key(itor)); } + /* * PARAMETERS * itor @@ -252,9 +246,7 @@ cl_map_key( * * SYNOPSIS */ -void -cl_map_construct( - IN cl_map_t* const p_map ); +void cl_map_construct(IN cl_map_t * const p_map); /* * PARAMETERS * p_map @@ -283,16 +275,15 @@ cl_map_construct( * * SYNOPSIS */ -static inline boolean_t -cl_is_map_inited( - IN const cl_map_t* const p_map ) +static inline boolean_t cl_is_map_inited(IN const cl_map_t * const p_map) { /* * The map's pool of map items is the last thing initialized. * We can therefore use it to test for initialization. */ - return( cl_is_qpool_inited( &p_map->pool ) ); + return (cl_is_qpool_inited(&p_map->pool)); } + /* * PARAMETERS * p_map @@ -321,10 +312,7 @@ cl_is_map_inited( * * SYNOPSIS */ -cl_status_t -cl_map_init( - IN cl_map_t* const p_map, - IN const uint32_t min_items ); +cl_status_t cl_map_init(IN cl_map_t * const p_map, IN const uint32_t min_items); /* * PARAMETERS * p_map @@ -354,9 +342,7 @@ cl_map_init( * * SYNOPSIS */ -void -cl_map_destroy( - IN cl_map_t* const p_map ); +void cl_map_destroy(IN cl_map_t * const p_map); /* * PARAMETERS * p_map @@ -386,13 +372,12 @@ cl_map_destroy( * * SYNOPSIS */ -static inline cl_map_iterator_t -cl_map_end( - IN const cl_map_t* const p_map ) +static inline cl_map_iterator_t cl_map_end(IN const cl_map_t * const p_map) { - CL_ASSERT( p_map ); - return( cl_qmap_end( &p_map->qmap ) ); + CL_ASSERT(p_map); + return (cl_qmap_end(&p_map->qmap)); } + /* * PARAMETERS * p_map @@ -423,13 +408,12 @@ cl_map_end( * * SYNOPSIS */ -static inline cl_map_iterator_t -cl_map_head( - IN const cl_map_t* const p_map ) +static inline cl_map_iterator_t cl_map_head(IN const cl_map_t * const p_map) { - CL_ASSERT( p_map ); - return( cl_qmap_head( &p_map->qmap ) ); + CL_ASSERT(p_map); + return (cl_qmap_head(&p_map->qmap)); } + /* * PARAMETERS * p_map @@ -457,13 +441,12 @@ cl_map_head( * * SYNOPSIS */ -static inline cl_map_iterator_t -cl_map_tail( - IN const cl_map_t* const p_map ) +static inline cl_map_iterator_t cl_map_tail(IN const cl_map_t * const p_map) { - CL_ASSERT( p_map ); - return( cl_qmap_tail( &p_map->qmap ) ); + CL_ASSERT(p_map); + return (cl_qmap_tail(&p_map->qmap)); } + /* * PARAMETERS * p_map @@ -492,13 +475,12 @@ cl_map_tail( * * SYNOPSIS */ -static inline cl_map_iterator_t -cl_map_next( - IN const cl_map_iterator_t itor ) +static inline cl_map_iterator_t cl_map_next(IN const cl_map_iterator_t itor) { - CL_ASSERT( itor ); - return( cl_qmap_next( itor ) ); + CL_ASSERT(itor); + return (cl_qmap_next(itor)); } + /* * PARAMETERS * itor @@ -528,13 +510,12 @@ cl_map_next( * * SYNOPSIS */ -static inline cl_map_iterator_t -cl_map_prev( - IN const cl_map_iterator_t itor ) +static inline cl_map_iterator_t cl_map_prev(IN const cl_map_iterator_t itor) { - CL_ASSERT( itor ); - return( cl_qmap_prev( itor ) ); + CL_ASSERT(itor); + return (cl_qmap_prev(itor)); } + /* * PARAMETERS * itor @@ -563,11 +544,8 @@ cl_map_prev( * * SYNOPSIS */ -void* -cl_map_insert( - IN cl_map_t* const p_map, - IN const uint64_t key, - IN const void* const p_object ); +void *cl_map_insert(IN cl_map_t * const p_map, + IN const uint64_t key, IN const void *const p_object); /* * PARAMETERS * p_map @@ -605,10 +583,7 @@ cl_map_insert( * * SYNOPSIS */ -void* -cl_map_get( - IN const cl_map_t* const p_map, - IN const uint64_t key ); +void *cl_map_get(IN const cl_map_t * const p_map, IN const uint64_t key); /* * PARAMETERS * p_map @@ -641,10 +616,7 @@ cl_map_get( * * SYNOPSIS */ -void* -cl_map_get_next( - IN const cl_map_t* const p_map, - IN const uint64_t key ); +void *cl_map_get_next(IN const cl_map_t * const p_map, IN const uint64_t key); /* * PARAMETERS * p_map @@ -678,9 +650,7 @@ cl_map_get_next( * SYNOPSIS */ void -cl_map_remove_item( - IN cl_map_t* const p_map, - IN const cl_map_iterator_t itor ); +cl_map_remove_item(IN cl_map_t * const p_map, IN const cl_map_iterator_t itor); /* * PARAMETERS * p_map @@ -716,10 +686,7 @@ cl_map_remove_item( * * SYNOPSIS */ -void* -cl_map_remove( - IN cl_map_t* const p_map, - IN const uint64_t key ); +void *cl_map_remove(IN cl_map_t * const p_map, IN const uint64_t key); /* * PARAMETERS * p_map @@ -749,9 +716,7 @@ cl_map_remove( * * SYNOPSIS */ -void -cl_map_remove_all( - IN cl_map_t* const p_map ); +void cl_map_remove_all(IN cl_map_t * const p_map); /* * PARAMETERS * p_map @@ -773,12 +738,11 @@ cl_map_remove_all( * * SYNOPSIS */ -static inline void* -cl_map_obj( - IN const cl_map_iterator_t itor ) +static inline void *cl_map_obj(IN const cl_map_iterator_t itor) { - return( cl_qmap_obj( PARENT_STRUCT( itor, cl_map_obj_t, item ) ) ); + return (cl_qmap_obj(PARENT_STRUCT(itor, cl_map_obj_t, item))); } + /* * PARAMETERS * itor @@ -805,9 +769,8 @@ cl_map_obj( * SYNOPSIS */ cl_status_t -cl_map_merge( - OUT cl_map_t* const p_dest_map, - IN OUT cl_map_t* const p_src_map ); +cl_map_merge(OUT cl_map_t * const p_dest_map, + IN OUT cl_map_t * const p_src_map); /* * PARAMETERS * p_dest_map @@ -843,11 +806,9 @@ cl_map_merge( * SYNOPSIS */ cl_status_t -cl_map_delta( - IN OUT cl_map_t* const p_map1, - IN OUT cl_map_t* const p_map2, - OUT cl_map_t* const p_new, - OUT cl_map_t* const p_old ); +cl_map_delta(IN OUT cl_map_t * const p_map1, + IN OUT cl_map_t * const p_map2, + OUT cl_map_t * const p_new, OUT cl_map_t * const p_old); /* * PARAMETERS * p_map1 @@ -888,5 +849,4 @@ cl_map_delta( *********/ END_C_DECLS - -#endif /* _CL_MAP_H_ */ +#endif /* _CL_MAP_H_ */ diff --git a/opensm/include/complib/cl_math.h b/opensm/include/complib/cl_math.h index 123bd22..9201dfa 100644 --- a/opensm/include/complib/cl_math.h +++ b/opensm/include/complib/cl_math.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****d* Component Library: Math/MAX * NAME * MAX @@ -84,7 +83,6 @@ BEGIN_C_DECLS #ifndef MAX #define MAX(x,y) ((x) > (y) ? (x) : (y)) #endif - /****d* Component Library: Math/MIN * NAME * MIN @@ -111,7 +109,6 @@ BEGIN_C_DECLS #ifndef MIN #define MIN(x,y) ((x) < (y) ? (x) : (y)) #endif - /****d* Component Library: Math/ROUNDUP * NAME * ROUNDUP @@ -142,7 +139,5 @@ BEGIN_C_DECLS #define ROUNDUP(val, align) \ ((((val) / (align))*(align)) + (((val) % (align)) ? (align) : 0)) #endif - -END_C_DECLS - -#endif /* _CL_MATH_H_ */ + END_C_DECLS +#endif /* _CL_MATH_H_ */ diff --git a/opensm/include/complib/cl_passivelock.h b/opensm/include/complib/cl_passivelock.h index b1ac973..982fd95 100644 --- a/opensm/include/complib/cl_passivelock.h +++ b/opensm/include/complib/cl_passivelock.h @@ -54,13 +54,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Passive Lock * NAME * Passive Lock @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Manipulation * cl_plock_acquire, cl_plock_excl_acquire, cl_plock_release *********/ - /****s* Component Library: Passive Lock/cl_plock_t * NAME * cl_plock_t @@ -99,10 +97,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_plock -{ - pthread_rwlock_t lock; - cl_state_t state; + typedef struct _cl_plock { + pthread_rwlock_t lock; + cl_state_t state; } cl_plock_t; /* * FIELDS @@ -127,14 +124,13 @@ typedef struct _cl_plock * * SYNOPSIS */ -static inline void -cl_plock_construct( - IN cl_plock_t* const p_lock ) +static inline void cl_plock_construct(IN cl_plock_t * const p_lock) { - CL_ASSERT( p_lock ); + CL_ASSERT(p_lock); - p_lock->state = CL_UNINITIALIZED; + p_lock->state = CL_UNINITIALIZED; } + /* * PARAMETERS * p_lock @@ -163,15 +159,14 @@ cl_plock_construct( * * SYNOPSIS */ -static inline void -cl_plock_destroy( - IN cl_plock_t* const p_lock ) +static inline void cl_plock_destroy(IN cl_plock_t * const p_lock) { - CL_ASSERT( p_lock ); - p_lock->state = CL_DESTROYING; - pthread_rwlock_destroy( &p_lock->lock ); - p_lock->state = CL_DESTROYED; + CL_ASSERT(p_lock); + p_lock->state = CL_DESTROYING; + pthread_rwlock_destroy(&p_lock->lock); + p_lock->state = CL_DESTROYED; } + /* * PARAMETERS * p_lock @@ -201,18 +196,18 @@ cl_plock_destroy( * * SYNOPSIS */ -static inline cl_status_t -cl_plock_init( - IN cl_plock_t* const p_lock ) +static inline cl_status_t cl_plock_init(IN cl_plock_t * const p_lock) { - cl_status_t status; + cl_status_t status; - CL_ASSERT( p_lock ); - status = (cl_status_t)pthread_rwlock_init( &p_lock->lock, NULL ); - if (status) return CL_ERROR; - p_lock->state = CL_INITIALIZED; - return( CL_SUCCESS ); + CL_ASSERT(p_lock); + status = (cl_status_t) pthread_rwlock_init(&p_lock->lock, NULL); + if (status) + return CL_ERROR; + p_lock->state = CL_INITIALIZED; + return (CL_SUCCESS); } + /* * PARAMETERS * p_lock @@ -242,17 +237,16 @@ cl_plock_init( * * SYNOPSIS */ -static inline void -cl_plock_acquire( - IN cl_plock_t* const p_lock ) +static inline void cl_plock_acquire(IN cl_plock_t * const p_lock) { - cl_status_t status; - CL_ASSERT( p_lock ); - CL_ASSERT( p_lock->state == CL_INITIALIZED ); + cl_status_t status; + CL_ASSERT(p_lock); + CL_ASSERT(p_lock->state == CL_INITIALIZED); - status = (cl_status_t)pthread_rwlock_rdlock(&p_lock->lock); - CL_ASSERT( status == 0 ); + status = (cl_status_t) pthread_rwlock_rdlock(&p_lock->lock); + CL_ASSERT(status == 0); } + /* * PARAMETERS * p_lock @@ -275,18 +269,17 @@ cl_plock_acquire( * * SYNOPSIS */ -static inline void -cl_plock_excl_acquire( - IN cl_plock_t* const p_lock ) +static inline void cl_plock_excl_acquire(IN cl_plock_t * const p_lock) { - cl_status_t status; + cl_status_t status; - CL_ASSERT( p_lock ); - CL_ASSERT( p_lock->state == CL_INITIALIZED ); + CL_ASSERT(p_lock); + CL_ASSERT(p_lock->state == CL_INITIALIZED); - status = (cl_status_t)pthread_rwlock_wrlock(&p_lock->lock); - CL_ASSERT( status == 0 ); + status = (cl_status_t) pthread_rwlock_wrlock(&p_lock->lock); + CL_ASSERT(status == 0); } + /* * PARAMETERS * p_lock @@ -309,17 +302,16 @@ cl_plock_excl_acquire( * * SYNOPSIS */ -static inline void -cl_plock_release( - IN cl_plock_t* const p_lock ) +static inline void cl_plock_release(IN cl_plock_t * const p_lock) { - cl_status_t status; - CL_ASSERT( p_lock ); - CL_ASSERT( p_lock->state == CL_INITIALIZED ); + cl_status_t status; + CL_ASSERT(p_lock); + CL_ASSERT(p_lock->state == CL_INITIALIZED); - status = (cl_status_t)pthread_rwlock_unlock(&p_lock->lock); - CL_ASSERT( status == 0 ); + status = (cl_status_t) pthread_rwlock_unlock(&p_lock->lock); + CL_ASSERT(status == 0); } + /* * PARAMETERS * p_lock @@ -333,5 +325,4 @@ cl_plock_release( *********/ END_C_DECLS - -#endif /* _CL_PASSIVE_LOCK_H_ */ +#endif /* _CL_PASSIVE_LOCK_H_ */ diff --git a/opensm/include/complib/cl_pool.h b/opensm/include/complib/cl_pool.h index 2e196fb..435db48 100644 --- a/opensm/include/complib/cl_pool.h +++ b/opensm/include/complib/cl_pool.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Pool * NAME * Pool @@ -96,7 +95,6 @@ BEGIN_C_DECLS * Attributes: * cl_is_pool_inited, cl_pool_count *********/ - /****d* Component Library: Pool/cl_pfn_pool_init_t * NAME * cl_pfn_pool_init_t @@ -109,9 +107,7 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_pool_init_t)( - IN void* const p_object, - IN void* context ); + (*cl_pfn_pool_init_t) (IN void *const p_object, IN void *context); /* * PARAMETERS * p_object @@ -155,9 +151,7 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_pool_dtor_t)( - IN void* const p_object, - IN void* context ); + (*cl_pfn_pool_dtor_t) (IN void *const p_object, IN void *context); /* * PARAMETERS * p_object @@ -195,12 +189,11 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_pool -{ - cl_qcpool_t qcpool; - cl_pfn_pool_init_t pfn_init; - cl_pfn_pool_dtor_t pfn_dtor; - const void *context; +typedef struct _cl_pool { + cl_qcpool_t qcpool; + cl_pfn_pool_init_t pfn_init; + cl_pfn_pool_dtor_t pfn_dtor; + const void *context; } cl_pool_t; /* @@ -235,9 +228,7 @@ typedef struct _cl_pool * * SYNOPSIS */ -void -cl_pool_construct( - IN cl_pool_t* const p_pool ); +void cl_pool_construct(IN cl_pool_t * const p_pool); /* * PARAMETERS * p_pool @@ -266,14 +257,13 @@ cl_pool_construct( * * SYNOPSIS */ -static inline uint32_t -cl_is_pool_inited( - IN const cl_pool_t* const p_pool ) +static inline uint32_t cl_is_pool_inited(IN const cl_pool_t * const p_pool) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_pool ); - return( cl_is_qcpool_inited( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_is_qcpool_inited(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -303,15 +293,14 @@ cl_is_pool_inited( * SYNOPSIS */ cl_status_t -cl_pool_init( - IN cl_pool_t* const p_pool, - IN const size_t min_count, - IN const size_t max_count, - IN const size_t grow_size, - IN const size_t object_size, - IN cl_pfn_pool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_pool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ); +cl_pool_init(IN cl_pool_t * const p_pool, + IN const size_t min_count, + IN const size_t max_count, + IN const size_t grow_size, + IN const size_t object_size, + IN cl_pfn_pool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_pool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_pool @@ -380,13 +369,12 @@ cl_pool_init( * * SYNOPSIS */ -static inline void -cl_pool_destroy( - IN cl_pool_t* const p_pool ) +static inline void cl_pool_destroy(IN cl_pool_t * const p_pool) { - CL_ASSERT( p_pool ); - cl_qcpool_destroy( &p_pool->qcpool ); + CL_ASSERT(p_pool); + cl_qcpool_destroy(&p_pool->qcpool); } + /* * PARAMETERS * p_pool @@ -421,13 +409,12 @@ cl_pool_destroy( * * SYNOPSIS */ -static inline size_t -cl_pool_count( - IN cl_pool_t* const p_pool ) +static inline size_t cl_pool_count(IN cl_pool_t * const p_pool) { - CL_ASSERT( p_pool ); - return( cl_qcpool_count( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_count(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -450,21 +437,20 @@ cl_pool_count( * * SYNOPSIS */ -static inline void* -cl_pool_get( - IN cl_pool_t* const p_pool ) +static inline void *cl_pool_get(IN cl_pool_t * const p_pool) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - p_pool_obj = (cl_pool_obj_t*)cl_qcpool_get( &p_pool->qcpool ); - if( !p_pool_obj ) - return( NULL ); + p_pool_obj = (cl_pool_obj_t *) cl_qcpool_get(&p_pool->qcpool); + if (!p_pool_obj) + return (NULL); - CL_ASSERT( p_pool_obj->p_object ); - return( (void*)p_pool_obj->p_object ); + CL_ASSERT(p_pool_obj->p_object); + return ((void *)p_pool_obj->p_object); } + /* * PARAMETERS * p_pool @@ -495,24 +481,23 @@ cl_pool_get( * SYNOPSIS */ static inline void -cl_pool_put( - IN cl_pool_t* const p_pool, - IN void* const p_object ) +cl_pool_put(IN cl_pool_t * const p_pool, IN void *const p_object) { - cl_pool_obj_t *p_pool_obj; + cl_pool_obj_t *p_pool_obj; - CL_ASSERT( p_pool ); - CL_ASSERT( p_object ); + CL_ASSERT(p_pool); + CL_ASSERT(p_object); /* Calculate the offset to the list object representing this object. */ - p_pool_obj = (cl_pool_obj_t*) - (((uint8_t*)p_object) - sizeof(cl_pool_obj_t)); + p_pool_obj = (cl_pool_obj_t *) + (((uint8_t *) p_object) - sizeof(cl_pool_obj_t)); /* good sanity check */ - CL_ASSERT( p_pool_obj->p_object == p_object ); + CL_ASSERT(p_pool_obj->p_object == p_object); - cl_qcpool_put( &p_pool->qcpool, &p_pool_obj->pool_item ); + cl_qcpool_put(&p_pool->qcpool, &p_pool_obj->pool_item); } + /* * PARAMETERS * p_pool @@ -546,13 +531,12 @@ cl_pool_put( * SYNOPSIS */ static inline cl_status_t -cl_pool_grow( - IN cl_pool_t* const p_pool, - IN const size_t obj_count ) +cl_pool_grow(IN cl_pool_t * const p_pool, IN const size_t obj_count) { - CL_ASSERT( p_pool ); - return( cl_qcpool_grow( &p_pool->qcpool, obj_count ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_grow(&p_pool->qcpool, obj_count)); } + /* * PARAMETERS * p_pool @@ -580,5 +564,4 @@ cl_pool_grow( *********/ END_C_DECLS - -#endif /* _CL_POOL_H_ */ +#endif /* _CL_POOL_H_ */ diff --git a/opensm/include/complib/cl_ptr_vector.h b/opensm/include/complib/cl_ptr_vector.h index e64dc30..cacf99b 100644 --- a/opensm/include/complib/cl_ptr_vector.h +++ b/opensm/include/complib/cl_ptr_vector.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Pointer Vector * NAME * Pointer Vector @@ -94,7 +93,6 @@ BEGIN_C_DECLS * cl_ptr_vector_find_from_start, cl_ptr_vector_find_from_end * cl_ptr_vector_apply_func *********/ - /****d* Component Library: Pointer Vector/cl_pfn_ptr_vec_apply_t * NAME * cl_pfn_ptr_vec_apply_t @@ -105,11 +103,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef void -(*cl_pfn_ptr_vec_apply_t)( - IN const size_t index, - IN void* const element, - IN void* context ); + typedef void + (*cl_pfn_ptr_vec_apply_t) (IN const size_t index, + IN void *const element, IN void *context); /* * PARAMETERS * index @@ -144,10 +140,8 @@ typedef void * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_ptr_vec_find_t)( - IN const size_t index, - IN const void* const element, - IN void* context ); + (*cl_pfn_ptr_vec_find_t) (IN const size_t index, + IN const void *const element, IN void *context); /* * PARAMETERS * index @@ -188,13 +182,12 @@ typedef cl_status_t * * SYNOPSIS */ -typedef struct _cl_ptr_vector -{ - size_t size; - size_t grow_size; - size_t capacity; - const void **p_ptr_array; - cl_state_t state; +typedef struct _cl_ptr_vector { + size_t size; + size_t grow_size; + size_t capacity; + const void **p_ptr_array; + cl_state_t state; } cl_ptr_vector_t; /* @@ -230,9 +223,7 @@ typedef struct _cl_ptr_vector * * SYNOPSIS */ -void -cl_ptr_vector_construct( - IN cl_ptr_vector_t* const p_vector ); +void cl_ptr_vector_construct(IN cl_ptr_vector_t * const p_vector); /* * PARAMETERS * p_vector @@ -262,10 +253,8 @@ cl_ptr_vector_construct( * SYNOPSIS */ cl_status_t -cl_ptr_vector_init( - IN cl_ptr_vector_t* const p_vector, - IN const size_t min_size, - IN const size_t grow_size ); +cl_ptr_vector_init(IN cl_ptr_vector_t * const p_vector, + IN const size_t min_size, IN const size_t grow_size); /* * PARAMETERS * p_vector @@ -297,9 +286,7 @@ cl_ptr_vector_init( * * SYNOPSIS */ -void -cl_ptr_vector_destroy( - IN cl_ptr_vector_t* const p_vector ); +void cl_ptr_vector_destroy(IN cl_ptr_vector_t * const p_vector); /* * PARAMETERS * p_vector @@ -329,14 +316,14 @@ cl_ptr_vector_destroy( * SYNOPSIS */ static inline size_t -cl_ptr_vector_get_capacity( - IN const cl_ptr_vector_t* const p_vector ) +cl_ptr_vector_get_capacity(IN const cl_ptr_vector_t * const p_vector) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - return( p_vector->capacity ); + return (p_vector->capacity); } + /* * PARAMETERS * p_vector @@ -364,14 +351,14 @@ cl_ptr_vector_get_capacity( * SYNOPSIS */ static inline uint32_t -cl_ptr_vector_get_size( - IN const cl_ptr_vector_t* const p_vector ) +cl_ptr_vector_get_size(IN const cl_ptr_vector_t * const p_vector) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - return( (uint32_t)p_vector->size ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + return ((uint32_t) p_vector->size); } + /* * PARAMETERS * p_vector @@ -394,17 +381,16 @@ cl_ptr_vector_get_size( * * SYNOPSIS */ -static inline void* -cl_ptr_vector_get( - IN const cl_ptr_vector_t* const p_vector, - IN const size_t index ) +static inline void *cl_ptr_vector_get(IN const cl_ptr_vector_t * const p_vector, + IN const size_t index) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( p_vector->size > index ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(p_vector->size > index); - return( (void*)p_vector->p_ptr_array[index] ); + return ((void *)p_vector->p_ptr_array[index]); } + /* * PARAMETERS * p_vector @@ -439,10 +425,8 @@ cl_ptr_vector_get( * SYNOPSIS */ cl_status_t -cl_ptr_vector_at( - IN const cl_ptr_vector_t* const p_vector, - IN const size_t index, - OUT void** const p_element ); +cl_ptr_vector_at(IN const cl_ptr_vector_t * const p_vector, + IN const size_t index, OUT void **const p_element); /* * PARAMETERS * p_vector @@ -482,10 +466,8 @@ cl_ptr_vector_at( * SYNOPSIS */ cl_status_t -cl_ptr_vector_set( - IN cl_ptr_vector_t* const p_vector, - IN const size_t index, - IN const void* const element ); +cl_ptr_vector_set(IN cl_ptr_vector_t * const p_vector, + IN const size_t index, IN const void *const element); /* * PARAMETERS * p_vector @@ -523,22 +505,22 @@ cl_ptr_vector_set( * SYNOPSIS */ static inline cl_status_t -cl_ptr_vector_insert( - IN cl_ptr_vector_t* const p_vector, - IN const void* const element, - OUT size_t* const p_index OPTIONAL ) +cl_ptr_vector_insert(IN cl_ptr_vector_t * const p_vector, + IN const void *const element, + OUT size_t * const p_index OPTIONAL) { - cl_status_t status; + cl_status_t status; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - status = cl_ptr_vector_set( p_vector, p_vector->size, element ); - if( status == CL_SUCCESS && p_index ) + status = cl_ptr_vector_set(p_vector, p_vector->size, element); + if (status == CL_SUCCESS && p_index) *p_index = p_vector->size - 1; - return( status ); + return (status); } + /* * PARAMETERS * p_vector @@ -581,10 +563,8 @@ cl_ptr_vector_insert( * * SYNOPSIS */ -void* -cl_ptr_vector_remove( - IN cl_ptr_vector_t* const p_vector, - IN const size_t index ); +void *cl_ptr_vector_remove(IN cl_ptr_vector_t * const p_vector, + IN const size_t index); /* * PARAMETERS * p_vector @@ -617,9 +597,8 @@ cl_ptr_vector_remove( * SYNOPSIS */ cl_status_t -cl_ptr_vector_set_capacity( - IN cl_ptr_vector_t* const p_vector, - IN const size_t new_capacity ); +cl_ptr_vector_set_capacity(IN cl_ptr_vector_t * const p_vector, + IN const size_t new_capacity); /* * PARAMETERS * p_vector @@ -657,9 +636,8 @@ cl_ptr_vector_set_capacity( * SYNOPSIS */ cl_status_t -cl_ptr_vector_set_size( - IN cl_ptr_vector_t* const p_vector, - IN const size_t size ); +cl_ptr_vector_set_size(IN cl_ptr_vector_t * const p_vector, + IN const size_t size); /* * PARAMETERS * p_vector @@ -697,9 +675,8 @@ cl_ptr_vector_set_size( * SYNOPSIS */ cl_status_t -cl_ptr_vector_set_min_size( - IN cl_ptr_vector_t* const p_vector, - IN const size_t min_size ); +cl_ptr_vector_set_min_size(IN cl_ptr_vector_t * const p_vector, + IN const size_t min_size); /* * PARAMETERS * p_vector @@ -738,10 +715,9 @@ cl_ptr_vector_set_min_size( * SYNOPSIS */ void -cl_ptr_vector_apply_func( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_apply_t pfn_callback, - IN const void* const context ); +cl_ptr_vector_apply_func(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_apply_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -778,10 +754,9 @@ cl_ptr_vector_apply_func( * SYNOPSIS */ size_t -cl_ptr_vector_find_from_start( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_find_t pfn_callback, - IN const void* const context ); +cl_ptr_vector_find_from_start(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_find_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -821,10 +796,9 @@ cl_ptr_vector_find_from_start( * SYNOPSIS */ size_t -cl_ptr_vector_find_from_end( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_find_t pfn_callback, - IN const void* const context ); +cl_ptr_vector_find_from_end(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_find_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -854,5 +828,4 @@ cl_ptr_vector_find_from_end( *********/ END_C_DECLS - -#endif /* _CL_PTR_VECTOR_H_ */ +#endif /* _CL_PTR_VECTOR_H_ */ diff --git a/opensm/include/complib/cl_qcomppool.h b/opensm/include/complib/cl_qcomppool.h index 818523f..91f5867 100644 --- a/opensm/include/complib/cl_qcomppool.h +++ b/opensm/include/complib/cl_qcomppool.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Quick Composite Pool * NAME * Quick Composite Pool @@ -109,7 +108,6 @@ BEGIN_C_DECLS * Attributes: * cl_is_qcpool_inited, cl_qcpool_count *********/ - /****s* Component Library: Quick Composite Pool/cl_pool_item_t * NAME * cl_pool_item_t @@ -119,12 +117,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_pool_item -{ - cl_list_item_t list_item; + typedef struct _cl_pool_item { + cl_list_item_t list_item; #ifdef _DEBUG_ /* Pointer to the owner pool used for sanity checks. */ - struct _cl_qcpool *p_pool; + struct _cl_qcpool *p_pool; #endif } cl_pool_item_t; @@ -154,11 +151,10 @@ typedef struct _cl_pool_item * * SYNOPSIS */ -typedef struct _cl_pool_obj -{ +typedef struct _cl_pool_obj { /* The pool item must be the first item to allow casting. */ - cl_pool_item_t pool_item; - const void *p_object; + cl_pool_item_t pool_item; + const void *p_object; } cl_pool_obj_t; /* @@ -188,11 +184,10 @@ typedef struct _cl_pool_obj * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_qcpool_init_t)( - IN void** const p_comp_array, - IN const uint32_t num_components, - IN void* context, - OUT cl_pool_item_t** const pp_pool_item ); + (*cl_pfn_qcpool_init_t) (IN void **const p_comp_array, + IN const uint32_t num_components, + IN void *context, + OUT cl_pool_item_t ** const pp_pool_item); /* * PARAMETERS * p_comp_array @@ -256,9 +251,8 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_qcpool_dtor_t)( - IN const cl_pool_item_t* const p_pool_item, - IN void* context ); + (*cl_pfn_qcpool_dtor_t) (IN const cl_pool_item_t * const p_pool_item, + IN void *context); /* * PARAMETERS * p_pool_item @@ -296,20 +290,19 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_qcpool -{ - uint32_t num_components; - size_t *component_sizes; - void **p_components; - size_t num_objects; - size_t max_objects; - size_t grow_size; - cl_pfn_qcpool_init_t pfn_init; - cl_pfn_qcpool_dtor_t pfn_dtor; - const void *context; - cl_qlist_t free_list; - cl_qlist_t alloc_list; - cl_state_t state; +typedef struct _cl_qcpool { + uint32_t num_components; + size_t *component_sizes; + void **p_components; + size_t num_objects; + size_t max_objects; + size_t grow_size; + cl_pfn_qcpool_init_t pfn_init; + cl_pfn_qcpool_dtor_t pfn_dtor; + const void *context; + cl_qlist_t free_list; + cl_qlist_t alloc_list; + cl_state_t state; } cl_qcpool_t; /* @@ -363,9 +356,7 @@ typedef struct _cl_qcpool * * SYNOPSIS */ -void -cl_qcpool_construct( - IN cl_qcpool_t* const p_pool ); +void cl_qcpool_construct(IN cl_qcpool_t * const p_pool); /* * PARAMETERS * p_pool @@ -395,17 +386,16 @@ cl_qcpool_construct( * * SYNOPSIS */ -static inline uint32_t -cl_is_qcpool_inited( - IN const cl_qcpool_t* const p_pool ) +static inline uint32_t cl_is_qcpool_inited(IN const cl_qcpool_t * const p_pool) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); /* CL_ASSERT that the pool is not in some invalid state. */ - CL_ASSERT( cl_is_state_valid( p_pool->state ) ); + CL_ASSERT(cl_is_state_valid(p_pool->state)); - return( p_pool->state == CL_INITIALIZED ); + return (p_pool->state == CL_INITIALIZED); } + /* * PARAMETERS * p_pool @@ -434,16 +424,15 @@ cl_is_qcpool_inited( * SYNOPSIS */ cl_status_t -cl_qcpool_init( - IN cl_qcpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t* const component_sizes, - IN const uint32_t num_components, - IN cl_pfn_qcpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_qcpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ); +cl_qcpool_init(IN cl_qcpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t * const component_sizes, + IN const uint32_t num_components, + IN cl_pfn_qcpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_qcpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_pool @@ -522,9 +511,7 @@ cl_qcpool_init( * * SYNOPSIS */ -void -cl_qcpool_destroy( - IN cl_qcpool_t* const p_pool ); +void cl_qcpool_destroy(IN cl_qcpool_t * const p_pool); /* * PARAMETERS * p_pool @@ -559,15 +546,14 @@ cl_qcpool_destroy( * * SYNOPSIS */ -static inline size_t -cl_qcpool_count( - IN cl_qcpool_t* const p_pool ) +static inline size_t cl_qcpool_count(IN cl_qcpool_t * const p_pool) { - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); - return( cl_qlist_count( &p_pool->free_list ) ); + return (cl_qlist_count(&p_pool->free_list)); } + /* * PARAMETERS * p_pool @@ -592,9 +578,7 @@ cl_qcpool_count( * * SYNOPSIS */ -cl_pool_item_t* -cl_qcpool_get( - IN cl_qcpool_t* const p_pool ); +cl_pool_item_t *cl_qcpool_get(IN cl_qcpool_t * const p_pool); /* * PARAMETERS * p_pool @@ -626,19 +610,19 @@ cl_qcpool_get( * SYNOPSIS */ static inline void -cl_qcpool_put( - IN cl_qcpool_t* const p_pool, - IN cl_pool_item_t* const p_pool_item ) +cl_qcpool_put(IN cl_qcpool_t * const p_pool, + IN cl_pool_item_t * const p_pool_item) { - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); - CL_ASSERT( p_pool_item ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); + CL_ASSERT(p_pool_item); /* Make sure items being returned came from the specified pool. */ - CL_ASSERT( p_pool_item->p_pool == p_pool ); + CL_ASSERT(p_pool_item->p_pool == p_pool); /* return this lil' doggy to the pool */ - cl_qlist_insert_head( &p_pool->free_list, &p_pool_item->list_item ); + cl_qlist_insert_head(&p_pool->free_list, &p_pool_item->list_item); } + /* * PARAMETERS * p_pool @@ -673,31 +657,29 @@ cl_qcpool_put( * SYNOPSIS */ static inline void -cl_qcpool_put_list( - IN cl_qcpool_t* const p_pool, - IN cl_qlist_t* const p_list ) +cl_qcpool_put_list(IN cl_qcpool_t * const p_pool, IN cl_qlist_t * const p_list) { #ifdef _DEBUG_ - cl_list_item_t *p_item; + cl_list_item_t *p_item; #endif - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); - CL_ASSERT( p_list ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); + CL_ASSERT(p_list); #ifdef _DEBUG_ /* Chech that all items in the list came from this pool. */ - p_item = cl_qlist_head( p_list ); - while( p_item != cl_qlist_end( p_list ) ) - { - CL_ASSERT( ((cl_pool_item_t*)p_item)->p_pool == p_pool ); - p_item = cl_qlist_next( p_item ); + p_item = cl_qlist_head(p_list); + while (p_item != cl_qlist_end(p_list)) { + CL_ASSERT(((cl_pool_item_t *) p_item)->p_pool == p_pool); + p_item = cl_qlist_next(p_item); } #endif /* return these lil' doggies to the pool */ - cl_qlist_insert_list_head( &p_pool->free_list, p_list ); + cl_qlist_insert_list_head(&p_pool->free_list, p_list); } + /* * PARAMETERS * p_pool @@ -731,10 +713,7 @@ cl_qcpool_put_list( * * SYNOPSIS */ -cl_status_t -cl_qcpool_grow( - IN cl_qcpool_t* const p_pool, - IN size_t obj_count ); +cl_status_t cl_qcpool_grow(IN cl_qcpool_t * const p_pool, IN size_t obj_count); /* * PARAMETERS * p_pool @@ -762,5 +741,4 @@ cl_qcpool_grow( *********/ END_C_DECLS - -#endif /* _CL_QUICK_COMPOSITE_POOL_H_ */ +#endif /* _CL_QUICK_COMPOSITE_POOL_H_ */ diff --git a/opensm/include/complib/cl_qlist.h b/opensm/include/complib/cl_qlist.h index 273aff7..63f1b3a 100644 --- a/opensm/include/complib/cl_qlist.h +++ b/opensm/include/complib/cl_qlist.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Quick List * NAME * Quick List @@ -110,7 +109,6 @@ BEGIN_C_DECLS * Attributes: * cl_qlist_count, cl_is_qlist_empty *********/ - /****s* Component Library: Quick List/cl_list_item_t * NAME * cl_list_item_t @@ -120,12 +118,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_list_item -{ - struct _cl_list_item *p_next; - struct _cl_list_item *p_prev; + typedef struct _cl_list_item { + struct _cl_list_item *p_next; + struct _cl_list_item *p_prev; #ifdef _DEBUG_ - struct _cl_qlist *p_list; + struct _cl_qlist *p_list; #endif } cl_list_item_t; @@ -150,10 +147,9 @@ typedef struct _cl_list_item * * SYNOPSIS */ -typedef struct _cl_list_obj -{ - cl_list_item_t list_item; - const void *p_object; /* User's context */ +typedef struct _cl_list_obj { + cl_list_item_t list_item; + const void *p_object; /* User's context */ } cl_list_obj_t; /* @@ -186,11 +182,10 @@ typedef struct _cl_list_obj * * SYNOPSIS */ -typedef struct _cl_qlist -{ - cl_list_item_t end; - size_t count; - cl_state_t state; +typedef struct _cl_qlist { + cl_list_item_t end; + size_t count; + cl_state_t state; } cl_qlist_t; /* @@ -219,9 +214,8 @@ typedef struct _cl_qlist * SYNOPSIS */ typedef void -(*cl_pfn_qlist_apply_t)( - IN cl_list_item_t* const p_list_item, - IN void* context ); + (*cl_pfn_qlist_apply_t) (IN cl_list_item_t * const p_list_item, + IN void *context); /* * PARAMETERS * p_list_item @@ -253,9 +247,8 @@ typedef void * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_qlist_find_t)( - IN const cl_list_item_t* const p_list_item, - IN void* context ); + (*cl_pfn_qlist_find_t) (IN const cl_list_item_t * const p_list_item, + IN void *context); /* * PARAMETERS * p_list_item @@ -291,20 +284,20 @@ typedef cl_status_t * SYNOPSIS */ static inline void -__cl_primitive_insert( - IN cl_list_item_t* const p_list_item, - IN cl_list_item_t* const p_new_item ) +__cl_primitive_insert(IN cl_list_item_t * const p_list_item, + IN cl_list_item_t * const p_new_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_new_item ); + CL_ASSERT(p_new_item); p_new_item->p_next = p_list_item; p_new_item->p_prev = p_list_item->p_prev; p_list_item->p_prev = p_new_item; p_new_item->p_prev->p_next = p_new_item; } + /* * PARAMETERS * p_list_item @@ -327,27 +320,25 @@ __cl_primitive_insert( * * SYNOPSIS */ -static inline void -__cl_primitive_remove( - IN cl_list_item_t* const p_list_item ) +static inline void __cl_primitive_remove(IN cl_list_item_t * const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* set the back pointer */ - p_list_item->p_next->p_prev= p_list_item->p_prev; + p_list_item->p_next->p_prev = p_list_item->p_prev; /* set the next pointer */ - p_list_item->p_prev->p_next= p_list_item->p_next; + p_list_item->p_prev->p_next = p_list_item->p_next; /* if we're debugging, spruce up the pointers to help find bugs */ #if defined( _DEBUG_ ) - if( p_list_item != p_list_item->p_next ) - { + if (p_list_item != p_list_item->p_next) { p_list_item->p_next = NULL; p_list_item->p_prev = NULL; } -#endif /* defined( _DEBUG_ ) */ +#endif /* defined( _DEBUG_ ) */ } + /* * PARAMETERS * p_list_item @@ -371,14 +362,14 @@ __cl_primitive_remove( * SYNOPSIS */ static inline void -cl_qlist_set_obj( - IN cl_list_obj_t* const p_list_obj, - IN const void* const p_object ) +cl_qlist_set_obj(IN cl_list_obj_t * const p_list_obj, + IN const void *const p_object) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_obj ); + CL_ASSERT(p_list_obj); p_list_obj->p_object = p_object; } + /* * PARAMETERS * p_list_obj @@ -403,15 +394,14 @@ cl_qlist_set_obj( * * SYNOPSIS */ -static inline void* -cl_qlist_obj( - IN const cl_list_obj_t* const p_list_obj ) +static inline void *cl_qlist_obj(IN const cl_list_obj_t * const p_list_obj) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_obj ); + CL_ASSERT(p_list_obj); - return( (void*)p_list_obj->p_object ); + return ((void *)p_list_obj->p_object); } + /* * PARAMETERS * p_list_obj @@ -424,9 +414,7 @@ cl_qlist_obj( * Quick List, cl_qlist_set_obj *********/ -static inline void -__cl_qlist_reset( - IN cl_qlist_t* const p_list ) +static inline void __cl_qlist_reset(IN cl_qlist_t * const p_list) { /* Point the end item to itself. */ p_list->end.p_next = &p_list->end; @@ -448,18 +436,17 @@ __cl_qlist_reset( * * SYNOPSIS */ -static inline void -cl_qlist_init( - IN cl_qlist_t* const p_list ) +static inline void cl_qlist_init(IN cl_qlist_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); p_list->state = CL_INITIALIZED; /* Reset the quick list data structure. */ - __cl_qlist_reset( p_list ); + __cl_qlist_reset(p_list); } + /* * PARAMETERS * p_list @@ -486,17 +473,16 @@ cl_qlist_init( * * SYNOPSIS */ -static inline uint32_t -cl_qlist_count( - IN const cl_qlist_t* const p_list ) +static inline uint32_t cl_qlist_count(IN const cl_qlist_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); - return( (uint32_t)p_list->count ); + CL_ASSERT(p_list->state == CL_INITIALIZED); + return ((uint32_t) p_list->count); } + /* * PARAMETERS * p_list @@ -519,17 +505,16 @@ cl_qlist_count( * * SYNOPSIS */ -static inline boolean_t -cl_is_qlist_empty( - IN const cl_qlist_t* const p_list ) +static inline boolean_t cl_is_qlist_empty(IN const cl_qlist_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - return( !cl_qlist_count( p_list ) ); + return (!cl_qlist_count(p_list)); } + /* * PARAMETERS * p_list @@ -554,18 +539,18 @@ cl_is_qlist_empty( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_next( - IN const cl_list_item_t* const p_list_item ) +static inline cl_list_item_t *cl_qlist_next(IN const cl_list_item_t * + const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list_item->p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list_item->p_list->state == CL_INITIALIZED); /* Return the next item. */ - return( p_list_item->p_next ); + return (p_list_item->p_next); } + /* * PARAMETERS * p_list_item @@ -592,18 +577,18 @@ cl_qlist_next( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_prev( - IN const cl_list_item_t* const p_list_item ) +static inline cl_list_item_t *cl_qlist_prev(IN const cl_list_item_t * + const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list_item->p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list_item->p_list->state == CL_INITIALIZED); /* Return the previous item. */ - return( p_list_item->p_prev ); + return (p_list_item->p_prev); } + /* * PARAMETERS * p_list_item @@ -630,17 +615,16 @@ cl_qlist_prev( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_head( - IN const cl_qlist_t* const p_list ) +static inline cl_list_item_t *cl_qlist_head(IN const cl_qlist_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - return( cl_qlist_next( &p_list->end ) ); + return (cl_qlist_next(&p_list->end)); } + /* * PARAMETERS * p_list @@ -669,17 +653,16 @@ cl_qlist_head( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_tail( - IN const cl_qlist_t* const p_list ) +static inline cl_list_item_t *cl_qlist_tail(IN const cl_qlist_t * const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - return( cl_qlist_prev( &p_list->end ) ); + return (cl_qlist_prev(&p_list->end)); } + /* * PARAMETERS * p_list @@ -707,17 +690,17 @@ cl_qlist_tail( * * SYNOPSIS */ -static inline const cl_list_item_t* -cl_qlist_end( - IN const cl_qlist_t* const p_list ) +static inline const cl_list_item_t *cl_qlist_end(IN const cl_qlist_t * + const p_list) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - return( &p_list->end ); + return (&p_list->end); } + /* * PARAMETERS * p_list @@ -750,16 +733,15 @@ cl_qlist_end( * SYNOPSIS */ static inline void -cl_qlist_insert_head( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_list_item ) +cl_qlist_insert_head(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* * The list item must not already be part of the list. Note that this @@ -767,17 +749,18 @@ cl_qlist_insert_head( * list pointer equal to the specified list. The chances of this * happening are acceptable in light of the value of this check. */ - CL_ASSERT( p_list_item->p_list != p_list ); + CL_ASSERT(p_list_item->p_list != p_list); #if defined( _DEBUG_ ) p_list_item->p_list = p_list; #endif /* Insert before the head. */ - __cl_primitive_insert( cl_qlist_head( p_list ), p_list_item ); + __cl_primitive_insert(cl_qlist_head(p_list), p_list_item); p_list->count++; } + /* * PARAMETERS * p_list @@ -811,16 +794,15 @@ cl_qlist_insert_head( * SYNOPSIS */ static inline void -cl_qlist_insert_tail( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_list_item ) +cl_qlist_insert_tail(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* * The list item must not already be part of the list. Note that this @@ -828,7 +810,7 @@ cl_qlist_insert_tail( * list pointer equal to the specified list. The chances of this * happening are acceptable in light of the value of this check. */ - CL_ASSERT( p_list_item->p_list != p_list ); + CL_ASSERT(p_list_item->p_list != p_list); #if defined( _DEBUG_ ) p_list_item->p_list = p_list; @@ -838,10 +820,11 @@ cl_qlist_insert_tail( * Put the new element in front of the end which is the same * as being at the tail */ - __cl_primitive_insert( &p_list->end, p_list_item ); + __cl_primitive_insert(&p_list->end, p_list_item); p_list->count++; } + /* * PARAMETERS * p_list @@ -875,9 +858,8 @@ cl_qlist_insert_tail( * SYNOPSIS */ void -cl_qlist_insert_list_head( - IN cl_qlist_t* const p_dest_list, - IN cl_qlist_t* const p_src_list ); +cl_qlist_insert_list_head(IN cl_qlist_t * const p_dest_list, + IN cl_qlist_t * const p_src_list); /* * PARAMETERS * p_dest_list @@ -914,9 +896,8 @@ cl_qlist_insert_list_head( * SYNOPSIS */ void -cl_qlist_insert_list_tail( - IN cl_qlist_t* const p_dest_list, - IN cl_qlist_t* const p_src_list ); +cl_qlist_insert_list_tail(IN cl_qlist_t * const p_dest_list, + IN cl_qlist_t * const p_src_list); /* * PARAMETERS * p_dest_list @@ -953,11 +934,9 @@ cl_qlist_insert_list_tail( * SYNOPSIS */ void -cl_qlist_insert_array_head( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ); +cl_qlist_insert_array_head(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_array, + IN uint32_t item_count, IN const uint32_t item_size); /* * PARAMETERS * p_list @@ -1004,11 +983,9 @@ cl_qlist_insert_array_head( * SYNOPSIS */ void -cl_qlist_insert_array_tail( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size); +cl_qlist_insert_array_tail(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_array, + IN uint32_t item_count, IN const uint32_t item_size); /* * PARAMETERS * p_list @@ -1055,19 +1032,18 @@ cl_qlist_insert_array_tail( * SYNOPSIS */ static inline void -cl_qlist_insert_prev( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_list_item, - IN cl_list_item_t* const p_new_item ) +cl_qlist_insert_prev(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_list_item, + IN cl_list_item_t * const p_new_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_new_item ); + CL_ASSERT(p_new_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* * The list item must not already be part of the list. Note that this @@ -1075,16 +1051,17 @@ cl_qlist_insert_prev( * list pointer equal to the specified list. The chances of this * happening are acceptable in light of the value of this check. */ - CL_ASSERT( p_new_item->p_list != p_list ); + CL_ASSERT(p_new_item->p_list != p_list); #if defined( _DEBUG_ ) p_new_item->p_list = p_list; #endif - __cl_primitive_insert( p_list_item, p_new_item ); + __cl_primitive_insert(p_list_item, p_new_item); p_list->count++; } + /* * PARAMETERS * p_list @@ -1119,19 +1096,18 @@ cl_qlist_insert_prev( * SYNOPSIS */ static inline void -cl_qlist_insert_next( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_list_item, - IN cl_list_item_t* const p_new_item ) +cl_qlist_insert_next(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_list_item, + IN cl_list_item_t * const p_new_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_new_item ); + CL_ASSERT(p_new_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* * The list item must not already be part of the list. Note that this @@ -1139,16 +1115,17 @@ cl_qlist_insert_next( * list pointer equal to the specified list. The chances of this * happening are acceptable in light of the value of this check. */ - CL_ASSERT( p_new_item->p_list != p_list ); + CL_ASSERT(p_new_item->p_list != p_list); #if defined( _DEBUG_ ) p_new_item->p_list = p_list; #endif - __cl_primitive_insert( cl_qlist_next( p_list_item ), p_new_item ); + __cl_primitive_insert(cl_qlist_next(p_list_item), p_new_item); p_list->count++; } + /* * PARAMETERS * p_list @@ -1183,35 +1160,34 @@ cl_qlist_insert_next( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_remove_head( - IN cl_qlist_t* const p_list ) +static inline cl_list_item_t *cl_qlist_remove_head(IN cl_qlist_t * const p_list) { - cl_list_item_t *p_item; + cl_list_item_t *p_item; /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - p_item = cl_qlist_head( p_list ); + p_item = cl_qlist_head(p_list); /* CL_ASSERT that the list item is part of the list. */ - CL_ASSERT( p_item->p_list == p_list ); + CL_ASSERT(p_item->p_list == p_list); - if( p_item == cl_qlist_end( p_list ) ) - return( p_item ); + if (p_item == cl_qlist_end(p_list)) + return (p_item); #if defined( _DEBUG_ ) /* Clear the item's link to the list. */ p_item->p_list = NULL; #endif - __cl_primitive_remove( p_item ); + __cl_primitive_remove(p_item); p_list->count--; - return( p_item ); + return (p_item); } + /* * PARAMETERS * p_list @@ -1237,35 +1213,34 @@ cl_qlist_remove_head( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_remove_tail( - IN cl_qlist_t* const p_list ) +static inline cl_list_item_t *cl_qlist_remove_tail(IN cl_qlist_t * const p_list) { - cl_list_item_t *p_item; + cl_list_item_t *p_item; /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); - p_item = cl_qlist_tail( p_list ); + p_item = cl_qlist_tail(p_list); /* CL_ASSERT that the list item is part of the list. */ - CL_ASSERT( p_item->p_list == p_list ); + CL_ASSERT(p_item->p_list == p_list); - if( p_item == cl_qlist_end( p_list ) ) - return( p_item ); + if (p_item == cl_qlist_end(p_list)) + return (p_item); #if defined( _DEBUG_ ) /* Clear the item's link to the list. */ p_item->p_list = NULL; #endif - __cl_primitive_remove( p_item ); + __cl_primitive_remove(p_item); p_list->count--; - return( p_item ); + return (p_item); } + /* * PARAMETERS * p_list @@ -1291,20 +1266,19 @@ cl_qlist_remove_tail( * SYNOPSIS */ static inline void -cl_qlist_remove_item( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_list_item ) +cl_qlist_remove_item(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_list_item) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* CL_ASSERT that the list item is part of the list. */ - CL_ASSERT( p_list_item->p_list == p_list ); + CL_ASSERT(p_list_item->p_list == p_list); - if( p_list_item == cl_qlist_end( p_list ) ) + if (p_list_item == cl_qlist_end(p_list)) return; #if defined( _DEBUG_ ) @@ -1312,10 +1286,11 @@ cl_qlist_remove_item( p_list_item->p_list = NULL; #endif - __cl_primitive_remove( p_list_item ); + __cl_primitive_remove(p_list_item); p_list->count--; } + /* * PARAMETERS * p_list @@ -1345,27 +1320,25 @@ cl_qlist_remove_item( * * SYNOPSIS */ -static inline void -cl_qlist_remove_all( - IN cl_qlist_t* const p_list ) +static inline void cl_qlist_remove_all(IN cl_qlist_t * const p_list) { #if defined( _DEBUG_ ) - cl_list_item_t *p_list_item; + cl_list_item_t *p_list_item; /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); - p_list_item = cl_qlist_head( p_list ); - while( p_list_item != cl_qlist_end( p_list ) ) - { - p_list_item = cl_qlist_next( p_list_item ); - cl_qlist_prev( p_list_item )->p_list = NULL; + CL_ASSERT(p_list->state == CL_INITIALIZED); + p_list_item = cl_qlist_head(p_list); + while (p_list_item != cl_qlist_end(p_list)) { + p_list_item = cl_qlist_next(p_list_item); + cl_qlist_prev(p_list_item)->p_list = NULL; } #endif - __cl_qlist_reset( p_list ); + __cl_qlist_reset(p_list); } + /* * PARAMETERS * p_list @@ -1390,9 +1363,8 @@ cl_qlist_remove_all( * SYNOPSIS */ boolean_t -cl_is_item_in_qlist( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item ); +cl_is_item_in_qlist(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item); /* * PARAMETERS * p_list @@ -1420,12 +1392,10 @@ cl_is_item_in_qlist( * * SYNOPSIS */ -cl_list_item_t* -cl_qlist_find_next( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ); +cl_list_item_t *cl_qlist_find_next(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1474,12 +1444,10 @@ cl_qlist_find_next( * * SYNOPSIS */ -cl_list_item_t* -cl_qlist_find_prev( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ); +cl_list_item_t *cl_qlist_find_prev(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1528,22 +1496,24 @@ cl_qlist_find_prev( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_find_from_head( - IN const cl_qlist_t* const p_list, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ) +static inline cl_list_item_t *cl_qlist_find_from_head(IN const cl_qlist_t * + const p_list, + IN cl_pfn_qlist_find_t + pfn_func, + IN const void *const + context) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* CL_ASSERT that a find function is provided. */ - CL_ASSERT( pfn_func ); + CL_ASSERT(pfn_func); - return( cl_qlist_find_next( p_list, cl_qlist_end( p_list ), pfn_func, - context ) ); + return (cl_qlist_find_next(p_list, cl_qlist_end(p_list), pfn_func, + context)); } + /* * PARAMETERS * p_list @@ -1588,22 +1558,24 @@ cl_qlist_find_from_head( * * SYNOPSIS */ -static inline cl_list_item_t* -cl_qlist_find_from_tail( - IN const cl_qlist_t* const p_list, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ) +static inline cl_list_item_t *cl_qlist_find_from_tail(IN const cl_qlist_t * + const p_list, + IN cl_pfn_qlist_find_t + pfn_func, + IN const void *const + context) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_list ); + CL_ASSERT(p_list); /* CL_ASSERT that the list was initialized. */ - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* CL_ASSERT that a find function is provided. */ - CL_ASSERT( pfn_func ); + CL_ASSERT(pfn_func); - return( cl_qlist_find_prev( p_list, cl_qlist_end( p_list ), pfn_func, - context ) ); + return (cl_qlist_find_prev(p_list, cl_qlist_end(p_list), pfn_func, + context)); } + /* * PARAMETERS * p_list @@ -1649,10 +1621,9 @@ cl_qlist_find_from_tail( * SYNOPSIS */ void -cl_qlist_apply_func( - IN const cl_qlist_t* const p_list, - IN cl_pfn_qlist_apply_t pfn_func, - IN const void* const context ); +cl_qlist_apply_func(IN const cl_qlist_t * const p_list, + IN cl_pfn_qlist_apply_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_list @@ -1689,11 +1660,10 @@ cl_qlist_apply_func( * SYNOPSIS */ void -cl_qlist_move_items( - IN cl_qlist_t* const p_src_list, - IN cl_qlist_t* const p_dest_list, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ); +cl_qlist_move_items(IN cl_qlist_t * const p_src_list, + IN cl_qlist_t * const p_dest_list, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_src_list @@ -1733,5 +1703,4 @@ cl_qlist_move_items( *********/ END_C_DECLS - -#endif /* _CL_QUICK_LIST_H_ */ +#endif /* _CL_QUICK_LIST_H_ */ diff --git a/opensm/include/complib/cl_qlockpool.h b/opensm/include/complib/cl_qlockpool.h index 3404d05..58a9b4b 100644 --- a/opensm/include/complib/cl_qlockpool.h +++ b/opensm/include/complib/cl_qlockpool.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Quick Locking Pool * NAME * Quick Locking Pool @@ -80,7 +79,6 @@ BEGIN_C_DECLS * Manipulation * cl_qlock_pool_get, cl_qlock_pool_put *********/ - /****s* Component Library: Quick Locking Pool/cl_qlock_pool_t * NAME * cl_qlock_pool_t @@ -93,10 +91,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_qlock_pool -{ - cl_spinlock_t lock; - cl_qpool_t pool; + typedef struct _cl_qlock_pool { + cl_spinlock_t lock; + cl_qpool_t pool; } cl_qlock_pool_t; /* @@ -120,12 +117,10 @@ typedef struct _cl_qlock_pool * * SYNOPSIS */ -static inline void -cl_qlock_pool_construct( - IN cl_qlock_pool_t* const p_pool ) +static inline void cl_qlock_pool_construct(IN cl_qlock_pool_t * const p_pool) { - cl_qpool_construct( &p_pool->pool ); - cl_spinlock_construct( &p_pool->lock ); + cl_qpool_construct(&p_pool->pool); + cl_spinlock_construct(&p_pool->lock); } /* @@ -156,25 +151,22 @@ cl_qlock_pool_construct( * * SYNOPSIS */ -static inline void -cl_qlock_pool_destroy( - IN cl_qlock_pool_t* const p_pool ) +static inline void cl_qlock_pool_destroy(IN cl_qlock_pool_t * const p_pool) { /* - If the pool has already been put into use, grab the lock - to sync with other threads before we blow everything away. - */ - if( cl_is_qpool_inited( &p_pool->pool ) ) - { - cl_spinlock_acquire( &p_pool->lock ); - cl_qpool_destroy( &p_pool->pool ); - cl_spinlock_release( &p_pool->lock ); - } - else - cl_qpool_destroy( &p_pool->pool ); + If the pool has already been put into use, grab the lock + to sync with other threads before we blow everything away. + */ + if (cl_is_qpool_inited(&p_pool->pool)) { + cl_spinlock_acquire(&p_pool->lock); + cl_qpool_destroy(&p_pool->pool); + cl_spinlock_release(&p_pool->lock); + } else + cl_qpool_destroy(&p_pool->pool); - cl_spinlock_destroy( &p_pool->lock ); + cl_spinlock_destroy(&p_pool->lock); } + /* * PARAMETERS * p_pool @@ -203,29 +195,30 @@ cl_qlock_pool_destroy( * SYNOPSIS */ static inline cl_status_t -cl_qlock_pool_init( - IN cl_qlock_pool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t object_size, - IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ) +cl_qlock_pool_init(IN cl_qlock_pool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t object_size, + IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context) { cl_status_t status; - cl_qlock_pool_construct( p_pool ); + cl_qlock_pool_construct(p_pool); - status = cl_spinlock_init( &p_pool->lock ); - if( status ) - return( status ); + status = cl_spinlock_init(&p_pool->lock); + if (status) + return (status); - status = cl_qpool_init( &p_pool->pool, min_size, max_size, grow_size, - object_size, pfn_initializer, pfn_destructor, context ); + status = cl_qpool_init(&p_pool->pool, min_size, max_size, grow_size, + object_size, pfn_initializer, pfn_destructor, + context); - return( status ); + return (status); } + /* * PARAMETERS * p_pool @@ -292,15 +285,14 @@ cl_qlock_pool_init( * * SYNOPSIS */ -static inline cl_pool_item_t* -cl_qlock_pool_get( - IN cl_qlock_pool_t* const p_pool ) +static inline cl_pool_item_t *cl_qlock_pool_get(IN cl_qlock_pool_t * + const p_pool) { - cl_pool_item_t* p_item; - cl_spinlock_acquire( &p_pool->lock ); - p_item = cl_qpool_get( &p_pool->pool ); - cl_spinlock_release( &p_pool->lock ); - return( p_item ); + cl_pool_item_t *p_item; + cl_spinlock_acquire(&p_pool->lock); + p_item = cl_qpool_get(&p_pool->pool); + cl_spinlock_release(&p_pool->lock); + return (p_item); } /* @@ -332,14 +324,14 @@ cl_qlock_pool_get( * SYNOPSIS */ static inline void -cl_qlock_pool_put( - IN cl_qlock_pool_t* const p_pool, - IN cl_pool_item_t* const p_item ) +cl_qlock_pool_put(IN cl_qlock_pool_t * const p_pool, + IN cl_pool_item_t * const p_item) { - cl_spinlock_acquire( &p_pool->lock ); - cl_qpool_put( &p_pool->pool, p_item ); - cl_spinlock_release( &p_pool->lock ); + cl_spinlock_acquire(&p_pool->lock); + cl_qpool_put(&p_pool->pool, p_item); + cl_spinlock_release(&p_pool->lock); } + /* * PARAMETERS * p_pool @@ -361,5 +353,4 @@ cl_qlock_pool_put( *********/ END_C_DECLS - -#endif /* _CL_QLOCKPOOL_H_ */ +#endif /* _CL_QLOCKPOOL_H_ */ diff --git a/opensm/include/complib/cl_qmap.h b/opensm/include/complib/cl_qmap.h index e069d6f..959c3ca 100644 --- a/opensm/include/complib/cl_qmap.h +++ b/opensm/include/complib/cl_qmap.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Quick Map * NAME * Quick Map @@ -106,7 +105,6 @@ BEGIN_C_DECLS * Attributes: * cl_qmap_count, cl_is_qmap_empty, *********/ - /****i* Component Library: Quick Map/cl_map_color_t * NAME * cl_map_color_t @@ -117,11 +115,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef enum _cl_map_color -{ + typedef enum _cl_map_color { CL_MAP_RED, CL_MAP_BLACK - } cl_map_color_t; /* * VALUES @@ -147,17 +143,16 @@ typedef enum _cl_map_color * * SYNOPSIS */ -typedef struct _cl_map_item -{ +typedef struct _cl_map_item { /* Must be first to allow casting. */ - cl_pool_item_t pool_item; - struct _cl_map_item *p_left; - struct _cl_map_item *p_right; - struct _cl_map_item *p_up; - cl_map_color_t color; - uint64_t key; + cl_pool_item_t pool_item; + struct _cl_map_item *p_left; + struct _cl_map_item *p_right; + struct _cl_map_item *p_up; + cl_map_color_t color; + uint64_t key; #ifdef _DEBUG_ - struct _cl_qmap *p_map; + struct _cl_qmap *p_map; #endif } cl_map_item_t; @@ -216,10 +211,9 @@ typedef struct _cl_map_item * * SYNOPSIS */ -typedef struct _cl_map_obj -{ - cl_map_item_t item; - const void *p_object; +typedef struct _cl_map_obj { + cl_map_item_t item; + const void *p_object; } cl_map_obj_t; /* @@ -256,12 +250,11 @@ typedef struct _cl_map_obj * * SYNOPSIS */ -typedef struct _cl_qmap -{ - cl_map_item_t root; - cl_map_item_t nil; - cl_state_t state; - size_t count; +typedef struct _cl_qmap { + cl_map_item_t root; + cl_map_item_t nil; + cl_state_t state; + size_t count; } cl_qmap_t; /* @@ -297,9 +290,7 @@ typedef struct _cl_qmap * SYNOPSIS */ typedef void -(*cl_pfn_qmap_apply_t)( - IN cl_map_item_t* const p_map_item, - IN void* context ); + (*cl_pfn_qmap_apply_t) (IN cl_map_item_t * const p_map_item, IN void *context); /* * PARAMETERS * p_map_item @@ -330,14 +321,13 @@ typedef void * * SYNOPSIS */ -static inline uint32_t -cl_qmap_count( - IN const cl_qmap_t* const p_map ) +static inline uint32_t cl_qmap_count(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( (uint32_t)p_map->count ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return ((uint32_t) p_map->count); } + /* * PARAMETERS * p_map @@ -359,15 +349,14 @@ cl_qmap_count( * * SYNOPSIS */ -static inline boolean_t -cl_is_qmap_empty( - IN const cl_qmap_t* const p_map ) +static inline boolean_t cl_is_qmap_empty(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - return( p_map->count == 0 ); + return (p_map->count == 0); } + /* * PARAMETERS * p_map @@ -392,13 +381,13 @@ cl_is_qmap_empty( * SYNOPSIS */ static inline void -cl_qmap_set_obj( - IN cl_map_obj_t* const p_map_obj, - IN const void* const p_object ) +cl_qmap_set_obj(IN cl_map_obj_t * const p_map_obj, + IN const void *const p_object) { - CL_ASSERT( p_map_obj ); + CL_ASSERT(p_map_obj); p_map_obj->p_object = p_object; } + /* * PARAMETERS * p_map_obj @@ -424,13 +413,12 @@ cl_qmap_set_obj( * * SYNOPSIS */ -static inline void* -cl_qmap_obj( - IN const cl_map_obj_t* const p_map_obj ) +static inline void *cl_qmap_obj(IN const cl_map_obj_t * const p_map_obj) { - CL_ASSERT( p_map_obj ); - return( (void*)p_map_obj->p_object ); + CL_ASSERT(p_map_obj); + return ((void *)p_map_obj->p_object); } + /* * PARAMETERS * p_map_obj @@ -452,13 +440,12 @@ cl_qmap_obj( * * SYNOPSIS */ -static inline uint64_t -cl_qmap_key( - IN const cl_map_item_t* const p_item ) +static inline uint64_t cl_qmap_key(IN const cl_map_item_t * const p_item) { - CL_ASSERT( p_item ); - return( p_item->key ); + CL_ASSERT(p_item); + return (p_item->key); } + /* * PARAMETERS * p_item @@ -483,9 +470,7 @@ cl_qmap_key( * * SYNOPSIS */ -void -cl_qmap_init( - IN cl_qmap_t* const p_map ); +void cl_qmap_init(IN cl_qmap_t * const p_map); /* * PARAMETERS * p_map @@ -510,15 +495,14 @@ cl_qmap_init( * * SYNOPSIS */ -static inline const cl_map_item_t* -cl_qmap_end( - IN const cl_qmap_t* const p_map ) +static inline const cl_map_item_t *cl_qmap_end(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); /* Nil is the end of the map. */ - return( &p_map->nil ); + return (&p_map->nil); } + /* * PARAMETERS * p_map @@ -549,14 +533,13 @@ cl_qmap_end( * * SYNOPSIS */ -static inline cl_map_item_t* -cl_qmap_head( - IN const cl_qmap_t* const p_map ) +static inline cl_map_item_t *cl_qmap_head(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( (cl_map_item_t*)p_map->nil.pool_item.list_item.p_next ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return ((cl_map_item_t *) p_map->nil.pool_item.list_item.p_next); } + /* * PARAMETERS * p_map @@ -586,14 +569,13 @@ cl_qmap_head( * * SYNOPSIS */ -static inline cl_map_item_t* -cl_qmap_tail( - IN const cl_qmap_t* const p_map ) +static inline cl_map_item_t *cl_qmap_tail(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - return( (cl_map_item_t*)p_map->nil.pool_item.list_item.p_prev ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + return ((cl_map_item_t *) p_map->nil.pool_item.list_item.p_prev); } + /* * PARAMETERS * p_map @@ -623,13 +605,12 @@ cl_qmap_tail( * * SYNOPSIS */ -static inline cl_map_item_t* -cl_qmap_next( - IN const cl_map_item_t* const p_item ) +static inline cl_map_item_t *cl_qmap_next(IN const cl_map_item_t * const p_item) { - CL_ASSERT( p_item ); - return( (cl_map_item_t*)p_item->pool_item.list_item.p_next ); + CL_ASSERT(p_item); + return ((cl_map_item_t *) p_item->pool_item.list_item.p_next); } + /* * PARAMETERS * p_item @@ -656,13 +637,12 @@ cl_qmap_next( * * SYNOPSIS */ -static inline cl_map_item_t* -cl_qmap_prev( - IN const cl_map_item_t* const p_item ) +static inline cl_map_item_t *cl_qmap_prev(IN const cl_map_item_t * const p_item) { - CL_ASSERT( p_item ); - return( (cl_map_item_t*)p_item->pool_item.list_item.p_prev ); + CL_ASSERT(p_item); + return ((cl_map_item_t *) p_item->pool_item.list_item.p_prev); } + /* * PARAMETERS * p_item @@ -689,11 +669,9 @@ cl_qmap_prev( * * SYNOPSIS */ -cl_map_item_t* -cl_qmap_insert( - IN cl_qmap_t* const p_map, - IN const uint64_t key, - IN cl_map_item_t* const p_item ); +cl_map_item_t *cl_qmap_insert(IN cl_qmap_t * const p_map, + IN const uint64_t key, + IN cl_map_item_t * const p_item); /* * PARAMETERS * p_map @@ -727,10 +705,8 @@ cl_qmap_insert( * * SYNOPSIS */ -cl_map_item_t* -cl_qmap_get( - IN const cl_qmap_t* const p_map, - IN const uint64_t key ); +cl_map_item_t *cl_qmap_get(IN const cl_qmap_t * const p_map, + IN const uint64_t key); /* * PARAMETERS * p_map @@ -763,10 +739,8 @@ cl_qmap_get( * * SYNOPSIS */ -cl_map_item_t* -cl_qmap_get_next( - IN const cl_qmap_t* const p_map, - IN const uint64_t key ); +cl_map_item_t *cl_qmap_get_next(IN const cl_qmap_t * const p_map, + IN const uint64_t key); /* * PARAMETERS * p_map @@ -800,9 +774,8 @@ cl_qmap_get_next( * SYNOPSIS */ void -cl_qmap_remove_item( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* const p_item ); +cl_qmap_remove_item(IN cl_qmap_t * const p_map, + IN cl_map_item_t * const p_item); /* * PARAMETERS * p_item @@ -831,10 +804,8 @@ cl_qmap_remove_item( * * SYNOPSIS */ -cl_map_item_t* -cl_qmap_remove( - IN cl_qmap_t* const p_map, - IN const uint64_t key ); +cl_map_item_t *cl_qmap_remove(IN cl_qmap_t * const p_map, + IN const uint64_t key); /* * PARAMETERS * p_map @@ -864,18 +835,17 @@ cl_qmap_remove( * * SYNOPSIS */ -static inline void -cl_qmap_remove_all( - IN cl_qmap_t* const p_map ) +static inline void cl_qmap_remove_all(IN cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); p_map->root.p_left = &p_map->nil; p_map->nil.pool_item.list_item.p_next = &p_map->nil.pool_item.list_item; p_map->nil.pool_item.list_item.p_prev = &p_map->nil.pool_item.list_item; p_map->count = 0; } + /* * PARAMETERS * p_map @@ -899,9 +869,8 @@ cl_qmap_remove_all( * SYNOPSIS */ void -cl_qmap_merge( - OUT cl_qmap_t* const p_dest_map, - IN OUT cl_qmap_t* const p_src_map ); +cl_qmap_merge(OUT cl_qmap_t * const p_dest_map, + IN OUT cl_qmap_t * const p_src_map); /* * PARAMETERS * p_dest_map @@ -934,11 +903,9 @@ cl_qmap_merge( * SYNOPSIS */ void -cl_qmap_delta( - IN OUT cl_qmap_t* const p_map1, - IN OUT cl_qmap_t* const p_map2, - OUT cl_qmap_t* const p_new, - OUT cl_qmap_t* const p_old ); +cl_qmap_delta(IN OUT cl_qmap_t * const p_map1, + IN OUT cl_qmap_t * const p_map2, + OUT cl_qmap_t * const p_new, OUT cl_qmap_t * const p_old); /* * PARAMETERS * p_map1 @@ -985,10 +952,9 @@ cl_qmap_delta( * SYNOPSIS */ void -cl_qmap_apply_func( - IN const cl_qmap_t* const p_map, - IN cl_pfn_qmap_apply_t pfn_func, - IN const void* const context ); +cl_qmap_apply_func(IN const cl_qmap_t * const p_map, + IN cl_pfn_qmap_apply_t pfn_func, + IN const void *const context); /* * PARAMETERS * p_map @@ -1014,5 +980,4 @@ cl_qmap_apply_func( *********/ END_C_DECLS - -#endif /* _CL_QMAP_H_ */ +#endif /* _CL_QMAP_H_ */ diff --git a/opensm/include/complib/cl_qpool.h b/opensm/include/complib/cl_qpool.h index 0b3f756..5a38206 100644 --- a/opensm/include/complib/cl_qpool.h +++ b/opensm/include/complib/cl_qpool.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Quick Pool * NAME * Quick Pool @@ -102,7 +101,6 @@ BEGIN_C_DECLS * Attributes: * cl_is_qpool_inited, cl_qpool_count *********/ - /****d* Component Library: Quick Pool/cl_pfn_qpool_init_t * NAME * cl_pfn_qpool_init_t @@ -115,10 +113,9 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_qpool_init_t)( - IN void* const p_object, - IN void* context, - OUT cl_pool_item_t** const pp_pool_item ); + (*cl_pfn_qpool_init_t) (IN void *const p_object, + IN void *context, + OUT cl_pool_item_t ** const pp_pool_item); /* * PARAMETERS * p_object @@ -172,9 +169,8 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_qpool_dtor_t)( - IN const cl_pool_item_t* const p_pool_item, - IN void* context ); + (*cl_pfn_qpool_dtor_t) (IN const cl_pool_item_t * const p_pool_item, + IN void *context); /* * PARAMETERS * p_pool_item @@ -212,12 +208,11 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_qpool -{ - cl_qcpool_t qcpool; - cl_pfn_qpool_init_t pfn_init; - cl_pfn_qpool_dtor_t pfn_dtor; - const void *context; +typedef struct _cl_qpool { + cl_qcpool_t qcpool; + cl_pfn_qpool_init_t pfn_init; + cl_pfn_qpool_dtor_t pfn_dtor; + const void *context; } cl_qpool_t; /* @@ -252,9 +247,7 @@ typedef struct _cl_qpool * * SYNOPSIS */ -void -cl_qpool_construct( - IN cl_qpool_t* const p_pool ); +void cl_qpool_construct(IN cl_qpool_t * const p_pool); /* * PARAMETERS * p_pool @@ -283,14 +276,13 @@ cl_qpool_construct( * * SYNOPSIS */ -static inline uint32_t -cl_is_qpool_inited( - IN const cl_qpool_t* const p_pool ) +static inline uint32_t cl_is_qpool_inited(IN const cl_qpool_t * const p_pool) { /* CL_ASSERT that a non-null pointer is provided. */ - CL_ASSERT( p_pool ); - return( cl_is_qcpool_inited( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_is_qcpool_inited(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -320,15 +312,14 @@ cl_is_qpool_inited( * SYNOPSIS */ cl_status_t -cl_qpool_init( - IN cl_qpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t object_size, - IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ); +cl_qpool_init(IN cl_qpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t object_size, + IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_pool @@ -399,13 +390,12 @@ cl_qpool_init( * * SYNOPSIS */ -static inline void -cl_qpool_destroy( - IN cl_qpool_t* const p_pool ) +static inline void cl_qpool_destroy(IN cl_qpool_t * const p_pool) { - CL_ASSERT( p_pool ); - cl_qcpool_destroy( &p_pool->qcpool ); + CL_ASSERT(p_pool); + cl_qcpool_destroy(&p_pool->qcpool); } + /* * PARAMETERS * p_pool @@ -440,13 +430,12 @@ cl_qpool_destroy( * * SYNOPSIS */ -static inline size_t -cl_qpool_count( - IN cl_qpool_t* const p_pool ) +static inline size_t cl_qpool_count(IN cl_qpool_t * const p_pool) { - CL_ASSERT( p_pool ); - return( cl_qcpool_count( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_count(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -470,13 +459,12 @@ cl_qpool_count( * * SYNOPSIS */ -static inline cl_pool_item_t* -cl_qpool_get( - IN cl_qpool_t* const p_pool ) +static inline cl_pool_item_t *cl_qpool_get(IN cl_qpool_t * const p_pool) { - CL_ASSERT( p_pool ); - return( cl_qcpool_get( &p_pool->qcpool ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_get(&p_pool->qcpool)); } + /* * PARAMETERS * p_pool @@ -507,13 +495,13 @@ cl_qpool_get( * SYNOPSIS */ static inline void -cl_qpool_put( - IN cl_qpool_t* const p_pool, - IN cl_pool_item_t* const p_pool_item ) +cl_qpool_put(IN cl_qpool_t * const p_pool, + IN cl_pool_item_t * const p_pool_item) { - CL_ASSERT( p_pool ); - cl_qcpool_put( &p_pool->qcpool, p_pool_item ); + CL_ASSERT(p_pool); + cl_qcpool_put(&p_pool->qcpool, p_pool_item); } + /* * PARAMETERS * p_pool @@ -548,13 +536,12 @@ cl_qpool_put( * SYNOPSIS */ static inline void -cl_qpool_put_list( - IN cl_qpool_t* const p_pool, - IN cl_qlist_t* const p_list ) +cl_qpool_put_list(IN cl_qpool_t * const p_pool, IN cl_qlist_t * const p_list) { - CL_ASSERT( p_pool ); - cl_qcpool_put_list( &p_pool->qcpool, p_list ); + CL_ASSERT(p_pool); + cl_qcpool_put_list(&p_pool->qcpool, p_list); } + /* * PARAMETERS * p_pool @@ -589,13 +576,12 @@ cl_qpool_put_list( * SYNOPSIS */ static inline cl_status_t -cl_qpool_grow( - IN cl_qpool_t* const p_pool, - IN const size_t obj_count ) +cl_qpool_grow(IN cl_qpool_t * const p_pool, IN const size_t obj_count) { - CL_ASSERT( p_pool ); - return( cl_qcpool_grow( &p_pool->qcpool, obj_count ) ); + CL_ASSERT(p_pool); + return (cl_qcpool_grow(&p_pool->qcpool, obj_count)); } + /* * PARAMETERS * p_pool @@ -623,5 +609,4 @@ cl_qpool_grow( *********/ END_C_DECLS - -#endif /* _CL_QUICK_POOL_H_ */ +#endif /* _CL_QUICK_POOL_H_ */ diff --git a/opensm/include/complib/cl_spinlock.h b/opensm/include/complib/cl_spinlock.h index 89ca2b5..2e4cdf3 100644 --- a/opensm/include/complib/cl_spinlock.h +++ b/opensm/include/complib/cl_spinlock.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Public/Spinlock * NAME * Spinlock @@ -81,7 +80,6 @@ BEGIN_C_DECLS * cl_spinlock_acquire, cl_spinlock_release * cl_spinlock_acquire_irq, cl_spinlock_release_irq *********/ - /****f* Component Library: Spinlock/cl_spinlock_construct * NAME * cl_spinlock_construct @@ -92,9 +90,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -void -cl_spinlock_construct( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_construct(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -123,9 +119,7 @@ cl_spinlock_construct( * * SYNOPSIS */ -cl_status_t -cl_spinlock_init( - IN cl_spinlock_t* const p_spinlock ); +cl_status_t cl_spinlock_init(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -159,9 +153,7 @@ cl_spinlock_init( * * SYNOPSIS */ -void -cl_spinlock_destroy( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_destroy(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -192,9 +184,7 @@ cl_spinlock_destroy( * * SYNOPSIS */ -void -cl_spinlock_acquire( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_acquire(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -220,9 +210,7 @@ cl_spinlock_acquire( * * SYNOPSIS */ -void -cl_spinlock_acquire_irq( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_acquire_irq(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -244,9 +232,7 @@ cl_spinlock_acquire_irq( * * SYNOPSIS */ -void -cl_spinlock_release( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_release(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -271,9 +257,7 @@ cl_spinlock_release( * * SYNOPSIS */ -void -cl_spinlock_release_irq( - IN cl_spinlock_t* const p_spinlock ); +void cl_spinlock_release_irq(IN cl_spinlock_t * const p_spinlock); /* * PARAMETERS * p_spin_lock @@ -290,5 +274,4 @@ cl_spinlock_release_irq( *********/ END_C_DECLS - -#endif /* _CL_SPINLOCK_H_ */ +#endif /* _CL_SPINLOCK_H_ */ diff --git a/opensm/include/complib/cl_spinlock_osd.h b/opensm/include/complib/cl_spinlock_osd.h index ff8cded..5800340 100644 --- a/opensm/include/complib/cl_spinlock_osd.h +++ b/opensm/include/complib/cl_spinlock_osd.h @@ -49,23 +49,19 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #include -#include /* usr/include/ */ - -typedef struct _cl_spinlock_t -{ - pthread_mutex_t mutex; - cl_state_t state; +#include /* usr/include/ */ + typedef struct _cl_spinlock_t { + pthread_mutex_t mutex; + cl_state_t state; } cl_spinlock_t; END_C_DECLS - -#endif /* _CL_SPINLOCK_OSD_H_ */ +#endif /* _CL_SPINLOCK_OSD_H_ */ diff --git a/opensm/include/complib/cl_thread.h b/opensm/include/complib/cl_thread.h index ac90b0a..fdd73a9 100644 --- a/opensm/include/complib/cl_thread.h +++ b/opensm/include/complib/cl_thread.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****i* Component Library/Thread * NAME * Thread @@ -68,7 +67,6 @@ BEGIN_C_DECLS * The cl_thread_t structure should be treated as opaque and should be * manipulated only through the provided functions. *********/ - /****d* Component Library: Thread/cl_pfn_thread_callback_t * NAME * cl_pfn_thread_callback_t @@ -79,9 +77,8 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef void -(*cl_pfn_thread_callback_t)( - IN void* context ); + typedef void + (*cl_pfn_thread_callback_t) (IN void *context); /* * PARAMETERS * context @@ -110,12 +107,11 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_thread -{ - cl_thread_osd_t osd; - cl_pfn_thread_callback_t pfn_callback; - const void *context; - char name[16]; +typedef struct _cl_thread { + cl_thread_osd_t osd; + cl_pfn_thread_callback_t pfn_callback; + const void *context; + char name[16]; } cl_thread_t; /* @@ -145,9 +141,7 @@ typedef struct _cl_thread * * SYNOPSIS */ -void -cl_thread_construct( - IN cl_thread_t* const p_thread ); +void cl_thread_construct(IN cl_thread_t * const p_thread); /* * PARAMETERS * p_thread @@ -176,11 +170,9 @@ cl_thread_construct( * SYNOPSIS */ cl_status_t -cl_thread_init( - IN cl_thread_t* const p_thread, - IN cl_pfn_thread_callback_t pfn_callback, - IN const void* const context, - IN const char* const name ); +cl_thread_init(IN cl_thread_t * const p_thread, + IN cl_pfn_thread_callback_t pfn_callback, + IN const void *const context, IN const char *const name); /* * PARAMETERS * p_thread @@ -228,9 +220,7 @@ cl_thread_init( * * SYNOPSIS */ -void -cl_thread_destroy( - IN cl_thread_t* const p_thread ); +void cl_thread_destroy(IN cl_thread_t * const p_thread); /* * PARAMETERS * p_thread @@ -261,9 +251,7 @@ cl_thread_destroy( * * SYNOPSIS */ -void -cl_thread_suspend( - IN const uint32_t pause_ms ); +void cl_thread_suspend(IN const uint32_t pause_ms); /* * PARAMETERS * pause_ms @@ -291,9 +279,7 @@ cl_thread_suspend( * * SYNOPSIS */ -void -cl_thread_stall( - IN const uint32_t pause_us ); +void cl_thread_stall(IN const uint32_t pause_us); /* * PARAMETERS * pause_us @@ -321,8 +307,7 @@ cl_thread_stall( * * SYNOPSIS */ -int -cl_proc_count( void ); +int cl_proc_count(void); /* * RETURN VALUE * Returns the number of processors in the system. @@ -338,9 +323,7 @@ cl_proc_count( void ); * * SYNOPSIS */ -boolean_t -cl_is_current_thread( - IN const cl_thread_t* const p_thread ); +boolean_t cl_is_current_thread(IN const cl_thread_t * const p_thread); /* * PARAMETERS * p_thread @@ -367,8 +350,7 @@ cl_is_current_thread( * * SYNOPSIS */ -boolean_t -cl_is_blockable( void ); +boolean_t cl_is_blockable(void); /* * RETURN VALUE * TRUE @@ -378,5 +360,4 @@ cl_is_blockable( void ); *********/ END_C_DECLS - -#endif /* _CL_THREAD_H_ */ +#endif /* _CL_THREAD_H_ */ diff --git a/opensm/include/complib/cl_thread_osd.h b/opensm/include/complib/cl_thread_osd.h index a67b157..69ae1d2 100644 --- a/opensm/include/complib/cl_thread_osd.h +++ b/opensm/include/complib/cl_thread_osd.h @@ -49,31 +49,26 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #include #include #include - /* Linux user mode thread object structure definition. */ -typedef struct _cl_thread_osd_t -{ - pthread_t id; - cl_state_t state; + typedef struct _cl_thread_osd_t { + pthread_t id; + cl_state_t state; } cl_thread_osd_t; -static inline boolean_t -cl_is_blockable ( void ) +static inline boolean_t cl_is_blockable(void) { return TRUE; } END_C_DECLS - -#endif /* _CL_THREAD_OSD_H_ */ +#endif /* _CL_THREAD_OSD_H_ */ diff --git a/opensm/include/complib/cl_threadpool.h b/opensm/include/complib/cl_threadpool.h index eafbfa7..c136d2d 100644 --- a/opensm/include/complib/cl_threadpool.h +++ b/opensm/include/complib/cl_threadpool.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Thread Pool * NAME * Thread Pool @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Manipulation * cl_thread_pool_signal *********/ - /****s* Component Library: Thread Pool/cl_thread_pool_t * NAME * cl_thread_pool_t @@ -98,9 +96,8 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _cl_thread_pool -{ - void (*pfn_callback)(void*); + typedef struct _cl_thread_pool { + void (*pfn_callback) (void *); void *context; unsigned running_count; unsigned events; @@ -146,12 +143,10 @@ typedef struct _cl_thread_pool * SYNOPSIS */ cl_status_t -cl_thread_pool_init( - IN cl_thread_pool_t* const p_thread_pool, - IN unsigned count, - IN void (*pfn_callback)(void*), - IN void *context, - IN const char* const name ); +cl_thread_pool_init(IN cl_thread_pool_t * const p_thread_pool, + IN unsigned count, + IN void (*pfn_callback) (void *), + IN void *context, IN const char *const name); /* * PARAMETERS * p_thread_pool @@ -201,9 +196,7 @@ cl_thread_pool_init( * * SYNOPSIS */ -void -cl_thread_pool_destroy( - IN cl_thread_pool_t* const p_thread_pool ); +void cl_thread_pool_destroy(IN cl_thread_pool_t * const p_thread_pool); /* * PARAMETERS * p_thread_pool @@ -235,9 +228,7 @@ cl_thread_pool_destroy( * * SYNOPSIS */ -cl_status_t -cl_thread_pool_signal( - IN cl_thread_pool_t* const p_thread_pool ); +cl_status_t cl_thread_pool_signal(IN cl_thread_pool_t * const p_thread_pool); /* * PARAMETERS * p_thread_pool @@ -259,5 +250,4 @@ cl_thread_pool_signal( *********/ END_C_DECLS - -#endif /* _CL_THREAD_POOL_H_ */ +#endif /* _CL_THREAD_POOL_H_ */ diff --git a/opensm/include/complib/cl_timer.h b/opensm/include/complib/cl_timer.h index 29fef88..fdbfb31 100644 --- a/opensm/include/complib/cl_timer.h +++ b/opensm/include/complib/cl_timer.h @@ -51,13 +51,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Timer * NAME * Timer @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Manipulation: * cl_timer_start, cl_timer_stop *********/ - /****d* Component Library: Timer/cl_pfn_timer_callback_t * NAME * cl_pfn_timer_callback_t @@ -96,9 +94,8 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef void -(*cl_pfn_timer_callback_t)( - IN void* context ); + typedef void + (*cl_pfn_timer_callback_t) (IN void *context); /* * PARAMETERS * context @@ -130,9 +127,7 @@ typedef void * * SYNOPSIS */ -void -cl_timer_construct( - IN cl_timer_t* const p_timer ); +void cl_timer_construct(IN cl_timer_t * const p_timer); /* * PARAMETERS * p_timer @@ -161,10 +156,9 @@ cl_timer_construct( * SYNOPSIS */ cl_status_t -cl_timer_init( - IN cl_timer_t* const p_timer, - IN cl_pfn_timer_callback_t pfn_callback, - IN const void* const context ); +cl_timer_init(IN cl_timer_t * const p_timer, + IN cl_pfn_timer_callback_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_timer @@ -200,9 +194,7 @@ cl_timer_init( * * SYNOPSIS */ -void -cl_timer_destroy( - IN cl_timer_t* const p_timer ); +void cl_timer_destroy(IN cl_timer_t * const p_timer); /* * PARAMETERS * p_timer @@ -231,9 +223,7 @@ cl_timer_destroy( * SYNOPSIS */ cl_status_t -cl_timer_start( - IN cl_timer_t* const p_timer, - IN const uint32_t time_ms ); +cl_timer_start(IN cl_timer_t * const p_timer, IN const uint32_t time_ms); /* * PARAMETERS * p_timer @@ -267,9 +257,7 @@ cl_timer_start( * * SYNOPSIS */ -void -cl_timer_stop( - IN cl_timer_t* const p_timer ); +void cl_timer_stop(IN cl_timer_t * const p_timer); /* * PARAMETERS * p_timer @@ -297,9 +285,7 @@ cl_timer_stop( * SYNOPSIS */ cl_status_t -cl_timer_trim( - IN cl_timer_t* const p_timer, - IN const uint32_t time_ms ); +cl_timer_trim(IN cl_timer_t * const p_timer, IN const uint32_t time_ms); /* * PARAMETERS * p_timer @@ -337,8 +323,7 @@ cl_timer_trim( * * SYNOPSIS */ -uint64_t -cl_get_time_stamp( void ); +uint64_t cl_get_time_stamp(void); /* * RETURN VALUE * Time elapsed, in microseconds, since the system was booted. @@ -357,8 +342,7 @@ cl_get_time_stamp( void ); * * SYNOPSIS */ -uint32_t -cl_get_time_stamp_sec( void ); +uint32_t cl_get_time_stamp_sec(void); /* * RETURN VALUE * Time elapsed, in seconds, since the system was booted. @@ -368,5 +352,4 @@ cl_get_time_stamp_sec( void ); *********/ END_C_DECLS - -#endif /* _CL_TIMER_H_ */ +#endif /* _CL_TIMER_H_ */ diff --git a/opensm/include/complib/cl_timer_osd.h b/opensm/include/complib/cl_timer_osd.h index a5a3802..962cdfb 100644 --- a/opensm/include/complib/cl_timer_osd.h +++ b/opensm/include/complib/cl_timer_osd.h @@ -52,44 +52,36 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #include #include - -typedef enum _cl_timer_state -{ + typedef enum _cl_timer_state { CL_TIMER_IDLE, CL_TIMER_QUEUED, CL_TIMER_RUNNING - } cl_timer_state_t; -typedef struct _cl_timer_t -{ - cl_list_item_t list_item; - cl_timer_state_t timer_state; - cl_state_t state; - cl_pfn_timer_callback_t pfn_callback; - const void *context; - pthread_cond_t cond; - struct timespec timeout; +typedef struct _cl_timer_t { + cl_list_item_t list_item; + cl_timer_state_t timer_state; + cl_state_t state; + cl_pfn_timer_callback_t pfn_callback; + const void *context; + pthread_cond_t cond; + struct timespec timeout; } cl_timer_t; /* Internal functions to create the timer provider. */ -cl_status_t -__cl_timer_prov_create( void ); +cl_status_t __cl_timer_prov_create(void); /* Internal function to destroy the timer provider. */ -void -__cl_timer_prov_destroy( void ); +void __cl_timer_prov_destroy(void); END_C_DECLS - -#endif /* _CL_TIMER_OSD_H_ */ +#endif /* _CL_TIMER_OSD_H_ */ diff --git a/opensm/include/complib/cl_types.h b/opensm/include/complib/cl_types.h index 549250a..2663989 100644 --- a/opensm/include/complib/cl_types.h +++ b/opensm/include/complib/cl_types.h @@ -53,34 +53,32 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #include #include - -typedef uint16_t net16_t; -typedef uint32_t net32_t; -typedef uint64_t net64_t; +typedef uint16_t net16_t; +typedef uint32_t net32_t; +typedef uint64_t net64_t; /* explicit cast of void* to uint32_t */ #ifndef ASSERT_VOIDP2UINTN #if __WORDSIZE == 64 #define ASSERT_VOIDP2UINTN(var) \ CL_ASSERT( (intptr_t)var <= 0xffffffffffffffffL ) -#else /* __WORDSIZE == 64 */ +#else /* __WORDSIZE == 64 */ #if __WORDSIZE == 32 /* need to cast carefully to avoid the warining of un-needed check */ #define ASSERT_VOIDP2UINTN(var) \ CL_ASSERT( (intptr_t)var <= 0x100000000ULL ) -#else /* __WORDSIZE == 32 */ +#else /* __WORDSIZE == 32 */ #error "Need to know WORDSIZE to tell how to cast to unsigned long int" -#endif /* __WORDSIZE == 32 */ -#endif /* __WORDSIZE == 64 */ +#endif /* __WORDSIZE == 32 */ +#endif /* __WORDSIZE == 64 */ #endif /* explicit casting of void* to long */ @@ -188,10 +186,10 @@ typedef uint64_t net64_t; * keyword, if used, follows the parameter name. *********/ #ifndef IN -#define IN /* Function input parameter */ +#define IN /* Function input parameter */ #endif #ifndef OUT -#define OUT /* Function output parameter */ +#define OUT /* Function output parameter */ #endif #ifndef OPTIONAL #define OPTIONAL /* Optional function parameter - NULL if not used */ @@ -278,8 +276,7 @@ typedef uint64_t net64_t; * * SYNOPSIS */ -typedef enum _cl_status -{ +typedef enum _cl_status { CL_SUCCESS = 0, CL_ERROR, CL_INVALID_STATE, @@ -302,8 +299,7 @@ typedef enum _cl_status CL_DISCONNECT, CL_DUPLICATE, - CL_STATUS_COUNT /* should be the last value */ - + CL_STATUS_COUNT /* should be the last value */ } cl_status_t; /* * SEE ALSO @@ -311,7 +307,7 @@ typedef enum _cl_status *********/ /* Status values above converted to text for easier printing. */ -extern const char* cl_status_text[]; +extern const char *cl_status_text[]; #ifndef cl_panic /****f* Component Library: Error Trapping/cl_panic @@ -324,10 +320,7 @@ extern const char* cl_status_text[]; * * SYNOPSIS */ -void -cl_panic( - IN const char* const message, - IN ... ); +void cl_panic(IN const char *const message, IN ...); /* * PARAMETERS * message @@ -346,7 +339,7 @@ cl_panic( * * cl_panic sends the message to the current message logging target. *********/ -#endif /* cl_panic */ +#endif /* cl_panic */ /****d* Component Library: Data Types/CL_STATUS_MSG * NAME @@ -382,11 +375,11 @@ cl_panic( #if !defined( FALSE ) #define FALSE 0 -#endif /* !defined( TRUE ) */ +#endif /* !defined( TRUE ) */ #if !defined( TRUE ) #define TRUE (!FALSE) -#endif /* !defined( TRUE ) */ +#endif /* !defined( TRUE ) */ /****d* Component Library: Unreferenced Parameters/UNUSED_PARAM * NAME @@ -415,13 +408,11 @@ cl_panic( * * SYNOPSIS */ -typedef enum _cl_state -{ +typedef enum _cl_state { CL_UNINITIALIZED = 1, CL_INITIALIZED, CL_DESTROYING, CL_DESTROYED - } cl_state_t; /* * VALUES @@ -449,13 +440,12 @@ typedef enum _cl_state * * SYNOPSIS */ -static inline boolean_t -cl_is_state_valid( - IN const cl_state_t state ) +static inline boolean_t cl_is_state_valid(IN const cl_state_t state) { - return( (state == CL_UNINITIALIZED) || (state == CL_INITIALIZED) || - (state == CL_DESTROYING) || (state == CL_DESTROYED) ); + return ((state == CL_UNINITIALIZED) || (state == CL_INITIALIZED) || + (state == CL_DESTROYING) || (state == CL_DESTROYED)); } + /* * PARAMETERS * state @@ -476,5 +466,4 @@ cl_is_state_valid( *********/ END_C_DECLS - -#endif /* _DATA_TYPES_H_ */ +#endif /* _DATA_TYPES_H_ */ diff --git a/opensm/include/complib/cl_types_osd.h b/opensm/include/complib/cl_types_osd.h index 9d22847..fe7e829 100644 --- a/opensm/include/complib/cl_types_osd.h +++ b/opensm/include/complib/cl_types_osd.h @@ -52,40 +52,36 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if defined (_DEBUG_) #ifdef __IA64__ #define cl_break() asm(" break 0") -#else /* __IA64__ */ +#else /* __IA64__ */ #define cl_break() asm(" int $3") -#endif /* __IA64__ */ -#else /* _DEBUG_ */ +#endif /* __IA64__ */ +#else /* _DEBUG_ */ #define cl_break #endif - #include #include #include - #if defined (_DEBUG_) #define CL_ASSERT assert -#else /* _DEBUG_ */ +#else /* _DEBUG_ */ #define CL_ASSERT( __exp__ ) -#endif /* _DEBUG_ */ - +#endif /* _DEBUG_ */ /* * Types not explicitly defined are native to the platform. */ -typedef unsigned long uintn_t; -typedef long intn_t; -typedef int boolean_t; -typedef volatile int32_t atomic32_t; +typedef unsigned long uintn_t; +typedef long intn_t; +typedef int boolean_t; +typedef volatile int32_t atomic32_t; #ifndef NULL #define NULL (void*)0 @@ -94,5 +90,4 @@ typedef volatile int32_t atomic32_t; #define UNUSED_PARAM( P ) END_C_DECLS - -#endif /* _CL_TYPES_OSD_H_ */ +#endif /* _CL_TYPES_OSD_H_ */ diff --git a/opensm/include/complib/cl_vector.h b/opensm/include/complib/cl_vector.h index 049e223..e1e8ca7 100644 --- a/opensm/include/complib/cl_vector.h +++ b/opensm/include/complib/cl_vector.h @@ -53,13 +53,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* Component Library/Vector * NAME * Vector @@ -108,7 +107,6 @@ BEGIN_C_DECLS * cl_vector_find_from_start, cl_vector_find_from_end * cl_vector_apply_func *********/ - /****d* Component Library: Vector/cl_pfn_vec_init_t * NAME * cl_pfn_vec_init_t @@ -120,9 +118,7 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_vec_init_t)( - IN void* const p_element, - IN void* context ); + (*cl_pfn_vec_init_t) (IN void *const p_element, IN void *context); /* * PARAMETERS * p_element @@ -161,9 +157,7 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_vec_dtor_t)( - IN void* const p_element, - IN void* context ); + (*cl_pfn_vec_dtor_t) (IN void *const p_element, IN void *context); /* * PARAMETERS * p_element @@ -195,10 +189,8 @@ typedef void * SYNOPSIS */ typedef void -(*cl_pfn_vec_apply_t)( - IN const size_t index, - IN void* const p_element, - IN void* context ); + (*cl_pfn_vec_apply_t) (IN const size_t index, + IN void *const p_element, IN void *context); /* * PARAMETERS * index @@ -233,10 +225,8 @@ typedef void * SYNOPSIS */ typedef cl_status_t -(*cl_pfn_vec_find_t)( - IN const size_t index, - IN const void* const p_element, - IN void* context ); + (*cl_pfn_vec_find_t) (IN const size_t index, + IN const void *const p_element, IN void *context); /* * PARAMETERS * index @@ -274,10 +264,8 @@ typedef cl_status_t * SYNOPSIS */ typedef void -(*cl_pfn_vec_copy_t)( - IN void* const p_dest, - IN const void* const p_src, - IN const size_t size ); + (*cl_pfn_vec_copy_t) (IN void *const p_dest, + IN const void *const p_src, IN const size_t size); /* * PARAMETERS * p_dest @@ -308,19 +296,18 @@ typedef void * * SYNOPSIS */ -typedef struct _cl_vector -{ - size_t size; - size_t grow_size; - size_t capacity; - size_t element_size; - cl_pfn_vec_init_t pfn_init; - cl_pfn_vec_dtor_t pfn_dtor; - cl_pfn_vec_copy_t pfn_copy; - const void *context; - cl_qlist_t alloc_list; - void **p_ptr_array; - cl_state_t state; +typedef struct _cl_vector { + size_t size; + size_t grow_size; + size_t capacity; + size_t element_size; + cl_pfn_vec_init_t pfn_init; + cl_pfn_vec_dtor_t pfn_dtor; + cl_pfn_vec_copy_t pfn_copy; + const void *context; + cl_qlist_t alloc_list; + void **p_ptr_array; + cl_state_t state; } cl_vector_t; /* @@ -371,9 +358,7 @@ typedef struct _cl_vector * * SYNOPSIS */ -void -cl_vector_construct( - IN cl_vector_t* const p_vector ); +void cl_vector_construct(IN cl_vector_t * const p_vector); /* * PARAMETERS * p_vector @@ -402,14 +387,13 @@ cl_vector_construct( * SYNOPSIS */ cl_status_t -cl_vector_init( - IN cl_vector_t* const p_vector, - IN const size_t min_size, - IN const size_t grow_size, - IN const size_t element_size, - IN cl_pfn_vec_init_t pfn_init OPTIONAL, - IN cl_pfn_vec_dtor_t pfn_dtor OPTIONAL, - IN const void* const context ); +cl_vector_init(IN cl_vector_t * const p_vector, + IN const size_t min_size, + IN const size_t grow_size, + IN const size_t element_size, + IN cl_pfn_vec_init_t pfn_init OPTIONAL, + IN cl_pfn_vec_dtor_t pfn_dtor OPTIONAL, + IN const void *const context); /* * PARAMETERS * p_vector @@ -464,9 +448,7 @@ cl_vector_init( * * SYNOPSIS */ -void -cl_vector_destroy( - IN cl_vector_t* const p_vector ); +void cl_vector_destroy(IN cl_vector_t * const p_vector); /* * PARAMETERS * p_vector @@ -496,14 +478,14 @@ cl_vector_destroy( * SYNOPSIS */ static inline size_t -cl_vector_get_capacity( - IN const cl_vector_t* const p_vector ) +cl_vector_get_capacity(IN const cl_vector_t * const p_vector) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - return( p_vector->capacity ); + return (p_vector->capacity); } + /* * PARAMETERS * p_vector @@ -530,15 +512,14 @@ cl_vector_get_capacity( * * SYNOPSIS */ -static inline size_t -cl_vector_get_size( - IN const cl_vector_t* const p_vector ) +static inline size_t cl_vector_get_size(IN const cl_vector_t * const p_vector) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - return( p_vector->size ); + return (p_vector->size); } + /* * PARAMETERS * p_vector @@ -561,16 +542,15 @@ cl_vector_get_size( * * SYNOPSIS */ -static inline void* -cl_vector_get_ptr( - IN const cl_vector_t* const p_vector, - IN const size_t index ) +static inline void *cl_vector_get_ptr(IN const cl_vector_t * const p_vector, + IN const size_t index) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - return( p_vector->p_ptr_array[index] ); + return (p_vector->p_ptr_array[index]); } + /* * PARAMETERS * p_vector @@ -604,21 +584,20 @@ cl_vector_get_ptr( * SYNOPSIS */ static inline void -cl_vector_get( - IN const cl_vector_t* const p_vector, - IN const size_t index, - OUT void* const p_element ) +cl_vector_get(IN const cl_vector_t * const p_vector, + IN const size_t index, OUT void *const p_element) { void *p_src; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( p_element ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(p_element); /* Get a pointer to the element. */ - p_src = cl_vector_get_ptr( p_vector, index ); - p_vector->pfn_copy( p_src, p_element, p_vector->element_size ); + p_src = cl_vector_get_ptr(p_vector, index); + p_vector->pfn_copy(p_src, p_element, p_vector->element_size); } + /* * PARAMETERS * p_vector @@ -661,10 +640,8 @@ cl_vector_get( * SYNOPSIS */ cl_status_t -cl_vector_at( - IN const cl_vector_t* const p_vector, - IN const size_t index, - OUT void* const p_element ); +cl_vector_at(IN const cl_vector_t * const p_vector, + IN const size_t index, OUT void *const p_element); /* * PARAMETERS * p_vector @@ -703,10 +680,8 @@ cl_vector_at( * SYNOPSIS */ cl_status_t -cl_vector_set( - IN cl_vector_t* const p_vector, - IN const size_t index, - IN void* const p_element ); +cl_vector_set(IN cl_vector_t * const p_vector, + IN const size_t index, IN void *const p_element); /* * PARAMETERS * p_vector @@ -745,9 +720,8 @@ cl_vector_set( * SYNOPSIS */ cl_status_t -cl_vector_set_capacity( - IN cl_vector_t* const p_vector, - IN const size_t new_capacity ); +cl_vector_set_capacity(IN cl_vector_t * const p_vector, + IN const size_t new_capacity); /* * PARAMETERS * p_vector @@ -784,9 +758,7 @@ cl_vector_set_capacity( * SYNOPSIS */ cl_status_t -cl_vector_set_size( - IN cl_vector_t* const p_vector, - IN const size_t size ); +cl_vector_set_size(IN cl_vector_t * const p_vector, IN const size_t size); /* * PARAMETERS * p_vector @@ -826,9 +798,8 @@ cl_vector_set_size( * SYNOPSIS */ cl_status_t -cl_vector_set_min_size( - IN cl_vector_t* const p_vector, - IN const size_t min_size ); +cl_vector_set_min_size(IN cl_vector_t * const p_vector, + IN const size_t min_size); /* * PARAMETERS * p_vector @@ -865,10 +836,9 @@ cl_vector_set_min_size( * SYNOPSIS */ void -cl_vector_apply_func( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_apply_t pfn_callback, - IN const void* const context ); +cl_vector_apply_func(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_apply_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -905,10 +875,9 @@ cl_vector_apply_func( * SYNOPSIS */ size_t -cl_vector_find_from_start( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_find_t pfn_callback, - IN const void* const context ); +cl_vector_find_from_start(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_find_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -947,10 +916,9 @@ cl_vector_find_from_start( * SYNOPSIS */ size_t -cl_vector_find_from_end( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_find_t pfn_callback, - IN const void* const context ); +cl_vector_find_from_end(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_find_t pfn_callback, + IN const void *const context); /* * PARAMETERS * p_vector @@ -980,5 +948,4 @@ cl_vector_find_from_end( *********/ END_C_DECLS - -#endif /* _CL_VECTOR_H_ */ +#endif /* _CL_VECTOR_H_ */ -- 1.5.3.rc2.38.g11308 From kliteyn at mellanox.co.il Sun Aug 12 15:39:33 2007 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 13 Aug 2007 01:39:33 +0300 Subject: [ofa-general] Re: nightly osm_sim report 2007-08-12:normal completion In-Reply-To: <20070812174203.GG16267@sashak.voltaire.com> References: <20070812174203.GG16267@sashak.voltaire.com> Message-ID: <46BF8C25.20505@mellanox.co.il> Sasha Khapyorsky wrote: > Hi Yevgeny, > > On 07:44 Sun 12 Aug , kliteyn at mellanox.co.il wrote: > >> OSM Simulation Regression Summary >> >> [Generated mail - please do NOT reply] >> >> >> OpenSM rev = Wed_Aug_8_22:48:15_2007 [83d1f5ea339704f30e26918a897f9ead5a10678e] >> ibutils rev = Tue_Mar_13_14:36:32_2007 [80aaff94f0eb65117db39b9db7d609ffdcc055de] >> > > I see that ibutils already have the fix which makes pkey test compatible > with OpenSM. I think it should be useful to upgrade ibutils used in this > regression tests, so we will not see anymore such failures: > Thanks. I've just upgraded ibutils. Since it's a middle of the night, and some tests have already ran, we will see some failures tomorrow anyway. -- Yevgeny >> Failures: >> 42 Pkey IS1-16.topo >> 14 Pkey IS3-128.topo >> > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From kliteyn at mellanox.co.il Sun Aug 12 21:09:02 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 13 Aug 2007 07:09:02 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-13:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_9_19:03:32_2007 [9b2202bb79b153e51fda06860fcb8e45aaf43718] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=520 Pass=506 Fail=14 Pass: 39 Stability IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 38 LidMgr IS1-16.topo 30 Pkey IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo 12 OsmStress IS3-128.topo 10 Pkey IS3-128.topo Failures: 9 Pkey IS1-16.topo 3 Pkey IS3-128.topo 1 OsmStress IS3-128.topo 1 LidMgr IS1-16.topo From pradeeps at linux.vnet.ibm.com Sun Aug 12 22:56:24 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Sun, 12 Aug 2007 22:56:24 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46BBFCF3.1070109@linux.vnet.ibm.com> References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> Message-ID: <46BFF288.5020304@linux.vnet.ibm.com> Pradeep Satyanarayana wrote: > Michael S. Tsirkin wrote: >>> Quoting Pradeep Satyanarayana : >>> Subject: Re: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ >>> >>> Roland, Michael, >>> >>> Could you please review this patch and provide your comments? I would like to finish >>> up this work at the soonest and have this integrated into the for-2.6.24 tree. >> Naturally, with just a single packet pre-posted, RC QP will likely get RNR >> errors and close down quite soon. Wouldn't it make more sense to prepost >> some more WRs? >> > > This is only for completeness sake. Performance will be poor compared to two (or more) > Linux systems using this. This is expected to be really a corner and do not expect this > to happen. Hence is there for completeness only. If I interpreted this correctly, this > is was what Roland suggested (and I agree with the suggestion). > > Pradeep > Roland, Are there any other suggestions that you would like me to incorporate? Else, should I just roll up the patches into one and submit it? Pradeep From Colen.pald at solmser-pfadfinder.de Sun Aug 12 23:01:23 2007 From: Colen.pald at solmser-pfadfinder.de (Colen) Date: Mon, 13 Aug 2007 06:01:23 -0000 Subject: [ofa-general] result-openib-general Message-ID: -------------- next part -------------- A non-text attachment was scrubbed... Name: new agreements.pdf Type: application/octet-stream Size: 11900 bytes Desc: not available URL: From joe at perches.com Sun Aug 12 23:22:42 2007 From: joe at perches.com (joe at perches.com) Date: Sun, 12 Aug 2007 23:22:42 -0700 Subject: [ofa-general] [PATCH] [42/2many] MAINTAINERS - AMSO1100 RNIC DRIVER Message-ID: <46bff8b2.WLKhHT0C2uIfdSf+%joe@perches.com> Add file pattern to MAINTAINER entry Signed-off-by: Joe Perches diff --git a/MAINTAINERS b/MAINTAINERS index 635d812..e2e88f3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -412,6 +412,7 @@ P: Steve Wise M: swise at opengridcomputing.com L: general at lists.openfabrics.org S: Maintained +F: drivers/infiniband/hw/amso1100/ AOA (Apple Onboard Audio) ALSA DRIVER P: Johannes Berg From joe at perches.com Sun Aug 12 23:27:11 2007 From: joe at perches.com (joe at perches.com) Date: Sun, 12 Aug 2007 23:27:11 -0700 Subject: [ofa-general] [PATCH] [181/2many] MAINTAINERS - EHCA (IBM GX bus InfiniBand adapter) DRIVER: Message-ID: <46bff9bf.jz1f6LaulAG1xTlK%joe@perches.com> Add file pattern to MAINTAINER entry Signed-off-by: Joe Perches diff --git a/MAINTAINERS b/MAINTAINERS index ad6dc93..d5bceb3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1751,6 +1751,7 @@ P: Christoph Raisch M: raisch at de.ibm.com L: general at lists.openfabrics.org S: Supported +F: drivers/infiniband/hw/ehca/ EMULEX LPFC FC SCSI DRIVER P: James Smart From joe at perches.com Sun Aug 12 23:29:33 2007 From: joe at perches.com (joe at perches.com) Date: Sun, 12 Aug 2007 23:29:33 -0700 Subject: [ofa-general] [PATCH] [247/2many] MAINTAINERS - INFINIBAND SUBSYSTEM Message-ID: <46bffa4d.lrrIPefFmHe22+xM%joe@perches.com> Add file pattern to MAINTAINER entry Signed-off-by: Joe Perches diff --git a/MAINTAINERS b/MAINTAINERS index afc7780..7460778 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2341,6 +2341,9 @@ L: general at lists.openfabrics.org W: http://www.openib.org/ T: git kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git S: Supported +F: Documentation/infiniband/ +F: drivers/infiniband/ +F: include/linux/if_infiniband.h INPUT (KEYBOARD, MOUSE, JOYSTICK, TOUCHSCREEN) DRIVERS P: Dmitry Torokhov From joe at perches.com Sun Aug 12 23:30:11 2007 From: joe at perches.com (joe at perches.com) Date: Sun, 12 Aug 2007 23:30:11 -0700 Subject: [ofa-general] [PATCH] [263/2many] MAINTAINERS - IPATH DRIVER: Message-ID: <46bffa73.TlHrGgz5328/gmg4%joe@perches.com> Add file pattern to MAINTAINER entry Signed-off-by: Joe Perches diff --git a/MAINTAINERS b/MAINTAINERS index 629b64c..4157218 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2516,6 +2516,7 @@ M: infinipath at qlogic.com L: general at lists.openfabrics.org T: git git://git.qlogic.com/ipath-linux-2.6 S: Supported +F: drivers/infiniband/hw/ipath/ IPMI SUBSYSTEM P: Corey Minyard From jackm at dev.mellanox.co.il Sun Aug 12 23:34:07 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Mon, 13 Aug 2007 09:34:07 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <20070812164949.GD27355@mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> <20070812164949.GD27355@mellanox.co.il> Message-ID: <200708130934.08215.jackm@dev.mellanox.co.il> On Sunday 12 August 2007 19:49, Michael S. Tsirkin wrote: > > Why not add xrc_domain to the driver-data area? Because the xrc_domain is a ib verbs-layer construct. > > It doesn't follow, necessarily. > Will we be able to void breaking kernel-user ABI if we stick domain > handle in driver-specific area? Are you suggesting the following: 1. Do NOT increment the kernel-user ABI 2. Put all src-related changes into the driver-specific area 3. Rely the fact that if userspace is using driver libraries which do not support SRC, the src-related functions will not be present, and libibverbs will reject the src-related function calls. NOTE: This may be the case for SRC function calls. However, there is no check on qp-type in userspace during ibv_create_qp. Its possible for the user to indicate IBV_QPT_SRC, have it go all the way to kernel-space -- and kernel space will take the (garbage) value for the src-domain number. I think we will find other such holes if we don't increment the kernel-user ABI version. - Jack From vlad at lists.openfabrics.org Mon Aug 13 01:39:51 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 13 Aug 2007 01:39:51 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070813-0100 daily build status Message-ID: <20070813083951.1F81AE6083B@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.18 Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From HNGUYEN at de.ibm.com Mon Aug 13 01:42:30 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Mon, 13 Aug 2007 10:42:30 +0200 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070811181330.GC17843@mellanox.co.il> Message-ID: > > Could you please provide Michael and Vladimir with a URL to download > > above kernel source tree in order to perform daily build of ofed > > code suite? This will help us to prevent build issues in very early > > stage. > > Thanks much in advance! > > Nam, > why can't you do this yourself? I may not upload code, for which I'm not a distributor, to the (public) ofa server. Please understand that. > I think that IBM's the one that needs this most of all. I think it's ofed group who should ask redhat and suse for help in this matter, because in the long run, this is about ofed development and build. Tziporet, please add this topic to today's ofed call. Nam From vlad at lists.openfabrics.org Mon Aug 13 02:50:39 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 13 Aug 2007 02:50:39 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070813-0200 daily build status Message-ID: <20070813095039.4FDEDE603C1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070813-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From hal.rosenstock at gmail.com Mon Aug 13 05:00:28 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 08:00:28 -0400 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <20070812163623.GB27355@mellanox.co.il> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <20070811182120.GE17843@mellanox.co.il> <20070812163623.GB27355@mellanox.co.il> Message-ID: On 8/12/07, Michael S. Tsirkin wrote: > > > > Shouldn't SRP targets be per port too ? > > Since ib_cm_id object is per-device, I don't think so. > Why do you ask? Aren't SRP targets per port rather than per device ? Is port number of the device an additional part of the target discovery so it is handled properly ? > -- > MST > From mst at dev.mellanox.co.il Mon Aug 13 06:07:05 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 13 Aug 2007 16:07:05 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <200708130934.08215.jackm@dev.mellanox.co.il> References: <20070808071910.GC23514@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> <20070812164949.GD27355@mellanox.co.il> <200708130934.08215.jackm@dev.mellanox.co.il> Message-ID: <20070813130705.GA21592@mellanox.co.il> > Quoting Jack Morgenstein : > Subject: Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation > > On Sunday 12 August 2007 19:49, Michael S. Tsirkin wrote: > > > Why not add xrc_domain to the driver-data area? Because the xrc_domain is a ib verbs-layer construct. > > > > It doesn't follow, necessarily. > > Will we be able to void breaking kernel-user ABI if we stick domain > > handle in driver-specific area? > > Are you suggesting the following: > > 1. Do NOT increment the kernel-user ABI > 2. Put all src-related changes into the driver-specific area I was just generally saying that if we can avoid breaking the ABI, we should. What you describe seems OK to me. > 3. Rely the fact that if userspace is using driver libraries which do not support SRC, > the src-related functions will not be present, and libibverbs will reject the src-related > function calls. You mean, for devices that do not support SRC? No, I think it's kernel's job to validate this case. > NOTE: This may be the case for SRC function calls. However, there is no > check on qp-type in userspace during ibv_create_qp. Its possible for the > user to indicate IBV_QPT_SRC, have it go all the way to kernel-space -- and > kernel space will take the (garbage) value for the src-domain number. > > I think we will find other such holes if we don't increment the kernel-user > ABI version. So, kernel has to validate the SRC domain handle. This does not look like an issue to me, at all. -- MST From mst at dev.mellanox.co.il Mon Aug 13 06:08:23 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 13 Aug 2007 16:08:23 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: References: <20070811181330.GC17843@mellanox.co.il> Message-ID: <20070813130823.GB21592@mellanox.co.il> > Quoting Hoang-Nam Nguyen : > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > Could you please provide Michael and Vladimir with a URL to download > > > above kernel source tree in order to perform daily build of ofed > > > code suite? This will help us to prevent build issues in very early > > > stage. > > > Thanks much in advance! > > > > Nam, > > why can't you do this yourself? > > I may not upload code, for which I'm not a distributor, to the > (public) ofa server. Please understand that. Oh? GPL clearly gives you the right to distribute kernel code. > > I think that IBM's the one that needs this most of all. > > I think it's ofed group who should ask redhat and suse for help in this > matter, because in the long run, this is about ofed development and build. > Tziporet, please add this topic to today's ofed call. -- MST From hal.rosenstock at gmail.com Mon Aug 13 06:19:07 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 09:19:07 -0400 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: <11869538263299-git-send-email-sashak@voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> Message-ID: On 8/12/07, Sasha Khapyorsky wrote: > Anyway OpenSM will request resweep when there are suspected nodes > with duplicate GUID on the subnet. And because we cannot be 100% sure > that detected GUIDs duplication is not some corner case of port moving > I prefer to not exit. Endless (re)discovery and syslog messages should > be good indication if it is indeed this case. Couldn't there be some duplication state kept per GUID so the messages only get logged on change of state to duplicated rather than continually spewing into the log ? > > Signed-off-by: Sasha Khapyorsky > --- > opensm/opensm/osm_node_info_rcv.c | 6 ------ > 1 files changed, 0 insertions(+), 6 deletions(-) > > diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c > index 715c0f4..ae95964 100644 > --- a/opensm/opensm/osm_node_info_rcv.c > +++ b/opensm/opensm/osm_node_info_rcv.c > @@ -129,12 +129,6 @@ report_duplicated_guid( > > osm_log( p_rcv->p_log, OSM_LOG_SYS, > "FATAL: duplicated guids or 12x lane reversal\n"); > - > - if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) > - { > - osm_log( p_rcv->p_log, OSM_LOG_SYS, "Exiting\n"); > - exit( 1 ); > - } > } > > static void requery_dup_node_info( > -- > 1.5.3.rc2.38.g11308 > > From mxpcxoqtzbh at direcpc.com Sun Aug 12 18:24:46 2007 From: mxpcxoqtzbh at direcpc.com (Lhomme fait) Date: Mon, 13 Aug 2007 07:24:46 +0600 Subject: [ofa-general] Re: Message-ID: Gary Paulsen Hershey MoreLast Section: Spring CrimeThe Tin Blowdown JAMES LEE click Remove offers Third treaties Worlds Guitar Added:Aug RELATED CONTENT Memory advances coming benefit response PRICES Simple Drag Setups Matrix Industry Highest making readable bare boards Rough Metal Castings Router Netsim CCNABoson simulated Bosons newest simulator CCNP saves tutorData Entry Typing -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bgimg.gif Type: image/gif Size: 4883 bytes Desc: not available URL: From kweku.davey at bsalg.dk Mon Aug 13 09:20:33 2007 From: kweku.davey at bsalg.dk (Lamont Shoemaker) Date: Mon, 13 Aug 2007 14:20:33 -0200 Subject: [ofa-general] Always ready Message-ID: <01c7ddb5$1c34f440$a975cf52@kweku.davey> -------------- next part -------------- A non-text attachment was scrubbed... Name: kvakva.gif Type: image/gif Size: 14167 bytes Desc: not available URL: From hal.rosenstock at gmail.com Mon Aug 13 07:24:31 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 10:24:31 -0400 Subject: [ofa-general] Re: [PATCH][TIRIVIAL] ibdm/src/osm_check.cpp: Add missing include file In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901FF1344@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901FF1344@mtlexch01.mtl.com> Message-ID: On 7/31/07, Eitan Zahavi wrote: > Applied. Thanks Hal. Looks like the same thing is needed for OFED 1.2/1.2.c > Eitan Zahavi > > -----Original Message----- > > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com] > > Sent: Monday, July 30, 2007 8:54 PM > > To: Eitan Zahavi > > Cc: OpenFabrics General > > Subject: [PATCH][TIRIVIAL] ibdm/src/osm_check.cpp: Add > > missing include file > > > > ibdm/src/osm_check.cpp: Add missing include file > > > > Signed-off-by: Hal Rosenstock > > > > diff --git a/ibdm/src/osm_check.cpp b/ibdm/src/osm_check.cpp > > index 49215c2..f24eec6 100644 > > --- a/ibdm/src/osm_check.cpp > > +++ b/ibdm/src/osm_check.cpp > > @@ -35,6 +35,7 @@ > > #include "Fabric.h" > > #include "SubnMgt.h" > > #include "CredLoops.h" > > +#include > > #include > > #include > > > From sashak at voltaire.com Mon Aug 13 07:32:06 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 13 Aug 2007 17:32:06 +0300 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> Message-ID: <20070813143206.GA461@sashak.voltaire.com> Hi Hal, On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > On 8/12/07, Sasha Khapyorsky wrote: > > Anyway OpenSM will request resweep when there are suspected nodes > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > that detected GUIDs duplication is not some corner case of port moving > > I prefer to not exit. Endless (re)discovery and syslog messages should > > be good indication if it is indeed this case. > > Couldn't there be some duplication state kept per GUID so the messages > only get logged on change of state to duplicated rather than > continually spewing into the log ? There should be one message per duplicated GUID in the sweep. The sweep will be repeated and in the case of real duplication the message will appear again - so it is per sweep. I hope it is not too much. Sasha From hal.rosenstock at gmail.com Mon Aug 13 07:50:32 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 10:50:32 -0400 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: <20070813143206.GA461@sashak.voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> <20070813143206.GA461@sashak.voltaire.com> Message-ID: On 8/13/07, Sasha Khapyorsky wrote: > Hi Hal, > > On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > > On 8/12/07, Sasha Khapyorsky wrote: > > > Anyway OpenSM will request resweep when there are suspected nodes > > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > > that detected GUIDs duplication is not some corner case of port moving > > > I prefer to not exit. Endless (re)discovery and syslog messages should > > > be good indication if it is indeed this case. > > > > Couldn't there be some duplication state kept per GUID so the messages > > only get logged on change of state to duplicated rather than > > continually spewing into the log ? > > There should be one message per duplicated GUID in the sweep. The sweep > will be repeated and in the case of real duplication the message will > appear again - so it is per sweep. I hope it is not too much. Once per sweep is too much IMO. It still fills the log over time. > Sasha > From eli at mellanox.co.il Mon Aug 13 07:57:03 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Mon, 13 Aug 2007 17:57:03 +0300 Subject: [ofa-general] [PATCH] mlx4_core - delay after reset Message-ID: <1187017023.16253.31.camel@mtls03> Put a 1000 msec delay after resetting the device before attempting to do config cycles on it. Not doing this causes system hangs on some chipsets, e.g. Intel Corporation E7520, when the driver is loaded. Signed-off-by: Eli Cohen --- ofa_1_3_dev_kernel.orig/drivers/net/mlx4/reset.c 2007-08-13 16:25:31.000000000 +0300 +++ ofa_1_3_dev_kernel/drivers/net/mlx4/reset.c 2007-08-13 17:54:24.000000000 +0300 @@ -119,6 +119,7 @@ writel(MLX4_RESET_VALUE, reset + MLX4_RESET_OFFSET); iounmap(reset); + msleep(1000); end = jiffies + MLX4_RESET_TIMEOUT_JIFFIES; do { if (!pci_read_config_word(dev->pdev, PCI_VENDOR_ID, &vendor) && From rdreier at cisco.com Mon Aug 13 08:49:07 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 08:49:07 -0700 Subject: [ofa-general] [PATCH] mlx4_core - delay after reset In-Reply-To: <1187017023.16253.31.camel@mtls03> (Eli Cohen's message of "Mon, 13 Aug 2007 17:57:03 +0300") References: <1187017023.16253.31.camel@mtls03> Message-ID: thanks, applied (by hand, since you seem to have corrupted the patch header, since there's no "--- " line) From rdreier at cisco.com Mon Aug 13 08:49:44 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 08:49:44 -0700 Subject: [ofa-general] Re: [stable] [PATCH -stable] IB/core: Ignore membership bit in ib_find_pkey() In-Reply-To: <20070810060359.GA23664@kroah.com> (Greg KH's message of "Thu, 9 Aug 2007 23:03:59 -0700") References: <46B9ACF7.6030907@gmail.com> <20070810060359.GA23664@kroah.com> Message-ID: > I don't see this in Linus's tree, is there some reason it has not been > accepted into mainline yet? It's in my tree, but Linus dropped the pull request. I'll resend to Linus. From rdreier at cisco.com Mon Aug 13 08:52:05 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 08:52:05 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> (Sean Hefty's message of "Fri, 10 Aug 2007 17:22:29 -0700") References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> Message-ID: > >This concept is supported by the spec through SA redirection; however, I > >propose that we also allow the SA LID to be set manually by an administrator. > Is the following an acceptable approach for this? > > Add a class device file to the ib_sa that allows setting all SA redirection > parameters (SL, LID, PKey, QP - with processing similar to SRP's add_target). > > /sys/class/infiniband_sa/sa-mthca-0/redirect_sa My first reaction to this is to wonder why we wouldn't just use redirection as already specified in the IB architecture? It seems that having something that has to be set manually and that breaks all communication if it is set wrong is a bad idea. - R. From rdreier at cisco.com Mon Aug 13 08:55:13 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 08:55:13 -0700 Subject: [ofa-general] Re: [PATCH] libibverbs: Initialize reserved attributes in modify QP command In-Reply-To: <200708081627.08791.dotanb@dev.mellanox.co.il> (Dotan Barak's message of "Wed, 8 Aug 2007 16:27:08 +0300") References: <200708081627.08791.dotanb@dev.mellanox.co.il> Message-ID: thanks, applied. From rdreier at cisco.com Mon Aug 13 08:58:11 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 08:58:11 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git for-linus branch Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get some small fixes for 2.6.23: Dotan Barak (3): IB: Include from IB: Include and from IB: Move the macro IB_UMEM_MAX_PAGE_CHUNK() to umem.c Eli Cohen (1): mlx4_core: Wait 1 second after reset before accessing device Hal Rosenstock (3): IB/mad: Fix error path if response alloc fails in ib_mad_recv_done_handler() IB/mad: Fix memory leak in switch handling in ib_mad_recv_done_handler() IB/mad: agent_send_response() should be void Jack Morgenstein (1): IPoIB: Fix leak in ipoib_transport_dev_init() error path Moni Shoua (1): IB/core: Ignore membership bit in ib_find_pkey() Raghava Kondapalli (1): IB/srp: Add OUI for new Cisco targets Roland Dreier (2): IB/sa: Don't need to check for default P_Key twice IB/srp: Wrap OUI checking for workarounds in helper functions Sean Hefty (1): IB/mad: Fix address handle leak in mad_rmpp Steve Wise (1): RDMA/cxgb3: Always call low level send function via cxgb3_ofld_send() Vu Pham (1): IB/mlx4: Fix opcode returned in RDMA read completion drivers/infiniband/core/agent.c | 24 +++++++++------------ drivers/infiniband/core/agent.h | 6 ++-- drivers/infiniband/core/device.c | 2 +- drivers/infiniband/core/mad.c | 25 +++++++++++---------- drivers/infiniband/core/mad_rmpp.c | 8 +++--- drivers/infiniband/core/sa_query.c | 4 +-- drivers/infiniband/core/umem.c | 5 ++++ drivers/infiniband/hw/cxgb3/iwch_cm.c | 16 +++++++------- drivers/infiniband/hw/mlx4/cq.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 1 + drivers/infiniband/ulp/srp/ib_srp.c | 31 +++++++++++++++++++-------- drivers/net/mlx4/reset.c | 3 ++ include/rdma/ib_mad.h | 2 + include/rdma/ib_verbs.h | 7 +---- 14 files changed, 76 insertions(+), 60 deletions(-) diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c index db2633e..ae7c288 100644 --- a/drivers/infiniband/core/agent.c +++ b/drivers/infiniband/core/agent.c @@ -78,15 +78,14 @@ ib_get_agent_port(struct ib_device *device, int port_num) return entry; } -int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, - struct ib_wc *wc, struct ib_device *device, - int port_num, int qpn) +void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, + struct ib_wc *wc, struct ib_device *device, + int port_num, int qpn) { struct ib_agent_port_private *port_priv; struct ib_mad_agent *agent; struct ib_mad_send_buf *send_buf; struct ib_ah *ah; - int ret; struct ib_mad_send_wr_private *mad_send_wr; if (device->node_type == RDMA_NODE_IB_SWITCH) @@ -96,23 +95,21 @@ int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, if (!port_priv) { printk(KERN_ERR SPFX "Unable to find port agent\n"); - return -ENODEV; + return; } agent = port_priv->agent[qpn]; ah = ib_create_ah_from_wc(agent->qp->pd, wc, grh, port_num); if (IS_ERR(ah)) { - ret = PTR_ERR(ah); - printk(KERN_ERR SPFX "ib_create_ah_from_wc error:%d\n", ret); - return ret; + printk(KERN_ERR SPFX "ib_create_ah_from_wc error\n"); + return; } send_buf = ib_create_send_mad(agent, wc->src_qp, wc->pkey_index, 0, IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA, GFP_KERNEL); if (IS_ERR(send_buf)) { - ret = PTR_ERR(send_buf); - printk(KERN_ERR SPFX "ib_create_send_mad error:%d\n", ret); + printk(KERN_ERR SPFX "ib_create_send_mad error\n"); goto err1; } @@ -126,16 +123,15 @@ int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, mad_send_wr->send_wr.wr.ud.port_num = port_num; } - if ((ret = ib_post_send_mad(send_buf, NULL))) { - printk(KERN_ERR SPFX "ib_post_send_mad error:%d\n", ret); + if (ib_post_send_mad(send_buf, NULL)) { + printk(KERN_ERR SPFX "ib_post_send_mad error\n"); goto err2; } - return 0; + return; err2: ib_free_send_mad(send_buf); err1: ib_destroy_ah(ah); - return ret; } static void agent_send_handler(struct ib_mad_agent *mad_agent, diff --git a/drivers/infiniband/core/agent.h b/drivers/infiniband/core/agent.h index 86d72fa..fb9ed14 100644 --- a/drivers/infiniband/core/agent.h +++ b/drivers/infiniband/core/agent.h @@ -46,8 +46,8 @@ extern int ib_agent_port_open(struct ib_device *device, int port_num); extern int ib_agent_port_close(struct ib_device *device, int port_num); -extern int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, - struct ib_wc *wc, struct ib_device *device, - int port_num, int qpn); +extern void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, + struct ib_wc *wc, struct ib_device *device, + int port_num, int qpn); #endif /* __AGENT_H_ */ diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 3ada17c..2506c43 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -702,7 +702,7 @@ int ib_find_pkey(struct ib_device *device, if (ret) return ret; - if (pkey == tmp_pkey) { + if ((pkey & 0x7fff) == (tmp_pkey & 0x7fff)) { *index = i; return 0; } diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index bc547f1..6f42877 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -1842,16 +1842,11 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, { struct ib_mad_qp_info *qp_info; struct ib_mad_private_header *mad_priv_hdr; - struct ib_mad_private *recv, *response; + struct ib_mad_private *recv, *response = NULL; struct ib_mad_list_head *mad_list; struct ib_mad_agent_private *mad_agent; int port_num; - response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL); - if (!response) - printk(KERN_ERR PFX "ib_mad_recv_done_handler no memory " - "for response buffer\n"); - mad_list = (struct ib_mad_list_head *)(unsigned long)wc->wr_id; qp_info = mad_list->mad_queue->qp_info; dequeue_mad(mad_list); @@ -1879,6 +1874,13 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, if (!validate_mad(&recv->mad.mad, qp_info->qp->qp_num)) goto out; + response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL); + if (!response) { + printk(KERN_ERR PFX "ib_mad_recv_done_handler no memory " + "for response buffer\n"); + goto out; + } + if (port_priv->device->node_type == RDMA_NODE_IB_SWITCH) port_num = wc->port_num; else @@ -1914,12 +1916,11 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, response->header.recv_wc.recv_buf.mad = &response->mad.mad; response->header.recv_wc.recv_buf.grh = &response->grh; - if (!agent_send_response(&response->mad.mad, - &response->grh, wc, - port_priv->device, - smi_get_fwd_port(&recv->mad.smp), - qp_info->qp->qp_num)) - response = NULL; + agent_send_response(&response->mad.mad, + &response->grh, wc, + port_priv->device, + smi_get_fwd_port(&recv->mad.smp), + qp_info->qp->qp_num); goto out; } diff --git a/drivers/infiniband/core/mad_rmpp.c b/drivers/infiniband/core/mad_rmpp.c index 3663fd7..d43bc62 100644 --- a/drivers/infiniband/core/mad_rmpp.c +++ b/drivers/infiniband/core/mad_rmpp.c @@ -163,8 +163,10 @@ static struct ib_mad_send_buf *alloc_response_msg(struct ib_mad_agent *agent, hdr_len, 0, GFP_KERNEL); if (IS_ERR(msg)) ib_destroy_ah(ah); - else + else { msg->ah = ah; + msg->context[0] = ah; + } return msg; } @@ -197,9 +199,7 @@ static void ack_ds_ack(struct ib_mad_agent_private *agent, void ib_rmpp_send_handler(struct ib_mad_send_wc *mad_send_wc) { - struct ib_rmpp_mad *rmpp_mad = mad_send_wc->send_buf->mad; - - if (rmpp_mad->rmpp_hdr.rmpp_type != IB_MGMT_RMPP_TYPE_ACK) + if (mad_send_wc->send_buf->context[0] == mad_send_wc->send_buf->ah) ib_destroy_ah(mad_send_wc->send_buf->ah); ib_free_send_mad(mad_send_wc->send_buf); } diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index 20ab6b3..d271bd7 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -385,9 +385,7 @@ static void update_sm_ah(struct work_struct *work) new_ah->pkey_index = 0; if (ib_find_pkey(port->agent->device, port->port_num, - IB_DEFAULT_PKEY_FULL, &new_ah->pkey_index) && - ib_find_pkey(port->agent->device, port->port_num, - IB_DEFAULT_PKEY_PARTIAL, &new_ah->pkey_index)) + IB_DEFAULT_PKEY_FULL, &new_ah->pkey_index)) printk(KERN_ERR "Couldn't find index for default PKey\n"); memset(&ah_attr, 0, sizeof ah_attr); diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 26d0470..664d2fa 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -40,6 +40,11 @@ #include "uverbs.h" +#define IB_UMEM_MAX_PAGE_CHUNK \ + ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ + ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ + (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) + static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty) { struct ib_umem_chunk *chunk, *tmp; diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index 9574088..1cdfcd4 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -139,7 +139,7 @@ static void release_tid(struct t3cdev *tdev, u32 hwtid, struct sk_buff *skb) req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_FORWARD)); OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_TID_RELEASE, hwtid)); skb->priority = CPL_PRIORITY_SETUP; - tdev->send(tdev, skb); + cxgb3_ofld_send(tdev, skb); return; } @@ -161,7 +161,7 @@ int iwch_quiesce_tid(struct iwch_ep *ep) req->val = cpu_to_be64(1 << S_TCB_RX_QUIESCE); skb->priority = CPL_PRIORITY_DATA; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -183,7 +183,7 @@ int iwch_resume_tid(struct iwch_ep *ep) req->val = 0; skb->priority = CPL_PRIORITY_DATA; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -784,7 +784,7 @@ static int update_rx_credits(struct iwch_ep *ep, u32 credits) OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_RX_DATA_ACK, ep->hwtid)); req->credit_dack = htonl(V_RX_CREDITS(credits) | V_RX_FORCE_ACK(1)); skb->priority = CPL_PRIORITY_ACK; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return credits; } @@ -1152,7 +1152,7 @@ static int listen_start(struct iwch_listen_ep *ep) req->opt1 = htonl(V_CONN_POLICY(CPL_CONN_POLICY_ASK)); skb->priority = 1; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -1186,7 +1186,7 @@ static int listen_stop(struct iwch_listen_ep *ep) req->cpu_idx = 0; OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_CLOSE_LISTSRV_REQ, ep->stid)); skb->priority = 1; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -1264,7 +1264,7 @@ static void reject_cr(struct t3cdev *tdev, u32 hwtid, __be32 peer_ip, rpl->opt0l_status = htonl(CPL_PASS_OPEN_REJECT); rpl->opt2 = 0; rpl->rsvd = rpl->opt2; - tdev->send(tdev, skb); + cxgb3_ofld_send(tdev, skb); } } @@ -1557,7 +1557,7 @@ static int peer_abort(struct t3cdev *tdev, struct sk_buff *skb, void *ctx) rpl->wr.wr_lo = htonl(V_WR_TID(ep->hwtid)); OPCODE_TID(rpl) = htonl(MK_OPCODE_TID(CPL_ABORT_RPL, ep->hwtid)); rpl->cmd = CPL_ABORT_NO_RST; - ep->com.tdev->send(ep->com.tdev, rpl_skb); + cxgb3_ofld_send(ep->com.tdev, rpl_skb); if (state != ABORTING) { state_set(&ep->com, DEAD); release_ep_resources(ep); diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 660b27a..8bf44da 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -389,7 +389,7 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, wc->opcode = IB_WC_SEND; break; case MLX4_OPCODE_RDMA_READ: - wc->opcode = IB_WC_SEND; + wc->opcode = IB_WC_RDMA_READ; wc->byte_len = be32_to_cpu(cqe->byte_cnt); break; case MLX4_OPCODE_ATOMIC_CS: diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 982eb88..563aeac 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -211,6 +211,7 @@ out_free_cq: out_free_mr: ib_dereg_mr(priv->mr); + ipoib_cm_dev_cleanup(dev); out_free_pd: ib_dealloc_pd(priv->pd); diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index f01ca18..f6a0514 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -75,16 +75,12 @@ module_param(topspin_workarounds, int, 0444); MODULE_PARM_DESC(topspin_workarounds, "Enable workarounds for Topspin/Cisco SRP target bugs if != 0"); -static const u8 topspin_oui[3] = { 0x00, 0x05, 0xad }; - static int mellanox_workarounds = 1; module_param(mellanox_workarounds, int, 0444); MODULE_PARM_DESC(mellanox_workarounds, "Enable workarounds for Mellanox SRP target bugs if != 0"); -static const u8 mellanox_oui[3] = { 0x00, 0x02, 0xc9 }; - static void srp_add_one(struct ib_device *device); static void srp_remove_one(struct ib_device *device); static void srp_completion(struct ib_cq *cq, void *target_ptr); @@ -108,6 +104,24 @@ static const char *srp_target_info(struct Scsi_Host *host) return host_to_target(host)->target_name; } +static int srp_target_is_topspin(struct srp_target_port *target) +{ + static const u8 topspin_oui[3] = { 0x00, 0x05, 0xad }; + static const u8 cisco_oui[3] = { 0x00, 0x1b, 0x0d }; + + return topspin_workarounds && + (!memcmp(&target->ioc_guid, topspin_oui, sizeof topspin_oui) || + !memcmp(&target->ioc_guid, cisco_oui, sizeof cisco_oui)); +} + +static int srp_target_is_mellanox(struct srp_target_port *target) +{ + static const u8 mellanox_oui[3] = { 0x00, 0x02, 0xc9 }; + + return mellanox_workarounds && + !memcmp(&target->ioc_guid, mellanox_oui, sizeof mellanox_oui); +} + static struct srp_iu *srp_alloc_iu(struct srp_host *host, size_t size, gfp_t gfp_mask, enum dma_data_direction direction) @@ -360,7 +374,7 @@ static int srp_send_req(struct srp_target_port *target) * zero out the first 8 bytes of our initiator port ID and set * the second 8 bytes to the local node GUID. */ - if (topspin_workarounds && !memcmp(&target->ioc_guid, topspin_oui, 3)) { + if (srp_target_is_topspin(target)) { printk(KERN_DEBUG PFX "Topspin/Cisco initiator port ID workaround " "activated for target GUID %016llx\n", (unsigned long long) be64_to_cpu(target->ioc_guid)); @@ -585,8 +599,8 @@ static int srp_map_fmr(struct srp_target_port *target, struct scatterlist *scat, if (!dev->fmr_pool) return -ENODEV; - if ((ib_sg_dma_address(ibdev, &scat[0]) & ~dev->fmr_page_mask) && - mellanox_workarounds && !memcmp(&target->ioc_guid, mellanox_oui, 3)) + if (srp_target_is_mellanox(target) && + (ib_sg_dma_address(ibdev, &scat[0]) & ~dev->fmr_page_mask)) return -EINVAL; len = page_cnt = 0; @@ -1087,8 +1101,7 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id, break; case IB_CM_REJ_PORT_REDIRECT: - if (topspin_workarounds && - !memcmp(&target->ioc_guid, topspin_oui, 3)) { + if (srp_target_is_topspin(target)) { /* * Topspin/Cisco SRP gateways incorrectly send * reject reason code 25 when they mean 24 diff --git a/drivers/net/mlx4/reset.c b/drivers/net/mlx4/reset.c index e4dfd4b..e199715 100644 --- a/drivers/net/mlx4/reset.c +++ b/drivers/net/mlx4/reset.c @@ -119,6 +119,9 @@ int mlx4_reset(struct mlx4_dev *dev) writel(MLX4_RESET_VALUE, reset + MLX4_RESET_OFFSET); iounmap(reset); + /* Docs say to wait one second before accessing device */ + msleep(1000); + end = jiffies + MLX4_RESET_TIMEOUT_JIFFIES; do { if (!pci_read_config_word(dev->pdev, PCI_VENDOR_ID, &vendor) && diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h index 30712dd..8ec3799 100644 --- a/include/rdma/ib_mad.h +++ b/include/rdma/ib_mad.h @@ -39,6 +39,8 @@ #if !defined( IB_MAD_H ) #define IB_MAD_H +#include + #include /* Management base version */ diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 0627a6a..4bea182 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -46,6 +46,8 @@ #include #include #include +#include +#include #include #include @@ -731,11 +733,6 @@ struct ib_udata { size_t outlen; }; -#define IB_UMEM_MAX_PAGE_CHUNK \ - ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ - ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ - (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) - struct ib_pd { struct ib_device *device; struct ib_uobject *uobject; From mshefty at ichips.intel.com Mon Aug 13 10:06:50 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:06:50 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> Message-ID: <46C08FAA.3070702@ichips.intel.com> > My first reaction to this is to wonder why we wouldn't just use > redirection as already specified in the IB architecture? It seems > that having something that has to be set manually and that breaks all > communication if it is set wrong is a bad idea. This requires changes to the SAs to send the redirect message in response to a query initiated by the host. I was trying to avoid creating an actual distributed SA. - Sean From mshefty at ichips.intel.com Mon Aug 13 10:10:01 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:10:01 -0700 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <20070811182120.GE17843@mellanox.co.il> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <20070811182120.GE17843@mellanox.co.il> Message-ID: <46C09069.8030006@ichips.intel.com> > Hopefully, that would be something like sa/mthca0 so that > one does not need to perform tricky string parsing to find the interface > for a specific device. > > Further, won't you want to have this per-port? > It's easy to imagine separate ports redirecting to different SA's. Yes - this should be per port. I was looking more just for concept approval. I would also like to export multicast and SA event registration to user space, but this could be done separately. - Sean From rdreier at cisco.com Mon Aug 13 10:13:52 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 10:13:52 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <46C08FAA.3070702@ichips.intel.com> (Sean Hefty's message of "Mon, 13 Aug 2007 10:06:50 -0700") References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> Message-ID: > > My first reaction to this is to wonder why we wouldn't just use > > redirection as already specified in the IB architecture? It seems > > that having something that has to be set manually and that breaks all > > communication if it is set wrong is a bad idea. > > This requires changes to the SAs to send the redirect message in > response to a query initiated by the host. I was trying to avoid > creating an actual distributed SA. Don't you need a distributed SA for your idea to work? Otherwise what happens if node A sets a service record at SA LID X, and then node B sends a query for that service record at SA LID Y? - R. From mshefty at ichips.intel.com Mon Aug 13 10:15:14 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:15:14 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> Message-ID: <46C091A2.5080205@ichips.intel.com> > There are implications on the "deployment" model for this based on the PKey. The intent is to match ClassPortInfo fields. - Sean From mshefty at ichips.intel.com Mon Aug 13 10:19:31 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:19:31 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> Message-ID: <46C092A3.4060406@ichips.intel.com> > In that mode, I suppose it also requires an admin to reset it when the > node for the ib_sa module fails. Not necessarily. See C13-43.1.1: When a request for a particular MADHeader:MgmtClass has been redirected to another location, that location shall continue to service requests for the MADHeader:MgmtClass until either the location becomes inoperable for some reason or the requests are redirected again away from that location. Separately from admin or SA controlled redirection, the ib_sa module will need to determine when to fail back to the master SM/SA. I would do this based on X number of retries/timeouts. - Sean From rdreier at cisco.com Mon Aug 13 10:20:04 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 10:20:04 -0700 Subject: [ofa-general] Re: [PATCH] IB/ipoib: drain cq in dev_stop In-Reply-To: (Roland Dreier's message of "Thu, 24 May 2007 11:41:27 -0700") References: <20070524153246.GD7940@mellanox.co.il> <20070524155008.GB23535@mellanox.co.il> Message-ID: FWIW, I just added this to my for-2.6.24 branch: commit c529c1d566008981622518405d2b60026f3a7b7e Author: Roland Dreier Date: Mon Aug 13 10:19:36 2007 -0700 IPoIB: Make sure no receives are handled when stopping device The current IPoIB code might process receive completions from ipoib_drain_cq() when bringing down the interface. This could cause packets to be passed up the stack without the device's poll method being called. Avoid this by setting the status of any successful completions to IB_WC_WR_FLUSH_ERR. Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 1094488..5a70e28 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -558,6 +558,14 @@ void ipoib_drain_cq(struct net_device *dev) do { n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { + /* + * Convert any successful completions to flush + * errors to avoid passing packets up the + * stack after bringing the device down. + */ + if (priv->ibwc[i].status == IB_WC_SUCCESS) + priv->ibwc[i].status = IB_WC_WR_FLUSH_ERR; + if (priv->ibwc[i].wr_id & IPOIB_CM_OP_SRQ) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) From mst at dev.mellanox.co.il Mon Aug 13 10:36:38 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 13 Aug 2007 20:36:38 +0300 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <46C08FAA.3070702@ichips.intel.com> References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> Message-ID: <20070813173638.GB9924@mellanox.co.il> > Quoting Sean Hefty : > Subject: Re: [RFC] the never ending search for SA scalability > > >My first reaction to this is to wonder why we wouldn't just use > >redirection as already specified in the IB architecture? It seems > >that having something that has to be set manually and that breaks all > >communication if it is set wrong is a bad idea. > > This requires changes to the SAs to send the redirect message in > response to a query initiated by the host. I was trying to avoid > creating an actual distributed SA. I think my first reaction, too, is that I agree with Roland here. If all you want is to answer a query with the redirect to the source LID, it's probably quite easy to extend opensm in this way. -- MST From mshefty at ichips.intel.com Mon Aug 13 10:42:20 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:42:20 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> Message-ID: <46C097FC.7080503@ichips.intel.com> > Don't you need a distributed SA for your idea to work? Otherwise what > happens if node A sets a service record at SA LID X, and then node B > sends a query for that service record at SA LID Y? My initial hope is that we can get away with replicating the data, and using the existing SA protocol to keep it in sync. For the example you gave, SA LID X could forward the set to the master SA before recording it locally. SA LID Y would check with the master SA for the data if it weren't found local. - Sean From hal.rosenstock at gmail.com Mon Aug 13 10:44:33 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 13:44:33 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <46C092A3.4060406@ichips.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <46C092A3.4060406@ichips.intel.com> Message-ID: On 8/13/07, Sean Hefty wrote: > > In that mode, I suppose it also requires an admin to reset it when the > > node for the ib_sa module fails. > > Not necessarily. See C13-43.1.1: > > When a request for a particular MADHeader:MgmtClass has been redirected > to another location, that location shall continue to service requests > for the MADHeader:MgmtClass until either the location becomes inoperable > for some reason or the requests are redirected again away from that > location. > > Separately from admin or SA controlled redirection, the ib_sa module > will need to determine when to fail back to the master SM/SA. I would > do this based on X number of retries/timeouts. Yes, that was what I was getting at. > > - Sean > From hal.rosenstock at gmail.com Mon Aug 13 10:47:41 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 13:47:41 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <46C091A2.5080205@ichips.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C091A2.5080205@ichips.intel.com> Message-ID: On 8/13/07, Sean Hefty wrote: > > There are implications on the "deployment" model for this based on the PKey. > > The intent is to match ClassPortInfo fields. In terms of the redirection fields (for the local subnet), right ? I was referring to the implications on the partition configuration in terms of the ports for the ib_sa module. > - Sean > From mshefty at ichips.intel.com Mon Aug 13 10:53:28 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 10:53:28 -0700 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <20070813173638.GB9924@mellanox.co.il> References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> <20070813173638.GB9924@mellanox.co.il> Message-ID: <46C09A98.8050307@ichips.intel.com> > If all you want is to answer a query with the redirect to the source > LID, it's probably quite easy to extend opensm in this way. It's not quite that simple. The actual data eventually needs to come from the master SA, so opensm can't simply redirect every query. I believe as soon as the master SA becomes involved in the redirection, it would need to run some distributed SA protocol. The master SA would need to know that the requested data can be found at the redirected location. - Sean From hal.rosenstock at gmail.com Mon Aug 13 10:59:32 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 13:59:32 -0400 Subject: [ofa-general] Re: [RFC] the never ending search for SA scalability In-Reply-To: <46C09A98.8050307@ichips.intel.com> References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> <20070813173638.GB9924@mellanox.co.il> <46C09A98.8050307@ichips.intel.com> Message-ID: On 8/13/07, Sean Hefty wrote: > > If all you want is to answer a query with the redirect to the source > > LID, it's probably quite easy to extend opensm in this way. > > It's not quite that simple. The actual data eventually needs to come > from the master SA, so opensm can't simply redirect every query. > > I believe as soon as the master SA becomes involved in the redirection, > it would need to run some distributed SA protocol. The master SA would > need to know that the requested data can be found at the redirected > location. Also, when there are multiple SAs available, it would "need" some basis (e.g. policy) on which to base its redirection ("nearest" "approx equal distributed load" SA). > - Sean > From mshefty at ichips.intel.com Mon Aug 13 11:00:10 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 11:00:10 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C091A2.5080205@ichips.intel.com> Message-ID: <46C09C2A.9010108@ichips.intel.com> > In terms of the redirection fields (for the local subnet), right ? Yes - any field in ClassPortInfo named Redirect*. I don't see that it needs to be limited to the local subnet. > I was referring to the implications on the partition configuration in > terms of the ports for the ib_sa module. I don't know that an implementation needs to handle different PKeys or GRH fields right away, but I don't believe that general SA redirection prohibits this. (I really only care about the LID at the moment.) - Sean From hal.rosenstock at gmail.com Mon Aug 13 11:07:04 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 14:07:04 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <46C09C2A.9010108@ichips.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C091A2.5080205@ichips.intel.com> <46C09C2A.9010108@ichips.intel.com> Message-ID: On 8/13/07, Sean Hefty wrote: > > In terms of the redirection fields (for the local subnet), right ? > > Yes - any field in ClassPortInfo named Redirect*. I don't see that it > needs to be limited to the local subnet. > > > I was referring to the implications on the partition configuration in > > terms of the ports for the ib_sa module. > > I don't know that an implementation needs to handle different PKeys or > GRH fields right away, but I don't believe that general SA redirection > prohibits this. (I really only care about the LID at the moment.) Even in this case, it requires that the port(s) running the ib_sa module are in the full members of the default partition whereas before they might have been limited members. > - Sean > From hal.rosenstock at gmail.com Mon Aug 13 13:55:38 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 13 Aug 2007 16:55:38 -0400 Subject: [ofa-general] [PATCH][TRIVIAL] ibdiag/src/ibdebug_if.tcl: Fix typo Message-ID: ibdiag/src/ibdebug_if.tcl: Fix typo Signed-off-by: Hal Rosenstock diff --git a/ibdiag/src/ibdebug_if.tcl b/ibdiag/src/ibdebug_if.tcl index fa769af..2504874 100644 --- a/ibdiag/src/ibdebug_if.tcl +++ b/ibdiag/src/ibdebug_if.tcl @@ -1726,7 +1726,7 @@ proc inform { msgCode args } { } "-W-ibdiagnet:bad.pm.counter.report" { append msgText "$NODE(0,FullNamePort_Last)%n" - append msgText " Performence Monitor counter" + append msgText " Performance Monitor counter" append msgText "[string repeat " " [expr [GetLengthMaxWord $G(var:list append msgText "Value" From rdreier at cisco.com Mon Aug 13 14:24:09 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 14:24:09 -0700 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <46C097FC.7080503@ichips.intel.com> (Sean Hefty's message of "Mon, 13 Aug 2007 10:42:20 -0700") References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> <46C097FC.7080503@ichips.intel.com> Message-ID: > > Don't you need a distributed SA for your idea to work? Otherwise what > > happens if node A sets a service record at SA LID X, and then node B > > sends a query for that service record at SA LID Y? > > My initial hope is that we can get away with replicating the data, and > using the existing SA protocol to keep it in sync. For the example > you gave, SA LID X could forward the set to the master SA before > recording it locally. SA LID Y would check with the master SA for the > data if it weren't found local. You would need a protocol to invalidate any locally cached service records when a service record is deleted. And similarly for everything else that needs to be kept coherent. That sounds like what I would call a distributed SA. In any case, I don't see what having a mechanism for manual SA redirect buys you. I'm probably missing the point, but obviously you're not planning on having sysadmins manually set the SA LID on each node, which implies having some automatic agent that can set the SA LID. And given the existence of that agent, I don't see the difficulty in putting that agent in the real SA and using the existing SA redirect protocol to handle setting the SA LID. - R. From rdreier at cisco.com Mon Aug 13 14:28:40 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 14:28:40 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46BFF288.5020304@linux.vnet.ibm.com> (Pradeep Satyanarayana's message of "Sun, 12 Aug 2007 22:56:24 -0700") References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> Message-ID: > This is only for completeness sake. Performance will be poor compared to two (or more) > Linux systems using this. This is expected to be really a corner and do not expect this > to happen. Hence is there for completeness only. If I interpreted this correctly, this > is was what Roland suggested (and I agree with the suggestion). No, this isn't really what I suggested. I suggested one receive to start with, and if this receive is consumed, then post a full set of receives to fill the QP receive queue and use it normally. No separate CQ, no kernel log message. This still might run into RNR issues if more than one message is sent right away. I woudn't be that surprised if AIX or Solaris or some other OS chooses to use the same QP for sending and receiving, and I'm not sure we want to be in a situation where that doesn't work well. Although maybe we can defer worrying about this until someone runs into the issue in practice. I guess you should post your latest rolled up patch sowe can see where we stand on this. - R. From rdreier at cisco.com Mon Aug 13 14:37:38 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 14:37:38 -0700 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <200708121536.47152.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Sun, 12 Aug 2007 15:36:46 +0300") References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> Message-ID: > Having looked at implementation issues, it will be much simpler to just, as you say, > mimic the consumer API -- in kernel space as well. Yes, in fact I think we might be able to get away with not breaking the user-kernel ABI. If I understand things correctly, XRC QPs do not have an SRQ attached to them, so we could overload the srq_handle member of struct ib_uverbs_create_qp to hold the xrc_domain_handle when creating an XRC QP. Then if we're OK with having create_xrc_srq be a separate operation from create_srq, I think everything else doesn't break the ABI. Does that seem like a good plan? The only ugly thing is making the interface a little fatter than it needs to be with the duplicated create_xrc_srq and create_srq operations; but we save the pain of bumping the ABI and avoid the ugliness of putting generic objects into the driver-specific data (which would force us to export the lookup of XRC objects from the uverbs module, etc). - R. From dledford at redhat.com Mon Aug 13 14:08:57 2007 From: dledford at redhat.com (Doug Ledford) Date: Mon, 13 Aug 2007 17:08:57 -0400 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070811181330.GC17843@mellanox.co.il> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061530.54392.hnguyen@linux.vnet.ibm.com> <20070811181330.GC17843@mellanox.co.il> Message-ID: <1187039337.14384.11.camel@firewall.xsintricity.com> On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > Quoting Hoang-Nam Nguyen : > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > Hello Doug and Scott! > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > ehca backports for kernel.org kernels seem to be broken. > > > 1. Does anyone care enough to fix them? If not we'll disable > > > ehca in build for these kernels. > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? Don't you guys already have RHEL4U5? It had a backports directory in the OFED 1.2 release...and it's been out for quite a while... > > > This would make it possible for us to add it to nightly builds. > > Could you please provide Michael and Vladimir with a URL to download > > above kernel source tree in order to perform daily build of ofed > > code suite? This will help us to prevent build issues in very early > > stage. > > Thanks much in advance! > > > > Regards > > Nam > > Nam, > why can't you do this yourself? > I think that IBM's the one that needs this most of all. > -- Doug Ledford GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From dledford at redhat.com Mon Aug 13 14:07:41 2007 From: dledford at redhat.com (Doug Ledford) Date: Mon, 13 Aug 2007 17:07:41 -0400 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070813130823.GB21592@mellanox.co.il> References: <20070811181330.GC17843@mellanox.co.il> <20070813130823.GB21592@mellanox.co.il> Message-ID: <1187039261.14384.9.camel@firewall.xsintricity.com> On Mon, 2007-08-13 at 16:08 +0300, Michael S. Tsirkin wrote: > > Quoting Hoang-Nam Nguyen : > > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > Could you please provide Michael and Vladimir with a URL to download > > > > above kernel source tree in order to perform daily build of ofed > > > > code suite? This will help us to prevent build issues in very early > > > > stage. > > > > Thanks much in advance! > > > > > > Nam, > > > why can't you do this yourself? > > > > I may not upload code, for which I'm not a distributor, to the > > (public) ofa server. Please understand that. > > Oh? > GPL clearly gives you the right to distribute kernel code. I think IBM's attorneys have probably banned uploading code to public servers unless you are part of their IT department and uploading an officially blessed product. Technically, if Nam uploaded code to a public server, then IBM can be seen as a distributor of that code, and that's when certain parts of the GPL kick in and obligate the distributor to various things. I'm guessing the lawyers didn't want the company to be accidentally obligated to anything, and the only safe way to avoid that is not to allow developers to upload code. > > > I think that IBM's the one that needs this most of all. > > > > I think it's ofed group who should ask redhat and suse for help in this > > matter, because in the long run, this is about ofed development and build. > > Tziporet, please add this topic to today's ofed call. > > > -- Doug Ledford GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From pradeeps at linux.vnet.ibm.com Mon Aug 13 17:09:11 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Mon, 13 Aug 2007 17:09:11 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> Message-ID: <46C0F2A7.5010807@linux.vnet.ibm.com> Roland Dreier wrote: >> This is only for completeness sake. Performance will be poor compared to two (or more) >> Linux systems using this. This is expected to be really a corner and do not expect this >> to happen. Hence is there for completeness only. If I interpreted this correctly, this >> is was what Roland suggested (and I agree with the suggestion). > > No, this isn't really what I suggested. I suggested one receive to > start with, and if this receive is consumed, then post a full set of > receives to fill the QP receive queue and use it normally. No > separate CQ, no kernel log message. This still might run into RNR > issues if more than one message is sent right away. > > I woudn't be that surprised if AIX or Solaris or some other OS chooses > to use the same QP for sending and receiving, and I'm not sure we want > to be in a situation where that doesn't work well. Although maybe we > can defer worrying about this until someone runs into the issue in > practice. > > I guess you should post your latest rolled up patch sowe can see where > we stand on this. > Without a separate CQ, how do the receive work completion handlers distinguish between the different receive queues of rx_qp and tx_qp -any suggestions? Pradeep From mshefty at ichips.intel.com Mon Aug 13 17:10:05 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 13 Aug 2007 17:10:05 -0700 Subject: [ofa-general] removing PR caching from IPoIB Message-ID: <46C0F2DD.6070109@ichips.intel.com> I'm looking at removing the PR caching in IPoIB. Is there any reason why structs ipoib_path and ipoib_neigh cannot be merged? (And ipoib_path removed?) Is there any issue with the PR/AH scope aligning with that of ipoib_neigh? - Sean From rdreier at cisco.com Mon Aug 13 20:33:24 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 20:33:24 -0700 Subject: [ofa-general] Re: sparse output In-Reply-To: <20070807112921.GC13863@mellanox.co.il> (Michael S. Tsirkin's message of "Tue, 7 Aug 2007 14:29:21 +0300") References: <20070807112921.GC13863@mellanox.co.il> Message-ID: I added at least this to my 2.6.24 tree: commit d76781d993412d0d1f05894fe0749622b37b1329 Author: Roland Dreier Date: Mon Aug 13 20:32:31 2007 -0700 IB: find_first_zero_bit() takes unsigned pointer Fix sparse warning drivers/infiniband/core/device.c:142:6: warning: incorrect type in argument 1 (different signedness) drivers/infiniband/core/device.c:142:6: expected unsigned long const *addr drivers/infiniband/core/device.c:142:6: got long *[assigned] inuse by making the local variable inuse unsigned. Does not affect generated code at all. Signed-off-by: Roland Dreier diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 3ada17c..1a6233f 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -120,12 +120,12 @@ static struct ib_device *__ib_device_get_by_name(const char *name) static int alloc_name(char *name) { - long *inuse; + unsigned long *inuse; char buf[IB_DEVICE_NAME_MAX]; struct ib_device *device; int i; - inuse = (long *) get_zeroed_page(GFP_KERNEL); + inuse = (unsigned long *) get_zeroed_page(GFP_KERNEL); if (!inuse) return -ENOMEM; From rdreier at cisco.com Mon Aug 13 21:02:25 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 13 Aug 2007 21:02:25 -0700 Subject: [ofa-general] Re: [PATCH] mlx4/net: Use the enumerated value of GO_BIT_TIMEOUT for the Command Interface timeout In-Reply-To: <200708071118.53055.dotanb@dev.mellanox.co.il> (Dotan Barak's message of "Tue, 7 Aug 2007 11:18:52 +0300") References: <200708071118.53055.dotanb@dev.mellanox.co.il> Message-ID: thanks, applied for 2.6.24 From kliteyn at mellanox.co.il Mon Aug 13 21:06:26 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 14 Aug 2007 07:06:26 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-14:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Sat_Aug_11_22:20:30_2007 [10976b5f0a105db0c12a0132178362f19f95823e] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=520 Pass=520 Fail=0 Pass: 39 Stability IS1-16.topo 39 Pkey IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 Pkey IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From mst at dev.mellanox.co.il Mon Aug 13 21:53:59 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 07:53:59 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <1187039337.14384.11.camel@firewall.xsintricity.com> References: <20070802095603.48040E608CB@openfabrics.org> <20070802160847.GC9607@mellanox.co.il> <200708061530.54392.hnguyen@linux.vnet.ibm.com> <20070811181330.GC17843@mellanox.co.il> <1187039337.14384.11.camel@firewall.xsintricity.com> Message-ID: <20070814045359.GC9924@mellanox.co.il> > Quoting Doug Ledford : > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > Quoting Hoang-Nam Nguyen : > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > Hello Doug and Scott! > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > ehca backports for kernel.org kernels seem to be broken. > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > ehca in build for these kernels. > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > Don't you guys already have RHEL4U5? It had a backports directory in > the OFED 1.2 release...and it's been out for quite a while... Our cross build environment has the headers from the x86_64 version but not the ppc version. -- MST From mst at dev.mellanox.co.il Mon Aug 13 22:00:22 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 08:00:22 +0300 Subject: [ofa-general] Re: OFED Aug 13 meeting summary In-Reply-To: <46C088BE.1090100@mellanox.co.il> References: <46C088BE.1090100@mellanox.co.il> Message-ID: <20070814050022.GD9924@mellanox.co.il> > 1. OFED 1.2.5 (was 1.2.c) is ready for release: > An issue with ehca: There are patches form kernel 2.6.23 that were > inserted by mistake and must be removed before the release There aren't, really. The snapshot generating scripts seem to be broken and seem to put code from ofed_kernel branch under the 1.2.c name. -- MST From mst at dev.mellanox.co.il Mon Aug 13 22:50:57 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 08:50:57 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <20070809072418.GA5673@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> Message-ID: <20070814055057.GE9924@mellanox.co.il> > Quoting Roland Dreier : > Subject: Re: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation > > > Having looked at implementation issues, it will be much simpler to just, as you say, > > mimic the consumer API -- in kernel space as well. > > Yes, in fact I think we might be able to get away with not breaking > the user-kernel ABI. If I understand things correctly, XRC QPs do not > have an SRQ attached to them, so we could overload the srq_handle > member of struct ib_uverbs_create_qp to hold the xrc_domain_handle > when creating an XRC QP. Correct. -- MST From HNGUYEN at de.ibm.com Tue Aug 14 00:59:34 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Tue, 14 Aug 2007 09:59:34 +0200 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <1187039337.14384.11.camel@firewall.xsintricity.com> Message-ID: Hi Doug! > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > Quoting Hoang-Nam Nguyen : > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > Hello Doug and Scott! > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > ehca backports for kernel.org kernels seem to be broken. > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > ehca in build for these kernels. > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > Don't you guys already have RHEL4U5? It had a backports directory in > the OFED 1.2 release...and it's been out for quite a while... Some part of this thread might confuse. And really, it's not about any specific backport issue from ehca or other component(s). It's a general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 in their daily build runs too. Thanks Nam From dledford at redhat.com Tue Aug 14 01:14:23 2007 From: dledford at redhat.com (Doug Ledford) Date: Tue, 14 Aug 2007 04:14:23 -0400 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: References: Message-ID: <1187079263.14384.22.camel@firewall.xsintricity.com> On Tue, 2007-08-14 at 09:59 +0200, Hoang-Nam Nguyen wrote: > Hi Doug! > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > > Quoting Hoang-Nam Nguyen : > > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > > > Hello Doug and Scott! > > > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > > ehca backports for kernel.org kernels seem to be broken. > > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > > ehca in build for these kernels. > > > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > > Don't you guys already have RHEL4U5? It had a backports directory in > > the OFED 1.2 release...and it's been out for quite a while... > Some part of this thread might confuse. And really, it's not about > any specific backport issue from ehca or other component(s). It's a > general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 > in their daily build runs too. > Thanks > Nam All of the kernel rpms from our U5 kernel have been on my web page in my sig for *ages*. All you need to do is download the needed rpms and install. -- Doug Ledford GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From mst at dev.mellanox.co.il Tue Aug 14 01:23:19 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 11:23:19 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <1187079263.14384.22.camel@firewall.xsintricity.com> References: <1187079263.14384.22.camel@firewall.xsintricity.com> Message-ID: <20070814082319.GA21380@mellanox.co.il> > Quoting Doug Ledford : > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > On Tue, 2007-08-14 at 09:59 +0200, Hoang-Nam Nguyen wrote: > > Hi Doug! > > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > > > Quoting Hoang-Nam Nguyen : > > > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > > > > > Hello Doug and Scott! > > > > > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > > > ehca backports for kernel.org kernels seem to be broken. > > > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > > > ehca in build for these kernels. > > > > > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > > > > Don't you guys already have RHEL4U5? It had a backports directory in > > > the OFED 1.2 release...and it's been out for quite a while... > > Some part of this thread might confuse. And really, it's not about > > any specific backport issue from ehca or other component(s). It's a > > general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 > > in their daily build runs too. > > Thanks > > Nam > > All of the kernel rpms from our U5 kernel have been on my web page in my > sig for *ages*. All you need to do is download the needed rpms and > install. I think there's no way to unpack these without a ppc machine, though. Is that right? -- MST From jackm at dev.mellanox.co.il Tue Aug 14 01:30:22 2007 From: jackm at dev.mellanox.co.il (Jack Morgenstein) Date: Tue, 14 Aug 2007 11:30:22 +0300 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: References: <20070808071910.GC23514@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> Message-ID: <200708141130.23254.jackm@dev.mellanox.co.il> On Tuesday 14 August 2007 00:37, Roland Dreier wrote: > Yes, in fact I think we might be able to get away with not breaking > the user-kernel ABI. If I understand things correctly, XRC QPs do not > have an SRQ attached to them, so we could overload the srq_handle > member of struct ib_uverbs_create_qp to hold the xrc_domain_handle > when creating an XRC QP. Only caveat -- I'll need to put a comment in the code regarding the overloading (to avoid confusion). > Then if we're OK with having create_xrc_srq be a separate operation > from create_srq, I think everything else doesn't break the ABI. I've checked -- everything else seems OK. > Does that seem like a good plan? The only ugly thing is making the > interface a little fatter than it needs to be with the duplicated > create_xrc_srq and create_srq operations; but we save the pain of > bumping the ABI and avoid the ugliness of putting generic objects into > the driver-specific data (which would force us to export the lookup of > XRC objects from the uverbs module, etc). I think its the only reasonable plan -- all the other options were much nastier. I'll implement this as described above. - Jack From ogerlitz at voltaire.com Tue Aug 14 01:27:38 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 14 Aug 2007 11:27:38 +0300 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: References: <000001c7dbad$b3a61660$ff0da8c0@amr.corp.intel.com> <46C08FAA.3070702@ichips.intel.com> <46C097FC.7080503@ichips.intel.com> Message-ID: <46C1677A.7060802@voltaire.com> Roland Dreier wrote: >> My initial hope is that we can get away with replicating the data, and >> using the existing SA protocol to keep it in sync. For the example >> you gave, SA LID X could forward the set to the master SA before >> recording it locally. SA LID Y would check with the master SA for the >> data if it weren't found local. > In any case, I don't see what having a mechanism for manual SA > redirect buys you. I'm probably missing the point, but obviously > you're not planning on having sysadmins manually set the SA LID on > each node, which implies having some automatic agent that can set the > SA LID. And given the existence of that agent, I don't see the > difficulty in putting that agent in the real SA and using the existing > SA redirect protocol to handle setting the SA LID. Sean, Knowing the long way you have passed in order to solve the order-n-squared-load-on-the-SA-with-all-to-all-path-query-on-mpi-job-startup I really liked your idea which allows for implementing what you have called the "local sa" as user space service which --if-- installed and running allows for offloading the SA in the mentioned scenario. From many aspects which were discussed over the previous threads, it --really-- makes a difference if the "local sa" resides in the kernel or in user space, and my take is put it in user space. Roland, What this mechanism buys Sean is solving the PR problem. Indeed sysadmins would have to install/enable the user space rpm that does the job, if they want this feature. Other then that, I don't see any manual work: a possible design I see here, is that the "user space SA" would use the SM LID for each port such that it does PR offload as the actual SA LID to replicate/invalidate/etc PR info and "proxy" non PR queries to. Indeed, end-in-mind, this package may or may not be the basis for real distributed SA for those many K ports IB clusters. However, assuming its possible to implement the concept with only a little change in the kernel ib_sa (ie conditioned on a module param etc), I would not block this approach at this stage, and let Sean suggest a concrete design. Or. From vlad at lists.openfabrics.org Tue Aug 14 01:41:04 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 14 Aug 2007 01:41:04 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070814-0100 daily build status Message-ID: <20070814084104.2E5EDE60883@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From krkumar2 at in.ibm.com Tue Aug 14 02:02:31 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Tue, 14 Aug 2007 14:32:31 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070808.034900.85820906.davem@davemloft.net> Message-ID: Hi Dave, David Miller wrote on 08/08/2007 04:19:00 PM: > From: Krishna Kumar > Date: Wed, 08 Aug 2007 15:01:14 +0530 > > > RESULTS: The performance improvement for TCP No Delay is in the range of -8% > > to 320% (with -8% being the sole negative), with many individual tests > > giving 50% or more improvement (I think it is to do with the hw slots > > getting full quicker resulting in more batching when the queue gets > > woken). The results for TCP is in the range of -11% to 93%, with most > > of the tests (8/12) giving improvements. > > Not because I think it obviates your work, but rather because I'm > curious, could you test a TSO-in-hardware driver converted to > batching and see how TSO alone compares to batching for a pure > TCP workload? > > I personally don't think it will help for that case at all as > TSO likely does better job of coalescing the work _and_ reducing > bus traffic as well as work in the TCP stack. I used E1000 (guess the choice is OK as e1000_tso returns TRUE. My hw is 82547GI). You are right, it doesn't help TSO case at all (infact degrades). Two things to note though: - E1000 may not be suitable for adding batching (which is no longer a new API, as I have changed it already). - Small skbs where TSO doesn't come into picture still seems to improve. A couple of cases for large skbs did result in some improved (like 4K, TCP No Delay, 32 procs). [Total segments retransmission for original code test run: 2220 & for new code test run: 1620. So the retransmission problem that I was getting seems to be an IPoIB bug, though I did have to fix one bug in my networking component where I was calling qdisc_run(NULL) for regular xmit path and change to always use batching. The problem is that skb1 - skb10 may be present in the queue after each of them failed to be sent out, then net_tx_action fires which batches all of these into the blist and tries to send them out again, which also fails (eg tx lock fail or queue full), then the next single skb xmit will send the latest skb ignoring the 10 skbs that are already waiting in the batching list. These 10 skbs are sent out only the next time net_tx_action is called, so out of order skbs result. This fix reduced retransmissions from 180,000 to 55,000 or so. When I changed IPoIB driver to use iterative sends of each skb instead of creating multiple Work Request's, that number went down to 15]. I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will run a longer one tonight). The results are (results in KB/s, and %): Test Case Org BW New BW % Change TCP -------- Size:32 Procs:1 1848 3918 112.01 Size:32 Procs:8 21888 21555 -1.52 Size:32 Procs:32 19317 22433 16.13 Size:256 Procs:1 15584 25991 66.78 Size:256 Procs:8 110937 74565 -32.78 Size:256 Procs:32 105767 98967 -6.42 Size:4096 Procs:1 81910 96073 17.29 Size:4096 Procs:8 113302 94040 -17.00 Size:4096 Procs:32 109664 105522 -3.77 TCP No Delay: -------------- Size:32 Procs:1 2688 3177 18.19 Size:32 Procs:8 6568 10588 61.20 Size:32 Procs:32 6573 7838 19.24 Size:256 Procs:1 7869 12724 61.69 Size:256 Procs:8 65652 45652 -30.46 Size:256 Procs:32 95114 112279 18.04 Size:4096 Procs:1 95302 84664 -11.16 Size:4096 Procs:8 111119 89111 -19.80 Size:4096 Procs:32 109249 113919 4.27 I will submit Rev4 with suggested changes (including single merged API) on Thursday after some more testing. Thanks, - KK From sbahling at novell.com Tue Aug 14 02:16:15 2007 From: sbahling at novell.com (Scott Bahling) Date: Tue, 14 Aug 2007 11:16:15 +0200 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: OFD1BF683E.8B666A76-ONC1257337.002B467C-C1257337.002BE4C2@de.ibm.com References: OFD1BF683E.8B666A76-ONC1257337.002B467C-C1257337.002BE4C2@de.ibm.com Message-ID: <1187082976.12674.24.camel@trogdor.suse.de> On Tue, 2007-08-14 at 09:59 +0200, Hoang-Nam Nguyen wrote: > Hi Doug! > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > > Quoting Hoang-Nam Nguyen : > > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > > > Hello Doug and Scott! > > > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > > ehca backports for kernel.org kernels seem to be broken. > > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > > ehca in build for these kernels. > > > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > > Don't you guys already have RHEL4U5? It had a backports directory in > > the OFED 1.2 release...and it's been out for quite a while... > Some part of this thread might confuse. And really, it's not about > any specific backport issue from ehca or other component(s). It's a > general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 > in their daily build runs too. I have checked, and Mellanox still has an active partner account with Novell which means they can download our product isos (which contain the kernel-source rpms) at http://download.novell.com and any package updates at support.novell.com http://support.novell.com/linux/psdb/ using proper Mellanox logins. You can contact me offline of you need help here. Also, our current cvs snapshots can be found at ftp://ftp.suse.com/pub/projects/kernel/kotd/ or the following mirror ftp://ftp.gwdg.de/pub/linux/suse/projects/kernel/kotd/ These are unreleased bits, but give you a peek into what changes might be coming in the next kernel update. Not sure if this is of use to you. -Scott From mst at dev.mellanox.co.il Tue Aug 14 02:40:05 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 12:40:05 +0300 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <1187082976.12674.24.camel@trogdor.suse.de> References: <1187082976.12674.24.camel@trogdor.suse.de> Message-ID: <20070814094005.GB1301@mellanox.co.il> > Quoting Scott Bahling : > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > On Tue, 2007-08-14 at 09:59 +0200, Hoang-Nam Nguyen wrote: > > Hi Doug! > > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > > > Quoting Hoang-Nam Nguyen : > > > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > > > > > Hello Doug and Scott! > > > > > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > > > ehca backports for kernel.org kernels seem to be broken. > > > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > > > ehca in build for these kernels. > > > > > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > > > > Don't you guys already have RHEL4U5? It had a backports directory in > > > the OFED 1.2 release...and it's been out for quite a while... > > Some part of this thread might confuse. And really, it's not about > > any specific backport issue from ehca or other component(s). It's a > > general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 > > in their daily build runs too. > > I have checked, and Mellanox still has an active partner account with > Novell which means they can download our product isos (which contain the > kernel-source rpms) at http://download.novell.com and any package > updates at support.novell.com http://support.novell.com/linux/psdb/ > using proper Mellanox logins. You can contact me offline of you need > help here. > > Also, our current cvs snapshots can be found at > > ftp://ftp.suse.com/pub/projects/kernel/kotd/ > or the following mirror > ftp://ftp.gwdg.de/pub/linux/suse/projects/kernel/kotd/ > > These are unreleased bits, but give you a peek into what changes might > be coming in the next kernel update. Not sure if this is of use to you. I don't have a ppc machine so I can't use your install ISOs. If someone cares enough about SLES/RHEL on ppc support in OFED, she can upload the unpacked kernel devel headers for these distros to ofa server and we'll use these to cross-compile OFED kernel bits, nightly. -- MST From krkumar2 at in.ibm.com Tue Aug 14 02:39:24 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Tue, 14 Aug 2007 15:09:24 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB Message-ID: Forgot to mention one thing: > This fix reduced > retransmissions from 180,000 to 55,000 or so. When I changed IPoIB > driver to use iterative sends of each skb instead of creating multiple > Work Request's, that number went down to 15]. This also reduced TCP No Delay performance from huge percentages like 200-400% and now is almost the same as original code. So fixing this problem in IPoIB (driver?) will enable to use the multiple Work Request & Work Completion, rather than limiting batching to single WR/WC. thanks, - KK __________________ Hi Dave, David Miller wrote on 08/08/2007 04:19:00 PM: > From: Krishna Kumar > Date: Wed, 08 Aug 2007 15:01:14 +0530 > > > RESULTS: The performance improvement for TCP No Delay is in the range of -8% > > to 320% (with -8% being the sole negative), with many individual tests > > giving 50% or more improvement (I think it is to do with the hw slots > > getting full quicker resulting in more batching when the queue gets > > woken). The results for TCP is in the range of -11% to 93%, with most > > of the tests (8/12) giving improvements. > > Not because I think it obviates your work, but rather because I'm > curious, could you test a TSO-in-hardware driver converted to > batching and see how TSO alone compares to batching for a pure > TCP workload? > > I personally don't think it will help for that case at all as > TSO likely does better job of coalescing the work _and_ reducing > bus traffic as well as work in the TCP stack. I used E1000 (guess the choice is OK as e1000_tso returns TRUE. My hw is 82547GI). You are right, it doesn't help TSO case at all (infact degrades). Two things to note though: - E1000 may not be suitable for adding batching (which is no longer a new API, as I have changed it already). - Small skbs where TSO doesn't come into picture still seems to improve. A couple of cases for large skbs did result in some improved (like 4K, TCP No Delay, 32 procs). [Total segments retransmission for original code test run: 2220 & for new code test run: 1620. So the retransmission problem that I was getting seems to be an IPoIB bug, though I did have to fix one bug in my networking component where I was calling qdisc_run(NULL) for regular xmit path and change to always use batching. The problem is that skb1 - skb10 may be present in the queue after each of them failed to be sent out, then net_tx_action fires which batches all of these into the blist and tries to send them out again, which also fails (eg tx lock fail or queue full), then the next single skb xmit will send the latest skb ignoring the 10 skbs that are already waiting in the batching list. These 10 skbs are sent out only the next time net_tx_action is called, so out of order skbs result. This fix reduced retransmissions from 180,000 to 55,000 or so. When I changed IPoIB driver to use iterative sends of each skb instead of creating multiple Work Request's, that number went down to 15]. I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will run a longer one tonight). The results are (results in KB/s, and %): Test Case Org BW New BW % Change TCP -------- Size:32 Procs:1 1848 3918 112.01 Size:32 Procs:8 21888 21555 -1.52 Size:32 Procs:32 19317 22433 16.13 Size:256 Procs:1 15584 25991 66.78 Size:256 Procs:8 110937 74565 -32.78 Size:256 Procs:32 105767 98967 -6.42 Size:4096 Procs:1 81910 96073 17.29 Size:4096 Procs:8 113302 94040 -17.00 Size:4096 Procs:32 109664 105522 -3.77 TCP No Delay: -------------- Size:32 Procs:1 2688 3177 18.19 Size:32 Procs:8 6568 10588 61.20 Size:32 Procs:32 6573 7838 19.24 Size:256 Procs:1 7869 12724 61.69 Size:256 Procs:8 65652 45652 -30.46 Size:256 Procs:32 95114 112279 18.04 Size:4096 Procs:1 95302 84664 -11.16 Size:4096 Procs:8 111119 89111 -19.80 Size:4096 Procs:32 109249 113919 4.27 I will submit Rev4 with suggested changes (including single merged API) on Thursday after some more testing. Thanks, - KK - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From vlad at lists.openfabrics.org Tue Aug 14 02:51:07 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 14 Aug 2007 02:51:07 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070814-0200 daily build status Message-ID: <20070814095107.18051E6088F@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070814-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dotanb at dev.mellanox.co.il Tue Aug 14 04:00:52 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 14 Aug 2007 14:00:52 +0300 Subject: [ofa-general] [PATCH] libibverbs: fix several issues that were reported by valgrind Message-ID: <200708141400.52402.dotanb@dev.mellanox.co.il> Fix several issues that were reported by valgrind: * Initialize the reserved attributes * fixing the pointer + size when calling to VALGRIND_MAKE_MEM_DEFINED * adding VALGRIND_MAKE_MEM_DEFINED to the buffers which were filled with the system call "write". Signed-off-by: Dotan Barak --- diff --git a/src/cmd.c b/src/cmd.c index 6d4331f..31b6092 100644 --- a/src/cmd.c +++ b/src/cmd.c @@ -248,7 +248,7 @@ int ibv_cmd_reg_mr(struct ibv_pd *pd, void *addr, size_t length, if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; - VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); mr->handle = resp->mr_handle; mr->lkey = resp->lkey; @@ -291,7 +291,7 @@ static int ibv_cmd_create_cq_v2(struct ibv_context *context, int cqe, if (write(context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; - VALGRIND_MAKE_MEM_DEFINED(resp, sizeof resp_size); + VALGRIND_MAKE_MEM_DEFINED(resp, resp_size); cq->handle = resp->cq_handle; cq->cqe = resp->cqe; @@ -432,6 +432,7 @@ int ibv_cmd_destroy_cq(struct ibv_cq *cq) IBV_INIT_CMD_RESP(&cmd, sizeof cmd, DESTROY_CQ, &resp, sizeof resp); cmd.cq_handle = cq->handle; + cmd.reserved = 0; if (write(cq->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) return errno; @@ -539,10 +540,13 @@ int ibv_cmd_query_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, IBV_INIT_CMD_RESP(cmd, cmd_size, QUERY_SRQ, &resp, sizeof resp); cmd->srq_handle = srq->handle; + cmd->reserved = 0; if (write(srq->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; + VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + srq_attr->max_wr = resp.max_wr; srq_attr->max_sge = resp.max_sge; srq_attr->srq_limit = resp.srq_limit; @@ -573,10 +577,13 @@ int ibv_cmd_destroy_srq(struct ibv_srq *srq) IBV_INIT_CMD_RESP(&cmd, sizeof cmd, DESTROY_SRQ, &resp, sizeof resp); cmd.srq_handle = srq->handle; + cmd.reserved = 0; if (write(srq->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) return errno; + VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + pthread_mutex_lock(&srq->mutex); while (srq->events_completed != resp.events_reported) pthread_cond_wait(&srq->cond, &srq->mutex); @@ -657,6 +664,8 @@ int ibv_cmd_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, if (write(qp->context->cmd_fd, cmd, cmd_size) != cmd_size) return errno; + VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + attr->qkey = resp.qkey; attr->rq_psn = resp.rq_psn; attr->sq_psn = resp.sq_psn; @@ -1064,6 +1073,7 @@ int ibv_cmd_destroy_qp(struct ibv_qp *qp) IBV_INIT_CMD_RESP(&cmd, sizeof cmd, DESTROY_QP, &resp, sizeof resp); cmd.qp_handle = qp->handle; + cmd.reserved = 0; if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) return errno; @@ -1086,6 +1096,7 @@ int ibv_cmd_attach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) memcpy(cmd.gid, gid->raw, sizeof cmd.gid); cmd.qp_handle = qp->handle; cmd.mlid = lid; + cmd.reserved = 0; if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) return errno; @@ -1101,6 +1112,7 @@ int ibv_cmd_detach_mcast(struct ibv_qp *qp, union ibv_gid *gid, uint16_t lid) memcpy(cmd.gid, gid->raw, sizeof cmd.gid); cmd.qp_handle = qp->handle; cmd.mlid = lid; + cmd.reserved = 0; if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) return errno; From dotanb at dev.mellanox.co.il Tue Aug 14 05:04:34 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 14 Aug 2007 15:04:34 +0300 Subject: [ofa-general] Re: are you planning to add support for valgrind in librdmacm and libibcm? In-Reply-To: <46BB3CB7.1040702@ichips.intel.com> References: <46BB1EDE.9030502@dev.mellanox.co.il> <46BB3CB7.1040702@ichips.intel.com> Message-ID: <46C19A52.2010506@dev.mellanox.co.il> Sean Hefty wrote: >> Do you plan to add support for valgrind in librdmacm and libibcm? > > I have no objections to it being added, but have no plans to add it > myself at this time. > > - Sean > I must admit that this is not the answer that i expected to get: you can't always get anything that you want... ;) I will send you patches that will add valgrind support to those libraries ( + fixes to issues that i may find) Dotan From dotanb at dev.mellanox.co.il Tue Aug 14 05:00:15 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 14 Aug 2007 15:00:15 +0300 Subject: [ofa-general] [PATCH] librdmacm: Fix memory leak reported by valgrind Message-ID: <200708141500.15178.dotanb@dev.mellanox.co.il> Fix memory leak reported by valgrind: ==6239== 16 bytes in 1 blocks are definitely lost in loss record 2 of 10 ==6239== at 0x4A04CBF: calloc (vg_replace_malloc.c:279) ==6239== by 0x4E386C4: ibv_get_device_list@@IBVERBS_1.1 (device.c:65) ==6239== by 0x4C2D868: ucma_init (cma.c:221) ==6239== by 0x4C2F831: rdma_create_event_channel (cma.c:299) ==6239== by 0x4018A5: main (cmatose.c:650) Signed-off-by: Dotan Barak --- diff --git a/src/cma.c b/src/cma.c index 3579530..32edc1f 100644 --- a/src/cma.c +++ b/src/cma.c @@ -238,6 +238,7 @@ static int ucma_init(void) } out: pthread_mutex_unlock(&mut); + ibv_free_device_list(dev_list); return 0; err: ucma_cleanup(); From tziporet at dev.mellanox.co.il Tue Aug 14 05:06:44 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 14 Aug 2007 15:06:44 +0300 Subject: [ofa-general] Re: OFED Aug 13 meeting summary In-Reply-To: <20070814050022.GD9924@mellanox.co.il> References: <46C088BE.1090100@mellanox.co.il> <20070814050022.GD9924@mellanox.co.il> Message-ID: <46C19AD4.4030301@mellanox.co.il> Michael S. Tsirkin wrote: >> 1. OFED 1.2.5 (was 1.2.c) is ready for release: >> An issue with ehca: There are patches form kernel 2.6.23 that were >> inserted by mistake and must be removed before the release >> > > There aren't, really. The snapshot generating scripts seem > to be broken and seem to put code from ofed_kernel branch > under the 1.2.c name. > Good - so we will release 1.2.5 today Tziporet From d.freatis at tshirtpunk.com Tue Aug 14 07:34:49 2007 From: d.freatis at tshirtpunk.com (Kristi Wilson) Date: Tue, 14 Aug 2007 13:34:49 -0100 Subject: [ofa-general] our price - $89 save $909.05 ADOBE CS3 EXTENDED Message-ID: <01c7de77$e2f5e9f0$364fc8d4@d.freatis> Our price: $69 retail price US $ 599.00 adobe cs2 http://ktonasbyet.cn From ossrosch at linux.vnet.ibm.com Tue Aug 14 06:45:33 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Tue, 14 Aug 2007 15:45:33 +0200 Subject: [ofa-general] Re: [ewg] Re: OFED Aug 13 meeting summary In-Reply-To: <46C19AD4.4030301@mellanox.co.il> References: <46C088BE.1090100@mellanox.co.il> <20070814050022.GD9924@mellanox.co.il> <46C19AD4.4030301@mellanox.co.il> Message-ID: <200708141545.33576.ossrosch@linux.vnet.ibm.com> On Tuesday 14 August 2007 14:06, Tziporet Koren wrote: > Michael S. Tsirkin wrote: > >> 1. OFED 1.2.5 (was 1.2.c) is ready for release: > >> An issue with ehca: There are patches form kernel 2.6.23 that were > >> inserted by mistake and must be removed before the release > >> > > > > There aren't, really. The snapshot generating scripts seem > > to be broken and seem to put code from ofed_kernel branch > > under the 1.2.c name. > > > Good - so we will release 1.2.5 today Hi Tziporet, can we ensure that this patch http://lists.openfabrics.org/pipermail/ewg/2007-August/004299.html is apllied? Without this patch we have a broken ehca build on rhel-4.5. regards Stefan From sashak at voltaire.com Tue Aug 14 07:22:22 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 14 Aug 2007 17:22:22 +0300 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> <20070813143206.GA461@sashak.voltaire.com> Message-ID: <20070814142222.GB2184@sashak.voltaire.com> On 10:50 Mon 13 Aug , Hal Rosenstock wrote: > On 8/13/07, Sasha Khapyorsky wrote: > > Hi Hal, > > > > On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > > > On 8/12/07, Sasha Khapyorsky wrote: > > > > Anyway OpenSM will request resweep when there are suspected nodes > > > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > > > that detected GUIDs duplication is not some corner case of port moving > > > > I prefer to not exit. Endless (re)discovery and syslog messages should > > > > be good indication if it is indeed this case. > > > > > > Couldn't there be some duplication state kept per GUID so the messages > > > only get logged on change of state to duplicated rather than > > > continually spewing into the log ? > > > > There should be one message per duplicated GUID in the sweep. The sweep > > will be repeated and in the case of real duplication the message will > > appear again - so it is per sweep. I hope it is not too much. > > Once per sweep is too much IMO. It still fills the log over time. Hmm, I cannot find how to limit those printing in an elegant way. When there is real GUID duplication it is fatal error and setup must be fixed, so it is not something which could let us to work normally. Also I guess the case itself is pretty esoteric one. Do you think it is critical? Sasha From hal.rosenstock at gmail.com Tue Aug 14 07:44:30 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 14 Aug 2007 10:44:30 -0400 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: <20070814142222.GB2184@sashak.voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> <20070813143206.GA461@sashak.voltaire.com> <20070814142222.GB2184@sashak.voltaire.com> Message-ID: On 8/14/07, Sasha Khapyorsky wrote: > On 10:50 Mon 13 Aug , Hal Rosenstock wrote: > > On 8/13/07, Sasha Khapyorsky wrote: > > > Hi Hal, > > > > > > On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > > > > On 8/12/07, Sasha Khapyorsky wrote: > > > > > Anyway OpenSM will request resweep when there are suspected nodes > > > > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > > > > that detected GUIDs duplication is not some corner case of port moving > > > > > I prefer to not exit. Endless (re)discovery and syslog messages should > > > > > be good indication if it is indeed this case. > > > > > > > > Couldn't there be some duplication state kept per GUID so the messages > > > > only get logged on change of state to duplicated rather than > > > > continually spewing into the log ? > > > > > > There should be one message per duplicated GUID in the sweep. The sweep > > > will be repeated and in the case of real duplication the message will > > > appear again - so it is per sweep. I hope it is not too much. > > > > Once per sweep is too much IMO. It still fills the log over time. > > Hmm, I cannot find how to limit those printing in an elegant way. > When there is real GUID duplication it is fatal error and setup must be > fixed, so it is not something which could let us to work normally. Also I > guess the case itself is pretty esoteric one. Do you think it is > critical? Critical no but important since when it does occur, it fills the log with these repeated messages obscuring the important ones. -- Hal > Sasha > From mst at dev.mellanox.co.il Tue Aug 14 08:06:10 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 18:06:10 +0300 Subject: [ofa-general] Re: [ewg] Re: OFED Aug 13 meeting summary In-Reply-To: <200708141545.33576.ossrosch@linux.vnet.ibm.com> References: <46C088BE.1090100@mellanox.co.il> <20070814050022.GD9924@mellanox.co.il> <46C19AD4.4030301@mellanox.co.il> <200708141545.33576.ossrosch@linux.vnet.ibm.com> Message-ID: <20070814150610.GA16827@mellanox.co.il> > Quoting Stefan Roscher : > Subject: Re: [ewg] Re: OFED Aug 13 meeting summary > > On Tuesday 14 August 2007 14:06, Tziporet Koren wrote: > > Michael S. Tsirkin wrote: > > >> 1. OFED 1.2.5 (was 1.2.c) is ready for release: > > >> An issue with ehca: There are patches form kernel 2.6.23 that were > > >> inserted by mistake and must be removed before the release > > >> > > > > > > There aren't, really. The snapshot generating scripts seem > > > to be broken and seem to put code from ofed_kernel branch > > > under the 1.2.c name. > > > > > Good - so we will release 1.2.5 today > > Hi Tziporet, > > can we ensure that this patch > http://lists.openfabrics.org/pipermail/ewg/2007-August/004299.html is apllied? > Without this patch we have a broken ehca build on rhel-4.5. > > regards Stefan I pushed a fix for this out in my tree, so Vlad will take it in 1.2.c branch. -- MST From sashak at voltaire.com Tue Aug 14 08:10:04 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 14 Aug 2007 18:10:04 +0300 Subject: [ofa-general] [PATCH] opensm: remove light_sweep_physp_list list Message-ID: <20070814151004.GC2184@sashak.voltaire.com> Remove light_sweep_physp_list, we have all information in place - no need for this additional list creation step. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_subnet.h | 6 ----- opensm/opensm/osm_drop_mgr.c | 39 ---------------------------------- opensm/opensm/osm_state_mgr.c | 41 +++++++++++++++++++++++++---------- opensm/opensm/osm_subnet.c | 5 ---- 4 files changed, 29 insertions(+), 62 deletions(-) diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index e8b9f96..277db7a 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -550,7 +550,6 @@ typedef struct _osm_subn cl_qmap_t prtn_pkey_tbl; cl_qmap_t mgrp_mlid_tbl; cl_qmap_t sm_guid_tbl; - cl_list_t light_sweep_physp_list; cl_qlist_t sa_sr_list; cl_qlist_t sa_infr_list; cl_ptr_vector_t port_lid_tbl; @@ -603,11 +602,6 @@ typedef struct _osm_subn * Container of pointers to SM objects representing other SMs * on the subnet. * -* light_sweep_physp_list -* A list of all phys ports to scan for a change in remote -* side state in next light sweep. These ports are not down -* but for some reason the remote side did not answer. -* * node_lid_tbl * Container of pointers to all Node objects in the subent. * Indexed by node LID. diff --git a/opensm/opensm/osm_drop_mgr.c b/opensm/opensm/osm_drop_mgr.c index f61769b..cb9568f 100644 --- a/opensm/opensm/osm_drop_mgr.c +++ b/opensm/opensm/osm_drop_mgr.c @@ -528,15 +528,12 @@ osm_drop_mgr_process( { cl_qmap_t *p_node_guid_tbl; cl_qmap_t *p_port_guid_tbl; - cl_list_t *p_lsweep_ports; osm_port_t *p_port; osm_port_t *p_next_port; osm_node_t *p_node; osm_node_t *p_next_node; ib_net64_t port_guid; ib_net64_t node_guid; - uint8_t port_num; - osm_physp_t *p_physp; CL_ASSERT( p_mgr ); @@ -544,7 +541,6 @@ osm_drop_mgr_process( p_node_guid_tbl = &p_mgr->p_subn->node_guid_tbl; p_port_guid_tbl = &p_mgr->p_subn->port_guid_tbl; - p_lsweep_ports = &p_mgr->p_subn->light_sweep_physp_list; CL_PLOCK_EXCL_ACQUIRE( p_mgr->p_lock ); @@ -629,41 +625,6 @@ osm_drop_mgr_process( __osm_drop_mgr_remove_port( p_mgr, p_port ); } - /* - scan through all the ports left - if the port is not DOWN and - it does not have a valid remote port - we need to track it for - next light sweep scan... - */ - cl_list_remove_all( p_lsweep_ports ); - p_next_node = (osm_node_t*)cl_qmap_head( p_node_guid_tbl ); - while( p_next_node != (osm_node_t*)cl_qmap_end( p_node_guid_tbl ) ) - { - p_node = p_next_node; - p_next_node = (osm_node_t*)cl_qmap_next( &p_next_node->map_item ); - - for (port_num = 1; port_num < osm_node_get_num_physp(p_node); port_num++) - { - p_physp = osm_node_get_physp_ptr(p_node, port_num); - if (osm_physp_is_valid(p_physp) && - (osm_physp_get_port_state(p_physp) != IB_LINK_DOWN) && - ! osm_physp_get_remote(p_physp)) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_drop_mgr_process: ERR 0108: " - "Unknown remote side for node 0x%016" PRIx64 - " port %u. Adding to light sweep sampling list\n", - cl_ntoh64( osm_node_get_node_guid( p_node )), - port_num); - - osm_dump_dr_path(p_mgr->p_log, - osm_physp_get_dr_path_ptr( p_physp ), - OSM_LOG_ERROR); - - cl_list_insert_head( p_lsweep_ports, p_physp ); - } - } - } - CL_PLOCK_RELEASE( p_mgr->p_lock ); OSM_LOG_EXIT( p_mgr->p_log ); } diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 17b66d7..28b960c 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -1011,16 +1011,15 @@ __osm_state_mgr_light_sweep_start( ib_api_status_t status = IB_SUCCESS; osm_bind_handle_t h_bind; cl_qmap_t *p_sw_tbl; - cl_list_t *p_no_rem_port_list; - cl_list_iterator_t list_iter; - uint8_t path_array[IB_SUBNET_PATH_HOPS_MAX]; + cl_map_item_t *p_next; + osm_node_t *p_node; + osm_physp_t *p_physp; + uint8_t port_num; OSM_LOG_ENTER( p_mgr->p_log, __osm_state_mgr_light_sweep_start ); p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; - memset( path_array, 0, sizeof( path_array ) ); - /* * First, get the bind handle. */ @@ -1034,14 +1033,32 @@ __osm_state_mgr_light_sweep_start( /* now scan the list of physical ports that were not down but have no remote port */ CL_PLOCK_ACQUIRE( p_mgr->p_lock ); - p_no_rem_port_list = &p_mgr->p_subn->light_sweep_physp_list; - list_iter = cl_list_head( p_no_rem_port_list ); - while( list_iter != cl_list_end( p_no_rem_port_list ) ) + p_next = cl_qmap_head( &p_mgr->p_subn->node_guid_tbl ); + while( p_next != cl_qmap_end( &p_mgr->p_subn->node_guid_tbl ) ) { - __osm_state_mgr_get_remote_port_info( p_mgr, - ( osm_physp_t * ) - cl_list_obj( list_iter ) ); - list_iter = cl_list_next( list_iter ); + p_node = (osm_node_t *)p_next; + p_next = cl_qmap_next( p_next ); + + for (port_num = 1; port_num < osm_node_get_num_physp(p_node); port_num++) + { + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (osm_physp_is_valid(p_physp) && + (osm_physp_get_port_state(p_physp) != IB_LINK_DOWN) && + ! osm_physp_get_remote(p_physp)) + { + osm_log( p_mgr->p_log, OSM_LOG_ERROR, + "osm_drop_mgr_process: ERR 0108: " + "Unknown remote side for node 0x%016" PRIx64 + " port %u. Adding to light sweep sampling list\n", + cl_ntoh64( osm_node_get_node_guid( p_node )), port_num); + + osm_dump_dr_path(p_mgr->p_log, + osm_physp_get_dr_path_ptr( p_physp ), + OSM_LOG_ERROR); + + __osm_state_mgr_get_remote_port_info( p_mgr, p_physp ); + } + } } CL_PLOCK_RELEASE( p_mgr->p_lock ); } diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index f95864c..bd612f3 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -95,8 +95,6 @@ osm_subn_construct( cl_qmap_init( &p_subn->rtr_guid_tbl ); cl_qmap_init( &p_subn->prtn_pkey_tbl ); cl_qmap_init( &p_subn->mgrp_mlid_tbl ); - cl_list_construct( &p_subn->light_sweep_physp_list ); - cl_list_init( &p_subn->light_sweep_physp_list, 5 ); } /********************************************************************** @@ -170,9 +168,6 @@ osm_subn_destroy( osm_infr_delete( p_infr ); } - cl_list_remove_all( &p_subn->light_sweep_physp_list ); - cl_list_destroy( &p_subn->light_sweep_physp_list ); - cl_ptr_vector_destroy( &p_subn->port_lid_tbl ); cl_map_remove_all(&(p_subn->opt.port_prof_ignore_guids)); cl_map_destroy(&(p_subn->opt.port_prof_ignore_guids)); -- 1.5.3.rc2.38.g11308 From mst at dev.mellanox.co.il Tue Aug 14 08:09:42 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 14 Aug 2007 18:09:42 +0300 Subject: [ofa-general] [PATCH for-2.6.24] IB.ehca: include linux/mutex.h from ehca_classes.h Message-ID: <20070814150941.GB16827@mellanox.co.il> ehca_classes.h uses struct mutex, so while linux/mutex.h seems to be pulled in indirectly by one of the headers it includes, the right thing is to include linux/mutex.h directly. Signed-off-by: Michael S. Tsirkin --- diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 1d286d3..e63711f 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h @@ -53,6 +53,7 @@ struct ehca_pd; struct ehca_av; #include +#include #include #include -- MST From sashak at voltaire.com Tue Aug 14 08:19:51 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 14 Aug 2007 18:19:51 +0300 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> <20070813143206.GA461@sashak.voltaire.com> <20070814142222.GB2184@sashak.voltaire.com> Message-ID: <20070814151951.GD2184@sashak.voltaire.com> On 10:44 Tue 14 Aug , Hal Rosenstock wrote: > On 8/14/07, Sasha Khapyorsky wrote: > > On 10:50 Mon 13 Aug , Hal Rosenstock wrote: > > > On 8/13/07, Sasha Khapyorsky wrote: > > > > Hi Hal, > > > > > > > > On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > > > > > On 8/12/07, Sasha Khapyorsky wrote: > > > > > > Anyway OpenSM will request resweep when there are suspected nodes > > > > > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > > > > > that detected GUIDs duplication is not some corner case of port moving > > > > > > I prefer to not exit. Endless (re)discovery and syslog messages should > > > > > > be good indication if it is indeed this case. > > > > > > > > > > Couldn't there be some duplication state kept per GUID so the messages > > > > > only get logged on change of state to duplicated rather than > > > > > continually spewing into the log ? > > > > > > > > There should be one message per duplicated GUID in the sweep. The sweep > > > > will be repeated and in the case of real duplication the message will > > > > appear again - so it is per sweep. I hope it is not too much. > > > > > > Once per sweep is too much IMO. It still fills the log over time. > > > > Hmm, I cannot find how to limit those printing in an elegant way. > > When there is real GUID duplication it is fatal error and setup must be > > fixed, so it is not something which could let us to work normally. Also I > > guess the case itself is pretty esoteric one. Do you think it is > > critical? > > Critical no but important since when it does occur, it fills the log > with these repeated messages obscuring the important ones. When it does occur this will be _most_ important message. There were the changes in the code, now if "duplicated GUID" is reported repeatedly it is _not_ "false alarm". OpenSM will not work in such conditions. Sasha From hal.rosenstock at gmail.com Tue Aug 14 08:24:38 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 14 Aug 2007 11:24:38 -0400 Subject: [ofa-general] Re: [PATCH 3/4] opensm: resweep instead of exit when duplicated guid suspected In-Reply-To: <20070814151951.GD2184@sashak.voltaire.com> References: <1186953826888-git-send-email-sashak@voltaire.com> <11869538263299-git-send-email-sashak@voltaire.com> <20070813143206.GA461@sashak.voltaire.com> <20070814142222.GB2184@sashak.voltaire.com> <20070814151951.GD2184@sashak.voltaire.com> Message-ID: On 8/14/07, Sasha Khapyorsky wrote: > On 10:44 Tue 14 Aug , Hal Rosenstock wrote: > > On 8/14/07, Sasha Khapyorsky wrote: > > > On 10:50 Mon 13 Aug , Hal Rosenstock wrote: > > > > On 8/13/07, Sasha Khapyorsky wrote: > > > > > Hi Hal, > > > > > > > > > > On 09:19 Mon 13 Aug , Hal Rosenstock wrote: > > > > > > On 8/12/07, Sasha Khapyorsky wrote: > > > > > > > Anyway OpenSM will request resweep when there are suspected nodes > > > > > > > with duplicate GUID on the subnet. And because we cannot be 100% sure > > > > > > > that detected GUIDs duplication is not some corner case of port moving > > > > > > > I prefer to not exit. Endless (re)discovery and syslog messages should > > > > > > > be good indication if it is indeed this case. > > > > > > > > > > > > Couldn't there be some duplication state kept per GUID so the messages > > > > > > only get logged on change of state to duplicated rather than > > > > > > continually spewing into the log ? > > > > > > > > > > There should be one message per duplicated GUID in the sweep. The sweep > > > > > will be repeated and in the case of real duplication the message will > > > > > appear again - so it is per sweep. I hope it is not too much. > > > > > > > > Once per sweep is too much IMO. It still fills the log over time. > > > > > > Hmm, I cannot find how to limit those printing in an elegant way. > > > When there is real GUID duplication it is fatal error and setup must be > > > fixed, so it is not something which could let us to work normally. Also I > > > guess the case itself is pretty esoteric one. Do you think it is > > > critical? > > > > Critical no but important since when it does occur, it fills the log > > with these repeated messages obscuring the important ones. > > When it does occur this will be _most_ important message. There were > the changes in the code, now if "duplicated GUID" is reported repeatedly > it is _not_ "false alarm". OpenSM will not work in such conditions. OK; that is different from before when there were false alarms and there were some scenarios where it would continue to "work". -- Hal > > Sasha > From sashak at voltaire.com Tue Aug 14 08:42:23 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 14 Aug 2007 18:42:23 +0300 Subject: [ofa-general] [PATCH] opensm/include/ida: indentation changes Message-ID: <20070814154223.GE2184@sashak.voltaire.com> Produced by opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs). Signed-off-by: Sasha Khapyorsky --- opensm/include/iba/ib_cm_types.h | 21 +- opensm/include/iba/ib_types.h | 3993 ++++++++++++++++++-------------------- 2 files changed, 1891 insertions(+), 2123 deletions(-) diff --git a/opensm/include/iba/ib_cm_types.h b/opensm/include/iba/ib_cm_types.h index f4fb139..c1fbfaf 100644 --- a/opensm/include/iba/ib_cm_types.h +++ b/opensm/include/iba/ib_cm_types.h @@ -43,19 +43,17 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * Defines known Communication management class versions */ #define IB_MCLASS_CM_VER_2 2 #define IB_MCLASS_CM_VER_1 1 - /* * Defines the size of user available data in communication management MADs */ @@ -70,7 +68,6 @@ BEGIN_C_DECLS #define IB_DREP_PDATA_SIZE_VER2 224 #define IB_SIDR_REQ_PDATA_SIZE_VER2 216 #define IB_SIDR_REP_PDATA_SIZE_VER2 136 - #define IB_REQ_PDATA_SIZE_VER1 92 #define IB_MRA_PDATA_SIZE_VER1 222 #define IB_REJ_PDATA_SIZE_VER1 148 @@ -82,10 +79,8 @@ BEGIN_C_DECLS #define IB_DREP_PDATA_SIZE_VER1 224 #define IB_SIDR_REQ_PDATA_SIZE_VER1 216 #define IB_SIDR_REP_PDATA_SIZE_VER1 140 - #define IB_ARI_SIZE 72 // redefine #define IB_APR_INFO_SIZE 72 - /****d* Access Layer/ib_rej_status_t * NAME * ib_rej_status_t @@ -95,7 +90,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef ib_net16_t ib_rej_status_t; +typedef ib_net16_t ib_rej_status_t; /* * SEE ALSO * ib_cm_rej, ib_cm_rej_rec_t @@ -147,7 +142,7 @@ typedef ib_net16_t ib_rej_status_t; * * SYNOPSIS */ -typedef uint8_t ib_apr_status_t; +typedef uint8_t ib_apr_status_t; /* * SEE ALSO * ib_cm_apr, ib_cm_apr_rec_t @@ -195,7 +190,7 @@ typedef uint8_t ib_apr_status_t; /* * Service ID resolution status */ -typedef uint16_t ib_sidr_status_t; +typedef uint16_t ib_sidr_status_t; #define IB_SIDR_SUCCESS 0 #define IB_SIDR_UNSUPPORTED 1 #define IB_SIDR_REJECT 2 @@ -204,7 +199,5 @@ typedef uint16_t ib_sidr_status_t; #define IB_SIDR_UNSUPPORTED_VER 5 END_C_DECLS - -#endif /* ndef WIN32 */ - -#endif /* __IB_CM_TYPES_H__ */ +#endif /* ndef WIN32 */ +#endif /* __IB_CM_TYPES_H__ */ diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 358cd62..490e3fc 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -43,28 +43,26 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if defined( WIN32 ) || defined( _WIN64 ) - #if defined( EXPORT_AL_SYMBOLS ) - #define OSM_EXPORT __declspec(dllexport) - #else - #define OSM_EXPORT __declspec(dllimport) - #endif - #define OSM_API __stdcall - #define OSM_CDECL __cdecl +#if defined( EXPORT_AL_SYMBOLS ) +#define OSM_EXPORT __declspec(dllexport) #else - #define OSM_EXPORT extern - #define OSM_API - #define OSM_CDECL - #define __ptr64 +#define OSM_EXPORT __declspec(dllimport) +#endif +#define OSM_API __stdcall +#define OSM_CDECL __cdecl +#else +#define OSM_EXPORT extern +#define OSM_API +#define OSM_CDECL +#define __ptr64 #endif - /****h* IBA Base/Constants * NAME * Constants @@ -75,7 +73,6 @@ BEGIN_C_DECLS * Definitions are from the InfiniBand Architecture Specification v1.2 * *********/ - /****d* IBA Base: Constants/MAD_BLOCK_SIZE * NAME * MAD_BLOCK_SIZE @@ -87,7 +84,6 @@ BEGIN_C_DECLS */ #define MAD_BLOCK_SIZE 256 /**********/ - /****d* IBA Base: Constants/MAD_RMPP_HDR_SIZE * NAME * MAD_RMPP_HDR_SIZE @@ -99,7 +95,6 @@ BEGIN_C_DECLS */ #define MAD_RMPP_HDR_SIZE 36 /**********/ - /****d* IBA Base: Constants/MAD_RMPP_DATA_SIZE * NAME * MAD_RMPP_DATA_SIZE @@ -111,7 +106,6 @@ BEGIN_C_DECLS */ #define MAD_RMPP_DATA_SIZE (MAD_BLOCK_SIZE - MAD_RMPP_HDR_SIZE) /**********/ - /****d* IBA Base: Constants/MAD_BLOCK_GRH_SIZE * NAME * MAD_BLOCK_GRH_SIZE @@ -123,7 +117,6 @@ BEGIN_C_DECLS */ #define MAD_BLOCK_GRH_SIZE 296 /**********/ - /****d* IBA Base: Constants/IB_LID_PERMISSIVE * NAME * IB_LID_PERMISSIVE @@ -135,7 +128,6 @@ BEGIN_C_DECLS */ #define IB_LID_PERMISSIVE 0xFFFF /**********/ - /****d* IBA Base: Constants/IB_DEFAULT_PKEY * NAME * IB_DEFAULT_PKEY @@ -147,7 +139,6 @@ BEGIN_C_DECLS */ #define IB_DEFAULT_PKEY 0xFFFF /**********/ - /****d* IBA Base: Constants/IB_QP1_WELL_KNOWN_Q_KEY * NAME * IB_QP1_WELL_KNOWN_Q_KEY @@ -159,12 +150,9 @@ BEGIN_C_DECLS */ #define IB_QP1_WELL_KNOWN_Q_KEY CL_HTON32(0x80010000) /*********/ - #define IB_QP0 0 #define IB_QP1 CL_HTON32(1) - #define IB_QP_PRIVILEGED_Q_KEY CL_HTON32(0x80000000) - /****d* IBA Base: Constants/IB_LID_UCAST_START * NAME * IB_LID_UCAST_START @@ -177,7 +165,6 @@ BEGIN_C_DECLS #define IB_LID_UCAST_START_HO 0x0001 #define IB_LID_UCAST_START (CL_HTON16(IB_LID_UCAST_START_HO)) /**********/ - /****d* IBA Base: Constants/IB_LID_UCAST_END * NAME * IB_LID_UCAST_END @@ -190,7 +177,6 @@ BEGIN_C_DECLS #define IB_LID_UCAST_END_HO 0xBFFF #define IB_LID_UCAST_END (CL_HTON16(IB_LID_UCAST_END_HO)) /**********/ - /****d* IBA Base: Constants/IB_LID_MCAST_START * NAME * IB_LID_MCAST_START @@ -203,7 +189,6 @@ BEGIN_C_DECLS #define IB_LID_MCAST_START_HO 0xC000 #define IB_LID_MCAST_START (CL_HTON16(IB_LID_MCAST_START_HO)) /**********/ - /****d* IBA Base: Constants/IB_LID_MCAST_END * NAME * IB_LID_MCAST_END @@ -216,7 +201,6 @@ BEGIN_C_DECLS #define IB_LID_MCAST_END_HO 0xFFFE #define IB_LID_MCAST_END (CL_HTON16(IB_LID_MCAST_END_HO)) /**********/ - /****d* IBA Base: Constants/IB_DEFAULT_SUBNET_PREFIX * NAME * IB_DEFAULT_SUBNET_PREFIX @@ -228,7 +212,6 @@ BEGIN_C_DECLS */ #define IB_DEFAULT_SUBNET_PREFIX (CL_HTON64(0xFE80000000000000ULL)) /**********/ - /****d* IBA Base: Constants/IB_NODE_NUM_PORTS_MAX * NAME * IB_NODE_NUM_PORTS_MAX @@ -239,7 +222,6 @@ BEGIN_C_DECLS */ #define IB_NODE_NUM_PORTS_MAX 0xFE /**********/ - /****d* IBA Base: Constants/IB_INVALID_PORT_NUM * NAME * IB_INVALID_PORT_NUM @@ -251,7 +233,6 @@ BEGIN_C_DECLS */ #define IB_INVALID_PORT_NUM 0xFF /*********/ - /****d* IBA Base: Constants/IB_SUBNET_PATH_HOPS_MAX * NAME * IB_SUBNET_PATH_HOPS_MAX @@ -263,7 +244,6 @@ BEGIN_C_DECLS */ #define IB_SUBNET_PATH_HOPS_MAX 64 /*********/ - /****d* IBA Base: Constants/IB_HOPLIMIT_MAX * NAME * IB_HOPLIMIT_MAX @@ -275,7 +255,6 @@ BEGIN_C_DECLS */ #define IB_HOPLIMIT_MAX 255 /*********/ - /****d* IBA Base: Constants/IB_PKEY_MAX_BLOCKS * NAME * IB_PKEY_MAX_BLOCKS @@ -287,7 +266,6 @@ BEGIN_C_DECLS */ #define IB_PKEY_MAX_BLOCKS 2048 /*********/ - /****d* IBA Base: Constants/IB_MCAST_MAX_BLOCK_ID * NAME * IB_MCAST_MAX_BLOCK_ID @@ -299,7 +277,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_MAX_BLOCK_ID 511 /*********/ - /****d* IBA Base: Constants/IB_MCAST_BLOCK_ID_MASK_HO * NAME * IB_MCAST_BLOCK_ID_MASK_HO @@ -311,7 +288,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_BLOCK_ID_MASK_HO 0x000001FF /*********/ - /****d* IBA Base: Constants/IB_MCAST_BLOCK_SIZE * NAME * IB_MCAST_BLOCK_SIZE @@ -323,7 +299,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_BLOCK_SIZE 32 /*********/ - /****d* IBA Base: Constants/IB_MCAST_MASK_SIZE * NAME * IB_MCAST_MASK_SIZE @@ -335,7 +310,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_MASK_SIZE 16 /*********/ - /****d* IBA Base: Constants/IB_MCAST_POSITION_MASK_HO * NAME * IB_MCAST_POSITION_MASK_HO @@ -347,7 +321,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_POSITION_MASK_HO 0xF0000000 /*********/ - /****d* IBA Base: Constants/IB_MCAST_POSITION_MAX * NAME * IB_MCAST_POSITION_MAX @@ -359,7 +332,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_POSITION_MAX 0xF /*********/ - /****d* IBA Base: Constants/IB_MCAST_POSITION_SHIFT * NAME * IB_MCAST_POSITION_SHIFT @@ -371,7 +343,6 @@ BEGIN_C_DECLS */ #define IB_MCAST_POSITION_SHIFT 28 /*********/ - /****d* IBA Base: Constants/IB_PKEY_ENTRIES_MAX * NAME * IB_PKEY_ENTRIES_MAX @@ -383,7 +354,6 @@ BEGIN_C_DECLS */ #define IB_PKEY_ENTRIES_MAX (IB_PKEY_MAX_BLOCKS * IB_NUM_PKEY_ELEMENTS_IN_BLOCK) /*********/ - /****d* IBA Base: Constants/IB_PKEY_BASE_MASK * NAME * IB_PKEY_BASE_MASK @@ -395,7 +365,6 @@ BEGIN_C_DECLS */ #define IB_PKEY_BASE_MASK (CL_HTON16(0x7FFF)) /*********/ - /****d* IBA Base: Constants/IB_PKEY_TYPE_MASK * NAME * IB_PKEY_TYPE_MASK @@ -407,7 +376,6 @@ BEGIN_C_DECLS */ #define IB_PKEY_TYPE_MASK (CL_HTON16(0x8000)) /*********/ - /****d* IBA Base: Constants/IB_DEFAULT_PARTIAL_PKEY * NAME * IB_DEFAULT_PARTIAL_PKEY @@ -419,7 +387,6 @@ BEGIN_C_DECLS */ #define IB_DEFAULT_PARTIAL_PKEY (CL_HTON16(0x7FFF)) /**********/ - /****d* IBA Base: Constants/IB_MCLASS_SUBN_LID * NAME * IB_MCLASS_SUBN_LID @@ -431,7 +398,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_SUBN_LID 0x01 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_SUBN_DIR * NAME * IB_MCLASS_SUBN_DIR @@ -443,7 +409,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_SUBN_DIR 0x81 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_SUBN_ADM * NAME * IB_MCLASS_SUBN_ADM @@ -455,7 +420,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_SUBN_ADM 0x03 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_PERF * NAME * IB_MCLASS_PERF @@ -467,7 +431,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_PERF 0x04 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_BM * NAME * IB_MCLASS_BM @@ -479,7 +442,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_BM 0x05 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_DEV_MGMT * NAME * IB_MCLASS_DEV_MGMT @@ -491,7 +453,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_DEV_MGMT 0x06 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_COMM_MGMT * NAME * IB_MCLASS_COMM_MGMT @@ -503,7 +464,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_COMM_MGMT 0x07 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_SNMP * NAME * IB_MCLASS_SNMP @@ -515,7 +475,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_SNMP 0x08 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_VENDOR_LOW_RANGE_MIN * NAME * IB_MCLASS_VENDOR_LOW_RANGE_MIN @@ -527,7 +486,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_VENDOR_LOW_RANGE_MIN 0x09 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_VENDOR_LOW_RANGE_MAX * NAME * IB_MCLASS_VENDOR_LOW_RANGE_MAX @@ -539,7 +497,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_VENDOR_LOW_RANGE_MAX 0x0f /**********/ - /****d* IBA Base: Constants/IB_MCLASS_DEV_ADM * NAME * IB_MCLASS_DEV_ADM @@ -551,7 +508,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_DEV_ADM 0x10 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_BIS * NAME * IB_MCLASS_BIS @@ -563,7 +519,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_BIS 0x12 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_VENDOR_HIGH_RANGE_MIN * NAME * IB_MCLASS_VENDOR_HIGH_RANGE_MIN @@ -575,7 +530,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_VENDOR_HIGH_RANGE_MIN 0x30 /**********/ - /****d* IBA Base: Constants/IB_MCLASS_VENDOR_HIGH_RANGE_MAX * NAME * IB_MCLASS_VENDOR_HIGH_RANGE_MAX @@ -587,7 +541,6 @@ BEGIN_C_DECLS */ #define IB_MCLASS_VENDOR_HIGH_RANGE_MAX 0x4f /**********/ - /****f* IBA Base: Types/ib_class_is_vendor_specific_low * NAME * ib_class_is_vendor_specific_low @@ -598,13 +551,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_class_is_vendor_specific_low( - IN const uint8_t class_code ) +static inline boolean_t OSM_API +ib_class_is_vendor_specific_low(IN const uint8_t class_code) { - return( (class_code >= IB_MCLASS_VENDOR_LOW_RANGE_MIN) && - (class_code <= IB_MCLASS_VENDOR_LOW_RANGE_MAX) ); + return ((class_code >= IB_MCLASS_VENDOR_LOW_RANGE_MIN) && + (class_code <= IB_MCLASS_VENDOR_LOW_RANGE_MAX)); } + /* * PARAMETERS * class_code @@ -630,13 +583,13 @@ ib_class_is_vendor_specific_low( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_class_is_vendor_specific_high( - IN const uint8_t class_code ) +static inline boolean_t OSM_API +ib_class_is_vendor_specific_high(IN const uint8_t class_code) { - return( (class_code >= IB_MCLASS_VENDOR_HIGH_RANGE_MIN) && - (class_code <= IB_MCLASS_VENDOR_HIGH_RANGE_MAX) ); + return ((class_code >= IB_MCLASS_VENDOR_HIGH_RANGE_MIN) && + (class_code <= IB_MCLASS_VENDOR_HIGH_RANGE_MAX)); } + /* * PARAMETERS * class_code @@ -652,7 +605,6 @@ ib_class_is_vendor_specific_high( * IB_MCLASS_VENDOR_HIGH_RANGE_MIN, IB_MCLASS_VENDOR_HIGH_RANGE_MAX *********/ - /****f* IBA Base: Types/ib_class_is_vendor_specific * NAME * ib_class_is_vendor_specific @@ -662,13 +614,13 @@ ib_class_is_vendor_specific_high( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_class_is_vendor_specific( - IN const uint8_t class_code ) +static inline boolean_t OSM_API +ib_class_is_vendor_specific(IN const uint8_t class_code) { - return( ib_class_is_vendor_specific_low(class_code) || - ib_class_is_vendor_specific_high(class_code) ); + return (ib_class_is_vendor_specific_low(class_code) || + ib_class_is_vendor_specific_high(class_code)); } + /* * PARAMETERS * class_code @@ -693,16 +645,15 @@ ib_class_is_vendor_specific( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_class_is_rmpp( - IN const uint8_t class_code ) +static inline boolean_t OSM_API ib_class_is_rmpp(IN const uint8_t class_code) { - return( (class_code == IB_MCLASS_SUBN_ADM) || + return ((class_code == IB_MCLASS_SUBN_ADM) || (class_code == IB_MCLASS_DEV_MGMT) || (class_code == IB_MCLASS_DEV_ADM) || (class_code == IB_MCLASS_BIS) || - ib_class_is_vendor_specific_high( class_code ) ); + ib_class_is_vendor_specific_high(class_code)); } + /* * PARAMETERS * class_code @@ -811,7 +762,6 @@ ib_class_is_rmpp( #define IB_MAD_METHOD_GETMULTI 0x14 #define IB_MAD_METHOD_GETMULTI_RESP 0x94 - /****d* IBA Base: Constants/IB_MAD_METHOD_SEND * NAME * IB_MAD_METHOD_SEND @@ -1766,7 +1716,7 @@ ib_class_is_rmpp( * * SOURCE */ -typedef uint16_t ib_net16_t; +typedef uint16_t ib_net16_t; /**********/ /****d* IBA Base: Types/ib_net32_t @@ -1778,7 +1728,7 @@ typedef uint16_t ib_net16_t; * * SOURCE */ -typedef uint32_t ib_net32_t; +typedef uint32_t ib_net32_t; /**********/ /****d* IBA Base: Types/ib_net64_t @@ -1790,7 +1740,7 @@ typedef uint32_t ib_net32_t; * * SOURCE */ -typedef uint64_t ib_net64_t; +typedef uint64_t ib_net64_t; /**********/ /****d* IBA Base: Types/ib_gid_prefix_t @@ -1801,7 +1751,7 @@ typedef uint64_t ib_net64_t; * * SOURCE */ -typedef ib_net64_t ib_gid_prefix_t; +typedef ib_net64_t ib_gid_prefix_t; /**********/ /****d* IBA Base: Constants/ib_link_states_t @@ -1821,8 +1771,7 @@ typedef ib_net64_t ib_gid_prefix_t; #define IB_LINK_ACT_DEFER 5 /**********/ -static const char* const __ib_node_type_str[] = -{ +static const char *const __ib_node_type_str[] = { "UNKNOWN", "Channel Adapter", "Switch", @@ -1839,14 +1788,13 @@ static const char* const __ib_node_type_str[] = * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_node_type_str( - IN uint8_t node_type ) +static inline const char *OSM_API ib_get_node_type_str(IN uint8_t node_type) { - if( node_type > IB_NODE_TYPE_ROUTER ) + if (node_type > IB_NODE_TYPE_ROUTER) node_type = 0; - return( __ib_node_type_str[node_type] ); + return (__ib_node_type_str[node_type]); } + /* * PARAMETERS * node_type @@ -1861,14 +1809,14 @@ ib_get_node_type_str( * ib_node_info_t *********/ -static const char* const __ib_producer_type_str[] = -{ +static const char *const __ib_producer_type_str[] = { "UNKNOWN", "Channel Adapter", "Switch", "Router", "Class Manager" }; + /****f* IBA Base: Types/ib_get_producer_type_str * NAME * ib_get_producer_type_str @@ -1880,14 +1828,15 @@ static const char* const __ib_producer_type_str[] = * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_producer_type_str( - IN ib_net32_t producer_type ) +static inline const char *OSM_API +ib_get_producer_type_str(IN ib_net32_t producer_type) { - if(cl_ntoh32(producer_type) > CL_NTOH32(IB_NOTICE_PRODUCER_TYPE_CLASS_MGR)) + if (cl_ntoh32(producer_type) > + CL_NTOH32(IB_NOTICE_PRODUCER_TYPE_CLASS_MGR)) producer_type = 0; - return( __ib_producer_type_str[cl_ntoh32(producer_type)] ); + return (__ib_producer_type_str[cl_ntoh32(producer_type)]); } + /* * PARAMETERS * producer_type @@ -1902,8 +1851,7 @@ ib_get_producer_type_str( * ib_notice_get_prod_type *********/ -static const char* const __ib_port_state_str[] = -{ +static const char *const __ib_port_state_str[] = { "No State Change (NOP)", "DOWN", "INIT", @@ -1922,14 +1870,13 @@ static const char* const __ib_port_state_str[] = * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_port_state_str( - IN uint8_t port_state ) +static inline const char *OSM_API ib_get_port_state_str(IN uint8_t port_state) { - if( port_state > IB_LINK_ACTIVE ) + if (port_state > IB_LINK_ACTIVE) port_state = IB_LINK_ACTIVE + 1; - return( __ib_port_state_str[port_state] ); + return (__ib_port_state_str[port_state]); } + /* * PARAMETERS * port_state @@ -1953,24 +1900,24 @@ ib_get_port_state_str( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_get_port_state_from_str( - IN char* p_port_state_str ) -{ - if( !strncmp(p_port_state_str,"No State Change (NOP)", 12) ) - return(0); - else if( !strncmp(p_port_state_str, "DOWN", 4) ) - return(1); - else if( !strncmp(p_port_state_str, "INIT", 4) ) - return(2); - else if( !strncmp(p_port_state_str, "ARMED" , 5) ) - return(3); - else if( !strncmp(p_port_state_str, "ACTIVE", 6) ) - return(4); - else if( !strncmp(p_port_state_str, "ACTDEFER", 8) ) - return(5); - return(6); +static inline uint8_t OSM_API +ib_get_port_state_from_str(IN char *p_port_state_str) +{ + if (!strncmp(p_port_state_str, "No State Change (NOP)", 12)) + return (0); + else if (!strncmp(p_port_state_str, "DOWN", 4)) + return (1); + else if (!strncmp(p_port_state_str, "INIT", 4)) + return (2); + else if (!strncmp(p_port_state_str, "ARMED", 5)) + return (3); + else if (!strncmp(p_port_state_str, "ACTIVE", 6)) + return (4); + else if (!strncmp(p_port_state_str, "ACTDEFER", 8)) + return (5); + return (6); } + /* * PARAMETERS * p_port_state_str @@ -2008,12 +1955,11 @@ ib_get_port_state_from_str( * * SYNOPSIS */ -static inline ib_net16_t OSM_API -ib_pkey_get_base( - IN const ib_net16_t pkey ) +static inline ib_net16_t OSM_API ib_pkey_get_base(IN const ib_net16_t pkey) { - return( (ib_net16_t)(pkey & IB_PKEY_BASE_MASK) ); + return ((ib_net16_t) (pkey & IB_PKEY_BASE_MASK)); } + /* * PARAMETERS * pkey @@ -2036,12 +1982,11 @@ ib_pkey_get_base( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_pkey_is_full_member( - IN const ib_net16_t pkey ) +static inline boolean_t OSM_API ib_pkey_is_full_member(IN const ib_net16_t pkey) { - return( (pkey & IB_PKEY_TYPE_MASK) == IB_PKEY_TYPE_MASK ); + return ((pkey & IB_PKEY_TYPE_MASK) == IB_PKEY_TYPE_MASK); } + /* * PARAMETERS * pkey @@ -2068,15 +2013,14 @@ ib_pkey_is_full_member( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_pkey_is_invalid( - IN const ib_net16_t pkey ) +static inline boolean_t OSM_API ib_pkey_is_invalid(IN const ib_net16_t pkey) { - if (ib_pkey_get_base(pkey) == 0x0000) - return TRUE; + if (ib_pkey_get_base(pkey) == 0x0000) + return TRUE; - return FALSE; + return FALSE; } + /* * PARAMETERS * pkey @@ -2099,24 +2043,21 @@ ib_pkey_is_invalid( * SYNOPSIS */ #include -typedef union _ib_gid -{ - uint8_t raw[16]; - struct _ib_gid_unicast - { - ib_gid_prefix_t prefix; - ib_net64_t interface_id; +typedef union _ib_gid { + uint8_t raw[16]; + struct _ib_gid_unicast { + ib_gid_prefix_t prefix; + ib_net64_t interface_id; } PACK_SUFFIX unicast; - struct _ib_gid_multicast - { - uint8_t header[2]; - uint8_t raw_group_id[14]; + struct _ib_gid_multicast { + uint8_t header[2]; + uint8_t raw_group_id[14]; } PACK_SUFFIX multicast; -} PACK_SUFFIX ib_gid_t; +} PACK_SUFFIX ib_gid_t; #include /* * FIELDS @@ -2142,11 +2083,9 @@ typedef union _ib_gid * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_gid_is_multicast( - IN const ib_gid_t* p_gid ) +static inline boolean_t OSM_API ib_gid_is_multicast(IN const ib_gid_t * p_gid) { - return( p_gid->raw[0] == 0xFF ); + return (p_gid->raw[0] == 0xFF); } /****f* IBA Base: Types/ib_gid_get_scope @@ -2158,11 +2097,9 @@ ib_gid_is_multicast( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_mgid_get_scope( - IN const ib_gid_t* p_gid ) +static inline uint8_t OSM_API ib_mgid_get_scope(IN const ib_gid_t * p_gid) { - return( p_gid->raw[1] & 0x0F ); + return (p_gid->raw[1] & 0x0F); } /****f* IBA Base: Types/ib_gid_set_scope @@ -2175,9 +2112,7 @@ ib_mgid_get_scope( * SYNOPSIS */ static inline void OSM_API -ib_mgid_set_scope( - IN ib_gid_t* const p_gid, - IN const uint8_t scope ) +ib_mgid_set_scope(IN ib_gid_t * const p_gid, IN const uint8_t scope) { p_gid->raw[1] &= 0xF0; p_gid->raw[1] |= scope & 0x0F; @@ -2192,14 +2127,13 @@ ib_mgid_set_scope( * * SYNOPSIS */ -static inline void OSM_API -ib_gid_set_default( - IN ib_gid_t* const p_gid, - IN const ib_net64_t interface_id ) +static inline void OSM_API +ib_gid_set_default(IN ib_gid_t * const p_gid, IN const ib_net64_t interface_id) { p_gid->unicast.prefix = IB_DEFAULT_SUBNET_PREFIX; p_gid->unicast.interface_id = interface_id; } + /* * PARAMETERS * p_gid @@ -2226,12 +2160,12 @@ ib_gid_set_default( * * SYNOPSIS */ -static inline ib_net64_t OSM_API -ib_gid_get_subnet_prefix( - IN const ib_gid_t* const p_gid ) +static inline ib_net64_t OSM_API +ib_gid_get_subnet_prefix(IN const ib_gid_t * const p_gid) { - return( p_gid->unicast.prefix ); + return (p_gid->unicast.prefix); } + /* * PARAMETERS * p_gid @@ -2256,13 +2190,13 @@ ib_gid_get_subnet_prefix( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_gid_is_link_local( - IN const ib_gid_t* const p_gid ) +static inline boolean_t OSM_API +ib_gid_is_link_local(IN const ib_gid_t * const p_gid) { - return( ( ib_gid_get_subnet_prefix( p_gid ) & - CL_HTON64( 0xFFC0000000000000ULL ) ) == IB_DEFAULT_SUBNET_PREFIX ); + return ((ib_gid_get_subnet_prefix(p_gid) & + CL_HTON64(0xFFC0000000000000ULL)) == IB_DEFAULT_SUBNET_PREFIX); } + /* * PARAMETERS * p_gid @@ -2288,13 +2222,14 @@ ib_gid_is_link_local( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_gid_is_site_local( - IN const ib_gid_t* const p_gid ) +static inline boolean_t OSM_API +ib_gid_is_site_local(IN const ib_gid_t * const p_gid) { - return( ( ib_gid_get_subnet_prefix( p_gid ) & - CL_HTON64( 0xFFFFFFFFFFFF0000ULL ) ) == CL_HTON64( 0xFEC0000000000000ULL ) ); + return ((ib_gid_get_subnet_prefix(p_gid) & + CL_HTON64(0xFFFFFFFFFFFF0000ULL)) == + CL_HTON64(0xFEC0000000000000ULL)); } + /* * PARAMETERS * p_gid @@ -2319,12 +2254,12 @@ ib_gid_is_site_local( * * SYNOPSIS */ -static inline ib_net64_t OSM_API -ib_gid_get_guid( - IN const ib_gid_t* const p_gid ) +static inline ib_net64_t OSM_API +ib_gid_get_guid(IN const ib_gid_t * const p_gid) { - return( p_gid->unicast.interface_id ); + return (p_gid->unicast.interface_id); } + /* * PARAMETERS * p_gid @@ -2350,25 +2285,24 @@ ib_gid_get_guid( * SYNOPSIS */ #include -typedef struct _ib_path_rec -{ - uint8_t resv0[8]; - ib_gid_t dgid; - ib_gid_t sgid; - ib_net16_t dlid; - ib_net16_t slid; - ib_net32_t hop_flow_raw; - uint8_t tclass; - uint8_t num_path; - ib_net16_t pkey; - ib_net16_t sl; - uint8_t mtu; - uint8_t rate; - uint8_t pkt_life; - uint8_t preference; - uint8_t resv2[6]; - -} PACK_SUFFIX ib_path_rec_t; +typedef struct _ib_path_rec { + uint8_t resv0[8]; + ib_gid_t dgid; + ib_gid_t sgid; + ib_net16_t dlid; + ib_net16_t slid; + ib_net32_t hop_flow_raw; + uint8_t tclass; + uint8_t num_path; + ib_net16_t pkey; + ib_net16_t sl; + uint8_t mtu; + uint8_t rate; + uint8_t pkt_life; + uint8_t preference; + uint8_t resv2[6]; + +} PACK_SUFFIX ib_path_rec_t; #include /* * FIELDS @@ -2687,23 +2621,21 @@ typedef struct _ib_path_rec * * SYNOPSIS */ -static inline void OSM_API -ib_path_rec_init_local( - IN ib_path_rec_t* const p_rec, - IN ib_gid_t* const p_dgid, - IN ib_gid_t* const p_sgid, - IN ib_net16_t dlid, - IN ib_net16_t slid, - IN uint8_t num_path, - IN ib_net16_t pkey, - IN uint8_t sl, - IN uint8_t mtu_selector, - IN uint8_t mtu, - IN uint8_t rate_selector, - IN uint8_t rate, - IN uint8_t pkt_life_selector, - IN uint8_t pkt_life, - IN uint8_t preference ) +static inline void OSM_API +ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, + IN ib_gid_t * const p_dgid, + IN ib_gid_t * const p_sgid, + IN ib_net16_t dlid, + IN ib_net16_t slid, + IN uint8_t num_path, + IN ib_net16_t pkey, + IN uint8_t sl, + IN uint8_t mtu_selector, + IN uint8_t mtu, + IN uint8_t rate_selector, + IN uint8_t rate, + IN uint8_t pkt_life_selector, + IN uint8_t pkt_life, IN uint8_t preference) { p_rec->dgid = *p_dgid; p_rec->sgid = *p_sgid; @@ -2712,23 +2644,24 @@ ib_path_rec_init_local( p_rec->num_path = num_path; p_rec->pkey = pkey; /* Lower 4 bits of path rec's SL are reserved. */ - p_rec->sl = cl_ntoh16( sl ); - p_rec->mtu = (uint8_t)((mtu & IB_PATH_REC_BASE_MASK) | - (uint8_t)(mtu_selector << 6)); - p_rec->rate = (uint8_t)((rate & IB_PATH_REC_BASE_MASK) | - (uint8_t)(rate_selector << 6)); - p_rec->pkt_life = (uint8_t)((pkt_life & IB_PATH_REC_BASE_MASK) | - (uint8_t)(pkt_life_selector << 6)); + p_rec->sl = cl_ntoh16(sl); + p_rec->mtu = (uint8_t) ((mtu & IB_PATH_REC_BASE_MASK) | + (uint8_t) (mtu_selector << 6)); + p_rec->rate = (uint8_t) ((rate & IB_PATH_REC_BASE_MASK) | + (uint8_t) (rate_selector << 6)); + p_rec->pkt_life = (uint8_t) ((pkt_life & IB_PATH_REC_BASE_MASK) | + (uint8_t) (pkt_life_selector << 6)); p_rec->preference = preference; /* Clear global routing fields for local path records */ p_rec->hop_flow_raw = 0; p_rec->tclass = 0; - *((uint64_t*)p_rec->resv0) = 0; - *((uint32_t*)p_rec->resv2) = 0; - *((uint16_t*)p_rec->resv2 + 2) = 0; + *((uint64_t *) p_rec->resv0) = 0; + *((uint32_t *) p_rec->resv2) = 0; + *((uint16_t *) p_rec->resv2 + 2) = 0; } + /* * PARAMETERS * p_rec @@ -2797,12 +2730,12 @@ ib_path_rec_init_local( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_num_path( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) { - return( p_rec->num_path &0x7F ); + return (p_rec->num_path & 0x7F); } + /* * PARAMETERS * p_rec @@ -2826,12 +2759,12 @@ ib_path_rec_num_path( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_sl( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)((cl_ntoh16( p_rec->sl )) & 0xF) ); + return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); } + /* * PARAMETERS * p_rec @@ -2855,12 +2788,12 @@ ib_path_rec_sl( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_mtu( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_mtu(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)(p_rec->mtu & IB_PATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->mtu & IB_PATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -2890,12 +2823,12 @@ ib_path_rec_mtu( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_mtu_sel( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_mtu_sel(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)((p_rec->mtu & IB_PATH_REC_SELECTOR_MASK) >> 6) ); + return ((uint8_t) ((p_rec->mtu & IB_PATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -2923,12 +2856,12 @@ ib_path_rec_mtu_sel( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_rate( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_rate(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)(p_rec->rate & IB_PATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->rate & IB_PATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -2962,12 +2895,12 @@ ib_path_rec_rate( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_rate_sel( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_rate_sel(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)((p_rec->rate & IB_PATH_REC_SELECTOR_MASK) >> 6) ); + return ((uint8_t) ((p_rec->rate & IB_PATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -2995,12 +2928,12 @@ ib_path_rec_rate_sel( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_pkt_life( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_pkt_life(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)(p_rec->pkt_life & IB_PATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->pkt_life & IB_PATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -3024,12 +2957,12 @@ ib_path_rec_pkt_life( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_pkt_life_sel( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_pkt_life_sel(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)((p_rec->pkt_life & IB_PATH_REC_SELECTOR_MASK) >> 6 )); + return ((uint8_t) ((p_rec->pkt_life & IB_PATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -3057,12 +2990,12 @@ ib_path_rec_pkt_life_sel( * * SYNOPSIS */ -static inline uint32_t OSM_API -ib_path_rec_flow_lbl( - IN const ib_path_rec_t* const p_rec ) +static inline uint32_t OSM_API +ib_path_rec_flow_lbl(IN const ib_path_rec_t * const p_rec) { - return( ((cl_ntoh32(p_rec->hop_flow_raw) >> 8) & 0x000FFFFF) ); + return (((cl_ntoh32(p_rec->hop_flow_raw) >> 8) & 0x000FFFFF)); } + /* * PARAMETERS * p_rec @@ -3086,12 +3019,12 @@ ib_path_rec_flow_lbl( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_rec_hop_limit( - IN const ib_path_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_path_rec_hop_limit(IN const ib_path_rec_t * const p_rec) { - return( (uint8_t)( cl_ntoh32(p_rec->hop_flow_raw) & 0x000000FF ) ); + return ((uint8_t) (cl_ntoh32(p_rec->hop_flow_raw) & 0x000000FF)); } + /* * PARAMETERS * p_rec @@ -3165,27 +3098,26 @@ ib_path_rec_hop_limit( * SYNOPSIS */ #include -typedef struct _ib_class_port_info -{ - uint8_t base_ver; - uint8_t class_ver; - ib_net16_t cap_mask; - uint8_t reserved[3]; - uint8_t resp_time_val; - ib_gid_t redir_gid; - ib_net32_t redir_tc_sl_fl; - ib_net16_t redir_lid; - ib_net16_t redir_pkey; - ib_net32_t redir_qp; - ib_net32_t redir_qkey; - ib_gid_t trap_gid; - ib_net32_t trap_tc_sl_fl; - ib_net16_t trap_lid; - ib_net16_t trap_pkey; - ib_net32_t trap_hop_qp; - ib_net32_t trap_qkey; - -} PACK_SUFFIX ib_class_port_info_t; +typedef struct _ib_class_port_info { + uint8_t base_ver; + uint8_t class_ver; + ib_net16_t cap_mask; + uint8_t reserved[3]; + uint8_t resp_time_val; + ib_gid_t redir_gid; + ib_net32_t redir_tc_sl_fl; + ib_net16_t redir_lid; + ib_net16_t redir_pkey; + ib_net32_t redir_qp; + ib_net32_t redir_qkey; + ib_gid_t trap_gid; + ib_net32_t trap_tc_sl_fl; + ib_net16_t trap_lid; + ib_net16_t trap_pkey; + ib_net32_t trap_hop_qp; + ib_net32_t trap_qkey; + +} PACK_SUFFIX ib_class_port_info_t; #include /* * FIELDS @@ -3255,14 +3187,13 @@ typedef struct _ib_class_port_info * SYNOPSIS */ #include -typedef struct _ib_sm_info -{ - ib_net64_t guid; - ib_net64_t sm_key; - ib_net32_t act_count; - uint8_t pri_state; +typedef struct _ib_sm_info { + ib_net64_t guid; + ib_net64_t sm_key; + ib_net32_t act_count; + uint8_t pri_state; -} PACK_SUFFIX ib_sm_info_t; +} PACK_SUFFIX ib_sm_info_t; #include /* * FIELDS @@ -3290,12 +3221,12 @@ typedef struct _ib_sm_info * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_sminfo_get_priority( - IN const ib_sm_info_t* const p_smi ) +static inline uint8_t OSM_API +ib_sminfo_get_priority(IN const ib_sm_info_t * const p_smi) { - return( (uint8_t)((p_smi->pri_state & 0xF0)>>4) ); + return ((uint8_t) ((p_smi->pri_state & 0xF0) >> 4)); } + /* * PARAMETERS * p_smi @@ -3318,12 +3249,12 @@ ib_sminfo_get_priority( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_sminfo_get_state( - IN const ib_sm_info_t* const p_smi ) +static inline uint8_t OSM_API +ib_sminfo_get_state(IN const ib_sm_info_t * const p_smi) { - return( (uint8_t)(p_smi->pri_state & 0x0F) ); + return ((uint8_t) (p_smi->pri_state & 0x0F)); } + /* * PARAMETERS * p_smi @@ -3347,19 +3278,18 @@ ib_sminfo_get_state( * SYNOPSIS */ #include -typedef struct _ib_mad -{ - uint8_t base_ver; - uint8_t mgmt_class; - uint8_t class_ver; - uint8_t method; - ib_net16_t status; - ib_net16_t class_spec; - ib_net64_t trans_id; - ib_net16_t attr_id; - ib_net16_t resv; - ib_net32_t attr_mod; -} PACK_SUFFIX ib_mad_t; +typedef struct _ib_mad { + uint8_t base_ver; + uint8_t mgmt_class; + uint8_t class_ver; + uint8_t method; + ib_net16_t status; + ib_net16_t class_spec; + ib_net64_t trans_id; + ib_net16_t attr_id; + ib_net16_t resv; + ib_net32_t attr_mod; +} PACK_SUFFIX ib_mad_t; #include /* * FIELDS @@ -3396,7 +3326,6 @@ typedef struct _ib_mad * SEE ALSO *********/ - /****s* IBA Base: Types/ib_rmpp_mad_t * NAME * ib_rmpp_mad_t @@ -3407,26 +3336,24 @@ typedef struct _ib_mad * SYNOPSIS */ #include -typedef struct _ib_rmpp_mad -{ - ib_mad_t common_hdr; +typedef struct _ib_rmpp_mad { + ib_mad_t common_hdr; - uint8_t rmpp_version; - uint8_t rmpp_type; - uint8_t rmpp_flags; - uint8_t rmpp_status; + uint8_t rmpp_version; + uint8_t rmpp_type; + uint8_t rmpp_flags; + uint8_t rmpp_status; - ib_net32_t seg_num; - ib_net32_t paylen_newwin; + ib_net32_t seg_num; + ib_net32_t paylen_newwin; -} PACK_SUFFIX ib_rmpp_mad_t; +} PACK_SUFFIX ib_rmpp_mad_t; #include /* * SEE ALSO * ib_mad_t *********/ - /****f* IBA Base: Types/ib_mad_init_new * NAME * ib_mad_init_new @@ -3436,17 +3363,15 @@ typedef struct _ib_rmpp_mad * * SYNOPSIS */ -static inline void OSM_API -ib_mad_init_new( - IN ib_mad_t* const p_mad, - IN const uint8_t mgmt_class, - IN const uint8_t class_ver, - IN const uint8_t method, - IN const ib_net64_t trans_id, - IN const ib_net16_t attr_id, - IN const ib_net32_t attr_mod ) -{ - CL_ASSERT( p_mad ); +static inline void OSM_API +ib_mad_init_new(IN ib_mad_t * const p_mad, + IN const uint8_t mgmt_class, + IN const uint8_t class_ver, + IN const uint8_t method, + IN const ib_net64_t trans_id, + IN const ib_net16_t attr_id, IN const ib_net32_t attr_mod) +{ + CL_ASSERT(p_mad); p_mad->base_ver = 1; p_mad->mgmt_class = mgmt_class; p_mad->class_ver = class_ver; @@ -3458,6 +3383,7 @@ ib_mad_init_new( p_mad->resv = 0; p_mad->attr_mod = attr_mod; } + /* * PARAMETERS * p_mad @@ -3499,20 +3425,19 @@ ib_mad_init_new( * * SYNOPSIS */ -static inline void OSM_API -ib_mad_init_response( - IN const ib_mad_t* const p_req_mad, - IN ib_mad_t* const p_mad, - IN const ib_net16_t status ) +static inline void OSM_API +ib_mad_init_response(IN const ib_mad_t * const p_req_mad, + IN ib_mad_t * const p_mad, IN const ib_net16_t status) { - CL_ASSERT( p_req_mad ); - CL_ASSERT( p_mad ); + CL_ASSERT(p_req_mad); + CL_ASSERT(p_mad); *p_mad = *p_req_mad; p_mad->status = status; - if( p_mad->method == IB_MAD_METHOD_SET ) + if (p_mad->method == IB_MAD_METHOD_SET) p_mad->method = IB_MAD_METHOD_GET; p_mad->method |= IB_MAD_METHOD_RESP_MASK; } + /* * PARAMETERS * p_req_mad @@ -3544,14 +3469,14 @@ ib_mad_init_response( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_mad_is_response( - IN const ib_mad_t* const p_mad ) +static inline boolean_t OSM_API +ib_mad_is_response(IN const ib_mad_t * const p_mad) { - CL_ASSERT( p_mad ); - return( (p_mad->method & IB_MAD_METHOD_RESP_MASK) == - IB_MAD_METHOD_RESP_MASK ); + CL_ASSERT(p_mad); + return ((p_mad->method & IB_MAD_METHOD_RESP_MASK) == + IB_MAD_METHOD_RESP_MASK); } + /* * PARAMETERS * p_mad @@ -3578,17 +3503,17 @@ ib_mad_is_response( #define IB_RMPP_FLAG_LAST 0x04 #define IB_RMPP_STATUS_SUCCESS 0 -#define IB_RMPP_STATUS_RESX 1 /* resources exhausted */ -#define IB_RMPP_STATUS_T2L 118 /* time too long */ -#define IB_RMPP_STATUS_BAD_LEN 119 /* incon. last and payload len */ -#define IB_RMPP_STATUS_BAD_SEG 120 /* incon. first and segment no */ -#define IB_RMPP_STATUS_BADT 121 /* bad rmpp type */ -#define IB_RMPP_STATUS_W2S 122 /* newwindowlast too small */ -#define IB_RMPP_STATUS_S2B 123 /* segment no too big */ -#define IB_RMPP_STATUS_BAD_STATUS 124 /* illegal status */ -#define IB_RMPP_STATUS_UNV 125 /* unsupported version */ -#define IB_RMPP_STATUS_TMR 126 /* too many retries */ -#define IB_RMPP_STATUS_UNSPEC 127 /* unspecified */ +#define IB_RMPP_STATUS_RESX 1 /* resources exhausted */ +#define IB_RMPP_STATUS_T2L 118 /* time too long */ +#define IB_RMPP_STATUS_BAD_LEN 119 /* incon. last and payload len */ +#define IB_RMPP_STATUS_BAD_SEG 120 /* incon. first and segment no */ +#define IB_RMPP_STATUS_BADT 121 /* bad rmpp type */ +#define IB_RMPP_STATUS_W2S 122 /* newwindowlast too small */ +#define IB_RMPP_STATUS_S2B 123 /* segment no too big */ +#define IB_RMPP_STATUS_BAD_STATUS 124 /* illegal status */ +#define IB_RMPP_STATUS_UNV 125 /* unsupported version */ +#define IB_RMPP_STATUS_TMR 126 /* too many retries */ +#define IB_RMPP_STATUS_UNSPEC 127 /* unspecified */ /****f* IBA Base: Types/ib_rmpp_is_flag_set * NAME @@ -3599,14 +3524,14 @@ ib_mad_is_response( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_rmpp_is_flag_set( - IN const ib_rmpp_mad_t* const p_rmpp_mad, - IN const uint8_t flag ) +static inline boolean_t OSM_API +ib_rmpp_is_flag_set(IN const ib_rmpp_mad_t * const p_rmpp_mad, + IN const uint8_t flag) { - CL_ASSERT( p_rmpp_mad ); - return( (p_rmpp_mad->rmpp_flags & flag) == flag ); + CL_ASSERT(p_rmpp_mad); + return ((p_rmpp_mad->rmpp_flags & flag) == flag); } + /* * PARAMETERS * ib_rmpp_mad_t @@ -3624,22 +3549,19 @@ ib_rmpp_is_flag_set( * ib_mad_t, ib_rmpp_mad_t *********/ -static inline void OSM_API -ib_rmpp_set_resp_time( - IN ib_rmpp_mad_t* const p_rmpp_mad, - IN const uint8_t resp_time ) +static inline void OSM_API +ib_rmpp_set_resp_time(IN ib_rmpp_mad_t * const p_rmpp_mad, + IN const uint8_t resp_time) { - CL_ASSERT( p_rmpp_mad ); + CL_ASSERT(p_rmpp_mad); p_rmpp_mad->rmpp_flags |= (resp_time << 3); } - -static inline uint8_t OSM_API -ib_rmpp_get_resp_time( - IN const ib_rmpp_mad_t* const p_rmpp_mad ) +static inline uint8_t OSM_API +ib_rmpp_get_resp_time(IN const ib_rmpp_mad_t * const p_rmpp_mad) { - CL_ASSERT( p_rmpp_mad ); - return( (uint8_t)(p_rmpp_mad->rmpp_flags >> 3) ); + CL_ASSERT(p_rmpp_mad); + return ((uint8_t) (p_rmpp_mad->rmpp_flags >> 3)); } /****d* IBA Base: Constants/IB_SMP_DIRECTION @@ -3679,28 +3601,27 @@ ib_rmpp_get_resp_time( */ #define IB_SMP_DATA_SIZE 64 #include -typedef struct _ib_smp -{ - uint8_t base_ver; - uint8_t mgmt_class; - uint8_t class_ver; - uint8_t method; - ib_net16_t status; - uint8_t hop_ptr; - uint8_t hop_count; - ib_net64_t trans_id; - ib_net16_t attr_id; - ib_net16_t resv; - ib_net32_t attr_mod; - ib_net64_t m_key; - ib_net16_t dr_slid; - ib_net16_t dr_dlid; - uint32_t resv1[7]; - uint8_t data[IB_SMP_DATA_SIZE]; - uint8_t initial_path[IB_SUBNET_PATH_HOPS_MAX]; - uint8_t return_path[IB_SUBNET_PATH_HOPS_MAX]; - -} PACK_SUFFIX ib_smp_t; +typedef struct _ib_smp { + uint8_t base_ver; + uint8_t mgmt_class; + uint8_t class_ver; + uint8_t method; + ib_net16_t status; + uint8_t hop_ptr; + uint8_t hop_count; + ib_net64_t trans_id; + ib_net16_t attr_id; + ib_net16_t resv; + ib_net32_t attr_mod; + ib_net64_t m_key; + ib_net16_t dr_slid; + ib_net16_t dr_dlid; + uint32_t resv1[7]; + uint8_t data[IB_SMP_DATA_SIZE]; + uint8_t initial_path[IB_SUBNET_PATH_HOPS_MAX]; + uint8_t return_path[IB_SUBNET_PATH_HOPS_MAX]; + +} PACK_SUFFIX ib_smp_t; #include /* * FIELDS @@ -3770,12 +3691,12 @@ typedef struct _ib_smp * * SYNOPSIS */ -static inline ib_net16_t OSM_API -ib_smp_get_status( - IN const ib_smp_t* const p_smp ) +static inline ib_net16_t OSM_API +ib_smp_get_status(IN const ib_smp_t * const p_smp) { - return( (ib_net16_t)(p_smp->status & IB_SMP_STATUS_MASK) ); + return ((ib_net16_t) (p_smp->status & IB_SMP_STATUS_MASK)); } + /* * PARAMETERS * p_smp @@ -3799,12 +3720,12 @@ ib_smp_get_status( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_smp_is_response( - IN const ib_smp_t* const p_smp ) +static inline boolean_t OSM_API +ib_smp_is_response(IN const ib_smp_t * const p_smp) { - return( ib_mad_is_response( (const ib_mad_t*)p_smp ) ); + return (ib_mad_is_response((const ib_mad_t *)p_smp)); } + /* * PARAMETERS * p_smp @@ -3828,12 +3749,11 @@ ib_smp_is_response( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_smp_is_d( - IN const ib_smp_t* const p_smp ) +static inline boolean_t OSM_API ib_smp_is_d(IN const ib_smp_t * const p_smp) { - return( (p_smp->status & IB_SMP_DIRECTION) == IB_SMP_DIRECTION ); + return ((p_smp->status & IB_SMP_DIRECTION) == IB_SMP_DIRECTION); } + /* * PARAMETERS * p_smp @@ -3861,21 +3781,19 @@ ib_smp_is_d( * * SYNOPSIS */ -static inline void OSM_API -ib_smp_init_new( - IN ib_smp_t* const p_smp, - IN const uint8_t method, - IN const ib_net64_t trans_id, - IN const ib_net16_t attr_id, - IN const ib_net32_t attr_mod, - IN const uint8_t hop_count, - IN const ib_net64_t m_key, - IN const uint8_t* path_out, - IN const ib_net16_t dr_slid, - IN const ib_net16_t dr_dlid ) -{ - CL_ASSERT( p_smp ); - CL_ASSERT( hop_count < IB_SUBNET_PATH_HOPS_MAX ); +static inline void OSM_API +ib_smp_init_new(IN ib_smp_t * const p_smp, + IN const uint8_t method, + IN const ib_net64_t trans_id, + IN const ib_net16_t attr_id, + IN const ib_net32_t attr_mod, + IN const uint8_t hop_count, + IN const ib_net64_t m_key, + IN const uint8_t * path_out, + IN const ib_net16_t dr_slid, IN const ib_net16_t dr_dlid) +{ + CL_ASSERT(p_smp); + CL_ASSERT(hop_count < IB_SUBNET_PATH_HOPS_MAX); p_smp->base_ver = 1; p_smp->mgmt_class = IB_MCLASS_SUBN_DIR; p_smp->class_ver = 1; @@ -3891,16 +3809,15 @@ ib_smp_init_new( p_smp->dr_slid = dr_slid; p_smp->dr_dlid = dr_dlid; - memset( p_smp->resv1, 0, - sizeof(p_smp->resv1) + - sizeof(p_smp->data) + - sizeof(p_smp->initial_path) + - sizeof(p_smp->return_path) ); + memset(p_smp->resv1, 0, + sizeof(p_smp->resv1) + + sizeof(p_smp->data) + + sizeof(p_smp->initial_path) + sizeof(p_smp->return_path)); /* copy the path */ - memcpy( &p_smp->initial_path, path_out, - sizeof( p_smp->initial_path ) ); + memcpy(&p_smp->initial_path, path_out, sizeof(p_smp->initial_path)); } + /* * PARAMETERS * p_smp @@ -3948,12 +3865,12 @@ ib_smp_init_new( * * SYNOPSIS */ -static inline void* OSM_API -ib_smp_get_payload_ptr( - IN const ib_smp_t* const p_smp ) +static inline void *OSM_API +ib_smp_get_payload_ptr(IN const ib_smp_t * const p_smp) { - return( (void*)p_smp->data ); + return ((void *)p_smp->data); } + /* * PARAMETERS * p_smp @@ -3978,21 +3895,20 @@ ib_smp_get_payload_ptr( * SYNOPSIS */ #include -typedef struct _ib_node_info -{ - uint8_t base_version; - uint8_t class_version; - uint8_t node_type; - uint8_t num_ports; - ib_net64_t sys_guid; - ib_net64_t node_guid; - ib_net64_t port_guid; - ib_net16_t partition_cap; - ib_net16_t device_id; - ib_net32_t revision; - ib_net32_t port_num_vendor_id; - -} PACK_SUFFIX ib_node_info_t; +typedef struct _ib_node_info { + uint8_t base_version; + uint8_t class_version; + uint8_t node_type; + uint8_t num_ports; + ib_net64_t sys_guid; + ib_net64_t node_guid; + ib_net64_t port_guid; + ib_net16_t partition_cap; + ib_net16_t device_id; + ib_net32_t revision; + ib_net32_t port_num_vendor_id; + +} PACK_SUFFIX ib_node_info_t; #include /************/ @@ -4008,52 +3924,47 @@ typedef struct _ib_node_info #define IB_SA_DATA_SIZE 200 #include -typedef struct _ib_sa_mad -{ - uint8_t base_ver; - uint8_t mgmt_class; - uint8_t class_ver; - uint8_t method; - ib_net16_t status; - ib_net16_t resv; - ib_net64_t trans_id; - ib_net16_t attr_id; - ib_net16_t resv1; - ib_net32_t attr_mod; - - uint8_t rmpp_version; - uint8_t rmpp_type; - uint8_t rmpp_flags; - uint8_t rmpp_status; - - ib_net32_t seg_num; - ib_net32_t paylen_newwin; - - ib_net64_t sm_key; - - ib_net16_t attr_offset; - ib_net16_t resv3; - - ib_net64_t comp_mask; - - uint8_t data[IB_SA_DATA_SIZE]; -} PACK_SUFFIX ib_sa_mad_t; +typedef struct _ib_sa_mad { + uint8_t base_ver; + uint8_t mgmt_class; + uint8_t class_ver; + uint8_t method; + ib_net16_t status; + ib_net16_t resv; + ib_net64_t trans_id; + ib_net16_t attr_id; + ib_net16_t resv1; + ib_net32_t attr_mod; + + uint8_t rmpp_version; + uint8_t rmpp_type; + uint8_t rmpp_flags; + uint8_t rmpp_status; + + ib_net32_t seg_num; + ib_net32_t paylen_newwin; + + ib_net64_t sm_key; + + ib_net16_t attr_offset; + ib_net16_t resv3; + + ib_net64_t comp_mask; + + uint8_t data[IB_SA_DATA_SIZE]; +} PACK_SUFFIX ib_sa_mad_t; #include /**********/ #define IB_SA_MAD_HDR_SIZE (sizeof(ib_sa_mad_t) - IB_SA_DATA_SIZE) -static inline uint32_t OSM_API -ib_get_attr_size( - IN const ib_net16_t attr_offset ) +static inline uint32_t OSM_API ib_get_attr_size(IN const ib_net16_t attr_offset) { - return( ((uint32_t)cl_ntoh16( attr_offset )) << 3 ); + return (((uint32_t) cl_ntoh16(attr_offset)) << 3); } -static inline ib_net16_t OSM_API -ib_get_attr_offset( - IN const uint32_t attr_size ) +static inline ib_net16_t OSM_API ib_get_attr_offset(IN const uint32_t attr_size) { - return( cl_hton16( (uint16_t)(attr_size >> 3) ) ); + return (cl_hton16((uint16_t) (attr_size >> 3))); } /****f* IBA Base: Types/ib_sa_mad_get_payload_ptr @@ -4065,12 +3976,12 @@ ib_get_attr_offset( * * SYNOPSIS */ -static inline void* OSM_API -ib_sa_mad_get_payload_ptr( - IN const ib_sa_mad_t* const p_sa_mad ) +static inline void *OSM_API +ib_sa_mad_get_payload_ptr(IN const ib_sa_mad_t * const p_sa_mad) { - return( (void*)p_sa_mad->data ); + return ((void *)p_sa_mad->data); } + /* * PARAMETERS * p_smp @@ -4088,9 +3999,9 @@ ib_sa_mad_get_payload_ptr( #define IB_NODE_INFO_PORT_NUM_MASK (CL_HTON32(0xFF000000)) #define IB_NODE_INFO_VEND_ID_MASK (CL_HTON32(0x00FFFFFF)) #if CPU_LE - #define IB_NODE_INFO_PORT_NUM_SHIFT 0 +#define IB_NODE_INFO_PORT_NUM_SHIFT 0 #else - #define IB_NODE_INFO_PORT_NUM_SHIFT 24 +#define IB_NODE_INFO_PORT_NUM_SHIFT 24 #endif /****f* IBA Base: Types/ib_node_info_get_local_port_num @@ -4102,14 +4013,14 @@ ib_sa_mad_get_payload_ptr( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_node_info_get_local_port_num( - IN const ib_node_info_t* const p_ni ) +static inline uint8_t OSM_API +ib_node_info_get_local_port_num(IN const ib_node_info_t * const p_ni) { - return( (uint8_t)(( p_ni->port_num_vendor_id & - IB_NODE_INFO_PORT_NUM_MASK ) - >> IB_NODE_INFO_PORT_NUM_SHIFT )); + return ((uint8_t) ((p_ni->port_num_vendor_id & + IB_NODE_INFO_PORT_NUM_MASK) + >> IB_NODE_INFO_PORT_NUM_SHIFT)); } + /* * PARAMETERS * p_ni @@ -4133,13 +4044,13 @@ ib_node_info_get_local_port_num( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_node_info_get_vendor_id( - IN const ib_node_info_t* const p_ni ) +static inline ib_net32_t OSM_API +ib_node_info_get_vendor_id(IN const ib_node_info_t * const p_ni) { - return( (ib_net32_t)( p_ni->port_num_vendor_id & - IB_NODE_INFO_VEND_ID_MASK ) ); + return ((ib_net32_t) (p_ni->port_num_vendor_id & + IB_NODE_INFO_VEND_ID_MASK)); } + /* * PARAMETERS * p_ni @@ -4157,26 +4068,24 @@ ib_node_info_get_vendor_id( #define IB_NODE_DESCRIPTION_SIZE 64 #include -typedef struct _ib_node_desc -{ +typedef struct _ib_node_desc { // Node String is an array of UTF-8 character that // describes the node in text format // Note that this string is NOT NULL TERMINATED! - uint8_t description[IB_NODE_DESCRIPTION_SIZE]; + uint8_t description[IB_NODE_DESCRIPTION_SIZE]; -} PACK_SUFFIX ib_node_desc_t; +} PACK_SUFFIX ib_node_desc_t; #include #include -typedef struct _ib_node_record_t -{ - ib_net16_t lid; - ib_net16_t resv; - ib_node_info_t node_info; - ib_node_desc_t node_desc; - uint8_t pad[4]; - -} PACK_SUFFIX ib_node_record_t; +typedef struct _ib_node_record_t { + ib_net16_t lid; + ib_net16_t resv; + ib_node_info_t node_info; + ib_node_desc_t node_desc; + uint8_t pad[4]; + +} PACK_SUFFIX ib_node_record_t; #include /****s* IBA Base: Types/ib_port_info_t @@ -4189,41 +4098,40 @@ typedef struct _ib_node_record_t * SYNOPSIS */ #include -typedef struct _ib_port_info -{ - ib_net64_t m_key; - ib_net64_t subnet_prefix; - ib_net16_t base_lid; - ib_net16_t master_sm_base_lid; - ib_net32_t capability_mask; - ib_net16_t diag_code; - ib_net16_t m_key_lease_period; - uint8_t local_port_num; - uint8_t link_width_enabled; - uint8_t link_width_supported; - uint8_t link_width_active; - uint8_t state_info1; /* LinkSpeedSupported and PortState */ - uint8_t state_info2; /* PortPhysState and LinkDownDefaultState */ - uint8_t mkey_lmc; - uint8_t link_speed; /* LinkSpeedEnabled and LinkSpeedActive */ - uint8_t mtu_smsl; - uint8_t vl_cap; /* VLCap and InitType */ - uint8_t vl_high_limit; - uint8_t vl_arb_high_cap; - uint8_t vl_arb_low_cap; - uint8_t mtu_cap; - uint8_t vl_stall_life; - uint8_t vl_enforce; - ib_net16_t m_key_violations; - ib_net16_t p_key_violations; - ib_net16_t q_key_violations; - uint8_t guid_cap; - uint8_t subnet_timeout; /* cli_rereg(1b), resrv( -2b), timeout(5b) */ - uint8_t resp_time_value; - uint8_t error_threshold; - -} PACK_SUFFIX ib_port_info_t; +typedef struct _ib_port_info { + ib_net64_t m_key; + ib_net64_t subnet_prefix; + ib_net16_t base_lid; + ib_net16_t master_sm_base_lid; + ib_net32_t capability_mask; + ib_net16_t diag_code; + ib_net16_t m_key_lease_period; + uint8_t local_port_num; + uint8_t link_width_enabled; + uint8_t link_width_supported; + uint8_t link_width_active; + uint8_t state_info1; /* LinkSpeedSupported and PortState */ + uint8_t state_info2; /* PortPhysState and LinkDownDefaultState */ + uint8_t mkey_lmc; + uint8_t link_speed; /* LinkSpeedEnabled and LinkSpeedActive */ + uint8_t mtu_smsl; + uint8_t vl_cap; /* VLCap and InitType */ + uint8_t vl_high_limit; + uint8_t vl_arb_high_cap; + uint8_t vl_arb_low_cap; + uint8_t mtu_cap; + uint8_t vl_stall_life; + uint8_t vl_enforce; + ib_net16_t m_key_violations; + ib_net16_t p_key_violations; + ib_net16_t q_key_violations; + uint8_t guid_cap; + uint8_t subnet_timeout; /* cli_rereg(1b), resrv( + 2b), timeout(5b) */ + uint8_t resp_time_value; + uint8_t error_threshold; + +} PACK_SUFFIX ib_port_info_t; #include /************/ @@ -4290,12 +4198,12 @@ typedef struct _ib_port_info * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_port_state( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_port_state(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->state_info1 & IB_PORT_STATE_MASK) ); + return ((uint8_t) (p_pi->state_info1 & IB_PORT_STATE_MASK)); } + /* * PARAMETERS * p_pi @@ -4318,13 +4226,13 @@ ib_port_info_get_port_state( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_port_state( - IN ib_port_info_t* const p_pi, - IN const uint8_t port_state ) +static inline void OSM_API +ib_port_info_set_port_state(IN ib_port_info_t * const p_pi, + IN const uint8_t port_state) { - p_pi->state_info1 = (uint8_t)((p_pi->state_info1 & 0xF0) | port_state ); + p_pi->state_info1 = (uint8_t) ((p_pi->state_info1 & 0xF0) | port_state); } + /* * PARAMETERS * p_pi @@ -4350,12 +4258,12 @@ ib_port_info_set_port_state( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_vl_cap( - IN const ib_port_info_t* const p_pi) +static inline uint8_t OSM_API +ib_port_info_get_vl_cap(IN const ib_port_info_t * const p_pi) { - return((p_pi->vl_cap >> 4) & 0x0F); + return ((p_pi->vl_cap >> 4) & 0x0F); } + /* * PARAMETERS * p_pi @@ -4378,12 +4286,12 @@ ib_port_info_get_vl_cap( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_init_type( - IN const ib_port_info_t* const p_pi) +static inline uint8_t OSM_API +ib_port_info_get_init_type(IN const ib_port_info_t * const p_pi) { return (uint8_t) (p_pi->vl_cap & 0x0F); } + /* * PARAMETERS * p_pi @@ -4406,12 +4314,12 @@ ib_port_info_get_init_type( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_op_vls( - IN const ib_port_info_t* const p_pi) +static inline uint8_t OSM_API +ib_port_info_get_op_vls(IN const ib_port_info_t * const p_pi) { - return((p_pi->vl_enforce >> 4) & 0x0F); + return ((p_pi->vl_enforce >> 4) & 0x0F); } + /* * PARAMETERS * p_pi @@ -4434,13 +4342,13 @@ ib_port_info_get_op_vls( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_op_vls( - IN ib_port_info_t* const p_pi, - IN const uint8_t op_vls ) +static inline void OSM_API +ib_port_info_set_op_vls(IN ib_port_info_t * const p_pi, IN const uint8_t op_vls) { - p_pi->vl_enforce = (uint8_t)((p_pi->vl_enforce & 0x0F) | (op_vls << 4) ); + p_pi->vl_enforce = + (uint8_t) ((p_pi->vl_enforce & 0x0F) | (op_vls << 4)); } + /* * PARAMETERS * p_pi @@ -4466,13 +4374,13 @@ ib_port_info_set_op_vls( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_state_no_change( - IN ib_port_info_t* const p_pi ) +static inline void OSM_API +ib_port_info_set_state_no_change(IN ib_port_info_t * const p_pi) { - ib_port_info_set_port_state( p_pi, IB_LINK_NO_CHANGE ); + ib_port_info_set_port_state(p_pi, IB_LINK_NO_CHANGE); p_pi->state_info2 = 0; } + /* * PARAMETERS * p_pi @@ -4495,14 +4403,14 @@ ib_port_info_set_state_no_change( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_link_speed_sup( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_link_speed_sup(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)((p_pi->state_info1 & - IB_PORT_LINK_SPEED_SUPPORTED_MASK) >> - IB_PORT_LINK_SPEED_SHIFT) ); + return ((uint8_t) ((p_pi->state_info1 & + IB_PORT_LINK_SPEED_SUPPORTED_MASK) >> + IB_PORT_LINK_SPEED_SHIFT)); } + /* * PARAMETERS * p_pi @@ -4526,16 +4434,16 @@ ib_port_info_get_link_speed_sup( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_link_speed_sup( - IN uint8_t const speed, - IN ib_port_info_t* p_pi ) +static inline void OSM_API +ib_port_info_set_link_speed_sup(IN uint8_t const speed, + IN ib_port_info_t * p_pi) { p_pi->state_info1 = - ( ~IB_PORT_LINK_SPEED_SUPPORTED_MASK & p_pi->state_info1 ) | - ( IB_PORT_LINK_SPEED_SUPPORTED_MASK & - (speed << IB_PORT_LINK_SPEED_SHIFT) ); + (~IB_PORT_LINK_SPEED_SUPPORTED_MASK & p_pi->state_info1) | + (IB_PORT_LINK_SPEED_SUPPORTED_MASK & + (speed << IB_PORT_LINK_SPEED_SHIFT)); } + /* * PARAMETERS * speed @@ -4561,14 +4469,14 @@ ib_port_info_set_link_speed_sup( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_port_phys_state( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_port_phys_state(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)((p_pi->state_info2 & - IB_PORT_PHYS_STATE_MASK) >> - IB_PORT_PHYS_STATE_SHIFT) ); + return ((uint8_t) ((p_pi->state_info2 & + IB_PORT_PHYS_STATE_MASK) >> + IB_PORT_PHYS_STATE_SHIFT)); } + /* * PARAMETERS * p_pi @@ -4592,16 +4500,16 @@ ib_port_info_get_port_phys_state( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_port_phys_state( - IN uint8_t const phys_state, - IN ib_port_info_t* p_pi ) +static inline void OSM_API +ib_port_info_set_port_phys_state(IN uint8_t const phys_state, + IN ib_port_info_t * p_pi) { p_pi->state_info2 = - ( ~IB_PORT_PHYS_STATE_MASK & p_pi->state_info2 ) | - ( IB_PORT_PHYS_STATE_MASK & - (phys_state << IB_PORT_PHYS_STATE_SHIFT) ); + (~IB_PORT_PHYS_STATE_MASK & p_pi->state_info2) | + (IB_PORT_PHYS_STATE_MASK & + (phys_state << IB_PORT_PHYS_STATE_SHIFT)); } + /* * PARAMETERS * phys_state @@ -4627,12 +4535,12 @@ ib_port_info_set_port_phys_state( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_link_down_def_state( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_link_down_def_state(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->state_info2 & IB_PORT_LNKDWNDFTSTATE_MASK) ); + return ((uint8_t) (p_pi->state_info2 & IB_PORT_LNKDWNDFTSTATE_MASK)); } + /* * PARAMETERS * p_pi @@ -4655,13 +4563,14 @@ ib_port_info_get_link_down_def_state( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_link_down_def_state( - IN ib_port_info_t* const p_pi, - IN const uint8_t link_dwn_state ) +static inline void OSM_API +ib_port_info_set_link_down_def_state(IN ib_port_info_t * const p_pi, + IN const uint8_t link_dwn_state) { - p_pi->state_info2 = (uint8_t)((p_pi->state_info2 & 0xF0) | link_dwn_state ); + p_pi->state_info2 = + (uint8_t) ((p_pi->state_info2 & 0xF0) | link_dwn_state); } + /* * PARAMETERS * p_pi @@ -4687,14 +4596,14 @@ ib_port_info_set_link_down_def_state( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_link_speed_active( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_link_speed_active(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)((p_pi->link_speed & - IB_PORT_LINK_SPEED_ACTIVE_MASK) >> - IB_PORT_LINK_SPEED_SHIFT) ); + return ((uint8_t) ((p_pi->link_speed & + IB_PORT_LINK_SPEED_ACTIVE_MASK) >> + IB_PORT_LINK_SPEED_SHIFT)); } + /* * PARAMETERS * p_pi @@ -4739,93 +4648,89 @@ ib_port_info_get_link_speed_active( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_compute_rate( - IN const ib_port_info_t* const p_pi ) -{ - uint8_t rate = 0; - - switch (ib_port_info_get_link_speed_active(p_pi)) - { - case IB_LINK_SPEED_ACTIVE_2_5: - switch (p_pi->link_width_active) - { - case IB_LINK_WIDTH_ACTIVE_1X: - rate = IB_PATH_RECORD_RATE_2_5_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_4X: - rate = IB_PATH_RECORD_RATE_10_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_8X: - rate = IB_PATH_RECORD_RATE_20_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_12X: - rate = IB_PATH_RECORD_RATE_30_GBS; - break; - - default: - rate = IB_PATH_RECORD_RATE_2_5_GBS; - break; - } - break; - case IB_LINK_SPEED_ACTIVE_5: - switch (p_pi->link_width_active) - { - case IB_LINK_WIDTH_ACTIVE_1X: - rate = IB_PATH_RECORD_RATE_5_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_4X: - rate = IB_PATH_RECORD_RATE_20_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_8X: - rate = IB_PATH_RECORD_RATE_40_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_12X: - rate = IB_PATH_RECORD_RATE_60_GBS; - break; - - default: - rate = IB_PATH_RECORD_RATE_5_GBS; - break; - } - break; - case IB_LINK_SPEED_ACTIVE_10: - switch (p_pi->link_width_active) - { - case IB_LINK_WIDTH_ACTIVE_1X: - rate = IB_PATH_RECORD_RATE_10_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_4X: - rate = IB_PATH_RECORD_RATE_40_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_8X: - rate = IB_PATH_RECORD_RATE_80_GBS; - break; - - case IB_LINK_WIDTH_ACTIVE_12X: - rate =IB_PATH_RECORD_RATE_120_GBS; - break; - - default: - rate = IB_PATH_RECORD_RATE_10_GBS; - break; - } - break; - default: - rate = IB_PATH_RECORD_RATE_2_5_GBS; - break; - } - - return rate; +static inline uint8_t OSM_API +ib_port_info_compute_rate(IN const ib_port_info_t * const p_pi) +{ + uint8_t rate = 0; + + switch (ib_port_info_get_link_speed_active(p_pi)) { + case IB_LINK_SPEED_ACTIVE_2_5: + switch (p_pi->link_width_active) { + case IB_LINK_WIDTH_ACTIVE_1X: + rate = IB_PATH_RECORD_RATE_2_5_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_4X: + rate = IB_PATH_RECORD_RATE_10_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_8X: + rate = IB_PATH_RECORD_RATE_20_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_12X: + rate = IB_PATH_RECORD_RATE_30_GBS; + break; + + default: + rate = IB_PATH_RECORD_RATE_2_5_GBS; + break; + } + break; + case IB_LINK_SPEED_ACTIVE_5: + switch (p_pi->link_width_active) { + case IB_LINK_WIDTH_ACTIVE_1X: + rate = IB_PATH_RECORD_RATE_5_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_4X: + rate = IB_PATH_RECORD_RATE_20_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_8X: + rate = IB_PATH_RECORD_RATE_40_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_12X: + rate = IB_PATH_RECORD_RATE_60_GBS; + break; + + default: + rate = IB_PATH_RECORD_RATE_5_GBS; + break; + } + break; + case IB_LINK_SPEED_ACTIVE_10: + switch (p_pi->link_width_active) { + case IB_LINK_WIDTH_ACTIVE_1X: + rate = IB_PATH_RECORD_RATE_10_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_4X: + rate = IB_PATH_RECORD_RATE_40_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_8X: + rate = IB_PATH_RECORD_RATE_80_GBS; + break; + + case IB_LINK_WIDTH_ACTIVE_12X: + rate = IB_PATH_RECORD_RATE_120_GBS; + break; + + default: + rate = IB_PATH_RECORD_RATE_10_GBS; + break; + } + break; + default: + rate = IB_PATH_RECORD_RATE_2_5_GBS; + break; + } + + return rate; } + /* * PARAMETERS * p_pi @@ -4848,52 +4753,48 @@ ib_port_info_compute_rate( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_path_get_ipd( - IN uint8_t local_link_width_supported, - IN uint8_t path_rec_rate ) +static inline uint8_t OSM_API +ib_path_get_ipd(IN uint8_t local_link_width_supported, IN uint8_t path_rec_rate) { uint8_t ipd = 0; - switch(local_link_width_supported) - { + switch (local_link_width_supported) { /* link_width_supported = 1: 1x */ - case 1: - break; + case 1: + break; /* link_width_supported = 3: 1x or 4x */ - case 3: - switch(path_rec_rate & 0x3F) - { - case IB_PATH_RECORD_RATE_2_5_GBS: - ipd = 3; - break; - default: - break; - } + case 3: + switch (path_rec_rate & 0x3F) { + case IB_PATH_RECORD_RATE_2_5_GBS: + ipd = 3; + break; + default: break; + } + break; /* link_width_supported = 11: 1x or 4x or 12x */ - case 11: - switch(path_rec_rate & 0x3F) - { - case IB_PATH_RECORD_RATE_2_5_GBS: - ipd = 11; - break; - case IB_PATH_RECORD_RATE_10_GBS: - ipd = 2; - break; - default: - break; - } + case 11: + switch (path_rec_rate & 0x3F) { + case IB_PATH_RECORD_RATE_2_5_GBS: + ipd = 11; + break; + case IB_PATH_RECORD_RATE_10_GBS: + ipd = 2; break; - default: break; + } + break; + + default: + break; } return ipd; } + /* * PARAMETERS * local_link_width_supported @@ -4919,12 +4820,12 @@ ib_path_get_ipd( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_mtu_cap( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_mtu_cap(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->mtu_cap & 0x0F) ); + return ((uint8_t) (p_pi->mtu_cap & 0x0F)); } + /* * PARAMETERS * p_pi @@ -4947,12 +4848,12 @@ ib_port_info_get_mtu_cap( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_neighbor_mtu( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_neighbor_mtu(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)((p_pi->mtu_smsl & 0xF0) >> 4) ); + return ((uint8_t) ((p_pi->mtu_smsl & 0xF0) >> 4)); } + /* * PARAMETERS * p_pi @@ -4975,15 +4876,15 @@ ib_port_info_get_neighbor_mtu( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_neighbor_mtu( - IN ib_port_info_t* const p_pi, - IN const uint8_t mtu ) +static inline void OSM_API +ib_port_info_set_neighbor_mtu(IN ib_port_info_t * const p_pi, + IN const uint8_t mtu) { - CL_ASSERT( mtu <= 5 ); - CL_ASSERT( mtu != 0 ); - p_pi->mtu_smsl = (uint8_t)((p_pi->mtu_smsl & 0x0F) | (mtu << 4)); + CL_ASSERT(mtu <= 5); + CL_ASSERT(mtu != 0); + p_pi->mtu_smsl = (uint8_t) ((p_pi->mtu_smsl & 0x0F) | (mtu << 4)); } + /* * PARAMETERS * p_pi @@ -5009,12 +4910,12 @@ ib_port_info_set_neighbor_mtu( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_master_smsl( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_master_smsl(IN const ib_port_info_t * const p_pi) { - return (uint8_t) (p_pi->mtu_smsl & 0x0F); + return (uint8_t) (p_pi->mtu_smsl & 0x0F); } + /* * PARAMETERS * p_pi @@ -5037,13 +4938,13 @@ ib_port_info_get_master_smsl( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_master_smsl( - IN ib_port_info_t* const p_pi, - IN const uint8_t smsl ) +static inline void OSM_API +ib_port_info_set_master_smsl(IN ib_port_info_t * const p_pi, + IN const uint8_t smsl) { - p_pi->mtu_smsl = (uint8_t)((p_pi->mtu_smsl & 0xF0) | smsl ); + p_pi->mtu_smsl = (uint8_t) ((p_pi->mtu_smsl & 0xF0) | smsl); } + /* * PARAMETERS * p_pi @@ -5069,16 +4970,15 @@ ib_port_info_set_master_smsl( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_timeout( - IN ib_port_info_t* const p_pi, - IN const uint8_t timeout ) +static inline void OSM_API +ib_port_info_set_timeout(IN ib_port_info_t * const p_pi, + IN const uint8_t timeout) { - CL_ASSERT( timeout <= 0x1F ); + CL_ASSERT(timeout <= 0x1F); p_pi->subnet_timeout = - (uint8_t)( - (p_pi->subnet_timeout & 0x80) | (timeout & 0x1F)); + (uint8_t) ((p_pi->subnet_timeout & 0x80) | (timeout & 0x1F)); } + /* * PARAMETERS * p_pi @@ -5104,16 +5004,16 @@ ib_port_info_set_timeout( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_client_rereg( - IN ib_port_info_t* const p_pi, - IN const uint8_t client_rereg ) +static inline void OSM_API +ib_port_info_set_client_rereg(IN ib_port_info_t * const p_pi, + IN const uint8_t client_rereg) { - CL_ASSERT( client_rereg <= 0x1 ); + CL_ASSERT(client_rereg <= 0x1); p_pi->subnet_timeout = - (uint8_t)( - (p_pi->subnet_timeout & 0x1F) | ((client_rereg << 7) & 0x80)); + (uint8_t) ((p_pi-> + subnet_timeout & 0x1F) | ((client_rereg << 7) & 0x80)); } + /* * PARAMETERS * p_pi @@ -5139,12 +5039,12 @@ ib_port_info_set_client_rereg( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_timeout( - IN ib_port_info_t const* p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_timeout(IN ib_port_info_t const *p_pi) { - return(p_pi->subnet_timeout & 0x1F ); + return (p_pi->subnet_timeout & 0x1F); } + /* * PARAMETERS * p_pi @@ -5167,12 +5067,12 @@ ib_port_info_get_timeout( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_client_rereg( - IN ib_port_info_t const* p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_client_rereg(IN ib_port_info_t const *p_pi) { - return ( (p_pi->subnet_timeout & 0x80 ) >> 7); + return ((p_pi->subnet_timeout & 0x80) >> 7); } + /* * PARAMETERS * p_pi @@ -5196,14 +5096,14 @@ ib_port_info_get_client_rereg( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_hoq_lifetime( - IN ib_port_info_t* const p_pi, - IN const uint8_t hoq_life ) +static inline void OSM_API +ib_port_info_set_hoq_lifetime(IN ib_port_info_t * const p_pi, + IN const uint8_t hoq_life) { - p_pi->vl_stall_life = (uint8_t)((hoq_life & 0x1f) | - (p_pi->vl_stall_life & 0xe0)); + p_pi->vl_stall_life = (uint8_t) ((hoq_life & 0x1f) | + (p_pi->vl_stall_life & 0xe0)); } + /* * PARAMETERS * p_pi @@ -5230,11 +5130,10 @@ ib_port_info_set_hoq_lifetime( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_hoq_lifetime( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_hoq_lifetime(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->vl_stall_life & 0x1f) ); + return ((uint8_t) (p_pi->vl_stall_life & 0x1f)); } /* @@ -5260,14 +5159,14 @@ ib_port_info_get_hoq_lifetime( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_vl_stall_count( - IN ib_port_info_t* const p_pi, - IN const uint8_t vl_stall_count ) +static inline void OSM_API +ib_port_info_set_vl_stall_count(IN ib_port_info_t * const p_pi, + IN const uint8_t vl_stall_count) { - p_pi->vl_stall_life = (uint8_t)((p_pi->vl_stall_life & 0x1f) | - ((vl_stall_count << 5) & 0xe0)); + p_pi->vl_stall_life = (uint8_t) ((p_pi->vl_stall_life & 0x1f) | + ((vl_stall_count << 5) & 0xe0)); } + /* * PARAMETERS * p_pi @@ -5294,11 +5193,10 @@ ib_port_info_set_vl_stall_count( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_vl_stall_count( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_vl_stall_count(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->vl_stall_life & 0xe0) >> 5); + return ((uint8_t) (p_pi->vl_stall_life & 0xe0) >> 5); } /* @@ -5323,12 +5221,12 @@ ib_port_info_get_vl_stall_count( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_lmc( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_lmc(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->mkey_lmc & IB_PORT_LMC_MASK) ); + return ((uint8_t) (p_pi->mkey_lmc & IB_PORT_LMC_MASK)); } + /* * PARAMETERS * p_pi @@ -5351,14 +5249,13 @@ ib_port_info_get_lmc( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_lmc( - IN ib_port_info_t* const p_pi, - IN const uint8_t lmc ) +static inline void OSM_API +ib_port_info_set_lmc(IN ib_port_info_t * const p_pi, IN const uint8_t lmc) { - CL_ASSERT( lmc <= IB_PORT_LMC_MAX ); - p_pi->mkey_lmc = (uint8_t)((p_pi->mkey_lmc & 0xF8) | lmc); + CL_ASSERT(lmc <= IB_PORT_LMC_MAX); + p_pi->mkey_lmc = (uint8_t) ((p_pi->mkey_lmc & 0xF8) | lmc); } + /* * PARAMETERS * p_pi @@ -5384,12 +5281,12 @@ ib_port_info_set_lmc( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_link_speed_enabled( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_link_speed_enabled(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)(p_pi->link_speed & IB_PORT_LINK_SPEED_ENABLED_MASK) ); + return ((uint8_t) (p_pi->link_speed & IB_PORT_LINK_SPEED_ENABLED_MASK)); } + /* * PARAMETERS * p_pi @@ -5412,13 +5309,14 @@ ib_port_info_get_link_speed_enabled( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_link_speed_enabled( - IN ib_port_info_t* const p_pi, - IN const uint8_t link_speed_enabled ) +static inline void OSM_API +ib_port_info_set_link_speed_enabled(IN ib_port_info_t * const p_pi, + IN const uint8_t link_speed_enabled) { - p_pi->link_speed = (uint8_t)((p_pi->link_speed & 0xF0) | link_speed_enabled ); + p_pi->link_speed = + (uint8_t) ((p_pi->link_speed & 0xF0) | link_speed_enabled); } + /* * PARAMETERS * p_pi @@ -5444,13 +5342,13 @@ ib_port_info_set_link_speed_enabled( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_mpb( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_mpb(IN const ib_port_info_t * const p_pi) { - return( (uint8_t)((p_pi->mkey_lmc & IB_PORT_MPB_MASK) >> - IB_PORT_MPB_SHIFT) ); + return ((uint8_t) ((p_pi->mkey_lmc & IB_PORT_MPB_MASK) >> + IB_PORT_MPB_SHIFT)); } + /* * PARAMETERS * p_ni @@ -5473,15 +5371,14 @@ ib_port_info_get_mpb( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_mpb( - IN ib_port_info_t* p_pi, - IN uint8_t mpb ) +static inline void OSM_API +ib_port_info_set_mpb(IN ib_port_info_t * p_pi, IN uint8_t mpb) { p_pi->mkey_lmc = - (~IB_PORT_MPB_MASK & p_pi->mkey_lmc) | - ( IB_PORT_MPB_MASK & (mpb << IB_PORT_MPB_SHIFT) ); + (~IB_PORT_MPB_MASK & p_pi->mkey_lmc) | + (IB_PORT_MPB_MASK & (mpb << IB_PORT_MPB_SHIFT)); } + /* * PARAMETERS * mpb @@ -5505,12 +5402,12 @@ ib_port_info_set_mpb( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_local_phy_err_thd( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_local_phy_err_thd(IN const ib_port_info_t * const p_pi) { - return (uint8_t)( (p_pi->error_threshold & 0xF0) >> 4); + return (uint8_t) ((p_pi->error_threshold & 0xF0) >> 4); } + /* * PARAMETERS * p_pi @@ -5533,12 +5430,12 @@ ib_port_info_get_local_phy_err_thd( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_port_info_get_overrun_err_thd( - IN const ib_port_info_t* const p_pi ) +static inline uint8_t OSM_API +ib_port_info_get_overrun_err_thd(IN const ib_port_info_t * const p_pi) { - return (uint8_t)(p_pi->error_threshold & 0x0F); + return (uint8_t) (p_pi->error_threshold & 0x0F); } + /* * PARAMETERS * p_pi @@ -5561,15 +5458,16 @@ ib_port_info_get_overrun_err_thd( * * SYNOPSIS */ -static inline void OSM_API -ib_port_info_set_phy_and_overrun_err_thd( - IN ib_port_info_t* const p_pi, - IN uint8_t phy_threshold, - IN uint8_t overrun_threshold ) +static inline void OSM_API +ib_port_info_set_phy_and_overrun_err_thd(IN ib_port_info_t * const p_pi, + IN uint8_t phy_threshold, + IN uint8_t overrun_threshold) { - p_pi->error_threshold = - (uint8_t)( ((phy_threshold & 0x0F) << 4) | (overrun_threshold & 0x0F) ); + p_pi->error_threshold = + (uint8_t) (((phy_threshold & 0x0F) << 4) | + (overrun_threshold & 0x0F)); } + /* * PARAMETERS * p_pi @@ -5589,59 +5487,55 @@ ib_port_info_set_phy_and_overrun_err_thd( * SEE ALSO *********/ -typedef uint8_t ib_svc_name_t[64]; +typedef uint8_t ib_svc_name_t[64]; #include -typedef struct _ib_service_record -{ - ib_net64_t service_id; - ib_gid_t service_gid; - ib_net16_t service_pkey; - ib_net16_t resv; - ib_net32_t service_lease; - uint8_t service_key[16]; - ib_svc_name_t service_name; - uint8_t service_data8[16]; - ib_net16_t service_data16[8]; - ib_net32_t service_data32[4]; - ib_net64_t service_data64[2]; - -} PACK_SUFFIX ib_service_record_t; +typedef struct _ib_service_record { + ib_net64_t service_id; + ib_gid_t service_gid; + ib_net16_t service_pkey; + ib_net16_t resv; + ib_net32_t service_lease; + uint8_t service_key[16]; + ib_svc_name_t service_name; + uint8_t service_data8[16]; + ib_net16_t service_data16[8]; + ib_net32_t service_data32[4]; + ib_net64_t service_data64[2]; + +} PACK_SUFFIX ib_service_record_t; #include #include -typedef struct _ib_portinfo_record -{ - ib_net16_t lid; - uint8_t port_num; - uint8_t resv; - ib_port_info_t port_info; - uint8_t pad[6]; - -} PACK_SUFFIX ib_portinfo_record_t; +typedef struct _ib_portinfo_record { + ib_net16_t lid; + uint8_t port_num; + uint8_t resv; + ib_port_info_t port_info; + uint8_t pad[6]; + +} PACK_SUFFIX ib_portinfo_record_t; #include #include -typedef struct _ib_link_record -{ - ib_net16_t from_lid; - uint8_t from_port_num; - uint8_t to_port_num; - ib_net16_t to_lid; - uint8_t pad[2]; - -} PACK_SUFFIX ib_link_record_t; +typedef struct _ib_link_record { + ib_net16_t from_lid; + uint8_t from_port_num; + uint8_t to_port_num; + ib_net16_t to_lid; + uint8_t pad[2]; + +} PACK_SUFFIX ib_link_record_t; #include #include -typedef struct _ib_sminfo_record -{ - ib_net16_t lid; - uint16_t resv0; - ib_sm_info_t sm_info; - uint8_t pad[7]; +typedef struct _ib_sminfo_record { + ib_net16_t lid; + uint16_t resv0; + ib_sm_info_t sm_info; + uint8_t pad[7]; -} PACK_SUFFIX ib_sminfo_record_t; +} PACK_SUFFIX ib_sminfo_record_t; #include /****s* IBA Base: Types/ib_lft_record_t @@ -5654,13 +5548,12 @@ typedef struct _ib_sminfo_record * SYNOPSIS */ #include -typedef struct _ib_lft_record -{ - ib_net16_t lid; - ib_net16_t block_num; - uint32_t resv0; - uint8_t lft[64]; -} PACK_SUFFIX ib_lft_record_t; +typedef struct _ib_lft_record { + ib_net16_t lid; + ib_net16_t block_num; + uint32_t resv0; + uint8_t lft[64]; +} PACK_SUFFIX ib_lft_record_t; #include /************/ @@ -5674,13 +5567,12 @@ typedef struct _ib_lft_record * SYNOPSIS */ #include -typedef struct _ib_mft_record -{ - ib_net16_t lid; - ib_net16_t position_block_num; - uint32_t resv0; - ib_net16_t mft[IB_MCAST_BLOCK_SIZE]; -} PACK_SUFFIX ib_mft_record_t; +typedef struct _ib_mft_record { + ib_net16_t lid; + ib_net16_t position_block_num; + uint32_t resv0; + ib_net16_t mft[IB_MCAST_BLOCK_SIZE]; +} PACK_SUFFIX ib_mft_record_t; #include /************/ @@ -5694,33 +5586,31 @@ typedef struct _ib_mft_record * SYNOPSIS */ #include -typedef struct _ib_switch_info -{ - ib_net16_t lin_cap; - ib_net16_t rand_cap; - ib_net16_t mcast_cap; - ib_net16_t lin_top; - uint8_t def_port; - uint8_t def_mcast_pri_port; - uint8_t def_mcast_not_port; - uint8_t life_state; - ib_net16_t lids_per_port; - ib_net16_t enforce_cap; - uint8_t flags; - -} PACK_SUFFIX ib_switch_info_t; +typedef struct _ib_switch_info { + ib_net16_t lin_cap; + ib_net16_t rand_cap; + ib_net16_t mcast_cap; + ib_net16_t lin_top; + uint8_t def_port; + uint8_t def_mcast_pri_port; + uint8_t def_mcast_not_port; + uint8_t life_state; + ib_net16_t lids_per_port; + ib_net16_t enforce_cap; + uint8_t flags; + +} PACK_SUFFIX ib_switch_info_t; #include /************/ #include -typedef struct _ib_switch_info_record -{ - ib_net16_t lid; - uint16_t resv0; - ib_switch_info_t switch_info; - uint8_t pad[3]; +typedef struct _ib_switch_info_record { + ib_net16_t lid; + uint16_t resv0; + ib_switch_info_t switch_info; + uint8_t pad[3]; -} PACK_SUFFIX ib_switch_info_record_t; +} PACK_SUFFIX ib_switch_info_record_t; #include #define IB_SWITCH_PSC 0x04 @@ -5734,12 +5624,12 @@ typedef struct _ib_switch_info_record * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_switch_info_get_state_change( - IN const ib_switch_info_t* const p_si ) +static inline boolean_t OSM_API +ib_switch_info_get_state_change(IN const ib_switch_info_t * const p_si) { - return( (p_si->life_state & IB_SWITCH_PSC) == IB_SWITCH_PSC ); + return ((p_si->life_state & IB_SWITCH_PSC) == IB_SWITCH_PSC); } + /* * PARAMETERS * p_si @@ -5762,12 +5652,12 @@ ib_switch_info_get_state_change( * * SYNOPSIS */ -static inline void OSM_API -ib_switch_info_clear_state_change( - IN ib_switch_info_t* const p_si ) +static inline void OSM_API +ib_switch_info_clear_state_change(IN ib_switch_info_t * const p_si) { - p_si->life_state = (uint8_t)(p_si->life_state & 0xFB); + p_si->life_state = (uint8_t) (p_si->life_state & 0xFB); } + /* * PARAMETERS * p_ni @@ -5792,12 +5682,12 @@ ib_switch_info_clear_state_change( * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_switch_info_is_enhanced_port0( - IN const ib_switch_info_t* const p_si ) +static inline boolean_t OSM_API +ib_switch_info_is_enhanced_port0(IN const ib_switch_info_t * const p_si) { - return( (p_si->flags & 0x08) == 0x08 ); + return ((p_si->flags & 0x08) == 0x08); } + /* * PARAMETERS * p_si @@ -5823,46 +5713,43 @@ ib_switch_info_is_enhanced_port0( #define GUID_TABLE_MAX_ENTRIES 8 #include -typedef struct _ib_guid_info -{ - ib_net64_t guid[GUID_TABLE_MAX_ENTRIES]; +typedef struct _ib_guid_info { + ib_net64_t guid[GUID_TABLE_MAX_ENTRIES]; -} PACK_SUFFIX ib_guid_info_t; +} PACK_SUFFIX ib_guid_info_t; #include /************/ #include -typedef struct _ib_guidinfo_record -{ - ib_net16_t lid; - uint8_t block_num; - uint8_t resv; - uint32_t reserved; - ib_guid_info_t guid_info; -} PACK_SUFFIX ib_guidinfo_record_t; +typedef struct _ib_guidinfo_record { + ib_net16_t lid; + uint8_t block_num; + uint8_t resv; + uint32_t reserved; + ib_guid_info_t guid_info; +} PACK_SUFFIX ib_guidinfo_record_t; #include #define IB_MULTIPATH_MAX_GIDS 11 /* Support max that can fit into first MAD (for now) */ #include -typedef struct _ib_multipath_rec_t -{ - ib_net32_t hop_flow_raw; - uint8_t tclass; - uint8_t num_path; - ib_net16_t pkey; - uint8_t resv0; - uint8_t sl; - uint8_t mtu; - uint8_t rate; - uint8_t pkt_life; - uint8_t resv1; - uint8_t independence; /* formerly resv2 */ - uint8_t sgid_count; - uint8_t dgid_count; - uint8_t resv3[7]; - ib_gid_t gids[IB_MULTIPATH_MAX_GIDS]; -} PACK_SUFFIX ib_multipath_rec_t; +typedef struct _ib_multipath_rec_t { + ib_net32_t hop_flow_raw; + uint8_t tclass; + uint8_t num_path; + ib_net16_t pkey; + uint8_t resv0; + uint8_t sl; + uint8_t mtu; + uint8_t rate; + uint8_t pkt_life; + uint8_t resv1; + uint8_t independence; /* formerly resv2 */ + uint8_t sgid_count; + uint8_t dgid_count; + uint8_t resv3[7]; + ib_gid_t gids[IB_MULTIPATH_MAX_GIDS]; +} PACK_SUFFIX ib_multipath_rec_t; #include /* * FIELDS @@ -5907,12 +5794,12 @@ typedef struct _ib_multipath_rec_t * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_num_path( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_num_path(IN const ib_multipath_rec_t * const p_rec) { - return( p_rec->num_path &0x7F ); + return (p_rec->num_path & 0x7F); } + /* * PARAMETERS * p_rec @@ -5936,12 +5823,12 @@ ib_multipath_rec_num_path( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_sl( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_sl(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)((cl_ntoh16( p_rec->sl )) & 0xF) ); + return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); } + /* * PARAMETERS * p_rec @@ -5965,12 +5852,12 @@ ib_multipath_rec_sl( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_mtu( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_mtu(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)(p_rec->mtu & IB_MULTIPATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->mtu & IB_MULTIPATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -6000,12 +5887,12 @@ ib_multipath_rec_mtu( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_mtu_sel( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_mtu_sel(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)((p_rec->mtu & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6) ); + return ((uint8_t) ((p_rec->mtu & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -6033,12 +5920,12 @@ ib_multipath_rec_mtu_sel( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_rate( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_rate(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)(p_rec->rate & IB_MULTIPATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->rate & IB_MULTIPATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -6066,12 +5953,13 @@ ib_multipath_rec_rate( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_rate_sel( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_rate_sel(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)((p_rec->rate & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6) ); + return ((uint8_t) + ((p_rec->rate & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -6099,12 +5987,12 @@ ib_multipath_rec_rate_sel( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_pkt_life( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_pkt_life(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)(p_rec->pkt_life & IB_MULTIPATH_REC_BASE_MASK) ); + return ((uint8_t) (p_rec->pkt_life & IB_MULTIPATH_REC_BASE_MASK)); } + /* * PARAMETERS * p_rec @@ -6128,12 +6016,13 @@ ib_multipath_rec_pkt_life( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_multipath_rec_pkt_life_sel( - IN const ib_multipath_rec_t* const p_rec ) +static inline uint8_t OSM_API +ib_multipath_rec_pkt_life_sel(IN const ib_multipath_rec_t * const p_rec) { - return( (uint8_t)((p_rec->pkt_life & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6 )); + return ((uint8_t) + ((p_rec->pkt_life & IB_MULTIPATH_REC_SELECTOR_MASK) >> 6)); } + /* * PARAMETERS * p_rec @@ -6164,11 +6053,10 @@ ib_multipath_rec_pkt_life_sel( */ #include -typedef struct _ib_pkey_table -{ - ib_net16_t pkey_entry[IB_NUM_PKEY_ELEMENTS_IN_BLOCK]; +typedef struct _ib_pkey_table { + ib_net16_t pkey_entry[IB_NUM_PKEY_ELEMENTS_IN_BLOCK]; -} PACK_SUFFIX ib_pkey_table_t; +} PACK_SUFFIX ib_pkey_table_t; #include /************/ @@ -6182,16 +6070,15 @@ typedef struct _ib_pkey_table * SYNOPSIS */ #include -typedef struct _ib_pkey_table_record -{ - ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 - uint16_t block_num; - uint8_t port_num; // for switch: port number, for CA: reserved - uint8_t reserved1; - uint16_t reserved2; - ib_pkey_table_t pkey_tbl; - -} PACK_SUFFIX ib_pkey_table_record_t; +typedef struct _ib_pkey_table_record { + ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 + uint16_t block_num; + uint8_t port_num; // for switch: port number, for CA: reserved + uint8_t reserved1; + uint16_t reserved2; + ib_pkey_table_t pkey_tbl; + +} PACK_SUFFIX ib_pkey_table_record_t; #include /************/ @@ -6207,10 +6094,9 @@ typedef struct _ib_pkey_table_record * SYNOPSIS */ #include -typedef struct _ib_slvl_table -{ - uint8_t raw_vl_by_sl[IB_MAX_NUM_VLS/2]; -} PACK_SUFFIX ib_slvl_table_t; +typedef struct _ib_slvl_table { + uint8_t raw_vl_by_sl[IB_MAX_NUM_VLS / 2]; +} PACK_SUFFIX ib_slvl_table_t; #include /************/ @@ -6224,15 +6110,14 @@ typedef struct _ib_slvl_table * SYNOPSIS */ #include -typedef struct _ib_slvl_table_record -{ - ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 - uint8_t in_port_num; // reserved for CAs - uint8_t out_port_num; // reserved for CAs - uint32_t resv; - ib_slvl_table_t slvl_tbl; - -} PACK_SUFFIX ib_slvl_table_record_t; +typedef struct _ib_slvl_table_record { + ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 + uint8_t in_port_num; // reserved for CAs + uint8_t out_port_num; // reserved for CAs + uint32_t resv; + ib_slvl_table_t slvl_tbl; + +} PACK_SUFFIX ib_slvl_table_record_t; #include /************/ @@ -6245,27 +6130,25 @@ typedef struct _ib_slvl_table_record * * SYNOPSIS */ -static inline void OSM_API -ib_slvl_table_set( - IN ib_slvl_table_t* p_slvl_tbl, - IN uint8_t sl_index, - IN uint8_t vl ) -{ - uint8_t idx = sl_index/2; - CL_ASSERT(vl <= 15); - CL_ASSERT(sl_index <= 15); - - if (sl_index%2) - { - /* this is an odd sl. Need to update the ls bits */ - p_slvl_tbl->raw_vl_by_sl[idx] = ( p_slvl_tbl->raw_vl_by_sl[idx] & 0xF0 ) | vl ; - } - else - { - /* this is an even sl. Need to update the ms bits */ - p_slvl_tbl->raw_vl_by_sl[idx] = ( vl << 4 ) | ( p_slvl_tbl->raw_vl_by_sl[idx] & 0x0F ); - } +static inline void OSM_API +ib_slvl_table_set(IN ib_slvl_table_t * p_slvl_tbl, + IN uint8_t sl_index, IN uint8_t vl) +{ + uint8_t idx = sl_index / 2; + CL_ASSERT(vl <= 15); + CL_ASSERT(sl_index <= 15); + + if (sl_index % 2) { + /* this is an odd sl. Need to update the ls bits */ + p_slvl_tbl->raw_vl_by_sl[idx] = + (p_slvl_tbl->raw_vl_by_sl[idx] & 0xF0) | vl; + } else { + /* this is an even sl. Need to update the ms bits */ + p_slvl_tbl->raw_vl_by_sl[idx] = + (vl << 4) | (p_slvl_tbl->raw_vl_by_sl[idx] & 0x0F); + } } + /* * PARAMETERS * p_slvl_tbl @@ -6295,25 +6178,21 @@ ib_slvl_table_set( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_slvl_table_get( - IN const ib_slvl_table_t* p_slvl_tbl, - IN uint8_t sl_index ) +static inline uint8_t OSM_API +ib_slvl_table_get(IN const ib_slvl_table_t * p_slvl_tbl, IN uint8_t sl_index) { - uint8_t idx = sl_index/2; - CL_ASSERT(sl_index <= 15); + uint8_t idx = sl_index / 2; + CL_ASSERT(sl_index <= 15); - if (sl_index%2) - { - /* this is an odd sl. Need to return the ls bits. */ - return ( p_slvl_tbl->raw_vl_by_sl[idx] & 0x0F ); - } - else - { - /* this is an even sl. Need to return the ms bits. */ - return ( (p_slvl_tbl->raw_vl_by_sl[idx] & 0xF0) >> 4 ); - } + if (sl_index % 2) { + /* this is an odd sl. Need to return the ls bits. */ + return (p_slvl_tbl->raw_vl_by_sl[idx] & 0x0F); + } else { + /* this is an even sl. Need to return the ms bits. */ + return ((p_slvl_tbl->raw_vl_by_sl[idx] & 0xF0) >> 4); + } } + /* * PARAMETERS * p_slvl_tbl @@ -6341,11 +6220,10 @@ ib_slvl_table_get( * SYNOPSIS */ #include -typedef struct _ib_vl_arb_element -{ - uint8_t vl; - uint8_t weight; -} PACK_SUFFIX ib_vl_arb_element_t; +typedef struct _ib_vl_arb_element { + uint8_t vl; + uint8_t weight; +} PACK_SUFFIX ib_vl_arb_element_t; #include /************/ @@ -6361,10 +6239,9 @@ typedef struct _ib_vl_arb_element * SYNOPSIS */ #include -typedef struct _ib_vl_arb_table -{ +typedef struct _ib_vl_arb_table { ib_vl_arb_element_t vl_entry[IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]; -} PACK_SUFFIX ib_vl_arb_table_t; +} PACK_SUFFIX ib_vl_arb_table_t; #include /************/ @@ -6378,14 +6255,13 @@ typedef struct _ib_vl_arb_table * SYNOPSIS */ #include -typedef struct _ib_vl_arb_table_record -{ - ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 - uint8_t port_num; - uint8_t block_num; - uint32_t reserved; - ib_vl_arb_table_t vl_arb_tbl; -} PACK_SUFFIX ib_vl_arb_table_record_t; +typedef struct _ib_vl_arb_table_record { + ib_net16_t lid; // for CA: lid of port, for switch lid of port 0 + uint8_t port_num; + uint8_t block_num; + uint32_t reserved; + ib_vl_arb_table_t vl_arb_tbl; +} PACK_SUFFIX ib_vl_arb_table_record_t; #include /************/ @@ -6393,15 +6269,14 @@ typedef struct _ib_vl_arb_table_record * Global route header information received with unreliable datagram messages */ #include -typedef struct _ib_grh -{ - ib_net32_t ver_class_flow; - ib_net16_t resv1; - uint8_t resv2; - uint8_t hop_limit; - ib_gid_t src_gid; - ib_gid_t dest_gid; -} PACK_SUFFIX ib_grh_t; +typedef struct _ib_grh { + ib_net32_t ver_class_flow; + ib_net16_t resv1; + uint8_t resv2; + uint8_t hop_limit; + ib_gid_t src_gid; + ib_gid_t dest_gid; +} PACK_SUFFIX ib_grh_t; #include /****f* IBA Base: Types/ib_grh_get_ver_class_flow @@ -6413,28 +6288,28 @@ typedef struct _ib_grh * * SYNOPSIS */ -static inline void OSM_API -ib_grh_get_ver_class_flow( - IN const ib_net32_t ver_class_flow, - OUT uint8_t* const p_ver, - OUT uint8_t* const p_tclass, - OUT uint32_t* const p_flow_lbl ) +static inline void OSM_API +ib_grh_get_ver_class_flow(IN const ib_net32_t ver_class_flow, + OUT uint8_t * const p_ver, + OUT uint8_t * const p_tclass, + OUT uint32_t * const p_flow_lbl) { ib_net32_t tmp_ver_class_flow; if (p_ver) - *p_ver = (uint8_t)(ver_class_flow & 0x0f); + *p_ver = (uint8_t) (ver_class_flow & 0x0f); tmp_ver_class_flow = ver_class_flow >> 4; if (p_tclass) - *p_tclass = (uint8_t)(tmp_ver_class_flow & 0xff); + *p_tclass = (uint8_t) (tmp_ver_class_flow & 0xff); tmp_ver_class_flow = tmp_ver_class_flow >> 8; if (p_flow_lbl) *p_flow_lbl = tmp_ver_class_flow & 0xfffff; } + /* * PARAMETERS * ver_class_flow @@ -6465,13 +6340,11 @@ ib_grh_get_ver_class_flow( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_grh_set_ver_class_flow( - IN const uint8_t ver, - IN const uint8_t tclass, - IN const uint32_t flow_lbl ) +static inline ib_net32_t OSM_API +ib_grh_set_ver_class_flow(IN const uint8_t ver, + IN const uint8_t tclass, IN const uint32_t flow_lbl) { - ib_net32_t ver_class_flow; + ib_net32_t ver_class_flow; ver_class_flow = flow_lbl; ver_class_flow = ver_class_flow << 8; @@ -6480,6 +6353,7 @@ ib_grh_set_ver_class_flow( ver_class_flow = ver_class_flow | ver; return (ver_class_flow); } + /* * PARAMETERS * ver @@ -6512,24 +6386,23 @@ ib_grh_set_ver_class_flow( * SYNOPSIS */ #include -typedef struct _ib_member_rec -{ - ib_gid_t mgid; - ib_gid_t port_gid; - ib_net32_t qkey; - ib_net16_t mlid; - uint8_t mtu; - uint8_t tclass; - ib_net16_t pkey; - uint8_t rate; - uint8_t pkt_life; - ib_net32_t sl_flow_hop; - uint8_t scope_state; - uint8_t proxy_join:1; - uint8_t reserved[2]; - uint8_t pad[4]; - -} PACK_SUFFIX ib_member_rec_t; +typedef struct _ib_member_rec { + ib_gid_t mgid; + ib_gid_t port_gid; + ib_net32_t qkey; + ib_net16_t mlid; + uint8_t mtu; + uint8_t tclass; + ib_net16_t pkey; + uint8_t rate; + uint8_t pkt_life; + ib_net32_t sl_flow_hop; + uint8_t scope_state; + uint8_t proxy_join:1; + uint8_t reserved[2]; + uint8_t pad[4]; + +} PACK_SUFFIX ib_member_rec_t; #include /* * FIELDS @@ -6581,27 +6454,27 @@ typedef struct _ib_member_rec * * SYNOPSIS */ -static inline void OSM_API -ib_member_get_sl_flow_hop( - IN const ib_net32_t sl_flow_hop, - OUT uint8_t* const p_sl, - OUT uint32_t* const p_flow_lbl, - OUT uint8_t* const p_hop ) +static inline void OSM_API +ib_member_get_sl_flow_hop(IN const ib_net32_t sl_flow_hop, + OUT uint8_t * const p_sl, + OUT uint32_t * const p_flow_lbl, + OUT uint8_t * const p_hop) { uint32_t tmp; tmp = cl_ntoh32(sl_flow_hop); if (p_hop) - *p_hop = (uint8_t)tmp; + *p_hop = (uint8_t) tmp; tmp >>= 8; if (p_flow_lbl) - *p_flow_lbl = (uint32_t)(tmp & 0xfffff); + *p_flow_lbl = (uint32_t) (tmp & 0xfffff); tmp >>= 20; if (p_sl) - *p_sl = (uint8_t)tmp; + *p_sl = (uint8_t) tmp; } + /* * PARAMETERS * sl_flow_hop @@ -6632,17 +6505,17 @@ ib_member_get_sl_flow_hop( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_member_set_sl_flow_hop( - IN const uint8_t sl, - IN const uint32_t flow_label, - IN const uint8_t hop_limit ) +static inline ib_net32_t OSM_API +ib_member_set_sl_flow_hop(IN const uint8_t sl, + IN const uint32_t flow_label, + IN const uint8_t hop_limit) { uint32_t tmp; tmp = (sl << 28) | ((flow_label & 0xfffff) << 8) | hop_limit; return cl_hton32(tmp); } + /* * PARAMETERS * sl @@ -6673,23 +6546,23 @@ ib_member_set_sl_flow_hop( * * SYNOPSIS */ -static inline void OSM_API -ib_member_get_scope_state( - IN const uint8_t scope_state, - OUT uint8_t* const p_scope, - OUT uint8_t* const p_state ) +static inline void OSM_API +ib_member_get_scope_state(IN const uint8_t scope_state, + OUT uint8_t * const p_scope, + OUT uint8_t * const p_state) { - uint8_t tmp_scope_state; + uint8_t tmp_scope_state; if (p_state) - *p_state = (uint8_t)(scope_state & 0x0f); + *p_state = (uint8_t) (scope_state & 0x0f); tmp_scope_state = scope_state >> 4; if (p_scope) - *p_scope = (uint8_t)(tmp_scope_state & 0x0f); + *p_scope = (uint8_t) (tmp_scope_state & 0x0f); } + /* * PARAMETERS * scope_state @@ -6717,18 +6590,17 @@ ib_member_get_scope_state( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_member_set_scope_state( - IN const uint8_t scope, - IN const uint8_t state ) +static inline uint8_t OSM_API +ib_member_set_scope_state(IN const uint8_t scope, IN const uint8_t state) { - uint8_t scope_state; + uint8_t scope_state; scope_state = scope; scope_state = scope_state << 4; scope_state = scope_state | state; return (scope_state); } + /* * PARAMETERS * scope @@ -6756,14 +6628,14 @@ ib_member_set_scope_state( * * SYNOPSIS */ -static inline void OSM_API -ib_member_set_join_state( - IN OUT ib_member_rec_t *p_mc_rec, - IN const uint8_t state ) +static inline void OSM_API +ib_member_set_join_state(IN OUT ib_member_rec_t * p_mc_rec, + IN const uint8_t state) { /* keep the scope as it is */ p_mc_rec->scope_state = (p_mc_rec->scope_state & 0xF0) | (0x0f & state); } + /* * PARAMETERS * p_mc_rec @@ -6799,115 +6671,111 @@ ib_member_set_join_state( #define IB_NOTICE_TYPE_EMPTY 0x7F #include -typedef struct _ib_mad_notice_attr // Total Size calc Accumulated -{ - uint8_t generic_type; // 1 1 - - union _notice_g_or_v - { - struct _notice_generic // 5 6 - { - uint8_t prod_type_msb; - ib_net16_t prod_type_lsb; - ib_net16_t trap_num; - } PACK_SUFFIX generic; - - struct _notice_vend - { - uint8_t vend_id_msb; - ib_net16_t vend_id_lsb; - ib_net16_t dev_id; - } PACK_SUFFIX vend; - } g_or_v; - - ib_net16_t issuer_lid; // 2 8 - ib_net16_t toggle_count; // 2 10 - - union _data_details // 54 64 +typedef struct _ib_mad_notice_attr // Total Size calc Accumulated +{ + uint8_t generic_type; // 1 1 + + union _notice_g_or_v { + struct _notice_generic // 5 6 + { + uint8_t prod_type_msb; + ib_net16_t prod_type_lsb; + ib_net16_t trap_num; + } PACK_SUFFIX generic; + + struct _notice_vend { + uint8_t vend_id_msb; + ib_net16_t vend_id_lsb; + ib_net16_t dev_id; + } PACK_SUFFIX vend; + } g_or_v; + + ib_net16_t issuer_lid; // 2 8 + ib_net16_t toggle_count; // 2 10 + + union _data_details // 54 64 { - struct _raw_data - { - uint8_t details[54]; - } PACK_SUFFIX raw_data; - - struct _ntc_64_67 - { - uint8_t res[6]; - ib_gid_t gid; // the Node or Multicast Group that came in/out - } PACK_SUFFIX ntc_64_67; - - struct _ntc_128 { - ib_net16_t sw_lid; // the sw lid of which link state changed - } PACK_SUFFIX ntc_128; - - struct _ntc_129_131 { - ib_net16_t pad; - ib_net16_t lid; // lid and port number of the violation - uint8_t port_num; - } PACK_SUFFIX ntc_129_131; - - struct _ntc_144 { - ib_net16_t pad1; - ib_net16_t lid; // lid where capability mask changed - ib_net16_t pad2; - ib_net32_t new_cap_mask; // new capability mask - } PACK_SUFFIX ntc_144; - - struct _ntc_145 { - ib_net16_t pad1; - ib_net16_t lid; // lid where sys guid changed - ib_net16_t pad2; - ib_net64_t new_sys_guid; // new system image guid - } PACK_SUFFIX ntc_145; - - struct _ntc_256 { // total: 54 - ib_net16_t pad1; // 2 - ib_net16_t lid; // 2 - ib_net16_t pad2; // 2 - uint8_t method; // 1 - uint8_t pad3; // 1 - ib_net16_t attr_id; // 2 - ib_net32_t attr_mod; // 4 - ib_net64_t mkey; // 8 - uint8_t dr_slid; // 1 - uint8_t dr_trunc_hop; // 1 - uint8_t dr_rtn_path[30]; // 30 - } PACK_SUFFIX ntc_256; - - struct _ntc_257_258 // violation of p/q_key // 49 - { - ib_net16_t pad1; // 2 - ib_net16_t lid1; // 2 - ib_net16_t lid2; // 2 - ib_net32_t key; // 2 - uint8_t sl; // 1 - ib_net32_t qp1; // 4 - ib_net32_t qp2; // 4 - ib_gid_t gid1; // 16 - ib_gid_t gid2; // 16 - } PACK_SUFFIX ntc_257_258; - - struct _ntc_259 // p/q_key violation with sw info 53 - { - ib_net16_t data_valid; // 2 - ib_net16_t lid1; // 2 - ib_net16_t lid2; // 2 - ib_net32_t key; // 4 - uint8_t sl; // 1 - ib_net32_t qp1; // 4 - uint8_t qp2_msb; // 1 - ib_net16_t qp2_lsb; // 2 - ib_gid_t gid1; // 16 - ib_gid_t gid2; // 16 - ib_net16_t sw_lid; // 2 - uint8_t port_no; // 1 - } PACK_SUFFIX ntc_259; + struct _raw_data { + uint8_t details[54]; + } PACK_SUFFIX raw_data; + + struct _ntc_64_67 { + uint8_t res[6]; + ib_gid_t gid; // the Node or Multicast Group that came in/out + } PACK_SUFFIX ntc_64_67; + + struct _ntc_128 { + ib_net16_t sw_lid; // the sw lid of which link state changed + } PACK_SUFFIX ntc_128; + + struct _ntc_129_131 { + ib_net16_t pad; + ib_net16_t lid; // lid and port number of the violation + uint8_t port_num; + } PACK_SUFFIX ntc_129_131; + + struct _ntc_144 { + ib_net16_t pad1; + ib_net16_t lid; // lid where capability mask changed + ib_net16_t pad2; + ib_net32_t new_cap_mask; // new capability mask + } PACK_SUFFIX ntc_144; + + struct _ntc_145 { + ib_net16_t pad1; + ib_net16_t lid; // lid where sys guid changed + ib_net16_t pad2; + ib_net64_t new_sys_guid; // new system image guid + } PACK_SUFFIX ntc_145; + + struct _ntc_256 { // total: 54 + ib_net16_t pad1; // 2 + ib_net16_t lid; // 2 + ib_net16_t pad2; // 2 + uint8_t method; // 1 + uint8_t pad3; // 1 + ib_net16_t attr_id; // 2 + ib_net32_t attr_mod; // 4 + ib_net64_t mkey; // 8 + uint8_t dr_slid; // 1 + uint8_t dr_trunc_hop; // 1 + uint8_t dr_rtn_path[30]; // 30 + } PACK_SUFFIX ntc_256; + + struct _ntc_257_258 // violation of p/q_key // 49 + { + ib_net16_t pad1; // 2 + ib_net16_t lid1; // 2 + ib_net16_t lid2; // 2 + ib_net32_t key; // 2 + uint8_t sl; // 1 + ib_net32_t qp1; // 4 + ib_net32_t qp2; // 4 + ib_gid_t gid1; // 16 + ib_gid_t gid2; // 16 + } PACK_SUFFIX ntc_257_258; + + struct _ntc_259 // p/q_key violation with sw info 53 + { + ib_net16_t data_valid; // 2 + ib_net16_t lid1; // 2 + ib_net16_t lid2; // 2 + ib_net32_t key; // 4 + uint8_t sl; // 1 + ib_net32_t qp1; // 4 + uint8_t qp2_msb; // 1 + ib_net16_t qp2_lsb; // 2 + ib_gid_t gid1; // 16 + ib_gid_t gid2; // 16 + ib_net16_t sw_lid; // 2 + uint8_t port_no; // 1 + } PACK_SUFFIX ntc_259; } data_details; - ib_gid_t issuer_gid; // 16 80 + ib_gid_t issuer_gid; // 16 80 -} PACK_SUFFIX ib_mad_notice_attr_t; +} PACK_SUFFIX ib_mad_notice_attr_t; #include /****f* IBA Base: Types/ib_notice_is_generic @@ -6919,12 +6787,12 @@ typedef struct _ib_mad_notice_attr // Total Size calc Accumulated * * SYNOPSIS */ -static inline boolean_t OSM_API -ib_notice_is_generic( - IN const ib_mad_notice_attr_t *p_ntc ) +static inline boolean_t OSM_API +ib_notice_is_generic(IN const ib_mad_notice_attr_t * p_ntc) { - return (p_ntc->generic_type & 0x80); + return (p_ntc->generic_type & 0x80); } + /* * PARAMETERS * p_ntc @@ -6946,12 +6814,12 @@ ib_notice_is_generic( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_notice_get_type( - IN const ib_mad_notice_attr_t *p_ntc ) +static inline uint8_t OSM_API +ib_notice_get_type(IN const ib_mad_notice_attr_t * p_ntc) { - return p_ntc->generic_type & 0x7f; + return p_ntc->generic_type & 0x7f; } + /* * PARAMETERS * p_ntc @@ -6973,16 +6841,16 @@ ib_notice_get_type( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_notice_get_prod_type( - IN const ib_mad_notice_attr_t *p_ntc ) +static inline ib_net32_t OSM_API +ib_notice_get_prod_type(IN const ib_mad_notice_attr_t * p_ntc) { - uint32_t pt; + uint32_t pt; - pt = cl_ntoh16(p_ntc->g_or_v.generic.prod_type_lsb) | - (p_ntc->g_or_v.generic.prod_type_msb << 16); - return cl_hton32(pt); + pt = cl_ntoh16(p_ntc->g_or_v.generic.prod_type_lsb) | + (p_ntc->g_or_v.generic.prod_type_msb << 16); + return cl_hton32(pt); } + /* * PARAMETERS * p_ntc @@ -7004,15 +6872,17 @@ ib_notice_get_prod_type( * * SYNOPSIS */ -static inline void OSM_API -ib_notice_set_prod_type( - IN ib_mad_notice_attr_t *p_ntc, - IN ib_net32_t prod_type_val ) -{ - uint32_t ptv = cl_ntoh32(prod_type_val); - p_ntc->g_or_v.generic.prod_type_lsb = cl_hton16((uint16_t)(ptv & 0x0000ffff)); - p_ntc->g_or_v.generic.prod_type_msb = (uint8_t)( (ptv & 0x00ff0000) >> 16); +static inline void OSM_API +ib_notice_set_prod_type(IN ib_mad_notice_attr_t * p_ntc, + IN ib_net32_t prod_type_val) +{ + uint32_t ptv = cl_ntoh32(prod_type_val); + p_ntc->g_or_v.generic.prod_type_lsb = + cl_hton16((uint16_t) (ptv & 0x0000ffff)); + p_ntc->g_or_v.generic.prod_type_msb = + (uint8_t) ((ptv & 0x00ff0000) >> 16); } + /* * PARAMETERS * p_ntc @@ -7037,16 +6907,16 @@ ib_notice_set_prod_type( * * SYNOPSIS */ -static inline void OSM_API -ib_notice_set_prod_type_ho( - IN ib_mad_notice_attr_t *p_ntc, - IN uint32_t prod_type_val_ho ) +static inline void OSM_API +ib_notice_set_prod_type_ho(IN ib_mad_notice_attr_t * p_ntc, + IN uint32_t prod_type_val_ho) { - p_ntc->g_or_v.generic.prod_type_lsb = - cl_hton16( (uint16_t)(prod_type_val_ho & 0x0000ffff) ); - p_ntc->g_or_v.generic.prod_type_msb = - (uint8_t)( (prod_type_val_ho & 0x00ff0000) >> 16); + p_ntc->g_or_v.generic.prod_type_lsb = + cl_hton16((uint16_t) (prod_type_val_ho & 0x0000ffff)); + p_ntc->g_or_v.generic.prod_type_msb = + (uint8_t) ((prod_type_val_ho & 0x00ff0000) >> 16); } + /* * PARAMETERS * p_ntc @@ -7071,16 +6941,16 @@ ib_notice_set_prod_type_ho( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_notice_get_vend_id( - IN const ib_mad_notice_attr_t *p_ntc ) +static inline ib_net32_t OSM_API +ib_notice_get_vend_id(IN const ib_mad_notice_attr_t * p_ntc) { - uint32_t vi; + uint32_t vi; - vi = cl_ntoh16(p_ntc->g_or_v.vend.vend_id_lsb) | - (p_ntc->g_or_v.vend.vend_id_msb << 16); - return cl_hton32(vi); + vi = cl_ntoh16(p_ntc->g_or_v.vend.vend_id_lsb) | + (p_ntc->g_or_v.vend.vend_id_msb << 16); + return cl_hton32(vi); } + /* * PARAMETERS * p_ntc @@ -7102,15 +6972,15 @@ ib_notice_get_vend_id( * * SYNOPSIS */ -static inline void OSM_API -ib_notice_set_vend_id( - IN ib_mad_notice_attr_t *p_ntc, - IN ib_net32_t vend_id ) +static inline void OSM_API +ib_notice_set_vend_id(IN ib_mad_notice_attr_t * p_ntc, IN ib_net32_t vend_id) { - uint32_t vi = cl_ntoh32(vend_id); - p_ntc->g_or_v.vend.vend_id_lsb = cl_hton16((uint16_t)(vi & 0x0000ffff)); - p_ntc->g_or_v.vend.vend_id_msb = (uint8_t)((vi & 0x00ff0000) >> 16); + uint32_t vi = cl_ntoh32(vend_id); + p_ntc->g_or_v.vend.vend_id_lsb = + cl_hton16((uint16_t) (vi & 0x0000ffff)); + p_ntc->g_or_v.vend.vend_id_msb = (uint8_t) ((vi & 0x00ff0000) >> 16); } + /* * PARAMETERS * p_ntc @@ -7135,16 +7005,16 @@ ib_notice_set_vend_id( * * SYNOPSIS */ -static inline void OSM_API -ib_notice_set_vend_id_ho( - IN ib_mad_notice_attr_t *p_ntc, - IN uint32_t vend_id_ho ) +static inline void OSM_API +ib_notice_set_vend_id_ho(IN ib_mad_notice_attr_t * p_ntc, + IN uint32_t vend_id_ho) { - p_ntc->g_or_v.vend.vend_id_lsb = - cl_hton16((uint16_t)(vend_id_ho & 0x0000ffff)); - p_ntc->g_or_v.vend.vend_id_msb = - (uint8_t)((vend_id_ho & 0x00ff0000) >> 16); + p_ntc->g_or_v.vend.vend_id_lsb = + cl_hton16((uint16_t) (vend_id_ho & 0x0000ffff)); + p_ntc->g_or_v.vend.vend_id_msb = + (uint8_t) ((vend_id_ho & 0x00ff0000) >> 16); } + /* * PARAMETERS * p_ntc @@ -7161,38 +7031,34 @@ ib_notice_set_vend_id_ho( *********/ #include -typedef struct _ib_inform_info -{ - ib_gid_t gid; - ib_net16_t lid_range_begin; - ib_net16_t lid_range_end; - ib_net16_t reserved1; - uint8_t is_generic; - uint8_t subscribe; - ib_net16_t trap_type; - union _inform_g_or_v - { - struct _inform_generic - { - ib_net16_t trap_num; - ib_net32_t qpn_resp_time_val; - uint8_t reserved2; - uint8_t node_type_msb; - ib_net16_t node_type_lsb; - } PACK_SUFFIX generic; - - struct _inform_vend - { - ib_net16_t dev_id; - ib_net32_t qpn_resp_time_val; - uint8_t reserved2; - uint8_t vendor_id_msb; - ib_net16_t vendor_id_lsb; - } PACK_SUFFIX vend; - - } PACK_SUFFIX g_or_v; - -} PACK_SUFFIX ib_inform_info_t; +typedef struct _ib_inform_info { + ib_gid_t gid; + ib_net16_t lid_range_begin; + ib_net16_t lid_range_end; + ib_net16_t reserved1; + uint8_t is_generic; + uint8_t subscribe; + ib_net16_t trap_type; + union _inform_g_or_v { + struct _inform_generic { + ib_net16_t trap_num; + ib_net32_t qpn_resp_time_val; + uint8_t reserved2; + uint8_t node_type_msb; + ib_net16_t node_type_lsb; + } PACK_SUFFIX generic; + + struct _inform_vend { + ib_net16_t dev_id; + ib_net32_t qpn_resp_time_val; + uint8_t reserved2; + uint8_t vendor_id_msb; + ib_net16_t vendor_id_lsb; + } PACK_SUFFIX vend; + + } PACK_SUFFIX g_or_v; + +} PACK_SUFFIX ib_inform_info_t; #include /****f* IBA Base: Types/ib_inform_info_get_qpn_resp_time @@ -7204,20 +7070,20 @@ typedef struct _ib_inform_info * * SYNOPSIS */ -static inline void OSM_API -ib_inform_info_get_qpn_resp_time( - IN const ib_net32_t qpn_resp_time_val, - OUT ib_net32_t* const p_qpn, - OUT uint8_t* const p_resp_time_val ) +static inline void OSM_API +ib_inform_info_get_qpn_resp_time(IN const ib_net32_t qpn_resp_time_val, + OUT ib_net32_t * const p_qpn, + OUT uint8_t * const p_resp_time_val) { - uint32_t tmp = cl_ntoh32(qpn_resp_time_val); + uint32_t tmp = cl_ntoh32(qpn_resp_time_val); - if (p_qpn) - *p_qpn = cl_hton32((tmp & 0xffffff00) >> 8); + if (p_qpn) + *p_qpn = cl_hton32((tmp & 0xffffff00) >> 8); if (p_resp_time_val) - *p_resp_time_val = (uint8_t)(tmp & 0x0000001f); + *p_resp_time_val = (uint8_t) (tmp & 0x0000001f); } + /* * PARAMETERS * qpn_resp_time_val @@ -7245,19 +7111,16 @@ ib_inform_info_get_qpn_resp_time( * * SYNOPSIS */ -static inline void OSM_API -ib_inform_info_set_qpn( - IN ib_inform_info_t *p_ii, - IN ib_net32_t const qpn) +static inline void OSM_API +ib_inform_info_set_qpn(IN ib_inform_info_t * p_ii, IN ib_net32_t const qpn) { - uint32_t tmp = cl_ntoh32(p_ii->g_or_v.generic.qpn_resp_time_val); + uint32_t tmp = cl_ntoh32(p_ii->g_or_v.generic.qpn_resp_time_val); - p_ii->g_or_v.generic.qpn_resp_time_val = - cl_hton32( - (tmp & 0x000000ff) | - ((cl_ntoh32(qpn) << 8) & 0xffffff00) - ); + p_ii->g_or_v.generic.qpn_resp_time_val = + cl_hton32((tmp & 0x000000ff) | ((cl_ntoh32(qpn) << 8) & 0xffffff00) + ); } + /* * PARAMETERS * @@ -7277,16 +7140,16 @@ ib_inform_info_set_qpn( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_inform_info_get_prod_type( - IN const ib_inform_info_t *p_inf) +static inline ib_net32_t OSM_API +ib_inform_info_get_prod_type(IN const ib_inform_info_t * p_inf) { - uint32_t nt; + uint32_t nt; - nt = cl_ntoh16(p_inf->g_or_v.generic.node_type_lsb) | - (p_inf->g_or_v.generic.node_type_msb << 16); - return cl_hton32(nt); + nt = cl_ntoh16(p_inf->g_or_v.generic.node_type_lsb) | + (p_inf->g_or_v.generic.node_type_msb << 16); + return cl_hton32(nt); } + /* * PARAMETERS * p_inf @@ -7310,16 +7173,16 @@ ib_inform_info_get_prod_type( * * SYNOPSIS */ -static inline ib_net32_t OSM_API -ib_inform_info_get_vend_id( - IN const ib_inform_info_t *p_inf) +static inline ib_net32_t OSM_API +ib_inform_info_get_vend_id(IN const ib_inform_info_t * p_inf) { - uint32_t vi; + uint32_t vi; - vi = cl_ntoh16(p_inf->g_or_v.vend.vendor_id_lsb) | - (p_inf->g_or_v.vend.vendor_id_msb << 16); - return cl_hton32(vi); + vi = cl_ntoh16(p_inf->g_or_v.vend.vendor_id_lsb) | + (p_inf->g_or_v.vend.vendor_id_msb << 16); + return cl_hton32(vi); } + /* * PARAMETERS * p_inf @@ -7344,14 +7207,13 @@ ib_inform_info_get_vend_id( * SYNOPSIS */ #include -typedef struct _ib_inform_info_record -{ - ib_gid_t subscriber_gid; - ib_net16_t subscriber_enum; - uint8_t reserved[6]; - ib_inform_info_t inform_info; - uint8_t pad[4]; -} PACK_SUFFIX ib_inform_info_record_t; +typedef struct _ib_inform_info_record { + ib_gid_t subscriber_gid; + ib_net16_t subscriber_enum; + uint8_t reserved[6]; + ib_inform_info_t inform_info; + uint8_t pad[4]; +} PACK_SUFFIX ib_inform_info_record_t; #include /****s* IBA Base: Types/ib_perfmgt_mad_t @@ -7364,15 +7226,14 @@ typedef struct _ib_inform_info_record * SYNOPSIS */ #include -typedef struct _ib_perfmgt_mad -{ - ib_mad_t header; - uint8_t resv[40]; +typedef struct _ib_perfmgt_mad { + ib_mad_t header; + uint8_t resv[40]; #define IB_PM_DATA_SIZE 192 - uint8_t data[IB_PM_DATA_SIZE]; + uint8_t data[IB_PM_DATA_SIZE]; -} PACK_SUFFIX ib_perfmgt_mad_t; +} PACK_SUFFIX ib_perfmgt_mad_t; #include /* * FIELDS @@ -7400,29 +7261,28 @@ typedef struct _ib_perfmgt_mad * SYNOPSIS */ #include -typedef struct _ib_port_counters -{ - uint8_t reserved; - uint8_t port_select; - ib_net16_t counter_select; - ib_net16_t symbol_err_cnt; - uint8_t link_err_recover; - uint8_t link_downed; - ib_net16_t rcv_err; - ib_net16_t rcv_rem_phys_err; - ib_net16_t rcv_switch_relay_err; - ib_net16_t xmit_discards; - uint8_t xmit_constraint_err; - uint8_t rcv_constraint_err; - uint8_t res1; - uint8_t link_int_buffer_overrun; - ib_net16_t res2; - ib_net16_t vl15_dropped; - ib_net32_t xmit_data; - ib_net32_t rcv_data; - ib_net32_t xmit_pkts; - ib_net32_t rcv_pkts; -} PACK_SUFFIX ib_port_counters_t; +typedef struct _ib_port_counters { + uint8_t reserved; + uint8_t port_select; + ib_net16_t counter_select; + ib_net16_t symbol_err_cnt; + uint8_t link_err_recover; + uint8_t link_downed; + ib_net16_t rcv_err; + ib_net16_t rcv_rem_phys_err; + ib_net16_t rcv_switch_relay_err; + ib_net16_t xmit_discards; + uint8_t xmit_constraint_err; + uint8_t rcv_constraint_err; + uint8_t res1; + uint8_t link_int_buffer_overrun; + ib_net16_t res2; + ib_net16_t vl15_dropped; + ib_net32_t xmit_data; + ib_net32_t rcv_data; + ib_net32_t xmit_pkts; + ib_net32_t rcv_pkts; +} PACK_SUFFIX ib_port_counters_t; #include #define PC_LINK_INT(integ_buf_over) ((integ_buf_over & 0xF0) >> 4) @@ -7438,21 +7298,20 @@ typedef struct _ib_port_counters * SYNOPSIS */ #include -typedef struct _ib_port_counters_ext -{ - uint8_t reserved; - uint8_t port_select; - ib_net16_t counter_select; - ib_net32_t reserved2; - ib_net64_t xmit_data; - ib_net64_t rcv_data; - ib_net64_t xmit_pkts; - ib_net64_t rcv_pkts; - ib_net64_t unicast_xmit_pkts; - ib_net64_t unicast_rcv_pkts; - ib_net64_t multicast_xmit_pkts; - ib_net64_t multicast_rcv_pkts; -} PACK_SUFFIX ib_port_counters_ext_t; +typedef struct _ib_port_counters_ext { + uint8_t reserved; + uint8_t port_select; + ib_net16_t counter_select; + ib_net32_t reserved2; + ib_net64_t xmit_data; + ib_net64_t rcv_data; + ib_net64_t xmit_pkts; + ib_net64_t rcv_pkts; + ib_net64_t unicast_xmit_pkts; + ib_net64_t unicast_rcv_pkts; + ib_net64_t multicast_xmit_pkts; + ib_net64_t multicast_rcv_pkts; +} PACK_SUFFIX ib_port_counters_ext_t; #include /****s* IBA Base: Types/ib_port_samples_control @@ -7465,16 +7324,15 @@ typedef struct _ib_port_counters_ext * SYNOPSIS */ #include -typedef struct _ib_port_samples_control -{ - uint8_t op_code; - uint8_t port_select; - uint8_t tick; - uint8_t counter_width; /* 5 bits res : 3bits counter_width */ - ib_net32_t counter_mask; /* 2 bits res : 3 bits counter_mask : 27 bits counter_masks_1to9 */ - ib_net16_t counter_mask_10to14; /* 1 bits res : 15 bits counter_masks_10to14 */ - uint8_t sample_mech; - uint8_t sample_status; /* 6 bits res : 2 bits sample_status */ +typedef struct _ib_port_samples_control { + uint8_t op_code; + uint8_t port_select; + uint8_t tick; + uint8_t counter_width; /* 5 bits res : 3bits counter_width */ + ib_net32_t counter_mask; /* 2 bits res : 3 bits counter_mask : 27 bits counter_masks_1to9 */ + ib_net16_t counter_mask_10to14; /* 1 bits res : 15 bits counter_masks_10to14 */ + uint8_t sample_mech; + uint8_t sample_status; /* 6 bits res : 2 bits sample_status */ ib_net64_t option_mask; ib_net64_t vendor_mask; ib_net32_t sample_start; @@ -7523,10 +7381,9 @@ typedef struct _ib_port_samples_control * SYNOPSIS */ #include -typedef struct _ib_port_samples_result -{ +typedef struct _ib_port_samples_result { ib_net16_t tag; - ib_net16_t sample_status; /* 14 bits res : 2 bits sample_status */ + ib_net16_t sample_status; /* 14 bits res : 2 bits sample_status */ ib_net32_t counter0; ib_net32_t counter1; ib_net32_t counter2; @@ -7545,8 +7402,6 @@ typedef struct _ib_port_samples_result } PACK_SUFFIX ib_port_samples_result_t; #include - - /****d* IBA Base: Types/DM_SVC_NAME * NAME * DM_SVC_NAME @@ -7571,15 +7426,14 @@ typedef struct _ib_port_samples_result * SYNOPSIS */ #include -typedef struct _ib_dm_mad -{ - ib_mad_t header; - uint8_t resv[40]; +typedef struct _ib_dm_mad { + ib_mad_t header; + uint8_t resv[40]; #define IB_DM_DATA_SIZE 192 - uint8_t data[IB_DM_DATA_SIZE]; + uint8_t data[IB_DM_DATA_SIZE]; -} PACK_SUFFIX ib_dm_mad_t; +} PACK_SUFFIX ib_dm_mad_t; #include /* * FIELDS @@ -7607,21 +7461,20 @@ typedef struct _ib_dm_mad * SYNOPSIS */ #include -typedef struct _ib_iou_info -{ - ib_net16_t change_id; - uint8_t max_controllers; - uint8_t diag_rom; +typedef struct _ib_iou_info { + ib_net16_t change_id; + uint8_t max_controllers; + uint8_t diag_rom; #define IB_DM_CTRL_LIST_SIZE 128 - uint8_t controller_list[IB_DM_CTRL_LIST_SIZE]; + uint8_t controller_list[IB_DM_CTRL_LIST_SIZE]; #define IOC_NOT_INSTALLED 0x0 #define IOC_INSTALLED 0x1 -// Reserved values 0x02-0xE +// Reserved values 0x02-0xE #define SLOT_DOES_NOT_EXIST 0xF -} PACK_SUFFIX ib_iou_info_t; +} PACK_SUFFIX ib_iou_info_t; #include /* * FIELDS @@ -7654,12 +7507,12 @@ typedef struct _ib_iou_info * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_iou_info_diag_dev_id( - IN const ib_iou_info_t* const p_iou_info ) +static inline uint8_t OSM_API +ib_iou_info_diag_dev_id(IN const ib_iou_info_t * const p_iou_info) { - return( (uint8_t)(p_iou_info->diag_rom >> 6 & 1) ); + return ((uint8_t) (p_iou_info->diag_rom >> 6 & 1)); } + /* * PARAMETERS * p_iou_info @@ -7683,12 +7536,12 @@ ib_iou_info_diag_dev_id( * * SYNOPSIS */ -static inline uint8_t OSM_API -ib_iou_info_option_rom( - IN const ib_iou_info_t* const p_iou_info ) +static inline uint8_t OSM_API +ib_iou_info_option_rom(IN const ib_iou_info_t * const p_iou_info) { - return( (uint8_t)(p_iou_info->diag_rom >> 7) ); + return ((uint8_t) (p_iou_info->diag_rom >> 7)); } + /* * PARAMETERS * p_iou_info @@ -7712,17 +7565,18 @@ ib_iou_info_option_rom( * * SYNOPSIS */ -static inline uint8_t OSM_API -ioc_at_slot( - IN const ib_iou_info_t* const p_iou_info, - IN uint8_t slot ) -{ - if( slot >= IB_DM_CTRL_LIST_SIZE ) return SLOT_DOES_NOT_EXIST; - else return (int8_t) - ( (slot%2) ? - ((p_iou_info->controller_list[slot/2] & 0xf0) >> 4 ): - (p_iou_info->controller_list[slot/2] & 0x0f) ); +static inline uint8_t OSM_API +ioc_at_slot(IN const ib_iou_info_t * const p_iou_info, IN uint8_t slot) +{ + if (slot >= IB_DM_CTRL_LIST_SIZE) + return SLOT_DOES_NOT_EXIST; + else + return (int8_t) + ((slot % 2) ? + ((p_iou_info->controller_list[slot / 2] & 0xf0) >> 4) : + (p_iou_info->controller_list[slot / 2] & 0x0f)); } + /* * PARAMETERS * p_iou_info @@ -7750,32 +7604,31 @@ ioc_at_slot( * SYNOPSIS */ #include -typedef struct _ib_ioc_profile -{ - ib_net64_t ioc_guid; +typedef struct _ib_ioc_profile { + ib_net64_t ioc_guid; - ib_net32_t vend_id; + ib_net32_t vend_id; - ib_net32_t dev_id; - ib_net16_t dev_ver; - ib_net16_t resv2; + ib_net32_t dev_id; + ib_net16_t dev_ver; + ib_net16_t resv2; - ib_net32_t subsys_vend_id; - ib_net32_t subsys_id; + ib_net32_t subsys_vend_id; + ib_net32_t subsys_id; - ib_net16_t io_class; - ib_net16_t io_subclass; - ib_net16_t protocol; - ib_net16_t protocol_ver; + ib_net16_t io_class; + ib_net16_t io_subclass; + ib_net16_t protocol; + ib_net16_t protocol_ver; - ib_net32_t resv3; - ib_net16_t send_msg_depth; - uint8_t resv4; - uint8_t rdma_read_depth; - ib_net32_t send_msg_size; - ib_net32_t rdma_size; + ib_net32_t resv3; + ib_net16_t send_msg_depth; + uint8_t resv4; + uint8_t rdma_read_depth; + ib_net32_t send_msg_size; + ib_net32_t rdma_size; - uint8_t ctrl_ops_cap; + uint8_t ctrl_ops_cap; #define CTRL_OPS_CAP_ST 0x01 #define CTRL_OPS_CAP_SF 0x02 #define CTRL_OPS_CAP_RT 0x04 @@ -7785,17 +7638,17 @@ typedef struct _ib_ioc_profile #define CTRL_OPS_CAP_AT 0x40 #define CTRL_OPS_CAP_AF 0x80 - uint8_t resv5; + uint8_t resv5; - uint8_t num_svc_entries; + uint8_t num_svc_entries; #define MAX_NUM_SVC_ENTRIES 0xff - uint8_t resv6[9]; + uint8_t resv6[9]; #define CTRL_ID_STRING_LEN 64 - char id_string[CTRL_ID_STRING_LEN]; + char id_string[CTRL_ID_STRING_LEN]; -} PACK_SUFFIX ib_ioc_profile_t; +} PACK_SUFFIX ib_ioc_profile_t; #include /* * FIELDS @@ -7858,18 +7711,15 @@ typedef struct _ib_ioc_profile * ib_dm_mad_t *********/ -static inline uint32_t OSM_API -ib_ioc_profile_get_vend_id( - IN const ib_ioc_profile_t* const p_ioc_profile ) +static inline uint32_t OSM_API +ib_ioc_profile_get_vend_id(IN const ib_ioc_profile_t * const p_ioc_profile) { - return( cl_ntoh32(p_ioc_profile->vend_id) >> 8 ); + return (cl_ntoh32(p_ioc_profile->vend_id) >> 8); } - -static inline void OSM_API -ib_ioc_profile_set_vend_id( - IN ib_ioc_profile_t* const p_ioc_profile, - IN const uint32_t vend_id ) +static inline void OSM_API +ib_ioc_profile_set_vend_id(IN ib_ioc_profile_t * const p_ioc_profile, + IN const uint32_t vend_id) { p_ioc_profile->vend_id = (cl_hton32(vend_id) << 8); } @@ -7884,14 +7734,13 @@ ib_ioc_profile_set_vend_id( * SYNOPSIS */ #include -typedef struct _ib_svc_entry -{ +typedef struct _ib_svc_entry { #define MAX_SVC_ENTRY_NAME_LEN 40 - char name[MAX_SVC_ENTRY_NAME_LEN]; + char name[MAX_SVC_ENTRY_NAME_LEN]; - ib_net64_t id; + ib_net64_t id; -} PACK_SUFFIX ib_svc_entry_t; +} PACK_SUFFIX ib_svc_entry_t; #include /* * FIELDS @@ -7915,12 +7764,11 @@ typedef struct _ib_svc_entry * SYNOPSIS */ #include -typedef struct _ib_svc_entries -{ +typedef struct _ib_svc_entries { #define SVC_ENTRY_COUNT 4 - ib_svc_entry_t service_entry[SVC_ENTRY_COUNT]; + ib_svc_entry_t service_entry[SVC_ENTRY_COUNT]; -} PACK_SUFFIX ib_svc_entries_t; +} PACK_SUFFIX ib_svc_entries_t; #include /* * FIELDS @@ -7931,54 +7779,51 @@ typedef struct _ib_svc_entries * ib_dm_mad_t, ib_svc_entry_t *********/ -static inline void OSM_API -ib_dm_get_slot_lo_hi( - IN const ib_net32_t slot_lo_hi, - OUT uint8_t *const p_slot, - OUT uint8_t *const p_lo, - OUT uint8_t *const p_hi ) +static inline void OSM_API +ib_dm_get_slot_lo_hi(IN const ib_net32_t slot_lo_hi, + OUT uint8_t * const p_slot, + OUT uint8_t * const p_lo, OUT uint8_t * const p_hi) { - ib_net32_t tmp_slot_lo_hi = CL_NTOH32( slot_lo_hi ); + ib_net32_t tmp_slot_lo_hi = CL_NTOH32(slot_lo_hi); - if( p_slot ) - *p_slot = (uint8_t)( ( tmp_slot_lo_hi >> 16 ) & 0x0f ); + if (p_slot) + *p_slot = (uint8_t) ((tmp_slot_lo_hi >> 16) & 0x0f); - if( p_hi ) - *p_hi = (uint8_t)( ( tmp_slot_lo_hi >> 8 ) & 0xff ); + if (p_hi) + *p_hi = (uint8_t) ((tmp_slot_lo_hi >> 8) & 0xff); - if( p_lo ) - *p_lo = (uint8_t)( ( tmp_slot_lo_hi >> 0 ) & 0xff ); + if (p_lo) + *p_lo = (uint8_t) ((tmp_slot_lo_hi >> 0) & 0xff); } /* * IBA defined information describing an I/O controller */ #include -typedef struct _ib_ioc_info -{ - ib_net64_t module_guid; - ib_net64_t iou_guid; - ib_ioc_profile_t ioc_profile; - ib_net64_t access_key; - uint16_t initiators_conf; - uint8_t resv[38]; - -} PACK_SUFFIX ib_ioc_info_t; +typedef struct _ib_ioc_info { + ib_net64_t module_guid; + ib_net64_t iou_guid; + ib_ioc_profile_t ioc_profile; + ib_net64_t access_key; + uint16_t initiators_conf; + uint8_t resv[38]; + +} PACK_SUFFIX ib_ioc_info_t; #include /* * The following definitions are shared between the Access Layer and VPD */ -typedef struct _ib_ca* __ptr64 ib_ca_handle_t; -typedef struct _ib_pd* __ptr64 ib_pd_handle_t; -typedef struct _ib_rdd* __ptr64 ib_rdd_handle_t; -typedef struct _ib_mr* __ptr64 ib_mr_handle_t; -typedef struct _ib_mw* __ptr64 ib_mw_handle_t; -typedef struct _ib_qp* __ptr64 ib_qp_handle_t; -typedef struct _ib_eec* __ptr64 ib_eec_handle_t; -typedef struct _ib_cq* __ptr64 ib_cq_handle_t; -typedef struct _ib_av* __ptr64 ib_av_handle_t; -typedef struct _ib_mcast* __ptr64 ib_mcast_handle_t; +typedef struct _ib_ca *__ptr64 ib_ca_handle_t; +typedef struct _ib_pd *__ptr64 ib_pd_handle_t; +typedef struct _ib_rdd *__ptr64 ib_rdd_handle_t; +typedef struct _ib_mr *__ptr64 ib_mr_handle_t; +typedef struct _ib_mw *__ptr64 ib_mw_handle_t; +typedef struct _ib_qp *__ptr64 ib_qp_handle_t; +typedef struct _ib_eec *__ptr64 ib_eec_handle_t; +typedef struct _ib_cq *__ptr64 ib_cq_handle_t; +typedef struct _ib_av *__ptr64 ib_av_handle_t; +typedef struct _ib_mcast *__ptr64 ib_mcast_handle_t; /* Currently for windows branch, use the extended version of ib special verbs struct in order to be compliant with Infinicon ib_types; later we'll change it to support @@ -8000,8 +7845,7 @@ typedef struct _ib_mcast* __ptr64 ib_mcast_handle_t; * * SYNOPSIS */ -typedef enum _ib_api_status_t -{ +typedef enum _ib_api_status_t { IB_SUCCESS, IB_INSUFFICIENT_RESOURCES, IB_INSUFFICIENT_MEMORY, @@ -8042,22 +7886,21 @@ typedef enum _ib_api_status_t IB_INVALID_RDD_HANDLE, IB_INVALID_MCAST_HANDLE, IB_INVALID_CALLBACK, - IB_INVALID_AL_HANDLE, /* InfiniBand Access Layer */ - IB_INVALID_HANDLE, /* InfiniBand Access Layer */ - IB_ERROR, /* InfiniBand Access Layer */ - IB_REMOTE_ERROR, /* Infiniband Access Layer */ - IB_VERBS_PROCESSING_DONE, /* See Notes above */ + IB_INVALID_AL_HANDLE, /* InfiniBand Access Layer */ + IB_INVALID_HANDLE, /* InfiniBand Access Layer */ + IB_ERROR, /* InfiniBand Access Layer */ + IB_REMOTE_ERROR, /* Infiniband Access Layer */ + IB_VERBS_PROCESSING_DONE, /* See Notes above */ IB_INVALID_WR_TYPE, IB_QP_IN_TIMEWAIT, IB_EE_IN_TIMEWAIT, IB_INVALID_PORT, IB_NOT_DONE, - IB_UNKNOWN_ERROR /* ALWAYS LAST ENUM VALUE! */ - -} ib_api_status_t; + IB_UNKNOWN_ERROR /* ALWAYS LAST ENUM VALUE! */ +} ib_api_status_t; /*****/ -OSM_EXPORT const char* ib_error_str[]; +OSM_EXPORT const char *ib_error_str[]; /****f* IBA Base: Types/ib_get_err_str * NAME @@ -8068,14 +7911,13 @@ OSM_EXPORT const char* ib_error_str[]; * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_err_str( - IN ib_api_status_t status ) +static inline const char *OSM_API ib_get_err_str(IN ib_api_status_t status) { - if( status > IB_UNKNOWN_ERROR ) + if (status > IB_UNKNOWN_ERROR) status = IB_UNKNOWN_ERROR; - return( ib_error_str[status] ); + return (ib_error_str[status]); } + /* * PARAMETERS * status @@ -8101,8 +7943,7 @@ ib_get_err_str( * * SYNOPSIS */ -typedef enum _ib_async_event_t -{ +typedef enum _ib_async_event_t { IB_AE_SQ_ERROR = 1, IB_AE_SQ_DRAINED, IB_AE_RQ_ERROR, @@ -8130,8 +7971,7 @@ typedef enum _ib_async_event_t IB_AE_PORT_ACTIVE, IB_AE_PORT_DOWN, IB_AE_UNKNOWN /* ALWAYS LAST ENUM VALUE */ - -} ib_async_event_t; +} ib_async_event_t; /* * VALUES * IB_AE_SQ_ERROR @@ -8244,7 +8084,7 @@ typedef enum _ib_async_event_t * *****/ -OSM_EXPORT const char* ib_async_event_str[]; +OSM_EXPORT const char *ib_async_event_str[]; /****f* IBA Base: Types/ib_get_async_event_str * NAME @@ -8255,14 +8095,14 @@ OSM_EXPORT const char* ib_async_event_str[]; * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_async_event_str( - IN ib_async_event_t event ) +static inline const char *OSM_API +ib_get_async_event_str(IN ib_async_event_t event) { - if( event > IB_AE_UNKNOWN ) + if (event > IB_AE_UNKNOWN) event = IB_AE_UNKNOWN; - return( ib_async_event_str[event] ); + return (ib_async_event_str[event]); } + /* * PARAMETERS * event @@ -8293,52 +8133,47 @@ ib_get_async_event_str( * * SYNOPSIS */ -typedef struct _ib_event_rec -{ - void *context; - ib_async_event_t type; +typedef struct _ib_event_rec { + void *context; + ib_async_event_t type; /* HCA vendor specific event information. */ - uint64_t vendor_specific; + uint64_t vendor_specific; /* The following structures are valid only for trap types. */ - union _trap - { - struct - { - uint16_t lid; - ib_net64_t port_guid; - uint8_t port_num; + union _trap { + struct { + uint16_t lid; + ib_net64_t port_guid; + uint8_t port_num; /* * The following structure is valid only for * P_KEY, Q_KEY, and M_KEY violation traps. */ - struct - { - uint8_t sl; - uint16_t src_lid; - uint16_t dest_lid; - union _key - { - uint16_t pkey; - uint32_t qkey; - uint64_t mkey; + struct { + uint8_t sl; + uint16_t src_lid; + uint16_t dest_lid; + union _key { + uint16_t pkey; + uint32_t qkey; + uint64_t mkey; } key; - uint32_t src_qp; - uint32_t dest_qp; - ib_gid_t src_gid; - ib_gid_t dest_gid; + uint32_t src_qp; + uint32_t dest_qp; + ib_gid_t src_gid; + ib_gid_t dest_gid; - } violation; + } violation; } info; - ib_net64_t sysimg_guid; + ib_net64_t sysimg_guid; - } trap; + } trap; -} ib_event_rec_t; +} ib_event_rec_t; /*******/ /****d* Access Layer/ib_atomic_t @@ -8350,13 +8185,11 @@ typedef struct _ib_event_rec * * SYNOPSIS */ -typedef enum _ib_atomic_t -{ +typedef enum _ib_atomic_t { IB_ATOMIC_NONE, IB_ATOMIC_LOCAL, IB_ATOMIC_GLOBAL - -} ib_atomic_t; +} ib_atomic_t; /* * VALUES * IB_ATOMIC_NONE @@ -8380,31 +8213,30 @@ typedef enum _ib_atomic_t * * SYNOPSIS */ -typedef struct _ib_port_cap -{ - boolean_t cm; - boolean_t snmp; - boolean_t dev_mgmt; - boolean_t vend; - boolean_t sm; - boolean_t sm_disable; - boolean_t qkey_ctr; - boolean_t pkey_ctr; - boolean_t notice; - boolean_t trap; - boolean_t apm; - boolean_t slmap; - boolean_t pkey_nvram; - boolean_t mkey_nvram; - boolean_t sysguid; - boolean_t dr_notice; - boolean_t boot_mgmt; - boolean_t capm_notice; - boolean_t reinit; - boolean_t ledinfo; - boolean_t port_active; - -} ib_port_cap_t; +typedef struct _ib_port_cap { + boolean_t cm; + boolean_t snmp; + boolean_t dev_mgmt; + boolean_t vend; + boolean_t sm; + boolean_t sm_disable; + boolean_t qkey_ctr; + boolean_t pkey_ctr; + boolean_t notice; + boolean_t trap; + boolean_t apm; + boolean_t slmap; + boolean_t pkey_nvram; + boolean_t mkey_nvram; + boolean_t sysguid; + boolean_t dr_notice; + boolean_t boot_mgmt; + boolean_t capm_notice; + boolean_t reinit; + boolean_t ledinfo; + boolean_t port_active; + +} ib_port_cap_t; /*****/ /****d* Access Layer/ib_init_type_t @@ -8420,7 +8252,7 @@ typedef struct _ib_port_cap * * SYNOPSIS */ -typedef uint8_t ib_init_type_t; +typedef uint8_t ib_init_type_t; #define IB_INIT_TYPE_NO_LOAD 0x01 #define IB_INIT_TYPE_PRESERVE_CONTENT 0x02 #define IB_INIT_TYPE_PRESERVE_PRESENCE 0x04 @@ -8436,16 +8268,15 @@ typedef uint8_t ib_init_type_t; * * SYNOPSIS */ -typedef struct _ib_port_attr_mod -{ - ib_port_cap_t cap; - uint16_t pkey_ctr; - uint16_t qkey_ctr; +typedef struct _ib_port_attr_mod { + ib_port_cap_t cap; + uint16_t pkey_ctr; + uint16_t qkey_ctr; - ib_init_type_t init_type; - ib_net64_t system_image_guid; + ib_init_type_t init_type; + ib_net64_t system_image_guid; -} ib_port_attr_mod_t; +} ib_port_attr_mod_t; /* * SEE ALSO * ib_port_cap_t @@ -8460,28 +8291,27 @@ typedef struct _ib_port_attr_mod * * SYNOPSIS */ -typedef struct _ib_port_attr -{ - ib_net64_t port_guid; - uint8_t port_num; - uint8_t mtu; - uint64_t max_msg_size; - ib_net16_t lid; - uint8_t lmc; +typedef struct _ib_port_attr { + ib_net64_t port_guid; + uint8_t port_num; + uint8_t mtu; + uint64_t max_msg_size; + ib_net16_t lid; + uint8_t lmc; /* * LinkWidthSupported as defined in PortInfo. Required to calculate * inter-packet delay (a.k.a. static rate). */ - uint8_t link_width_supported; + uint8_t link_width_supported; - uint16_t max_vls; + uint16_t max_vls; - ib_net16_t sm_lid; - uint8_t sm_sl; - uint8_t link_state; + ib_net16_t sm_lid; + uint8_t sm_sl; + uint8_t link_state; - ib_init_type_t init_type_reply; /* Optional */ + ib_init_type_t init_type_reply; /* Optional */ /* * subnet_timeout: @@ -8491,22 +8321,22 @@ typedef struct _ib_port_attr * * timeout = 4.096 microseconds * 2^subnet_timeout */ - uint8_t subnet_timeout; + uint8_t subnet_timeout; - ib_port_cap_t cap; - uint16_t pkey_ctr; - uint16_t qkey_ctr; + ib_port_cap_t cap; + uint16_t pkey_ctr; + uint16_t qkey_ctr; - uint16_t num_gids; - uint16_t num_pkeys; + uint16_t num_gids; + uint16_t num_pkeys; /* * Pointers at the end of the structure to allow doing a simple * memory comparison of contents up to the first pointer. */ - ib_gid_t *p_gid_table; - ib_net16_t *p_pkey_table; + ib_gid_t *p_gid_table; + ib_net16_t *p_pkey_table; -} ib_port_attr_t; +} ib_port_attr_t; /* * SEE ALSO * uint8_t, ib_port_cap_t, ib_link_states_t @@ -8521,58 +8351,57 @@ typedef struct _ib_port_attr * * SYNOPSIS */ -typedef struct _ib_ca_attr -{ - ib_net64_t ca_guid; +typedef struct _ib_ca_attr { + ib_net64_t ca_guid; - uint32_t vend_id; - uint16_t dev_id; - uint16_t revision; - uint64_t fw_ver; + uint32_t vend_id; + uint16_t dev_id; + uint16_t revision; + uint64_t fw_ver; /* * Total size of the ca attributes in bytes */ - uint32_t size; - uint32_t max_qps; - uint32_t max_wrs; + uint32_t size; + uint32_t max_qps; + uint32_t max_wrs; - uint32_t max_sges; - uint32_t max_rd_sges; + uint32_t max_sges; + uint32_t max_rd_sges; - uint32_t max_cqs; - uint32_t max_cqes; + uint32_t max_cqs; + uint32_t max_cqes; - uint32_t max_pds; + uint32_t max_pds; - uint32_t init_regions; - uint64_t init_region_size; + uint32_t init_regions; + uint64_t init_region_size; - uint32_t init_windows; - uint32_t max_addr_handles; + uint32_t init_windows; + uint32_t max_addr_handles; - uint32_t max_partitions; + uint32_t max_partitions; - ib_atomic_t atomicity; + ib_atomic_t atomicity; - uint8_t max_qp_resp_res; - uint8_t max_eec_resp_res; - uint8_t max_resp_res; + uint8_t max_qp_resp_res; + uint8_t max_eec_resp_res; + uint8_t max_resp_res; - uint8_t max_qp_init_depth; - uint8_t max_eec_init_depth; + uint8_t max_qp_init_depth; + uint8_t max_eec_init_depth; - uint32_t max_eecs; - uint32_t max_rdds; + uint32_t max_eecs; + uint32_t max_rdds; - uint32_t max_ipv6_qps; - uint32_t max_ether_qps; + uint32_t max_ipv6_qps; + uint32_t max_ether_qps; - uint32_t max_mcast_grps; - uint32_t max_mcast_qps; - uint32_t max_qps_per_mcast_grp; - uint32_t max_fmr; - uint32_t max_map_per_fmr; + uint32_t max_mcast_grps; + uint32_t max_mcast_qps; + uint32_t max_qps_per_mcast_grp; + uint32_t max_fmr; + uint32_t max_map_per_fmr; /* * local_ack_delay: @@ -8581,31 +8410,31 @@ typedef struct _ib_ca_attr * * timeout = 4.096 microseconds * 2^local_ack_delay */ - uint8_t local_ack_delay; + uint8_t local_ack_delay; - boolean_t bad_pkey_ctr_support; - boolean_t bad_qkey_ctr_support; - boolean_t raw_mcast_support; - boolean_t apm_support; - boolean_t av_port_check; - boolean_t change_primary_port; - boolean_t modify_wr_depth; - boolean_t current_qp_state_support; - boolean_t shutdown_port_capability; - boolean_t init_type_support; - boolean_t port_active_event_support; - boolean_t system_image_guid_support; - boolean_t hw_agents; + boolean_t bad_pkey_ctr_support; + boolean_t bad_qkey_ctr_support; + boolean_t raw_mcast_support; + boolean_t apm_support; + boolean_t av_port_check; + boolean_t change_primary_port; + boolean_t modify_wr_depth; + boolean_t current_qp_state_support; + boolean_t shutdown_port_capability; + boolean_t init_type_support; + boolean_t port_active_event_support; + boolean_t system_image_guid_support; + boolean_t hw_agents; - ib_net64_t system_image_guid; + ib_net64_t system_image_guid; - uint32_t num_page_sizes; - uint8_t num_ports; + uint32_t num_page_sizes; + uint8_t num_ports; - uint32_t *p_page_size; - ib_port_attr_t *p_port_attr; + uint32_t *p_page_size; + ib_port_attr_t *p_port_attr; -} ib_ca_attr_t; +} ib_ca_attr_t; /* * FIELDS * ca_guid @@ -8786,10 +8615,8 @@ typedef struct _ib_ca_attr * * SYNOPSIS */ -ib_ca_attr_t* -ib_copy_ca_attr( - IN ib_ca_attr_t* const p_dest, - IN const ib_ca_attr_t* const p_src ); +ib_ca_attr_t *ib_copy_ca_attr(IN ib_ca_attr_t * const p_dest, + IN const ib_ca_attr_t * const p_src); /* * PARAMETERS * p_dest @@ -8818,28 +8645,26 @@ ib_copy_ca_attr( * * SYNOPSIS */ -typedef struct _ib_av_attr -{ - uint8_t port_num; +typedef struct _ib_av_attr { + uint8_t port_num; - uint8_t sl; - ib_net16_t dlid; + uint8_t sl; + ib_net16_t dlid; - boolean_t grh_valid; - ib_grh_t grh; - uint8_t static_rate; - uint8_t path_bits; + boolean_t grh_valid; + ib_grh_t grh; + uint8_t static_rate; + uint8_t path_bits; - struct _av_conn - { - uint8_t path_mtu; - uint8_t local_ack_timeout; - uint8_t seq_err_retry_cnt; - uint8_t rnr_retry_cnt; + struct _av_conn { + uint8_t path_mtu; + uint8_t local_ack_timeout; + uint8_t seq_err_retry_cnt; + uint8_t rnr_retry_cnt; - } conn; + } conn; -} ib_av_attr_t; +} ib_av_attr_t; /* * SEE ALSO * ib_gid_t @@ -8854,21 +8679,19 @@ typedef struct _ib_av_attr * * SYNOPSIS */ -typedef enum _ib_qp_type -{ - IB_QPT_RELIABLE_CONN = 0, /* Matches CM REQ transport type */ - IB_QPT_UNRELIABLE_CONN = 1, /* Matches CM REQ transport type */ - IB_QPT_RELIABLE_DGRM = 2, /* Matches CM REQ transport type */ +typedef enum _ib_qp_type { + IB_QPT_RELIABLE_CONN = 0, /* Matches CM REQ transport type */ + IB_QPT_UNRELIABLE_CONN = 1, /* Matches CM REQ transport type */ + IB_QPT_RELIABLE_DGRM = 2, /* Matches CM REQ transport type */ IB_QPT_UNRELIABLE_DGRM, IB_QPT_QP0, IB_QPT_QP1, IB_QPT_RAW_IPV6, IB_QPT_RAW_ETHER, - IB_QPT_MAD, /* InfiniBand Access Layer */ - IB_QPT_QP0_ALIAS, /* InfiniBand Access Layer */ - IB_QPT_QP1_ALIAS /* InfiniBand Access Layer */ - -} ib_qp_type_t; + IB_QPT_MAD, /* InfiniBand Access Layer */ + IB_QPT_QP0_ALIAS, /* InfiniBand Access Layer */ + IB_QPT_QP1_ALIAS /* InfiniBand Access Layer */ +} ib_qp_type_t; /* * VALUES * IB_QPT_RELIABLE_CONN @@ -8921,7 +8744,7 @@ typedef enum _ib_qp_type * * SYNOPSIS */ -typedef uint32_t ib_access_t; +typedef uint32_t ib_access_t; #define IB_AC_RDMA_READ 0x00000001 #define IB_AC_RDMA_WRITE 0x00000002 #define IB_AC_ATOMIC 0x00000004 @@ -8945,7 +8768,7 @@ typedef uint32_t ib_access_t; * * SYNOPSIS */ -typedef uint32_t ib_qp_state_t; +typedef uint32_t ib_qp_state_t; #define IB_QPS_RESET 0x00000001 #define IB_QPS_INIT 0x00000002 #define IB_QPS_RTR 0x00000004 @@ -8967,13 +8790,11 @@ typedef uint32_t ib_qp_state_t; * * SYNOPSIS */ -typedef enum _ib_apm_state -{ +typedef enum _ib_apm_state { IB_APM_MIGRATED = 1, IB_APM_REARM, IB_APM_ARMED - -} ib_apm_state_t; +} ib_apm_state_t; /*****/ /****s* Access Layer/ib_qp_create_t @@ -8985,23 +8806,22 @@ typedef enum _ib_apm_state * * SYNOPSIS */ -typedef struct _ib_qp_create -{ - ib_qp_type_t qp_type; +typedef struct _ib_qp_create { + ib_qp_type_t qp_type; - ib_rdd_handle_t h_rdd; + ib_rdd_handle_t h_rdd; - uint32_t sq_depth; - uint32_t rq_depth; - uint32_t sq_sge; - uint32_t rq_sge; + uint32_t sq_depth; + uint32_t rq_depth; + uint32_t sq_sge; + uint32_t rq_sge; - ib_cq_handle_t h_sq_cq; - ib_cq_handle_t h_rq_cq; + ib_cq_handle_t h_sq_cq; + ib_cq_handle_t h_rq_cq; - boolean_t sq_signaled; + boolean_t sq_signaled; -} ib_qp_create_t; +} ib_qp_create_t; /* * FIELDS * type @@ -9067,41 +8887,40 @@ typedef struct _ib_qp_create * * SYNOPSIS */ -typedef struct _ib_qp_attr -{ - ib_pd_handle_t h_pd; - ib_qp_type_t qp_type; - ib_access_t access_ctrl; - uint16_t pkey_index; +typedef struct _ib_qp_attr { + ib_pd_handle_t h_pd; + ib_qp_type_t qp_type; + ib_access_t access_ctrl; + uint16_t pkey_index; - uint32_t sq_depth; - uint32_t rq_depth; - uint32_t sq_sge; - uint32_t rq_sge; - uint8_t init_depth; - uint8_t resp_res; + uint32_t sq_depth; + uint32_t rq_depth; + uint32_t sq_sge; + uint32_t rq_sge; + uint8_t init_depth; + uint8_t resp_res; - ib_cq_handle_t h_sq_cq; - ib_cq_handle_t h_rq_cq; - ib_rdd_handle_t h_rdd; + ib_cq_handle_t h_sq_cq; + ib_cq_handle_t h_rq_cq; + ib_rdd_handle_t h_rdd; - boolean_t sq_signaled; + boolean_t sq_signaled; - ib_qp_state_t state; - ib_net32_t num; - ib_net32_t dest_num; - ib_net32_t qkey; + ib_qp_state_t state; + ib_net32_t num; + ib_net32_t dest_num; + ib_net32_t qkey; - ib_net32_t sq_psn; - ib_net32_t rq_psn; + ib_net32_t sq_psn; + ib_net32_t rq_psn; - uint8_t primary_port; - uint8_t alternate_port; - ib_av_attr_t primary_av; - ib_av_attr_t alternate_av; - ib_apm_state_t apm_state; + uint8_t primary_port; + uint8_t alternate_port; + ib_av_attr_t primary_av; + ib_av_attr_t alternate_av; + ib_apm_state_t apm_state; -} ib_qp_attr_t; +} ib_qp_attr_t; /* * FIELDS * h_pd @@ -9124,7 +8943,7 @@ typedef struct _ib_qp_attr * * SYNOPSIS */ -typedef uint32_t ib_qp_opts_t; +typedef uint32_t ib_qp_opts_t; #define IB_MOD_QP_ALTERNATE_AV 0x00000001 #define IB_MOD_QP_PKEY 0x00000002 #define IB_MOD_QP_APM_STATE 0x00000004 @@ -9156,86 +8975,79 @@ typedef uint32_t ib_qp_opts_t; * * SYNOPSIS */ -typedef struct _ib_qp_mod -{ - ib_qp_state_t req_state; +typedef struct _ib_qp_mod { + ib_qp_state_t req_state; - union _qp_state - { - struct _qp_reset - { + union _qp_state { + struct _qp_reset { /* * Time, in milliseconds, that the QP needs to spend in * the time wait state before being reused. */ - uint32_t timewait; + uint32_t timewait; - } reset; + } reset; - struct _qp_init - { - ib_qp_opts_t opts; - uint8_t primary_port; - ib_net32_t qkey; - uint16_t pkey_index; - ib_access_t access_ctrl; + struct _qp_init { + ib_qp_opts_t opts; + uint8_t primary_port; + ib_net32_t qkey; + uint16_t pkey_index; + ib_access_t access_ctrl; - } init; + } init; - struct _qp_rtr - { - ib_net32_t rq_psn; - ib_net32_t dest_qp; - ib_av_attr_t primary_av; - uint8_t resp_res; - - ib_qp_opts_t opts; - ib_av_attr_t alternate_av; - ib_net32_t qkey; - uint16_t pkey_index; - ib_access_t access_ctrl; - uint32_t sq_depth; - uint32_t rq_depth; - uint8_t rnr_nak_timeout; - - } rtr; - - struct _qp_rts - { - ib_net32_t sq_psn; - uint8_t retry_cnt; - uint8_t rnr_retry_cnt; - uint8_t rnr_nak_timeout; - uint8_t local_ack_timeout; - uint8_t init_depth; + struct _qp_rtr { + ib_net32_t rq_psn; + ib_net32_t dest_qp; + ib_av_attr_t primary_av; + uint8_t resp_res; - ib_qp_opts_t opts; - ib_qp_state_t current_state; - ib_net32_t qkey; - ib_access_t access_ctrl; - uint8_t resp_res; + ib_qp_opts_t opts; + ib_av_attr_t alternate_av; + ib_net32_t qkey; + uint16_t pkey_index; + ib_access_t access_ctrl; + uint32_t sq_depth; + uint32_t rq_depth; + uint8_t rnr_nak_timeout; - ib_av_attr_t primary_av; - ib_av_attr_t alternate_av; + } rtr; - uint32_t sq_depth; - uint32_t rq_depth; + struct _qp_rts { + ib_net32_t sq_psn; + uint8_t retry_cnt; + uint8_t rnr_retry_cnt; + uint8_t rnr_nak_timeout; + uint8_t local_ack_timeout; + uint8_t init_depth; - ib_apm_state_t apm_state; - uint8_t primary_port; - uint16_t pkey_index; + ib_qp_opts_t opts; + ib_qp_state_t current_state; + ib_net32_t qkey; + ib_access_t access_ctrl; + uint8_t resp_res; - } rts; + ib_av_attr_t primary_av; + ib_av_attr_t alternate_av; - struct _qp_sqd - { - boolean_t sqd_event; + uint32_t sq_depth; + uint32_t rq_depth; + + ib_apm_state_t apm_state; + uint8_t primary_port; + uint16_t pkey_index; + + } rts; - } sqd; + struct _qp_sqd { + boolean_t sqd_event; - } state; + } sqd; -} ib_qp_mod_t; + } state; + +} ib_qp_mod_t; /* * SEE ALSO * ib_qp_state_t, ib_access_t, ib_av_attr_t, ib_apm_state_t @@ -9250,25 +9062,24 @@ typedef struct _ib_qp_mod * * SYNOPSIS */ -typedef struct _ib_eec_attr -{ - ib_qp_state_t state; - ib_rdd_handle_t h_rdd; - ib_net32_t local_eecn; +typedef struct _ib_eec_attr { + ib_qp_state_t state; + ib_rdd_handle_t h_rdd; + ib_net32_t local_eecn; - ib_net32_t sq_psn; - ib_net32_t rq_psn; - uint8_t primary_port; - uint16_t pkey_index; - uint32_t resp_res; - ib_net32_t remote_eecn; - uint32_t init_depth; - uint32_t dest_num; // ??? What is this? - ib_av_attr_t primary_av; - ib_av_attr_t alternate_av; - ib_apm_state_t apm_state; + ib_net32_t sq_psn; + ib_net32_t rq_psn; + uint8_t primary_port; + uint16_t pkey_index; + uint32_t resp_res; + ib_net32_t remote_eecn; + uint32_t init_depth; + uint32_t dest_num; // ??? What is this? + ib_av_attr_t primary_av; + ib_av_attr_t alternate_av; + ib_apm_state_t apm_state; -} ib_eec_attr_t; +} ib_eec_attr_t; /* * SEE ALSO * ib_qp_state_t, ib_av_attr_t, ib_apm_state_t @@ -9283,7 +9094,7 @@ typedef struct _ib_eec_attr * * SYNOPSIS */ -typedef uint32_t ib_eec_opts_t; +typedef uint32_t ib_eec_opts_t; #define IB_MOD_EEC_ALTERNATE_AV 0x00000001 #define IB_MOD_EEC_PKEY 0x00000002 #define IB_MOD_EEC_APM_STATE 0x00000004 @@ -9308,59 +9119,53 @@ typedef uint32_t ib_eec_opts_t; * * SYNOPSIS */ -typedef struct _ib_eec_mod -{ - ib_qp_state_t req_state; +typedef struct _ib_eec_mod { + ib_qp_state_t req_state; - union _eec_state - { - struct _eec_init - { - uint8_t primary_port; - uint16_t pkey_index; + union _eec_state { + struct _eec_init { + uint8_t primary_port; + uint16_t pkey_index; - } init; + } init; - struct _eec_rtr - { - ib_net32_t rq_psn; - ib_net32_t remote_eecn; - ib_av_attr_t primary_av; - uint8_t resp_res; + struct _eec_rtr { + ib_net32_t rq_psn; + ib_net32_t remote_eecn; + ib_av_attr_t primary_av; + uint8_t resp_res; - ib_eec_opts_t opts; - ib_av_attr_t alternate_av; - uint16_t pkey_index; + ib_eec_opts_t opts; + ib_av_attr_t alternate_av; + uint16_t pkey_index; - } rtr; + } rtr; - struct _eec_rts - { - ib_net32_t sq_psn; - uint8_t retry_cnt; - uint8_t rnr_retry_cnt; - uint8_t local_ack_timeout; - uint8_t init_depth; + struct _eec_rts { + ib_net32_t sq_psn; + uint8_t retry_cnt; + uint8_t rnr_retry_cnt; + uint8_t local_ack_timeout; + uint8_t init_depth; - ib_eec_opts_t opts; - ib_av_attr_t alternate_av; - ib_apm_state_t apm_state; + ib_eec_opts_t opts; + ib_av_attr_t alternate_av; + ib_apm_state_t apm_state; - ib_av_attr_t primary_av; - uint16_t pkey_index; - uint8_t primary_port; + ib_av_attr_t primary_av; + uint16_t pkey_index; + uint8_t primary_port; - } rts; + } rts; - struct _eec_sqd - { - boolean_t sqd_event; + struct _eec_sqd { + boolean_t sqd_event; - } sqd; + } sqd; - } state; + } state; -} ib_eec_mod_t; +} ib_eec_mod_t; /* * SEE ALSO * ib_qp_state_t, ib_av_attr_t, ib_apm_state_t @@ -9375,15 +9180,13 @@ typedef struct _ib_eec_mod * * SYNOPSIS */ -typedef enum _ib_wr_type_t -{ +typedef enum _ib_wr_type_t { WR_SEND = 1, WR_RDMA_WRITE, WR_RDMA_READ, WR_COMPARE_SWAP, WR_FETCH_ADD - -} ib_wr_type_t; +} ib_wr_type_t; /*****/ /****s* Access Layer/ib_local_ds_t @@ -9397,13 +9200,12 @@ typedef enum _ib_wr_type_t * * SYNOPSIS */ -typedef struct _ib_local_ds -{ - void *vaddr; - uint32_t length; - uint32_t lkey; +typedef struct _ib_local_ds { + void *vaddr; + uint32_t length; + uint32_t lkey; -} ib_local_ds_t; +} ib_local_ds_t; /*****/ /****d* Access Layer/ib_send_opt_t @@ -9416,7 +9218,7 @@ typedef struct _ib_local_ds * * SYNOPSIS */ -typedef uint32_t ib_send_opt_t; +typedef uint32_t ib_send_opt_t; #define IB_SEND_OPT_IMMEDIATE 0x00000001 #define IB_SEND_OPT_FENCE 0x00000002 #define IB_SEND_OPT_SIGNALED 0x00000004 @@ -9476,66 +9278,59 @@ typedef uint32_t ib_send_opt_t; * * SYNOPSIS */ -typedef struct _ib_send_wr -{ - struct _ib_send_wr *p_next; - uint64_t wr_id; - ib_wr_type_t wr_type; - ib_send_opt_t send_opt; - uint32_t num_ds; - ib_local_ds_t *ds_array; - ib_net32_t immediate_data; +typedef struct _ib_send_wr { + struct _ib_send_wr *p_next; + uint64_t wr_id; + ib_wr_type_t wr_type; + ib_send_opt_t send_opt; + uint32_t num_ds; + ib_local_ds_t *ds_array; + ib_net32_t immediate_data; - union _send_dgrm - { - struct _send_ud - { - ib_net32_t remote_qp; - ib_net32_t remote_qkey; - ib_av_handle_t h_av; + union _send_dgrm { + struct _send_ud { + ib_net32_t remote_qp; + ib_net32_t remote_qkey; + ib_av_handle_t h_av; - } ud; + } ud; - struct _send_rd - { - ib_net32_t remote_qp; - ib_net32_t remote_qkey; - ib_net32_t eecn; + struct _send_rd { + ib_net32_t remote_qp; + ib_net32_t remote_qkey; + ib_net32_t eecn; - } rd; + } rd; - struct _send_raw_ether - { - ib_net16_t dest_lid; - uint8_t path_bits; - uint8_t sl; - uint8_t max_static_rate; - ib_net16_t ether_type; + struct _send_raw_ether { + ib_net16_t dest_lid; + uint8_t path_bits; + uint8_t sl; + uint8_t max_static_rate; + ib_net16_t ether_type; - } raw_ether; + } raw_ether; - struct _send_raw_ipv6 - { - ib_net16_t dest_lid; - uint8_t path_bits; - uint8_t sl; - uint8_t max_static_rate; + struct _send_raw_ipv6 { + ib_net16_t dest_lid; + uint8_t path_bits; + uint8_t sl; + uint8_t max_static_rate; - } raw_ipv6; + } raw_ipv6; - } dgrm; + } dgrm; - struct _send_remote_ops - { - uint64_t vaddr; - uint32_t rkey; + struct _send_remote_ops { + uint64_t vaddr; + uint32_t rkey; - ib_net64_t atomic1; - ib_net64_t atomic2; + ib_net64_t atomic1; + ib_net64_t atomic2; - } remote_ops; + } remote_ops; -} ib_send_wr_t; +} ib_send_wr_t; /* * FIELDS * p_next @@ -9650,13 +9445,12 @@ typedef struct _ib_send_wr * * SYNOPSIS */ -typedef struct _ib_recv_wr -{ - struct _ib_recv_wr *p_next; - uint64_t wr_id; - uint32_t num_ds; - ib_local_ds_t *ds_array; -} ib_recv_wr_t; +typedef struct _ib_recv_wr { + struct _ib_recv_wr *p_next; + uint64_t wr_id; + uint32_t num_ds; + ib_local_ds_t *ds_array; +} ib_recv_wr_t; /* * FIELDS * p_next @@ -9689,18 +9483,17 @@ typedef struct _ib_recv_wr * * SYNOPSIS */ -typedef struct _ib_bind_wr -{ - uint64_t wr_id; - ib_send_opt_t send_opt; +typedef struct _ib_bind_wr { + uint64_t wr_id; + ib_send_opt_t send_opt; - ib_mr_handle_t h_mr; - ib_access_t access_ctrl; - uint32_t current_rkey; + ib_mr_handle_t h_mr; + ib_access_t access_ctrl; + uint32_t current_rkey; - ib_local_ds_t local_ds; + ib_local_ds_t local_ds; -} ib_bind_wr_t; +} ib_bind_wr_t; /* * FIELDS * wr_id @@ -9737,8 +9530,7 @@ typedef struct _ib_bind_wr * * SYNOPSIS */ -typedef enum _ib_wc_status_t -{ +typedef enum _ib_wc_status_t { IB_WCS_SUCCESS, IB_WCS_LOCAL_LEN_ERR, IB_WCS_LOCAL_OP_ERR, @@ -9754,11 +9546,10 @@ typedef enum _ib_wc_status_t IB_WCS_REM_INVALID_RD_REQ_ERR, IB_WCS_INVALID_EECN, IB_WCS_INVALID_EEC_STATE, - IB_WCS_UNMATCHED_RESPONSE, /* InfiniBand Access Layer */ - IB_WCS_CANCELED, /* InfiniBand Access Layer */ - IB_WCS_UNKNOWN /* Must be last. */ - -} ib_wc_status_t; + IB_WCS_UNMATCHED_RESPONSE, /* InfiniBand Access Layer */ + IB_WCS_CANCELED, /* InfiniBand Access Layer */ + IB_WCS_UNKNOWN /* Must be last. */ +} ib_wc_status_t; /* * VALUES * IB_WCS_SUCCESS @@ -9841,7 +9632,7 @@ typedef enum _ib_wc_status_t * The completed work request was canceled by the user. *****/ -OSM_EXPORT const char* ib_wc_status_str[]; +OSM_EXPORT const char *ib_wc_status_str[]; /****f* IBA Base: Types/ib_get_wc_status_str * NAME @@ -9852,14 +9643,14 @@ OSM_EXPORT const char* ib_wc_status_str[]; * * SYNOPSIS */ -static inline const char* OSM_API -ib_get_wc_status_str( - IN ib_wc_status_t wc_status ) +static inline const char *OSM_API +ib_get_wc_status_str(IN ib_wc_status_t wc_status) { - if( wc_status > IB_WCS_UNKNOWN ) + if (wc_status > IB_WCS_UNKNOWN) wc_status = IB_WCS_UNKNOWN; - return( ib_wc_status_str[wc_status] ); + return (ib_wc_status_str[wc_status]); } + /* * PARAMETERS * wc_status @@ -9882,8 +9673,7 @@ ib_get_wc_status_str( * * SYNOPSIS */ -typedef enum _ib_wc_type_t -{ +typedef enum _ib_wc_type_t { IB_WC_SEND, IB_WC_RDMA_WRITE, IB_WC_RECV, @@ -9892,8 +9682,7 @@ typedef enum _ib_wc_type_t IB_WC_FETCH_ADD, IB_WC_COMPARE_SWAP, IB_WC_RECV_RDMA_WRITE - -} ib_wc_type_t; +} ib_wc_type_t; /*****/ /****d* Access Layer/ib_recv_opt_t @@ -9905,7 +9694,7 @@ typedef enum _ib_wc_type_t * * SYNOPSIS */ -typedef uint32_t ib_recv_opt_t; +typedef uint32_t ib_recv_opt_t; #define IB_RECV_OPT_IMMEDIATE 0x00000001 #define IB_RECV_OPT_FORWARD 0x00000002 #define IB_RECV_OPT_GRH_VALID 0x00000004 @@ -9938,67 +9727,60 @@ typedef uint32_t ib_recv_opt_t; * * SYNOPSIS */ -typedef struct _ib_wc -{ - struct _ib_wc *p_next; - uint64_t wr_id; - ib_wc_type_t wc_type; +typedef struct _ib_wc { + struct _ib_wc *p_next; + uint64_t wr_id; + ib_wc_type_t wc_type; - uint32_t length; - ib_wc_status_t status; - uint64_t vendor_specific; + uint32_t length; + ib_wc_status_t status; + uint64_t vendor_specific; - union _wc_recv - { - struct _wc_conn - { - ib_recv_opt_t recv_opt; - ib_net32_t immediate_data; + union _wc_recv { + struct _wc_conn { + ib_recv_opt_t recv_opt; + ib_net32_t immediate_data; - } conn; + } conn; - struct _wc_ud - { - ib_recv_opt_t recv_opt; - ib_net32_t immediate_data; - ib_net32_t remote_qp; - uint16_t pkey_index; - ib_net16_t remote_lid; - uint8_t remote_sl; - uint8_t path_bits; + struct _wc_ud { + ib_recv_opt_t recv_opt; + ib_net32_t immediate_data; + ib_net32_t remote_qp; + uint16_t pkey_index; + ib_net16_t remote_lid; + uint8_t remote_sl; + uint8_t path_bits; - } ud; + } ud; - struct _wc_rd - { - ib_net32_t remote_eecn; - ib_net32_t remote_qp; - ib_net16_t remote_lid; - uint8_t remote_sl; - uint32_t free_cnt; + struct _wc_rd { + ib_net32_t remote_eecn; + ib_net32_t remote_qp; + ib_net16_t remote_lid; + uint8_t remote_sl; + uint32_t free_cnt; - } rd; + } rd; - struct _wc_raw_ipv6 - { - ib_net16_t remote_lid; - uint8_t remote_sl; - uint8_t path_bits; + struct _wc_raw_ipv6 { + ib_net16_t remote_lid; + uint8_t remote_sl; + uint8_t path_bits; - } raw_ipv6; + } raw_ipv6; - struct _wc_raw_ether - { - ib_net16_t remote_lid; - uint8_t remote_sl; - uint8_t path_bits; - ib_net16_t ether_type; + struct _wc_raw_ether { + ib_net16_t remote_lid; + uint8_t remote_sl; + uint8_t path_bits; + ib_net16_t ether_type; - } raw_ether; + } raw_ether; - } recv; + } recv; -} ib_wc_t; +} ib_wc_t; /* * FIELDS * p_next @@ -10117,12 +9899,11 @@ typedef struct _ib_wc * * SYNOPSIS */ -typedef struct _ib_mr_create -{ - void *vaddr; - uint64_t length; - ib_access_t access_ctrl; -} ib_mr_create_t; +typedef struct _ib_mr_create { + void *vaddr; + uint64_t length; + ib_access_t access_ctrl; +} ib_mr_create_t; /* * FIELDS * vaddr @@ -10147,15 +9928,14 @@ typedef struct _ib_mr_create * * SYNOPSIS */ -typedef struct _ib_phys_create -{ - uint64_t length; - uint32_t num_bufs; - uint64_t *buf_array; - uint32_t buf_offset; - uint32_t page_size; - ib_access_t access_ctrl; -} ib_phys_create_t; +typedef struct _ib_phys_create { + uint64_t length; + uint32_t num_bufs; + uint64_t *buf_array; + uint32_t buf_offset; + uint32_t page_size; + ib_access_t access_ctrl; +} ib_phys_create_t; /* * length * The length of the memory region in bytes. @@ -10190,17 +9970,16 @@ typedef struct _ib_phys_create * * SYNOPSIS */ -typedef struct _ib_mr_attr -{ - ib_pd_handle_t h_pd; - void *local_lb; - void *local_ub; - void *remote_lb; - void *remote_ub; - ib_access_t access_ctrl; - uint32_t lkey; - uint32_t rkey; -} ib_mr_attr_t; +typedef struct _ib_mr_attr { + ib_pd_handle_t h_pd; + void *local_lb; + void *local_ub; + void *remote_lb; + void *remote_ub; + ib_access_t access_ctrl; + uint32_t lkey; + uint32_t rkey; +} ib_mr_attr_t; /* * DESCRIPTION * h_pd @@ -10248,7 +10027,7 @@ typedef struct _ib_mr_attr * * SYNOPSIS */ -typedef uint32_t ib_ca_mod_t; +typedef uint32_t ib_ca_mod_t; #define IB_CA_MOD_IS_CM_SUPPORTED 0x00000001 #define IB_CA_MOD_IS_SNMP_SUPPORTED 0x00000002 #define IB_CA_MOD_IS_DEV_MGMT_SUPPORTED 0x00000004 @@ -10365,7 +10144,7 @@ typedef uint32_t ib_ca_mod_t; * * SYNOPSIS */ -typedef uint32_t ib_mr_mod_t; +typedef uint32_t ib_mr_mod_t; #define IB_MR_MOD_ADDR 0x00000001 #define IB_MR_MOD_PD 0x00000002 #define IB_MR_MOD_ACCESS 0x00000004 @@ -10463,15 +10242,14 @@ typedef uint32_t ib_mr_mod_t; * * SYNOPSIS */ -typedef struct _ib_ci_op -{ - IN uint32_t command; - IN OUT void* p_buf OPTIONAL; - IN uint32_t buf_size; - IN OUT uint32_t num_bytes_ret; - IN OUT int32_t status; +typedef struct _ib_ci_op { + IN uint32_t command; + IN OUT void *p_buf OPTIONAL; + IN uint32_t buf_size; + IN OUT uint32_t num_bytes_ret; + IN OUT int32_t status; -} ib_ci_op_t; +} ib_ci_op_t; /* * FIELDS * command @@ -10505,11 +10283,8 @@ typedef struct _ib_ci_op *****/ END_C_DECLS - -#endif /* ndef WIN32 */ - +#endif /* ndef WIN32 */ #if defined( __WIN__ ) - #include +#include #endif - -#endif /* __IB_TYPES_H__ */ +#endif /* __IB_TYPES_H__ */ -- 1.5.3.rc2.38.g11308 From kbaxley at redhat.com Tue Aug 14 08:36:55 2007 From: kbaxley at redhat.com (Kent Baxley) Date: Tue, 14 Aug 2007 11:36:55 -0400 Subject: [ofa-general] Can Mellanox cards handle scatter gather I/O? Message-ID: <20070814113655.68c317d8@dhcp-243-71.rdu.redhat.com> Evidently, we're seeing performance problems when a card on one side is capable of scatter gather IO but the mellanox driver is not able to handle it. Then a whole lot of time is spent gathering the data into one skbuff. This is currently being observed on RHEL4.5 with OFED 1.2. Is the Mellanox card capable of doing scatter gather IO and this just a driver limitation or is the mellanox card just not able to do scatter gather IO? Here is the original description of the problem in case the original synopsis was oversimplified: A second issue arises in how current kernels and specifically NETDEV handle scatter/gather and the related SKBs. Kernel.org is currently in the middle of a massive re-write of this section but existing ports may incur an extra buffer copy. Of the existing backports there are two skbuff variations. One is simply a pointer to the SKB while the other is a list of SKBs. If a driver supports the list and forwards a packet to a driver which supports only a pointer then NETDEV is stuck copying the list of buffers into a new single SKB. At 10gbs this copy can sap a processor's capacity to move data. The identified case where this conflict arises is the Myri10GE driver which supports lists of SKBs and Mellanox IB (IPoIB) driver which does not. Initial search through the web suggests that the IB verbs layer does support scatter gather IO and that it has a limit of 60 segments. However, we've as of yet been unable to determine if that is implemented in the verbs layer or if it is done in hardware. If anyone has any info that might help here, it will be appreciated. Thanks. From Dionisio at ieko.co.uk Tue Aug 14 08:42:18 2007 From: Dionisio at ieko.co.uk (Dionisio naismith) Date: Tue, 14 Aug 2007 11:42:18 -0400 Subject: [ofa-general] This syntax is neither worse, nor better than the Intel syntax. Message-ID: <20070814154226.55F54E608B7@openfabrics.org> T,h-i*s Tue-sday i.t*s CY TV!!! G*e.t on C,Y,T*V Fir,st Thi-ng on TU-E_SDAY, i_t*',s goin-g to exploa+d! C omp_any: CH_INA Y OUTV C,O R'P (_O.T+C BB:CYT V..OB) Sym*bol: C+Y*T-V Cur,rent Pri ce: $_0 ..4'6 Mo+nday Mo+ve: ('U_p Over(__12.20%) T+h-e pr+ice is at a min_imum it w+i l*l b'o+o'm on Tuesda+y! Re+com*mendation: "ST'RONG--BUY" star,.ting on Tu+es+day, AUGU.ST 1'4', 2,0+0+6'. Br,*eaking N*e+w.s*: C,Y T V MAK,ES MAJ OR ANNOUN,C,EMENTS: C*hina YouT V's Cn.Boo W'e-b S i*t_e Ra_nks N,o+.'1 on Mi*cr*osoft L_i'v_e S earch Eng'ine Agg.r,esive Tr'aders G e,t in earl_y, t h*i+s o,n_e c.a*n f+l_i,p h,i-g,h retu_rns f_a*s't+. T h-e f+i+s_h fl,ickere,d o*u+t of phas e, h'e,r syst_em lau ochm.ng a sub rout'ine. W h+y do n't y,o_u m+arry me instea-d. There'_s no fa+-cility f-o+r automa tical,ly detec',ting w*h-a-t a-*ddresses a'r e fa-iling. I*''m dead,-" s'a_i*d Russel'l.. "I w.a's up a_l l l+a*s*t n+ight w'i+t,h dysente'r-y. I s e_e y*o+u d_i_d n_o't r*e+a d y+o*u,r les*son care+fully.. From Tammie467 at damen-bergum.nl Tue Aug 14 08:56:29 2007 From: Tammie467 at damen-bergum.nl (Tammie hater) Date: Tue, 14 Aug 2007 12:56:29 -0300 Subject: [ofa-general] It reads from the INI file the buttons it should display and what command is associated with each one. Message-ID: <000a01c7de8b$ad529780$cd7a18bd@junior> T+h,i*s Tu esday i*t_s C.YTV!!! G_e.t on C Y'T_V Fi*rst Th.ing on TU*E-SDAY, i,t_'*s goi.ng to expload,! Co.mpa+ny: C.HINA YOU,TV C+O_R_P (,O_T-C BB:.CY*TV.OB) Symb.ol: C_Y,T V Curr+ent Pr ice: $.0_.+4-6 Monda_y M ove: ( U*p Over(12.,2,0%) T,h'e pric.e is at a minim,um it w+i'l_l b*o o m on Tue+sday! Reco,mmendat*ion: "ST RONG-B'UY" start i-ng on T,ues*day, AU*GUST 1-4_, 2_0+0 6,. Breaki*n+g N,e-w-s,: C.Y+T'V MA*KES M AJOR ANN OUNCEMENTS+: Chin.a You*TV's C+nBoo W_e,b S,i+t+e R-anks N.o'._1 on M*i_crosoft L i_v_e Sear*ch En,gine A'ggre-sive Tr-aders G e+t in earl_y, t-h-i_s o-n*e c-a,n f_l+i p h*i*g-h return,s f+a+s-t_. I_t-' s st,ill b'etter to u*s,e 10. T-h i_s is accomplishe +d by usi*ng a D_O,S driv-er a,n+d a V D'D-. Wh ere d,i+d s,u.c'h a half',-educated m-a+n l'a,y h i+s hand,s on so m'u_c'h mon+ey. Thei.r tender+, dark-b'lu*e corol la_s w.i+t-h t h-e go*lden spri+-nkling of t,h.e fin est star.s, e+a c+h w-i,t h up to s*i*x petal_s, loo ked do*wn, d,a.ngling on bow,ed st'ems. T-h e "S*ave to f_ile" fiel-d is sel'f-expl_aining. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dledford at redhat.com Tue Aug 14 08:56:17 2007 From: dledford at redhat.com (Doug Ledford) Date: Tue, 14 Aug 2007 11:56:17 -0400 Subject: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status In-Reply-To: <20070814094005.GB1301@mellanox.co.il> References: <1187082976.12674.24.camel@trogdor.suse.de> <20070814094005.GB1301@mellanox.co.il> Message-ID: <1187106980.14384.33.camel@firewall.xsintricity.com> On Tue, 2007-08-14 at 12:40 +0300, Michael S. Tsirkin wrote: > > Quoting Scott Bahling : > > Subject: Re: [ofa-general] Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > On Tue, 2007-08-14 at 09:59 +0200, Hoang-Nam Nguyen wrote: > > > Hi Doug! > > > > On Sat, 2007-08-11 at 21:13 +0300, Michael S. Tsirkin wrote: > > > > > > Quoting Hoang-Nam Nguyen : > > > > > > Subject: Re: ofa_1_2_c_kernel 20070802-0201 daily build status > > > > > > > > > > > > Hello Doug and Scott! > > > > > > > > > > > > On Thursday 02 August 2007 18:08, Michael S. Tsirkin wrote: > > > > > > > ehca backports for kernel.org kernels seem to be broken. > > > > > > > 1. Does anyone care enough to fix them? If not we'll disable > > > > > > > ehca in build for these kernels. > > > > > > > > > > > > > > 2. Could you upload kernels for RHEL4U5 and SLES10 ppc64? > > > > > > > > Don't you guys already have RHEL4U5? It had a backports directory in > > > > the OFED 1.2 release...and it's been out for quite a while... > > > Some part of this thread might confuse. And really, it's not about > > > any specific backport issue from ehca or other component(s). It's a > > > general prereq for ofed's daily build to have rhel4.5 resp sles10 ppc64 > > > in their daily build runs too. > > > > I have checked, and Mellanox still has an active partner account with > > Novell which means they can download our product isos (which contain the > > kernel-source rpms) at http://download.novell.com and any package > > updates at support.novell.com http://support.novell.com/linux/psdb/ > > using proper Mellanox logins. You can contact me offline of you need > > help here. > > > > Also, our current cvs snapshots can be found at > > > > ftp://ftp.suse.com/pub/projects/kernel/kotd/ > > or the following mirror > > ftp://ftp.gwdg.de/pub/linux/suse/projects/kernel/kotd/ > > > > These are unreleased bits, but give you a peek into what changes might > > be coming in the next kernel update. Not sure if this is of use to you. > > I don't have a ppc machine so I can't use your install ISOs. Nonsense. Of course you can. At a minimum rpm2cpio | cpio -ivd gets around the arch of the package regardless of what machine you are on and installs it without involving the rpm database. > If someone cares enough about SLES/RHEL on ppc support in OFED, > she can upload the unpacked kernel devel headers for these distros > to ofa server and we'll use these to cross-compile OFED kernel bits, > nightly. -- Doug Ledford GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From tziporet at dev.mellanox.co.il Tue Aug 14 09:12:04 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 14 Aug 2007 19:12:04 +0300 Subject: [ofa-general] Re: [ewg] Re: OFED Aug 13 meeting summary In-Reply-To: <20070814150610.GA16827@mellanox.co.il> References: <46C088BE.1090100@mellanox.co.il> <20070814050022.GD9924@mellanox.co.il> <46C19AD4.4030301@mellanox.co.il> <200708141545.33576.ossrosch@linux.vnet.ibm.com> <20070814150610.GA16827@mellanox.co.il> Message-ID: <46C1D454.7080603@mellanox.co.il> Michael S. Tsirkin wrote: >> Quoting Stefan Roscher : >> Subject: Re: [ewg] Re: OFED Aug 13 meeting summary >> >> can we ensure that this patch >> http://lists.openfabrics.org/pipermail/ewg/2007-August/004299.html is apllied? >> Without this patch we have a broken ehca build on rhel-4.5. >> >> regards Stefan >> > > I pushed a fix for this out in my tree, so Vlad will take it in 1.2.c branch. > > Vlad will take this patch, and we will do a new build with it. This will delay the 1.2.5 release for tomorrow :-( ehca maintainers: For next release please check that all your required patches are in on the release candidates and not wait for the last day of the release Thanks, Tziporet From sean.hefty at intel.com Tue Aug 14 09:21:47 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Tue, 14 Aug 2007 09:21:47 -0700 Subject: [ofa-general] RE: [PATCH] librdmacm: Fix memory leak reported by valgrind In-Reply-To: <200708141500.15178.dotanb@dev.mellanox.co.il> Message-ID: <000001c7de8f$361390d0$ff0da8c0@amr.corp.intel.com> Thanks - applied to master - and will be in next release - 1.0.3. - Sean From sashak at voltaire.com Tue Aug 14 10:13:05 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 14 Aug 2007 20:13:05 +0300 Subject: [ofa-general] [PATCH] complib: indentation changes Message-ID: <20070814171305.GI2184@sashak.voltaire.com> Formatted with opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs) Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_complib.c | 37 +- opensm/complib/cl_dispatcher.c | 526 +++++++-------- opensm/complib/cl_event.c | 144 ++--- opensm/complib/cl_event_wheel.c | 1090 +++++++++++++++---------------- opensm/complib/cl_list.c | 545 +++++++--------- opensm/complib/cl_log.c | 68 +-- opensm/complib/cl_map.c | 1369 +++++++++++++++++---------------------- opensm/complib/cl_pool.c | 544 ++++++++-------- opensm/complib/cl_ptr_vector.c | 277 ++++----- opensm/complib/cl_spinlock.c | 65 +-- opensm/complib/cl_statustext.c | 5 +- opensm/complib/cl_thread.c | 88 ++-- opensm/complib/cl_threadpool.c | 53 +- opensm/complib/cl_timer.c | 594 ++++++++--------- opensm/complib/cl_vector.c | 420 ++++++------- opensm/complib/ib_statustext.c | 22 +- 16 files changed, 2681 insertions(+), 3166 deletions(-) diff --git a/opensm/complib/cl_complib.c b/opensm/complib/cl_complib.c index 0677544..b9aca19 100644 --- a/opensm/complib/cl_complib.c +++ b/opensm/complib/cl_complib.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -50,48 +50,43 @@ */ extern -cl_status_t -__cl_timer_prov_create( void ); + cl_status_t __cl_timer_prov_create(void); extern -void -__cl_timer_prov_destroy( void ); +void __cl_timer_prov_destroy(void); cl_spinlock_t cl_atomic_spinlock; -void -complib_init(void) +void complib_init(void) { - cl_status_t status = CL_SUCCESS; + cl_status_t status = CL_SUCCESS; status = cl_spinlock_init(&cl_atomic_spinlock); if (status != CL_SUCCESS) goto _error; status = __cl_timer_prov_create(); - if( status != CL_SUCCESS) + if (status != CL_SUCCESS) goto _error; return; - _error: - cl_msg_out( "__init: failed to create complib (%s)\n", - CL_STATUS_MSG( status ) ); + _error: + cl_msg_out("__init: failed to create complib (%s)\n", + CL_STATUS_MSG(status)); exit(1); } -void -complib_exit(void) +void complib_exit(void) { - __cl_timer_prov_destroy(); - cl_spinlock_destroy(&cl_atomic_spinlock); + __cl_timer_prov_destroy(); + cl_spinlock_destroy(&cl_atomic_spinlock); } -boolean_t -cl_is_debug( void ) +boolean_t cl_is_debug(void) { #if defined( _DEBUG_ ) - return TRUE; + return TRUE; #else - return FALSE; -#endif /* defined( _DEBUG_ ) */ + return FALSE; +#endif /* defined( _DEBUG_ ) */ } diff --git a/opensm/complib/cl_dispatcher.c b/opensm/complib/cl_dispatcher.c index 4a1960c..1d865d5 100644 --- a/opensm/complib/cl_dispatcher.c +++ b/opensm/complib/cl_dispatcher.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -76,330 +76,306 @@ Returns: None ********************************************************************/ -void -__cl_disp_worker( - IN void* context ) +void __cl_disp_worker(IN void *context) { - cl_disp_msg_t *p_msg; - cl_dispatcher_t *p_disp = (cl_dispatcher_t*)context; - - cl_spinlock_acquire( &p_disp->lock ); - - /* Process the FIFO until we drain it dry. */ - while( cl_qlist_count( &p_disp->msg_fifo ) ) - { - /* Pop the message at the head from the FIFO. */ - p_msg = (cl_disp_msg_t*)cl_qlist_remove_head( &p_disp->msg_fifo ); - - /* we track the tim ethe last message spent in the queue */ - p_disp->last_msg_queue_time_us = cl_get_time_stamp() - p_msg->in_time; - - /* - * Release the spinlock while the message is processed. - * The user's callback may reenter the dispatcher - * and cause the lock to be reaquired. - */ - cl_spinlock_release( &p_disp->lock ); - p_msg->p_dest_reg->pfn_rcv_callback( - (void*)p_msg->p_dest_reg->context, (void*)p_msg->p_data ); - - cl_atomic_dec( &p_msg->p_dest_reg->ref_cnt ); - - /* The client has seen the data. Notify the sender as appropriate. */ - if( p_msg->pfn_xmt_callback ) - { - p_msg->pfn_xmt_callback( (void*)p_msg->context, - (void*)p_msg->p_data ); - cl_atomic_dec( &p_msg->p_src_reg->ref_cnt ); - } - - /* Grab the lock for the next iteration through the list. */ - cl_spinlock_acquire(&p_disp->lock); - - /* Return this message to the pool. */ - cl_qpool_put( &p_disp->msg_pool, (cl_pool_item_t *)p_msg ); - } - - cl_spinlock_release( &p_disp->lock ); + cl_disp_msg_t *p_msg; + cl_dispatcher_t *p_disp = (cl_dispatcher_t *) context; + + cl_spinlock_acquire(&p_disp->lock); + + /* Process the FIFO until we drain it dry. */ + while (cl_qlist_count(&p_disp->msg_fifo)) { + /* Pop the message at the head from the FIFO. */ + p_msg = + (cl_disp_msg_t *) cl_qlist_remove_head(&p_disp->msg_fifo); + + /* we track the tim ethe last message spent in the queue */ + p_disp->last_msg_queue_time_us = + cl_get_time_stamp() - p_msg->in_time; + + /* + * Release the spinlock while the message is processed. + * The user's callback may reenter the dispatcher + * and cause the lock to be reaquired. + */ + cl_spinlock_release(&p_disp->lock); + p_msg->p_dest_reg->pfn_rcv_callback((void *)p_msg->p_dest_reg-> + context, + (void *)p_msg->p_data); + + cl_atomic_dec(&p_msg->p_dest_reg->ref_cnt); + + /* The client has seen the data. Notify the sender as appropriate. */ + if (p_msg->pfn_xmt_callback) { + p_msg->pfn_xmt_callback((void *)p_msg->context, + (void *)p_msg->p_data); + cl_atomic_dec(&p_msg->p_src_reg->ref_cnt); + } + + /* Grab the lock for the next iteration through the list. */ + cl_spinlock_acquire(&p_disp->lock); + + /* Return this message to the pool. */ + cl_qpool_put(&p_disp->msg_pool, (cl_pool_item_t *) p_msg); + } + + cl_spinlock_release(&p_disp->lock); } /******************************************************************** ********************************************************************/ -void -cl_disp_construct( - IN cl_dispatcher_t* const p_disp ) +void cl_disp_construct(IN cl_dispatcher_t * const p_disp) { - CL_ASSERT( p_disp ); + CL_ASSERT(p_disp); - cl_qlist_init( &p_disp->reg_list ); - cl_ptr_vector_construct( &p_disp->reg_vec ); - cl_qlist_init( &p_disp->msg_fifo ); - cl_spinlock_construct( &p_disp->lock ); - cl_qpool_construct( &p_disp->msg_pool ); + cl_qlist_init(&p_disp->reg_list); + cl_ptr_vector_construct(&p_disp->reg_vec); + cl_qlist_init(&p_disp->msg_fifo); + cl_spinlock_construct(&p_disp->lock); + cl_qpool_construct(&p_disp->msg_pool); } /******************************************************************** ********************************************************************/ -void -cl_disp_shutdown( - IN cl_dispatcher_t* const p_disp ) +void cl_disp_shutdown(IN cl_dispatcher_t * const p_disp) { - CL_ASSERT( p_disp ); + CL_ASSERT(p_disp); - /* Stop the thread pool. */ - cl_thread_pool_destroy( &p_disp->worker_threads ); + /* Stop the thread pool. */ + cl_thread_pool_destroy(&p_disp->worker_threads); - /* Process all outstanding callbacks. */ - __cl_disp_worker( p_disp ); + /* Process all outstanding callbacks. */ + __cl_disp_worker(p_disp); - /* Free all registration info. */ - while( !cl_is_qlist_empty( &p_disp->reg_list ) ) - free( cl_qlist_remove_head( &p_disp->reg_list ) ); + /* Free all registration info. */ + while (!cl_is_qlist_empty(&p_disp->reg_list)) + free(cl_qlist_remove_head(&p_disp->reg_list)); } /******************************************************************** ********************************************************************/ -void -cl_disp_destroy( - IN cl_dispatcher_t* const p_disp ) +void cl_disp_destroy(IN cl_dispatcher_t * const p_disp) { - CL_ASSERT( p_disp ); + CL_ASSERT(p_disp); - cl_spinlock_destroy( &p_disp->lock ); - /* Destroy the message pool */ - cl_qpool_destroy( &p_disp->msg_pool ); - /* Destroy the pointer vector of registrants. */ - cl_ptr_vector_destroy( &p_disp->reg_vec ); + cl_spinlock_destroy(&p_disp->lock); + /* Destroy the message pool */ + cl_qpool_destroy(&p_disp->msg_pool); + /* Destroy the pointer vector of registrants. */ + cl_ptr_vector_destroy(&p_disp->reg_vec); } /******************************************************************** ********************************************************************/ cl_status_t -cl_disp_init( - IN cl_dispatcher_t* const p_disp, - IN const uint32_t thread_count, - IN const char* const name ) +cl_disp_init(IN cl_dispatcher_t * const p_disp, + IN const uint32_t thread_count, IN const char *const name) { - cl_status_t status; - - CL_ASSERT( p_disp ); - - cl_disp_construct( p_disp ); - - status = cl_spinlock_init( &p_disp->lock ); - if( status != CL_SUCCESS ) - { - cl_disp_destroy( p_disp ); - return( status ); - } - - /* Specify no upper limit to the number of messages in the pool */ - status = cl_qpool_init( &p_disp->msg_pool, CL_DISP_INITIAL_MSG_COUNT, - 0, CL_DISP_MSG_GROW_SIZE, sizeof(cl_disp_msg_t), NULL, - NULL, NULL ); - if( status != CL_SUCCESS ) - { - cl_disp_destroy( p_disp ); - return( status ); - } - - status = cl_ptr_vector_init( &p_disp->reg_vec, CL_DISP_INITIAL_REG_COUNT, - CL_DISP_REG_GROW_SIZE ); - if( status != CL_SUCCESS ) - { - cl_disp_destroy( p_disp ); - return( status ); - } - - status = cl_thread_pool_init( &p_disp->worker_threads, thread_count, - __cl_disp_worker, p_disp, name ); - if( status != CL_SUCCESS ) - cl_disp_destroy( p_disp ); - - return( status ); + cl_status_t status; + + CL_ASSERT(p_disp); + + cl_disp_construct(p_disp); + + status = cl_spinlock_init(&p_disp->lock); + if (status != CL_SUCCESS) { + cl_disp_destroy(p_disp); + return (status); + } + + /* Specify no upper limit to the number of messages in the pool */ + status = cl_qpool_init(&p_disp->msg_pool, CL_DISP_INITIAL_MSG_COUNT, + 0, CL_DISP_MSG_GROW_SIZE, sizeof(cl_disp_msg_t), + NULL, NULL, NULL); + if (status != CL_SUCCESS) { + cl_disp_destroy(p_disp); + return (status); + } + + status = cl_ptr_vector_init(&p_disp->reg_vec, CL_DISP_INITIAL_REG_COUNT, + CL_DISP_REG_GROW_SIZE); + if (status != CL_SUCCESS) { + cl_disp_destroy(p_disp); + return (status); + } + + status = cl_thread_pool_init(&p_disp->worker_threads, thread_count, + __cl_disp_worker, p_disp, name); + if (status != CL_SUCCESS) + cl_disp_destroy(p_disp); + + return (status); } /******************************************************************** ********************************************************************/ cl_disp_reg_handle_t -cl_disp_register( - IN cl_dispatcher_t* const p_disp, - IN const cl_disp_msgid_t msg_id, - IN cl_pfn_msgrcv_cb_t pfn_callback OPTIONAL, - IN const void* const context OPTIONAL ) +cl_disp_register(IN cl_dispatcher_t * const p_disp, + IN const cl_disp_msgid_t msg_id, + IN cl_pfn_msgrcv_cb_t pfn_callback OPTIONAL, + IN const void *const context OPTIONAL) { - cl_disp_reg_info_t *p_reg; - cl_status_t status; - - CL_ASSERT( p_disp ); - - /* Check that the requested registrant ID is available. */ - cl_spinlock_acquire( &p_disp->lock ); - if( (msg_id != CL_DISP_MSGID_NONE ) && - (msg_id < cl_ptr_vector_get_size( &p_disp->reg_vec )) && - (cl_ptr_vector_get( &p_disp->reg_vec, msg_id )) ) - { - cl_spinlock_release( &p_disp->lock ); - return( NULL ); - } - - /* Get a registration info from the pool. */ - p_reg = (cl_disp_reg_info_t*)malloc( sizeof(cl_disp_reg_info_t) ); - if( !p_reg ) - { - cl_spinlock_release( &p_disp->lock ); - return( NULL ); - } - else - { - memset( p_reg, 0, sizeof(cl_disp_reg_info_t) ); - } - - p_reg->p_disp = p_disp; - p_reg->ref_cnt = 0; - p_reg->pfn_rcv_callback = pfn_callback; - p_reg->context = context; - p_reg->msg_id = msg_id; - - /* Insert the registration in the list. */ - cl_qlist_insert_tail( &p_disp->reg_list, (cl_list_item_t*)p_reg ); - - /* Set the array entry to the registrant. */ - /* The ptr_vector grow automatically as necessary. */ - if( msg_id != CL_DISP_MSGID_NONE ) - { - status = cl_ptr_vector_set( &p_disp->reg_vec, msg_id, p_reg ); - if( status != CL_SUCCESS ) - { - free( p_reg ); - cl_spinlock_release( &p_disp->lock ); - return( NULL ); - } - } - - cl_spinlock_release( &p_disp->lock ); - - return( p_reg ); + cl_disp_reg_info_t *p_reg; + cl_status_t status; + + CL_ASSERT(p_disp); + + /* Check that the requested registrant ID is available. */ + cl_spinlock_acquire(&p_disp->lock); + if ((msg_id != CL_DISP_MSGID_NONE) && + (msg_id < cl_ptr_vector_get_size(&p_disp->reg_vec)) && + (cl_ptr_vector_get(&p_disp->reg_vec, msg_id))) { + cl_spinlock_release(&p_disp->lock); + return (NULL); + } + + /* Get a registration info from the pool. */ + p_reg = (cl_disp_reg_info_t *) malloc(sizeof(cl_disp_reg_info_t)); + if (!p_reg) { + cl_spinlock_release(&p_disp->lock); + return (NULL); + } else { + memset(p_reg, 0, sizeof(cl_disp_reg_info_t)); + } + + p_reg->p_disp = p_disp; + p_reg->ref_cnt = 0; + p_reg->pfn_rcv_callback = pfn_callback; + p_reg->context = context; + p_reg->msg_id = msg_id; + + /* Insert the registration in the list. */ + cl_qlist_insert_tail(&p_disp->reg_list, (cl_list_item_t *) p_reg); + + /* Set the array entry to the registrant. */ + /* The ptr_vector grow automatically as necessary. */ + if (msg_id != CL_DISP_MSGID_NONE) { + status = cl_ptr_vector_set(&p_disp->reg_vec, msg_id, p_reg); + if (status != CL_SUCCESS) { + free(p_reg); + cl_spinlock_release(&p_disp->lock); + return (NULL); + } + } + + cl_spinlock_release(&p_disp->lock); + + return (p_reg); } /******************************************************************** ********************************************************************/ -void -cl_disp_unregister( - IN const cl_disp_reg_handle_t handle ) +void cl_disp_unregister(IN const cl_disp_reg_handle_t handle) { - cl_disp_reg_info_t *p_reg; - cl_dispatcher_t *p_disp; - - if( handle == CL_DISP_INVALID_HANDLE ) - return; - - p_reg = (cl_disp_reg_info_t*)handle; - p_disp = p_reg->p_disp; - CL_ASSERT( p_disp ); - - cl_spinlock_acquire( &p_disp->lock ); - /* - * Clear the registrant vector entry. This will cause any further - * post calls to fail. - */ - if( p_reg->msg_id != CL_DISP_MSGID_NONE ) - { - CL_ASSERT( p_reg->msg_id < cl_ptr_vector_get_size(&p_disp->reg_vec)); - cl_ptr_vector_set( &p_disp->reg_vec, p_reg->msg_id, NULL ); - } - cl_spinlock_release( &p_disp->lock ); - - while( p_reg->ref_cnt > 0) - cl_thread_suspend( 1 ); - - cl_spinlock_acquire(&p_disp->lock); - /* Remove the registrant from the list. */ - cl_qlist_remove_item( &p_disp->reg_list, (cl_list_item_t*)p_reg ); - /* Return the registration info to the pool */ - free( p_reg ); - - cl_spinlock_release( &p_disp->lock ); + cl_disp_reg_info_t *p_reg; + cl_dispatcher_t *p_disp; + + if (handle == CL_DISP_INVALID_HANDLE) + return; + + p_reg = (cl_disp_reg_info_t *) handle; + p_disp = p_reg->p_disp; + CL_ASSERT(p_disp); + + cl_spinlock_acquire(&p_disp->lock); + /* + * Clear the registrant vector entry. This will cause any further + * post calls to fail. + */ + if (p_reg->msg_id != CL_DISP_MSGID_NONE) { + CL_ASSERT(p_reg->msg_id < + cl_ptr_vector_get_size(&p_disp->reg_vec)); + cl_ptr_vector_set(&p_disp->reg_vec, p_reg->msg_id, NULL); + } + cl_spinlock_release(&p_disp->lock); + + while (p_reg->ref_cnt > 0) + cl_thread_suspend(1); + + cl_spinlock_acquire(&p_disp->lock); + /* Remove the registrant from the list. */ + cl_qlist_remove_item(&p_disp->reg_list, (cl_list_item_t *) p_reg); + /* Return the registration info to the pool */ + free(p_reg); + + cl_spinlock_release(&p_disp->lock); } /******************************************************************** ********************************************************************/ cl_status_t -cl_disp_post( - IN const cl_disp_reg_handle_t handle, - IN const cl_disp_msgid_t msg_id, - IN const void* const p_data, - IN cl_pfn_msgdone_cb_t pfn_callback OPTIONAL, - IN const void* const context OPTIONAL ) +cl_disp_post(IN const cl_disp_reg_handle_t handle, + IN const cl_disp_msgid_t msg_id, + IN const void *const p_data, + IN cl_pfn_msgdone_cb_t pfn_callback OPTIONAL, + IN const void *const context OPTIONAL) { - cl_disp_reg_info_t *p_src_reg = (cl_disp_reg_info_t*)handle; - cl_disp_reg_info_t *p_dest_reg; - cl_dispatcher_t *p_disp; - cl_disp_msg_t *p_msg; - - p_disp = handle->p_disp; - CL_ASSERT( p_disp ); - CL_ASSERT( msg_id != CL_DISP_MSGID_NONE ); - - cl_spinlock_acquire( &p_disp->lock ); - /* Check that the recipient exists. */ - p_dest_reg = cl_ptr_vector_get( &p_disp->reg_vec, msg_id ); - if( !p_dest_reg ) - { - cl_spinlock_release( &p_disp->lock ); - return( CL_NOT_FOUND ); - } - - /* Get a free message from the pool. */ - p_msg = (cl_disp_msg_t*)cl_qpool_get( &p_disp->msg_pool ); - if( !p_msg ) - { - cl_spinlock_release( &p_disp->lock ); - return( CL_INSUFFICIENT_MEMORY ); - } - - /* Initialize the message */ - p_msg->p_src_reg = p_src_reg; - p_msg->p_dest_reg = p_dest_reg; - p_msg->p_data = p_data; - p_msg->pfn_xmt_callback = pfn_callback; - p_msg->context = context; - p_msg->in_time = cl_get_time_stamp(); - - /* - * Increment the sender's reference count if they request a completion - * notification. - */ - if( pfn_callback ) - cl_atomic_inc( &p_src_reg->ref_cnt ); - - /* Increment the recipient's reference count. */ - cl_atomic_inc( &p_dest_reg->ref_cnt ); - - /* Queue the message in the FIFO. */ - cl_qlist_insert_tail( &p_disp->msg_fifo, (cl_list_item_t *)p_msg ); - cl_spinlock_release( &p_disp->lock ); - - /* Signal the thread pool that there is work to be done. */ - cl_thread_pool_signal( &p_disp->worker_threads ); - return( CL_SUCCESS ); + cl_disp_reg_info_t *p_src_reg = (cl_disp_reg_info_t *) handle; + cl_disp_reg_info_t *p_dest_reg; + cl_dispatcher_t *p_disp; + cl_disp_msg_t *p_msg; + + p_disp = handle->p_disp; + CL_ASSERT(p_disp); + CL_ASSERT(msg_id != CL_DISP_MSGID_NONE); + + cl_spinlock_acquire(&p_disp->lock); + /* Check that the recipient exists. */ + p_dest_reg = cl_ptr_vector_get(&p_disp->reg_vec, msg_id); + if (!p_dest_reg) { + cl_spinlock_release(&p_disp->lock); + return (CL_NOT_FOUND); + } + + /* Get a free message from the pool. */ + p_msg = (cl_disp_msg_t *) cl_qpool_get(&p_disp->msg_pool); + if (!p_msg) { + cl_spinlock_release(&p_disp->lock); + return (CL_INSUFFICIENT_MEMORY); + } + + /* Initialize the message */ + p_msg->p_src_reg = p_src_reg; + p_msg->p_dest_reg = p_dest_reg; + p_msg->p_data = p_data; + p_msg->pfn_xmt_callback = pfn_callback; + p_msg->context = context; + p_msg->in_time = cl_get_time_stamp(); + + /* + * Increment the sender's reference count if they request a completion + * notification. + */ + if (pfn_callback) + cl_atomic_inc(&p_src_reg->ref_cnt); + + /* Increment the recipient's reference count. */ + cl_atomic_inc(&p_dest_reg->ref_cnt); + + /* Queue the message in the FIFO. */ + cl_qlist_insert_tail(&p_disp->msg_fifo, (cl_list_item_t *) p_msg); + cl_spinlock_release(&p_disp->lock); + + /* Signal the thread pool that there is work to be done. */ + cl_thread_pool_signal(&p_disp->worker_threads); + return (CL_SUCCESS); } void -cl_disp_get_queue_status( - IN const cl_disp_reg_handle_t handle, - OUT uint32_t *p_num_queued_msgs, - OUT uint64_t *p_last_msg_queue_time_ms) +cl_disp_get_queue_status(IN const cl_disp_reg_handle_t handle, + OUT uint32_t * p_num_queued_msgs, + OUT uint64_t * p_last_msg_queue_time_ms) { - cl_dispatcher_t *p_disp = ((cl_disp_reg_info_t*)handle)->p_disp; + cl_dispatcher_t *p_disp = ((cl_disp_reg_info_t *) handle)->p_disp; - cl_spinlock_acquire( &p_disp->lock ); + cl_spinlock_acquire(&p_disp->lock); - if (p_last_msg_queue_time_ms) - *p_last_msg_queue_time_ms = p_disp->last_msg_queue_time_us/1000; + if (p_last_msg_queue_time_ms) + *p_last_msg_queue_time_ms = + p_disp->last_msg_queue_time_us / 1000; - if (p_num_queued_msgs) - *p_num_queued_msgs = cl_qlist_count( &p_disp->msg_fifo ); + if (p_num_queued_msgs) + *p_num_queued_msgs = cl_qlist_count(&p_disp->msg_fifo); - cl_spinlock_release( &p_disp->lock ); + cl_spinlock_release(&p_disp->lock); } diff --git a/opensm/complib/cl_event.c b/opensm/complib/cl_event.c index 20f0992..1b38654 100644 --- a/opensm/complib/cl_event.c +++ b/opensm/complib/cl_event.c @@ -35,158 +35,142 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include #include #include -void -cl_event_construct( - IN cl_event_t *p_event ) +void cl_event_construct(IN cl_event_t * p_event) { - CL_ASSERT( p_event ); + CL_ASSERT(p_event); - cl_spinlock_construct( &p_event->spinlock ); + cl_spinlock_construct(&p_event->spinlock); p_event->state = CL_UNINITIALIZED; } cl_status_t -cl_event_init( - IN cl_event_t* const p_event, - IN const boolean_t manual_reset ) +cl_event_init(IN cl_event_t * const p_event, IN const boolean_t manual_reset) { - CL_ASSERT( p_event ); + CL_ASSERT(p_event); - cl_event_construct( p_event ); + cl_event_construct(p_event); - pthread_cond_init( &p_event->condvar, NULL ); - cl_spinlock_init( &p_event->spinlock ); + pthread_cond_init(&p_event->condvar, NULL); + cl_spinlock_init(&p_event->spinlock); p_event->signaled = FALSE; p_event->manual_reset = manual_reset; p_event->state = CL_INITIALIZED; - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -cl_event_destroy( - IN cl_event_t* const p_event ) +void cl_event_destroy(IN cl_event_t * const p_event) { - CL_ASSERT( cl_is_state_valid( p_event->state ) ); + CL_ASSERT(cl_is_state_valid(p_event->state)); /* Destroy only if the event was constructed */ - if( p_event->state == CL_INITIALIZED ) - { - pthread_cond_broadcast( &p_event->condvar ); - pthread_cond_destroy( &p_event->condvar ); + if (p_event->state == CL_INITIALIZED) { + pthread_cond_broadcast(&p_event->condvar); + pthread_cond_destroy(&p_event->condvar); } - cl_spinlock_destroy( &p_event->spinlock ); + cl_spinlock_destroy(&p_event->spinlock); p_event->state = CL_UNINITIALIZED; } -cl_status_t -cl_event_signal( - IN cl_event_t* const p_event ) +cl_status_t cl_event_signal(IN cl_event_t * const p_event) { /* Make sure that the event was started */ - CL_ASSERT( p_event->state == CL_INITIALIZED ); + CL_ASSERT(p_event->state == CL_INITIALIZED); - cl_spinlock_acquire( &p_event->spinlock ); + cl_spinlock_acquire(&p_event->spinlock); p_event->signaled = TRUE; /* Wake up one or all depending on whether the event is auto-resetting. */ - if( p_event->manual_reset ) - pthread_cond_broadcast( &p_event->condvar ); + if (p_event->manual_reset) + pthread_cond_broadcast(&p_event->condvar); else - pthread_cond_signal( &p_event->condvar ); + pthread_cond_signal(&p_event->condvar); - cl_spinlock_release( &p_event->spinlock ); + cl_spinlock_release(&p_event->spinlock); - return( CL_SUCCESS ); + return (CL_SUCCESS); } -cl_status_t -cl_event_reset( - IN cl_event_t* const p_event ) +cl_status_t cl_event_reset(IN cl_event_t * const p_event) { /* Make sure that the event was started */ - CL_ASSERT( p_event->state == CL_INITIALIZED ); + CL_ASSERT(p_event->state == CL_INITIALIZED); - cl_spinlock_acquire( &p_event->spinlock ); + cl_spinlock_acquire(&p_event->spinlock); p_event->signaled = FALSE; - cl_spinlock_release( &p_event->spinlock ); + cl_spinlock_release(&p_event->spinlock); - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_event_wait_on( - IN cl_event_t* const p_event, - IN const uint32_t wait_us, - IN const boolean_t interruptible ) +cl_event_wait_on(IN cl_event_t * const p_event, + IN const uint32_t wait_us, IN const boolean_t interruptible) { - cl_status_t status; - int wait_ret; - struct timespec timeout; - struct timeval curtime; + cl_status_t status; + int wait_ret; + struct timespec timeout; + struct timeval curtime; /* Make sure that the event was Started */ - CL_ASSERT( p_event->state == CL_INITIALIZED ); + CL_ASSERT(p_event->state == CL_INITIALIZED); - cl_spinlock_acquire( &p_event->spinlock ); + cl_spinlock_acquire(&p_event->spinlock); /* Return immediately if the event is signalled. */ - if( p_event->signaled ) - { - if( !p_event->manual_reset ) + if (p_event->signaled) { + if (!p_event->manual_reset) p_event->signaled = FALSE; - cl_spinlock_release( &p_event->spinlock ); - return( CL_SUCCESS ); + cl_spinlock_release(&p_event->spinlock); + return (CL_SUCCESS); } /* If just testing the state, return CL_TIMEOUT. */ - if( wait_us == 0 ) - { - cl_spinlock_release( &p_event->spinlock ); - return( CL_TIMEOUT ); + if (wait_us == 0) { + cl_spinlock_release(&p_event->spinlock); + return (CL_TIMEOUT); } - if( wait_us == EVENT_NO_TIMEOUT ) - { + if (wait_us == EVENT_NO_TIMEOUT) { /* Wait for condition variable to be signaled or broadcast. */ - if( pthread_cond_wait( &p_event->condvar, &p_event->spinlock.mutex ) ) + if (pthread_cond_wait + (&p_event->condvar, &p_event->spinlock.mutex)) status = CL_NOT_DONE; else status = CL_SUCCESS; - } - else - { + } else { /* Get the current time */ - if( gettimeofday( &curtime, NULL ) == 0 ) - { + if (gettimeofday(&curtime, NULL) == 0) { timeout.tv_sec = curtime.tv_sec + (wait_us / 1000000); - timeout.tv_nsec = (curtime.tv_usec + (wait_us % 1000000)) * 1000; - - wait_ret = pthread_cond_timedwait( &p_event->condvar, - &p_event->spinlock.mutex, &timeout ); - if( wait_ret == 0 ) - status = (p_event->signaled ? CL_SUCCESS : CL_NOT_DONE); - else if( wait_ret == ETIMEDOUT ) + timeout.tv_nsec = + (curtime.tv_usec + (wait_us % 1000000)) * 1000; + + wait_ret = pthread_cond_timedwait(&p_event->condvar, + &p_event->spinlock. + mutex, &timeout); + if (wait_ret == 0) + status = + (p_event-> + signaled ? CL_SUCCESS : CL_NOT_DONE); + else if (wait_ret == ETIMEDOUT) status = CL_TIMEOUT; else status = CL_NOT_DONE; - } - else - { + } else { status = CL_ERROR; } } - if( !p_event->manual_reset ) + if (!p_event->manual_reset) p_event->signaled = FALSE; - cl_spinlock_release( &p_event->spinlock ); - return( status ); + cl_spinlock_release(&p_event->spinlock); + return (status); } diff --git a/opensm/complib/cl_event_wheel.c b/opensm/complib/cl_event_wheel.c index d5f4142..4ad7817 100644 --- a/opensm/complib/cl_event_wheel.c +++ b/opensm/complib/cl_event_wheel.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -43,547 +43,527 @@ #include cl_status_t -__event_will_age_before( - IN const cl_list_item_t* const p_list_item, - IN void* context ) +__event_will_age_before(IN const cl_list_item_t * const p_list_item, + IN void *context) { - uint64_t aging_time = *((uint64_t*)context); - cl_event_wheel_reg_info_t *p_event; + uint64_t aging_time = *((uint64_t *) context); + cl_event_wheel_reg_info_t *p_event; - p_event = - PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - if (p_event->aging_time < aging_time) - return CL_SUCCESS; - else - return CL_NOT_FOUND; + if (p_event->aging_time < aging_time) + return CL_SUCCESS; + else + return CL_NOT_FOUND; } -void -__cl_event_wheel_callback( IN void* context ) +void __cl_event_wheel_callback(IN void *context) { - cl_event_wheel_t *p_event_wheel = (cl_event_wheel_t *)context; - cl_list_item_t *p_list_item, *p_prev_event_list_item; - cl_list_item_t *p_list_next_item; - cl_event_wheel_reg_info_t *p_event; - uint64_t current_time; - uint64_t next_aging_time; - uint32_t new_timeout; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_event_wheel->p_log, __cl_event_wheel_callback); - - /* might be during closing ... */ - if (p_event_wheel->closing) - { - goto JustExit; - } - - current_time = cl_get_time_stamp(); - - if (NULL != p_event_wheel->p_external_lock) { - - /* Take care of the order of acquiring locks to avoid the deadlock! - * The external lock goes first. - */ - CL_SPINLOCK_ACQUIRE(p_event_wheel->p_external_lock); - } - - CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); - - p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); - if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) - { - /* the list is empty - nothing to do */ - goto Exit; - } - - /* we found such an item. get the p_event */ - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - - while (p_event->aging_time <= current_time) - { - /* this object has aged - invoke it's callback */ - if (p_event->pfn_aged_callback) - { - next_aging_time = - p_event->pfn_aged_callback( - p_event->key, p_event->num_regs, p_event->context); - } - else - { - next_aging_time = 0; - } - - /* point to the next object in the wheel */ - p_list_next_item = cl_qlist_next(p_list_item); - - /* We need to retire the event if the next aging time passed */ - if (next_aging_time < current_time) - { - /* remove it from the map */ - cl_qmap_remove_item(&p_event_wheel->events_map, &(p_event->map_item)); - - /* pop p_event from the wheel */ - cl_qlist_remove_head(&p_event_wheel->events_wheel); - - /* delete the event info object - allocated by cl_event_wheel_reg */ - free(p_event); - } - else - { - /* update the required aging time */ - p_event->aging_time = next_aging_time; - p_event->num_regs++; - - /* do not remove from the map - but remove from the list head and - place in the correct position */ - - /* pop p_event from the wheel */ - cl_qlist_remove_head(&p_event_wheel->events_wheel); - - /* find the event that ages just before */ - p_prev_event_list_item = cl_qlist_find_from_tail( - &p_event_wheel->events_wheel, - __event_will_age_before, - &p_event->aging_time); - - /* insert just after */ - cl_qlist_insert_next( - &p_event_wheel->events_wheel, - p_prev_event_list_item , - &p_event->list_item); - - /* as we have modified the list - restart from first item: */ - p_list_next_item = cl_qlist_head(&p_event_wheel->events_wheel); - } - - /* advance to next event */ - p_list_item = p_list_next_item; - if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) - { - /* the list is empty - nothing to do */ - break; - } - - /* get the p_event */ - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - } - - /* We need to restart the timer only if the list is not empty now */ - if (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) - { - /* get the p_event */ - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - - /* start the timer to the timeout [msec] */ - new_timeout = (uint32_t)(((p_event->aging_time - current_time)/1000)+0.5); - osm_log (p_event_wheel->p_log, OSM_LOG_DEBUG, - "__cl_event_wheel_callback : " - "Restart timer in : %u [msec]\n", - new_timeout); - cl_status = cl_timer_start(&p_event_wheel->timer, new_timeout); - if (cl_status != CL_SUCCESS) - { - osm_log (p_event_wheel->p_log, OSM_LOG_ERROR, - "__cl_event_wheel_callback : ERR 6100: " - "Failed to start timer\n" ); - } - } - - /* release the lock */ - Exit: - CL_SPINLOCK_RELEASE( &p_event_wheel->lock ); - if (NULL != p_event_wheel->p_external_lock) { - CL_SPINLOCK_RELEASE(p_event_wheel->p_external_lock); - } - JustExit: - OSM_LOG_EXIT( p_event_wheel->p_log ); + cl_event_wheel_t *p_event_wheel = (cl_event_wheel_t *) context; + cl_list_item_t *p_list_item, *p_prev_event_list_item; + cl_list_item_t *p_list_next_item; + cl_event_wheel_reg_info_t *p_event; + uint64_t current_time; + uint64_t next_aging_time; + uint32_t new_timeout; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_event_wheel->p_log, __cl_event_wheel_callback); + + /* might be during closing ... */ + if (p_event_wheel->closing) { + goto JustExit; + } + + current_time = cl_get_time_stamp(); + + if (NULL != p_event_wheel->p_external_lock) { + + /* Take care of the order of acquiring locks to avoid the deadlock! + * The external lock goes first. + */ + CL_SPINLOCK_ACQUIRE(p_event_wheel->p_external_lock); + } + + CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + + p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); + if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) { + /* the list is empty - nothing to do */ + goto Exit; + } + + /* we found such an item. get the p_event */ + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); + + while (p_event->aging_time <= current_time) { + /* this object has aged - invoke it's callback */ + if (p_event->pfn_aged_callback) { + next_aging_time = + p_event->pfn_aged_callback(p_event->key, + p_event->num_regs, + p_event->context); + } else { + next_aging_time = 0; + } + + /* point to the next object in the wheel */ + p_list_next_item = cl_qlist_next(p_list_item); + + /* We need to retire the event if the next aging time passed */ + if (next_aging_time < current_time) { + /* remove it from the map */ + cl_qmap_remove_item(&p_event_wheel->events_map, + &(p_event->map_item)); + + /* pop p_event from the wheel */ + cl_qlist_remove_head(&p_event_wheel->events_wheel); + + /* delete the event info object - allocated by cl_event_wheel_reg */ + free(p_event); + } else { + /* update the required aging time */ + p_event->aging_time = next_aging_time; + p_event->num_regs++; + + /* do not remove from the map - but remove from the list head and + place in the correct position */ + + /* pop p_event from the wheel */ + cl_qlist_remove_head(&p_event_wheel->events_wheel); + + /* find the event that ages just before */ + p_prev_event_list_item = + cl_qlist_find_from_tail(&p_event_wheel-> + events_wheel, + __event_will_age_before, + &p_event->aging_time); + + /* insert just after */ + cl_qlist_insert_next(&p_event_wheel->events_wheel, + p_prev_event_list_item, + &p_event->list_item); + + /* as we have modified the list - restart from first item: */ + p_list_next_item = + cl_qlist_head(&p_event_wheel->events_wheel); + } + + /* advance to next event */ + p_list_item = p_list_next_item; + if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) { + /* the list is empty - nothing to do */ + break; + } + + /* get the p_event */ + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, + list_item); + } + + /* We need to restart the timer only if the list is not empty now */ + if (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) { + /* get the p_event */ + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, + list_item); + + /* start the timer to the timeout [msec] */ + new_timeout = + (uint32_t) (((p_event->aging_time - current_time) / 1000) + + 0.5); + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "__cl_event_wheel_callback : " + "Restart timer in : %u [msec]\n", new_timeout); + cl_status = cl_timer_start(&p_event_wheel->timer, new_timeout); + if (cl_status != CL_SUCCESS) { + osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, + "__cl_event_wheel_callback : ERR 6100: " + "Failed to start timer\n"); + } + } + + /* release the lock */ + Exit: + CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + if (NULL != p_event_wheel->p_external_lock) { + CL_SPINLOCK_RELEASE(p_event_wheel->p_external_lock); + } + JustExit: + OSM_LOG_EXIT(p_event_wheel->p_log); } /* * Construct and Initialize */ -void -cl_event_wheel_construct( - IN cl_event_wheel_t* const p_event_wheel ) +void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel) { - cl_spinlock_construct( &(p_event_wheel->lock) ); - cl_timer_construct( &(p_event_wheel->timer) ); + cl_spinlock_construct(&(p_event_wheel->lock)); + cl_timer_construct(&(p_event_wheel->timer)); } cl_status_t -cl_event_wheel_init( - IN cl_event_wheel_t* const p_event_wheel, - IN osm_log_t *p_log) +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, + IN osm_log_t * p_log) { - cl_status_t cl_status = CL_SUCCESS; - - OSM_LOG_ENTER( p_log, cl_event_wheel_init ); - - /* initialize */ - p_event_wheel->p_log = p_log; - p_event_wheel->p_external_lock = NULL; - p_event_wheel->closing = FALSE; - cl_status = cl_spinlock_init( &(p_event_wheel->lock) ); - if (cl_status != CL_SUCCESS) - { - osm_log (p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_init : ERR 6101: " - "Failed to initialize cl_spinlock\n" ); - goto Exit; - } - cl_qlist_init( &p_event_wheel->events_wheel); - cl_qmap_init( &p_event_wheel->events_map ); - - /* init the timer with timeout */ - cl_status = cl_timer_init(&p_event_wheel->timer, - __cl_event_wheel_callback, - p_event_wheel ); /* cb context */ - - if (cl_status != CL_SUCCESS) - { - osm_log (p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_init : ERR 6102: " - "Failed to initialize cl_timer\n" ); - goto Exit; - } - Exit: - OSM_LOG_EXIT( p_event_wheel->p_log ); - return(cl_status); + cl_status_t cl_status = CL_SUCCESS; + + OSM_LOG_ENTER(p_log, cl_event_wheel_init); + + /* initialize */ + p_event_wheel->p_log = p_log; + p_event_wheel->p_external_lock = NULL; + p_event_wheel->closing = FALSE; + cl_status = cl_spinlock_init(&(p_event_wheel->lock)); + if (cl_status != CL_SUCCESS) { + osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, + "cl_event_wheel_init : ERR 6101: " + "Failed to initialize cl_spinlock\n"); + goto Exit; + } + cl_qlist_init(&p_event_wheel->events_wheel); + cl_qmap_init(&p_event_wheel->events_map); + + /* init the timer with timeout */ + cl_status = cl_timer_init(&p_event_wheel->timer, __cl_event_wheel_callback, p_event_wheel); /* cb context */ + + if (cl_status != CL_SUCCESS) { + osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, + "cl_event_wheel_init : ERR 6102: " + "Failed to initialize cl_timer\n"); + goto Exit; + } + Exit: + OSM_LOG_EXIT(p_event_wheel->p_log); + return (cl_status); } cl_status_t -cl_event_wheel_init_ex( - IN cl_event_wheel_t* const p_event_wheel, - IN osm_log_t *p_log, - IN cl_spinlock_t *p_external_lock) +cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, + IN osm_log_t * p_log, IN cl_spinlock_t * p_external_lock) { - cl_status_t cl_status; + cl_status_t cl_status; - cl_status = cl_event_wheel_init(p_event_wheel, p_log); - if (CL_SUCCESS != cl_status) - { - return cl_status; - } + cl_status = cl_event_wheel_init(p_event_wheel, p_log); + if (CL_SUCCESS != cl_status) { + return cl_status; + } - p_event_wheel->p_external_lock = p_external_lock; - return cl_status; + p_event_wheel->p_external_lock = p_external_lock; + return cl_status; } -void -cl_event_wheel_dump( - IN cl_event_wheel_t* const p_event_wheel ) +void cl_event_wheel_dump(IN cl_event_wheel_t * const p_event_wheel) { - cl_list_item_t *p_list_item; - cl_event_wheel_reg_info_t *p_event; - - OSM_LOG_ENTER( p_event_wheel->p_log, cl_event_wheel_dump ); - - p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_dump: " - "event_wheel ptr:%p\n", - p_event_wheel); - - while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel) ) - { - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_dump: " - "Found event key:<0x%"PRIx64">, aging time:%" PRIu64"\n", - p_event->key, p_event->aging_time ); - p_list_item = cl_qlist_next( p_list_item ); - } - OSM_LOG_EXIT( p_event_wheel->p_log ); + cl_list_item_t *p_list_item; + cl_event_wheel_reg_info_t *p_event; + + OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_dump); + + p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_dump: " "event_wheel ptr:%p\n", p_event_wheel); + + while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) { + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, + list_item); + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_dump: " "Found event key:<0x%" PRIx64 + ">, aging time:%" PRIu64 "\n", p_event->key, + p_event->aging_time); + p_list_item = cl_qlist_next(p_list_item); + } + OSM_LOG_EXIT(p_event_wheel->p_log); } -void -cl_event_wheel_destroy( - IN cl_event_wheel_t* const p_event_wheel ) +void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel) { - cl_list_item_t *p_list_item; - cl_map_item_t *p_map_item; - cl_event_wheel_reg_info_t *p_event; - - OSM_LOG_ENTER( p_event_wheel->p_log, cl_event_wheel_destroy ); - - /* we need to get a lock */ - CL_SPINLOCK_ACQUIRE( &p_event_wheel->lock ); - - cl_event_wheel_dump( p_event_wheel); - - /* go over all the items in the list and remove them */ - p_list_item = cl_qlist_remove_head(&p_event_wheel->events_wheel); - while ( p_list_item != cl_qlist_end(&p_event_wheel->events_wheel) ) - { - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_destroy: " - "Found outstanding event key:<0x%"PRIx64">\n", - p_event->key ); - - /* remove it from the map */ - p_map_item = &(p_event->map_item); - cl_qmap_remove_item(&p_event_wheel->events_map, p_map_item); - free(p_event); /* allocated by cl_event_wheel_reg */ - p_list_item = cl_qlist_remove_head(&p_event_wheel->events_wheel); - } - - /* destroy the timer */ - cl_timer_destroy( &p_event_wheel->timer ); - - /* destroy the lock (this should be done without releasing - we don't want - any other run to grab the lock at this point. */ - CL_SPINLOCK_RELEASE( &p_event_wheel->lock ); - cl_spinlock_destroy( &(p_event_wheel->lock) ); - - OSM_LOG_EXIT( p_event_wheel->p_log ); + cl_list_item_t *p_list_item; + cl_map_item_t *p_map_item; + cl_event_wheel_reg_info_t *p_event; + + OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_destroy); + + /* we need to get a lock */ + CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + + cl_event_wheel_dump(p_event_wheel); + + /* go over all the items in the list and remove them */ + p_list_item = cl_qlist_remove_head(&p_event_wheel->events_wheel); + while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) { + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, + list_item); + + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_destroy: " + "Found outstanding event key:<0x%" PRIx64 ">\n", + p_event->key); + + /* remove it from the map */ + p_map_item = &(p_event->map_item); + cl_qmap_remove_item(&p_event_wheel->events_map, p_map_item); + free(p_event); /* allocated by cl_event_wheel_reg */ + p_list_item = + cl_qlist_remove_head(&p_event_wheel->events_wheel); + } + + /* destroy the timer */ + cl_timer_destroy(&p_event_wheel->timer); + + /* destroy the lock (this should be done without releasing - we don't want + any other run to grab the lock at this point. */ + CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + cl_spinlock_destroy(&(p_event_wheel->lock)); + + OSM_LOG_EXIT(p_event_wheel->p_log); } cl_status_t -cl_event_wheel_reg( - IN cl_event_wheel_t* const p_event_wheel, - IN const uint64_t key, - IN const uint64_t aging_time_usec, - IN cl_pfn_event_aged_cb_t pfn_callback, - IN void* const context ) +cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, + IN const uint64_t key, + IN const uint64_t aging_time_usec, + IN cl_pfn_event_aged_cb_t pfn_callback, + IN void *const context) { - cl_event_wheel_reg_info_t *p_event; - uint64_t timeout; - uint32_t to; - cl_status_t cl_status = CL_SUCCESS; - cl_list_item_t *prev_event_list_item; - cl_map_item_t *p_map_item; - - OSM_LOG_ENTER( p_event_wheel->p_log, cl_event_wheel_reg ); - - /* Get the lock on the manager */ - CL_SPINLOCK_ACQUIRE( &(p_event_wheel->lock) ); - - cl_event_wheel_dump( p_event_wheel); - - /* Make sure such a key does not exists */ - p_map_item = cl_qmap_get( &p_event_wheel->events_map, key ); - if (p_map_item != cl_qmap_end( &p_event_wheel->events_map )) - { - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_reg: " - "Already exists key:0x%"PRIx64"\n", key); - - /* already there - remove it from the list as it is getting a new time */ - p_event = PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, map_item); - - /* remove the item from the qlist */ - cl_qlist_remove_item( &p_event_wheel->events_wheel, &p_event->list_item ); - /* and the qmap */ - cl_qmap_remove_item( &p_event_wheel->events_map, &p_event->map_item ); - } - else - { - /* make a new one */ - p_event = (cl_event_wheel_reg_info_t *) - malloc( sizeof (cl_event_wheel_reg_info_t) ); - p_event->num_regs = 0; - } - - p_event->key = key; - p_event->aging_time = aging_time_usec; - p_event->pfn_aged_callback = pfn_callback; - p_event->context = context; - p_event->num_regs++; - - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_reg: " - "Registering event key:0x%"PRIx64" aging in %u [msec]\n", - p_event->key, - (uint32_t)((p_event->aging_time - cl_get_time_stamp()) / 1000 )); - - /* If the list is empty - need to start the timer */ - if (cl_is_qlist_empty(&p_event_wheel->events_wheel)) - { - /* Edward Bortnikov 03/29/2003 - * ++TBD Consider moving the timer manipulation behind the list manipulation. - */ - - /* calculate the new timeout */ - timeout = (p_event->aging_time - cl_get_time_stamp() + 500) / 1000; - - /* stop the timer if it is running */ - - /* Edward Bortnikov 03/29/2003 - * Don't call cl_timer_stop() because it spins forever. - * cl_timer_start() will invoke cl_timer_stop() by itself. - * - * The problematic scenario is when __cl_event_wheel_callback() - * is in race condition with this code. It sets timer.in_timer_cb - * to TRUE and then blocks on p_event_wheel->lock. Following this, - * the call to cl_timer_stop() hangs. Following this, the whole system - * enters into a deadlock. - * - * cl_timer_stop(&p_event_wheel->timer); - */ - - /* The timeout for the cl_timer_start should be given as uint32_t. - if there is an overflow - warn about it. */ - to = (uint32_t)timeout; - if ( timeout > (uint32_t)timeout ) - { - to = 0xffffffff; /* max 32 bit timer */ - osm_log (p_event_wheel->p_log, OSM_LOG_INFO, - "cl_event_wheel_reg: " - "timeout requested is too large. Using timeout: %u\n", - to ); - } - - /* start the timer to the timeout [msec] */ - cl_status = cl_timer_start(&p_event_wheel->timer, to); - if (cl_status != CL_SUCCESS) - { - osm_log (p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_reg : ERR 6103: " - "Failed to start timer\n" ); - goto Exit; - } - } - - /* insert the object to the qlist and the qmap */ - - /* BUT WE MUST INSERT IT IN A SORTED MANNER */ - prev_event_list_item = cl_qlist_find_from_tail( - &p_event_wheel->events_wheel, - __event_will_age_before, - &p_event->aging_time); - - cl_qlist_insert_next( - &p_event_wheel->events_wheel, - prev_event_list_item , - &p_event->list_item); - - cl_qmap_insert( &p_event_wheel->events_map, key, &(p_event->map_item)); - - Exit: - CL_SPINLOCK_RELEASE( &p_event_wheel->lock ); - OSM_LOG_EXIT( p_event_wheel->p_log ); - - return cl_status; + cl_event_wheel_reg_info_t *p_event; + uint64_t timeout; + uint32_t to; + cl_status_t cl_status = CL_SUCCESS; + cl_list_item_t *prev_event_list_item; + cl_map_item_t *p_map_item; + + OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_reg); + + /* Get the lock on the manager */ + CL_SPINLOCK_ACQUIRE(&(p_event_wheel->lock)); + + cl_event_wheel_dump(p_event_wheel); + + /* Make sure such a key does not exists */ + p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); + if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_reg: " + "Already exists key:0x%" PRIx64 "\n", key); + + /* already there - remove it from the list as it is getting a new time */ + p_event = + PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, + map_item); + + /* remove the item from the qlist */ + cl_qlist_remove_item(&p_event_wheel->events_wheel, + &p_event->list_item); + /* and the qmap */ + cl_qmap_remove_item(&p_event_wheel->events_map, + &p_event->map_item); + } else { + /* make a new one */ + p_event = (cl_event_wheel_reg_info_t *) + malloc(sizeof(cl_event_wheel_reg_info_t)); + p_event->num_regs = 0; + } + + p_event->key = key; + p_event->aging_time = aging_time_usec; + p_event->pfn_aged_callback = pfn_callback; + p_event->context = context; + p_event->num_regs++; + + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_reg: " + "Registering event key:0x%" PRIx64 " aging in %u [msec]\n", + p_event->key, + (uint32_t) ((p_event->aging_time - + cl_get_time_stamp()) / 1000)); + + /* If the list is empty - need to start the timer */ + if (cl_is_qlist_empty(&p_event_wheel->events_wheel)) { + /* Edward Bortnikov 03/29/2003 + * ++TBD Consider moving the timer manipulation behind the list manipulation. + */ + + /* calculate the new timeout */ + timeout = + (p_event->aging_time - cl_get_time_stamp() + 500) / 1000; + + /* stop the timer if it is running */ + + /* Edward Bortnikov 03/29/2003 + * Don't call cl_timer_stop() because it spins forever. + * cl_timer_start() will invoke cl_timer_stop() by itself. + * + * The problematic scenario is when __cl_event_wheel_callback() + * is in race condition with this code. It sets timer.in_timer_cb + * to TRUE and then blocks on p_event_wheel->lock. Following this, + * the call to cl_timer_stop() hangs. Following this, the whole system + * enters into a deadlock. + * + * cl_timer_stop(&p_event_wheel->timer); + */ + + /* The timeout for the cl_timer_start should be given as uint32_t. + if there is an overflow - warn about it. */ + to = (uint32_t) timeout; + if (timeout > (uint32_t) timeout) { + to = 0xffffffff; /* max 32 bit timer */ + osm_log(p_event_wheel->p_log, OSM_LOG_INFO, + "cl_event_wheel_reg: " + "timeout requested is too large. Using timeout: %u\n", + to); + } + + /* start the timer to the timeout [msec] */ + cl_status = cl_timer_start(&p_event_wheel->timer, to); + if (cl_status != CL_SUCCESS) { + osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, + "cl_event_wheel_reg : ERR 6103: " + "Failed to start timer\n"); + goto Exit; + } + } + + /* insert the object to the qlist and the qmap */ + + /* BUT WE MUST INSERT IT IN A SORTED MANNER */ + prev_event_list_item = + cl_qlist_find_from_tail(&p_event_wheel->events_wheel, + __event_will_age_before, + &p_event->aging_time); + + cl_qlist_insert_next(&p_event_wheel->events_wheel, + prev_event_list_item, &p_event->list_item); + + cl_qmap_insert(&p_event_wheel->events_map, key, &(p_event->map_item)); + + Exit: + CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + OSM_LOG_EXIT(p_event_wheel->p_log); + + return cl_status; } void -cl_event_wheel_unreg( - IN cl_event_wheel_t* const p_event_wheel, - IN uint64_t key ) +cl_event_wheel_unreg(IN cl_event_wheel_t * const p_event_wheel, IN uint64_t key) { - cl_event_wheel_reg_info_t *p_event; - cl_map_item_t* p_map_item; - - OSM_LOG_ENTER( p_event_wheel->p_log, cl_event_wheel_unreg ); - - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " - "Removing key:0x%"PRIx64"\n", key ); - - CL_SPINLOCK_ACQUIRE( &p_event_wheel->lock ); - p_map_item = cl_qmap_get( &p_event_wheel->events_map, key ); - if (p_map_item != cl_qmap_end( &p_event_wheel->events_map )) - { - /* we found such an item. */ - p_event = PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, map_item); - - /* remove the item from the qlist */ - cl_qlist_remove_item( &p_event_wheel->events_wheel, &(p_event->list_item)); - /* remove the item from the qmap */ - cl_qmap_remove_item( &p_event_wheel->events_map, &(p_event->map_item) ); - - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " - "Removed key:0x%"PRIx64"\n", key ); - - /* free the item */ - free(p_event); - } - else - { - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " - "Did not find key:0x%"PRIx64"\n", key ); - } - - CL_SPINLOCK_RELEASE( &p_event_wheel->lock ); - OSM_LOG_EXIT( p_event_wheel->p_log ); + cl_event_wheel_reg_info_t *p_event; + cl_map_item_t *p_map_item; + + OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_unreg); + + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_unreg: " "Removing key:0x%" PRIx64 "\n", key); + + CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); + if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { + /* we found such an item. */ + p_event = + PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, + map_item); + + /* remove the item from the qlist */ + cl_qlist_remove_item(&p_event_wheel->events_wheel, + &(p_event->list_item)); + /* remove the item from the qmap */ + cl_qmap_remove_item(&p_event_wheel->events_map, + &(p_event->map_item)); + + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_unreg: " + "Removed key:0x%" PRIx64 "\n", key); + + /* free the item */ + free(p_event); + } else { + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_unreg: " + "Did not find key:0x%" PRIx64 "\n", key); + } + + CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + OSM_LOG_EXIT(p_event_wheel->p_log); } uint32_t -cl_event_wheel_num_regs( - IN cl_event_wheel_t* const p_event_wheel, - IN uint64_t key ) +cl_event_wheel_num_regs(IN cl_event_wheel_t * const p_event_wheel, + IN uint64_t key) { - cl_event_wheel_reg_info_t *p_event; - cl_map_item_t* p_map_item; - uint32_t num_regs = 0; - - OSM_LOG_ENTER( p_event_wheel->p_log, cl_event_wheel_num_regs ); - - /* try to find the key in the map */ - osm_log( p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_num_regs: " - "Looking for key:0x%"PRIx64"\n", key ); - - CL_SPINLOCK_ACQUIRE( &p_event_wheel->lock ); - p_map_item = cl_qmap_get( &p_event_wheel->events_map, key ); - if (p_map_item != cl_qmap_end( &p_event_wheel->events_map )) - { - /* ok so we can simply return it's num_regs */ - p_event = PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, map_item); - num_regs = p_event->num_regs; - } - - CL_SPINLOCK_RELEASE( &p_event_wheel->lock ); - OSM_LOG_EXIT( p_event_wheel->p_log ); - return(num_regs); + cl_event_wheel_reg_info_t *p_event; + cl_map_item_t *p_map_item; + uint32_t num_regs = 0; + + OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_num_regs); + + /* try to find the key in the map */ + osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, + "cl_event_wheel_num_regs: " + "Looking for key:0x%" PRIx64 "\n", key); + + CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); + if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { + /* ok so we can simply return it's num_regs */ + p_event = + PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, + map_item); + num_regs = p_event->num_regs; + } + + CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + OSM_LOG_EXIT(p_event_wheel->p_log); + return (num_regs); } #ifdef __CL_EVENT_WHEEL_TEST__ /* Dump out the complete state of the event wheel */ -void __cl_event_wheel_dump( - IN cl_event_wheel_t* const p_event_wheel) +void __cl_event_wheel_dump(IN cl_event_wheel_t * const p_event_wheel) { - cl_list_item_t *p_list_item; - cl_map_item_t *p_map_item; - cl_event_wheel_reg_info_t *p_event; - - printf("************** Event Wheel Dump ***********************\n"); - printf("Event Wheel List has %u items:\n", - cl_qlist_count( &p_event_wheel->events_wheel )); - - p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); - while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) - { - p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - printf("Event key:0x%"PRIx64" Conetxt:%s NumRegs:%u\n", - p_event->key, (char *)p_event->context, p_event->num_regs); - - /* next */ - p_list_item = cl_qlist_next(p_list_item); - } - - printf("Event Map has %u items:\n", - cl_qmap_count( &p_event_wheel->events_map )); - - p_map_item = cl_qmap_head(&p_event_wheel->events_map); - while (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) - { - p_event = PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, map_item); - printf("Event key:0x%"PRIx64" Conetxt:%s NumRegs:%u\n", - p_event->key, (char *)p_event->context, p_event->num_regs); - - /* next */ - p_map_item = cl_qmap_next(p_map_item); - } + cl_list_item_t *p_list_item; + cl_map_item_t *p_map_item; + cl_event_wheel_reg_info_t *p_event; + + printf("************** Event Wheel Dump ***********************\n"); + printf("Event Wheel List has %u items:\n", + cl_qlist_count(&p_event_wheel->events_wheel)); + + p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); + while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) { + p_event = + PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, + list_item); + printf("Event key:0x%" PRIx64 " Conetxt:%s NumRegs:%u\n", + p_event->key, (char *)p_event->context, + p_event->num_regs); + + /* next */ + p_list_item = cl_qlist_next(p_list_item); + } + + printf("Event Map has %u items:\n", + cl_qmap_count(&p_event_wheel->events_map)); + + p_map_item = cl_qmap_head(&p_event_wheel->events_map); + while (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { + p_event = + PARENT_STRUCT(p_map_item, cl_event_wheel_reg_info_t, + map_item); + printf("Event key:0x%" PRIx64 " Conetxt:%s NumRegs:%u\n", + p_event->key, (char *)p_event->context, + p_event->num_regs); + + /* next */ + p_map_item = cl_qmap_next(p_map_item); + } } @@ -591,73 +571,65 @@ void __cl_event_wheel_dump( /* We assume we pass a text context */ void __test_event_aging(uint64_t key, void *context) { - printf("*****************************************************\n"); - printf("Aged key: 0x%"PRIx64" Conetxt:%s\n", key, (char *)context); + printf("*****************************************************\n"); + printf("Aged key: 0x%" PRIx64 " Conetxt:%s\n", key, (char *)context); } -int -main () +int main() { - osm_log_t log; - cl_event_wheel_t event_wheel; - /* uint64_t key; */ - - /* construct */ - osm_log_construct( &log ); - cl_event_wheel_construct( &event_wheel ); - - /* init */ - osm_log_init_v2( &log, TRUE, 0xff, NULL, 0, FALSE); - cl_event_wheel_init( &event_wheel, &log ); - - /* Start Playing */ - cl_event_wheel_reg( &event_wheel, - 1, /* key */ - cl_get_time_stamp() + 3000000, /* 3 sec lifetime */ - __test_event_aging, /* cb */ - "The first Aging Event" - ); - - cl_event_wheel_reg( &event_wheel, - 2, /* key */ - cl_get_time_stamp() + 3000000, /* 3 sec lifetime */ - __test_event_aging, /* cb */ - "The Second Aging Event" - ); - - cl_event_wheel_reg( &event_wheel, - 3, /* key */ - cl_get_time_stamp() + 3500000, /* 3 sec lifetime */ - __test_event_aging, /* cb */ - "The Third Aging Event" - ); - - __cl_event_wheel_dump(&event_wheel); - - sleep(2); - cl_event_wheel_reg( &event_wheel, - 2, /* key */ - cl_get_time_stamp() + 8000000, /* 3 sec lifetime */ - __test_event_aging, /* cb */ - "The Second Aging Event Moved" - ); - - __cl_event_wheel_dump(&event_wheel); - - sleep(1); - /* remove the third event */ - cl_event_wheel_unreg( &event_wheel, - 3); /* key */ - - /* get the number of registrations for the keys */ - printf("Event 1 Registered: %u\n", cl_event_wheel_num_regs(&event_wheel, 1)); - printf("Event 2 Registered: %u\n", cl_event_wheel_num_regs(&event_wheel, 2)); - - sleep(5); - /* destroy */ - cl_event_wheel_destroy( &event_wheel ); - - return(0); + osm_log_t log; + cl_event_wheel_t event_wheel; + /* uint64_t key; */ + + /* construct */ + osm_log_construct(&log); + cl_event_wheel_construct(&event_wheel); + + /* init */ + osm_log_init_v2(&log, TRUE, 0xff, NULL, 0, FALSE); + cl_event_wheel_init(&event_wheel, &log); + + /* Start Playing */ + cl_event_wheel_reg(&event_wheel, 1, /* key */ + cl_get_time_stamp() + 3000000, /* 3 sec lifetime */ + __test_event_aging, /* cb */ + "The first Aging Event"); + + cl_event_wheel_reg(&event_wheel, 2, /* key */ + cl_get_time_stamp() + 3000000, /* 3 sec lifetime */ + __test_event_aging, /* cb */ + "The Second Aging Event"); + + cl_event_wheel_reg(&event_wheel, 3, /* key */ + cl_get_time_stamp() + 3500000, /* 3 sec lifetime */ + __test_event_aging, /* cb */ + "The Third Aging Event"); + + __cl_event_wheel_dump(&event_wheel); + + sleep(2); + cl_event_wheel_reg(&event_wheel, 2, /* key */ + cl_get_time_stamp() + 8000000, /* 3 sec lifetime */ + __test_event_aging, /* cb */ + "The Second Aging Event Moved"); + + __cl_event_wheel_dump(&event_wheel); + + sleep(1); + /* remove the third event */ + cl_event_wheel_unreg(&event_wheel, 3); /* key */ + + /* get the number of registrations for the keys */ + printf("Event 1 Registered: %u\n", + cl_event_wheel_num_regs(&event_wheel, 1)); + printf("Event 2 Registered: %u\n", + cl_event_wheel_num_regs(&event_wheel, 2)); + + sleep(5); + /* destroy */ + cl_event_wheel_destroy(&event_wheel); + + return (0); } -#endif /* __CL_EVENT_WHEEL_TEST__ */ +#endif /* __CL_EVENT_WHEEL_TEST__ */ diff --git a/opensm/complib/cl_list.c b/opensm/complib/cl_list.c index d0cef8a..7a25e4b 100644 --- a/opensm/complib/cl_list.c +++ b/opensm/complib/cl_list.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -60,110 +60,102 @@ ******************************************************************************* ******************************************************************************/ void -cl_qlist_insert_array_head( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ) +cl_qlist_insert_array_head(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_array, + IN uint32_t item_count, IN const uint32_t item_size) { - cl_list_item_t *p_item; + cl_list_item_t *p_item; - CL_ASSERT( p_list ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); - CL_ASSERT( p_array ); - CL_ASSERT( item_size >= sizeof(cl_list_item_t) ); - CL_ASSERT( item_count ); + CL_ASSERT(p_list); + CL_ASSERT(p_list->state == CL_INITIALIZED); + CL_ASSERT(p_array); + CL_ASSERT(item_size >= sizeof(cl_list_item_t)); + CL_ASSERT(item_count); /* * To add items from the array to the list in the same order as * the elements appear in the array, we add them starting with * the last one first. Locate the last item. */ - p_item = (cl_list_item_t*)( - (uint8_t*)p_array + (item_size * (item_count - 1))); + p_item = (cl_list_item_t *) ((uint8_t *) p_array + + (item_size * (item_count - 1))); /* Continue to add all items to the list. */ - while( item_count-- ) - { - cl_qlist_insert_head( p_list, p_item ); + while (item_count--) { + cl_qlist_insert_head(p_list, p_item); /* Get the next object to add to the list. */ - p_item = (cl_list_item_t*)((uint8_t*)p_item - item_size); + p_item = (cl_list_item_t *) ((uint8_t *) p_item - item_size); } } void -cl_qlist_insert_array_tail( - IN cl_qlist_t* const p_list, - IN cl_list_item_t* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ) +cl_qlist_insert_array_tail(IN cl_qlist_t * const p_list, + IN cl_list_item_t * const p_array, + IN uint32_t item_count, IN const uint32_t item_size) { - cl_list_item_t *p_item; + cl_list_item_t *p_item; - CL_ASSERT( p_list ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); - CL_ASSERT( p_array ); - CL_ASSERT( item_size >= sizeof(cl_list_item_t) ); - CL_ASSERT( item_count ); + CL_ASSERT(p_list); + CL_ASSERT(p_list->state == CL_INITIALIZED); + CL_ASSERT(p_array); + CL_ASSERT(item_size >= sizeof(cl_list_item_t)); + CL_ASSERT(item_count); /* Set the first item to add to the list. */ p_item = p_array; /* Continue to add all items to the list. */ - while( item_count-- ) - { - cl_qlist_insert_tail( p_list, p_item ); + while (item_count--) { + cl_qlist_insert_tail(p_list, p_item); /* Get the next object to add to the list. */ - p_item = (cl_list_item_t*)((uint8_t*)p_item + item_size); + p_item = (cl_list_item_t *) ((uint8_t *) p_item + item_size); } } void -cl_qlist_insert_list_head( - IN cl_qlist_t* const p_dest_list, - IN cl_qlist_t* const p_src_list ) +cl_qlist_insert_list_head(IN cl_qlist_t * const p_dest_list, + IN cl_qlist_t * const p_src_list) { #if defined( _DEBUG_ ) - cl_list_item_t *p_item; + cl_list_item_t *p_item; #endif - CL_ASSERT( p_dest_list ); - CL_ASSERT( p_src_list ); - CL_ASSERT( p_dest_list->state == CL_INITIALIZED ); - CL_ASSERT( p_src_list->state == CL_INITIALIZED ); + CL_ASSERT(p_dest_list); + CL_ASSERT(p_src_list); + CL_ASSERT(p_dest_list->state == CL_INITIALIZED); + CL_ASSERT(p_src_list->state == CL_INITIALIZED); /* * Is the src list empty? * We must have this check here for code below to work. */ - if( cl_is_qlist_empty( p_src_list ) ) + if (cl_is_qlist_empty(p_src_list)) return; #if defined( _DEBUG_ ) /* Check that all items in the source list belong there. */ - p_item = cl_qlist_head( p_src_list ); - while( p_item != cl_qlist_end( p_src_list ) ) - { + p_item = cl_qlist_head(p_src_list); + while (p_item != cl_qlist_end(p_src_list)) { /* All list items in the source list must point to it. */ - CL_ASSERT( p_item->p_list == p_src_list ); + CL_ASSERT(p_item->p_list == p_src_list); /* Point them all to the destination list. */ p_item->p_list = p_dest_list; - p_item = cl_qlist_next( p_item ); + p_item = cl_qlist_next(p_item); } #endif /* Chain the destination list to the tail of the source list. */ - cl_qlist_tail( p_src_list )->p_next = cl_qlist_head( p_dest_list ); - cl_qlist_head( p_dest_list )->p_prev = cl_qlist_tail( p_src_list ); + cl_qlist_tail(p_src_list)->p_next = cl_qlist_head(p_dest_list); + cl_qlist_head(p_dest_list)->p_prev = cl_qlist_tail(p_src_list); /* * Update the head of the destination list to the head of * the source list. */ - p_dest_list->end.p_next = cl_qlist_head( p_src_list ); - cl_qlist_head( p_src_list )->p_prev = &p_dest_list->end; + p_dest_list->end.p_next = cl_qlist_head(p_src_list); + cl_qlist_head(p_src_list)->p_prev = &p_dest_list->end; /* * Update the count of the destination to reflect the source items having @@ -172,53 +164,51 @@ cl_qlist_insert_list_head( p_dest_list->count += p_src_list->count; /* Update source list to reflect being empty. */ - __cl_qlist_reset( p_src_list ); + __cl_qlist_reset(p_src_list); } void -cl_qlist_insert_list_tail( - IN cl_qlist_t* const p_dest_list, - IN cl_qlist_t* const p_src_list ) +cl_qlist_insert_list_tail(IN cl_qlist_t * const p_dest_list, + IN cl_qlist_t * const p_src_list) { #if defined( _DEBUG_ ) - cl_list_item_t *p_item; + cl_list_item_t *p_item; #endif - CL_ASSERT( p_dest_list ); - CL_ASSERT( p_src_list ); - CL_ASSERT( p_dest_list->state == CL_INITIALIZED ); - CL_ASSERT( p_src_list->state == CL_INITIALIZED ); + CL_ASSERT(p_dest_list); + CL_ASSERT(p_src_list); + CL_ASSERT(p_dest_list->state == CL_INITIALIZED); + CL_ASSERT(p_src_list->state == CL_INITIALIZED); /* * Is the src list empty? * We must have this check here for code below to work. */ - if( cl_is_qlist_empty( p_src_list ) ) + if (cl_is_qlist_empty(p_src_list)) return; #if defined( _DEBUG_ ) /* Check that all items in the source list belong there. */ - p_item = cl_qlist_head( p_src_list ); - while( p_item != cl_qlist_end( p_src_list ) ) - { + p_item = cl_qlist_head(p_src_list); + while (p_item != cl_qlist_end(p_src_list)) { /* All list items in the source list must point to it. */ - CL_ASSERT( p_item->p_list == p_src_list ); + CL_ASSERT(p_item->p_list == p_src_list); /* Point them all to the destination list. */ p_item->p_list = p_dest_list; - p_item = cl_qlist_next( p_item ); + p_item = cl_qlist_next(p_item); } #endif /* Chain the source list to the tail of the destination list. */ - cl_qlist_tail( p_dest_list )->p_next = cl_qlist_head( p_src_list ); - cl_qlist_head( p_src_list )->p_prev = cl_qlist_tail( p_dest_list ); + cl_qlist_tail(p_dest_list)->p_next = cl_qlist_head(p_src_list); + cl_qlist_head(p_src_list)->p_prev = cl_qlist_tail(p_dest_list); /* * Update the tail of the destination list to the tail of * the source list. */ - p_dest_list->end.p_prev = cl_qlist_tail( p_src_list ); - cl_qlist_tail( p_src_list )->p_next = &p_dest_list->end; + p_dest_list->end.p_prev = cl_qlist_tail(p_src_list); + cl_qlist_tail(p_src_list)->p_next = &p_dest_list->end; /* * Update the count of the destination to reflect the source items having @@ -227,148 +217,134 @@ cl_qlist_insert_list_tail( p_dest_list->count += p_src_list->count; /* Update source list to reflect being empty. */ - __cl_qlist_reset( p_src_list ); + __cl_qlist_reset(p_src_list); } boolean_t -cl_is_item_in_qlist( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item ) +cl_is_item_in_qlist(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item) { - const cl_list_item_t* p_temp; + const cl_list_item_t *p_temp; - CL_ASSERT( p_list ); - CL_ASSERT( p_list_item ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); + CL_ASSERT(p_list); + CL_ASSERT(p_list_item); + CL_ASSERT(p_list->state == CL_INITIALIZED); /* Traverse looking for a match */ - p_temp = cl_qlist_head( p_list ); - while( p_temp != cl_qlist_end( p_list ) ) - { - if( p_temp == p_list_item ) - { - CL_ASSERT( p_list_item->p_list == p_list ); - return( TRUE ); + p_temp = cl_qlist_head(p_list); + while (p_temp != cl_qlist_end(p_list)) { + if (p_temp == p_list_item) { + CL_ASSERT(p_list_item->p_list == p_list); + return (TRUE); } - p_temp = cl_qlist_next( p_temp ); + p_temp = cl_qlist_next(p_temp); } - return( FALSE ); + return (FALSE); } -cl_list_item_t* -cl_qlist_find_next( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ) +cl_list_item_t *cl_qlist_find_next(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context) { - cl_list_item_t *p_found_item; + cl_list_item_t *p_found_item; - CL_ASSERT( p_list ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); - CL_ASSERT( p_list_item ); - CL_ASSERT( p_list_item->p_list == p_list ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_list); + CL_ASSERT(p_list->state == CL_INITIALIZED); + CL_ASSERT(p_list_item); + CL_ASSERT(p_list_item->p_list == p_list); + CL_ASSERT(pfn_func); - p_found_item = cl_qlist_next( p_list_item ); + p_found_item = cl_qlist_next(p_list_item); /* The user provided a compare function */ - while( p_found_item != cl_qlist_end( p_list ) ) - { - CL_ASSERT( p_found_item->p_list == p_list ); + while (p_found_item != cl_qlist_end(p_list)) { + CL_ASSERT(p_found_item->p_list == p_list); - if( pfn_func( p_found_item, (void*)context ) == CL_SUCCESS ) + if (pfn_func(p_found_item, (void *)context) == CL_SUCCESS) break; - p_found_item = cl_qlist_next( p_found_item ); + p_found_item = cl_qlist_next(p_found_item); } /* No match */ - return( p_found_item ); + return (p_found_item); } -cl_list_item_t* -cl_qlist_find_prev( - IN const cl_qlist_t* const p_list, - IN const cl_list_item_t* const p_list_item, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ) +cl_list_item_t *cl_qlist_find_prev(IN const cl_qlist_t * const p_list, + IN const cl_list_item_t * const p_list_item, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context) { - cl_list_item_t *p_found_item; + cl_list_item_t *p_found_item; - CL_ASSERT( p_list ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); - CL_ASSERT( p_list_item ); - CL_ASSERT( p_list_item->p_list == p_list ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_list); + CL_ASSERT(p_list->state == CL_INITIALIZED); + CL_ASSERT(p_list_item); + CL_ASSERT(p_list_item->p_list == p_list); + CL_ASSERT(pfn_func); - p_found_item = cl_qlist_prev( p_list_item ); + p_found_item = cl_qlist_prev(p_list_item); /* The user provided a compare function */ - while( p_found_item != cl_qlist_end( p_list ) ) - { - CL_ASSERT( p_found_item->p_list == p_list ); + while (p_found_item != cl_qlist_end(p_list)) { + CL_ASSERT(p_found_item->p_list == p_list); - if( pfn_func( p_found_item, (void*)context ) == CL_SUCCESS ) + if (pfn_func(p_found_item, (void *)context) == CL_SUCCESS) break; - p_found_item = cl_qlist_prev( p_found_item ); + p_found_item = cl_qlist_prev(p_found_item); } /* No match */ - return( p_found_item ); + return (p_found_item); } void -cl_qlist_apply_func( - IN const cl_qlist_t* const p_list, - IN cl_pfn_qlist_apply_t pfn_func, - IN const void* const context ) +cl_qlist_apply_func(IN const cl_qlist_t * const p_list, + IN cl_pfn_qlist_apply_t pfn_func, + IN const void *const context) { - cl_list_item_t* p_list_item; + cl_list_item_t *p_list_item; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_list ); - CL_ASSERT( p_list->state == CL_INITIALIZED ); - CL_ASSERT( pfn_func ); - - p_list_item = cl_qlist_head( p_list ); - while( p_list_item != cl_qlist_end( p_list ) ) - { - pfn_func( p_list_item, (void*)context ); - p_list_item = cl_qlist_next( p_list_item ); + CL_ASSERT(p_list); + CL_ASSERT(p_list->state == CL_INITIALIZED); + CL_ASSERT(pfn_func); + + p_list_item = cl_qlist_head(p_list); + while (p_list_item != cl_qlist_end(p_list)) { + pfn_func(p_list_item, (void *)context); + p_list_item = cl_qlist_next(p_list_item); } } void -cl_qlist_move_items( - IN cl_qlist_t* const p_src_list, - IN cl_qlist_t* const p_dest_list, - IN cl_pfn_qlist_find_t pfn_func, - IN const void* const context ) +cl_qlist_move_items(IN cl_qlist_t * const p_src_list, + IN cl_qlist_t * const p_dest_list, + IN cl_pfn_qlist_find_t pfn_func, + IN const void *const context) { - cl_list_item_t *p_current_item, *p_next; + cl_list_item_t *p_current_item, *p_next; - CL_ASSERT( p_src_list ); - CL_ASSERT( p_dest_list ); - CL_ASSERT( p_src_list->state == CL_INITIALIZED ); - CL_ASSERT( p_dest_list->state == CL_INITIALIZED ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_src_list); + CL_ASSERT(p_dest_list); + CL_ASSERT(p_src_list->state == CL_INITIALIZED); + CL_ASSERT(p_dest_list->state == CL_INITIALIZED); + CL_ASSERT(pfn_func); - p_current_item = cl_qlist_head( p_src_list ); + p_current_item = cl_qlist_head(p_src_list); - while( p_current_item != cl_qlist_end( p_src_list ) ) - { + while (p_current_item != cl_qlist_end(p_src_list)) { /* Before we do anything, get a pointer to the next item. */ - p_next = cl_qlist_next( p_current_item ); + p_next = cl_qlist_next(p_current_item); - if( pfn_func( p_current_item, (void*)context ) == CL_SUCCESS ) - { + if (pfn_func(p_current_item, (void *)context) == CL_SUCCESS) { /* Move the item from one list to the other. */ - cl_qlist_remove_item( p_src_list, p_current_item ); - cl_qlist_insert_tail( p_dest_list, p_current_item ); + cl_qlist_remove_item(p_src_list, p_current_item); + cl_qlist_insert_tail(p_dest_list, p_current_item); } p_current_item = p_next; } @@ -381,252 +357,229 @@ cl_qlist_move_items( ************** ************ ******************************************************************************* ******************************************************************************/ -void -cl_list_construct( - IN cl_list_t* const p_list ) +void cl_list_construct(IN cl_list_t * const p_list) { - CL_ASSERT( p_list ); + CL_ASSERT(p_list); - cl_qpool_construct( &p_list->list_item_pool ); + cl_qpool_construct(&p_list->list_item_pool); } -cl_status_t -cl_list_init( - IN cl_list_t* const p_list, - IN const size_t min_items ) +cl_status_t cl_list_init(IN cl_list_t * const p_list, IN const size_t min_items) { - uint32_t grow_size; + uint32_t grow_size; - CL_ASSERT( p_list ); - cl_qlist_init( &p_list->list ); + CL_ASSERT(p_list); + cl_qlist_init(&p_list->list); /* * We will grow by min_items/8 items at a time, with a minimum of * FREE_ITEM_GROW_SIZE. */ grow_size = (uint32_t) min_items >> 3; - if( grow_size < FREE_ITEM_GROW_SIZE ) + if (grow_size < FREE_ITEM_GROW_SIZE) grow_size = FREE_ITEM_GROW_SIZE; /* Initialize the pool of list items. */ - return( cl_qpool_init( &p_list->list_item_pool, min_items, 0, grow_size, - sizeof(cl_pool_obj_t), NULL, NULL, NULL ) ); + return (cl_qpool_init(&p_list->list_item_pool, min_items, 0, grow_size, + sizeof(cl_pool_obj_t), NULL, NULL, NULL)); } -void -cl_list_destroy( - IN cl_list_t* const p_list ) +void cl_list_destroy(IN cl_list_t * const p_list) { - CL_ASSERT( p_list ); + CL_ASSERT(p_list); - cl_qpool_destroy( &p_list->list_item_pool ); + cl_qpool_destroy(&p_list->list_item_pool); } static cl_status_t -cl_list_find_cb( - IN const cl_list_item_t* const p_list_item, - IN void* const context ) +cl_list_find_cb(IN const cl_list_item_t * const p_list_item, + IN void *const context) { - CL_ASSERT( p_list_item ); + CL_ASSERT(p_list_item); - if( cl_list_obj( p_list_item ) == context ) - return( CL_SUCCESS ); + if (cl_list_obj(p_list_item) == context) + return (CL_SUCCESS); - return( CL_NOT_FOUND ); + return (CL_NOT_FOUND); } cl_status_t -cl_list_remove_object( - IN cl_list_t* const p_list, - IN const void* const p_object ) +cl_list_remove_object(IN cl_list_t * const p_list, + IN const void *const p_object) { - cl_list_item_t *p_list_item; + cl_list_item_t *p_list_item; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); /* find the item in question */ p_list_item = - cl_qlist_find_from_head( &p_list->list, cl_list_find_cb, p_object ); - if( p_list_item != cl_qlist_end( &p_list->list ) ) - { + cl_qlist_find_from_head(&p_list->list, cl_list_find_cb, p_object); + if (p_list_item != cl_qlist_end(&p_list->list)) { /* remove this item */ - cl_qlist_remove_item( &p_list->list, p_list_item ); - cl_qpool_put( &p_list->list_item_pool, (cl_pool_item_t*)p_list_item ); - return( CL_SUCCESS ); + cl_qlist_remove_item(&p_list->list, p_list_item); + cl_qpool_put(&p_list->list_item_pool, + (cl_pool_item_t *) p_list_item); + return (CL_SUCCESS); } - return( CL_NOT_FOUND ); + return (CL_NOT_FOUND); } boolean_t -cl_is_object_in_list( - IN const cl_list_t* const p_list, - IN const void* const p_object ) +cl_is_object_in_list(IN const cl_list_t * const p_list, + IN const void *const p_object) { - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); - return( cl_qlist_find_from_head( &p_list->list, cl_list_find_cb, p_object ) - != cl_qlist_end( &p_list->list ) ); + return (cl_qlist_find_from_head + (&p_list->list, cl_list_find_cb, p_object) + != cl_qlist_end(&p_list->list)); } cl_status_t -cl_list_insert_array_head( - IN cl_list_t* const p_list, - IN const void* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ) +cl_list_insert_array_head(IN cl_list_t * const p_list, + IN const void *const p_array, + IN uint32_t item_count, IN const uint32_t item_size) { - cl_status_t status; - void *p_object; + cl_status_t status; + void *p_object; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - CL_ASSERT( p_array ); - CL_ASSERT( item_size ); - CL_ASSERT( item_count ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + CL_ASSERT(p_array); + CL_ASSERT(item_size); + CL_ASSERT(item_count); /* * To add items from the array to the list in the same order as * the elements appear in the array, we add them starting with * the last one first. Locate the last item. */ - p_object = ((uint8_t*)p_array + (item_size * (item_count - 1))); + p_object = ((uint8_t *) p_array + (item_size * (item_count - 1))); /* Continue to add all items to the list. */ - while( item_count-- ) - { - status = cl_list_insert_head( p_list, p_object ); - if( status != CL_SUCCESS ) - { + while (item_count--) { + status = cl_list_insert_head(p_list, p_object); + if (status != CL_SUCCESS) { /* Remove all items that have been inserted. */ - while( item_count++ < item_count ) - cl_list_remove_head( p_list ); - return( status ); + while (item_count++ < item_count) + cl_list_remove_head(p_list); + return (status); } /* Get the next object to add to the list. */ - p_object = ((uint8_t*)p_object - item_size); + p_object = ((uint8_t *) p_object - item_size); } - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_list_insert_array_tail( - IN cl_list_t* const p_list, - IN const void* const p_array, - IN uint32_t item_count, - IN const uint32_t item_size ) +cl_list_insert_array_tail(IN cl_list_t * const p_list, + IN const void *const p_array, + IN uint32_t item_count, IN const uint32_t item_size) { - cl_status_t status; - void *p_object; + cl_status_t status; + void *p_object; - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - CL_ASSERT( p_array ); - CL_ASSERT( item_size ); - CL_ASSERT( item_count ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + CL_ASSERT(p_array); + CL_ASSERT(item_size); + CL_ASSERT(item_count); /* Set the first item to add to the list. */ - p_object = (void*)p_array; + p_object = (void *)p_array; /* Continue to add all items to the list. */ - while( item_count-- ) - { - status = cl_list_insert_tail( p_list, p_object ); - if( status != CL_SUCCESS ) - { + while (item_count--) { + status = cl_list_insert_tail(p_list, p_object); + if (status != CL_SUCCESS) { /* Remove all items that have been inserted. */ - while( item_count++ < item_count ) - cl_list_remove_tail( p_list ); - return( status ); + while (item_count++ < item_count) + cl_list_remove_tail(p_list); + return (status); } /* Get the next object to add to the list. */ - p_object = ((uint8_t*)p_object + item_size); + p_object = ((uint8_t *) p_object + item_size); } - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_list_iterator_t -cl_list_find_from_head( - IN const cl_list_t* const p_list, - IN cl_pfn_list_find_t pfn_func, - IN const void* const context ) +cl_list_find_from_head(IN const cl_list_t * const p_list, + IN cl_pfn_list_find_t pfn_func, + IN const void *const context) { - cl_status_t status; - cl_list_iterator_t itor; + cl_status_t status; + cl_list_iterator_t itor; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + CL_ASSERT(pfn_func); - itor = cl_list_head( p_list ); + itor = cl_list_head(p_list); - while( itor != cl_list_end( p_list ) ) - { - status = pfn_func( cl_list_obj( itor ), (void*)context ); - if( status == CL_SUCCESS ) + while (itor != cl_list_end(p_list)) { + status = pfn_func(cl_list_obj(itor), (void *)context); + if (status == CL_SUCCESS) break; - itor = cl_list_next( itor ); + itor = cl_list_next(itor); } /* no match */ - return( itor ); + return (itor); } cl_list_iterator_t -cl_list_find_from_tail( - IN const cl_list_t* const p_list, - IN cl_pfn_list_find_t pfn_func, - IN const void* const context ) +cl_list_find_from_tail(IN const cl_list_t * const p_list, + IN cl_pfn_list_find_t pfn_func, + IN const void *const context) { - cl_status_t status; - cl_list_iterator_t itor; + cl_status_t status; + cl_list_iterator_t itor; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + CL_ASSERT(pfn_func); - itor = cl_list_tail( p_list ); + itor = cl_list_tail(p_list); - while( itor != cl_list_end( p_list ) ) - { - status = pfn_func( cl_list_obj( itor ), (void*)context ); - if( status == CL_SUCCESS ) + while (itor != cl_list_end(p_list)) { + status = pfn_func(cl_list_obj(itor), (void *)context); + if (status == CL_SUCCESS) break; - itor = cl_list_prev( itor ); + itor = cl_list_prev(itor); } /* no match */ - return( itor ); + return (itor); } void -cl_list_apply_func( - IN const cl_list_t* const p_list, - IN cl_pfn_list_apply_t pfn_func, - IN const void* const context ) +cl_list_apply_func(IN const cl_list_t * const p_list, + IN cl_pfn_list_apply_t pfn_func, + IN const void *const context) { - cl_list_iterator_t itor; + cl_list_iterator_t itor; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_list ); - CL_ASSERT( cl_is_qpool_inited( &p_list->list_item_pool ) ); - CL_ASSERT( pfn_func ); + CL_ASSERT(p_list); + CL_ASSERT(cl_is_qpool_inited(&p_list->list_item_pool)); + CL_ASSERT(pfn_func); - itor = cl_list_head( p_list ); + itor = cl_list_head(p_list); - while( itor != cl_list_end( p_list ) ) - { - pfn_func( cl_list_obj( itor ), (void*)context ); + while (itor != cl_list_end(p_list)) { + pfn_func(cl_list_obj(itor), (void *)context); - itor = cl_list_next( itor ); + itor = cl_list_next(itor); } } diff --git a/opensm/complib/cl_log.c b/opensm/complib/cl_log.c index dd3a4c6..a2a1372 100644 --- a/opensm/complib/cl_log.c +++ b/opensm/complib/cl_log.c @@ -39,7 +39,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,24 +56,21 @@ #define CL_LOG_DATA_SIZE (CL_MAX_LOG_DATA + (CL_MAX_LOG_DATA/4)) void -cl_log_event( - IN const char* const name, - IN const cl_log_type_t type, - IN const char* const message, - IN const void* const p_data OPTIONAL, - IN const uint32_t data_len ) +cl_log_event(IN const char *const name, + IN const cl_log_type_t type, + IN const char *const message, + IN const void *const p_data OPTIONAL, IN const uint32_t data_len) { - int priority, i; - char data[CL_LOG_DATA_SIZE]; - char *p_buf; - uint8_t *p_int_data = (uint8_t*)p_data; + int priority, i; + char data[CL_LOG_DATA_SIZE]; + char *p_buf; + uint8_t *p_int_data = (uint8_t *) p_data; - CL_ASSERT( name ); - CL_ASSERT( message ); + CL_ASSERT(name); + CL_ASSERT(message); - openlog( name, LOG_NDELAY | LOG_PID, LOG_USER ); - switch( type ) - { + openlog(name, LOG_NDELAY | LOG_PID, LOG_USER); + switch (type) { case CL_LOG_ERROR: priority = LOG_ERR; break; @@ -88,41 +85,34 @@ cl_log_event( break; } - if( p_data ) - { - CL_ASSERT( data_len ); - if( data_len < CL_MAX_LOG_DATA ) - { + if (p_data) { + CL_ASSERT(data_len); + if (data_len < CL_MAX_LOG_DATA) { p_buf = data; /* Format the data into ASCII. */ - for( i = 0; i < data_len; i++ ) - { - sprintf( p_buf, "%02x", *p_int_data++ ); + for (i = 0; i < data_len; i++) { + sprintf(p_buf, "%02x", *p_int_data++); p_buf += 2; /* Add line break after 8 DWORDS. */ - if( i % 32 ) - { - sprintf( p_buf++, "\n" ); + if (i % 32) { + sprintf(p_buf++, "\n"); continue; } /* Add a space between DWORDS. */ - if( i % 4 ) - sprintf( p_buf++, " " ); + if (i % 4) + sprintf(p_buf++, " "); } - syslog( priority, "%s data:\n%s\n", message, p_buf ); - } - else - { + syslog(priority, "%s data:\n%s\n", message, p_buf); + } else { /* The data portion is too large to log. */ - cl_msg_out( "cl_log() - WARNING: data too large to log.\n" ); - syslog( priority, "%s\n", message ); + cl_msg_out + ("cl_log() - WARNING: data too large to log.\n"); + syslog(priority, "%s\n", message); } - } - else - { - syslog( priority, "%s\n", message ); + } else { + syslog(priority, "%s\n", message); } closelog(); } diff --git a/opensm/complib/cl_map.c b/opensm/complib/cl_map.c index 7138a43..d8b7254 100644 --- a/opensm/complib/cl_map.c +++ b/opensm/complib/cl_map.c @@ -64,7 +64,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -82,44 +82,39 @@ /* * Get the root. */ -static inline cl_map_item_t* -__cl_map_root( - IN const cl_qmap_t* const p_map ) +static inline cl_map_item_t *__cl_map_root(IN const cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); - return( p_map->root.p_left ); + CL_ASSERT(p_map); + return (p_map->root.p_left); } /* * Returns whether a given item is on the left of its parent. */ -static boolean_t -__cl_map_is_left_child( - IN const cl_map_item_t* const p_item ) +static boolean_t __cl_map_is_left_child(IN const cl_map_item_t * const p_item) { - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_up ); - CL_ASSERT( p_item->p_up != p_item ); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_up); + CL_ASSERT(p_item->p_up != p_item); - return( p_item->p_up->p_left == p_item ); + return (p_item->p_up->p_left == p_item); } /* * Retrieve the pointer to the parent's pointer to an item. */ -static cl_map_item_t** -__cl_map_get_parent_ptr_to_item( - IN cl_map_item_t* const p_item ) +static cl_map_item_t **__cl_map_get_parent_ptr_to_item(IN cl_map_item_t * + const p_item) { - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_up ); - CL_ASSERT( p_item->p_up != p_item ); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_up); + CL_ASSERT(p_item->p_up != p_item); - if( __cl_map_is_left_child( p_item ) ) - return( &p_item->p_up->p_left ); + if (__cl_map_is_left_child(p_item)) + return (&p_item->p_up->p_left); - CL_ASSERT( p_item->p_up->p_right == p_item ); - return( &p_item->p_up->p_right ); + CL_ASSERT(p_item->p_up->p_right == p_item); + return (&p_item->p_up->p_right); } /* @@ -138,17 +133,15 @@ __cl_map_get_parent_ptr_to_item( * X Y X Y */ static void -__cl_map_rot_left( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* const p_item ) +__cl_map_rot_left(IN cl_qmap_t * const p_map, IN cl_map_item_t * const p_item) { - cl_map_item_t **pp_root; + cl_map_item_t **pp_root; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_right != &p_map->nil ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_right != &p_map->nil); - pp_root = __cl_map_get_parent_ptr_to_item( p_item ); + pp_root = __cl_map_get_parent_ptr_to_item(p_item); /* Point R to C instead of A. */ *pp_root = p_item->p_right; @@ -161,7 +154,7 @@ __cl_map_rot_left( * Set B's parent to A. We trap for B being NIL since the * caller may depend on NIL not changing. */ - if( (*pp_root)->p_left != &p_map->nil ) + if ((*pp_root)->p_left != &p_map->nil) (*pp_root)->p_left->p_up = p_item; /* Set C's left to A. */ @@ -186,18 +179,16 @@ __cl_map_rot_left( * X Y X Y */ static void -__cl_map_rot_right( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* const p_item ) +__cl_map_rot_right(IN cl_qmap_t * const p_map, IN cl_map_item_t * const p_item) { - cl_map_item_t **pp_root; + cl_map_item_t **pp_root; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_left != &p_map->nil ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_left != &p_map->nil); /* Point R to A instead of C. */ - pp_root = __cl_map_get_parent_ptr_to_item( p_item ); + pp_root = __cl_map_get_parent_ptr_to_item(p_item); (*pp_root) = p_item->p_left; /* Set A's parent to R. */ (*pp_root)->p_up = p_item->p_up; @@ -208,7 +199,7 @@ __cl_map_rot_right( * Set B's parent to C. We trap for B being NIL since the * caller may depend on NIL not changing. */ - if( (*pp_root)->p_right != &p_map->nil ) + if ((*pp_root)->p_right != &p_map->nil) (*pp_root)->p_right->p_up = p_item; /* Set A's right to C. */ @@ -217,13 +208,11 @@ __cl_map_rot_right( p_item->p_up = *pp_root; } -void -cl_qmap_init( - IN cl_qmap_t* const p_map ) +void cl_qmap_init(IN cl_qmap_t * const p_map) { - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - memset( p_map, 0, sizeof(cl_qmap_t) ); + memset(p_map, 0, sizeof(cl_qmap_t)); /* special setup for the root node */ p_map->root.p_up = &p_map->root; @@ -239,80 +228,72 @@ cl_qmap_init( p_map->state = CL_INITIALIZED; - cl_qmap_remove_all( p_map ); + cl_qmap_remove_all(p_map); } -cl_map_item_t* -cl_qmap_get( - IN const cl_qmap_t* const p_map, - IN const uint64_t key ) +cl_map_item_t *cl_qmap_get(IN const cl_qmap_t * const p_map, + IN const uint64_t key) { - cl_map_item_t *p_item; + cl_map_item_t *p_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - p_item = __cl_map_root( p_map ); + p_item = __cl_map_root(p_map); - while( p_item != &p_map->nil ) - { - if( key == p_item->key ) - break; /* just right */ + while (p_item != &p_map->nil) { + if (key == p_item->key) + break; /* just right */ - if( key < p_item->key ) + if (key < p_item->key) p_item = p_item->p_left; /* too small */ else p_item = p_item->p_right; /* too big */ } - return( p_item ); + return (p_item); } -cl_map_item_t* -cl_qmap_get_next( - IN const cl_qmap_t* const p_map, - IN const uint64_t key ) +cl_map_item_t *cl_qmap_get_next(IN const cl_qmap_t * const p_map, + IN const uint64_t key) { - cl_map_item_t *p_item; - cl_map_item_t *p_item_found; + cl_map_item_t *p_item; + cl_map_item_t *p_item_found; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - p_item = __cl_map_root( p_map ); - p_item_found = (cl_map_item_t*)&p_map->nil; + p_item = __cl_map_root(p_map); + p_item_found = (cl_map_item_t *) & p_map->nil; - while( p_item != &p_map->nil ) - { - if( key < p_item->key ){ + while (p_item != &p_map->nil) { + if (key < p_item->key) { p_item_found = p_item; p_item = p_item->p_left; - }else{ + } else { p_item = p_item->p_right; } } - return( p_item_found ); + return (p_item_found); } void -cl_qmap_apply_func( - IN const cl_qmap_t* const p_map, - IN cl_pfn_qmap_apply_t pfn_func, - IN const void* const context ) +cl_qmap_apply_func(IN const cl_qmap_t * const p_map, + IN cl_pfn_qmap_apply_t pfn_func, + IN const void *const context) { - cl_map_item_t* p_map_item; + cl_map_item_t *p_map_item; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( pfn_func ); - - p_map_item = cl_qmap_head( p_map ); - while( p_map_item != cl_qmap_end( p_map ) ) - { - pfn_func( p_map_item, (void*)context ); - p_map_item = cl_qmap_next( p_map_item ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(pfn_func); + + p_map_item = cl_qmap_head(p_map); + while (p_map_item != cl_qmap_end(p_map)) { + pfn_func(p_map_item, (void *)context); + p_map_item = cl_qmap_next(p_map_item); } } @@ -320,24 +301,19 @@ cl_qmap_apply_func( * Balance a tree starting at a given item back to the root. */ static void -__cl_map_ins_bal( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* p_item ) +__cl_map_ins_bal(IN cl_qmap_t * const p_map, IN cl_map_item_t * p_item) { - cl_map_item_t* p_grand_uncle; + cl_map_item_t *p_grand_uncle; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item != &p_map->root ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item != &p_map->root); - while( p_item->p_up->color == CL_MAP_RED ) - { - if( __cl_map_is_left_child( p_item->p_up ) ) - { + while (p_item->p_up->color == CL_MAP_RED) { + if (__cl_map_is_left_child(p_item->p_up)) { p_grand_uncle = p_item->p_up->p_up->p_right; - CL_ASSERT( p_grand_uncle ); - if( p_grand_uncle->color == CL_MAP_RED ) - { + CL_ASSERT(p_grand_uncle); + if (p_grand_uncle->color == CL_MAP_RED) { p_grand_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; @@ -345,21 +321,17 @@ __cl_map_ins_bal( continue; } - if( !__cl_map_is_left_child( p_item ) ) - { + if (!__cl_map_is_left_child(p_item)) { p_item = p_item->p_up; - __cl_map_rot_left( p_map, p_item ); + __cl_map_rot_left(p_map, p_item); } p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; - __cl_map_rot_right( p_map, p_item->p_up->p_up ); - } - else - { + __cl_map_rot_right(p_map, p_item->p_up->p_up); + } else { p_grand_uncle = p_item->p_up->p_up->p_left; - CL_ASSERT( p_grand_uncle ); - if( p_grand_uncle->color == CL_MAP_RED ) - { + CL_ASSERT(p_grand_uncle); + if (p_grand_uncle->color == CL_MAP_RED) { p_grand_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; @@ -367,32 +339,29 @@ __cl_map_ins_bal( continue; } - if( __cl_map_is_left_child( p_item ) ) - { + if (__cl_map_is_left_child(p_item)) { p_item = p_item->p_up; - __cl_map_rot_right( p_map, p_item ); + __cl_map_rot_right(p_map, p_item); } p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; - __cl_map_rot_left( p_map, p_item->p_up->p_up ); + __cl_map_rot_left(p_map, p_item->p_up->p_up); } } } -cl_map_item_t* -cl_qmap_insert( - IN cl_qmap_t* const p_map, - IN const uint64_t key, - IN cl_map_item_t* const p_item ) +cl_map_item_t *cl_qmap_insert(IN cl_qmap_t * const p_map, + IN const uint64_t key, + IN cl_map_item_t * const p_item) { - cl_map_item_t *p_insert_at, *p_comp_item; + cl_map_item_t *p_insert_at, *p_comp_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( p_item ); - CL_ASSERT( p_map->root.p_up == &p_map->root ); - CL_ASSERT( p_map->root.color != CL_MAP_RED ); - CL_ASSERT( p_map->nil.color != CL_MAP_RED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(p_item); + CL_ASSERT(p_map->root.p_up == &p_map->root); + CL_ASSERT(p_map->root.color != CL_MAP_RED); + CL_ASSERT(p_map->nil.color != CL_MAP_RED); p_item->p_left = &p_map->nil; p_item->p_right = &p_map->nil; @@ -401,54 +370,48 @@ cl_qmap_insert( /* Find the insertion location. */ p_insert_at = &p_map->root; - p_comp_item = __cl_map_root( p_map ); + p_comp_item = __cl_map_root(p_map); - while( p_comp_item != &p_map->nil ) - { + while (p_comp_item != &p_map->nil) { p_insert_at = p_comp_item; - if( key == p_insert_at->key ) - return( p_insert_at ); + if (key == p_insert_at->key) + return (p_insert_at); /* Traverse the tree until the correct insertion point is found. */ - if( key < p_insert_at->key ) + if (key < p_insert_at->key) p_comp_item = p_insert_at->p_left; else p_comp_item = p_insert_at->p_right; } - CL_ASSERT( p_insert_at != &p_map->nil ); - CL_ASSERT( p_comp_item == &p_map->nil ); + CL_ASSERT(p_insert_at != &p_map->nil); + CL_ASSERT(p_comp_item == &p_map->nil); /* Insert the item. */ - if( p_insert_at == &p_map->root ) - { + if (p_insert_at == &p_map->root) { p_insert_at->p_left = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( &p_map->nil.pool_item.list_item, - &p_item->pool_item.list_item ); - } - else if( key < p_insert_at->key ) - { + __cl_primitive_insert(&p_map->nil.pool_item.list_item, + &p_item->pool_item.list_item); + } else if (key < p_insert_at->key) { p_insert_at->p_left = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( &p_insert_at->pool_item.list_item, - &p_item->pool_item.list_item ); - } - else - { + __cl_primitive_insert(&p_insert_at->pool_item.list_item, + &p_item->pool_item.list_item); + } else { p_insert_at->p_right = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( p_insert_at->pool_item.list_item.p_next, - &p_item->pool_item.list_item ); + __cl_primitive_insert(p_insert_at->pool_item.list_item.p_next, + &p_item->pool_item.list_item); } /* Increase the count. */ p_map->count++; @@ -460,9 +423,9 @@ cl_qmap_insert( * Rebalance as necessary as we retrace our path through the tree * and update colors. */ - __cl_map_ins_bal( p_map, p_item ); + __cl_map_ins_bal(p_map, p_item); - __cl_map_root( p_map )->color = CL_MAP_BLACK; + __cl_map_root(p_map)->color = CL_MAP_BLACK; /* * Note that it is not necessary to re-color the nil node black because all @@ -475,34 +438,27 @@ cl_qmap_insert( p_item->p_map = p_map; #endif - return( p_item ); + return (p_item); } static void -__cl_map_del_bal( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* p_item ) +__cl_map_del_bal(IN cl_qmap_t * const p_map, IN cl_map_item_t * p_item) { - cl_map_item_t *p_uncle; + cl_map_item_t *p_uncle; - while( (p_item->color != CL_MAP_RED) && (p_item->p_up != &p_map->root) ) - { - if( __cl_map_is_left_child( p_item ) ) - { + while ((p_item->color != CL_MAP_RED) && (p_item->p_up != &p_map->root)) { + if (__cl_map_is_left_child(p_item)) { p_uncle = p_item->p_up->p_right; - if( p_uncle->color == CL_MAP_RED ) - { + if (p_uncle->color == CL_MAP_RED) { p_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_RED; - __cl_map_rot_left( p_map, p_item->p_up ); + __cl_map_rot_left(p_map, p_item->p_up); p_uncle = p_item->p_up->p_right; } - if( p_uncle->p_right->color != CL_MAP_RED ) - { - if( p_uncle->p_left->color != CL_MAP_RED ) - { + if (p_uncle->p_right->color != CL_MAP_RED) { + if (p_uncle->p_left->color != CL_MAP_RED) { p_uncle->color = CL_MAP_RED; p_item = p_item->p_up; continue; @@ -510,31 +466,26 @@ __cl_map_del_bal( p_uncle->p_left->color = CL_MAP_BLACK; p_uncle->color = CL_MAP_RED; - __cl_map_rot_right( p_map, p_uncle ); + __cl_map_rot_right(p_map, p_uncle); p_uncle = p_item->p_up->p_right; } p_uncle->color = p_item->p_up->color; p_item->p_up->color = CL_MAP_BLACK; p_uncle->p_right->color = CL_MAP_BLACK; - __cl_map_rot_left( p_map, p_item->p_up ); + __cl_map_rot_left(p_map, p_item->p_up); break; - } - else - { + } else { p_uncle = p_item->p_up->p_left; - if( p_uncle->color == CL_MAP_RED ) - { + if (p_uncle->color == CL_MAP_RED) { p_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_RED; - __cl_map_rot_right( p_map, p_item->p_up ); + __cl_map_rot_right(p_map, p_item->p_up); p_uncle = p_item->p_up->p_left; } - if( p_uncle->p_left->color != CL_MAP_RED ) - { - if( p_uncle->p_right->color != CL_MAP_RED ) - { + if (p_uncle->p_left->color != CL_MAP_RED) { + if (p_uncle->p_right->color != CL_MAP_RED) { p_uncle->color = CL_MAP_RED; p_item = p_item->p_up; continue; @@ -542,13 +493,13 @@ __cl_map_del_bal( p_uncle->p_right->color = CL_MAP_BLACK; p_uncle->color = CL_MAP_RED; - __cl_map_rot_left( p_map, p_uncle ); + __cl_map_rot_left(p_map, p_uncle); p_uncle = p_item->p_up->p_left; } p_uncle->color = p_item->p_up->color; p_item->p_up->color = CL_MAP_BLACK; p_uncle->p_left->color = CL_MAP_BLACK; - __cl_map_rot_right( p_map, p_item->p_up ); + __cl_map_rot_right(p_map, p_item->p_up); break; } } @@ -556,30 +507,25 @@ __cl_map_del_bal( } void -cl_qmap_remove_item( - IN cl_qmap_t* const p_map, - IN cl_map_item_t* const p_item ) +cl_qmap_remove_item(IN cl_qmap_t * const p_map, IN cl_map_item_t * const p_item) { - cl_map_item_t *p_child, *p_del_item; + cl_map_item_t *p_child, *p_del_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( p_item ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(p_item); - if( p_item == cl_qmap_end( p_map ) ) + if (p_item == cl_qmap_end(p_map)) return; - /* must be checked after comparing to cl_qmap_end, since - the end is not a valid item. */ - CL_ASSERT( p_item->p_map == p_map ); + /* must be checked after comparing to cl_qmap_end, since + the end is not a valid item. */ + CL_ASSERT(p_item->p_map == p_map); - if( (p_item->p_right == &p_map->nil) || (p_item->p_left == &p_map->nil ) ) - { + if ((p_item->p_right == &p_map->nil) || (p_item->p_left == &p_map->nil)) { /* The item being removed has children on at most on side. */ p_del_item = p_item; - } - else - { + } else { /* * The item being removed has children on both side. * We select the item that will replace it. After removing @@ -587,17 +533,17 @@ cl_qmap_remove_item( * correct topology. Exchanging the substitute for the item * will finalize the removal. */ - p_del_item = cl_qmap_next( p_item ); - CL_ASSERT( p_del_item != &p_map->nil ); + p_del_item = cl_qmap_next(p_item); + CL_ASSERT(p_del_item != &p_map->nil); } /* Remove the item from the list. */ - __cl_primitive_remove( &p_item->pool_item.list_item ); + __cl_primitive_remove(&p_item->pool_item.list_item); /* Decrement the item count. */ p_map->count--; /* Get the pointer to the new root's child, if any. */ - if( p_del_item->p_left != &p_map->nil ) + if (p_del_item->p_left != &p_map->nil) p_child = p_del_item->p_left; else p_child = p_del_item->p_right; @@ -607,10 +553,10 @@ cl_qmap_remove_item( * This is inconsequential. */ p_child->p_up = p_del_item->p_up; - (*__cl_map_get_parent_ptr_to_item( p_del_item )) = p_child; + (*__cl_map_get_parent_ptr_to_item(p_del_item)) = p_child; - if( p_del_item->color != CL_MAP_RED ) - __cl_map_del_bal( p_map, p_child ); + if (p_del_item->color != CL_MAP_RED) + __cl_map_del_bal(p_map, p_child); /* * Note that the splicing done below does not need to occur before @@ -618,8 +564,7 @@ cl_qmap_remove_item( * preceding code. The topology is preserved by the color assignment made * below (reader should be reminded that p_del_item == p_item in some cases). */ - if( p_del_item != p_item ) - { + if (p_del_item != p_item) { /* * Finalize the removal of the specified item by exchanging it with * the substitute which we removed above. @@ -627,13 +572,13 @@ cl_qmap_remove_item( p_del_item->p_up = p_item->p_up; p_del_item->p_left = p_item->p_left; p_del_item->p_right = p_item->p_right; - (*__cl_map_get_parent_ptr_to_item( p_item )) = p_del_item; + (*__cl_map_get_parent_ptr_to_item(p_item)) = p_del_item; p_item->p_right->p_up = p_del_item; p_item->p_left->p_up = p_del_item; p_del_item->color = p_item->color; } - CL_ASSERT( p_map->nil.color != CL_MAP_RED ); + CL_ASSERT(p_map->nil.color != CL_MAP_RED); #ifdef _DEBUG_ /* Clear the pointer to the map since the item has been removed. */ @@ -641,127 +586,113 @@ cl_qmap_remove_item( #endif } -cl_map_item_t* -cl_qmap_remove( - IN cl_qmap_t* const p_map, - IN const uint64_t key ) +cl_map_item_t *cl_qmap_remove(IN cl_qmap_t * const p_map, IN const uint64_t key) { - cl_map_item_t *p_item; + cl_map_item_t *p_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); /* Seek the node with the specified key */ - p_item = cl_qmap_get( p_map, key ); + p_item = cl_qmap_get(p_map, key); - cl_qmap_remove_item( p_map, p_item ); + cl_qmap_remove_item(p_map, p_item); - return( p_item ); + return (p_item); } void -cl_qmap_merge( - OUT cl_qmap_t* const p_dest_map, - IN OUT cl_qmap_t* const p_src_map ) +cl_qmap_merge(OUT cl_qmap_t * const p_dest_map, + IN OUT cl_qmap_t * const p_src_map) { - cl_map_item_t *p_item, *p_item2, *p_next; + cl_map_item_t *p_item, *p_item2, *p_next; - CL_ASSERT( p_dest_map ); - CL_ASSERT( p_src_map ); + CL_ASSERT(p_dest_map); + CL_ASSERT(p_src_map); - p_item = cl_qmap_head( p_src_map ); + p_item = cl_qmap_head(p_src_map); - while( p_item != cl_qmap_end( p_src_map ) ) - { - p_next = cl_qmap_next( p_item ); + while (p_item != cl_qmap_end(p_src_map)) { + p_next = cl_qmap_next(p_item); /* Remove the item from its current map. */ - cl_qmap_remove_item( p_src_map, p_item ); + cl_qmap_remove_item(p_src_map, p_item); /* Insert the item into the destination map. */ - p_item2 = cl_qmap_insert( p_dest_map, cl_qmap_key( p_item ), p_item ); + p_item2 = + cl_qmap_insert(p_dest_map, cl_qmap_key(p_item), p_item); /* Check that the item was successfully inserted. */ - if( p_item2 != p_item ) - { + if (p_item2 != p_item) { /* Put the item in back in the source map. */ p_item2 = - cl_qmap_insert( p_src_map, cl_qmap_key( p_item ), p_item ); - CL_ASSERT( p_item2 == p_item ); + cl_qmap_insert(p_src_map, cl_qmap_key(p_item), + p_item); + CL_ASSERT(p_item2 == p_item); } p_item = p_next; } } static void -__cl_qmap_delta_move( - IN OUT cl_qmap_t* const p_dest, - IN OUT cl_qmap_t* const p_src, - IN OUT cl_map_item_t** const pp_item ) +__cl_qmap_delta_move(IN OUT cl_qmap_t * const p_dest, + IN OUT cl_qmap_t * const p_src, + IN OUT cl_map_item_t ** const pp_item) { - cl_map_item_t *p_temp, *p_next; + cl_map_item_t *p_temp, *p_next; /* * Get the next item so that we can ensure that pp_item points to * a valid item upon return from the function. */ - p_next = cl_qmap_next( *pp_item ); + p_next = cl_qmap_next(*pp_item); /* Move the old item from its current map the the old map. */ - cl_qmap_remove_item( p_src, *pp_item ); - p_temp = cl_qmap_insert( p_dest, cl_qmap_key( *pp_item ), *pp_item ); + cl_qmap_remove_item(p_src, *pp_item); + p_temp = cl_qmap_insert(p_dest, cl_qmap_key(*pp_item), *pp_item); /* We should never have duplicates. */ - CL_ASSERT( p_temp == *pp_item ); + CL_ASSERT(p_temp == *pp_item); /* Point pp_item to a valid item in the source map. */ (*pp_item) = p_next; } void -cl_qmap_delta( - IN OUT cl_qmap_t* const p_map1, - IN OUT cl_qmap_t* const p_map2, - OUT cl_qmap_t* const p_new, - OUT cl_qmap_t* const p_old ) +cl_qmap_delta(IN OUT cl_qmap_t * const p_map1, + IN OUT cl_qmap_t * const p_map2, + OUT cl_qmap_t * const p_new, OUT cl_qmap_t * const p_old) { - cl_map_item_t *p_item1, *p_item2; - uint64_t key1, key2; - - CL_ASSERT( p_map1 ); - CL_ASSERT( p_map2 ); - CL_ASSERT( p_new ); - CL_ASSERT( p_old ); - CL_ASSERT( cl_is_qmap_empty( p_new ) ); - CL_ASSERT( cl_is_qmap_empty( p_old ) ); - - p_item1 = cl_qmap_head( p_map1 ); - p_item2 = cl_qmap_head( p_map2 ); - - while( p_item1 != cl_qmap_end( p_map1 ) && - p_item2 != cl_qmap_end( p_map2 ) ) - { - key1 = cl_qmap_key( p_item1 ); - key2 = cl_qmap_key( p_item2 ); - if( key1 < key2 ) - { + cl_map_item_t *p_item1, *p_item2; + uint64_t key1, key2; + + CL_ASSERT(p_map1); + CL_ASSERT(p_map2); + CL_ASSERT(p_new); + CL_ASSERT(p_old); + CL_ASSERT(cl_is_qmap_empty(p_new)); + CL_ASSERT(cl_is_qmap_empty(p_old)); + + p_item1 = cl_qmap_head(p_map1); + p_item2 = cl_qmap_head(p_map2); + + while (p_item1 != cl_qmap_end(p_map1) && p_item2 != cl_qmap_end(p_map2)) { + key1 = cl_qmap_key(p_item1); + key2 = cl_qmap_key(p_item2); + if (key1 < key2) { /* We found an old item. */ - __cl_qmap_delta_move( p_old, p_map1, &p_item1 ); - } - else if( key1 > key2 ) - { + __cl_qmap_delta_move(p_old, p_map1, &p_item1); + } else if (key1 > key2) { /* We found a new item. */ - __cl_qmap_delta_move( p_new, p_map2, &p_item2 ); - } - else - { + __cl_qmap_delta_move(p_new, p_map2, &p_item2); + } else { /* Move both forward since they have the same key. */ - p_item1 = cl_qmap_next( p_item1 ); - p_item2 = cl_qmap_next( p_item2 ); + p_item1 = cl_qmap_next(p_item1); + p_item2 = cl_qmap_next(p_item2); } } /* Process the remainder if the end of either source map was reached. */ - while( p_item2 != cl_qmap_end( p_map2 ) ) - __cl_qmap_delta_move( p_new, p_map2, &p_item2 ); + while (p_item2 != cl_qmap_end(p_map2)) + __cl_qmap_delta_move(p_new, p_map2, &p_item2); - while( p_item1 != cl_qmap_end( p_map1 ) ) - __cl_qmap_delta_move( p_old, p_map1, &p_item1 ); + while (p_item1 != cl_qmap_end(p_map1)) + __cl_qmap_delta_move(p_old, p_map1, &p_item1); } /****************************************************************************** @@ -774,348 +705,301 @@ cl_qmap_delta( #define MAP_GROW_SIZE 32 -void -cl_map_construct( - IN cl_map_t* const p_map ) +void cl_map_construct(IN cl_map_t * const p_map) { - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - cl_qpool_construct( &p_map->pool ); + cl_qpool_construct(&p_map->pool); } -cl_status_t -cl_map_init( - IN cl_map_t* const p_map, - IN const uint32_t min_items ) +cl_status_t cl_map_init(IN cl_map_t * const p_map, IN const uint32_t min_items) { - uint32_t grow_size; + uint32_t grow_size; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - cl_qmap_init( &p_map->qmap ); + cl_qmap_init(&p_map->qmap); /* * We will grow by min_items/8 items at a time, with a minimum of * MAP_GROW_SIZE. */ grow_size = min_items >> 3; - if( grow_size < MAP_GROW_SIZE ) + if (grow_size < MAP_GROW_SIZE) grow_size = MAP_GROW_SIZE; - return( cl_qpool_init( &p_map->pool, min_items, 0, grow_size, - sizeof(cl_map_obj_t), NULL, NULL, NULL ) ); + return (cl_qpool_init(&p_map->pool, min_items, 0, grow_size, + sizeof(cl_map_obj_t), NULL, NULL, NULL)); } -void -cl_map_destroy( - IN cl_map_t* const p_map ) +void cl_map_destroy(IN cl_map_t * const p_map) { - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - cl_qpool_destroy( &p_map->pool ); + cl_qpool_destroy(&p_map->pool); } -void* -cl_map_insert( - IN cl_map_t* const p_map, - IN const uint64_t key, - IN const void* const p_object ) +void *cl_map_insert(IN cl_map_t * const p_map, + IN const uint64_t key, IN const void *const p_object) { - cl_map_obj_t *p_map_obj, *p_obj_at_key; + cl_map_obj_t *p_map_obj, *p_obj_at_key; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - p_map_obj = (cl_map_obj_t*)cl_qpool_get( &p_map->pool ); + p_map_obj = (cl_map_obj_t *) cl_qpool_get(&p_map->pool); - if( !p_map_obj ) - return( NULL ); + if (!p_map_obj) + return (NULL); - cl_qmap_set_obj( p_map_obj, p_object ); + cl_qmap_set_obj(p_map_obj, p_object); p_obj_at_key = - (cl_map_obj_t*)cl_qmap_insert( &p_map->qmap, key, &p_map_obj->item ); + (cl_map_obj_t *) cl_qmap_insert(&p_map->qmap, key, + &p_map_obj->item); /* Return the item to the pool if insertion failed. */ - if( p_obj_at_key != p_map_obj ) - cl_qpool_put( &p_map->pool, &p_map_obj->item.pool_item ); + if (p_obj_at_key != p_map_obj) + cl_qpool_put(&p_map->pool, &p_map_obj->item.pool_item); - return( cl_qmap_obj( p_obj_at_key ) ); + return (cl_qmap_obj(p_obj_at_key)); } -void* -cl_map_get( - IN const cl_map_t* const p_map, - IN const uint64_t key ) +void *cl_map_get(IN const cl_map_t * const p_map, IN const uint64_t key) { - cl_map_item_t *p_item; + cl_map_item_t *p_item; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - p_item = cl_qmap_get( &p_map->qmap, key ); + p_item = cl_qmap_get(&p_map->qmap, key); - if( p_item == cl_qmap_end( &p_map->qmap ) ) - return( NULL ); + if (p_item == cl_qmap_end(&p_map->qmap)) + return (NULL); - return( cl_qmap_obj( PARENT_STRUCT( p_item, cl_map_obj_t, item ) ) ); + return (cl_qmap_obj(PARENT_STRUCT(p_item, cl_map_obj_t, item))); } -void* -cl_map_get_next( - IN const cl_map_t* const p_map, - IN const uint64_t key ) +void *cl_map_get_next(IN const cl_map_t * const p_map, IN const uint64_t key) { - cl_map_item_t *p_item; + cl_map_item_t *p_item; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - p_item = cl_qmap_get_next( &p_map->qmap, key ); + p_item = cl_qmap_get_next(&p_map->qmap, key); - if( p_item == cl_qmap_end( &p_map->qmap ) ) - return( NULL ); + if (p_item == cl_qmap_end(&p_map->qmap)) + return (NULL); - return( cl_qmap_obj( PARENT_STRUCT( p_item, cl_map_obj_t, item ) ) ); + return (cl_qmap_obj(PARENT_STRUCT(p_item, cl_map_obj_t, item))); } void -cl_map_remove_item( - IN cl_map_t* const p_map, - IN const cl_map_iterator_t itor ) +cl_map_remove_item(IN cl_map_t * const p_map, IN const cl_map_iterator_t itor) { - CL_ASSERT( itor->p_map == &p_map->qmap ); + CL_ASSERT(itor->p_map == &p_map->qmap); - if( itor == cl_map_end( p_map ) ) + if (itor == cl_map_end(p_map)) return; - cl_qmap_remove_item( &p_map->qmap, (cl_map_item_t*)itor ); - cl_qpool_put( &p_map->pool, &((cl_map_item_t*)itor)->pool_item ); + cl_qmap_remove_item(&p_map->qmap, (cl_map_item_t *) itor); + cl_qpool_put(&p_map->pool, &((cl_map_item_t *) itor)->pool_item); } -void* -cl_map_remove( - IN cl_map_t* const p_map, - IN const uint64_t key ) +void *cl_map_remove(IN cl_map_t * const p_map, IN const uint64_t key) { - cl_map_item_t *p_item; - void *p_obj; + cl_map_item_t *p_item; + void *p_obj; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); - p_item = cl_qmap_remove( &p_map->qmap, key ); + p_item = cl_qmap_remove(&p_map->qmap, key); - if( p_item == cl_qmap_end( &p_map->qmap ) ) - return( NULL ); + if (p_item == cl_qmap_end(&p_map->qmap)) + return (NULL); - p_obj = cl_qmap_obj( (cl_map_obj_t*)p_item ); - cl_qpool_put( &p_map->pool, &p_item->pool_item ); + p_obj = cl_qmap_obj((cl_map_obj_t *) p_item); + cl_qpool_put(&p_map->pool, &p_item->pool_item); - return( p_obj ); + return (p_obj); } -void -cl_map_remove_all( - IN cl_map_t* const p_map ) +void cl_map_remove_all(IN cl_map_t * const p_map) { - cl_map_item_t *p_item; + cl_map_item_t *p_item; - CL_ASSERT( p_map ); + CL_ASSERT(p_map); /* Return all map items to the pool. */ - while( !cl_is_qmap_empty( &p_map->qmap ) ) - { - p_item = cl_qmap_head( &p_map->qmap ); - cl_qmap_remove_item( &p_map->qmap, p_item ); - cl_qpool_put( &p_map->pool, &p_item->pool_item ); - - if( !cl_is_qmap_empty( &p_map->qmap ) ) - { - p_item = cl_qmap_tail( &p_map->qmap ); - cl_qmap_remove_item( &p_map->qmap, p_item ); - cl_qpool_put( &p_map->pool, &p_item->pool_item ); + while (!cl_is_qmap_empty(&p_map->qmap)) { + p_item = cl_qmap_head(&p_map->qmap); + cl_qmap_remove_item(&p_map->qmap, p_item); + cl_qpool_put(&p_map->pool, &p_item->pool_item); + + if (!cl_is_qmap_empty(&p_map->qmap)) { + p_item = cl_qmap_tail(&p_map->qmap); + cl_qmap_remove_item(&p_map->qmap, p_item); + cl_qpool_put(&p_map->pool, &p_item->pool_item); } } } cl_status_t -cl_map_merge( - OUT cl_map_t* const p_dest_map, - IN OUT cl_map_t* const p_src_map ) +cl_map_merge(OUT cl_map_t * const p_dest_map, IN OUT cl_map_t * const p_src_map) { - cl_status_t status = CL_SUCCESS; - cl_map_iterator_t itor, next; - uint64_t key; - void *p_obj, *p_obj2; + cl_status_t status = CL_SUCCESS; + cl_map_iterator_t itor, next; + uint64_t key; + void *p_obj, *p_obj2; - CL_ASSERT( p_dest_map ); - CL_ASSERT( p_src_map ); + CL_ASSERT(p_dest_map); + CL_ASSERT(p_src_map); - itor = cl_map_head( p_src_map ); - while( itor != cl_map_end( p_src_map ) ) - { - next = cl_map_next( itor ); + itor = cl_map_head(p_src_map); + while (itor != cl_map_end(p_src_map)) { + next = cl_map_next(itor); - p_obj = cl_map_obj( itor ); - key = cl_map_key( itor ); + p_obj = cl_map_obj(itor); + key = cl_map_key(itor); - cl_map_remove_item( p_src_map, itor ); + cl_map_remove_item(p_src_map, itor); /* Insert the object into the destination map. */ - p_obj2 = cl_map_insert( p_dest_map, key, p_obj ); + p_obj2 = cl_map_insert(p_dest_map, key, p_obj); /* Trap for failure. */ - if( p_obj != p_obj2 ) - { - if( !p_obj2 ) + if (p_obj != p_obj2) { + if (!p_obj2) status = CL_INSUFFICIENT_MEMORY; /* Put the object back in the source map. This must succeed. */ - p_obj2 = cl_map_insert( p_src_map, key, p_obj ); - CL_ASSERT( p_obj == p_obj2 ); + p_obj2 = cl_map_insert(p_src_map, key, p_obj); + CL_ASSERT(p_obj == p_obj2); /* If the failure was due to insufficient memory, return. */ - if( status != CL_SUCCESS ) - return( status ); + if (status != CL_SUCCESS) + return (status); } itor = next; } - return( CL_SUCCESS ); + return (CL_SUCCESS); } static void -__cl_map_revert( - IN OUT cl_map_t* const p_map1, - IN OUT cl_map_t* const p_map2, - IN OUT cl_map_t* const p_new, - IN OUT cl_map_t* const p_old ) +__cl_map_revert(IN OUT cl_map_t * const p_map1, + IN OUT cl_map_t * const p_map2, + IN OUT cl_map_t * const p_new, IN OUT cl_map_t * const p_old) { - cl_status_t status; + cl_status_t status; /* Restore the initial state. */ - status = cl_map_merge( p_map1, p_old ); - CL_ASSERT( status == CL_SUCCESS ); - status = cl_map_merge( p_map2, p_new ); - CL_ASSERT( status == CL_SUCCESS ); + status = cl_map_merge(p_map1, p_old); + CL_ASSERT(status == CL_SUCCESS); + status = cl_map_merge(p_map2, p_new); + CL_ASSERT(status == CL_SUCCESS); } static cl_status_t -__cl_map_delta_move( - OUT cl_map_t* const p_dest, - IN OUT cl_map_t* const p_src, - IN OUT cl_map_iterator_t* const p_itor ) +__cl_map_delta_move(OUT cl_map_t * const p_dest, + IN OUT cl_map_t * const p_src, + IN OUT cl_map_iterator_t * const p_itor) { - cl_map_iterator_t next; - void *p_obj, *p_obj2; - uint64_t key; + cl_map_iterator_t next; + void *p_obj, *p_obj2; + uint64_t key; /* Get a valid iterator so we can continue the loop. */ - next = cl_map_next( *p_itor ); + next = cl_map_next(*p_itor); /* Get the pointer to the object for insertion. */ - p_obj = cl_map_obj( *p_itor ); + p_obj = cl_map_obj(*p_itor); /* Get the key for the object. */ - key = cl_map_key( *p_itor ); + key = cl_map_key(*p_itor); /* Move the object. */ - cl_map_remove_item( p_src, *p_itor ); - p_obj2 = cl_map_insert( p_dest, key, p_obj ); + cl_map_remove_item(p_src, *p_itor); + p_obj2 = cl_map_insert(p_dest, key, p_obj); /* Check for failure. We should never get a duplicate. */ - if( !p_obj2 ) - { - p_obj2 = cl_map_insert( p_src, key, p_obj ); - CL_ASSERT( p_obj2 == p_obj ); - return( CL_INSUFFICIENT_MEMORY ); + if (!p_obj2) { + p_obj2 = cl_map_insert(p_src, key, p_obj); + CL_ASSERT(p_obj2 == p_obj); + return (CL_INSUFFICIENT_MEMORY); } /* We should never get a duplicate */ - CL_ASSERT( p_obj == p_obj2 ); + CL_ASSERT(p_obj == p_obj2); /* Update the iterator so that it is valid. */ (*p_itor) = next; - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_map_delta( - IN OUT cl_map_t* const p_map1, - IN OUT cl_map_t* const p_map2, - OUT cl_map_t* const p_new, - OUT cl_map_t* const p_old ) +cl_map_delta(IN OUT cl_map_t * const p_map1, + IN OUT cl_map_t * const p_map2, + OUT cl_map_t * const p_new, OUT cl_map_t * const p_old) { - cl_map_iterator_t itor1, itor2; - uint64_t key1, key2; - cl_status_t status; + cl_map_iterator_t itor1, itor2; + uint64_t key1, key2; + cl_status_t status; - CL_ASSERT( p_map1 ); - CL_ASSERT( p_map2 ); - CL_ASSERT( p_new ); - CL_ASSERT( p_old ); - CL_ASSERT( cl_is_map_empty( p_new ) ); - CL_ASSERT( cl_is_map_empty( p_old ) ); + CL_ASSERT(p_map1); + CL_ASSERT(p_map2); + CL_ASSERT(p_new); + CL_ASSERT(p_old); + CL_ASSERT(cl_is_map_empty(p_new)); + CL_ASSERT(cl_is_map_empty(p_old)); - itor1 = cl_map_head( p_map1 ); - itor2 = cl_map_head( p_map2 ); + itor1 = cl_map_head(p_map1); + itor2 = cl_map_head(p_map2); /* * Note that the check is for the end, since duplicate items will remain * in their respective maps. */ - while( itor1 != cl_map_end( p_map1 ) && - itor2 != cl_map_end( p_map2 ) ) - { - key1 = cl_map_key( itor1 ); - key2 = cl_map_key( itor2 ); - if( key1 < key2 ) - { - status = __cl_map_delta_move( p_old, p_map1, &itor1 ); + while (itor1 != cl_map_end(p_map1) && itor2 != cl_map_end(p_map2)) { + key1 = cl_map_key(itor1); + key2 = cl_map_key(itor2); + if (key1 < key2) { + status = __cl_map_delta_move(p_old, p_map1, &itor1); /* Check for failure. */ - if( status != CL_SUCCESS ) - { + if (status != CL_SUCCESS) { /* Restore the initial state. */ - __cl_map_revert( p_map1, p_map2, p_new, p_old ); + __cl_map_revert(p_map1, p_map2, p_new, p_old); /* Return the failure status. */ - return( status ); + return (status); } - } - else if( key1 > key2 ) - { - status = __cl_map_delta_move( p_new, p_map2, &itor2 ); - if( status != CL_SUCCESS ) - { + } else if (key1 > key2) { + status = __cl_map_delta_move(p_new, p_map2, &itor2); + if (status != CL_SUCCESS) { /* Restore the initial state. */ - __cl_map_revert( p_map1, p_map2, p_new, p_old ); + __cl_map_revert(p_map1, p_map2, p_new, p_old); /* Return the failure status. */ - return( status ); + return (status); } - } - else - { + } else { /* Move both forward since they have the same key. */ - itor1 = cl_map_next( itor1 ); - itor2 = cl_map_next( itor2 ); + itor1 = cl_map_next(itor1); + itor2 = cl_map_next(itor2); } } /* Process the remainder if either source map is empty. */ - while( itor2 != cl_map_end( p_map2 ) ) - { - status = __cl_map_delta_move( p_new, p_map2, &itor2 ); - if( status != CL_SUCCESS ) - { + while (itor2 != cl_map_end(p_map2)) { + status = __cl_map_delta_move(p_new, p_map2, &itor2); + if (status != CL_SUCCESS) { /* Restore the initial state. */ - __cl_map_revert( p_map1, p_map2, p_new, p_old ); + __cl_map_revert(p_map1, p_map2, p_new, p_old); /* Return the failure status. */ - return( status ); + return (status); } } - while( itor1 != cl_map_end( p_map1 ) ) - { - status = __cl_map_delta_move( p_old, p_map1, &itor1 ); - if( status != CL_SUCCESS ) - { + while (itor1 != cl_map_end(p_map1)) { + status = __cl_map_delta_move(p_old, p_map1, &itor1); + if (status != CL_SUCCESS) { /* Restore the initial state. */ - __cl_map_revert( p_map1, p_map2, p_new, p_old ); + __cl_map_revert(p_map1, p_map2, p_new, p_old); /* Return the failure status. */ - return( status ); + return (status); } } - return( CL_SUCCESS ); + return (CL_SUCCESS); } /****************************************************************************** @@ -1129,44 +1013,39 @@ cl_map_delta( /* * Get the root. */ -static inline cl_fmap_item_t* -__cl_fmap_root( - IN const cl_fmap_t* const p_map ) +static inline cl_fmap_item_t *__cl_fmap_root(IN const cl_fmap_t * const p_map) { - CL_ASSERT( p_map ); - return( p_map->root.p_left ); + CL_ASSERT(p_map); + return (p_map->root.p_left); } /* * Returns whether a given item is on the left of its parent. */ -static boolean_t -__cl_fmap_is_left_child( - IN const cl_fmap_item_t* const p_item ) +static boolean_t __cl_fmap_is_left_child(IN const cl_fmap_item_t * const p_item) { - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_up ); - CL_ASSERT( p_item->p_up != p_item ); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_up); + CL_ASSERT(p_item->p_up != p_item); - return( p_item->p_up->p_left == p_item ); + return (p_item->p_up->p_left == p_item); } /* * Retrieve the pointer to the parent's pointer to an item. */ -static cl_fmap_item_t** -__cl_fmap_get_parent_ptr_to_item( - IN cl_fmap_item_t* const p_item ) +static cl_fmap_item_t **__cl_fmap_get_parent_ptr_to_item(IN cl_fmap_item_t * + const p_item) { - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_up ); - CL_ASSERT( p_item->p_up != p_item ); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_up); + CL_ASSERT(p_item->p_up != p_item); - if( __cl_fmap_is_left_child( p_item ) ) - return( &p_item->p_up->p_left ); + if (__cl_fmap_is_left_child(p_item)) + return (&p_item->p_up->p_left); - CL_ASSERT( p_item->p_up->p_right == p_item ); - return( &p_item->p_up->p_right ); + CL_ASSERT(p_item->p_up->p_right == p_item); + return (&p_item->p_up->p_right); } /* @@ -1185,17 +1064,15 @@ __cl_fmap_get_parent_ptr_to_item( * X Y X Y */ static void -__cl_fmap_rot_left( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* const p_item ) +__cl_fmap_rot_left(IN cl_fmap_t * const p_map, IN cl_fmap_item_t * const p_item) { - cl_fmap_item_t **pp_root; + cl_fmap_item_t **pp_root; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_right != &p_map->nil ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_right != &p_map->nil); - pp_root = __cl_fmap_get_parent_ptr_to_item( p_item ); + pp_root = __cl_fmap_get_parent_ptr_to_item(p_item); /* Point R to C instead of A. */ *pp_root = p_item->p_right; @@ -1208,7 +1085,7 @@ __cl_fmap_rot_left( * Set B's parent to A. We trap for B being NIL since the * caller may depend on NIL not changing. */ - if( (*pp_root)->p_left != &p_map->nil ) + if ((*pp_root)->p_left != &p_map->nil) (*pp_root)->p_left->p_up = p_item; /* Set C's left to A. */ @@ -1233,18 +1110,17 @@ __cl_fmap_rot_left( * X Y X Y */ static void -__cl_fmap_rot_right( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* const p_item ) +__cl_fmap_rot_right(IN cl_fmap_t * const p_map, + IN cl_fmap_item_t * const p_item) { - cl_fmap_item_t **pp_root; + cl_fmap_item_t **pp_root; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_left != &p_map->nil ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_left != &p_map->nil); /* Point R to A instead of C. */ - pp_root = __cl_fmap_get_parent_ptr_to_item( p_item ); + pp_root = __cl_fmap_get_parent_ptr_to_item(p_item); (*pp_root) = p_item->p_left; /* Set A's parent to R. */ (*pp_root)->p_up = p_item->p_up; @@ -1255,7 +1131,7 @@ __cl_fmap_rot_right( * Set B's parent to C. We trap for B being NIL since the * caller may depend on NIL not changing. */ - if( (*pp_root)->p_right != &p_map->nil ) + if ((*pp_root)->p_right != &p_map->nil) (*pp_root)->p_right->p_up = p_item; /* Set A's right to C. */ @@ -1264,15 +1140,12 @@ __cl_fmap_rot_right( p_item->p_up = *pp_root; } -void -cl_fmap_init( - IN cl_fmap_t* const p_map, - IN cl_pfn_fmap_cmp_t pfn_compare ) +void cl_fmap_init(IN cl_fmap_t * const p_map, IN cl_pfn_fmap_cmp_t pfn_compare) { - CL_ASSERT( p_map ); - CL_ASSERT( pfn_compare ); + CL_ASSERT(p_map); + CL_ASSERT(pfn_compare); - memset( p_map, 0, sizeof(cl_fmap_t) ); + memset(p_map, 0, sizeof(cl_fmap_t)); /* special setup for the root node */ p_map->root.p_up = &p_map->root; @@ -1291,86 +1164,78 @@ cl_fmap_init( p_map->state = CL_INITIALIZED; - cl_fmap_remove_all( p_map ); + cl_fmap_remove_all(p_map); } -cl_fmap_item_t* -cl_fmap_get( - IN const cl_fmap_t* const p_map, - IN const void* const p_key ) +cl_fmap_item_t *cl_fmap_get(IN const cl_fmap_t * const p_map, + IN const void *const p_key) { - cl_fmap_item_t *p_item; - intn_t cmp; + cl_fmap_item_t *p_item; + intn_t cmp; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - p_item = __cl_fmap_root( p_map ); + p_item = __cl_fmap_root(p_map); - while( p_item != &p_map->nil ) - { - cmp = p_map->pfn_compare( p_key, p_item->p_key ); + while (p_item != &p_map->nil) { + cmp = p_map->pfn_compare(p_key, p_item->p_key); - if( !cmp ) - break; /* just right */ + if (!cmp) + break; /* just right */ - if( cmp < 0 ) + if (cmp < 0) p_item = p_item->p_left; /* too small */ else p_item = p_item->p_right; /* too big */ } - return( p_item ); + return (p_item); } -cl_fmap_item_t* -cl_fmap_get_next( - IN const cl_fmap_t* const p_map, - IN const void* const p_key ) +cl_fmap_item_t *cl_fmap_get_next(IN const cl_fmap_t * const p_map, + IN const void *const p_key) { - cl_fmap_item_t *p_item; - cl_fmap_item_t *p_item_found; - intn_t cmp; + cl_fmap_item_t *p_item; + cl_fmap_item_t *p_item_found; + intn_t cmp; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); - p_item = __cl_fmap_root( p_map ); - p_item_found = (cl_fmap_item_t*)&p_map->nil; + p_item = __cl_fmap_root(p_map); + p_item_found = (cl_fmap_item_t *) & p_map->nil; - while( p_item != &p_map->nil ) - { - cmp = p_map->pfn_compare( p_key, p_item->p_key ); + while (p_item != &p_map->nil) { + cmp = p_map->pfn_compare(p_key, p_item->p_key); - if( cmp < 0 ){ + if (cmp < 0) { p_item_found = p_item; p_item = p_item->p_left; /* too small */ - }else{ + } else { p_item = p_item->p_right; /* too big or match */ } } - return( p_item_found ); + return (p_item_found); } void -cl_fmap_apply_func( - IN const cl_fmap_t* const p_map, - IN cl_pfn_fmap_apply_t pfn_func, - IN const void* const context ) +cl_fmap_apply_func(IN const cl_fmap_t * const p_map, + IN cl_pfn_fmap_apply_t pfn_func, + IN const void *const context) { - cl_fmap_item_t* p_fmap_item; + cl_fmap_item_t *p_fmap_item; /* Note that context can have any arbitrary value. */ - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( pfn_func ); - - p_fmap_item = cl_fmap_head( p_map ); - while( p_fmap_item != cl_fmap_end( p_map ) ) - { - pfn_func( p_fmap_item, (void*)context ); - p_fmap_item = cl_fmap_next( p_fmap_item ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(pfn_func); + + p_fmap_item = cl_fmap_head(p_map); + while (p_fmap_item != cl_fmap_end(p_map)) { + pfn_func(p_fmap_item, (void *)context); + p_fmap_item = cl_fmap_next(p_fmap_item); } } @@ -1378,24 +1243,19 @@ cl_fmap_apply_func( * Balance a tree starting at a given item back to the root. */ static void -__cl_fmap_ins_bal( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* p_item ) +__cl_fmap_ins_bal(IN cl_fmap_t * const p_map, IN cl_fmap_item_t * p_item) { - cl_fmap_item_t* p_grand_uncle; + cl_fmap_item_t *p_grand_uncle; - CL_ASSERT( p_map ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item != &p_map->root ); + CL_ASSERT(p_map); + CL_ASSERT(p_item); + CL_ASSERT(p_item != &p_map->root); - while( p_item->p_up->color == CL_MAP_RED ) - { - if( __cl_fmap_is_left_child( p_item->p_up ) ) - { + while (p_item->p_up->color == CL_MAP_RED) { + if (__cl_fmap_is_left_child(p_item->p_up)) { p_grand_uncle = p_item->p_up->p_up->p_right; - CL_ASSERT( p_grand_uncle ); - if( p_grand_uncle->color == CL_MAP_RED ) - { + CL_ASSERT(p_grand_uncle); + if (p_grand_uncle->color == CL_MAP_RED) { p_grand_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; @@ -1403,21 +1263,17 @@ __cl_fmap_ins_bal( continue; } - if( !__cl_fmap_is_left_child( p_item ) ) - { + if (!__cl_fmap_is_left_child(p_item)) { p_item = p_item->p_up; - __cl_fmap_rot_left( p_map, p_item ); + __cl_fmap_rot_left(p_map, p_item); } p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; - __cl_fmap_rot_right( p_map, p_item->p_up->p_up ); - } - else - { + __cl_fmap_rot_right(p_map, p_item->p_up->p_up); + } else { p_grand_uncle = p_item->p_up->p_up->p_left; - CL_ASSERT( p_grand_uncle ); - if( p_grand_uncle->color == CL_MAP_RED ) - { + CL_ASSERT(p_grand_uncle); + if (p_grand_uncle->color == CL_MAP_RED) { p_grand_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; @@ -1425,33 +1281,30 @@ __cl_fmap_ins_bal( continue; } - if( __cl_fmap_is_left_child( p_item ) ) - { + if (__cl_fmap_is_left_child(p_item)) { p_item = p_item->p_up; - __cl_fmap_rot_right( p_map, p_item ); + __cl_fmap_rot_right(p_map, p_item); } p_item->p_up->color = CL_MAP_BLACK; p_item->p_up->p_up->color = CL_MAP_RED; - __cl_fmap_rot_left( p_map, p_item->p_up->p_up ); + __cl_fmap_rot_left(p_map, p_item->p_up->p_up); } } } -cl_fmap_item_t* -cl_fmap_insert( - IN cl_fmap_t* const p_map, - IN const void* const p_key, - IN cl_fmap_item_t* const p_item ) +cl_fmap_item_t *cl_fmap_insert(IN cl_fmap_t * const p_map, + IN const void *const p_key, + IN cl_fmap_item_t * const p_item) { - cl_fmap_item_t *p_insert_at, *p_comp_item; - intn_t cmp = 0; + cl_fmap_item_t *p_insert_at, *p_comp_item; + intn_t cmp = 0; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( p_item ); - CL_ASSERT( p_map->root.p_up == &p_map->root ); - CL_ASSERT( p_map->root.color != CL_MAP_RED ); - CL_ASSERT( p_map->nil.color != CL_MAP_RED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(p_item); + CL_ASSERT(p_map->root.p_up == &p_map->root); + CL_ASSERT(p_map->root.color != CL_MAP_RED); + CL_ASSERT(p_map->nil.color != CL_MAP_RED); p_item->p_left = &p_map->nil; p_item->p_right = &p_map->nil; @@ -1460,56 +1313,50 @@ cl_fmap_insert( /* Find the insertion location. */ p_insert_at = &p_map->root; - p_comp_item = __cl_fmap_root( p_map ); + p_comp_item = __cl_fmap_root(p_map); - while( p_comp_item != &p_map->nil ) - { + while (p_comp_item != &p_map->nil) { p_insert_at = p_comp_item; - cmp = p_map->pfn_compare( p_key, p_insert_at->p_key ); + cmp = p_map->pfn_compare(p_key, p_insert_at->p_key); - if( !cmp ) - return( p_insert_at ); + if (!cmp) + return (p_insert_at); /* Traverse the tree until the correct insertion point is found. */ - if( cmp < 0 ) + if (cmp < 0) p_comp_item = p_insert_at->p_left; else p_comp_item = p_insert_at->p_right; } - CL_ASSERT( p_insert_at != &p_map->nil ); - CL_ASSERT( p_comp_item == &p_map->nil ); + CL_ASSERT(p_insert_at != &p_map->nil); + CL_ASSERT(p_comp_item == &p_map->nil); /* Insert the item. */ - if( p_insert_at == &p_map->root ) - { + if (p_insert_at == &p_map->root) { p_insert_at->p_left = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( &p_map->nil.pool_item.list_item, - &p_item->pool_item.list_item ); - } - else if( cmp < 0 ) - { + __cl_primitive_insert(&p_map->nil.pool_item.list_item, + &p_item->pool_item.list_item); + } else if (cmp < 0) { p_insert_at->p_left = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( &p_insert_at->pool_item.list_item, - &p_item->pool_item.list_item ); - } - else - { + __cl_primitive_insert(&p_insert_at->pool_item.list_item, + &p_item->pool_item.list_item); + } else { p_insert_at->p_right = p_item; /* * Primitive insert places the new item in front of * the existing item. */ - __cl_primitive_insert( p_insert_at->pool_item.list_item.p_next, - &p_item->pool_item.list_item ); + __cl_primitive_insert(p_insert_at->pool_item.list_item.p_next, + &p_item->pool_item.list_item); } /* Increase the count. */ p_map->count++; @@ -1521,9 +1368,9 @@ cl_fmap_insert( * Rebalance as necessary as we retrace our path through the tree * and update colors. */ - __cl_fmap_ins_bal( p_map, p_item ); + __cl_fmap_ins_bal(p_map, p_item); - __cl_fmap_root( p_map )->color = CL_MAP_BLACK; + __cl_fmap_root(p_map)->color = CL_MAP_BLACK; /* * Note that it is not necessary to re-color the nil node black because all @@ -1536,34 +1383,27 @@ cl_fmap_insert( p_item->p_map = p_map; #endif - return( p_item ); + return (p_item); } static void -__cl_fmap_del_bal( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* p_item ) +__cl_fmap_del_bal(IN cl_fmap_t * const p_map, IN cl_fmap_item_t * p_item) { - cl_fmap_item_t *p_uncle; + cl_fmap_item_t *p_uncle; - while( (p_item->color != CL_MAP_RED) && (p_item->p_up != &p_map->root) ) - { - if( __cl_fmap_is_left_child( p_item ) ) - { + while ((p_item->color != CL_MAP_RED) && (p_item->p_up != &p_map->root)) { + if (__cl_fmap_is_left_child(p_item)) { p_uncle = p_item->p_up->p_right; - if( p_uncle->color == CL_MAP_RED ) - { + if (p_uncle->color == CL_MAP_RED) { p_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_RED; - __cl_fmap_rot_left( p_map, p_item->p_up ); + __cl_fmap_rot_left(p_map, p_item->p_up); p_uncle = p_item->p_up->p_right; } - if( p_uncle->p_right->color != CL_MAP_RED ) - { - if( p_uncle->p_left->color != CL_MAP_RED ) - { + if (p_uncle->p_right->color != CL_MAP_RED) { + if (p_uncle->p_left->color != CL_MAP_RED) { p_uncle->color = CL_MAP_RED; p_item = p_item->p_up; continue; @@ -1571,31 +1411,26 @@ __cl_fmap_del_bal( p_uncle->p_left->color = CL_MAP_BLACK; p_uncle->color = CL_MAP_RED; - __cl_fmap_rot_right( p_map, p_uncle ); + __cl_fmap_rot_right(p_map, p_uncle); p_uncle = p_item->p_up->p_right; } p_uncle->color = p_item->p_up->color; p_item->p_up->color = CL_MAP_BLACK; p_uncle->p_right->color = CL_MAP_BLACK; - __cl_fmap_rot_left( p_map, p_item->p_up ); + __cl_fmap_rot_left(p_map, p_item->p_up); break; - } - else - { + } else { p_uncle = p_item->p_up->p_left; - if( p_uncle->color == CL_MAP_RED ) - { + if (p_uncle->color == CL_MAP_RED) { p_uncle->color = CL_MAP_BLACK; p_item->p_up->color = CL_MAP_RED; - __cl_fmap_rot_right( p_map, p_item->p_up ); + __cl_fmap_rot_right(p_map, p_item->p_up); p_uncle = p_item->p_up->p_left; } - if( p_uncle->p_left->color != CL_MAP_RED ) - { - if( p_uncle->p_right->color != CL_MAP_RED ) - { + if (p_uncle->p_left->color != CL_MAP_RED) { + if (p_uncle->p_right->color != CL_MAP_RED) { p_uncle->color = CL_MAP_RED; p_item = p_item->p_up; continue; @@ -1603,13 +1438,13 @@ __cl_fmap_del_bal( p_uncle->p_right->color = CL_MAP_BLACK; p_uncle->color = CL_MAP_RED; - __cl_fmap_rot_left( p_map, p_uncle ); + __cl_fmap_rot_left(p_map, p_uncle); p_uncle = p_item->p_up->p_left; } p_uncle->color = p_item->p_up->color; p_item->p_up->color = CL_MAP_BLACK; p_uncle->p_left->color = CL_MAP_BLACK; - __cl_fmap_rot_right( p_map, p_item->p_up ); + __cl_fmap_rot_right(p_map, p_item->p_up); break; } } @@ -1617,27 +1452,23 @@ __cl_fmap_del_bal( } void -cl_fmap_remove_item( - IN cl_fmap_t* const p_map, - IN cl_fmap_item_t* const p_item ) +cl_fmap_remove_item(IN cl_fmap_t * const p_map, + IN cl_fmap_item_t * const p_item) { - cl_fmap_item_t *p_child, *p_del_item; + cl_fmap_item_t *p_child, *p_del_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); - CL_ASSERT( p_item ); - CL_ASSERT( p_item->p_map == p_map ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); + CL_ASSERT(p_item); + CL_ASSERT(p_item->p_map == p_map); - if( p_item == cl_fmap_end( p_map ) ) + if (p_item == cl_fmap_end(p_map)) return; - if( (p_item->p_right == &p_map->nil) || (p_item->p_left == &p_map->nil ) ) - { + if ((p_item->p_right == &p_map->nil) || (p_item->p_left == &p_map->nil)) { /* The item being removed has children on at most on side. */ p_del_item = p_item; - } - else - { + } else { /* * The item being removed has children on both side. * We select the item that will replace it. After removing @@ -1645,17 +1476,17 @@ cl_fmap_remove_item( * correct topology. Exchanging the substitute for the item * will finalize the removal. */ - p_del_item = cl_fmap_next( p_item ); - CL_ASSERT( p_del_item != &p_map->nil ); + p_del_item = cl_fmap_next(p_item); + CL_ASSERT(p_del_item != &p_map->nil); } /* Remove the item from the list. */ - __cl_primitive_remove( &p_item->pool_item.list_item ); + __cl_primitive_remove(&p_item->pool_item.list_item); /* Decrement the item count. */ p_map->count--; /* Get the pointer to the new root's child, if any. */ - if( p_del_item->p_left != &p_map->nil ) + if (p_del_item->p_left != &p_map->nil) p_child = p_del_item->p_left; else p_child = p_del_item->p_right; @@ -1665,10 +1496,10 @@ cl_fmap_remove_item( * This is inconsequential. */ p_child->p_up = p_del_item->p_up; - (*__cl_fmap_get_parent_ptr_to_item( p_del_item )) = p_child; + (*__cl_fmap_get_parent_ptr_to_item(p_del_item)) = p_child; - if( p_del_item->color != CL_MAP_RED ) - __cl_fmap_del_bal( p_map, p_child ); + if (p_del_item->color != CL_MAP_RED) + __cl_fmap_del_bal(p_map, p_child); /* * Note that the splicing done below does not need to occur before @@ -1676,8 +1507,7 @@ cl_fmap_remove_item( * preceding code. The topology is preserved by the color assignment made * below (reader should be reminded that p_del_item == p_item in some cases). */ - if( p_del_item != p_item ) - { + if (p_del_item != p_item) { /* * Finalize the removal of the specified item by exchanging it with * the substitute which we removed above. @@ -1685,13 +1515,13 @@ cl_fmap_remove_item( p_del_item->p_up = p_item->p_up; p_del_item->p_left = p_item->p_left; p_del_item->p_right = p_item->p_right; - (*__cl_fmap_get_parent_ptr_to_item( p_item )) = p_del_item; + (*__cl_fmap_get_parent_ptr_to_item(p_item)) = p_del_item; p_item->p_right->p_up = p_del_item; p_item->p_left->p_up = p_del_item; p_del_item->color = p_item->color; } - CL_ASSERT( p_map->nil.color != CL_MAP_RED ); + CL_ASSERT(p_map->nil.color != CL_MAP_RED); #ifdef _DEBUG_ /* Clear the pointer to the map since the item has been removed. */ @@ -1699,125 +1529,112 @@ cl_fmap_remove_item( #endif } -cl_fmap_item_t* -cl_fmap_remove( - IN cl_fmap_t* const p_map, - IN const void* const p_key ) +cl_fmap_item_t *cl_fmap_remove(IN cl_fmap_t * const p_map, + IN const void *const p_key) { - cl_fmap_item_t *p_item; + cl_fmap_item_t *p_item; - CL_ASSERT( p_map ); - CL_ASSERT( p_map->state == CL_INITIALIZED ); + CL_ASSERT(p_map); + CL_ASSERT(p_map->state == CL_INITIALIZED); /* Seek the node with the specified key */ - p_item = cl_fmap_get( p_map, p_key ); + p_item = cl_fmap_get(p_map, p_key); - cl_fmap_remove_item( p_map, p_item ); + cl_fmap_remove_item(p_map, p_item); - return( p_item ); + return (p_item); } void -cl_fmap_merge( - OUT cl_fmap_t* const p_dest_map, - IN OUT cl_fmap_t* const p_src_map ) +cl_fmap_merge(OUT cl_fmap_t * const p_dest_map, + IN OUT cl_fmap_t * const p_src_map) { - cl_fmap_item_t *p_item, *p_item2, *p_next; + cl_fmap_item_t *p_item, *p_item2, *p_next; - CL_ASSERT( p_dest_map ); - CL_ASSERT( p_src_map ); + CL_ASSERT(p_dest_map); + CL_ASSERT(p_src_map); - p_item = cl_fmap_head( p_src_map ); + p_item = cl_fmap_head(p_src_map); - while( p_item != cl_fmap_end( p_src_map ) ) - { - p_next = cl_fmap_next( p_item ); + while (p_item != cl_fmap_end(p_src_map)) { + p_next = cl_fmap_next(p_item); /* Remove the item from its current map. */ - cl_fmap_remove_item( p_src_map, p_item ); + cl_fmap_remove_item(p_src_map, p_item); /* Insert the item into the destination map. */ - p_item2 = cl_fmap_insert( p_dest_map, cl_fmap_key( p_item ), p_item ); + p_item2 = + cl_fmap_insert(p_dest_map, cl_fmap_key(p_item), p_item); /* Check that the item was successfully inserted. */ - if( p_item2 != p_item ) - { + if (p_item2 != p_item) { /* Put the item in back in the source map. */ p_item2 = - cl_fmap_insert( p_src_map, cl_fmap_key( p_item ), p_item ); - CL_ASSERT( p_item2 == p_item ); + cl_fmap_insert(p_src_map, cl_fmap_key(p_item), + p_item); + CL_ASSERT(p_item2 == p_item); } p_item = p_next; } } static void -__cl_fmap_delta_move( - IN OUT cl_fmap_t* const p_dest, - IN OUT cl_fmap_t* const p_src, - IN OUT cl_fmap_item_t** const pp_item ) +__cl_fmap_delta_move(IN OUT cl_fmap_t * const p_dest, + IN OUT cl_fmap_t * const p_src, + IN OUT cl_fmap_item_t ** const pp_item) { - cl_fmap_item_t *p_temp, *p_next; + cl_fmap_item_t *p_temp, *p_next; /* * Get the next item so that we can ensure that pp_item points to * a valid item upon return from the function. */ - p_next = cl_fmap_next( *pp_item ); + p_next = cl_fmap_next(*pp_item); /* Move the old item from its current map the the old map. */ - cl_fmap_remove_item( p_src, *pp_item ); - p_temp = cl_fmap_insert( p_dest, cl_fmap_key( *pp_item ), *pp_item ); + cl_fmap_remove_item(p_src, *pp_item); + p_temp = cl_fmap_insert(p_dest, cl_fmap_key(*pp_item), *pp_item); /* We should never have duplicates. */ - CL_ASSERT( p_temp == *pp_item ); + CL_ASSERT(p_temp == *pp_item); /* Point pp_item to a valid item in the source map. */ (*pp_item) = p_next; } void -cl_fmap_delta( - IN OUT cl_fmap_t* const p_map1, - IN OUT cl_fmap_t* const p_map2, - OUT cl_fmap_t* const p_new, - OUT cl_fmap_t* const p_old ) +cl_fmap_delta(IN OUT cl_fmap_t * const p_map1, + IN OUT cl_fmap_t * const p_map2, + OUT cl_fmap_t * const p_new, OUT cl_fmap_t * const p_old) { - cl_fmap_item_t *p_item1, *p_item2; - intn_t cmp; - - CL_ASSERT( p_map1 ); - CL_ASSERT( p_map2 ); - CL_ASSERT( p_new ); - CL_ASSERT( p_old ); - CL_ASSERT( cl_is_fmap_empty( p_new ) ); - CL_ASSERT( cl_is_fmap_empty( p_old ) ); - - p_item1 = cl_fmap_head( p_map1 ); - p_item2 = cl_fmap_head( p_map2 ); - - while( p_item1 != cl_fmap_end( p_map1 ) && - p_item2 != cl_fmap_end( p_map2 ) ) - { - cmp = p_map1->pfn_compare( cl_fmap_key( p_item1 ), - cl_fmap_key( p_item2 ) ); - if( cmp < 0 ) - { + cl_fmap_item_t *p_item1, *p_item2; + intn_t cmp; + + CL_ASSERT(p_map1); + CL_ASSERT(p_map2); + CL_ASSERT(p_new); + CL_ASSERT(p_old); + CL_ASSERT(cl_is_fmap_empty(p_new)); + CL_ASSERT(cl_is_fmap_empty(p_old)); + + p_item1 = cl_fmap_head(p_map1); + p_item2 = cl_fmap_head(p_map2); + + while (p_item1 != cl_fmap_end(p_map1) && p_item2 != cl_fmap_end(p_map2)) { + cmp = p_map1->pfn_compare(cl_fmap_key(p_item1), + cl_fmap_key(p_item2)); + if (cmp < 0) { /* We found an old item. */ - __cl_fmap_delta_move( p_old, p_map1, &p_item1 ); - } - else if( cmp > 0 ) - { + __cl_fmap_delta_move(p_old, p_map1, &p_item1); + } else if (cmp > 0) { /* We found a new item. */ - __cl_fmap_delta_move( p_new, p_map2, &p_item2 ); - } - else - { + __cl_fmap_delta_move(p_new, p_map2, &p_item2); + } else { /* Move both forward since they have the same key. */ - p_item1 = cl_fmap_next( p_item1 ); - p_item2 = cl_fmap_next( p_item2 ); + p_item1 = cl_fmap_next(p_item1); + p_item2 = cl_fmap_next(p_item2); } } /* Process the remainder if the end of either source map was reached. */ - while( p_item2 != cl_fmap_end( p_map2 ) ) - __cl_fmap_delta_move( p_new, p_map2, &p_item2 ); + while (p_item2 != cl_fmap_end(p_map2)) + __cl_fmap_delta_move(p_new, p_map2, &p_item2); - while( p_item1 != cl_fmap_end( p_map1 ) ) - __cl_fmap_delta_move( p_old, p_map1, &p_item1 ); + while (p_item1 != cl_fmap_end(p_map1)) + __cl_fmap_delta_move(p_old, p_map1, &p_item1); } diff --git a/opensm/complib/cl_pool.c b/opensm/complib/cl_pool.c index 6ee1d98..1b3db3f 100644 --- a/opensm/complib/cl_pool.c +++ b/opensm/complib/cl_pool.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -59,88 +59,85 @@ /* * IMPLEMENTATION OF QUICK COMPOSITE POOL */ -void -cl_qcpool_construct( - IN cl_qcpool_t* const p_pool ) +void cl_qcpool_construct(IN cl_qcpool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - memset( p_pool, 0, sizeof(cl_qcpool_t) ); + memset(p_pool, 0, sizeof(cl_qcpool_t)); p_pool->state = CL_UNINITIALIZED; } cl_status_t -cl_qcpool_init( - IN cl_qcpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t* const component_sizes, - IN const uint32_t num_components, - IN cl_pfn_qcpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_qcpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ) +cl_qcpool_init(IN cl_qcpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t * const component_sizes, + IN const uint32_t num_components, + IN cl_pfn_qcpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_qcpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context) { - cl_status_t status; - uint32_t i; + cl_status_t status; + uint32_t i; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); /* Must have a minimum of 1 component. */ - CL_ASSERT( num_components ); + CL_ASSERT(num_components); /* A component size array is required. */ - CL_ASSERT( component_sizes ); + CL_ASSERT(component_sizes); /* * If no initializer is provided, the first component must be large * enough to hold a pool item. */ - CL_ASSERT( pfn_initializer || - (component_sizes[0] >= sizeof(cl_pool_item_t)) ); + CL_ASSERT(pfn_initializer || + (component_sizes[0] >= sizeof(cl_pool_item_t))); - cl_qcpool_construct( p_pool ); + cl_qcpool_construct(p_pool); - if( num_components > 1 && !pfn_initializer ) - return( CL_INVALID_SETTING ); + if (num_components > 1 && !pfn_initializer) + return (CL_INVALID_SETTING); - if( max_size && max_size < min_size ) - return( CL_INVALID_SETTING ); + if (max_size && max_size < min_size) + return (CL_INVALID_SETTING); /* * Allocate the array of component sizes and component pointers all * in one allocation. */ - p_pool->component_sizes = (size_t*)malloc( - (sizeof(size_t) + sizeof(void*)) * num_components ); + p_pool->component_sizes = (size_t *) malloc((sizeof(size_t) + + sizeof(void *)) * + num_components); - if( !p_pool->component_sizes ) - return( CL_INSUFFICIENT_MEMORY ); + if (!p_pool->component_sizes) + return (CL_INSUFFICIENT_MEMORY); else - memset( p_pool->component_sizes, 0, - (sizeof(size_t) + sizeof(void*)) * num_components ); + memset(p_pool->component_sizes, 0, + (sizeof(size_t) + sizeof(void *)) * num_components); /* Calculate the pointer to the array of pointers, used for callbacks. */ p_pool->p_components = - (void**)(p_pool->component_sizes + num_components); + (void **)(p_pool->component_sizes + num_components); /* Copy the user's sizes into our array for future use. */ - memcpy( p_pool->component_sizes, component_sizes, - sizeof(component_sizes[0]) * num_components ); + memcpy(p_pool->component_sizes, component_sizes, + sizeof(component_sizes[0]) * num_components); /* Store the number of components per object. */ p_pool->num_components = num_components; /* Round up and store the size of the components. */ - for( i = 0; i < num_components; i++ ) - { + for (i = 0; i < num_components; i++) { /* * We roundup each component size so that all components * are aligned on a natural boundary. */ p_pool->component_sizes[i] = - ROUNDUP( p_pool->component_sizes[i], sizeof(uintn_t) ); + ROUNDUP(p_pool->component_sizes[i], sizeof(uintn_t)); } - p_pool->max_objects = max_size? max_size : ~(size_t)0; + p_pool->max_objects = max_size ? max_size : ~(size_t) 0; p_pool->grow_size = grow_size; /* Store callback function pointers. */ @@ -148,9 +145,9 @@ cl_qcpool_init( p_pool->pfn_dtor = pfn_destructor; /* may be NULL */ p_pool->context = context; - cl_qlist_init( &p_pool->alloc_list ); + cl_qlist_init(&p_pool->alloc_list); - cl_qlist_init( &p_pool->free_list ); + cl_qlist_init(&p_pool->free_list); /* * We are now initialized. We change the initialized flag before @@ -159,28 +156,25 @@ cl_qcpool_init( p_pool->state = CL_INITIALIZED; /* Allocate the minimum number of objects as requested. */ - if( !min_size ) - return( CL_SUCCESS ); + if (!min_size) + return (CL_SUCCESS); - status = cl_qcpool_grow( p_pool, min_size ); + status = cl_qcpool_grow(p_pool, min_size); /* Trap for error and cleanup if necessary. */ - if( status != CL_SUCCESS ) - cl_qcpool_destroy( p_pool ); + if (status != CL_SUCCESS) + cl_qcpool_destroy(p_pool); - return( status ); + return (status); } -void -cl_qcpool_destroy( - IN cl_qcpool_t* const p_pool ) +void cl_qcpool_destroy(IN cl_qcpool_t * const p_pool) { /* CL_ASSERT that a non-NULL pointer was provided. */ - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); /* CL_ASSERT that we are in a valid state (not uninitialized memory). */ - CL_ASSERT( cl_is_state_valid( p_pool->state ) ); + CL_ASSERT(cl_is_state_valid(p_pool->state)); - if( p_pool->state == CL_INITIALIZED ) - { + if (p_pool->state == CL_INITIALIZED) { /* * Assert if the user hasn't put everything back in the pool * before destroying it @@ -188,31 +182,27 @@ cl_qcpool_destroy( * that will be freed, and the destructor will not be called! */ #ifdef _DEBUG_ - /* but we do not want "free" version to assert on this one */ - CL_ASSERT( cl_qcpool_count( p_pool ) == p_pool->num_objects ); + /* but we do not want "free" version to assert on this one */ + CL_ASSERT(cl_qcpool_count(p_pool) == p_pool->num_objects); #endif /* call the user's destructor for each object in the pool */ - if( p_pool->pfn_dtor ) - { - while( !cl_is_qlist_empty( &p_pool->free_list ) ) - { - p_pool->pfn_dtor( (cl_pool_item_t*) - cl_qlist_remove_head( &p_pool->free_list ), - (void*)p_pool->context ); + if (p_pool->pfn_dtor) { + while (!cl_is_qlist_empty(&p_pool->free_list)) { + p_pool->pfn_dtor((cl_pool_item_t *) + cl_qlist_remove_head(&p_pool-> + free_list), + (void *)p_pool->context); } - } - else - { - cl_qlist_remove_all( &p_pool->free_list ); + } else { + cl_qlist_remove_all(&p_pool->free_list); } /* Free all allocated memory blocks. */ - while( !cl_is_qlist_empty( &p_pool->alloc_list ) ) - free( cl_qlist_remove_head( &p_pool->alloc_list ) ); + while (!cl_is_qlist_empty(&p_pool->alloc_list)) + free(cl_qlist_remove_head(&p_pool->alloc_list)); - if( p_pool->component_sizes ) - { - free( p_pool->component_sizes ); + if (p_pool->component_sizes) { + free(p_pool->component_sizes); p_pool->component_sizes = NULL; } } @@ -220,93 +210,90 @@ cl_qcpool_destroy( p_pool->state = CL_UNINITIALIZED; } -cl_status_t -cl_qcpool_grow( - IN cl_qcpool_t* const p_pool, - IN size_t obj_count ) +cl_status_t cl_qcpool_grow(IN cl_qcpool_t * const p_pool, IN size_t obj_count) { - cl_status_t status = CL_SUCCESS; - uint8_t *p_objects; - cl_pool_item_t *p_pool_item; - uint32_t i; - size_t obj_size; + cl_status_t status = CL_SUCCESS; + uint8_t *p_objects; + cl_pool_item_t *p_pool_item; + uint32_t i; + size_t obj_size; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); - CL_ASSERT( obj_count ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); + CL_ASSERT(obj_count); /* Validate that growth is possible. */ - if( p_pool->num_objects == p_pool->max_objects ) - return( CL_INSUFFICIENT_MEMORY ); + if (p_pool->num_objects == p_pool->max_objects) + return (CL_INSUFFICIENT_MEMORY); /* Cap the growth to the desired maximum. */ - if( obj_count > (p_pool->max_objects - p_pool->num_objects) ) + if (obj_count > (p_pool->max_objects - p_pool->num_objects)) obj_count = p_pool->max_objects - p_pool->num_objects; /* Calculate the size of an object. */ obj_size = 0; - for( i = 0; i < p_pool->num_components; i++ ) + for (i = 0; i < p_pool->num_components; i++) obj_size += p_pool->component_sizes[i]; /* Allocate the buffer for the new objects. */ - p_objects = (uint8_t*) - malloc( sizeof(cl_list_item_t) + (obj_size * obj_count) ); + p_objects = (uint8_t *) + malloc(sizeof(cl_list_item_t) + (obj_size * obj_count)); /* Make sure the allocation succeeded. */ - if( !p_objects ) - return( CL_INSUFFICIENT_MEMORY ); + if (!p_objects) + return (CL_INSUFFICIENT_MEMORY); else - memset( p_objects, 0, - sizeof(cl_list_item_t) + (obj_size * obj_count) ); + memset(p_objects, 0, + sizeof(cl_list_item_t) + (obj_size * obj_count)); /* Insert the allocation in our list. */ - cl_qlist_insert_tail( &p_pool->alloc_list, (cl_list_item_t*)p_objects ); + cl_qlist_insert_tail(&p_pool->alloc_list, (cl_list_item_t *) p_objects); p_objects += sizeof(cl_list_item_t); /* initialize the new elements and add them to the free list */ - while( obj_count-- ) - { + while (obj_count--) { /* Setup the array of components for the current object. */ p_pool->p_components[0] = p_objects; - for( i = 1; i < p_pool->num_components; i++ ) - { + for (i = 1; i < p_pool->num_components; i++) { /* Calculate the pointer to the next component. */ - p_pool->p_components[i] = (uint8_t*)p_pool->p_components[i-1] + - p_pool->component_sizes[i-1]; + p_pool->p_components[i] = + (uint8_t *) p_pool->p_components[i - 1] + + p_pool->component_sizes[i - 1]; } /* * call the user's initializer * this can fail! */ - if( p_pool->pfn_init ) - { + if (p_pool->pfn_init) { p_pool_item = NULL; - status = p_pool->pfn_init( p_pool->p_components, - p_pool->num_components, (void*)p_pool->context, &p_pool_item ); - if( status != CL_SUCCESS ) - { + status = p_pool->pfn_init(p_pool->p_components, + p_pool->num_components, + (void *)p_pool->context, + &p_pool_item); + if (status != CL_SUCCESS) { /* * User initialization failed * we may have only grown the pool by some partial amount * Invoke the destructor for the object that failed * initialization. */ - if( p_pool->pfn_dtor ) - p_pool->pfn_dtor( p_pool_item, (void*)p_pool->context ); + if (p_pool->pfn_dtor) + p_pool->pfn_dtor(p_pool_item, + (void *)p_pool-> + context); /* Return the user's status. */ - return( status ); + return (status); } - CL_ASSERT( p_pool_item ); - } - else - { + CL_ASSERT(p_pool_item); + } else { /* * If no initializer is provided, assume that the pool item * is stored at the beginning of the first component. */ - p_pool_item = (cl_pool_item_t*)p_pool->p_components[0]; + p_pool_item = + (cl_pool_item_t *) p_pool->p_components[0]; } #ifdef _DEBUG_ @@ -318,7 +305,8 @@ cl_qcpool_grow( #endif /* Insert the new item in the free list, traping for failure. */ - cl_qlist_insert_head( &p_pool->free_list, &p_pool_item->list_item ); + cl_qlist_insert_head(&p_pool->free_list, + &p_pool_item->list_item); p_pool->num_objects++; @@ -326,75 +314,69 @@ cl_qcpool_grow( p_objects += obj_size; } - return( status ); + return (status); } -cl_pool_item_t* -cl_qcpool_get( - IN cl_qcpool_t* const p_pool ) +cl_pool_item_t *cl_qcpool_get(IN cl_qcpool_t * const p_pool) { - cl_list_item_t *p_list_item; + cl_list_item_t *p_list_item; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); - if( cl_is_qlist_empty( &p_pool->free_list ) ) - { + if (cl_is_qlist_empty(&p_pool->free_list)) { /* * No object is available. * Return NULL if the user does not want automatic growth. */ - if( !p_pool->grow_size ) - return( NULL ); + if (!p_pool->grow_size) + return (NULL); /* We ran out of elements. Get more */ - cl_qcpool_grow( p_pool, p_pool->grow_size ); + cl_qcpool_grow(p_pool, p_pool->grow_size); /* * We may not have gotten everything we wanted but we might have * gotten something. */ - if( cl_is_qlist_empty( &p_pool->free_list ) ) - return( NULL ); + if (cl_is_qlist_empty(&p_pool->free_list)) + return (NULL); } - p_list_item = cl_qlist_remove_head( &p_pool->free_list ); + p_list_item = cl_qlist_remove_head(&p_pool->free_list); /* OK, at this point we have an object */ - CL_ASSERT( p_list_item != cl_qlist_end( &p_pool->free_list ) ); - return( (cl_pool_item_t*)p_list_item ); + CL_ASSERT(p_list_item != cl_qlist_end(&p_pool->free_list)); + return ((cl_pool_item_t *) p_list_item); } -cl_pool_item_t* -cl_qcpool_get_tail( - IN cl_qcpool_t* const p_pool ) +cl_pool_item_t *cl_qcpool_get_tail(IN cl_qcpool_t * const p_pool) { - cl_list_item_t *p_list_item; + cl_list_item_t *p_list_item; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->state == CL_INITIALIZED ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->state == CL_INITIALIZED); - if( cl_is_qlist_empty( &p_pool->free_list ) ) - { + if (cl_is_qlist_empty(&p_pool->free_list)) { /* * No object is available. * Return NULL if the user does not want automatic growth. */ - if( !p_pool->grow_size ) - return( NULL ); + if (!p_pool->grow_size) + return (NULL); /* We ran out of elements. Get more */ - cl_qcpool_grow( p_pool, p_pool->grow_size ); + cl_qcpool_grow(p_pool, p_pool->grow_size); /* * We may not have gotten everything we wanted but we might have * gotten something. */ - if( cl_is_qlist_empty( &p_pool->free_list ) ) - return( NULL ); + if (cl_is_qlist_empty(&p_pool->free_list)) + return (NULL); } - p_list_item = cl_qlist_remove_tail( &p_pool->free_list ); + p_list_item = cl_qlist_remove_tail(&p_pool->free_list); /* OK, at this point we have an object */ - CL_ASSERT( p_list_item != cl_qlist_end( &p_pool->free_list ) ); - return( (cl_pool_item_t*)p_list_item ); + CL_ASSERT(p_list_item != cl_qlist_end(&p_pool->free_list)); + return ((cl_pool_item_t *) p_list_item); } /* @@ -406,22 +388,21 @@ cl_qcpool_get_tail( * initializer callback. */ static cl_status_t -__cl_qpool_init_cb( - IN void** const p_comp_array, - IN const uint32_t num_components, - IN void* const context, - OUT cl_pool_item_t** const pp_pool_item ) +__cl_qpool_init_cb(IN void **const p_comp_array, + IN const uint32_t num_components, + IN void *const context, + OUT cl_pool_item_t ** const pp_pool_item) { - cl_qpool_t *p_pool = (cl_qpool_t*)context; + cl_qpool_t *p_pool = (cl_qpool_t *) context; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->pfn_init ); - CL_ASSERT( num_components == 1 ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->pfn_init); + CL_ASSERT(num_components == 1); - UNUSED_PARAM( num_components ); + UNUSED_PARAM(num_components); - return( p_pool->pfn_init( p_comp_array[0], (void*)p_pool->context, - pp_pool_item ) ); + return (p_pool->pfn_init(p_comp_array[0], (void *)p_pool->context, + pp_pool_item)); } /* @@ -429,51 +410,49 @@ __cl_qpool_init_cb( * destructor callback. */ static void -__cl_qpool_dtor_cb( - IN const cl_pool_item_t* const p_pool_item, - IN void* const context ) +__cl_qpool_dtor_cb(IN const cl_pool_item_t * const p_pool_item, + IN void *const context) { - cl_qpool_t *p_pool = (cl_qpool_t*)context; + cl_qpool_t *p_pool = (cl_qpool_t *) context; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->pfn_dtor ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->pfn_dtor); - p_pool->pfn_dtor( p_pool_item, (void*)p_pool->context ); + p_pool->pfn_dtor(p_pool_item, (void *)p_pool->context); } -void -cl_qpool_construct( - IN cl_qpool_t* const p_pool ) +void cl_qpool_construct(IN cl_qpool_t * const p_pool) { - memset( p_pool, 0, sizeof(cl_qpool_t) ); + memset(p_pool, 0, sizeof(cl_qpool_t)); - cl_qcpool_construct( &p_pool->qcpool ); + cl_qcpool_construct(&p_pool->qcpool); } cl_status_t -cl_qpool_init( - IN cl_qpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t object_size, - IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ) +cl_qpool_init(IN cl_qpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t object_size, + IN cl_pfn_qpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_qpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context) { cl_status_t status; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); p_pool->pfn_init = pfn_initializer; /* may be NULL */ p_pool->pfn_dtor = pfn_destructor; /* may be NULL */ p_pool->context = context; - status = cl_qcpool_init( &p_pool->qcpool, min_size, max_size, grow_size, - &object_size, 1, pfn_initializer ? __cl_qpool_init_cb : NULL, - pfn_destructor ? __cl_qpool_dtor_cb : NULL, p_pool ); + status = cl_qcpool_init(&p_pool->qcpool, min_size, max_size, grow_size, + &object_size, 1, + pfn_initializer ? __cl_qpool_init_cb : NULL, + pfn_destructor ? __cl_qpool_dtor_cb : NULL, + p_pool); - return( status ); + return (status); } /* @@ -485,28 +464,27 @@ cl_qpool_init( * initializer callback. */ static cl_status_t -__cl_cpool_init_cb( - IN void** const p_comp_array, - IN const uint32_t num_components, - IN void* const context, - OUT cl_pool_item_t** const pp_pool_item ) +__cl_cpool_init_cb(IN void **const p_comp_array, + IN const uint32_t num_components, + IN void *const context, + OUT cl_pool_item_t ** const pp_pool_item) { - cl_cpool_t *p_pool = (cl_cpool_t*)context; - cl_pool_obj_t *p_pool_obj; - cl_status_t status = CL_SUCCESS; + cl_cpool_t *p_pool = (cl_cpool_t *) context; + cl_pool_obj_t *p_pool_obj; + cl_status_t status = CL_SUCCESS; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); /* * Set our pointer to the list item, which is stored at the beginning of * the first component. */ - p_pool_obj = (cl_pool_obj_t*)p_comp_array[0]; + p_pool_obj = (cl_pool_obj_t *) p_comp_array[0]; /* Set the pool item pointer for the caller. */ *pp_pool_item = &p_pool_obj->pool_item; /* Calculate the pointer to the user's first component. */ - p_comp_array[0] = ((uint8_t*)p_comp_array[0]) + sizeof(cl_pool_obj_t); + p_comp_array[0] = ((uint8_t *) p_comp_array[0]) + sizeof(cl_pool_obj_t); /* * Set the object pointer in the pool object to point to the first of the @@ -515,13 +493,12 @@ __cl_cpool_init_cb( p_pool_obj->p_object = p_comp_array[0]; /* Invoke the user's constructor callback. */ - if( p_pool->pfn_init ) - { - status = p_pool->pfn_init( p_comp_array, num_components, - (void*)p_pool->context ); + if (p_pool->pfn_init) { + status = p_pool->pfn_init(p_comp_array, num_components, + (void *)p_pool->context); } - return( status ); + return (status); } /* @@ -529,49 +506,45 @@ __cl_cpool_init_cb( * destructor callback. */ static void -__cl_cpool_dtor_cb( - IN const cl_pool_item_t* const p_pool_item, - IN void* const context ) +__cl_cpool_dtor_cb(IN const cl_pool_item_t * const p_pool_item, + IN void *const context) { - cl_cpool_t *p_pool = (cl_cpool_t*)context; + cl_cpool_t *p_pool = (cl_cpool_t *) context; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->pfn_dtor ); - CL_ASSERT( ((cl_pool_obj_t*)p_pool_item)->p_object ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->pfn_dtor); + CL_ASSERT(((cl_pool_obj_t *) p_pool_item)->p_object); /* Invoke the user's destructor callback. */ - p_pool->pfn_dtor( (void*)((cl_pool_obj_t*)p_pool_item)->p_object, - (void*)p_pool->context ); + p_pool->pfn_dtor((void *)((cl_pool_obj_t *) p_pool_item)->p_object, + (void *)p_pool->context); } -void -cl_cpool_construct( - IN cl_cpool_t* const p_pool ) +void cl_cpool_construct(IN cl_cpool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - memset( p_pool, 0, sizeof(cl_cpool_t) ); + memset(p_pool, 0, sizeof(cl_cpool_t)); - cl_qcpool_construct( &p_pool->qcpool ); + cl_qcpool_construct(&p_pool->qcpool); } cl_status_t -cl_cpool_init( - IN cl_cpool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN size_t* const component_sizes, - IN const uint32_t num_components, - IN cl_pfn_cpool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_cpool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ) +cl_cpool_init(IN cl_cpool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN size_t * const component_sizes, + IN const uint32_t num_components, + IN cl_pfn_cpool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_cpool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context) { - cl_status_t status; + cl_status_t status; - CL_ASSERT( p_pool ); - CL_ASSERT( num_components ); - CL_ASSERT( component_sizes ); + CL_ASSERT(p_pool); + CL_ASSERT(num_components); + CL_ASSERT(component_sizes); /* Add the size of the pool object to the first component. */ component_sizes[0] += sizeof(cl_pool_obj_t); @@ -581,15 +554,16 @@ cl_cpool_init( p_pool->pfn_dtor = pfn_destructor; /* may be NULL */ p_pool->context = context; - status = cl_qcpool_init( &p_pool->qcpool, min_size, max_size, grow_size, - component_sizes, num_components, __cl_cpool_init_cb, - pfn_destructor ? __cl_cpool_dtor_cb : NULL, - p_pool ); + status = cl_qcpool_init(&p_pool->qcpool, min_size, max_size, grow_size, + component_sizes, num_components, + __cl_cpool_init_cb, + pfn_destructor ? __cl_cpool_dtor_cb : NULL, + p_pool); /* Restore the original value of the first component. */ component_sizes[0] -= sizeof(cl_pool_obj_t); - return( status ); + return (status); } /* @@ -600,31 +574,30 @@ cl_cpool_init( * Callback to translate quick composite to grow pool constructor callback. */ static cl_status_t -__cl_pool_init_cb( - IN void** const pp_obj, - IN const uint32_t count, - IN void* const context, - OUT cl_pool_item_t** const pp_pool_item ) +__cl_pool_init_cb(IN void **const pp_obj, + IN const uint32_t count, + IN void *const context, + OUT cl_pool_item_t ** const pp_pool_item) { - cl_pool_t *p_pool = (cl_pool_t*)context; - cl_pool_obj_t *p_pool_obj; - cl_status_t status = CL_SUCCESS; + cl_pool_t *p_pool = (cl_pool_t *) context; + cl_pool_obj_t *p_pool_obj; + cl_status_t status = CL_SUCCESS; - CL_ASSERT( p_pool ); - CL_ASSERT( pp_obj ); - CL_ASSERT( count == 1 ); + CL_ASSERT(p_pool); + CL_ASSERT(pp_obj); + CL_ASSERT(count == 1); - UNUSED_PARAM( count ); + UNUSED_PARAM(count); /* * Set our pointer to the list item, which is stored at the beginning of * the first component. */ - p_pool_obj = (cl_pool_obj_t*)*pp_obj; + p_pool_obj = (cl_pool_obj_t *) * pp_obj; *pp_pool_item = &p_pool_obj->pool_item; /* Calculate the pointer to the user's first component. */ - *pp_obj = ((uint8_t*)*pp_obj) + sizeof(cl_pool_obj_t); + *pp_obj = ((uint8_t *) * pp_obj) + sizeof(cl_pool_obj_t); /* * Set the object pointer in the pool item to point to the first of the @@ -633,57 +606,53 @@ __cl_pool_init_cb( p_pool_obj->p_object = *pp_obj; /* Invoke the user's constructor callback. */ - if( p_pool->pfn_init ) - status = p_pool->pfn_init( *pp_obj, (void*)p_pool->context ); + if (p_pool->pfn_init) + status = p_pool->pfn_init(*pp_obj, (void *)p_pool->context); - return( status ); + return (status); } /* * Callback to translate quick composite to grow pool destructor callback. */ static void -__cl_pool_dtor_cb( - IN const cl_pool_item_t* const p_pool_item, - IN void* const context ) +__cl_pool_dtor_cb(IN const cl_pool_item_t * const p_pool_item, + IN void *const context) { - cl_pool_t *p_pool = (cl_pool_t*)context; + cl_pool_t *p_pool = (cl_pool_t *) context; - CL_ASSERT( p_pool ); - CL_ASSERT( p_pool->pfn_dtor ); - CL_ASSERT( ((cl_pool_obj_t*)p_pool_item)->p_object ); + CL_ASSERT(p_pool); + CL_ASSERT(p_pool->pfn_dtor); + CL_ASSERT(((cl_pool_obj_t *) p_pool_item)->p_object); /* Invoke the user's destructor callback. */ - p_pool->pfn_dtor( (void*)((cl_pool_obj_t*)p_pool_item)->p_object, - (void*)p_pool->context ); + p_pool->pfn_dtor((void *)((cl_pool_obj_t *) p_pool_item)->p_object, + (void *)p_pool->context); } -void -cl_pool_construct( - IN cl_pool_t* const p_pool ) +void cl_pool_construct(IN cl_pool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - memset( p_pool, 0, sizeof(cl_pool_t) ); + memset(p_pool, 0, sizeof(cl_pool_t)); - cl_qcpool_construct( &p_pool->qcpool ); + cl_qcpool_construct(&p_pool->qcpool); } cl_status_t -cl_pool_init( - IN cl_pool_t* const p_pool, - IN const size_t min_size, - IN const size_t max_size, - IN const size_t grow_size, - IN const size_t object_size, - IN cl_pfn_pool_init_t pfn_initializer OPTIONAL, - IN cl_pfn_pool_dtor_t pfn_destructor OPTIONAL, - IN const void* const context ) +cl_pool_init(IN cl_pool_t * const p_pool, + IN const size_t min_size, + IN const size_t max_size, + IN const size_t grow_size, + IN const size_t object_size, + IN cl_pfn_pool_init_t pfn_initializer OPTIONAL, + IN cl_pfn_pool_dtor_t pfn_destructor OPTIONAL, + IN const void *const context) { - cl_status_t status; - size_t total_size; + cl_status_t status; + size_t total_size; - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); /* Add the size of the list item to the first component. */ total_size = object_size + sizeof(cl_pool_obj_t); @@ -697,9 +666,10 @@ cl_pool_init( * We need an initializer in all cases for quick composite pool, since * the user pointer must be manipulated to hide the prefixed cl_pool_obj_t. */ - status = cl_qcpool_init( &p_pool->qcpool, min_size, max_size, grow_size, - &total_size, 1, __cl_pool_init_cb, - pfn_destructor ? __cl_pool_dtor_cb : NULL, p_pool ); + status = cl_qcpool_init(&p_pool->qcpool, min_size, max_size, grow_size, + &total_size, 1, __cl_pool_init_cb, + pfn_destructor ? __cl_pool_dtor_cb : NULL, + p_pool); - return( status ); + return (status); } diff --git a/opensm/complib/cl_ptr_vector.c b/opensm/complib/cl_ptr_vector.c index 1ee7bfb..333cf0f 100644 --- a/opensm/complib/cl_ptr_vector.c +++ b/opensm/complib/cl_ptr_vector.c @@ -45,34 +45,30 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include #include -void -cl_ptr_vector_construct( - IN cl_ptr_vector_t* const p_vector ) +void cl_ptr_vector_construct(IN cl_ptr_vector_t * const p_vector) { - CL_ASSERT( p_vector ); + CL_ASSERT(p_vector); - memset( p_vector, 0, sizeof(cl_ptr_vector_t) ); + memset(p_vector, 0, sizeof(cl_ptr_vector_t)); p_vector->state = CL_UNINITIALIZED; } cl_status_t -cl_ptr_vector_init( - IN cl_ptr_vector_t* const p_vector, - IN const size_t min_size, - IN const size_t grow_size ) +cl_ptr_vector_init(IN cl_ptr_vector_t * const p_vector, + IN const size_t min_size, IN const size_t grow_size) { - cl_status_t status = CL_SUCCESS; + cl_status_t status = CL_SUCCESS; - CL_ASSERT( p_vector ); + CL_ASSERT(p_vector); - cl_ptr_vector_construct( p_vector ); + cl_ptr_vector_construct(p_vector); p_vector->grow_size = grow_size; @@ -83,30 +79,25 @@ cl_ptr_vector_init( p_vector->state = CL_INITIALIZED; /* get the storage needed by the user */ - if( min_size ) - { - status = cl_ptr_vector_set_size( p_vector, min_size ); - if( status != CL_SUCCESS ) - cl_ptr_vector_destroy( p_vector ); + if (min_size) { + status = cl_ptr_vector_set_size(p_vector, min_size); + if (status != CL_SUCCESS) + cl_ptr_vector_destroy(p_vector); } - return( status ); + return (status); } -void -cl_ptr_vector_destroy( - IN cl_ptr_vector_t* const p_vector ) +void cl_ptr_vector_destroy(IN cl_ptr_vector_t * const p_vector) { - CL_ASSERT( p_vector ); - CL_ASSERT( cl_is_state_valid( p_vector->state ) ); + CL_ASSERT(p_vector); + CL_ASSERT(cl_is_state_valid(p_vector->state)); /* Call the user's destructor for each element in the array. */ - if( p_vector->state == CL_INITIALIZED ) - { + if (p_vector->state == CL_INITIALIZED) { /* Destroy the page vector. */ - if( p_vector->p_ptr_array ) - { - free( (void*)p_vector->p_ptr_array ); + if (p_vector->p_ptr_array) { + free((void *)p_vector->p_ptr_array); p_vector->p_ptr_array = NULL; } } @@ -115,108 +106,98 @@ cl_ptr_vector_destroy( } cl_status_t -cl_ptr_vector_at( - IN const cl_ptr_vector_t* const p_vector, - IN const size_t index, - OUT void** const p_element ) +cl_ptr_vector_at(IN const cl_ptr_vector_t * const p_vector, + IN const size_t index, OUT void **const p_element) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Range check */ - if( index >= p_vector->size ) - return( CL_INVALID_PARAMETER ); + if (index >= p_vector->size) + return (CL_INVALID_PARAMETER); - *p_element = cl_ptr_vector_get( p_vector, index ); - return( CL_SUCCESS ); + *p_element = cl_ptr_vector_get(p_vector, index); + return (CL_SUCCESS); } cl_status_t -cl_ptr_vector_set( - IN cl_ptr_vector_t* const p_vector, - IN const size_t index, - IN const void* const element ) +cl_ptr_vector_set(IN cl_ptr_vector_t * const p_vector, + IN const size_t index, IN const void *const element) { - cl_status_t status; + cl_status_t status; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Determine if the vector has room for this element. */ - if( index >= p_vector->size ) - { + if (index >= p_vector->size) { /* Resize to accomodate the given index. */ - status = cl_ptr_vector_set_size( p_vector, index + 1 ); + status = cl_ptr_vector_set_size(p_vector, index + 1); /* Check for failure on or before the given index. */ - if( (status != CL_SUCCESS) && (p_vector->size < index) ) - return( status ); + if ((status != CL_SUCCESS) && (p_vector->size < index)) + return (status); } /* At this point, the array is guaranteed to be big enough */ p_vector->p_ptr_array[index] = element; - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void* -cl_ptr_vector_remove( - IN cl_ptr_vector_t* const p_vector, - IN const size_t index ) +void *cl_ptr_vector_remove(IN cl_ptr_vector_t * const p_vector, + IN const size_t index) { - size_t src; - const void *element; + size_t src; + const void *element; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( p_vector->size > index ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(p_vector->size > index); /* Store a copy of the element to return. */ element = p_vector->p_ptr_array[index]; /* Shift all items above the removed item down. */ - if( index < --p_vector->size ) - { - for( src = index; src < p_vector->size; src++ ) - p_vector->p_ptr_array[src] = p_vector->p_ptr_array[src + 1]; + if (index < --p_vector->size) { + for (src = index; src < p_vector->size; src++) + p_vector->p_ptr_array[src] = + p_vector->p_ptr_array[src + 1]; } /* Clear the entry for the element just outside of the new upper bound. */ p_vector->p_ptr_array[p_vector->size] = NULL; - return( (void*)element ); + return ((void *)element); } cl_status_t -cl_ptr_vector_set_capacity( - IN cl_ptr_vector_t* const p_vector, - IN const size_t new_capacity ) +cl_ptr_vector_set_capacity(IN cl_ptr_vector_t * const p_vector, + IN const size_t new_capacity) { - void *p_new_ptr_array; + void *p_new_ptr_array; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Do we have to do anything here? */ - if( new_capacity <= p_vector->capacity ) - { + if (new_capacity <= p_vector->capacity) { /* Nope */ - return( CL_SUCCESS ); + return (CL_SUCCESS); } /* Allocate our pointer array. */ - p_new_ptr_array = malloc( new_capacity * sizeof(void*) ); - if( !p_new_ptr_array ) - return( CL_INSUFFICIENT_MEMORY ); + p_new_ptr_array = malloc(new_capacity * sizeof(void *)); + if (!p_new_ptr_array) + return (CL_INSUFFICIENT_MEMORY); else - memset( p_new_ptr_array, 0, new_capacity * sizeof(void*) ); + memset(p_new_ptr_array, 0, new_capacity * sizeof(void *)); - if( p_vector->p_ptr_array ) - { + if (p_vector->p_ptr_array) { /* Copy the old pointer array into the new. */ - memcpy( p_new_ptr_array, p_vector->p_ptr_array, - p_vector->capacity * sizeof(void*) ); + memcpy(p_new_ptr_array, p_vector->p_ptr_array, + p_vector->capacity * sizeof(void *)); /* Free the old pointer array. */ - free( (void*)p_vector->p_ptr_array ); + free((void *)p_vector->p_ptr_array); } /* Set the new array. */ @@ -225,130 +206,118 @@ cl_ptr_vector_set_capacity( /* Update the vector with the new capactity. */ p_vector->capacity = new_capacity; - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_ptr_vector_set_size( - IN cl_ptr_vector_t* const p_vector, - IN const size_t size ) +cl_ptr_vector_set_size(IN cl_ptr_vector_t * const p_vector, + IN const size_t size) { - cl_status_t status; - size_t new_capacity; + cl_status_t status; + size_t new_capacity; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Check to see if the requested size is the same as the existing size. */ - if( size == p_vector->size ) - return( CL_SUCCESS ); + if (size == p_vector->size) + return (CL_SUCCESS); /* Determine if the vector has room for this element. */ - if( size >= p_vector->capacity ) - { - if( !p_vector->grow_size ) - return( CL_INSUFFICIENT_MEMORY ); + if (size >= p_vector->capacity) { + if (!p_vector->grow_size) + return (CL_INSUFFICIENT_MEMORY); /* Calculate the new capacity, taking into account the grow size. */ new_capacity = size; - if( size % p_vector->grow_size ) - { + if (size % p_vector->grow_size) { /* Round up to nearest grow_size boundary. */ new_capacity += p_vector->grow_size - - (size % p_vector->grow_size); + (size % p_vector->grow_size); } - status = cl_ptr_vector_set_capacity( p_vector, new_capacity ); - if( status != CL_SUCCESS ) - return( status ); + status = cl_ptr_vector_set_capacity(p_vector, new_capacity); + if (status != CL_SUCCESS) + return (status); } p_vector->size = size; - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_ptr_vector_set_min_size( - IN cl_ptr_vector_t* const p_vector, - IN const size_t min_size ) +cl_ptr_vector_set_min_size(IN cl_ptr_vector_t * const p_vector, + IN const size_t min_size) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - if( min_size > p_vector->size ) - { + if (min_size > p_vector->size) { /* We have to resize the array */ - return( cl_ptr_vector_set_size( p_vector, min_size ) ); + return (cl_ptr_vector_set_size(p_vector, min_size)); } /* We didn't have to do anything */ - return( CL_SUCCESS ); + return (CL_SUCCESS); } void -cl_ptr_vector_apply_func( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_apply_t pfn_callback, - IN const void* const context ) +cl_ptr_vector_apply_func(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_apply_t pfn_callback, + IN const void *const context) { - size_t i; + size_t i; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); - for( i = 0; i < p_vector->size; i++ ) - pfn_callback( i, (void*)p_vector->p_ptr_array[i], (void*)context ); + for (i = 0; i < p_vector->size; i++) + pfn_callback(i, (void *)p_vector->p_ptr_array[i], + (void *)context); } size_t -cl_ptr_vector_find_from_start( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_find_t pfn_callback, - IN const void* const context ) +cl_ptr_vector_find_from_start(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_find_t pfn_callback, + IN const void *const context) { - size_t i; + size_t i; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); - for( i = 0; i < p_vector->size; i++ ) - { + for (i = 0; i < p_vector->size; i++) { /* Invoke the callback */ - if( pfn_callback( i, (void*)p_vector->p_ptr_array[i], - (void*)context ) == CL_SUCCESS ) - { + if (pfn_callback(i, (void *)p_vector->p_ptr_array[i], + (void *)context) == CL_SUCCESS) { break; } } - return( i ); + return (i); } size_t -cl_ptr_vector_find_from_end( - IN const cl_ptr_vector_t* const p_vector, - IN cl_pfn_ptr_vec_find_t pfn_callback, - IN const void* const context ) +cl_ptr_vector_find_from_end(IN const cl_ptr_vector_t * const p_vector, + IN cl_pfn_ptr_vec_find_t pfn_callback, + IN const void *const context) { - size_t i; + size_t i; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); i = p_vector->size; - while( i ) - { + while (i) { /* Invoke the callback for the current element. */ - if( pfn_callback( i, (void*)p_vector->p_ptr_array[--i], - (void*)context ) == CL_SUCCESS ) - { - return( i ); + if (pfn_callback(i, (void *)p_vector->p_ptr_array[--i], + (void *)context) == CL_SUCCESS) { + return (i); } } - return( p_vector->size ); + return (p_vector->size); } - diff --git a/opensm/complib/cl_spinlock.c b/opensm/complib/cl_spinlock.c index b453f52..2d81696 100644 --- a/opensm/complib/cl_spinlock.c +++ b/opensm/complib/cl_spinlock.c @@ -35,68 +35,57 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include -void -cl_spinlock_construct( - IN cl_spinlock_t* const p_spinlock ) +void cl_spinlock_construct(IN cl_spinlock_t * const p_spinlock) { - CL_ASSERT( p_spinlock ); + CL_ASSERT(p_spinlock); p_spinlock->state = CL_UNINITIALIZED; } -cl_status_t -cl_spinlock_init( - IN cl_spinlock_t* const p_spinlock ) +cl_status_t cl_spinlock_init(IN cl_spinlock_t * const p_spinlock) { - CL_ASSERT( p_spinlock ); + CL_ASSERT(p_spinlock); - cl_spinlock_construct( p_spinlock ); + cl_spinlock_construct(p_spinlock); /* Initialize with pthread_mutexattr_t = NULL */ - if( pthread_mutex_init( &p_spinlock->mutex, NULL ) ) - return( CL_ERROR ); + if (pthread_mutex_init(&p_spinlock->mutex, NULL)) + return (CL_ERROR); p_spinlock->state = CL_INITIALIZED; - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -cl_spinlock_destroy( - IN cl_spinlock_t* const p_spinlock ) +void cl_spinlock_destroy(IN cl_spinlock_t * const p_spinlock) { - CL_ASSERT( p_spinlock ); - CL_ASSERT( cl_is_state_valid( p_spinlock->state ) ); + CL_ASSERT(p_spinlock); + CL_ASSERT(cl_is_state_valid(p_spinlock->state)); - if( p_spinlock->state == CL_INITIALIZED ) - { - p_spinlock->state = CL_UNINITIALIZED; - pthread_mutex_lock( &p_spinlock->mutex ); - pthread_mutex_unlock( &p_spinlock->mutex ); - pthread_mutex_destroy( &p_spinlock->mutex ); - } - p_spinlock->state = CL_UNINITIALIZED; + if (p_spinlock->state == CL_INITIALIZED) { + p_spinlock->state = CL_UNINITIALIZED; + pthread_mutex_lock(&p_spinlock->mutex); + pthread_mutex_unlock(&p_spinlock->mutex); + pthread_mutex_destroy(&p_spinlock->mutex); + } + p_spinlock->state = CL_UNINITIALIZED; } -void -cl_spinlock_acquire( - IN cl_spinlock_t* const p_spinlock ) +void cl_spinlock_acquire(IN cl_spinlock_t * const p_spinlock) { - CL_ASSERT( p_spinlock ); - CL_ASSERT( p_spinlock->state == CL_INITIALIZED ); + CL_ASSERT(p_spinlock); + CL_ASSERT(p_spinlock->state == CL_INITIALIZED); - pthread_mutex_lock( &p_spinlock->mutex ); + pthread_mutex_lock(&p_spinlock->mutex); } -void -cl_spinlock_release( - IN cl_spinlock_t* const p_spinlock ) +void cl_spinlock_release(IN cl_spinlock_t * const p_spinlock) { - CL_ASSERT( p_spinlock ); - CL_ASSERT( p_spinlock->state == CL_INITIALIZED ); + CL_ASSERT(p_spinlock); + CL_ASSERT(p_spinlock->state == CL_INITIALIZED); - pthread_mutex_unlock( &p_spinlock->mutex ); + pthread_mutex_unlock(&p_spinlock->mutex); } diff --git a/opensm/complib/cl_statustext.c b/opensm/complib/cl_statustext.c index 1d68db3..0a7d32b 100644 --- a/opensm/complib/cl_statustext.c +++ b/opensm/complib/cl_statustext.c @@ -45,13 +45,12 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include /* Status values above converted to text for easier printing. */ -const char* cl_status_text[] = -{ +const char *cl_status_text[] = { "CL_SUCCESS", "CL_ERROR", "CL_INVALID_STATE", diff --git a/opensm/complib/cl_thread.c b/opensm/complib/cl_thread.c index 0e5ef3c..004b118 100644 --- a/opensm/complib/cl_thread.c +++ b/opensm/complib/cl_thread.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -47,109 +47,95 @@ * This function is always run as a result of creation a new user mode thread. * Its main job is to synchronize the creation and running of the new thread. */ -void* -__cl_thread_wrapper( void* arg ) +void *__cl_thread_wrapper(void *arg) { - cl_thread_t *p_thread = (cl_thread_t*)arg; + cl_thread_t *p_thread = (cl_thread_t *) arg; - CL_ASSERT( p_thread ); - CL_ASSERT( p_thread->pfn_callback ); + CL_ASSERT(p_thread); + CL_ASSERT(p_thread->pfn_callback); - p_thread->pfn_callback( (void*)p_thread->context ); + p_thread->pfn_callback((void *)p_thread->context); - return( NULL ); + return (NULL); } -void -cl_thread_construct( - IN cl_thread_t* const p_thread ) +void cl_thread_construct(IN cl_thread_t * const p_thread) { - CL_ASSERT( p_thread ); + CL_ASSERT(p_thread); p_thread->osd.state = CL_UNINITIALIZED; } cl_status_t -cl_thread_init( - IN cl_thread_t* const p_thread, - IN cl_pfn_thread_callback_t pfn_callback, - IN const void* const context, - IN const char* const name ) +cl_thread_init(IN cl_thread_t * const p_thread, + IN cl_pfn_thread_callback_t pfn_callback, + IN const void *const context, IN const char *const name) { - int ret; + int ret; - CL_ASSERT( p_thread ); + CL_ASSERT(p_thread); - cl_thread_construct( p_thread ); + cl_thread_construct(p_thread); /* Initialize the thread structure */ p_thread->pfn_callback = pfn_callback; p_thread->context = context; - ret = pthread_create( &p_thread->osd.id, NULL, - __cl_thread_wrapper, (void*)p_thread ); + ret = pthread_create(&p_thread->osd.id, NULL, + __cl_thread_wrapper, (void *)p_thread); - if( ret != 0 ) /* pthread_create returns a "0" for success */ - return( CL_ERROR ); + if (ret != 0) /* pthread_create returns a "0" for success */ + return (CL_ERROR); p_thread->osd.state = CL_INITIALIZED; - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -cl_thread_destroy( - IN cl_thread_t* const p_thread ) +void cl_thread_destroy(IN cl_thread_t * const p_thread) { - CL_ASSERT( p_thread ); - CL_ASSERT( cl_is_state_valid( p_thread->osd.state ) ); + CL_ASSERT(p_thread); + CL_ASSERT(cl_is_state_valid(p_thread->osd.state)); - if( p_thread->osd.state == CL_INITIALIZED ) - pthread_join( p_thread->osd.id, NULL ); + if (p_thread->osd.state == CL_INITIALIZED) + pthread_join(p_thread->osd.id, NULL); p_thread->osd.state = CL_UNINITIALIZED; } -void -cl_thread_suspend( - IN const uint32_t pause_ms ) +void cl_thread_suspend(IN const uint32_t pause_ms) { /* Convert to micro seconds */ - usleep( pause_ms * 1000 ); + usleep(pause_ms * 1000); } -void -cl_thread_stall( - IN const uint32_t pause_us ) +void cl_thread_stall(IN const uint32_t pause_us) { /* * Not quite a busy wait, but Linux is lacking in terms of high * resolution time stamp information in user mode. */ - usleep( pause_us ); + usleep(pause_us); } -int -cl_proc_count( void ) +int cl_proc_count(void) { uint32_t ret; ret = get_nprocs(); - if( !ret) - return 1;/* Workaround for PPC where get_nprocs() returns 0 */ + if (!ret) + return 1; /* Workaround for PPC where get_nprocs() returns 0 */ return ret; } -boolean_t -cl_is_current_thread( - IN const cl_thread_t* const p_thread ) +boolean_t cl_is_current_thread(IN const cl_thread_t * const p_thread) { - pthread_t current; + pthread_t current; - CL_ASSERT( p_thread ); - CL_ASSERT( p_thread->osd.state == CL_INITIALIZED ); + CL_ASSERT(p_thread); + CL_ASSERT(p_thread->osd.state == CL_INITIALIZED); current = pthread_self(); - return( pthread_equal( current, p_thread->osd.id ) ); + return (pthread_equal(current, p_thread->osd.id)); } diff --git a/opensm/complib/cl_threadpool.c b/opensm/complib/cl_threadpool.c index 6fe05b6..34fe322 100644 --- a/opensm/complib/cl_threadpool.c +++ b/opensm/complib/cl_threadpool.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -54,44 +54,42 @@ static void cleanup_mutex(void *arg) { - pthread_mutex_unlock(&((cl_thread_pool_t *)arg)->mutex); + pthread_mutex_unlock(&((cl_thread_pool_t *) arg)->mutex); } -static void *thread_pool_routine(void* context) +static void *thread_pool_routine(void *context) { - cl_thread_pool_t *p_thread_pool = (cl_thread_pool_t*)context; + cl_thread_pool_t *p_thread_pool = (cl_thread_pool_t *) context; do { pthread_mutex_lock(&p_thread_pool->mutex); pthread_cleanup_push(cleanup_mutex, p_thread_pool); - while(!p_thread_pool->events) + while (!p_thread_pool->events) pthread_cond_wait(&p_thread_pool->cond, &p_thread_pool->mutex); p_thread_pool->events--; pthread_cleanup_pop(1); /* The event has been signalled. Invoke the callback. */ - (*p_thread_pool->pfn_callback)(p_thread_pool->context); + (*p_thread_pool->pfn_callback) (p_thread_pool->context); } while (1); return NULL; } cl_status_t -cl_thread_pool_init( - IN cl_thread_pool_t* const p_thread_pool, - IN unsigned count, - IN void (*pfn_callback)(void*), - IN void *context, - IN const char* const name ) +cl_thread_pool_init(IN cl_thread_pool_t * const p_thread_pool, + IN unsigned count, + IN void (*pfn_callback) (void *), + IN void *context, IN const char *const name) { int i; - CL_ASSERT( p_thread_pool ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_thread_pool); + CL_ASSERT(pfn_callback); memset(p_thread_pool, 0, sizeof(*p_thread_pool)); - if(!count) + if (!count) count = cl_proc_count(); pthread_mutex_init(&p_thread_pool->mutex, NULL); @@ -104,37 +102,34 @@ cl_thread_pool_init( p_thread_pool->tid = calloc(count, sizeof(*p_thread_pool->tid)); if (!p_thread_pool->tid) { - cl_thread_pool_destroy( p_thread_pool ); + cl_thread_pool_destroy(p_thread_pool); return CL_INSUFFICIENT_MEMORY; } p_thread_pool->running_count = count; - for( i = 0; i < count; i++ ) - { + for (i = 0; i < count; i++) { if (pthread_create(&p_thread_pool->tid[i], NULL, thread_pool_routine, p_thread_pool) < 0) { - cl_thread_pool_destroy( p_thread_pool ); + cl_thread_pool_destroy(p_thread_pool); return CL_INSUFFICIENT_RESOURCES; } } - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -cl_thread_pool_destroy( - IN cl_thread_pool_t* const p_thread_pool ) +void cl_thread_pool_destroy(IN cl_thread_pool_t * const p_thread_pool) { int i; - CL_ASSERT( p_thread_pool ); + CL_ASSERT(p_thread_pool); - for (i = 0 ; i < p_thread_pool->running_count; i++) + for (i = 0; i < p_thread_pool->running_count; i++) if (p_thread_pool->tid[i]) pthread_cancel(p_thread_pool->tid[i]); - for (i = 0 ; i < p_thread_pool->running_count; i++) + for (i = 0; i < p_thread_pool->running_count; i++) if (p_thread_pool->tid[i]) pthread_join(p_thread_pool->tid[i], NULL); @@ -145,12 +140,10 @@ cl_thread_pool_destroy( p_thread_pool->events = 0; } -cl_status_t -cl_thread_pool_signal( - IN cl_thread_pool_t* const p_thread_pool ) +cl_status_t cl_thread_pool_signal(IN cl_thread_pool_t * const p_thread_pool) { int ret; - CL_ASSERT( p_thread_pool ); + CL_ASSERT(p_thread_pool); pthread_mutex_lock(&p_thread_pool->mutex); p_thread_pool->events++; ret = pthread_cond_signal(&p_thread_pool->cond); diff --git a/opensm/complib/cl_timer.c b/opensm/complib/cl_timer.c index c8e2feb..7393aee 100644 --- a/opensm/complib/cl_timer.c +++ b/opensm/complib/cl_timer.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -55,222 +55,208 @@ #include /* Timer provider (emulates timers in user mode). */ -typedef struct _cl_timer_prov -{ - pthread_t thread; - pthread_mutex_t mutex; - pthread_cond_t cond; - cl_qlist_t queue; +typedef struct _cl_timer_prov { + pthread_t thread; + pthread_mutex_t mutex; + pthread_cond_t cond; + cl_qlist_t queue; - boolean_t exit; + boolean_t exit; } cl_timer_prov_t; /* Global timer provider. */ -static cl_timer_prov_t *gp_timer_prov = NULL; +static cl_timer_prov_t *gp_timer_prov = NULL; -static void* -__cl_timer_prov_cb( - IN void* const context ); +static void *__cl_timer_prov_cb(IN void *const context); /* * Creates the process global timer provider. Must be called by the shared * object framework to solve all serialization issues. */ -cl_status_t -__cl_timer_prov_create( void ) +cl_status_t __cl_timer_prov_create(void) { - CL_ASSERT( gp_timer_prov == NULL ); + CL_ASSERT(gp_timer_prov == NULL); - gp_timer_prov = malloc( sizeof(cl_timer_prov_t) ); - if( !gp_timer_prov ) - return( CL_INSUFFICIENT_MEMORY ); - else - memset( gp_timer_prov, 0, sizeof(cl_timer_prov_t) ); + gp_timer_prov = malloc(sizeof(cl_timer_prov_t)); + if (!gp_timer_prov) + return (CL_INSUFFICIENT_MEMORY); + else + memset(gp_timer_prov, 0, sizeof(cl_timer_prov_t)); - cl_qlist_init( &gp_timer_prov->queue ); + cl_qlist_init(&gp_timer_prov->queue); - pthread_mutex_init( &gp_timer_prov->mutex, NULL ); - pthread_cond_init( &gp_timer_prov->cond, NULL ); + pthread_mutex_init(&gp_timer_prov->mutex, NULL); + pthread_cond_init(&gp_timer_prov->cond, NULL); - if( pthread_create( &gp_timer_prov->thread, NULL, - __cl_timer_prov_cb, NULL ) ) - { - __cl_timer_prov_destroy(); - return( CL_ERROR ); - } + if (pthread_create(&gp_timer_prov->thread, NULL, + __cl_timer_prov_cb, NULL)) { + __cl_timer_prov_destroy(); + return (CL_ERROR); + } - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -__cl_timer_prov_destroy( void ) +void __cl_timer_prov_destroy(void) { - pthread_t tid; + pthread_t tid; - if( !gp_timer_prov ) - return; + if (!gp_timer_prov) + return; - tid = gp_timer_prov->thread; - pthread_mutex_lock( &gp_timer_prov->mutex ); - gp_timer_prov->exit = TRUE; - pthread_cond_broadcast( &gp_timer_prov->cond ); - pthread_mutex_unlock( &gp_timer_prov->mutex ); - pthread_join( tid , NULL ); + tid = gp_timer_prov->thread; + pthread_mutex_lock(&gp_timer_prov->mutex); + gp_timer_prov->exit = TRUE; + pthread_cond_broadcast(&gp_timer_prov->cond); + pthread_mutex_unlock(&gp_timer_prov->mutex); + pthread_join(tid, NULL); - /* Destroy the mutex and condition variable. */ - pthread_mutex_destroy( &gp_timer_prov->mutex ); - pthread_cond_destroy( &gp_timer_prov->cond ); + /* Destroy the mutex and condition variable. */ + pthread_mutex_destroy(&gp_timer_prov->mutex); + pthread_cond_destroy(&gp_timer_prov->cond); - /* Free the memory and reset the global pointer. */ - free( gp_timer_prov ); - gp_timer_prov = NULL; + /* Free the memory and reset the global pointer. */ + free(gp_timer_prov); + gp_timer_prov = NULL; } /* * This is the internal work function executed by the timer's thread. */ -static void* -__cl_timer_prov_cb( - IN void* const context ) +static void *__cl_timer_prov_cb(IN void *const context) { - int ret; - cl_timer_t *p_timer; - - pthread_mutex_lock( &gp_timer_prov->mutex ); - while( !gp_timer_prov->exit ) - { - if( cl_is_qlist_empty( &gp_timer_prov->queue ) ) - { - /* Wait until we exit or a timer is queued. */ - /* cond wait does: - * pthread_cond_wait atomically unlocks the mutex (as per - * pthread_unlock_mutex) and waits for the condition variable - * cond to be signaled. The thread execution is suspended and - * does not consume any CPU time until the condition variable is - * signaled. The mutex must be locked by the calling thread on - * entrance to pthread_cond_wait. Before RETURNING TO THE - * CALLING THREAD, PTHREAD_COND_WAIT RE-ACQUIRES MUTEX (as per - * pthread_lock_mutex). - */ - ret = pthread_cond_wait( &gp_timer_prov->cond, - &gp_timer_prov->mutex ); - } - else - { - /* - * The timer elements are on the queue in expiration order. - * Get the first in the list to determine how long to wait. - */ - - p_timer = (cl_timer_t*)cl_qlist_head( &gp_timer_prov->queue ); - ret = pthread_cond_timedwait( &gp_timer_prov->cond, - &gp_timer_prov->mutex, &p_timer->timeout ); - - /* - Sleep again on every event other than timeout and invalid - Note: EINVAL means that we got behind. This can occur when - we are very busy... - */ - if( ret != ETIMEDOUT && ret != EINVAL ) - continue; - - /* - * The timer expired. Check the state in case it was cancelled - * after it expired but before we got a chance to invoke the - * callback. - */ - if( p_timer->timer_state != CL_TIMER_QUEUED ) - continue; - - /* - * Mark the timer as running to synchronize with its - * cancelation since we can't hold the mutex during the - * callback. - */ - p_timer->timer_state = CL_TIMER_RUNNING; - - /* Remove the item from the timer queue. */ - cl_qlist_remove_item( &gp_timer_prov->queue, - &p_timer->list_item ); - pthread_mutex_unlock( &gp_timer_prov->mutex ); - /* Invoke the callback. */ - p_timer->pfn_callback( (void*)p_timer->context ); - - /* Acquire the mutex again. */ - pthread_mutex_lock( &gp_timer_prov->mutex ); - /* - * Only set the state to idle if the timer has not been accessed - * from the callback - */ - if( p_timer->timer_state == CL_TIMER_RUNNING ) - p_timer->timer_state = CL_TIMER_IDLE; - - /* - * Signal any thread trying to manipulate the timer - * that expired. - */ - pthread_cond_signal( &p_timer->cond ); - } - } - gp_timer_prov->thread = 0; - pthread_mutex_unlock( &gp_timer_prov->mutex ); - pthread_exit( NULL ); + int ret; + cl_timer_t *p_timer; + + pthread_mutex_lock(&gp_timer_prov->mutex); + while (!gp_timer_prov->exit) { + if (cl_is_qlist_empty(&gp_timer_prov->queue)) { + /* Wait until we exit or a timer is queued. */ + /* cond wait does: + * pthread_cond_wait atomically unlocks the mutex (as per + * pthread_unlock_mutex) and waits for the condition variable + * cond to be signaled. The thread execution is suspended and + * does not consume any CPU time until the condition variable is + * signaled. The mutex must be locked by the calling thread on + * entrance to pthread_cond_wait. Before RETURNING TO THE + * CALLING THREAD, PTHREAD_COND_WAIT RE-ACQUIRES MUTEX (as per + * pthread_lock_mutex). + */ + ret = pthread_cond_wait(&gp_timer_prov->cond, + &gp_timer_prov->mutex); + } else { + /* + * The timer elements are on the queue in expiration order. + * Get the first in the list to determine how long to wait. + */ + + p_timer = + (cl_timer_t *) cl_qlist_head(&gp_timer_prov->queue); + ret = + pthread_cond_timedwait(&gp_timer_prov->cond, + &gp_timer_prov->mutex, + &p_timer->timeout); + + /* + Sleep again on every event other than timeout and invalid + Note: EINVAL means that we got behind. This can occur when + we are very busy... + */ + if (ret != ETIMEDOUT && ret != EINVAL) + continue; + + /* + * The timer expired. Check the state in case it was cancelled + * after it expired but before we got a chance to invoke the + * callback. + */ + if (p_timer->timer_state != CL_TIMER_QUEUED) + continue; + + /* + * Mark the timer as running to synchronize with its + * cancelation since we can't hold the mutex during the + * callback. + */ + p_timer->timer_state = CL_TIMER_RUNNING; + + /* Remove the item from the timer queue. */ + cl_qlist_remove_item(&gp_timer_prov->queue, + &p_timer->list_item); + pthread_mutex_unlock(&gp_timer_prov->mutex); + /* Invoke the callback. */ + p_timer->pfn_callback((void *)p_timer->context); + + /* Acquire the mutex again. */ + pthread_mutex_lock(&gp_timer_prov->mutex); + /* + * Only set the state to idle if the timer has not been accessed + * from the callback + */ + if (p_timer->timer_state == CL_TIMER_RUNNING) + p_timer->timer_state = CL_TIMER_IDLE; + + /* + * Signal any thread trying to manipulate the timer + * that expired. + */ + pthread_cond_signal(&p_timer->cond); + } + } + gp_timer_prov->thread = 0; + pthread_mutex_unlock(&gp_timer_prov->mutex); + pthread_exit(NULL); } /* Timer implementation. */ -void -cl_timer_construct( - IN cl_timer_t* const p_timer ) +void cl_timer_construct(IN cl_timer_t * const p_timer) { - memset( p_timer, 0, sizeof(cl_timer_t) ); - p_timer->state = CL_UNINITIALIZED; + memset(p_timer, 0, sizeof(cl_timer_t)); + p_timer->state = CL_UNINITIALIZED; } cl_status_t -cl_timer_init( - IN cl_timer_t* const p_timer, - IN cl_pfn_timer_callback_t pfn_callback, - IN const void* const context ) +cl_timer_init(IN cl_timer_t * const p_timer, + IN cl_pfn_timer_callback_t pfn_callback, + IN const void *const context) { - CL_ASSERT( p_timer ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_timer); + CL_ASSERT(pfn_callback); - cl_timer_construct( p_timer ); + cl_timer_construct(p_timer); - if( !gp_timer_prov ) - return( CL_ERROR ); + if (!gp_timer_prov) + return (CL_ERROR); - /* Store timer parameters. */ - p_timer->pfn_callback = pfn_callback; - p_timer->context = context; + /* Store timer parameters. */ + p_timer->pfn_callback = pfn_callback; + p_timer->context = context; - /* Mark the timer as idle. */ - p_timer->timer_state = CL_TIMER_IDLE; + /* Mark the timer as idle. */ + p_timer->timer_state = CL_TIMER_IDLE; - /* Create the condition variable that is used when cancelling a timer. */ - pthread_cond_init( &p_timer->cond, NULL ); + /* Create the condition variable that is used when cancelling a timer. */ + pthread_cond_init(&p_timer->cond, NULL); - p_timer->state = CL_INITIALIZED; + p_timer->state = CL_INITIALIZED; - return( CL_SUCCESS ); + return (CL_SUCCESS); } -void -cl_timer_destroy( - IN cl_timer_t* const p_timer ) +void cl_timer_destroy(IN cl_timer_t * const p_timer) { - CL_ASSERT( p_timer ); - CL_ASSERT( cl_is_state_valid( p_timer->state ) ); + CL_ASSERT(p_timer); + CL_ASSERT(cl_is_state_valid(p_timer->state)); - if( p_timer->state == CL_INITIALIZED ) - cl_timer_stop( p_timer ); + if (p_timer->state == CL_INITIALIZED) + cl_timer_stop(p_timer); - p_timer->state = CL_UNINITIALIZED; + p_timer->state = CL_UNINITIALIZED; - /* is it possible we have some threads waiting on the cond now? */ - pthread_cond_broadcast( &p_timer->cond ); - pthread_cond_destroy( &p_timer->cond ); + /* is it possible we have some threads waiting on the cond now? */ + pthread_cond_broadcast(&p_timer->cond); + pthread_cond_destroy(&p_timer->cond); } @@ -278,13 +264,12 @@ cl_timer_destroy( * Return TRUE if timeout value 1 is earlier than timeout value 2. */ static __inline boolean_t -__cl_timer_is_earlier( - IN struct timespec *p_timeout1, - IN struct timespec *p_timeout2 ) +__cl_timer_is_earlier(IN struct timespec *p_timeout1, + IN struct timespec *p_timeout2) { - return( (p_timeout1->tv_sec < p_timeout2->tv_sec) || - ( (p_timeout1->tv_sec == p_timeout2->tv_sec) && - (p_timeout1->tv_nsec < p_timeout2->tv_nsec) ) ); + return ((p_timeout1->tv_sec < p_timeout2->tv_sec) || + ((p_timeout1->tv_sec == p_timeout2->tv_sec) && + (p_timeout1->tv_nsec < p_timeout2->tv_nsec))); } /* @@ -293,188 +278,173 @@ __cl_timer_is_earlier( * a cl_timer_t structure with valid timeouts. */ static cl_status_t -__cl_timer_find( - IN const cl_list_item_t* const p_list_item, - IN void* const context ) +__cl_timer_find(IN const cl_list_item_t * const p_list_item, + IN void *const context) { - cl_timer_t *p_in_list; - cl_timer_t *p_new; + cl_timer_t *p_in_list; + cl_timer_t *p_new; - CL_ASSERT( p_list_item ); - CL_ASSERT( context ); + CL_ASSERT(p_list_item); + CL_ASSERT(context); - p_in_list = (cl_timer_t*)p_list_item; - p_new = (cl_timer_t*)context; + p_in_list = (cl_timer_t *) p_list_item; + p_new = (cl_timer_t *) context; - CL_ASSERT( p_in_list->state == CL_INITIALIZED ); - CL_ASSERT( p_new->state == CL_INITIALIZED ); + CL_ASSERT(p_in_list->state == CL_INITIALIZED); + CL_ASSERT(p_new->state == CL_INITIALIZED); - CL_ASSERT( p_in_list->timer_state == CL_TIMER_QUEUED ); + CL_ASSERT(p_in_list->timer_state == CL_TIMER_QUEUED); - if( __cl_timer_is_earlier( &p_in_list->timeout, &p_new->timeout ) ) - return( CL_SUCCESS ); + if (__cl_timer_is_earlier(&p_in_list->timeout, &p_new->timeout)) + return (CL_SUCCESS); - return( CL_NOT_FOUND ); + return (CL_NOT_FOUND); } cl_status_t -cl_timer_start( - IN cl_timer_t* const p_timer, - IN const uint32_t time_ms ) +cl_timer_start(IN cl_timer_t * const p_timer, IN const uint32_t time_ms) { - struct timeval curtime; - cl_list_item_t *p_list_item; - uint32_t delta_time = time_ms; + struct timeval curtime; + cl_list_item_t *p_list_item; + uint32_t delta_time = time_ms; - CL_ASSERT( p_timer ); - CL_ASSERT( p_timer->state == CL_INITIALIZED ); + CL_ASSERT(p_timer); + CL_ASSERT(p_timer->state == CL_INITIALIZED); - pthread_mutex_lock( &gp_timer_prov->mutex ); - /* Signal the timer provider thread to wake up. */ - pthread_cond_signal( &gp_timer_prov->cond ); + pthread_mutex_lock(&gp_timer_prov->mutex); + /* Signal the timer provider thread to wake up. */ + pthread_cond_signal(&gp_timer_prov->cond); - /* Remove the timer from the queue if currently queued. */ - if( p_timer->timer_state == CL_TIMER_QUEUED ) - cl_qlist_remove_item( &gp_timer_prov->queue, &p_timer->list_item); + /* Remove the timer from the queue if currently queued. */ + if (p_timer->timer_state == CL_TIMER_QUEUED) + cl_qlist_remove_item(&gp_timer_prov->queue, + &p_timer->list_item); - /* Get the current time */ + /* Get the current time */ #ifndef timerclear #define timerclear(tvp) (tvp)->tv_sec = (time_t)0, (tvp)->tv_usec = 0L #endif - timerclear( &curtime ); - gettimeofday( &curtime, NULL ); - - /* do not do 0 wait ! */ - /* if (delta_time < 1000.0) {delta_time = 1000;} */ - - /* Calculate the timeout. */ - p_timer->timeout.tv_sec = - curtime.tv_sec + (delta_time / 1000); - p_timer->timeout.tv_nsec = - (curtime.tv_usec + ((delta_time % 1000) * 1000)) * 1000; - - /* Add the timer to the queue. */ - if( cl_is_qlist_empty( &gp_timer_prov->queue ) ) - { - /* The timer list is empty. Add to the head. */ - cl_qlist_insert_head( &gp_timer_prov->queue, &p_timer->list_item ); - } - else - { - /* Find the correct insertion place in the list for the timer. */ - p_list_item = cl_qlist_find_from_tail( &gp_timer_prov->queue, - __cl_timer_find, p_timer ); - - /* Insert the timer. */ - cl_qlist_insert_next( &gp_timer_prov->queue, p_list_item, - &p_timer->list_item ); - } - /* Set the state. */ - p_timer->timer_state = CL_TIMER_QUEUED; - pthread_mutex_unlock( &gp_timer_prov->mutex ); - - return( CL_SUCCESS ); + timerclear(&curtime); + gettimeofday(&curtime, NULL); + + /* do not do 0 wait ! */ + /* if (delta_time < 1000.0) {delta_time = 1000;} */ + + /* Calculate the timeout. */ + p_timer->timeout.tv_sec = curtime.tv_sec + (delta_time / 1000); + p_timer->timeout.tv_nsec = + (curtime.tv_usec + ((delta_time % 1000) * 1000)) * 1000; + + /* Add the timer to the queue. */ + if (cl_is_qlist_empty(&gp_timer_prov->queue)) { + /* The timer list is empty. Add to the head. */ + cl_qlist_insert_head(&gp_timer_prov->queue, + &p_timer->list_item); + } else { + /* Find the correct insertion place in the list for the timer. */ + p_list_item = cl_qlist_find_from_tail(&gp_timer_prov->queue, + __cl_timer_find, p_timer); + + /* Insert the timer. */ + cl_qlist_insert_next(&gp_timer_prov->queue, p_list_item, + &p_timer->list_item); + } + /* Set the state. */ + p_timer->timer_state = CL_TIMER_QUEUED; + pthread_mutex_unlock(&gp_timer_prov->mutex); + + return (CL_SUCCESS); } -void -cl_timer_stop( - IN cl_timer_t* const p_timer ) +void cl_timer_stop(IN cl_timer_t * const p_timer) { - CL_ASSERT( p_timer ); - CL_ASSERT( p_timer->state == CL_INITIALIZED ); - - pthread_mutex_lock( &gp_timer_prov->mutex ); - switch( p_timer->timer_state ) - { - case CL_TIMER_RUNNING: - /* Wait for the callback to complete. */ - pthread_cond_wait( &p_timer->cond, &gp_timer_prov->mutex ); - /* Timer could have been queued while we were waiting. */ - if( p_timer->timer_state != CL_TIMER_QUEUED ) - break; - - case CL_TIMER_QUEUED: - /* Change the state of the timer. */ - p_timer->timer_state = CL_TIMER_IDLE; - /* Remove the timer from the queue. */ - cl_qlist_remove_item( &gp_timer_prov->queue, &p_timer->list_item ); - /* - * Signal the timer provider thread to move onto the - * next timer in the queue. - */ - pthread_cond_signal( &gp_timer_prov->cond ); - break; - - case CL_TIMER_IDLE: - break; - } - pthread_mutex_unlock( &gp_timer_prov->mutex ); + CL_ASSERT(p_timer); + CL_ASSERT(p_timer->state == CL_INITIALIZED); + + pthread_mutex_lock(&gp_timer_prov->mutex); + switch (p_timer->timer_state) { + case CL_TIMER_RUNNING: + /* Wait for the callback to complete. */ + pthread_cond_wait(&p_timer->cond, &gp_timer_prov->mutex); + /* Timer could have been queued while we were waiting. */ + if (p_timer->timer_state != CL_TIMER_QUEUED) + break; + + case CL_TIMER_QUEUED: + /* Change the state of the timer. */ + p_timer->timer_state = CL_TIMER_IDLE; + /* Remove the timer from the queue. */ + cl_qlist_remove_item(&gp_timer_prov->queue, + &p_timer->list_item); + /* + * Signal the timer provider thread to move onto the + * next timer in the queue. + */ + pthread_cond_signal(&gp_timer_prov->cond); + break; + + case CL_TIMER_IDLE: + break; + } + pthread_mutex_unlock(&gp_timer_prov->mutex); } cl_status_t -cl_timer_trim( - IN cl_timer_t* const p_timer, - IN const uint32_t time_ms ) +cl_timer_trim(IN cl_timer_t * const p_timer, IN const uint32_t time_ms) { - struct timeval curtime; - struct timespec newtime; - cl_status_t status; + struct timeval curtime; + struct timespec newtime; + cl_status_t status; - CL_ASSERT( p_timer ); - CL_ASSERT( p_timer->state == CL_INITIALIZED ); + CL_ASSERT(p_timer); + CL_ASSERT(p_timer->state == CL_INITIALIZED); - pthread_mutex_lock( &gp_timer_prov->mutex ); + pthread_mutex_lock(&gp_timer_prov->mutex); - /* Get the current time */ - timerclear( &curtime ); - gettimeofday( &curtime, NULL ); + /* Get the current time */ + timerclear(&curtime); + gettimeofday(&curtime, NULL); - /* Calculate the timeout. */ - newtime.tv_sec = - curtime.tv_sec + (time_ms / 1000); - newtime.tv_nsec = - (curtime.tv_usec + ((time_ms % 1000) * 1000)) * 1000; + /* Calculate the timeout. */ + newtime.tv_sec = curtime.tv_sec + (time_ms / 1000); + newtime.tv_nsec = (curtime.tv_usec + ((time_ms % 1000) * 1000)) * 1000; - if( p_timer->timer_state == CL_TIMER_QUEUED ) - { - /* If the old time is earlier, do not trim it. Just return. */ - if( __cl_timer_is_earlier( &p_timer->timeout, &newtime ) ) - { - pthread_mutex_unlock( &gp_timer_prov->mutex ); - return( CL_SUCCESS ); - } - } + if (p_timer->timer_state == CL_TIMER_QUEUED) { + /* If the old time is earlier, do not trim it. Just return. */ + if (__cl_timer_is_earlier(&p_timer->timeout, &newtime)) { + pthread_mutex_unlock(&gp_timer_prov->mutex); + return (CL_SUCCESS); + } + } - /* Reset the timer to the new timeout value. */ + /* Reset the timer to the new timeout value. */ - pthread_mutex_unlock( &gp_timer_prov->mutex ); - status = cl_timer_start( p_timer, time_ms ); + pthread_mutex_unlock(&gp_timer_prov->mutex); + status = cl_timer_start(p_timer, time_ms); - return( status ); + return (status); } -uint64_t -cl_get_time_stamp( void ) +uint64_t cl_get_time_stamp(void) { - uint64_t tstamp; - struct timeval tv; + uint64_t tstamp; + struct timeval tv; - timerclear( &tv ); - gettimeofday( &tv, NULL ); + timerclear(&tv); + gettimeofday(&tv, NULL); - /* Convert the time of day into a microsecond timestamp. */ - tstamp = ((uint64_t)tv.tv_sec * 1000000) + (uint64_t)tv.tv_usec; + /* Convert the time of day into a microsecond timestamp. */ + tstamp = ((uint64_t) tv.tv_sec * 1000000) + (uint64_t) tv.tv_usec; - return( tstamp ); + return (tstamp); } -uint32_t -cl_get_time_stamp_sec( void ) +uint32_t cl_get_time_stamp_sec(void) { - struct timeval tv; + struct timeval tv; - timerclear( &tv ); - gettimeofday( &tv, NULL ); + timerclear(&tv); + gettimeofday(&tv, NULL); - return( tv.tv_sec ); + return (tv.tv_sec); } diff --git a/opensm/complib/cl_vector.c b/opensm/complib/cl_vector.c index eb14c12..6e8e428 100644 --- a/opensm/complib/cl_vector.c +++ b/opensm/complib/cl_vector.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -75,12 +75,10 @@ * */ static void -cl_vector_copy_general( - OUT void* const p_dest, - IN const void* const p_src, - IN const size_t size ) +cl_vector_copy_general(OUT void *const p_dest, + IN const void *const p_src, IN const size_t size) { - memcpy( p_dest, p_src, size ); + memcpy(p_dest, p_src, size); } /* @@ -100,15 +98,13 @@ cl_vector_copy_general( * */ static void -cl_vector_copy8( - OUT void* const p_dest, - IN const void* const p_src, - IN const size_t size ) +cl_vector_copy8(OUT void *const p_dest, + IN const void *const p_src, IN const size_t size) { - CL_ASSERT( size == sizeof(uint8_t) ); - UNUSED_PARAM( size ); + CL_ASSERT(size == sizeof(uint8_t)); + UNUSED_PARAM(size); - *(uint8_t*)p_dest = *(uint8_t*)p_src; + *(uint8_t *) p_dest = *(uint8_t *) p_src; } /* @@ -128,15 +124,13 @@ cl_vector_copy8( * */ void -cl_vector_copy16( - OUT void* const p_dest, - IN const void* const p_src, - IN const size_t size ) +cl_vector_copy16(OUT void *const p_dest, + IN const void *const p_src, IN const size_t size) { - CL_ASSERT( size == sizeof(uint16_t) ); - UNUSED_PARAM( size ); + CL_ASSERT(size == sizeof(uint16_t)); + UNUSED_PARAM(size); - *(uint16_t*)p_dest = *(uint16_t*)p_src; + *(uint16_t *) p_dest = *(uint16_t *) p_src; } /* @@ -156,15 +150,13 @@ cl_vector_copy16( * */ void -cl_vector_copy32( - OUT void* const p_dest, - IN const void* const p_src, - IN const size_t size ) +cl_vector_copy32(OUT void *const p_dest, + IN const void *const p_src, IN const size_t size) { - CL_ASSERT( size == sizeof(uint32_t) ); - UNUSED_PARAM( size ); + CL_ASSERT(size == sizeof(uint32_t)); + UNUSED_PARAM(size); - *(uint32_t*)p_dest = *(uint32_t*)p_src; + *(uint32_t *) p_dest = *(uint32_t *) p_src; } /* @@ -184,44 +176,39 @@ cl_vector_copy32( * */ void -cl_vector_copy64( - OUT void* const p_dest, - IN const void* const p_src, - IN const size_t size ) +cl_vector_copy64(OUT void *const p_dest, + IN const void *const p_src, IN const size_t size) { - CL_ASSERT( size == sizeof(uint64_t) ); - UNUSED_PARAM( size ); + CL_ASSERT(size == sizeof(uint64_t)); + UNUSED_PARAM(size); - *(uint64_t*)p_dest = *(uint64_t*)p_src; + *(uint64_t *) p_dest = *(uint64_t *) p_src; } -void -cl_vector_construct( - IN cl_vector_t* const p_vector ) +void cl_vector_construct(IN cl_vector_t * const p_vector) { - CL_ASSERT( p_vector ); + CL_ASSERT(p_vector); - memset( p_vector, 0, sizeof(cl_vector_t) ); + memset(p_vector, 0, sizeof(cl_vector_t)); p_vector->state = CL_UNINITIALIZED; } cl_status_t -cl_vector_init( - IN cl_vector_t* const p_vector, - IN const size_t min_size, - IN const size_t grow_size, - IN const size_t element_size, - IN cl_pfn_vec_init_t pfn_init OPTIONAL, - IN cl_pfn_vec_dtor_t pfn_dtor OPTIONAL, - IN const void* const context ) +cl_vector_init(IN cl_vector_t * const p_vector, + IN const size_t min_size, + IN const size_t grow_size, + IN const size_t element_size, + IN cl_pfn_vec_init_t pfn_init OPTIONAL, + IN cl_pfn_vec_dtor_t pfn_dtor OPTIONAL, + IN const void *const context) { - cl_status_t status = CL_SUCCESS; + cl_status_t status = CL_SUCCESS; - CL_ASSERT( p_vector ); - CL_ASSERT( element_size ); + CL_ASSERT(p_vector); + CL_ASSERT(element_size); - cl_vector_construct( p_vector ); + cl_vector_construct(p_vector); p_vector->grow_size = grow_size; p_vector->element_size = element_size; @@ -233,8 +220,7 @@ cl_vector_init( * Try to choose a smart copy operator * someday, we could simply let the users pass one in */ - switch( element_size ) - { + switch (element_size) { case sizeof(uint8_t): p_vector->pfn_copy = cl_vector_copy8; break; @@ -263,51 +249,45 @@ cl_vector_init( p_vector->state = CL_INITIALIZED; /* Initialize the allocation list */ - cl_qlist_init( &p_vector->alloc_list ); + cl_qlist_init(&p_vector->alloc_list); /* get the storage needed by the user */ - if( min_size ) - { - status = cl_vector_set_size( p_vector, min_size ); - if( status != CL_SUCCESS ) - cl_vector_destroy( p_vector ); + if (min_size) { + status = cl_vector_set_size(p_vector, min_size); + if (status != CL_SUCCESS) + cl_vector_destroy(p_vector); } - return( status ); + return (status); } -void -cl_vector_destroy( - IN cl_vector_t* const p_vector ) +void cl_vector_destroy(IN cl_vector_t * const p_vector) { - size_t i; - void *p_element; + size_t i; + void *p_element; - CL_ASSERT( p_vector ); - CL_ASSERT( cl_is_state_valid( p_vector->state ) ); + CL_ASSERT(p_vector); + CL_ASSERT(cl_is_state_valid(p_vector->state)); /* Call the user's destructor for each element in the array. */ - if( p_vector->state == CL_INITIALIZED ) - { - if( p_vector->pfn_dtor ) - { - for( i = 0; i < p_vector->size; i++ ) - { + if (p_vector->state == CL_INITIALIZED) { + if (p_vector->pfn_dtor) { + for (i = 0; i < p_vector->size; i++) { p_element = p_vector->p_ptr_array[i]; /* Sanity check! */ - CL_ASSERT( p_element ); - p_vector->pfn_dtor( p_element, (void*)p_vector->context ); + CL_ASSERT(p_element); + p_vector->pfn_dtor(p_element, + (void *)p_vector->context); } } /* Deallocate the pages */ - while( !cl_is_qlist_empty( &p_vector->alloc_list ) ) - free( cl_qlist_remove_head( &p_vector->alloc_list ) ); + while (!cl_is_qlist_empty(&p_vector->alloc_list)) + free(cl_qlist_remove_head(&p_vector->alloc_list)); /* Destroy the page vector. */ - if( p_vector->p_ptr_array ) - { - free( p_vector->p_ptr_array ); + if (p_vector->p_ptr_array) { + free(p_vector->p_ptr_array); p_vector->p_ptr_array = NULL; } } @@ -316,93 +296,85 @@ cl_vector_destroy( } cl_status_t -cl_vector_at( - IN const cl_vector_t* const p_vector, - IN const size_t index, - OUT void* const p_element ) +cl_vector_at(IN const cl_vector_t * const p_vector, + IN const size_t index, OUT void *const p_element) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Range check */ - if( index >= p_vector->size ) - return( CL_INVALID_PARAMETER ); + if (index >= p_vector->size) + return (CL_INVALID_PARAMETER); - cl_vector_get( p_vector, index, p_element ); - return( CL_SUCCESS ); + cl_vector_get(p_vector, index, p_element); + return (CL_SUCCESS); } cl_status_t -cl_vector_set( - IN cl_vector_t* const p_vector, - IN const size_t index, - IN void* const p_element ) +cl_vector_set(IN cl_vector_t * const p_vector, + IN const size_t index, IN void *const p_element) { - cl_status_t status; - void *p_dest; + cl_status_t status; + void *p_dest; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( p_element ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(p_element); /* Determine if the vector has room for this element. */ - if( index >= p_vector->size ) - { + if (index >= p_vector->size) { /* Resize to accomodate the given index. */ - status = cl_vector_set_size( p_vector, index + 1 ); + status = cl_vector_set_size(p_vector, index + 1); /* Check for failure on or before the given index. */ - if( (status != CL_SUCCESS) && (p_vector->size < index) ) - return( status ); + if ((status != CL_SUCCESS) && (p_vector->size < index)) + return (status); } /* At this point, the array is guaranteed to be big enough */ - p_dest = cl_vector_get_ptr( p_vector, index ); + p_dest = cl_vector_get_ptr(p_vector, index); /* Sanity check! */ - CL_ASSERT( p_dest ); + CL_ASSERT(p_dest); /* Copy the data into the array */ - p_vector->pfn_copy( p_dest, p_element, p_vector->element_size ); + p_vector->pfn_copy(p_dest, p_element, p_vector->element_size); - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_vector_set_capacity( - IN cl_vector_t* const p_vector, - IN const size_t new_capacity ) +cl_vector_set_capacity(IN cl_vector_t * const p_vector, + IN const size_t new_capacity) { - size_t new_elements; - size_t alloc_size; - size_t i; - cl_list_item_t *p_buf; - void *p_new_ptr_array; + size_t new_elements; + size_t alloc_size; + size_t i; + cl_list_item_t *p_buf; + void *p_new_ptr_array; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Do we have to do anything here? */ - if( new_capacity <= p_vector->capacity ) - { + if (new_capacity <= p_vector->capacity) { /* Nope */ - return( CL_SUCCESS ); + return (CL_SUCCESS); } /* Allocate our pointer array. */ - p_new_ptr_array = malloc( new_capacity * sizeof(void*) ); - if( !p_new_ptr_array ) - return( CL_INSUFFICIENT_MEMORY ); + p_new_ptr_array = malloc(new_capacity * sizeof(void *)); + if (!p_new_ptr_array) + return (CL_INSUFFICIENT_MEMORY); else - memset( p_new_ptr_array, 0, new_capacity * sizeof(void*) ); + memset(p_new_ptr_array, 0, new_capacity * sizeof(void *)); - if( p_vector->p_ptr_array ) - { + if (p_vector->p_ptr_array) { /* Copy the old pointer array into the new. */ - memcpy( p_new_ptr_array, p_vector->p_ptr_array, - p_vector->capacity * sizeof(void*) ); + memcpy(p_new_ptr_array, p_vector->p_ptr_array, + p_vector->capacity * sizeof(void *)); /* Free the old pointer array. */ - free( p_vector->p_ptr_array ); + free(p_vector->p_ptr_array); } /* Set the new array. */ @@ -416,192 +388,178 @@ cl_vector_set_capacity( /* Determine the allocation size for the new array elements. */ alloc_size = new_elements * p_vector->element_size; - p_buf = (cl_list_item_t*)malloc( alloc_size + sizeof(cl_list_item_t) ); - if( !p_buf ) - return( CL_INSUFFICIENT_MEMORY ); + p_buf = (cl_list_item_t *) malloc(alloc_size + sizeof(cl_list_item_t)); + if (!p_buf) + return (CL_INSUFFICIENT_MEMORY); else - memset( p_buf, 0, alloc_size + sizeof(cl_list_item_t) ); + memset(p_buf, 0, alloc_size + sizeof(cl_list_item_t)); - cl_qlist_insert_tail( &p_vector->alloc_list, p_buf ); + cl_qlist_insert_tail(&p_vector->alloc_list, p_buf); /* Advance the buffer pointer past the list item. */ p_buf++; - for( i = p_vector->capacity; i < new_capacity; i++ ) - { + for (i = p_vector->capacity; i < new_capacity; i++) { p_vector->p_ptr_array[i] = p_buf; /* Move the buffer pointer to the next element. */ - p_buf = (void*)(((uint8_t*)p_buf) + p_vector->element_size); + p_buf = (void *)(((uint8_t *) p_buf) + p_vector->element_size); } /* Update the vector with the new capactity. */ p_vector->capacity = new_capacity; - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_vector_set_size( - IN cl_vector_t* const p_vector, - IN const size_t size ) +cl_vector_set_size(IN cl_vector_t * const p_vector, IN const size_t size) { - cl_status_t status; - size_t new_capacity; - size_t index; - void *p_element; + cl_status_t status; + size_t new_capacity; + size_t index; + void *p_element; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); /* Check to see if the requested size is the same as the existing size. */ - if( size == p_vector->size ) - return( CL_SUCCESS ); + if (size == p_vector->size) + return (CL_SUCCESS); /* Determine if the vector has room for this element. */ - if( size >= p_vector->capacity ) - { - if( !p_vector->grow_size ) - return( CL_INSUFFICIENT_MEMORY ); + if (size >= p_vector->capacity) { + if (!p_vector->grow_size) + return (CL_INSUFFICIENT_MEMORY); /* Calculate the new capacity, taking into account the grow size. */ new_capacity = size; - if( size % p_vector->grow_size ) - { + if (size % p_vector->grow_size) { /* Round up to nearest grow_size boundary. */ new_capacity += p_vector->grow_size - - (size % p_vector->grow_size); + (size % p_vector->grow_size); } - status = cl_vector_set_capacity( p_vector, new_capacity ); - if( status != CL_SUCCESS ) - return( status ); + status = cl_vector_set_capacity(p_vector, new_capacity); + if (status != CL_SUCCESS) + return (status); } /* Are we growing the array and need to invoke an initializer callback? */ - if( size > p_vector->size && p_vector->pfn_init ) - { - for( index = p_vector->size; index < size; index++ ) - { + if (size > p_vector->size && p_vector->pfn_init) { + for (index = p_vector->size; index < size; index++) { /* Get a pointer to this element */ - p_element = cl_vector_get_ptr( p_vector, index ); + p_element = cl_vector_get_ptr(p_vector, index); /* Call the user's initializer and trap failures. */ - status = p_vector->pfn_init( p_element, (void*)p_vector->context ); - if( status != CL_SUCCESS ) - { + status = + p_vector->pfn_init(p_element, + (void *)p_vector->context); + if (status != CL_SUCCESS) { /* Call the destructor for this object */ - if( p_vector->pfn_dtor ) - p_vector->pfn_dtor( p_element, (void*)p_vector->context ); + if (p_vector->pfn_dtor) + p_vector->pfn_dtor(p_element, + (void *)p_vector-> + context); /* Return the failure status to the caller. */ - return( status ); + return (status); } /* The array just grew by one element */ p_vector->size++; } - } - else if( p_vector->pfn_dtor ) - { + } else if (p_vector->pfn_dtor) { /* The array is shrinking and there is a destructor to invoke. */ - for( index = size; index < p_vector->size; index++ ) - { + for (index = size; index < p_vector->size; index++) { /* compute the address of the new elements */ - p_element = cl_vector_get_ptr( p_vector, index ); + p_element = cl_vector_get_ptr(p_vector, index); /* call the user's destructor */ - p_vector->pfn_dtor( p_element, (void*)p_vector->context ); + p_vector->pfn_dtor(p_element, + (void *)p_vector->context); } } p_vector->size = size; - return( CL_SUCCESS ); + return (CL_SUCCESS); } cl_status_t -cl_vector_set_min_size( - IN cl_vector_t* const p_vector, - IN const size_t min_size ) +cl_vector_set_min_size(IN cl_vector_t * const p_vector, + IN const size_t min_size) { - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); - if( min_size > p_vector->size ) - { + if (min_size > p_vector->size) { /* We have to resize the array */ - return( cl_vector_set_size( p_vector, min_size ) ); + return (cl_vector_set_size(p_vector, min_size)); } /* We didn't have to do anything */ - return( CL_SUCCESS ); + return (CL_SUCCESS); } void -cl_vector_apply_func( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_apply_t pfn_callback, - IN const void* const context ) +cl_vector_apply_func(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_apply_t pfn_callback, + IN const void *const context) { - size_t i; - void *p_element; + size_t i; + void *p_element; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); - for( i = 0; i < p_vector->size; i++ ) - { - p_element = cl_vector_get_ptr( p_vector, i ); - pfn_callback( i, p_element, (void*)context ); + for (i = 0; i < p_vector->size; i++) { + p_element = cl_vector_get_ptr(p_vector, i); + pfn_callback(i, p_element, (void *)context); } } size_t -cl_vector_find_from_start( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_find_t pfn_callback, - IN const void* const context ) +cl_vector_find_from_start(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_find_t pfn_callback, + IN const void *const context) { - size_t i; - void *p_element; + size_t i; + void *p_element; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); - for( i = 0; i < p_vector->size; i++ ) - { - p_element = cl_vector_get_ptr( p_vector, i ); + for (i = 0; i < p_vector->size; i++) { + p_element = cl_vector_get_ptr(p_vector, i); /* Invoke the callback */ - if( pfn_callback( i, p_element, (void*)context ) == CL_SUCCESS ) + if (pfn_callback(i, p_element, (void *)context) == CL_SUCCESS) break; } - return( i ); + return (i); } size_t -cl_vector_find_from_end( - IN const cl_vector_t* const p_vector, - IN cl_pfn_vec_find_t pfn_callback, - IN const void* const context ) +cl_vector_find_from_end(IN const cl_vector_t * const p_vector, + IN cl_pfn_vec_find_t pfn_callback, + IN const void *const context) { - size_t i; - void *p_element; + size_t i; + void *p_element; - CL_ASSERT( p_vector ); - CL_ASSERT( p_vector->state == CL_INITIALIZED ); - CL_ASSERT( pfn_callback ); + CL_ASSERT(p_vector); + CL_ASSERT(p_vector->state == CL_INITIALIZED); + CL_ASSERT(pfn_callback); i = p_vector->size; - while( i ) - { + while (i) { /* Get a pointer to the element in the array. */ - p_element = cl_vector_get_ptr( p_vector, --i ); - CL_ASSERT( p_element ); + p_element = cl_vector_get_ptr(p_vector, --i); + CL_ASSERT(p_element); /* Invoke the callback for the current element. */ - if( pfn_callback( i, p_element, (void*)context ) == CL_SUCCESS ) - return( i ); + if (pfn_callback(i, p_element, (void *)context) == CL_SUCCESS) + return (i); } - return( p_vector->size ); + return (p_vector->size); } diff --git a/opensm/complib/ib_statustext.c b/opensm/complib/ib_statustext.c index 07aee7a..0bedff1 100644 --- a/opensm/complib/ib_statustext.c +++ b/opensm/complib/ib_statustext.c @@ -45,13 +45,12 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include /* ib_api_status_t values above converted to text for easier printing. */ -const char* ib_error_str[] = -{ +const char *ib_error_str[] = { "IB_SUCCESS", "IB_INSUFFICIENT_RESOURCES", "IB_INSUFFICIENT_MEMORY", @@ -95,7 +94,7 @@ const char* ib_error_str[] = "IB_INVALID_AL_HANDLE", "IB_INVALID_HANDLE", "IB_ERROR", - "IB_REMOTE_ERROR", /* Infiniband Access Layer */ + "IB_REMOTE_ERROR", /* Infiniband Access Layer */ "IB_VERBS_PROCESSING_DONE", "IB_INVALID_WR_TYPE", "IB_QP_IN_TIMEWAIT", @@ -103,12 +102,10 @@ const char* ib_error_str[] = "IB_INVALID_PORT", "IB_NOT_DONE", "IB_UNKNOWN_ERROR" - }; /* ib_async_event_t values above converted to text for easier printing. */ -const char* ib_async_event_str[] = -{ +const char *ib_async_event_str[] = { "IB_AE_SQ_ERROR", "IB_AE_SQ_DRAINED", "IB_AE_RQ_ERROR", @@ -133,14 +130,12 @@ const char* ib_async_event_str[] = "IB_AE_EEC_APM_ERROR", "IB_AE_WQ_REQ_ERROR", "IB_AE_WQ_ACCESS_ERROR", - "IB_AE_PORT_ACTIVE", /* ACTIVE STATE */ - "IB_AE_PORT_DOWN", /* INIT", ARMED", DOWN */ + "IB_AE_PORT_ACTIVE", /* ACTIVE STATE */ + "IB_AE_PORT_DOWN", /* INIT", ARMED", DOWN */ "IB_AE_UNKNOWN" - }; -const char* ib_wc_status_str[] = -{ +const char *ib_wc_status_str[] = { "IB_WCS_SUCCESS", "IB_WCS_LOCAL_LEN_ERR", "IB_WCS_LOCAL_OP_ERR", @@ -157,7 +152,6 @@ const char* ib_wc_status_str[] = "IB_WCS_INVALID_EECN", "IB_WCS_INVALID_EEC_STATE", "IB_WCS_UNMATCHED_RESPONSE", /* InfiniBand Access Layer */ - "IB_WCS_CANCELED", /* InfiniBand Access Layer */ + "IB_WCS_CANCELED", /* InfiniBand Access Layer */ "IB_WCS_UNKNOWN" - }; -- 1.5.3.rc2.38.g11308 From rdreier at cisco.com Tue Aug 14 11:00:24 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 11:00:24 -0700 Subject: [ofa-general] Re: [PATCHv4 RFC] Scalable Reliable Connection: API and documentation In-Reply-To: <200708141130.23254.jackm@dev.mellanox.co.il> (Jack Morgenstein's message of "Tue, 14 Aug 2007 11:30:22 +0300") References: <20070808071910.GC23514@mellanox.co.il> <200708121536.47152.jackm@dev.mellanox.co.il> <200708141130.23254.jackm@dev.mellanox.co.il> Message-ID: > Only caveat -- I'll need to put a comment in the code regarding the > overloading (to avoid confusion). Yes... in fact I would rename the field to something like __u32 srq_or_xrcdomain_handle; - R. From afriedle at open-mpi.org Tue Aug 14 11:12:41 2007 From: afriedle at open-mpi.org (Andrew Friedley) Date: Tue, 14 Aug 2007 11:12:41 -0700 Subject: [ofa-general] multicast group join limits -- test code Message-ID: <46C1F099.5070403@open-mpi.org> I've attached a simple test program that should demonstrate the limitations I'm seeing when joining multiple multicast groups; the idea being to allow others to see the weirdness I'm seeing and make some progress. An MPI is needed to compile/run the test. No arguments are needed; the test repeatedly joins groups (without leaving them) until an error occurs, then intentionally hangs. Here's some of the different behaviors I see with this test (OFED v1.2 is always used): mpirun -np 1 ./jointest On my 128 node machine 'odin' running OpenSM, I was able to join 891 groups quite a few times in a row. Then suddenly running the same test again I was able to join only 5 groups. This behavior persists on this node. I can go to another node on the same machine, and again be able to join 891 groups. If I run the test separately on two different nodes (that can still join 891 each), I am able to join a total of 891 groups between both nodes before both tests error. If I run on one node that errors after 5 groups and another that errors at 891 groups, the first node joins 5 groups and the second joins 886 groups. On a separate 8 node machine 'thor' running Cisco's SM on a Topspin switch, I can join 14 groups. mpirun -np 2 ./jointest (one node) On odin I can join 892 groups, the thor machine is able to join 5 groups. mpirun -np 2 ./jointest (two nodes) Odin was able to join 4 groups for the first 3 runs, then was able to join 14 groups repeatedly. Thor is able to join 5 groups consistently. None of these results seem to match with any of the hardcoded limits people have mentioned to me. I really need to figure out the cause of this strange behavior, as most cases severely limit the usability of IB multicast in MPI. Is my test code correct? Does anybody know what is causing this, or where I could look/test to try and nail it down? I've gotten suggestions that the problem lies in the SM, though I haven't found anything blatantly wrong when reading relevant parts of the OpenSM code. Andrew -------------- next part -------------- A non-text attachment was scrubbed... Name: jointest.tar.gz Type: application/x-gzip Size: 5907 bytes Desc: not available URL: From rdreier at cisco.com Tue Aug 14 11:14:29 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 11:14:29 -0700 Subject: [ofa-general] removing PR caching from IPoIB In-Reply-To: <46C0F2DD.6070109@ichips.intel.com> (Sean Hefty's message of "Mon, 13 Aug 2007 17:10:05 -0700") References: <46C0F2DD.6070109@ichips.intel.com> Message-ID: > I'm looking at removing the PR caching in IPoIB. > > Is there any reason why structs ipoib_path and ipoib_neigh cannot be > merged? (And ipoib_path removed?) > > Is there any issue with the PR/AH scope aligning with that of ipoib_neigh? There are some packets that are sent with no neighbour structure; unicast ARPs are the case that I know of, although I'm not positive there aren't other such cases. See unicast_arp_send() in the IPoIB source for details. So you may not be able to merge ipoib_path and ipoib_neigh. However it should be possible to only keep the AH once a path record lookup is done. OTOH I'm not sure it will end up simplifying anything, since you would have to deal with allocating/freeing somewhere to hold the struct ib_sa_path_rec between starting the path record lookup and when it finishes. - R. From rdreier at cisco.com Tue Aug 14 11:14:54 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 11:14:54 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46C0F2A7.5010807@linux.vnet.ibm.com> (Pradeep Satyanarayana's message of "Mon, 13 Aug 2007 17:09:11 -0700") References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> Message-ID: > Without a separate CQ, how do the receive work completion handlers distinguish > between the different receive queues of rx_qp and tx_qp -any suggestions? You have the work request ID in the work completion -- is that not enough? From Loree_krutkewich at heike-presse.de Tue Aug 14 19:22:24 2007 From: Loree_krutkewich at heike-presse.de (Loree krutkewich) Date: Tue, 14 Aug 2007 19:22:24 -0700 Subject: [ofa-general] Definitely Israeli passport is not worth that. Message-ID: <000601c7dee3$1e2656c0$079ec25a@patrickkenny> T+h i s Tue*sday i*t.s CY'TV!!! G,e.t on C*Y.T'V Fir*st Thi+ng on TU+ES+DAY, i*t_'+s go.ing to exp,load! Comp any*: CHI-NA YOUT*V C+O,R*P (+O.T.C BB:CYT-V.O+B) Symbo_l: C.Y'T'V Cu.rrent Pr+ice: $_0,.,4*6 Mond,ay M,ove: (+U'p Over(1+2.2.0%) T*h-e p rice is at a mi-nimum it w+i,l.l b_o o*m on Tuesda*y! Recomm+e ndation: "*STRON+G-BUY" start._ing on Tuesda-y., A*UGUST 1+4_, 2 0*0 6_. Brea,k,ing N e+w*s-: C,Y,T_V M,AKES MAJ-OR ANNOUNCEMEN.+TS: Chin.a Y*ouTV's Cn'Boo W*e+b S+i+t.e Rank,s N o_.'1 on Micros-of t L'i v'e Sea_rch Engin_e Aggres*i.ve Tr,aders G,e.t in early*, t'h'i*s o+n'e c'a+n f_l_i+p h'i+g h retur+ns f+a*s+t+. F+o-r m o r-e informat.ion_, ch.eck o_u-t o u_r KSOC.KS D a,t.a Shee_t. If y,o'u k+n-e_w h_o_w m-u_c*h t'h*i s mean-s to me. We u s*e n'amed not,at_ion f+o*r t h-e paramet.e+rs. G*e*t+s t.h-e ti_me, in second++s, si,nce a memb+er''s l*ogon to t-h'e P_.assport logo*n server.. T,h-e*y w-ouldn+'t h.a_v_e stopp_ed h*e'r,. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at cisco.com Tue Aug 14 11:31:45 2007 From: jsquyres at cisco.com (Jeff Squyres) Date: Tue, 14 Aug 2007 14:31:45 -0400 Subject: [ofa-general] multicast group join limits -- test code In-Reply-To: <46C1F099.5070403@open-mpi.org> References: <46C1F099.5070403@open-mpi.org> Message-ID: On Aug 14, 2007, at 2:12 PM, Andrew Friedley wrote: > An MPI is needed to compile/run the test. No arguments are needed; > the test repeatedly joins groups (without leaving them) until an > error occurs, then intentionally hangs. Just to clarify for those not familiar with MPI -- MPI is not used in the multicast portion of the test. It's only used to bootstrap / launch the test and used as an "out of band" messaging mechanism so that you can know when the group has been joined, etc. So you can even use a TCP-only MPI to run this test to ensure that you are not skewing any IB stack issues. > Here's some of the different behaviors I see with this test (OFED > v1.2 is always used): FWIW, I've been trying to help Andrew run this test, and I always run into one of two errors: - Running 1 proc each on 2 nodes joins 2 groups, then: 0 ERROR rdma_join_multicast(): 99 Cannot assign requested address - Running 4 procs on 1 node joins a few 10s of groups (different every time) and then: ERROR event 13, status -110 Operation now in progress, forcing job to hang The nodes are all RHEL4U4 running OFED 1.2; each node has 4 cores. -- Jeff Squyres Cisco Systems From pradeeps at linux.vnet.ibm.com Tue Aug 14 11:50:05 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Tue, 14 Aug 2007 11:50:05 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> Message-ID: <46C1F95D.2050508@linux.vnet.ibm.com> Roland Dreier wrote: > > Without a separate CQ, how do the receive work completion handlers distinguish > > between the different receive queues of rx_qp and tx_qp -any suggestions? > > You have the work request ID in the work completion -- is that not enough? > That maybe a possibility ...in the NOSRQ case for the rx_qp I play around with the bits in the wr_id, so I may be able to check upon the receipt of a packet if the bits are mangled or not. However, that means I need to ascertain on receipt of every packet the destination receive queue. I thought a cleaner solution (and better performing) would be to implement a separate CQ and handler. Could you explain your reservations to this approach? Would it be enough if I change my previous patch to post 1 WR to start with and then as you suggested upon receipt of that packet post a full set of WRs? Pradeep From mshefty at ichips.intel.com Tue Aug 14 12:27:06 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 14 Aug 2007 12:27:06 -0700 Subject: [ofa-general] removing PR caching from IPoIB In-Reply-To: References: <46C0F2DD.6070109@ichips.intel.com> Message-ID: <46C2020A.3080709@ichips.intel.com> > There are some packets that are sent with no neighbour structure; > unicast ARPs are the case that I know of, although I'm not positive > there aren't other such cases. See unicast_arp_send() in the IPoIB > source for details. My first thought was to tie the IPoIB PR lifetime to ipoib_neigh and treat unicast ARPs separately. For unicast ARPs, I was thinking of setting the skb as the context of the PR query and send it after it completed, but I hadn't considered cleanup. As an alternate approach, I could try adding a reference count to ipoib_path that's incremented once for each neighbour or when sending a unicast ARP. This may be equivalent to merging ipoib_ah and ipoib_path, but I haven't looked at the details of trying this. Do you have a preference or any other suggestions that would limit the PR lifetime? - Sean From rdreier at cisco.com Tue Aug 14 12:30:49 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 12:30:49 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46C1F95D.2050508@linux.vnet.ibm.com> (Pradeep Satyanarayana's message of "Tue, 14 Aug 2007 11:50:05 -0700") References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> <46C1F95D.2050508@linux.vnet.ibm.com> Message-ID: > I thought a cleaner solution (and better performing) would be to implement a separate > CQ and handler. Could you explain your reservations to this approach? We want to have all completion handling go through the NAPI poll routine to avoid having the system collapse under high interrupt load. And yes, having a separate send CQ for CM QPs is a problem we need to fix. - R. From rdreier at cisco.com Tue Aug 14 12:33:06 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 12:33:06 -0700 Subject: [ofa-general] removing PR caching from IPoIB In-Reply-To: <46C2020A.3080709@ichips.intel.com> (Sean Hefty's message of "Tue, 14 Aug 2007 12:27:06 -0700") References: <46C0F2DD.6070109@ichips.intel.com> <46C2020A.3080709@ichips.intel.com> Message-ID: > My first thought was to tie the IPoIB PR lifetime to ipoib_neigh and > treat unicast ARPs separately. For unicast ARPs, I was thinking of > setting the skb as the context of the PR query and send it after it > completed, but I hadn't considered cleanup. So you would do a new path record lookup for every unicast ARP that gets sent? What is the advantage of that? > As an alternate approach, I could try adding a reference count to > ipoib_path that's incremented once for each neighbour or when sending > a unicast ARP. This may be equivalent to merging ipoib_ah and > ipoib_path, but I haven't looked at the details of trying this. > > Do you have a preference or any other suggestions that would limit the > PR lifetime? What is the motivation for these changes? If you explain what you're trying to fix, then maybe I can give a more intelligent answer. - R. From pradeeps at linux.vnet.ibm.com Tue Aug 14 13:23:11 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Tue, 14 Aug 2007 13:23:11 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> <46C1F95D.2050508@linux.vnet.ibm.com> Message-ID: <46C20F2F.1050003@linux.vnet.ibm.com> Roland Dreier wrote: > > I thought a cleaner solution (and better performing) would be to implement a separate > > CQ and handler. Could you explain your reservations to this approach? > > We want to have all completion handling go through the NAPI poll > routine to avoid having the system collapse under high interrupt load. > I purposely chose not to use NAPI. In fact I also chose not to use scatter- gather lists for the skbs (of the rq of tx_qp), because I believe this really is a corner case. There hasn't been a strong requirement for heterogeneous OS support. When SRQ will be supported on the IBM HCA, there will be no need for this. I am implementing this only for completeness sake and want to do it with minimal effort. Given the above, do you still see it necessary to use NAPI? Is it acceptable that I roll up the previous NOSRQ patches (other than this one) into a single patch and can that one be integrated first? We can deal with this one after that. The corner case one is holding up the main patch and I would like to find a way to get the main patch in ASAP. Pradeep From rdreier at cisco.com Tue Aug 14 13:26:47 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 13:26:47 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: <46C20F2F.1050003@linux.vnet.ibm.com> (Pradeep Satyanarayana's message of "Tue, 14 Aug 2007 13:23:11 -0700") References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> <46C1F95D.2050508@linux.vnet.ibm.com> <46C20F2F.1050003@linux.vnet.ibm.com> Message-ID: > I am implementing this only for completeness sake and want to do it with minimal > effort. Given the above, do you still see it necessary to use NAPI? > Is it acceptable that I roll up the previous NOSRQ patches (other than this one) into > a single patch and can that one be integrated first? Given that you don't want to actually have something that really works, I guess there's no point in doing it at all. Since there are no other IPoIB CM implementations out there yet, maybe it's acceptable to say that we don't implement a fully RFC-compliant version of the protocol. Anwyay, yes, let's see your latest patch and see how far away that is. - R. From mshefty at ichips.intel.com Tue Aug 14 13:52:20 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Tue, 14 Aug 2007 13:52:20 -0700 Subject: [ofa-general] removing PR caching from IPoIB In-Reply-To: References: <46C0F2DD.6070109@ichips.intel.com> <46C2020A.3080709@ichips.intel.com> Message-ID: <46C21604.9050108@ichips.intel.com> > What is the motivation for these changes? If you explain what you're > trying to fix, then maybe I can give a more intelligent answer. IPoIB permanently caches PRs with no mechanism in place to keep them in sync with the SA. The goal is to limit the PR lifetime to prevent the use of stale paths. A side benefit of limiting the PR scope to a neighbour is that different neighbours could use different paths, versus using a single path per DGID. - Sean From P.Selzner at KRZ.DE Tue Aug 14 15:21:17 2007 From: P.Selzner at KRZ.DE (Selzner, Peter (KRZ)) Date: Wed, 15 Aug 2007 00:21:17 +0200 Subject: [ofa-general] ethernet an infiniband bonds ? Message-ID: <1c0af451000048f8@krz.de> Hi, can enyone describe the right way to build both (ethernet and infiniband/ipoib) bonds on an linux machine PLEASE. We have problems to realize our wanted architecture with 2 eth-bonds (bond0 - eth0,eth2 and bond1 - eth1,eth3) and 1 inf-bond (ibbond0 - ib0,ib1). Should we make this with the "classic" way over the configuration files in /etc/sysconfig/network (SLES 10) or with the /etc/inifiniabnd/openib.conf file, but is this the right file to configure eth-bonds? If necessery to publish my configuration files let me known and i will do this, but for the first time it would be very helpfull to get basic information about this stuff. Many thanks in advance. Peter Selzner -------------- next part -------------- An HTML attachment was scrubbed... URL: From or.gerlitz at gmail.com Tue Aug 14 15:29:57 2007 From: or.gerlitz at gmail.com (Or Gerlitz) Date: Wed, 15 Aug 2007 01:29:57 +0300 Subject: [ofa-general] ethernet an infiniband bonds ? In-Reply-To: <1c0af451000048f8@krz.de> References: <1c0af451000048f8@krz.de> Message-ID: <15ddcffd0708141529w5d7d7db3qfce1f34c5a10320c@mail.gmail.com> On 8/15/07, Selzner, Peter (KRZ) wrote: > > can enyone describe the right way to build both (ethernet and > infiniband/ipoib) bonds on an linux machine PLEASE. > We have problems to realize our wanted architecture with 2 eth-bonds > (bond0 - eth0,eth2 and bond1 - eth1,eth3) and 1 inf-bond > > (ibbond0 - ib0,ib1). Should we make this with the "classic" way over the > configuration files in /etc/sysconfig/network (SLES 10) > > or with the /etc/inifiniabnd/openib.conf file, but is this the right file > to configure eth-bonds? > If necessery to publish my configuration files let me known and i will do > this, but for the first time it would be very helpfull to get > > basic information about this stuff. > Hi Peter, The bonding driver that comes with OFED 1.2 is based on the 2.6.21 upstream kernel driver. However, we did not ported the somehow deprecated ifenslave program which is used in SLES10 to configure bonding when you set a network script for a bond. This means that for "IB bonds" you must not set such script and use the mechanism in /etc/inifiniabnd/openib.conf and for "Eth bonds" you must not use openib.conf but rather a standard network script... Its next in our todo to provide a solution for IB bonds in the standard framework. I am cc-ing Moni Shoua who is the OFED bonding maintainer, please approach him and cc the list on any problem you face with the IB bonding solution. Or. -------------- next part -------------- An HTML attachment was scrubbed... URL: From weiny2 at llnl.gov Tue Aug 14 16:21:48 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 14 Aug 2007 16:21:48 -0700 Subject: [ofa-general] [PATCH] Fix the osm_epi_node_id_t structure name. Message-ID: <20070814162148.1012c706.weiny2@llnl.gov> >From 6e01f304c1d1cbd8d1163708d907ec0550a0ca31 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Wed, 25 Jul 2007 11:33:08 -0700 Subject: [PATCH] Fix the osm_epi_node_id_t structure name. The structure osm_epi_node_id_t describes a port. Signed-off-by: Ira K. Weiny --- opensm/include/opensm/osm_event_plugin.h | 20 ++++++++++---------- opensm/opensm/osm_perfmgr_db.c | 4 ++-- opensm/osmeventplugin/src/osmeventplugin.c | 24 ++++++++++++------------ 3 files changed, 24 insertions(+), 24 deletions(-) diff --git a/opensm/include/opensm/osm_event_plugin.h b/opensm/include/opensm/osm_event_plugin.h index 57d9e05..b575d2a 100644 --- a/opensm/include/opensm/osm_event_plugin.h +++ b/opensm/include/opensm/osm_event_plugin.h @@ -83,7 +83,7 @@ typedef struct { uint64_t node_guid; uint8_t port_num; char node_name[OSM_EPI_NODE_NAME_LEN]; -} osm_epi_node_id_t; +} osm_epi_port_id_t; /** ========================================================================= * Port error event @@ -91,7 +91,7 @@ typedef struct { * This is a difference from the last reading. NOT an absolute reading. */ typedef struct { - osm_epi_node_id_t node_id; + osm_epi_port_id_t port_id; uint64_t symbol_err_cnt; uint64_t link_err_recover; uint64_t link_downed; @@ -112,7 +112,7 @@ typedef struct { * This is a difference from the last reading. NOT an absolute reading. */ typedef struct { - osm_epi_node_id_t node_id; + osm_epi_port_id_t port_id; uint64_t xmit_data; uint64_t rcv_data; uint64_t xmit_pkts; @@ -129,7 +129,7 @@ typedef struct { * This is a difference from the last reading. NOT an absolute reading. */ typedef struct { - osm_epi_node_id_t node_id; + osm_epi_port_id_t port_id; uint64_t xmit_wait; time_t time_diff_s; } osm_epi_ps_event_t; @@ -138,7 +138,7 @@ typedef struct { * Trap events */ typedef struct { - osm_epi_node_id_t node_id; + osm_epi_port_id_t port_id; uint8_t type; uint32_t prod_type; uint16_t trap_num; @@ -189,13 +189,13 @@ void osm_epi_report(osm_epi_plugin_t *plugin, * Helper functions */ static inline void -osm_epi_create_node_id(osm_epi_node_id_t *node_id, uint64_t node_guid, +osm_epi_create_port_id(osm_epi_port_id_t *port_id, uint64_t node_guid, uint8_t port_num, char *node_name) { - node_id->node_guid = node_guid; - node_id->port_num = port_num; - strncpy(node_id->node_name, node_name, OSM_EPI_NODE_NAME_LEN); - node_id->node_name[OSM_EPI_NODE_NAME_LEN-1] = '\0'; + port_id->node_guid = node_guid; + port_id->port_num = port_num; + strncpy(port_id->node_name, node_name, OSM_EPI_NODE_NAME_LEN); + port_id->node_name[OSM_EPI_NODE_NAME_LEN-1] = '\0'; } END_C_DECLS diff --git a/opensm/opensm/osm_perfmgr_db.c b/opensm/opensm/osm_perfmgr_db.c index 2a1c94b..4c82673 100644 --- a/opensm/opensm/osm_perfmgr_db.c +++ b/opensm/opensm/osm_perfmgr_db.c @@ -253,7 +253,7 @@ perfmgr_db_add_err_reading(perfmgr_db_t *db, uint64_t guid, debug_dump_err_reading(db, guid, port, p_port, reading); epi_pe_data.time_diff_s = (reading->time - previous->time); - osm_epi_create_node_id(&(epi_pe_data.node_id), guid, port, node->node_name); + osm_epi_create_port_id(&(epi_pe_data.port_id), guid, port, node->node_name); /* calculate changes from previous reading */ epi_pe_data.symbol_err_cnt = (reading->symbol_err_cnt - previous->symbol_err_cnt); @@ -378,7 +378,7 @@ perfmgr_db_add_dc_reading(perfmgr_db_t *db, uint64_t guid, debug_dump_dc_reading(db, guid, port, p_port, reading); epi_dc_data.time_diff_s = (reading->time - previous->time); - osm_epi_create_node_id(&(epi_dc_data.node_id), guid, port, node->node_name); + osm_epi_create_port_id(&(epi_dc_data.port_id), guid, port, node->node_name); /* calculate changes from previous reading */ epi_dc_data.xmit_data = (reading->xmit_data - previous->xmit_data); diff --git a/opensm/osmeventplugin/src/osmeventplugin.c b/opensm/osmeventplugin/src/osmeventplugin.c index d315eb6..9307639 100644 --- a/opensm/osmeventplugin/src/osmeventplugin.c +++ b/opensm/osmeventplugin/src/osmeventplugin.c @@ -108,9 +108,9 @@ handle_port_counter(_log_events_t *log, osm_epi_pe_event_t *pc) || pc->vl15_dropped > 0) { fprintf(log->log_file, "Port counter errors for node 0x%"PRIx64" (%s) port %d\n", - pc->node_id.node_guid, - pc->node_id.node_name, - pc->node_id.port_num); + pc->port_id.node_guid, + pc->port_id.node_name, + pc->port_id.port_num); } } @@ -121,9 +121,9 @@ handle_port_counter_ext(_log_events_t *log, osm_epi_dc_event_t *epc) { fprintf(log->log_file, "Recieved Data counters for node 0x%"PRIx64" (%s) port %d\n", - epc->node_id.node_guid, - epc->node_id.node_name, - epc->node_id.port_num); + epc->port_id.node_guid, + epc->port_id.node_name, + epc->port_id.port_num); } /** ========================================================================= @@ -134,9 +134,9 @@ handle_port_select(_log_events_t *log, osm_epi_ps_event_t *ps) if (ps->xmit_wait > 0) { fprintf(log->log_file, "Port select Xmit Wait counts for node 0x%"PRIx64" (%s) port %d\n", - ps->node_id.node_guid, - ps->node_id.node_name, - ps->node_id.port_num); + ps->port_id.node_guid, + ps->port_id.node_name, + ps->port_id.port_num); } } @@ -148,9 +148,9 @@ handle_trap_event(_log_events_t *log, osm_epi_trap_event_t *trap) fprintf(log->log_file, "Trap event %d from 0x%"PRIx64" (%s) port %d\n", trap->trap_num, - trap->node_id.node_guid, - trap->node_id.node_name, - trap->node_id.port_num); + trap->port_id.node_guid, + trap->port_id.node_name, + trap->port_id.port_num); } /** ========================================================================= -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-the-osm_epi_node_id_t-structure-name.patch Type: application/octet-stream Size: 5856 bytes Desc: not available URL: From weiny2 at llnl.gov Tue Aug 14 18:06:14 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 14 Aug 2007 18:06:14 -0700 Subject: [ofa-general] [PATCH] iblinkinfo.pl: Remove annoying double quote around switch name. Message-ID: <20070814180614.62293fe3.weiny2@llnl.gov> >From fc0251b5150afdd5d40d55facdf85c645939b482 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 14 Aug 2007 18:04:43 -0700 Subject: [PATCH] iblinkinfo.pl: Remove annoying double quote around switch name. Signed-off-by: Ira K. Weiny --- infiniband-diags/scripts/iblinkinfo.pl | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/infiniband-diags/scripts/iblinkinfo.pl b/infiniband-diags/scripts/iblinkinfo.pl index 2413842..73ac585 100755 --- a/infiniband-diags/scripts/iblinkinfo.pl +++ b/infiniband-diags/scripts/iblinkinfo.pl @@ -108,7 +108,7 @@ sub main if ($switch_prompt eq "no" && !$line_mode) { push (@output_lines, - sprintf ("Switch %18s \"%s\"%s:\n", $switch, $hr->{loc_desc}, $pkt_life_prompt)); + sprintf ("Switch %18s %s%s:\n", $switch, $hr->{loc_desc}, $pkt_life_prompt)); $switch_prompt = "yes"; } my $data = `smpquery -G portinfo $switch $port`; -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-iblinkinfo.pl-Remove-annoying-double-quote-around-s.patch Type: application/octet-stream Size: 1034 bytes Desc: not available URL: From kohls at kohls.chtah.com Tue Aug 14 20:15:02 2007 From: kohls at kohls.chtah.com (Kohls.com) Date: Wed, 15 Aug 2007 03:15:02 -0000 Subject: [ofa-general] Back-to-School Savings + Free Shipping Message-ID: http://kohls.chtah.com/a/tBGwmxRBBZVhBBVLYjfBVHokNiC/kohl46-3 ************************************************************************** FREE* Standard Shipping when you spend $75 or more! ************************************************************************** Save an EXTRA 30%, 20% or 15% on EVERYTHING** Shop Wednesday, August 15 through Tuesday, August 21, and save an extra 30%, 20% or 15% on all sale-, regular- and clearance-priced merchandise every time you use your Kohl's Charge in-store and online! It's easy! Printed on your Kohl's Charge mailer is a Kohls.com Promo Code. Just enter this Promo Code when you checkout with your Kohl's Charge and you'll save! Even if you don't have a Promo Code, you'll automatically save an extra 15% with your Kohl's Charge. Visit Kohls.com for complete details. ************************************************************************** School Days Sale We're making back to school fun with the latest fashions and gear at prices that won't bust your budget! Make Kohl's your back-to-school headquarters for apparel, backpacks, dorm decor and more! Shop in-store and online! ************************************************************************** 30-40% Off ENTIRE STOCK Tony Hawk Collection For Young Men For Boys 4-7 For Boys 8-20 20-50% Off ENTIRE STOCK Backpacks & Duffel Bags 50-60% Off ENTIRE STOCK Student Lounge Bedding Bath Dorm Accents, Furniture & Storage ************************************************************************** Find even more deals from other departments! $24.99-$34.99 Juniors' 5-pkt. Jeans, Fashion Jeans & Casual Pants 40-50% Off Young Men's Knit & Woven Tops 20-50% Off Men's Team Apparel 30-40% Off ENTIRE STOCK Chaps Collections For Infants For Boys 4-7 For Boys 8-20 For Girls 4-6x For Girls 7-16 30-40% Off Girls' 7-16 Fashion Tops $14.99-$34.99 Kids' Skechers Shoes ************************************************************************** Beauty - Buy SMOOTH 365 and TRI-AKTILINE and get 50% off any good skin product! ************************************************************************** Back-to-School Shop - My Transformation - Shop for back to school now! ************************************************************************** Back-to-School Gift Card - One Card Fits All! Give them a Gift Card for Back to School! ************************************************************************** 60-80% Clearance Shop Kohls.com Clearance for up to 60-80% off*** original prices on items from every department. But hurry--quantities are limited and deals like these won't last. ************************************************************************** *Surcharges may apply due to size, weight or special handling required. If your item has a surcharge, it will appear on the product page. **Discount is nontransferable. This offer is not valid for price adjustments on prior purchases, as payment toward your Kohl's Charge or in conjunction with other percent-off offers. This offer is also not valid on purchases of Kohl's Gift Cards or on purchases of Kohl's Cares for Kids merchandise. Excludes shipping and sales tax. Subject to credit approval. ***Clearance prices represent savings off original prices. Interim markdowns may have been taken. Sorry, no price adjustments on prior purchases. This mailbox is unattended, so please do not reply to this message. Instead, e-mail us at myaccount.help at kohls.com, or write to us at Kohl's Department Stores, Attention: Customer Service, N56 W17000 Ridgewood Drive, Menomonee Falls, WI 53051. If you no longer wish to receive e-mails from Kohls.com, unsubscribe by pasting this link into the Address field of your Internet browser: http://kohls.chtah.com/a/tBGwmxRBBZVhBBVLYjfBVHokNiC/kohl40-3 School Days Sale prices good August 15-25, 2007. Free Standard Shipping offer good August 15-18, 2007. Kohl's Charge offer good August 15-21, 2007. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Tue Aug 14 21:05:20 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:05:20 -0700 Subject: [ofa-general] [PATCH] ipoib - high dma support In-Reply-To: <1186918971.5556.35.camel@mtls03> (Eli Cohen's message of "Sun, 12 Aug 2007 14:42:51 +0300") References: <1186918971.5556.35.camel@mtls03> Message-ID: Overall this series looks like interesting stuff. However a few general comments: - There seems to be a fair bit of whitespace damage and other formatting glitches. Running scripts/checkpatch.pl on all your patches would be worthwhile. - The changes need to be split up in a better way. For each feature (eg checksum offload), I would have four patches: "add core support for X," which just has the core changes to ib_verbs.h etc; "use X in IPoIB," which uses the feature in IPoIB; and "implement X in mthca" and "implement X in mlx4," which add low-level driver support for the feature. Also more specific comments below: > + pdev = to_pci_dev(hca->dma_device); this doesn't make sense... not all IB devices are PCI devices (eg ehca is not a PCI device). > + if (pdev->dma_mask & DMA_64BIT_MASK) > + priv->dev->features |= NETIF_F_HIGHDMA; I don't think we need this test. The assumption that IB devices can do 64-bit DMA is pretty safe; I think we can enable NETIF_F_HIGHDMA unconditionally. - R. From rdreier at cisco.com Tue Aug 14 21:12:25 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:12:25 -0700 Subject: [ofa-general] Re: [PATCH] ipoib - scatter/gather suopport In-Reply-To: <1186919013.5556.36.camel@mtls03> (Eli Cohen's message of "Sun, 12 Aug 2007 14:43:33 +0300") References: <1186919013.5556.36.camel@mtls03> Message-ID: > + priv->dev->features |= NETIF_F_HIGHDMA | NETIF_F_SG; Does this accomplish anything? I would think the code in net/core/dev.c below would just remove NETIF_F_SG immediately: if ((dev->features & NETIF_F_SG) && !(dev->features & NETIF_F_ALL_CSUM)) { printk(KERN_NOTICE "%s: Dropping NETIF_F_SG since no checksum feature.\n", dev->name); dev->features &= ~NETIF_F_SG; } so is there any point to this change? > - .max_send_sge = 1, > + .max_send_sge = MAX_SKB_FRAGS + 1, this makes WQEs much bigger -- do we want to do this for all devices? Or is it going to hurt on something like Tavor, which doesn't do checksum offload and hence can't take advantage of scatter/gather anyway? > --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-02 11:44:43.000000000 +0300 > +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-02 14:31:02.000000000 +0300 > @@ -495,8 +495,8 @@ static inline int post_send(struct ipoib > { > struct ib_send_wr *bad_wr; > > - priv->tx_sge.addr = addr; > - priv->tx_sge.length = len; > + priv->tx_sge[0].addr = addr; > + priv->tx_sge[0].length = len; Can't connected mode destinations get multi-fragment skbs to send too? In fact it's much more likely given the bigger MTU, although given that we don't have checksum offload for RC sends, we might not be able to enable scatter/gather and CM at the same time. - R. From rdreier at cisco.com Tue Aug 14 21:19:13 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:19:13 -0700 Subject: [ofa-general] Re: [PATCH] ipoib checksum offload In-Reply-To: <1186919051.5556.39.camel@mtls03> (Eli Cohen's message of "Sun, 12 Aug 2007 14:44:11 +0300") References: <1186919051.5556.39.camel@mtls03> Message-ID: > +static DEVICE_ATTR(tx_csum, S_IWUSR | S_IRUGO, show_tx_csum, NULL); Looks like you left out the set_tx_csum function here. But in any case I think checksum features should be set with the standard ethtool interface if it at all possible -- I think adding ethtool support to ipoib would definitely be worthwhile for other reasons too. - R. From kliteyn at mellanox.co.il Tue Aug 14 21:21:42 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 15 Aug 2007 07:21:42 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-15:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_13_00:33:19_2007 [1094c660253d5749331c30c82817e96a6faf939d] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=520 Pass=520 Fail=0 Pass: 39 Stability IS1-16.topo 39 Pkey IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 Pkey IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From rdreier at cisco.com Tue Aug 14 21:22:46 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:22:46 -0700 Subject: [ofa-general] [PATCH] mlx4_ib QP max msg size In-Reply-To: <1186919059.5556.40.camel@mtls03> (Eli Cohen's message of "Sun, 12 Aug 2007 14:44:19 +0300") References: <1186919059.5556.40.camel@mtls03> Message-ID: This needs a better description -- a patch's changelog entry should explain why we want to make a change, what the change does and why it fixes/improves whatever the issue is. For this patch I don't understand why we want to make these changes. > - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || > - ibqp->qp_type == IB_QPT_UD) > + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) > + context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; > + else if (ibqp->qp_type == IB_QPT_UD) > context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; This is especially mysterious: you seem to have changed if (A || B || C) foo(); into if (A || B) foo(); else if (C) foo(); which seems to be exactly equivalent code. From rdreier at cisco.com Tue Aug 14 21:27:48 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:27:48 -0700 Subject: [ofa-general] Re: [PATCH] ipoib large send offload In-Reply-To: <1186919067.5556.41.camel@mtls03> (Eli Cohen's message of "Sun, 12 Aug 2007 14:44:27 +0300") References: <1186919067.5556.41.camel@mtls03> Message-ID: > Add LSO supprt to ipoib This is a perfect example of what I mentioned before about splitting up patches. This patch should be three: "add LSO support to core," "Use LSO in IPoIB" and "implement LSO in mlx4." > + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), Why a new opcode vs. just a send flag? Seems like a strange interface to present to the consumer. > + > + if (wr->opcode == IB_WR_LSO) { > + int halign; > + > + memcpy(((struct mlx4_lso_seg *)wqe)->header, > + wr->wr.ud.header, wr->wr.ud.hlen); > + wmb(); > + ((struct mlx4_lso_seg *)wqe)->mss_hdr_size = > + cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) << 16) | > + wr->wr.ud.hlen); > + > + halign = ALIGN(wr->wr.ud.hlen, 16); > + wqe += halign; > + size += halign >> 4; > + > + if (unlikely(wr->num_sge > qp->sq.max_gs - (halign >> 4))) { > + err = -EINVAL; > + *bad_wr = wr; > + goto out; > + } > + } > + Please try to wrap this up into a helper function so the post send function doesn't get too cluttered. > @@ -1365,6 +1389,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp > ctrl->owner_opcode = mlx4_ib_opcode[wr->opcode] | > (ind & qp->sq.wqe_cnt ? cpu_to_be32(1 << 31) : 0); > > + > /* > * We can improve latency by not stamping the last > * send queue WQE until after ringing the doorbell, so Please try to avoid extranoeous changes like this. > +// printk("%s: [%d] addr = 0x%llx, size = %d\n", __func__, i, addr, frag->size); seems like some debugging code escaped into the wild. - R. From rdreier at cisco.com Tue Aug 14 21:32:15 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 14 Aug 2007 21:32:15 -0700 Subject: [ofa-general] removing PR caching from IPoIB In-Reply-To: <46C21604.9050108@ichips.intel.com> (Sean Hefty's message of "Tue, 14 Aug 2007 13:52:20 -0700") References: <46C0F2DD.6070109@ichips.intel.com> <46C2020A.3080709@ichips.intel.com> <46C21604.9050108@ichips.intel.com> Message-ID: > IPoIB permanently caches PRs with no mechanism in place to keep them > in sync with the SA. The goal is to limit the PR lifetime to prevent > the use of stale paths. > > A side benefit of limiting the PR scope to a neighbour is that > different neighbours could use different paths, versus using a single > path per DGID. OK, that makes sense. However, I don't think we want to do a path record lookup for every unicast ARP send, right? So I think we have to have some table of DGID -> AH mappings independent of the ipoib_neigh stuff. Adding a reference count to the path structure as you mentioned before makes sense to me. The only slightly ugly thing is handling dropping the reference when an ARP send completes, but I guess that should be possible to do somehow. Or were you suggesting using the existing AH reference counting and letting that handle things? That makes sense to me and actually seems pretty elegant. - R. From xma at us.ibm.com Tue Aug 14 22:55:48 2007 From: xma at us.ibm.com (Shirley Ma) Date: Tue, 14 Aug 2007 22:55:48 -0700 Subject: [ofa-general] Re: [PATCH] ipoib checksum offload In-Reply-To: Message-ID: >I think adding ethtool support to ipoib would definitely be worthwhile for other reasons too. Yes, we definitely need ethtool to support other features like GSO, LRO as well. Thanks Shirley -------------- next part -------------- An HTML attachment was scrubbed... URL: From mst at dev.mellanox.co.il Tue Aug 14 23:05:47 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 15 Aug 2007 09:05:47 +0300 Subject: [ofa-general] Re: Can Mellanox cards handle scatter gather I/O? In-Reply-To: <20070814113655.68c317d8@dhcp-243-71.rdu.redhat.com> References: <20070814113655.68c317d8@dhcp-243-71.rdu.redhat.com> Message-ID: <20070815060547.GB24899@mellanox.co.il> > Quoting Kent Baxley : > Subject: Can Mellanox cards handle scatter gather I/O? > > > Evidently, we're seeing performance problems when a card on one side > is capable of scatter gather IO but the mellanox driver is not able to > handle it. Then a whole lot of time is spent gathering the data into > one skbuff. I guess you are speaking about a router configuration where you stick 2 network cards in the same machine - right? > This is currently being observed on RHEL4.5 with OFED 1.2. > > Is the Mellanox card capable of doing scatter gather IO and this just > a driver limitation or is the mellanox card just not able to do scatter > gather IO? AFAIK all Mellanox cards are capable of s/g IO in hardware. However, speaking of IPoIB, the code in net/core/dev.c below would just remove NETIF_F_SG immediately: if ((dev->features & NETIF_F_SG) && !(dev->features & NETIF_F_ALL_CSUM)) { printk(KERN_NOTICE "%s: Dropping NETIF_F_SG since no checksum feature.\n", dev->name); dev->features &= ~NETIF_F_SG; } we can work around this code in OFED, but I think it would make sense to first have a discussion on netdev and lkml list about why is this code there in the first place. -- MST From vlad at lists.openfabrics.org Wed Aug 15 01:39:28 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 15 Aug 2007 01:39:28 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070815-0100 daily build status Message-ID: <20070815083929.14240E608A1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.18 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Wed Aug 15 02:50:52 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 15 Aug 2007 02:50:52 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070815-0200 daily build status Message-ID: <20070815095052.A2B0DE6089B@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070815-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From mst at dev.mellanox.co.il Wed Aug 15 02:53:09 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Wed, 15 Aug 2007 12:53:09 +0300 Subject: [ofa-general] Re: [PATCH] ipoib - scatter/gather suopport In-Reply-To: References: <1186919013.5556.36.camel@mtls03> Message-ID: <20070815095309.GC1995@mellanox.co.il> > > - .max_send_sge = 1, > > + .max_send_sge = MAX_SKB_FRAGS + 1, > > this makes WQEs much bigger -- do we want to do this for all devices? > Or is it going to hurt on something like Tavor, which doesn't do > checksum offload and hence can't take advantage of scatter/gather > anyway? Should be measured, but I would expect the cost to be negligeable. And as was recently mentioned on this list, some users might want to use ethtool to enable s/g even if the device does not support checksum in hardware. How about we disable s/g by default, and let the user enable it with ethtool as appropriate? -- MST From dotanb at dev.mellanox.co.il Wed Aug 15 03:52:41 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 15 Aug 2007 13:52:41 +0300 Subject: [ofa-general] [PATCH] librdmacm 1/2: add valgrind support to auto-tools configuration file Message-ID: <200708151352.42026.dotanb@dev.mellanox.co.il> Added valgrind support to the auto-tools configuration file. Signed-off-by: Dotan Barak --- diff --git a/configure.in b/configure.in index ce15f44..9171319 100644 --- a/configure.in +++ b/configure.in @@ -15,6 +15,19 @@ AC_ARG_ENABLE(libcheck, [ --disable-libcheck do not test for presence of i fi ]) +AC_ARG_WITH([valgrind], + AC_HELP_STRING([--with-valgrind], + [Enable Valgrind annotations (small runtime overhead, default NO)])) +if test x$with_valgrind = x || test x$with_valgrind = xno; then + want_valgrind=no + AC_DEFINE([NVALGRIND], 1, [Define to 1 to disable Valgrind annotations.]) +else + want_valgrind=yes + if test -d $with_valgrind; then + CPPFLAGS="$CPPFLAGS -I$with_valgrind/include" + fi +fi + dnl Checks for programs AC_PROG_CC @@ -37,6 +50,13 @@ AC_CHECK_HEADER(infiniband/verbs.h, [], AC_MSG_ERROR([ not found. Is libibverbs installed?])) fi +AC_CHECK_HEADER(valgrind/memcheck.h, + [AC_DEFINE(HAVE_VALGRIND_MEMCHECK_H, 1, + [Define to 1 if you have the header file.])], + [if test $want_valgrind = yes; then + AC_MSG_ERROR([Valgrind memcheck support requested, but not found.]) + fi]) + AC_CACHE_CHECK(whether ld accepts --version-script, ac_cv_version_script, if test -n "`$LD --help < /dev/null 2>/dev/null | grep version-script`"; then ac_cv_version_script=yes From dotanb at dev.mellanox.co.il Wed Aug 15 03:53:56 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 15 Aug 2007 13:53:56 +0300 Subject: [ofa-general] [PATCH] librdmacm 2/2: add valgrind support to the cma code Message-ID: <200708151353.56687.dotanb@dev.mellanox.co.il> Added valgrind support to the cma code and mark buffers that were filled by the kernel level with the macro VALGRIND_MAKE_MEM_DEFINED. Signed-off-by: Dotan Barak --- diff --git a/src/cma.c b/src/cma.c index 32edc1f..42fa162 100644 --- a/src/cma.c +++ b/src/cma.c @@ -57,6 +57,20 @@ #define PFX "librdmacm: " +#ifdef HAVE_VALGRIND_MEMCHECK_H + +# include + +# ifndef VALGRIND_MAKE_MEM_DEFINED +# warning "Valgrind support requested, but VALGRIND_MAKE_MEM_DEFINED not available" +# endif + +#endif /* HAVE_VALGRIND_MEMCHECK_H */ + +#ifndef VALGRIND_MAKE_MEM_DEFINED +# define VALGRIND_MAKE_MEM_DEFINED(addr,len) +#endif + #if __BYTE_ORDER == __LITTLE_ENDIAN static inline uint64_t htonll(uint64_t x) { return bswap_64(x); } static inline uint64_t ntohll(uint64_t x) { return bswap_64(x); } @@ -383,6 +397,8 @@ int rdma_create_id(struct rdma_event_channel *channel, if (ret != size) goto err; + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + id_priv->handle = resp->id; *id = &id_priv->id; return 0; @@ -405,6 +421,8 @@ static int ucma_destroy_kern_id(int fd, uint32_t handle) if (ret != size) return (ret > 0) ? -ENODATA : ret; + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + return resp->events_reported; } @@ -458,6 +476,8 @@ static int ucma_query_route(struct rdma_cm_id *id) if (ret != size) return (ret > 0) ? -ENODATA : ret; + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + if (resp->num_paths) { id->route.path_rec = malloc(sizeof *id->route.path_rec * resp->num_paths); @@ -583,6 +603,8 @@ static int rdma_init_qp_attr(struct rdma_cm_id *id, struct ibv_qp_attr *qp_attr, if (ret != size) return (ret > 0) ? -ENODATA : ret; + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + ibv_copy_qp_attr_from_kern(qp_attr, resp); *qp_attr_mask = resp->qp_attr_mask; return 0; @@ -1010,6 +1032,8 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr, goto err2; } + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + mc->handle = resp->id; return 0; err2: @@ -1061,6 +1085,8 @@ int rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) goto free; } + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); + pthread_mutex_lock(&id_priv->mut); while (mc->events_completed < resp->events_reported) pthread_cond_wait(&mc->cond, &id_priv->mut); @@ -1255,6 +1281,8 @@ retry: free(evt); return (ret > 0) ? -ENODATA : ret; } + + VALGRIND_MAKE_MEM_DEFINED(resp, sizeof *resp); evt->event.event = resp->event; switch (resp->event) { From hal.rosenstock at gmail.com Wed Aug 15 05:07:39 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 15 Aug 2007 08:07:39 -0400 Subject: [ofa-general] [RFC] the never ending search for SA scalability In-Reply-To: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> References: <000301c7dab5$d2648fd0$ff0da8c0@amr.corp.intel.com> Message-ID: On 8/9/07, Sean Hefty wrote: > I'd like to propose the following change as a simple solution for handling SA > scalability problems: > > Modify the ib_sa module to support an SA LID that's separate from the SM LID. > > This concept is supported by the spec through SA redirection; however, I propose > that we also allow the SA LID to be set manually by an administrator. > Additional details are below. > > --- > > The SA LID can be set to a local or remote LID - it doesn't matter to the > kernel. All SA MADs (PR queries, MC joins, event registration, etc.) would be > sent to that destination for processing. > Initially, I envision a user space library capable of responding to PR queries, > but it could be expanded to respond to other types of requests. How the library > responds to requests (forwarding them to the SM/SA, using lookup tables, etc.) > is outside the scope of the proposal. Other than PRs, what SA requests are planned to be handled without reforwarding to the "real" SM/SA ? Is it just PRs ? Even PRs in QoS mode will be a challenge and likely be forwarded. -- Hal > - Sean > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From pradeeps at linux.vnet.ibm.com Wed Aug 15 07:15:12 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Wed, 15 Aug 2007 07:15:12 -0700 Subject: [ofa-general] [PATCH][RFC] Handle packet received on RQ of tx_qp with NOSRQ In-Reply-To: References: <46B3C97F.6030007@linux.vnet.ibm.com> <46BB365F.6070604@linux.vnet.ibm.com> <20070809155144.GA32251@mellanox.co.il> <46BBFCF3.1070109@linux.vnet.ibm.com> <46BFF288.5020304@linux.vnet.ibm.com> <46C0F2A7.5010807@linux.vnet.ibm.com> <46C1F95D.2050508@linux.vnet.ibm.com> <46C20F2F.1050003@linux.vnet.ibm.com> Message-ID: <46C30A70.1020102@linux.vnet.ibm.com> Roland Dreier wrote: > > I am implementing this only for completeness sake and want to do it with minimal > > effort. Given the above, do you still see it necessary to use NAPI? > > > Is it acceptable that I roll up the previous NOSRQ patches (other than this one) into > > a single patch and can that one be integrated first? > > Given that you don't want to actually have something that really > works, I guess there's no point in doing it at all. Since there are > no other IPoIB CM implementations out there yet, maybe it's acceptable > to say that we don't implement a fully RFC-compliant version of the > protocol. > I believe you may have misinterpreted. If the existing handlers cannot be reused, then I have reservations about having to come up with a complete parallel implementation that will deal with all the issues covered previously -memory usage, falling back to UD mode and the like for what I believe will be seldom used. I therefore want to come up with a "minimal" implementation that will be RFC compliant. Pradeep From swise at opengridcomputing.com Wed Aug 15 07:42:41 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 15 Aug 2007 09:42:41 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070809.145534.102938208.davem@davemloft.net> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> <20070809.145534.102938208.davem@davemloft.net> Message-ID: <46C310E1.7020503@opengridcomputing.com> David Miller wrote: > From: Sean Hefty > Date: Thu, 09 Aug 2007 14:40:16 -0700 > >> Steve Wise wrote: >>> Any more comments? >> Does anyone have ideas on how to reserve the port space without using a >> struct socket? > > How about we just remove the RDMA stack altogether? I am not at all > kidding. If you guys can't stay in your sand box and need to cause > problems for the normal network stack, it's unacceptable. We were > told all along the if RDMA went into the tree none of this kind of > stuff would be an issue. I think removing the RDMA stack is the wrong thing to do, and you shouldn't just threaten to yank entire subsystems because you don't like the technology. Lets keep this constructive, can we? RDMA should get the respect of any other technology in Linux. Maybe its a niche in your opinion, but come on, there's more RDMA users than say, the sparc64 port. Eh? > > These are exactly the kinds of problems for which people like myself > were dreading. These subsystems have no buisness using the TCP port > space of the Linux software stack, absolutely none. > Ok, although IMO its the correct solution. But I'll propose other solutions below. I ask for your feedback (and everyones!) on these alternate solutions. > After TCP port reservation, what's next? It seems an at least > bi-monthly event that the RDMA folks need to put their fingers > into something else in the normal networking stack. No more. > The only other change requested and commited, if I recall correctly, was for netevents, and that enabled both Infiniband and iWARP to integrate with the neighbour subsystem. I think that was a useful and needed change. Prior to that, these subsystems were snooping ARP replies to trigger events. That was back in 2.6.18 or 2.6.19 I think... > I will NACK any patch that opens up sockets to eat up ports or > anything stupid like that. Got it. Here are alternate solutions that avoid the need to share the port space: Solution 1) 1) admins must setup an alias interface on the iwarp device for use with rdma. This interface will have to be a separate subnet from the "TCP used" interface. And with a canonical name that indicates its "for rdma only". Like eth2:iw or eth2:rdma. There can be many of these per device. 2) admins make sure their sockets/tcp services don't use the interface configured in #1, and their rdma service do use said interface. 3) iwarp providers must translation binds to ipaddr 0.0.0.0 to the associated "for rdma only" ip addresses. They can do this by searching for all aliases of the canonical name that are aliases of the TCP interface for their nic device. Or: somehow not handle incoming connections to any address but the "for rdma use" addresses and instead pass them up and not offload them. This will avoid the collisions as long as the above steps are followed. Solution 2) Another possibility would be for the driver to create two net devices (and hence two interace names) like "eth2" and "iw2", and artificially separate the RDMA stuff that way. These two solutions are similar in that they create a "rdma only" interface. Pros: - is not intrusive into the core networking code - very minimal changes needed and in the iwarp provider's code, who are the ones with this problem - makes it clear which subnets are RDMA only Cons: - relies on system admin to set it up correctly. - native stack can still "use" this rdma-only interface and the same port space issue will exist. For the record, here are possible port-sharing solutions Dave sez he'll NAK: Solution NAK-1) The rdma-cma just allocates a socket and binds it to reserve TCP ports. Pros: - minimal changes needed to implement (always a plus in my mind :) - simple, clean, and it works (KISS) - if no RDMA is in use, there is no impact on the native stack - no need for a seperate RDMA interface Cons: - wastes memory - puts a TCP socket in the "CLOSED" state in the pcb tables. - Dave will NAK it :) Solution NAK-2) Create a low-level sockets-agnostic port allocation service that is shared by both TCP and RDMA. This way, the rdma-cm can reserve ports in an efficient manor instead of doing it via kernel_bind() using a sock struct. Pros: - probably the correct solution (my opinion :) if we went down the path of sharing port space - if no RDMA is in use, there is no impact on the native stack - no need for a separate RDMA interface Cons: - very intrusive change because the port allocations stuff is tightly bound to the host stack and sock struct, etc. - Dave will NAK it :) Steve. From chas at cmf.nrl.navy.mil Wed Aug 15 07:58:06 2007 From: chas at cmf.nrl.navy.mil (chas williams - CONTRACTOR) Date: Wed, 15 Aug 2007 10:58:06 -0400 Subject: [ofa-general] Re: Can Mellanox cards handle scatter gather I/O? In-Reply-To: <20070815060547.GB24899@mellanox.co.il> Message-ID: <200708151458.l7FEw6jX008617@cmf.nrl.navy.mil> In message <20070815060547.GB24899 at mellanox.co.il>,"Michael S. Tsirkin" writes: > if ((dev->features & NETIF_F_SG) && > !(dev->features & NETIF_F_ALL_CSUM)) { > printk(KERN_NOTICE "%s: Dropping NETIF_F_SG since no checksum >feature.\n", > dev->name); > dev->features &= ~NETIF_F_SG; > } > >we can work around this code in OFED, but I think it would >make sense to first have a discussion on netdev and lkml list about why is >this code there in the first place. as i recall, preferred software combined copy and checksum routine doesnt typically handle scatter/gather. From dotanb at dev.mellanox.co.il Wed Aug 15 08:00:21 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Wed, 15 Aug 2007 18:00:21 +0300 Subject: [ofa-general] [PATCH] libibverbs/devinfo: free the device list to prevent memory leaks Message-ID: <200708151800.21678.dotanb@dev.mellanox.co.il> Free the device list to prevent memory leaks (and give a good code example). Signed-off-by: Dotan Barak --- diff --git a/examples/devinfo.c b/examples/devinfo.c index d054999..4e4316a 100644 --- a/examples/devinfo.c +++ b/examples/devinfo.c @@ -323,7 +323,7 @@ int main(int argc, char *argv[]) { char *ib_devname = NULL; int ret = 0; - struct ibv_device **dev_list; + struct ibv_device **dev_list, **orig_dev_list; int num_of_hcas; int ib_port = 0; @@ -360,7 +360,7 @@ int main(int argc, char *argv[]) break; case 'l': - dev_list = ibv_get_device_list(&num_of_hcas); + dev_list = orig_dev_list = ibv_get_device_list(&num_of_hcas); if (!dev_list) { fprintf(stderr, "Failed to get IB devices list"); return -1; @@ -375,6 +375,9 @@ int main(int argc, char *argv[]) } printf("\n"); + + ibv_free_device_list(orig_dev_list); + return 0; default: @@ -383,7 +386,7 @@ int main(int argc, char *argv[]) } } - dev_list = ibv_get_device_list(NULL); + dev_list = orig_dev_list = ibv_get_device_list(NULL); if (!dev_list) { fprintf(stderr, "Failed to get IB device list\n"); return -1; @@ -417,5 +420,7 @@ int main(int argc, char *argv[]) if (ib_devname) free(ib_devname); + ibv_free_device_list(orig_dev_list); + return ret; } From ralph.campbell at qlogic.com Wed Aug 15 09:38:32 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Wed, 15 Aug 2007 09:38:32 -0700 Subject: [ofa-general] Re: Can Mellanox cards handle scatter gather I/O? In-Reply-To: <20070815060547.GB24899@mellanox.co.il> References: <20070814113655.68c317d8@dhcp-243-71.rdu.redhat.com> <20070815060547.GB24899@mellanox.co.il> Message-ID: <1187195912.6293.541.camel@brick.pathscale.com> On Wed, 2007-08-15 at 09:05 +0300, Michael S. Tsirkin wrote: > > Quoting Kent Baxley : > > Subject: Can Mellanox cards handle scatter gather I/O? > > > > > > Evidently, we're seeing performance problems when a card on one side > > is capable of scatter gather IO but the mellanox driver is not able to > > handle it. Then a whole lot of time is spent gathering the data into > > one skbuff. > > I guess you are speaking about a router configuration where > you stick 2 network cards in the same machine - right? > > > This is currently being observed on RHEL4.5 with OFED 1.2. > > > > Is the Mellanox card capable of doing scatter gather IO and this just > > a driver limitation or is the mellanox card just not able to do scatter > > gather IO? > > AFAIK all Mellanox cards are capable of s/g IO in hardware. > > However, speaking of IPoIB, the code in > net/core/dev.c below would just remove NETIF_F_SG immediately: > > if ((dev->features & NETIF_F_SG) && > !(dev->features & NETIF_F_ALL_CSUM)) { > printk(KERN_NOTICE "%s: Dropping NETIF_F_SG since no checksum feature.\n", > dev->name); > dev->features &= ~NETIF_F_SG; > } > > we can work around this code in OFED, but I think it would > make sense to first have a discussion on netdev and lkml list about why is > this code there in the first place. I believe earlier Linux kernels forced HW checksum off unless the device also supported scatter/gather. Recent kernels have changed the code so that the two features can be separately turned on/off. From eli at mellanox.co.il Wed Aug 15 11:03:39 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:03:39 +0300 Subject: [ofa-general] offload series of patches Message-ID: <1187201019.16253.165.camel@mtls03> Thank you all for all the comments! - and I know there will be more. I have reworked the patches so that they pass the checkpatch.pl barrier (except for one case), split the patches to core, ulp or hw driver and in some cases referred to your comments on the implementation. Also they are self contained and do not "do preparations" for subsequent patches. However they do need to be applied in order to be cleanly applied. The patches are have [PATH - n] in the subject field to define the order. They are now built with the following dependencies enforced: checksum, scatter/gather, tso. If a feature can't be used, the following will not be used. From eli at mellanox.co.il Wed Aug 15 11:03:44 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:03:44 +0300 Subject: [ofa-general] [PATCH - 1] ipoib - add high dma support Message-ID: <1187201024.16253.166.camel@mtls03> Add high dma support to ipoib This patch assumes all IB devices support 64 bit DMA. Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 09:08:14.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 09:11:13.000000000 +0300 @@ -1079,6 +1079,8 @@ static struct net_device *ipoib_add_port SET_NETDEV_DEV(priv->dev, hca->dma_device); + priv->dev->features |= NETIF_F_HIGHDMA; + result = ib_query_pkey(hca, port, 0, &priv->pkey); if (result) { printk(KERN_WARNING "%s: ib_query_pkey port %d failed (ret = %d)\n", From eli at mellanox.co.il Wed Aug 15 11:05:16 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:05:16 +0300 Subject: [ofa-general] [PATCH -2] ib core - checksum offload Message-ID: <1187201116.16253.168.camel@mtls03> Add checksum offload support to the core Signed-off-by: Eli Cohen --- A device that publishes IB_DEVICE_IP_CSUM actually supports calculating checksum on transmit and provides indication whether the checksum is OK on receive. Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h =================================================================== --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-08-15 20:50:16.000000000 +0300 +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-15 20:50:28.000000000 +0300 @@ -93,7 +93,8 @@ enum ib_device_cap_flags { IB_DEVICE_N_NOTIFY_CQ = (1<<14), IB_DEVICE_ZERO_STAG = (1<<15), IB_DEVICE_SEND_W_INV = (1<<16), - IB_DEVICE_MEM_WINDOW = (1<<17) + IB_DEVICE_MEM_WINDOW = (1<<17), + IB_DEVICE_IP_CSUM = (1<<18) }; enum ib_atomic_cap { @@ -429,6 +430,8 @@ struct ib_wc { u8 sl; u8 dlid_path_bits; u8 port_num; /* valid only for DR SMPs on switches */ + u16 csum; + int csum_ok; }; enum ib_cq_notify_flags { @@ -613,7 +616,9 @@ enum ib_send_flags { IB_SEND_FENCE = 1, IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), - IB_SEND_INLINE = (1<<3) + IB_SEND_INLINE = (1<<3), + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_UDP_TCP_CSUM = (1<<5) }; struct ib_sge { From eli at mellanox.co.il Wed Aug 15 11:11:35 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:11:35 +0300 Subject: [ofa-general] [PATCH -2] ib core - checksum offload Message-ID: <1187201495.16253.173.camel@mtls03> Add checksum offload support to the core Signed-off-by: Eli Cohen --- resending this one - I suspect the previous one had formatting problems. A device that publishes IB_DEVICE_IP_CSUM actually supports calculating checksum on transmit and provides indication whether the checksum is OK on receive. Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h =================================================================== --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-08-15 20:50:16.000000000 +0300 +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-15 20:50:28.000000000 +0300 @@ -93,7 +93,8 @@ enum ib_device_cap_flags { IB_DEVICE_N_NOTIFY_CQ = (1<<14), IB_DEVICE_ZERO_STAG = (1<<15), IB_DEVICE_SEND_W_INV = (1<<16), - IB_DEVICE_MEM_WINDOW = (1<<17) + IB_DEVICE_MEM_WINDOW = (1<<17), + IB_DEVICE_IP_CSUM = (1<<18) }; enum ib_atomic_cap { @@ -429,6 +430,8 @@ struct ib_wc { u8 sl; u8 dlid_path_bits; u8 port_num; /* valid only for DR SMPs on switches */ + u16 csum; + int csum_ok; }; enum ib_cq_notify_flags { @@ -613,7 +616,9 @@ enum ib_send_flags { IB_SEND_FENCE = 1, IB_SEND_SIGNALED = (1<<1), IB_SEND_SOLICITED = (1<<2), - IB_SEND_INLINE = (1<<3) + IB_SEND_INLINE = (1<<3), + IB_SEND_IP_CSUM = (1<<4), + IB_SEND_UDP_TCP_CSUM = (1<<5) }; struct ib_sge { From eli at mellanox.co.il Wed Aug 15 11:12:32 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:12:32 +0300 Subject: [ofa-general] [PATCH -3] ib_mthca checksum offload support Message-ID: <1187201552.16253.175.camel@mtls03> Add checksum offload support in mthca Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-15 20:50:29.000000000 +0300 @@ -1377,6 +1377,9 @@ int mthca_INIT_HCA(struct mthca_dev *dev MTHCA_PUT(inbox, param->uarc_base, INIT_HCA_UAR_CTX_BASE_OFFSET); } + if (dev->device_cap_flags & IB_DEVICE_IP_CSUM) + *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) |= cpu_to_be32(7 << 3); + err = mthca_cmd(dev, mailbox->dma, 0, 0, CMD_INIT_HCA, HZ, status); mthca_free_mailbox(dev, mailbox); Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-15 20:50:29.000000000 +0300 @@ -103,6 +103,7 @@ enum { DEV_LIM_FLAG_RAW_IPV6 = 1 << 4, DEV_LIM_FLAG_RAW_ETHER = 1 << 5, DEV_LIM_FLAG_SRQ = 1 << 6, + DEV_LIM_FLAG_IPOIB_CSUM = 1 << 7, DEV_LIM_FLAG_BAD_PKEY_CNTR = 1 << 8, DEV_LIM_FLAG_BAD_QKEY_CNTR = 1 << 9, DEV_LIM_FLAG_MW = 1 << 16, Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-15 20:50:29.000000000 +0300 @@ -119,7 +119,8 @@ struct mthca_cqe { __be32 my_qpn; __be32 my_ee; __be32 rqpn; - __be16 sl_g_mlpath; + u8 sl_ipok; + u8 g_mlpath; __be16 rlid; __be32 imm_etype_pkey_eec; __be32 byte_cnt; @@ -498,6 +499,7 @@ static inline int mthca_poll_one(struct int is_send; int free_cqe = 1; int err = 0; + u16 checksum; cqe = next_cqe_sw(cq); if (!cqe) @@ -646,6 +648,10 @@ static inline int mthca_poll_one(struct entry->pkey_index = be32_to_cpu(cqe->imm_etype_pkey_eec) >> 16; entry->wc_flags |= be16_to_cpu(cqe->sl_g_mlpath) & 0x80 ? IB_WC_GRH : 0; + checksum = (be32_to_cpu(cqe->rqpn) >> 24) | + ((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00); + entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff); + entry->csum = checksum; } entry->status = IB_WC_SUCCESS; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-15 20:50:29.000000000 +0300 @@ -289,6 +289,10 @@ static int mthca_dev_lim(struct mthca_de if (dev_lim->flags & DEV_LIM_FLAG_SRQ) mdev->mthca_flags |= MTHCA_FLAG_SRQ; + if (mthca_is_memfree(mdev)) + if (dev_lim->flags & DEV_LIM_FLAG_IPOIB_CSUM) + mdev->device_cap_flags |= IB_DEVICE_IP_CSUM; + return 0; } Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-15 20:50:29.000000000 +0300 @@ -2024,6 +2024,10 @@ int mthca_arbel_post_send(struct ib_qp * cpu_to_be32(MTHCA_NEXT_CQ_UPDATE) : 0) | ((wr->send_flags & IB_SEND_SOLICITED) ? cpu_to_be32(MTHCA_NEXT_SOLICIT) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_IP_CSUM) : 0) | + ((wr->send_flags & IB_SEND_UDP_TCP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_TCP_UDP_CSUM) : 0) | cpu_to_be32(1); if (wr->opcode == IB_WR_SEND_WITH_IMM || wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-15 20:50:29.000000000 +0300 @@ -38,14 +38,15 @@ #include enum { - MTHCA_NEXT_DBD = 1 << 7, - MTHCA_NEXT_FENCE = 1 << 6, - MTHCA_NEXT_CQ_UPDATE = 1 << 3, - MTHCA_NEXT_EVENT_GEN = 1 << 2, - MTHCA_NEXT_SOLICIT = 1 << 1, - - MTHCA_MLX_VL15 = 1 << 17, - MTHCA_MLX_SLR = 1 << 16 + MTHCA_NEXT_DBD = 1 << 7, + MTHCA_NEXT_FENCE = 1 << 6, + MTHCA_NEXT_CQ_UPDATE = 1 << 3, + MTHCA_NEXT_EVENT_GEN = 1 << 2, + MTHCA_NEXT_SOLICIT = 1 << 1, + MTHCA_NEXT_IP_CSUM = 1 << 4, + MTHCA_NEXT_TCP_UDP_CSUM = 1 << 5, + MTHCA_MLX_VL15 = 1 << 17, + MTHCA_MLX_SLR = 1 << 16 }; enum { From eli at mellanox.co.il Wed Aug 15 11:13:27 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:13:27 +0300 Subject: [ofa-general] [PATCH -4] mlx4 - checksum offload Message-ID: <1187201607.16253.177.camel@mtls03> Add checksum offload support to mlx4 Signed-off-by: Ali Ayub Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/include/linux/mlx4/cq.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/cq.h 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/cq.h 2007-08-15 20:50:30.000000000 +0300 @@ -45,11 +45,11 @@ struct mlx4_cqe { u8 sl; u8 reserved1; __be16 rlid; - u32 reserved2; + __be32 ipoib_status; __be32 byte_cnt; __be16 wqe_index; __be16 checksum; - u8 reserved3[3]; + u8 reserved2[3]; u8 owner_sr_opcode; }; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/cq.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c 2007-08-15 20:50:30.000000000 +0300 @@ -431,6 +431,9 @@ static int mlx4_ib_poll_one(struct mlx4_ wc->wc_flags |= be32_to_cpu(cqe->g_mlpath_rqpn) & 0x80000000 ? IB_WC_GRH : 0; wc->pkey_index = be32_to_cpu(cqe->immed_rss_invalid) >> 16; + wc->csum = be16_to_cpu(cqe->checksum); + wc->csum_ok = be32_to_cpu(cqe->ipoib_status) & 0x10000000 && + wc->csum == 0xffff; } return 0; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/main.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c 2007-08-15 20:50:30.000000000 +0300 @@ -99,6 +99,8 @@ static int mlx4_ib_query_device(struct i props->device_cap_flags |= IB_DEVICE_AUTO_PATH_MIG; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_UD_AV_PORT) props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; + if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + props->device_cap_flags |= IB_DEVICE_IP_CSUM; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -568,6 +570,9 @@ static void *mlx4_ib_add(struct mlx4_dev ibdev->ib_dev.detach_mcast = mlx4_ib_mcg_detach; ibdev->ib_dev.process_mad = mlx4_ib_process_mad; + if (ibdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + ibdev->ib_dev.flags |= IB_DEVICE_IP_CSUM; + if (init_node_data(ibdev)) goto err_map; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:30.000000000 +0300 @@ -1258,6 +1258,10 @@ int mlx4_ib_post_send(struct ib_qp *ibqp cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE) : 0) | (wr->send_flags & IB_SEND_SOLICITED ? cpu_to_be32(MLX4_WQE_CTRL_SOLICITED) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM) : 0) | + ((wr->send_flags & IB_SEND_UDP_TCP_CSUM) ? + cpu_to_be32(MLX4_WQE_CTRL_TCP_UDP_CSUM) : 0) | qp->sq_signal_bits; if (wr->opcode == IB_WR_SEND_WITH_IMM || Index: linux-2.6.23-rc1/include/linux/mlx4/qp.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/qp.h 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/qp.h 2007-08-15 20:50:30.000000000 +0300 @@ -155,9 +155,11 @@ struct mlx4_qp_context { }; enum { - MLX4_WQE_CTRL_FENCE = 1 << 6, - MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, - MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_FENCE = 1 << 6, + MLX4_WQE_CTRL_CQ_UPDATE = 3 << 2, + MLX4_WQE_CTRL_SOLICITED = 1 << 1, + MLX4_WQE_CTRL_IP_CSUM = 1 << 4, + MLX4_WQE_CTRL_TCP_UDP_CSUM = 1 << 5, }; struct mlx4_wqe_ctrl_seg { Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.c 2007-08-15 20:50:15.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.c 2007-08-15 20:50:30.000000000 +0300 @@ -736,6 +736,9 @@ int mlx4_INIT_HCA(struct mlx4_dev *dev, MLX4_PUT(inbox, (u8) (PAGE_SHIFT - 12), INIT_HCA_UAR_PAGE_SZ_OFFSET); MLX4_PUT(inbox, param->log_uar_sz, INIT_HCA_LOG_UAR_SZ_OFFSET); + if (dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) + *(inbox + INIT_HCA_FLAGS_OFFSET / 4) |= cpu_to_be32(1 << 3); + err = mlx4_cmd(dev, mailbox->dma, 0, 0, MLX4_CMD_INIT_HCA, 1000); if (err) From eli at mellanox.co.il Wed Aug 15 11:14:18 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:14:18 +0300 Subject: [ofa-general] [PATCH - 5] ipoib checksum offlad Message-ID: <1187201658.16253.179.camel@mtls03> Add checksum offload support to ipoib Signed-off-by: Eli Cohen Signed-off-by: Ali Ayub --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:14.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:31.000000000 +0300 @@ -86,6 +86,7 @@ enum { IPOIB_MCAST_STARTED = 8, IPOIB_FLAG_NETIF_STOPPED = 9, IPOIB_FLAG_ADMIN_CM = 10, + IPOIB_FLAG_RX_CSUM = 11, IPOIB_MAX_BACKOFF_SECONDS = 16, Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 20:50:14.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 20:50:31.000000000 +0300 @@ -1252,6 +1252,8 @@ static ssize_t set_mode(struct device *d /* flush paths if we switch modes so that connections are restarted */ if (IPOIB_CM_SUPPORTED(dev->dev_addr) && !strcmp(buf, "connected\n")) { set_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); + /* on CM mode, turn off tx_csum offloading */ + dev->features &= ~NETIF_F_HW_CSUM; ipoib_warn(priv, "enabling connected mode " "will cause multicast packet drops\n"); ipoib_flush_paths(dev); Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:14.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:31.000000000 +0300 @@ -37,6 +37,7 @@ #include #include +#include #include @@ -231,6 +232,14 @@ static void ipoib_ib_handle_rx_wc(struct skb->dev = dev; /* XXX get correct PACKET_ type here */ skb->pkt_type = PACKET_HOST; + + /* check rx csum */ + if (test_bit(IPOIB_FLAG_RX_CSUM, &priv->flags)) + if (likely(wc->csum_ok)) { + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum = wc->csum; + } + netif_receive_skb(skb); repost: @@ -394,6 +403,16 @@ void ipoib_send(struct net_device *dev, } tx_req->mapping = addr; + if (dev->features & NETIF_F_HW_CSUM) { + if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) + priv->tx_wr.send_flags |= + IB_SEND_UDP_TCP_CSUM | IB_SEND_IP_CSUM; + else + priv->tx_wr.send_flags &= + ~(IB_SEND_UDP_TCP_CSUM | IB_SEND_IP_CSUM); + } + + if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, addr, skb->len))) { ipoib_warn(priv, "post_send failed\n"); Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:27.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:31.000000000 +0300 @@ -1067,6 +1067,29 @@ int ipoib_add_pkey_attr(struct net_devic return device_create_file(&dev->dev, &dev_attr_pkey); } +static void set_tx_csum(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (test_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags)) + return; + + if (!(priv->ca->flags & IB_DEVICE_IP_CSUM)) + return; + + dev->features |= NETIF_F_HW_CSUM; +} + +static void set_rx_csum(struct net_device *dev) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (!(priv->ca->flags & IB_DEVICE_IP_CSUM)) + return; + + set_bit(IPOIB_FLAG_RX_CSUM, &priv->flags); +} + static struct net_device *ipoib_add_port(const char *format, struct ib_device *hca, u8 port) { @@ -1123,6 +1146,9 @@ static struct net_device *ipoib_add_port goto event_failed; } + set_tx_csum(priv->dev); + set_rx_csum(priv->dev); + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", From eli at mellanox.co.il Wed Aug 15 11:15:24 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:15:24 +0300 Subject: [ofa-general] [PATCH -6] ipoib - fix typo Message-ID: <1187201724.16253.182.camel@mtls03> Fix type - comma instead of semicolon Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:14.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:32.000000000 +0300 @@ -185,7 +185,7 @@ int ipoib_transport_dev_init(struct net_ goto out_free_cq; init_attr.send_cq = priv->cq; - init_attr.recv_cq = priv->cq, + init_attr.recv_cq = priv->cq; priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { From eli at mellanox.co.il Wed Aug 15 11:16:14 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:16:14 +0300 Subject: [ofa-general] [PATCH - 6] ipoib scatter/gather Message-ID: <1187201774.16253.183.camel@mtls03> Add scatter/gather support to ipoib Signed-off-by: Eli Cohen --- If the net device does not use scatter/gather then the QP uses a single send sge. Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:31.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:33.000000000 +0300 @@ -121,9 +121,14 @@ struct ipoib_rx_buf { u64 mapping; }; +struct ipoib_mapping_st { + u64 addr; + u16 size; +}; + struct ipoib_tx_buf { struct sk_buff *skb; - u64 mapping; + struct ipoib_mapping_st mapping[MAX_SKB_FRAGS + 1]; }; struct ib_cm_id; @@ -270,7 +275,7 @@ struct ipoib_dev_priv { struct ipoib_tx_buf *tx_ring; unsigned tx_head; unsigned tx_tail; - struct ib_sge tx_sge; + struct ib_sge tx_sge[MAX_SKB_FRAGS + 1]; struct ib_send_wr tx_wr; struct ib_wc ibwc[IPOIB_NUM_WC]; Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:31.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:33.000000000 +0300 @@ -248,6 +248,24 @@ repost: "for buf %d\n", wr_id); } +static int dma_unmap_list(struct ib_device *ca, struct ipoib_mapping_st *map, + u16 n) +{ + int i; + int len; + + ib_dma_unmap_single(ca, map[0].addr, map[0].size, DMA_TO_DEVICE); + len = map[0].size; + + for (i = 1; i < n; ++i) { + ib_dma_unmap_page(ca, map[i].addr, map[i].size, + DMA_TO_DEVICE); + len += map[i].size; + } + + return len; +} + static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -265,12 +283,9 @@ static void ipoib_ib_handle_tx_wc(struct } tx_req = &priv->tx_ring[wr_id]; - - ib_dma_unmap_single(priv->ca, tx_req->mapping, - tx_req->skb->len, DMA_TO_DEVICE); - + priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags + 1); ++priv->stats.tx_packets; - priv->stats.tx_bytes += tx_req->skb->len; dev_kfree_skb_any(tx_req->skb); @@ -352,13 +367,17 @@ void ipoib_ib_completion(struct ib_cq *c static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 addr, int len) + struct ipoib_mapping_st *mapping, int ngather) { struct ib_send_wr *bad_wr; + int i; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + for (i = 0; i < ngather; ++i) { + priv->tx_sge[i].addr = mapping[i].addr; + priv->tx_sge[i].length = mapping[i].size; + } + priv->tx_wr.num_sge = ngather; priv->tx_wr.wr_id = wr_id; priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; @@ -371,7 +390,9 @@ void ipoib_send(struct net_device *dev, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; + struct skb_frag_struct *frag; u64 addr; + unsigned short i; if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -394,14 +415,27 @@ void ipoib_send(struct net_device *dev, */ tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, + addr = ib_dma_map_single(priv->ca, skb->data, skb_headlen(skb), DMA_TO_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); return; } - tx_req->mapping = addr; + + tx_req->mapping[0].addr = addr; + tx_req->mapping[0].size = skb_headlen(skb); + + for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { + frag = &skb_shinfo(skb)->frags[i]; + addr = ib_dma_map_page(priv->ca, frag->page, frag->page_offset, + frag->size, DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(priv->ca, addr))) + goto map_err; + + tx_req->mapping[i + 1].addr = addr; + tx_req->mapping[i + 1].size = frag->size; + } if (dev->features & NETIF_F_HW_CSUM) { if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) @@ -414,11 +448,10 @@ void ipoib_send(struct net_device *dev, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, addr, skb->len))) { + address->ah, qpn, tx_req->mapping, + skb_shinfo(skb)->nr_frags + 1))) { ipoib_warn(priv, "post_send failed\n"); - ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); - dev_kfree_skb_any(skb); + goto map_err; } else { dev->trans_start = jiffies; @@ -431,6 +464,11 @@ void ipoib_send(struct net_device *dev, set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); } } + return; + +map_err: + dma_unmap_list(priv->ca, tx_req->mapping, i + 1); + dev_kfree_skb_any(skb); } static void __ipoib_reap_ah(struct net_device *dev) @@ -593,6 +631,7 @@ int ipoib_ib_dev_stop(struct net_device struct ib_qp_attr qp_attr; unsigned long begin; struct ipoib_tx_buf *tx_req; + struct sk_buff *skb; int i; clear_bit(IPOIB_FLAG_INITIALIZED, &priv->flags); @@ -623,11 +662,10 @@ int ipoib_ib_dev_stop(struct net_device while ((int) priv->tx_tail - (int) priv->tx_head < 0) { tx_req = &priv->tx_ring[priv->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, - tx_req->mapping, - tx_req->skb->len, - DMA_TO_DEVICE); - dev_kfree_skb_any(tx_req->skb); + skb = tx_req->skb; + dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(skb)->nr_frags + 1); + dev_kfree_skb_any(skb); ++priv->tx_tail; } @@ -635,13 +673,14 @@ int ipoib_ib_dev_stop(struct net_device struct ipoib_rx_buf *rx_req; rx_req = &priv->rx_ring[i]; - if (!rx_req->skb) + skb = rx_req->skb; + if (!skb) continue; ib_dma_unmap_single(priv->ca, rx_req->mapping, IPOIB_BUF_SIZE, DMA_FROM_DEVICE); - dev_kfree_skb_any(rx_req->skb); + dev_kfree_skb_any(skb); rx_req->skb = NULL; } Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:32.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:33.000000000 +0300 @@ -149,14 +149,15 @@ int ipoib_transport_dev_init(struct net_ .cap = { .max_send_wr = ipoib_sendq_size, .max_recv_wr = ipoib_recvq_size, - .max_send_sge = 1, .max_recv_sge = 1 }, .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_UD }; - int ret, size; + int ret; + int size; + int i; priv->pd = ib_alloc_pd(priv->ca); if (IS_ERR(priv->pd)) { @@ -187,6 +188,11 @@ int ipoib_transport_dev_init(struct net_ init_attr.send_cq = priv->cq; init_attr.recv_cq = priv->cq; + if (priv->dev->features & NETIF_F_SG) + init_attr.cap.max_send_sge = MAX_SKB_FRAGS + 1; + else + init_attr.cap.max_send_sge = 1; + priv->qp = ib_create_qp(priv->pd, &init_attr); if (IS_ERR(priv->qp)) { printk(KERN_WARNING "%s: failed to create QP\n", ca->name); @@ -197,11 +203,11 @@ int ipoib_transport_dev_init(struct net_ priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - priv->tx_sge.lkey = priv->mr->lkey; + for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) + priv->tx_sge[i].lkey = priv->mr->lkey; priv->tx_wr.opcode = IB_WR_SEND; - priv->tx_wr.sg_list = &priv->tx_sge; - priv->tx_wr.num_sge = 1; + priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; return 0; Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 20:50:31.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 20:50:33.000000000 +0300 @@ -495,8 +495,8 @@ static inline int post_send(struct ipoib { struct ib_send_wr *bad_wr; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + priv->tx_sge[0].addr = addr; + priv->tx_sge[0].length = len; priv->tx_wr.wr_id = wr_id; @@ -537,7 +537,7 @@ void ipoib_cm_send(struct net_device *de return; } - tx_req->mapping = addr; + tx_req->mapping[0].addr = addr; if (unlikely(post_send(priv, tx, tx->tx_head & (ipoib_sendq_size - 1), addr, skb->len))) { @@ -577,7 +577,8 @@ static void ipoib_cm_handle_tx_wc(struct tx_req = &tx->tx_ring[wr_id]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, DMA_TO_DEVICE); + ib_dma_unmap_single(priv->ca, tx_req->mapping[0].addr, + tx_req->skb->len, DMA_TO_DEVICE); /* FIXME: is this right? Shouldn't we only increment on success? */ ++priv->stats.tx_packets; @@ -981,8 +982,8 @@ static void ipoib_cm_tx_destroy(struct i if (p->tx_ring) { while ((int) p->tx_tail - (int) p->tx_head < 0) { tx_req = &p->tx_ring[p->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, - DMA_TO_DEVICE); + ib_dma_unmap_single(priv->ca, tx_req->mapping[0].addr, + tx_req->skb->len, DMA_TO_DEVICE); dev_kfree_skb_any(tx_req->skb); ++p->tx_tail; } Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:31.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:33.000000000 +0300 @@ -1067,17 +1067,18 @@ int ipoib_add_pkey_attr(struct net_devic return device_create_file(&dev->dev, &dev_attr_pkey); } -static void set_tx_csum(struct net_device *dev) +static int set_tx_csum(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); if (test_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags)) - return; + return -EINVAL; if (!(priv->ca->flags & IB_DEVICE_IP_CSUM)) - return; + return -EINVAL; dev->features |= NETIF_F_HW_CSUM; + return 0; } static void set_rx_csum(struct net_device *dev) @@ -1128,6 +1129,11 @@ static struct net_device *ipoib_add_port } else memcpy(priv->dev->dev_addr + 4, priv->local_gid.raw, sizeof (union ib_gid)); + priv->ca = hca; + if (!set_tx_csum(priv->dev)) + priv->dev->features |= NETIF_F_SG; + + set_rx_csum(priv->dev); result = ipoib_dev_init(priv->dev, hca, port); if (result < 0) { @@ -1146,9 +1152,6 @@ static struct net_device *ipoib_add_port goto event_failed; } - set_tx_csum(priv->dev); - set_rx_csum(priv->dev); - result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", From eli at mellanox.co.il Wed Aug 15 11:19:43 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:19:43 +0300 Subject: [ofa-general] [PATCH - 8] mlx4_ib configure QP's max nessage size Message-ID: <1187201983.16253.188.camel@mtls03> Configure QP's max message size according to the value queried by query dev cap. Signed-off-by: Eli Cohen --- Please note the two previous patches were numbered 6. "fix typo" is 6 and "s/g" is 7. Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:34.000000000 +0300 @@ -746,7 +746,8 @@ static int __mlx4_ib_modify_qp(struct ib attr->path_mtu); goto out; } - context->mtu_msgmax = (attr->path_mtu << 5) | 31; + context->mtu_msgmax = (attr->path_mtu << 5) | + ilog2(dev->dev->caps.max_msg_sz); } if (qp->rq.wqe_cnt) From eli at mellanox.co.il Wed Aug 15 11:20:40 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:20:40 +0300 Subject: [ofa-general] [PATCH - 9] ib core - add lso support Message-ID: <1187202040.16253.190.camel@mtls03> Add LSO supprt at the core This patch provides support at the core level for devices that support TCP large send offload fragmentation. Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h =================================================================== --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-08-15 20:50:28.000000000 +0300 +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-15 20:50:35.000000000 +0300 @@ -94,7 +94,8 @@ enum ib_device_cap_flags { IB_DEVICE_ZERO_STAG = (1<<15), IB_DEVICE_SEND_W_INV = (1<<16), IB_DEVICE_MEM_WINDOW = (1<<17), - IB_DEVICE_IP_CSUM = (1<<18) + IB_DEVICE_IP_CSUM = (1<<18), + IB_DEVICE_TCP_GSO = (1<<19) }; enum ib_atomic_cap { @@ -402,6 +403,7 @@ enum ib_wc_opcode { IB_WC_COMP_SWAP, IB_WC_FETCH_ADD, IB_WC_BIND_MW, + IB_WC_LSO, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -606,6 +608,7 @@ enum ib_wr_opcode { IB_WR_RDMA_WRITE, IB_WR_RDMA_WRITE_WITH_IMM, IB_WR_SEND, + IB_WR_LSO, IB_WR_SEND_WITH_IMM, IB_WR_RDMA_READ, IB_WR_ATOMIC_CMP_AND_SWP, @@ -618,7 +621,8 @@ enum ib_send_flags { IB_SEND_SOLICITED = (1<<2), IB_SEND_INLINE = (1<<3), IB_SEND_IP_CSUM = (1<<4), - IB_SEND_UDP_TCP_CSUM = (1<<5) + IB_SEND_UDP_TCP_CSUM = (1<<5), + IB_SEND_UDP_LSO = (1<<6) }; struct ib_sge { @@ -648,6 +652,9 @@ struct ib_send_wr { } atomic; struct { struct ib_ah *ah; + void *header; + int hlen; + int mss; u32 remote_qpn; u32 remote_qkey; u16 pkey_index; /* valid for GSI only */ From eli at mellanox.co.il Wed Aug 15 11:21:49 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:21:49 +0300 Subject: [ofa-general] [PATCH - 10] mlx4 - add LSO support Message-ID: <1187202109.16253.193.camel@mtls03> Add LSO support to mlx4 Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.c 2007-08-15 20:50:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.c 2007-08-15 20:50:36.000000000 +0300 @@ -133,6 +133,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * #define QUERY_DEV_CAP_MAX_AV_OFFSET 0x27 #define QUERY_DEV_CAP_MAX_REQ_QP_OFFSET 0x29 #define QUERY_DEV_CAP_MAX_RES_QP_OFFSET 0x2b +#define QUERY_DEV_CAP_MAX_GSO_OFFSET 0x2d #define QUERY_DEV_CAP_MAX_RDMA_OFFSET 0x2f #define QUERY_DEV_CAP_RSZ_SRQ_OFFSET 0x33 #define QUERY_DEV_CAP_ACK_DELAY_OFFSET 0x35 @@ -215,6 +216,13 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * dev_cap->max_requester_per_qp = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RES_QP_OFFSET); dev_cap->max_responder_per_qp = 1 << (field & 0x3f); + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_GSO_OFFSET); + field &= 0x1f; + if (!field) + dev_cap->max_gso_sz = 0; + else + dev_cap->max_gso_sz = 1 << field; + MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_RDMA_OFFSET); dev_cap->max_rdma_global = 1 << (field & 0x3f); MLX4_GET(field, outbox, QUERY_DEV_CAP_ACK_DELAY_OFFSET); @@ -377,6 +385,7 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev * dev_cap->max_sq_desc_sz, dev_cap->max_sq_sg); mlx4_dbg(dev, "Max RQ desc size: %d, max RQ S/G: %d\n", dev_cap->max_rq_desc_sz, dev_cap->max_rq_sg); + mlx4_dbg(dev, "Max GSO size: %d\n", dev_cap->max_gso_sz); dump_dev_cap_flags(dev, dev_cap->flags); Index: linux-2.6.23-rc1/drivers/net/mlx4/fw.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/fw.h 2007-08-15 20:50:12.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/fw.h 2007-08-15 20:50:36.000000000 +0300 @@ -96,6 +96,7 @@ struct mlx4_dev_cap { u8 bmme_flags; u32 reserved_lkey; u64 max_icm_sz; + int max_gso_sz; }; struct mlx4_adapter { Index: linux-2.6.23-rc1/drivers/net/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/net/mlx4/main.c 2007-08-15 20:50:12.000000000 +0300 +++ linux-2.6.23-rc1/drivers/net/mlx4/main.c 2007-08-15 20:50:36.000000000 +0300 @@ -158,6 +158,7 @@ static int __devinit mlx4_dev_cap(struct dev->caps.page_size_cap = ~(u32) (dev_cap->min_page_sz - 1); dev->caps.flags = dev_cap->flags; dev->caps.stat_rate_support = dev_cap->stat_rate_support; + dev->caps.max_gso_sz = dev_cap->max_gso_sz; return 0; } Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/main.c 2007-08-15 20:50:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/main.c 2007-08-15 20:50:36.000000000 +0300 @@ -101,6 +101,8 @@ static int mlx4_ib_query_device(struct i props->device_cap_flags |= IB_DEVICE_UD_AV_PORT_ENFORCE; if (dev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) props->device_cap_flags |= IB_DEVICE_IP_CSUM; + if (dev->dev->caps.max_gso_sz) + props->device_cap_flags |= IB_DEVICE_TCP_GSO; props->vendor_id = be32_to_cpup((__be32 *) (out_mad->data + 36)) & 0xffffff; @@ -572,6 +574,8 @@ static void *mlx4_ib_add(struct mlx4_dev if (ibdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IPOIB_CSUM) ibdev->ib_dev.flags |= IB_DEVICE_IP_CSUM; + if (ibdev->dev->caps.max_gso_sz) + ibdev->ib_dev.flags |= IB_DEVICE_TCP_GSO; if (init_node_data(ibdev)) goto err_map; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:34.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/qp.c 2007-08-15 20:50:36.000000000 +0300 @@ -65,6 +65,7 @@ struct mlx4_ib_sqp { static const __be32 mlx4_ib_opcode[] = { [IB_WR_SEND] = __constant_cpu_to_be32(MLX4_OPCODE_SEND), + [IB_WR_LSO] = __constant_cpu_to_be32(MLX4_OPCODE_LSO), [IB_WR_SEND_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_SEND_IMM), [IB_WR_RDMA_WRITE] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE), [IB_WR_RDMA_WRITE_WITH_IMM] = __constant_cpu_to_be32(MLX4_OPCODE_RDMA_WRITE_IMM), @@ -737,9 +738,11 @@ static int __mlx4_ib_modify_qp(struct ib } } - if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI || - ibqp->qp_type == IB_QPT_UD) + if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI) context->mtu_msgmax = (IB_MTU_4096 << 5) | 11; + else if (ibqp->qp_type == IB_QPT_UD) + context->mtu_msgmax = (IB_MTU_4096 << 5) | + ilog2(dev->dev->caps.max_gso_sz); else if (attr_mask & IB_QP_PATH_MTU) { if (attr->path_mtu < IB_MTU_256 || attr->path_mtu > IB_MTU_4096) { printk(KERN_ERR "path MTU (%u) is invalid\n", @@ -1221,6 +1224,29 @@ static void set_data_seg(struct mlx4_wqe dseg->addr = cpu_to_be64(sg->addr); } +static int build_lso_seg(struct mlx4_lso_seg *wqe, struct ib_send_wr *wr, + struct mlx4_ib_qp *qp, int *lso_seg_len) +{ + int halign; + + memcpy(wqe->header, wr->wr.ud.header, wr->wr.ud.hlen); + + /* make sure LSO header is written before + overwriting stamping */ + wmb(); + + wqe->mss_hdr_size = cpu_to_be32(((wr->wr.ud.mss - wr->wr.ud.hlen) + << 16) | wr->wr.ud.hlen); + + halign = ALIGN(wr->wr.ud.hlen, 16); + + if (unlikely(wr->num_sge > qp->sq.max_gs - (halign >> 4))) + return -EINVAL; + + *lso_seg_len = halign; + return 0; +} + int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, struct ib_send_wr **bad_wr) { @@ -1233,6 +1259,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp int ind; int size; int i; + int hlen; spin_lock_irqsave(&qp->rq.lock, flags); @@ -1311,6 +1338,17 @@ int mlx4_ib_post_send(struct ib_qp *ibqp set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + + if (wr->opcode == IB_WR_LSO) { + err = build_lso_seg(wqe, wr, qp, &hlen); + if (err) { + *bad_wr = wr; + goto out; + } + wqe += hlen; + size += hlen >> 4; + } + break; case IB_QPT_SMI: @@ -1364,6 +1402,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp ctrl->owner_opcode = mlx4_ib_opcode[wr->opcode] | (ind & qp->sq.wqe_cnt ? cpu_to_be32(1 << 31) : 0); + /* * We can improve latency by not stamping the last * send queue WQE until after ringing the doorbell, so Index: linux-2.6.23-rc1/include/linux/mlx4/device.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/device.h 2007-08-15 20:50:12.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/device.h 2007-08-15 20:50:36.000000000 +0300 @@ -177,6 +177,7 @@ struct mlx4_caps { u32 flags; u16 stat_rate_support; u8 port_width_cap[MLX4_MAX_PORTS + 1]; + int max_gso_sz; }; struct mlx4_buf_list { Index: linux-2.6.23-rc1/include/linux/mlx4/qp.h =================================================================== --- linux-2.6.23-rc1.orig/include/linux/mlx4/qp.h 2007-08-15 20:50:30.000000000 +0300 +++ linux-2.6.23-rc1/include/linux/mlx4/qp.h 2007-08-15 20:50:36.000000000 +0300 @@ -215,6 +215,11 @@ struct mlx4_wqe_datagram_seg { __be32 reservd[2]; }; +struct mlx4_lso_seg { + __be32 mss_hdr_size; + __be32 header[0]; +}; + struct mlx4_wqe_bind_seg { __be32 flags1; __be32 flags2; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mlx4/cq.c 2007-08-15 20:50:30.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mlx4/cq.c 2007-08-15 20:50:36.000000000 +0300 @@ -403,6 +403,9 @@ static int mlx4_ib_poll_one(struct mlx4_ case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + case MLX4_OPCODE_LSO: + wc->opcode = IB_WC_LSO; + break; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); From eli at mellanox.co.il Wed Aug 15 11:22:39 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:22:39 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support Message-ID: <1187202159.16253.195.camel@mtls03> Add LSO support to ipoib Using LSO improves performance by allowing the software to not fragment the payload to mtu sized patckets and also results in lower rate of interrupts since each such work request has just one CQE. Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:33.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:38.000000000 +0300 @@ -704,7 +704,13 @@ static int ipoib_start_xmit(struct sk_bu goto out; } - ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(skb->dst->neighbour->ha)); + if (skb_is_gso(skb)) + ipoib_send_gso(dev, skb, neigh->ah, + IPOIB_QPN(skb->dst->neighbour->ha)); + else + ipoib_send(dev, skb, neigh->ah, + IPOIB_QPN(skb->dst->neighbour->ha)); + goto out; } @@ -1152,6 +1158,10 @@ static struct net_device *ipoib_add_port goto event_failed; } + if (priv->dev->features & NETIF_F_SG) + if (priv->ca->flags & IB_DEVICE_TCP_GSO) + priv->dev->features |= NETIF_F_TSO; + result = register_netdev(priv->dev); if (result) { printk(KERN_WARNING "%s: couldn't register ipoib port %d; error %d\n", Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:33.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:38.000000000 +0300 @@ -373,6 +373,10 @@ int ipoib_add_pkey_attr(struct net_devic void ipoib_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_ah *address, u32 qpn); + +void ipoib_send_gso(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn); + void ipoib_reap_ah(struct work_struct *work); void ipoib_flush_paths(struct net_device *dev); Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:33.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-15 20:50:38.000000000 +0300 @@ -38,6 +38,7 @@ #include #include #include +#include #include @@ -249,15 +250,24 @@ repost: } static int dma_unmap_list(struct ib_device *ca, struct ipoib_mapping_st *map, - u16 n) + u16 n, int gso) { int i; int len; + int first; - ib_dma_unmap_single(ca, map[0].addr, map[0].size, DMA_TO_DEVICE); - len = map[0].size; + if (!gso) { + ib_dma_unmap_single(ca, map[0].addr, map[0].size, + DMA_TO_DEVICE); + len = map[0].size; + first = 1; + } else { + len = 0; + first = 0; + } + + for (i = first; i < n; ++i) { - for (i = 1; i < n; ++i) { ib_dma_unmap_page(ca, map[i].addr, map[i].size, DMA_TO_DEVICE); len += map[i].size; @@ -276,6 +286,7 @@ static void ipoib_ib_handle_tx_wc(struct ipoib_dbg_data(priv, "send completion: id %d, status: %d\n", wr_id, wc->status); + if (unlikely(wr_id >= ipoib_sendq_size)) { ipoib_warn(priv, "send completion event with wrid %d (> %d)\n", wr_id, ipoib_sendq_size); @@ -283,8 +294,16 @@ static void ipoib_ib_handle_tx_wc(struct } tx_req = &priv->tx_ring[wr_id]; - priv->stats.tx_bytes += dma_unmap_list(priv->ca, tx_req->mapping, - skb_shinfo(tx_req->skb)->nr_frags + 1); + if (skb_is_gso(tx_req->skb)) + priv->stats.tx_bytes += + dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags, 1); + else + priv->stats.tx_bytes += + dma_unmap_list(priv->ca, tx_req->mapping, + skb_shinfo(tx_req->skb)->nr_frags + 1, + 0); + ++priv->stats.tx_packets; dev_kfree_skb_any(tx_req->skb); @@ -367,7 +386,8 @@ void ipoib_ib_completion(struct ib_cq *c static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - struct ipoib_mapping_st *mapping, int ngather) + struct ipoib_mapping_st *mapping, int ngather, + void *lso_header, int h_len) { struct ib_send_wr *bad_wr; int i; @@ -382,9 +402,88 @@ static inline int post_send(struct ipoib priv->tx_wr.wr.ud.remote_qpn = qpn; priv->tx_wr.wr.ud.ah = address; + if (lso_header) { + priv->tx_wr.wr.ud.mss = priv->dev->mtu; + priv->tx_wr.wr.ud.header = lso_header; + priv->tx_wr.wr.ud.hlen = h_len; + priv->tx_wr.opcode = IB_WR_LSO; + } else + priv->tx_wr.opcode = IB_WR_SEND; + return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } + +void ipoib_send_gso(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ipoib_tx_buf *tx_req; + struct skb_frag_struct *frag; + u64 addr; + unsigned short i; + + ipoib_dbg_data(priv, "sending gso packet, length=%d address=%p" + " qpn=0x%06x\n", skb->len, address, qpn); + + if (unlikely((skb_headlen(skb) - IPOIB_ENCAP_LEN) != + ((ip_hdr(skb)->ihl + tcp_hdr(skb)->doff) << 2))) { + ipoib_warn(priv, "headlen (%d) does not match ip (%d)and " + "tcp headers(%d), dropping skb\n", + skb_headlen(skb) - IPOIB_ENCAP_LEN, + ip_hdr(skb)->ihl << 2, tcp_hdr(skb)->doff << 2); + ++priv->stats.tx_errors; + dev_kfree_skb_any(skb); + return; + } + + /* + * We put the skb into the tx_ring _before_ we call post_send() + * because it's entirely possible that the completion handler will + * run before we execute anything after the post_send(). That + * means we have to make sure everything is properly recorded and + * our state is consistent before we call post_send(). + */ + tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; + tx_req->skb = skb; + + for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) { + frag = &skb_shinfo(skb)->frags[i]; + addr = ib_dma_map_page(priv->ca, frag->page, frag->page_offset, + frag->size, DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(priv->ca, addr))) + goto map_err; + + tx_req->mapping[i].addr = addr; + tx_req->mapping[i].size = frag->size; + } + + if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), + address->ah, qpn, tx_req->mapping, + skb_shinfo(skb)->nr_frags, skb->data, + skb_headlen(skb)))) { + ipoib_warn(priv, "post_send failed\n"); + goto map_err; + } else { + dev->trans_start = jiffies; + + address->last_send = priv->tx_head; + ++priv->tx_head; + + if (priv->tx_head - priv->tx_tail == ipoib_sendq_size) { + ipoib_dbg(priv, "TX ring full, stopping kernel" + " net queue\n"); + netif_stop_queue(dev); + set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + } + } + return; + +map_err: + dma_unmap_list(priv->ca, tx_req->mapping, i, 1); + dev_kfree_skb_any(skb); +} + void ipoib_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_ah *address, u32 qpn) { @@ -449,7 +548,7 @@ void ipoib_send(struct net_device *dev, if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), address->ah, qpn, tx_req->mapping, - skb_shinfo(skb)->nr_frags + 1))) { + skb_shinfo(skb)->nr_frags + 1, NULL, 0))) { ipoib_warn(priv, "post_send failed\n"); goto map_err; } else { @@ -467,7 +566,7 @@ void ipoib_send(struct net_device *dev, return; map_err: - dma_unmap_list(priv->ca, tx_req->mapping, i + 1); + dma_unmap_list(priv->ca, tx_req->mapping, i + 1, 0); dev_kfree_skb_any(skb); } @@ -664,7 +763,8 @@ int ipoib_ib_dev_stop(struct net_device (ipoib_sendq_size - 1)]; skb = tx_req->skb; dma_unmap_list(priv->ca, tx_req->mapping, - skb_shinfo(skb)->nr_frags + 1); + skb_shinfo(skb)->nr_frags + 1, + skb_is_gso(skb)); dev_kfree_skb_any(skb); ++priv->tx_tail; } Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:33.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-15 20:50:38.000000000 +0300 @@ -206,7 +206,6 @@ int ipoib_transport_dev_init(struct net_ for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) priv->tx_sge[i].lkey = priv->mr->lkey; - priv->tx_wr.opcode = IB_WR_SEND; priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; From eli at mellanox.co.il Wed Aug 15 11:23:47 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Wed, 15 Aug 2007 21:23:47 +0300 Subject: [ofa-general] [PATCH - 12] - ipoib add ethtool support Message-ID: <1187202227.16253.197.camel@mtls03> Add ethtool support to ipoib Signed-off-by: Eli Cohen --- This one is actually the foundation with no real contecxt. I think we can add here all the logic of wheather to allow using a certain feature, e.g. checksum offload, scatter/gather etc. and decide on all the dependencies. Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/Makefile =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/Makefile 2007-08-15 20:50:11.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/Makefile 2007-08-15 20:50:39.000000000 +0300 @@ -4,7 +4,8 @@ ib_ipoib-y := ipoib_main.o \ ipoib_ib.o \ ipoib_multicast.o \ ipoib_verbs.o \ - ipoib_vlan.o + ipoib_vlan.o \ + ipoib_etool.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_CM) += ipoib_cm.o ib_ipoib-$(CONFIG_INFINIBAND_IPOIB_DEBUG) += ipoib_fs.o Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_etool.c =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_etool.c 2007-08-15 20:50:39.000000000 +0300 @@ -0,0 +1,55 @@ +/* + * Copyright (c) 2007 Mellanox Technologies. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + * $Id: ipoib_etool.c $ + */ + +#include +#include +#include + +#include "ipoib.h" + +static void ipoib_get_drvinfo(struct net_device *netdev, + struct ethtool_drvinfo *drvinfo) +{ + strncpy(drvinfo->driver, "ipoib", sizeof(drvinfo->driver) - 1); +} + +static const struct ethtool_ops ipoib_ethtool_ops = { + .get_drvinfo = ipoib_get_drvinfo, + .get_tso = ethtool_op_get_tso, +}; + +void ipoib_set_ethtool_ops(struct net_device *dev) +{ + SET_ETHTOOL_OPS(dev, &ipoib_ethtool_ops); +} Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:38.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 20:50:39.000000000 +0300 @@ -440,6 +440,8 @@ void ipoib_pkey_poll(struct work_struct int ipoib_pkey_dev_delay_open(struct net_device *dev); void ipoib_drain_cq(struct net_device *dev); +void ipoib_set_ethtool_ops(struct net_device *dev); + #ifdef CONFIG_INFINIBAND_IPOIB_CM #define IPOIB_FLAGS_RC 0x80 Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:38.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 20:50:39.000000000 +0300 @@ -956,6 +956,7 @@ static void ipoib_setup(struct net_devic dev->neigh_setup = ipoib_neigh_setup_dev; dev->poll = ipoib_poll; dev->weight = 100; + ipoib_set_ethtool_ops(dev); dev->watchdog_timeo = HZ; From ossrosch at linux.vnet.ibm.com Wed Aug 15 11:29:45 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 15 Aug 2007 20:29:45 +0200 Subject: [ofa-general] Re: [ewg] Re: OFED Aug 13 meeting summary In-Reply-To: <46C1D454.7080603@mellanox.co.il> References: <46C088BE.1090100@mellanox.co.il> <20070814150610.GA16827@mellanox.co.il> <46C1D454.7080603@mellanox.co.il> Message-ID: <200708152029.46315.ossrosch@linux.vnet.ibm.com> On Tuesday 14 August 2007 18:12, Tziporet Koren wrote: > > > > I pushed a fix for this out in my tree, so Vlad will take it in 1.2.c branch. > > > > > Vlad will take this patch, and we will do a new build with it. > This will delay the 1.2.5 release for tomorrow :-( Hi, we tested the ofa_1_2_c_kernel-20070815-0200.tgz from today. The changes are applied and we successfully built the package on SLES10 and rhel-4.5. All works fine for us. regards Stefan From sashak at voltaire.com Wed Aug 15 12:25:23 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 15 Aug 2007 22:25:23 +0300 Subject: [ofa-general] Re: [PATCH] Fix the osm_epi_node_id_t structure name. In-Reply-To: <20070814162148.1012c706.weiny2@llnl.gov> References: <20070814162148.1012c706.weiny2@llnl.gov> Message-ID: <20070815192523.GX27000@sashak.voltaire.com> On 16:21 Tue 14 Aug , Ira Weiny wrote: > From 6e01f304c1d1cbd8d1163708d907ec0550a0ca31 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Wed, 25 Jul 2007 11:33:08 -0700 > Subject: [PATCH] Fix the osm_epi_node_id_t structure name. > > The structure osm_epi_node_id_t describes a port. > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Wed Aug 15 12:25:47 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 15 Aug 2007 22:25:47 +0300 Subject: [ofa-general] Re: [PATCH] iblinkinfo.pl: Remove annoying double quote around switch name. In-Reply-To: <20070814180614.62293fe3.weiny2@llnl.gov> References: <20070814180614.62293fe3.weiny2@llnl.gov> Message-ID: <20070815192547.GY27000@sashak.voltaire.com> On 18:06 Tue 14 Aug , Ira Weiny wrote: > From fc0251b5150afdd5d40d55facdf85c645939b482 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 14 Aug 2007 18:04:43 -0700 > Subject: [PATCH] iblinkinfo.pl: Remove annoying double quote around switch name. > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From changquing.tang at hp.com Wed Aug 15 13:44:54 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Wed, 15 Aug 2007 20:44:54 -0000 Subject: [ofa-general] Post receive WR to QP in error state In-Reply-To: References: <20070729140431.GG16915@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA8403020C6484@G3W0634.americas.hpqcorp.net> Roland: I read the IB specification, it does say posting send WR to QP in error state is OK, and the WR will be completed in FLUSH error, but it does not say that for posting receive WR. I want to check with you if "posting receive WR to QP in error state returns SUCCESS, and the WR will be completed with FLUSH error". Thanks. --CQ From display at palmyrastatebank.net Wed Aug 15 11:29:56 2007 From: display at palmyrastatebank.net (Consuelo Kessler) Date: Wed, 15 Aug 2007 21:29:56 +0300 Subject: [ofa-general] Haben Sie wieder Spass am Leben! that I have been -- Best of all, in a way that won't Message-ID: <01c7df83$6cc46ab0$5a073ac9@display> Sie leben nur einmal - warum dann nicht was neues ausprobieren? Preise die keine Konkurrenz kennen - Kein peinlicher Arztbesuch erforderlich - Kostenlose, arztliche Telefon-Beratung - Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen - Bequem und diskret online bestellen. - Visa verifizierter Onlineshop - Diskrete Verpackung und Zahlung - keine versteckte Kosten Originalmedikamente Ciiaaaaaalis 10 Pack. 27,00 Euro Viiaaaagra 10 Pack. 21,00 Euro Klicken Sie HIER und Sie erhalten vier Dosen umsonst http://hduho.eatless.cn/?804277434270 (bitte warten Sie einen Moment bis die Seite vollstandig geladen wird) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Wed Aug 15 14:53:56 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 16 Aug 2007 00:53:56 +0300 Subject: [ofa-general] [PATCH] opensm/libvendor: syntax fixes Message-ID: <20070815215356.GZ27000@sashak.voltaire.com> Fix trivial syntax errors - extra ")" and unmatched #endif. Signed-off-by: Sasha Khapyorsky --- opensm/libvendor/osm_vendor_al.c | 1 - opensm/libvendor/osm_vendor_ts.c | 2 -- 2 files changed, 0 insertions(+), 3 deletions(-) diff --git a/opensm/libvendor/osm_vendor_al.c b/opensm/libvendor/osm_vendor_al.c index c170d79..1b748ae 100644 --- a/opensm/libvendor/osm_vendor_al.c +++ b/opensm/libvendor/osm_vendor_al.c @@ -1424,7 +1424,6 @@ void osm_vendor_set_sm( IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val ) - ) { osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *)h_bind; osm_vendor_t *p_vend = p_bind->p_vend; diff --git a/opensm/libvendor/osm_vendor_ts.c b/opensm/libvendor/osm_vendor_ts.c index d5a9173..763073b 100644 --- a/opensm/libvendor/osm_vendor_ts.c +++ b/opensm/libvendor/osm_vendor_ts.c @@ -990,5 +990,3 @@ osm_vendor_set_debug( { } - -#endif /* OSM_VENDOR_INTF_TS */ -- 1.5.3.rc2.29.gc4640f From sashak at voltaire.com Wed Aug 15 14:55:01 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 16 Aug 2007 00:55:01 +0300 Subject: [ofa-general] [PATCH] opensm/include/vendor: indentation fixes Message-ID: <20070815215501.GA27000@sashak.voltaire.com> include/vendor header files are formatted with opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs) Signed-off-by: Sasha Khapyorsky --- opensm/include/vendor/osm_mtl_bind.h | 50 ++-- opensm/include/vendor/osm_pkt_randomizer.h | 42 ++-- opensm/include/vendor/osm_vendor.h | 2 - opensm/include/vendor/osm_vendor_al.h | 94 +++---- opensm/include/vendor/osm_vendor_api.h | 93 +++---- opensm/include/vendor/osm_vendor_ibumad.h | 49 ++-- opensm/include/vendor/osm_vendor_mlx.h | 39 ++-- opensm/include/vendor/osm_vendor_mlx_defs.h | 52 ++-- opensm/include/vendor/osm_vendor_mlx_dispatcher.h | 13 +- opensm/include/vendor/osm_vendor_mlx_hca.h | 21 +- opensm/include/vendor/osm_vendor_mlx_inout.h | 10 +- opensm/include/vendor/osm_vendor_mlx_rmpp_ctx.h | 120 ++++----- opensm/include/vendor/osm_vendor_mlx_sar.h | 34 ++-- opensm/include/vendor/osm_vendor_mlx_sender.h | 56 ++--- opensm/include/vendor/osm_vendor_mlx_svc.h | 196 +++++++------- opensm/include/vendor/osm_vendor_mlx_transport.h | 24 +- .../vendor/osm_vendor_mlx_transport_anafa.h | 19 +- opensm/include/vendor/osm_vendor_mlx_txn.h | 185 ++++++------- opensm/include/vendor/osm_vendor_mtl.h | 71 ++--- opensm/include/vendor/osm_vendor_mtl_hca_guid.h | 39 ++-- .../vendor/osm_vendor_mtl_transaction_mgr.h | 72 ++--- opensm/include/vendor/osm_vendor_sa_api.h | 286 ++++++++++---------- opensm/include/vendor/osm_vendor_select.h | 17 +- opensm/include/vendor/osm_vendor_test.h | 45 ++-- opensm/include/vendor/osm_vendor_ts.h | 102 +++---- opensm/include/vendor/osm_vendor_umadt.h | 26 +- 26 files changed, 762 insertions(+), 995 deletions(-) diff --git a/opensm/include/vendor/osm_mtl_bind.h b/opensm/include/vendor/osm_mtl_bind.h index 81b434b..3994d59 100644 --- a/opensm/include/vendor/osm_mtl_bind.h +++ b/opensm/include/vendor/osm_mtl_bind.h @@ -45,13 +45,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Vendor/osm_vendor_mgt_bind * NAME * osm_vendor_mgt_bind_t @@ -62,14 +61,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_vendor_mgt_bind -{ - boolean_t smi_init, gsi_init; - IB_MGT_mad_hndl_t smi_mads_hdl; - IB_MGT_mad_hndl_t gsi_mads_hdl; - struct _osm_mtl_bind_info *smi_p_bind; -} -osm_vendor_mgt_bind_t; +typedef struct _osm_vendor_mgt_bind { + boolean_t smi_init, gsi_init; + IB_MGT_mad_hndl_t smi_mads_hdl; + IB_MGT_mad_hndl_t gsi_mads_hdl; + struct _osm_mtl_bind_info *smi_p_bind; +} osm_vendor_mgt_bind_t; /* * FIELDS @@ -91,19 +88,17 @@ osm_vendor_mgt_bind_t; * * SYNOPSIS */ -typedef struct _osm_mtl_bind_info -{ - IB_MGT_mad_hndl_t mad_hndl; - osm_vendor_t *p_vend; - void *client_context; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_id_t hca_id; - uint8_t port_num; - osm_vend_mad_recv_callback_t rcv_callback; - osm_vend_mad_send_err_callback_t send_err_callback; - osm_mad_pool_t *p_osm_pool; -} -osm_mtl_bind_info_t; +typedef struct _osm_mtl_bind_info { + IB_MGT_mad_hndl_t mad_hndl; + osm_vendor_t *p_vend; + void *client_context; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_id_t hca_id; + uint8_t port_num; + osm_vend_mad_recv_callback_t rcv_callback; + osm_vend_mad_send_err_callback_t send_err_callback; + osm_mad_pool_t *p_osm_pool; +} osm_mtl_bind_info_t; /* * FIELDS @@ -135,10 +130,7 @@ osm_mtl_bind_info_t; * SEE ALSO *********/ ib_api_status_t -osm_mtl_send_mad( - IN osm_mtl_bind_info_t *p_bind, - IN osm_madw_t * const p_madw); +osm_mtl_send_mad(IN osm_mtl_bind_info_t * p_bind, IN osm_madw_t * const p_madw); END_C_DECLS - -#endif // _OSM_BIND_H_ +#endif // _OSM_BIND_H_ diff --git a/opensm/include/vendor/osm_pkt_randomizer.h b/opensm/include/vendor/osm_pkt_randomizer.h index 8c32b1b..0c831e0 100644 --- a/opensm/include/vendor/osm_pkt_randomizer.h +++ b/opensm/include/vendor/osm_pkt_randomizer.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Declaration of osm_subn_t. @@ -49,7 +48,6 @@ #ifndef _OSM_PKT_RANDOMIZER_H_ #define _OSM_PKT_RANDOMIZER_H_ - #include #include #include @@ -62,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Packet Randomizer * NAME * Packet Randomizer @@ -84,7 +81,6 @@ BEGIN_C_DECLS * Yael Kalka, Mellanox * *********/ - /****d* OpenSM: Pkt_Randomizer/osm_pkt_randomizer_t * NAME * osm_pkt_randomizer_t @@ -95,13 +91,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pkt_randomizer -{ - uint8_t osm_pkt_drop_rate; - uint8_t osm_pkt_num_unstable_links; - uint8_t osm_pkt_unstable_link_rate; - osm_dr_path_t *fault_dr_paths; - uint8_t num_paths_initialized; +typedef struct _osm_pkt_randomizer { + uint8_t osm_pkt_drop_rate; + uint8_t osm_pkt_num_unstable_links; + uint8_t osm_pkt_unstable_link_rate; + osm_dr_path_t *fault_dr_paths; + uint8_t num_paths_initialized; } osm_pkt_randomizer_t; /* @@ -143,9 +138,8 @@ typedef struct _osm_pkt_randomizer * SYNOPSIS */ ib_api_status_t -osm_pkt_randomizer_init( - IN OUT osm_pkt_randomizer_t **pp_pkt_randomizer, - IN osm_log_t *p_log ); +osm_pkt_randomizer_init(IN OUT osm_pkt_randomizer_t ** pp_pkt_randomizer, + IN osm_log_t * p_log); /* * PARAMETERS * p_pkt_randomizer @@ -163,7 +157,6 @@ osm_pkt_randomizer_init( * *********/ - /****f* OpenSM: Pkt_Randomizer/osm_pkt_randomizer_destroy * NAME * osm_pkt_randomizer_destroy @@ -174,9 +167,8 @@ osm_pkt_randomizer_init( * SYNOPSIS */ void -osm_pkt_randomizer_destroy( - IN osm_pkt_randomizer_t **pp_pkt_randomizer, - IN osm_log_t *p_log ); +osm_pkt_randomizer_destroy(IN osm_pkt_randomizer_t ** pp_pkt_randomizer, + IN osm_log_t * p_log); /* * PARAMETERS * p_pkt_randomizer @@ -210,10 +202,9 @@ osm_pkt_randomizer_destroy( * SYNOPSIS */ boolean_t -osm_pkt_randomizer_mad_drop( - IN osm_log_t *p_log, - IN osm_pkt_randomizer_t *p_pkt_randomizer, - IN const ib_mad_t *p_mad ); +osm_pkt_randomizer_mad_drop(IN osm_log_t * p_log, + IN osm_pkt_randomizer_t * p_pkt_randomizer, + IN const ib_mad_t * p_mad); /* * PARAMETERS * p_subn @@ -235,5 +226,4 @@ osm_pkt_randomizer_mad_drop( *********/ END_C_DECLS - -#endif /* _OSM_PKT_RANDOMIZER_H */ +#endif /* _OSM_PKT_RANDOMIZER_H */ diff --git a/opensm/include/vendor/osm_vendor.h b/opensm/include/vendor/osm_vendor.h index 6a4f5c6..eb7665e 100644 --- a/opensm/include/vendor/osm_vendor.h +++ b/opensm/include/vendor/osm_vendor.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Include file used by OpenSM to pull in the correct vendor file. @@ -75,4 +74,3 @@ #error No MAD Interface selected! #error Choose an interface in osm_vendor_select.h #endif - diff --git a/opensm/include/vendor/osm_vendor_al.h b/opensm/include/vendor/osm_vendor_al.h index ad529f3..1f80006 100644 --- a/opensm/include/vendor/osm_vendor_al.h +++ b/opensm/include/vendor/osm_vendor_al.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Declaration of osm_mad_wrapper_t. @@ -46,9 +45,6 @@ * $Revision: 1.4 $ */ - - - #ifndef _OSM_VENDOR_AL_H_ #define _OSM_VENDOR_AL_H_ @@ -62,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Vendor AL * NAME * Vendor AL @@ -87,9 +82,6 @@ BEGIN_C_DECLS * * *********/ - - - /****h* OpenSM/Vendor Access Layer (AL) * NAME * Vendor AL @@ -101,14 +93,11 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - #define OSM_AL_SQ_SGE 256 #define OSM_AL_RQ_SGE 256 #define OSM_DEFAULT_RETRY_COUNT 3 - /* AL supports RMPP */ #define VENDOR_RMPP_SUPPORT 1 - /****s* OpenSM: Vendor AL/osm_ca_info_t * NAME * osm_ca_info_t @@ -118,11 +107,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; } osm_ca_info_t; /* @@ -149,11 +137,11 @@ typedef struct _osm_ca_info * SYNOPSIS */ static inline uint8_t -osm_ca_info_get_num_ports( - IN const osm_ca_info_t* const p_ca_info ) +osm_ca_info_get_num_ports(IN const osm_ca_info_t * const p_ca_info) { - return( p_ca_info->p_attr->num_ports ); + return (p_ca_info->p_attr->num_ports); } + /* * PARAMETERS * p_ca_info @@ -177,12 +165,12 @@ osm_ca_info_get_num_ports( * SYNOPSIS */ static inline ib_net64_t -osm_ca_info_get_port_guid( - IN const osm_ca_info_t* const p_ca_info, - IN const uint8_t index ) +osm_ca_info_get_port_guid(IN const osm_ca_info_t * const p_ca_info, + IN const uint8_t index) { - return( p_ca_info->p_attr->p_port_attr[index].port_guid ); + return (p_ca_info->p_attr->p_port_attr[index].port_guid); } + /* * PARAMETERS * p_ca_info @@ -212,12 +200,12 @@ osm_ca_info_get_port_guid( * SYNOPSIS */ static inline uint8_t -osm_ca_info_get_port_num( - IN const osm_ca_info_t* const p_ca_info, - IN const uint8_t index ) +osm_ca_info_get_port_num(IN const osm_ca_info_t * const p_ca_info, + IN const uint8_t index) { - return( p_ca_info->p_attr->p_port_attr[index].port_num ); + return (p_ca_info->p_attr->p_port_attr[index].port_num); } + /* * PARAMETERS * p_ca_info @@ -246,11 +234,11 @@ osm_ca_info_get_port_num( * SYNOPSIS */ static inline ib_net64_t -osm_ca_info_get_ca_guid( - IN const osm_ca_info_t* const p_ca_info ) +osm_ca_info_get_ca_guid(IN const osm_ca_info_t * const p_ca_info) { - return( p_ca_info->p_attr->ca_guid ); + return (p_ca_info->p_attr->ca_guid); } + /* * PARAMETERS * p_ca_info @@ -273,15 +261,14 @@ osm_ca_info_get_ca_guid( * * SYNOPSIS */ -typedef struct _osm_vendor -{ - ib_al_handle_t h_al; - osm_log_t *p_log; - uint32_t ca_count; - osm_ca_info_t *p_ca_info; - uint32_t timeout; - ib_ca_handle_t h_ca; - ib_pd_handle_t h_pd; +typedef struct _osm_vendor { + ib_al_handle_t h_al; + osm_log_t *p_log; + uint32_t ca_count; + osm_ca_info_t *p_ca_info; + uint32_t timeout; + ib_ca_handle_t h_ca; + ib_pd_handle_t h_pd; } osm_vendor_t; /* @@ -307,10 +294,8 @@ typedef struct _osm_vendor * SEE ALSO *********/ - #define OSM_BIND_INVALID_HANDLE 0 - /****s* OpenSM: Vendor AL/osm_bind_handle_t * NAME * osm_bind_handle_t @@ -320,7 +305,7 @@ typedef struct _osm_vendor * * SYNOPSIS */ -typedef void* osm_bind_handle_t; +typedef void *osm_bind_handle_t; /***********/ /****s* OpenSM/osm_vend_wrap_t @@ -333,15 +318,14 @@ typedef void* osm_bind_handle_t; * * SYNOPSIS *********/ -typedef struct _osm_vend_wrap_t -{ - uint32_t size; - osm_bind_handle_t h_bind; - ib_mad_element_t *p_elem; - ib_av_handle_t h_av; - void* p_resp_madw; - -}osm_vend_wrap_t; +typedef struct _osm_vend_wrap_t { + uint32_t size; + osm_bind_handle_t h_bind; + ib_mad_element_t *p_elem; + ib_av_handle_t h_av; + void *p_resp_madw; + +} osm_vend_wrap_t; /* * FIELDS * size @@ -365,7 +349,5 @@ typedef struct _osm_vend_wrap_t * SEE ALSO *********/ - END_C_DECLS - -#endif /* _OSM_VENDOR_AL_H_ */ +#endif /* _OSM_VENDOR_AL_H_ */ diff --git a/opensm/include/vendor/osm_vendor_api.h b/opensm/include/vendor/osm_vendor_api.h index 2451baa..8e3011d 100644 --- a/opensm/include/vendor/osm_vendor_api.h +++ b/opensm/include/vendor/osm_vendor_api.h @@ -54,13 +54,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM Vendor API/osm_vend_mad_recv_callback_t * NAME * osm_vend_mad_recv_callback_t @@ -71,10 +70,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef void (*osm_vend_mad_recv_callback_t)( - IN osm_madw_t *p_madw, - IN void* bind_context, - IN osm_madw_t *p_req_madw ); +typedef void (*osm_vend_mad_recv_callback_t) (IN osm_madw_t * p_madw, + IN void *bind_context, + IN osm_madw_t * p_req_madw); /* * PARAMETERS * p_madw @@ -107,9 +105,8 @@ typedef void (*osm_vend_mad_recv_callback_t)( * * SYNOPSIS */ -typedef void (*osm_vend_mad_send_err_callback_t)( - IN void* bind_context, - IN osm_madw_t *p_madw ); +typedef void (*osm_vend_mad_send_err_callback_t) (IN void *bind_context, + IN osm_madw_t * p_madw); /* * PARAMETERS * bind_context @@ -139,10 +136,8 @@ typedef void (*osm_vend_mad_send_err_callback_t)( * * SYNOPSIS */ -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ); +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout); /* * PARAMETERS * p_log @@ -168,9 +163,7 @@ osm_vendor_new( * * SYNOPSIS */ -void -osm_vendor_delete( - IN osm_vendor_t** const pp_vend ); +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend); /* * PARAMETERS * pp_vend @@ -194,10 +187,9 @@ osm_vendor_delete( * SYNOPSIS */ ib_api_status_t -osm_vendor_get_all_port_attr( - IN osm_vendor_t* const p_vend, - IN ib_port_attr_t* const p_attr_array, - IN uint32_t* const p_num_ports ); +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports); /* * PARAMETERS * p_vend @@ -232,10 +224,8 @@ osm_vendor_get_all_port_attr( * SYNOPSIS */ ib_api_status_t -osm_vendor_init( - IN osm_vendor_t* const p_vend, - IN osm_log_t * const p_log, - IN const uint32_t timeout ); +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout); /* * PARAMETERS * p_vend @@ -267,13 +257,12 @@ osm_vendor_init( * SYNOPSIS */ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_bind_info, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void* context ); +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_bind_info, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context); /* * PARAMETERS * p_vend @@ -313,9 +302,7 @@ osm_vendor_bind( * * SYNOPSIS */ -void -osm_vendor_unbind( - IN osm_bind_handle_t h_bind ); +void osm_vendor_unbind(IN osm_bind_handle_t h_bind); /* * PARAMETERS * h_bind @@ -339,11 +326,9 @@ osm_vendor_unbind( * * SYNOPSIS */ -ib_mad_t* -osm_vendor_get( - IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t* const p_vend_wrap ); +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vend_wrap); /* * PARAMETERS * h_bind @@ -372,10 +357,8 @@ osm_vendor_get( * SYNOPSIS */ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t const resp_expected); +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected); /* * PARAMETERS * h_bind @@ -409,9 +392,8 @@ osm_vendor_send( * SYNOPSIS */ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vend_wrap ); +osm_vendor_put(IN osm_bind_handle_t h_bind, + IN osm_vend_wrap_t * const p_vend_wrap); /* * PARAMETERS * h_bind @@ -439,9 +421,7 @@ osm_vendor_put( * * SYNOPSIS */ -ib_api_status_t -osm_vendor_local_lid_change( - IN osm_bind_handle_t h_bind ); +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind); /* * PARAMETERS * h_bind @@ -464,10 +444,7 @@ osm_vendor_local_lid_change( * * SYNOPSIS */ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ); +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val); /* * PARAMETERS * h_bind @@ -494,10 +471,7 @@ osm_vendor_set_sm( * * SYNOPSIS */ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ); +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level); /* * PARAMETERS * p_vend @@ -515,5 +489,4 @@ osm_vendor_set_debug( *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_API_H_ */ +#endif /* _OSM_VENDOR_API_H_ */ diff --git a/opensm/include/vendor/osm_vendor_ibumad.h b/opensm/include/vendor/osm_vendor_ibumad.h index 0c00d1f..f86aeef 100644 --- a/opensm/include/vendor/osm_vendor_ibumad.h +++ b/opensm/include/vendor/osm_vendor_ibumad.h @@ -47,13 +47,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Vendor Access Layer (UMAD) * NAME * Vendor UMAD @@ -65,12 +64,9 @@ BEGIN_C_DECLS * * *********/ - #define OSM_DEFAULT_RETRY_COUNT 3 - #define OSM_UMAD_MAX_CAS 32 #define OSM_UMAD_MAX_PORTS_PER_CA 2 - /****s* OpenSM: Vendor UMAD/osm_ca_info_t * NAME * osm_ca_info_t @@ -80,11 +76,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; } osm_ca_info_t; /* * FIELDS @@ -110,11 +105,11 @@ typedef struct _osm_ca_info * SYNOPSIS */ static inline uint8_t -osm_ca_info_get_num_ports( - IN const osm_ca_info_t* const p_ca_info ) +osm_ca_info_get_num_ports(IN const osm_ca_info_t * const p_ca_info) { - return( p_ca_info->p_attr->num_ports ); + return (p_ca_info->p_attr->num_ports); } + /* * PARAMETERS * p_ca_info @@ -137,7 +132,7 @@ osm_ca_info_get_num_ports( * * SYNOPSIS */ -typedef void *osm_bind_handle_t; +typedef void *osm_bind_handle_t; /***********/ typedef struct _umad_match { @@ -148,20 +143,18 @@ typedef struct _umad_match { #define OSM_UMAD_MAX_PENDING 1000 -typedef struct vendor_match_tbl -{ +typedef struct vendor_match_tbl { umad_match_t tbl[OSM_UMAD_MAX_PENDING]; uint32_t last_version; int max; } vendor_match_tbl_t; -typedef struct _osm_vendor -{ - osm_log_t *p_log; - uint32_t ca_count; - osm_ca_info_t *p_ca_info; - uint32_t timeout; - int max_retries; +typedef struct _osm_vendor { + osm_log_t *p_log; + uint32_t ca_count; + osm_ca_info_t *p_ca_info; + uint32_t timeout; + int max_retries; osm_bind_handle_t agents[UMAD_CA_MAX_AGENTS]; char ca_names[OSM_UMAD_MAX_CAS][UMAD_CA_NAME_LEN]; vendor_match_tbl_t mtbl; @@ -176,15 +169,13 @@ typedef struct _osm_vendor #define OSM_BIND_INVALID_HANDLE 0 -typedef struct _osm_vend_wrap -{ +typedef struct _osm_vend_wrap { int agent; int size; int retries; void *umad; - osm_bind_handle_t h_bind; + osm_bind_handle_t h_bind; } osm_vend_wrap_t; END_C_DECLS - -#endif /* _OSM_VENDOR_UMAD_H_ */ +#endif /* _OSM_VENDOR_UMAD_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx.h b/opensm/include/vendor/osm_vendor_mlx.h index b3794cd..106e0b4 100644 --- a/opensm/include/vendor/osm_vendor_mlx.h +++ b/opensm/include/vendor/osm_vendor_mlx.h @@ -43,13 +43,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* Forward reference */ @@ -59,35 +58,34 @@ struct _osm_pkt_randomizer; typedef struct _osm_vendor { - /* for holding common transport info - useful at ibmgt transport*/ - void *p_transport_info; + /* for holding common transport info - useful at ibmgt transport */ + void *p_transport_info; - osm_log_t *p_log; + osm_log_t *p_log; - /* Uniform timeout for every ACK/single MAD */ - uint32_t resp_timeout; + /* Uniform timeout for every ACK/single MAD */ + uint32_t resp_timeout; - /* Uniform timeout for every rmpp transaction */ - uint32_t ttime_timeout; + /* Uniform timeout for every rmpp transaction */ + uint32_t ttime_timeout; - /* All the bind handles associated with the vendor */ - cl_qlist_t bind_handles; + /* All the bind handles associated with the vendor */ + cl_qlist_t bind_handles; - /* run randomizer flag */ - boolean_t run_randomizer; + /* run randomizer flag */ + boolean_t run_randomizer; - /* Packet Randomizer object */ - struct _osm_pkt_randomizer *p_pkt_randomizer; + /* Packet Randomizer object */ + struct _osm_pkt_randomizer *p_pkt_randomizer; } osm_vendor_t; /* Repeating the definitions in osm_vendor_api.h */ -typedef void* osm_bind_handle_t; +typedef void *osm_bind_handle_t; -typedef struct _osm_vend_wrap -{ - ib_mad_t* p_mad; +typedef struct _osm_vend_wrap { + ib_mad_t *p_mad; } osm_vend_wrap_t; #ifndef OSM_BIND_INVALID_HANDLE @@ -95,5 +93,4 @@ typedef struct _osm_vend_wrap #endif END_C_DECLS - #endif diff --git a/opensm/include/vendor/osm_vendor_mlx_defs.h b/opensm/include/vendor/osm_vendor_mlx_defs.h index db5aef5..d4c2c30 100644 --- a/opensm/include/vendor/osm_vendor_mlx_defs.h +++ b/opensm/include/vendor/osm_vendor_mlx_defs.h @@ -44,22 +44,18 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* The maximum number of outstanding MADs an RMPP sender can transmit */ #define OSMV_RMPP_RECV_WIN 16 - /* The maximum number of retransmissions of the same MAD */ #define OSMV_MAX_RETRANSMIT 3 - /* Transaction Timeout = OSMV_TXN_TIMEOUT_FACTOR * Response Timeout */ #define OSMV_TXN_TIMEOUT_FACTOR 128 - /************/ /****s* OSM Vendor: Types/osmv_bind_obj_t * NAME @@ -71,39 +67,37 @@ BEGIN_C_DECLS * * SYNOPSIS */ - typedef struct _osmv_bind_obj { - /* Used to signal when the struct is being destroyed */ - struct _osmv_bind_obj* magic_ptr; + /* Used to signal when the struct is being destroyed */ + struct _osmv_bind_obj *magic_ptr; - osm_vendor_t /*const */ *p_vendor; + osm_vendor_t /*const */ * p_vendor; - uint32_t hca_hndl; - uint32_t port_num; + uint32_t hca_hndl; + uint32_t port_num; - /* Atomic access protector */ - cl_spinlock_t lock; + /* Atomic access protector */ + cl_spinlock_t lock; - /* is_closing == TRUE --> the handle is being unbound */ - boolean_t is_closing; + /* is_closing == TRUE --> the handle is being unbound */ + boolean_t is_closing; - /* Event callbacks */ - osm_vend_mad_recv_callback_t recv_cb; - osm_vend_mad_send_err_callback_t send_err_cb; - /* ... and their context */ - void *cb_context; + /* Event callbacks */ + osm_vend_mad_recv_callback_t recv_cb; + osm_vend_mad_send_err_callback_t send_err_cb; + /* ... and their context */ + void *cb_context; - /* A pool to manage MAD wrappers*/ - osm_mad_pool_t *p_osm_pool; + /* A pool to manage MAD wrappers */ + osm_mad_pool_t *p_osm_pool; - /* each subvendor implements its own transport mgr */ - void *p_transp_mgr; + /* each subvendor implements its own transport mgr */ + void *p_transp_mgr; - /* The transaction DB */ - osmv_txn_mgr_t txn_mgr; + /* The transaction DB */ + osmv_txn_mgr_t txn_mgr; } osmv_bind_obj_t; END_C_DECLS - -#endif /* _OSMV_DEFS_H_ */ +#endif /* _OSMV_DEFS_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_dispatcher.h b/opensm/include/vendor/osm_vendor_mlx_dispatcher.h index 5a14d48..ba83f30 100644 --- a/opensm/include/vendor/osm_vendor_mlx_dispatcher.h +++ b/opensm/include/vendor/osm_vendor_mlx_dispatcher.h @@ -39,13 +39,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * NAME * osmv_dispatch_mad @@ -62,12 +61,10 @@ BEGIN_C_DECLS * p_mad_buf The 256 byte buffer of individual MAD * p_mad_addr The MAD originator's address */ - ib_api_status_t -osmv_dispatch_mad(IN osm_bind_handle_t h_bind, - IN const void *p_mad_buf, - IN const osm_mad_addr_t *p_mad_addr); +osmv_dispatch_mad(IN osm_bind_handle_t h_bind, + IN const void *p_mad_buf, + IN const osm_mad_addr_t * p_mad_addr); END_C_DECLS - #endif diff --git a/opensm/include/vendor/osm_vendor_mlx_hca.h b/opensm/include/vendor/osm_vendor_mlx_hca.h index 1052135..9b56943 100644 --- a/opensm/include/vendor/osm_vendor_mlx_hca.h +++ b/opensm/include/vendor/osm_vendor_mlx_hca.h @@ -42,26 +42,23 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if defined( OSM_VENDOR_INTF_TS_NO_VAPI ) || defined( OSM_VENDOR_INTF_SIM ) #define VAPI_hca_hndl_t uint32_t #define VAPI_hca_id_t char* #endif - ib_api_status_t -osm_vendor_get_guid_ca_and_port( IN osm_vendor_t const *p_vend, - IN ib_net64_t const guid, - OUT uint32_t *p_hca_hndl, - OUT char *p_hca_id, - OUT uint8_t *p_hca_idx, - OUT uint32_t *p_port_num ); +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t const *p_vend, + IN ib_net64_t const guid, + OUT uint32_t * p_hca_hndl, + OUT char *p_hca_id, + OUT uint8_t * p_hca_idx, + OUT uint32_t * p_port_num); END_C_DECLS - -#endif /* _OSMV_HCA_H_ */ +#endif /* _OSMV_HCA_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_inout.h b/opensm/include/vendor/osm_vendor_mlx_inout.h index b88fadc..868639b 100644 --- a/opensm/include/vendor/osm_vendor_mlx_inout.h +++ b/opensm/include/vendor/osm_vendor_mlx_inout.h @@ -39,13 +39,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #ifdef IN #undef IN #endif @@ -65,7 +64,6 @@ BEGIN_C_DECLS #ifndef OUT #define OUT #endif - #ifndef OSM_VENDOR_INTF_TS_NO_VAPI #ifndef OSM_VENDOR_INTF_ANAFA #ifndef OSM_VENDOR_INTF_SIM @@ -74,7 +72,5 @@ BEGIN_C_DECLS #endif #endif #endif - END_C_DECLS - -#endif /* _OSMV_INOUT_H_ */ +#endif /* _OSMV_INOUT_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_rmpp_ctx.h b/opensm/include/vendor/osm_vendor_mlx_rmpp_ctx.h index b056eb8..dac1f13 100644 --- a/opensm/include/vendor/osm_vendor_mlx_rmpp_ctx.h +++ b/opensm/include/vendor/osm_vendor_mlx_rmpp_ctx.h @@ -44,48 +44,45 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS typedef struct _osmv_rmpp_send_ctx { - uint8_t status; + uint8_t status; - uint32_t window_first; - uint32_t window_last; + uint32_t window_first; + uint32_t window_last; - uint32_t mad_sz; - boolean_t is_sa_mad; + uint32_t mad_sz; + boolean_t is_sa_mad; - cl_event_t event; + cl_event_t event; - /* Segmentation engine */ - osmv_rmpp_sar_t sar; - osm_log_t *p_log; + /* Segmentation engine */ + osmv_rmpp_sar_t sar; + osm_log_t *p_log; } osmv_rmpp_send_ctx_t; - typedef struct _osmv_rmpp_recv_ctx { - boolean_t is_sa_mad; - - uint32_t expected_seg; - - /* Reassembly buffer */ - cl_qlist_t *p_rbuf; + boolean_t is_sa_mad; - /* Reassembly engine */ - osmv_rmpp_sar_t sar; - osm_log_t* p_log; + uint32_t expected_seg; -} osmv_rmpp_recv_ctx_t; + /* Reassembly buffer */ + cl_qlist_t *p_rbuf; + /* Reassembly engine */ + osmv_rmpp_sar_t sar; + osm_log_t *p_log; +} osmv_rmpp_recv_ctx_t; /* * NAME @@ -98,9 +95,8 @@ typedef struct _osmv_rmpp_recv_ctx { * */ ib_api_status_t -osmv_rmpp_send_ctx_init(osmv_rmpp_send_ctx_t* p_ctx, void* arbt_mad, uint32_t mad_sz, - osm_log_t* p_log); - +osmv_rmpp_send_ctx_init(osmv_rmpp_send_ctx_t * p_ctx, void *arbt_mad, + uint32_t mad_sz, osm_log_t * p_log); /* * NAME @@ -112,9 +108,7 @@ osmv_rmpp_send_ctx_init(osmv_rmpp_send_ctx_t* p_ctx, void* arbt_mad, uint32_t ma * SEE ALSO * */ -void -osmv_rmpp_send_ctx_done(IN osmv_rmpp_send_ctx_t *ctx); - +void osmv_rmpp_send_ctx_done(IN osmv_rmpp_send_ctx_t * ctx); /* * NAME @@ -126,10 +120,10 @@ osmv_rmpp_send_ctx_done(IN osmv_rmpp_send_ctx_t *ctx); * */ static inline uint32_t -osmv_rmpp_send_ctx_get_wf(IN const osmv_rmpp_send_ctx_t *p_ctx) +osmv_rmpp_send_ctx_get_wf(IN const osmv_rmpp_send_ctx_t * p_ctx) { - CL_ASSERT( p_ctx); - return p_ctx->window_first; + CL_ASSERT(p_ctx); + return p_ctx->window_first; } /* @@ -142,10 +136,10 @@ osmv_rmpp_send_ctx_get_wf(IN const osmv_rmpp_send_ctx_t *p_ctx) * */ static inline void -osmv_rmpp_send_ctx_set_wf(IN osmv_rmpp_send_ctx_t *p_ctx, IN uint32_t val) +osmv_rmpp_send_ctx_set_wf(IN osmv_rmpp_send_ctx_t * p_ctx, IN uint32_t val) { - CL_ASSERT( p_ctx); - p_ctx->window_first = val; + CL_ASSERT(p_ctx); + p_ctx->window_first = val; } /* @@ -158,10 +152,10 @@ osmv_rmpp_send_ctx_set_wf(IN osmv_rmpp_send_ctx_t *p_ctx, IN uint32_t val) * */ static inline uint32_t -osmv_rmpp_send_ctx_get_wl(IN const osmv_rmpp_send_ctx_t *p_send_ctx) +osmv_rmpp_send_ctx_get_wl(IN const osmv_rmpp_send_ctx_t * p_send_ctx) { - CL_ASSERT(p_send_ctx); - return p_send_ctx->window_last; + CL_ASSERT(p_send_ctx); + return p_send_ctx->window_last; } /* @@ -174,10 +168,10 @@ osmv_rmpp_send_ctx_get_wl(IN const osmv_rmpp_send_ctx_t *p_send_ctx) * */ static inline void -osmv_rmpp_send_ctx_set_wl(IN osmv_rmpp_send_ctx_t *p_ctx, IN uint32_t val) +osmv_rmpp_send_ctx_set_wl(IN osmv_rmpp_send_ctx_t * p_ctx, IN uint32_t val) { - CL_ASSERT( p_ctx); - p_ctx->window_last = val; + CL_ASSERT(p_ctx); + p_ctx->window_last = val; } /* @@ -189,8 +183,7 @@ osmv_rmpp_send_ctx_set_wl(IN osmv_rmpp_send_ctx_t *p_ctx, IN uint32_t val) * SEE ALSO * */ -uint32_t -osmv_rmpp_send_ctx_get_num_segs(IN osmv_rmpp_send_ctx_t *p_send_ctx); +uint32_t osmv_rmpp_send_ctx_get_num_segs(IN osmv_rmpp_send_ctx_t * p_send_ctx); /* * NAME @@ -202,8 +195,9 @@ osmv_rmpp_send_ctx_get_num_segs(IN osmv_rmpp_send_ctx_t *p_send_ctx); * */ ib_api_status_t -osmv_rmpp_send_ctx_get_seg(IN osmv_rmpp_send_ctx_t *p_send_ctx,IN uint32_t seg_idx, - IN uint32_t resp_timeout, OUT void* p_mad); +osmv_rmpp_send_ctx_get_seg(IN osmv_rmpp_send_ctx_t * p_send_ctx, + IN uint32_t seg_idx, IN uint32_t resp_timeout, + OUT void *p_mad); /* * NAME @@ -215,7 +209,7 @@ osmv_rmpp_send_ctx_get_seg(IN osmv_rmpp_send_ctx_t *p_send_ctx,IN uint32_t seg_i * */ ib_api_status_t -osmv_rmpp_recv_ctx_init(osmv_rmpp_recv_ctx_t *p_ctx,osm_log_t* p_log ); +osmv_rmpp_recv_ctx_init(osmv_rmpp_recv_ctx_t * p_ctx, osm_log_t * p_log); /* * NAME @@ -226,8 +220,7 @@ osmv_rmpp_recv_ctx_init(osmv_rmpp_recv_ctx_t *p_ctx,osm_log_t* p_log ); * SEE ALSO * */ -void -osmv_rmpp_recv_ctx_done(IN osmv_rmpp_recv_ctx_t *p_ctx); +void osmv_rmpp_recv_ctx_done(IN osmv_rmpp_recv_ctx_t * p_ctx); /* * NAME @@ -238,10 +231,10 @@ osmv_rmpp_recv_ctx_done(IN osmv_rmpp_recv_ctx_t *p_ctx); * */ static inline uint32_t -osmv_rmpp_recv_ctx_get_es(IN const osmv_rmpp_recv_ctx_t *p_recv_ctx) +osmv_rmpp_recv_ctx_get_es(IN const osmv_rmpp_recv_ctx_t * p_recv_ctx) { - CL_ASSERT( p_recv_ctx); - return p_recv_ctx->expected_seg; + CL_ASSERT(p_recv_ctx); + return p_recv_ctx->expected_seg; } /* @@ -253,11 +246,10 @@ osmv_rmpp_recv_ctx_get_es(IN const osmv_rmpp_recv_ctx_t *p_recv_ctx) * */ static inline void -osmv_rmpp_recv_ctx_set_es(IN osmv_rmpp_recv_ctx_t *p_recv_ctx, - IN uint32_t val) +osmv_rmpp_recv_ctx_set_es(IN osmv_rmpp_recv_ctx_t * p_recv_ctx, IN uint32_t val) { - CL_ASSERT( p_recv_ctx); - p_recv_ctx->expected_seg = val; + CL_ASSERT(p_recv_ctx); + p_recv_ctx->expected_seg = val; } /* @@ -269,20 +261,18 @@ osmv_rmpp_recv_ctx_set_es(IN osmv_rmpp_recv_ctx_t *p_recv_ctx, * */ ib_api_status_t -osmv_rmpp_recv_ctx_store_mad_seg(IN osmv_rmpp_recv_ctx_t *p_recv_ctx, - IN void* p_mad); - +osmv_rmpp_recv_ctx_store_mad_seg(IN osmv_rmpp_recv_ctx_t * p_recv_ctx, + IN void *p_mad); uint32_t -osmv_rmpp_recv_ctx_get_cur_byte_num(IN osmv_rmpp_recv_ctx_t *p_recv_ctx); +osmv_rmpp_recv_ctx_get_cur_byte_num(IN osmv_rmpp_recv_ctx_t * p_recv_ctx); uint32_t -osmv_rmpp_recv_ctx_get_byte_num_from_first(IN osmv_rmpp_recv_ctx_t *p_recv_ctx); +osmv_rmpp_recv_ctx_get_byte_num_from_first(IN osmv_rmpp_recv_ctx_t * + p_recv_ctx); uint32_t -osmv_rmpp_recv_ctx_get_byte_num_from_last(IN osmv_rmpp_recv_ctx_t *p_recv_ctx); - - +osmv_rmpp_recv_ctx_get_byte_num_from_last(IN osmv_rmpp_recv_ctx_t * p_recv_ctx); /* * NAME @@ -292,10 +282,8 @@ osmv_rmpp_recv_ctx_get_byte_num_from_last(IN osmv_rmpp_recv_ctx_t *p_recv_ctx); * reassembles all rmpp buffs to one big arbitrary mad */ ib_api_status_t -osmv_rmpp_recv_ctx_reassemble_arbt_mad( IN osmv_rmpp_recv_ctx_t *p_recv_ctx, - IN uint32_t size, - IN void* p_arbt_mad); +osmv_rmpp_recv_ctx_reassemble_arbt_mad(IN osmv_rmpp_recv_ctx_t * p_recv_ctx, + IN uint32_t size, IN void *p_arbt_mad); END_C_DECLS - #endif diff --git a/opensm/include/vendor/osm_vendor_mlx_sar.h b/opensm/include/vendor/osm_vendor_mlx_sar.h index 448054a..a65b4f2 100644 --- a/opensm/include/vendor/osm_vendor_mlx_sar.h +++ b/opensm/include/vendor/osm_vendor_mlx_sar.h @@ -42,22 +42,21 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS -typedef struct _osmv_rmpp_sar{ - void* p_arbt_mad; - uint32_t data_len; /* total data len in all the mads */ - /* these data members contain only constants */ - uint32_t hdr_sz; - uint32_t data_sz; /*typical data sz for this kind of mad (sa or regular */ - -}osmv_rmpp_sar_t; +typedef struct _osmv_rmpp_sar { + void *p_arbt_mad; + uint32_t data_len; /* total data len in all the mads */ + /* these data members contain only constants */ + uint32_t hdr_sz; + uint32_t data_sz; /*typical data sz for this kind of mad (sa or regular */ +} osmv_rmpp_sar_t; /* * NAME @@ -70,8 +69,8 @@ typedef struct _osmv_rmpp_sar{ * */ ib_api_status_t -osmv_rmpp_sar_init(osmv_rmpp_sar_t* p_sar, void* p_arbt_mad, - uint32_t mad_size, boolean_t is_sa_mad); +osmv_rmpp_sar_init(osmv_rmpp_sar_t * p_sar, void *p_arbt_mad, + uint32_t mad_size, boolean_t is_sa_mad); /* * NAME @@ -83,8 +82,7 @@ osmv_rmpp_sar_init(osmv_rmpp_sar_t* p_sar, void* p_arbt_mad, * SEE ALSO * */ -void -osmv_rmpp_sar_done(osmv_rmpp_sar_t* p_sar); +void osmv_rmpp_sar_done(osmv_rmpp_sar_t * p_sar); /* * NAME @@ -97,7 +95,8 @@ osmv_rmpp_sar_done(osmv_rmpp_sar_t* p_sar); * */ ib_api_status_t -osmv_rmpp_sar_get_mad_seg(osmv_rmpp_sar_t* p_sar,uint32_t seg_idx,void* p_buf); +osmv_rmpp_sar_get_mad_seg(osmv_rmpp_sar_t * p_sar, uint32_t seg_idx, + void *p_buf); /* * NAME @@ -111,8 +110,7 @@ osmv_rmpp_sar_get_mad_seg(osmv_rmpp_sar_t* p_sar,uint32_t seg_idx,void* p_buf); * */ ib_api_status_t -osmv_rmpp_sar_reassemble_arbt_mad(osmv_rmpp_sar_t* p_sar,cl_qlist_t* p_bufs); +osmv_rmpp_sar_reassemble_arbt_mad(osmv_rmpp_sar_t * p_sar, cl_qlist_t * p_bufs); END_C_DECLS - -#endif /* _OSMV_SAR_H_ */ +#endif /* _OSMV_SAR_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_sender.h b/opensm/include/vendor/osm_vendor_mlx_sender.h index fb72750..e849744 100644 --- a/opensm/include/vendor/osm_vendor_mlx_sender.h +++ b/opensm/include/vendor/osm_vendor_mlx_sender.h @@ -43,13 +43,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****d* OSM Vendor/osmv_simple_send_madw * NAME * osmv_simple_send_madw @@ -62,12 +61,9 @@ BEGIN_C_DECLS * */ ib_api_status_t -osmv_simple_send_madw( - IN osm_bind_handle_t h_bind, - IN osm_madw_t *const p_madw, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_retry - ); +osmv_simple_send_madw(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN osmv_txn_ctx_t * p_txn, IN boolean_t is_retry); /****d* OSM Vendor/osmv_rmpp_send_madw * NAME @@ -82,10 +78,9 @@ osmv_simple_send_madw( * has been acknowledged, or upon error. */ ib_api_status_t -osmv_rmpp_send_madw(IN osm_bind_handle_t h_bind, - IN osm_madw_t *const p_madw, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_rmpp_ds); +osmv_rmpp_send_madw(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN osmv_txn_ctx_t * p_txn, IN boolean_t is_rmpp_ds); /* * NAME osmv_rmpp_send_ack @@ -94,11 +89,10 @@ osmv_rmpp_send_madw(IN osm_bind_handle_t h_bind, */ ib_api_status_t -osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN uint32_t seg_num, - IN uint32_t nwl, - IN const osm_mad_addr_t *p_mad_addr); +osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN uint32_t seg_num, + IN uint32_t nwl, IN const osm_mad_addr_t * p_mad_addr); /* * NAME osmv_rmpp_send_nak @@ -107,11 +101,10 @@ osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, */ ib_api_status_t -osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t nak_type, - IN uint8_t status); +osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t nak_type, IN uint8_t status); /* * NAME osmv_rmpp_snd_error @@ -120,17 +113,16 @@ osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, */ static inline void -osmv_rmpp_snd_error(IN osmv_rmpp_send_ctx_t *p_send_ctx, - IN ib_api_status_t status) +osmv_rmpp_snd_error(IN osmv_rmpp_send_ctx_t * p_send_ctx, + IN ib_api_status_t status) { - p_send_ctx->status = status; + p_send_ctx->status = status; - /* Release the thread waiting on send() - * It will release the transaction's context - */ - cl_event_signal(&p_send_ctx->event); + /* Release the thread waiting on send() + * It will release the transaction's context + */ + cl_event_signal(&p_send_ctx->event); } END_C_DECLS - -#endif /* _OSMV_RMPP_SENDER_H_ */ +#endif /* _OSMV_RMPP_SENDER_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_svc.h b/opensm/include/vendor/osm_vendor_mlx_svc.h index c74c6f3..f23a77d 100644 --- a/opensm/include/vendor/osm_vendor_mlx_svc.h +++ b/opensm/include/vendor/osm_vendor_mlx_svc.h @@ -44,172 +44,158 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - -inline static boolean_t -osmv_mad_is_response(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_mad_is_response(IN const ib_mad_t * p_mad) { - return (ib_mad_is_response(p_mad) || - (p_mad->method == IB_MAD_METHOD_TRAP_REPRESS)); + return (ib_mad_is_response(p_mad) || + (p_mad->method == IB_MAD_METHOD_TRAP_REPRESS)); } -inline static uint8_t -osmv_invert_method(IN uint8_t req_method) +inline static uint8_t osmv_invert_method(IN uint8_t req_method) { - switch(req_method) - { - case IB_MAD_METHOD_GET_RESP: - /* Not a 1-1 mapping! */ - return IB_MAD_METHOD_GET; + switch (req_method) { + case IB_MAD_METHOD_GET_RESP: + /* Not a 1-1 mapping! */ + return IB_MAD_METHOD_GET; - case IB_MAD_METHOD_GET: - return IB_MAD_METHOD_GET_RESP; + case IB_MAD_METHOD_GET: + return IB_MAD_METHOD_GET_RESP; - case IB_MAD_METHOD_SET: - return IB_MAD_METHOD_GET_RESP; + case IB_MAD_METHOD_SET: + return IB_MAD_METHOD_GET_RESP; - case IB_MAD_METHOD_GETTABLE_RESP: - return IB_MAD_METHOD_GETTABLE; + case IB_MAD_METHOD_GETTABLE_RESP: + return IB_MAD_METHOD_GETTABLE; - case IB_MAD_METHOD_GETTABLE: - return IB_MAD_METHOD_GETTABLE_RESP; + case IB_MAD_METHOD_GETTABLE: + return IB_MAD_METHOD_GETTABLE_RESP; - case IB_MAD_METHOD_GETMULTI_RESP: - /* Not a 1-1 mapping! */ - return IB_MAD_METHOD_GETMULTI; + case IB_MAD_METHOD_GETMULTI_RESP: + /* Not a 1-1 mapping! */ + return IB_MAD_METHOD_GETMULTI; - case IB_MAD_METHOD_GETTRACETABLE: - case IB_MAD_METHOD_GETMULTI: - return IB_MAD_METHOD_GETMULTI_RESP; + case IB_MAD_METHOD_GETTRACETABLE: + case IB_MAD_METHOD_GETMULTI: + return IB_MAD_METHOD_GETMULTI_RESP; - case IB_MAD_METHOD_TRAP: - return IB_MAD_METHOD_TRAP_REPRESS; + case IB_MAD_METHOD_TRAP: + return IB_MAD_METHOD_TRAP_REPRESS; - case IB_MAD_METHOD_TRAP_REPRESS: - return IB_MAD_METHOD_TRAP; + case IB_MAD_METHOD_TRAP_REPRESS: + return IB_MAD_METHOD_TRAP; - case IB_MAD_METHOD_REPORT: - return IB_MAD_METHOD_REPORT_RESP; + case IB_MAD_METHOD_REPORT: + return IB_MAD_METHOD_REPORT_RESP; - case IB_MAD_METHOD_REPORT_RESP: - return IB_MAD_METHOD_REPORT; + case IB_MAD_METHOD_REPORT_RESP: + return IB_MAD_METHOD_REPORT; - /* IB_MAD_METHOD_SEND does not have a response */ - case IB_MAD_METHOD_SEND: - return IB_MAD_METHOD_SEND; + /* IB_MAD_METHOD_SEND does not have a response */ + case IB_MAD_METHOD_SEND: + return IB_MAD_METHOD_SEND; - default: CL_ASSERT(FALSE); - } + default: + CL_ASSERT(FALSE); + } - return 0; /* Just make the compiler happy */ + return 0; /* Just make the compiler happy */ } -inline static boolean_t -osmv_mad_is_rmpp(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_mad_is_rmpp(IN const ib_mad_t * p_mad) { - uint8_t rmpp_flags; - CL_ASSERT(NULL != p_mad); - - rmpp_flags = ((ib_rmpp_mad_t*)p_mad)->rmpp_flags; - /* HACK - JUST SA and DevMgt for now - need to add BIS and DevAdm */ - if ( (p_mad->mgmt_class != IB_MCLASS_SUBN_ADM) && - (p_mad->mgmt_class != IB_MCLASS_DEV_MGMT) ) - return(0); - return (0 != (rmpp_flags & IB_RMPP_FLAG_ACTIVE)); + uint8_t rmpp_flags; + CL_ASSERT(NULL != p_mad); + + rmpp_flags = ((ib_rmpp_mad_t *) p_mad)->rmpp_flags; + /* HACK - JUST SA and DevMgt for now - need to add BIS and DevAdm */ + if ((p_mad->mgmt_class != IB_MCLASS_SUBN_ADM) && + (p_mad->mgmt_class != IB_MCLASS_DEV_MGMT)) + return (0); + return (0 != (rmpp_flags & IB_RMPP_FLAG_ACTIVE)); } -inline static boolean_t -osmv_mad_is_multi_resp(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_mad_is_multi_resp(IN const ib_mad_t * p_mad) { - CL_ASSERT(NULL != p_mad); - return (IB_MAD_METHOD_GETMULTI == p_mad->method - || - IB_MAD_METHOD_GETTRACETABLE == p_mad->method); + CL_ASSERT(NULL != p_mad); + return (IB_MAD_METHOD_GETMULTI == p_mad->method + || IB_MAD_METHOD_GETTRACETABLE == p_mad->method); } -inline static boolean_t -osmv_mad_is_sa(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_mad_is_sa(IN const ib_mad_t * p_mad) { - CL_ASSERT(NULL != p_mad); - return (IB_MCLASS_SUBN_ADM == p_mad->mgmt_class); + CL_ASSERT(NULL != p_mad); + return (IB_MCLASS_SUBN_ADM == p_mad->mgmt_class); } -inline static boolean_t -osmv_rmpp_is_abort_stop(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_rmpp_is_abort_stop(IN const ib_mad_t * p_mad) { - uint8_t rmpp_type; - CL_ASSERT(p_mad); + uint8_t rmpp_type; + CL_ASSERT(p_mad); - rmpp_type = ((ib_rmpp_mad_t*)p_mad)->rmpp_type; - return (IB_RMPP_TYPE_STOP == rmpp_type || IB_RMPP_TYPE_ABORT == rmpp_type); + rmpp_type = ((ib_rmpp_mad_t *) p_mad)->rmpp_type; + return (IB_RMPP_TYPE_STOP == rmpp_type + || IB_RMPP_TYPE_ABORT == rmpp_type); } -inline static boolean_t -osmv_rmpp_is_data(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_rmpp_is_data(IN const ib_mad_t * p_mad) { - CL_ASSERT(p_mad); - return (IB_RMPP_TYPE_DATA == ((ib_rmpp_mad_t*)p_mad)->rmpp_type); + CL_ASSERT(p_mad); + return (IB_RMPP_TYPE_DATA == ((ib_rmpp_mad_t *) p_mad)->rmpp_type); } -inline static boolean_t -osmv_rmpp_is_ack(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_rmpp_is_ack(IN const ib_mad_t * p_mad) { - CL_ASSERT(p_mad); - return (IB_RMPP_TYPE_ACK == ((ib_rmpp_mad_t*)p_mad)->rmpp_type); + CL_ASSERT(p_mad); + return (IB_RMPP_TYPE_ACK == ((ib_rmpp_mad_t *) p_mad)->rmpp_type); } -inline static boolean_t -osmv_rmpp_is_first(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_rmpp_is_first(IN const ib_mad_t * p_mad) { - uint8_t rmpp_flags; - CL_ASSERT(NULL != p_mad); + uint8_t rmpp_flags; + CL_ASSERT(NULL != p_mad); - rmpp_flags = ((ib_rmpp_mad_t*)p_mad)->rmpp_flags; - return (0 != (IB_RMPP_FLAG_FIRST & rmpp_flags)); + rmpp_flags = ((ib_rmpp_mad_t *) p_mad)->rmpp_flags; + return (0 != (IB_RMPP_FLAG_FIRST & rmpp_flags)); } -inline static boolean_t -osmv_rmpp_is_last(IN const ib_mad_t *p_mad) +inline static boolean_t osmv_rmpp_is_last(IN const ib_mad_t * p_mad) { - uint8_t rmpp_flags; - CL_ASSERT(NULL != p_mad); + uint8_t rmpp_flags; + CL_ASSERT(NULL != p_mad); - rmpp_flags = ((ib_rmpp_mad_t*)p_mad)->rmpp_flags; - return (0 != (IB_RMPP_FLAG_LAST & rmpp_flags)); + rmpp_flags = ((ib_rmpp_mad_t *) p_mad)->rmpp_flags; + return (0 != (IB_RMPP_FLAG_LAST & rmpp_flags)); } -inline static uint8_t* -osmv_mad_copy(IN const ib_mad_t *p_mad) +inline static uint8_t *osmv_mad_copy(IN const ib_mad_t * p_mad) { - uint8_t *p_copy; + uint8_t *p_copy; - CL_ASSERT(p_mad); - p_copy = malloc(MAD_BLOCK_SIZE); + CL_ASSERT(p_mad); + p_copy = malloc(MAD_BLOCK_SIZE); - if (NULL != p_copy) { - memset(p_copy, 0, MAD_BLOCK_SIZE); - memcpy(p_copy, p_mad, MAD_BLOCK_SIZE); - } + if (NULL != p_copy) { + memset(p_copy, 0, MAD_BLOCK_SIZE); + memcpy(p_copy, p_mad, MAD_BLOCK_SIZE); + } - return p_copy; + return p_copy; } /* Should be passed externally from the Makefile */ /* #define OSMV_RANDOM_DROP 1 */ #define OSMV_DROP_RATE 0.3 -inline static boolean_t -osmv_random_drop(void) +inline static boolean_t osmv_random_drop(void) { - srand(1); /* Pick a new base */ - return (rand() / (double)RAND_MAX < OSMV_DROP_RATE); + srand(1); /* Pick a new base */ + return (rand() / (double)RAND_MAX < OSMV_DROP_RATE); } END_C_DECLS - -#endif /* _OSMV_SVC_H_ */ +#endif /* _OSMV_SVC_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_transport.h b/opensm/include/vendor/osm_vendor_mlx_transport.h index 4dcd1eb..2840e49 100644 --- a/opensm/include/vendor/osm_vendor_mlx_transport.h +++ b/opensm/include/vendor/osm_vendor_mlx_transport.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * NAME * osmv_transport_init @@ -66,13 +65,11 @@ BEGIN_C_DECLS * DESCRIPTION * Setup the MAD transport infrastructure (filters, callbacks etc). */ - #define VENDOR_HCA_MAXNAMES 32 ib_api_status_t -osmv_transport_init( IN osm_bind_info_t *p_info, - IN char hca_id[VENDOR_HCA_MAXNAMES], - IN uint8_t hca_idx, - IN osmv_bind_obj_t *p_bo); +osmv_transport_init(IN osm_bind_info_t * p_info, + IN char hca_id[VENDOR_HCA_MAXNAMES], + IN uint8_t hca_idx, IN osmv_bind_obj_t * p_bo); /* * NAME @@ -82,9 +79,8 @@ osmv_transport_init( IN osm_bind_info_t *p_info, * Send a single MAD (256 byte) */ ib_api_status_t -osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, - IN void *p_mad, - IN const osm_mad_addr_t *p_mad_addr); +osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, + IN void *p_mad, IN const osm_mad_addr_t * p_mad_addr); /* * NAME @@ -93,9 +89,7 @@ osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, * DESCRIPTION * deallocator of transportation infrastructure */ -void -osmv_transport_done(IN const osm_bind_handle_t h_bind); +void osmv_transport_done(IN const osm_bind_handle_t h_bind); END_C_DECLS - -#endif /* _OSMV_TRANSPORT_H_ */ +#endif /* _OSMV_TRANSPORT_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_transport_anafa.h b/opensm/include/vendor/osm_vendor_mlx_transport_anafa.h index 5587a91..dac26ed 100644 --- a/opensm/include/vendor/osm_vendor_mlx_transport_anafa.h +++ b/opensm/include/vendor/osm_vendor_mlx_transport_anafa.h @@ -49,24 +49,21 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSMV_ANAFA_ID 0 - typedef struct _osmv_TOPSPIN_ANAFA_transport_mgr_ { - int device_fd; - cl_thread_t receiver; -}osmv_TOPSPIN_ANAFA_transport_mgr_t; + int device_fd; + cl_thread_t receiver; +} osmv_TOPSPIN_ANAFA_transport_mgr_t; typedef struct _osmv_TOPSPIN_ANAFA_transport_info_ { - int device_fd; -}osmv_TOPSPIN_ANAFA_transport_info_t; + int device_fd; +} osmv_TOPSPIN_ANAFA_transport_info_t; END_C_DECLS - -#endif /* _OSMV_TRANSPORT_ANAFA_H_ */ +#endif /* _OSMV_TRANSPORT_ANAFA_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mlx_txn.h b/opensm/include/vendor/osm_vendor_mlx_txn.h index d05fbfa..ce59191 100644 --- a/opensm/include/vendor/osm_vendor_mlx_txn.h +++ b/opensm/include/vendor/osm_vendor_mlx_txn.h @@ -48,62 +48,58 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS typedef enum _osmv_txn_rmpp_state { - OSMV_TXN_RMPP_NONE = 0, /* Not part of RMPP transaction */ - - OSMV_TXN_RMPP_SENDER, - OSMV_TXN_RMPP_RECEIVER + OSMV_TXN_RMPP_NONE = 0, /* Not part of RMPP transaction */ + OSMV_TXN_RMPP_SENDER, + OSMV_TXN_RMPP_RECEIVER } osmv_txn_rmpp_state_t; - typedef struct _osmv_rmpp_txfr { - osmv_txn_rmpp_state_t rmpp_state; - boolean_t is_rmpp_init_by_peer; - osmv_rmpp_send_ctx_t *p_rmpp_send_ctx; - osmv_rmpp_recv_ctx_t *p_rmpp_recv_ctx; + osmv_txn_rmpp_state_t rmpp_state; + boolean_t is_rmpp_init_by_peer; + osmv_rmpp_send_ctx_t *p_rmpp_send_ctx; + osmv_rmpp_recv_ctx_t *p_rmpp_recv_ctx; } osmv_rmpp_txfr_t; - typedef struct _osmv_txn_ctx { - /* The original Transaction ID */ - uint64_t tid; - /* The key by which the Transaction is stored */ - uint64_t key; + /* The original Transaction ID */ + uint64_t tid; + /* The key by which the Transaction is stored */ + uint64_t key; - /* RMPP Send/Receive contexts, if applicable */ - osmv_rmpp_txfr_t rmpp_txfr; + /* RMPP Send/Receive contexts, if applicable */ + osmv_rmpp_txfr_t rmpp_txfr; - /* A MAD that was sent during the transaction (request or response) */ - osm_madw_t *p_madw; + /* A MAD that was sent during the transaction (request or response) */ + osm_madw_t *p_madw; - /* Reference to a log to enable tracing */ - osm_log_t *p_log; + /* Reference to a log to enable tracing */ + osm_log_t *p_log; } osmv_txn_ctx_t; - typedef struct _osmv_txn_mgr { - /* Container of all the transactions */ - cl_qmap_t *p_txn_map; + /* Container of all the transactions */ + cl_qmap_t *p_txn_map; - /* The timeouts DB */ - cl_event_wheel_t *p_event_wheel; + /* The timeouts DB */ + cl_event_wheel_t *p_event_wheel; - /* Reference to a log to enable tracing */ - osm_log_t *p_log; + /* Reference to a log to enable tracing */ + osm_log_t *p_log; } osmv_txn_mgr_t; @@ -119,11 +115,8 @@ typedef struct _osmv_txn_mgr { * */ ib_api_status_t -osmv_txn_init( IN osm_bind_handle_t h_bind, - IN uint64_t tid, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn); - +osmv_txn_init(IN osm_bind_handle_t h_bind, + IN uint64_t tid, IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn); /* * NAME @@ -136,10 +129,8 @@ osmv_txn_init( IN osm_bind_handle_t h_bind, * */ ib_api_status_t -osmv_txn_init_rmpp_sender(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN osm_madw_t *p_madw); - +osmv_txn_init_rmpp_sender(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, IN osm_madw_t * p_madw); /* * NAME @@ -152,9 +143,9 @@ osmv_txn_init_rmpp_sender(IN osm_bind_handle_t h_bind, * */ ib_api_status_t -osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_init_by_peer); +osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, + IN boolean_t is_init_by_peer); /* * NAME @@ -167,9 +158,8 @@ osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, * */ void -osmv_txn_done(IN osm_bind_handle_t h_bind, - IN uint64_t key, - IN boolean_t is_in_cb); +osmv_txn_done(IN osm_bind_handle_t h_bind, + IN uint64_t key, IN boolean_t is_in_cb); /* * NAME * osmv_txn_get_tid @@ -179,11 +169,10 @@ osmv_txn_done(IN osm_bind_handle_t h_bind, * SEE ALSO * */ -static inline uint64_t -osmv_txn_get_tid(IN osmv_txn_ctx_t* p_txn) +static inline uint64_t osmv_txn_get_tid(IN osmv_txn_ctx_t * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->tid; + CL_ASSERT(NULL != p_txn); + return p_txn->tid; } /* @@ -196,12 +185,12 @@ osmv_txn_get_tid(IN osmv_txn_ctx_t* p_txn) * */ -static inline uint64_t -osmv_txn_get_key(IN osmv_txn_ctx_t* p_txn) +static inline uint64_t osmv_txn_get_key(IN osmv_txn_ctx_t * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->key; + CL_ASSERT(NULL != p_txn); + return p_txn->key; } + /* * NAME * osmv_txn_is_rmpp_init_by_peer @@ -212,11 +201,10 @@ osmv_txn_get_key(IN osmv_txn_ctx_t* p_txn) * SEE ALSO * */ -static inline boolean_t -osmv_txn_is_rmpp_init_by_peer(IN osmv_txn_ctx_t* p_txn) +static inline boolean_t osmv_txn_is_rmpp_init_by_peer(IN osmv_txn_ctx_t * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->rmpp_txfr.is_rmpp_init_by_peer; + CL_ASSERT(NULL != p_txn); + return p_txn->rmpp_txfr.is_rmpp_init_by_peer; } /* @@ -228,11 +216,11 @@ osmv_txn_is_rmpp_init_by_peer(IN osmv_txn_ctx_t* p_txn) * SEE ALSO * */ -static inline osmv_rmpp_send_ctx_t* -osmv_txn_get_rmpp_send_ctx(IN osmv_txn_ctx_t* p_txn) +static inline osmv_rmpp_send_ctx_t *osmv_txn_get_rmpp_send_ctx(IN osmv_txn_ctx_t + * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->rmpp_txfr.p_rmpp_send_ctx; + CL_ASSERT(NULL != p_txn); + return p_txn->rmpp_txfr.p_rmpp_send_ctx; } /* @@ -244,11 +232,11 @@ osmv_txn_get_rmpp_send_ctx(IN osmv_txn_ctx_t* p_txn) * SEE ALSO * */ -static inline osmv_rmpp_recv_ctx_t* -osmv_txn_get_rmpp_recv_ctx(IN osmv_txn_ctx_t* p_txn) +static inline osmv_rmpp_recv_ctx_t *osmv_txn_get_rmpp_recv_ctx(IN osmv_txn_ctx_t + * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->rmpp_txfr.p_rmpp_recv_ctx; + CL_ASSERT(NULL != p_txn); + return p_txn->rmpp_txfr.p_rmpp_recv_ctx; } /* @@ -261,10 +249,10 @@ osmv_txn_get_rmpp_recv_ctx(IN osmv_txn_ctx_t* p_txn) * */ static inline osmv_txn_rmpp_state_t -osmv_txn_get_rmpp_state(IN osmv_txn_ctx_t *p_txn) +osmv_txn_get_rmpp_state(IN osmv_txn_ctx_t * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->rmpp_txfr.rmpp_state; + CL_ASSERT(NULL != p_txn); + return p_txn->rmpp_txfr.rmpp_state; } /* @@ -277,11 +265,11 @@ osmv_txn_get_rmpp_state(IN osmv_txn_ctx_t *p_txn) * */ static inline void -osmv_txn_set_rmpp_state(IN osmv_txn_ctx_t *p_txn, - IN osmv_txn_rmpp_state_t state) +osmv_txn_set_rmpp_state(IN osmv_txn_ctx_t * p_txn, + IN osmv_txn_rmpp_state_t state) { - CL_ASSERT(NULL != p_txn); - p_txn->rmpp_txfr.rmpp_state = state; + CL_ASSERT(NULL != p_txn); + p_txn->rmpp_txfr.rmpp_state = state; } /* @@ -293,11 +281,10 @@ osmv_txn_set_rmpp_state(IN osmv_txn_ctx_t *p_txn, * SEE ALSO * */ -static inline osm_madw_t* -osmv_txn_get_madw(IN osmv_txn_ctx_t *p_txn) +static inline osm_madw_t *osmv_txn_get_madw(IN osmv_txn_ctx_t * p_txn) { - CL_ASSERT(NULL != p_txn); - return p_txn->p_madw; + CL_ASSERT(NULL != p_txn); + return p_txn->p_madw; } /* @@ -310,11 +297,10 @@ osmv_txn_get_madw(IN osmv_txn_ctx_t *p_txn) * */ static inline void -osmv_txn_set_madw(IN osmv_txn_ctx_t *p_txn, - IN osm_madw_t *p_madw) +osmv_txn_set_madw(IN osmv_txn_ctx_t * p_txn, IN osm_madw_t * p_madw) { - CL_ASSERT(NULL != p_txn); - p_txn->p_madw = p_madw; + CL_ASSERT(NULL != p_txn); + p_txn->p_madw = p_madw; } /* @@ -327,9 +313,8 @@ osmv_txn_set_madw(IN osmv_txn_ctx_t *p_txn, * */ ib_api_status_t -osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, - IN uint64_t key, - IN uint64_t msec); +osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, + IN uint64_t key, IN uint64_t msec); /* * NAME * osmv_txn_remove_timeout_ev @@ -339,9 +324,7 @@ osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, * SEE ALSO * */ -void -osmv_txn_remove_timeout_ev(IN osm_bind_handle_t h_bind, - IN uint64_t key); +void osmv_txn_remove_timeout_ev(IN osm_bind_handle_t h_bind, IN uint64_t key); /* * NAME * osmv_txn_lookup @@ -353,14 +336,10 @@ osmv_txn_remove_timeout_ev(IN osm_bind_handle_t h_bind, * */ ib_api_status_t -osmv_txn_lookup(IN osm_bind_handle_t h_bind, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn); - -void -osmv_txn_abort_rmpp_txns(IN osm_bind_handle_t h_bind); - +osmv_txn_lookup(IN osm_bind_handle_t h_bind, + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn); +void osmv_txn_abort_rmpp_txns(IN osm_bind_handle_t h_bind); /* * * * * * * * * * * * */ /* @@ -373,9 +352,8 @@ osmv_txn_abort_rmpp_txns(IN osm_bind_handle_t h_bind); * */ ib_api_status_t -osmv_txnmgr_init(IN osmv_txn_mgr_t *p_tx_mgr, - IN osm_log_t *p_log, - IN cl_spinlock_t *p_lock); +osmv_txnmgr_init(IN osmv_txn_mgr_t * p_tx_mgr, + IN osm_log_t * p_log, IN cl_spinlock_t * p_lock); /* * NAME @@ -386,20 +364,17 @@ osmv_txnmgr_init(IN osmv_txn_mgr_t *p_tx_mgr, * SEE ALSO * */ -void -osmv_txnmgr_done(IN osm_bind_handle_t h_bind); +void osmv_txnmgr_done(IN osm_bind_handle_t h_bind); -void osmv_txn_lock(IN osm_bind_handle_t h_bind); -void osmv_txn_unlock(IN osm_bind_handle_t h_bind); +void osmv_txn_lock(IN osm_bind_handle_t h_bind); +void osmv_txn_unlock(IN osm_bind_handle_t h_bind); -inline static uint64_t -osmv_txn_uniq_key(IN uint64_t tid) +inline static uint64_t osmv_txn_uniq_key(IN uint64_t tid) { - uint64_t pid = getpid(); + uint64_t pid = getpid(); - return ((pid << 32) | (tid & 0xFFFFFFFF)); + return ((pid << 32) | (tid & 0xFFFFFFFF)); } END_C_DECLS - -#endif /* _OSMV_TXN_H_ */ +#endif /* _OSMV_TXN_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mtl.h b/opensm/include/vendor/osm_vendor_mtl.h index 5e3b416..e740b2e 100644 --- a/opensm/include/vendor/osm_vendor_mtl.h +++ b/opensm/include/vendor/osm_vendor_mtl.h @@ -33,8 +33,6 @@ * */ - - /* * Abstract: * Definition of interface for the MTL Vendor @@ -65,13 +63,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Vendor MTL * NAME * Vendor MTL @@ -88,7 +85,6 @@ BEGIN_C_DECLS * * *********/ - /****s* OpenSM: Vendor MTL/osm_ca_info_t * NAME * osm_ca_info_t @@ -98,14 +94,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ca_info -{ +typedef struct _osm_ca_info { ib_net64_t guid; size_t attr_size; ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +} osm_ca_info_t; /* * FIELDS @@ -123,7 +117,6 @@ osm_ca_info_t; #define OSM_DEFAULT_RETRY_COUNT 3 - /***** OpenSM: Vendor MTL/osm_vendor_t * NAME * osm_vendor_t @@ -133,16 +126,14 @@ osm_ca_info_t; * * SYNOPSIS */ -typedef struct _osm_vendor -{ +typedef struct _osm_vendor { ib_al_handle_t h_al; osm_log_t *p_log; uint32_t ca_count; osm_ca_info_t *p_ca_info; uint32_t timeout; - struct osm_transaction_mgr_t *p_transaction_mgr; -} -osm_vendor_t; + struct osm_transaction_mgr_t *p_transaction_mgr; +} osm_vendor_t; /* * FIELDS @@ -167,8 +158,6 @@ osm_vendor_t; * SEE ALSO *********/ - - /****f* OpenSM: Vendor MTL/CA Info/osm_ca_info_get_port_guid * NAME * osm_ca_info_get_port_guid @@ -179,10 +168,10 @@ osm_vendor_t; * SYNOPSIS */ static inline ib_net64_t -osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +osm_ca_info_get_port_guid(IN const osm_ca_info_t * const p_ca_info, + IN const uint8_t index) { - return ( p_ca_info->p_attr->p_port_attr[index].port_guid ); + return (p_ca_info->p_attr->p_port_attr[index].port_guid); } /* @@ -213,9 +202,9 @@ osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, * SYNOPSIS */ static inline uint8_t -osm_ca_info_get_num_ports( IN const osm_ca_info_t * const p_ca_info ) +osm_ca_info_get_num_ports(IN const osm_ca_info_t * const p_ca_info) { - return ( p_ca_info->p_attr->num_ports ); + return (p_ca_info->p_attr->num_ports); } /* @@ -242,11 +231,11 @@ osm_ca_info_get_num_ports( IN const osm_ca_info_t * const p_ca_info ) * SYNOPSIS */ ib_api_status_t -osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT VAPI_hca_hndl_t * p_hca_hndl, - OUT VAPI_hca_id_t * p_hca_id, - OUT uint32_t * p_port_num ); +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT VAPI_hca_hndl_t * p_hca_hndl, + OUT VAPI_hca_id_t * p_hca_id, + OUT uint32_t * p_port_num); /* * PARAMETERS @@ -283,11 +272,10 @@ osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, * * SYNOPSIS */ -ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * - const p_attr_array, - IN uint32_t * - const p_num_ports ); +ib_api_status_t osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * + const p_attr_array, + IN uint32_t * const p_num_ports); /* * PARAMETERS @@ -309,7 +297,6 @@ ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, * SEE ALSO *********/ - #define OSM_BIND_INVALID_HANDLE 0 /****s* OpenSM: Vendor MTL/osm_bind_handle_t @@ -335,13 +322,12 @@ typedef void *osm_bind_handle_t; * * SYNOPSIS *********/ -typedef struct _osm_vend_wrap_t -{ - uint32_t size; - osm_bind_handle_t h_bind; - // ib_av_handle_t h_av; - ib_mad_t *mad_buf_p; - void *p_resp_madw; +typedef struct _osm_vend_wrap_t { + uint32_t size; + osm_bind_handle_t h_bind; + // ib_av_handle_t h_av; + ib_mad_t *mad_buf_p; + void *p_resp_madw; } osm_vend_wrap_t; /* @@ -364,5 +350,4 @@ typedef struct _osm_vend_wrap_t *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_MTL_H_ */ +#endif /* _OSM_VENDOR_MTL_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mtl_hca_guid.h b/opensm/include/vendor/osm_vendor_mtl_hca_guid.h index f27aa54..59bfad5 100644 --- a/opensm/include/vendor/osm_vendor_mtl_hca_guid.h +++ b/opensm/include/vendor/osm_vendor_mtl_hca_guid.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Provides interface over VAPI for obtaining the local ports guids or from guid @@ -51,13 +50,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Vendor AL/osm_ca_info_t * NAME * osm_ca_info_t @@ -67,14 +65,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ca_info -{ +typedef struct _osm_ca_info { ib_net64_t guid; size_t attr_size; ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +} osm_ca_info_t; /* * FIELDS @@ -90,7 +86,6 @@ osm_ca_info_t; * SEE ALSO *********/ - /****f* OpenSM: CA Info/osm_ca_info_get_port_guid * NAME * osm_ca_info_get_port_guid @@ -101,10 +96,10 @@ osm_ca_info_t; * SYNOPSIS */ static inline ib_net64_t -osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +osm_ca_info_get_port_guid(IN const osm_ca_info_t * const p_ca_info, + IN const uint8_t index) { - return ( p_ca_info->p_attr->p_port_attr[index].port_guid ); + return (p_ca_info->p_attr->p_port_attr[index].port_guid); } /* @@ -136,10 +131,10 @@ osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, * SYNOPSIS */ ib_api_status_t -osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT VAPI_hca_id_t * p_hca_id, - OUT uint32_t * p_port_num ); +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT VAPI_hca_id_t * p_hca_id, + OUT uint32_t * p_port_num); /* * PARAMETERS @@ -176,11 +171,10 @@ osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, * * SYNOPSIS */ -ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * - const p_attr_array, - IN uint32_t * - const p_num_ports ); +ib_api_status_t osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * + const p_attr_array, + IN uint32_t * const p_num_ports); /* * PARAMETERS @@ -203,5 +197,4 @@ ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_HCA_GUID_H_ */ +#endif /* _OSM_VENDOR_HCA_GUID_H_ */ diff --git a/opensm/include/vendor/osm_vendor_mtl_transaction_mgr.h b/opensm/include/vendor/osm_vendor_mtl_transaction_mgr.h index 126177c..bdd0714 100644 --- a/opensm/include/vendor/osm_vendor_mtl_transaction_mgr.h +++ b/opensm/include/vendor/osm_vendor_mtl_transaction_mgr.h @@ -33,8 +33,6 @@ * */ - - /* * Abstract: * Definition of interface for the MTL Vendor @@ -50,9 +48,9 @@ #define _OSM_TRANSACTION_MGR_H_ /* -#include -#include -*/ + #include + #include + */ #include #include @@ -73,13 +71,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Transaction Manager/osm_madw_req_t * NAME * osm_madw_req_t @@ -90,14 +87,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_madw_req -{ - cl_list_item_t list_item; - cl_map_item_t map_item; - osm_madw_t *p_madw; - uint64_t waking_time; - uint8_t retry_cnt; - osm_bind_handle_t *p_bind; +typedef struct _osm_madw_req { + cl_list_item_t list_item; + cl_map_item_t map_item; + osm_madw_t *p_madw; + uint64_t waking_time; + uint8_t retry_cnt; + osm_bind_handle_t *p_bind; } osm_madw_req_t; /* @@ -143,14 +139,12 @@ typedef struct _osm_madw_req * * SYNOPSIS */ -typedef struct _osm_transaction_mgr -{ - cl_qmap_t *madw_by_tid_map_p; - cl_qlist_t *madw_reqs_list_p; - cl_spinlock_t transaction_mgr_lock; - cl_timer_t madw_list_timer; -} -osm_transaction_mgr_t; +typedef struct _osm_transaction_mgr { + cl_qmap_t *madw_by_tid_map_p; + cl_qlist_t *madw_reqs_list_p; + cl_spinlock_t transaction_mgr_lock; + cl_timer_t madw_list_timer; +} osm_transaction_mgr_t; /* * FIELDS @@ -178,8 +172,7 @@ osm_transaction_mgr_t; * * SYNOPSIS */ -void -osm_transaction_mgr_init( IN osm_vendor_t * const p_vend ); +void osm_transaction_mgr_init(IN osm_vendor_t * const p_vend); /* * PARAMETERS @@ -188,7 +181,6 @@ osm_transaction_mgr_init( IN osm_vendor_t * const p_vend ); * *********/ - /****f* OpenSM: Transaction Manager/osm_transaction_mgr_destroy * NAME * osm_transaction_mgr_destroy @@ -200,8 +192,7 @@ osm_transaction_mgr_init( IN osm_vendor_t * const p_vend ); * * SYNOPSIS */ -void -osm_transaction_mgr_destroy ( IN osm_vendor_t * const p_vend ); +void osm_transaction_mgr_destroy(IN osm_vendor_t * const p_vend); /* * PARAMETERS @@ -224,8 +215,8 @@ osm_transaction_mgr_destroy ( IN osm_vendor_t * const p_vend ); * SYNOPSIS */ ib_api_status_t -osm_transaction_mgr_insert_madw( IN osm_bind_handle_t *p_bind, - IN osm_madw_t * p_madw ); +osm_transaction_mgr_insert_madw(IN osm_bind_handle_t * p_bind, + IN osm_madw_t * p_madw); /* * PARAMETERS * p_vend @@ -236,7 +227,6 @@ osm_transaction_mgr_insert_madw( IN osm_bind_handle_t *p_bind, * *********/ - /****f* OpenSM: Transaction Manager/osm_transaction_mgr_erase_madw * NAME * osm_transaction_mgr_erase_madw @@ -250,8 +240,8 @@ osm_transaction_mgr_insert_madw( IN osm_bind_handle_t *p_bind, * SYNOPSIS */ ib_api_status_t -osm_transaction_mgr_erase_madw( IN osm_vendor_t * const p_vend, - IN ib_mad_t * p_mad ); +osm_transaction_mgr_erase_madw(IN osm_vendor_t * const p_vend, + IN ib_mad_t * p_mad); /* * PARAMETERS * p_vend @@ -272,9 +262,9 @@ osm_transaction_mgr_erase_madw( IN osm_vendor_t * const p_vend, * SYNOPSIS */ ib_api_status_t -osm_transaction_mgr_get_madw_for_tid( IN osm_vendor_t * const p_vend, - IN ib_mad_t * const p_mad, - OUT osm_madw_t ** req_madw_p ); +osm_transaction_mgr_get_madw_for_tid(IN osm_vendor_t * const p_vend, + IN ib_mad_t * const p_mad, + OUT osm_madw_t ** req_madw_p); /* * PARAMETERS * p_vend @@ -288,7 +278,6 @@ osm_transaction_mgr_get_madw_for_tid( IN osm_vendor_t * const p_vend, * *********/ - /****f* OpenSM: Transaction Manager/osm_transaction_mgr_callback * NAME * osm_transaction_mgr_callback @@ -303,8 +292,7 @@ osm_transaction_mgr_get_madw_for_tid( IN osm_vendor_t * const p_vend, * * SYNOPSIS */ -void -osm_transaction_mgr_callback( IN void* context ); +void osm_transaction_mgr_callback(IN void *context); /* * PARAMETERS * context @@ -313,6 +301,4 @@ osm_transaction_mgr_callback( IN void* context ); *********/ END_C_DECLS - -#endif /* _OSM_TRANSACTION_MGR_H_ */ - +#endif /* _OSM_TRANSACTION_MGR_H_ */ diff --git a/opensm/include/vendor/osm_vendor_sa_api.h b/opensm/include/vendor/osm_vendor_sa_api.h index a260333..3de7fe3 100644 --- a/opensm/include/vendor/osm_vendor_sa_api.h +++ b/opensm/include/vendor/osm_vendor_sa_api.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****d* OpenSM Vendor SA Client/osmv_flags_t * NAME * osmv_flags_t @@ -68,7 +67,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef uint32_t osmv_flags_t; +typedef uint32_t osmv_flags_t; #define OSM_SA_FLAGS_SYNC 0x00000001 /* * VALUES @@ -90,8 +89,7 @@ typedef uint32_t osmv_flags_t; * * SYNOPSIS */ -typedef enum _osmv_query_type -{ +typedef enum _osmv_query_type { OSMV_QUERY_USER_DEFINED, OSMV_QUERY_ALL_SVC_RECS, @@ -116,7 +114,7 @@ typedef enum _osmv_query_type OSMV_QUERY_MULTIPATH_REC, -} osmv_query_type_t; +} osmv_query_type_t; /* * VALUES * OSMV_QUERY_USER_DEFINED @@ -185,15 +183,14 @@ typedef enum _osmv_query_type * * SYNOPSIS */ -typedef struct _osmv_user_query -{ - uint8_t method; - ib_net16_t attr_id; - ib_net16_t attr_offset; - ib_net32_t attr_mod; - ib_net64_t comp_mask; - void *p_attr; -} osmv_user_query_t; +typedef struct _osmv_user_query { + uint8_t method; + ib_net16_t attr_id; + ib_net16_t attr_offset; + ib_net32_t attr_mod; + ib_net64_t comp_mask; + void *p_attr; +} osmv_user_query_t; /* * FIELDS * @@ -238,11 +235,10 @@ typedef struct _osmv_user_query * * SYNOPSIS */ -typedef struct _osmv_gid_pair -{ - ib_gid_t src_gid; - ib_gid_t dest_gid; -} osmv_gid_pair_t; +typedef struct _osmv_gid_pair { + ib_gid_t src_gid; + ib_gid_t dest_gid; +} osmv_gid_pair_t; /* * FIELDS * src_gid @@ -267,11 +263,10 @@ typedef struct _osmv_gid_pair * * SYNOPSIS */ -typedef struct _osmv_lid_pair -{ - ib_net16_t src_lid; - ib_net16_t dest_lid; -} osmv_lid_pair_t; +typedef struct _osmv_lid_pair { + ib_net16_t src_lid; + ib_net16_t dest_lid; +} osmv_lid_pair_t; /* * FIELDS * src_lid @@ -294,11 +289,10 @@ typedef struct _osmv_lid_pair * * SYNOPSIS */ -typedef struct _osmv_guid_pair -{ - ib_net64_t src_guid; - ib_net64_t dest_guid; -} osmv_guid_pair_t; +typedef struct _osmv_guid_pair { + ib_net64_t src_guid; + ib_net64_t dest_guid; +} osmv_guid_pair_t; /* * FIELDS * src_guid @@ -324,17 +318,16 @@ typedef struct _osmv_guid_pair * * SYNOPSIS */ -typedef struct _osmv_multipath_req_t -{ - ib_net64_t comp_mask; - uint16_t pkey; - boolean_t reversible; - uint8_t num_path; - uint8_t sl; - uint8_t independence; - uint8_t sgid_count; - uint8_t dgid_count; - ib_gid_t gids[IB_MULTIPATH_MAX_GIDS]; +typedef struct _osmv_multipath_req_t { + ib_net64_t comp_mask; + uint16_t pkey; + boolean_t reversible; + uint8_t num_path; + uint8_t sl; + uint8_t independence; + uint8_t sgid_count; + uint8_t dgid_count; + ib_gid_t gids[IB_MULTIPATH_MAX_GIDS]; } osmv_multipath_req_t; /* * FIELDS @@ -354,14 +347,13 @@ typedef struct _osmv_multipath_req_t * * SYNOPSIS */ -typedef struct _osmv_query_res -{ - const void *query_context; - ib_api_status_t status; - osmv_query_type_t query_type; - uint32_t result_cnt; - osm_madw_t *p_result_madw; -} osmv_query_res_t; +typedef struct _osmv_query_res { + const void *query_context; + ib_api_status_t status; + osmv_query_type_t query_type; + uint32_t result_cnt; + osm_madw_t *p_result_madw; +} osmv_query_res_t; /* * FIELDS * query_context @@ -413,22 +405,21 @@ typedef struct _osmv_query_res * * SYNOPSIS */ -static inline void* -osmv_get_query_result( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline void *osmv_get_query_result(IN osm_madw_t * p_result_madw, + IN uint32_t result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad ); - CL_ASSERT( ib_get_attr_size( p_sa_mad->attr_offset ) * (result_index + 1) + - IB_SA_MAD_HDR_SIZE <= p_result_madw->mad_size ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad); + CL_ASSERT(ib_get_attr_size(p_sa_mad->attr_offset) * (result_index + 1) + + IB_SA_MAD_HDR_SIZE <= p_result_madw->mad_size); - return( p_sa_mad->data + - (ib_get_attr_size( p_sa_mad->attr_offset ) * result_index) ); + return (p_sa_mad->data + + (ib_get_attr_size(p_sa_mad->attr_offset) * result_index)); } + /* * PARAMETERS * p_result_madw @@ -458,19 +449,20 @@ osmv_get_query_result( * * SYNOPSIS */ -static inline ib_path_rec_t* -osmv_get_query_path_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_path_rec_t *osmv_get_query_path_rec(IN osm_madw_t * + p_result_madw, + IN uint32_t result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_PATH_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_PATH_RECORD); - return( (ib_path_rec_t*)osmv_get_query_result( p_result_madw, result_index ) ); + return ((ib_path_rec_t *) + osmv_get_query_result(p_result_madw, result_index)); } + /* * PARAMETERS * p_result_madw @@ -498,20 +490,21 @@ osmv_get_query_path_rec( * * SYNOPSIS */ -static inline ib_portinfo_record_t* -osmv_get_query_portinfo_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_portinfo_record_t *osmv_get_query_portinfo_rec(IN osm_madw_t * + p_result_madw, + IN uint32_t + result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_PORTINFO_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_PORTINFO_RECORD); - return( (ib_portinfo_record_t*)osmv_get_query_result( p_result_madw, - result_index ) ); + return ((ib_portinfo_record_t *) osmv_get_query_result(p_result_madw, + result_index)); } + /* * PARAMETERS * p_result_madw @@ -539,20 +532,21 @@ osmv_get_query_portinfo_rec( * * SYNOPSIS */ -static inline ib_node_record_t* -osmv_get_query_node_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_node_record_t *osmv_get_query_node_rec(IN osm_madw_t * + p_result_madw, + IN uint32_t + result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_NODE_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_NODE_RECORD); - return( (ib_node_record_t*)osmv_get_query_result( p_result_madw, - result_index ) ); + return ((ib_node_record_t *) osmv_get_query_result(p_result_madw, + result_index)); } + /* * PARAMETERS * p_result_madw @@ -580,20 +574,21 @@ osmv_get_query_node_rec( * * SYNOPSIS */ -static inline ib_service_record_t* -osmv_get_query_svc_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_service_record_t *osmv_get_query_svc_rec(IN osm_madw_t * + p_result_madw, + IN uint32_t + result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_SERVICE_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_SERVICE_RECORD); - return( (ib_service_record_t*)osmv_get_query_result( p_result_madw, - result_index ) ); + return ((ib_service_record_t *) osmv_get_query_result(p_result_madw, + result_index)); } + /* * PARAMETERS * p_result_madw @@ -621,20 +616,20 @@ osmv_get_query_svc_rec( * * SYNOPSIS */ -static inline ib_member_rec_t* -osmv_get_query_mc_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_member_rec_t *osmv_get_query_mc_rec(IN osm_madw_t * + p_result_madw, + IN uint32_t result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_MCMEMBER_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_MCMEMBER_RECORD); - return( (ib_member_rec_t*)osmv_get_query_result( p_result_madw, - result_index ) ); + return ((ib_member_rec_t *) osmv_get_query_result(p_result_madw, + result_index)); } + /* * PARAMETERS * p_result_madw @@ -662,20 +657,25 @@ osmv_get_query_mc_rec( * * SYNOPSIS */ -static inline ib_inform_info_record_t* -osmv_get_query_inform_info_rec( - IN osm_madw_t *p_result_madw, - IN uint32_t result_index ) +static inline ib_inform_info_record_t *osmv_get_query_inform_info_rec(IN + osm_madw_t + * + p_result_madw, + IN + uint32_t + result_index) { - ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_sa_mad; - CL_ASSERT( p_result_madw ); - p_sa_mad = (ib_sa_mad_t*)osm_madw_get_mad_ptr( p_result_madw ); - CL_ASSERT( p_sa_mad && p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO_RECORD ); + CL_ASSERT(p_result_madw); + p_sa_mad = (ib_sa_mad_t *) osm_madw_get_mad_ptr(p_result_madw); + CL_ASSERT(p_sa_mad + && p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO_RECORD); - return( (ib_inform_info_record_t*)osmv_get_query_result( p_result_madw, - result_index ) ); + return ((ib_inform_info_record_t *) osmv_get_query_result(p_result_madw, + result_index)); } + /* * PARAMETERS * p_result_madw @@ -704,8 +704,7 @@ osmv_get_query_inform_info_rec( * SYNOPSIS */ typedef void -(*osmv_pfn_query_cb_t)( - IN osmv_query_res_t *p_query_res ); + (*osmv_pfn_query_cb_t) (IN osmv_query_res_t * p_query_res); /* * PARAMETERS * p_query_res @@ -735,19 +734,18 @@ typedef void * * SYNOPSIS */ -typedef struct _osmv_query_req -{ - osmv_query_type_t query_type; - const void *p_query_input; - ib_net64_t sm_key; - - uint32_t timeout_ms; - uint32_t retry_cnt; - osmv_flags_t flags; - - const void *query_context; - osmv_pfn_query_cb_t pfn_query_cb; -} osmv_query_req_t; +typedef struct _osmv_query_req { + osmv_query_type_t query_type; + const void *p_query_input; + ib_net64_t sm_key; + + uint32_t timeout_ms; + uint32_t retry_cnt; + osmv_flags_t flags; + + const void *query_context; + osmv_pfn_query_cb_t pfn_query_cb; +} osmv_query_req_t; /* * FIELDS * query_type @@ -816,11 +814,8 @@ typedef struct _osmv_query_req * SYNOPSIS */ osm_bind_handle_t -osmv_bind_sa( - IN osm_vendor_t * const p_vend, - IN osm_mad_pool_t * const p_mad_pool, - IN ib_net64_t port_guid - ); +osmv_bind_sa(IN osm_vendor_t * const p_vend, + IN osm_mad_pool_t * const p_mad_pool, IN ib_net64_t port_guid); /* * PARAMETERS * p_vend @@ -851,10 +846,8 @@ osmv_bind_sa( * SYNOPSIS */ ib_api_status_t -osmv_query_sa( - IN osm_bind_handle_t h_bind, - IN const osmv_query_req_t * const p_query_req - ); +osmv_query_sa(IN osm_bind_handle_t h_bind, + IN const osmv_query_req_t * const p_query_req); /* * PARAMETERS * h_bind @@ -875,5 +868,4 @@ osmv_query_sa( *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_SA_API_H_ */ +#endif /* _OSM_VENDOR_SA_API_H_ */ diff --git a/opensm/include/vendor/osm_vendor_select.h b/opensm/include/vendor/osm_vendor_select.h index c63cfc4..5bd6ba6 100644 --- a/opensm/include/vendor/osm_vendor_select.h +++ b/opensm/include/vendor/osm_vendor_select.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Include file that defines which vendor files to compile. @@ -65,12 +64,12 @@ #ifndef OSM_VENDOR_INTF_SIM #ifndef OSM_VENDOR_INTF_AL #define OSM_VENDOR_INTF_AL -#endif /* AL */ -#endif /* TS */ -#endif /* SIM */ -#endif /* MTL */ -#endif /* UMADT */ -#endif /* TEST */ -#endif /* OPENIB */ +#endif /* AL */ +#endif /* TS */ +#endif /* SIM */ +#endif /* MTL */ +#endif /* UMADT */ +#endif /* TEST */ +#endif /* OPENIB */ -#endif /* _OSM_VENDOR_SELECT_H_ */ +#endif /* _OSM_VENDOR_SELECT_H_ */ diff --git a/opensm/include/vendor/osm_vendor_test.h b/opensm/include/vendor/osm_vendor_test.h index 6757ef8..26e6314 100644 --- a/opensm/include/vendor/osm_vendor_test.h +++ b/opensm/include/vendor/osm_vendor_test.h @@ -33,7 +33,6 @@ * */ - #ifndef _OSM_VENDOR_TEST_H_ #define _OSM_VENDOR_TEST_H_ @@ -44,17 +43,14 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* This value must be zero for the TEST transport. */ #define OSM_BIND_INVALID_HANDLE 0 - - /* * Abstract: * Declaration of vendor specific transport interface. @@ -67,7 +63,6 @@ BEGIN_C_DECLS * * $Revision: 1.4 $ */ - /****h* OpenSM/Vendor Test * NAME * Vendor Test @@ -91,9 +86,8 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_vend_wrap -{ - uint32_t dummy; +typedef struct _osm_vend_wrap { + uint32_t dummy; } osm_vend_wrap_t; /*********/ @@ -110,28 +104,25 @@ typedef struct _osm_vend_wrap * * SYNOPSIS */ -typedef struct _osm_vendor -{ - osm_log_t *p_log; - uint32_t timeout; +typedef struct _osm_vendor { + osm_log_t *p_log; + uint32_t timeout; } osm_vendor_t; /*********/ -typedef struct _osm_bind_handle -{ - osm_vendor_t *p_vend; - ib_net64_t port_guid; - uint8_t mad_class; - uint8_t class_version; - boolean_t is_responder; - boolean_t is_trap_processor; - boolean_t is_report_processor; - uint32_t send_q_size; - uint32_t recv_q_size; +typedef struct _osm_bind_handle { + osm_vendor_t *p_vend; + ib_net64_t port_guid; + uint8_t mad_class; + uint8_t class_version; + boolean_t is_responder; + boolean_t is_trap_processor; + boolean_t is_report_processor; + uint32_t send_q_size; + uint32_t recv_q_size; } *osm_bind_handle_t; END_C_DECLS - -#endif /* _OSM_VENDOR_TEST_H_ */ +#endif /* _OSM_VENDOR_TEST_H_ */ diff --git a/opensm/include/vendor/osm_vendor_ts.h b/opensm/include/vendor/osm_vendor_ts.h index 35e28c1..2c46fb9 100644 --- a/opensm/include/vendor/osm_vendor_ts.h +++ b/opensm/include/vendor/osm_vendor_ts.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Definition of interface for the TS Vendor @@ -65,13 +64,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Vendor TS/osm_bind_handle_t * NAME * osm_bind_handle_t @@ -95,17 +93,16 @@ typedef void *osm_bind_handle_t; * * SYNOPSIS */ -typedef struct _osm_ts_bind_info -{ - int ul_dev_fd; - VAPI_hca_hndl_t hca_hndl; - struct _osm_vendor *p_vend; - void *client_context; - uint8_t port_num; - void *rcv_callback; - void *send_err_callback; - struct _osm_mad_pool *p_osm_pool; - cl_thread_t poller; +typedef struct _osm_ts_bind_info { + int ul_dev_fd; + VAPI_hca_hndl_t hca_hndl; + struct _osm_vendor *p_vend; + void *client_context; + uint8_t port_num; + void *rcv_callback; + void *send_err_callback; + struct _osm_mad_pool *p_osm_pool; + cl_thread_t poller; } osm_ts_bind_info_t; /* * FIELDS @@ -168,14 +165,12 @@ typedef struct _osm_ts_bind_info * * SYNOPSIS */ -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +} osm_ca_info_t; /* * FIELDS @@ -191,7 +186,6 @@ osm_ca_info_t; * SEE ALSO *********/ - /***** OpenSM: Vendor TS/osm_vendor_t * NAME * osm_vendor_t @@ -201,15 +195,14 @@ osm_ca_info_t; * * SYNOPSIS */ -typedef struct _osm_vendor -{ - osm_log_t *p_log; - uint32_t ca_count; - osm_ca_info_t *p_ca_info; - uint32_t timeout; - struct _osm_transaction_mgr *p_transaction_mgr; - osm_ts_bind_info_t smi_bind; - osm_ts_bind_info_t gsi_bind; +typedef struct _osm_vendor { + osm_log_t *p_log; + uint32_t ca_count; + osm_ca_info_t *p_ca_info; + uint32_t timeout; + struct _osm_transaction_mgr *p_transaction_mgr; + osm_ts_bind_info_t smi_bind; + osm_ts_bind_info_t gsi_bind; } osm_vendor_t; /* @@ -241,7 +234,6 @@ typedef struct _osm_vendor * SEE ALSO *********/ - /****f* OpenSM: Vendor TS/CA Info/osm_ca_info_get_port_guid * NAME * osm_ca_info_get_port_guid @@ -252,10 +244,10 @@ typedef struct _osm_vendor * SYNOPSIS */ static inline ib_net64_t -osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +osm_ca_info_get_port_guid(IN const osm_ca_info_t * const p_ca_info, + IN const uint8_t index) { - return ( p_ca_info->p_attr->p_port_attr[index].port_guid ); + return (p_ca_info->p_attr->p_port_attr[index].port_guid); } /* @@ -286,9 +278,9 @@ osm_ca_info_get_port_guid( IN const osm_ca_info_t * const p_ca_info, * SYNOPSIS */ static inline uint8_t -osm_ca_info_get_num_ports( IN const osm_ca_info_t * const p_ca_info ) +osm_ca_info_get_num_ports(IN const osm_ca_info_t * const p_ca_info) { - return ( p_ca_info->p_attr->num_ports ); + return (p_ca_info->p_attr->num_ports); } /* @@ -315,11 +307,11 @@ osm_ca_info_get_num_ports( IN const osm_ca_info_t * const p_ca_info ) * SYNOPSIS */ ib_api_status_t -osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT VAPI_hca_hndl_t * p_hca_hndl, - OUT VAPI_hca_id_t * p_hca_id, - OUT uint32_t * p_port_num ); +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT VAPI_hca_hndl_t * p_hca_hndl, + OUT VAPI_hca_id_t * p_hca_id, + OUT uint32_t * p_port_num); /* * PARAMETERS @@ -356,11 +348,10 @@ osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, * * SYNOPSIS */ -ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * - const p_attr_array, - IN uint32_t * - const p_num_ports ); +ib_api_status_t osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * + const p_attr_array, + IN uint32_t * const p_num_ports); /* * PARAMETERS @@ -382,7 +373,6 @@ ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, * SEE ALSO *********/ - #define OSM_BIND_INVALID_HANDLE 0 /****s* OpenSM: Vendor TS/osm_vend_wrap_t @@ -395,12 +385,11 @@ ib_api_status_t osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, * * SYNOPSIS *********/ -typedef struct _osm_vend_wrap_t -{ - uint32_t size; - osm_bind_handle_t h_bind; - ib_mad_t *p_mad_buf; - void *p_resp_madw; +typedef struct _osm_vend_wrap_t { + uint32_t size; + osm_bind_handle_t h_bind; + ib_mad_t *p_mad_buf; + void *p_resp_madw; } osm_vend_wrap_t; /* @@ -423,5 +412,4 @@ typedef struct _osm_vend_wrap_t *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_TS_H_ */ +#endif /* _OSM_VENDOR_TS_H_ */ diff --git a/opensm/include/vendor/osm_vendor_umadt.h b/opensm/include/vendor/osm_vendor_umadt.h index 4964947..68b7dfb 100644 --- a/opensm/include/vendor/osm_vendor_umadt.h +++ b/opensm/include/vendor/osm_vendor_umadt.h @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Declaration of osm_mad_wrapper_t. @@ -58,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/ Vendor Umadt * NAME * MAD Wrapper @@ -76,10 +74,9 @@ BEGIN_C_DECLS * Ranjit Pandit, Intel * *********/ -typedef void* osm_vendor_t; +typedef void *osm_vendor_t; #define OSM_BIND_INVALID_HANDLE 0 - /****s* OpenSM: Vendor Umadt /osm_bind_handle_t * NAME * osm_bind_handle_t @@ -90,7 +87,7 @@ typedef void* osm_vendor_t; * SYNOPSIS */ -typedef void* osm_bind_handle_t; +typedef void *osm_bind_handle_t; /****s* OpenSM: Vendor Umadt /mad_direction_t * NAME @@ -104,8 +101,8 @@ typedef void* osm_bind_handle_t; * SYNOPSIS */ typedef enum _mad_direction_t { - SEND =0, - RECEIVE, + SEND = 0, + RECEIVE, } mad_direction_t; /****s* OpenSM/ osm_vend_wrap_t @@ -119,10 +116,10 @@ typedef enum _mad_direction_t { * SYNOPSIS *********/ typedef struct _osm_vend_wrap_t { - MadtStruct *p_madt_struct; - mad_direction_t direction; // send or receive - uint32_t size; -}osm_vend_wrap_t; + MadtStruct *p_madt_struct; + mad_direction_t direction; // send or receive + uint32_t size; +} osm_vend_wrap_t; /* * FIELDS * p_madt_struct @@ -135,5 +132,4 @@ typedef struct _osm_vend_wrap_t { *********/ END_C_DECLS - -#endif /* _OSM_VENDOR_UMADT_h_ */ +#endif /* _OSM_VENDOR_UMADT_h_ */ -- 1.5.3.rc2.29.gc4640f From sashak at voltaire.com Wed Aug 15 14:55:42 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 16 Aug 2007 00:55:42 +0300 Subject: [ofa-general] [PATCH] opensm/libvendor: indentation changes Message-ID: <20070815215542.GB27000@sashak.voltaire.com> Formatted with opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs) Signed-off-by: Sasha Khapyorsky --- opensm/libvendor/osm_pkt_randomizer.c | 474 +++--- opensm/libvendor/osm_vendor_al.c | 2185 ++++++++++----------- opensm/libvendor/osm_vendor_ibumad.c | 542 +++--- opensm/libvendor/osm_vendor_ibumad_sa.c | 1411 +++++++------- opensm/libvendor/osm_vendor_mlx.c | 1192 ++++++------ opensm/libvendor/osm_vendor_mlx_anafa.c | 517 +++--- opensm/libvendor/osm_vendor_mlx_dispatcher.c | 1115 +++++------ opensm/libvendor/osm_vendor_mlx_hca.c | 843 ++++----- opensm/libvendor/osm_vendor_mlx_hca_anafa.c | 74 +- opensm/libvendor/osm_vendor_mlx_hca_pfs.c | 1192 ++++++------ opensm/libvendor/osm_vendor_mlx_hca_sim.c | 1389 +++++++------- opensm/libvendor/osm_vendor_mlx_ibmgt.c | 1358 +++++++------- opensm/libvendor/osm_vendor_mlx_rmpp_ctx.c | 542 +++--- opensm/libvendor/osm_vendor_mlx_sa.c | 1352 ++++++------- opensm/libvendor/osm_vendor_mlx_sar.c | 198 +- opensm/libvendor/osm_vendor_mlx_sender.c | 538 +++--- opensm/libvendor/osm_vendor_mlx_sim.c | 659 +++---- opensm/libvendor/osm_vendor_mlx_ts.c | 791 ++++---- opensm/libvendor/osm_vendor_mlx_ts_anafa.c | 203 +- opensm/libvendor/osm_vendor_mlx_txn.c | 1016 +++++----- opensm/libvendor/osm_vendor_mtl.c | 1863 +++++++++--------- opensm/libvendor/osm_vendor_mtl_hca_guid.c | 995 +++++----- opensm/libvendor/osm_vendor_mtl_transaction_mgr.c | 897 +++++----- opensm/libvendor/osm_vendor_test.c | 303 ++-- opensm/libvendor/osm_vendor_ts.c | 1479 +++++++-------- opensm/libvendor/osm_vendor_umadt.c | 1877 +++++++++--------- 26 files changed, 12076 insertions(+), 12929 deletions(-) diff --git a/opensm/libvendor/osm_pkt_randomizer.c b/opensm/libvendor/osm_pkt_randomizer.c index 33993b6..70962e7 100644 --- a/opensm/libvendor/osm_pkt_randomizer.c +++ b/opensm/libvendor/osm_pkt_randomizer.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_pkt_randomizer_t. @@ -46,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -67,46 +66,45 @@ * the given paths: 0,1 or 0,3,1,4 - the return value will be FALSE. **********************************************************************/ boolean_t -__osm_pkt_randomizer_is_path_in_fault_paths( - IN osm_log_t *p_log, - IN osm_dr_path_t *p_dr_path, - IN osm_pkt_randomizer_t *p_pkt_rand ) +__osm_pkt_randomizer_is_path_in_fault_paths(IN osm_log_t * p_log, + IN osm_dr_path_t * p_dr_path, + IN osm_pkt_randomizer_t * + p_pkt_rand) { - boolean_t res = FALSE, found_path; - osm_dr_path_t *p_found_dr_path; - uint8_t ind1, ind2; - - OSM_LOG_ENTER( p_log, __osm_pkt_randomizer_is_path_in_fault_paths ); - - for ( ind1 = 0 ; ind1 < p_pkt_rand->num_paths_initialized ; ind1++ ) - { - found_path = TRUE; - p_found_dr_path = &(p_pkt_rand->fault_dr_paths[ind1]); - /* if the hop count of the found path is greater than the - hop count of the input path - then it is not part of it. - Check the next path. */ - if (p_found_dr_path->hop_count > p_dr_path->hop_count) - continue; - - /* go over all the ports in the found path and see if they match - the ports in the input path */ - for ( ind2 = 0 ; ind2 <= p_found_dr_path->hop_count ; ind2++ ) - if (p_found_dr_path->path[ind2] != p_dr_path->path[ind2]) - found_path = FALSE; - - /* If found_path is TRUE then there is a full match of the path */ - if ( found_path == TRUE ) - { - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_is_path_in_fault_paths: " - "Given path is in a fault path\n"); - res = TRUE; - break; - } - } - - OSM_LOG_EXIT( p_log ); - return res; + boolean_t res = FALSE, found_path; + osm_dr_path_t *p_found_dr_path; + uint8_t ind1, ind2; + + OSM_LOG_ENTER(p_log, __osm_pkt_randomizer_is_path_in_fault_paths); + + for (ind1 = 0; ind1 < p_pkt_rand->num_paths_initialized; ind1++) { + found_path = TRUE; + p_found_dr_path = &(p_pkt_rand->fault_dr_paths[ind1]); + /* if the hop count of the found path is greater than the + hop count of the input path - then it is not part of it. + Check the next path. */ + if (p_found_dr_path->hop_count > p_dr_path->hop_count) + continue; + + /* go over all the ports in the found path and see if they match + the ports in the input path */ + for (ind2 = 0; ind2 <= p_found_dr_path->hop_count; ind2++) + if (p_found_dr_path->path[ind2] != + p_dr_path->path[ind2]) + found_path = FALSE; + + /* If found_path is TRUE then there is a full match of the path */ + if (found_path == TRUE) { + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_pkt_randomizer_is_path_in_fault_paths: " + "Given path is in a fault path\n"); + res = TRUE; + break; + } + } + + OSM_LOG_EXIT(p_log); + return res; } /********************************************************************** @@ -121,228 +119,224 @@ __osm_pkt_randomizer_is_path_in_fault_paths( * if to drop it or not. **********************************************************************/ boolean_t -__osm_pkt_randomizer_process_path( - IN osm_log_t *p_log, - IN osm_pkt_randomizer_t *p_pkt_rand, - IN osm_dr_path_t *p_dr_path ) +__osm_pkt_randomizer_process_path(IN osm_log_t * p_log, + IN osm_pkt_randomizer_t * p_pkt_rand, + IN osm_dr_path_t * p_dr_path) { - boolean_t res = FALSE; - static boolean_t rand_value_init = FALSE; - static int rand_value; - boolean_t in_fault_paths; - uint8_t i; - char buf[BUF_SIZE]; - char line[BUF_SIZE]; - - OSM_LOG_ENTER( p_log, __osm_pkt_randomizer_process_path ); - - if (rand_value_init == FALSE) - { - int seed; + boolean_t res = FALSE; + static boolean_t rand_value_init = FALSE; + static int rand_value; + boolean_t in_fault_paths; + uint8_t i; + char buf[BUF_SIZE]; + char line[BUF_SIZE]; + + OSM_LOG_ENTER(p_log, __osm_pkt_randomizer_process_path); + + if (rand_value_init == FALSE) { + int seed; #ifdef WIN32 - SYSTEMTIME st; + SYSTEMTIME st; #else - struct timeval tv; - struct timezone tz; -#endif /* WIN32 */ + struct timeval tv; + struct timezone tz; +#endif /* WIN32 */ - /* initiate the rand_value according to timeofday */ - rand_value_init = TRUE; + /* initiate the rand_value according to timeofday */ + rand_value_init = TRUE; #ifdef WIN32 - GetLocalTime(&st); - seed = st.wMilliseconds; + GetLocalTime(&st); + seed = st.wMilliseconds; #else - gettimeofday( &tv, &tz ); - seed = tv.tv_usec; -#endif /* WIN32 */ - - srand(seed); - } - - /* If the hop_count is 1 - then this is a mad down to our local port - don't drop it */ - if ( p_dr_path->hop_count <= 1 ) - goto Exit; - - rand_value = rand(); - - sprintf( buf, "Path: " ); - /* update the dr_path into the buf */ - for (i = 0 ; i <= p_dr_path->hop_count; i++ ) - { - sprintf( line, "[%X]", p_dr_path->path[i] ); - strcat( buf, line ); - } - - /* Check if the path given is in one of the fault paths */ - in_fault_paths = __osm_pkt_randomizer_is_path_in_fault_paths( - p_log, p_dr_path, p_pkt_rand ); - - /* Check if all paths are initialized */ - if ( p_pkt_rand->num_paths_initialized < p_pkt_rand->osm_pkt_num_unstable_links ) - { - /* Not all packets are initialized. */ - if ( in_fault_paths == FALSE ) - { - /* the path is not in the false paths. Check using the rand value - if to update it there or not. */ - if ( rand_value%(p_pkt_rand->osm_pkt_unstable_link_rate) == 0 ) - { - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " - "%s added to the fault_dr_paths list\n" - "\t\t\t rand_value:%u, unstable_link_rate:%u \n", - buf, rand_value, p_pkt_rand->osm_pkt_unstable_link_rate ); - - /* update the path in the fault paths */ - memcpy( &(p_pkt_rand->fault_dr_paths[p_pkt_rand->num_paths_initialized]), - p_dr_path, sizeof(osm_dr_path_t) ); - p_pkt_rand->num_paths_initialized++; - in_fault_paths = TRUE; - } - } - } - - if ( in_fault_paths == FALSE ) - { - /* If in_fault_paths is FALSE - just ignore the path */ - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " - "%s not in fault paths\n", buf ); - goto Exit; - } - - /* The path is in the fault paths. Need to choose (randomally if to drop it - or not. */ - rand_value = rand(); - - if ( rand_value%(p_pkt_rand->osm_pkt_drop_rate) == 0 ) - { - /* drop the current packet */ - res = TRUE; - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_pkt_randomizer_process_path: " - "Dropping path:%s\n", buf ); - } - - Exit: - OSM_LOG_EXIT( p_log ); - return res; + gettimeofday(&tv, &tz); + seed = tv.tv_usec; +#endif /* WIN32 */ + + srand(seed); + } + + /* If the hop_count is 1 - then this is a mad down to our local port - don't drop it */ + if (p_dr_path->hop_count <= 1) + goto Exit; + + rand_value = rand(); + + sprintf(buf, "Path: "); + /* update the dr_path into the buf */ + for (i = 0; i <= p_dr_path->hop_count; i++) { + sprintf(line, "[%X]", p_dr_path->path[i]); + strcat(buf, line); + } + + /* Check if the path given is in one of the fault paths */ + in_fault_paths = + __osm_pkt_randomizer_is_path_in_fault_paths(p_log, p_dr_path, + p_pkt_rand); + + /* Check if all paths are initialized */ + if (p_pkt_rand->num_paths_initialized < + p_pkt_rand->osm_pkt_num_unstable_links) { + /* Not all packets are initialized. */ + if (in_fault_paths == FALSE) { + /* the path is not in the false paths. Check using the rand value + if to update it there or not. */ + if (rand_value % + (p_pkt_rand->osm_pkt_unstable_link_rate) == 0) { + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_pkt_randomizer_process_path: " + "%s added to the fault_dr_paths list\n" + "\t\t\t rand_value:%u, unstable_link_rate:%u \n", + buf, rand_value, + p_pkt_rand->osm_pkt_unstable_link_rate); + + /* update the path in the fault paths */ + memcpy(& + (p_pkt_rand-> + fault_dr_paths[p_pkt_rand-> + num_paths_initialized]), + p_dr_path, sizeof(osm_dr_path_t)); + p_pkt_rand->num_paths_initialized++; + in_fault_paths = TRUE; + } + } + } + + if (in_fault_paths == FALSE) { + /* If in_fault_paths is FALSE - just ignore the path */ + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_pkt_randomizer_process_path: " + "%s not in fault paths\n", buf); + goto Exit; + } + + /* The path is in the fault paths. Need to choose (randomally if to drop it + or not. */ + rand_value = rand(); + + if (rand_value % (p_pkt_rand->osm_pkt_drop_rate) == 0) { + /* drop the current packet */ + res = TRUE; + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_pkt_randomizer_process_path: " + "Dropping path:%s\n", buf); + } + + Exit: + OSM_LOG_EXIT(p_log); + return res; } /********************************************************************** **********************************************************************/ boolean_t -osm_pkt_randomizer_mad_drop( - IN osm_log_t *p_log, - IN osm_pkt_randomizer_t *p_pkt_randomizer, - IN const ib_mad_t *p_mad ) +osm_pkt_randomizer_mad_drop(IN osm_log_t * p_log, + IN osm_pkt_randomizer_t * p_pkt_randomizer, + IN const ib_mad_t * p_mad) { - const ib_smp_t* p_smp; - boolean_t res = FALSE; - osm_dr_path_t dr_path; - - OSM_LOG_ENTER( p_log, osm_pkt_randomizer_mad_drop ); - - p_smp = (ib_smp_t*)p_mad; - - if ( p_smp->mgmt_class != IB_MCLASS_SUBN_DIR ) - /* This is a lid route mad. Don't drop it */ - goto Exit; - - osm_dr_path_init( &dr_path, - 0, /* The h_bind is not really important for us to save */ - p_smp->hop_count, p_smp->initial_path ); - - if (__osm_pkt_randomizer_process_path( p_log, p_pkt_randomizer, &dr_path ) ) - { - /* the mad should be dropped o*/ - osm_log ( p_log, OSM_LOG_VERBOSE, - "osm_pkt_randomizer_mad_drop: " - "mad TID: 0x%" PRIx64 " is being dropped\n", - cl_ntoh64(p_smp->trans_id) ); - res = TRUE; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return res; + const ib_smp_t *p_smp; + boolean_t res = FALSE; + osm_dr_path_t dr_path; + + OSM_LOG_ENTER(p_log, osm_pkt_randomizer_mad_drop); + + p_smp = (ib_smp_t *) p_mad; + + if (p_smp->mgmt_class != IB_MCLASS_SUBN_DIR) + /* This is a lid route mad. Don't drop it */ + goto Exit; + + osm_dr_path_init(&dr_path, 0, /* The h_bind is not really important for us to save */ + p_smp->hop_count, p_smp->initial_path); + + if (__osm_pkt_randomizer_process_path + (p_log, p_pkt_randomizer, &dr_path)) { + /* the mad should be dropped o */ + osm_log(p_log, OSM_LOG_VERBOSE, + "osm_pkt_randomizer_mad_drop: " + "mad TID: 0x%" PRIx64 " is being dropped\n", + cl_ntoh64(p_smp->trans_id)); + res = TRUE; + } + + Exit: + OSM_LOG_EXIT(p_log); + return res; } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pkt_randomizer_init( - IN OUT osm_pkt_randomizer_t **pp_pkt_randomizer, - IN osm_log_t *p_log ) +osm_pkt_randomizer_init(IN OUT osm_pkt_randomizer_t ** pp_pkt_randomizer, + IN osm_log_t * p_log) { - uint8_t tmp; - ib_api_status_t res = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_pkt_randomizer_init ); - - *pp_pkt_randomizer = malloc( sizeof( osm_pkt_randomizer_t ) ); - if ( *pp_pkt_randomizer == NULL ) - { - res = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - memset( *pp_pkt_randomizer, 0, sizeof(osm_pkt_randomizer_t) ); - (*pp_pkt_randomizer)->num_paths_initialized = 0; - - tmp = atol( getenv("OSM_PKT_DROP_RATE") ); - (*pp_pkt_randomizer)->osm_pkt_drop_rate = tmp; - - if ( getenv("OSM_PKT_NUM_UNSTABLE_LINKS") != NULL && (tmp = atol( getenv("OSM_PKT_NUM_UNSTABLE_LINKS") )) > 0 ) - (*pp_pkt_randomizer)->osm_pkt_num_unstable_links = tmp; - else - (*pp_pkt_randomizer)->osm_pkt_num_unstable_links = 1; - - if ( getenv("OSM_PKT_UNSTABLE_LINK_RATE") != NULL && (tmp = atol( getenv("OSM_PKT_UNSTABLE_LINK_RATE") )) > 0 ) - (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate = tmp; - else - (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate = 20; - - osm_log( p_log, OSM_LOG_VERBOSE, - "osm_pkt_randomizer_init: " - "Using OSM_PKT_DROP_RATE=%u \n" - "\t\t\t\t OSM_PKT_NUM_UNSTABLE_LINKS=%u \n" - "\t\t\t\t OSM_PKT_UNSTABLE_LINK_RATE=%u \n", - (*pp_pkt_randomizer)->osm_pkt_drop_rate, - (*pp_pkt_randomizer)->osm_pkt_num_unstable_links, - (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate ); - - /* allocate the fault_dr_paths variable */ - /* It is the number of the paths that will be saved as fault = osm_pkt_num_unstable_links */ - (*pp_pkt_randomizer)->fault_dr_paths = malloc( sizeof( osm_dr_path_t ) * - (*pp_pkt_randomizer)->osm_pkt_num_unstable_links ); - if ( (*pp_pkt_randomizer)->fault_dr_paths == NULL ) - { - res = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - memset( (*pp_pkt_randomizer)->fault_dr_paths, 0, - sizeof( osm_dr_path_t ) * (*pp_pkt_randomizer)->osm_pkt_num_unstable_links ); - - Exit: - OSM_LOG_EXIT( p_log ); - return (res); + uint8_t tmp; + ib_api_status_t res = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_pkt_randomizer_init); + + *pp_pkt_randomizer = malloc(sizeof(osm_pkt_randomizer_t)); + if (*pp_pkt_randomizer == NULL) { + res = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + memset(*pp_pkt_randomizer, 0, sizeof(osm_pkt_randomizer_t)); + (*pp_pkt_randomizer)->num_paths_initialized = 0; + + tmp = atol(getenv("OSM_PKT_DROP_RATE")); + (*pp_pkt_randomizer)->osm_pkt_drop_rate = tmp; + + if (getenv("OSM_PKT_NUM_UNSTABLE_LINKS") != NULL + && (tmp = atol(getenv("OSM_PKT_NUM_UNSTABLE_LINKS"))) > 0) + (*pp_pkt_randomizer)->osm_pkt_num_unstable_links = tmp; + else + (*pp_pkt_randomizer)->osm_pkt_num_unstable_links = 1; + + if (getenv("OSM_PKT_UNSTABLE_LINK_RATE") != NULL + && (tmp = atol(getenv("OSM_PKT_UNSTABLE_LINK_RATE"))) > 0) + (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate = tmp; + else + (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate = 20; + + osm_log(p_log, OSM_LOG_VERBOSE, + "osm_pkt_randomizer_init: " + "Using OSM_PKT_DROP_RATE=%u \n" + "\t\t\t\t OSM_PKT_NUM_UNSTABLE_LINKS=%u \n" + "\t\t\t\t OSM_PKT_UNSTABLE_LINK_RATE=%u \n", + (*pp_pkt_randomizer)->osm_pkt_drop_rate, + (*pp_pkt_randomizer)->osm_pkt_num_unstable_links, + (*pp_pkt_randomizer)->osm_pkt_unstable_link_rate); + + /* allocate the fault_dr_paths variable */ + /* It is the number of the paths that will be saved as fault = osm_pkt_num_unstable_links */ + (*pp_pkt_randomizer)->fault_dr_paths = malloc(sizeof(osm_dr_path_t) * + (*pp_pkt_randomizer)-> + osm_pkt_num_unstable_links); + if ((*pp_pkt_randomizer)->fault_dr_paths == NULL) { + res = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + memset((*pp_pkt_randomizer)->fault_dr_paths, 0, + sizeof(osm_dr_path_t) * + (*pp_pkt_randomizer)->osm_pkt_num_unstable_links); + + Exit: + OSM_LOG_EXIT(p_log); + return (res); } /********************************************************************** **********************************************************************/ void -osm_pkt_randomizer_destroy( - IN OUT osm_pkt_randomizer_t **pp_pkt_randomizer, - IN osm_log_t *p_log ) +osm_pkt_randomizer_destroy(IN OUT osm_pkt_randomizer_t ** pp_pkt_randomizer, + IN osm_log_t * p_log) { - OSM_LOG_ENTER( p_log, osm_pkt_randomizer_destroy ); - - if ( *pp_pkt_randomizer != NULL ) - { - free( (*pp_pkt_randomizer)->fault_dr_paths ); - free( *pp_pkt_randomizer ); - } - OSM_LOG_EXIT( p_log ); + OSM_LOG_ENTER(p_log, osm_pkt_randomizer_destroy); + + if (*pp_pkt_randomizer != NULL) { + free((*pp_pkt_randomizer)->fault_dr_paths); + free(*pp_pkt_randomizer); + } + OSM_LOG_EXIT(p_log); } diff --git a/opensm/libvendor/osm_vendor_al.c b/opensm/libvendor/osm_vendor_al.c index 1b748ae..7ad8547 100644 --- a/opensm/libvendor/osm_vendor_al.c +++ b/opensm/libvendor/osm_vendor_al.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_req_t. @@ -52,7 +51,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -79,18 +78,17 @@ * * SYNOPSIS */ -typedef struct _osm_al_bind_info -{ - osm_vendor_t *p_vend; - void *client_context; - ib_qp_handle_t h_qp; - ib_mad_svc_handle_t h_svc; - uint8_t port_num; - ib_pool_key_t pool_key; - osm_vend_mad_recv_callback_t rcv_callback; - osm_vend_mad_send_err_callback_t send_err_callback; - osm_mad_pool_t *p_osm_pool; - ib_av_handle_t h_dr_av; +typedef struct _osm_al_bind_info { + osm_vendor_t *p_vend; + void *client_context; + ib_qp_handle_t h_qp; + ib_mad_svc_handle_t h_svc; + uint8_t port_num; + ib_pool_key_t pool_key; + osm_vend_mad_recv_callback_t rcv_callback; + osm_vend_mad_send_err_callback_t send_err_callback; + osm_mad_pool_t *p_osm_pool; + ib_av_handle_t h_dr_av; } osm_al_bind_info_t; /* @@ -122,500 +120,453 @@ typedef struct _osm_al_bind_info /********************************************************************** **********************************************************************/ inline static ib_api_status_t -__osm_al_convert_wcs( - IN ib_wc_status_t const wc_status ) +__osm_al_convert_wcs(IN ib_wc_status_t const wc_status) { - switch( wc_status ) - { - case IB_WCS_SUCCESS: - return( IB_SUCCESS ); + switch (wc_status) { + case IB_WCS_SUCCESS: + return (IB_SUCCESS); - case IB_WCS_TIMEOUT_RETRY_ERR: - return( IB_TIMEOUT ); + case IB_WCS_TIMEOUT_RETRY_ERR: + return (IB_TIMEOUT); - default: - return( IB_ERROR ); - } + default: + return (IB_ERROR); + } } /********************************************************************** **********************************************************************/ -static void -__osm_al_ca_err_callback( - IN ib_async_event_rec_t *p_async_rec ) +static void __osm_al_ca_err_callback(IN ib_async_event_rec_t * p_async_rec) { - osm_vendor_t *p_vend = (osm_vendor_t*)p_async_rec->context; - OSM_LOG_ENTER( p_vend->p_log, __osm_al_ca_err_callback ); + osm_vendor_t *p_vend = (osm_vendor_t *) p_async_rec->context; + OSM_LOG_ENTER(p_vend->p_log, __osm_al_ca_err_callback); - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_al_ca_err_callback: ERR 3B01: " - "Event on channel adapter (%s).\n", - ib_get_async_event_str( p_async_rec->code ) ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_al_ca_err_callback: ERR 3B01: " + "Event on channel adapter (%s).\n", + ib_get_async_event_str(p_async_rec->code)); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -static void -__osm_al_ca_destroy_callback( - IN void *context ) +static void __osm_al_ca_destroy_callback(IN void *context) { - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t*)context; - osm_vendor_t *p_vend = p_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, __osm_al_ca_destroy_callback ); + osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *) context; + osm_vendor_t *p_vend = p_bind->p_vend; + OSM_LOG_ENTER(p_vend->p_log, __osm_al_ca_destroy_callback); - osm_log( p_vend->p_log, OSM_LOG_INFO, - "__osm_al_ca_destroy_callback: " - "Closing local channel adapter.\n" ); + osm_log(p_vend->p_log, OSM_LOG_INFO, + "__osm_al_ca_destroy_callback: " + "Closing local channel adapter.\n"); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -static void -__osm_al_err_callback( - IN ib_async_event_rec_t *p_async_rec ) +static void __osm_al_err_callback(IN ib_async_event_rec_t * p_async_rec) { - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t*)p_async_rec->context; - osm_vendor_t *p_vend = p_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, __osm_al_err_callback ); + osm_al_bind_info_t *p_bind = + (osm_al_bind_info_t *) p_async_rec->context; + osm_vendor_t *p_vend = p_bind->p_vend; + OSM_LOG_ENTER(p_vend->p_log, __osm_al_err_callback); - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_al_err_callback: ERR 3B02: " - "Error on QP (%s).\n", - ib_get_async_event_str( p_async_rec->code ) ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_al_err_callback: ERR 3B02: " + "Error on QP (%s).\n", + ib_get_async_event_str(p_async_rec->code)); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_al_send_callback( - IN void *mad_svc_context, - IN ib_mad_element_t *p_elem ) +__osm_al_send_callback(IN void *mad_svc_context, IN ib_mad_element_t * p_elem) { - osm_al_bind_info_t* const p_bind = (osm_al_bind_info_t*)mad_svc_context; - osm_vendor_t* const p_vend = p_bind->p_vend; - osm_madw_t* const p_madw = (osm_madw_t*)p_elem->context1; - osm_vend_wrap_t* const p_vw = osm_madw_get_vend_ptr( p_madw ); - ib_mad_t *p_mad; - - OSM_LOG_ENTER( p_vend->p_log, __osm_al_send_callback ); - - CL_ASSERT( p_vw ); - CL_ASSERT( p_vw->h_av ); - - /* - Destroy the address vector as necessary. - */ - if( p_vw->h_av != p_bind->h_dr_av ) - { - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_al_send_callback: " - "Destroying av handle %p.\n", p_vw->h_av ); - } - - ib_destroy_av( p_vw->h_av ); - } - - p_mad = ib_get_mad_buf( p_elem ); - - if( p_elem->resp_expected ) - { - /* - If the send was unsuccessful, notify the user - for MADs that were expecting a response. - A NULL mad wrapper parameter is the user's clue - that the transaction turned sour. - - Otherwise, do nothing for successful sends when a - reponse is expected. The mad will be returned to the - pool later. - */ - p_madw->status = __osm_al_convert_wcs( p_elem->status ); - if( p_elem->status != IB_WCS_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_al_send_callback: " - "MAD completed with work queue error: %s.\n", - ib_get_wc_status_str( p_elem->status ) ); - /* - Return any wrappers to the pool that may have been - pre-emptively allocated to handle a receive. - */ - if( p_vw->p_resp_madw ) - { - osm_mad_pool_put( p_bind->p_osm_pool, p_vw->p_resp_madw ); - p_vw->p_resp_madw = NULL; - } - - p_bind->send_err_callback( p_bind->client_context, p_madw ); - } - } - else - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_al_send_callback: " - "Returning MAD to pool, TID = 0x%" PRIx64 ".\n", - cl_ntoh64( p_mad->trans_id ) ); - osm_mad_pool_put( p_bind->p_osm_pool, p_madw ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); + osm_al_bind_info_t *const p_bind = + (osm_al_bind_info_t *) mad_svc_context; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_madw_t *const p_madw = (osm_madw_t *) p_elem->context1; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + ib_mad_t *p_mad; + + OSM_LOG_ENTER(p_vend->p_log, __osm_al_send_callback); + + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->h_av); + + /* + Destroy the address vector as necessary. + */ + if (p_vw->h_av != p_bind->h_dr_av) { + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_al_send_callback: " + "Destroying av handle %p.\n", p_vw->h_av); + } + + ib_destroy_av(p_vw->h_av); + } + + p_mad = ib_get_mad_buf(p_elem); + + if (p_elem->resp_expected) { + /* + If the send was unsuccessful, notify the user + for MADs that were expecting a response. + A NULL mad wrapper parameter is the user's clue + that the transaction turned sour. + + Otherwise, do nothing for successful sends when a + reponse is expected. The mad will be returned to the + pool later. + */ + p_madw->status = __osm_al_convert_wcs(p_elem->status); + if (p_elem->status != IB_WCS_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_al_send_callback: " + "MAD completed with work queue error: %s.\n", + ib_get_wc_status_str(p_elem->status)); + /* + Return any wrappers to the pool that may have been + pre-emptively allocated to handle a receive. + */ + if (p_vw->p_resp_madw) { + osm_mad_pool_put(p_bind->p_osm_pool, + p_vw->p_resp_madw); + p_vw->p_resp_madw = NULL; + } + + p_bind->send_err_callback(p_bind->client_context, + p_madw); + } + } else { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_al_send_callback: " + "Returning MAD to pool, TID = 0x%" PRIx64 ".\n", + cl_ntoh64(p_mad->trans_id)); + osm_mad_pool_put(p_bind->p_osm_pool, p_madw); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_al_rcv_callback( - IN void *mad_svc_context, - IN ib_mad_element_t *p_elem ) +__osm_al_rcv_callback(IN void *mad_svc_context, IN ib_mad_element_t * p_elem) { - osm_al_bind_info_t* const p_bind = (osm_al_bind_info_t*)mad_svc_context; - osm_vendor_t* const p_vend = p_bind->p_vend; - osm_madw_t *p_old_madw; - osm_madw_t *p_new_madw; - osm_vend_wrap_t* p_old_vw; - osm_vend_wrap_t* p_new_vw; - ib_mad_t *p_new_mad; - osm_mad_addr_t mad_addr; - - OSM_LOG_ENTER( p_vend->p_log, __osm_al_rcv_callback ); - - CL_ASSERT( p_elem->context1 == NULL ); - CL_ASSERT( p_elem->context2 == NULL ); - - p_new_mad = ib_get_mad_buf( p_elem ); - - /* - In preperation for initializing the new mad wrapper, - Initialize the mad_addr structure for the received wire MAD. - */ - mad_addr.dest_lid = p_elem->remote_lid; - mad_addr.path_bits = p_elem->path_bits; - - /* TO DO - figure out which #define to use for the 2.5 Gb rate... */ - mad_addr.static_rate = 0; - - if( p_new_mad->mgmt_class == IB_MCLASS_SUBN_LID || - p_new_mad->mgmt_class == IB_MCLASS_SUBN_DIR ) - { - mad_addr.addr_type.smi.source_lid = p_elem->remote_lid; - } - else - { - mad_addr.addr_type.gsi.remote_qp = p_elem->remote_qp; - mad_addr.addr_type.gsi.remote_qkey = p_elem->remote_qkey; - mad_addr.addr_type.gsi.pkey = p_elem->pkey_index; - mad_addr.addr_type.gsi.service_level = p_elem->remote_sl; - mad_addr.addr_type.gsi.global_route = FALSE; - } - - /* - If this MAD is a response to a previous request, - then grab our pre-allocated MAD wrapper. - Otherwise, allocate a new MAD wrapper. - */ - if( ib_mad_is_response( p_new_mad ) ) - { - CL_ASSERT( p_elem->send_context1 != NULL ); - CL_ASSERT( p_elem->send_context2 == NULL ); - - p_old_madw = (osm_madw_t*)p_elem->send_context1; - p_old_vw = osm_madw_get_vend_ptr( p_old_madw ); - p_new_madw = p_old_vw->p_resp_madw; - - CL_ASSERT( p_new_madw ); - - osm_madw_init( p_new_madw, p_bind, p_elem->size, - &mad_addr ); - osm_madw_set_mad( p_new_madw, p_new_mad ); - } - else - { - CL_ASSERT( p_elem->send_context1 == NULL ); - CL_ASSERT( p_elem->send_context2 == NULL ); - - p_new_madw = osm_mad_pool_get_wrapper( p_bind->p_osm_pool, - p_bind, p_elem->size, p_new_mad, &mad_addr ); - } - - CL_ASSERT(p_new_madw); - p_new_vw = osm_madw_get_vend_ptr( p_new_madw ); - - - p_new_vw->h_bind = p_bind; - p_new_vw->size = p_elem->size; - p_new_vw->p_elem = p_elem; - p_new_vw->h_av = 0; - p_new_vw->p_resp_madw = NULL; - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_al_rcv_callback: " - "Calling receive callback function %p.\n", - p_bind->rcv_callback ); - - p_bind->rcv_callback( p_new_madw, p_bind->client_context, - p_elem->send_context1 ); - - OSM_LOG_EXIT( p_vend->p_log ); + osm_al_bind_info_t *const p_bind = + (osm_al_bind_info_t *) mad_svc_context; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_madw_t *p_old_madw; + osm_madw_t *p_new_madw; + osm_vend_wrap_t *p_old_vw; + osm_vend_wrap_t *p_new_vw; + ib_mad_t *p_new_mad; + osm_mad_addr_t mad_addr; + + OSM_LOG_ENTER(p_vend->p_log, __osm_al_rcv_callback); + + CL_ASSERT(p_elem->context1 == NULL); + CL_ASSERT(p_elem->context2 == NULL); + + p_new_mad = ib_get_mad_buf(p_elem); + + /* + In preperation for initializing the new mad wrapper, + Initialize the mad_addr structure for the received wire MAD. + */ + mad_addr.dest_lid = p_elem->remote_lid; + mad_addr.path_bits = p_elem->path_bits; + + /* TO DO - figure out which #define to use for the 2.5 Gb rate... */ + mad_addr.static_rate = 0; + + if (p_new_mad->mgmt_class == IB_MCLASS_SUBN_LID || + p_new_mad->mgmt_class == IB_MCLASS_SUBN_DIR) { + mad_addr.addr_type.smi.source_lid = p_elem->remote_lid; + } else { + mad_addr.addr_type.gsi.remote_qp = p_elem->remote_qp; + mad_addr.addr_type.gsi.remote_qkey = p_elem->remote_qkey; + mad_addr.addr_type.gsi.pkey = p_elem->pkey_index; + mad_addr.addr_type.gsi.service_level = p_elem->remote_sl; + mad_addr.addr_type.gsi.global_route = FALSE; + } + + /* + If this MAD is a response to a previous request, + then grab our pre-allocated MAD wrapper. + Otherwise, allocate a new MAD wrapper. + */ + if (ib_mad_is_response(p_new_mad)) { + CL_ASSERT(p_elem->send_context1 != NULL); + CL_ASSERT(p_elem->send_context2 == NULL); + + p_old_madw = (osm_madw_t *) p_elem->send_context1; + p_old_vw = osm_madw_get_vend_ptr(p_old_madw); + p_new_madw = p_old_vw->p_resp_madw; + + CL_ASSERT(p_new_madw); + + osm_madw_init(p_new_madw, p_bind, p_elem->size, &mad_addr); + osm_madw_set_mad(p_new_madw, p_new_mad); + } else { + CL_ASSERT(p_elem->send_context1 == NULL); + CL_ASSERT(p_elem->send_context2 == NULL); + + p_new_madw = osm_mad_pool_get_wrapper(p_bind->p_osm_pool, + p_bind, p_elem->size, + p_new_mad, &mad_addr); + } + + CL_ASSERT(p_new_madw); + p_new_vw = osm_madw_get_vend_ptr(p_new_madw); + + p_new_vw->h_bind = p_bind; + p_new_vw->size = p_elem->size; + p_new_vw->p_elem = p_elem; + p_new_vw->h_av = 0; + p_new_vw->p_resp_madw = NULL; + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_al_rcv_callback: " + "Calling receive callback function %p.\n", + p_bind->rcv_callback); + + p_bind->rcv_callback(p_new_madw, p_bind->client_context, + p_elem->send_context1); + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_init( - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_vendor_init ); - - p_vend->p_log = p_log; - - /* - Open our instance of AL. - */ - status = ib_open_al( &p_vend->h_al ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: ERR 3B03: " - "Error opening AL (%s).\n", - ib_get_err_str( status ) ); - - goto Exit; - } - - p_vend->timeout = timeout; - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status; + OSM_LOG_ENTER(p_log, osm_vendor_init); + + p_vend->p_log = p_log; + + /* + Open our instance of AL. + */ + status = ib_open_al(&p_vend->h_al); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_init: ERR 3B03: " + "Error opening AL (%s).\n", ib_get_err_str(status)); + + goto Exit; + } + + p_vend->timeout = timeout; + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - ib_api_status_t status; - osm_vendor_t *p_vend; - - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - p_vend = malloc( sizeof(*p_vend) ); - if( p_vend == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 3B04: " - "Unable to allocate vendor object.\n" ); - goto Exit; - } - - memset( p_vend, 0, sizeof(*p_vend) ); - - status = osm_vendor_init( p_vend, p_log, timeout ); - if( status != IB_SUCCESS ) - { - free( p_vend ); - p_vend = NULL; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( p_vend ); + ib_api_status_t status; + osm_vendor_t *p_vend; + + OSM_LOG_ENTER(p_log, osm_vendor_new); + + p_vend = malloc(sizeof(*p_vend)); + if (p_vend == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 3B04: " + "Unable to allocate vendor object.\n"); + goto Exit; + } + + memset(p_vend, 0, sizeof(*p_vend)); + + status = osm_vendor_init(p_vend, p_log, timeout); + if (status != IB_SUCCESS) { + free(p_vend); + p_vend = NULL; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (p_vend); } /********************************************************************** **********************************************************************/ -void -osm_vendor_delete( - IN osm_vendor_t** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - /* TO DO - fill this in */ - ib_close_al( (*pp_vend)->h_al ); - free( *pp_vend ); - *pp_vend = NULL; + /* TO DO - fill this in */ + ib_close_al((*pp_vend)->h_al); + free(*pp_vend); + *pp_vend = NULL; } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_ca_info_init( - IN osm_vendor_t* const p_vend, - IN osm_ca_info_t* const p_ca_info, - IN const ib_net64_t ca_guid ) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info, + IN const ib_net64_t ca_guid) { - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, __osm_ca_info_init ); - - p_ca_info->guid = ca_guid; - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "__osm_ca_info_init: " - "Querying CA 0x%" PRIx64 ".\n", - cl_ntoh64( ca_guid ) ); - } - - status = ib_query_ca_by_guid( p_vend->h_al, ca_guid, NULL, - &p_ca_info->attr_size ); - if( (status != IB_INSUFFICIENT_MEMORY ) && (status != IB_SUCCESS ) ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3B05: " - "Unexpected status getting CA attributes (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - CL_ASSERT( p_ca_info->attr_size ); - - p_ca_info->p_attr = malloc( p_ca_info->attr_size ); - if( p_ca_info->p_attr == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3B06: " - "Unable to allocate attribute storage.\n" ); - goto Exit; - } - - status = ib_query_ca_by_guid( p_vend->h_al, ca_guid, p_ca_info->p_attr, - &p_ca_info->attr_size ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3B07: " - "Unexpected status getting CA attributes (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); + + p_ca_info->guid = ca_guid; + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "__osm_ca_info_init: " + "Querying CA 0x%" PRIx64 ".\n", cl_ntoh64(ca_guid)); + } + + status = ib_query_ca_by_guid(p_vend->h_al, ca_guid, NULL, + &p_ca_info->attr_size); + if ((status != IB_INSUFFICIENT_MEMORY) && (status != IB_SUCCESS)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3B05: " + "Unexpected status getting CA attributes (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + CL_ASSERT(p_ca_info->attr_size); + + p_ca_info->p_attr = malloc(p_ca_info->attr_size); + if (p_ca_info->p_attr == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3B06: " + "Unable to allocate attribute storage.\n"); + goto Exit; + } + + status = ib_query_ca_by_guid(p_vend->h_al, ca_guid, p_ca_info->p_attr, + &p_ca_info->attr_size); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3B07: " + "Unexpected status getting CA attributes (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } - /********************************************************************** **********************************************************************/ void -osm_ca_info_destroy( - IN osm_vendor_t* const p_vend, - IN osm_ca_info_t* const p_ca_info ) +osm_ca_info_destroy(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info) { - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_destroy ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_destroy); - if( p_ca_info->p_attr ) - free( p_ca_info->p_attr ); + if (p_ca_info->p_attr) + free(p_ca_info->p_attr); - free( p_ca_info ); + free(p_ca_info); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -osm_ca_info_t* -osm_ca_info_new( - IN osm_vendor_t* const p_vend, - IN const ib_net64_t ca_guid ) +osm_ca_info_t *osm_ca_info_new(IN osm_vendor_t * const p_vend, + IN const ib_net64_t ca_guid) { - ib_api_status_t status; - osm_ca_info_t *p_ca_info; + ib_api_status_t status; + osm_ca_info_t *p_ca_info; - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_new ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_new); - CL_ASSERT( ca_guid ); + CL_ASSERT(ca_guid); - p_ca_info = malloc( sizeof(*p_ca_info) ); - if( p_ca_info == NULL ) - goto Exit; + p_ca_info = malloc(sizeof(*p_ca_info)); + if (p_ca_info == NULL) + goto Exit; - memset( p_ca_info, 0, sizeof(*p_ca_info) ); + memset(p_ca_info, 0, sizeof(*p_ca_info)); - status = __osm_ca_info_init( p_vend, p_ca_info, ca_guid ); - if( status != IB_SUCCESS ) - { - osm_ca_info_destroy( p_vend, p_ca_info ); - p_ca_info = NULL; - goto Exit; - } + status = __osm_ca_info_init(p_vend, p_ca_info, ca_guid); + if (status != IB_SUCCESS) { + osm_ca_info_destroy(p_vend, p_ca_info); + p_ca_info = NULL; + goto Exit; + } - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( p_ca_info ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (p_ca_info); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_vendor_get_ca_guids( - IN osm_vendor_t* const p_vend, - IN ib_net64_t** const p_guids, - IN uintn_t* const p_num_guids ) +__osm_vendor_get_ca_guids(IN osm_vendor_t * const p_vend, + IN ib_net64_t ** const p_guids, + IN uintn_t * const p_num_guids) { - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_get_ca_guids ); - - CL_ASSERT( p_guids ); - CL_ASSERT( p_num_guids ); - - status = ib_get_ca_guids( p_vend->h_al, NULL, p_num_guids ); - if( (status != IB_INSUFFICIENT_MEMORY ) && (status != IB_SUCCESS ) ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_guids: ERR 3B08: " - "Unexpected status getting CA GUID array (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - if( *p_num_guids == 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_guids: ERR 3B09: " - "No available channel adapters.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - *p_guids = malloc( *p_num_guids * sizeof(**p_guids) ); - if( *p_guids == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_guids: ERR 3B10: " - "Unable to allocate CA GUID array.\n" ); - goto Exit; - } - - status = ib_get_ca_guids( p_vend->h_al, *p_guids, p_num_guids ); - CL_ASSERT( *p_num_guids ); - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "__osm_vendor_get_ca_guids: " - "Detected %u local channel adapters.\n", *p_num_guids ); - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_get_ca_guids); + + CL_ASSERT(p_guids); + CL_ASSERT(p_num_guids); + + status = ib_get_ca_guids(p_vend->h_al, NULL, p_num_guids); + if ((status != IB_INSUFFICIENT_MEMORY) && (status != IB_SUCCESS)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_guids: ERR 3B08: " + "Unexpected status getting CA GUID array (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + if (*p_num_guids == 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_guids: ERR 3B09: " + "No available channel adapters.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + *p_guids = malloc(*p_num_guids * sizeof(**p_guids)); + if (*p_guids == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_guids: ERR 3B10: " + "Unable to allocate CA GUID array.\n"); + goto Exit; + } + + status = ib_get_ca_guids(p_vend->h_al, *p_guids, p_num_guids); + CL_ASSERT(*p_num_guids); + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "__osm_vendor_get_ca_guids: " + "Detected %u local channel adapters.\n", *p_num_guids); + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /****f* OpenSM: CA Info/osm_ca_info_get_pi_ptr @@ -628,13 +579,13 @@ __osm_vendor_get_ca_guids( * * SYNOPSIS */ -static ib_port_attr_t* -__osm_ca_info_get_port_attr_ptr( - IN const osm_ca_info_t* const p_ca_info, - IN const uint8_t index ) +static ib_port_attr_t *__osm_ca_info_get_port_attr_ptr(IN const osm_ca_info_t * + const p_ca_info, + IN const uint8_t index) { - return( &p_ca_info->p_attr->p_port_attr[index] ); + return (&p_ca_info->p_attr->p_port_attr[index]); } + /* * PARAMETERS * p_ca_info @@ -657,807 +608,719 @@ __osm_ca_info_get_port_attr_ptr( /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( - IN osm_vendor_t* const p_vend, - IN ib_port_attr_t* const p_attr_array, - IN uint32_t* const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_api_status_t status; - - uint32_t ca; - uintn_t ca_count; - uint32_t port_count = 0; - uint8_t port_num; - uint32_t total_ports = 0; - ib_net64_t *p_ca_guid = NULL; - osm_ca_info_t *p_ca_info; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); - - CL_ASSERT( p_vend ); - CL_ASSERT( p_vend->p_ca_info == NULL ); - - /* - 1) Determine the number of CA's - 2) Allocate an array big enough to hold the ca info objects. - 3) Call again to retrieve the guids. - */ - status = __osm_vendor_get_ca_guids( p_vend, &p_ca_guid, &ca_count ); - - p_vend->p_ca_info = malloc( ca_count * sizeof(*p_vend->p_ca_info) ); - if( p_vend->p_ca_info == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 3B11: " - "Unable to allocate CA information array.\n" ); - goto Exit; - } - - memset( p_vend->p_ca_info, 0, ca_count * sizeof(*p_vend->p_ca_info) ); - p_vend->ca_count = ca_count; - - /* - For each CA, retrieve the port info attributes - */ - for( ca = 0; ca < ca_count; ca++ ) - { - p_ca_info = &p_vend->p_ca_info[ca]; - - status = __osm_ca_info_init( - p_vend, - p_ca_info, - p_ca_guid[ca] ); - - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 3B12: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str( status ) ); - } - - total_ports += osm_ca_info_get_num_ports( p_ca_info ); - } - - /* - If the user supplied enough storage, return the port guids, - otherwise, return the appropriate error. - */ - if( *p_num_ports >= total_ports ) - { - for( ca = 0; ca < ca_count; ca++ ) - { - uint32_t num_ports; - - p_ca_info = &p_vend->p_ca_info[ca]; - - num_ports = osm_ca_info_get_num_ports( p_ca_info ); - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_attr_array[port_count] = *__osm_ca_info_get_port_attr_ptr( - p_ca_info, port_num ); - port_count++; - } - } - } - else - { - status = IB_INSUFFICIENT_MEMORY; - } - - - *p_num_ports = total_ports; - - - Exit: - if( p_ca_guid ) - free( p_ca_guid ); - - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + ib_api_status_t status; + + uint32_t ca; + uintn_t ca_count; + uint32_t port_count = 0; + uint8_t port_num; + uint32_t total_ports = 0; + ib_net64_t *p_ca_guid = NULL; + osm_ca_info_t *p_ca_info; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + + CL_ASSERT(p_vend); + CL_ASSERT(p_vend->p_ca_info == NULL); + + /* + 1) Determine the number of CA's + 2) Allocate an array big enough to hold the ca info objects. + 3) Call again to retrieve the guids. + */ + status = __osm_vendor_get_ca_guids(p_vend, &p_ca_guid, &ca_count); + + p_vend->p_ca_info = malloc(ca_count * sizeof(*p_vend->p_ca_info)); + if (p_vend->p_ca_info == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 3B11: " + "Unable to allocate CA information array.\n"); + goto Exit; + } + + memset(p_vend->p_ca_info, 0, ca_count * sizeof(*p_vend->p_ca_info)); + p_vend->ca_count = ca_count; + + /* + For each CA, retrieve the port info attributes + */ + for (ca = 0; ca < ca_count; ca++) { + p_ca_info = &p_vend->p_ca_info[ca]; + + status = __osm_ca_info_init(p_vend, p_ca_info, p_ca_guid[ca]); + + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 3B12: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); + } + + total_ports += osm_ca_info_get_num_ports(p_ca_info); + } + + /* + If the user supplied enough storage, return the port guids, + otherwise, return the appropriate error. + */ + if (*p_num_ports >= total_ports) { + for (ca = 0; ca < ca_count; ca++) { + uint32_t num_ports; + + p_ca_info = &p_vend->p_ca_info[ca]; + + num_ports = osm_ca_info_get_num_ports(p_ca_info); + + for (port_num = 0; port_num < num_ports; port_num++) { + p_attr_array[port_count] = + *__osm_ca_info_get_port_attr_ptr(p_ca_info, + port_num); + port_count++; + } + } + } else { + status = IB_INSUFFICIENT_MEMORY; + } + + *p_num_ports = total_ports; + + Exit: + if (p_ca_guid) + free(p_ca_guid); + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_net64_t -osm_vendor_get_ca_guid( - IN osm_vendor_t* const p_vend, - IN const ib_net64_t port_guid ) +osm_vendor_get_ca_guid(IN osm_vendor_t * const p_vend, + IN const ib_net64_t port_guid) { - uint8_t index; - uint8_t num_ports; - uint32_t num_guids = 0; - osm_ca_info_t *p_ca_info; - uint32_t ca; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_ca_guid ); - - CL_ASSERT( port_guid ); - /* - First, locate the HCA that owns this port. - */ - if( p_vend->p_ca_info == NULL ) - { - /* - Initialize the osm_ca_info_t array which allows - us to match port GUID to CA. - */ - osm_vendor_get_all_port_attr( p_vend, NULL, &num_guids ); - } - - CL_ASSERT( p_vend->p_ca_info ); - CL_ASSERT( p_vend->ca_count ); - - for( ca = 0; ca < p_vend->ca_count; ca++ ) - { - p_ca_info = &p_vend->p_ca_info[ca]; - - num_ports = osm_ca_info_get_num_ports( p_ca_info ); - CL_ASSERT( num_ports ); - - for( index = 0; index < num_ports; index++ ) - { - if( port_guid == - osm_ca_info_get_port_guid( p_ca_info, index ) ) - { - OSM_LOG_EXIT( p_vend->p_log ); - return( osm_ca_info_get_ca_guid( p_ca_info ) ); - } - } - } - - /* - No local CA owns this guid! - */ - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_ca_guid: ERR 3B13: " - "Unable to determine CA guid.\n" ); - - OSM_LOG_EXIT( p_vend->p_log ); - return( 0 ); + uint8_t index; + uint8_t num_ports; + uint32_t num_guids = 0; + osm_ca_info_t *p_ca_info; + uint32_t ca; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_ca_guid); + + CL_ASSERT(port_guid); + /* + First, locate the HCA that owns this port. + */ + if (p_vend->p_ca_info == NULL) { + /* + Initialize the osm_ca_info_t array which allows + us to match port GUID to CA. + */ + osm_vendor_get_all_port_attr(p_vend, NULL, &num_guids); + } + + CL_ASSERT(p_vend->p_ca_info); + CL_ASSERT(p_vend->ca_count); + + for (ca = 0; ca < p_vend->ca_count; ca++) { + p_ca_info = &p_vend->p_ca_info[ca]; + + num_ports = osm_ca_info_get_num_ports(p_ca_info); + CL_ASSERT(num_ports); + + for (index = 0; index < num_ports; index++) { + if (port_guid == + osm_ca_info_get_port_guid(p_ca_info, index)) { + OSM_LOG_EXIT(p_vend->p_log); + return (osm_ca_info_get_ca_guid(p_ca_info)); + } + } + } + + /* + No local CA owns this guid! + */ + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_ca_guid: ERR 3B13: " + "Unable to determine CA guid.\n"); + + OSM_LOG_EXIT(p_vend->p_log); + return (0); } /********************************************************************** **********************************************************************/ uint8_t -osm_vendor_get_port_num( - IN osm_vendor_t* const p_vend, - IN const ib_net64_t port_guid ) +osm_vendor_get_port_num(IN osm_vendor_t * const p_vend, + IN const ib_net64_t port_guid) { - uint8_t index; - uint8_t num_ports; - uint32_t num_guids = 0; - osm_ca_info_t *p_ca_info; - uint32_t ca; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_port_num ); - - CL_ASSERT( port_guid ); - /* - First, locate the HCA that owns this port. - */ - if( p_vend->p_ca_info == NULL ) - { - /* - Initialize the osm_ca_info_t array which allows - us to match port GUID to CA. - */ - osm_vendor_get_all_port_attr( p_vend, NULL, &num_guids ); - } - - CL_ASSERT( p_vend->p_ca_info ); - CL_ASSERT( p_vend->ca_count ); - - for( ca = 0; ca < p_vend->ca_count; ca++ ) - { - p_ca_info = &p_vend->p_ca_info[ca]; - - num_ports = osm_ca_info_get_num_ports( p_ca_info ); - CL_ASSERT( num_ports ); - - for( index = 0; index < num_ports; index++ ) - { - if( port_guid == - osm_ca_info_get_port_guid( p_ca_info, index ) ) - { - OSM_LOG_EXIT( p_vend->p_log ); - return( osm_ca_info_get_port_num( p_ca_info, index ) ); - } - } - } - - /* - No local CA owns this guid! - */ - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_port_num: ERR 3B30: " - "Unable to determine CA guid.\n" ); - - OSM_LOG_EXIT( p_vend->p_log ); - return( 0 ); + uint8_t index; + uint8_t num_ports; + uint32_t num_guids = 0; + osm_ca_info_t *p_ca_info; + uint32_t ca; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_port_num); + + CL_ASSERT(port_guid); + /* + First, locate the HCA that owns this port. + */ + if (p_vend->p_ca_info == NULL) { + /* + Initialize the osm_ca_info_t array which allows + us to match port GUID to CA. + */ + osm_vendor_get_all_port_attr(p_vend, NULL, &num_guids); + } + + CL_ASSERT(p_vend->p_ca_info); + CL_ASSERT(p_vend->ca_count); + + for (ca = 0; ca < p_vend->ca_count; ca++) { + p_ca_info = &p_vend->p_ca_info[ca]; + + num_ports = osm_ca_info_get_num_ports(p_ca_info); + CL_ASSERT(num_ports); + + for (index = 0; index < num_ports; index++) { + if (port_guid == + osm_ca_info_get_port_guid(p_ca_info, index)) { + OSM_LOG_EXIT(p_vend->p_log); + return (osm_ca_info_get_port_num + (p_ca_info, index)); + } + } + } + + /* + No local CA owns this guid! + */ + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_port_num: ERR 3B30: " + "Unable to determine CA guid.\n"); + + OSM_LOG_EXIT(p_vend->p_log); + return (0); } - /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_vendor_open_ca( - IN osm_vendor_t* const p_vend, - IN const ib_net64_t port_guid ) +__osm_vendor_open_ca(IN osm_vendor_t * const p_vend, + IN const ib_net64_t port_guid) { - ib_net64_t ca_guid; - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_open_ca ); - - ca_guid = osm_vendor_get_ca_guid( p_vend, port_guid ); - if( ca_guid == 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_open_ca: ERR 3B31: " - "Bad port GUID value 0x%" PRIx64 ".\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "__osm_vendor_open_ca: " - "Opening HCA 0x%" PRIx64 ".\n", cl_ntoh64( ca_guid ) ); - - status = ib_open_ca( p_vend->h_al, - ca_guid, - __osm_al_ca_err_callback, - p_vend, - &p_vend->h_ca ); - - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_open_ca: ERR 3B15: " - "Unable to open CA (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - CL_ASSERT( p_vend->h_ca ); - - status = ib_alloc_pd( - p_vend->h_ca, - IB_PDT_ALIAS, - p_vend, - &p_vend->h_pd ); - - if( status != IB_SUCCESS ) - { - ib_close_ca( p_vend->h_ca, __osm_al_ca_destroy_callback ); - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_open_ca: ERR 3B16: " - "Unable to allocate protection domain (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - CL_ASSERT( p_vend->h_pd ); - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + ib_net64_t ca_guid; + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_open_ca); + + ca_guid = osm_vendor_get_ca_guid(p_vend, port_guid); + if (ca_guid == 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_open_ca: ERR 3B31: " + "Bad port GUID value 0x%" PRIx64 ".\n", + cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "__osm_vendor_open_ca: " + "Opening HCA 0x%" PRIx64 ".\n", cl_ntoh64(ca_guid)); + + status = ib_open_ca(p_vend->h_al, + ca_guid, + __osm_al_ca_err_callback, p_vend, &p_vend->h_ca); + + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_open_ca: ERR 3B15: " + "Unable to open CA (%s).\n", ib_get_err_str(status)); + goto Exit; + } + + CL_ASSERT(p_vend->h_ca); + + status = ib_alloc_pd(p_vend->h_ca, IB_PDT_ALIAS, p_vend, &p_vend->h_pd); + + if (status != IB_SUCCESS) { + ib_close_ca(p_vend->h_ca, __osm_al_ca_destroy_callback); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_open_ca: ERR 3B16: " + "Unable to allocate protection domain (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + CL_ASSERT(p_vend->h_pd); + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_vendor_init_av( - IN const osm_al_bind_info_t* p_bind, - IN ib_av_attr_t* p_av ) +__osm_vendor_init_av(IN const osm_al_bind_info_t * p_bind, + IN ib_av_attr_t * p_av) { - memset( p_av, 0, sizeof(*p_av) ); - p_av->port_num = p_bind->port_num; - p_av->dlid = IB_LID_PERMISSIVE; + memset(p_av, 0, sizeof(*p_av)); + p_av->port_num = p_bind->port_num; + p_av->dlid = IB_LID_PERMISSIVE; } /********************************************************************** **********************************************************************/ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_user_bind, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void* context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_user_bind, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { - ib_net64_t port_guid; - osm_al_bind_info_t *p_bind = 0; - ib_api_status_t status; - ib_qp_create_t qp_create; - ib_mad_svc_t mad_svc; - ib_av_attr_t av; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_bind ); - - CL_ASSERT( p_user_bind ); - CL_ASSERT( p_mad_pool ); - CL_ASSERT( mad_recv_callback ); - CL_ASSERT( send_err_callback ); - - port_guid = p_user_bind->port_guid; - - osm_log( p_vend->p_log, OSM_LOG_INFO, - "osm_vendor_bind: " - "Binding to port 0x%" PRIx64 ".\n", - cl_ntoh64( port_guid ) ); - - if( p_vend->h_ca == 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Opening CA that owns port 0x%" PRIx64 ".\n", port_guid ); - - status = __osm_vendor_open_ca( p_vend, port_guid ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3B17: " - "Unable to Open CA (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - } - - p_bind = malloc( sizeof(*p_bind) ); - if( p_bind == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3B18: " - "Unable to allocate internal bind object.\n" ); - goto Exit; - } - - memset( p_bind, 0, sizeof(*p_bind) ); - p_bind->p_vend = p_vend; - p_bind->client_context = context; - p_bind->port_num = osm_vendor_get_port_num( p_vend, port_guid ); - p_bind->rcv_callback = mad_recv_callback; - p_bind->send_err_callback = send_err_callback; - p_bind->p_osm_pool = p_mad_pool; - - CL_ASSERT( p_bind->port_num ); - - /* - Get the proper QP. - */ - memset( &qp_create, 0, sizeof(qp_create) ); - - switch( p_user_bind->mad_class ) - { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - qp_create.qp_type = IB_QPT_QP0_ALIAS; - break; - - case IB_MCLASS_SUBN_ADM: - default: - qp_create.qp_type = IB_QPT_QP1_ALIAS; - break; - } - - qp_create.sq_depth = p_user_bind->send_q_size; - qp_create.rq_depth = p_user_bind->recv_q_size; - qp_create.sq_sge = OSM_AL_SQ_SGE; - qp_create.rq_sge = OSM_AL_RQ_SGE; - - status = ib_get_spl_qp( - p_vend->h_pd, - port_guid, - &qp_create, - p_bind, - __osm_al_err_callback, - &p_bind->pool_key, - &p_bind->h_qp ); - - if( status != IB_SUCCESS ) - { - free( p_bind ); - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3B19: " - "Unable to get QP handle (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - CL_ASSERT( p_bind->h_qp ); - CL_ASSERT( p_bind->pool_key ); - - memset( &mad_svc, 0, sizeof(mad_svc) ); - - mad_svc.mad_svc_context = p_bind; - mad_svc.pfn_mad_send_cb = __osm_al_send_callback; - mad_svc.pfn_mad_recv_cb = __osm_al_rcv_callback; - mad_svc.mgmt_class = p_user_bind->mad_class; - mad_svc.mgmt_version = p_user_bind->class_version; - mad_svc.support_unsol = p_user_bind->is_responder; - mad_svc.method_array[IB_MAD_METHOD_GET] = TRUE; - mad_svc.method_array[IB_MAD_METHOD_SET] = TRUE; - mad_svc.method_array[IB_MAD_METHOD_DELETE] = TRUE; - mad_svc.method_array[IB_MAD_METHOD_TRAP] = TRUE; - mad_svc.method_array[IB_MAD_METHOD_GETTABLE] = TRUE; - - - status = ib_reg_mad_svc( - p_bind->h_qp, - &mad_svc, - &p_bind->h_svc ); - - if( status != IB_SUCCESS ) - { - free( p_bind ); - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3B21: " - "Unable to register QP0 MAD service (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - - __osm_vendor_init_av( p_bind, &av ); - - status = ib_create_av( p_vend->h_pd, &av, &p_bind->h_dr_av ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3B22: " - "Unable to create address vector (%s).\n", - ib_get_err_str( status ) ); - - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Allocating av handle %p.\n", p_bind->h_dr_av ); - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( (osm_bind_handle_t)p_bind ); + ib_net64_t port_guid; + osm_al_bind_info_t *p_bind = 0; + ib_api_status_t status; + ib_qp_create_t qp_create; + ib_mad_svc_t mad_svc; + ib_av_attr_t av; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_bind); + + CL_ASSERT(p_user_bind); + CL_ASSERT(p_mad_pool); + CL_ASSERT(mad_recv_callback); + CL_ASSERT(send_err_callback); + + port_guid = p_user_bind->port_guid; + + osm_log(p_vend->p_log, OSM_LOG_INFO, + "osm_vendor_bind: " + "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64(port_guid)); + + if (p_vend->h_ca == 0) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Opening CA that owns port 0x%" PRIx64 ".\n", + port_guid); + + status = __osm_vendor_open_ca(p_vend, port_guid); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3B17: " + "Unable to Open CA (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + } + + p_bind = malloc(sizeof(*p_bind)); + if (p_bind == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3B18: " + "Unable to allocate internal bind object.\n"); + goto Exit; + } + + memset(p_bind, 0, sizeof(*p_bind)); + p_bind->p_vend = p_vend; + p_bind->client_context = context; + p_bind->port_num = osm_vendor_get_port_num(p_vend, port_guid); + p_bind->rcv_callback = mad_recv_callback; + p_bind->send_err_callback = send_err_callback; + p_bind->p_osm_pool = p_mad_pool; + + CL_ASSERT(p_bind->port_num); + + /* + Get the proper QP. + */ + memset(&qp_create, 0, sizeof(qp_create)); + + switch (p_user_bind->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + qp_create.qp_type = IB_QPT_QP0_ALIAS; + break; + + case IB_MCLASS_SUBN_ADM: + default: + qp_create.qp_type = IB_QPT_QP1_ALIAS; + break; + } + + qp_create.sq_depth = p_user_bind->send_q_size; + qp_create.rq_depth = p_user_bind->recv_q_size; + qp_create.sq_sge = OSM_AL_SQ_SGE; + qp_create.rq_sge = OSM_AL_RQ_SGE; + + status = ib_get_spl_qp(p_vend->h_pd, + port_guid, + &qp_create, + p_bind, + __osm_al_err_callback, + &p_bind->pool_key, &p_bind->h_qp); + + if (status != IB_SUCCESS) { + free(p_bind); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3B19: " + "Unable to get QP handle (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + CL_ASSERT(p_bind->h_qp); + CL_ASSERT(p_bind->pool_key); + + memset(&mad_svc, 0, sizeof(mad_svc)); + + mad_svc.mad_svc_context = p_bind; + mad_svc.pfn_mad_send_cb = __osm_al_send_callback; + mad_svc.pfn_mad_recv_cb = __osm_al_rcv_callback; + mad_svc.mgmt_class = p_user_bind->mad_class; + mad_svc.mgmt_version = p_user_bind->class_version; + mad_svc.support_unsol = p_user_bind->is_responder; + mad_svc.method_array[IB_MAD_METHOD_GET] = TRUE; + mad_svc.method_array[IB_MAD_METHOD_SET] = TRUE; + mad_svc.method_array[IB_MAD_METHOD_DELETE] = TRUE; + mad_svc.method_array[IB_MAD_METHOD_TRAP] = TRUE; + mad_svc.method_array[IB_MAD_METHOD_GETTABLE] = TRUE; + + status = ib_reg_mad_svc(p_bind->h_qp, &mad_svc, &p_bind->h_svc); + + if (status != IB_SUCCESS) { + free(p_bind); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3B21: " + "Unable to register QP0 MAD service (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + + __osm_vendor_init_av(p_bind, &av); + + status = ib_create_av(p_vend->h_pd, &av, &p_bind->h_dr_av); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3B22: " + "Unable to create address vector (%s).\n", + ib_get_err_str(status)); + + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Allocating av handle %p.\n", p_bind->h_dr_av); + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return ((osm_bind_handle_t) p_bind); } /********************************************************************** **********************************************************************/ -ib_mad_t* -osm_vendor_get( - IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t* const p_vw ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { - ib_mad_t *p_mad; - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *)h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get ); - - CL_ASSERT( p_vw ); - - p_vw->size = mad_size; - p_vw->h_bind = h_bind; - - /* - Retrieve a MAD element from the pool and give the user direct - access to its buffer. - */ - status = ib_get_mad( p_bind->pool_key, mad_size, &p_vw->p_elem ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get: ERR 3B25: " - "Unable to acquire MAD (%s).\n", - ib_get_err_str( status ) ); - - p_mad = NULL; - goto Exit; - } - - CL_ASSERT( p_vw->p_elem ); - p_mad = ib_get_mad_buf( p_vw->p_elem ); - - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get: " - "Acquired MAD %p, size = %u.\n", p_mad, mad_size ); - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( p_mad ); + ib_mad_t *p_mad; + osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); + + CL_ASSERT(p_vw); + + p_vw->size = mad_size; + p_vw->h_bind = h_bind; + + /* + Retrieve a MAD element from the pool and give the user direct + access to its buffer. + */ + status = ib_get_mad(p_bind->pool_key, mad_size, &p_vw->p_elem); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get: ERR 3B25: " + "Unable to acquire MAD (%s).\n", + ib_get_err_str(status)); + + p_mad = NULL; + goto Exit; + } + + CL_ASSERT(p_vw->p_elem); + p_mad = ib_get_mad_buf(p_vw->p_elem); + + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get: " + "Acquired MAD %p, size = %u.\n", p_mad, mad_size); + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (p_mad); } /********************************************************************** **********************************************************************/ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vw ) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *)h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_put ); - - CL_ASSERT( p_vw ); - CL_ASSERT( p_vw->p_elem ); - CL_ASSERT( p_vw->h_bind == h_bind ); - - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_put: " - "Retiring MAD %p.\n", ib_get_mad_buf( p_vw->p_elem ) ); - } - - status = ib_put_mad( p_vw->p_elem ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_put: ERR 3B26: " - "Unable to retire MAD (%s).\n", - ib_get_err_str( status ) ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); + + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->p_elem); + CL_ASSERT(p_vw->h_bind == h_bind); + + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_put: " + "Retiring MAD %p.\n", ib_get_mad_buf(p_vw->p_elem)); + } + + status = ib_put_mad(p_vw->p_elem); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_put: ERR 3B26: " + "Unable to retire MAD (%s).\n", ib_get_err_str(status)); + } + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { - osm_al_bind_info_t* const p_bind = h_bind; - osm_vendor_t* const p_vend = p_bind->p_vend; - osm_vend_wrap_t* const p_vw = osm_madw_get_vend_ptr( p_madw ); - osm_mad_addr_t* const p_mad_addr = osm_madw_get_mad_addr_ptr( p_madw ); - ib_mad_t* const p_mad = osm_madw_get_mad_ptr( p_madw ); - ib_api_status_t status; - ib_mad_element_t *p_elem; - ib_av_attr_t av; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_send ); - - CL_ASSERT( p_vw->h_bind == h_bind ); - CL_ASSERT( p_vw->p_elem ); - - p_elem = p_vw->p_elem; - - /* - If a response is expected to this MAD, then preallocate - a mad wrapper to contain the wire MAD received in the - response. Allocating a wrapper here allows for easier - failure paths than after we already received the wire mad. - */ - if( resp_expected ) - { - p_vw->p_resp_madw = osm_mad_pool_get_wrapper_raw( - p_bind->p_osm_pool ); - if( p_vw->p_resp_madw == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 3B27: " - "Unable to allocate MAD wrapper.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - } - else - p_vw->p_resp_madw = NULL; - - /* - For all sends other than directed route SM MADs, - acquire an address vector for the destination. - */ - if( p_mad->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - memset( &av, 0, sizeof(av) ); - av.port_num = p_bind->port_num; - av.dlid = p_mad_addr->dest_lid; - av.static_rate = p_mad_addr->static_rate; - av.path_bits = p_mad_addr->path_bits; - - if( (p_mad->mgmt_class != IB_MCLASS_SUBN_LID) && - (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR) ){ - av.sl = p_mad_addr->addr_type.gsi.service_level; - - - if(p_mad_addr->addr_type.gsi.global_route) - { - av.grh_valid = TRUE; - /* ANIL */ - /* av.grh = p_mad_addr->addr_type.gsi.grh_info; */ - } - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_send: " - "av.port_num 0x%X, " - "av.dlid 0x%X, " - "av.static_rate %d, " - "av.path_bits %d.\n", - av.port_num, cl_ntoh16(av.dlid), - av.static_rate, av.path_bits); - } - - status = ib_create_av( p_vend->h_pd, &av, &p_vw->h_av ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 3B28: " - "Unable to create address vector (%s).\n", - ib_get_err_str( status ) ); - - if( p_vw->p_resp_madw ) - osm_mad_pool_put( p_bind->p_osm_pool, p_vw->p_resp_madw ); - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_send: " - "Allocating av handle %p.\n", p_vw->h_av ); - } - } - else - { - p_vw->h_av = p_bind->h_dr_av; - } - - p_elem->h_av = p_vw->h_av; - - p_elem->context1 = p_madw; - p_elem->context2 = NULL; - - p_elem->immediate_data = 0; - p_elem->p_grh = NULL; - p_elem->resp_expected = resp_expected; - p_elem->retry_cnt = OSM_DEFAULT_RETRY_COUNT; - - p_elem->send_opt = IB_SEND_OPT_SIGNALED; - p_elem->timeout_ms = p_vend->timeout; - - /* Completion information. */ - p_elem->status = 0; /* Not trusting AL */ - - - if( (p_mad->mgmt_class == IB_MCLASS_SUBN_LID) || - (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) ) - { - p_elem->remote_qp = 0; - p_elem->remote_qkey = 0; - } - else - { - p_elem->remote_qp = p_mad_addr->addr_type.gsi.remote_qp; - p_elem->remote_qkey = p_mad_addr->addr_type.gsi.remote_qkey; - osm_log(p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_send: " - "remote qp = 0x%X, remote qkey = 0x%X.\n", - cl_ntoh32(p_elem->remote_qp), - cl_ntoh32(p_elem->remote_qkey) ); - } - - status = ib_send_mad( p_bind->h_svc, p_elem, NULL ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 3B29: " - "Send failed (%s).\n", - ib_get_err_str( status ) ); - if( p_vw->p_resp_madw ) - osm_mad_pool_put( p_bind->p_osm_pool, p_vw->p_resp_madw ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + osm_al_bind_info_t *const p_bind = h_bind; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + ib_mad_t *const p_mad = osm_madw_get_mad_ptr(p_madw); + ib_api_status_t status; + ib_mad_element_t *p_elem; + ib_av_attr_t av; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_send); + + CL_ASSERT(p_vw->h_bind == h_bind); + CL_ASSERT(p_vw->p_elem); + + p_elem = p_vw->p_elem; + + /* + If a response is expected to this MAD, then preallocate + a mad wrapper to contain the wire MAD received in the + response. Allocating a wrapper here allows for easier + failure paths than after we already received the wire mad. + */ + if (resp_expected) { + p_vw->p_resp_madw = + osm_mad_pool_get_wrapper_raw(p_bind->p_osm_pool); + if (p_vw->p_resp_madw == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 3B27: " + "Unable to allocate MAD wrapper.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + } else + p_vw->p_resp_madw = NULL; + + /* + For all sends other than directed route SM MADs, + acquire an address vector for the destination. + */ + if (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR) { + memset(&av, 0, sizeof(av)); + av.port_num = p_bind->port_num; + av.dlid = p_mad_addr->dest_lid; + av.static_rate = p_mad_addr->static_rate; + av.path_bits = p_mad_addr->path_bits; + + if ((p_mad->mgmt_class != IB_MCLASS_SUBN_LID) && + (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR)) { + av.sl = p_mad_addr->addr_type.gsi.service_level; + + if (p_mad_addr->addr_type.gsi.global_route) { + av.grh_valid = TRUE; + /* ANIL */ + /* av.grh = p_mad_addr->addr_type.gsi.grh_info; */ + } + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_send: " + "av.port_num 0x%X, " + "av.dlid 0x%X, " + "av.static_rate %d, " + "av.path_bits %d.\n", + av.port_num, cl_ntoh16(av.dlid), + av.static_rate, av.path_bits); + } + + status = ib_create_av(p_vend->h_pd, &av, &p_vw->h_av); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 3B28: " + "Unable to create address vector (%s).\n", + ib_get_err_str(status)); + + if (p_vw->p_resp_madw) + osm_mad_pool_put(p_bind->p_osm_pool, + p_vw->p_resp_madw); + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_send: " + "Allocating av handle %p.\n", p_vw->h_av); + } + } else { + p_vw->h_av = p_bind->h_dr_av; + } + + p_elem->h_av = p_vw->h_av; + + p_elem->context1 = p_madw; + p_elem->context2 = NULL; + + p_elem->immediate_data = 0; + p_elem->p_grh = NULL; + p_elem->resp_expected = resp_expected; + p_elem->retry_cnt = OSM_DEFAULT_RETRY_COUNT; + + p_elem->send_opt = IB_SEND_OPT_SIGNALED; + p_elem->timeout_ms = p_vend->timeout; + + /* Completion information. */ + p_elem->status = 0; /* Not trusting AL */ + + if ((p_mad->mgmt_class == IB_MCLASS_SUBN_LID) || + (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR)) { + p_elem->remote_qp = 0; + p_elem->remote_qkey = 0; + } else { + p_elem->remote_qp = p_mad_addr->addr_type.gsi.remote_qp; + p_elem->remote_qkey = p_mad_addr->addr_type.gsi.remote_qkey; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_send: " + "remote qp = 0x%X, remote qkey = 0x%X.\n", + cl_ntoh32(p_elem->remote_qp), + cl_ntoh32(p_elem->remote_qkey)); + } + + status = ib_send_mad(p_bind->h_svc, p_elem, NULL); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 3B29: " + "Send failed (%s).\n", ib_get_err_str(status)); + if (p_vw->p_resp_madw) + osm_mad_pool_put(p_bind->p_osm_pool, p_vw->p_resp_madw); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_vendor_local_lid_change( - IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *)h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - ib_av_attr_t av; - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); - - /* - The only thing we need to do is refresh the directed - route address vector. - */ - __osm_vendor_init_av( p_bind, &av ); - - status = ib_destroy_av( p_bind->h_dr_av ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_local_lid_change: ERR 3B32: " - "Unable to destroy address vector (%s).\n", - ib_get_err_str( status ) ); - - goto Exit; - } - - status = ib_create_av( p_vend->h_pd, &av, &p_bind->h_dr_av ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_local_lid_change: ERR 3B33: " - "Unable to create address vector (%s).\n", - ib_get_err_str( status ) ); - - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( status ); + osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + ib_av_attr_t av; + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); + + /* + The only thing we need to do is refresh the directed + route address vector. + */ + __osm_vendor_init_av(p_bind, &av); + + status = ib_destroy_av(p_bind->h_dr_av); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_local_lid_change: ERR 3B32: " + "Unable to destroy address vector (%s).\n", + ib_get_err_str(status)); + + goto Exit; + } + + status = ib_create_av(p_vend->h_pd, &av, &p_bind->h_dr_av); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_local_lid_change: ERR 3B33: " + "Unable to create address vector (%s).\n", + ib_get_err_str(status)); + + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *)h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - ib_api_status_t status; - ib_port_attr_mod_t attr_mod; + osm_al_bind_info_t *p_bind = (osm_al_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + ib_api_status_t status; + ib_port_attr_mod_t attr_mod; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); - memset( &attr_mod, 0, sizeof(attr_mod) ); + memset(&attr_mod, 0, sizeof(attr_mod)); - attr_mod.cap.sm = is_sm_val; + attr_mod.cap.sm = is_sm_val; - status = ib_modify_ca( p_vend->h_ca, p_bind->port_num, - IB_CA_MOD_IS_SM, &attr_mod ); + status = ib_modify_ca(p_vend->h_ca, p_bind->port_num, + IB_CA_MOD_IS_SM, &attr_mod); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 3B34: " - "Unable set 'IS_SM' bit to:%u in port attributes (%s).\n", - is_sm_val, ib_get_err_str( status ) ); - } + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 3B34: " + "Unable set 'IS_SM' bit to:%u in port attributes (%s).\n", + is_sm_val, ib_get_err_str(status)); + } - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } -#endif /* OSM_VENDOR_INTF_AL */ +#endif /* OSM_VENDOR_INTF_AL */ diff --git a/opensm/libvendor/osm_vendor_ibumad.c b/opensm/libvendor/osm_vendor_ibumad.c index 686da26..a5b2fce 100644 --- a/opensm/libvendor/osm_vendor_ibumad.c +++ b/opensm/libvendor/osm_vendor_ibumad.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -76,8 +76,7 @@ * * SYNOPSIS */ -typedef struct _osm_umad_bind_info -{ +typedef struct _osm_umad_bind_info { osm_vendor_t *p_vend; void *client_context; osm_mad_pool_t *p_mad_pool; @@ -90,47 +89,44 @@ typedef struct _osm_umad_bind_info int agent_id1; /* SMI requires two agents */ } osm_umad_bind_info_t; -typedef struct _umad_receiver -{ +typedef struct _umad_receiver { pthread_t tid; - osm_vendor_t *p_vend; - osm_log_t *p_log; + osm_vendor_t *p_vend; + osm_log_t *p_log; } umad_receiver_t; -static void osm_vendor_close_port(osm_vendor_t* const p_vend); +static void osm_vendor_close_port(osm_vendor_t * const p_vend); -static void -clear_madw(osm_vendor_t *p_vend) +static void clear_madw(osm_vendor_t * p_vend) { umad_match_t *m, *e, *old_m; ib_net64_t old_tid; - OSM_LOG_ENTER( p_vend->p_log, clear_madw ); + OSM_LOG_ENTER(p_vend->p_log, clear_madw); pthread_mutex_lock(&p_vend->match_tbl_mutex); for (m = p_vend->mtbl.tbl, e = m + p_vend->mtbl.max; m < e; m++) { if (m->tid) { old_m = m; old_tid = m->tid; m->tid = 0; - osm_mad_pool_put( - ((osm_umad_bind_info_t *)((osm_madw_t *)m->v)->h_bind)->p_mad_pool, - m->v); + osm_mad_pool_put(((osm_umad_bind_info_t + *) ((osm_madw_t *) m->v)->h_bind)-> + p_mad_pool, m->v); pthread_mutex_unlock(&p_vend->match_tbl_mutex); osm_log(p_vend->p_log, OSM_LOG_ERROR, "clear_madw: ERR 5401: " - "evicting entry %p (tid was 0x%"PRIx64")\n", + "evicting entry %p (tid was 0x%" PRIx64 ")\n", old_m, old_tid); goto Exit; } } pthread_mutex_unlock(&p_vend->match_tbl_mutex); -Exit: - OSM_LOG_EXIT( p_vend->p_log ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); } -static osm_madw_t * -get_madw(osm_vendor_t *p_vend, ib_net64_t *tid) +static osm_madw_t *get_madw(osm_vendor_t * p_vend, ib_net64_t * tid) { umad_match_t *m, *e; ib_net64_t mtid = (*tid & cl_ntoh64(0x00000000ffffffffllu)); @@ -159,7 +155,7 @@ get_madw(osm_vendor_t *p_vend, ib_net64_t *tid) } static void -put_madw(osm_vendor_t *p_vend, osm_madw_t *p_madw, ib_net64_t *tid) +put_madw(osm_vendor_t * p_vend, osm_madw_t * p_madw, ib_net64_t * tid) { umad_match_t *m, *e, *old_lru, *lru = 0; osm_madw_t *p_req_madw; @@ -172,7 +168,9 @@ put_madw(osm_vendor_t *p_vend, osm_madw_t *p_madw, ib_net64_t *tid) if (m->tid == 0) { m->tid = *tid; m->v = p_madw; - m->version = cl_atomic_inc((atomic32_t *)&p_vend->mtbl.last_version); + m->version = + cl_atomic_inc((atomic32_t *) & p_vend->mtbl. + last_version); pthread_mutex_unlock(&p_vend->match_tbl_mutex); return; } @@ -188,19 +186,22 @@ put_madw(osm_vendor_t *p_vend, osm_madw_t *p_madw, ib_net64_t *tid) p_bind = p_req_madw->h_bind; p_req_madw->status = IB_CANCELED; pthread_mutex_lock(&p_vend->cb_mutex); - (*p_bind->send_err_callback)(p_bind->client_context, old_lru->v); + (*p_bind->send_err_callback) (p_bind->client_context, old_lru->v); pthread_mutex_unlock(&p_vend->cb_mutex); lru->tid = *tid; lru->v = p_madw; - lru->version = cl_atomic_inc((atomic32_t *)&p_vend->mtbl.last_version); + lru->version = + cl_atomic_inc((atomic32_t *) & p_vend->mtbl.last_version); pthread_mutex_unlock(&p_vend->match_tbl_mutex); osm_log(p_vend->p_log, OSM_LOG_ERROR, "put_madw: ERR 5402: " - "evicting entry %p (tid was 0x%"PRIx64")\n", old_lru, old_tid); + "evicting entry %p (tid was 0x%" PRIx64 ")\n", old_lru, + old_tid); } static void -ib_mad_addr_conv(ib_mad_addr_t *ib_mad_addr, osm_mad_addr_t *osm_mad_addr, int is_smi) +ib_mad_addr_conv(ib_mad_addr_t * ib_mad_addr, osm_mad_addr_t * osm_mad_addr, + int is_smi) { osm_mad_addr->dest_lid = ib_mad_addr->lid; osm_mad_addr->path_bits = ib_mad_addr->path_bits; @@ -216,13 +217,12 @@ ib_mad_addr_conv(ib_mad_addr_t *ib_mad_addr, osm_mad_addr_t *osm_mad_addr, int i osm_mad_addr->addr_type.gsi.remote_qkey = ib_mad_addr->qkey; osm_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; /* FIXME: support real pkey */ osm_mad_addr->addr_type.gsi.service_level = ib_mad_addr->sl; - osm_mad_addr->addr_type.gsi.global_route = 0; /* FIXME: handle GRH */ + osm_mad_addr->addr_type.gsi.global_route = 0; /* FIXME: handle GRH */ memset(&osm_mad_addr->addr_type.gsi.grh_info, 0, - sizeof osm_mad_addr->addr_type.gsi.grh_info); + sizeof osm_mad_addr->addr_type.gsi.grh_info); } -static void * -swap_mad_bufs(osm_madw_t *p_madw, void *umad) +static void *swap_mad_bufs(osm_madw_t * p_madw, void *umad) { void *old; @@ -238,10 +238,9 @@ static void unlock_mutex(void *arg) pthread_mutex_unlock(arg); } -void * -umad_receiver(void *p_ptr) +void *umad_receiver(void *p_ptr) { - umad_receiver_t* const p_ur = (umad_receiver_t *)p_ptr; + umad_receiver_t *const p_ur = (umad_receiver_t *) p_ptr; osm_vendor_t *p_vend = p_ur->p_vend; osm_umad_bind_info_t *p_bind; ib_mad_addr_t *ib_mad_addr; @@ -251,12 +250,12 @@ umad_receiver(void *p_ptr) void *umad = 0; int mad_agent, length; - OSM_LOG_ENTER( p_ur->p_log, umad_receiver ); + OSM_LOG_ENTER(p_ur->p_log, umad_receiver); for (;;) { if (!umad && !(umad = umad_alloc(1, umad_size() + MAD_BLOCK_SIZE))) { - osm_log(p_ur->p_log, OSM_LOG_ERROR, + osm_log(p_ur->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5403: " "can't alloc MAD sized umad\n"); break; @@ -277,15 +276,18 @@ umad_receiver(void *p_ptr) if (!umad) { osm_log(p_ur->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5405: " - "can't alloc umad length %d\n", length); + "can't alloc umad length %d\n", + length); continue; } if ((mad_agent = umad_recv(p_vend->umad_port_id, - umad, &length, -1)) < 0) { + umad, &length, + -1)) < 0) { osm_log(p_ur->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5406: " - "recv error on umad length %d (%m)\n", length); + "recv error on umad length %d (%m)\n", + length); continue; } } @@ -299,20 +301,20 @@ umad_receiver(void *p_ptr) continue; } - mad = (ib_mad_t*)umad_get_mad(umad); + mad = (ib_mad_t *) umad_get_mad(umad); ib_mad_addr = umad_get_mad_addr(umad); ib_mad_addr_conv(ib_mad_addr, &osm_addr, mad->mgmt_class == IB_MCLASS_SUBN_LID || mad->mgmt_class == IB_MCLASS_SUBN_DIR); - if (!(p_madw = osm_mad_pool_get(p_bind->p_mad_pool, - (osm_bind_handle_t)p_bind, + if (!(p_madw = osm_mad_pool_get(p_bind->p_mad_pool, + (osm_bind_handle_t) p_bind, MAX(length, MAD_BLOCK_SIZE), &osm_addr))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5408: " - "request for a new madw failed -- dropping packet\n" ); + "request for a new madw failed -- dropping packet\n"); continue; } @@ -328,7 +330,7 @@ umad_receiver(void *p_ptr) /* if status != 0 then we are handling recv timeout on send */ if (umad_status(p_madw->vend_wrap.umad)) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5409: " "send completed with error" " (method=0x%X attr=0x%X trans_id=0x%" PRIx64 @@ -337,7 +339,7 @@ umad_receiver(void *p_ptr) cl_ntoh64(mad->trans_id)); if (mad->mgmt_class != IB_MCLASS_SUBN_DIR) { /* LID routed */ - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5410: " "class 0x%x LID 0x%x\n", mad->mgmt_class, @@ -346,14 +348,16 @@ umad_receiver(void *p_ptr) ib_smp_t *smp; /* Direct routed SMP */ - smp = (ib_smp_t *)mad; + smp = (ib_smp_t *) mad; osm_log(p_vend->p_log, OSM_LOG_ERROR, - "umad_receiver: ERR 5411: DR SMP Hop Ptr: 0x%X\n", smp->hop_ptr); - osm_dump_smp_dr_path(p_vend->p_log, smp, OSM_LOG_ERROR); + "umad_receiver: ERR 5411: DR SMP Hop Ptr: 0x%X\n", + smp->hop_ptr); + osm_dump_smp_dr_path(p_vend->p_log, smp, + OSM_LOG_ERROR); } if (!(p_req_madw = get_madw(p_vend, &mad->trans_id))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5412: " "Failed to obtain request madw for timed out MAD" "(method=0x%X attr=0x%X) -- dropping\n", @@ -362,8 +366,11 @@ umad_receiver(void *p_ptr) p_req_madw->status = IB_TIMEOUT; /* cb frees req_madw */ pthread_mutex_lock(&p_vend->cb_mutex); - pthread_cleanup_push(unlock_mutex, &p_vend->cb_mutex); - (*p_bind->send_err_callback)(p_bind->client_context, p_req_madw); + pthread_cleanup_push(unlock_mutex, + &p_vend->cb_mutex); + (*p_bind->send_err_callback) (p_bind-> + client_context, + p_req_madw); pthread_cleanup_pop(1); } @@ -374,7 +381,7 @@ umad_receiver(void *p_ptr) p_req_madw = 0; if (ib_mad_is_response(mad) && !(p_req_madw = get_madw(p_vend, &mad->trans_id))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5413: " "Failed to obtain request madw for received MAD" "(method=0x%X attr=0x%X) -- dropping\n", @@ -382,22 +389,21 @@ umad_receiver(void *p_ptr) osm_mad_pool_put(p_bind->p_mad_pool, p_madw); continue; } - #ifndef VENDOR_RMPP_SUPPORT if ((mad->mgmt_class != IB_MCLASS_SUBN_DIR) && (mad->mgmt_class != IB_MCLASS_SUBN_LID) && - (ib_rmpp_is_flag_set((ib_rmpp_mad_t *)mad, - IB_RMPP_FLAG_ACTIVE))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + (ib_rmpp_is_flag_set((ib_rmpp_mad_t *) mad, + IB_RMPP_FLAG_ACTIVE))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, "umad_receiver: ERR 5414: " "class 0x%x method 0x%x" " RMPP version %d type %d flags 0x%x" " received -- dropping\n", mad->mgmt_class, mad->method, - ((ib_rmpp_mad_t *)mad)->rmpp_version, - ((ib_rmpp_mad_t *)mad)->rmpp_type, - ((ib_rmpp_mad_t *)mad)->rmpp_flags); + ((ib_rmpp_mad_t *) mad)->rmpp_version, + ((ib_rmpp_mad_t *) mad)->rmpp_type, + ((ib_rmpp_mad_t *) mad)->rmpp_flags); osm_mad_pool_put(p_bind->p_mad_pool, p_madw); continue; } @@ -406,15 +412,16 @@ umad_receiver(void *p_ptr) /* call the CB */ pthread_mutex_lock(&p_vend->cb_mutex); pthread_cleanup_push(unlock_mutex, &p_vend->cb_mutex); - (*p_bind->mad_recv_callback)(p_madw, p_bind->client_context, p_req_madw); + (*p_bind->mad_recv_callback) (p_madw, p_bind->client_context, + p_req_madw); pthread_cleanup_pop(1); } - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); return NULL; } -static int umad_receiver_start(osm_vendor_t *p_vend) +static int umad_receiver_start(osm_vendor_t * p_vend) { umad_receiver_t *p_ur = p_vend->receiver; @@ -427,7 +434,7 @@ static int umad_receiver_start(osm_vendor_t *p_vend) return 0; } -static void umad_receiver_stop(umad_receiver_t *p_ur) +static void umad_receiver_stop(umad_receiver_t * p_ur) { pthread_cancel(p_ur->tid); pthread_join(p_ur->tid, NULL); @@ -435,17 +442,16 @@ static void umad_receiver_stop(umad_receiver_t *p_ur) p_ur->p_vend = NULL; p_ur->p_log = NULL; } + /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_init( - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { int r, n_cas; - OSM_LOG_ENTER( p_log, osm_vendor_init ); + OSM_LOG_ENTER(p_log, osm_vendor_init); p_vend->p_log = p_log; p_vend->timeout = timeout; @@ -459,13 +465,13 @@ osm_vendor_init( * Open our instance of UMAD. */ if ((r = umad_init()) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: ERR 5415: Error opening UMAD\n"); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_init: ERR 5415: Error opening UMAD\n"); } if ((n_cas = umad_get_cas_names(p_vend->ca_names, OSM_UMAD_MAX_CAS)) < 0) { - osm_log(p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_init: ERR 5416: umad_get_cas_names failed\n"); r = n_cas; goto Exit; @@ -474,78 +480,71 @@ osm_vendor_init( p_vend->ca_count = n_cas; p_vend->mtbl.max = OSM_UMAD_MAX_PENDING; -Exit: - OSM_LOG_EXIT( p_log ); - return( r ); + Exit: + OSM_LOG_EXIT(p_log); + return (r); } /********************************************************************** **********************************************************************/ -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - osm_vendor_t *p_vend = NULL; + osm_vendor_t *p_vend = NULL; - OSM_LOG_ENTER( p_log, osm_vendor_new ); + OSM_LOG_ENTER(p_log, osm_vendor_new); - if ( !timeout ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 5433: " - "transaction timeout cannot be 0\n"); + if (!timeout) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 5433: " + "transaction timeout cannot be 0\n"); goto Exit; } - p_vend = malloc( sizeof(*p_vend) ); - if( p_vend == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, + p_vend = malloc(sizeof(*p_vend)); + if (p_vend == NULL) { + osm_log(p_log, OSM_LOG_ERROR, "osm_vendor_new: ERR 5417: " - "Unable to allocate vendor object\n" ); + "Unable to allocate vendor object\n"); goto Exit; } - memset( p_vend, 0, sizeof(*p_vend) ); + memset(p_vend, 0, sizeof(*p_vend)); - if (osm_vendor_init( p_vend, p_log, timeout ) < 0) { - free( p_vend ); + if (osm_vendor_init(p_vend, p_log, timeout) < 0) { + free(p_vend); p_vend = NULL; } -Exit: - OSM_LOG_EXIT( p_log ); - return( p_vend ); + Exit: + OSM_LOG_EXIT(p_log); + return (p_vend); } /********************************************************************** **********************************************************************/ -void -osm_vendor_delete( - IN osm_vendor_t** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { osm_vendor_close_port(*pp_vend); - clear_madw( *pp_vend ); + clear_madw(*pp_vend); /* make sure all ports are closed */ umad_done(); pthread_mutex_destroy(&(*pp_vend)->cb_mutex); pthread_mutex_destroy(&(*pp_vend)->match_tbl_mutex); - free( *pp_vend ); + free(*pp_vend); *pp_vend = NULL; } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( - IN osm_vendor_t* const p_vend, - IN ib_port_attr_t* const p_attr_array, - IN uint32_t* const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_net64_t portguids[*p_num_ports]; + ib_net64_t portguids[*p_num_ports]; ib_net64_t *p_guid = portguids, *e = portguids + *p_num_ports; umad_ca_t ca; int lids[*p_num_ports]; @@ -554,17 +553,17 @@ osm_vendor_get_all_port_attr( int *p_lid = lids; int *p_linkstates = linkstates; int *p_portnum = portnums; - umad_port_t def_port = {""}; + umad_port_t def_port = { "" }; int r, i, j; int sm_lid = 0; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); - CL_ASSERT( p_vend && p_num_ports ); + CL_ASSERT(p_vend && p_num_ports); if (!*p_num_ports) { r = IB_INVALID_PARAMETER; - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_get_all_port_attr: ERR 5418: " "Ports in should be > 0\n"); goto Exit; @@ -576,9 +575,9 @@ osm_vendor_get_all_port_attr( */ if ((r = umad_get_ca_portguids(p_vend->ca_names[i], p_guid, e - p_guid)) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5419: " - "Unable to get CA %s port guids (%s)\n", + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5419: " + "Unable to get CA %s port guids (%s)\n", p_vend->ca_names[i], strerror(r)); goto Exit; } @@ -615,11 +614,11 @@ osm_vendor_get_all_port_attr( portnums[0] = def_port.portnum; sm_lid = def_port.sm_lid; - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_all_port_attr: " - "assign CA %s port %d guid (0x%"PRIx64") as the default port\n", - def_port.ca_name, def_port.portnum, - cl_hton64(def_port.port_guid)); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_all_port_attr: " + "assign CA %s port %d guid (0x%" PRIx64 + ") as the default port\n", def_port.ca_name, + def_port.portnum, cl_hton64(def_port.port_guid)); umad_release_port(&def_port); } @@ -636,7 +635,8 @@ osm_vendor_get_all_port_attr( if (j == 0) p_attr_array[j].sm_lid = sm_lid; else - p_attr_array[j].sm_lid = p_vend->umad_port.sm_lid; + p_attr_array[j].sm_lid = + p_vend->umad_port.sm_lid; p_attr_array[j].link_state = linkstates[i]; j++; } @@ -645,27 +645,26 @@ osm_vendor_get_all_port_attr( } else r = IB_INSUFFICIENT_MEMORY; -Exit: - OSM_LOG_EXIT( p_vend->p_log ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); return r; } /********************************************************************** **********************************************************************/ static int -osm_vendor_open_port( - IN osm_vendor_t* const p_vend, - IN const ib_net64_t port_guid ) +osm_vendor_open_port(IN osm_vendor_t * const p_vend, + IN const ib_net64_t port_guid) { - ib_net64_t portguids[OSM_UMAD_MAX_PORTS_PER_CA + 1]; + ib_net64_t portguids[OSM_UMAD_MAX_PORTS_PER_CA + 1]; int i = 0, umad_port_id = -1, found = 0; int ca, r; - CL_ASSERT( p_vend ); + CL_ASSERT(p_vend); - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_open_port ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_open_port); - CL_ASSERT( port_guid ); + CL_ASSERT(port_guid); if (p_vend->umad_port_id >= 0) { umad_port_id = p_vend->umad_port_id; @@ -676,9 +675,9 @@ osm_vendor_open_port( if ((r = umad_get_ca_portguids(p_vend->ca_names[ca], portguids, OSM_UMAD_MAX_CAS)) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5421: " - "Unable to get CA %s port guids (%s)\n", + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 5421: " + "Unable to get CA %s port guids (%s)\n", p_vend->ca_names[ca], strerror(r)); goto Exit; } @@ -697,72 +696,71 @@ osm_vendor_open_port( /* * No local CA owns this guid! */ - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_open_port: ERR 5422: " - "Unable to find requested CA guid 0x%"PRIx64"\n", - cl_ntoh64(port_guid) ); + "Unable to find requested CA guid 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); goto Exit; } /* Port found, try to open it */ if (umad_get_ca(p_vend->ca_names[ca], &p_vend->umad_ca) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542A: " - "umad_get_ca() failed\n" ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 542A: " + "umad_get_ca() failed\n"); goto Exit; } /* Validate that node is an IB node type */ - if (p_vend->umad_ca.node_type < 1 || - p_vend->umad_ca.node_type > 3) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542D: " - "Node type %d is not an IB node type\n", - p_vend->umad_ca.node_type ); - fprintf( stderr, "Node type %d is not an IB node type\n", - p_vend->umad_ca.node_type ); + if (p_vend->umad_ca.node_type < 1 || p_vend->umad_ca.node_type > 3) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 542D: " + "Node type %d is not an IB node type\n", + p_vend->umad_ca.node_type); + fprintf(stderr, "Node type %d is not an IB node type\n", + p_vend->umad_ca.node_type); goto Exit; } if (umad_get_port(p_vend->ca_names[ca], i, &p_vend->umad_port) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542B: " - "umad_get_port() failed\n" ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 542B: " + "umad_get_port() failed\n"); goto Exit; } if ((umad_port_id = umad_open_port(p_vend->ca_names[ca], i)) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 542C: " - "umad_open_port() failed\n" ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 542C: " + "umad_open_port() failed\n"); goto Exit; } p_vend->umad_port_id = umad_port_id; /* start receiver thread */ - if (!(p_vend->receiver = calloc(1, sizeof (umad_receiver_t)))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + if (!(p_vend->receiver = calloc(1, sizeof(umad_receiver_t)))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_open_port: ERR 5423: " - "Unable to alloc receiver struct\n" ); + "Unable to alloc receiver struct\n"); umad_close_port(umad_port_id); p_vend->umad_port_id = umad_port_id = -1; goto Exit; } if (umad_receiver_start(p_vend) != 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_open_port: ERR 5420: " - "umad_receiver_init failed\n" ); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_open_port: ERR 5420: " + "umad_receiver_init failed\n"); umad_close_port(umad_port_id); p_vend->umad_port_id = umad_port_id = -1; } -Exit: - OSM_LOG_EXIT( p_vend->p_log ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); return umad_port_id; } -static void osm_vendor_close_port(osm_vendor_t* const p_vend) +static void osm_vendor_close_port(osm_vendor_t * const p_vend) { umad_receiver_t *p_ur; int i; @@ -798,13 +796,12 @@ static int set_bit(int nr, void *method_mask) /********************************************************************** **********************************************************************/ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_user_bind, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void* context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_user_bind, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { ib_net64_t port_guid; osm_umad_bind_info_t *p_bind = 0; @@ -812,33 +809,34 @@ osm_vendor_bind( int umad_port_id; uint8_t rmpp_version; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_bind ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_bind); - CL_ASSERT( p_user_bind ); - CL_ASSERT( p_mad_pool ); - CL_ASSERT( mad_recv_callback ); - CL_ASSERT( send_err_callback ); + CL_ASSERT(p_user_bind); + CL_ASSERT(p_mad_pool); + CL_ASSERT(mad_recv_callback); + CL_ASSERT(send_err_callback); port_guid = p_user_bind->port_guid; - osm_log( p_vend->p_log, OSM_LOG_INFO, "osm_vendor_bind: " - "Binding to port 0x%" PRIx64 "\n", cl_ntoh64( port_guid ) ); + osm_log(p_vend->p_log, OSM_LOG_INFO, "osm_vendor_bind: " + "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); - if ((umad_port_id = osm_vendor_open_port( p_vend, port_guid )) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + if ((umad_port_id = osm_vendor_open_port(p_vend, port_guid)) < 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5424: " - "Unable to open port 0x%" PRIx64 "\n", cl_ntoh64( port_guid )); + "Unable to open port 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); goto Exit; } - if (!(p_bind = malloc( sizeof(*p_bind) ))) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + if (!(p_bind = malloc(sizeof(*p_bind)))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5425: " - "Unable to allocate internal bind object\n" ); + "Unable to allocate internal bind object\n"); goto Exit; } - memset( p_bind, 0, sizeof(*p_bind) ); + memset(p_bind, 0, sizeof(*p_bind)); p_bind->p_vend = p_vend; p_bind->port_id = umad_port_id; p_bind->client_context = context; @@ -867,7 +865,6 @@ osm_vendor_bind( set_bit(IB_MAD_METHOD_TRAP, &method_mask); set_bit(IB_MAD_METHOD_TRAP_REPRESS, &method_mask); } - #ifndef VENDOR_RMPP_SUPPORT rmpp_version = 0; #else @@ -882,7 +879,7 @@ osm_vendor_bind( p_user_bind->mad_class, p_user_bind->class_version, rmpp_version, method_mask)) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5426: " "Unable to register class %u version %u\n", p_user_bind->mad_class, p_user_bind->class_version); @@ -893,7 +890,7 @@ osm_vendor_bind( if (p_bind->agent_id >= UMAD_CA_MAX_AGENTS || p_vend->agents[p_bind->agent_id]) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5427: " "bad agent id %u or duplicate agent for class %u vers %u\n", p_bind->agent_id, p_user_bind->mad_class, @@ -909,9 +906,10 @@ osm_vendor_bind( if (p_user_bind->mad_class == IB_MCLASS_SUBN_DIR) { if ((p_bind->agent_id1 = umad_register(p_vend->umad_port_id, IB_MCLASS_SUBN_LID, - p_user_bind->class_version, - 0, method_mask)) < 0) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + p_user_bind-> + class_version, 0, + method_mask)) < 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5428: " "Unable to register class 1 version %u\n", p_user_bind->class_version); @@ -922,7 +920,7 @@ osm_vendor_bind( if (p_bind->agent_id1 >= UMAD_CA_MAX_AGENTS || p_vend->agents[p_bind->agent_id1]) { - osm_log( p_vend->p_log, OSM_LOG_ERROR, + osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_bind: ERR 5429: " "bad agent id %u or duplicate agent for class 1 vers %u\n", p_bind->agent_id1, p_user_bind->class_version); @@ -934,101 +932,94 @@ osm_vendor_bind( p_vend->agents[p_bind->agent_id1] = p_bind; } -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( (osm_bind_handle_t)p_bind ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return ((osm_bind_handle_t) p_bind); } /********************************************************************** **********************************************************************/ void -__osm_vendor_recv_dummy_cb( - IN osm_madw_t *p_madw, - IN void *bind_context, - IN osm_madw_t *p_req_madw ) +__osm_vendor_recv_dummy_cb(IN osm_madw_t * p_madw, + IN void *bind_context, IN osm_madw_t * p_req_madw) { #ifdef _DEBUG_ - fprintf(stderr, "__osm_vendor_recv_dummy_cb: Ignoring received MAD after osm_vendor_unbind\n"); + fprintf(stderr, + "__osm_vendor_recv_dummy_cb: Ignoring received MAD after osm_vendor_unbind\n"); #endif } /********************************************************************** **********************************************************************/ void -__osm_vendor_send_err_dummy_cb( - IN void* bind_context, - IN osm_madw_t *p_req_madw ) +__osm_vendor_send_err_dummy_cb(IN void *bind_context, + IN osm_madw_t * p_req_madw) { #ifdef _DEBUG_ - fprintf(stderr, "__osm_vendor_send_err_dummy_cb: Ignoring send error after osm_vendor_unbind\n"); + fprintf(stderr, + "__osm_vendor_send_err_dummy_cb: Ignoring send error after osm_vendor_unbind\n"); #endif } /********************************************************************** **********************************************************************/ -void -osm_vendor_unbind( - IN osm_bind_handle_t h_bind) +void osm_vendor_unbind(IN osm_bind_handle_t h_bind) { - osm_umad_bind_info_t *p_bind = ( osm_umad_bind_info_t * ) h_bind; + osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *) h_bind; osm_vendor_t *p_vend = p_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_unbind ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_unbind); pthread_mutex_lock(&p_vend->cb_mutex); p_bind->mad_recv_callback = __osm_vendor_recv_dummy_cb; p_bind->send_err_callback = __osm_vendor_send_err_dummy_cb; pthread_mutex_unlock(&p_vend->cb_mutex); - OSM_LOG_EXIT( p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -ib_mad_t* -osm_vendor_get( - IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t* const p_vw ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { - osm_umad_bind_info_t *p_bind = ( osm_umad_bind_info_t * ) h_bind; + osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *) h_bind; osm_vendor_t *p_vend = p_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " - "Acquiring UMAD for p_madw = %p, size = %u\n", p_vw, mad_size ); - CL_ASSERT( p_vw ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " + "Acquiring UMAD for p_madw = %p, size = %u\n", p_vw, mad_size); + CL_ASSERT(p_vw); p_vw->size = mad_size; p_vw->umad = umad_alloc(1, mad_size + umad_size()); /* track locally */ p_vw->h_bind = h_bind; - osm_log( p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " - "Acquired UMAD %p, size = %u\n", p_vw->umad, p_vw->size ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_get: " + "Acquired UMAD %p, size = %u\n", p_vw->umad, p_vw->size); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); return umad_get_mad(p_vw->umad); } /********************************************************************** **********************************************************************/ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vw ) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { - osm_umad_bind_info_t *p_bind = ( osm_umad_bind_info_t * ) h_bind; + osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *) h_bind; osm_vendor_t *p_vend = p_bind->p_vend; osm_madw_t *p_madw; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_put ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); - CL_ASSERT( p_vw ); + CL_ASSERT(p_vw); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_put: " - "Retiring UMAD %p\n", p_vw->umad ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_put: " + "Retiring UMAD %p\n", p_vw->umad); /* * We moved the removal of the transaction to immediately after @@ -1038,26 +1029,24 @@ osm_vendor_put( /* free the mad but the wrapper is part of the madw object */ umad_free(p_vw->umad); p_vw->umad = 0; - p_madw = PARENT_STRUCT( p_vw, osm_madw_t, vend_wrap); + p_madw = PARENT_STRUCT(p_vw, osm_madw_t, vend_wrap); p_madw->p_mad = NULL; - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { - osm_umad_bind_info_t* const p_bind = h_bind; - osm_vendor_t* const p_vend = p_bind->p_vend; - osm_vend_wrap_t* const p_vw = osm_madw_get_vend_ptr( p_madw ); - osm_mad_addr_t* const p_mad_addr = osm_madw_get_mad_addr_ptr( p_madw ); - ib_mad_t* const p_mad = osm_madw_get_mad_ptr( p_madw ); - ib_sa_mad_t* const p_sa = (ib_sa_mad_t *)p_mad; + osm_umad_bind_info_t *const p_bind = h_bind; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + ib_mad_t *const p_mad = osm_madw_get_mad_ptr(p_madw); + ib_sa_mad_t *const p_sa = (ib_sa_mad_t *) p_mad; int ret = -1; int is_rmpp = 0; uint32_t sent_mad_size; @@ -1065,10 +1054,10 @@ osm_vendor_send( uint32_t paylen = 0; #endif - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_send ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_send); - CL_ASSERT( p_vw->h_bind == h_bind ); - CL_ASSERT( p_mad == umad_get_mad(p_vw->umad) ); + CL_ASSERT(p_vw->h_bind == h_bind); + CL_ASSERT(p_mad == umad_get_mad(p_vw->umad)); if (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) { umad_set_addr_net(p_vw->umad, 0xffff, 0, 0, 0); @@ -1087,9 +1076,9 @@ osm_vendor_send( IB_QP1_WELL_KNOWN_Q_KEY); umad_set_grh(p_vw->umad, 0); /* FIXME: GRH support */ umad_set_pkey(p_vw->umad, 0); - /* FIXME: p_mad_addr->addr_type.gsi.pkey to index */ - if (ib_class_is_rmpp(p_mad->mgmt_class)) { /* RMPP GSI classes FIXME: no GRH */ - if (!ib_rmpp_is_flag_set((ib_rmpp_mad_t *)p_sa, + /* FIXME: p_mad_addr->addr_type.gsi.pkey to index */ + if (ib_class_is_rmpp(p_mad->mgmt_class)) { /* RMPP GSI classes FIXME: no GRH */ + if (!ib_rmpp_is_flag_set((ib_rmpp_mad_t *) p_sa, IB_RMPP_FLAG_ACTIVE)) { /* Clear RMPP header when RMPP not ACTIVE */ p_sa->rmpp_version = 0; @@ -1099,16 +1088,16 @@ osm_vendor_send( #ifdef VENDOR_RMPP_SUPPORT } else is_rmpp = 1; - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, "osm_vendor_send: RMPP %d length %d\n", - ib_rmpp_is_flag_set((ib_rmpp_mad_t *)p_sa, - IB_RMPP_FLAG_ACTIVE), - p_madw->mad_size); + ib_rmpp_is_flag_set((ib_rmpp_mad_t *) p_sa, + IB_RMPP_FLAG_ACTIVE), + p_madw->mad_size); #else } else { p_sa->rmpp_version = 1; - p_sa->seg_num = cl_ntoh32(1); /* first DATA is seg 1 */ - p_sa->rmpp_flags |= (uint8_t)0x70; /* RRespTime of 14 (high 5 bits) */ + p_sa->seg_num = cl_ntoh32(1); /* first DATA is seg 1 */ + p_sa->rmpp_flags |= (uint8_t) 0x70; /* RRespTime of 14 (high 5 bits) */ p_sa->rmpp_status = 0; paylen = p_madw->mad_size - IB_SA_MAD_HDR_SIZE; paylen += (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); @@ -1117,7 +1106,7 @@ osm_vendor_send( #endif } -Resp: + Resp: if (resp_expected) put_madw(p_vend, p_madw, &p_mad->trans_id); @@ -1125,13 +1114,12 @@ Resp: sent_mad_size = p_madw->mad_size; #else sent_mad_size = is_rmpp ? p_madw->mad_size - IB_SA_MAD_HDR_SIZE : - p_madw->mad_size, + p_madw->mad_size, #endif - - if ((ret = umad_send(p_bind->port_id, p_bind->agent_id, p_vw->umad, - sent_mad_size, - resp_expected ? p_vend->timeout : 0, - p_vend->max_retries)) < 0) { + if ((ret = umad_send(p_bind->port_id, p_bind->agent_id, p_vw->umad, + sent_mad_size, + resp_expected ? p_vend->timeout : 0, + p_vend->max_retries)) < 0) { if (resp_expected) get_madw(p_vend, &p_mad->trans_id); /* remove from aging table */ osm_log(p_vend->p_log, OSM_LOG_ERROR, @@ -1140,49 +1128,44 @@ Resp: p_madw, sent_mad_size, ret); p_madw->status = IB_ERROR; pthread_mutex_lock(&p_vend->cb_mutex); - (*p_bind->send_err_callback)(p_bind->client_context, p_madw); /* cb frees madw */ + (*p_bind->send_err_callback) (p_bind->client_context, p_madw); /* cb frees madw */ pthread_mutex_unlock(&p_vend->cb_mutex); goto Exit; } if (!resp_expected) - osm_mad_pool_put(p_bind->p_mad_pool, p_madw); + osm_mad_pool_put(p_bind->p_mad_pool, p_madw); osm_log(p_vend->p_log, OSM_LOG_DEBUG, "osm_vendor_send: " "Completed sending %s p_madw = %p\n", resp_expected ? "request" : "response or unsolicited", p_madw); -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( ret ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (ret); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_vendor_local_lid_change( - IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *)h_bind; + osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *) h_bind; osm_vendor_t *p_vend = p_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); ; - OSM_LOG_EXIT( p_vend->p_log ); - return( 0 ); + OSM_LOG_EXIT(p_vend->p_log); + return (0); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *)h_bind; + osm_umad_bind_info_t *p_bind = (osm_umad_bind_info_t *) h_bind; osm_vendor_t *p_vend = p_bind->p_vend; char issmstring[24]; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); sprintf(issmstring, "/dev/infiniband/issm%d", p_vend->umad_port_id); if (TRUE == is_sm_val) { p_vend->issmfd = open(issmstring, O_NONBLOCK); @@ -1203,15 +1186,12 @@ osm_vendor_set_sm( } p_vend->issmfd = -1; } - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { umad_debug(level); } -#endif /* OSM_VENDOR_INTF_OPENIB */ +#endif /* OSM_VENDOR_INTF_OPENIB */ diff --git a/opensm/libvendor/osm_vendor_ibumad_sa.c b/opensm/libvendor/osm_vendor_ibumad_sa.c index 1abca91..42a6d3a 100644 --- a/opensm/libvendor/osm_vendor_ibumad_sa.c +++ b/opensm/libvendor/osm_vendor_ibumad_sa.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -51,15 +51,15 @@ /* this struct is the internal rep of the bind handle */ typedef struct _osmv_sa_bind_info { - osm_bind_handle_t h_bind; - osm_log_t *p_log; - osm_vendor_t *p_vendor; - osm_mad_pool_t *p_mad_pool; - uint64_t port_guid; - cl_event_t sync_event; - uint64_t last_lids_update_sec; - uint16_t lid; - uint16_t sm_lid; + osm_bind_handle_t h_bind; + osm_log_t *p_log; + osm_vendor_t *p_vendor; + osm_mad_pool_t *p_mad_pool; + uint64_t port_guid; + cl_event_t sync_event; + uint64_t last_lids_update_sec; + uint16_t lid; + uint16_t sm_lid; } osmv_sa_bind_info_t; /***************************************************************************** @@ -75,117 +75,107 @@ typedef struct _osmv_sa_bind_info { request structure) is attached as the p_madw->context.ni_context.node_guid */ void -__osmv_sa_mad_rcv_cb( - IN osm_madw_t *p_madw, - IN void* bind_context, - IN osm_madw_t *p_req_madw) +__osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, + IN void *bind_context, IN osm_madw_t * p_req_madw) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)bind_context; - osmv_query_req_t *p_query_req_copy = NULL; - osmv_query_res_t query_res; - ib_sa_mad_t *p_sa_mad; - ib_net16_t mad_status; - - OSM_LOG_ENTER( p_bind->p_log, __osmv_sa_mad_rcv_cb ); - - if (! p_req_madw) - { - osm_log( p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: " - "Ignoring a non-response mad\n"); - osm_mad_pool_put(p_bind->p_mad_pool, p_madw); - goto Exit; - } - - /* obtain the sent context since we store it during send in the ni_ctx */ - p_query_req_copy = - (osmv_query_req_t *)(long*)(long)(p_req_madw->context.ni_context.node_guid); - - /* provide the context of the original request in the result */ - query_res.query_context = p_query_req_copy->query_context; - - /* provide the resulting madw */ - query_res.p_result_madw = p_madw; - - /* update the req fields */ - p_sa_mad = ( ib_sa_mad_t * ) p_madw->p_mad; - - /* if we got a remote error track it in the status */ - mad_status = ( ib_net16_t ) ( p_sa_mad->status & IB_SMP_STATUS_MASK ); - if (mad_status != IB_SUCCESS) - { - osm_log( p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 5501: " - "Remote error:0x%04X\n", mad_status - ); - query_res.status = IB_REMOTE_ERROR; - } - else - { - query_res.status = IB_SUCCESS; - } - - /* what if we have got back an empty mad ? */ - if (! p_madw->mad_size) - { - osm_log( p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 5502: " - "Got an empty mad\n" - ); - query_res.status = IB_ERROR; - } - - if (IB_SUCCESS == mad_status) - { - - /* if we are in not in a method response of an rmpp nature we must get only 1 */ - /* HACK: in the future we might need to be smarter for other methods... */ - if (p_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) - { - query_res.result_cnt = 1; - } - else - { + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) bind_context; + osmv_query_req_t *p_query_req_copy = NULL; + osmv_query_res_t query_res; + ib_sa_mad_t *p_sa_mad; + ib_net16_t mad_status; + + OSM_LOG_ENTER(p_bind->p_log, __osmv_sa_mad_rcv_cb); + + if (!p_req_madw) { + osm_log(p_bind->p_log, OSM_LOG_DEBUG, + "__osmv_sa_mad_rcv_cb: " + "Ignoring a non-response mad\n"); + osm_mad_pool_put(p_bind->p_mad_pool, p_madw); + goto Exit; + } + + /* obtain the sent context since we store it during send in the ni_ctx */ + p_query_req_copy = + (osmv_query_req_t *) (long *)(long)(p_req_madw->context.ni_context. + node_guid); + + /* provide the context of the original request in the result */ + query_res.query_context = p_query_req_copy->query_context; + + /* provide the resulting madw */ + query_res.p_result_madw = p_madw; + + /* update the req fields */ + p_sa_mad = (ib_sa_mad_t *) p_madw->p_mad; + + /* if we got a remote error track it in the status */ + mad_status = (ib_net16_t) (p_sa_mad->status & IB_SMP_STATUS_MASK); + if (mad_status != IB_SUCCESS) { + osm_log(p_bind->p_log, OSM_LOG_ERROR, + "__osmv_sa_mad_rcv_cb: ERR 5501: " + "Remote error:0x%04X\n", mad_status); + query_res.status = IB_REMOTE_ERROR; + } else { + query_res.status = IB_SUCCESS; + } + + /* what if we have got back an empty mad ? */ + if (!p_madw->mad_size) { + osm_log(p_bind->p_log, OSM_LOG_ERROR, + "__osmv_sa_mad_rcv_cb: ERR 5502: " + "Got an empty mad\n"); + query_res.status = IB_ERROR; + } + + if (IB_SUCCESS == mad_status) { + + /* if we are in not in a method response of an rmpp nature we must get only 1 */ + /* HACK: in the future we might need to be smarter for other methods... */ + if (p_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) { + query_res.result_cnt = 1; + } else { #ifndef VENDOR_RMPP_SUPPORT - if (mad_status != IB_SUCCESS) - query_res.result_cnt = 0; - else - query_res.result_cnt = 1; + if (mad_status != IB_SUCCESS) + query_res.result_cnt = 0; + else + query_res.result_cnt = 1; #else - /* we used the offset value to calculate the number of - records in here */ - query_res.result_cnt = - (uintn_t) - ( ( p_madw->mad_size - IB_SA_MAD_HDR_SIZE ) / - ib_get_attr_size( p_sa_mad->attr_offset ) ); - osm_log( p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: Count = %u = %zu / %u (%zu)\n", - query_res.result_cnt, p_madw->mad_size - IB_SA_MAD_HDR_SIZE, - ib_get_attr_size( p_sa_mad->attr_offset ), - ( p_madw->mad_size - IB_SA_MAD_HDR_SIZE ) % - ib_get_attr_size( p_sa_mad->attr_offset ) - ); + /* we used the offset value to calculate the number of + records in here */ + query_res.result_cnt = (uintn_t) + ((p_madw->mad_size - IB_SA_MAD_HDR_SIZE) / + ib_get_attr_size(p_sa_mad->attr_offset)); + osm_log(p_bind->p_log, OSM_LOG_DEBUG, + "__osmv_sa_mad_rcv_cb: Count = %u = %zu / %u (%zu)\n", + query_res.result_cnt, + p_madw->mad_size - IB_SA_MAD_HDR_SIZE, + ib_get_attr_size(p_sa_mad->attr_offset), + (p_madw->mad_size - + IB_SA_MAD_HDR_SIZE) % + ib_get_attr_size(p_sa_mad->attr_offset) + ); #endif - } - } + } + } - query_res.query_type = p_query_req_copy->query_type; + query_res.query_type = p_query_req_copy->query_type; - p_query_req_copy->pfn_query_cb( &query_res ); + p_query_req_copy->pfn_query_cb(&query_res); - if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) - cl_event_signal( &p_bind->sync_event ); + if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) + cl_event_signal(&p_bind->sync_event); - Exit: + Exit: - /* free the copied query request if found */ - if (p_query_req_copy) free(p_query_req_copy); + /* free the copied query request if found */ + if (p_query_req_copy) + free(p_query_req_copy); - /* put back the request madw */ - if (p_req_madw) - osm_mad_pool_put(p_bind->p_mad_pool, p_req_madw); + /* put back the request madw */ + if (p_req_madw) + osm_mad_pool_put(p_bind->p_mad_pool, p_req_madw); - OSM_LOG_EXIT( p_bind->p_log ); + OSM_LOG_EXIT(p_bind->p_log); } /***************************************************************************** @@ -195,38 +185,37 @@ __osmv_sa_mad_rcv_cb( Only report the error and get rid of the mad wrapper */ -void -__osmv_sa_mad_err_cb( - IN void* bind_context, - IN osm_madw_t *p_madw) +void __osmv_sa_mad_err_cb(IN void *bind_context, IN osm_madw_t * p_madw) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)bind_context; - osmv_query_req_t *p_query_req_copy = NULL; - osmv_query_res_t query_res; + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) bind_context; + osmv_query_req_t *p_query_req_copy = NULL; + osmv_query_res_t query_res; - OSM_LOG_ENTER( p_bind->p_log, __osmv_sa_mad_err_cb ); + OSM_LOG_ENTER(p_bind->p_log, __osmv_sa_mad_err_cb); - /* Obtain the sent context etc */ - p_query_req_copy = - (osmv_query_req_t *)(long*)(long)(p_madw->context.ni_context.node_guid); + /* Obtain the sent context etc */ + p_query_req_copy = + (osmv_query_req_t *) (long *)(long)(p_madw->context.ni_context. + node_guid); - /* provide the context of the original request in the result */ - query_res.query_context = p_query_req_copy->query_context; + /* provide the context of the original request in the result */ + query_res.query_context = p_query_req_copy->query_context; - query_res.p_result_madw = p_madw; + query_res.p_result_madw = p_madw; - query_res.status = IB_TIMEOUT; - query_res.result_cnt = 0; + query_res.status = IB_TIMEOUT; + query_res.result_cnt = 0; - query_res.query_type = p_query_req_copy->query_type; + query_res.query_type = p_query_req_copy->query_type; - p_query_req_copy->pfn_query_cb( &query_res ); + p_query_req_copy->pfn_query_cb(&query_res); - if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) - cl_event_signal( &p_bind->sync_event ); + if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) + cl_event_signal(&p_bind->sync_event); - if (p_query_req_copy) free(p_query_req_copy); - OSM_LOG_EXIT( p_bind->p_log ); + if (p_query_req_copy) + free(p_query_req_copy); + OSM_LOG_EXIT(p_bind->p_log); } /***************************************************************************** @@ -238,196 +227,175 @@ __osmv_sa_mad_err_cb( we query for multiple guids. *****************************************************************************/ ib_api_status_t -__osmv_get_lid_and_sm_lid_by_port_guid( - IN osm_vendor_t* const p_vend, - IN ib_net64_t port_guid, - IN OUT uint64_t* p_lids_update_time_sec, - OUT uint16_t* lid, - OUT uint16_t* sm_lid) +__osmv_get_lid_and_sm_lid_by_port_guid(IN osm_vendor_t * const p_vend, + IN ib_net64_t port_guid, + IN OUT uint64_t * p_lids_update_time_sec, + OUT uint16_t * lid, + OUT uint16_t * sm_lid) { - ib_api_status_t status; - ib_port_attr_t *p_attr_array; - uint32_t num_ports = MAX_PORTS; - uint32_t port_num; - - OSM_LOG_ENTER( p_vend->p_log, __osmv_get_lid_and_sm_lid_by_port_guid ); - - /* use previous values if current time is close enough to previous query */ - if ((time(NULL) <= *p_lids_update_time_sec + 30) && *lid && *sm_lid) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Using previously stored lid:0x%04x sm_lid:0x%04x\n", - *lid, *sm_lid - ); - status = IB_SUCCESS; - goto Exit; - } - - /* obtain the number of available ports */ - num_ports = MAX_PORTS; - status = osm_vendor_get_all_port_attr(p_vend, NULL, &num_ports); - if (status != IB_INSUFFICIENT_MEMORY) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 5503: " - "expected to get IB_INSUFFICIENT_MEMORY but got: %s\n", - ib_get_err_str(status) - ); - status = IB_ERROR; - goto Exit; - } - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Found total of %u ports. Looking for guid:0x%016" PRIx64 "\n", - num_ports, cl_ntoh64(port_guid) - ); - - /* allocate the attributes */ - p_attr_array = - (ib_port_attr_t *)malloc(sizeof(ib_port_attr_t)*num_ports); - - /* obtain the attributes */ - status = osm_vendor_get_all_port_attr(p_vend, p_attr_array, &num_ports); - if (status != IB_SUCCESS) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 5504: " - "Failed to get port attributes (error: %s)\n", - ib_get_err_str(status) - ); - free(p_attr_array); - goto Exit; - } - - status = IB_ERROR; - /* find the port requested in the list */ - for (port_num = 0; (port_nump_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Found guid:0x%016" PRIx64 " with idx:%d\n", - cl_ntoh64(port_guid), port_num); - } - } - - free(p_attr_array); - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + ib_port_attr_t *p_attr_array; + uint32_t num_ports = MAX_PORTS; + uint32_t port_num; + + OSM_LOG_ENTER(p_vend->p_log, __osmv_get_lid_and_sm_lid_by_port_guid); + + /* use previous values if current time is close enough to previous query */ + if ((time(NULL) <= *p_lids_update_time_sec + 30) && *lid && *sm_lid) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Using previously stored lid:0x%04x sm_lid:0x%04x\n", + *lid, *sm_lid); + status = IB_SUCCESS; + goto Exit; + } + + /* obtain the number of available ports */ + num_ports = MAX_PORTS; + status = osm_vendor_get_all_port_attr(p_vend, NULL, &num_ports); + if (status != IB_INSUFFICIENT_MEMORY) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 5503: " + "expected to get IB_INSUFFICIENT_MEMORY but got: %s\n", + ib_get_err_str(status) + ); + status = IB_ERROR; + goto Exit; + } + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Found total of %u ports. Looking for guid:0x%016" PRIx64 "\n", + num_ports, cl_ntoh64(port_guid) + ); + + /* allocate the attributes */ + p_attr_array = + (ib_port_attr_t *) malloc(sizeof(ib_port_attr_t) * num_ports); + + /* obtain the attributes */ + status = osm_vendor_get_all_port_attr(p_vend, p_attr_array, &num_ports); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 5504: " + "Failed to get port attributes (error: %s)\n", + ib_get_err_str(status) + ); + free(p_attr_array); + goto Exit; + } + + status = IB_ERROR; + /* find the port requested in the list */ + for (port_num = 0; (port_num < num_ports) && (status == IB_ERROR); + port_num++) { + if (p_attr_array[port_num].port_guid == port_guid) { + *lid = p_attr_array[port_num].lid; + *sm_lid = p_attr_array[port_num].sm_lid; + *p_lids_update_time_sec = time(NULL); + status = IB_SUCCESS; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Found guid:0x%016" PRIx64 " with idx:%d\n", + cl_ntoh64(port_guid), port_num); + } + } + + free(p_attr_array); + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /***************************************************************************** *****************************************************************************/ osm_bind_handle_t -osmv_bind_sa( - IN osm_vendor_t* const p_vend, - IN osm_mad_pool_t* const p_mad_pool, - IN ib_net64_t port_guid - ) +osmv_bind_sa(IN osm_vendor_t * const p_vend, + IN osm_mad_pool_t * const p_mad_pool, IN ib_net64_t port_guid) { - osm_bind_info_t bind_info; - osm_log_t *p_log = p_vend->p_log; - ib_api_status_t status = IB_SUCCESS; - osmv_sa_bind_info_t *p_sa_bind_info; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_log, osmv_bind_sa ); - - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_bind_sa: " - "Binding to port 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - - bind_info.port_guid = port_guid; - bind_info.mad_class = IB_MCLASS_SUBN_ADM; - bind_info.class_version = 2; - bind_info.is_responder = FALSE; - bind_info.is_trap_processor = FALSE; - bind_info.is_report_processor = FALSE; - bind_info.send_q_size = OSM_SM_DEFAULT_QP1_RCV_SIZE; - bind_info.recv_q_size = OSM_SM_DEFAULT_QP1_SEND_SIZE; - - /* allocate the new sa bind info */ - p_sa_bind_info = - (osmv_sa_bind_info_t *)malloc(sizeof(osmv_sa_bind_info_t)); - if (! p_sa_bind_info) - { - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5505: " - "Failed to allocate new bind structure\n" ); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - goto Exit; - } - - /* store some important context */ - p_sa_bind_info->p_log = p_log; - p_sa_bind_info->port_guid = port_guid; - p_sa_bind_info->p_mad_pool = p_mad_pool; - p_sa_bind_info->p_vendor = p_vend; - p_sa_bind_info->last_lids_update_sec = 0; - - /* Bind to the lower level */ - p_sa_bind_info->h_bind = - osm_vendor_bind( p_vend, - &bind_info, - p_mad_pool, - __osmv_sa_mad_rcv_cb, - __osmv_sa_mad_err_cb, - p_sa_bind_info); /* context provided to CBs */ - - if (p_sa_bind_info->h_bind == OSM_BIND_INVALID_HANDLE) - { - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5506: " - "Failed to bind to vendor GSI\n" ); - goto Exit; - } - - /* obtain the sm_lid from the vendor */ - status = - __osmv_get_lid_and_sm_lid_by_port_guid( - p_vend, port_guid, - &p_sa_bind_info->last_lids_update_sec, - &p_sa_bind_info->lid, - &p_sa_bind_info->sm_lid); - if (status != IB_SUCCESS) - { - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5507: " - "Failed to obtain the SM lid\n" ); - goto Exit; - } - - /* initialize the sync_event */ - cl_event_construct( &p_sa_bind_info->sync_event ); - cl_status = cl_event_init( &p_sa_bind_info->sync_event, TRUE ); - if( cl_status != CL_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 5508: " - "cl_init_event failed: %s\n", - ib_get_err_str(cl_status) - ); - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return ( p_sa_bind_info ); + osm_bind_info_t bind_info; + osm_log_t *p_log = p_vend->p_log; + ib_api_status_t status = IB_SUCCESS; + osmv_sa_bind_info_t *p_sa_bind_info; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_log, osmv_bind_sa); + + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_bind_sa: " + "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + + bind_info.port_guid = port_guid; + bind_info.mad_class = IB_MCLASS_SUBN_ADM; + bind_info.class_version = 2; + bind_info.is_responder = FALSE; + bind_info.is_trap_processor = FALSE; + bind_info.is_report_processor = FALSE; + bind_info.send_q_size = OSM_SM_DEFAULT_QP1_RCV_SIZE; + bind_info.recv_q_size = OSM_SM_DEFAULT_QP1_SEND_SIZE; + + /* allocate the new sa bind info */ + p_sa_bind_info = + (osmv_sa_bind_info_t *) malloc(sizeof(osmv_sa_bind_info_t)); + if (!p_sa_bind_info) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 5505: " + "Failed to allocate new bind structure\n"); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + goto Exit; + } + + /* store some important context */ + p_sa_bind_info->p_log = p_log; + p_sa_bind_info->port_guid = port_guid; + p_sa_bind_info->p_mad_pool = p_mad_pool; + p_sa_bind_info->p_vendor = p_vend; + p_sa_bind_info->last_lids_update_sec = 0; + + /* Bind to the lower level */ + p_sa_bind_info->h_bind = osm_vendor_bind(p_vend, &bind_info, p_mad_pool, __osmv_sa_mad_rcv_cb, __osmv_sa_mad_err_cb, p_sa_bind_info); /* context provided to CBs */ + + if (p_sa_bind_info->h_bind == OSM_BIND_INVALID_HANDLE) { + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 5506: " + "Failed to bind to vendor GSI\n"); + goto Exit; + } + + /* obtain the sm_lid from the vendor */ + status = + __osmv_get_lid_and_sm_lid_by_port_guid(p_vend, port_guid, + &p_sa_bind_info-> + last_lids_update_sec, + &p_sa_bind_info->lid, + &p_sa_bind_info->sm_lid); + if (status != IB_SUCCESS) { + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 5507: " + "Failed to obtain the SM lid\n"); + goto Exit; + } + + /* initialize the sync_event */ + cl_event_construct(&p_sa_bind_info->sync_event); + cl_status = cl_event_init(&p_sa_bind_info->sync_event, TRUE); + if (cl_status != CL_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 5508: " + "cl_init_event failed: %s\n", ib_get_err_str(cl_status) + ); + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (p_sa_bind_info); } /***************************************************************************** @@ -443,15 +411,14 @@ osmv_bind_sa( * * SYNOPSIS */ -typedef struct _osmv_sa_mad_data -{ - /* MAD data. */ - uint8_t method; - ib_net16_t attr_id; - ib_net16_t attr_offset; - ib_net32_t attr_mod; - ib_net64_t comp_mask; - void *p_attr; +typedef struct _osmv_sa_mad_data { + /* MAD data. */ + uint8_t method; + ib_net16_t attr_id; + ib_net16_t attr_offset; + ib_net32_t attr_mod; + ib_net64_t comp_mask; + void *p_attr; } osmv_sa_mad_data_t; /* * method @@ -478,140 +445,128 @@ typedef struct _osmv_sa_mad_data *****************************************************************************/ /* Send a MAD out on the GSI interface */ ib_api_status_t -__osmv_send_sa_req( - IN osmv_sa_bind_info_t* p_bind, - IN const osmv_sa_mad_data_t * const p_sa_mad_data, - IN const osmv_query_req_t * const p_query_req ) +__osmv_send_sa_req(IN osmv_sa_bind_info_t * p_bind, + IN const osmv_sa_mad_data_t * const p_sa_mad_data, + IN const osmv_query_req_t * const p_query_req) { - ib_api_status_t status; - ib_mad_t *p_mad_hdr; - ib_sa_mad_t *p_sa_mad; - osm_madw_t *p_madw; - osm_log_t *p_log = p_bind->p_log; - static atomic32_t trans_id; - boolean_t sync; - osmv_query_req_t *p_query_req_copy; - - OSM_LOG_ENTER( p_log, __osmv_send_sa_req ); - - /* - since the sm_lid might change we obtain it every send - (actually it is cached in the bind object and refreshed - every 30sec by this proc ) - */ - status = - __osmv_get_lid_and_sm_lid_by_port_guid( - p_bind->p_vendor, p_bind->port_guid, - &p_bind->last_lids_update_sec, - &p_bind->lid, - &p_bind->sm_lid ); - if (status != IB_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 5509: " - "Failed to obtain the SM lid\n" ); - goto Exit; - } - - /* Get a MAD wrapper for the send */ - p_madw = osm_mad_pool_get( - p_bind->p_mad_pool, - p_bind->h_bind, - MAD_BLOCK_SIZE, - NULL ); - - if( p_madw == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 5510: " - "Unable to acquire MAD\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* Initialize the Sent MAD: */ - - /* Initialize the MAD buffer for the send operation. */ - p_mad_hdr = osm_madw_get_mad_ptr( p_madw ); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - /* Get a new transaction Id */ - cl_atomic_inc( &trans_id ); - - /* Cleanup the MAD from any residue */ - memset(p_sa_mad, 0, MAD_BLOCK_SIZE); - - /* Initialize the standard MAD header. */ - ib_mad_init_new( - p_mad_hdr, /* mad pointer */ - IB_MCLASS_SUBN_ADM, /* class */ - ( uint8_t ) 2, /* version */ - p_sa_mad_data->method, /* method */ - cl_hton64( ( uint64_t ) trans_id ),/* tid */ - p_sa_mad_data->attr_id, /* attr id */ - p_sa_mad_data->attr_mod /* attr mod */ - ); - - /* Set the query information. */ - p_sa_mad->sm_key = p_query_req->sm_key; - p_sa_mad->attr_offset = 0; - p_sa_mad->comp_mask = p_sa_mad_data->comp_mask; + ib_api_status_t status; + ib_mad_t *p_mad_hdr; + ib_sa_mad_t *p_sa_mad; + osm_madw_t *p_madw; + osm_log_t *p_log = p_bind->p_log; + static atomic32_t trans_id; + boolean_t sync; + osmv_query_req_t *p_query_req_copy; + + OSM_LOG_ENTER(p_log, __osmv_send_sa_req); + + /* + since the sm_lid might change we obtain it every send + (actually it is cached in the bind object and refreshed + every 30sec by this proc ) + */ + status = + __osmv_get_lid_and_sm_lid_by_port_guid(p_bind->p_vendor, + p_bind->port_guid, + &p_bind-> + last_lids_update_sec, + &p_bind->lid, + &p_bind->sm_lid); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osmv_send_sa_req: ERR 5509: " + "Failed to obtain the SM lid\n"); + goto Exit; + } + + /* Get a MAD wrapper for the send */ + p_madw = osm_mad_pool_get(p_bind->p_mad_pool, + p_bind->h_bind, MAD_BLOCK_SIZE, NULL); + + if (p_madw == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osmv_send_sa_req: ERR 5510: " + "Unable to acquire MAD\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* Initialize the Sent MAD: */ + + /* Initialize the MAD buffer for the send operation. */ + p_mad_hdr = osm_madw_get_mad_ptr(p_madw); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + /* Get a new transaction Id */ + cl_atomic_inc(&trans_id); + + /* Cleanup the MAD from any residue */ + memset(p_sa_mad, 0, MAD_BLOCK_SIZE); + + /* Initialize the standard MAD header. */ + ib_mad_init_new(p_mad_hdr, /* mad pointer */ + IB_MCLASS_SUBN_ADM, /* class */ + (uint8_t) 2, /* version */ + p_sa_mad_data->method, /* method */ + cl_hton64((uint64_t) trans_id), /* tid */ + p_sa_mad_data->attr_id, /* attr id */ + p_sa_mad_data->attr_mod /* attr mod */ + ); + + /* Set the query information. */ + p_sa_mad->sm_key = p_query_req->sm_key; + p_sa_mad->attr_offset = 0; + p_sa_mad->comp_mask = p_sa_mad_data->comp_mask; #ifdef DUAL_SIDED_RMPP - if( p_sa_mad->method == IB_MAD_METHOD_GETMULTI ) - p_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + if (p_sa_mad->method == IB_MAD_METHOD_GETMULTI) + p_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - if( p_sa_mad->comp_mask ) - { - memcpy( p_sa_mad->data, p_sa_mad_data->p_attr, - ib_get_attr_size(p_sa_mad_data->attr_offset)); - } - - /* - Provide the address to send to - */ - p_madw->mad_addr.dest_lid = cl_hton16(p_bind->sm_lid); - p_madw->mad_addr.addr_type.smi.source_lid = - cl_hton16(p_bind->lid); - p_madw->mad_addr.addr_type.gsi.remote_qp = CL_HTON32(1); - p_madw->resp_expected = TRUE; - p_madw->fail_msg = CL_DISP_MSGID_NONE; - - /* - Provide MAD context such that the call back will know what to do. - We have to keep the entire request structure so we know the CB. - Since we can not rely on the client to keep it around until - the response - we duplicate it and will later dispose it (in CB). - To store on the MADW we cast it into what opensm has: - p_madw->context.ni_context.node_guid - */ - p_query_req_copy = malloc(sizeof(*p_query_req_copy)); - *p_query_req_copy = *p_query_req; - p_madw->context.ni_context.node_guid = - (ib_net64_t)(long)p_query_req_copy; - - /* we can support async as well as sync calls */ - sync = ((p_query_req->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC); - - /* send the mad asynchronously */ - status = osm_vendor_send( - osm_madw_get_bind_handle( p_madw ), - p_madw, - p_madw->resp_expected ); - - /* if synchronous - wait on the event */ - if (sync) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__osmv_send_sa_req: " - "Waiting for async event\n" ); - cl_event_wait_on( &p_bind->sync_event, EVENT_NO_TIMEOUT, FALSE ); - cl_event_reset(&p_bind->sync_event); - status = p_madw->status; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return status; + if (p_sa_mad->comp_mask) { + memcpy(p_sa_mad->data, p_sa_mad_data->p_attr, + ib_get_attr_size(p_sa_mad_data->attr_offset)); + } + + /* + Provide the address to send to + */ + p_madw->mad_addr.dest_lid = cl_hton16(p_bind->sm_lid); + p_madw->mad_addr.addr_type.smi.source_lid = cl_hton16(p_bind->lid); + p_madw->mad_addr.addr_type.gsi.remote_qp = CL_HTON32(1); + p_madw->resp_expected = TRUE; + p_madw->fail_msg = CL_DISP_MSGID_NONE; + + /* + Provide MAD context such that the call back will know what to do. + We have to keep the entire request structure so we know the CB. + Since we can not rely on the client to keep it around until + the response - we duplicate it and will later dispose it (in CB). + To store on the MADW we cast it into what opensm has: + p_madw->context.ni_context.node_guid + */ + p_query_req_copy = malloc(sizeof(*p_query_req_copy)); + *p_query_req_copy = *p_query_req; + p_madw->context.ni_context.node_guid = + (ib_net64_t) (long)p_query_req_copy; + + /* we can support async as well as sync calls */ + sync = ((p_query_req->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC); + + /* send the mad asynchronously */ + status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), + p_madw, p_madw->resp_expected); + + /* if synchronous - wait on the event */ + if (sync) { + osm_log(p_log, OSM_LOG_DEBUG, + "__osmv_send_sa_req: " "Waiting for async event\n"); + cl_event_wait_on(&p_bind->sync_event, EVENT_NO_TIMEOUT, FALSE); + cl_event_reset(&p_bind->sync_event); + status = p_madw->status; + } + + Exit: + OSM_LOG_EXIT(p_log); + return status; } /***************************************************************************** @@ -620,280 +575,292 @@ __osmv_send_sa_req( * Query the SA based on the user's request. */ ib_api_status_t -osmv_query_sa( - IN osm_bind_handle_t h_bind, - IN const osmv_query_req_t * const p_query_req - ) +osmv_query_sa(IN osm_bind_handle_t h_bind, + IN const osmv_query_req_t * const p_query_req) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)h_bind; - osmv_sa_mad_data_t sa_mad_data; - osmv_user_query_t *p_user_query; - ib_service_record_t svc_rec; - ib_node_record_t node_rec; - ib_portinfo_record_t port_info; - ib_path_rec_t path_rec; + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) h_bind; + osmv_sa_mad_data_t sa_mad_data; + osmv_user_query_t *p_user_query; + ib_service_record_t svc_rec; + ib_node_record_t node_rec; + ib_portinfo_record_t port_info; + ib_path_rec_t path_rec; #ifdef DUAL_SIDED_RMPP - ib_multipath_rec_t multipath_rec; - osmv_multipath_req_t *p_mpr_req; - int i, j; + ib_multipath_rec_t multipath_rec; + osmv_multipath_req_t *p_mpr_req; + int i, j; #endif - ib_class_port_info_t class_port_info; - osm_log_t *p_log = p_bind->p_log; - ib_api_status_t status; - - OSM_LOG_ENTER( p_log, osmv_query_sa ); - - /* Set the request information. */ - sa_mad_data.method = IB_MAD_METHOD_GETTABLE; - sa_mad_data.attr_mod = 0; - - /* Set the MAD attributes and component mask correctly. */ - switch ( p_query_req->query_type ) - { - - case OSMV_QUERY_USER_DEFINED: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "USER_DEFINED\n" ); - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - if (p_user_query->method) - sa_mad_data.method = p_user_query->method; - sa_mad_data.attr_offset = p_user_query->attr_offset; - sa_mad_data.attr_id = p_user_query->attr_id; - sa_mad_data.attr_mod = p_user_query->attr_mod; - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_ALL_SVC_RECS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.comp_mask = 0; - sa_mad_data.p_attr = &svc_rec; - break; - - case OSMV_QUERY_SVC_REC_BY_NAME: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n" ); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.p_attr = &svc_rec; - memcpy( svc_rec.service_name, p_query_req->p_query_input, - sizeof( ib_svc_name_t ) ); - break; - - case OSMV_QUERY_SVC_REC_BY_ID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_ID\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.comp_mask = IB_SR_COMPMASK_SID; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.p_attr = &svc_rec; - svc_rec.service_id = *( ib_net64_t * ) ( p_query_req->p_query_input ); - break; - - case OSMV_QUERY_CLASS_PORT_INFO: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","CLASS_PORT_INFO\n" ); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_CLASS_PORT_INFO; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_class_port_info_t ) ); - sa_mad_data.comp_mask = 0; - sa_mad_data.p_attr = &class_port_info; - break; - - case OSMV_QUERY_NODE_REC_BY_NODE_GUID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","NODE_REC_BY_NODE_GUID\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_NODE_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_node_record_t ) ); - sa_mad_data.comp_mask = IB_NR_COMPMASK_NODEGUID; - sa_mad_data.p_attr = &node_rec; - node_rec.node_info.node_guid = - *( ib_net64_t * ) ( p_query_req->p_query_input ); - break; - - case OSMV_QUERY_PORT_REC_BY_LID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PORT_REC_BY_LID\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_portinfo_record_t ) ); - sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID; - sa_mad_data.p_attr = &port_info; - port_info.lid = *( ib_net16_t * ) ( p_query_req->p_query_input ); - break; - - case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PORT_REC_BY_LID_AND_NUM\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_portinfo_record_t ) ); - sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID | IB_PIR_COMPMASK_PORTNUM; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_VLARB_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_vl_arb_table_record_t ) ); - sa_mad_data.comp_mask = IB_VLA_COMPMASK_LID | IB_VLA_COMPMASK_OUT_PORT | IB_VLA_COMPMASK_BLOCK; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_SLVL_BY_LID_AND_PORTS: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_SLVL_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_slvl_table_record_t ) ); - sa_mad_data.comp_mask = IB_SLVL_COMPMASK_LID | IB_SLVL_COMPMASK_OUT_PORT | IB_SLVL_COMPMASK_IN_PORT; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_PORT_GUIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID ); - sa_mad_data.p_attr = &path_rec; - ib_gid_set_default( &path_rec.dgid, - ( ( osmv_guid_pair_t * ) ( p_query_req-> - p_query_input ) )-> - dest_guid ); - ib_gid_set_default( &path_rec.sgid, - ( ( osmv_guid_pair_t * ) ( p_query_req-> - p_query_input ) )-> - src_guid ); - break; - - case OSMV_QUERY_PATH_REC_BY_GIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_GIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID ); - sa_mad_data.p_attr = &path_rec; - memcpy( &path_rec.dgid, - &( ( osmv_gid_pair_t * ) ( p_query_req->p_query_input ) )-> - dest_gid, sizeof( ib_gid_t ) ); - memcpy( &path_rec.sgid, - &( ( osmv_gid_pair_t * ) ( p_query_req->p_query_input ) )-> - src_gid, sizeof( ib_gid_t ) ); - break; - - case OSMV_QUERY_PATH_REC_BY_LIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_LIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DLID | IB_PR_COMPMASK_SLID ); - sa_mad_data.p_attr = &path_rec; - path_rec.dlid = - ( ( osmv_lid_pair_t * ) ( p_query_req->p_query_input ) )->dest_lid; - path_rec.slid = - ( ( osmv_lid_pair_t * ) ( p_query_req->p_query_input ) )->src_lid; - break; - - case OSMV_QUERY_UD_MULTICAST_SET: - sa_mad_data.method = IB_MAD_METHOD_SET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_UD_MULTICAST_SET\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_member_rec_t ) ); - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_UD_MULTICAST_DELETE: - sa_mad_data.method = IB_MAD_METHOD_DELETE; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_UD_MULTICAST_DELETE\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_member_rec_t ) ); - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; + ib_class_port_info_t class_port_info; + osm_log_t *p_log = p_bind->p_log; + ib_api_status_t status; + + OSM_LOG_ENTER(p_log, osmv_query_sa); + + /* Set the request information. */ + sa_mad_data.method = IB_MAD_METHOD_GETTABLE; + sa_mad_data.attr_mod = 0; + + /* Set the MAD attributes and component mask correctly. */ + switch (p_query_req->query_type) { + + case OSMV_QUERY_USER_DEFINED: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "USER_DEFINED\n"); + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + if (p_user_query->method) + sa_mad_data.method = p_user_query->method; + sa_mad_data.attr_offset = p_user_query->attr_offset; + sa_mad_data.attr_id = p_user_query->attr_id; + sa_mad_data.attr_mod = p_user_query->attr_mod; + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_ALL_SVC_RECS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.comp_mask = 0; + sa_mad_data.p_attr = &svc_rec; + break; + + case OSMV_QUERY_SVC_REC_BY_NAME: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.p_attr = &svc_rec; + memcpy(svc_rec.service_name, p_query_req->p_query_input, + sizeof(ib_svc_name_t)); + break; + + case OSMV_QUERY_SVC_REC_BY_ID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_ID\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.comp_mask = IB_SR_COMPMASK_SID; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.p_attr = &svc_rec; + svc_rec.service_id = + *(ib_net64_t *) (p_query_req->p_query_input); + break; + + case OSMV_QUERY_CLASS_PORT_INFO: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "CLASS_PORT_INFO\n"); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_CLASS_PORT_INFO; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_class_port_info_t)); + sa_mad_data.comp_mask = 0; + sa_mad_data.p_attr = &class_port_info; + break; + + case OSMV_QUERY_NODE_REC_BY_NODE_GUID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "NODE_REC_BY_NODE_GUID\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_NODE_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_node_record_t)); + sa_mad_data.comp_mask = IB_NR_COMPMASK_NODEGUID; + sa_mad_data.p_attr = &node_rec; + node_rec.node_info.node_guid = + *(ib_net64_t *) (p_query_req->p_query_input); + break; + + case OSMV_QUERY_PORT_REC_BY_LID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PORT_REC_BY_LID\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_portinfo_record_t)); + sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID; + sa_mad_data.p_attr = &port_info; + port_info.lid = *(ib_net16_t *) (p_query_req->p_query_input); + break; + + case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "PORT_REC_BY_LID_AND_NUM\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_portinfo_record_t)); + sa_mad_data.comp_mask = + IB_PIR_COMPMASK_LID | IB_PIR_COMPMASK_PORTNUM; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_VLARB_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_vl_arb_table_record_t)); + sa_mad_data.comp_mask = + IB_VLA_COMPMASK_LID | IB_VLA_COMPMASK_OUT_PORT | + IB_VLA_COMPMASK_BLOCK; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_SLVL_BY_LID_AND_PORTS: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_SLVL_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_slvl_table_record_t)); + sa_mad_data.comp_mask = + IB_SLVL_COMPMASK_LID | IB_SLVL_COMPMASK_OUT_PORT | + IB_SLVL_COMPMASK_IN_PORT; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_PORT_GUIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID); + sa_mad_data.p_attr = &path_rec; + ib_gid_set_default(&path_rec.dgid, + ((osmv_guid_pair_t *) (p_query_req-> + p_query_input))-> + dest_guid); + ib_gid_set_default(&path_rec.sgid, + ((osmv_guid_pair_t *) (p_query_req-> + p_query_input))-> + src_guid); + break; + + case OSMV_QUERY_PATH_REC_BY_GIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_GIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID); + sa_mad_data.p_attr = &path_rec; + memcpy(&path_rec.dgid, + &((osmv_gid_pair_t *) (p_query_req->p_query_input))-> + dest_gid, sizeof(ib_gid_t)); + memcpy(&path_rec.sgid, + &((osmv_gid_pair_t *) (p_query_req->p_query_input))-> + src_gid, sizeof(ib_gid_t)); + break; + + case OSMV_QUERY_PATH_REC_BY_LIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_LIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DLID | IB_PR_COMPMASK_SLID); + sa_mad_data.p_attr = &path_rec; + path_rec.dlid = + ((osmv_lid_pair_t *) (p_query_req->p_query_input))-> + dest_lid; + path_rec.slid = + ((osmv_lid_pair_t *) (p_query_req->p_query_input))->src_lid; + break; + + case OSMV_QUERY_UD_MULTICAST_SET: + sa_mad_data.method = IB_MAD_METHOD_SET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_UD_MULTICAST_SET\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_UD_MULTICAST_DELETE: + sa_mad_data.method = IB_MAD_METHOD_DELETE; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_UD_MULTICAST_DELETE\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; #ifdef DUAL_SIDED_RMPP - case OSMV_QUERY_MULTIPATH_REC: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","MULTIPATH_REC\n" ); - /* Validate sgid/dgid counts against SA client limit */ - p_mpr_req = ( osmv_multipath_req_t * ) p_query_req->p_query_input; - if ( p_mpr_req->sgid_count + p_mpr_req->dgid_count > IB_MULTIPATH_MAX_GIDS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osmv_query_sa DBG:001 MULTIPATH_REC " - "SGID count %d DGID count %d max count %d\n", - p_mpr_req->sgid_count, p_mpr_req->dgid_count, - IB_MULTIPATH_MAX_GIDS ); - CL_ASSERT( 0 ); - return IB_ERROR; - } - memset(&multipath_rec, 0, sizeof(ib_multipath_rec_t )); - sa_mad_data.method = IB_MAD_METHOD_GETMULTI; - sa_mad_data.attr_id = IB_MAD_ATTR_MULTIPATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_multipath_rec_t ) ); - sa_mad_data.p_attr = &multipath_rec; - sa_mad_data.comp_mask = p_mpr_req->comp_mask; - multipath_rec.num_path = p_mpr_req->num_path; - if ( p_mpr_req->reversible ) - multipath_rec.num_path |= 0x80; - else - multipath_rec.num_path &= ~0x80; - multipath_rec.pkey = p_mpr_req->pkey; - multipath_rec.sl = p_mpr_req->sl; - multipath_rec.independence = p_mpr_req->independence; - multipath_rec.sgid_count = p_mpr_req->sgid_count; - multipath_rec.dgid_count = p_mpr_req->dgid_count; - j = 0; - for (i = 0; i < p_mpr_req->sgid_count; i++, j++) - multipath_rec.gids[j] = p_mpr_req->gids[j]; - for (i = 0; i < p_mpr_req->dgid_count; i++, j++) - multipath_rec.gids[j] = p_mpr_req->gids[j]; - break; + case OSMV_QUERY_MULTIPATH_REC: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "MULTIPATH_REC\n"); + /* Validate sgid/dgid counts against SA client limit */ + p_mpr_req = (osmv_multipath_req_t *) p_query_req->p_query_input; + if (p_mpr_req->sgid_count + p_mpr_req->dgid_count > + IB_MULTIPATH_MAX_GIDS) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_query_sa DBG:001 MULTIPATH_REC " + "SGID count %d DGID count %d max count %d\n", + p_mpr_req->sgid_count, p_mpr_req->dgid_count, + IB_MULTIPATH_MAX_GIDS); + CL_ASSERT(0); + return IB_ERROR; + } + memset(&multipath_rec, 0, sizeof(ib_multipath_rec_t)); + sa_mad_data.method = IB_MAD_METHOD_GETMULTI; + sa_mad_data.attr_id = IB_MAD_ATTR_MULTIPATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_multipath_rec_t)); + sa_mad_data.p_attr = &multipath_rec; + sa_mad_data.comp_mask = p_mpr_req->comp_mask; + multipath_rec.num_path = p_mpr_req->num_path; + if (p_mpr_req->reversible) + multipath_rec.num_path |= 0x80; + else + multipath_rec.num_path &= ~0x80; + multipath_rec.pkey = p_mpr_req->pkey; + multipath_rec.sl = p_mpr_req->sl; + multipath_rec.independence = p_mpr_req->independence; + multipath_rec.sgid_count = p_mpr_req->sgid_count; + multipath_rec.dgid_count = p_mpr_req->dgid_count; + j = 0; + for (i = 0; i < p_mpr_req->sgid_count; i++, j++) + multipath_rec.gids[j] = p_mpr_req->gids[j]; + for (i = 0; i < p_mpr_req->dgid_count; i++, j++) + multipath_rec.gids[j] = p_mpr_req->gids[j]; + break; #endif - default: - osm_log( p_log, OSM_LOG_ERROR, - "osmv_query_sa DBG:001 %s", "UNKNOWN\n" ); - CL_ASSERT( 0 ); - return IB_ERROR; - } + default: + osm_log(p_log, OSM_LOG_ERROR, + "osmv_query_sa DBG:001 %s", "UNKNOWN\n"); + CL_ASSERT(0); + return IB_ERROR; + } - status = __osmv_send_sa_req( h_bind, &sa_mad_data, p_query_req ); + status = __osmv_send_sa_req(h_bind, &sa_mad_data, p_query_req); - OSM_LOG_EXIT( p_log ); - return status; + OSM_LOG_EXIT(p_log); + return status; } diff --git a/opensm/libvendor/osm_vendor_mlx.c b/opensm/libvendor/osm_vendor_mlx.c index ea2d9b3..e3eee2b 100644 --- a/opensm/libvendor/osm_vendor_mlx.c +++ b/opensm/libvendor/osm_vendor_mlx.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -50,14 +50,12 @@ * FORWARD REFERENCES */ static ib_api_status_t -__osmv_get_send_txn(IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t is_rmpp, - IN boolean_t resp_expected, - OUT osmv_txn_ctx_t **pp_txn); +__osmv_get_send_txn(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN boolean_t is_rmpp, + IN boolean_t resp_expected, OUT osmv_txn_ctx_t ** pp_txn); -static void -__osm_vendor_internal_unbind(osm_bind_handle_t h_bind); +static void __osm_vendor_internal_unbind(osm_bind_handle_t h_bind); /* * NAME osm_vendor_new @@ -65,38 +63,32 @@ __osm_vendor_internal_unbind(osm_bind_handle_t h_bind); * DESCRIPTION Create and Initialize the osm_vendor_t Object */ -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - ib_api_status_t status; - osm_vendor_t *p_vend; - - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - CL_ASSERT( p_log ); - - p_vend = malloc( sizeof( *p_vend ) ); - if ( p_vend != NULL ) - { - memset( p_vend, 0, sizeof( *p_vend ) ); - - status = osm_vendor_init( p_vend, p_log, timeout ); - if ( status != IB_SUCCESS ) - { - osm_vendor_delete( &p_vend ); - } - } - else - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 7301: " - "Fail to allocate vendor object.\n" ); - } - - OSM_LOG_EXIT( p_log ); - return( p_vend ); + ib_api_status_t status; + osm_vendor_t *p_vend; + + OSM_LOG_ENTER(p_log, osm_vendor_new); + + CL_ASSERT(p_log); + + p_vend = malloc(sizeof(*p_vend)); + if (p_vend != NULL) { + memset(p_vend, 0, sizeof(*p_vend)); + + status = osm_vendor_init(p_vend, p_log, timeout); + if (status != IB_SUCCESS) { + osm_vendor_delete(&p_vend); + } + } else { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 7301: " + "Fail to allocate vendor object.\n"); + } + + OSM_LOG_EXIT(p_log); + return (p_vend); } /* @@ -105,47 +97,46 @@ osm_vendor_new( * DESCRIPTION Delete all the binds behind the vendor + free the vendor object */ -void -osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - cl_list_item_t *p_item; - cl_list_obj_t* p_obj; - osm_bind_handle_t bind_h; - osm_log_t* p_log; + cl_list_item_t *p_item; + cl_list_obj_t *p_obj; + osm_bind_handle_t bind_h; + osm_log_t *p_log; - OSM_LOG_ENTER( (*pp_vend)->p_log, osm_vendor_delete ); - p_log = (*pp_vend)->p_log; + OSM_LOG_ENTER((*pp_vend)->p_log, osm_vendor_delete); + p_log = (*pp_vend)->p_log; - /* go over the bind handles , unbind them and remove from list */ - p_item = cl_qlist_remove_head(&((*pp_vend)->bind_handles)); - while ( p_item != cl_qlist_end(&((*pp_vend)->bind_handles))) { + /* go over the bind handles , unbind them and remove from list */ + p_item = cl_qlist_remove_head(&((*pp_vend)->bind_handles)); + while (p_item != cl_qlist_end(&((*pp_vend)->bind_handles))) { - p_obj = PARENT_STRUCT(p_item,cl_list_obj_t,list_item); - bind_h = (osm_bind_handle_t*)cl_qlist_obj(p_obj); - osm_log(p_log,OSM_LOG_DEBUG, - "osm_vendor_delete: unbinding bind_h:%p \n",bind_h); + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + bind_h = (osm_bind_handle_t *) cl_qlist_obj(p_obj); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_vendor_delete: unbinding bind_h:%p \n", bind_h); - __osm_vendor_internal_unbind(bind_h); + __osm_vendor_internal_unbind(bind_h); - free(p_obj); - /*removing from list */ - p_item = cl_qlist_remove_head(&((*pp_vend)->bind_handles)); - } + free(p_obj); + /*removing from list */ + p_item = cl_qlist_remove_head(&((*pp_vend)->bind_handles)); + } - if (NULL != ((*pp_vend)->p_transport_info)) - { - free((*pp_vend)->p_transport_info); - (*pp_vend)->p_transport_info = NULL; - } + if (NULL != ((*pp_vend)->p_transport_info)) { + free((*pp_vend)->p_transport_info); + (*pp_vend)->p_transport_info = NULL; + } - /* remove the packet randomizer object */ - if ( (*pp_vend)->run_randomizer == TRUE ) - osm_pkt_randomizer_destroy( &((*pp_vend)->p_pkt_randomizer), p_log ); + /* remove the packet randomizer object */ + if ((*pp_vend)->run_randomizer == TRUE) + osm_pkt_randomizer_destroy(&((*pp_vend)->p_pkt_randomizer), + p_log); - free( *pp_vend ); - *pp_vend = NULL; + free(*pp_vend); + *pp_vend = NULL; - OSM_LOG_EXIT( p_log ); + OSM_LOG_EXIT(p_log); } /* @@ -155,40 +146,38 @@ osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) */ ib_api_status_t -osm_vendor_init( IN osm_vendor_t * const p_vend, - IN osm_log_t * const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_vendor_init ); - - p_vend->p_transport_info = NULL; - p_vend->p_log = p_log; - p_vend->resp_timeout = timeout; - p_vend->ttime_timeout = timeout * OSMV_TXN_TIMEOUT_FACTOR; - - cl_qlist_init(&p_vend->bind_handles); - - /* update the run_randomizer flag */ - if ( getenv("OSM_PKT_DROP_RATE") != NULL && atol( getenv("OSM_PKT_DROP_RATE") ) != 0 ) - { - /* if the OSM_PKT_DROP_RATE global variable is defined to a non-zero value - - then the randomizer should be called. - Need to create the packet randomizer object */ - p_vend->run_randomizer = TRUE; - status = osm_pkt_randomizer_init( &(p_vend->p_pkt_randomizer), p_log ); - if ( status != IB_SUCCESS ) - return status; - } - else - { - p_vend->run_randomizer = FALSE; - p_vend->p_pkt_randomizer = NULL; - } - - OSM_LOG_EXIT( p_log ); - return( IB_SUCCESS ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_vendor_init); + + p_vend->p_transport_info = NULL; + p_vend->p_log = p_log; + p_vend->resp_timeout = timeout; + p_vend->ttime_timeout = timeout * OSMV_TXN_TIMEOUT_FACTOR; + + cl_qlist_init(&p_vend->bind_handles); + + /* update the run_randomizer flag */ + if (getenv("OSM_PKT_DROP_RATE") != NULL + && atol(getenv("OSM_PKT_DROP_RATE")) != 0) { + /* if the OSM_PKT_DROP_RATE global variable is defined to a non-zero value - + then the randomizer should be called. + Need to create the packet randomizer object */ + p_vend->run_randomizer = TRUE; + status = + osm_pkt_randomizer_init(&(p_vend->p_pkt_randomizer), p_log); + if (status != IB_SUCCESS) + return status; + } else { + p_vend->run_randomizer = FALSE; + p_vend->p_pkt_randomizer = NULL; + } + + OSM_LOG_EXIT(p_log); + return (IB_SUCCESS); } /* @@ -198,134 +187,126 @@ osm_vendor_init( IN osm_vendor_t * const p_vend, */ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_bind_info, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void* context) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_bind_info, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { - osmv_bind_obj_t *p_bo; - ib_api_status_t status; - char hca_id[32]; - cl_status_t cl_st; - cl_list_obj_t* p_obj; - uint8_t hca_index; - - if (NULL == p_vend || NULL == p_bind_info || NULL == p_mad_pool - || NULL == mad_recv_callback || NULL == send_err_callback ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7302: " - "NULL parameter passed in: p_vend=%p p_bind_info=%p p_mad_pool=%p recv_cb=%p send_err_cb=%p\n", - p_vend, p_bind_info, p_mad_pool, mad_recv_callback, send_err_callback); - - return OSM_BIND_INVALID_HANDLE; - } - - p_bo = malloc(sizeof(osmv_bind_obj_t)); - if (NULL == p_bo) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7303: could not allocate the bind object\n"); - return OSM_BIND_INVALID_HANDLE; - } - - memset(p_bo, 0, sizeof(osmv_bind_obj_t)); - p_bo->p_vendor = p_vend; - p_bo->recv_cb = mad_recv_callback; - p_bo->send_err_cb = send_err_callback; - p_bo->cb_context = context; - p_bo->p_osm_pool = p_mad_pool; - - /* obtain the hca name and port num from the guid */ - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", - cl_ntoh64(p_bind_info->port_guid) ); - - status = osm_vendor_get_guid_ca_and_port( - p_bo->p_vendor, - p_bind_info->port_guid, - &(p_bo->hca_hndl), - hca_id, - &hca_index, - &(p_bo->port_num)); - if (status != IB_SUCCESS) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7304: " - "Fail to find port number of port guid:0x%016"PRIx64"\n", - p_bind_info->port_guid - ); - free(p_bo); - return OSM_BIND_INVALID_HANDLE; - } - - /* Initialize the magic_ptr to the pointer of the p_bo info. - This will be used to signal when the object is being destroyed, so no - real action will be done then. */ - p_bo->magic_ptr = p_bo; - - p_bo->is_closing = FALSE; - - cl_spinlock_construct(&(p_bo->lock)); - cl_st = cl_spinlock_init(&(p_bo->lock)); - if (cl_st != CL_SUCCESS) - { - osm_log(p_bo->p_vendor->p_log,OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7305: " - "could not initialize the spinlock ...\n"); - free(p_bo); - return OSM_BIND_INVALID_HANDLE; - } - - osm_log(p_bo->p_vendor->p_log,OSM_LOG_DEBUG, - "osm_vendor_bind: osmv_txnmgr_init ... \n"); - if (osmv_txnmgr_init(&p_bo->txn_mgr, p_vend->p_log, &(p_bo->lock)) != IB_SUCCESS) - { - osm_log(p_bo->p_vendor->p_log,OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7306: " - "osmv_txnmgr_init failed \n"); - cl_spinlock_destroy(&p_bo->lock); - free(p_bo); - return OSM_BIND_INVALID_HANDLE; - } - - /* Do the real job! (Transport-dependent) */ - if (IB_SUCCESS != osmv_transport_init(p_bind_info, hca_id, hca_index, p_bo)) - { - osm_log(p_bo->p_vendor->p_log,OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7307: " - "osmv_transport_init failed \n"); - osmv_txnmgr_done((osm_bind_handle_t) p_bo); - cl_spinlock_destroy(&p_bo->lock); - free(p_bo); - return OSM_BIND_INVALID_HANDLE; - } - - /* insert bind handle into db */ - p_obj = malloc(sizeof(cl_list_obj_t)); - if (NULL == p_obj) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7308: " - "osm_vendor_bind: could not allocate the list object\n"); - - osmv_transport_done(p_bo->p_transp_mgr); - osmv_txnmgr_done((osm_bind_handle_t) p_bo); - cl_spinlock_destroy(&p_bo->lock); - free(p_bo); - return OSM_BIND_INVALID_HANDLE; - } - memset(p_obj, 0, sizeof(cl_list_obj_t)); - cl_qlist_set_obj(p_obj, p_bo); - - cl_qlist_insert_head(&p_vend->bind_handles,&p_obj->list_item); - - return(osm_bind_handle_t)p_bo; + osmv_bind_obj_t *p_bo; + ib_api_status_t status; + char hca_id[32]; + cl_status_t cl_st; + cl_list_obj_t *p_obj; + uint8_t hca_index; + + if (NULL == p_vend || NULL == p_bind_info || NULL == p_mad_pool + || NULL == mad_recv_callback || NULL == send_err_callback) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7302: " + "NULL parameter passed in: p_vend=%p p_bind_info=%p p_mad_pool=%p recv_cb=%p send_err_cb=%p\n", + p_vend, p_bind_info, p_mad_pool, mad_recv_callback, + send_err_callback); + + return OSM_BIND_INVALID_HANDLE; + } + + p_bo = malloc(sizeof(osmv_bind_obj_t)); + if (NULL == p_bo) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7303: could not allocate the bind object\n"); + return OSM_BIND_INVALID_HANDLE; + } + + memset(p_bo, 0, sizeof(osmv_bind_obj_t)); + p_bo->p_vendor = p_vend; + p_bo->recv_cb = mad_recv_callback; + p_bo->send_err_cb = send_err_callback; + p_bo->cb_context = context; + p_bo->p_osm_pool = p_mad_pool; + + /* obtain the hca name and port num from the guid */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", + cl_ntoh64(p_bind_info->port_guid)); + + status = osm_vendor_get_guid_ca_and_port(p_bo->p_vendor, + p_bind_info->port_guid, + &(p_bo->hca_hndl), + hca_id, + &hca_index, &(p_bo->port_num)); + if (status != IB_SUCCESS) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7304: " + "Fail to find port number of port guid:0x%016" PRIx64 + "\n", p_bind_info->port_guid); + free(p_bo); + return OSM_BIND_INVALID_HANDLE; + } + + /* Initialize the magic_ptr to the pointer of the p_bo info. + This will be used to signal when the object is being destroyed, so no + real action will be done then. */ + p_bo->magic_ptr = p_bo; + + p_bo->is_closing = FALSE; + + cl_spinlock_construct(&(p_bo->lock)); + cl_st = cl_spinlock_init(&(p_bo->lock)); + if (cl_st != CL_SUCCESS) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7305: " + "could not initialize the spinlock ...\n"); + free(p_bo); + return OSM_BIND_INVALID_HANDLE; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: osmv_txnmgr_init ... \n"); + if (osmv_txnmgr_init(&p_bo->txn_mgr, p_vend->p_log, &(p_bo->lock)) != + IB_SUCCESS) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7306: " + "osmv_txnmgr_init failed \n"); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); + return OSM_BIND_INVALID_HANDLE; + } + + /* Do the real job! (Transport-dependent) */ + if (IB_SUCCESS != + osmv_transport_init(p_bind_info, hca_id, hca_index, p_bo)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7307: " + "osmv_transport_init failed \n"); + osmv_txnmgr_done((osm_bind_handle_t) p_bo); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); + return OSM_BIND_INVALID_HANDLE; + } + + /* insert bind handle into db */ + p_obj = malloc(sizeof(cl_list_obj_t)); + if (NULL == p_obj) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7308: " + "osm_vendor_bind: could not allocate the list object\n"); + + osmv_transport_done(p_bo->p_transp_mgr); + osmv_txnmgr_done((osm_bind_handle_t) p_bo); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); + return OSM_BIND_INVALID_HANDLE; + } + memset(p_obj, 0, sizeof(cl_list_obj_t)); + cl_qlist_set_obj(p_obj, p_bo); + + cl_qlist_insert_head(&p_vend->bind_handles, &p_obj->list_item); + + return (osm_bind_handle_t) p_bo; } /* @@ -334,40 +315,39 @@ osm_vendor_bind( * DESCRIPTION Destroy the bind object and remove it from the vendor's list */ -void -osm_vendor_unbind(IN osm_bind_handle_t h_bind) +void osm_vendor_unbind(IN osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osm_log_t* p_log = p_bo->p_vendor->p_log; - cl_list_obj_t* p_obj = NULL; - cl_list_item_t* p_item,*p_item_tmp; - cl_qlist_t *const p_bh_list = (cl_qlist_t *const)&p_bo->p_vendor->bind_handles; - - OSM_LOG_ENTER(p_log, osm_vendor_unbind ); - - /* go over all the items in the list and remove the specific item */ - p_item = cl_qlist_head(p_bh_list); - while (p_item != cl_qlist_end(p_bh_list)) { - p_obj = PARENT_STRUCT(p_item,cl_list_obj_t,list_item); - if (cl_qlist_obj(p_obj) == h_bind) - { - break; - } - p_item_tmp = cl_qlist_next(p_item); - p_item = p_item_tmp; - } - - CL_ASSERT(p_item != cl_qlist_end(p_bh_list)); - - cl_qlist_remove_item(p_bh_list,p_item); - if (p_obj) free(p_obj); - - if (h_bind != 0) - { - __osm_vendor_internal_unbind(h_bind); - } - - OSM_LOG_EXIT(p_log); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_log_t *p_log = p_bo->p_vendor->p_log; + cl_list_obj_t *p_obj = NULL; + cl_list_item_t *p_item, *p_item_tmp; + cl_qlist_t *const p_bh_list = + (cl_qlist_t * const)&p_bo->p_vendor->bind_handles; + + OSM_LOG_ENTER(p_log, osm_vendor_unbind); + + /* go over all the items in the list and remove the specific item */ + p_item = cl_qlist_head(p_bh_list); + while (p_item != cl_qlist_end(p_bh_list)) { + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + if (cl_qlist_obj(p_obj) == h_bind) { + break; + } + p_item_tmp = cl_qlist_next(p_item); + p_item = p_item_tmp; + } + + CL_ASSERT(p_item != cl_qlist_end(p_bh_list)); + + cl_qlist_remove_item(p_bh_list, p_item); + if (p_obj) + free(p_obj); + + if (h_bind != 0) { + __osm_vendor_internal_unbind(h_bind); + } + + OSM_LOG_EXIT(p_log); } /* @@ -376,53 +356,47 @@ osm_vendor_unbind(IN osm_bind_handle_t h_bind) * DESCRIPTION Allocate the space for a new MAD */ -ib_mad_t * -osm_vendor_get( IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t * const p_vw ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { - ib_mad_t *p_mad; - osmv_bind_obj_t *p_bo = ( osmv_bind_obj_t * ) h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - uint32_t act_mad_size; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get ); - - CL_ASSERT( p_vw ); - - if (mad_size < MAD_BLOCK_SIZE) - { - /* Stupid, but the applications want that! */ - act_mad_size = MAD_BLOCK_SIZE; - } - else - { - act_mad_size = mad_size; - } - - /* allocate it */ - p_mad = ( ib_mad_t * ) malloc( act_mad_size ); - if ( p_mad == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get: ERR 7309: " - "Error Obtaining MAD buffer.\n" ); - goto Exit; - } - - memset( p_mad, 0, act_mad_size ); - - if ( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get: " - "Allocated MAD %p, size = %u.\n", p_mad, act_mad_size ); - } - p_vw->p_mad = p_mad; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return( p_mad ); + ib_mad_t *p_mad; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + uint32_t act_mad_size; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); + + CL_ASSERT(p_vw); + + if (mad_size < MAD_BLOCK_SIZE) { + /* Stupid, but the applications want that! */ + act_mad_size = MAD_BLOCK_SIZE; + } else { + act_mad_size = mad_size; + } + + /* allocate it */ + p_mad = (ib_mad_t *) malloc(act_mad_size); + if (p_mad == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get: ERR 7309: " + "Error Obtaining MAD buffer.\n"); + goto Exit; + } + + memset(p_mad, 0, act_mad_size); + + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get: " + "Allocated MAD %p, size = %u.\n", p_mad, act_mad_size); + } + p_vw->p_mad = p_mad; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (p_mad); } /* @@ -438,126 +412,112 @@ osm_vendor_get( IN osm_bind_handle_t h_bind, */ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t const resp_expected) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { - ib_api_status_t ret = IB_SUCCESS; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - boolean_t is_rmpp=FALSE, is_rmpp_ds=FALSE; - osmv_txn_ctx_t *p_txn = NULL; - ib_mad_t *p_mad; - osm_log_t *p_log = p_bo->p_vendor->p_log; - osm_mad_pool_t *p_mad_pool = p_bo->p_osm_pool; - OSM_LOG_ENTER(p_log, osm_vendor_send); - - if (NULL == h_bind || NULL == p_madw || - NULL == (p_mad = osm_madw_get_mad_ptr(p_madw)) || - NULL == osm_madw_get_mad_addr_ptr(p_madw)) { - - return IB_INVALID_PARAMETER; - } - - is_rmpp = (p_madw->mad_size > MAD_BLOCK_SIZE || osmv_mad_is_rmpp(p_mad)); - /* is this rmpp double sided? This means we expect a response that can be - an rmpp or not */ - is_rmpp_ds = (TRUE == is_rmpp && TRUE == resp_expected); - - /* Make our operations with the send context atomic */ - osmv_txn_lock(p_bo); - - if (TRUE == p_bo->is_closing) - { - - osm_log(p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 7310: " - "The handle %p is being unbound, cannot send.\n", h_bind); - ret = IB_INTERRUPTED; - /* When closing p_bo could be detroyed or is going to , thus could not refer to it */ - goto send_done; - } - - if (TRUE == resp_expected || TRUE == is_rmpp) - { - - /* We must run under a transaction framework. - * Get the transaction object (old or new) */ - ret = __osmv_get_send_txn(h_bind, p_madw, is_rmpp, - resp_expected, &p_txn); - if (IB_SUCCESS != ret) - { - goto send_done; - } - } - - if (TRUE == is_rmpp) - { - /* Do the job - RMPP! - * The call returns as all the packets are ACK'ed/upon error - * The txn lock will be released each time the function sleeps - * and re-acquired when it wakes up - */ - ret = osmv_rmpp_send_madw(h_bind, p_madw, p_txn, is_rmpp_ds); - } - else { - - /* Do the job - single MAD! - * The call returns as soon as the MAD is put on the wire - */ - ret = osmv_simple_send_madw(h_bind, p_madw, p_txn, FALSE); - } - - if (IB_SUCCESS == ret) - { - - if ((TRUE == is_rmpp) && (FALSE == is_rmpp_ds)) - { - /* For double-sided sends, the txn continues to live */ - osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), - FALSE /*not in callback*/); - } - - if (FALSE == resp_expected) - { - osm_mad_pool_put(p_mad_pool, p_madw ); - } - } - else if (IB_INTERRUPTED != ret) - { - if (NULL != p_txn) - { - osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), - FALSE /*not in callback*/); - } - - osm_log(p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 7311: failed to send MADW %p\n", p_madw); - - if (TRUE == resp_expected) - { - /* Change the status on the p_madw */ - p_madw->status = ret; - /* Only the requester expects the error callback */ - p_bo->send_err_cb(p_bo->cb_context, p_madw); - } - else - { - /* put back the mad - it is useless ... */ - osm_mad_pool_put(p_mad_pool, p_madw ); - } - } - else /* the transaction was aborted due to p_bo exit */ - { - osm_mad_pool_put(p_mad_pool, p_madw ); - goto aborted; - } - send_done: - - osmv_txn_unlock(p_bo); - aborted: - OSM_LOG_EXIT(p_log); - return ret; + ib_api_status_t ret = IB_SUCCESS; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + boolean_t is_rmpp = FALSE, is_rmpp_ds = FALSE; + osmv_txn_ctx_t *p_txn = NULL; + ib_mad_t *p_mad; + osm_log_t *p_log = p_bo->p_vendor->p_log; + osm_mad_pool_t *p_mad_pool = p_bo->p_osm_pool; + OSM_LOG_ENTER(p_log, osm_vendor_send); + + if (NULL == h_bind || NULL == p_madw || + NULL == (p_mad = osm_madw_get_mad_ptr(p_madw)) || + NULL == osm_madw_get_mad_addr_ptr(p_madw)) { + + return IB_INVALID_PARAMETER; + } + + is_rmpp = (p_madw->mad_size > MAD_BLOCK_SIZE + || osmv_mad_is_rmpp(p_mad)); + /* is this rmpp double sided? This means we expect a response that can be + an rmpp or not */ + is_rmpp_ds = (TRUE == is_rmpp && TRUE == resp_expected); + + /* Make our operations with the send context atomic */ + osmv_txn_lock(p_bo); + + if (TRUE == p_bo->is_closing) { + + osm_log(p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 7310: " + "The handle %p is being unbound, cannot send.\n", + h_bind); + ret = IB_INTERRUPTED; + /* When closing p_bo could be detroyed or is going to , thus could not refer to it */ + goto send_done; + } + + if (TRUE == resp_expected || TRUE == is_rmpp) { + + /* We must run under a transaction framework. + * Get the transaction object (old or new) */ + ret = __osmv_get_send_txn(h_bind, p_madw, is_rmpp, + resp_expected, &p_txn); + if (IB_SUCCESS != ret) { + goto send_done; + } + } + + if (TRUE == is_rmpp) { + /* Do the job - RMPP! + * The call returns as all the packets are ACK'ed/upon error + * The txn lock will be released each time the function sleeps + * and re-acquired when it wakes up + */ + ret = osmv_rmpp_send_madw(h_bind, p_madw, p_txn, is_rmpp_ds); + } else { + + /* Do the job - single MAD! + * The call returns as soon as the MAD is put on the wire + */ + ret = osmv_simple_send_madw(h_bind, p_madw, p_txn, FALSE); + } + + if (IB_SUCCESS == ret) { + + if ((TRUE == is_rmpp) && (FALSE == is_rmpp_ds)) { + /* For double-sided sends, the txn continues to live */ + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), + FALSE /*not in callback */ ); + } + + if (FALSE == resp_expected) { + osm_mad_pool_put(p_mad_pool, p_madw); + } + } else if (IB_INTERRUPTED != ret) { + if (NULL != p_txn) { + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), + FALSE /*not in callback */ ); + } + + osm_log(p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 7311: failed to send MADW %p\n", + p_madw); + + if (TRUE == resp_expected) { + /* Change the status on the p_madw */ + p_madw->status = ret; + /* Only the requester expects the error callback */ + p_bo->send_err_cb(p_bo->cb_context, p_madw); + } else { + /* put back the mad - it is useless ... */ + osm_mad_pool_put(p_mad_pool, p_madw); + } + } else { /* the transaction was aborted due to p_bo exit */ + + osm_mad_pool_put(p_mad_pool, p_madw); + goto aborted; + } + send_done: + + osmv_txn_unlock(p_bo); + aborted: + OSM_LOG_EXIT(p_log); + return ret; } /* @@ -567,32 +527,29 @@ osm_vendor_send( */ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vw ) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { - osmv_bind_obj_t *p_bo = ( osmv_bind_obj_t * ) h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; - if (p_bo->is_closing != TRUE) - { - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_put ); + if (p_bo->is_closing != TRUE) { + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); - CL_ASSERT( p_vw ); - CL_ASSERT( p_vw->p_mad); + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->p_mad); - if ( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_put: " "Retiring MAD %p.\n", p_vw->p_mad ); - } + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_put: " "Retiring MAD %p.\n", + p_vw->p_mad); + } - free( p_vw->p_mad ); - p_vw->p_mad = NULL; + free(p_vw->p_mad); + p_vw->p_mad = NULL; - OSM_LOG_EXIT( p_vend->p_log ); - } + OSM_LOG_EXIT(p_vend->p_log); + } } /* @@ -603,19 +560,17 @@ osm_vendor_put( * housekeeping functions such as address vector updates. */ -ib_api_status_t -osm_vendor_local_lid_change( - IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_vendor_t const *p_vend = ( ( osmv_bind_obj_t * ) h_bind )->p_vendor; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); + osm_vendor_t const *p_vend = ((osmv_bind_obj_t *) h_bind)->p_vendor; + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_local_lid_change: " "Change of LID.\n" ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_local_lid_change: " "Change of LID.\n"); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); - return( IB_SUCCESS ); + return (IB_SUCCESS); } @@ -626,37 +581,33 @@ osm_vendor_local_lid_change( * according to the value given (TRUE or FALSE). */ #if !(defined(OSM_VENDOR_INTF_TS_NO_VAPI) || defined(OSM_VENDOR_INTF_SIM) || defined(OSM_VENDOR_INTF_TS)) -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osmv_bind_obj_t *p_bo = ( osmv_bind_obj_t * ) h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - VAPI_ret_t status; - VAPI_hca_attr_t attr_mod; - VAPI_hca_attr_mask_t attr_mask; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); - - memset( &attr_mod, 0, sizeof( attr_mod ) ); - memset( &attr_mask, 0, sizeof( attr_mask ) ); - - attr_mod.is_sm = is_sm_val; - attr_mask = HCA_ATTR_IS_SM; - - status = - VAPI_modify_hca_attr( p_bo->hca_hndl, p_bo->port_num, &attr_mod, - &attr_mask ); - if ( status != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 7312: " - "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", - is_sm_val, status ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + VAPI_ret_t status; + VAPI_hca_attr_t attr_mod; + VAPI_hca_attr_mask_t attr_mask; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); + + memset(&attr_mod, 0, sizeof(attr_mod)); + memset(&attr_mask, 0, sizeof(attr_mask)); + + attr_mod.is_sm = is_sm_val; + attr_mask = HCA_ATTR_IS_SM; + + status = + VAPI_modify_hca_attr(p_bo->hca_hndl, p_bo->port_num, &attr_mod, + &attr_mask); + if (status != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 7312: " + "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", + is_sm_val, status); + } + + OSM_LOG_EXIT(p_vend->p_log); } #endif @@ -669,54 +620,53 @@ osm_vendor_set_sm( * (2) Destroy the associated data structures */ -static void -__osm_vendor_internal_unbind(osm_bind_handle_t h_bind) +static void __osm_vendor_internal_unbind(osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osm_log_t* p_log = p_bo->p_vendor->p_log; - - OSM_LOG_ENTER(p_log,__osm_vendor_internal_unbind); - - /* "notifying" all that from now on no new sends can be done */ - p_bo->txn_mgr.p_event_wheel->closing = TRUE; - - osmv_txn_lock(p_bo); - - /* - the is_closing is set under lock we we know we only need to - check for it after obtaining the lock - */ - p_bo->is_closing = TRUE; - - /* notifying all sleeping rmpp sends to exit */ - osmv_txn_abort_rmpp_txns(h_bind); - - /* unlock the bo to allow for any residual mads to be dispatched */ - osmv_txn_unlock(p_bo); - osm_log(p_log,OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying transport mgr.. \n"); - /* wait for the receiver thread to exit */ - osmv_transport_done(h_bind); - - /* lock to avoid any collissions while we cleanup the structs */ - osmv_txn_lock(p_bo); - osm_log(p_log,OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying txn mgr.. \n"); - osmv_txnmgr_done(h_bind); - osm_log(p_log,OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying bind lock.. \n"); - osmv_txn_unlock(p_bo); - - /* - we intentionally let the p_bo and its lock leak - - as we did not implement a way to track active bind handles provided to - the client - and the client might use them - - cl_spinlock_destroy(&p_bo->lock); - free(p_bo); - */ - - OSM_LOG_EXIT(p_log); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_log_t *p_log = p_bo->p_vendor->p_log; + + OSM_LOG_ENTER(p_log, __osm_vendor_internal_unbind); + + /* "notifying" all that from now on no new sends can be done */ + p_bo->txn_mgr.p_event_wheel->closing = TRUE; + + osmv_txn_lock(p_bo); + + /* + the is_closing is set under lock we we know we only need to + check for it after obtaining the lock + */ + p_bo->is_closing = TRUE; + + /* notifying all sleeping rmpp sends to exit */ + osmv_txn_abort_rmpp_txns(h_bind); + + /* unlock the bo to allow for any residual mads to be dispatched */ + osmv_txn_unlock(p_bo); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying transport mgr.. \n"); + /* wait for the receiver thread to exit */ + osmv_transport_done(h_bind); + + /* lock to avoid any collissions while we cleanup the structs */ + osmv_txn_lock(p_bo); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying txn mgr.. \n"); + osmv_txnmgr_done(h_bind); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying bind lock.. \n"); + osmv_txn_unlock(p_bo); + + /* + we intentionally let the p_bo and its lock leak - + as we did not implement a way to track active bind handles provided to + the client - and the client might use them + + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); + */ + + OSM_LOG_EXIT(p_log); } /* @@ -727,104 +677,94 @@ __osm_vendor_internal_unbind(osm_bind_handle_t h_bind) */ static ib_api_status_t -__osmv_get_send_txn(IN osm_bind_handle_t h_bind, - IN osm_madw_t* const p_madw, - IN boolean_t is_rmpp, - IN boolean_t resp_expected, - OUT osmv_txn_ctx_t **pp_txn) +__osmv_get_send_txn(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN boolean_t is_rmpp, + IN boolean_t resp_expected, OUT osmv_txn_ctx_t ** pp_txn) { - ib_api_status_t ret; - uint64_t tid, key; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - ib_mad_t *p_mad = osm_madw_get_mad_ptr(p_madw); - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_get_send_txn); - CL_ASSERT(NULL != pp_txn); - - key = tid = cl_ntoh64(p_mad->trans_id); - if (TRUE == resp_expected) - { - /* Create a unique identifier at the requester side */ - key = osmv_txn_uniq_key(tid); - } - - /* We must run under a transaction framework */ - ret = osmv_txn_lookup(h_bind, key, pp_txn); - if (IB_NOT_FOUND == ret) - { - /* Generally, we start a new transaction */ - ret = osmv_txn_init(h_bind, tid, key, pp_txn); - if (IB_SUCCESS != ret) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7313: " - "The transaction id=0x%llX failed to init.\n", tid); - goto get_send_txn_done; - } - } - else - { - CL_ASSERT(NULL != *pp_txn); - /* The transaction context exists. - * This is legal only if I am going to return an - * (RMPP?) reply to an RMPP request sent by the other part - * (double-sided RMPP transfer) - */ - if (FALSE == is_rmpp || FALSE == osmv_txn_is_rmpp_init_by_peer(*pp_txn)) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7314: " - "The transaction id=0x%llX is not unique. Send failed.\n", tid); - - ret = IB_INVALID_SETTING; - goto get_send_txn_done; - } - - if (TRUE == resp_expected) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7315: " - "The transaction id=%llX can't expect a response. Send failed.\n", - tid); - - ret = IB_INVALID_PARAMETER; - goto get_send_txn_done; - } - } - - if (TRUE == is_rmpp) - { - ret = osmv_txn_init_rmpp_sender(h_bind, *pp_txn, p_madw); - if (IB_SUCCESS != ret) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7316: " - "The transaction id=%llX failed to init the rmpp mad. Send failed.\n", - tid); - osmv_txn_done(h_bind, tid, FALSE); - goto get_send_txn_done; - } - } - - /* Save a reference to the MAD in the txn context - * We'll need to match it in two cases: - * (1) When the response is returned, if I am the requester - * (2) In RMPP retransmissions - */ - osmv_txn_set_madw(*pp_txn, p_madw); - - get_send_txn_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - - return ret; + ib_api_status_t ret; + uint64_t tid, key; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + ib_mad_t *p_mad = osm_madw_get_mad_ptr(p_madw); + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_get_send_txn); + CL_ASSERT(NULL != pp_txn); + + key = tid = cl_ntoh64(p_mad->trans_id); + if (TRUE == resp_expected) { + /* Create a unique identifier at the requester side */ + key = osmv_txn_uniq_key(tid); + } + + /* We must run under a transaction framework */ + ret = osmv_txn_lookup(h_bind, key, pp_txn); + if (IB_NOT_FOUND == ret) { + /* Generally, we start a new transaction */ + ret = osmv_txn_init(h_bind, tid, key, pp_txn); + if (IB_SUCCESS != ret) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7313: " + "The transaction id=0x%llX failed to init.\n", + tid); + goto get_send_txn_done; + } + } else { + CL_ASSERT(NULL != *pp_txn); + /* The transaction context exists. + * This is legal only if I am going to return an + * (RMPP?) reply to an RMPP request sent by the other part + * (double-sided RMPP transfer) + */ + if (FALSE == is_rmpp + || FALSE == osmv_txn_is_rmpp_init_by_peer(*pp_txn)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7314: " + "The transaction id=0x%llX is not unique. Send failed.\n", + tid); + + ret = IB_INVALID_SETTING; + goto get_send_txn_done; + } + + if (TRUE == resp_expected) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7315: " + "The transaction id=%llX can't expect a response. Send failed.\n", + tid); + + ret = IB_INVALID_PARAMETER; + goto get_send_txn_done; + } + } + + if (TRUE == is_rmpp) { + ret = osmv_txn_init_rmpp_sender(h_bind, *pp_txn, p_madw); + if (IB_SUCCESS != ret) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7316: " + "The transaction id=%llX failed to init the rmpp mad. Send failed.\n", + tid); + osmv_txn_done(h_bind, tid, FALSE); + goto get_send_txn_done; + } + } + + /* Save a reference to the MAD in the txn context + * We'll need to match it in two cases: + * (1) When the response is returned, if I am the requester + * (2) In RMPP retransmissions + */ + osmv_txn_set_madw(*pp_txn, p_madw); + + get_send_txn_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + + return ret; } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } diff --git a/opensm/libvendor/osm_vendor_mlx_anafa.c b/opensm/libvendor/osm_vendor_mlx_anafa.c index 722e446..bc79af8 100644 --- a/opensm/libvendor/osm_vendor_mlx_anafa.c +++ b/opensm/libvendor/osm_vendor_mlx_anafa.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -46,8 +45,6 @@ #include #include - - #include #include #include @@ -61,14 +58,12 @@ * FORWARD REFERENCES */ static ib_api_status_t -__osmv_get_send_txn (IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t is_rmpp, - IN boolean_t resp_expected, - OUT osmv_txn_ctx_t ** pp_txn); +__osmv_get_send_txn(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN boolean_t is_rmpp, + IN boolean_t resp_expected, OUT osmv_txn_ctx_t ** pp_txn); -static void - __osm_vendor_internal_unbind (osm_bind_handle_t h_bind); +static void __osm_vendor_internal_unbind(osm_bind_handle_t h_bind); /* * NAME osm_vendor_new @@ -76,30 +71,30 @@ static void * DESCRIPTION Create and Initialize the osm_vendor_t Object */ -osm_vendor_t * -osm_vendor_new (IN osm_log_t * const p_log, IN const uint32_t timeout) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { ib_api_status_t status; osm_vendor_t *p_vend; - OSM_LOG_ENTER (p_log, osm_vendor_new); + OSM_LOG_ENTER(p_log, osm_vendor_new); - CL_ASSERT (p_log); + CL_ASSERT(p_log); - p_vend = malloc (sizeof (*p_vend)); + p_vend = malloc(sizeof(*p_vend)); if (p_vend != NULL) { - memset(p_vend, 0, sizeof (*p_vend)); - status = osm_vendor_init (p_vend, p_log, timeout); + memset(p_vend, 0, sizeof(*p_vend)); + status = osm_vendor_init(p_vend, p_log, timeout); if (status != IB_SUCCESS) { - osm_vendor_delete (&p_vend); + osm_vendor_delete(&p_vend); } } else { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 7401: " - "Fail to allocate vendor object.\n"); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 7401: " + "Fail to allocate vendor object.\n"); } - OSM_LOG_EXIT (p_log); + OSM_LOG_EXIT(p_log); return (p_vend); } @@ -109,52 +104,52 @@ osm_vendor_new (IN osm_log_t * const p_log, IN const uint32_t timeout) * DESCRIPTION Delete all the binds behind the vendor + free the vendor object */ -void -osm_vendor_delete (IN osm_vendor_t ** const pp_vend) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { cl_list_item_t *p_item; cl_list_obj_t *p_obj; osm_bind_handle_t bind_h; osm_log_t *p_log; - OSM_LOG_ENTER ((*pp_vend)->p_log, osm_vendor_delete); + OSM_LOG_ENTER((*pp_vend)->p_log, osm_vendor_delete); p_log = (*pp_vend)->p_log; /* go over the bind handles , unbind them and remove from list */ /* Note that if we reached here due to problem in the init, then - the bind_handles list is not initialized yet */ - if ((*pp_vend)->bind_handles.state == CL_INITIALIZED) - { - p_item = cl_qlist_remove_head (&((*pp_vend)->bind_handles)); - while (p_item != cl_qlist_end (&((*pp_vend)->bind_handles))) { - - p_obj = PARENT_STRUCT (p_item, cl_list_obj_t, list_item); - bind_h = (osm_bind_handle_t *) cl_qlist_obj (p_obj); - osm_log (p_log, OSM_LOG_DEBUG, - "osm_vendor_delete: unbinding bind_h:%p \n", bind_h); - - __osm_vendor_internal_unbind (bind_h); - - free (p_obj); - /* removing from list */ - p_item = cl_qlist_remove_head (&((*pp_vend)->bind_handles)); - } - } + the bind_handles list is not initialized yet */ + if ((*pp_vend)->bind_handles.state == CL_INITIALIZED) { + p_item = cl_qlist_remove_head(&((*pp_vend)->bind_handles)); + while (p_item != cl_qlist_end(&((*pp_vend)->bind_handles))) { + + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + bind_h = (osm_bind_handle_t *) cl_qlist_obj(p_obj); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_vendor_delete: unbinding bind_h:%p \n", + bind_h); + + __osm_vendor_internal_unbind(bind_h); + + free(p_obj); + /* removing from list */ + p_item = + cl_qlist_remove_head(&((*pp_vend)->bind_handles)); + } + } if (NULL != ((*pp_vend)->p_transport_info)) { - free ((*pp_vend)->p_transport_info); + free((*pp_vend)->p_transport_info); (*pp_vend)->p_transport_info = NULL; } /* remove the packet randomizer object */ if ((*pp_vend)->run_randomizer == TRUE) - osm_pkt_randomizer_destroy (&((*pp_vend)->p_pkt_randomizer), - p_log); + osm_pkt_randomizer_destroy(&((*pp_vend)->p_pkt_randomizer), + p_log); - free (*pp_vend); + free(*pp_vend); *pp_vend = NULL; - OSM_LOG_EXIT (p_log); + OSM_LOG_EXIT(p_log); } /* @@ -164,39 +159,38 @@ osm_vendor_delete (IN osm_vendor_t ** const pp_vend) */ ib_api_status_t -osm_vendor_init (IN osm_vendor_t * const p_vend, - IN osm_log_t * const p_log, - IN const uint32_t timeout) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { ib_api_status_t status = IB_SUCCESS; char device_file[16]; int device_fd; - OSM_LOG_ENTER (p_log, osm_vendor_init); + OSM_LOG_ENTER(p_log, osm_vendor_init); p_vend->p_log = p_log; p_vend->resp_timeout = timeout; p_vend->ttime_timeout = timeout * OSMV_TXN_TIMEOUT_FACTOR; p_vend->p_transport_info = (osmv_TOPSPIN_ANAFA_transport_info_t *) - malloc (sizeof (osmv_TOPSPIN_ANAFA_transport_info_t)); + malloc(sizeof(osmv_TOPSPIN_ANAFA_transport_info_t)); if (!p_vend->p_transport_info) { return IB_ERROR; } - memset(p_vend->p_transport_info, 0, sizeof (osmv_TOPSPIN_ANAFA_transport_info_t)); + memset(p_vend->p_transport_info, 0, + sizeof(osmv_TOPSPIN_ANAFA_transport_info_t)); /* update the run_randomizer flag */ - if (getenv ("OSM_PKT_DROP_RATE") != NULL - && atol (getenv ("OSM_PKT_DROP_RATE")) != 0) { + if (getenv("OSM_PKT_DROP_RATE") != NULL + && atol(getenv("OSM_PKT_DROP_RATE")) != 0) { /* if the OSM_PKT_DROP_RATE global variable is defined to a non-zero value - then the randomizer should be called. Need to create the packet randomizer object */ p_vend->run_randomizer = TRUE; status = - osm_pkt_randomizer_init (&(p_vend->p_pkt_randomizer), - p_log); + osm_pkt_randomizer_init(&(p_vend->p_pkt_randomizer), p_log); if (status != IB_SUCCESS) return status; } else { @@ -205,22 +199,23 @@ osm_vendor_init (IN osm_vendor_t * const p_vend, } /* open TopSpin file device */ - sprintf (device_file, "/dev/ts_ua0"); - device_fd = open ("/dev/ts_ua0", O_RDWR); + sprintf(device_file, "/dev/ts_ua0"); + device_fd = open("/dev/ts_ua0", O_RDWR); if (device_fd < 0) { - fprintf (stderr, "Fatal: Fail to open the file:%s(%d)\n", - device_file, errno); + fprintf(stderr, "Fatal: Fail to open the file:%s(%d)\n", + device_file, errno); return IB_ERROR; } ((osmv_TOPSPIN_ANAFA_transport_info_t *) p_vend->p_transport_info)-> device_fd = device_fd; - cl_qlist_init (&p_vend->bind_handles); + cl_qlist_init(&p_vend->bind_handles); - OSM_LOG_EXIT (p_log); + OSM_LOG_EXIT(p_log); return (IB_SUCCESS); } + /* * NAME osm_vendor_bind * @@ -228,106 +223,106 @@ osm_vendor_init (IN osm_vendor_t * const p_vend, */ osm_bind_handle_t -osm_vendor_bind (IN osm_vendor_t * const p_vend, - IN osm_bind_info_t * const p_bind_info, - IN osm_mad_pool_t * const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void *context) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_bind_info, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { osmv_bind_obj_t *p_bo; cl_status_t cl_st; cl_list_obj_t *p_obj; - uint8_t hca_idx = 0; + uint8_t hca_idx = 0; if (NULL == p_vend || NULL == p_bind_info || NULL == p_mad_pool || NULL == mad_recv_callback || NULL == send_err_callback) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7402: " - "NULL parameter passed in: p_vend=%p p_bind_info=%p p_mad_pool=%p recv_cb=%p send_err_cb=%p\n", - p_vend, p_bind_info, p_mad_pool, mad_recv_callback, - send_err_callback); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7402: " + "NULL parameter passed in: p_vend=%p p_bind_info=%p p_mad_pool=%p recv_cb=%p send_err_cb=%p\n", + p_vend, p_bind_info, p_mad_pool, mad_recv_callback, + send_err_callback); return OSM_BIND_INVALID_HANDLE; } - p_bo = malloc (sizeof (osmv_bind_obj_t)); + p_bo = malloc(sizeof(osmv_bind_obj_t)); if (NULL == p_bo) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7403: " - "could not allocate the bind object\n"); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7403: " + "could not allocate the bind object\n"); return OSM_BIND_INVALID_HANDLE; } - memset (p_bo, 0, sizeof (osmv_bind_obj_t)); + memset(p_bo, 0, sizeof(osmv_bind_obj_t)); p_bo->p_vendor = p_vend; p_bo->recv_cb = mad_recv_callback; p_bo->send_err_cb = send_err_callback; p_bo->cb_context = context; p_bo->p_osm_pool = p_mad_pool; - p_bo->port_num = 1; /* anafa2 has one port */ - p_bo->hca_hndl = 0; /* only one ca on anafa system */ + p_bo->port_num = 1; /* anafa2 has one port */ + p_bo->hca_hndl = 0; /* only one ca on anafa system */ /* obtain the hca name and port num from the guid */ - osm_log (p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", - cl_ntoh64 (p_bind_info->port_guid)); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", + cl_ntoh64(p_bind_info->port_guid)); p_bo->is_closing = FALSE; - cl_spinlock_construct (&(p_bo->lock)); - cl_st = cl_spinlock_init (&(p_bo->lock)); + cl_spinlock_construct(&(p_bo->lock)); + cl_st = cl_spinlock_init(&(p_bo->lock)); if (cl_st != CL_SUCCESS) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7405: " - "could not initialize the spinlock ...\n"); - free (p_bo); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7405: " + "could not initialize the spinlock ...\n"); + free(p_bo); return OSM_BIND_INVALID_HANDLE; } - osm_log (p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: osmv_txnmgr_init ... \n"); - if (osmv_txnmgr_init (&p_bo->txn_mgr, p_vend->p_log, &(p_bo->lock)) != + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: osmv_txnmgr_init ... \n"); + if (osmv_txnmgr_init(&p_bo->txn_mgr, p_vend->p_log, &(p_bo->lock)) != IB_SUCCESS) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7406: " - "osmv_txnmgr_init failed \n"); - cl_spinlock_destroy (&p_bo->lock); - free (p_bo); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7406: " + "osmv_txnmgr_init failed \n"); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); return OSM_BIND_INVALID_HANDLE; } /* Do the real job! (Transport-dependent) */ if (IB_SUCCESS != - osmv_transport_init (p_bind_info, OSMV_ANAFA_ID, hca_idx, p_bo)) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7407: " - "osmv_transport_init failed \n"); - osmv_txnmgr_done ((osm_bind_handle_t) p_bo); - cl_spinlock_destroy (&p_bo->lock); - free (p_bo); + osmv_transport_init(p_bind_info, OSMV_ANAFA_ID, hca_idx, p_bo)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7407: " + "osmv_transport_init failed \n"); + osmv_txnmgr_done((osm_bind_handle_t) p_bo); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); return OSM_BIND_INVALID_HANDLE; } /* insert bind handle into db */ - p_obj = malloc (sizeof (cl_list_obj_t)); + p_obj = malloc(sizeof(cl_list_obj_t)); if (NULL == p_obj) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 7408: " - "osm_vendor_bind: could not allocate the list object\n"); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 7408: " + "osm_vendor_bind: could not allocate the list object\n"); - osmv_transport_done (p_bo->p_transp_mgr); - osmv_txnmgr_done ((osm_bind_handle_t) p_bo); - cl_spinlock_destroy (&p_bo->lock); - free (p_bo); + osmv_transport_done(p_bo->p_transp_mgr); + osmv_txnmgr_done((osm_bind_handle_t) p_bo); + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); return OSM_BIND_INVALID_HANDLE; } if (p_obj) - memset (p_obj, 0, sizeof (cl_list_obj_t)); - cl_qlist_set_obj (p_obj, p_bo); + memset(p_obj, 0, sizeof(cl_list_obj_t)); + cl_qlist_set_obj(p_obj, p_bo); - cl_qlist_insert_head (&p_vend->bind_handles, &p_obj->list_item); + cl_qlist_insert_head(&p_vend->bind_handles, &p_obj->list_item); return (osm_bind_handle_t) p_bo; } @@ -338,8 +333,7 @@ osm_vendor_bind (IN osm_vendor_t * const p_vend, * DESCRIPTION Destroy the bind object and remove it from the vendor's list */ -void -osm_vendor_unbind (IN osm_bind_handle_t h_bind) +void osm_vendor_unbind(IN osm_bind_handle_t h_bind) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osm_log_t *p_log = p_bo->p_vendor->p_log; @@ -348,27 +342,27 @@ osm_vendor_unbind (IN osm_bind_handle_t h_bind) cl_qlist_t *const p_bh_list = (cl_qlist_t * const)&p_bo->p_vendor->bind_handles; - OSM_LOG_ENTER (p_log, osm_vendor_unbind); + OSM_LOG_ENTER(p_log, osm_vendor_unbind); /* go over all the items in the list and remove the specific item */ - p_item = cl_qlist_head (&p_bo->p_vendor->bind_handles); - while (p_item != cl_qlist_end (&p_bo->p_vendor->bind_handles)) { - p_obj = PARENT_STRUCT (p_item, cl_list_obj_t, list_item); - if (cl_qlist_obj (p_obj) == h_bind) { + p_item = cl_qlist_head(&p_bo->p_vendor->bind_handles); + while (p_item != cl_qlist_end(&p_bo->p_vendor->bind_handles)) { + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + if (cl_qlist_obj(p_obj) == h_bind) { break; } - p_item_tmp = cl_qlist_next (p_item); + p_item_tmp = cl_qlist_next(p_item); p_item = p_item_tmp; } - CL_ASSERT (p_item != cl_qlist_end (p_bh_list)); + CL_ASSERT(p_item != cl_qlist_end(p_bh_list)); - cl_qlist_remove_item (p_bh_list, p_item); - free (p_obj); + cl_qlist_remove_item(p_bh_list, p_item); + free(p_obj); - __osm_vendor_internal_unbind (h_bind); + __osm_vendor_internal_unbind(h_bind); - OSM_LOG_EXIT (p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /* @@ -377,19 +371,18 @@ osm_vendor_unbind (IN osm_bind_handle_t h_bind) * DESCRIPTION Allocate the space for a new MAD */ -ib_mad_t * -osm_vendor_get (IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t * const p_vw) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { ib_mad_t *p_mad; osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osm_vendor_t const *p_vend = p_bo->p_vendor; uint32_t act_mad_size; - OSM_LOG_ENTER (p_vend->p_log, osm_vendor_get); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); - CL_ASSERT (p_vw); + CL_ASSERT(p_vw); if (mad_size < MAD_BLOCK_SIZE) { /* Stupid, but the applications want that! */ @@ -399,25 +392,25 @@ osm_vendor_get (IN osm_bind_handle_t h_bind, } /* allocate it */ - p_mad = (ib_mad_t *) malloc (act_mad_size); + p_mad = (ib_mad_t *) malloc(act_mad_size); if (p_mad == NULL) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get: ERR 7409: " - "Error Obtaining MAD buffer.\n"); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get: ERR 7409: " + "Error Obtaining MAD buffer.\n"); goto Exit; } - memset (p_mad, 0, act_mad_size); + memset(p_mad, 0, act_mad_size); - if (osm_log_get_level (p_vend->p_log) >= OSM_LOG_DEBUG) { - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get: " - "Allocated MAD %p, size = %u.\n", p_mad, act_mad_size); + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get: " + "Allocated MAD %p, size = %u.\n", p_mad, act_mad_size); } p_vw->p_mad = p_mad; Exit: - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); return (p_mad); } @@ -434,9 +427,8 @@ osm_vendor_get (IN osm_bind_handle_t h_bind, */ ib_api_status_t -osm_vendor_send (IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t const resp_expected) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { ib_api_status_t ret = IB_SUCCESS; osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; @@ -444,37 +436,37 @@ osm_vendor_send (IN osm_bind_handle_t h_bind, osmv_txn_ctx_t *p_txn = NULL; ib_mad_t *p_mad; - OSM_LOG_ENTER (p_bo->p_vendor->p_log, osm_vendor_send); + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osm_vendor_send); if (NULL == h_bind || NULL == p_madw || - NULL == (p_mad = osm_madw_get_mad_ptr (p_madw)) || - NULL == osm_madw_get_mad_addr_ptr (p_madw)) { + NULL == (p_mad = osm_madw_get_mad_ptr(p_madw)) || + NULL == osm_madw_get_mad_addr_ptr(p_madw)) { return IB_INVALID_PARAMETER; } is_rmpp = (p_madw->mad_size > MAD_BLOCK_SIZE - || osmv_mad_is_rmpp (p_mad)); + || osmv_mad_is_rmpp(p_mad)); is_rmpp_ds = (TRUE == is_rmpp && TRUE == resp_expected); /* Make our operations with the send context atomic */ - osmv_txn_lock (p_bo); + osmv_txn_lock(p_bo); if (TRUE == p_bo->is_closing) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 7410: " - "The handle %p is being unbound, cannot send.\n", - h_bind); - ret = IB_INTERRUPTED; - goto send_done; + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 7410: " + "The handle %p is being unbound, cannot send.\n", + h_bind); + ret = IB_INTERRUPTED; + goto send_done; } if (TRUE == resp_expected || TRUE == is_rmpp) { /* We must run under a transaction framework. * Get the transaction object (old or new) */ - ret = __osmv_get_send_txn (h_bind, p_madw, is_rmpp, - resp_expected, &p_txn); + ret = __osmv_get_send_txn(h_bind, p_madw, is_rmpp, + resp_expected, &p_txn); if (IB_SUCCESS != ret) { goto send_done; } @@ -486,52 +478,52 @@ osm_vendor_send (IN osm_bind_handle_t h_bind, * The txn lock will be released each time the function sleeps * and re-acquired when it wakes up */ - ret = osmv_rmpp_send_madw (h_bind, p_madw, p_txn, is_rmpp_ds); + ret = osmv_rmpp_send_madw(h_bind, p_madw, p_txn, is_rmpp_ds); } else { /* Do the job - single MAD! * The call returns as soon as the MAD is put on the wire */ - ret = osmv_simple_send_madw (h_bind, p_madw, p_txn , FALSE); /* anafa2 */ + ret = osmv_simple_send_madw(h_bind, p_madw, p_txn, FALSE); /* anafa2 */ } if (IB_SUCCESS == ret) { if ((TRUE == is_rmpp) && (FALSE == is_rmpp_ds)) { /* For double-sided sends, the txn continues to live */ - osmv_txn_done (h_bind, osmv_txn_get_key (p_txn), - FALSE /*not in callback */ ); + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), + FALSE /*not in callback */ ); } if (FALSE == resp_expected) { - osm_mad_pool_put (p_bo->p_osm_pool, p_madw); + osm_mad_pool_put(p_bo->p_osm_pool, p_madw); } } else { if (NULL != p_txn) { - osmv_txn_done (h_bind, osmv_txn_get_key (p_txn), - FALSE /*not in callback */ ); + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), + FALSE /*not in callback */ ); } - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 7411: failed to send MADW %p\n", - p_madw); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 7411: failed to send MADW %p\n", + p_madw); if (TRUE == resp_expected) { /* Change the status on the p_madw */ p_madw->status = ret; /* Only the requester expects the error callback */ - p_bo->send_err_cb (p_bo->cb_context, p_madw); + p_bo->send_err_cb(p_bo->cb_context, p_madw); } else { /* put back the mad - it is useless ... */ - osm_mad_pool_put (p_bo->p_osm_pool, p_madw); + osm_mad_pool_put(p_bo->p_osm_pool, p_madw); } } - send_done: + send_done: - osmv_txn_unlock (p_bo); + osmv_txn_unlock(p_bo); - OSM_LOG_EXIT (p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); return ret; } @@ -542,28 +534,28 @@ osm_vendor_send (IN osm_bind_handle_t h_bind, */ void -osm_vendor_put (IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t * const p_vw) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osm_vendor_t const *p_vend = p_bo->p_vendor; - OSM_LOG_ENTER (p_vend->p_log, osm_vendor_put); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); - CL_ASSERT (p_vw); - CL_ASSERT (p_vw->p_mad); + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->p_mad); - if (osm_log_get_level (p_vend->p_log) >= OSM_LOG_DEBUG) { - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_put: " "Retiring MAD %p.\n", p_vw->p_mad); + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_put: " "Retiring MAD %p.\n", p_vw->p_mad); } - free (p_vw->p_mad); + free(p_vw->p_mad); p_vw->p_mad = NULL; - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); } + /* * NAME osm_vendor_local_lid_change * @@ -572,16 +564,15 @@ osm_vendor_put (IN osm_bind_handle_t h_bind, * housekeeping functions such as address vector updates. */ -ib_api_status_t -osm_vendor_local_lid_change (IN osm_bind_handle_t h_bind) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { osm_vendor_t const *p_vend = ((osmv_bind_obj_t *) h_bind)->p_vendor; - OSM_LOG_ENTER (p_vend->p_log, osm_vendor_local_lid_change); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_local_lid_change: " "Change of LID.\n"); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_local_lid_change: " "Change of LID.\n"); - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); return (IB_SUCCESS); @@ -593,10 +584,7 @@ osm_vendor_local_lid_change (IN osm_bind_handle_t h_bind) * DESCRIPTION Modifies the port info for the bound port to set the "IS_SM" bit. */ -void -osm_vendor_set_sm ( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osm_vendor_t const *p_vend = p_bo->p_vendor; @@ -604,23 +592,23 @@ osm_vendor_set_sm ( int ioctl_ret; osm_ts_set_port_info_ioctl port_info; - OSM_LOG_ENTER (p_vend->p_log, osm_vendor_set_sm); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); - port_info.port = 0; /* anafa has only 1 port */ + port_info.port = 0; /* anafa has only 1 port */ port_info.port_info.valid_fields = IB_PORT_IS_SM; port_info.port_info.is_sm = is_sm_val; p_mgr = (osmv_TOPSPIN_ANAFA_transport_mgr_t *) p_bo->p_transp_mgr; - ioctl_ret = ioctl (p_mgr->device_fd, TS_IB_IOCSPORTINFO, &port_info); + ioctl_ret = ioctl(p_mgr->device_fd, TS_IB_IOCSPORTINFO, &port_info); if (ioctl_ret < 0) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 7412: " - "Unable set 'IS_SM' bit to:%u in port attributes (%d). errno=%d\n", - is_sm_val, ioctl_ret, errno); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 7412: " + "Unable set 'IS_SM' bit to:%u in port attributes (%d). errno=%d\n", + is_sm_val, ioctl_ret, errno); } - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); } /* @@ -631,47 +619,46 @@ osm_vendor_set_sm ( * (2) Destroy the associated data structures */ -static void -__osm_vendor_internal_unbind (osm_bind_handle_t h_bind) +static void __osm_vendor_internal_unbind(osm_bind_handle_t h_bind) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osm_log_t *p_log = p_bo->p_vendor->p_log; - OSM_LOG_ENTER (p_log, __osm_vendor_internal_unbind); + OSM_LOG_ENTER(p_log, __osm_vendor_internal_unbind); /* "notifying" all that from now on no new sends can be done */ - p_bo->txn_mgr.p_event_wheel->closing = TRUE; + p_bo->txn_mgr.p_event_wheel->closing = TRUE; - osmv_txn_lock (p_bo); + osmv_txn_lock(p_bo); p_bo->is_closing = TRUE; /* notifying all sleeping rmpp sends to exit */ - osmv_txn_abort_rmpp_txns (h_bind); + osmv_txn_abort_rmpp_txns(h_bind); /* frees all data in bind handle */ - osm_log (p_log, OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying transport mgr.. \n"); - osmv_txn_unlock(p_bo); - - osmv_transport_done (h_bind); - osm_log (p_log, OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying txn mgr.. \n"); - osmv_txn_lock (p_bo); - osmv_txnmgr_done (h_bind); - osm_log (p_log, OSM_LOG_DEBUG, - "__osm_vendor_internal_unbind: destroying bind lock.. \n"); - - osmv_txn_unlock(p_bo); - /* - we intentionally let the p_bo and its lock leak - - as we did not implement a way to track active bind handles provided to - the client - and the client might use them - - cl_spinlock_destroy(&p_bo->lock); - free(p_bo); - */ - - OSM_LOG_EXIT (p_log); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying transport mgr.. \n"); + osmv_txn_unlock(p_bo); + + osmv_transport_done(h_bind); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying txn mgr.. \n"); + osmv_txn_lock(p_bo); + osmv_txnmgr_done(h_bind); + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_vendor_internal_unbind: destroying bind lock.. \n"); + + osmv_txn_unlock(p_bo); + /* + we intentionally let the p_bo and its lock leak - + as we did not implement a way to track active bind handles provided to + the client - and the client might use them + + cl_spinlock_destroy(&p_bo->lock); + free(p_bo); + */ + + OSM_LOG_EXIT(p_log); } /* @@ -682,57 +669,56 @@ __osm_vendor_internal_unbind (osm_bind_handle_t h_bind) */ static ib_api_status_t -__osmv_get_send_txn (IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t is_rmpp, - IN boolean_t resp_expected, - OUT osmv_txn_ctx_t ** pp_txn) +__osmv_get_send_txn(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, + IN boolean_t is_rmpp, + IN boolean_t resp_expected, OUT osmv_txn_ctx_t ** pp_txn) { ib_api_status_t ret; uint64_t tid, key; osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; - ib_mad_t *p_mad = osm_madw_get_mad_ptr (p_madw); + ib_mad_t *p_mad = osm_madw_get_mad_ptr(p_madw); - OSM_LOG_ENTER (p_bo->p_vendor->p_log, __osmv_get_send_txn); - CL_ASSERT (NULL != pp_txn); + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_get_send_txn); + CL_ASSERT(NULL != pp_txn); - key = tid = cl_ntoh64 (p_mad->trans_id); + key = tid = cl_ntoh64(p_mad->trans_id); if (TRUE == resp_expected) { /* Create a unique identifier at the requester side */ - key = osmv_txn_uniq_key (tid); + key = osmv_txn_uniq_key(tid); } /* We must run under a transaction framework */ - ret = osmv_txn_lookup (h_bind, key, pp_txn); + ret = osmv_txn_lookup(h_bind, key, pp_txn); if (IB_NOT_FOUND == ret) { /* Generally, we start a new transaction */ - ret = osmv_txn_init (h_bind, tid, key, pp_txn); + ret = osmv_txn_init(h_bind, tid, key, pp_txn); if (IB_SUCCESS != ret) { goto get_send_txn_done; } } else { - CL_ASSERT (NULL != *pp_txn); + CL_ASSERT(NULL != *pp_txn); /* The transaction context exists. * This is legal only if I am going to return an * (RMPP?) reply to an RMPP request sent by the other part * (double-sided RMPP transfer) */ if (FALSE == is_rmpp - || FALSE == osmv_txn_is_rmpp_init_by_peer (*pp_txn)) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7413: " - "The transaction id=0x%llX is not unique. Send failed.\n", - tid); + || FALSE == osmv_txn_is_rmpp_init_by_peer(*pp_txn)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7413: " + "The transaction id=0x%llX is not unique. Send failed.\n", + tid); ret = IB_INVALID_SETTING; goto get_send_txn_done; } if (TRUE == resp_expected) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_get_send_txn: ERR 7414: " - "The transaction id=%llX can\'t expect a response. Send failed.\n", - tid); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_get_send_txn: ERR 7414: " + "The transaction id=%llX can\'t expect a response. Send failed.\n", + tid); ret = IB_INVALID_PARAMETER; goto get_send_txn_done; @@ -740,9 +726,9 @@ __osmv_get_send_txn (IN osm_bind_handle_t h_bind, } if (TRUE == is_rmpp) { - ret = osmv_txn_init_rmpp_sender (h_bind, *pp_txn, p_madw); + ret = osmv_txn_init_rmpp_sender(h_bind, *pp_txn, p_madw); if (IB_SUCCESS != ret) { - osmv_txn_done (h_bind, tid, FALSE); + osmv_txn_done(h_bind, tid, FALSE); goto get_send_txn_done; } } @@ -752,20 +738,17 @@ __osmv_get_send_txn (IN osm_bind_handle_t h_bind, * (1) When the response is returned, if I am the requester * (2) In RMPP retransmissions */ - osmv_txn_set_madw (*pp_txn, p_madw); + osmv_txn_set_madw(*pp_txn, p_madw); get_send_txn_done: - OSM_LOG_EXIT (p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); return ret; } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } diff --git a/opensm/libvendor/osm_vendor_mlx_dispatcher.c b/opensm/libvendor/osm_vendor_mlx_dispatcher.c index 65309ef..968a914 100644 --- a/opensm/libvendor/osm_vendor_mlx_dispatcher.c +++ b/opensm/libvendor/osm_vendor_mlx_dispatcher.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -47,9 +47,9 @@ typedef enum _osmv_disp_route { - OSMV_ROUTE_DROP, - OSMV_ROUTE_SIMPLE, - OSMV_ROUTE_RMPP, + OSMV_ROUTE_DROP, + OSMV_ROUTE_SIMPLE, + OSMV_ROUTE_RMPP, } osmv_disp_route_t; @@ -58,43 +58,42 @@ typedef enum _osmv_disp_route { */ static osmv_disp_route_t -__osmv_dispatch_route(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - OUT osmv_txn_ctx_t **pp_txn); +__osmv_dispatch_route(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, OUT osmv_txn_ctx_t ** pp_txn); static void -__osmv_dispatch_simple_mad(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr); +__osmv_dispatch_simple_mad(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr); static void -__osmv_dispatch_rmpp_mad(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr); +__osmv_dispatch_rmpp_mad(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr); static void -__osmv_dispatch_rmpp_snd(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr); +__osmv_dispatch_rmpp_snd(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr); static ib_api_status_t -__osmv_dispatch_rmpp_rcv(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr); +__osmv_dispatch_rmpp_rcv(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr); static ib_api_status_t -__osmv_dispatch_accept_seg(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN const ib_mad_t *p_mad); +__osmv_dispatch_accept_seg(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, + IN const ib_mad_t * p_mad); static void -__osmv_dispatch_send_ack(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr); +__osmv_dispatch_send_ack(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr); /* * NAME @@ -114,73 +113,72 @@ __osmv_dispatch_send_ack(IN osm_bind_handle_t h_bind, */ ib_api_status_t -osmv_dispatch_mad(IN osm_bind_handle_t h_bind, - IN const void *p_mad_buf, - IN const osm_mad_addr_t *p_mad_addr) +osmv_dispatch_mad(IN osm_bind_handle_t h_bind, + IN const void *p_mad_buf, + IN const osm_mad_addr_t * p_mad_addr) { - ib_api_status_t ret = IB_SUCCESS; - const ib_mad_t *p_mad = (ib_mad_t *)p_mad_buf; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - osmv_txn_ctx_t *p_txn = NULL; - osm_log_t* p_log = p_bo->p_vendor->p_log; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, osmv_dispatch_mad); - - CL_ASSERT(NULL != h_bind && NULL != p_mad && NULL != p_mad_addr); - - osmv_txn_lock(p_bo); - - if (TRUE == p_bo->is_closing ) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The bind handle %p is being closed. " - "The MAD will not be dispatched.\n", p_bo); - - ret = IB_INTERRUPTED; - goto dispatch_mad_done; - } - - /* - Add call for packet drop randomizer. - This is a testing feature. If run_randomizer flag is set to TRUE, - the randomizer will be called, and randomally will drop - a packet. This is used for simulating unstable fabric. - */ - if ( p_bo->p_vendor->run_randomizer == TRUE ) - { - /* Try the randomizer */ - if ( osm_pkt_randomizer_mad_drop( p_bo->p_vendor->p_log, - p_bo->p_vendor->p_pkt_randomizer, - p_mad ) == TRUE ) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The MAD will not be dispatched.\n"); - goto dispatch_mad_done; - } - } - - switch (__osmv_dispatch_route(h_bind, p_mad, &p_txn)) { - - case OSMV_ROUTE_DROP: - break; /* Do nothing */ - - case OSMV_ROUTE_SIMPLE: - __osmv_dispatch_simple_mad(h_bind, p_mad, p_txn, p_mad_addr); - break; - - case OSMV_ROUTE_RMPP: - __osmv_dispatch_rmpp_mad(h_bind, p_mad, p_txn, p_mad_addr); - break; - - default: CL_ASSERT(FALSE); - } - - dispatch_mad_done: - osmv_txn_unlock(p_bo); - - OSM_LOG_EXIT( p_log ); - return ret; + ib_api_status_t ret = IB_SUCCESS; + const ib_mad_t *p_mad = (ib_mad_t *) p_mad_buf; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osmv_txn_ctx_t *p_txn = NULL; + osm_log_t *p_log = p_bo->p_vendor->p_log; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_dispatch_mad); + + CL_ASSERT(NULL != h_bind && NULL != p_mad && NULL != p_mad_addr); + + osmv_txn_lock(p_bo); + + if (TRUE == p_bo->is_closing) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The bind handle %p is being closed. " + "The MAD will not be dispatched.\n", p_bo); + + ret = IB_INTERRUPTED; + goto dispatch_mad_done; + } + + /* + Add call for packet drop randomizer. + This is a testing feature. If run_randomizer flag is set to TRUE, + the randomizer will be called, and randomally will drop + a packet. This is used for simulating unstable fabric. + */ + if (p_bo->p_vendor->run_randomizer == TRUE) { + /* Try the randomizer */ + if (osm_pkt_randomizer_mad_drop(p_bo->p_vendor->p_log, + p_bo->p_vendor-> + p_pkt_randomizer, + p_mad) == TRUE) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The MAD will not be dispatched.\n"); + goto dispatch_mad_done; + } + } + + switch (__osmv_dispatch_route(h_bind, p_mad, &p_txn)) { + + case OSMV_ROUTE_DROP: + break; /* Do nothing */ + + case OSMV_ROUTE_SIMPLE: + __osmv_dispatch_simple_mad(h_bind, p_mad, p_txn, p_mad_addr); + break; + + case OSMV_ROUTE_RMPP: + __osmv_dispatch_rmpp_mad(h_bind, p_mad, p_txn, p_mad_addr); + break; + + default: + CL_ASSERT(FALSE); + } + + dispatch_mad_done: + osmv_txn_unlock(p_bo); + + OSM_LOG_EXIT(p_log); + return ret; } /* @@ -190,46 +188,42 @@ osmv_dispatch_mad(IN osm_bind_handle_t h_bind, */ static osmv_disp_route_t -__osmv_dispatch_route(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - OUT osmv_txn_ctx_t **pp_txn) +__osmv_dispatch_route(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, OUT osmv_txn_ctx_t ** pp_txn) { - ib_api_status_t ret; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - boolean_t is_resp = osmv_mad_is_response(p_mad); - boolean_t is_txn; - uint64_t key = cl_ntoh64(p_mad->trans_id); - - CL_ASSERT(NULL != pp_txn); - - ret = osmv_txn_lookup(h_bind, key, pp_txn); - is_txn = (IB_SUCCESS == ret); - - if (FALSE == is_txn && TRUE == is_resp) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Received a response to a non-started/aged-out transaction (tid=0x%llX). " - "Dropping the MAD.\n", key); - return OSMV_ROUTE_DROP; - } - - if (TRUE == osmv_mad_is_rmpp(p_mad)) - { - /* An RMPP transaction. The filtering is more delicate there */ - return OSMV_ROUTE_RMPP; - } - - if (TRUE == is_txn && FALSE == is_resp) - { - /* Does this MAD try to start a transaction with duplicate tid? */ - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Duplicate TID 0x%llX received (not a response). " - "Dropping the MAD.\n", key); - - return OSMV_ROUTE_DROP; - } - - return OSMV_ROUTE_SIMPLE; + ib_api_status_t ret; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + boolean_t is_resp = osmv_mad_is_response(p_mad); + boolean_t is_txn; + uint64_t key = cl_ntoh64(p_mad->trans_id); + + CL_ASSERT(NULL != pp_txn); + + ret = osmv_txn_lookup(h_bind, key, pp_txn); + is_txn = (IB_SUCCESS == ret); + + if (FALSE == is_txn && TRUE == is_resp) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Received a response to a non-started/aged-out transaction (tid=0x%llX). " + "Dropping the MAD.\n", key); + return OSMV_ROUTE_DROP; + } + + if (TRUE == osmv_mad_is_rmpp(p_mad)) { + /* An RMPP transaction. The filtering is more delicate there */ + return OSMV_ROUTE_RMPP; + } + + if (TRUE == is_txn && FALSE == is_resp) { + /* Does this MAD try to start a transaction with duplicate tid? */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Duplicate TID 0x%llX received (not a response). " + "Dropping the MAD.\n", key); + + return OSMV_ROUTE_DROP; + } + + return OSMV_ROUTE_SIMPLE; } /* @@ -239,62 +233,59 @@ __osmv_dispatch_route(IN osm_bind_handle_t h_bind, */ static void -__osmv_dispatch_simple_mad(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr) +__osmv_dispatch_simple_mad(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr) { - osm_madw_t *p_madw; - ib_mad_t *p_mad_buf; - osm_madw_t *p_req_madw = NULL; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_dispatch_simple_mad); - - /* Build the MAD wrapper to be returned to the user. - * The actual storage for the MAD is allocated there. - */ - p_madw = - osm_mad_pool_get(p_bo->p_osm_pool, h_bind, MAD_BLOCK_SIZE, p_mad_addr); - - if (NULL == p_madw) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_simple_mad: ERR 6501: " - "Out Of Memory - could not allocate a buffer of size %d\n", - MAD_BLOCK_SIZE); - - goto dispatch_simple_mad_done; - } - - p_mad_buf = osm_madw_get_mad_ptr(p_madw); - /* Copy the payload to the MAD buffer */ - memcpy((void*)p_mad_buf, (void*)p_mad, MAD_BLOCK_SIZE); - - if (NULL != p_txn) - { - /* This is a RESPONSE MAD. Pair it with the REQUEST MAD, pass upstream */ - p_req_madw = p_txn->p_madw; - CL_ASSERT(NULL != p_req_madw); - - p_mad_buf->trans_id = cl_hton64(osmv_txn_get_tid(p_txn)); - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Restoring the original TID to 0x%llX\n", - cl_ntoh64(p_mad_buf->trans_id)); - - /* Reply matched, transaction complete */ - osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); - } - else - { - /* This is a REQUEST MAD. Don't create a context, pass upstream */ - } - - /* Do the job !*/ - p_bo->recv_cb(p_madw, p_bo->cb_context, p_req_madw); - - dispatch_simple_mad_done: - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); + osm_madw_t *p_madw; + ib_mad_t *p_mad_buf; + osm_madw_t *p_req_madw = NULL; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_simple_mad); + + /* Build the MAD wrapper to be returned to the user. + * The actual storage for the MAD is allocated there. + */ + p_madw = + osm_mad_pool_get(p_bo->p_osm_pool, h_bind, MAD_BLOCK_SIZE, + p_mad_addr); + + if (NULL == p_madw) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_simple_mad: ERR 6501: " + "Out Of Memory - could not allocate a buffer of size %d\n", + MAD_BLOCK_SIZE); + + goto dispatch_simple_mad_done; + } + + p_mad_buf = osm_madw_get_mad_ptr(p_madw); + /* Copy the payload to the MAD buffer */ + memcpy((void *)p_mad_buf, (void *)p_mad, MAD_BLOCK_SIZE); + + if (NULL != p_txn) { + /* This is a RESPONSE MAD. Pair it with the REQUEST MAD, pass upstream */ + p_req_madw = p_txn->p_madw; + CL_ASSERT(NULL != p_req_madw); + + p_mad_buf->trans_id = cl_hton64(osmv_txn_get_tid(p_txn)); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Restoring the original TID to 0x%llX\n", + cl_ntoh64(p_mad_buf->trans_id)); + + /* Reply matched, transaction complete */ + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); + } else { + /* This is a REQUEST MAD. Don't create a context, pass upstream */ + } + + /* Do the job ! */ + p_bo->recv_cb(p_madw, p_bo->cb_context, p_req_madw); + + dispatch_simple_mad_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /* @@ -304,88 +295,86 @@ __osmv_dispatch_simple_mad(IN osm_bind_handle_t h_bind, */ static void -__osmv_dispatch_rmpp_mad(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr) +__osmv_dispatch_rmpp_mad(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr) { - ib_api_status_t status = IB_SUCCESS; - uint64_t key = cl_ntoh64(p_mad->trans_id); - boolean_t is_init_by_peer = FALSE; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - osm_madw_t *p_madw; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_mad); - - if (NULL == p_txn) - { - if (FALSE == osmv_rmpp_is_data(p_mad) || FALSE == osmv_rmpp_is_first(p_mad)) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The MAD does not match any transaction " - "and does not start a sender-initiated RMPP transfer.\n"); - goto dispatch_rmpp_mad_done; - } - - /* IB Spec 13.6.2.2. This is a Sender Initiated Transfer. - My peer is the requester and RMPP Sender. I am the RMPP Receiver. - */ - status = osmv_txn_init(h_bind, /*tid==key*/key, key, &p_txn); - if (IB_SUCCESS != status) - { - goto dispatch_rmpp_mad_done; - } - - is_init_by_peer = TRUE; - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "A new sender-initiated transfer (TID=0x%llX) started\n", key); - } - - if (OSMV_TXN_RMPP_NONE == osmv_txn_get_rmpp_state(p_txn)) - { - /* Case 1: Fall through from above. - * Case 2: When the transaction was initiated by me - * (a single request MAD), there was an uncertainty - * whether the reply will be RMPP. Now it's resolved, - * since the reply is RMPP! - */ - status = osmv_txn_init_rmpp_receiver(h_bind, p_txn, is_init_by_peer); - if (IB_SUCCESS != status) - { - goto dispatch_rmpp_mad_done; - } - } - - switch (osmv_txn_get_rmpp_state(p_txn)) { - - case OSMV_TXN_RMPP_RECEIVER: - status = __osmv_dispatch_rmpp_rcv(h_bind, p_mad, p_txn, p_mad_addr); - if (IB_SUCCESS != status) - { - if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn)) - { - /* This is a requester, still waiting for the reply. Apply the callback */ - /* update the status of the p_madw */ - p_madw = osmv_txn_get_madw(p_txn); - p_madw->status = status; - p_bo->send_err_cb(p_bo->cb_context, p_madw); - } - - /* ABORT/STOP/LOCAL ERROR */ - osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); - } - break; - - case OSMV_TXN_RMPP_SENDER: - __osmv_dispatch_rmpp_snd(h_bind, p_mad, p_txn, p_mad_addr); - /* If an error happens here, it's the sender thread to cleanup the txn */ - break; - - default: CL_ASSERT(FALSE); - } - - dispatch_rmpp_mad_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); + ib_api_status_t status = IB_SUCCESS; + uint64_t key = cl_ntoh64(p_mad->trans_id); + boolean_t is_init_by_peer = FALSE; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_madw_t *p_madw; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_mad); + + if (NULL == p_txn) { + if (FALSE == osmv_rmpp_is_data(p_mad) + || FALSE == osmv_rmpp_is_first(p_mad)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The MAD does not match any transaction " + "and does not start a sender-initiated RMPP transfer.\n"); + goto dispatch_rmpp_mad_done; + } + + /* IB Spec 13.6.2.2. This is a Sender Initiated Transfer. + My peer is the requester and RMPP Sender. I am the RMPP Receiver. + */ + status = osmv_txn_init(h_bind, /*tid==key */ key, key, &p_txn); + if (IB_SUCCESS != status) { + goto dispatch_rmpp_mad_done; + } + + is_init_by_peer = TRUE; + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "A new sender-initiated transfer (TID=0x%llX) started\n", + key); + } + + if (OSMV_TXN_RMPP_NONE == osmv_txn_get_rmpp_state(p_txn)) { + /* Case 1: Fall through from above. + * Case 2: When the transaction was initiated by me + * (a single request MAD), there was an uncertainty + * whether the reply will be RMPP. Now it's resolved, + * since the reply is RMPP! + */ + status = + osmv_txn_init_rmpp_receiver(h_bind, p_txn, is_init_by_peer); + if (IB_SUCCESS != status) { + goto dispatch_rmpp_mad_done; + } + } + + switch (osmv_txn_get_rmpp_state(p_txn)) { + + case OSMV_TXN_RMPP_RECEIVER: + status = + __osmv_dispatch_rmpp_rcv(h_bind, p_mad, p_txn, p_mad_addr); + if (IB_SUCCESS != status) { + if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn)) { + /* This is a requester, still waiting for the reply. Apply the callback */ + /* update the status of the p_madw */ + p_madw = osmv_txn_get_madw(p_txn); + p_madw->status = status; + p_bo->send_err_cb(p_bo->cb_context, p_madw); + } + + /* ABORT/STOP/LOCAL ERROR */ + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); + } + break; + + case OSMV_TXN_RMPP_SENDER: + __osmv_dispatch_rmpp_snd(h_bind, p_mad, p_txn, p_mad_addr); + /* If an error happens here, it's the sender thread to cleanup the txn */ + break; + + default: + CL_ASSERT(FALSE); + } + + dispatch_rmpp_mad_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /* @@ -395,100 +384,94 @@ __osmv_dispatch_rmpp_mad(IN osm_bind_handle_t h_bind, */ static void -__osmv_dispatch_rmpp_snd(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr) +__osmv_dispatch_rmpp_snd(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr) { - osmv_rmpp_send_ctx_t *p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); - - uint32_t old_wl = p_send_ctx->window_last; - uint32_t total_segs = osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); - uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t*)p_mad)->seg_num); - uint32_t new_wl = cl_ntoh32(((ib_rmpp_mad_t*)p_mad)->paylen_newwin); - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_snd); - - if (TRUE == osmv_rmpp_is_abort_stop(p_mad)) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_rmpp_snd: ERR 6502: " - "The remote side sent an ABORT/STOP indication.\n"); - osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); - goto dispatch_rmpp_snd_done; - } - - if (FALSE == osmv_rmpp_is_ack(p_mad)) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Not supposed to receive DATA packets --> dropping the MAD\n"); - goto dispatch_rmpp_snd_done; - } - - /* Continue processing the ACK */ - if (seg_num > old_wl) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_rmpp_snd: ERR 6503: " - "ACK received for a non-sent segment %d\n", seg_num); - - osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, - IB_RMPP_TYPE_ABORT, - IB_RMPP_STATUS_S2B); - - osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); - goto dispatch_rmpp_snd_done; - } - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_dispatch_rmpp_snd: " - "New WL = %u Old WL = %u Total Segs = %u\n", - new_wl, old_wl, total_segs); - - if (new_wl < old_wl) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_rmpp_snd: ERR 6508: " - "The receiver requests a smaller WL (%d) than before (%d)\n", - new_wl, old_wl); - - osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, - IB_RMPP_TYPE_ABORT, - IB_RMPP_STATUS_W2S); - - osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); - goto dispatch_rmpp_snd_done; - } - - /* Update the sender's window, and optionally wake up the sender thread - * Note! A single ACK can acknowledge a whole range of segments: [WF..SEG_NUM] - */ - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "ACK for seg_num #%d accepted.\n", seg_num); - - if (seg_num == old_wl) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The send window [%d:%d] is totally acknowledged.\n", - p_send_ctx->window_first, old_wl); - - p_send_ctx->window_first = seg_num+1; - p_send_ctx->window_last = (new_wl < total_segs) ? new_wl : total_segs; - - /* Remove the response timeout event for the window */ - osmv_txn_remove_timeout_ev(h_bind, osmv_txn_get_key(p_txn)); - - /* Wake up the sending thread */ - cl_event_signal(&p_send_ctx->event); - } - - dispatch_rmpp_snd_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); + osmv_rmpp_send_ctx_t *p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); + + uint32_t old_wl = p_send_ctx->window_last; + uint32_t total_segs = osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); + uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t *) p_mad)->seg_num); + uint32_t new_wl = cl_ntoh32(((ib_rmpp_mad_t *) p_mad)->paylen_newwin); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_snd); + + if (TRUE == osmv_rmpp_is_abort_stop(p_mad)) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_rmpp_snd: ERR 6502: " + "The remote side sent an ABORT/STOP indication.\n"); + osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); + goto dispatch_rmpp_snd_done; + } + + if (FALSE == osmv_rmpp_is_ack(p_mad)) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Not supposed to receive DATA packets --> dropping the MAD\n"); + goto dispatch_rmpp_snd_done; + } + + /* Continue processing the ACK */ + if (seg_num > old_wl) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_rmpp_snd: ERR 6503: " + "ACK received for a non-sent segment %d\n", seg_num); + + osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, + IB_RMPP_TYPE_ABORT, IB_RMPP_STATUS_S2B); + + osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); + goto dispatch_rmpp_snd_done; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_dispatch_rmpp_snd: " + "New WL = %u Old WL = %u Total Segs = %u\n", + new_wl, old_wl, total_segs); + + if (new_wl < old_wl) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_rmpp_snd: ERR 6508: " + "The receiver requests a smaller WL (%d) than before (%d)\n", + new_wl, old_wl); + + osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, + IB_RMPP_TYPE_ABORT, IB_RMPP_STATUS_W2S); + + osmv_rmpp_snd_error(p_send_ctx, IB_REMOTE_ERROR); + goto dispatch_rmpp_snd_done; + } + + /* Update the sender's window, and optionally wake up the sender thread + * Note! A single ACK can acknowledge a whole range of segments: [WF..SEG_NUM] + */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "ACK for seg_num #%d accepted.\n", seg_num); + + if (seg_num == old_wl) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The send window [%d:%d] is totally acknowledged.\n", + p_send_ctx->window_first, old_wl); + + p_send_ctx->window_first = seg_num + 1; + p_send_ctx->window_last = + (new_wl < total_segs) ? new_wl : total_segs; + + /* Remove the response timeout event for the window */ + osmv_txn_remove_timeout_ev(h_bind, osmv_txn_get_key(p_txn)); + + /* Wake up the sending thread */ + cl_event_signal(&p_send_ctx->event); + } + + dispatch_rmpp_snd_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /* @@ -498,165 +481,159 @@ __osmv_dispatch_rmpp_snd(IN osm_bind_handle_t h_bind, */ static ib_api_status_t -__osmv_dispatch_rmpp_rcv(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr) +__osmv_dispatch_rmpp_rcv(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr) { - ib_api_status_t status = IB_SUCCESS; - osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - boolean_t is_last1=FALSE, is_last2=FALSE; - osm_madw_t *p_new_madw=NULL, *p_req_madw=NULL; - ib_mad_t *p_mad_buf; - uint32_t size = 0; - uint64_t key = osmv_txn_get_key(p_txn); - uint64_t tid = osmv_txn_get_tid(p_txn); - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_rcv); - - if (TRUE == osmv_rmpp_is_ack(p_mad)) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Not supposed to receive ACK's --> dropping the MAD\n"); - - goto dispatch_rmpp_rcv_done; - } - - if (TRUE == osmv_rmpp_is_abort_stop(p_mad)) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_dispatch_rmpp_rcv: ERR 6504: " - "The Remote Side stopped sending\n"); - - status = IB_REMOTE_ERROR; - goto dispatch_rmpp_rcv_done; - } - - status = __osmv_dispatch_accept_seg(h_bind, p_txn, p_mad); - switch (status) { - - case IB_SUCCESS: - - /* Check wheter this is the legal last MAD */ - /* Criteria #1: the received MAD is marked last */ - is_last1 = osmv_rmpp_is_last(p_mad); - - /* Criteria #2: the total accumulated length hits the advertised one */ - is_last2 = is_last1; - - size = osmv_rmpp_recv_ctx_get_byte_num_from_first(p_recv_ctx); - if (size > 0) - { - is_last2 = (osmv_rmpp_recv_ctx_get_cur_byte_num(p_recv_ctx) >= size); - } - - if (is_last1 != is_last2) - { - - osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, - IB_RMPP_TYPE_ABORT, - IB_RMPP_STATUS_BAD_LEN); - - status = IB_ERROR; - goto dispatch_rmpp_rcv_done; - } - - /* TBD Consider an optimization - sending an ACK - * only for the last segment in the window - */ - __osmv_dispatch_send_ack(h_bind, p_mad, p_txn, p_mad_addr); - break; - - case IB_INSUFFICIENT_RESOURCES: - /* An out-of-order segment received. Send the ACK anyway */ - __osmv_dispatch_send_ack(h_bind, p_mad, p_txn, p_mad_addr); - status = IB_SUCCESS; - goto dispatch_rmpp_rcv_done; - - case IB_INSUFFICIENT_MEMORY: - osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, - IB_RMPP_TYPE_STOP, - IB_RMPP_STATUS_RESX); - goto dispatch_rmpp_rcv_done; - - default: - /* Illegal return code */ - CL_ASSERT(FALSE); - } - - if (TRUE != is_last1) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "RMPP MADW assembly continues, TID=0x%llX\n", tid); - goto dispatch_rmpp_rcv_done; - } - - /* This is the last packet. */ - if (0 == size) - { - /* The total size was not advertised in the first packet */ - size = osmv_rmpp_recv_ctx_get_byte_num_from_last(p_recv_ctx); - } - - /* - NOTE: the received mad might not be >= 256 bytes. - some MADs might contain several SA records but still be - less then a full MAD. - We have to use RMPP to send them over since on a regular - "simple" MAD there is no way to know how many records were sent - */ - - /* Build the MAD wrapper to be returned to the user. - * The actual storage for the MAD is allocated there. - */ - p_new_madw = osm_mad_pool_get(p_bo->p_osm_pool, h_bind, size, p_mad_addr); - if (NULL == p_new_madw) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_rmpp_rcv: ERR 6506: " - "Out Of Memory - could not allocate %d bytes for the MADW\n", - size); - - status = IB_INSUFFICIENT_MEMORY; - goto dispatch_rmpp_rcv_done; - } - - p_req_madw = osmv_txn_get_madw(p_txn); - p_mad_buf = osm_madw_get_mad_ptr(p_new_madw); - status = osmv_rmpp_recv_ctx_reassemble_arbt_mad(p_recv_ctx, size, - (uint8_t*)p_mad_buf); - if (IB_SUCCESS != status) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_dispatch_rmpp_rcv: ERR 6507: " - "Internal error - could not reassemble the result MAD\n"); - goto dispatch_rmpp_rcv_done; /* What can happen here? */ - } - - /* The MAD is assembled, we are about to apply the callback. - * Delete the transaction context, unless the transaction is double sided */ - if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn) - || - FALSE == osmv_mad_is_multi_resp(p_mad)) { - - osmv_txn_done(h_bind, key, FALSE); - } - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "RMPP MADW %p assembly complete, TID=0x%llX\n", p_new_madw, tid); - - p_mad_buf->trans_id = cl_hton64(tid); - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Restoring the original TID to 0x%llX\n", - cl_ntoh64(p_mad_buf->trans_id)); - - /* Finally, do the job! */ - p_bo->recv_cb(p_new_madw, p_bo->cb_context, p_req_madw); - - dispatch_rmpp_rcv_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - return status; + ib_api_status_t status = IB_SUCCESS; + osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + boolean_t is_last1 = FALSE, is_last2 = FALSE; + osm_madw_t *p_new_madw = NULL, *p_req_madw = NULL; + ib_mad_t *p_mad_buf; + uint32_t size = 0; + uint64_t key = osmv_txn_get_key(p_txn); + uint64_t tid = osmv_txn_get_tid(p_txn); + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_rmpp_rcv); + + if (TRUE == osmv_rmpp_is_ack(p_mad)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Not supposed to receive ACK's --> dropping the MAD\n"); + + goto dispatch_rmpp_rcv_done; + } + + if (TRUE == osmv_rmpp_is_abort_stop(p_mad)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_dispatch_rmpp_rcv: ERR 6504: " + "The Remote Side stopped sending\n"); + + status = IB_REMOTE_ERROR; + goto dispatch_rmpp_rcv_done; + } + + status = __osmv_dispatch_accept_seg(h_bind, p_txn, p_mad); + switch (status) { + + case IB_SUCCESS: + + /* Check wheter this is the legal last MAD */ + /* Criteria #1: the received MAD is marked last */ + is_last1 = osmv_rmpp_is_last(p_mad); + + /* Criteria #2: the total accumulated length hits the advertised one */ + is_last2 = is_last1; + + size = osmv_rmpp_recv_ctx_get_byte_num_from_first(p_recv_ctx); + if (size > 0) { + is_last2 = + (osmv_rmpp_recv_ctx_get_cur_byte_num(p_recv_ctx) >= + size); + } + + if (is_last1 != is_last2) { + + osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, + IB_RMPP_TYPE_ABORT, + IB_RMPP_STATUS_BAD_LEN); + + status = IB_ERROR; + goto dispatch_rmpp_rcv_done; + } + + /* TBD Consider an optimization - sending an ACK + * only for the last segment in the window + */ + __osmv_dispatch_send_ack(h_bind, p_mad, p_txn, p_mad_addr); + break; + + case IB_INSUFFICIENT_RESOURCES: + /* An out-of-order segment received. Send the ACK anyway */ + __osmv_dispatch_send_ack(h_bind, p_mad, p_txn, p_mad_addr); + status = IB_SUCCESS; + goto dispatch_rmpp_rcv_done; + + case IB_INSUFFICIENT_MEMORY: + osmv_rmpp_send_nak(h_bind, p_mad, p_mad_addr, + IB_RMPP_TYPE_STOP, IB_RMPP_STATUS_RESX); + goto dispatch_rmpp_rcv_done; + + default: + /* Illegal return code */ + CL_ASSERT(FALSE); + } + + if (TRUE != is_last1) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "RMPP MADW assembly continues, TID=0x%llX\n", tid); + goto dispatch_rmpp_rcv_done; + } + + /* This is the last packet. */ + if (0 == size) { + /* The total size was not advertised in the first packet */ + size = osmv_rmpp_recv_ctx_get_byte_num_from_last(p_recv_ctx); + } + + /* + NOTE: the received mad might not be >= 256 bytes. + some MADs might contain several SA records but still be + less then a full MAD. + We have to use RMPP to send them over since on a regular + "simple" MAD there is no way to know how many records were sent + */ + + /* Build the MAD wrapper to be returned to the user. + * The actual storage for the MAD is allocated there. + */ + p_new_madw = + osm_mad_pool_get(p_bo->p_osm_pool, h_bind, size, p_mad_addr); + if (NULL == p_new_madw) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_rmpp_rcv: ERR 6506: " + "Out Of Memory - could not allocate %d bytes for the MADW\n", + size); + + status = IB_INSUFFICIENT_MEMORY; + goto dispatch_rmpp_rcv_done; + } + + p_req_madw = osmv_txn_get_madw(p_txn); + p_mad_buf = osm_madw_get_mad_ptr(p_new_madw); + status = osmv_rmpp_recv_ctx_reassemble_arbt_mad(p_recv_ctx, size, + (uint8_t *) p_mad_buf); + if (IB_SUCCESS != status) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_dispatch_rmpp_rcv: ERR 6507: " + "Internal error - could not reassemble the result MAD\n"); + goto dispatch_rmpp_rcv_done; /* What can happen here? */ + } + + /* The MAD is assembled, we are about to apply the callback. + * Delete the transaction context, unless the transaction is double sided */ + if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn) + || FALSE == osmv_mad_is_multi_resp(p_mad)) { + + osmv_txn_done(h_bind, key, FALSE); + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "RMPP MADW %p assembly complete, TID=0x%llX\n", p_new_madw, + tid); + + p_mad_buf->trans_id = cl_hton64(tid); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Restoring the original TID to 0x%llX\n", + cl_ntoh64(p_mad_buf->trans_id)); + + /* Finally, do the job! */ + p_bo->recv_cb(p_new_madw, p_bo->cb_context, p_req_madw); + + dispatch_rmpp_rcv_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return status; } /* @@ -667,42 +644,38 @@ __osmv_dispatch_rmpp_rcv(IN osm_bind_handle_t h_bind, */ static ib_api_status_t -__osmv_dispatch_accept_seg(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN const ib_mad_t *p_mad) +__osmv_dispatch_accept_seg(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, IN const ib_mad_t * p_mad) { - ib_api_status_t ret = IB_SUCCESS; - uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t*)p_mad)->seg_num); - osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - uint64_t tid = osmv_txn_get_tid(p_txn); - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_dispatch_accept_seg); - - if (seg_num != p_recv_ctx->expected_seg) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "TID 0x%llX: can't accept this segment (%d) - " - "this is a Go-Back-N implementation\n", - tid, seg_num); - return IB_INSUFFICIENT_RESOURCES; - } - - /* Store the packet's copy in the reassembly list. - * Promote the expected segment counter. - */ - ret = osmv_rmpp_recv_ctx_store_mad_seg(p_recv_ctx, (uint8_t*)p_mad); - if (IB_SUCCESS != ret) - { - return ret; - } - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "TID 0x%llX: segment %d accepted\n", tid, seg_num); - p_recv_ctx->expected_seg = seg_num+1; - - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); - return IB_SUCCESS; + ib_api_status_t ret = IB_SUCCESS; + uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t *) p_mad)->seg_num); + osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + uint64_t tid = osmv_txn_get_tid(p_txn); + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_dispatch_accept_seg); + + if (seg_num != p_recv_ctx->expected_seg) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "TID 0x%llX: can't accept this segment (%d) - " + "this is a Go-Back-N implementation\n", tid, seg_num); + return IB_INSUFFICIENT_RESOURCES; + } + + /* Store the packet's copy in the reassembly list. + * Promote the expected segment counter. + */ + ret = osmv_rmpp_recv_ctx_store_mad_seg(p_recv_ctx, (uint8_t *) p_mad); + if (IB_SUCCESS != ret) { + return ret; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "TID 0x%llX: segment %d accepted\n", tid, seg_num); + p_recv_ctx->expected_seg = seg_num + 1; + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return IB_SUCCESS; } /* @@ -716,22 +689,22 @@ __osmv_dispatch_accept_seg(IN osm_bind_handle_t h_bind, */ static void -__osmv_dispatch_send_ack(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN osmv_txn_ctx_t *p_txn, - IN const osm_mad_addr_t *p_mad_addr) +__osmv_dispatch_send_ack(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN osmv_txn_ctx_t * p_txn, + IN const osm_mad_addr_t * p_mad_addr) { - osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); + osmv_rmpp_recv_ctx_t *p_recv_ctx = osmv_txn_get_rmpp_recv_ctx(p_txn); - /* ACK the segment # that was accepted */ - uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t*)p_req_mad)->seg_num); + /* ACK the segment # that was accepted */ + uint32_t seg_num = cl_ntoh32(((ib_rmpp_mad_t *) p_req_mad)->seg_num); - /* NOTE! The receiver can publish the New Window Last (NWL) value - * that is greater than the total number of segments to be sent. - * It's the sender's responsibility to compute the correct number - * of segments to send in the next burst. - */ - uint32_t nwl = p_recv_ctx->expected_seg + OSMV_RMPP_RECV_WIN - 1; + /* NOTE! The receiver can publish the New Window Last (NWL) value + * that is greater than the total number of segments to be sent. + * It's the sender's responsibility to compute the correct number + * of segments to send in the next burst. + */ + uint32_t nwl = p_recv_ctx->expected_seg + OSMV_RMPP_RECV_WIN - 1; - osmv_rmpp_send_ack(h_bind, p_req_mad, seg_num, nwl, p_mad_addr); + osmv_rmpp_send_ack(h_bind, p_req_mad, seg_num, nwl, p_mad_addr); } diff --git a/opensm/libvendor/osm_vendor_mlx_hca.c b/opensm/libvendor/osm_vendor_mlx_hca.c index 3e011fb..2bbb995 100644 --- a/opensm/libvendor/osm_vendor_mlx_hca.c +++ b/opensm/libvendor/osm_vendor_mlx_hca.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,34 +55,31 @@ * ********************************************************************************/ -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; +} osm_ca_info_t; /********************************************************************** * Convert the given GID to GUID by copy of it's upper 8 bytes **********************************************************************/ ib_api_status_t -__osm_vendor_gid_to_guid( IN u_int8_t * gid, - OUT VAPI_gid_t * guid ) +__osm_vendor_gid_to_guid(IN u_int8_t * gid, OUT VAPI_gid_t * guid) { - memcpy( guid, gid + 8, 8 ); - return ( IB_SUCCESS ); + memcpy(guid, gid + 8, 8); + return (IB_SUCCESS); } /********************************************************************** * Returns a pointer to the port attribute of the specified port * owned by this CA. ************************************************************************/ -static ib_port_attr_t * -__osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +static ib_port_attr_t *__osm_ca_info_get_port_attr_ptr(IN const osm_ca_info_t * + const p_ca_info, + IN const uint8_t index) { - return ( &p_ca_info->p_attr->p_port_attr[index] ); + return (&p_ca_info->p_attr->p_port_attr[index]); } /******************************************************************************** @@ -91,223 +87,207 @@ __osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, * NOTE: user of this function needs to deallocate p_hca_ids after usage. ********************************************************************************/ static ib_api_status_t -__osm_vendor_get_ca_ids( IN osm_vendor_t * const p_vend, - IN VAPI_hca_id_t ** const p_hca_ids, - IN uint32_t * const p_num_guids ) +__osm_vendor_get_ca_ids(IN osm_vendor_t * const p_vend, + IN VAPI_hca_id_t ** const p_hca_ids, + IN uint32_t * const p_num_guids) { - ib_api_status_t status; - VAPI_ret_t vapi_res; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_get_ca_ids ); - - CL_ASSERT( p_hca_ids ); - CL_ASSERT( p_num_guids ); - - /* first call is just to get the number */ - vapi_res = EVAPI_list_hcas( 0, p_num_guids, NULL ); - - /* fail ? */ - if( vapi_res == VAPI_EINVAL_PARAM ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 3D08: : " - "Bad parameter in calling: EVAPI_list_hcas. (%d)\n", - vapi_res ); - status = IB_ERROR; - goto Exit; - } - - /* NO HCA ? */ - if( *p_num_guids == 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 3D09: " - "No available channel adapters.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* allocate and really call - user of this function needs to deallocate it */ - *p_hca_ids = - ( VAPI_hca_id_t * ) malloc( *p_num_guids * - sizeof( VAPI_hca_id_t ) ); - - /* now call it really */ - vapi_res = EVAPI_list_hcas( *p_num_guids, p_num_guids, *p_hca_ids ); - - /* too many ? */ - if( vapi_res == VAPI_EAGAIN ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 3D10: " - "More CA GUIDs than allocated array (%d).\n", *p_num_guids ); - status = IB_ERROR; - goto Exit; - } - - /* fail ? */ - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 3D11: : " - "Bad parameter in calling: EVAPI_list_hcas.\n" ); - status = IB_ERROR; - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_vendor_get_ca_ids: " - "Detected %u local channel adapters.\n", *p_num_guids ); - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + VAPI_ret_t vapi_res; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_get_ca_ids); + + CL_ASSERT(p_hca_ids); + CL_ASSERT(p_num_guids); + + /* first call is just to get the number */ + vapi_res = EVAPI_list_hcas(0, p_num_guids, NULL); + + /* fail ? */ + if (vapi_res == VAPI_EINVAL_PARAM) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 3D08: : " + "Bad parameter in calling: EVAPI_list_hcas. (%d)\n", + vapi_res); + status = IB_ERROR; + goto Exit; + } + + /* NO HCA ? */ + if (*p_num_guids == 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 3D09: " + "No available channel adapters.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* allocate and really call - user of this function needs to deallocate it */ + *p_hca_ids = + (VAPI_hca_id_t *) malloc(*p_num_guids * sizeof(VAPI_hca_id_t)); + + /* now call it really */ + vapi_res = EVAPI_list_hcas(*p_num_guids, p_num_guids, *p_hca_ids); + + /* too many ? */ + if (vapi_res == VAPI_EAGAIN) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 3D10: " + "More CA GUIDs than allocated array (%d).\n", + *p_num_guids); + status = IB_ERROR; + goto Exit; + } + + /* fail ? */ + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 3D11: : " + "Bad parameter in calling: EVAPI_list_hcas.\n"); + status = IB_ERROR; + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_vendor_get_ca_ids: " + "Detected %u local channel adapters.\n", *p_num_guids); + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } - /********************************************************************** * Initialize an Info Struct for the Given HCA by its Id **********************************************************************/ static ib_api_status_t -__osm_ca_info_init( IN osm_vendor_t * const p_vend, - IN VAPI_hca_id_t ca_id, - OUT osm_ca_info_t * const p_ca_info ) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + IN VAPI_hca_id_t ca_id, OUT osm_ca_info_t * const p_ca_info) { - ib_api_status_t status = IB_ERROR; - VAPI_ret_t vapi_res; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_vendor_t hca_vendor; - VAPI_hca_cap_t hca_cap; - VAPI_hca_port_t hca_port; - uint8_t port_num; - IB_gid_t *p_port_gid; - uint16_t maxNumGids; - - OSM_LOG_ENTER( p_vend->p_log, __osm_ca_info_init ); - - /* get the HCA handle */ - vapi_res = EVAPI_get_hca_hndl( ca_id, &hca_hndl ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3D05: " - "Fail to get HCA handle (%u).\n", vapi_res ); - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_ca_info_init: " "Querying CA %s.\n", ca_id ); - } - - /* query and get the HCA capability */ - vapi_res = VAPI_query_hca_cap( hca_hndl, &hca_vendor, &hca_cap ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3D06: " - "Fail to get HCA Capabilities (%u).\n", vapi_res ); - goto Exit; - } - - /* get the guid of the HCA */ - memcpy( &( p_ca_info->guid ), hca_cap.node_guid, 8 * sizeof( u_int8_t ) ); - p_ca_info->attr_size = 1; - p_ca_info->p_attr = - ( ib_ca_attr_t * ) malloc( sizeof( ib_ca_attr_t ) ); - memcpy( &( p_ca_info->p_attr->ca_guid ), hca_cap.node_guid, - 8 * sizeof( u_int8_t ) ); - - /* now obtain the attributes of the ports */ - p_ca_info->p_attr->num_ports = hca_cap.phys_port_num; - p_ca_info->p_attr->p_port_attr = - ( ib_port_attr_t * ) malloc( hca_cap.phys_port_num * - sizeof( ib_port_attr_t ) ); - - for( port_num = 0; port_num < p_ca_info->p_attr->num_ports; port_num++ ) - { - - /* query the port attributes */ - vapi_res = - VAPI_query_hca_port_prop( hca_hndl, port_num + 1, &hca_port ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3D07: " - "Fail to get HCA Port Attributes (%d).\n", vapi_res ); - goto Exit; - } - - /* first call to know the size of the gid table */ - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, port_num + 1, 0, &maxNumGids, - NULL ); - p_port_gid = - ( IB_gid_t * ) malloc( maxNumGids * sizeof( IB_gid_t ) ); - - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, port_num + 1, maxNumGids, - &maxNumGids, p_port_gid ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 3D12: " - "Fail to get HCA Port GID (%d).\n", vapi_res ); - goto Exit; - } - - __osm_vendor_gid_to_guid( p_port_gid[0], - ( IB_gid_t * ) & p_ca_info->p_attr-> - p_port_attr[port_num].port_guid ); - p_ca_info->p_attr->p_port_attr[port_num].lid = hca_port.lid; - p_ca_info->p_attr->p_port_attr[port_num].link_state = hca_port.state; - p_ca_info->p_attr->p_port_attr[port_num].sm_lid = hca_port.sm_lid; - - free( p_port_gid ); - } - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_ERROR; + VAPI_ret_t vapi_res; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_vendor_t hca_vendor; + VAPI_hca_cap_t hca_cap; + VAPI_hca_port_t hca_port; + uint8_t port_num; + IB_gid_t *p_port_gid; + uint16_t maxNumGids; + + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); + + /* get the HCA handle */ + vapi_res = EVAPI_get_hca_hndl(ca_id, &hca_hndl); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3D05: " + "Fail to get HCA handle (%u).\n", vapi_res); + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_ca_info_init: " "Querying CA %s.\n", ca_id); + } + + /* query and get the HCA capability */ + vapi_res = VAPI_query_hca_cap(hca_hndl, &hca_vendor, &hca_cap); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3D06: " + "Fail to get HCA Capabilities (%u).\n", vapi_res); + goto Exit; + } + + /* get the guid of the HCA */ + memcpy(&(p_ca_info->guid), hca_cap.node_guid, 8 * sizeof(u_int8_t)); + p_ca_info->attr_size = 1; + p_ca_info->p_attr = (ib_ca_attr_t *) malloc(sizeof(ib_ca_attr_t)); + memcpy(&(p_ca_info->p_attr->ca_guid), hca_cap.node_guid, + 8 * sizeof(u_int8_t)); + + /* now obtain the attributes of the ports */ + p_ca_info->p_attr->num_ports = hca_cap.phys_port_num; + p_ca_info->p_attr->p_port_attr = + (ib_port_attr_t *) malloc(hca_cap.phys_port_num * + sizeof(ib_port_attr_t)); + + for (port_num = 0; port_num < p_ca_info->p_attr->num_ports; port_num++) { + + /* query the port attributes */ + vapi_res = + VAPI_query_hca_port_prop(hca_hndl, port_num + 1, &hca_port); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3D07: " + "Fail to get HCA Port Attributes (%d).\n", + vapi_res); + goto Exit; + } + + /* first call to know the size of the gid table */ + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, port_num + 1, 0, + &maxNumGids, NULL); + p_port_gid = (IB_gid_t *) malloc(maxNumGids * sizeof(IB_gid_t)); + + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, port_num + 1, maxNumGids, + &maxNumGids, p_port_gid); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 3D12: " + "Fail to get HCA Port GID (%d).\n", vapi_res); + goto Exit; + } + + __osm_vendor_gid_to_guid(p_port_gid[0], + (IB_gid_t *) & p_ca_info->p_attr-> + p_port_attr[port_num].port_guid); + p_ca_info->p_attr->p_port_attr[port_num].lid = hca_port.lid; + p_ca_info->p_attr->p_port_attr[port_num].link_state = + hca_port.state; + p_ca_info->p_attr->p_port_attr[port_num].sm_lid = + hca_port.sm_lid; + + free(p_port_gid); + } + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -osm_ca_info_destroy( IN osm_vendor_t * const p_vend, - IN osm_ca_info_t * const p_ca_info, - IN uint8_t num_ca) +osm_ca_info_destroy(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info, IN uint8_t num_ca) { - osm_ca_info_t *p_ca; - uint8_t i; + osm_ca_info_t *p_ca; + uint8_t i; - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_destroy ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_destroy); - for (i=0; i < num_ca; i++) - { - p_ca = &p_ca_info[i]; + for (i = 0; i < num_ca; i++) { + p_ca = &p_ca_info[i]; - if( NULL != p_ca->p_attr ) - { - if(0 != p_ca->p_attr->num_ports) - { - free( p_ca->p_attr->p_port_attr ); - } + if (NULL != p_ca->p_attr) { + if (0 != p_ca->p_attr->num_ports) { + free(p_ca->p_attr->p_port_attr); + } - free( p_ca->p_attr); - } - } + free(p_ca->p_attr); + } + } - free( p_ca_info ); + free(p_ca_info); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -317,108 +297,100 @@ osm_ca_info_destroy( IN osm_vendor_t * const p_vend, * Update the vendor object list of ca_info structs **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * const p_attr_array, - IN uint32_t * const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_api_status_t status; - - uint32_t ca; - uint32_t ca_count = 0; - uint32_t port_count = 0; - uint8_t port_num; - uint32_t total_ports = 0; - VAPI_hca_id_t *p_ca_ids = NULL; - osm_ca_info_t *p_ca_infos = NULL; - uint32_t attr_array_sz = *p_num_ports; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); - - CL_ASSERT( p_vend ); - - - /* determine the number of CA's */ - status = __osm_vendor_get_ca_ids( p_vend, &p_ca_ids, &ca_count ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 3D13: " - "Fail to get CA Ids.\n" ); - goto Exit; - } - - /* Allocate an array big enough to hold the ca info objects*/ - p_ca_infos = malloc( ca_count * sizeof( osm_ca_info_t ) ); - if( p_ca_infos == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 3D14: " - "Unable to allocate CA information array.\n" ); - goto Exit; - } - - memset( p_ca_infos, 0, ca_count * sizeof( osm_ca_info_t ) ); - - /* - * For each CA, retrieve the CA info attributes - */ - for( ca = 0; ca < ca_count; ca++ ) - { - status = __osm_ca_info_init( p_vend, p_ca_ids[ca], &p_ca_infos[ca] ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 3D15: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - total_ports += p_ca_infos[ca].p_attr->num_ports; - } - - *p_num_ports = total_ports; - osm_log(p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_all_port_attr: total ports:%u \n",total_ports); - - /* - * If the user supplied enough storage, return the port guids, - * otherwise, return the appropriate error. - */ - if( attr_array_sz >= total_ports ) - { - for( ca = 0; ca < ca_count; ca++ ) - { - uint32_t num_ports; - - num_ports = p_ca_infos[ca].p_attr->num_ports; - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_attr_array[port_count] = - *__osm_ca_info_get_port_attr_ptr( &p_ca_infos[ca], port_num ); - port_count++; - } - } - } - else - { - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - if( p_ca_ids ) - free( p_ca_ids ); - - if ( p_ca_infos ) - { - osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); - } - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + + uint32_t ca; + uint32_t ca_count = 0; + uint32_t port_count = 0; + uint8_t port_num; + uint32_t total_ports = 0; + VAPI_hca_id_t *p_ca_ids = NULL; + osm_ca_info_t *p_ca_infos = NULL; + uint32_t attr_array_sz = *p_num_ports; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + status = __osm_vendor_get_ca_ids(p_vend, &p_ca_ids, &ca_count); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 3D13: " + "Fail to get CA Ids.\n"); + goto Exit; + } + + /* Allocate an array big enough to hold the ca info objects */ + p_ca_infos = malloc(ca_count * sizeof(osm_ca_info_t)); + if (p_ca_infos == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 3D14: " + "Unable to allocate CA information array.\n"); + goto Exit; + } + + memset(p_ca_infos, 0, ca_count * sizeof(osm_ca_info_t)); + + /* + * For each CA, retrieve the CA info attributes + */ + for (ca = 0; ca < ca_count; ca++) { + status = + __osm_ca_info_init(p_vend, p_ca_ids[ca], &p_ca_infos[ca]); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 3D15: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + total_ports += p_ca_infos[ca].p_attr->num_ports; + } + + *p_num_ports = total_ports; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_all_port_attr: total ports:%u \n", total_ports); + + /* + * If the user supplied enough storage, return the port guids, + * otherwise, return the appropriate error. + */ + if (attr_array_sz >= total_ports) { + for (ca = 0; ca < ca_count; ca++) { + uint32_t num_ports; + + num_ports = p_ca_infos[ca].p_attr->num_ports; + + for (port_num = 0; port_num < num_ports; port_num++) { + p_attr_array[port_count] = + *__osm_ca_info_get_port_attr_ptr(&p_ca_infos + [ca], + port_num); + port_count++; + } + } + } else { + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + if (p_ca_ids) + free(p_ca_ids); + + if (p_ca_infos) { + osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); + } + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -427,131 +399,126 @@ osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, **********************************************************************/ ib_api_status_t -osm_vendor_get_guid_ca_and_port( - IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT VAPI_hca_hndl_t * p_hca_hndl, - OUT VAPI_hca_id_t * p_hca_id, - OUT uint8_t *p_hca_idx, - OUT uint32_t * p_port_num ) +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT VAPI_hca_hndl_t * p_hca_hndl, + OUT VAPI_hca_id_t * p_hca_id, + OUT uint8_t * p_hca_idx, + OUT uint32_t * p_port_num) { - ib_api_status_t status; - VAPI_hca_id_t *p_ca_ids = NULL; - VAPI_ret_t vapi_res; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_vendor_t hca_vendor; - VAPI_hca_cap_t hca_cap; - IB_gid_t *p_port_gid = NULL; - uint16_t maxNumGids; - ib_net64_t port_guid; - uint32_t ca, portIdx, ca_count; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_guid_ca_and_port ); - - CL_ASSERT( p_vend ); - - /* - * 1) Determine the number of CA's - * 2) Allocate an array big enough to hold the ca info objects. - * 3) Call again to retrieve the guids. - */ - status = __osm_vendor_get_ca_ids( p_vend, &p_ca_ids, &ca_count ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 3D16: " - "Fail to get CA Ids.\n" ); - goto Exit; - } - - /* - * For each CA, retrieve the CA info attributes - */ - for( ca = 0; ca < ca_count; ca++ ) - { - /* get the HCA handle */ - vapi_res = EVAPI_get_hca_hndl( p_ca_ids[ca], &hca_hndl ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 3D17: " - "Fail to get HCA handle (%u).\n", vapi_res ); - goto Exit; - } - - /* get the CA attributes - to know how many ports it has: */ - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_guid_ca_and_port: " - "Querying CA %s.\n", p_ca_ids[ca] ); - } - - /* query and get the HCA capability */ - vapi_res = VAPI_query_hca_cap( hca_hndl, &hca_vendor, &hca_cap ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 3D18: " - "Fail to get HCA Capabilities (%u).\n", vapi_res ); - goto Exit; - } - - /* go over all ports - to obtail their guids */ - for( portIdx = 0; portIdx < hca_cap.phys_port_num; portIdx++ ) - { - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, portIdx + 1, 0, &maxNumGids, - NULL ); - p_port_gid = - ( IB_gid_t * ) malloc( maxNumGids * sizeof( IB_gid_t ) ); - - /* get the port guid */ - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, portIdx + 1, maxNumGids, - &maxNumGids, p_port_gid ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 3D19: " - "Fail to get HCA Port GID (%d).\n", vapi_res ); - goto Exit; - } - - /* convert to SF style */ - __osm_vendor_gid_to_guid( p_port_gid[0], - ( VAPI_gid_t * ) & port_guid ); - - /* finally did we find it ? */ - if( port_guid == guid ) - { - *p_hca_hndl = hca_hndl; - memcpy( p_hca_id, p_ca_ids[ca], sizeof ( VAPI_hca_id_t ) ); - *p_hca_idx = ca; - *p_port_num = portIdx + 1; - status = IB_SUCCESS; - goto Exit; - } - - free( p_port_gid ); - p_port_gid = NULL; - } /* ALL PORTS */ - } /* all HCAs */ - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 3D20: " - "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", - cl_ntoh64(guid) ); - status = IB_INVALID_GUID; - - Exit: - if( p_ca_ids != NULL ) - free( p_ca_ids ); - if( p_port_gid != NULL ) - free( p_port_gid ); - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + VAPI_hca_id_t *p_ca_ids = NULL; + VAPI_ret_t vapi_res; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_vendor_t hca_vendor; + VAPI_hca_cap_t hca_cap; + IB_gid_t *p_port_gid = NULL; + uint16_t maxNumGids; + ib_net64_t port_guid; + uint32_t ca, portIdx, ca_count; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_guid_ca_and_port); + + CL_ASSERT(p_vend); + + /* + * 1) Determine the number of CA's + * 2) Allocate an array big enough to hold the ca info objects. + * 3) Call again to retrieve the guids. + */ + status = __osm_vendor_get_ca_ids(p_vend, &p_ca_ids, &ca_count); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 3D16: " + "Fail to get CA Ids.\n"); + goto Exit; + } + + /* + * For each CA, retrieve the CA info attributes + */ + for (ca = 0; ca < ca_count; ca++) { + /* get the HCA handle */ + vapi_res = EVAPI_get_hca_hndl(p_ca_ids[ca], &hca_hndl); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 3D17: " + "Fail to get HCA handle (%u).\n", vapi_res); + goto Exit; + } + + /* get the CA attributes - to know how many ports it has: */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_guid_ca_and_port: " + "Querying CA %s.\n", p_ca_ids[ca]); + } + + /* query and get the HCA capability */ + vapi_res = VAPI_query_hca_cap(hca_hndl, &hca_vendor, &hca_cap); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 3D18: " + "Fail to get HCA Capabilities (%u).\n", + vapi_res); + goto Exit; + } + + /* go over all ports - to obtail their guids */ + for (portIdx = 0; portIdx < hca_cap.phys_port_num; portIdx++) { + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, portIdx + 1, 0, + &maxNumGids, NULL); + p_port_gid = + (IB_gid_t *) malloc(maxNumGids * sizeof(IB_gid_t)); + + /* get the port guid */ + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, portIdx + 1, + maxNumGids, &maxNumGids, + p_port_gid); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 3D19: " + "Fail to get HCA Port GID (%d).\n", + vapi_res); + goto Exit; + } + + /* convert to SF style */ + __osm_vendor_gid_to_guid(p_port_gid[0], + (VAPI_gid_t *) & port_guid); + + /* finally did we find it ? */ + if (port_guid == guid) { + *p_hca_hndl = hca_hndl; + memcpy(p_hca_id, p_ca_ids[ca], + sizeof(VAPI_hca_id_t)); + *p_hca_idx = ca; + *p_port_num = portIdx + 1; + status = IB_SUCCESS; + goto Exit; + } + + free(p_port_gid); + p_port_gid = NULL; + } /* ALL PORTS */ + } /* all HCAs */ + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 3D20: " + "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", + cl_ntoh64(guid)); + status = IB_INVALID_GUID; + + Exit: + if (p_ca_ids != NULL) + free(p_ca_ids); + if (p_port_gid != NULL) + free(p_port_gid); + OSM_LOG_EXIT(p_vend->p_log); + return (status); } #endif diff --git a/opensm/libvendor/osm_vendor_mlx_hca_anafa.c b/opensm/libvendor/osm_vendor_mlx_hca_anafa.c index 64b9f64..ca3b4ea 100644 --- a/opensm/libvendor/osm_vendor_mlx_hca_anafa.c +++ b/opensm/libvendor/osm_vendor_mlx_hca_anafa.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #if defined(OSM_VENDOR_INTF_ANAFA) #undef IN @@ -69,9 +68,9 @@ typedef struct _osm_ca_info { * Convert the given GID to GUID by copy of it's upper 8 bytes **********************************************************************/ ib_api_status_t -__osm_vendor_gid_to_guid (IN tTS_IB_GID gid, OUT ib_net64_t * p_guid) +__osm_vendor_gid_to_guid(IN tTS_IB_GID gid, OUT ib_net64_t * p_guid) { - memcpy (p_guid, gid + 8, 8); + memcpy(p_guid, gid + 8, 8); return (IB_SUCCESS); } @@ -79,8 +78,8 @@ __osm_vendor_gid_to_guid (IN tTS_IB_GID gid, OUT ib_net64_t * p_guid) * Initialize an Info Struct for the Given HCA by its Id **********************************************************************/ static ib_api_status_t -__osm_ca_info_init (IN osm_vendor_t * const p_vend, - OUT osm_ca_info_t * const p_ca_info) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + OUT osm_ca_info_t * const p_ca_info) { ib_api_status_t status = IB_ERROR; int ioctl_ret = 0; @@ -90,50 +89,49 @@ __osm_ca_info_init (IN osm_vendor_t * const p_vend, osm_ts_get_port_info_ioctl port_info; struct ib_get_dev_info_ioctl dev_info; - OSM_LOG_ENTER (p_vend->p_log, __osm_ca_info_init); + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); /* query HCA guid */ - ioctl_ret = - ioctl (p_tpot_info->device_fd, TS_IB_IOCGDEVINFO, &dev_info); + ioctl_ret = ioctl(p_tpot_info->device_fd, TS_IB_IOCGDEVINFO, &dev_info); if (ioctl_ret != 0) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7001: " - "Fail to get HCA Capabilities (%d).\n", ioctl_ret); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7001: " + "Fail to get HCA Capabilities (%d).\n", ioctl_ret); goto Exit; } - memcpy (&(p_ca_info->attr.ca_guid), dev_info.dev_info.node_guid, - 8 * sizeof (uint8_t)); + memcpy(&(p_ca_info->attr.ca_guid), dev_info.dev_info.node_guid, + 8 * sizeof(uint8_t)); /* now obtain the attributes of the ports - on our case port 1*/ p_ca_info->attr.num_ports = 1; p_ca_info->attr.p_port_attr = - (ib_port_attr_t *) malloc (1 * sizeof (ib_port_attr_t)); + (ib_port_attr_t *) malloc(1 * sizeof(ib_port_attr_t)); port_info.port = 1; ioctl_ret = - ioctl (p_tpot_info->device_fd, TS_IB_IOCGPORTINFO, &port_info); + ioctl(p_tpot_info->device_fd, TS_IB_IOCGPORTINFO, &port_info); if (ioctl_ret) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7002: " - "Fail to get HCA Port Attributes (%d).\n", ioctl_ret); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7002: " + "Fail to get HCA Port Attributes (%d).\n", ioctl_ret); goto Exit; } gid_ioctl.port = 1; gid_ioctl.index = 0; ioctl_ret = - ioctl (p_tpot_info->device_fd, TS_IB_IOCGGIDENTRY, &gid_ioctl); + ioctl(p_tpot_info->device_fd, TS_IB_IOCGGIDENTRY, &gid_ioctl); if (ioctl_ret) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7003: " - "Fail to get HCA Port GID (%d).\n", ioctl_ret); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7003: " + "Fail to get HCA Port GID (%d).\n", ioctl_ret); goto Exit; } - __osm_vendor_gid_to_guid (gid_ioctl.gid_entry, - &(p_ca_info->attr.p_port_attr[0].port_guid)); + __osm_vendor_gid_to_guid(gid_ioctl.gid_entry, + &(p_ca_info->attr.p_port_attr[0].port_guid)); p_ca_info->attr.p_port_attr[0].lid = port_info.port_info.lid; p_ca_info->attr.p_port_attr[0].link_state = port_info.port_info.port_state; @@ -141,7 +139,7 @@ __osm_ca_info_init (IN osm_vendor_t * const p_vend, status = IB_SUCCESS; Exit: - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); return (status); } @@ -153,16 +151,16 @@ __osm_ca_info_init (IN osm_vendor_t * const p_vend, * Update the vendor object list of ca_info structs **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr (IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * const p_attr_array, - IN uint32_t *const p_num_ports) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { ib_api_status_t status; osm_ca_info_t ca_info; uint32_t attr_array_sz = *p_num_ports; - OSM_LOG_ENTER (p_vend->p_log, osm_vendor_get_all_port_attr); - CL_ASSERT (p_vend); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + CL_ASSERT(p_vend); /* anafa has one port - the user didnt supply enough storage space */ if (attr_array_sz < 1) { @@ -173,23 +171,23 @@ osm_vendor_get_all_port_attr (IN osm_vendor_t * const p_vend, /* * retrieve the CA info attributes */ - status = __osm_ca_info_init (p_vend, &ca_info); + status = __osm_ca_info_init(p_vend, &ca_info); if (status != IB_SUCCESS) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 7004: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str (status)); + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 7004: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); goto Exit; } *p_num_ports = 1; - p_attr_array[0] = ca_info.attr.p_port_attr[0]; /* anafa has only one port */ + p_attr_array[0] = ca_info.attr.p_port_attr[0]; /* anafa has only one port */ status = IB_SUCCESS; Exit: - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); return (status); } diff --git a/opensm/libvendor/osm_vendor_mlx_hca_pfs.c b/opensm/libvendor/osm_vendor_mlx_hca_pfs.c index af1cdb0..275bf92 100644 --- a/opensm/libvendor/osm_vendor_mlx_hca_pfs.c +++ b/opensm/libvendor/osm_vendor_mlx_hca_pfs.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #if defined(OSM_VENDOR_INTF_MTL) | defined(OSM_VENDOR_INTF_TS) #undef IN @@ -60,24 +59,22 @@ * ********************************************************************************/ -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +} osm_ca_info_t; /********************************************************************** * Returns a pointer to the port attribute of the specified port * owned by this CA. ************************************************************************/ -static ib_port_attr_t * -__osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +static ib_port_attr_t *__osm_ca_info_get_port_attr_ptr(IN const osm_ca_info_t * + const p_ca_info, + IN const uint8_t index) { - return ( &p_ca_info->p_attr->p_port_attr[index] ); + return (&p_ca_info->p_attr->p_port_attr[index]); } /********************************************************************** @@ -85,24 +82,22 @@ __osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, **********************************************************************/ int __hca_pfs_get_num_cas() { - int num_cas = 0; - DIR *dp; - struct dirent *ep; - - dp = opendir ("/proc/infiniband/core"); - if (dp != NULL) - { - while ((ep = readdir (dp))) - { - /* CAs are directories with the format ca[1-9][0-9]* */ - if ((ep->d_type == DT_DIR) && !strncmp(ep->d_name, "ca", 2)) - { - num_cas++; - } - } - closedir(dp); - } - return num_cas; + int num_cas = 0; + DIR *dp; + struct dirent *ep; + + dp = opendir("/proc/infiniband/core"); + if (dp != NULL) { + while ((ep = readdir(dp))) { + /* CAs are directories with the format ca[1-9][0-9]* */ + if ((ep->d_type == DT_DIR) + && !strncmp(ep->d_name, "ca", 2)) { + num_cas++; + } + } + closedir(dp); + } + return num_cas; } /* @@ -116,126 +111,120 @@ int __hca_pfs_get_num_cas() FW revision: 0x300020080 */ typedef struct _pfs_ca_info { - char name[32]; - char provider[32]; - uint64_t guid; - uint8_t num_ports; - uint32_t vend_id; - uint16_t dev_id; - uint16_t rev_id; - uint64_t fw_rev; + char name[32]; + char provider[32]; + uint64_t guid; + uint8_t num_ports; + uint32_t vend_id; + uint16_t dev_id; + uint16_t rev_id; + uint64_t fw_rev; } pfs_ca_info_t; /********************************************************************** * Parse the CA Info file available in /proc/infiniband/core/caN/info **********************************************************************/ static ib_api_status_t -__parse_ca_info_file( - IN osm_vendor_t * const p_vend, - IN uint32_t idx, - OUT pfs_ca_info_t *pfs_ca_info) +__parse_ca_info_file(IN osm_vendor_t * const p_vend, + IN uint32_t idx, OUT pfs_ca_info_t * pfs_ca_info) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char *p_ch; - int g1,g2,g3,g4; - int num_ports; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __parse_ca_info_file ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_ca_info_file: " "Querying CA %d.\n", idx ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "/proc/infiniband/core/ca%d/info", idx); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5205: " - "Fail to open HCA:%d info file:(%s).\n", idx, file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - name: InfiniHost0 - provider: tavor - node GUID: 0002:c900:0120:3470 - ports: 2 - vendor ID: 0x2c9 - device ID: 0x5a44 - HW revision: 0xa1 - FW revision: 0x300020080 - */ - if (!(p_ch = strstr(file_buffer, "name:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5206: " - "Fail to obtain HCA name. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"name: %s", pfs_ca_info->name) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5207: " - "Fail to parse name in info file:(%s).\n", p_ch); - goto Exit; - } - - /* get the guid of the HCA */ - if (!(p_ch = strstr(file_buffer, "node GUID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5208: " - "Fail to obtain GUID in info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch, "node GUID: %x:%x:%x:%x", &g1,&g2,&g3,&g4) != 4) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5209: " - "Fail to parse GUID in info file:(%s).\n", p_ch); - goto Exit; - } - pfs_ca_info->guid = (uint64_t)g1 << 48 | (uint64_t)g1 << 32 - | (uint64_t)g1 << 16 | (uint64_t)g3 ; - - /* obtain number of ports */ - if (!(p_ch = strstr(file_buffer, "ports:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5210: " - "Fail to obtain number of ports in info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch, "ports: %d", &num_ports) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5211: " - "Fail to parse num ports in info file:(%s).\n", p_ch); - goto Exit; - } - pfs_ca_info->num_ports = num_ports; - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_ca_info_file: " - "CA1 = name:%s guid:0x%016llx ports:%d\n", - pfs_ca_info->name, pfs_ca_info->guid, pfs_ca_info->num_ports - ); - - status = IB_SUCCESS; -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char *p_ch; + int g1, g2, g3, g4; + int num_ports; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __parse_ca_info_file); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_ca_info_file: " "Querying CA %d.\n", idx); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "/proc/infiniband/core/ca%d/info", idx); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5205: " + "Fail to open HCA:%d info file:(%s).\n", idx, + file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + name: InfiniHost0 + provider: tavor + node GUID: 0002:c900:0120:3470 + ports: 2 + vendor ID: 0x2c9 + device ID: 0x5a44 + HW revision: 0xa1 + FW revision: 0x300020080 + */ + if (!(p_ch = strstr(file_buffer, "name:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5206: " + "Fail to obtain HCA name. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "name: %s", pfs_ca_info->name) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5207: " + "Fail to parse name in info file:(%s).\n", p_ch); + goto Exit; + } + + /* get the guid of the HCA */ + if (!(p_ch = strstr(file_buffer, "node GUID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5208: " + "Fail to obtain GUID in info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "node GUID: %x:%x:%x:%x", &g1, &g2, &g3, &g4) != 4) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5209: " + "Fail to parse GUID in info file:(%s).\n", p_ch); + goto Exit; + } + pfs_ca_info->guid = (uint64_t) g1 << 48 | (uint64_t) g1 << 32 + | (uint64_t) g1 << 16 | (uint64_t) g3; + + /* obtain number of ports */ + if (!(p_ch = strstr(file_buffer, "ports:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5210: " + "Fail to obtain number of ports in info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "ports: %d", &num_ports) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5211: " + "Fail to parse num ports in info file:(%s).\n", p_ch); + goto Exit; + } + pfs_ca_info->num_ports = num_ports; + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_ca_info_file: " + "CA1 = name:%s guid:0x%016llx ports:%d\n", + pfs_ca_info->name, pfs_ca_info->guid, pfs_ca_info->num_ports); + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /* @@ -254,11 +243,11 @@ Exit: IsCapabilityMaskNoticeSupported */ typedef struct _pfs_port_info { - uint8_t state; - uint16_t lid; - uint8_t lmc; - uint16_t sm_lid; - uint8_t sm_sl; + uint8_t state; + uint16_t lid; + uint8_t lmc; + uint16_t sm_lid; + uint8_t sm_sl; } pfs_port_info_t; /********************************************************************** @@ -266,156 +255,150 @@ typedef struct _pfs_port_info { * Port num is 1..N **********************************************************************/ static ib_api_status_t -__parse_port_info_file( - IN osm_vendor_t * const p_vend, - IN uint32_t hca_idx, - IN uint8_t port_num, - OUT pfs_port_info_t *pfs_port_info) +__parse_port_info_file(IN osm_vendor_t * const p_vend, + IN uint32_t hca_idx, + IN uint8_t port_num, OUT pfs_port_info_t * pfs_port_info) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char state[12]; - char *p_ch; - int lid, sm_lid, lmc, sm_sl; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __parse_port_info_file ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_port_info_file: " - "Parsing Proc File System Port Info CA %d Port %d.\n", hca_idx, port_num ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "/proc/infiniband/core/ca%d/port%d/info", hca_idx, port_num); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5212: " - "Fail to open HCA:%d Port:%d info file:(%s).\n", hca_idx, port_num, - file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - state: ACTIVE - LID: 0x0001 - LMC: 0x0000 - SM LID: 0x0001 - SM SL: 0x0000 - ... - */ - if (!(p_ch = strstr(file_buffer, "state:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5213: " - "Fail to obtain port state. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"state: %s", state) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5214: " - "Fail to parse state from info file:(%s).\n", p_ch); - goto Exit; - } - - if (!strcmp(state, "ACTIVE")) - pfs_port_info->state = IB_LINK_ACTIVE; - else if (!strcmp(state, "DOWN")) - pfs_port_info->state = IB_LINK_DOWN; - else if (!strcmp(state, "INIT")) - pfs_port_info->state = IB_LINK_INIT; - else if (!strcmp(state, "ARMED")) - pfs_port_info->state = IB_LINK_ARMED; - else - pfs_port_info->state = 0; - - /* get lid */ - if (!(p_ch = strstr(file_buffer, "LID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5215: " - "Fail to obtain port lid. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"LID: %x", &lid) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5216: " - "Fail to parse lid from info file:(%s).\n", p_ch); - goto Exit; - } - pfs_port_info->lid = lid; - /* get LMC */ - if (!(p_ch = strstr(file_buffer, "LMC:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5217: " - "Fail to obtain port LMC. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"LMC: %x", &lmc) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5218: " - "Fail to parse LMC from info file:(%s).\n", p_ch); - goto Exit; - } - pfs_port_info->lmc = lmc; - - /* get SM LID */ - if (!(p_ch = strstr(file_buffer, "SM LID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5219: " - "Fail to obtain port SM LID. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"SM LID: %x", &sm_lid) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5220: " - "Fail to parse SM LID from info file:(%s).\n", p_ch); - goto Exit; - } - pfs_port_info->sm_lid = sm_lid; - - /* get SM LID */ - if (!(p_ch = strstr(file_buffer, "SM SL:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5221: " - "Fail to obtain port SM SL. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"SM SL: %x", &sm_sl) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5222: " - "Fail to parse SM SL from info file:(%s).\n", p_ch); - goto Exit; - } - pfs_port_info->sm_sl = sm_sl; - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_port_info_file: " - "Obtained Port:%d = state:%d, lid:0x%04X, lmc:%d, sm_lid:0x%04X, sm_sl:%d\n", - port_num, pfs_port_info->state, pfs_port_info->lid, - pfs_port_info->lmc, pfs_port_info->sm_lid, pfs_port_info->sm_sl - ); - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char state[12]; + char *p_ch; + int lid, sm_lid, lmc, sm_sl; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __parse_port_info_file); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_port_info_file: " + "Parsing Proc File System Port Info CA %d Port %d.\n", hca_idx, + port_num); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "/proc/infiniband/core/ca%d/port%d/info", hca_idx, + port_num); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5212: " + "Fail to open HCA:%d Port:%d info file:(%s).\n", + hca_idx, port_num, file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + state: ACTIVE + LID: 0x0001 + LMC: 0x0000 + SM LID: 0x0001 + SM SL: 0x0000 + ... + */ + if (!(p_ch = strstr(file_buffer, "state:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5213: " + "Fail to obtain port state. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "state: %s", state) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5214: " + "Fail to parse state from info file:(%s).\n", p_ch); + goto Exit; + } + + if (!strcmp(state, "ACTIVE")) + pfs_port_info->state = IB_LINK_ACTIVE; + else if (!strcmp(state, "DOWN")) + pfs_port_info->state = IB_LINK_DOWN; + else if (!strcmp(state, "INIT")) + pfs_port_info->state = IB_LINK_INIT; + else if (!strcmp(state, "ARMED")) + pfs_port_info->state = IB_LINK_ARMED; + else + pfs_port_info->state = 0; + + /* get lid */ + if (!(p_ch = strstr(file_buffer, "LID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5215: " + "Fail to obtain port lid. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "LID: %x", &lid) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5216: " + "Fail to parse lid from info file:(%s).\n", p_ch); + goto Exit; + } + pfs_port_info->lid = lid; + /* get LMC */ + if (!(p_ch = strstr(file_buffer, "LMC:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5217: " + "Fail to obtain port LMC. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "LMC: %x", &lmc) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5218: " + "Fail to parse LMC from info file:(%s).\n", p_ch); + goto Exit; + } + pfs_port_info->lmc = lmc; + + /* get SM LID */ + if (!(p_ch = strstr(file_buffer, "SM LID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5219: " + "Fail to obtain port SM LID. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "SM LID: %x", &sm_lid) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5220: " + "Fail to parse SM LID from info file:(%s).\n", p_ch); + goto Exit; + } + pfs_port_info->sm_lid = sm_lid; + + /* get SM LID */ + if (!(p_ch = strstr(file_buffer, "SM SL:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5221: " + "Fail to obtain port SM SL. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "SM SL: %x", &sm_sl) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5222: " + "Fail to parse SM SL from info file:(%s).\n", p_ch); + goto Exit; + } + pfs_port_info->sm_sl = sm_sl; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_port_info_file: " + "Obtained Port:%d = state:%d, lid:0x%04X, lmc:%d, sm_lid:0x%04X, sm_sl:%d\n", + port_num, pfs_port_info->state, pfs_port_info->lid, + pfs_port_info->lmc, pfs_port_info->sm_lid, + pfs_port_info->sm_sl); + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /********************************************************************** @@ -424,172 +407,172 @@ __parse_port_info_file( * [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 **********************************************************************/ static ib_api_status_t -__get_port_guid_from_port_gid_tbl( - IN osm_vendor_t * const p_vend, - IN uint32_t hca_idx, - IN uint8_t port_num, - OUT uint64_t *port_guid) +__get_port_guid_from_port_gid_tbl(IN osm_vendor_t * const p_vend, + IN uint32_t hca_idx, + IN uint8_t port_num, OUT uint64_t * port_guid) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char *p_ch; - int g[8]; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __get_port_guid_from_port_gid_tbl ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__get_port_guid_from_port_gid_tbl: " - "Parsing Proc File System Port Guid Table CA %d Port %d.\n", - hca_idx, port_num ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "/proc/infiniband/core/ca%d/port%d/gid_table", - hca_idx, port_num); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5223: " - "Fail to open HCA:%d Port:%d gid_table file:(%s).\n", hca_idx, port_num, - file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 - ... - */ - if (!(p_ch = strstr(file_buffer, "[ 0]"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5224: " - "Fail to obtain first gid index. In gid_table file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch+6,"%x:%x:%x:%x:%x:%x:%x:%x", - &g[7],&g[6],&g[5],&g[4],&g[3],&g[2],&g[1],&g[0]) != 8) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5225: " - "Fail to parse gid from gid_table file:(%s).\n", p_ch); - goto Exit; - } - - *port_guid = (uint64_t)g[3]<<48 | (uint64_t)g[2] << 32 | (uint64_t)g[1] << 16 | g[0]; - status = IB_SUCCESS; -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char *p_ch; + int g[8]; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __get_port_guid_from_port_gid_tbl); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__get_port_guid_from_port_gid_tbl: " + "Parsing Proc File System Port Guid Table CA %d Port %d.\n", + hca_idx, port_num); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "/proc/infiniband/core/ca%d/port%d/gid_table", + hca_idx, port_num); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5223: " + "Fail to open HCA:%d Port:%d gid_table file:(%s).\n", + hca_idx, port_num, file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 + ... + */ + if (!(p_ch = strstr(file_buffer, "[ 0]"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5224: " + "Fail to obtain first gid index. In gid_table file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch + 6, "%x:%x:%x:%x:%x:%x:%x:%x", + &g[7], &g[6], &g[5], &g[4], &g[3], &g[2], &g[1], &g[0]) != 8) + { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5225: " + "Fail to parse gid from gid_table file:(%s).\n", p_ch); + goto Exit; + } + + *port_guid = + (uint64_t) g[3] << 48 | (uint64_t) g[2] << 32 | (uint64_t) g[1] << + 16 | g[0]; + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /********************************************************************** * Initialize an Info Struct for the Given HCA by its index 1..N **********************************************************************/ static ib_api_status_t -__osm_ca_info_init( IN osm_vendor_t * const p_vend, - IN uint32_t const idx, - OUT osm_ca_info_t * const p_ca_info ) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + IN uint32_t const idx, OUT osm_ca_info_t * const p_ca_info) { - ib_api_status_t status = IB_ERROR; - uint8_t port_num; - uint64_t port_guid; - - pfs_ca_info_t pfs_ca_info; - - OSM_LOG_ENTER( p_vend->p_log, __osm_ca_info_init ); - - /* parse the CA info file */ - if (__parse_ca_info_file(p_vend, idx, &pfs_ca_info) != IB_SUCCESS) - goto Exit; - - p_ca_info->guid = cl_hton64( pfs_ca_info.guid ); - - /* set size of attributes and allocate them */ - p_ca_info->attr_size = 1; - p_ca_info->p_attr = ( ib_ca_attr_t * ) malloc( sizeof( ib_ca_attr_t ) ); - - p_ca_info->p_attr->ca_guid = p_ca_info->guid; - p_ca_info->p_attr->num_ports = pfs_ca_info.num_ports; - - /* now obtain the attributes of the ports */ - p_ca_info->p_attr->p_port_attr = - ( ib_port_attr_t * ) malloc( pfs_ca_info.num_ports * sizeof( ib_port_attr_t ) ); - - /* get all the ports info */ - for( port_num = 1; port_num <= pfs_ca_info.num_ports; port_num++ ) - { - pfs_port_info_t pfs_port_info; - /* query the port attributes */ - if (__parse_port_info_file(p_vend, idx, port_num, &pfs_port_info)) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 5226: " - "Fail to get HCA:%d Port:%d Attributes.\n", idx, port_num ); - goto Exit; - } - - /* HACK: the lids should have been converted to network but the rest of the code - is wrong and provdes them as is (host order) - so we stick with it. */ - p_ca_info->p_attr->p_port_attr[port_num-1].lid = pfs_port_info.lid; - p_ca_info->p_attr->p_port_attr[port_num-1].link_state = pfs_port_info.state; - p_ca_info->p_attr->p_port_attr[port_num-1].sm_lid = pfs_port_info.sm_lid; - - /* get the port guid */ - if (__get_port_guid_from_port_gid_tbl(p_vend, idx, port_num, &port_guid)) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 5227: " - "Fail to get HCA:%d Port:%d Guid.\n", idx, port_num ); - goto Exit; - } - p_ca_info->p_attr->p_port_attr[port_num-1].port_guid =cl_hton64( port_guid ); - } - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_ERROR; + uint8_t port_num; + uint64_t port_guid; + + pfs_ca_info_t pfs_ca_info; + + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); + + /* parse the CA info file */ + if (__parse_ca_info_file(p_vend, idx, &pfs_ca_info) != IB_SUCCESS) + goto Exit; + + p_ca_info->guid = cl_hton64(pfs_ca_info.guid); + + /* set size of attributes and allocate them */ + p_ca_info->attr_size = 1; + p_ca_info->p_attr = (ib_ca_attr_t *) malloc(sizeof(ib_ca_attr_t)); + + p_ca_info->p_attr->ca_guid = p_ca_info->guid; + p_ca_info->p_attr->num_ports = pfs_ca_info.num_ports; + + /* now obtain the attributes of the ports */ + p_ca_info->p_attr->p_port_attr = + (ib_port_attr_t *) malloc(pfs_ca_info.num_ports * + sizeof(ib_port_attr_t)); + + /* get all the ports info */ + for (port_num = 1; port_num <= pfs_ca_info.num_ports; port_num++) { + pfs_port_info_t pfs_port_info; + /* query the port attributes */ + if (__parse_port_info_file + (p_vend, idx, port_num, &pfs_port_info)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 5226: " + "Fail to get HCA:%d Port:%d Attributes.\n", idx, + port_num); + goto Exit; + } + + /* HACK: the lids should have been converted to network but the rest of the code + is wrong and provdes them as is (host order) - so we stick with it. */ + p_ca_info->p_attr->p_port_attr[port_num - 1].lid = + pfs_port_info.lid; + p_ca_info->p_attr->p_port_attr[port_num - 1].link_state = + pfs_port_info.state; + p_ca_info->p_attr->p_port_attr[port_num - 1].sm_lid = + pfs_port_info.sm_lid; + + /* get the port guid */ + if (__get_port_guid_from_port_gid_tbl + (p_vend, idx, port_num, &port_guid)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 5227: " + "Fail to get HCA:%d Port:%d Guid.\n", idx, + port_num); + goto Exit; + } + p_ca_info->p_attr->p_port_attr[port_num - 1].port_guid = + cl_hton64(port_guid); + } + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -osm_ca_info_destroy( IN osm_vendor_t * const p_vend, - IN osm_ca_info_t * const p_ca_info, - IN uint8_t num_ca) +osm_ca_info_destroy(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info, IN uint8_t num_ca) { - osm_ca_info_t *p_ca; - uint8_t i; + osm_ca_info_t *p_ca; + uint8_t i; - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_destroy ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_destroy); - for (i=0; i < num_ca; i++) - { - p_ca = &p_ca_info[i]; + for (i = 0; i < num_ca; i++) { + p_ca = &p_ca_info[i]; - if( NULL != p_ca->p_attr ) - { - if(0 != p_ca->p_attr->num_ports) - { - free( p_ca->p_attr->p_port_attr ); - } + if (NULL != p_ca->p_attr) { + if (0 != p_ca->p_attr->num_ports) { + free(p_ca->p_attr->p_port_attr); + } - free( p_ca->p_attr); - } - } + free(p_ca->p_attr); + } + } - free( p_ca_info ); + free(p_ca_info); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -597,105 +580,97 @@ osm_ca_info_destroy( IN osm_vendor_t * const p_vend, * avilable CAs on this machine. **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * const p_attr_array, - IN uint32_t * const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_api_status_t status = IB_SUCCESS; - - uint32_t caIdx; - uint32_t ca_count = 0; - uint32_t port_count = 0; - uint8_t port_num; - uint32_t total_ports = 0; - osm_ca_info_t *p_ca_infos = NULL; - uint32_t attr_array_sz = *p_num_ports; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); - - CL_ASSERT( p_vend ); - - - /* determine the number of CA's */ - ca_count = __hca_pfs_get_num_cas(); - if (! ca_count) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5228: " - "Fail to get Any CA Ids.\n" ); - goto Exit; - } - - /* Allocate an array big enough to hold the ca info objects*/ - p_ca_infos = malloc( ca_count * sizeof( osm_ca_info_t ) ); - if( p_ca_infos == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5229: " - "Unable to allocate CA information array.\n" ); - goto Exit; - } - - memset( p_ca_infos, 0, ca_count * sizeof( osm_ca_info_t ) ); - - /* - * For each CA, retrieve the CA info attributes - */ - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - status = __osm_ca_info_init( p_vend, caIdx, &p_ca_infos[caIdx-1] ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5230: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - total_ports += p_ca_infos[caIdx-1].p_attr->num_ports; - } - - *p_num_ports = total_ports; - osm_log(p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_all_port_attr: total ports:%u \n", - total_ports); - - /* - * If the user supplied enough storage, return the port guids, - * otherwise, return the appropriate error. - */ - if( attr_array_sz >= total_ports ) - { - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - uint32_t num_ports; - - num_ports = p_ca_infos[caIdx-1].p_attr->num_ports; - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_attr_array[port_count] = - *__osm_ca_info_get_port_attr_ptr( &p_ca_infos[caIdx-1], port_num ); - port_count++; - } - } - } - else - { - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - if ( p_ca_infos ) - { - osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); - } - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_SUCCESS; + + uint32_t caIdx; + uint32_t ca_count = 0; + uint32_t port_count = 0; + uint8_t port_num; + uint32_t total_ports = 0; + osm_ca_info_t *p_ca_infos = NULL; + uint32_t attr_array_sz = *p_num_ports; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + ca_count = __hca_pfs_get_num_cas(); + if (!ca_count) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5228: " + "Fail to get Any CA Ids.\n"); + goto Exit; + } + + /* Allocate an array big enough to hold the ca info objects */ + p_ca_infos = malloc(ca_count * sizeof(osm_ca_info_t)); + if (p_ca_infos == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5229: " + "Unable to allocate CA information array.\n"); + goto Exit; + } + + memset(p_ca_infos, 0, ca_count * sizeof(osm_ca_info_t)); + + /* + * For each CA, retrieve the CA info attributes + */ + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + status = + __osm_ca_info_init(p_vend, caIdx, &p_ca_infos[caIdx - 1]); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5230: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + total_ports += p_ca_infos[caIdx - 1].p_attr->num_ports; + } + + *p_num_ports = total_ports; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_all_port_attr: total ports:%u \n", total_ports); + + /* + * If the user supplied enough storage, return the port guids, + * otherwise, return the appropriate error. + */ + if (attr_array_sz >= total_ports) { + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + uint32_t num_ports; + + num_ports = p_ca_infos[caIdx - 1].p_attr->num_ports; + + for (port_num = 0; port_num < num_ports; port_num++) { + p_attr_array[port_count] = + *__osm_ca_info_get_port_attr_ptr(&p_ca_infos + [caIdx - + 1], + port_num); + port_count++; + } + } + } else { + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + if (p_ca_infos) { + osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); + } + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -704,74 +679,73 @@ osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, **********************************************************************/ ib_api_status_t -osm_vendor_get_guid_ca_and_port( - IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT uint32_t * p_hca_hndl, - OUT char * p_hca_id, - OUT uint8_t *p_hca_idx, - OUT uint32_t * p_port_num ) +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT uint32_t * p_hca_hndl, + OUT char *p_hca_id, + OUT uint8_t * p_hca_idx, + OUT uint32_t * p_port_num) { - uint32_t caIdx; - uint32_t ca_count = 0; - uint8_t port_num; - ib_api_status_t status = IB_ERROR; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_guid_ca_and_port ); - - CL_ASSERT( p_vend ); - - /* determine the number of CA's */ - ca_count = __hca_pfs_get_num_cas(); - if (! ca_count) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 5231: " - "Fail to get Any CA Ids.\n" ); - goto Exit; - } - - /* - * For each CA, retrieve the CA info attributes - */ - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - pfs_ca_info_t pfs_ca_info; - if (__parse_ca_info_file(p_vend, caIdx, &pfs_ca_info) == IB_SUCCESS) - { - /* get all the ports info */ - for( port_num = 1; port_num <= pfs_ca_info.num_ports; port_num++ ) - { - uint64_t port_guid; - if (! __get_port_guid_from_port_gid_tbl(p_vend, caIdx, port_num, &port_guid)) - { - if (cl_hton64(port_guid) == guid) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_guid_ca_and_port: " - "Found Matching guid on HCA:%d Port:%d.\n", caIdx, port_num ); - strcpy(p_hca_id, pfs_ca_info.name); - *p_port_num = port_num; - *p_hca_idx = caIdx - 1; - *p_hca_hndl = 0; - status = IB_SUCCESS; - goto Exit; - } - } - } - } - } - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 5232: " - "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", - cl_ntoh64(guid) ); - status = IB_INVALID_GUID; - - Exit: - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + uint32_t caIdx; + uint32_t ca_count = 0; + uint8_t port_num; + ib_api_status_t status = IB_ERROR; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_guid_ca_and_port); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + ca_count = __hca_pfs_get_num_cas(); + if (!ca_count) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 5231: " + "Fail to get Any CA Ids.\n"); + goto Exit; + } + + /* + * For each CA, retrieve the CA info attributes + */ + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + pfs_ca_info_t pfs_ca_info; + if (__parse_ca_info_file(p_vend, caIdx, &pfs_ca_info) == + IB_SUCCESS) { + /* get all the ports info */ + for (port_num = 1; port_num <= pfs_ca_info.num_ports; + port_num++) { + uint64_t port_guid; + if (!__get_port_guid_from_port_gid_tbl + (p_vend, caIdx, port_num, &port_guid)) { + if (cl_hton64(port_guid) == guid) { + osm_log(p_vend->p_log, + OSM_LOG_DEBUG, + "osm_vendor_get_guid_ca_and_port: " + "Found Matching guid on HCA:%d Port:%d.\n", + caIdx, port_num); + strcpy(p_hca_id, + pfs_ca_info.name); + *p_port_num = port_num; + *p_hca_idx = caIdx - 1; + *p_hca_hndl = 0; + status = IB_SUCCESS; + goto Exit; + } + } + } + } + } + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 5232: " + "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", + cl_ntoh64(guid)); + status = IB_INVALID_GUID; + + Exit: + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } #endif diff --git a/opensm/libvendor/osm_vendor_mlx_hca_sim.c b/opensm/libvendor/osm_vendor_mlx_hca_sim.c index 1c68937..e4b1117 100644 --- a/opensm/libvendor/osm_vendor_mlx_hca_sim.c +++ b/opensm/libvendor/osm_vendor_mlx_hca_sim.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #if defined(OSM_VENDOR_INTF_SIM) #undef IN @@ -64,57 +63,56 @@ char *__get_simulator_dir(void) { - static char *ibmgtSimDir = NULL; - static char *defaultIbmgtSimDir = "/tmp/ibmgtsim"; - static char *ibmgtSimNode = NULL; - static char dirName[1024]; - - /* we use the first pointer to know if we were here */ - if (ibmgtSimDir == NULL) - { - /* obtain the simulator directory */ - ibmgtSimDir = getenv("IBMGTSIM_DIR"); - if (ibmgtSimDir == NULL) - { - printf("-W- Environment variable: IBMGTSIM_DIR does not exist.\n"); - printf(" Please create one used by the simulator.\n"); - printf(" Using /tmp/ibmgtsim as default.\n"); - ibmgtSimDir = defaultIbmgtSimDir; - } - - /* obtain the node name we simulate */ - ibmgtSimNode = getenv("IBMGTSIM_NODE"); - if (ibmgtSimNode == NULL) - { - printf("-W- Environment variable: IBMGTSIM_NODE does not exist.\n"); - printf(" This variable should be the name of the node you wish to simulate.\n"); - printf(" Using H-1 as default.\n"); - ibmgtSimNode = "H-1"; - } - sprintf(dirName, "%s/%s", ibmgtSimDir, ibmgtSimNode); - } - - return dirName; + static char *ibmgtSimDir = NULL; + static char *defaultIbmgtSimDir = "/tmp/ibmgtsim"; + static char *ibmgtSimNode = NULL; + static char dirName[1024]; + + /* we use the first pointer to know if we were here */ + if (ibmgtSimDir == NULL) { + /* obtain the simulator directory */ + ibmgtSimDir = getenv("IBMGTSIM_DIR"); + if (ibmgtSimDir == NULL) { + printf + ("-W- Environment variable: IBMGTSIM_DIR does not exist.\n"); + printf + (" Please create one used by the simulator.\n"); + printf(" Using /tmp/ibmgtsim as default.\n"); + ibmgtSimDir = defaultIbmgtSimDir; + } + + /* obtain the node name we simulate */ + ibmgtSimNode = getenv("IBMGTSIM_NODE"); + if (ibmgtSimNode == NULL) { + printf + ("-W- Environment variable: IBMGTSIM_NODE does not exist.\n"); + printf + (" This variable should be the name of the node you wish to simulate.\n"); + printf(" Using H-1 as default.\n"); + ibmgtSimNode = "H-1"; + } + sprintf(dirName, "%s/%s", ibmgtSimDir, ibmgtSimNode); + } + + return dirName; } -typedef struct _osm_ca_info -{ - ib_net64_t guid; - size_t attr_size; - ib_ca_attr_t *p_attr; +typedef struct _osm_ca_info { + ib_net64_t guid; + size_t attr_size; + ib_ca_attr_t *p_attr; -} -osm_ca_info_t; +} osm_ca_info_t; /********************************************************************** * Returns a pointer to the port attribute of the specified port * owned by this CA. ************************************************************************/ -static ib_port_attr_t * -__osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +static ib_port_attr_t *__osm_ca_info_get_port_attr_ptr(IN const osm_ca_info_t * + const p_ca_info, + IN const uint8_t index) { - return ( &p_ca_info->p_attr->p_port_attr[index] ); + return (&p_ca_info->p_attr->p_port_attr[index]); } /********************************************************************** @@ -122,32 +120,30 @@ __osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, **********************************************************************/ int __hca_sim_get_num_cas(void) { - int num_cas = 0; - DIR *dp; - struct dirent *ep; - - dp = opendir(__get_simulator_dir()); - - if (dp != NULL) - { - while ((ep = readdir (dp))) - { - /* CAs are directories with the format ca[1-9][0-9]* */ - /* if ((ep->d_type == DT_DIR) && !strncmp(ep->d_name, "ca", 2)) */ - if (!strncmp(ep->d_name, "ca", 2)) - { - num_cas++; - } - } - closedir(dp); - } else { - printf("__hca_sim_get_num_cas: ERROR : ail to open dir %s\n", - __get_simulator_dir()); - exit(1); - } - - if (!num_cas) exit(1); - return num_cas; + int num_cas = 0; + DIR *dp; + struct dirent *ep; + + dp = opendir(__get_simulator_dir()); + + if (dp != NULL) { + while ((ep = readdir(dp))) { + /* CAs are directories with the format ca[1-9][0-9]* */ + /* if ((ep->d_type == DT_DIR) && !strncmp(ep->d_name, "ca", 2)) */ + if (!strncmp(ep->d_name, "ca", 2)) { + num_cas++; + } + } + closedir(dp); + } else { + printf("__hca_sim_get_num_cas: ERROR : ail to open dir %s\n", + __get_simulator_dir()); + exit(1); + } + + if (!num_cas) + exit(1); + return num_cas; } /* @@ -161,126 +157,120 @@ int __hca_sim_get_num_cas(void) FW revision: 0x300020080 */ typedef struct _sim_ca_info { - char name[32]; - char provider[32]; - uint64_t guid; - uint8_t num_ports; - uint32_t vend_id; - uint16_t dev_id; - uint16_t rev_id; - uint64_t fw_rev; + char name[32]; + char provider[32]; + uint64_t guid; + uint8_t num_ports; + uint32_t vend_id; + uint16_t dev_id; + uint16_t rev_id; + uint64_t fw_rev; } sim_ca_info_t; /********************************************************************** * Parse the CA Info file available in ibmgtSimDir/caN/info **********************************************************************/ static ib_api_status_t -__parse_ca_info_file( - IN osm_vendor_t * const p_vend, - IN uint32_t idx, - OUT sim_ca_info_t *sim_ca_info) +__parse_ca_info_file(IN osm_vendor_t * const p_vend, + IN uint32_t idx, OUT sim_ca_info_t * sim_ca_info) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char *p_ch; - int g1,g2,g3,g4; - int num_ports; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __parse_ca_info_file ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_ca_info_file: " "Querying CA %d.\n", idx ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "%s/ca%d/info", __get_simulator_dir(), idx); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5105: " - "Fail to open HCA:%d info file:(%s).\n", idx, file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - name: InfiniHost0 - provider: tavor - node GUID: 0002:c900:0120:3470 - ports: 2 - vendor ID: 0x2c9 - device ID: 0x5a44 - HW revision: 0xa1 - FW revision: 0x300020080 - */ - if (!(p_ch = strstr(file_buffer, "name:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5106: " - "Fail to obtain HCA name. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"name: %s", sim_ca_info->name) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5107: " - "Fail to parse name in info file:(%s).\n", p_ch); - goto Exit; - } - - /* get the guid of the HCA */ - if (!(p_ch = strstr(file_buffer, "node GUID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5108: " - "Fail to obtain GUID in info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch, "node GUID: %x:%x:%x:%x", &g1,&g2,&g3,&g4) != 4) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5109: " - "Fail to parse GUID in info file:(%s).\n", p_ch); - goto Exit; - } - sim_ca_info->guid = (uint64_t)g1 << 48 | (uint64_t)g1 << 32 - | (uint64_t)g1 << 16 | (uint64_t)g3 ; - - /* obtain number of ports */ - if (!(p_ch = strstr(file_buffer, "ports:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5110: " - "Fail to obtain number of ports in info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch, "ports: %d", &num_ports) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_ca_info_file: ERR 5111: " - "Fail to parse num ports in info file:(%s).\n", p_ch); - goto Exit; - } - sim_ca_info->num_ports = num_ports; - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_ca_info_file: " - "CA1 = name:%s guid:0x%016llx ports:%d\n", - sim_ca_info->name, sim_ca_info->guid, sim_ca_info->num_ports - ); - - status = IB_SUCCESS; -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char *p_ch; + int g1, g2, g3, g4; + int num_ports; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __parse_ca_info_file); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_ca_info_file: " "Querying CA %d.\n", idx); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "%s/ca%d/info", __get_simulator_dir(), idx); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5105: " + "Fail to open HCA:%d info file:(%s).\n", idx, + file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + name: InfiniHost0 + provider: tavor + node GUID: 0002:c900:0120:3470 + ports: 2 + vendor ID: 0x2c9 + device ID: 0x5a44 + HW revision: 0xa1 + FW revision: 0x300020080 + */ + if (!(p_ch = strstr(file_buffer, "name:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5106: " + "Fail to obtain HCA name. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "name: %s", sim_ca_info->name) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5107: " + "Fail to parse name in info file:(%s).\n", p_ch); + goto Exit; + } + + /* get the guid of the HCA */ + if (!(p_ch = strstr(file_buffer, "node GUID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5108: " + "Fail to obtain GUID in info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "node GUID: %x:%x:%x:%x", &g1, &g2, &g3, &g4) != 4) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5109: " + "Fail to parse GUID in info file:(%s).\n", p_ch); + goto Exit; + } + sim_ca_info->guid = (uint64_t) g1 << 48 | (uint64_t) g1 << 32 + | (uint64_t) g1 << 16 | (uint64_t) g3; + + /* obtain number of ports */ + if (!(p_ch = strstr(file_buffer, "ports:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5110: " + "Fail to obtain number of ports in info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "ports: %d", &num_ports) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_ca_info_file: ERR 5111: " + "Fail to parse num ports in info file:(%s).\n", p_ch); + goto Exit; + } + sim_ca_info->num_ports = num_ports; + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_ca_info_file: " + "CA1 = name:%s guid:0x%016llx ports:%d\n", + sim_ca_info->name, sim_ca_info->guid, sim_ca_info->num_ports); + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /* @@ -299,11 +289,11 @@ Exit: IsCapabilityMaskNoticeSupported */ typedef struct _sim_port_info { - uint8_t state; - uint16_t lid; - uint8_t lmc; - uint16_t sm_lid; - uint8_t sm_sl; + uint8_t state; + uint16_t lid; + uint8_t lmc; + uint16_t sm_lid; + uint8_t sm_sl; } sim_port_info_t; /********************************************************************** @@ -311,156 +301,150 @@ typedef struct _sim_port_info { * Port num is 1..N **********************************************************************/ static ib_api_status_t -__parse_port_info_file( - IN osm_vendor_t * const p_vend, - IN uint32_t hca_idx, - IN uint8_t port_num, - OUT sim_port_info_t *sim_port_info) +__parse_port_info_file(IN osm_vendor_t * const p_vend, + IN uint32_t hca_idx, + IN uint8_t port_num, OUT sim_port_info_t * sim_port_info) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char state[12]; - char *p_ch; - int lid, sm_lid, lmc, sm_sl; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __parse_port_info_file ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_port_info_file: " - "Parsing Proc File System Port Info CA %d Port %d.\n", hca_idx, port_num ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "%s/ca%d/port%d/info", __get_simulator_dir(), hca_idx, port_num); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5112: " - "Fail to open HCA:%d Port:%d info file:(%s).\n", hca_idx, port_num, - file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - state: ACTIVE - LID: 0x0001 - LMC: 0x0000 - SM LID: 0x0001 - SM SL: 0x0000 - ... - */ - if (!(p_ch = strstr(file_buffer, "state:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5113: " - "Fail to obtain port state. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"state: %s", state) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5114: " - "Fail to parse state from info file:(%s).\n", p_ch); - goto Exit; - } - - if (!strcmp(state, "ACTIVE")) - sim_port_info->state = IB_LINK_ACTIVE; - else if (!strcmp(state, "DOWN")) - sim_port_info->state = IB_LINK_DOWN; - else if (!strcmp(state, "INIT")) - sim_port_info->state = IB_LINK_INIT; - else if (!strcmp(state, "ARMED")) - sim_port_info->state = IB_LINK_ARMED; - else - sim_port_info->state = 0; - - /* get lid */ - if (!(p_ch = strstr(file_buffer, "LID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5115: " - "Fail to obtain port lid. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"LID: %x", &lid) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5116: " - "Fail to parse lid from info file:(%s).\n", p_ch); - goto Exit; - } - sim_port_info->lid = lid; - /* get LMC */ - if (!(p_ch = strstr(file_buffer, "LMC:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5117: " - "Fail to obtain port LMC. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"LMC: %x", &lmc) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5118: " - "Fail to parse LMC from info file:(%s).\n", p_ch); - goto Exit; - } - sim_port_info->lmc = lmc; - - /* get SM LID */ - if (!(p_ch = strstr(file_buffer, "SM LID:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5119: " - "Fail to obtain port SM LID. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"SM LID: %x", &sm_lid) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5120: " - "Fail to parse SM LID from info file:(%s).\n", p_ch); - goto Exit; - } - sim_port_info->sm_lid = sm_lid; - - /* get SM LID */ - if (!(p_ch = strstr(file_buffer, "SM SL:"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5121: " - "Fail to obtain port SM SL. In info file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch,"SM SL: %x", &sm_sl) != 1) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__parse_port_info_file: ERR 5122: " - "Fail to parse SM SL from info file:(%s).\n", p_ch); - goto Exit; - } - sim_port_info->sm_sl = sm_sl; - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__parse_port_info_file: " - "Obtained Port:%d = state:%d, lid:0x%04X, lmc:%d, sm_lid:0x%04X, sm_sl:%d\n", - port_num, sim_port_info->state, sim_port_info->lid, - sim_port_info->lmc, sim_port_info->sm_lid, sim_port_info->sm_sl - ); - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char state[12]; + char *p_ch; + int lid, sm_lid, lmc, sm_sl; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __parse_port_info_file); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_port_info_file: " + "Parsing Proc File System Port Info CA %d Port %d.\n", hca_idx, + port_num); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "%s/ca%d/port%d/info", __get_simulator_dir(), + hca_idx, port_num); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5112: " + "Fail to open HCA:%d Port:%d info file:(%s).\n", + hca_idx, port_num, file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + state: ACTIVE + LID: 0x0001 + LMC: 0x0000 + SM LID: 0x0001 + SM SL: 0x0000 + ... + */ + if (!(p_ch = strstr(file_buffer, "state:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5113: " + "Fail to obtain port state. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "state: %s", state) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5114: " + "Fail to parse state from info file:(%s).\n", p_ch); + goto Exit; + } + + if (!strcmp(state, "ACTIVE")) + sim_port_info->state = IB_LINK_ACTIVE; + else if (!strcmp(state, "DOWN")) + sim_port_info->state = IB_LINK_DOWN; + else if (!strcmp(state, "INIT")) + sim_port_info->state = IB_LINK_INIT; + else if (!strcmp(state, "ARMED")) + sim_port_info->state = IB_LINK_ARMED; + else + sim_port_info->state = 0; + + /* get lid */ + if (!(p_ch = strstr(file_buffer, "LID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5115: " + "Fail to obtain port lid. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "LID: %x", &lid) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5116: " + "Fail to parse lid from info file:(%s).\n", p_ch); + goto Exit; + } + sim_port_info->lid = lid; + /* get LMC */ + if (!(p_ch = strstr(file_buffer, "LMC:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5117: " + "Fail to obtain port LMC. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "LMC: %x", &lmc) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5118: " + "Fail to parse LMC from info file:(%s).\n", p_ch); + goto Exit; + } + sim_port_info->lmc = lmc; + + /* get SM LID */ + if (!(p_ch = strstr(file_buffer, "SM LID:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5119: " + "Fail to obtain port SM LID. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "SM LID: %x", &sm_lid) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5120: " + "Fail to parse SM LID from info file:(%s).\n", p_ch); + goto Exit; + } + sim_port_info->sm_lid = sm_lid; + + /* get SM LID */ + if (!(p_ch = strstr(file_buffer, "SM SL:"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5121: " + "Fail to obtain port SM SL. In info file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch, "SM SL: %x", &sm_sl) != 1) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__parse_port_info_file: ERR 5122: " + "Fail to parse SM SL from info file:(%s).\n", p_ch); + goto Exit; + } + sim_port_info->sm_sl = sm_sl; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__parse_port_info_file: " + "Obtained Port:%d = state:%d, lid:0x%04X, lmc:%d, sm_lid:0x%04X, sm_sl:%d\n", + port_num, sim_port_info->state, sim_port_info->lid, + sim_port_info->lmc, sim_port_info->sm_lid, + sim_port_info->sm_sl); + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /********************************************************************** @@ -469,172 +453,172 @@ __parse_port_info_file( * [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 **********************************************************************/ static ib_api_status_t -__get_port_guid_from_port_gid_tbl( - IN osm_vendor_t * const p_vend, - IN uint32_t hca_idx, - IN uint8_t port_num, - OUT uint64_t *port_guid) +__get_port_guid_from_port_gid_tbl(IN osm_vendor_t * const p_vend, + IN uint32_t hca_idx, + IN uint8_t port_num, OUT uint64_t * port_guid) { - ib_api_status_t status = IB_ERROR; - int info_file; - char file_name[256]; - char file_buffer[3200]; - char *p_ch; - int g[8]; - uint32_t len; - - OSM_LOG_ENTER( p_vend->p_log, __get_port_guid_from_port_gid_tbl ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__get_port_guid_from_port_gid_tbl: " - "Parsing Proc File System Port Guid Table CA %d Port %d.\n", - hca_idx, port_num ); - - /* we use the proc file system so we must be able to open the info file .. */ - sprintf(file_name, "%s/ca%d/port%d/gid_table", - __get_simulator_dir(), hca_idx, port_num); - info_file = open(file_name, O_RDONLY); - if (! info_file) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5123: " - "Fail to open HCA:%d Port:%d gid_table file:(%s).\n", hca_idx, port_num, - file_name); - goto Exit; - } - - /* read in the file */ - len = read(info_file, file_buffer, 3200); - close(info_file); - file_buffer[len] = '\0'; - - /* - parse the file ... - [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 - ... - */ - if (!(p_ch = strstr(file_buffer, "[ 0]"))) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5124: " - "Fail to obtain first gid index. In gid_table file:(%s).\n", file_buffer); - goto Exit; - } - if (sscanf(p_ch+6,"%x:%x:%x:%x:%x:%x:%x:%x", - &g[7],&g[6],&g[5],&g[4],&g[3],&g[2],&g[1],&g[0]) != 8) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__get_port_guid_from_port_gid_tbl: ERR 5125: " - "Fail to parse gid from gid_table file:(%s).\n", p_ch); - goto Exit; - } - - *port_guid = (uint64_t)g[3]<<48 | (uint64_t)g[2] << 32 | (uint64_t)g[1] << 16 | g[0]; - status = IB_SUCCESS; -Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return status; + ib_api_status_t status = IB_ERROR; + int info_file; + char file_name[256]; + char file_buffer[3200]; + char *p_ch; + int g[8]; + uint32_t len; + + OSM_LOG_ENTER(p_vend->p_log, __get_port_guid_from_port_gid_tbl); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__get_port_guid_from_port_gid_tbl: " + "Parsing Proc File System Port Guid Table CA %d Port %d.\n", + hca_idx, port_num); + + /* we use the proc file system so we must be able to open the info file .. */ + sprintf(file_name, "%s/ca%d/port%d/gid_table", + __get_simulator_dir(), hca_idx, port_num); + info_file = open(file_name, O_RDONLY); + if (!info_file) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5123: " + "Fail to open HCA:%d Port:%d gid_table file:(%s).\n", + hca_idx, port_num, file_name); + goto Exit; + } + + /* read in the file */ + len = read(info_file, file_buffer, 3200); + close(info_file); + file_buffer[len] = '\0'; + + /* + parse the file ... + [ 0] fe80:0000:0000:0000:0002:c900:0120:3472 + ... + */ + if (!(p_ch = strstr(file_buffer, "[ 0]"))) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5124: " + "Fail to obtain first gid index. In gid_table file:(%s).\n", + file_buffer); + goto Exit; + } + if (sscanf(p_ch + 6, "%x:%x:%x:%x:%x:%x:%x:%x", + &g[7], &g[6], &g[5], &g[4], &g[3], &g[2], &g[1], &g[0]) != 8) + { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__get_port_guid_from_port_gid_tbl: ERR 5125: " + "Fail to parse gid from gid_table file:(%s).\n", p_ch); + goto Exit; + } + + *port_guid = + (uint64_t) g[3] << 48 | (uint64_t) g[2] << 32 | (uint64_t) g[1] << + 16 | g[0]; + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return status; } /********************************************************************** * Initialize an Info Struct for the Given HCA by its index 1..N **********************************************************************/ static ib_api_status_t -__osm_ca_info_init( IN osm_vendor_t * const p_vend, - IN uint32_t const idx, - OUT osm_ca_info_t * const p_ca_info ) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + IN uint32_t const idx, OUT osm_ca_info_t * const p_ca_info) { - ib_api_status_t status = IB_ERROR; - uint8_t port_num; - uint64_t port_guid; - - sim_ca_info_t sim_ca_info; - - OSM_LOG_ENTER( p_vend->p_log, __osm_ca_info_init ); - - /* parse the CA info file */ - if (__parse_ca_info_file(p_vend, idx, &sim_ca_info) != IB_SUCCESS) - goto Exit; - - p_ca_info->guid = cl_hton64( sim_ca_info.guid ); - - /* set size of attributes and allocate them */ - p_ca_info->attr_size = 1; - p_ca_info->p_attr = ( ib_ca_attr_t * ) malloc( sizeof( ib_ca_attr_t ) ); - - p_ca_info->p_attr->ca_guid = p_ca_info->guid; - p_ca_info->p_attr->num_ports = sim_ca_info.num_ports; - - /* now obtain the attributes of the ports */ - p_ca_info->p_attr->p_port_attr = - ( ib_port_attr_t * ) malloc( sim_ca_info.num_ports * sizeof( ib_port_attr_t ) ); - - /* get all the ports info */ - for( port_num = 1; port_num <= sim_ca_info.num_ports; port_num++ ) - { - sim_port_info_t sim_port_info; - /* query the port attributes */ - if (__parse_port_info_file(p_vend, idx, port_num, &sim_port_info)) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 5126: " - "Fail to get HCA:%d Port:%d Attributes.\n", idx, port_num ); - goto Exit; - } - - /* HACK: the lids should have been converted to network but the rest of the code - is wrong and provdes them as is (host order) - so we stick with it. */ - p_ca_info->p_attr->p_port_attr[port_num-1].lid = sim_port_info.lid; - p_ca_info->p_attr->p_port_attr[port_num-1].link_state = sim_port_info.state; - p_ca_info->p_attr->p_port_attr[port_num-1].sm_lid = sim_port_info.sm_lid; - - /* get the port guid */ - if (__get_port_guid_from_port_gid_tbl(p_vend, idx, port_num, &port_guid)) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 5127: " - "Fail to get HCA:%d Port:%d Guid.\n", idx, port_num ); - goto Exit; - } - p_ca_info->p_attr->p_port_attr[port_num-1].port_guid =cl_hton64( port_guid ); - } - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_ERROR; + uint8_t port_num; + uint64_t port_guid; + + sim_ca_info_t sim_ca_info; + + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); + + /* parse the CA info file */ + if (__parse_ca_info_file(p_vend, idx, &sim_ca_info) != IB_SUCCESS) + goto Exit; + + p_ca_info->guid = cl_hton64(sim_ca_info.guid); + + /* set size of attributes and allocate them */ + p_ca_info->attr_size = 1; + p_ca_info->p_attr = (ib_ca_attr_t *) malloc(sizeof(ib_ca_attr_t)); + + p_ca_info->p_attr->ca_guid = p_ca_info->guid; + p_ca_info->p_attr->num_ports = sim_ca_info.num_ports; + + /* now obtain the attributes of the ports */ + p_ca_info->p_attr->p_port_attr = + (ib_port_attr_t *) malloc(sim_ca_info.num_ports * + sizeof(ib_port_attr_t)); + + /* get all the ports info */ + for (port_num = 1; port_num <= sim_ca_info.num_ports; port_num++) { + sim_port_info_t sim_port_info; + /* query the port attributes */ + if (__parse_port_info_file + (p_vend, idx, port_num, &sim_port_info)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 5126: " + "Fail to get HCA:%d Port:%d Attributes.\n", idx, + port_num); + goto Exit; + } + + /* HACK: the lids should have been converted to network but the rest of the code + is wrong and provdes them as is (host order) - so we stick with it. */ + p_ca_info->p_attr->p_port_attr[port_num - 1].lid = + sim_port_info.lid; + p_ca_info->p_attr->p_port_attr[port_num - 1].link_state = + sim_port_info.state; + p_ca_info->p_attr->p_port_attr[port_num - 1].sm_lid = + sim_port_info.sm_lid; + + /* get the port guid */ + if (__get_port_guid_from_port_gid_tbl + (p_vend, idx, port_num, &port_guid)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 5127: " + "Fail to get HCA:%d Port:%d Guid.\n", idx, + port_num); + goto Exit; + } + p_ca_info->p_attr->p_port_attr[port_num - 1].port_guid = + cl_hton64(port_guid); + } + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -osm_ca_info_destroy( IN osm_vendor_t * const p_vend, - IN osm_ca_info_t * const p_ca_info, - IN uint8_t num_ca) +osm_ca_info_destroy(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info, IN uint8_t num_ca) { - osm_ca_info_t *p_ca; - uint8_t i; + osm_ca_info_t *p_ca; + uint8_t i; - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_destroy ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_destroy); - for (i=0; i < num_ca; i++) - { - p_ca = &p_ca_info[i]; + for (i = 0; i < num_ca; i++) { + p_ca = &p_ca_info[i]; - if( NULL != p_ca->p_attr ) - { - if(0 != p_ca->p_attr->num_ports) - { - free( p_ca->p_attr->p_port_attr ); - } + if (NULL != p_ca->p_attr) { + if (0 != p_ca->p_attr->num_ports) { + free(p_ca->p_attr->p_port_attr); + } - free( p_ca->p_attr); - } - } + free(p_ca->p_attr); + } + } - free( p_ca_info ); + free(p_ca_info); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -642,104 +626,97 @@ osm_ca_info_destroy( IN osm_vendor_t * const p_vend, * avilable CAs on this machine. **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * const p_attr_array, - IN uint32_t * const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_api_status_t status = IB_SUCCESS; - - uint32_t caIdx; - uint32_t ca_count = 0; - uint32_t port_count = 0; - uint8_t port_num; - uint32_t total_ports = 0; - osm_ca_info_t *p_ca_infos = NULL; - uint32_t attr_array_sz = *p_num_ports; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); - - CL_ASSERT( p_vend ); - - /* determine the number of CA's */ - ca_count = __hca_sim_get_num_cas(); - if (! ca_count) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5128: " - "Fail to get Any CA Ids.\n" ); - goto Exit; - } - - /* Allocate an array big enough to hold the ca info objects*/ - p_ca_infos = malloc( ca_count * sizeof( osm_ca_info_t ) ); - if( p_ca_infos == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5129: " - "Unable to allocate CA information array.\n" ); - goto Exit; - } - - memset( p_ca_infos, 0, ca_count * sizeof( osm_ca_info_t ) ); - - /* - * For each CA, retrieve the CA info attributes - */ - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - status = __osm_ca_info_init( p_vend, caIdx, &p_ca_infos[caIdx-1] ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 5130: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str( status ) ); - goto Exit; - } - total_ports += p_ca_infos[caIdx-1].p_attr->num_ports; - } - - *p_num_ports = total_ports; - osm_log(p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_all_port_attr: total ports:%u \n", - total_ports); - - /* - * If the user supplied enough storage, return the port guids, - * otherwise, return the appropriate error. - */ - if( attr_array_sz >= total_ports ) - { - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - uint32_t num_ports; - - num_ports = p_ca_infos[caIdx-1].p_attr->num_ports; - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_attr_array[port_count] = - *__osm_ca_info_get_port_attr_ptr( &p_ca_infos[caIdx-1], port_num ); - port_count++; - } - } - } - else - { - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - if ( p_ca_infos ) - { - osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); - } - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_SUCCESS; + + uint32_t caIdx; + uint32_t ca_count = 0; + uint32_t port_count = 0; + uint8_t port_num; + uint32_t total_ports = 0; + osm_ca_info_t *p_ca_infos = NULL; + uint32_t attr_array_sz = *p_num_ports; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + ca_count = __hca_sim_get_num_cas(); + if (!ca_count) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5128: " + "Fail to get Any CA Ids.\n"); + goto Exit; + } + + /* Allocate an array big enough to hold the ca info objects */ + p_ca_infos = malloc(ca_count * sizeof(osm_ca_info_t)); + if (p_ca_infos == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5129: " + "Unable to allocate CA information array.\n"); + goto Exit; + } + + memset(p_ca_infos, 0, ca_count * sizeof(osm_ca_info_t)); + + /* + * For each CA, retrieve the CA info attributes + */ + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + status = + __osm_ca_info_init(p_vend, caIdx, &p_ca_infos[caIdx - 1]); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 5130: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); + goto Exit; + } + total_ports += p_ca_infos[caIdx - 1].p_attr->num_ports; + } + + *p_num_ports = total_ports; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_all_port_attr: total ports:%u \n", total_ports); + + /* + * If the user supplied enough storage, return the port guids, + * otherwise, return the appropriate error. + */ + if (attr_array_sz >= total_ports) { + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + uint32_t num_ports; + + num_ports = p_ca_infos[caIdx - 1].p_attr->num_ports; + + for (port_num = 0; port_num < num_ports; port_num++) { + p_attr_array[port_count] = + *__osm_ca_info_get_port_attr_ptr(&p_ca_infos + [caIdx - + 1], + port_num); + port_count++; + } + } + } else { + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + if (p_ca_infos) { + osm_ca_info_destroy(p_vend, p_ca_infos, ca_count); + } + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -748,74 +725,73 @@ osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, **********************************************************************/ ib_api_status_t -osm_vendor_get_guid_ca_and_port( - IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT uint32_t * p_hca_hndl, - OUT char * p_hca_id, - OUT uint8_t *p_hca_idx, - OUT uint32_t * p_port_num ) +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT uint32_t * p_hca_hndl, + OUT char *p_hca_id, + OUT uint8_t * p_hca_idx, + OUT uint32_t * p_port_num) { - uint32_t caIdx; - uint32_t ca_count = 0; - uint8_t port_num; - ib_api_status_t status = IB_ERROR; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_guid_ca_and_port ); - - CL_ASSERT( p_vend ); - - /* determine the number of CA's */ - ca_count = __hca_sim_get_num_cas(); - if (! ca_count) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 5131: " - "Fail to get Any CA Ids.\n" ); - goto Exit; - } - - /* - * For each CA, retrieve the CA info attributes - */ - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - sim_ca_info_t sim_ca_info; - if (__parse_ca_info_file(p_vend, caIdx, &sim_ca_info) == IB_SUCCESS) - { - /* get all the ports info */ - for( port_num = 1; port_num <= sim_ca_info.num_ports; port_num++ ) - { - uint64_t port_guid; - if (! __get_port_guid_from_port_gid_tbl(p_vend, caIdx, port_num, &port_guid)) - { - if (cl_hton64(port_guid) == guid) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_guid_ca_and_port: " - "Found Matching guid on HCA:%d Port:%d.\n", caIdx, port_num ); - strcpy(p_hca_id, sim_ca_info.name); - *p_port_num = port_num; - *p_hca_idx = caIdx - 1; - *p_hca_hndl = 0; - status = IB_SUCCESS; - goto Exit; - } - } - } - } - } - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 5132: " - "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", - cl_ntoh64(guid) ); - status = IB_INVALID_GUID; - - Exit: - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + uint32_t caIdx; + uint32_t ca_count = 0; + uint8_t port_num; + ib_api_status_t status = IB_ERROR; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_guid_ca_and_port); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + ca_count = __hca_sim_get_num_cas(); + if (!ca_count) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 5131: " + "Fail to get Any CA Ids.\n"); + goto Exit; + } + + /* + * For each CA, retrieve the CA info attributes + */ + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + sim_ca_info_t sim_ca_info; + if (__parse_ca_info_file(p_vend, caIdx, &sim_ca_info) == + IB_SUCCESS) { + /* get all the ports info */ + for (port_num = 1; port_num <= sim_ca_info.num_ports; + port_num++) { + uint64_t port_guid; + if (!__get_port_guid_from_port_gid_tbl + (p_vend, caIdx, port_num, &port_guid)) { + if (cl_hton64(port_guid) == guid) { + osm_log(p_vend->p_log, + OSM_LOG_DEBUG, + "osm_vendor_get_guid_ca_and_port: " + "Found Matching guid on HCA:%d Port:%d.\n", + caIdx, port_num); + strcpy(p_hca_id, + sim_ca_info.name); + *p_port_num = port_num; + *p_hca_idx = caIdx - 1; + *p_hca_hndl = 0; + status = IB_SUCCESS; + goto Exit; + } + } + } + } + } + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 5132: " + "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", + cl_ntoh64(guid)); + status = IB_INVALID_GUID; + + Exit: + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -824,68 +800,65 @@ osm_vendor_get_guid_ca_and_port( **********************************************************************/ ib_api_status_t -osm_vendor_get_guid_by_ca_and_port( IN osm_vendor_t * const p_vend, - IN char *hca_id, - IN uint32_t port_num, - OUT uint64_t *p_port_guid) +osm_vendor_get_guid_by_ca_and_port(IN osm_vendor_t * const p_vend, + IN char *hca_id, + IN uint32_t port_num, + OUT uint64_t * p_port_guid) { - uint32_t caIdx; - uint32_t ca_count = 0; - ib_api_status_t status = IB_ERROR; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_guid_by_ca_and_port ); - - CL_ASSERT( p_vend ); - - /* determine the number of CA's */ - ca_count = __hca_sim_get_num_cas(); - if (! ca_count) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_by_ca_and_port: ERR 5133: " - "Fail to get Any CA Ids.\n" ); - goto Exit; - } - - /* - * For each CA, retrieve the CA info attributes - */ - for( caIdx = 1; caIdx <= ca_count; caIdx++ ) - { - sim_ca_info_t sim_ca_info; - if (__parse_ca_info_file(p_vend, caIdx, &sim_ca_info) == IB_SUCCESS) - { - /* if not identical by id - go to next one */ - if (strcmp(sim_ca_info.name, hca_id)) continue; - - if( (port_num < 1) || (port_num > sim_ca_info.num_ports)) - { - return 1; - } - - - if (! __get_port_guid_from_port_gid_tbl( - p_vend, caIdx, port_num, p_port_guid)) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_guid_by_ca_and_port: " - "Found Matching guid on HCA:%d Port:%d.\n", caIdx, port_num); - status = IB_SUCCESS; - goto Exit; - } - } - } - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_by_ca_and_port: ERR 5134: " - "Fail to find HCA:%s\n", - hca_id); - status = IB_INVALID_GUID; - - Exit: - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + uint32_t caIdx; + uint32_t ca_count = 0; + ib_api_status_t status = IB_ERROR; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_guid_by_ca_and_port); + + CL_ASSERT(p_vend); + + /* determine the number of CA's */ + ca_count = __hca_sim_get_num_cas(); + if (!ca_count) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_by_ca_and_port: ERR 5133: " + "Fail to get Any CA Ids.\n"); + goto Exit; + } + + /* + * For each CA, retrieve the CA info attributes + */ + for (caIdx = 1; caIdx <= ca_count; caIdx++) { + sim_ca_info_t sim_ca_info; + if (__parse_ca_info_file(p_vend, caIdx, &sim_ca_info) == + IB_SUCCESS) { + /* if not identical by id - go to next one */ + if (strcmp(sim_ca_info.name, hca_id)) + continue; + + if ((port_num < 1) + || (port_num > sim_ca_info.num_ports)) { + return 1; + } + + if (!__get_port_guid_from_port_gid_tbl + (p_vend, caIdx, port_num, p_port_guid)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_guid_by_ca_and_port: " + "Found Matching guid on HCA:%d Port:%d.\n", + caIdx, port_num); + status = IB_SUCCESS; + goto Exit; + } + } + } + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_by_ca_and_port: ERR 5134: " + "Fail to find HCA:%s\n", hca_id); + status = IB_INVALID_GUID; + + Exit: + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } #endif diff --git a/opensm/libvendor/osm_vendor_mlx_ibmgt.c b/opensm/libvendor/osm_vendor_mlx_ibmgt.c index 45ec872..eca09c9 100644 --- a/opensm/libvendor/osm_vendor_mlx_ibmgt.c +++ b/opensm/libvendor/osm_vendor_mlx_ibmgt.c @@ -43,7 +43,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -54,46 +54,41 @@ #include typedef struct _osmv_IBMGT_transport_mgr_ { - IB_MGT_mad_type_t mad_type; - uint8_t mgmt_class; /* for gsi */ - /* for communication between send call back and send mad */ - boolean_t is_send_ok; - cl_event_t send_done; -}osmv_IBMGT_transport_mgr_t; + IB_MGT_mad_type_t mad_type; + uint8_t mgmt_class; /* for gsi */ + /* for communication between send call back and send mad */ + boolean_t is_send_ok; + cl_event_t send_done; +} osmv_IBMGT_transport_mgr_t; typedef struct _osmv_IBMGT_transport_info_ { - IB_MGT_mad_hndl_t smi_h; - cl_qlist_t* p_smi_list; + IB_MGT_mad_hndl_t smi_h; + cl_qlist_t *p_smi_list; - IB_MGT_mad_hndl_t gsi_h; - /* holds bind object list for every binded mgmt class */ - cl_qlist_t* gsi_mgmt_lists[15]; -}osmv_IBMGT_transport_info_t; + IB_MGT_mad_hndl_t gsi_h; + /* holds bind object list for every binded mgmt class */ + cl_qlist_t *gsi_mgmt_lists[15]; +} osmv_IBMGT_transport_info_t; static void -__osmv_IBMGT_rcv_desc_to_osm_addr( - IN IB_MGT_mad_rcv_desc_t *p_rcv_desc, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr); +__osmv_IBMGT_rcv_desc_to_osm_addr(IN IB_MGT_mad_rcv_desc_t * p_rcv_desc, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr); static void -__osmv_IBMGT_osm_addr_to_ibmgt_addr( - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT IB_ud_av_t *p_av); +__osmv_IBMGT_osm_addr_to_ibmgt_addr(IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, OUT IB_ud_av_t * p_av); void -__osmv_IBMGT_send_cb( IN IB_MGT_mad_hndl_t mad_hndl, - IN u_int64_t wrid, - IN IB_comp_status_t status, - IN void *private_ctx_p ); +__osmv_IBMGT_send_cb(IN IB_MGT_mad_hndl_t mad_hndl, + IN u_int64_t wrid, + IN IB_comp_status_t status, IN void *private_ctx_p); void -__osmv_IBMGT_rcv_cb( IN IB_MGT_mad_hndl_t mad_hndl, - IN void *private_ctx_p, - IN void *payload_p, - IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p ); - +__osmv_IBMGT_rcv_cb(IN IB_MGT_mad_hndl_t mad_hndl, + IN void *private_ctx_p, + IN void *payload_p, + IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p); /* * NAME @@ -104,262 +99,261 @@ __osmv_IBMGT_rcv_cb( IN IB_MGT_mad_hndl_t mad_hndl, */ ib_api_status_t -osmv_transport_init(IN osm_bind_info_t *p_info, - IN char hca_id[VENDOR_HCA_MAXNAMES], - IN uint8_t hca_idx, - IN osmv_bind_obj_t *p_bo) +osmv_transport_init(IN osm_bind_info_t * p_info, + IN char hca_id[VENDOR_HCA_MAXNAMES], + IN uint8_t hca_idx, IN osmv_bind_obj_t * p_bo) { - ib_api_status_t st = IB_SUCCESS; - IB_MGT_ret_t ret; - IB_MGT_mad_type_t mad_type; - osmv_IBMGT_transport_mgr_t* p_mgr; - osmv_IBMGT_transport_info_t* p_tpot_info; - cl_list_obj_t* p_obj = NULL; - osm_log_t* p_log = p_bo->p_vendor->p_log; - int i; - - UNUSED_PARAM( hca_idx ); - - /* if first bind, allocate tranport_info at vendor*/ - if (NULL == p_bo->p_vendor->p_transport_info) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: first bind() for the vendor\n"); - p_bo->p_vendor->p_transport_info - = (osmv_IBMGT_transport_info_t*) malloc(sizeof(osmv_IBMGT_transport_info_t)); - if (NULL == p_bo->p_vendor->p_transport_info) - { - return IB_INSUFFICIENT_MEMORY; - } - memset(p_bo->p_vendor->p_transport_info, 0, sizeof(osmv_IBMGT_transport_info_t)); - p_tpot_info = (osmv_IBMGT_transport_info_t*)(p_bo->p_vendor->p_transport_info); - - p_tpot_info->smi_h = 0xffffffff; - p_tpot_info->p_smi_list = NULL; - - p_tpot_info->gsi_h = 0xffffffff; - for (i=0; i< 15; i++) { - - p_tpot_info->gsi_mgmt_lists[i] = NULL; - } - - } else { - - p_tpot_info = (osmv_IBMGT_transport_info_t*)(p_bo->p_vendor->p_transport_info); - } - - /* Initialize the magic_ptr to the pointer of the p_bo info. - This will be used to signal when the object is being destroyed, so no - real action will be done then. */ - p_bo->magic_ptr = p_bo; - - - /* allocate transport mgr */ - p_mgr = malloc(sizeof(osmv_IBMGT_transport_mgr_t)); - if (NULL == p_mgr) - { - free(p_tpot_info); - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7201: " - "alloc failed \n"); - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_mgr, 0, sizeof(osmv_IBMGT_transport_mgr_t)); - - p_bo->p_transp_mgr = p_mgr; - - switch ( p_info->mad_class ) { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - mad_type = IB_MGT_SMI; - break; - - case IB_MCLASS_SUBN_ADM: - default: - mad_type = IB_MGT_GSI; - break; - } - - /* we only support one class registration per SMI/GSI !!! */ - switch (mad_type) { - case IB_MGT_SMI: - /* we do not need to bind the handle if already available */ - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: SMI bind\n"); - - if (p_tpot_info->smi_h == 0xffffffff) - { - ret = IB_MGT_get_handle(hca_id, - p_bo->port_num, - IB_MGT_SMI, - &(p_tpot_info->smi_h)); - if ( IB_MGT_OK != ret ) - { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7202: " - "IB_MGT_get_handle for smi failed \n"); - st = IB_ERROR; - free(p_mgr); - goto Exit; - } - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: got smi handle:%d \n", - p_tpot_info->smi_h); - - ret = IB_MGT_bind_sm( p_tpot_info->smi_h ); - if ( IB_MGT_OK != ret ) - { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7203: " - "IB_MGT_bind_sm failed \n"); - st = IB_ERROR; - free( p_mgr); - goto Exit; - } - - /* init smi list */ - p_tpot_info->p_smi_list = malloc(sizeof(cl_qlist_t)); - if (NULL == p_tpot_info->p_smi_list) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7204: " - "alloc failed \n"); - IB_MGT_unbind_sm(p_tpot_info->smi_h); - IB_MGT_release_handle(p_tpot_info->smi_h); - free(p_mgr); - return IB_INSUFFICIENT_MEMORY; - } - memset(p_tpot_info->p_smi_list, 0, sizeof(cl_qlist_t)); - cl_qlist_init(p_tpot_info->p_smi_list); - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: before reg_cb\n"); - ret = IB_MGT_reg_cb(p_tpot_info->smi_h, - &__osmv_IBMGT_rcv_cb, - p_bo, - &__osmv_IBMGT_send_cb, - p_tpot_info->p_smi_list, - IB_MGT_RCV_CB_MASK | IB_MGT_SEND_CB_MASK ); - if (ret != IB_SUCCESS) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7205: " - "reg_cb failed with return code:%x \n", - ret - ); - IB_MGT_unbind_sm(p_tpot_info->smi_h); - IB_MGT_release_handle(p_tpot_info->smi_h); - free(p_tpot_info->p_smi_list); - free(p_mgr); - st= IB_ERROR; - goto Exit; - } - - } - /* insert to list of smi's - for raising callbacks later on */ - p_obj = malloc(sizeof(cl_list_obj_t)); - if (p_obj) - memset(p_obj, 0, sizeof(cl_list_obj_t)); - cl_qlist_set_obj(p_obj, p_bo); - cl_qlist_insert_tail(p_tpot_info->p_smi_list, &p_obj->list_item); - - break; - - case IB_MGT_GSI: - /* we do not need to bind the handle if already available */ - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: ERR 7206: GSI bind\n" ); - if (p_tpot_info->gsi_h == 0xffffffff) - { - ret = IB_MGT_get_handle(hca_id, - p_bo->port_num, - IB_MGT_GSI, - &(p_tpot_info->gsi_h)); - if ( IB_MGT_OK != ret ) - { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7207: " - "IB_MGT_get_handle for gsi failed \n"); - st = IB_ERROR; - free(p_mgr); - goto Exit; - } - } - - /* this mgmt class was not binded yet */ - if (p_tpot_info->gsi_mgmt_lists[p_info->mad_class] == NULL) - { - ret = IB_MGT_bind_gsi_class( p_tpot_info->gsi_h,p_info->mad_class); - if ( IB_MGT_OK != ret ) - { - osm_log(p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7208: " - "IB_MGT_bind_gsi_class failed \n"); - st = IB_ERROR; - free( p_mgr); - goto Exit; - } - - p_tpot_info->gsi_mgmt_lists[p_info->mad_class] = malloc(sizeof(cl_qlist_t)); - if (NULL == p_tpot_info->gsi_mgmt_lists[p_info->mad_class]) - { - IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h,p_info->mad_class); - free(p_mgr); - return IB_INSUFFICIENT_MEMORY; - } - memset(p_tpot_info->gsi_mgmt_lists[p_info->mad_class], 0, sizeof(cl_qlist_t)); - cl_qlist_init(p_tpot_info->gsi_mgmt_lists[p_info->mad_class]); - } - /* insert to list of smi's - for raising callbacks later on */ - p_obj = malloc(sizeof(cl_list_obj_t)); - if (p_obj) - memset(p_obj, 0, sizeof(cl_list_obj_t)); - cl_qlist_set_obj(p_obj,p_bo); - cl_qlist_insert_tail(p_tpot_info->gsi_mgmt_lists[p_info->mad_class],&p_obj->list_item); - - p_mgr->mgmt_class = p_info->mad_class; - ret = IB_MGT_reg_cb(p_tpot_info->gsi_h, - &__osmv_IBMGT_rcv_cb, - p_bo, - &__osmv_IBMGT_send_cb, - p_bo, - IB_MGT_RCV_CB_MASK | IB_MGT_SEND_CB_MASK ); - - if (ret != IB_SUCCESS) - { - IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h,p_mgr->mgmt_class); - free(p_tpot_info->gsi_mgmt_lists[p_mgr->mgmt_class]); - free(p_mgr); - st= IB_ERROR; - goto Exit; - } - - break; - - default: - osm_log(p_log, OSM_LOG_ERROR, - "osmv_transport_init: ERR 7209: unrecognized mgmt class \n" ); - st = IB_ERROR; - free( p_mgr); - goto Exit; - } - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_init: GSI bind\n"); - cl_event_construct(&p_mgr->send_done); - cl_event_init(&p_mgr->send_done,TRUE); - p_mgr->is_send_ok = FALSE; - p_mgr->mad_type = mad_type; - - Exit: - /* OSM_LOG_EXIT(p_log ); */ - return(ib_api_status_t)st; + ib_api_status_t st = IB_SUCCESS; + IB_MGT_ret_t ret; + IB_MGT_mad_type_t mad_type; + osmv_IBMGT_transport_mgr_t *p_mgr; + osmv_IBMGT_transport_info_t *p_tpot_info; + cl_list_obj_t *p_obj = NULL; + osm_log_t *p_log = p_bo->p_vendor->p_log; + int i; + + UNUSED_PARAM(hca_idx); + + /* if first bind, allocate tranport_info at vendor */ + if (NULL == p_bo->p_vendor->p_transport_info) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: first bind() for the vendor\n"); + p_bo->p_vendor->p_transport_info + = + (osmv_IBMGT_transport_info_t *) + malloc(sizeof(osmv_IBMGT_transport_info_t)); + if (NULL == p_bo->p_vendor->p_transport_info) { + return IB_INSUFFICIENT_MEMORY; + } + memset(p_bo->p_vendor->p_transport_info, 0, + sizeof(osmv_IBMGT_transport_info_t)); + p_tpot_info = + (osmv_IBMGT_transport_info_t *) (p_bo->p_vendor-> + p_transport_info); + + p_tpot_info->smi_h = 0xffffffff; + p_tpot_info->p_smi_list = NULL; + + p_tpot_info->gsi_h = 0xffffffff; + for (i = 0; i < 15; i++) { + + p_tpot_info->gsi_mgmt_lists[i] = NULL; + } + + } else { + + p_tpot_info = + (osmv_IBMGT_transport_info_t *) (p_bo->p_vendor-> + p_transport_info); + } + + /* Initialize the magic_ptr to the pointer of the p_bo info. + This will be used to signal when the object is being destroyed, so no + real action will be done then. */ + p_bo->magic_ptr = p_bo; + + /* allocate transport mgr */ + p_mgr = malloc(sizeof(osmv_IBMGT_transport_mgr_t)); + if (NULL == p_mgr) { + free(p_tpot_info); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7201: " "alloc failed \n"); + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_mgr, 0, sizeof(osmv_IBMGT_transport_mgr_t)); + + p_bo->p_transp_mgr = p_mgr; + + switch (p_info->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + mad_type = IB_MGT_SMI; + break; + + case IB_MCLASS_SUBN_ADM: + default: + mad_type = IB_MGT_GSI; + break; + } + + /* we only support one class registration per SMI/GSI !!! */ + switch (mad_type) { + case IB_MGT_SMI: + /* we do not need to bind the handle if already available */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: SMI bind\n"); + + if (p_tpot_info->smi_h == 0xffffffff) { + ret = IB_MGT_get_handle(hca_id, + p_bo->port_num, + IB_MGT_SMI, + &(p_tpot_info->smi_h)); + if (IB_MGT_OK != ret) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7202: " + "IB_MGT_get_handle for smi failed \n"); + st = IB_ERROR; + free(p_mgr); + goto Exit; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: got smi handle:%d \n", + p_tpot_info->smi_h); + + ret = IB_MGT_bind_sm(p_tpot_info->smi_h); + if (IB_MGT_OK != ret) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7203: " + "IB_MGT_bind_sm failed \n"); + st = IB_ERROR; + free(p_mgr); + goto Exit; + } + + /* init smi list */ + p_tpot_info->p_smi_list = malloc(sizeof(cl_qlist_t)); + if (NULL == p_tpot_info->p_smi_list) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7204: " + "alloc failed \n"); + IB_MGT_unbind_sm(p_tpot_info->smi_h); + IB_MGT_release_handle(p_tpot_info->smi_h); + free(p_mgr); + return IB_INSUFFICIENT_MEMORY; + } + memset(p_tpot_info->p_smi_list, 0, sizeof(cl_qlist_t)); + cl_qlist_init(p_tpot_info->p_smi_list); + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: before reg_cb\n"); + ret = IB_MGT_reg_cb(p_tpot_info->smi_h, + &__osmv_IBMGT_rcv_cb, + p_bo, + &__osmv_IBMGT_send_cb, + p_tpot_info->p_smi_list, + IB_MGT_RCV_CB_MASK | + IB_MGT_SEND_CB_MASK); + if (ret != IB_SUCCESS) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7205: " + "reg_cb failed with return code:%x \n", + ret); + IB_MGT_unbind_sm(p_tpot_info->smi_h); + IB_MGT_release_handle(p_tpot_info->smi_h); + free(p_tpot_info->p_smi_list); + free(p_mgr); + st = IB_ERROR; + goto Exit; + } + + } + /* insert to list of smi's - for raising callbacks later on */ + p_obj = malloc(sizeof(cl_list_obj_t)); + if (p_obj) + memset(p_obj, 0, sizeof(cl_list_obj_t)); + cl_qlist_set_obj(p_obj, p_bo); + cl_qlist_insert_tail(p_tpot_info->p_smi_list, + &p_obj->list_item); + + break; + + case IB_MGT_GSI: + /* we do not need to bind the handle if already available */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: ERR 7206: GSI bind\n"); + if (p_tpot_info->gsi_h == 0xffffffff) { + ret = IB_MGT_get_handle(hca_id, + p_bo->port_num, + IB_MGT_GSI, + &(p_tpot_info->gsi_h)); + if (IB_MGT_OK != ret) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7207: " + "IB_MGT_get_handle for gsi failed \n"); + st = IB_ERROR; + free(p_mgr); + goto Exit; + } + } + + /* this mgmt class was not binded yet */ + if (p_tpot_info->gsi_mgmt_lists[p_info->mad_class] == NULL) { + ret = + IB_MGT_bind_gsi_class(p_tpot_info->gsi_h, + p_info->mad_class); + if (IB_MGT_OK != ret) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7208: " + "IB_MGT_bind_gsi_class failed \n"); + st = IB_ERROR; + free(p_mgr); + goto Exit; + } + + p_tpot_info->gsi_mgmt_lists[p_info->mad_class] = + malloc(sizeof(cl_qlist_t)); + if (NULL == + p_tpot_info->gsi_mgmt_lists[p_info->mad_class]) { + IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h, + p_info->mad_class); + free(p_mgr); + return IB_INSUFFICIENT_MEMORY; + } + memset(p_tpot_info->gsi_mgmt_lists[p_info->mad_class], + 0, sizeof(cl_qlist_t)); + cl_qlist_init(p_tpot_info-> + gsi_mgmt_lists[p_info->mad_class]); + } + /* insert to list of smi's - for raising callbacks later on */ + p_obj = malloc(sizeof(cl_list_obj_t)); + if (p_obj) + memset(p_obj, 0, sizeof(cl_list_obj_t)); + cl_qlist_set_obj(p_obj, p_bo); + cl_qlist_insert_tail(p_tpot_info-> + gsi_mgmt_lists[p_info->mad_class], + &p_obj->list_item); + + p_mgr->mgmt_class = p_info->mad_class; + ret = IB_MGT_reg_cb(p_tpot_info->gsi_h, + &__osmv_IBMGT_rcv_cb, + p_bo, + &__osmv_IBMGT_send_cb, + p_bo, + IB_MGT_RCV_CB_MASK | IB_MGT_SEND_CB_MASK); + + if (ret != IB_SUCCESS) { + IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h, + p_mgr->mgmt_class); + free(p_tpot_info->gsi_mgmt_lists[p_mgr->mgmt_class]); + free(p_mgr); + st = IB_ERROR; + goto Exit; + } + + break; + + default: + osm_log(p_log, OSM_LOG_ERROR, + "osmv_transport_init: ERR 7209: unrecognized mgmt class \n"); + st = IB_ERROR; + free(p_mgr); + goto Exit; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_init: GSI bind\n"); + cl_event_construct(&p_mgr->send_done); + cl_event_init(&p_mgr->send_done, TRUE); + p_mgr->is_send_ok = FALSE; + p_mgr->mad_type = mad_type; + + Exit: + /* OSM_LOG_EXIT(p_log ); */ + return (ib_api_status_t) st; } - - /* * NAME * osmv_transport_send_mad @@ -370,439 +364,421 @@ osmv_transport_init(IN osm_bind_info_t *p_info, ib_api_status_t osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, - IN void *p_ib_mad, - IN const osm_mad_addr_t *p_mad_addr){ - - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osmv_IBMGT_transport_info_t* p_tpot_info = - (osmv_IBMGT_transport_info_t*)(p_bo->p_vendor->p_transport_info); - osm_vendor_t const *p_vend = p_bo->p_vendor; - ib_api_status_t status; - IB_ud_av_t av; - IB_MGT_ret_t ret; - ib_mad_t* p_mad = p_ib_mad; - - OSM_LOG_ENTER( p_vend->p_log,osmv_transport_mad_send); - - CL_ASSERT(p_bo->p_vendor->p_transport_info); - - /* - * For all sends other than directed route SM MADs, - * acquire an address vector for the destination. - */ - if ( p_mad->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - __osmv_IBMGT_osm_addr_to_ibmgt_addr(p_mad_addr,p_mad->mgmt_class == IB_MCLASS_SUBN_LID,&av); - } - else - { - /* is a directed route - we need to construct a permissive address */ - memset( &av, 0, sizeof( av ) ); - /* we do not need port number since it is part of the mad_hndl */ - av.dlid = IB_LID_PERMISSIVE; - } - - /* send it */ - if ( (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || - (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { - - /* SMI CASE */ - if ( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osmv_transport_mad_send: " - "av.dlid:0x%X, " - "av.static_rate:%d, " - "av.path_bits:%d.\n", - cl_ntoh16( av.dlid ), av.static_rate, av.src_path_bits ); - } - - ret = IB_MGT_send_mad(p_tpot_info->smi_h,p_mad, /* actual payload */ - &av, /* address vector */ - (u_int64_t)CAST_P2LONG(p_bo), - IB_MGT_DEFAULT_SEND_TIME ); - } - else - { - /* GSI CASE - Support Remote QP */ - if ( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osmv_transport_mad_send: " - "av.dlid:0x%X, av.static_rate:%d, av.path_bits:%d, remote qp:%d \n", - cl_ntoh16( av.dlid ), av.static_rate, av.src_path_bits, - cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp) - ); - } - - ret = IB_MGT_send_mad_to_qp(p_tpot_info->gsi_h, - p_mad, /* actual payload */ - &av, /* address vector */ - (u_int64_t)CAST_P2LONG(p_bo), - IB_MGT_DEFAULT_SEND_TIME, - cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp)); - - - } - - status = IB_SUCCESS; - if ( ret != IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osmv_transport_mad_send: ERR 7210: " - "Error sending mad (%d).\n", ret ); - status = IB_ERROR; - } - else - { - osmv_IBMGT_transport_mgr_t* p_mgr = - (osmv_IBMGT_transport_mgr_t* ) (p_bo->p_transp_mgr); - - /* Let the others work when I am sleeping ... */ - osmv_txn_unlock(p_bo); - - cl_event_wait_on(&(p_mgr->send_done),0xffffffff,TRUE); - - /* Re-acquire the lock */ - osmv_txn_lock(p_bo); - - if (TRUE == p_bo->is_closing) - { - - osm_log(p_vend->p_log, OSM_LOG_ERROR, - "osmv_transport_mad_send: ERR 7211: " - "The handle %p is being unbound, cannot send.\n", h_bind); - status = IB_ERROR; - } - - if (p_mgr->is_send_ok == FALSE) - { - status = IB_ERROR; - } - } - - OSM_LOG_EXIT( p_vend->p_log ); - return(status); -} - - + IN void *p_ib_mad, IN const osm_mad_addr_t * p_mad_addr) +{ -void -osmv_transport_done(IN const osm_bind_handle_t h_bind){ - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osm_log_t* p_log = p_bo->p_vendor->p_log; - osmv_IBMGT_transport_mgr_t* p_mgr; - osmv_IBMGT_transport_info_t* p_tpot_info; - IB_MGT_ret_t ret; - cl_list_obj_t* p_obj = NULL; - cl_list_item_t* p_item,*p_item_tmp; - int i; - cl_qlist_t* p_list = NULL; - - OSM_LOG_ENTER( p_log, osmv_transport_done ); - - CL_ASSERT(p_bo); - - /* First of all - zero out the magic_ptr, so if a callback is called - - it'll know that we are currently closing down, and will not handle the - mad. */ - p_bo->magic_ptr = 0; - - p_mgr = (osmv_IBMGT_transport_mgr_t*)(p_bo->p_transp_mgr); - p_tpot_info = (osmv_IBMGT_transport_info_t*)(p_bo->p_vendor->p_transport_info); - - - - switch (p_mgr->mad_type) { - case IB_MGT_SMI: - p_list = p_tpot_info->p_smi_list; - - /* remove from the bindings list */ - p_item = cl_qlist_head(p_list); - while (p_item != cl_qlist_end(p_list)) { - p_obj = PARENT_STRUCT(p_item,cl_list_obj_t,list_item); - if (cl_qlist_obj(p_obj) == h_bind) - { - break; - } - p_item_tmp = cl_qlist_next(p_item); - p_item = p_item_tmp; - } - - CL_ASSERT(p_item != cl_qlist_end(p_list)); - cl_qlist_remove_item(p_list,p_item); - if (p_obj) free(p_obj); - - /* no one is binded to smi anymore - we can free the list, unbind & realease the hndl*/ - if (cl_is_qlist_empty(p_list) == TRUE) - { - free(p_list); - p_list = NULL; - - ret = IB_MGT_unbind_sm(p_tpot_info->smi_h); - if ( ret != IB_MGT_OK ) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_done: ERR 7212: " - "Failed to unbind sm\n" ); - } - - ret= IB_MGT_release_handle(p_tpot_info->smi_h); - if ( ret != IB_MGT_OK ) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_done: ERR 7213: " - "Failed to release smi handle\n" ); - } - p_tpot_info->smi_h = 0xffffffff; - } - break; - - case IB_MGT_GSI: - p_list = p_tpot_info->gsi_mgmt_lists[p_mgr->mgmt_class]; - /* remove from the bindings list */ - p_item = cl_qlist_head(p_list); - while (p_item != cl_qlist_end(p_list)) { - p_obj = PARENT_STRUCT(p_item,cl_list_obj_t,list_item); - if (cl_qlist_obj(p_obj) == h_bind) - { - break; - } - p_item_tmp = cl_qlist_next(p_item); - p_item = p_item_tmp; - } - - CL_ASSERT(p_item != cl_qlist_end(p_list)); - cl_qlist_remove_item(p_list,p_item); - if (p_obj) free(p_obj); - - /* no one is binded to this class anymore - we can free the list and unbind this class*/ - if (cl_is_qlist_empty(p_list) == TRUE) - { - free(p_list); - p_list = NULL; - - ret = IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h,p_mgr->mgmt_class); - if ( ret != IB_MGT_OK ) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_done: ERR 7214: " - "Failed to unbind gsi class\n" ); - } - } - - /* all the mgmt classes are unbinded - release gsi handle*/ - for (i=0; i< 15; i++) { - if (p_tpot_info->gsi_mgmt_lists[i] != NULL) - { - break; - } - } - - if (i==15) - { - ret= IB_MGT_release_handle(p_tpot_info->gsi_h); - if ( ret != IB_MGT_OK ) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_transport_done: ERR 7215: " - "Failed to release gsi handle\n" ); - } - p_tpot_info->gsi_h = 0xffffffff; - } - }/* end switch */ - - free(p_mgr); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osmv_IBMGT_transport_info_t *p_tpot_info = + (osmv_IBMGT_transport_info_t *) (p_bo->p_vendor->p_transport_info); + osm_vendor_t const *p_vend = p_bo->p_vendor; + ib_api_status_t status; + IB_ud_av_t av; + IB_MGT_ret_t ret; + ib_mad_t *p_mad = p_ib_mad; + + OSM_LOG_ENTER(p_vend->p_log, osmv_transport_mad_send); + + CL_ASSERT(p_bo->p_vendor->p_transport_info); + + /* + * For all sends other than directed route SM MADs, + * acquire an address vector for the destination. + */ + if (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR) { + __osmv_IBMGT_osm_addr_to_ibmgt_addr(p_mad_addr, + p_mad->mgmt_class == + IB_MCLASS_SUBN_LID, &av); + } else { + /* is a directed route - we need to construct a permissive address */ + memset(&av, 0, sizeof(av)); + /* we do not need port number since it is part of the mad_hndl */ + av.dlid = IB_LID_PERMISSIVE; + } + + /* send it */ + if ((p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || + (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { + + /* SMI CASE */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osmv_transport_mad_send: " + "av.dlid:0x%X, " + "av.static_rate:%d, " + "av.path_bits:%d.\n", + cl_ntoh16(av.dlid), av.static_rate, + av.src_path_bits); + } + + ret = IB_MGT_send_mad(p_tpot_info->smi_h, p_mad, /* actual payload */ + &av, /* address vector */ + (u_int64_t) CAST_P2LONG(p_bo), + IB_MGT_DEFAULT_SEND_TIME); + } else { + /* GSI CASE - Support Remote QP */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osmv_transport_mad_send: " + "av.dlid:0x%X, av.static_rate:%d, av.path_bits:%d, remote qp:%d \n", + cl_ntoh16(av.dlid), av.static_rate, + av.src_path_bits, + cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp) + ); + } + + ret = IB_MGT_send_mad_to_qp(p_tpot_info->gsi_h, p_mad, /* actual payload */ + &av, /* address vector */ + (u_int64_t) CAST_P2LONG(p_bo), + IB_MGT_DEFAULT_SEND_TIME, + cl_ntoh32(p_mad_addr->addr_type.gsi. + remote_qp)); + + } + + status = IB_SUCCESS; + if (ret != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osmv_transport_mad_send: ERR 7210: " + "Error sending mad (%d).\n", ret); + status = IB_ERROR; + } else { + osmv_IBMGT_transport_mgr_t *p_mgr = + (osmv_IBMGT_transport_mgr_t *) (p_bo->p_transp_mgr); + + /* Let the others work when I am sleeping ... */ + osmv_txn_unlock(p_bo); + + cl_event_wait_on(&(p_mgr->send_done), 0xffffffff, TRUE); + + /* Re-acquire the lock */ + osmv_txn_lock(p_bo); + + if (TRUE == p_bo->is_closing) { + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osmv_transport_mad_send: ERR 7211: " + "The handle %p is being unbound, cannot send.\n", + h_bind); + status = IB_ERROR; + } + + if (p_mgr->is_send_ok == FALSE) { + status = IB_ERROR; + } + } + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } +void osmv_transport_done(IN const osm_bind_handle_t h_bind) +{ + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_log_t *p_log = p_bo->p_vendor->p_log; + osmv_IBMGT_transport_mgr_t *p_mgr; + osmv_IBMGT_transport_info_t *p_tpot_info; + IB_MGT_ret_t ret; + cl_list_obj_t *p_obj = NULL; + cl_list_item_t *p_item, *p_item_tmp; + int i; + cl_qlist_t *p_list = NULL; + + OSM_LOG_ENTER(p_log, osmv_transport_done); + + CL_ASSERT(p_bo); + + /* First of all - zero out the magic_ptr, so if a callback is called - + it'll know that we are currently closing down, and will not handle the + mad. */ + p_bo->magic_ptr = 0; + + p_mgr = (osmv_IBMGT_transport_mgr_t *) (p_bo->p_transp_mgr); + p_tpot_info = + (osmv_IBMGT_transport_info_t *) (p_bo->p_vendor->p_transport_info); + + switch (p_mgr->mad_type) { + case IB_MGT_SMI: + p_list = p_tpot_info->p_smi_list; + + /* remove from the bindings list */ + p_item = cl_qlist_head(p_list); + while (p_item != cl_qlist_end(p_list)) { + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + if (cl_qlist_obj(p_obj) == h_bind) { + break; + } + p_item_tmp = cl_qlist_next(p_item); + p_item = p_item_tmp; + } + + CL_ASSERT(p_item != cl_qlist_end(p_list)); + cl_qlist_remove_item(p_list, p_item); + if (p_obj) + free(p_obj); + + /* no one is binded to smi anymore - we can free the list, unbind & realease the hndl */ + if (cl_is_qlist_empty(p_list) == TRUE) { + free(p_list); + p_list = NULL; + + ret = IB_MGT_unbind_sm(p_tpot_info->smi_h); + if (ret != IB_MGT_OK) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_done: ERR 7212: " + "Failed to unbind sm\n"); + } + + ret = IB_MGT_release_handle(p_tpot_info->smi_h); + if (ret != IB_MGT_OK) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_done: ERR 7213: " + "Failed to release smi handle\n"); + } + p_tpot_info->smi_h = 0xffffffff; + } + break; + + case IB_MGT_GSI: + p_list = p_tpot_info->gsi_mgmt_lists[p_mgr->mgmt_class]; + /* remove from the bindings list */ + p_item = cl_qlist_head(p_list); + while (p_item != cl_qlist_end(p_list)) { + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + if (cl_qlist_obj(p_obj) == h_bind) { + break; + } + p_item_tmp = cl_qlist_next(p_item); + p_item = p_item_tmp; + } + + CL_ASSERT(p_item != cl_qlist_end(p_list)); + cl_qlist_remove_item(p_list, p_item); + if (p_obj) + free(p_obj); + + /* no one is binded to this class anymore - we can free the list and unbind this class */ + if (cl_is_qlist_empty(p_list) == TRUE) { + free(p_list); + p_list = NULL; + + ret = + IB_MGT_unbind_gsi_class(p_tpot_info->gsi_h, + p_mgr->mgmt_class); + if (ret != IB_MGT_OK) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_done: ERR 7214: " + "Failed to unbind gsi class\n"); + } + } + + /* all the mgmt classes are unbinded - release gsi handle */ + for (i = 0; i < 15; i++) { + if (p_tpot_info->gsi_mgmt_lists[i] != NULL) { + break; + } + } + + if (i == 15) { + ret = IB_MGT_release_handle(p_tpot_info->gsi_h); + if (ret != IB_MGT_OK) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_transport_done: ERR 7215: " + "Failed to release gsi handle\n"); + } + p_tpot_info->gsi_h = 0xffffffff; + } + } /* end switch */ + + free(p_mgr); +} /********************************************************************** * IB_MGT Receive callback : invoked after each receive **********************************************************************/ void -__osmv_IBMGT_rcv_cb( IN IB_MGT_mad_hndl_t mad_hndl, - IN void *private_ctx_p, - IN void *payload_p, - IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p ){ - osmv_bind_obj_t *p_bo; - osm_mad_addr_t mad_addr; - cl_list_item_t* p_item; - cl_list_obj_t* p_obj; - cl_qlist_t* p_list; - ib_mad_t* p_mad = (ib_mad_t*)payload_p; - osm_vendor_t *p_vendor; - osmv_IBMGT_transport_info_t *p_tinfo; - - __osmv_IBMGT_rcv_desc_to_osm_addr(rcv_remote_info_p, - ( (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)|| - (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR)), - &mad_addr); - - /* different handling of SMI and GSI */ - if ( (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || - (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { - /* SMI CASE */ - p_bo = (osmv_bind_obj_t *)private_ctx_p; - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - p_vendor = p_bo->p_vendor; - p_tinfo = (osmv_IBMGT_transport_info_t *)p_vendor->p_transport_info; - p_list = p_tinfo->p_smi_list; - } - else - { - /* GSI CASE */ - p_bo = (osmv_bind_obj_t *)private_ctx_p; - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - p_vendor = p_bo->p_vendor; - p_tinfo = (osmv_IBMGT_transport_info_t *)p_vendor->p_transport_info; - p_list = p_tinfo->gsi_mgmt_lists[p_mad->mgmt_class]; - } - - /* go over the bindings list and send the mad, one of them will accept it, - the others will drope - */ - p_item = cl_qlist_head(p_list); - while (p_item != cl_qlist_end(p_list)) { - p_obj = PARENT_STRUCT(p_item,cl_list_obj_t,list_item); - p_bo = cl_qlist_obj(p_obj); - /* give upper layer the mad */ - osmv_dispatch_mad((osm_bind_handle_t)p_bo,payload_p,&mad_addr); - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - p_item = cl_qlist_next(p_item); - } +__osmv_IBMGT_rcv_cb(IN IB_MGT_mad_hndl_t mad_hndl, + IN void *private_ctx_p, + IN void *payload_p, + IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p) +{ + osmv_bind_obj_t *p_bo; + osm_mad_addr_t mad_addr; + cl_list_item_t *p_item; + cl_list_obj_t *p_obj; + cl_qlist_t *p_list; + ib_mad_t *p_mad = (ib_mad_t *) payload_p; + osm_vendor_t *p_vendor; + osmv_IBMGT_transport_info_t *p_tinfo; + + __osmv_IBMGT_rcv_desc_to_osm_addr(rcv_remote_info_p, + ((p_mad->mgmt_class == + IB_MCLASS_SUBN_LID) + || (p_mad->mgmt_class == + IB_MCLASS_SUBN_DIR)), &mad_addr); + + /* different handling of SMI and GSI */ + if ((p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || + (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { + /* SMI CASE */ + p_bo = (osmv_bind_obj_t *) private_ctx_p; + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + p_vendor = p_bo->p_vendor; + p_tinfo = + (osmv_IBMGT_transport_info_t *) p_vendor->p_transport_info; + p_list = p_tinfo->p_smi_list; + } else { + /* GSI CASE */ + p_bo = (osmv_bind_obj_t *) private_ctx_p; + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + p_vendor = p_bo->p_vendor; + p_tinfo = + (osmv_IBMGT_transport_info_t *) p_vendor->p_transport_info; + p_list = p_tinfo->gsi_mgmt_lists[p_mad->mgmt_class]; + } + + /* go over the bindings list and send the mad, one of them will accept it, + the others will drope + */ + p_item = cl_qlist_head(p_list); + while (p_item != cl_qlist_end(p_list)) { + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + p_bo = cl_qlist_obj(p_obj); + /* give upper layer the mad */ + osmv_dispatch_mad((osm_bind_handle_t) p_bo, payload_p, + &mad_addr); + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + p_item = cl_qlist_next(p_item); + } } /********************************************************************** * IB_MGT Send callback : invoked after each send **********************************************************************/ void -__osmv_IBMGT_send_cb( IN IB_MGT_mad_hndl_t mad_hndl, - IN u_int64_t wrid, - IN IB_comp_status_t status, - IN void *private_ctx_p ){ - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)CAST_P2LONG(wrid); - - osmv_IBMGT_transport_mgr_t* p_mgr = (osmv_IBMGT_transport_mgr_t*)p_bo->p_transp_mgr; - - /* Make sure the p_bo object is still relevant */ - if ( p_bo->magic_ptr != p_bo ) - return; - - /* we assume that each send on a bind object is synchronized, and no paralel sends - from diffrent threads with same object can be made */ - if (status == IB_COMP_SUCCESS) - { - p_mgr->is_send_ok = TRUE; - } else p_mgr->is_send_ok = FALSE; - cl_event_signal(&p_mgr->send_done); +__osmv_IBMGT_send_cb(IN IB_MGT_mad_hndl_t mad_hndl, + IN u_int64_t wrid, + IN IB_comp_status_t status, IN void *private_ctx_p) +{ + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) CAST_P2LONG(wrid); -} + osmv_IBMGT_transport_mgr_t *p_mgr = + (osmv_IBMGT_transport_mgr_t *) p_bo->p_transp_mgr; + /* Make sure the p_bo object is still relevant */ + if (p_bo->magic_ptr != p_bo) + return; + + /* we assume that each send on a bind object is synchronized, and no paralel sends + from diffrent threads with same object can be made */ + if (status == IB_COMP_SUCCESS) { + p_mgr->is_send_ok = TRUE; + } else + p_mgr->is_send_ok = FALSE; + cl_event_signal(&p_mgr->send_done); + +} /********************************************************************** * IB_MGT to OSM ADDRESS VECTOR **********************************************************************/ static void -__osmv_IBMGT_rcv_desc_to_osm_addr( - IN IB_MGT_mad_rcv_desc_t *p_rcv_desc, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr){ - /* p_mad_addr->dest_lid = p_osm->subn.sm_base_lid; - for resp we use the dest lid ... */ - p_mad_addr->dest_lid = cl_hton16(p_rcv_desc->remote_lid); - p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ - p_mad_addr->path_bits = p_rcv_desc->local_path_bits; - /* Clear the grh any way to avoid unset fields */ - memset(&p_mad_addr->addr_type.gsi.grh_info, 0, sizeof(p_mad_addr->addr_type.gsi.grh_info)); - - if (is_smi) - { - /* SMI */ - p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_rcv_desc->remote_lid); - p_mad_addr->addr_type.smi.port_num = 99; /* HACK - if used - should fail */ - } - else - { - /* GSI */ - /* seems to me there is a IBMGT bug reversing the QPN ... */ - /* Does IBMGT supposed to provide the QPN is network or HOST ? */ - p_mad_addr->addr_type.gsi.remote_qp = cl_hton32(p_rcv_desc->qp); - - - p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ - /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ - /* the full PKey table - than go by the index. */ - /* since this does not seem reasonable to me I simply use the default */ - /* There is a TAVOR limitation that only one P_KEY is supported per */ - /* QP - so QP1 must use IB_DEFAULT_PKEY */ - p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_mad_addr->addr_type.gsi.service_level = p_rcv_desc->sl; - - p_mad_addr->addr_type.gsi.global_route = p_rcv_desc->grh_flag; - /* copy the GRH data if relevant */ - if (p_mad_addr->addr_type.gsi.global_route) - { - p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = - ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, - p_rcv_desc->grh.traffic_class, - p_rcv_desc->grh.flow_label); - p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; - memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, - &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); - memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); - } - } +__osmv_IBMGT_rcv_desc_to_osm_addr(IN IB_MGT_mad_rcv_desc_t * p_rcv_desc, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr) +{ + /* p_mad_addr->dest_lid = p_osm->subn.sm_base_lid; - for resp we use the dest lid ... */ + p_mad_addr->dest_lid = cl_hton16(p_rcv_desc->remote_lid); + p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ + p_mad_addr->path_bits = p_rcv_desc->local_path_bits; + /* Clear the grh any way to avoid unset fields */ + memset(&p_mad_addr->addr_type.gsi.grh_info, 0, + sizeof(p_mad_addr->addr_type.gsi.grh_info)); + + if (is_smi) { + /* SMI */ + p_mad_addr->addr_type.smi.source_lid = + cl_hton16(p_rcv_desc->remote_lid); + p_mad_addr->addr_type.smi.port_num = 99; /* HACK - if used - should fail */ + } else { + /* GSI */ + /* seems to me there is a IBMGT bug reversing the QPN ... */ + /* Does IBMGT supposed to provide the QPN is network or HOST ? */ + p_mad_addr->addr_type.gsi.remote_qp = cl_hton32(p_rcv_desc->qp); + + p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ + /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ + /* the full PKey table - than go by the index. */ + /* since this does not seem reasonable to me I simply use the default */ + /* There is a TAVOR limitation that only one P_KEY is supported per */ + /* QP - so QP1 must use IB_DEFAULT_PKEY */ + p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_mad_addr->addr_type.gsi.service_level = p_rcv_desc->sl; + + p_mad_addr->addr_type.gsi.global_route = p_rcv_desc->grh_flag; + /* copy the GRH data if relevant */ + if (p_mad_addr->addr_type.gsi.global_route) { + p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = + ib_grh_set_ver_class_flow(p_rcv_desc->grh. + IP_version, + p_rcv_desc->grh. + traffic_class, + p_rcv_desc->grh. + flow_label); + p_mad_addr->addr_type.gsi.grh_info.hop_limit = + p_rcv_desc->grh.hop_limit; + memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, + &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); + memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); + } + } } /********************************************************************** * OSM ADDR VECTOR TO IB_MGT **********************************************************************/ void -__osmv_IBMGT_osm_addr_to_ibmgt_addr( - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT IB_ud_av_t *p_av){ - - /* For global destination or Multicast address:*/ - u_int8_t ver; - - memset( p_av, 0, sizeof( IB_ud_av_t ) ); - - p_av->src_path_bits = p_mad_addr->path_bits; - p_av->static_rate = p_mad_addr->static_rate; - p_av->dlid = cl_ntoh16(p_mad_addr->dest_lid); - - if (is_smi) - { - p_av->sl = 0; /* Just to note we use 0 here. */ - } - else - { - p_av->sl = p_mad_addr->addr_type.gsi.service_level; - p_av->grh_flag = p_mad_addr->addr_type.gsi.global_route; - - if (p_mad_addr->addr_type.gsi.global_route) - { - ib_grh_get_ver_class_flow(p_mad_addr->addr_type.gsi.grh_info.ver_class_flow, - &ver, - &p_av->traffic_class, - &p_av->flow_label); - p_av->hop_limit = p_mad_addr->addr_type.gsi.grh_info.hop_limit; - p_av->sgid_index = 0; /* we always use source GID 0 */ - memcpy(&p_av->dgid, - &p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - sizeof(ib_net64_t)); - - } - } +__osmv_IBMGT_osm_addr_to_ibmgt_addr(IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, OUT IB_ud_av_t * p_av) +{ + + /* For global destination or Multicast address: */ + u_int8_t ver; + + memset(p_av, 0, sizeof(IB_ud_av_t)); + + p_av->src_path_bits = p_mad_addr->path_bits; + p_av->static_rate = p_mad_addr->static_rate; + p_av->dlid = cl_ntoh16(p_mad_addr->dest_lid); + + if (is_smi) { + p_av->sl = 0; /* Just to note we use 0 here. */ + } else { + p_av->sl = p_mad_addr->addr_type.gsi.service_level; + p_av->grh_flag = p_mad_addr->addr_type.gsi.global_route; + + if (p_mad_addr->addr_type.gsi.global_route) { + ib_grh_get_ver_class_flow(p_mad_addr->addr_type.gsi. + grh_info.ver_class_flow, &ver, + &p_av->traffic_class, + &p_av->flow_label); + p_av->hop_limit = + p_mad_addr->addr_type.gsi.grh_info.hop_limit; + p_av->sgid_index = 0; /* we always use source GID 0 */ + memcpy(&p_av->dgid, + &p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + sizeof(ib_net64_t)); + + } + } } diff --git a/opensm/libvendor/osm_vendor_mlx_rmpp_ctx.c b/opensm/libvendor/osm_vendor_mlx_rmpp_ctx.c index b766713..303947e 100644 --- a/opensm/libvendor/osm_vendor_mlx_rmpp_ctx.c +++ b/opensm/libvendor/osm_vendor_mlx_rmpp_ctx.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -45,349 +45,317 @@ #include #include - ib_api_status_t -osmv_rmpp_send_ctx_init(osmv_rmpp_send_ctx_t* p_ctx, void* p_arbt_mad, uint32_t mad_sz, - osm_log_t* p_log) +osmv_rmpp_send_ctx_init(osmv_rmpp_send_ctx_t * p_ctx, void *p_arbt_mad, + uint32_t mad_sz, osm_log_t * p_log) { - ib_api_status_t st = IB_SUCCESS; - cl_status_t cl_st; - - CL_ASSERT(p_ctx); - if (NULL == p_arbt_mad) - { - return IB_INVALID_PARAMETER; - } - - if (osmv_mad_is_sa((ib_mad_t*)p_arbt_mad)) - { - p_ctx->is_sa_mad = TRUE; - }else p_ctx->is_sa_mad = FALSE; - - p_ctx->mad_sz = mad_sz; - - cl_event_construct(&p_ctx->event); - cl_st = cl_event_init(&p_ctx->event, FALSE); - if (cl_st != CL_SUCCESS) - { - return IB_ERROR; - } - - st= osmv_rmpp_sar_init(&p_ctx->sar,p_arbt_mad, p_ctx->mad_sz, p_ctx->is_sa_mad); - if (st == IB_SUCCESS) - { - p_ctx->window_first = 1; - p_ctx->window_last = 1; - } - - p_ctx->p_log = p_log; - return st; + ib_api_status_t st = IB_SUCCESS; + cl_status_t cl_st; + + CL_ASSERT(p_ctx); + if (NULL == p_arbt_mad) { + return IB_INVALID_PARAMETER; + } + + if (osmv_mad_is_sa((ib_mad_t *) p_arbt_mad)) { + p_ctx->is_sa_mad = TRUE; + } else + p_ctx->is_sa_mad = FALSE; + + p_ctx->mad_sz = mad_sz; + + cl_event_construct(&p_ctx->event); + cl_st = cl_event_init(&p_ctx->event, FALSE); + if (cl_st != CL_SUCCESS) { + return IB_ERROR; + } + + st = osmv_rmpp_sar_init(&p_ctx->sar, p_arbt_mad, p_ctx->mad_sz, + p_ctx->is_sa_mad); + if (st == IB_SUCCESS) { + p_ctx->window_first = 1; + p_ctx->window_last = 1; + } + + p_ctx->p_log = p_log; + return st; } -void -osmv_rmpp_send_ctx_done(IN osmv_rmpp_send_ctx_t *p_ctx) +void osmv_rmpp_send_ctx_done(IN osmv_rmpp_send_ctx_t * p_ctx) { - CL_ASSERT(p_ctx); - cl_event_destroy(&p_ctx->event); - osmv_rmpp_sar_done(&p_ctx->sar); - free(p_ctx); + CL_ASSERT(p_ctx); + cl_event_destroy(&p_ctx->event); + osmv_rmpp_sar_done(&p_ctx->sar); + free(p_ctx); } - -uint32_t -osmv_rmpp_send_ctx_get_num_segs(IN osmv_rmpp_send_ctx_t *p_send_ctx) +uint32_t osmv_rmpp_send_ctx_get_num_segs(IN osmv_rmpp_send_ctx_t * p_send_ctx) { - uint32_t data_len, data_sz, num; - - CL_ASSERT(p_send_ctx); - - if (p_send_ctx->is_sa_mad) - { - data_len = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; - data_sz = IB_SA_DATA_SIZE; - } - else - { - data_len = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; - data_sz = MAD_RMPP_DATA_SIZE; - } - - num= data_len / data_sz ; - if (0 == data_len || (data_len % data_sz) > 0) - { - num++; - } - - return num; -} + uint32_t data_len, data_sz, num; + + CL_ASSERT(p_send_ctx); + if (p_send_ctx->is_sa_mad) { + data_len = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; + data_sz = IB_SA_DATA_SIZE; + } else { + data_len = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; + data_sz = MAD_RMPP_DATA_SIZE; + } + + num = data_len / data_sz; + if (0 == data_len || (data_len % data_sz) > 0) { + num++; + } + + return num; +} ib_api_status_t -osmv_rmpp_send_ctx_get_seg(IN osmv_rmpp_send_ctx_t *p_send_ctx, - IN uint32_t seg_idx, - IN uint32_t resp_timeout, - OUT void* p_buf) +osmv_rmpp_send_ctx_get_seg(IN osmv_rmpp_send_ctx_t * p_send_ctx, + IN uint32_t seg_idx, + IN uint32_t resp_timeout, OUT void *p_buf) { - ib_api_status_t st = IB_SUCCESS; - uint32_t num_segs, paylen=0; - ib_rmpp_mad_t *p_rmpp_mad; - - OSM_LOG_ENTER(p_send_ctx->p_log,osmv_rmpp_send_ctx_get_seg); - CL_ASSERT(p_send_ctx); - - st = osmv_rmpp_sar_get_mad_seg(&p_send_ctx->sar,seg_idx, p_buf); - if (st != IB_SUCCESS) - { - goto Exit; - } - - p_rmpp_mad = (ib_rmpp_mad_t*)p_buf; - /* Set the relevant bits in the RMPP hdr */ - p_rmpp_mad->rmpp_status = IB_RMPP_STATUS_SUCCESS; - p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_ACTIVE; - p_rmpp_mad->rmpp_flags |= resp_timeout << 3; - - num_segs= osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); - - if (1 == seg_idx) - { - p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_FIRST; - - /* This is the first segment - - the reported paylen is the total amount of data. - */ - if (p_send_ctx->is_sa_mad) - { - /* sa mad hdr sz */ - paylen = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; - paylen += num_segs * (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); - } - else - { - /* mad hdr sz */ - paylen = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; - } - } - - if (seg_idx == num_segs) - { - p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_LAST; - - /* - This is the last segment - - the reported paylen is only the amount of data left on this segment. - */ - if (p_send_ctx->is_sa_mad) - { - paylen = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; - paylen -= (num_segs - 1) * IB_SA_DATA_SIZE; - paylen += (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); - } - else - { - paylen = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; - paylen -= (num_segs - 1) * (MAD_BLOCK_SIZE - MAD_RMPP_HDR_SIZE); - } - } - - p_rmpp_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_rmpp_mad->rmpp_version = 1; - p_rmpp_mad->paylen_newwin = cl_ntoh32(paylen); - p_rmpp_mad->seg_num = cl_ntoh32(seg_idx); - - Exit: - OSM_LOG_EXIT(p_send_ctx->p_log); - return st; + ib_api_status_t st = IB_SUCCESS; + uint32_t num_segs, paylen = 0; + ib_rmpp_mad_t *p_rmpp_mad; + + OSM_LOG_ENTER(p_send_ctx->p_log, osmv_rmpp_send_ctx_get_seg); + CL_ASSERT(p_send_ctx); + + st = osmv_rmpp_sar_get_mad_seg(&p_send_ctx->sar, seg_idx, p_buf); + if (st != IB_SUCCESS) { + goto Exit; + } + + p_rmpp_mad = (ib_rmpp_mad_t *) p_buf; + /* Set the relevant bits in the RMPP hdr */ + p_rmpp_mad->rmpp_status = IB_RMPP_STATUS_SUCCESS; + p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_ACTIVE; + p_rmpp_mad->rmpp_flags |= resp_timeout << 3; + + num_segs = osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); + + if (1 == seg_idx) { + p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_FIRST; + + /* This is the first segment - + the reported paylen is the total amount of data. + */ + if (p_send_ctx->is_sa_mad) { + /* sa mad hdr sz */ + paylen = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; + paylen += + num_segs * (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); + } else { + /* mad hdr sz */ + paylen = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; + } + } + + if (seg_idx == num_segs) { + p_rmpp_mad->rmpp_flags |= IB_RMPP_FLAG_LAST; + + /* + This is the last segment - + the reported paylen is only the amount of data left on this segment. + */ + if (p_send_ctx->is_sa_mad) { + paylen = p_send_ctx->mad_sz - IB_SA_MAD_HDR_SIZE; + paylen -= (num_segs - 1) * IB_SA_DATA_SIZE; + paylen += (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); + } else { + paylen = p_send_ctx->mad_sz - MAD_RMPP_HDR_SIZE; + paylen -= + (num_segs - 1) * (MAD_BLOCK_SIZE - + MAD_RMPP_HDR_SIZE); + } + } + + p_rmpp_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_rmpp_mad->rmpp_version = 1; + p_rmpp_mad->paylen_newwin = cl_ntoh32(paylen); + p_rmpp_mad->seg_num = cl_ntoh32(seg_idx); + + Exit: + OSM_LOG_EXIT(p_send_ctx->p_log); + return st; } - ib_api_status_t -osmv_rmpp_recv_ctx_init(osmv_rmpp_recv_ctx_t* p_ctx, osm_log_t* p_log) +osmv_rmpp_recv_ctx_init(osmv_rmpp_recv_ctx_t * p_ctx, osm_log_t * p_log) { - ib_api_status_t st = IB_SUCCESS; + ib_api_status_t st = IB_SUCCESS; - CL_ASSERT(p_ctx); + CL_ASSERT(p_ctx); - p_ctx->is_sa_mad = FALSE; + p_ctx->is_sa_mad = FALSE; - p_ctx->p_rbuf = malloc(sizeof(cl_qlist_t)); - if (p_ctx->p_rbuf) - { - memset(p_ctx->p_rbuf, 0, sizeof(cl_qlist_t)); - cl_qlist_init(p_ctx->p_rbuf); - p_ctx->expected_seg = 1; - } else st= IB_INSUFFICIENT_MEMORY; + p_ctx->p_rbuf = malloc(sizeof(cl_qlist_t)); + if (p_ctx->p_rbuf) { + memset(p_ctx->p_rbuf, 0, sizeof(cl_qlist_t)); + cl_qlist_init(p_ctx->p_rbuf); + p_ctx->expected_seg = 1; + } else + st = IB_INSUFFICIENT_MEMORY; - p_ctx->p_log = p_log; + p_ctx->p_log = p_log; - return st; + return st; } -void -osmv_rmpp_recv_ctx_done(IN osmv_rmpp_recv_ctx_t *p_ctx) +void osmv_rmpp_recv_ctx_done(IN osmv_rmpp_recv_ctx_t * p_ctx) { - cl_list_item_t *p_list_item; - cl_list_obj_t *p_obj; + cl_list_item_t *p_list_item; + cl_list_obj_t *p_obj; - CL_ASSERT(p_ctx); + CL_ASSERT(p_ctx); - /* go over all the items in the list and remove them */ - p_list_item = cl_qlist_remove_head(p_ctx->p_rbuf); - while ( p_list_item != cl_qlist_end(p_ctx->p_rbuf) ) { + /* go over all the items in the list and remove them */ + p_list_item = cl_qlist_remove_head(p_ctx->p_rbuf); + while (p_list_item != cl_qlist_end(p_ctx->p_rbuf)) { - p_obj = PARENT_STRUCT(p_list_item,cl_list_obj_t,list_item); + p_obj = PARENT_STRUCT(p_list_item, cl_list_obj_t, list_item); - free(cl_qlist_obj(p_obj)); - free(p_obj); + free(cl_qlist_obj(p_obj)); + free(p_obj); - p_list_item = cl_qlist_remove_head(p_ctx->p_rbuf); - } + p_list_item = cl_qlist_remove_head(p_ctx->p_rbuf); + } - osmv_rmpp_sar_done(&p_ctx->sar); + osmv_rmpp_sar_done(&p_ctx->sar); - free(p_ctx->p_rbuf); - free(p_ctx); + free(p_ctx->p_rbuf); + free(p_ctx); } - - ib_api_status_t -osmv_rmpp_recv_ctx_store_mad_seg(IN osmv_rmpp_recv_ctx_t *p_recv_ctx, IN void* p_mad) +osmv_rmpp_recv_ctx_store_mad_seg(IN osmv_rmpp_recv_ctx_t * p_recv_ctx, + IN void *p_mad) { - cl_list_obj_t* p_obj = NULL; - void* p_list_mad; - - OSM_LOG_ENTER(p_recv_ctx->p_log, osmv_rmpp_recv_ctx_store_mad_seg); - - CL_ASSERT(p_recv_ctx); - p_list_mad= malloc(MAD_BLOCK_SIZE); - if (NULL == p_list_mad) - { - return IB_INSUFFICIENT_MEMORY; - } - memset(p_list_mad, 0, MAD_BLOCK_SIZE); - memcpy(p_list_mad, p_mad, MAD_BLOCK_SIZE); - - p_obj = malloc(sizeof(cl_list_obj_t)); - if (NULL == p_obj) - { - free(p_list_mad); - return IB_INSUFFICIENT_MEMORY; - } - memset(p_obj, 0, sizeof(cl_list_obj_t)); - cl_qlist_set_obj(p_obj, p_list_mad); - - cl_qlist_insert_tail(p_recv_ctx->p_rbuf,&p_obj->list_item); - - if (osmv_mad_is_sa((ib_mad_t*)p_mad)) - { - p_recv_ctx->is_sa_mad = TRUE; - } - - return IB_SUCCESS; + cl_list_obj_t *p_obj = NULL; + void *p_list_mad; + + OSM_LOG_ENTER(p_recv_ctx->p_log, osmv_rmpp_recv_ctx_store_mad_seg); + + CL_ASSERT(p_recv_ctx); + p_list_mad = malloc(MAD_BLOCK_SIZE); + if (NULL == p_list_mad) { + return IB_INSUFFICIENT_MEMORY; + } + memset(p_list_mad, 0, MAD_BLOCK_SIZE); + memcpy(p_list_mad, p_mad, MAD_BLOCK_SIZE); + + p_obj = malloc(sizeof(cl_list_obj_t)); + if (NULL == p_obj) { + free(p_list_mad); + return IB_INSUFFICIENT_MEMORY; + } + memset(p_obj, 0, sizeof(cl_list_obj_t)); + cl_qlist_set_obj(p_obj, p_list_mad); + + cl_qlist_insert_tail(p_recv_ctx->p_rbuf, &p_obj->list_item); + + if (osmv_mad_is_sa((ib_mad_t *) p_mad)) { + p_recv_ctx->is_sa_mad = TRUE; + } + + return IB_SUCCESS; } uint32_t -osmv_rmpp_recv_ctx_get_cur_byte_num(IN osmv_rmpp_recv_ctx_t *p_recv_ctx) +osmv_rmpp_recv_ctx_get_cur_byte_num(IN osmv_rmpp_recv_ctx_t * p_recv_ctx) { - uint32_t num_segs; + uint32_t num_segs; - num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); - if (p_recv_ctx->is_sa_mad) - return ((num_segs * IB_SA_DATA_SIZE) + IB_SA_MAD_HDR_SIZE); - else return ((num_segs * MAD_RMPP_DATA_SIZE) + MAD_RMPP_HDR_SIZE); + num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); + if (p_recv_ctx->is_sa_mad) + return ((num_segs * IB_SA_DATA_SIZE) + IB_SA_MAD_HDR_SIZE); + else + return ((num_segs * MAD_RMPP_DATA_SIZE) + MAD_RMPP_HDR_SIZE); } uint32_t -osmv_rmpp_recv_ctx_get_byte_num_from_first(IN osmv_rmpp_recv_ctx_t *p_recv_ctx) +osmv_rmpp_recv_ctx_get_byte_num_from_first(IN osmv_rmpp_recv_ctx_t * p_recv_ctx) { - cl_list_item_t* p_item; - cl_list_obj_t* p_obj; - void* p_list_mad; - uint32_t num_bytes, num_segs; - - p_item = cl_qlist_head(p_recv_ctx->p_rbuf); - p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); - p_list_mad = cl_qlist_obj(p_obj); - - /* mad data sz */ - num_bytes = cl_ntoh32( ((ib_rmpp_mad_t*)p_list_mad)->paylen_newwin); - if (0 != num_bytes) - { - if (p_recv_ctx->is_sa_mad) - { - /* sa mad hdr sz */ - num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); - num_bytes -= num_segs * (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); - num_bytes += IB_SA_MAD_HDR_SIZE; - } - else - { - /* mad hdr sz */ - num_bytes += MAD_RMPP_HDR_SIZE; - } - } - - return num_bytes; + cl_list_item_t *p_item; + cl_list_obj_t *p_obj; + void *p_list_mad; + uint32_t num_bytes, num_segs; + + p_item = cl_qlist_head(p_recv_ctx->p_rbuf); + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + p_list_mad = cl_qlist_obj(p_obj); + + /* mad data sz */ + num_bytes = cl_ntoh32(((ib_rmpp_mad_t *) p_list_mad)->paylen_newwin); + if (0 != num_bytes) { + if (p_recv_ctx->is_sa_mad) { + /* sa mad hdr sz */ + num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); + num_bytes -= + num_segs * (IB_SA_MAD_HDR_SIZE - MAD_RMPP_HDR_SIZE); + num_bytes += IB_SA_MAD_HDR_SIZE; + } else { + /* mad hdr sz */ + num_bytes += MAD_RMPP_HDR_SIZE; + } + } + + return num_bytes; } uint32_t -osmv_rmpp_recv_ctx_get_byte_num_from_last(IN osmv_rmpp_recv_ctx_t *p_recv_ctx) +osmv_rmpp_recv_ctx_get_byte_num_from_last(IN osmv_rmpp_recv_ctx_t * p_recv_ctx) { - cl_list_item_t* p_item; - cl_list_obj_t* p_obj; - void* p_list_mad; - uint32_t num_bytes, num_segs; - - p_item = cl_qlist_tail(p_recv_ctx->p_rbuf); - p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); - p_list_mad = cl_qlist_obj(p_obj); - - /* mad data sz */ - num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); - num_bytes = cl_ntoh32( ((ib_rmpp_mad_t*)p_list_mad)->paylen_newwin); - - if (0 != num_bytes) - { - if (p_recv_ctx->is_sa_mad) - { - /* sa mad hdr sz */ - num_bytes += MAD_RMPP_HDR_SIZE; - num_bytes += (num_segs-1) * IB_SA_DATA_SIZE; - } - else - { - /* mad hdr sz */ - num_bytes += MAD_RMPP_HDR_SIZE; - num_bytes += (num_segs-1) * MAD_RMPP_DATA_SIZE; - } - } - - return num_bytes; + cl_list_item_t *p_item; + cl_list_obj_t *p_obj; + void *p_list_mad; + uint32_t num_bytes, num_segs; + + p_item = cl_qlist_tail(p_recv_ctx->p_rbuf); + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + p_list_mad = cl_qlist_obj(p_obj); + + /* mad data sz */ + num_segs = cl_qlist_count(p_recv_ctx->p_rbuf); + num_bytes = cl_ntoh32(((ib_rmpp_mad_t *) p_list_mad)->paylen_newwin); + + if (0 != num_bytes) { + if (p_recv_ctx->is_sa_mad) { + /* sa mad hdr sz */ + num_bytes += MAD_RMPP_HDR_SIZE; + num_bytes += (num_segs - 1) * IB_SA_DATA_SIZE; + } else { + /* mad hdr sz */ + num_bytes += MAD_RMPP_HDR_SIZE; + num_bytes += (num_segs - 1) * MAD_RMPP_DATA_SIZE; + } + } + + return num_bytes; } /* assuming that the last rmpp pkt arrived so that data member: total_bytes has the right value */ ib_api_status_t -osmv_rmpp_recv_ctx_reassemble_arbt_mad( IN osmv_rmpp_recv_ctx_t *p_recv_ctx, - IN uint32_t size, - IN void *p_arbt_mad) +osmv_rmpp_recv_ctx_reassemble_arbt_mad(IN osmv_rmpp_recv_ctx_t * p_recv_ctx, + IN uint32_t size, IN void *p_arbt_mad) { - ib_api_status_t st = IB_SUCCESS; + ib_api_status_t st = IB_SUCCESS; - CL_ASSERT(p_recv_ctx); + CL_ASSERT(p_recv_ctx); - st = osmv_rmpp_sar_init(&p_recv_ctx->sar,p_arbt_mad, size, p_recv_ctx->is_sa_mad); - if (st != IB_SUCCESS) - { - return st; - } + st = osmv_rmpp_sar_init(&p_recv_ctx->sar, p_arbt_mad, size, + p_recv_ctx->is_sa_mad); + if (st != IB_SUCCESS) { + return st; + } - st = osmv_rmpp_sar_reassemble_arbt_mad(&p_recv_ctx->sar,p_recv_ctx->p_rbuf); + st = osmv_rmpp_sar_reassemble_arbt_mad(&p_recv_ctx->sar, + p_recv_ctx->p_rbuf); - osmv_rmpp_sar_done(&p_recv_ctx->sar); + osmv_rmpp_sar_done(&p_recv_ctx->sar); - return st; + return st; } - - diff --git a/opensm/libvendor/osm_vendor_mlx_sa.c b/opensm/libvendor/osm_vendor_mlx_sa.c index 5b3f799..924a2b7 100644 --- a/opensm/libvendor/osm_vendor_mlx_sa.c +++ b/opensm/libvendor/osm_vendor_mlx_sa.c @@ -33,11 +33,9 @@ * */ - - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -47,21 +45,20 @@ #include #include - /***************************************************************************** *****************************************************************************/ /* this struct is the internal rep of the bind handle */ typedef struct _osmv_sa_bind_info { - osm_bind_handle_t h_bind; - osm_log_t *p_log; - osm_vendor_t *p_vendor; - osm_mad_pool_t *p_mad_pool; - uint64_t port_guid; - cl_event_t sync_event; - uint64_t last_lids_update_sec; - uint16_t lid; - uint16_t sm_lid; + osm_bind_handle_t h_bind; + osm_log_t *p_log; + osm_vendor_t *p_vendor; + osm_mad_pool_t *p_mad_pool; + uint64_t port_guid; + cl_event_t sync_event; + uint64_t last_lids_update_sec; + uint16_t lid; + uint16_t sm_lid; } osmv_sa_bind_info_t; /***************************************************************************** @@ -77,117 +74,106 @@ typedef struct _osmv_sa_bind_info { request structure) is attached as the p_madw->context.ni_context.node_guid */ void -__osmv_sa_mad_rcv_cb( - IN osm_madw_t *p_madw, - IN void* bind_context, - IN osm_madw_t *p_req_madw) +__osmv_sa_mad_rcv_cb(IN osm_madw_t * p_madw, + IN void *bind_context, IN osm_madw_t * p_req_madw) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)bind_context; - osmv_query_req_t *p_query_req_copy = NULL; - osmv_query_res_t query_res; - ib_sa_mad_t *p_sa_mad; - ib_net16_t mad_status; - - OSM_LOG_ENTER( p_bind->p_log, __osmv_sa_mad_rcv_cb ); - - if (! p_req_madw) - { - osm_log( p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: " - "Ignoring a non-response mad\n"); - osm_mad_pool_put(p_bind->p_mad_pool, p_madw); - goto Exit; - } - - /* obtain the sent context */ - p_query_req_copy = - (osmv_query_req_t *)(p_req_madw->context.arb_context.context1); - - /* provide the context of the original request in the result */ - query_res.query_context = p_query_req_copy->query_context; - - /* provide the resulting madw */ - query_res.p_result_madw = p_madw; - - /* update the req fields */ - p_sa_mad = ( ib_sa_mad_t * ) p_madw->p_mad; - - /* if we got a remote error track it in the status */ - mad_status = ( ib_net16_t ) ( p_sa_mad->status & IB_SMP_STATUS_MASK ); - if (mad_status != IB_SUCCESS) - { - osm_log( p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 0501: " - "Remote error:0x%04X .\n", mad_status - ); - query_res.status = IB_REMOTE_ERROR; - } - else - { - query_res.status = IB_SUCCESS; - } - - /* what if we have got back an empty mad ? */ - if (! p_madw->mad_size) - { - osm_log( p_bind->p_log, OSM_LOG_ERROR, - "__osmv_sa_mad_rcv_cb: ERR 0502: " - "Got an empty mad.\n" - ); - query_res.status = IB_ERROR; - } - - if (IB_SUCCESS == mad_status) - { - - /* if we are in not in a method response of an rmpp nature we must get only 1 */ - /* HACK: in the future we might need to be smarter for other methods... */ - if (p_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) - { - query_res.result_cnt = 1; - } - else - { + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) bind_context; + osmv_query_req_t *p_query_req_copy = NULL; + osmv_query_res_t query_res; + ib_sa_mad_t *p_sa_mad; + ib_net16_t mad_status; + + OSM_LOG_ENTER(p_bind->p_log, __osmv_sa_mad_rcv_cb); + + if (!p_req_madw) { + osm_log(p_bind->p_log, OSM_LOG_DEBUG, + "__osmv_sa_mad_rcv_cb: " + "Ignoring a non-response mad\n"); + osm_mad_pool_put(p_bind->p_mad_pool, p_madw); + goto Exit; + } + + /* obtain the sent context */ + p_query_req_copy = + (osmv_query_req_t *) (p_req_madw->context.arb_context.context1); + + /* provide the context of the original request in the result */ + query_res.query_context = p_query_req_copy->query_context; + + /* provide the resulting madw */ + query_res.p_result_madw = p_madw; + + /* update the req fields */ + p_sa_mad = (ib_sa_mad_t *) p_madw->p_mad; + + /* if we got a remote error track it in the status */ + mad_status = (ib_net16_t) (p_sa_mad->status & IB_SMP_STATUS_MASK); + if (mad_status != IB_SUCCESS) { + osm_log(p_bind->p_log, OSM_LOG_ERROR, + "__osmv_sa_mad_rcv_cb: ERR 0501: " + "Remote error:0x%04X .\n", mad_status); + query_res.status = IB_REMOTE_ERROR; + } else { + query_res.status = IB_SUCCESS; + } + + /* what if we have got back an empty mad ? */ + if (!p_madw->mad_size) { + osm_log(p_bind->p_log, OSM_LOG_ERROR, + "__osmv_sa_mad_rcv_cb: ERR 0502: " + "Got an empty mad.\n"); + query_res.status = IB_ERROR; + } + + if (IB_SUCCESS == mad_status) { + + /* if we are in not in a method response of an rmpp nature we must get only 1 */ + /* HACK: in the future we might need to be smarter for other methods... */ + if (p_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) { + query_res.result_cnt = 1; + } else { #ifndef VENDOR_RMPP_SUPPORT - if (mad_status != IB_SUCCESS) - query_res.result_cnt = 0; - else - query_res.result_cnt = 1; + if (mad_status != IB_SUCCESS) + query_res.result_cnt = 0; + else + query_res.result_cnt = 1; #else - /* we used the offset value to calculate the number of - records in here */ - query_res.result_cnt = - (uintn_t) - ( ( p_madw->mad_size - IB_SA_MAD_HDR_SIZE ) / - ib_get_attr_size( p_sa_mad->attr_offset ) ); - osm_log( p_bind->p_log, OSM_LOG_DEBUG, - "__osmv_sa_mad_rcv_cb: Count = %u = %u / %u (%u)\n", - query_res.result_cnt, p_madw->mad_size - IB_SA_MAD_HDR_SIZE, - ib_get_attr_size( p_sa_mad->attr_offset ), - ( p_madw->mad_size - IB_SA_MAD_HDR_SIZE ) % - ib_get_attr_size( p_sa_mad->attr_offset ) - ); + /* we used the offset value to calculate the number of + records in here */ + query_res.result_cnt = (uintn_t) + ((p_madw->mad_size - IB_SA_MAD_HDR_SIZE) / + ib_get_attr_size(p_sa_mad->attr_offset)); + osm_log(p_bind->p_log, OSM_LOG_DEBUG, + "__osmv_sa_mad_rcv_cb: Count = %u = %u / %u (%u)\n", + query_res.result_cnt, + p_madw->mad_size - IB_SA_MAD_HDR_SIZE, + ib_get_attr_size(p_sa_mad->attr_offset), + (p_madw->mad_size - + IB_SA_MAD_HDR_SIZE) % + ib_get_attr_size(p_sa_mad->attr_offset) + ); #endif - } - } + } + } - query_res.query_type = p_query_req_copy->query_type; + query_res.query_type = p_query_req_copy->query_type; - p_query_req_copy->pfn_query_cb( &query_res ); + p_query_req_copy->pfn_query_cb(&query_res); - if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) - cl_event_signal( &p_bind->sync_event ); + if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) + cl_event_signal(&p_bind->sync_event); - Exit: + Exit: - /* free the copied query request if found */ - if (p_query_req_copy) free(p_query_req_copy); + /* free the copied query request if found */ + if (p_query_req_copy) + free(p_query_req_copy); - /* put back the request madw */ - if (p_req_madw) - osm_mad_pool_put(p_bind->p_mad_pool, p_req_madw); + /* put back the request madw */ + if (p_req_madw) + osm_mad_pool_put(p_bind->p_mad_pool, p_req_madw); - OSM_LOG_EXIT( p_bind->p_log ); + OSM_LOG_EXIT(p_bind->p_log); } /***************************************************************************** @@ -197,39 +183,37 @@ __osmv_sa_mad_rcv_cb( Only report the error and get rid of the mad wrapper */ -void -__osmv_sa_mad_err_cb( - IN void* bind_context, - IN osm_madw_t *p_madw) +void __osmv_sa_mad_err_cb(IN void *bind_context, IN osm_madw_t * p_madw) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)bind_context; - osmv_query_req_t *p_query_req_copy = NULL; - osmv_query_res_t query_res; + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) bind_context; + osmv_query_req_t *p_query_req_copy = NULL; + osmv_query_res_t query_res; - OSM_LOG_ENTER( p_bind->p_log, __osmv_sa_mad_err_cb ); + OSM_LOG_ENTER(p_bind->p_log, __osmv_sa_mad_err_cb); - /* Obtain the sent context etc */ - p_query_req_copy = - (osmv_query_req_t *)(p_madw->context.arb_context.context1); + /* Obtain the sent context etc */ + p_query_req_copy = + (osmv_query_req_t *) (p_madw->context.arb_context.context1); - /* provide the context of the original request in the result */ - query_res.query_context = p_query_req_copy->query_context; + /* provide the context of the original request in the result */ + query_res.query_context = p_query_req_copy->query_context; - query_res.p_result_madw = p_madw; + query_res.p_result_madw = p_madw; - query_res.status = IB_TIMEOUT; - query_res.result_cnt = 0; - query_res.p_result_madw->status = IB_TIMEOUT; - p_madw->status = IB_TIMEOUT; - query_res.query_type = p_query_req_copy->query_type; + query_res.status = IB_TIMEOUT; + query_res.result_cnt = 0; + query_res.p_result_madw->status = IB_TIMEOUT; + p_madw->status = IB_TIMEOUT; + query_res.query_type = p_query_req_copy->query_type; - p_query_req_copy->pfn_query_cb( &query_res ); + p_query_req_copy->pfn_query_cb(&query_res); - if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) - cl_event_signal( &p_bind->sync_event ); + if ((p_query_req_copy->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC) + cl_event_signal(&p_bind->sync_event); - if (p_query_req_copy) free(p_query_req_copy); - OSM_LOG_EXIT( p_bind->p_log ); + if (p_query_req_copy) + free(p_query_req_copy); + OSM_LOG_EXIT(p_bind->p_log); } /***************************************************************************** @@ -241,197 +225,176 @@ __osmv_sa_mad_err_cb( we query for multiple guids. *****************************************************************************/ ib_api_status_t -__osmv_get_lid_and_sm_lid_by_port_guid( - IN osm_vendor_t* const p_vend, - IN ib_net64_t port_guid, - IN OUT uint64_t* p_lids_update_time_sec, - OUT uint16_t* lid, - OUT uint16_t* sm_lid) +__osmv_get_lid_and_sm_lid_by_port_guid(IN osm_vendor_t * const p_vend, + IN ib_net64_t port_guid, + IN OUT uint64_t * p_lids_update_time_sec, + OUT uint16_t * lid, + OUT uint16_t * sm_lid) { - ib_api_status_t status; - ib_port_attr_t *p_attr_array; - uint32_t num_ports; - uint32_t port_num; - - OSM_LOG_ENTER( p_vend->p_log, __osmv_get_lid_and_sm_lid_by_port_guid ); - - /* use prevous values if current time is close enough to previous query */ - if (cl_get_time_stamp_sec() <= *p_lids_update_time_sec + 30) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Using previously stored lid:0x%04x sm_lid:0x%04x\n", - *lid, *sm_lid - ); - status = IB_SUCCESS; - goto Exit; - } - - /* obtain the number of available ports */ - num_ports = 0; - status = osm_vendor_get_all_port_attr(p_vend, NULL, &num_ports); - if (status != IB_INSUFFICIENT_MEMORY) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 0503: " - "expected to get the IB_INSUFFICIENT_MEMORY but got: %s\n", - ib_get_err_str(status) - ); - status = IB_ERROR; - goto Exit; - } - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Found total of %u ports. Looking for guid:0x%016" PRIx64 "\n", - num_ports, cl_ntoh64(port_guid) - ); - - /* allocate the attributes */ - p_attr_array = - (ib_port_attr_t *)malloc(sizeof(ib_port_attr_t)*num_ports); - - /* obtain the attributes */ - status = osm_vendor_get_all_port_attr(p_vend, p_attr_array, &num_ports); - if (status != IB_SUCCESS) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 0504: " - "Fail to get port attributes (error: %s)\n", - ib_get_err_str(status) - ); - free(p_attr_array); - goto Exit; - } - - status = IB_ERROR; - /* find the port requested in the list */ - for (port_num = 0; (port_nump_log, OSM_LOG_DEBUG, - "__osmv_get_lid_and_sm_lid_by_port_guid: " - "Found guid:0x%016" PRIx64 " with idx:%d\n", - cl_ntoh64(port_guid), port_num); - } - } - - free(p_attr_array); - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + ib_port_attr_t *p_attr_array; + uint32_t num_ports; + uint32_t port_num; + + OSM_LOG_ENTER(p_vend->p_log, __osmv_get_lid_and_sm_lid_by_port_guid); + + /* use prevous values if current time is close enough to previous query */ + if (cl_get_time_stamp_sec() <= *p_lids_update_time_sec + 30) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Using previously stored lid:0x%04x sm_lid:0x%04x\n", + *lid, *sm_lid); + status = IB_SUCCESS; + goto Exit; + } + + /* obtain the number of available ports */ + num_ports = 0; + status = osm_vendor_get_all_port_attr(p_vend, NULL, &num_ports); + if (status != IB_INSUFFICIENT_MEMORY) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 0503: " + "expected to get the IB_INSUFFICIENT_MEMORY but got: %s\n", + ib_get_err_str(status) + ); + status = IB_ERROR; + goto Exit; + } + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Found total of %u ports. Looking for guid:0x%016" PRIx64 "\n", + num_ports, cl_ntoh64(port_guid) + ); + + /* allocate the attributes */ + p_attr_array = + (ib_port_attr_t *) malloc(sizeof(ib_port_attr_t) * num_ports); + + /* obtain the attributes */ + status = osm_vendor_get_all_port_attr(p_vend, p_attr_array, &num_ports); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osmv_get_lid_and_sm_lid_by_port_guid: ERR 0504: " + "Fail to get port attributes (error: %s)\n", + ib_get_err_str(status) + ); + free(p_attr_array); + goto Exit; + } + + status = IB_ERROR; + /* find the port requested in the list */ + for (port_num = 0; (port_num < num_ports) && (status == IB_ERROR); + port_num++) { + if (p_attr_array[port_num].port_guid == port_guid) { + *lid = p_attr_array[port_num].lid; + *sm_lid = p_attr_array[port_num].sm_lid; + *p_lids_update_time_sec = cl_get_time_stamp_sec(); + status = IB_SUCCESS; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osmv_get_lid_and_sm_lid_by_port_guid: " + "Found guid:0x%016" PRIx64 " with idx:%d\n", + cl_ntoh64(port_guid), port_num); + } + } + + free(p_attr_array); + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /***************************************************************************** *****************************************************************************/ osm_bind_handle_t -osmv_bind_sa( - IN osm_vendor_t* const p_vend, - IN osm_mad_pool_t* const p_mad_pool, - IN ib_net64_t port_guid - ) +osmv_bind_sa(IN osm_vendor_t * const p_vend, + IN osm_mad_pool_t * const p_mad_pool, IN ib_net64_t port_guid) { - osm_bind_info_t bind_info; - osm_log_t *p_log = p_vend->p_log; - ib_api_status_t status = IB_SUCCESS; - osmv_sa_bind_info_t *p_sa_bind_info; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_log, osmv_bind_sa ); - - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_bind_sa: " - "Binding to port 0x%" PRIx64 ".\n", - cl_ntoh64( port_guid ) ); - - bind_info.port_guid = port_guid; - bind_info.mad_class = IB_MCLASS_SUBN_ADM; - bind_info.class_version = 2; - bind_info.is_responder = TRUE; - bind_info.is_trap_processor = FALSE; - bind_info.is_report_processor = TRUE; - bind_info.send_q_size = 256; - bind_info.recv_q_size = 256; - - /* allocate the new sa bind info */ - p_sa_bind_info = - (osmv_sa_bind_info_t *)malloc(sizeof(osmv_sa_bind_info_t)); - if (! p_sa_bind_info) - { - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 0505: " - "Fail to allocate new bidn structure\n" ); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - goto Exit; - } - - /* store some important context */ - p_sa_bind_info->p_log = p_log; - p_sa_bind_info->port_guid = port_guid; - p_sa_bind_info->p_mad_pool = p_mad_pool; - p_sa_bind_info->p_vendor = p_vend; - p_sa_bind_info->last_lids_update_sec = 0; - - /* Bind to the lower level */ - p_sa_bind_info->h_bind = - osm_vendor_bind( p_vend, - &bind_info, - p_mad_pool, - __osmv_sa_mad_rcv_cb, - __osmv_sa_mad_err_cb, - p_sa_bind_info); /* context provided to CBs */ - - if (p_sa_bind_info->h_bind == OSM_BIND_INVALID_HANDLE) - { - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 0506: " - "Fail to bind to vendor SMI.\n" ); - goto Exit; - } - - /* obtain the sm_lid from the vendor */ - status = - __osmv_get_lid_and_sm_lid_by_port_guid( - p_vend, port_guid, - &p_sa_bind_info->last_lids_update_sec, - &p_sa_bind_info->lid, - &p_sa_bind_info->sm_lid); - if (status != IB_SUCCESS) - { - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 0507: " - "Fail to obtain the sm lid.\n" ); - goto Exit; - } - - /* initialize the sync_event */ - cl_event_construct( &p_sa_bind_info->sync_event ); - cl_status = cl_event_init( &p_sa_bind_info->sync_event, TRUE ); - if( cl_status != CL_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osmv_bind_sa: ERR 0508: " - "cl_init_event failed: %s\n", - ib_get_err_str(cl_status) - ); - free(p_sa_bind_info); - p_sa_bind_info = OSM_BIND_INVALID_HANDLE; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return ( p_sa_bind_info ); + osm_bind_info_t bind_info; + osm_log_t *p_log = p_vend->p_log; + ib_api_status_t status = IB_SUCCESS; + osmv_sa_bind_info_t *p_sa_bind_info; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_log, osmv_bind_sa); + + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_bind_sa: " + "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64(port_guid)); + + bind_info.port_guid = port_guid; + bind_info.mad_class = IB_MCLASS_SUBN_ADM; + bind_info.class_version = 2; + bind_info.is_responder = TRUE; + bind_info.is_trap_processor = FALSE; + bind_info.is_report_processor = TRUE; + bind_info.send_q_size = 256; + bind_info.recv_q_size = 256; + + /* allocate the new sa bind info */ + p_sa_bind_info = + (osmv_sa_bind_info_t *) malloc(sizeof(osmv_sa_bind_info_t)); + if (!p_sa_bind_info) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 0505: " + "Fail to allocate new bidn structure\n"); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + goto Exit; + } + + /* store some important context */ + p_sa_bind_info->p_log = p_log; + p_sa_bind_info->port_guid = port_guid; + p_sa_bind_info->p_mad_pool = p_mad_pool; + p_sa_bind_info->p_vendor = p_vend; + p_sa_bind_info->last_lids_update_sec = 0; + + /* Bind to the lower level */ + p_sa_bind_info->h_bind = osm_vendor_bind(p_vend, &bind_info, p_mad_pool, __osmv_sa_mad_rcv_cb, __osmv_sa_mad_err_cb, p_sa_bind_info); /* context provided to CBs */ + + if (p_sa_bind_info->h_bind == OSM_BIND_INVALID_HANDLE) { + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 0506: " + "Fail to bind to vendor SMI.\n"); + goto Exit; + } + + /* obtain the sm_lid from the vendor */ + status = + __osmv_get_lid_and_sm_lid_by_port_guid(p_vend, port_guid, + &p_sa_bind_info-> + last_lids_update_sec, + &p_sa_bind_info->lid, + &p_sa_bind_info->sm_lid); + if (status != IB_SUCCESS) { + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 0507: " + "Fail to obtain the sm lid.\n"); + goto Exit; + } + + /* initialize the sync_event */ + cl_event_construct(&p_sa_bind_info->sync_event); + cl_status = cl_event_init(&p_sa_bind_info->sync_event, TRUE); + if (cl_status != CL_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "osmv_bind_sa: ERR 0508: " + "cl_init_event failed: %s\n", ib_get_err_str(cl_status) + ); + free(p_sa_bind_info); + p_sa_bind_info = OSM_BIND_INVALID_HANDLE; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (p_sa_bind_info); } /***************************************************************************** @@ -447,15 +410,14 @@ osmv_bind_sa( * * SYNOPSIS */ -typedef struct _osmv_sa_mad_data -{ - /* MAD data. */ - uint8_t method; - ib_net16_t attr_id; - ib_net16_t attr_offset; - ib_net32_t attr_mod; - ib_net64_t comp_mask; - void *p_attr; +typedef struct _osmv_sa_mad_data { + /* MAD data. */ + uint8_t method; + ib_net16_t attr_id; + ib_net16_t attr_offset; + ib_net32_t attr_mod; + ib_net64_t comp_mask; + void *p_attr; } osmv_sa_mad_data_t; /* * method @@ -482,142 +444,130 @@ typedef struct _osmv_sa_mad_data *****************************************************************************/ /* Send a MAD out on the GSI interface */ ib_api_status_t -__osmv_send_sa_req( - IN osmv_sa_bind_info_t* p_bind, - IN const osmv_sa_mad_data_t * const p_sa_mad_data, - IN const osmv_query_req_t * const p_query_req ) +__osmv_send_sa_req(IN osmv_sa_bind_info_t * p_bind, + IN const osmv_sa_mad_data_t * const p_sa_mad_data, + IN const osmv_query_req_t * const p_query_req) { - ib_api_status_t status; - ib_mad_t *p_mad_hdr; - ib_sa_mad_t *p_sa_mad; - osm_madw_t *p_madw; - osm_log_t *p_log = p_bind->p_log; - static atomic32_t trans_id; - boolean_t sync; - osmv_query_req_t *p_query_req_copy; - - OSM_LOG_ENTER( p_log, __osmv_send_sa_req ); - - /* - since the sm_lid might change we obtain it every send - (actually it is cached in the bind object and refreshed - every 30sec by this proc ) - */ - status = - __osmv_get_lid_and_sm_lid_by_port_guid( - p_bind->p_vendor, p_bind->port_guid, - &p_bind->last_lids_update_sec, - &p_bind->lid, - &p_bind->sm_lid); - if (status != IB_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 0509: " - "Fail to obtain the sm lid.\n" ); - goto Exit; - } - - /* Get a MAD wrapper for the send */ - p_madw = osm_mad_pool_get( - p_bind->p_mad_pool, - p_bind->h_bind, - MAD_BLOCK_SIZE, - NULL ); - - if( p_madw == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osmv_send_sa_req: ERR 0510: " - "Unable to acquire MAD.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* Initialize the Sent MAD: */ - - /* Initialize the MAD buffer for the send operation. */ - p_mad_hdr = osm_madw_get_mad_ptr( p_madw ); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - /* Get a new transaction Id */ - cl_atomic_inc( &trans_id ); - - /* Cleanup the MAD from any residue */ - memset(p_sa_mad, 0, MAD_BLOCK_SIZE); - - /* Initialize the standard MAD header. */ - ib_mad_init_new( - p_mad_hdr, /* mad pointer */ - IB_MCLASS_SUBN_ADM, /* class */ - ( uint8_t ) 2, /* version */ - p_sa_mad_data->method, /* method */ - cl_hton64( ( uint64_t ) trans_id ),/* tid */ - p_sa_mad_data->attr_id, /* attr id */ - p_sa_mad_data->attr_mod /* attr mod */ - ); - - /* Set the query information. */ - p_sa_mad->sm_key = p_query_req->sm_key; - p_sa_mad->attr_offset = 0; - p_sa_mad->comp_mask = p_sa_mad_data->comp_mask; - if( p_sa_mad->comp_mask ) - { - memcpy( p_sa_mad->data, p_sa_mad_data->p_attr, - ib_get_attr_size(p_sa_mad_data->attr_offset)); - } - - /* - Provide the address to send to - */ - /* Patch to handle IBAL - host order , where it should take destination lid in network order */ + ib_api_status_t status; + ib_mad_t *p_mad_hdr; + ib_sa_mad_t *p_sa_mad; + osm_madw_t *p_madw; + osm_log_t *p_log = p_bind->p_log; + static atomic32_t trans_id; + boolean_t sync; + osmv_query_req_t *p_query_req_copy; + + OSM_LOG_ENTER(p_log, __osmv_send_sa_req); + + /* + since the sm_lid might change we obtain it every send + (actually it is cached in the bind object and refreshed + every 30sec by this proc ) + */ + status = + __osmv_get_lid_and_sm_lid_by_port_guid(p_bind->p_vendor, + p_bind->port_guid, + &p_bind-> + last_lids_update_sec, + &p_bind->lid, + &p_bind->sm_lid); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osmv_send_sa_req: ERR 0509: " + "Fail to obtain the sm lid.\n"); + goto Exit; + } + + /* Get a MAD wrapper for the send */ + p_madw = osm_mad_pool_get(p_bind->p_mad_pool, + p_bind->h_bind, MAD_BLOCK_SIZE, NULL); + + if (p_madw == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osmv_send_sa_req: ERR 0510: " + "Unable to acquire MAD.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* Initialize the Sent MAD: */ + + /* Initialize the MAD buffer for the send operation. */ + p_mad_hdr = osm_madw_get_mad_ptr(p_madw); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + /* Get a new transaction Id */ + cl_atomic_inc(&trans_id); + + /* Cleanup the MAD from any residue */ + memset(p_sa_mad, 0, MAD_BLOCK_SIZE); + + /* Initialize the standard MAD header. */ + ib_mad_init_new(p_mad_hdr, /* mad pointer */ + IB_MCLASS_SUBN_ADM, /* class */ + (uint8_t) 2, /* version */ + p_sa_mad_data->method, /* method */ + cl_hton64((uint64_t) trans_id), /* tid */ + p_sa_mad_data->attr_id, /* attr id */ + p_sa_mad_data->attr_mod /* attr mod */ + ); + + /* Set the query information. */ + p_sa_mad->sm_key = p_query_req->sm_key; + p_sa_mad->attr_offset = 0; + p_sa_mad->comp_mask = p_sa_mad_data->comp_mask; + if (p_sa_mad->comp_mask) { + memcpy(p_sa_mad->data, p_sa_mad_data->p_attr, + ib_get_attr_size(p_sa_mad_data->attr_offset)); + } + + /* + Provide the address to send to + */ + /* Patch to handle IBAL - host order , where it should take destination lid in network order */ #ifdef OSM_VENDOR_INTF_AL - p_madw->mad_addr.dest_lid = p_bind->sm_lid; + p_madw->mad_addr.dest_lid = p_bind->sm_lid; #else - p_madw->mad_addr.dest_lid = cl_hton16(p_bind->sm_lid); + p_madw->mad_addr.dest_lid = cl_hton16(p_bind->sm_lid); #endif - p_madw->mad_addr.addr_type.smi.source_lid = - cl_hton16(p_bind->lid); - p_madw->mad_addr.addr_type.gsi.remote_qp = CL_HTON32(1); - p_madw->mad_addr.addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - p_madw->mad_addr.addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_madw->resp_expected = TRUE; - p_madw->fail_msg = CL_DISP_MSGID_NONE; - - /* - Provide MAD context such that the call back will know what to do. - We have to keep the entire request structure so we know the CB. - Since we can not rely on the client to keep it arroud until - the response - we duplicate it and will later dispose it (in CB). - To store on the MADW we cast it into what opensm has: - p_madw->context.arb_context.context1 - */ - p_query_req_copy = malloc(sizeof(*p_query_req_copy)); - *p_query_req_copy = *p_query_req; - p_madw->context.arb_context.context1 = p_query_req_copy; - - /* we can support async as well as sync calls */ - sync = ((p_query_req->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC); - - /* send the mad asynchronously */ - status = osm_vendor_send( - osm_madw_get_bind_handle( p_madw ), - p_madw, - p_madw->resp_expected ); - - /* if synchronous - wait on the event */ - if (sync) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__osmv_send_sa_req: " - "Waiting for async event.\n" ); - cl_event_wait_on( &p_bind->sync_event, EVENT_NO_TIMEOUT, FALSE ); - cl_event_reset(&p_bind->sync_event); - status = p_madw->status; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return status; + p_madw->mad_addr.addr_type.smi.source_lid = cl_hton16(p_bind->lid); + p_madw->mad_addr.addr_type.gsi.remote_qp = CL_HTON32(1); + p_madw->mad_addr.addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + p_madw->mad_addr.addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_madw->resp_expected = TRUE; + p_madw->fail_msg = CL_DISP_MSGID_NONE; + + /* + Provide MAD context such that the call back will know what to do. + We have to keep the entire request structure so we know the CB. + Since we can not rely on the client to keep it arroud until + the response - we duplicate it and will later dispose it (in CB). + To store on the MADW we cast it into what opensm has: + p_madw->context.arb_context.context1 + */ + p_query_req_copy = malloc(sizeof(*p_query_req_copy)); + *p_query_req_copy = *p_query_req; + p_madw->context.arb_context.context1 = p_query_req_copy; + + /* we can support async as well as sync calls */ + sync = ((p_query_req->flags & OSM_SA_FLAGS_SYNC) == OSM_SA_FLAGS_SYNC); + + /* send the mad asynchronously */ + status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), + p_madw, p_madw->resp_expected); + + /* if synchronous - wait on the event */ + if (sync) { + osm_log(p_log, OSM_LOG_DEBUG, + "__osmv_send_sa_req: " "Waiting for async event.\n"); + cl_event_wait_on(&p_bind->sync_event, EVENT_NO_TIMEOUT, FALSE); + cl_event_reset(&p_bind->sync_event); + status = p_madw->status; + } + + Exit: + OSM_LOG_EXIT(p_log); + return status; } /***************************************************************************** @@ -626,243 +576,253 @@ __osmv_send_sa_req( * Query the SA based on the user's request. */ ib_api_status_t -osmv_query_sa( - IN osm_bind_handle_t h_bind, - IN const osmv_query_req_t * const p_query_req - ) +osmv_query_sa(IN osm_bind_handle_t h_bind, + IN const osmv_query_req_t * const p_query_req) { - osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *)h_bind; - osmv_sa_mad_data_t sa_mad_data; - osmv_user_query_t *p_user_query; - ib_service_record_t svc_rec; - ib_node_record_t node_rec; - ib_portinfo_record_t port_info; - ib_path_rec_t path_rec; - ib_class_port_info_t class_port_info; - osm_log_t *p_log = p_bind->p_log; - ib_api_status_t status; - - OSM_LOG_ENTER( p_log, osmv_query_sa ); - - /* Set the request information. */ - sa_mad_data.method = IB_MAD_METHOD_GETTABLE; - sa_mad_data.attr_mod = 0; - - /* Set the MAD attributes and component mask correctly. */ - switch ( p_query_req->query_type ) - { - - case OSMV_QUERY_USER_DEFINED: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "USER_DEFINED\n" ); - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - if (p_user_query->method) sa_mad_data.method = p_user_query->method; - sa_mad_data.attr_offset = p_user_query->attr_offset; - sa_mad_data.attr_id = p_user_query->attr_id; - sa_mad_data.attr_mod = p_user_query->attr_mod; - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_ALL_SVC_RECS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n" ); - sa_mad_data.method = IB_MAD_METHOD_GETTABLE; - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.comp_mask = 0; - sa_mad_data.p_attr = &svc_rec; - break; - - case OSMV_QUERY_SVC_REC_BY_NAME: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n" ); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.p_attr = &svc_rec; - memcpy( svc_rec.service_name, p_query_req->p_query_input, - sizeof( ib_svc_name_t ) ); - break; - - case OSMV_QUERY_SVC_REC_BY_ID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s", "SVC_REC_BY_ID\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; - sa_mad_data.comp_mask = IB_SR_COMPMASK_SID; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_service_record_t ) ); - sa_mad_data.p_attr = &svc_rec; - svc_rec.service_id = *( ib_net64_t * ) ( p_query_req->p_query_input ); - break; - - case OSMV_QUERY_CLASS_PORT_INFO: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","CLASS_PORT_INFO\n" ); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_CLASS_PORT_INFO; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_class_port_info_t ) ); - sa_mad_data.comp_mask = 0; - sa_mad_data.p_attr = &class_port_info; - - break; - - case OSMV_QUERY_NODE_REC_BY_NODE_GUID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","NODE_REC_BY_NODE_GUID\n" ); - sa_mad_data.method = IB_MAD_METHOD_GETTABLE; - sa_mad_data.attr_id = IB_MAD_ATTR_NODE_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_node_record_t ) ); - sa_mad_data.comp_mask = IB_NR_COMPMASK_NODEGUID; - sa_mad_data.p_attr = &node_rec; - node_rec.node_info.node_guid = - *( ib_net64_t * ) ( p_query_req->p_query_input ); - - break; - - case OSMV_QUERY_PORT_REC_BY_LID: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PORT_REC_BY_LID\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_portinfo_record_t ) ); - sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID; - sa_mad_data.p_attr = &port_info; - port_info.lid = *( ib_net16_t * ) ( p_query_req->p_query_input ); - break; - - case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PORT_REC_BY_LID_AND_NUM\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_portinfo_record_t ) ); - sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID | IB_PIR_COMPMASK_PORTNUM; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_VLARB_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_vl_arb_table_record_t ) ); - sa_mad_data.comp_mask = IB_VLA_COMPMASK_LID | IB_VLA_COMPMASK_OUT_PORT | IB_VLA_COMPMASK_BLOCK; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_SLVL_BY_LID_AND_PORTS: - sa_mad_data.method = IB_MAD_METHOD_GET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_SLVL_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_slvl_table_record_t ) ); - sa_mad_data.comp_mask = IB_SLVL_COMPMASK_LID | IB_SLVL_COMPMASK_OUT_PORT | IB_SLVL_COMPMASK_IN_PORT; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_PORT_GUIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID ); - sa_mad_data.p_attr = &path_rec; - ib_gid_set_default( &path_rec.dgid, - ( ( osmv_guid_pair_t * ) ( p_query_req-> - p_query_input ) )-> - dest_guid ); - ib_gid_set_default( &path_rec.sgid, - ( ( osmv_guid_pair_t * ) ( p_query_req-> - p_query_input ) )-> - src_guid ); - break; - - case OSMV_QUERY_PATH_REC_BY_GIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_GIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID ); - sa_mad_data.p_attr = &path_rec; - memcpy( &path_rec.dgid, - &( ( osmv_gid_pair_t * ) ( p_query_req->p_query_input ) )-> - dest_gid, sizeof( ib_gid_t ) ); - memcpy( &path_rec.sgid, - &( ( osmv_gid_pair_t * ) ( p_query_req->p_query_input ) )-> - src_gid, sizeof( ib_gid_t ) ); - break; - - case OSMV_QUERY_PATH_REC_BY_LIDS: - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","PATH_REC_BY_LIDS\n" ); - memset(&path_rec, 0, sizeof(ib_path_rec_t )); - sa_mad_data.method = IB_MAD_METHOD_GET; - sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_path_rec_t ) ); - sa_mad_data.comp_mask = ( IB_PR_COMPMASK_DLID | IB_PR_COMPMASK_SLID ); - sa_mad_data.p_attr = &path_rec; - path_rec.dlid = - ( ( osmv_lid_pair_t * ) ( p_query_req->p_query_input ) )->dest_lid; - path_rec.slid = - ( ( osmv_lid_pair_t * ) ( p_query_req->p_query_input ) )->src_lid; - break; - - case OSMV_QUERY_UD_MULTICAST_SET: - sa_mad_data.method = IB_MAD_METHOD_SET; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_UD_MULTICAST_SET\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_member_rec_t ) ); - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - case OSMV_QUERY_UD_MULTICAST_DELETE: - sa_mad_data.method = IB_MAD_METHOD_DELETE; - p_user_query = ( osmv_user_query_t * ) p_query_req->p_query_input; - osm_log( p_log, OSM_LOG_DEBUG, - "osmv_query_sa DBG:001 %s","OSMV_QUERY_UD_MULTICAST_DELETE\n" ); - sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; - sa_mad_data.attr_offset = - ib_get_attr_offset( sizeof( ib_member_rec_t ) ); - sa_mad_data.comp_mask = p_user_query->comp_mask; - sa_mad_data.p_attr = p_user_query->p_attr; - break; - - default: - osm_log( p_log, OSM_LOG_ERROR, - "osmv_query_sa DBG:001 %s","UNKNOWN\n" ); - CL_ASSERT( 0 ); - return IB_ERROR; - } - - status = __osmv_send_sa_req( h_bind, &sa_mad_data, p_query_req ); - - OSM_LOG_EXIT( p_log ); - return status; + osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) h_bind; + osmv_sa_mad_data_t sa_mad_data; + osmv_user_query_t *p_user_query; + ib_service_record_t svc_rec; + ib_node_record_t node_rec; + ib_portinfo_record_t port_info; + ib_path_rec_t path_rec; + ib_class_port_info_t class_port_info; + osm_log_t *p_log = p_bind->p_log; + ib_api_status_t status; + + OSM_LOG_ENTER(p_log, osmv_query_sa); + + /* Set the request information. */ + sa_mad_data.method = IB_MAD_METHOD_GETTABLE; + sa_mad_data.attr_mod = 0; + + /* Set the MAD attributes and component mask correctly. */ + switch (p_query_req->query_type) { + + case OSMV_QUERY_USER_DEFINED: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "USER_DEFINED\n"); + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + if (p_user_query->method) + sa_mad_data.method = p_user_query->method; + sa_mad_data.attr_offset = p_user_query->attr_offset; + sa_mad_data.attr_id = p_user_query->attr_id; + sa_mad_data.attr_mod = p_user_query->attr_mod; + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_ALL_SVC_RECS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + sa_mad_data.method = IB_MAD_METHOD_GETTABLE; + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.comp_mask = 0; + sa_mad_data.p_attr = &svc_rec; + break; + + case OSMV_QUERY_SVC_REC_BY_NAME: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_NAME\n"); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.p_attr = &svc_rec; + memcpy(svc_rec.service_name, p_query_req->p_query_input, + sizeof(ib_svc_name_t)); + break; + + case OSMV_QUERY_SVC_REC_BY_ID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "SVC_REC_BY_ID\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_SERVICE_RECORD; + sa_mad_data.comp_mask = IB_SR_COMPMASK_SID; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); + sa_mad_data.p_attr = &svc_rec; + svc_rec.service_id = + *(ib_net64_t *) (p_query_req->p_query_input); + break; + + case OSMV_QUERY_CLASS_PORT_INFO: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "CLASS_PORT_INFO\n"); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_CLASS_PORT_INFO; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_class_port_info_t)); + sa_mad_data.comp_mask = 0; + sa_mad_data.p_attr = &class_port_info; + + break; + + case OSMV_QUERY_NODE_REC_BY_NODE_GUID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "NODE_REC_BY_NODE_GUID\n"); + sa_mad_data.method = IB_MAD_METHOD_GETTABLE; + sa_mad_data.attr_id = IB_MAD_ATTR_NODE_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_node_record_t)); + sa_mad_data.comp_mask = IB_NR_COMPMASK_NODEGUID; + sa_mad_data.p_attr = &node_rec; + node_rec.node_info.node_guid = + *(ib_net64_t *) (p_query_req->p_query_input); + + break; + + case OSMV_QUERY_PORT_REC_BY_LID: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PORT_REC_BY_LID\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_portinfo_record_t)); + sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID; + sa_mad_data.p_attr = &port_info; + port_info.lid = *(ib_net16_t *) (p_query_req->p_query_input); + break; + + case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "PORT_REC_BY_LID_AND_NUM\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_PORTINFO_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_portinfo_record_t)); + sa_mad_data.comp_mask = + IB_PIR_COMPMASK_LID | IB_PIR_COMPMASK_PORTNUM; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_VLARB_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_vl_arb_table_record_t)); + sa_mad_data.comp_mask = + IB_VLA_COMPMASK_LID | IB_VLA_COMPMASK_OUT_PORT | + IB_VLA_COMPMASK_BLOCK; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_SLVL_BY_LID_AND_PORTS: + sa_mad_data.method = IB_MAD_METHOD_GET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_VLARB_BY_LID_PORT_BLOCK\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_SLVL_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_slvl_table_record_t)); + sa_mad_data.comp_mask = + IB_SLVL_COMPMASK_LID | IB_SLVL_COMPMASK_OUT_PORT | + IB_SLVL_COMPMASK_IN_PORT; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_PORT_GUIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID); + sa_mad_data.p_attr = &path_rec; + ib_gid_set_default(&path_rec.dgid, + ((osmv_guid_pair_t *) (p_query_req-> + p_query_input))-> + dest_guid); + ib_gid_set_default(&path_rec.sgid, + ((osmv_guid_pair_t *) (p_query_req-> + p_query_input))-> + src_guid); + break; + + case OSMV_QUERY_PATH_REC_BY_GIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_GIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID); + sa_mad_data.p_attr = &path_rec; + memcpy(&path_rec.dgid, + &((osmv_gid_pair_t *) (p_query_req->p_query_input))-> + dest_gid, sizeof(ib_gid_t)); + memcpy(&path_rec.sgid, + &((osmv_gid_pair_t *) (p_query_req->p_query_input))-> + src_gid, sizeof(ib_gid_t)); + break; + + case OSMV_QUERY_PATH_REC_BY_LIDS: + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", "PATH_REC_BY_LIDS\n"); + memset(&path_rec, 0, sizeof(ib_path_rec_t)); + sa_mad_data.method = IB_MAD_METHOD_GET; + sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_path_rec_t)); + sa_mad_data.comp_mask = + (IB_PR_COMPMASK_DLID | IB_PR_COMPMASK_SLID); + sa_mad_data.p_attr = &path_rec; + path_rec.dlid = + ((osmv_lid_pair_t *) (p_query_req->p_query_input))-> + dest_lid; + path_rec.slid = + ((osmv_lid_pair_t *) (p_query_req->p_query_input))->src_lid; + break; + + case OSMV_QUERY_UD_MULTICAST_SET: + sa_mad_data.method = IB_MAD_METHOD_SET; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_UD_MULTICAST_SET\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + case OSMV_QUERY_UD_MULTICAST_DELETE: + sa_mad_data.method = IB_MAD_METHOD_DELETE; + p_user_query = (osmv_user_query_t *) p_query_req->p_query_input; + osm_log(p_log, OSM_LOG_DEBUG, + "osmv_query_sa DBG:001 %s", + "OSMV_QUERY_UD_MULTICAST_DELETE\n"); + sa_mad_data.attr_id = IB_MAD_ATTR_MCMEMBER_RECORD; + sa_mad_data.attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); + sa_mad_data.comp_mask = p_user_query->comp_mask; + sa_mad_data.p_attr = p_user_query->p_attr; + break; + + default: + osm_log(p_log, OSM_LOG_ERROR, + "osmv_query_sa DBG:001 %s", "UNKNOWN\n"); + CL_ASSERT(0); + return IB_ERROR; + } + + status = __osmv_send_sa_req(h_bind, &sa_mad_data, p_query_req); + + OSM_LOG_EXIT(p_log); + return status; } - /***************************************************************************** *****************************************************************************/ - - diff --git a/opensm/libvendor/osm_vendor_mlx_sar.c b/opensm/libvendor/osm_vendor_mlx_sar.c index 4108ae3..5523284 100644 --- a/opensm/libvendor/osm_vendor_mlx_sar.c +++ b/opensm/libvendor/osm_vendor_mlx_sar.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -43,122 +43,112 @@ #include ib_api_status_t -osmv_rmpp_sar_init(osmv_rmpp_sar_t* p_sar, void* p_arbt_mad, - uint32_t mad_size, boolean_t is_sa_mad) +osmv_rmpp_sar_init(osmv_rmpp_sar_t * p_sar, void *p_arbt_mad, + uint32_t mad_size, boolean_t is_sa_mad) { - CL_ASSERT(p_sar); - p_sar->p_arbt_mad = p_arbt_mad; - if (is_sa_mad) - { - p_sar->data_len = mad_size - IB_SA_MAD_HDR_SIZE; - p_sar->hdr_sz = IB_SA_MAD_HDR_SIZE; - p_sar->data_sz = IB_SA_DATA_SIZE; - } - else - { - p_sar->data_len = mad_size - MAD_RMPP_HDR_SIZE; - p_sar->hdr_sz = MAD_RMPP_HDR_SIZE; - p_sar->data_sz = MAD_RMPP_DATA_SIZE; - } - return IB_SUCCESS; + CL_ASSERT(p_sar); + p_sar->p_arbt_mad = p_arbt_mad; + if (is_sa_mad) { + p_sar->data_len = mad_size - IB_SA_MAD_HDR_SIZE; + p_sar->hdr_sz = IB_SA_MAD_HDR_SIZE; + p_sar->data_sz = IB_SA_DATA_SIZE; + } else { + p_sar->data_len = mad_size - MAD_RMPP_HDR_SIZE; + p_sar->hdr_sz = MAD_RMPP_HDR_SIZE; + p_sar->data_sz = MAD_RMPP_DATA_SIZE; + } + return IB_SUCCESS; } - -void -osmv_rmpp_sar_done(osmv_rmpp_sar_t* p_sar) +void osmv_rmpp_sar_done(osmv_rmpp_sar_t * p_sar) { - p_sar->p_arbt_mad = NULL; + p_sar->p_arbt_mad = NULL; } - /* the big mad should be with mad header, rmpp header ( &sa hdr) space */ ib_api_status_t -osmv_rmpp_sar_get_mad_seg( - IN osmv_rmpp_sar_t* p_sar, - IN uint32_t seg_idx, - OUT void* p_buf) +osmv_rmpp_sar_get_mad_seg(IN osmv_rmpp_sar_t * p_sar, + IN uint32_t seg_idx, OUT void *p_buf) { - void* p_seg; - uint32_t sz_left; - uint32_t num_segs; - - CL_ASSERT(p_sar); - - num_segs = p_sar->data_len / p_sar->data_sz ; - if ((p_sar->data_len % p_sar->data_sz) > 0) - { - num_segs++; - } - - if ( (seg_idx > num_segs) && (seg_idx != 1) ) - { - return IB_NOT_FOUND; - } - - /* cleanup */ - memset(p_buf, 0, MAD_BLOCK_SIZE); - - /* attach header */ - memcpy(p_buf,p_sar->p_arbt_mad,p_sar->hdr_sz); - - /* fill data */ - p_seg = (char*)p_sar->p_arbt_mad + p_sar->hdr_sz + ((seg_idx-1) * p_sar->data_sz); - sz_left = p_sar->data_len - ((seg_idx -1) * p_sar->data_sz); - if (sz_left > p_sar->data_sz) - memcpy((char*)p_buf+p_sar->hdr_sz,(char*)p_seg,p_sar->data_sz); - else - memcpy((char*)p_buf+ p_sar->hdr_sz, (char*)p_seg, sz_left); - - return IB_SUCCESS; + void *p_seg; + uint32_t sz_left; + uint32_t num_segs; + + CL_ASSERT(p_sar); + + num_segs = p_sar->data_len / p_sar->data_sz; + if ((p_sar->data_len % p_sar->data_sz) > 0) { + num_segs++; + } + + if ((seg_idx > num_segs) && (seg_idx != 1)) { + return IB_NOT_FOUND; + } + + /* cleanup */ + memset(p_buf, 0, MAD_BLOCK_SIZE); + + /* attach header */ + memcpy(p_buf, p_sar->p_arbt_mad, p_sar->hdr_sz); + + /* fill data */ + p_seg = + (char *)p_sar->p_arbt_mad + p_sar->hdr_sz + + ((seg_idx - 1) * p_sar->data_sz); + sz_left = p_sar->data_len - ((seg_idx - 1) * p_sar->data_sz); + if (sz_left > p_sar->data_sz) + memcpy((char *)p_buf + p_sar->hdr_sz, (char *)p_seg, + p_sar->data_sz); + else + memcpy((char *)p_buf + p_sar->hdr_sz, (char *)p_seg, sz_left); + + return IB_SUCCESS; } - /* turns a list of mads to one big mad - including header */ /* ALSO - deallocates the list */ ib_api_status_t -osmv_rmpp_sar_reassemble_arbt_mad(osmv_rmpp_sar_t* p_sar, cl_qlist_t* p_bufs) +osmv_rmpp_sar_reassemble_arbt_mad(osmv_rmpp_sar_t * p_sar, cl_qlist_t * p_bufs) { - void* buf_tmp,*p_mad; - cl_list_item_t* p_item; - cl_list_obj_t* p_obj; - uint32_t space_left = p_sar->data_len + p_sar->hdr_sz; - - CL_ASSERT(p_sar); - CL_ASSERT(FALSE == cl_is_qlist_empty(p_bufs)); - - /* attach header */ - p_mad = p_sar->p_arbt_mad; - p_item = cl_qlist_head(p_bufs); - p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); - buf_tmp = cl_qlist_obj(p_obj); - memcpy(p_mad,buf_tmp,p_sar->hdr_sz); - p_mad = (char*)p_mad + p_sar->hdr_sz; - space_left-= p_sar->hdr_sz; - - /* reassemble data */ - while (FALSE == cl_is_qlist_empty(p_bufs)) { - - p_item = cl_qlist_remove_head(p_bufs); - p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); - buf_tmp = cl_qlist_obj(p_obj); - - if (FALSE == cl_is_qlist_empty(p_bufs)) - { - memcpy((char*)p_mad,(char*)buf_tmp+p_sar->hdr_sz,p_sar->data_sz); - p_mad = (char*)p_mad + p_sar->data_sz; - space_left-= p_sar->data_sz; - } - else - { - /* the last mad on the list */ - memcpy((char*)p_mad,(char*)buf_tmp+p_sar->hdr_sz,space_left); - p_mad= (char*)p_mad+space_left; - } - - free(buf_tmp); - free(p_obj); - } - - return IB_SUCCESS; + void *buf_tmp, *p_mad; + cl_list_item_t *p_item; + cl_list_obj_t *p_obj; + uint32_t space_left = p_sar->data_len + p_sar->hdr_sz; + + CL_ASSERT(p_sar); + CL_ASSERT(FALSE == cl_is_qlist_empty(p_bufs)); + + /* attach header */ + p_mad = p_sar->p_arbt_mad; + p_item = cl_qlist_head(p_bufs); + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + buf_tmp = cl_qlist_obj(p_obj); + memcpy(p_mad, buf_tmp, p_sar->hdr_sz); + p_mad = (char *)p_mad + p_sar->hdr_sz; + space_left -= p_sar->hdr_sz; + + /* reassemble data */ + while (FALSE == cl_is_qlist_empty(p_bufs)) { + + p_item = cl_qlist_remove_head(p_bufs); + p_obj = PARENT_STRUCT(p_item, cl_list_obj_t, list_item); + buf_tmp = cl_qlist_obj(p_obj); + + if (FALSE == cl_is_qlist_empty(p_bufs)) { + memcpy((char *)p_mad, (char *)buf_tmp + p_sar->hdr_sz, + p_sar->data_sz); + p_mad = (char *)p_mad + p_sar->data_sz; + space_left -= p_sar->data_sz; + } else { + /* the last mad on the list */ + memcpy((char *)p_mad, (char *)buf_tmp + p_sar->hdr_sz, + space_left); + p_mad = (char *)p_mad + space_left; + } + + free(buf_tmp); + free(p_obj); + } + + return IB_SUCCESS; } - diff --git a/opensm/libvendor/osm_vendor_mlx_sender.c b/opensm/libvendor/osm_vendor_mlx_sender.c index 624d3ee..5542e64 100644 --- a/opensm/libvendor/osm_vendor_mlx_sender.c +++ b/opensm/libvendor/osm_vendor_mlx_sender.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -44,9 +44,8 @@ #include static ib_api_status_t -__osmv_rmpp_send_segment(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN uint32_t seg_num); +__osmv_rmpp_send_segment(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, IN uint32_t seg_num); /****d* OSM Vendor/osmv_simple_send_madw * NAME @@ -62,67 +61,60 @@ __osmv_rmpp_send_segment(IN osm_bind_handle_t h_bind, ib_api_status_t osmv_simple_send_madw(IN osm_bind_handle_t h_bind, - IN osm_madw_t *const p_madw, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_retry) + IN osm_madw_t * const p_madw, + IN osmv_txn_ctx_t * p_txn, IN boolean_t is_retry) { - ib_api_status_t ret; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - osm_mad_addr_t *p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); - uint8_t mad_buf[MAD_BLOCK_SIZE]; - ib_mad_t *p_mad = (ib_mad_t*)mad_buf; - uint64_t key = 0; - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_simple_send_madw); - - CL_ASSERT( p_madw->mad_size <= MAD_BLOCK_SIZE ); - - memset(p_mad, 0, MAD_BLOCK_SIZE); - memcpy(p_mad, osm_madw_get_mad_ptr(p_madw), p_madw->mad_size); - - if (NULL != p_txn) - { - /* Push a fake txn id to the MAD */ - key = osmv_txn_get_key(p_txn); - p_mad->trans_id = cl_hton64(key); - } - - /* - Add call for packet drop randomizer. - This is a testing feature. If run_randomizer flag is set to TRUE, - the randomizer will be called, and randomally will drop - a packet. This is used for simulating unstable fabric. - */ - if ( p_bo->p_vendor->run_randomizer == TRUE ) - { - /* Try the randomizer */ - if ( osm_pkt_randomizer_mad_drop( p_bo->p_vendor->p_log, - p_bo->p_vendor->p_pkt_randomizer, - p_mad ) == TRUE ) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The MAD will not be sent. \n" ); - ret = IB_SUCCESS; - } - else - { - ret = osmv_transport_mad_send(h_bind, p_mad, p_mad_addr); - } - } - else - { - ret = osmv_transport_mad_send(h_bind, p_mad, p_mad_addr); - } - - if ((IB_SUCCESS == ret) && (NULL != p_txn) && (! is_retry)) - { - /* Set the timeout for receiving the response MAD */ - ret = osmv_txn_set_timeout_ev(h_bind, key, - p_bo->p_vendor->resp_timeout); - } - - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - return ret; + ib_api_status_t ret; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_mad_addr_t *p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + uint8_t mad_buf[MAD_BLOCK_SIZE]; + ib_mad_t *p_mad = (ib_mad_t *) mad_buf; + uint64_t key = 0; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_simple_send_madw); + + CL_ASSERT(p_madw->mad_size <= MAD_BLOCK_SIZE); + + memset(p_mad, 0, MAD_BLOCK_SIZE); + memcpy(p_mad, osm_madw_get_mad_ptr(p_madw), p_madw->mad_size); + + if (NULL != p_txn) { + /* Push a fake txn id to the MAD */ + key = osmv_txn_get_key(p_txn); + p_mad->trans_id = cl_hton64(key); + } + + /* + Add call for packet drop randomizer. + This is a testing feature. If run_randomizer flag is set to TRUE, + the randomizer will be called, and randomally will drop + a packet. This is used for simulating unstable fabric. + */ + if (p_bo->p_vendor->run_randomizer == TRUE) { + /* Try the randomizer */ + if (osm_pkt_randomizer_mad_drop(p_bo->p_vendor->p_log, + p_bo->p_vendor-> + p_pkt_randomizer, + p_mad) == TRUE) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The MAD will not be sent. \n"); + ret = IB_SUCCESS; + } else { + ret = + osmv_transport_mad_send(h_bind, p_mad, p_mad_addr); + } + } else { + ret = osmv_transport_mad_send(h_bind, p_mad, p_mad_addr); + } + + if ((IB_SUCCESS == ret) && (NULL != p_txn) && (!is_retry)) { + /* Set the timeout for receiving the response MAD */ + ret = osmv_txn_set_timeout_ev(h_bind, key, + p_bo->p_vendor->resp_timeout); + } + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return ret; } /***** OSM Vendor/osmv_rmpp_send_madw @@ -143,110 +135,107 @@ osmv_simple_send_madw(IN osm_bind_handle_t h_bind, ib_api_status_t osmv_rmpp_send_madw(IN osm_bind_handle_t h_bind, - IN osm_madw_t *const p_madw, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_rmpp_ds) + IN osm_madw_t * const p_madw, + IN osmv_txn_ctx_t * p_txn, IN boolean_t is_rmpp_ds) { - ib_api_status_t ret=IB_SUCCESS; - uint32_t i, total_segs; - - osmv_rmpp_send_ctx_t *p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_rmpp_send_madw); - - total_segs = osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); - CL_ASSERT(total_segs >= 1); - - /* In the double-sided transfer, wait for ACK 0 */ - - for (;;) { - - if (p_send_ctx->window_first > total_segs) - { - - /* Every segment is acknowledged */ - break; - } - - /* Send the next burst. */ - for (i = p_send_ctx->window_first; i <= p_send_ctx->window_last; i++) { - - /* Send a segment and setup a timeout timer */ - ret = __osmv_rmpp_send_segment(h_bind, p_txn, i); - if (IB_SUCCESS != ret) - { - goto send_done; - } - } - - /* Set the Response Timeout for the ACK on the last DATA segment */ - ret = osmv_txn_set_timeout_ev(h_bind, osmv_txn_get_key(p_txn), - p_bo->p_vendor->resp_timeout); - if (IB_SUCCESS != ret) - { - goto send_done; - } - - /* Going to sleep. Let the others access the transaction DB */ - osmv_txn_unlock(p_bo); - - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "RMPP Sender thread (madw=%p) going to sleep ...\n", p_madw); - - /* Await the next event to happen */ - cl_event_wait_on(&p_send_ctx->event, - EVENT_NO_TIMEOUT, - TRUE /* interruptible */); - - /* Got a signal from the MAD dispatcher/timeout handler */ - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "RMPP Sender thread (madw=%p) waking up on a signal ...\n", p_madw); - - /* Let's see what changed... Make this atomic - re-acquire the lock. */ - osmv_txn_lock(p_bo); - - if (TRUE == p_bo->is_closing) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_rmpp_send_madw: ERR 6601: " - "The bind handle %p is being closed. " - "Stopping the RMPP Send of MADW %p\n", - h_bind, p_madw); - - ret = IB_TIMEOUT; - return IB_INTERRUPTED; - } - - /* STOP? ABORT? TIMEOUT? */ - if (IB_SUCCESS != p_send_ctx->status) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_rmpp_send_madw: ERR 6602: " - "An error (%s) happened during the RMPP send of %p. Bailing out.\n", - ib_get_err_str(p_send_ctx->status), p_madw); - ret = p_send_ctx->status; - goto send_done; - } - } - - if (TRUE == is_rmpp_ds) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Double-sided RMPP - switching to be the receiver.\n"); - - ret = osmv_txn_init_rmpp_receiver(h_bind, p_txn, FALSE/*Send was initiated by me*/); - - if (IB_SUCCESS == ret) - { - /* Send ACK on the 0 segment */ - ret = __osmv_rmpp_send_segment(h_bind, p_txn, 0); - } - } - - send_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - return ret; + ib_api_status_t ret = IB_SUCCESS; + uint32_t i, total_segs; + + osmv_rmpp_send_ctx_t *p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_rmpp_send_madw); + + total_segs = osmv_rmpp_send_ctx_get_num_segs(p_send_ctx); + CL_ASSERT(total_segs >= 1); + + /* In the double-sided transfer, wait for ACK 0 */ + + for (;;) { + + if (p_send_ctx->window_first > total_segs) { + + /* Every segment is acknowledged */ + break; + } + + /* Send the next burst. */ + for (i = p_send_ctx->window_first; i <= p_send_ctx->window_last; + i++) { + + /* Send a segment and setup a timeout timer */ + ret = __osmv_rmpp_send_segment(h_bind, p_txn, i); + if (IB_SUCCESS != ret) { + goto send_done; + } + } + + /* Set the Response Timeout for the ACK on the last DATA segment */ + ret = osmv_txn_set_timeout_ev(h_bind, osmv_txn_get_key(p_txn), + p_bo->p_vendor->resp_timeout); + if (IB_SUCCESS != ret) { + goto send_done; + } + + /* Going to sleep. Let the others access the transaction DB */ + osmv_txn_unlock(p_bo); + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "RMPP Sender thread (madw=%p) going to sleep ...\n", + p_madw); + + /* Await the next event to happen */ + cl_event_wait_on(&p_send_ctx->event, + EVENT_NO_TIMEOUT, TRUE /* interruptible */ ); + + /* Got a signal from the MAD dispatcher/timeout handler */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "RMPP Sender thread (madw=%p) waking up on a signal ...\n", + p_madw); + + /* Let's see what changed... Make this atomic - re-acquire the lock. */ + osmv_txn_lock(p_bo); + + if (TRUE == p_bo->is_closing) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_rmpp_send_madw: ERR 6601: " + "The bind handle %p is being closed. " + "Stopping the RMPP Send of MADW %p\n", + h_bind, p_madw); + + ret = IB_TIMEOUT; + return IB_INTERRUPTED; + } + + /* STOP? ABORT? TIMEOUT? */ + if (IB_SUCCESS != p_send_ctx->status) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_rmpp_send_madw: ERR 6602: " + "An error (%s) happened during the RMPP send of %p. Bailing out.\n", + ib_get_err_str(p_send_ctx->status), p_madw); + ret = p_send_ctx->status; + goto send_done; + } + } + + if (TRUE == is_rmpp_ds) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Double-sided RMPP - switching to be the receiver.\n"); + + ret = + osmv_txn_init_rmpp_receiver(h_bind, p_txn, + FALSE + /*Send was initiated by me */ ); + + if (IB_SUCCESS == ret) { + /* Send ACK on the 0 segment */ + ret = __osmv_rmpp_send_segment(h_bind, p_txn, 0); + } + } + + send_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return ret; } /* @@ -257,33 +246,32 @@ osmv_rmpp_send_madw(IN osm_bind_handle_t h_bind, */ ib_api_status_t -osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN uint32_t seg_num, - IN uint32_t nwl, - IN const osm_mad_addr_t *p_mad_addr) +osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN uint32_t seg_num, + IN uint32_t nwl, IN const osm_mad_addr_t * p_mad_addr) { - uint8_t resp_mad[MAD_BLOCK_SIZE]; - ib_rmpp_mad_t *p_resp_mad = (ib_rmpp_mad_t*)resp_mad; + uint8_t resp_mad[MAD_BLOCK_SIZE]; + ib_rmpp_mad_t *p_resp_mad = (ib_rmpp_mad_t *) resp_mad; #ifdef OSMV_RANDOM_DROP - if (TRUE == osmv_random_drop()) - { - osm_log(((osmv_bind_obj_t*)h_bind)->p_vendor->p_log, OSM_LOG_DEBUG, - "Error injection - dropping the RMPP ACK\n"); - return IB_SUCCESS; - } + if (TRUE == osmv_random_drop()) { + osm_log(((osmv_bind_obj_t *) h_bind)->p_vendor->p_log, + OSM_LOG_DEBUG, + "Error injection - dropping the RMPP ACK\n"); + return IB_SUCCESS; + } #endif - memcpy(p_resp_mad, p_req_mad, MAD_BLOCK_SIZE); + memcpy(p_resp_mad, p_req_mad, MAD_BLOCK_SIZE); - p_resp_mad->common_hdr.method = osmv_invert_method(p_req_mad->method); - p_resp_mad->rmpp_type = IB_RMPP_TYPE_ACK; - p_resp_mad->seg_num = cl_hton32(seg_num); - p_resp_mad->paylen_newwin = cl_hton32(nwl); - p_resp_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + p_resp_mad->common_hdr.method = osmv_invert_method(p_req_mad->method); + p_resp_mad->rmpp_type = IB_RMPP_TYPE_ACK; + p_resp_mad->seg_num = cl_hton32(seg_num); + p_resp_mad->paylen_newwin = cl_hton32(nwl); + p_resp_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; - return osmv_transport_mad_send(h_bind, p_resp_mad, p_mad_addr); + return osmv_transport_mad_send(h_bind, p_resp_mad, p_mad_addr); } /* @@ -293,22 +281,21 @@ osmv_rmpp_send_ack(IN osm_bind_handle_t h_bind, */ ib_api_status_t -osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, - IN const ib_mad_t *p_req_mad, - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t nak_type, - IN uint8_t status) +osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, + IN const ib_mad_t * p_req_mad, + IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t nak_type, IN uint8_t status) { - uint8_t resp_mad[MAD_BLOCK_SIZE]; - ib_rmpp_mad_t *p_resp_mad = (ib_rmpp_mad_t*)resp_mad; + uint8_t resp_mad[MAD_BLOCK_SIZE]; + ib_rmpp_mad_t *p_resp_mad = (ib_rmpp_mad_t *) resp_mad; - memcpy(p_resp_mad, p_req_mad, MAD_BLOCK_SIZE); + memcpy(p_resp_mad, p_req_mad, MAD_BLOCK_SIZE); - p_resp_mad->common_hdr.method = osmv_invert_method(p_req_mad->method); - p_resp_mad->rmpp_type = nak_type; - p_resp_mad->rmpp_status = status; + p_resp_mad->common_hdr.method = osmv_invert_method(p_req_mad->method); + p_resp_mad->rmpp_type = nak_type; + p_resp_mad->rmpp_status = status; - return osmv_transport_mad_send(h_bind, p_resp_mad, p_mad_addr); + return osmv_transport_mad_send(h_bind, p_resp_mad, p_mad_addr); } /* @@ -318,86 +305,87 @@ osmv_rmpp_send_nak(IN osm_bind_handle_t h_bind, */ static ib_api_status_t -__osmv_rmpp_send_segment(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN uint32_t seg_num) +__osmv_rmpp_send_segment(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, IN uint32_t seg_num) { - ib_api_status_t ret; - osmv_rmpp_send_ctx_t *p_send_ctx; - uint8_t mad_buf[MAD_BLOCK_SIZE]; - ib_mad_t *p_mad = (ib_mad_t*)mad_buf; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - osm_mad_addr_t *p_mad_addr = osm_madw_get_mad_addr_ptr(osmv_txn_get_madw(p_txn)); - uint32_t timeout = p_bo->p_vendor->resp_timeout; - uint64_t key; - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_rmpp_send_segment); + ib_api_status_t ret; + osmv_rmpp_send_ctx_t *p_send_ctx; + uint8_t mad_buf[MAD_BLOCK_SIZE]; + ib_mad_t *p_mad = (ib_mad_t *) mad_buf; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_mad_addr_t *p_mad_addr = + osm_madw_get_mad_addr_ptr(osmv_txn_get_madw(p_txn)); + uint32_t timeout = p_bo->p_vendor->resp_timeout; + uint64_t key; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_rmpp_send_segment); #ifdef OSMV_RANDOM_DROP - if (TRUE == osmv_random_drop()) - { + if (TRUE == osmv_random_drop()) { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Error injection - simulating the RMPP segment drop\n"); - return IB_SUCCESS; - } + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Error injection - simulating the RMPP segment drop\n"); + return IB_SUCCESS; + } #endif - p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); - key = osmv_txn_get_key(p_txn); - - if (0 != seg_num) - { - ret = osmv_rmpp_send_ctx_get_seg(p_send_ctx, seg_num, timeout, p_mad); - CL_ASSERT(IB_SUCCESS == ret); - - /* Put the segment to the wire ! */ - p_mad->trans_id = cl_hton64(key); - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Sending RMPP segment #%d, on-wire TID=0x%llX\n", - seg_num, p_mad->trans_id); - - /* - Add call for packet drop randomizer. - This is a testing feature. If run_randomizer flag is set to TRUE, - the randomizer will be called, and randomally will drop - a packet. This is used for simulating unstable fabric. - */ - if ( p_bo->p_vendor->run_randomizer == TRUE ) - { - /* Try the randomizer */ - if ( osm_pkt_randomizer_mad_drop( p_bo->p_vendor->p_log, - p_bo->p_vendor->p_pkt_randomizer, - p_mad ) == TRUE ) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "The MAD will not be sent. \n" ); - ret = IB_SUCCESS; - } - else - { - ret = osmv_transport_mad_send((osm_bind_handle_t)p_bo, p_mad, p_mad_addr); - } - } - else - { - ret = osmv_transport_mad_send((osm_bind_handle_t)p_bo, p_mad, p_mad_addr); - } - } - else - { - /* This is an ACK for double-sided handshake. Give it a special treatment. */ - - /* It doesn't really matter which data to put. Only the header matters. */ - ret = osmv_rmpp_send_ctx_get_seg(p_send_ctx, 1, timeout, p_mad); - CL_ASSERT(IB_SUCCESS == ret); - - p_mad->trans_id = cl_hton64(key); - ret = osmv_rmpp_send_ack((osm_bind_handle_t)p_bo, p_mad, 0 /* segnum */, - OSMV_RMPP_RECV_WIN /* NWL */, p_mad_addr); - } - - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - return ret; + p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); + key = osmv_txn_get_key(p_txn); + + if (0 != seg_num) { + ret = + osmv_rmpp_send_ctx_get_seg(p_send_ctx, seg_num, timeout, + p_mad); + CL_ASSERT(IB_SUCCESS == ret); + + /* Put the segment to the wire ! */ + p_mad->trans_id = cl_hton64(key); + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Sending RMPP segment #%d, on-wire TID=0x%llX\n", + seg_num, p_mad->trans_id); + + /* + Add call for packet drop randomizer. + This is a testing feature. If run_randomizer flag is set to TRUE, + the randomizer will be called, and randomally will drop + a packet. This is used for simulating unstable fabric. + */ + if (p_bo->p_vendor->run_randomizer == TRUE) { + /* Try the randomizer */ + if (osm_pkt_randomizer_mad_drop(p_bo->p_vendor->p_log, + p_bo->p_vendor-> + p_pkt_randomizer, + p_mad) == TRUE) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "The MAD will not be sent. \n"); + ret = IB_SUCCESS; + } else { + ret = + osmv_transport_mad_send((osm_bind_handle_t) + p_bo, p_mad, + p_mad_addr); + } + } else { + ret = + osmv_transport_mad_send((osm_bind_handle_t) p_bo, + p_mad, p_mad_addr); + } + } else { + /* This is an ACK for double-sided handshake. Give it a special treatment. */ + + /* It doesn't really matter which data to put. Only the header matters. */ + ret = osmv_rmpp_send_ctx_get_seg(p_send_ctx, 1, timeout, p_mad); + CL_ASSERT(IB_SUCCESS == ret); + + p_mad->trans_id = cl_hton64(key); + ret = + osmv_rmpp_send_ack((osm_bind_handle_t) p_bo, p_mad, + 0 /* segnum */ , + OSMV_RMPP_RECV_WIN /* NWL */ , + p_mad_addr); + } + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return ret; } diff --git a/opensm/libvendor/osm_vendor_mlx_sim.c b/opensm/libvendor/osm_vendor_mlx_sim.c index 6159525..6d57c63 100644 --- a/opensm/libvendor/osm_vendor_mlx_sim.c +++ b/opensm/libvendor/osm_vendor_mlx_sim.c @@ -43,7 +43,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -63,76 +63,79 @@ #include typedef struct _osmv_ibms_transport_mgr { - ibms_conn_handle_t conHdl; /* the connection handle we talk to */ - ibms_bind_msg_t filter; /* the bind message defining the filtering */ - cl_thread_t receiver; /* the thread waiting for incomming messages */ + ibms_conn_handle_t conHdl; /* the connection handle we talk to */ + ibms_bind_msg_t filter; /* the bind message defining the filtering */ + cl_thread_t receiver; /* the thread waiting for incomming messages */ } osmv_ibms_transport_mgr_t; static void -__osmv_ibms_mad_addr_to_osm_addr( - IN osm_vendor_t const * p_vend, - IN struct _ibms_mad_addr *p_ibms_addr, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_osm_addr); +__osmv_ibms_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct _ibms_mad_addr *p_ibms_addr, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_osm_addr); static void -__osmv_ibms_osm_addr_to_mad_addr( - IN const osm_mad_addr_t *p_osm_addr, - IN uint8_t is_smi, - OUT struct _ibms_mad_addr *p_ibms_addr); +__osmv_ibms_osm_addr_to_mad_addr(IN const osm_mad_addr_t * p_osm_addr, + IN uint8_t is_smi, + OUT struct _ibms_mad_addr *p_ibms_addr); /* this is the callback function the "server" will call on incoming messages */ -void -__osmv_ibms_receiver_callback(void* p_ctx, ibms_mad_msg_t *p_mad) +void __osmv_ibms_receiver_callback(void *p_ctx, ibms_mad_msg_t * p_mad) { - osm_mad_addr_t mad_addr; - osmv_bind_obj_t * const p_bo = (osmv_bind_obj_t *)p_ctx; - ib_api_status_t status = IB_SUCCESS; - - /* Make sure the p_bo object is still relevant */ - if ( (p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - { - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_ibms_receiver_callback ); - - - /* some logging */ - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_ibms_receiver_callback: " - "MAD QPN:%d SLID:0x%04x class:0x%02x " - "method:0x%02x attr:0x%04x status:0x%04x " - "tid:0x%016" PRIx64 "\n", - p_mad->addr.dqpn, - cl_ntoh16(p_mad->addr.slid), - p_mad->header.mgmt_class, - p_mad->header.method, - cl_ntoh16(p_mad->header.attr_id), - cl_ntoh16(p_mad->header.status), - cl_ntoh64(p_mad->header.trans_id)); - - /* first arrange an address */ - __osmv_ibms_mad_addr_to_osm_addr( - p_bo->p_vendor, - &p_mad->addr, - ( ((ib_mad_t*)&p_mad->header)->mgmt_class == IB_MCLASS_SUBN_LID ) || - ( ((ib_mad_t*)&p_mad->header)->mgmt_class == IB_MCLASS_SUBN_DIR ), - &mad_addr); - - /* call the receiver callback */ - - status = osmv_dispatch_mad((osm_bind_handle_t)p_bo,(void*)&p_mad->header, &mad_addr); - - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); - } + osm_mad_addr_t mad_addr; + osmv_bind_obj_t *const p_bo = (osmv_bind_obj_t *) p_ctx; + ib_api_status_t status = IB_SUCCESS; + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + { + OSM_LOG_ENTER(p_bo->p_vendor->p_log, + __osmv_ibms_receiver_callback); + + /* some logging */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_ibms_receiver_callback: " + "MAD QPN:%d SLID:0x%04x class:0x%02x " + "method:0x%02x attr:0x%04x status:0x%04x " + "tid:0x%016" PRIx64 "\n", + p_mad->addr.dqpn, + cl_ntoh16(p_mad->addr.slid), + p_mad->header.mgmt_class, + p_mad->header.method, + cl_ntoh16(p_mad->header.attr_id), + cl_ntoh16(p_mad->header.status), + cl_ntoh64(p_mad->header.trans_id)); + + /* first arrange an address */ + __osmv_ibms_mad_addr_to_osm_addr(p_bo->p_vendor, + &p_mad->addr, + (((ib_mad_t *) & p_mad-> + header)->mgmt_class == + IB_MCLASS_SUBN_LID) + || + (((ib_mad_t *) & p_mad-> + header)->mgmt_class == + IB_MCLASS_SUBN_DIR), + &mad_addr); + + /* call the receiver callback */ + + status = + osmv_dispatch_mad((osm_bind_handle_t) p_bo, + (void *)&p_mad->header, &mad_addr); + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + } } ib_api_status_t -osm_vendor_get_guid_by_ca_and_port( IN osm_vendor_t * const p_vend, - IN char *hca_id, - IN uint32_t port_num, - OUT uint64_t *p_port_guid); +osm_vendor_get_guid_by_ca_and_port(IN osm_vendor_t * const p_vend, + IN char *hca_id, + IN uint32_t port_num, + OUT uint64_t * p_port_guid); /* * NAME @@ -143,98 +146,90 @@ osm_vendor_get_guid_by_ca_and_port( IN osm_vendor_t * const p_vend, */ ib_api_status_t -osmv_transport_init(IN osm_bind_info_t *p_info, - IN char hca_id[VENDOR_HCA_MAXNAMES], - IN uint8_t hca_idx, - IN osmv_bind_obj_t *p_bo) +osmv_transport_init(IN osm_bind_info_t * p_info, + IN char hca_id[VENDOR_HCA_MAXNAMES], + IN uint8_t hca_idx, IN osmv_bind_obj_t * p_bo) { - ibms_conn_handle_t conHdl; /* the connection we talk to the simulator through */ - osmv_ibms_transport_mgr_t* p_mgr = malloc(sizeof(osmv_ibms_transport_mgr_t)); - int qpn; - int ibms_status; - uint64_t port_guid; - - if (!p_mgr) - { - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_mgr, 0, sizeof(osmv_ibms_transport_mgr_t)); - - /* create the client socket connected to the simulator */ - /* also perform the "connect" message - such that we - validate the target guid */ - if (osm_vendor_get_guid_by_ca_and_port( - p_bo->p_vendor, hca_id, p_bo->port_num, &port_guid)) - { - return IB_INVALID_GUID; - } - - conHdl = - ibms_connect(port_guid, __osmv_ibms_receiver_callback, (void*)p_bo); - if (!conHdl) - { - printf("fail to connect to the server.\n"); - exit(1); - } - - /* - * Create the MAD filter on this file handle. - */ - - p_mgr->filter.port = p_bo->port_num; - p_mgr->filter.only_input = 1; - p_mgr->filter.mask = - IBMS_BIND_MASK_PORT | - IBMS_BIND_MASK_INPUT | - IBMS_BIND_MASK_QP | - IBMS_BIND_MASK_CLASS ; - - switch ( p_info->mad_class ) - { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - qpn = 0; - p_mgr->filter.qpn = qpn; - p_mgr->filter.mgt_class = IB_MCLASS_SUBN_LID; - ibms_status = ibms_bind(conHdl, &p_mgr->filter); - if (ibms_status) - { - return IB_ERROR; - } - - p_mgr->filter.mgt_class = IB_MCLASS_SUBN_DIR; - ibms_status = ibms_bind(conHdl, &p_mgr->filter); - if (ibms_status) - { - return IB_ERROR; - } - - break; - - case IB_MCLASS_SUBN_ADM: - default: - qpn = 1; - p_mgr->filter.qpn = qpn; - p_mgr->filter.mgt_class = p_info->mad_class; - ibms_status = ibms_bind(conHdl, &p_mgr->filter); - if (ibms_status) - { - return IB_ERROR; - } - break; - } - - p_mgr->conHdl = conHdl; - - p_bo->p_transp_mgr = p_mgr ; - - /* Initialize the magic_ptr to the pointer of the p_bo info. - This will be used to signal when the object is being destroyed, so no - real action will be done then. */ - p_bo->magic_ptr = p_bo; - - return IB_SUCCESS; + ibms_conn_handle_t conHdl; /* the connection we talk to the simulator through */ + osmv_ibms_transport_mgr_t *p_mgr = + malloc(sizeof(osmv_ibms_transport_mgr_t)); + int qpn; + int ibms_status; + uint64_t port_guid; + + if (!p_mgr) { + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_mgr, 0, sizeof(osmv_ibms_transport_mgr_t)); + + /* create the client socket connected to the simulator */ + /* also perform the "connect" message - such that we + validate the target guid */ + if (osm_vendor_get_guid_by_ca_and_port + (p_bo->p_vendor, hca_id, p_bo->port_num, &port_guid)) { + return IB_INVALID_GUID; + } + + conHdl = + ibms_connect(port_guid, __osmv_ibms_receiver_callback, + (void *)p_bo); + if (!conHdl) { + printf("fail to connect to the server.\n"); + exit(1); + } + + /* + * Create the MAD filter on this file handle. + */ + + p_mgr->filter.port = p_bo->port_num; + p_mgr->filter.only_input = 1; + p_mgr->filter.mask = + IBMS_BIND_MASK_PORT | + IBMS_BIND_MASK_INPUT | IBMS_BIND_MASK_QP | IBMS_BIND_MASK_CLASS; + + switch (p_info->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + qpn = 0; + p_mgr->filter.qpn = qpn; + p_mgr->filter.mgt_class = IB_MCLASS_SUBN_LID; + ibms_status = ibms_bind(conHdl, &p_mgr->filter); + if (ibms_status) { + return IB_ERROR; + } + + p_mgr->filter.mgt_class = IB_MCLASS_SUBN_DIR; + ibms_status = ibms_bind(conHdl, &p_mgr->filter); + if (ibms_status) { + return IB_ERROR; + } + + break; + + case IB_MCLASS_SUBN_ADM: + default: + qpn = 1; + p_mgr->filter.qpn = qpn; + p_mgr->filter.mgt_class = p_info->mad_class; + ibms_status = ibms_bind(conHdl, &p_mgr->filter); + if (ibms_status) { + return IB_ERROR; + } + break; + } + + p_mgr->conHdl = conHdl; + + p_bo->p_transp_mgr = p_mgr; + + /* Initialize the magic_ptr to the pointer of the p_bo info. + This will be used to signal when the object is being destroyed, so no + real action will be done then. */ + p_bo->magic_ptr = p_bo; + + return IB_SUCCESS; } /* @@ -246,183 +241,173 @@ osmv_transport_init(IN osm_bind_info_t *p_info, */ ib_api_status_t -osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, - IN void *p_mad, - IN const osm_mad_addr_t *p_mad_addr) +osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, + IN void *p_mad, IN const osm_mad_addr_t * p_mad_addr) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - int ret; - ibms_mad_msg_t mad_msg; - ib_api_status_t status; - - const ib_mad_t *p_mad_hdr = p_mad; - - OSM_LOG_ENTER( p_vend->p_log, osmv_transport_mad_send); - - memset(&mad_msg, 0, sizeof(mad_msg)); - - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return IB_INVALID_CALLBACK; - - /* - * Copy the MAD over to the sent mad - */ - memcpy(&mad_msg.header, p_mad_hdr, MAD_BLOCK_SIZE); - - /* - * For all sends other than directed route SM MADs, - * acquire an address vector for the destination. - */ - if( p_mad_hdr->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - - __osmv_ibms_osm_addr_to_mad_addr( - p_mad_addr, - p_mad_hdr->mgmt_class == IB_MCLASS_SUBN_LID, - &mad_msg.addr); - } - else - { - /* is a directed route - we need to construct a permissive address */ - /* we do not need port number since it is part of the mad_hndl */ - mad_msg.addr.dlid = IB_LID_PERMISSIVE; - mad_msg.addr.slid = IB_LID_PERMISSIVE; - mad_msg.addr.sqpn = 0; - mad_msg.addr.dqpn = 0; - } - - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_mad_send: " - "Sending QPN:%d DLID:0x%04x class:0x%02x " - "method:0x%02x attr:0x%04x status:0x%04x " - "tid:0x%016" PRIx64 "\n", - mad_msg.addr.dqpn, - cl_ntoh16(mad_msg.addr.dlid), - mad_msg.header.mgmt_class, - mad_msg.header.method, - cl_ntoh16(mad_msg.header.attr_id), - cl_ntoh16(mad_msg.header.status), - cl_ntoh64(mad_msg.header.trans_id) - ); - - /* send it */ - ret = ibms_send(((osmv_ibms_transport_mgr_t*)(p_bo->p_transp_mgr))->conHdl, - &mad_msg); - if( ret ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osmv_transport_mad_send: ERR 5304: " - "Error sending mad (%d).\n", ret ); - status = IB_ERROR; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return(status); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + int ret; + ibms_mad_msg_t mad_msg; + ib_api_status_t status; + + const ib_mad_t *p_mad_hdr = p_mad; + + OSM_LOG_ENTER(p_vend->p_log, osmv_transport_mad_send); + + memset(&mad_msg, 0, sizeof(mad_msg)); + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return IB_INVALID_CALLBACK; + + /* + * Copy the MAD over to the sent mad + */ + memcpy(&mad_msg.header, p_mad_hdr, MAD_BLOCK_SIZE); + + /* + * For all sends other than directed route SM MADs, + * acquire an address vector for the destination. + */ + if (p_mad_hdr->mgmt_class != IB_MCLASS_SUBN_DIR) { + + __osmv_ibms_osm_addr_to_mad_addr(p_mad_addr, + p_mad_hdr->mgmt_class == + IB_MCLASS_SUBN_LID, + &mad_msg.addr); + } else { + /* is a directed route - we need to construct a permissive address */ + /* we do not need port number since it is part of the mad_hndl */ + mad_msg.addr.dlid = IB_LID_PERMISSIVE; + mad_msg.addr.slid = IB_LID_PERMISSIVE; + mad_msg.addr.sqpn = 0; + mad_msg.addr.dqpn = 0; + } + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_mad_send: " + "Sending QPN:%d DLID:0x%04x class:0x%02x " + "method:0x%02x attr:0x%04x status:0x%04x " + "tid:0x%016" PRIx64 "\n", + mad_msg.addr.dqpn, + cl_ntoh16(mad_msg.addr.dlid), + mad_msg.header.mgmt_class, + mad_msg.header.method, + cl_ntoh16(mad_msg.header.attr_id), + cl_ntoh16(mad_msg.header.status), + cl_ntoh64(mad_msg.header.trans_id) + ); + + /* send it */ + ret = + ibms_send(((osmv_ibms_transport_mgr_t *) (p_bo->p_transp_mgr))-> + conHdl, &mad_msg); + if (ret) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osmv_transport_mad_send: ERR 5304: " + "Error sending mad (%d).\n", ret); + status = IB_ERROR; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } -void -osmv_transport_done(IN const osm_bind_handle_t h_bind) +void osmv_transport_done(IN const osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osmv_ibms_transport_mgr_t* p_tpot_mgr = - (osmv_ibms_transport_mgr_t*)(p_bo->p_transp_mgr); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osmv_ibms_transport_mgr_t *p_tpot_mgr = + (osmv_ibms_transport_mgr_t *) (p_bo->p_transp_mgr); - CL_ASSERT(p_bo); + CL_ASSERT(p_bo); - /* First of all - zero out the magic_ptr, so if a callback is called - - it'll know that we are currently closing down, and will not handle the - mad. */ - p_bo->magic_ptr = 0; - /* usleep(3000000); */ + /* First of all - zero out the magic_ptr, so if a callback is called - + it'll know that we are currently closing down, and will not handle the + mad. */ + p_bo->magic_ptr = 0; + /* usleep(3000000); */ - ibms_disconnect(p_tpot_mgr->conHdl); + ibms_disconnect(p_tpot_mgr->conHdl); - /* seems the only way to abort a blocking read is to make it read something */ - free(p_tpot_mgr); + /* seems the only way to abort a blocking read is to make it read something */ + free(p_tpot_mgr); } static void -__osmv_ibms_osm_addr_to_mad_addr( - IN const osm_mad_addr_t *p_osm_addr, - IN uint8_t is_smi, - OUT struct _ibms_mad_addr *p_ibms_addr) +__osmv_ibms_osm_addr_to_mad_addr(IN const osm_mad_addr_t * p_osm_addr, + IN uint8_t is_smi, + OUT struct _ibms_mad_addr *p_ibms_addr) { - /* For global destination or Multicast address:*/ - p_ibms_addr->dlid = cl_ntoh16(p_osm_addr->dest_lid); - p_ibms_addr->sl = p_osm_addr->addr_type.gsi.service_level; - if (is_smi) - { - p_ibms_addr->sqpn = 0; - p_ibms_addr->dqpn = 0; - } - else - { - p_ibms_addr->sqpn = 1; - p_ibms_addr->dqpn = cl_ntoh32(p_osm_addr->addr_type.gsi.remote_qp); - } - /* - HACK we limit to the first PKey Index assuming it will - always be the default PKey - */ - p_ibms_addr->pkey_index = 0; + /* For global destination or Multicast address: */ + p_ibms_addr->dlid = cl_ntoh16(p_osm_addr->dest_lid); + p_ibms_addr->sl = p_osm_addr->addr_type.gsi.service_level; + if (is_smi) { + p_ibms_addr->sqpn = 0; + p_ibms_addr->dqpn = 0; + } else { + p_ibms_addr->sqpn = 1; + p_ibms_addr->dqpn = + cl_ntoh32(p_osm_addr->addr_type.gsi.remote_qp); + } + /* + HACK we limit to the first PKey Index assuming it will + always be the default PKey + */ + p_ibms_addr->pkey_index = 0; } static void -__osmv_ibms_mad_addr_to_osm_addr( - IN osm_vendor_t const * p_vend, - IN struct _ibms_mad_addr *p_ibms_addr, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_osm_addr) +__osmv_ibms_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct _ibms_mad_addr *p_ibms_addr, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_osm_addr) { - memset(p_osm_addr, 0, sizeof(osm_mad_addr_t)); - p_osm_addr->dest_lid = cl_hton16(p_ibms_addr->slid); - p_osm_addr->static_rate = 0; - p_osm_addr->path_bits = 0; - if (is_smi) - { - /* SMI */ - p_osm_addr->addr_type.smi.source_lid = cl_hton16(p_ibms_addr->slid); - p_osm_addr->addr_type.smi.port_num = 1; /* TODO add if required p_ibms_addr->port; */ - } - else - { - /* GSI */ - p_osm_addr->addr_type.gsi.remote_qp = cl_ntoh32(p_ibms_addr->sqpn); - p_osm_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - /* we do have the p_osm_addr->pkey_ix but how to get the PKey by index ? */ - /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ - /* the full PKey table - than go by the index. */ - /* since this does not seem reasonable to me I simply use the default */ - /* There is a TAVOR limitation that only one P_KEY is supported per */ - /* QP - so QP1 must use IB_DEFAULT_PKEY */ - p_osm_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_osm_addr->addr_type.gsi.service_level = p_ibms_addr->sl; - - p_osm_addr->addr_type.gsi.global_route = FALSE; - /* copy the GRH data if relevant - TopSpin imp doesnt relate to GRH!!!*/ - /* - if (p_osm_addr->addr_type.gsi.global_route) - { - p_osm_addr->addr_type.gsi.grh_info.ver_class_flow = - ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, - p_rcv_desc->grh.traffic_class, - p_rcv_desc->grh.flow_label); - p_osm_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; - memcpy(&p_osm_addr->addr_type.gsi.grh_info.src_gid.raw, - &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); - memcpy(&p_osm_addr->addr_type.gsi.grh_info.dest_gid.raw, - p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); - } - */ - } + memset(p_osm_addr, 0, sizeof(osm_mad_addr_t)); + p_osm_addr->dest_lid = cl_hton16(p_ibms_addr->slid); + p_osm_addr->static_rate = 0; + p_osm_addr->path_bits = 0; + if (is_smi) { + /* SMI */ + p_osm_addr->addr_type.smi.source_lid = + cl_hton16(p_ibms_addr->slid); + p_osm_addr->addr_type.smi.port_num = 1; /* TODO add if required p_ibms_addr->port; */ + } else { + /* GSI */ + p_osm_addr->addr_type.gsi.remote_qp = + cl_ntoh32(p_ibms_addr->sqpn); + p_osm_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + /* we do have the p_osm_addr->pkey_ix but how to get the PKey by index ? */ + /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ + /* the full PKey table - than go by the index. */ + /* since this does not seem reasonable to me I simply use the default */ + /* There is a TAVOR limitation that only one P_KEY is supported per */ + /* QP - so QP1 must use IB_DEFAULT_PKEY */ + p_osm_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_osm_addr->addr_type.gsi.service_level = p_ibms_addr->sl; + + p_osm_addr->addr_type.gsi.global_route = FALSE; + /* copy the GRH data if relevant - TopSpin imp doesnt relate to GRH!!! */ + /* + if (p_osm_addr->addr_type.gsi.global_route) + { + p_osm_addr->addr_type.gsi.grh_info.ver_class_flow = + ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, + p_rcv_desc->grh.traffic_class, + p_rcv_desc->grh.flow_label); + p_osm_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; + memcpy(&p_osm_addr->addr_type.gsi.grh_info.src_gid.raw, + &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); + memcpy(&p_osm_addr->addr_type.gsi.grh_info.dest_gid.raw, + p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); + } + */ + } } /* @@ -432,36 +417,30 @@ __osmv_ibms_mad_addr_to_osm_addr( * according to the value given (TRUE or FALSE). */ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osmv_bind_obj_t *p_bo = ( osmv_bind_obj_t * ) h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - int ret; - ibms_cap_msg_t cap_msg; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); - - cap_msg.mask = IB_PORT_CAP_IS_SM; - if (is_sm_val) - cap_msg.capabilities = IB_PORT_CAP_IS_SM; - else - cap_msg.capabilities = 0; - - ret = ibms_set_cap( - ((osmv_ibms_transport_mgr_t*)(p_bo->p_transp_mgr))->conHdl, - &cap_msg); - - if( ret ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 5312: " - "Unable set 'IS_SM' bit to:%u in port attributes.\n", - is_sm_val ); - } - OSM_LOG_EXIT( p_vend->p_log ); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + int ret; + ibms_cap_msg_t cap_msg; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); + + cap_msg.mask = IB_PORT_CAP_IS_SM; + if (is_sm_val) + cap_msg.capabilities = IB_PORT_CAP_IS_SM; + else + cap_msg.capabilities = 0; + + ret = ibms_set_cap(((osmv_ibms_transport_mgr_t *) (p_bo-> + p_transp_mgr))-> + conHdl, &cap_msg); + + if (ret) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 5312: " + "Unable set 'IS_SM' bit to:%u in port attributes.\n", + is_sm_val); + } + OSM_LOG_EXIT(p_vend->p_log); } - - diff --git a/opensm/libvendor/osm_vendor_mlx_ts.c b/opensm/libvendor/osm_vendor_mlx_ts.c index a980d3a..83ed1b1 100644 --- a/opensm/libvendor/osm_vendor_mlx_ts.c +++ b/opensm/libvendor/osm_vendor_mlx_ts.c @@ -43,7 +43,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -61,112 +61,111 @@ #include typedef struct _osmv_TOPSPIN_transport_mgr_ { - int device_fd; - osm_ts_user_mad_filter filter; - cl_thread_t receiver; -}osmv_TOPSPIN_transport_mgr_t; + int device_fd; + osm_ts_user_mad_filter filter; + cl_thread_t receiver; +} osmv_TOPSPIN_transport_mgr_t; static void -__osmv_TOPSPIN_mad_addr_to_osm_addr( - IN osm_vendor_t const * p_vend, - IN struct ib_mad *p_mad, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr); +__osmv_TOPSPIN_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct ib_mad *p_mad, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr); static void -__osmv_TOPSPIN_osm_addr_to_mad_addr( - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT struct ib_mad *p_mad); +__osmv_TOPSPIN_osm_addr_to_mad_addr(IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, + OUT struct ib_mad *p_mad); - - -void -__osmv_TOPSPIN_receiver_thr(void* p_ctx) +void __osmv_TOPSPIN_receiver_thr(void *p_ctx) { - int ts_ret_code; - struct ib_mad mad; - osm_mad_addr_t mad_addr; - osmv_bind_obj_t * const p_bo = (osmv_bind_obj_t *)p_ctx; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_TOPSPIN_receiver_thr ); - - /* Make sure the p_bo object is still relevant */ - if ( (p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - /* we set the type of cancelation for this thread */ - /* pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); */ - - while( 1 ) - { - /* Make sure the p_bo object is still relevant */ - if ( (p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - /* we read one mad at a time and pass it to the read callback function */ - ts_ret_code = read(((osmv_TOPSPIN_transport_mgr_t*)(p_bo->p_transp_mgr))->device_fd, - &mad, sizeof(mad)); - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - if (ts_ret_code != sizeof(mad)) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_TOPSPIN_receiver_thr: ERR 6803: " - "error with read, bytes = %d, errno = %d\n", - ts_ret_code, errno - ); - break; - } - else - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_TOPSPIN_receiver_thr: " - "MAD QPN:%d SLID:0x%04x class:0x%02x " - "method:0x%02x attr:0x%04x status:0x%04x " - "tid:0x%016" PRIx64 "\n", - mad.dqpn, - cl_ntoh16(mad.slid), - mad.mgmt_class, - mad.r_method, - cl_ntoh16(mad.attribute_id), - cl_ntoh16(mad.status), - cl_ntoh64(mad.transaction_id)); - - /* first arrange an address */ - __osmv_TOPSPIN_mad_addr_to_osm_addr( - p_bo->p_vendor,&mad, - ( ((ib_mad_t*)&mad)->mgmt_class == IB_MCLASS_SUBN_LID ) || - ( ((ib_mad_t*)&mad)->mgmt_class == IB_MCLASS_SUBN_DIR ), - &mad_addr); - - /* call the receiver callback */ - - status = osmv_dispatch_mad((osm_bind_handle_t)p_bo,(void*)&mad,&mad_addr); - - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - if (IB_INTERRUPTED == status) - { - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_TOPSPIN_receiver_thr: " - "The bind handle %p is being closed. " - "Breaking the loop.\n", p_bo); - break; - } - } - } - - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); + int ts_ret_code; + struct ib_mad mad; + osm_mad_addr_t mad_addr; + osmv_bind_obj_t *const p_bo = (osmv_bind_obj_t *) p_ctx; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_TOPSPIN_receiver_thr); + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + /* we set the type of cancelation for this thread */ + /* pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); */ + + while (1) { + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + /* we read one mad at a time and pass it to the read callback function */ + ts_ret_code = + read(((osmv_TOPSPIN_transport_mgr_t *) (p_bo-> + p_transp_mgr))-> + device_fd, &mad, sizeof(mad)); + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + if (ts_ret_code != sizeof(mad)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_TOPSPIN_receiver_thr: ERR 6803: " + "error with read, bytes = %d, errno = %d\n", + ts_ret_code, errno); + break; + } else { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_TOPSPIN_receiver_thr: " + "MAD QPN:%d SLID:0x%04x class:0x%02x " + "method:0x%02x attr:0x%04x status:0x%04x " + "tid:0x%016" PRIx64 "\n", + mad.dqpn, + cl_ntoh16(mad.slid), + mad.mgmt_class, + mad.r_method, + cl_ntoh16(mad.attribute_id), + cl_ntoh16(mad.status), + cl_ntoh64(mad.transaction_id)); + + /* first arrange an address */ + __osmv_TOPSPIN_mad_addr_to_osm_addr(p_bo->p_vendor, + &mad, + (((ib_mad_t *) & + mad)-> + mgmt_class == + IB_MCLASS_SUBN_LID) + || + (((ib_mad_t *) & + mad)-> + mgmt_class == + IB_MCLASS_SUBN_DIR), + &mad_addr); + + /* call the receiver callback */ + + status = + osmv_dispatch_mad((osm_bind_handle_t) p_bo, + (void *)&mad, &mad_addr); + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + if (IB_INTERRUPTED == status) { + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_TOPSPIN_receiver_thr: " + "The bind handle %p is being closed. " + "Breaking the loop.\n", p_bo); + break; + } + } + } + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } - /* * NAME * osmv_transport_init @@ -176,100 +175,96 @@ __osmv_TOPSPIN_receiver_thr(void* p_ctx) */ ib_api_status_t -osmv_transport_init(IN osm_bind_info_t *p_info, - IN char hca_id[VENDOR_HCA_MAXNAMES], - IN uint8_t hca_idx, - IN osmv_bind_obj_t *p_bo) +osmv_transport_init(IN osm_bind_info_t * p_info, + IN char hca_id[VENDOR_HCA_MAXNAMES], + IN uint8_t hca_idx, IN osmv_bind_obj_t * p_bo) { - cl_status_t cl_st; - char device_file[16]; - int device_fd; - int ts_ioctl_ret; - osmv_TOPSPIN_transport_mgr_t* p_mgr = malloc(sizeof(osmv_TOPSPIN_transport_mgr_t)); - int qpn; - - if (!p_mgr) - { - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_mgr, 0, sizeof(osmv_TOPSPIN_transport_mgr_t)); - - /* open TopSpin file device */ - /* HACK: assume last char in hostid is the HCA index */ - sprintf(device_file, "/dev/ts_ua%u", hca_idx); - device_fd = open(device_file, O_RDWR ); - if (device_fd < 0) - { - fprintf(stderr, "Fatal: Fail to open the file:%s err:%d\n", device_file, errno); - return IB_ERROR; - } - - /* - * Create the MAD filter on this file handle. - */ - - p_mgr->filter.port = p_bo->port_num; - p_mgr->filter.direction = TS_IB_MAD_DIRECTION_IN; - p_mgr->filter.mask = - TS_IB_MAD_FILTER_DIRECTION | - TS_IB_MAD_FILTER_PORT | - TS_IB_MAD_FILTER_QPN | - TS_IB_MAD_FILTER_MGMT_CLASS; - - switch ( p_info->mad_class ) - { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - qpn = 0; - p_mgr->filter.qpn = qpn; - p_mgr->filter.mgmt_class = IB_MCLASS_SUBN_LID; - ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); - if (ts_ioctl_ret < 0) - { - return IB_ERROR; - } - - p_mgr->filter.mgmt_class = IB_MCLASS_SUBN_DIR; - ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); - if (ts_ioctl_ret < 0) - { - return IB_ERROR; - } - - break; - - case IB_MCLASS_SUBN_ADM: - default: - qpn = 1; - p_mgr->filter.qpn = qpn; - p_mgr->filter.mgmt_class = p_info->mad_class; - ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); - if (ts_ioctl_ret < 0) - { - return IB_ERROR; - } - break; - } - - p_mgr->device_fd = device_fd; - - p_bo->p_transp_mgr = p_mgr ; - - /* Initialize the magic_ptr to the pointer of the p_bo info. - This will be used to signal when the object is being destroyed, so no - real action will be done then. */ - p_bo->magic_ptr = p_bo; - - /* init receiver thread */ - cl_st = cl_thread_init(&p_mgr->receiver,__osmv_TOPSPIN_receiver_thr,(void*)p_bo, - "osmv TOPSPIN rcv thr"); - - return (ib_api_status_t)cl_st; + cl_status_t cl_st; + char device_file[16]; + int device_fd; + int ts_ioctl_ret; + osmv_TOPSPIN_transport_mgr_t *p_mgr = + malloc(sizeof(osmv_TOPSPIN_transport_mgr_t)); + int qpn; + + if (!p_mgr) { + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_mgr, 0, sizeof(osmv_TOPSPIN_transport_mgr_t)); + + /* open TopSpin file device */ + /* HACK: assume last char in hostid is the HCA index */ + sprintf(device_file, "/dev/ts_ua%u", hca_idx); + device_fd = open(device_file, O_RDWR); + if (device_fd < 0) { + fprintf(stderr, "Fatal: Fail to open the file:%s err:%d\n", + device_file, errno); + return IB_ERROR; + } + + /* + * Create the MAD filter on this file handle. + */ + + p_mgr->filter.port = p_bo->port_num; + p_mgr->filter.direction = TS_IB_MAD_DIRECTION_IN; + p_mgr->filter.mask = + TS_IB_MAD_FILTER_DIRECTION | + TS_IB_MAD_FILTER_PORT | + TS_IB_MAD_FILTER_QPN | TS_IB_MAD_FILTER_MGMT_CLASS; + + switch (p_info->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + qpn = 0; + p_mgr->filter.qpn = qpn; + p_mgr->filter.mgmt_class = IB_MCLASS_SUBN_LID; + ts_ioctl_ret = + ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); + if (ts_ioctl_ret < 0) { + return IB_ERROR; + } + + p_mgr->filter.mgmt_class = IB_MCLASS_SUBN_DIR; + ts_ioctl_ret = + ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); + if (ts_ioctl_ret < 0) { + return IB_ERROR; + } + + break; + + case IB_MCLASS_SUBN_ADM: + default: + qpn = 1; + p_mgr->filter.qpn = qpn; + p_mgr->filter.mgmt_class = p_info->mad_class; + ts_ioctl_ret = + ioctl(device_fd, TS_IB_IOCSMADFILTADD, &p_mgr->filter); + if (ts_ioctl_ret < 0) { + return IB_ERROR; + } + break; + } + + p_mgr->device_fd = device_fd; + + p_bo->p_transp_mgr = p_mgr; + + /* Initialize the magic_ptr to the pointer of the p_bo info. + This will be used to signal when the object is being destroyed, so no + real action will be done then. */ + p_bo->magic_ptr = p_bo; + + /* init receiver thread */ + cl_st = + cl_thread_init(&p_mgr->receiver, __osmv_TOPSPIN_receiver_thr, + (void *)p_bo, "osmv TOPSPIN rcv thr"); + + return (ib_api_status_t) cl_st; } - - /* * NAME * osmv_transport_send_mad @@ -279,88 +274,83 @@ osmv_transport_init(IN osm_bind_info_t *p_info, */ ib_api_status_t -osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, - IN void *p_mad, - IN const osm_mad_addr_t *p_mad_addr) +osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, + IN void *p_mad, IN const osm_mad_addr_t * p_mad_addr) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - struct ib_mad ts_mad; - int ret; - ib_api_status_t status; - - const ib_mad_t *p_mad_hdr = p_mad; - - OSM_LOG_ENTER( p_vend->p_log, osmv_transport_mad_send); - - memset(&ts_mad, 0, sizeof(ts_mad)); - - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return IB_INVALID_CALLBACK; - - /* - * Copy the MAD over to the sent mad - */ - memcpy(&ts_mad, p_mad_hdr, MAD_BLOCK_SIZE); - - /* - * For all sends other than directed route SM MADs, - * acquire an address vector for the destination. - */ - if( p_mad_hdr->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - - __osmv_TOPSPIN_osm_addr_to_mad_addr( - p_mad_addr, - p_mad_hdr->mgmt_class == IB_MCLASS_SUBN_LID, - &ts_mad); - } - else - { - /* is a directed route - we need to construct a permissive address */ - /* we do not need port number since it is part of the mad_hndl */ - ts_mad.dlid = IB_LID_PERMISSIVE; - ts_mad.slid = IB_LID_PERMISSIVE; - ts_mad.sqpn = 0; - ts_mad.dqpn = 0; - } - - ts_mad.port = p_bo->port_num; - - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "osmv_transport_mad_send: " - "Sending QPN:%d DLID:0x%04x class:0x%02x " - "method:0x%02x attr:0x%04x status:0x%04x " - "tid:0x%016" PRIx64 "\n", - ts_mad.dqpn, - cl_ntoh16(ts_mad.dlid), - ts_mad.mgmt_class, - ts_mad.r_method, - cl_ntoh16(ts_mad.attribute_id), - cl_ntoh16(ts_mad.status), - cl_ntoh64(ts_mad.transaction_id) - ); - - /* send it */ - ret = write(((osmv_TOPSPIN_transport_mgr_t*)(p_bo->p_transp_mgr))->device_fd, - &ts_mad, sizeof(ts_mad)); - - if( ret != sizeof(ts_mad) ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osmv_transport_mad_send: ERR 6804: " - "Error sending mad (%d).\n", ret ); - status = IB_ERROR; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return(status); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + struct ib_mad ts_mad; + int ret; + ib_api_status_t status; + + const ib_mad_t *p_mad_hdr = p_mad; + + OSM_LOG_ENTER(p_vend->p_log, osmv_transport_mad_send); + + memset(&ts_mad, 0, sizeof(ts_mad)); + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return IB_INVALID_CALLBACK; + + /* + * Copy the MAD over to the sent mad + */ + memcpy(&ts_mad, p_mad_hdr, MAD_BLOCK_SIZE); + + /* + * For all sends other than directed route SM MADs, + * acquire an address vector for the destination. + */ + if (p_mad_hdr->mgmt_class != IB_MCLASS_SUBN_DIR) { + + __osmv_TOPSPIN_osm_addr_to_mad_addr(p_mad_addr, + p_mad_hdr->mgmt_class == + IB_MCLASS_SUBN_LID, + &ts_mad); + } else { + /* is a directed route - we need to construct a permissive address */ + /* we do not need port number since it is part of the mad_hndl */ + ts_mad.dlid = IB_LID_PERMISSIVE; + ts_mad.slid = IB_LID_PERMISSIVE; + ts_mad.sqpn = 0; + ts_mad.dqpn = 0; + } + + ts_mad.port = p_bo->port_num; + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "osmv_transport_mad_send: " + "Sending QPN:%d DLID:0x%04x class:0x%02x " + "method:0x%02x attr:0x%04x status:0x%04x " + "tid:0x%016" PRIx64 "\n", + ts_mad.dqpn, + cl_ntoh16(ts_mad.dlid), + ts_mad.mgmt_class, + ts_mad.r_method, + cl_ntoh16(ts_mad.attribute_id), + cl_ntoh16(ts_mad.status), cl_ntoh64(ts_mad.transaction_id) + ); + + /* send it */ + ret = + write(((osmv_TOPSPIN_transport_mgr_t *) (p_bo->p_transp_mgr))-> + device_fd, &ts_mad, sizeof(ts_mad)); + + if (ret != sizeof(ts_mad)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osmv_transport_mad_send: ERR 6804: " + "Error sending mad (%d).\n", ret); + status = IB_ERROR; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /* @@ -368,129 +358,118 @@ osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, and send a mad through - the main idea is to make the filter catch it such that the read unblocks */ -void -__osm_transport_gen_dummy_mad(osmv_bind_obj_t* p_bo) +void __osm_transport_gen_dummy_mad(osmv_bind_obj_t * p_bo) { - struct ib_mad ts_mad; - osmv_TOPSPIN_transport_mgr_t* p_mgr = - (osmv_TOPSPIN_transport_mgr_t*)(p_bo->p_transp_mgr); - struct ib_get_port_info_ioctl port_data; - int ts_ioctl_ret; - - /* prepare the mad fields following the stored filter on the bind */ - memset(&ts_mad, 0, sizeof(ts_mad)); - ts_mad.format_version = 1; - ts_mad.mgmt_class = p_mgr->filter.mgmt_class; - ts_mad.attribute_id = 0x2; - ts_mad.class_version = 1; - ts_mad.r_method = cl_ntoh16(0x2); - ts_mad.port = p_bo->port_num; - ts_mad.sqpn = p_mgr->filter.qpn; - ts_mad.dqpn = p_mgr->filter.qpn; - ts_mad.slid = 0xffff; - /* we must send to our local lid ... */ - port_data.port = p_bo->port_num; - ts_ioctl_ret = ioctl(p_mgr->device_fd, TS_IB_IOCGPORTINFO, &port_data); - ts_mad.dlid = port_data.port_info.lid; - ts_mad.transaction_id = 0x9999; - write(p_mgr->device_fd, &ts_mad, sizeof(ts_mad)); + struct ib_mad ts_mad; + osmv_TOPSPIN_transport_mgr_t *p_mgr = + (osmv_TOPSPIN_transport_mgr_t *) (p_bo->p_transp_mgr); + struct ib_get_port_info_ioctl port_data; + int ts_ioctl_ret; + + /* prepare the mad fields following the stored filter on the bind */ + memset(&ts_mad, 0, sizeof(ts_mad)); + ts_mad.format_version = 1; + ts_mad.mgmt_class = p_mgr->filter.mgmt_class; + ts_mad.attribute_id = 0x2; + ts_mad.class_version = 1; + ts_mad.r_method = cl_ntoh16(0x2); + ts_mad.port = p_bo->port_num; + ts_mad.sqpn = p_mgr->filter.qpn; + ts_mad.dqpn = p_mgr->filter.qpn; + ts_mad.slid = 0xffff; + /* we must send to our local lid ... */ + port_data.port = p_bo->port_num; + ts_ioctl_ret = ioctl(p_mgr->device_fd, TS_IB_IOCGPORTINFO, &port_data); + ts_mad.dlid = port_data.port_info.lid; + ts_mad.transaction_id = 0x9999; + write(p_mgr->device_fd, &ts_mad, sizeof(ts_mad)); } -void -osmv_transport_done(IN const osm_bind_handle_t h_bind) +void osmv_transport_done(IN const osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - osmv_TOPSPIN_transport_mgr_t* p_tpot_mgr = - (osmv_TOPSPIN_transport_mgr_t*)(p_bo->p_transp_mgr); - - CL_ASSERT(p_bo); - - /* First of all - zero out the magic_ptr, so if a callback is called - - it'll know that we are currently closing down, and will not handle the - mad. */ - p_bo->magic_ptr = 0; - /* usleep(3000000); */ - - /* seems the only way to abort a blocking read is to make it read something */ - __osm_transport_gen_dummy_mad(p_bo); - cl_thread_destroy(&(p_tpot_mgr->receiver)); - free(p_tpot_mgr); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osmv_TOPSPIN_transport_mgr_t *p_tpot_mgr = + (osmv_TOPSPIN_transport_mgr_t *) (p_bo->p_transp_mgr); + + CL_ASSERT(p_bo); + + /* First of all - zero out the magic_ptr, so if a callback is called - + it'll know that we are currently closing down, and will not handle the + mad. */ + p_bo->magic_ptr = 0; + /* usleep(3000000); */ + + /* seems the only way to abort a blocking read is to make it read something */ + __osm_transport_gen_dummy_mad(p_bo); + cl_thread_destroy(&(p_tpot_mgr->receiver)); + free(p_tpot_mgr); } static void -__osmv_TOPSPIN_osm_addr_to_mad_addr( - IN const osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT struct ib_mad *p_mad) +__osmv_TOPSPIN_osm_addr_to_mad_addr(IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, OUT struct ib_mad *p_mad) { - /* For global destination or Multicast address:*/ - p_mad->dlid = cl_ntoh16(p_mad_addr->dest_lid); - p_mad->sl = p_mad_addr->addr_type.gsi.service_level; - if (is_smi) - { - p_mad->sqpn = 0; - p_mad->dqpn = 0; - } - else - { - p_mad->sqpn = 1; - p_mad->dqpn = cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp); - } - /* - HACK we limit to the first PKey Index assuming it will - always be the default PKey - */ - p_mad->pkey_index = 0; + /* For global destination or Multicast address: */ + p_mad->dlid = cl_ntoh16(p_mad_addr->dest_lid); + p_mad->sl = p_mad_addr->addr_type.gsi.service_level; + if (is_smi) { + p_mad->sqpn = 0; + p_mad->dqpn = 0; + } else { + p_mad->sqpn = 1; + p_mad->dqpn = cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp); + } + /* + HACK we limit to the first PKey Index assuming it will + always be the default PKey + */ + p_mad->pkey_index = 0; } static void -__osmv_TOPSPIN_mad_addr_to_osm_addr( - IN osm_vendor_t const * p_vend, - IN struct ib_mad *p_mad, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr) +__osmv_TOPSPIN_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct ib_mad *p_mad, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr) { - p_mad_addr->dest_lid = cl_hton16(p_mad->slid); - p_mad_addr->static_rate = 0; - p_mad_addr->path_bits = 0; - if (is_smi) - { - /* SMI */ - p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_mad->slid); - p_mad_addr->addr_type.smi.port_num = p_mad->port; - } - else - { - /* GSI */ - p_mad_addr->addr_type.gsi.remote_qp = cl_ntoh32(p_mad->sqpn); - p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ - /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ - /* the full PKey table - than go by the index. */ - /* since this does not seem reasonable to me I simply use the default */ - /* There is a TAVOR limitation that only one P_KEY is supported per */ - /* QP - so QP1 must use IB_DEFAULT_PKEY */ - p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_mad_addr->addr_type.gsi.service_level = p_mad->sl; - - p_mad_addr->addr_type.gsi.global_route = FALSE; - /* copy the GRH data if relevant - TopSpin imp doesnt relate to GRH!!!*/ - /* - if (p_mad_addr->addr_type.gsi.global_route) - { - p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = - ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, - p_rcv_desc->grh.traffic_class, - p_rcv_desc->grh.flow_label); - p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; - memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, - &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); - memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); - } - */ - } + p_mad_addr->dest_lid = cl_hton16(p_mad->slid); + p_mad_addr->static_rate = 0; + p_mad_addr->path_bits = 0; + if (is_smi) { + /* SMI */ + p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_mad->slid); + p_mad_addr->addr_type.smi.port_num = p_mad->port; + } else { + /* GSI */ + p_mad_addr->addr_type.gsi.remote_qp = cl_ntoh32(p_mad->sqpn); + p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ + /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ + /* the full PKey table - than go by the index. */ + /* since this does not seem reasonable to me I simply use the default */ + /* There is a TAVOR limitation that only one P_KEY is supported per */ + /* QP - so QP1 must use IB_DEFAULT_PKEY */ + p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_mad_addr->addr_type.gsi.service_level = p_mad->sl; + + p_mad_addr->addr_type.gsi.global_route = FALSE; + /* copy the GRH data if relevant - TopSpin imp doesnt relate to GRH!!! */ + /* + if (p_mad_addr->addr_type.gsi.global_route) + { + p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = + ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, + p_rcv_desc->grh.traffic_class, + p_rcv_desc->grh.flow_label); + p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; + memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, + &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); + memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); + } + */ + } } /* @@ -501,35 +480,31 @@ __osmv_TOPSPIN_mad_addr_to_osm_addr( */ #if (defined(OSM_VENDOR_INTF_TS_NO_VAPI) || defined(OSM_VENDOR_INTF_TS)) -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osmv_bind_obj_t *p_bo = ( osmv_bind_obj_t * ) h_bind; - osm_vendor_t const *p_vend = p_bo->p_vendor; - int ts_ioctl_ret; - int device_fd = ((osmv_TOPSPIN_transport_mgr_t*)(p_bo->p_transp_mgr))->device_fd; - struct ib_set_port_info_ioctl set_port_data; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); - - memset(&set_port_data, 0, sizeof(set_port_data)); - - set_port_data.port = p_bo->port_num; - set_port_data.port_info.valid_fields = IB_PORT_IS_SM; - set_port_data.port_info.is_sm = is_sm_val; - ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSPORTINFO, &set_port_data); - if ( ts_ioctl_ret < 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 6805: " - "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", - is_sm_val, ts_ioctl_ret ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + osm_vendor_t const *p_vend = p_bo->p_vendor; + int ts_ioctl_ret; + int device_fd = + ((osmv_TOPSPIN_transport_mgr_t *) (p_bo->p_transp_mgr))->device_fd; + struct ib_set_port_info_ioctl set_port_data; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); + + memset(&set_port_data, 0, sizeof(set_port_data)); + + set_port_data.port = p_bo->port_num; + set_port_data.port_info.valid_fields = IB_PORT_IS_SM; + set_port_data.port_info.is_sm = is_sm_val; + ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSPORTINFO, &set_port_data); + if (ts_ioctl_ret < 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 6805: " + "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", + is_sm_val, ts_ioctl_ret); + } + + OSM_LOG_EXIT(p_vend->p_log); } #endif - diff --git a/opensm/libvendor/osm_vendor_mlx_ts_anafa.c b/opensm/libvendor/osm_vendor_mlx_ts_anafa.c index f812405..f3569a7 100644 --- a/opensm/libvendor/osm_vendor_mlx_ts_anafa.c +++ b/opensm/libvendor/osm_vendor_mlx_ts_anafa.c @@ -43,7 +43,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -63,18 +63,17 @@ #include static void - __osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr (IN osm_vendor_t const *p_vend, - IN struct ib_mad *p_mad, - IN uint8_t is_smi, - OUT osm_mad_addr_t * p_mad_addr); +__osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct ib_mad *p_mad, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr); static void - __osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr (IN const osm_mad_addr_t * - p_mad_addr, IN uint8_t is_smi, - OUT struct ib_mad *p_mad); +__osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr(IN const osm_mad_addr_t * + p_mad_addr, IN uint8_t is_smi, + OUT struct ib_mad *p_mad); -void -__osmv_TOPSPIN_ANAFA_receiver_thr (void *p_ctx) +void __osmv_TOPSPIN_ANAFA_receiver_thr(void *p_ctx) { int ts_ret_code; struct ib_mad mad; @@ -82,63 +81,62 @@ __osmv_TOPSPIN_ANAFA_receiver_thr (void *p_ctx) osmv_bind_obj_t *const p_bo = (osmv_bind_obj_t *) p_ctx; ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER (p_bo->p_vendor->p_log, - __osmv_TOPSPIN_ANAFA_receiver_thr); + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_TOPSPIN_ANAFA_receiver_thr); /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) return; /* we set the type of cancelation for this thread */ /* pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); */ while (1) { - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; /* we read one mad at a time and pass it to the read callback function */ ts_ret_code = - read (((osmv_TOPSPIN_ANAFA_transport_mgr_t *) (p_bo-> - p_transp_mgr))-> - device_fd, &mad, sizeof (mad)); - - /* Make sure the p_bo object is still relevant */ - if (( p_bo->magic_ptr != p_bo) || p_bo->is_closing ) - return; - - if (ts_ret_code != sizeof (mad)) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_TOPSPIN_ANAFA_receiver_thr: ERR 6903: " - "error with read, bytes = %d\n", ts_ret_code); + read(((osmv_TOPSPIN_ANAFA_transport_mgr_t *) (p_bo-> + p_transp_mgr))-> + device_fd, &mad, sizeof(mad)); + + /* Make sure the p_bo object is still relevant */ + if ((p_bo->magic_ptr != p_bo) || p_bo->is_closing) + return; + + if (ts_ret_code != sizeof(mad)) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_TOPSPIN_ANAFA_receiver_thr: ERR 6903: " + "error with read, bytes = %d\n", ts_ret_code); break; } else { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_TOPSPIN_ANAFA_receiver_thr: " - "MAD QPN:%d SLID:0x%04x class:0x%02x " - "method:0x%02x attr:0x%04x status:0x%04x " - "tid:0x%016" PRIx64 "\n", - mad.dqpn, - cl_ntoh16 (mad.slid), - mad.mgmt_class, - mad.r_method, - cl_ntoh16 (mad.attribute_id), - cl_ntoh16 (mad.status), - cl_ntoh64 (mad.transaction_id)); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_TOPSPIN_ANAFA_receiver_thr: " + "MAD QPN:%d SLID:0x%04x class:0x%02x " + "method:0x%02x attr:0x%04x status:0x%04x " + "tid:0x%016" PRIx64 "\n", + mad.dqpn, + cl_ntoh16(mad.slid), + mad.mgmt_class, + mad.r_method, + cl_ntoh16(mad.attribute_id), + cl_ntoh16(mad.status), + cl_ntoh64(mad.transaction_id)); /* first arrange an address */ __osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr (p_bo->p_vendor, &mad, - (((ib_mad_t *) & mad)->mgmt_class == - IB_MCLASS_SUBN_LID) - || (((ib_mad_t *) & mad)->mgmt_class == - IB_MCLASS_SUBN_DIR), &mad_addr); + (((ib_mad_t *) & mad)->mgmt_class == + IB_MCLASS_SUBN_LID) + || (((ib_mad_t *) & mad)->mgmt_class == + IB_MCLASS_SUBN_DIR), &mad_addr); /* call the receiver callback */ status = - osmv_dispatch_mad ((osm_bind_handle_t) p_bo, - (void *)&mad, &mad_addr); + osmv_dispatch_mad((osm_bind_handle_t) p_bo, + (void *)&mad, &mad_addr); /* Make sure the p_bo object is still relevant */ if (p_bo->magic_ptr != p_bo) @@ -146,16 +144,16 @@ __osmv_TOPSPIN_ANAFA_receiver_thr (void *p_ctx) if (IB_INTERRUPTED == status) { - osm_log (p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_TOPSPIN_ANAFA_receiver_thr: " - "The bind handle %p is being closed. " - "Breaking the loop.\n", p_bo); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_TOPSPIN_ANAFA_receiver_thr: " + "The bind handle %p is being closed. " + "Breaking the loop.\n", p_bo); break; } } } - OSM_LOG_EXIT (p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /* @@ -167,11 +165,9 @@ __osmv_TOPSPIN_ANAFA_receiver_thr (void *p_ctx) */ ib_api_status_t -osmv_transport_init ( - IN osm_bind_info_t * p_info, - IN char hca_id[VENDOR_HCA_MAXNAMES], - IN uint8_t hca_idx, - IN osmv_bind_obj_t * p_bo) +osmv_transport_init(IN osm_bind_info_t * p_info, + IN char hca_id[VENDOR_HCA_MAXNAMES], + IN uint8_t hca_idx, IN osmv_bind_obj_t * p_bo) { cl_status_t cl_st; @@ -185,18 +181,18 @@ osmv_transport_init ( (osmv_TOPSPIN_ANAFA_transport_info_t *) p_bo->p_vendor-> p_transport_info; - p_mgr = malloc (sizeof (osmv_TOPSPIN_ANAFA_transport_mgr_t)); + p_mgr = malloc(sizeof(osmv_TOPSPIN_ANAFA_transport_mgr_t)); if (!p_mgr) { return IB_INSUFFICIENT_MEMORY; } - memset(p_mgr, 0, sizeof (osmv_TOPSPIN_ANAFA_transport_mgr_t)); + memset(p_mgr, 0, sizeof(osmv_TOPSPIN_ANAFA_transport_mgr_t)); /* open TopSpin file device */ - device_fd = open (device_file, O_RDWR); + device_fd = open(device_file, O_RDWR); if (device_fd < 0) { - fprintf (stderr, "Fatal: Fail to open the file:%s err:%d\n", - device_file, errno); + fprintf(stderr, "Fatal: Fail to open the file:%s err:%d\n", + device_file, errno); return IB_ERROR; } p_mgr->device_fd = device_fd; @@ -205,7 +201,7 @@ osmv_transport_init ( * Create the MAD filter on this file handle. */ - filter.port = 0; /* Victor */ + filter.port = 0; /* Victor */ filter.direction = TS_IB_MAD_DIRECTION_IN; filter.mask = TS_IB_MAD_FILTER_DIRECTION | @@ -217,13 +213,13 @@ osmv_transport_init ( case IB_MCLASS_SUBN_DIR: filter.qpn = 0; filter.mgmt_class = IB_MCLASS_SUBN_LID; - ts_ioctl_ret = ioctl (device_fd, TS_IB_IOCSMADFILTADD, &filter); + ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &filter); if (ts_ioctl_ret < 0) { return IB_ERROR; } filter.mgmt_class = IB_MCLASS_SUBN_DIR; - ts_ioctl_ret = ioctl (device_fd, TS_IB_IOCSMADFILTADD, &filter); + ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &filter); if (ts_ioctl_ret < 0) { return IB_ERROR; } @@ -234,7 +230,7 @@ osmv_transport_init ( default: filter.qpn = 1; filter.mgmt_class = p_info->mad_class; - ts_ioctl_ret = ioctl (device_fd, TS_IB_IOCSMADFILTADD, &filter); + ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &filter); if (ts_ioctl_ret < 0) { return IB_ERROR; } @@ -250,8 +246,8 @@ osmv_transport_init ( /* init receiver thread */ cl_st = - cl_thread_init (&p_mgr->receiver, __osmv_TOPSPIN_ANAFA_receiver_thr, - (void *)p_bo, "osmv TOPSPIN_ANAFA rcv thr"); + cl_thread_init(&p_mgr->receiver, __osmv_TOPSPIN_ANAFA_receiver_thr, + (void *)p_bo, "osmv TOPSPIN_ANAFA rcv thr"); return (ib_api_status_t) cl_st; } @@ -265,9 +261,8 @@ osmv_transport_init ( */ ib_api_status_t -osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, - IN void *p_mad, - IN const osm_mad_addr_t * p_mad_addr) +osmv_transport_mad_send(IN const osm_bind_handle_t h_bind, + IN void *p_mad, IN const osm_mad_addr_t * p_mad_addr) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; @@ -278,7 +273,7 @@ osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, const ib_mad_t *p_mad_hdr = p_mad; - OSM_LOG_ENTER (p_vend->p_log, osmv_transport_mad_send); + OSM_LOG_ENTER(p_vend->p_log, osmv_transport_mad_send); /* Make sure the p_bo object is still relevant */ if (p_bo->magic_ptr != p_bo) @@ -287,7 +282,7 @@ osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, /* * Copy the MAD over to the sent mad */ - memcpy (&ts_mad, p_mad_hdr, MAD_BLOCK_SIZE); + memcpy(&ts_mad, p_mad_hdr, MAD_BLOCK_SIZE); /* * For all sends other than directed route SM MADs, @@ -295,11 +290,11 @@ osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, */ if (p_mad_hdr->mgmt_class != IB_MCLASS_SUBN_DIR) { - __osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr (p_mad_addr, - p_mad_hdr-> - mgmt_class == - IB_MCLASS_SUBN_LID, - &ts_mad); + __osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr(p_mad_addr, + p_mad_hdr-> + mgmt_class == + IB_MCLASS_SUBN_LID, + &ts_mad); } else { /* is a directed route - we need to construct a permissive address */ /* we do not need port number since it is part of the mad_hndl */ @@ -316,18 +311,18 @@ osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, } /* ts_mad.port = p_bo->port_num; */ - ts_mad.port = 0; /* Victor */ + ts_mad.port = 0; /* Victor */ /* send it */ ret = - write (((osmv_TOPSPIN_ANAFA_transport_mgr_t *) (p_bo-> - p_transp_mgr))-> - device_fd, &ts_mad, sizeof (ts_mad)); - - if (ret != sizeof (ts_mad)) { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osmv_transport_mad_send: ERR 6904: " - "Error sending mad (%d).\n", ret); + write(((osmv_TOPSPIN_ANAFA_transport_mgr_t *) (p_bo-> + p_transp_mgr))-> + device_fd, &ts_mad, sizeof(ts_mad)); + + if (ret != sizeof(ts_mad)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osmv_transport_mad_send: ERR 6904: " + "Error sending mad (%d).\n", ret); status = IB_ERROR; goto Exit; } @@ -335,39 +330,38 @@ osmv_transport_mad_send (IN const osm_bind_handle_t h_bind, status = IB_SUCCESS; Exit: - OSM_LOG_EXIT (p_vend->p_log); + OSM_LOG_EXIT(p_vend->p_log); return (status); } -void -osmv_transport_done (IN const osm_bind_handle_t h_bind) +void osmv_transport_done(IN const osm_bind_handle_t h_bind) { osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; osmv_TOPSPIN_ANAFA_transport_mgr_t *p_tpot_mgr = (osmv_TOPSPIN_ANAFA_transport_mgr_t *) (p_bo->p_transp_mgr); - CL_ASSERT (p_bo); + CL_ASSERT(p_bo); /* First of all - zero out the magic_ptr, so if a callback is called - it'll know that we are currently closing down, and will not handle the mad. */ p_bo->magic_ptr = 0; - /* usleep(3000000); */ + /* usleep(3000000); */ /* pthread_cancel (p_tpot_mgr->receiver.osd.id); */ - cl_thread_destroy (&(p_tpot_mgr->receiver)); - free (p_tpot_mgr); + cl_thread_destroy(&(p_tpot_mgr->receiver)); + free(p_tpot_mgr); } static void -__osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr (IN const osm_mad_addr_t * p_mad_addr, - IN uint8_t is_smi, - OUT struct ib_mad *p_mad) +__osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr(IN const osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, + OUT struct ib_mad *p_mad) { /* For global destination or Multicast address: */ - p_mad->dlid = cl_ntoh16 (p_mad_addr->dest_lid); + p_mad->dlid = cl_ntoh16(p_mad_addr->dest_lid); p_mad->sl = p_mad_addr->addr_type.gsi.service_level; if (is_smi) { p_mad->sqpn = 0; @@ -384,17 +378,17 @@ __osmv_TOPSPIN_ANAFA_osm_addr_to_mad_addr (IN const osm_mad_addr_t * p_mad_addr, } static void -__osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr (IN osm_vendor_t const *p_vend, - IN struct ib_mad *p_mad, - IN uint8_t is_smi, - OUT osm_mad_addr_t * p_mad_addr) +__osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr(IN osm_vendor_t const *p_vend, + IN struct ib_mad *p_mad, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr) { - p_mad_addr->dest_lid = cl_hton16 (p_mad->slid); + p_mad_addr->dest_lid = cl_hton16(p_mad->slid); p_mad_addr->static_rate = 0; p_mad_addr->path_bits = 0; if (is_smi) { /* SMI */ - p_mad_addr->addr_type.smi.source_lid = cl_hton16 (p_mad->slid); + p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_mad->slid); p_mad_addr->addr_type.smi.port_num = p_mad->port; } else { /* GSI */ @@ -427,6 +421,3 @@ __osmv_TOPSPIN_ANAFA_mad_addr_to_osm_addr (IN osm_vendor_t const *p_vend, */ } } - - - diff --git a/opensm/libvendor/osm_vendor_mlx_txn.c b/opensm/libvendor/osm_vendor_mlx_txn.c index 234e33b..1399ad6 100644 --- a/opensm/libvendor/osm_vendor_mlx_txn.c +++ b/opensm/libvendor/osm_vendor_mlx_txn.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -46,147 +46,138 @@ #include static ib_api_status_t -__osmv_txnmgr_lookup(IN osmv_txn_mgr_t *p_tx_mgr, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn); +__osmv_txnmgr_lookup(IN osmv_txn_mgr_t * p_tx_mgr, + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn); static ib_api_status_t -__osmv_txnmgr_insert_txn(IN osmv_txn_mgr_t *p_tx_mgr, - IN osmv_txn_ctx_t *p_txn, - IN uint64_t key); +__osmv_txnmgr_insert_txn(IN osmv_txn_mgr_t * p_tx_mgr, + IN osmv_txn_ctx_t * p_txn, IN uint64_t key); static ib_api_status_t -__osmv_txnmgr_remove_txn(IN osmv_txn_mgr_t *p_tx_mgr, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn); +__osmv_txnmgr_remove_txn(IN osmv_txn_mgr_t * p_tx_mgr, + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn); -static void -__osmv_txn_all_done(osm_bind_handle_t h_bind); +static void __osmv_txn_all_done(osm_bind_handle_t h_bind); static uint64_t __osmv_txn_timeout_cb(IN uint64_t key, - IN uint32_t num_regs, - IN void *cb_context); + IN uint32_t num_regs, IN void *cb_context); ib_api_status_t -osmv_txn_init(IN osm_bind_handle_t h_bind, - IN uint64_t tid, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn) +osmv_txn_init(IN osm_bind_handle_t h_bind, + IN uint64_t tid, IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn) { - ib_api_status_t st; - osmv_txn_ctx_t *p_txn; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, osmv_txn_init); - - CL_ASSERT(NULL != h_bind && NULL != pp_txn); - - osm_log( p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Starting transaction 0x%llX (key=0x%llX)\n", tid, key); - - p_txn = malloc(sizeof(osmv_txn_ctx_t)); - if (! p_txn) - { - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_txn, 0, sizeof(osmv_txn_ctx_t)); - p_txn->p_log = p_bo->txn_mgr.p_log; - p_txn->tid = tid; - p_txn->key = key; - p_txn->p_madw = NULL; - p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_NONE; - - /* insert into transaction manager DB */ - st= __osmv_txnmgr_insert_txn(&p_bo->txn_mgr, p_txn, key); - if (IB_SUCCESS != st) - { - osm_log( p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "osmv_txn_init: ERR 6703: " - "Failed to insert to transaction 0x%llX (key=0x%llX) to manager DB\n", tid, key); - goto insert_txn_failed; - } - - *pp_txn = p_txn; - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); - return IB_SUCCESS; - - insert_txn_failed: - free(p_txn); - - OSM_LOG_EXIT( p_bo->p_vendor->p_log ); - return st; + ib_api_status_t st; + osmv_txn_ctx_t *p_txn; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_txn_init); + + CL_ASSERT(NULL != h_bind && NULL != pp_txn); + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Starting transaction 0x%llX (key=0x%llX)\n", tid, key); + + p_txn = malloc(sizeof(osmv_txn_ctx_t)); + if (!p_txn) { + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_txn, 0, sizeof(osmv_txn_ctx_t)); + p_txn->p_log = p_bo->txn_mgr.p_log; + p_txn->tid = tid; + p_txn->key = key; + p_txn->p_madw = NULL; + p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_NONE; + + /* insert into transaction manager DB */ + st = __osmv_txnmgr_insert_txn(&p_bo->txn_mgr, p_txn, key); + if (IB_SUCCESS != st) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "osmv_txn_init: ERR 6703: " + "Failed to insert to transaction 0x%llX (key=0x%llX) to manager DB\n", + tid, key); + goto insert_txn_failed; + } + + *pp_txn = p_txn; + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return IB_SUCCESS; + + insert_txn_failed: + free(p_txn); + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return st; } ib_api_status_t -osmv_txn_init_rmpp_sender(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN osm_madw_t *p_madw) +osmv_txn_init_rmpp_sender(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, IN osm_madw_t * p_madw) { - ib_api_status_t st; + ib_api_status_t st; - CL_ASSERT(p_txn); + CL_ASSERT(p_txn); - /* Double-Sided RMPP Direction Switch */ - osmv_txn_remove_timeout_ev(h_bind, osmv_txn_get_key(p_txn)); + /* Double-Sided RMPP Direction Switch */ + osmv_txn_remove_timeout_ev(h_bind, osmv_txn_get_key(p_txn)); - p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_SENDER; - p_txn->rmpp_txfr.p_rmpp_send_ctx = malloc(sizeof(osmv_rmpp_send_ctx_t)); + p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_SENDER; + p_txn->rmpp_txfr.p_rmpp_send_ctx = malloc(sizeof(osmv_rmpp_send_ctx_t)); - if (!p_txn->rmpp_txfr.p_rmpp_send_ctx) - { - return IB_INSUFFICIENT_MEMORY; - } + if (!p_txn->rmpp_txfr.p_rmpp_send_ctx) { + return IB_INSUFFICIENT_MEMORY; + } - memset(p_txn->rmpp_txfr.p_rmpp_send_ctx, 0, sizeof(osmv_rmpp_send_ctx_t)); + memset(p_txn->rmpp_txfr.p_rmpp_send_ctx, 0, + sizeof(osmv_rmpp_send_ctx_t)); - st = osmv_rmpp_send_ctx_init(p_txn->rmpp_txfr.p_rmpp_send_ctx, - (void*)p_madw->p_mad, - p_madw->mad_size, - p_txn->p_log); - return st; + st = osmv_rmpp_send_ctx_init(p_txn->rmpp_txfr.p_rmpp_send_ctx, + (void *)p_madw->p_mad, + p_madw->mad_size, p_txn->p_log); + return st; } ib_api_status_t -osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, - IN osmv_txn_ctx_t *p_txn, - IN boolean_t is_init_by_peer) +osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, + IN osmv_txn_ctx_t * p_txn, + IN boolean_t is_init_by_peer) { - ib_api_status_t st; - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *)h_bind; - uint64_t key = osmv_txn_get_key(p_txn); + ib_api_status_t st; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + uint64_t key = osmv_txn_get_key(p_txn); - CL_ASSERT(p_txn); + CL_ASSERT(p_txn); - /* Double-Sided RMPP Direction Switch */ - osmv_txn_remove_timeout_ev(h_bind, key); + /* Double-Sided RMPP Direction Switch */ + osmv_txn_remove_timeout_ev(h_bind, key); - /* Set the Transaction Timeout value */ - st = osmv_txn_set_timeout_ev(h_bind, key, p_bo->p_vendor->ttime_timeout); - if (IB_SUCCESS != st) - { + /* Set the Transaction Timeout value */ + st = osmv_txn_set_timeout_ev(h_bind, key, + p_bo->p_vendor->ttime_timeout); + if (IB_SUCCESS != st) { - return st; - } + return st; + } - p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_RECEIVER; - p_txn->rmpp_txfr.is_rmpp_init_by_peer = is_init_by_peer; + p_txn->rmpp_txfr.rmpp_state = OSMV_TXN_RMPP_RECEIVER; + p_txn->rmpp_txfr.is_rmpp_init_by_peer = is_init_by_peer; - p_txn->rmpp_txfr.p_rmpp_recv_ctx = malloc(sizeof(osmv_rmpp_recv_ctx_t)); + p_txn->rmpp_txfr.p_rmpp_recv_ctx = malloc(sizeof(osmv_rmpp_recv_ctx_t)); - if (!p_txn->rmpp_txfr.p_rmpp_recv_ctx) - { + if (!p_txn->rmpp_txfr.p_rmpp_recv_ctx) { - osmv_txn_remove_timeout_ev(h_bind, key); - return IB_INSUFFICIENT_MEMORY; - } + osmv_txn_remove_timeout_ev(h_bind, key); + return IB_INSUFFICIENT_MEMORY; + } - memset(p_txn->rmpp_txfr.p_rmpp_recv_ctx, 0, sizeof(osmv_rmpp_recv_ctx_t)); + memset(p_txn->rmpp_txfr.p_rmpp_recv_ctx, 0, + sizeof(osmv_rmpp_recv_ctx_t)); - st = osmv_rmpp_recv_ctx_init(p_txn->rmpp_txfr.p_rmpp_recv_ctx,p_txn->p_log); + st = osmv_rmpp_recv_ctx_init(p_txn->rmpp_txfr.p_rmpp_recv_ctx, + p_txn->p_log); - return st; + return st; } /* @@ -199,21 +190,18 @@ osmv_txn_init_rmpp_receiver(IN osm_bind_handle_t h_bind, * */ ib_api_status_t -osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, - IN uint64_t key, - IN uint64_t msec) +osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, + IN uint64_t key, IN uint64_t msec) { - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *)h_bind; - cl_event_wheel_t *p_event_wheel = p_bo->txn_mgr.p_event_wheel; - cl_status_t status; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + cl_event_wheel_t *p_event_wheel = p_bo->txn_mgr.p_event_wheel; + cl_status_t status; - status = cl_event_wheel_reg(p_event_wheel, - key, - cl_get_time_stamp() + 1000*msec, /* TTL */ - __osmv_txn_timeout_cb, - p_bo /* The context */); + status = cl_event_wheel_reg(p_event_wheel, key, cl_get_time_stamp() + 1000 * msec, /* TTL */ + __osmv_txn_timeout_cb, + p_bo /* The context */ ); - return (ib_api_status_t)status; + return (ib_api_status_t) status; } /* @@ -225,493 +213,463 @@ osmv_txn_set_timeout_ev(IN osm_bind_handle_t h_bind, * SEE ALSO * */ -void -osmv_txn_remove_timeout_ev(IN osm_bind_handle_t h_bind, - IN uint64_t key) +void osmv_txn_remove_timeout_ev(IN osm_bind_handle_t h_bind, IN uint64_t key) { - cl_event_wheel_t *p_event_wheel = ((osmv_bind_obj_t *)h_bind)->txn_mgr.p_event_wheel; - cl_event_wheel_unreg(p_event_wheel, key); + cl_event_wheel_t *p_event_wheel = + ((osmv_bind_obj_t *) h_bind)->txn_mgr.p_event_wheel; + cl_event_wheel_unreg(p_event_wheel, key); } void osmv_txn_done(IN osm_bind_handle_t h_bind, - IN uint64_t key, - IN boolean_t is_in_cb) + IN uint64_t key, IN boolean_t is_in_cb) { - osmv_txn_ctx_t* p_ctx; - osmv_bind_obj_t * const p_bo = (osmv_bind_obj_t *)h_bind; - - OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_txn_done); - - CL_ASSERT(h_bind); - - /* Cancel the (single) timeout possibly outstanding for this txn - * Don't do this if you are in the callback context, for 2 reasons: - * (1) The event wheel will remove the context itself. - * (2) If we try to, there is a deadlock in the event wheel - */ - if (FALSE == is_in_cb) - { - osmv_txn_remove_timeout_ev(h_bind, key); - } - - /* Remove from DB */ - if (IB_NOT_FOUND == __osmv_txnmgr_remove_txn(&p_bo->txn_mgr, key, &p_ctx)) - { - return; - } - - /* Destroy the transaction's RMPP contexts - * (can be more than one in the case of double sided transfer) - */ - - if (p_ctx->rmpp_txfr.p_rmpp_send_ctx) - { - osmv_rmpp_send_ctx_done(p_ctx->rmpp_txfr.p_rmpp_send_ctx); - } - - if (p_ctx->rmpp_txfr.p_rmpp_recv_ctx) - { - osmv_rmpp_recv_ctx_done(p_ctx->rmpp_txfr.p_rmpp_recv_ctx); - } - - free(p_ctx); - - OSM_LOG_EXIT(p_bo->p_vendor->p_log); + osmv_txn_ctx_t *p_ctx; + osmv_bind_obj_t *const p_bo = (osmv_bind_obj_t *) h_bind; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_txn_done); + + CL_ASSERT(h_bind); + + /* Cancel the (single) timeout possibly outstanding for this txn + * Don't do this if you are in the callback context, for 2 reasons: + * (1) The event wheel will remove the context itself. + * (2) If we try to, there is a deadlock in the event wheel + */ + if (FALSE == is_in_cb) { + osmv_txn_remove_timeout_ev(h_bind, key); + } + + /* Remove from DB */ + if (IB_NOT_FOUND == + __osmv_txnmgr_remove_txn(&p_bo->txn_mgr, key, &p_ctx)) { + return; + } + + /* Destroy the transaction's RMPP contexts + * (can be more than one in the case of double sided transfer) + */ + + if (p_ctx->rmpp_txfr.p_rmpp_send_ctx) { + osmv_rmpp_send_ctx_done(p_ctx->rmpp_txfr.p_rmpp_send_ctx); + } + + if (p_ctx->rmpp_txfr.p_rmpp_recv_ctx) { + osmv_rmpp_recv_ctx_done(p_ctx->rmpp_txfr.p_rmpp_recv_ctx); + } + + free(p_ctx); + + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } ib_api_status_t osmv_txn_lookup(IN osm_bind_handle_t h_bind, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn) + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn) { - return __osmv_txnmgr_lookup(&(((osmv_bind_obj_t*)h_bind)->txn_mgr), key, pp_txn); + return __osmv_txnmgr_lookup(&(((osmv_bind_obj_t *) h_bind)->txn_mgr), + key, pp_txn); } - -void -osmv_txn_abort_rmpp_txns(osm_bind_handle_t h_bind) +void osmv_txn_abort_rmpp_txns(osm_bind_handle_t h_bind) { - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)h_bind; - cl_map_item_t *p_item; - cl_map_obj_t *p_obj; - osmv_txn_ctx_t *p_txn; - osmv_rmpp_send_ctx_t *p_send_ctx; - cl_qmap_t *p_map = p_bo->txn_mgr.p_txn_map; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + cl_map_item_t *p_item; + cl_map_obj_t *p_obj; + osmv_txn_ctx_t *p_txn; + osmv_rmpp_send_ctx_t *p_send_ctx; + cl_qmap_t *p_map = p_bo->txn_mgr.p_txn_map; - OSM_LOG_ENTER( p_bo->p_vendor->p_log, osmv_txn_abort_rmpp_txns); + OSM_LOG_ENTER(p_bo->p_vendor->p_log, osmv_txn_abort_rmpp_txns); - while (FALSE == cl_is_qmap_empty(p_map)) { + while (FALSE == cl_is_qmap_empty(p_map)) { - p_item = cl_qmap_head(p_map); - p_obj = PARENT_STRUCT(p_item,cl_map_obj_t,item); - p_txn = (osmv_txn_ctx_t*)cl_qmap_obj(p_obj); - p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); + p_item = cl_qmap_head(p_map); + p_obj = PARENT_STRUCT(p_item, cl_map_obj_t, item); + p_txn = (osmv_txn_ctx_t *) cl_qmap_obj(p_obj); + p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); - if (NULL != p_send_ctx) - { + if (NULL != p_send_ctx) { - p_send_ctx->status = IB_INTERRUPTED; + p_send_ctx->status = IB_INTERRUPTED; - /* Wake up the sender thread to let it break out */ - cl_event_signal(&p_send_ctx->event); - } + /* Wake up the sender thread to let it break out */ + cl_event_signal(&p_send_ctx->event); + } - cl_qmap_remove_item(p_map, p_item); - } + cl_qmap_remove_item(p_map, p_item); + } - OSM_LOG_EXIT(p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } - ib_api_status_t -osmv_txnmgr_init(IN osmv_txn_mgr_t *p_tx_mgr, - IN osm_log_t *p_log, - IN cl_spinlock_t *p_lock) +osmv_txnmgr_init(IN osmv_txn_mgr_t * p_tx_mgr, + IN osm_log_t * p_log, IN cl_spinlock_t * p_lock) { - cl_status_t cl_st = CL_SUCCESS; - - p_tx_mgr->p_event_wheel = malloc(sizeof(cl_event_wheel_t)); - if (!p_tx_mgr->p_event_wheel) - { - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_tx_mgr->p_event_wheel, 0, sizeof(cl_event_wheel_t)); - - cl_event_wheel_construct(p_tx_mgr->p_event_wheel); - - /* NOTE! We are using an extended constructor. - * We tell the Event Wheel run in a non-protected manner in the reg/unreg calls, - * and acquire an external lock in the asynchronous callback. - */ - cl_st = cl_event_wheel_init_ex(p_tx_mgr->p_event_wheel, p_log, p_lock); - if (cl_st != CL_SUCCESS) - { - free(p_tx_mgr->p_event_wheel); - return (ib_api_status_t)cl_st; - } - - p_tx_mgr->p_txn_map = malloc(sizeof(cl_qmap_t)); - if (!p_tx_mgr->p_txn_map) - { - cl_event_wheel_destroy(p_tx_mgr->p_event_wheel); - free(p_tx_mgr->p_event_wheel); - return IB_INSUFFICIENT_MEMORY; - } - - memset(p_tx_mgr->p_txn_map, 0, sizeof(cl_qmap_t)); - - cl_qmap_init(p_tx_mgr->p_txn_map); - p_tx_mgr->p_log = p_log; - - return cl_st; + cl_status_t cl_st = CL_SUCCESS; + + p_tx_mgr->p_event_wheel = malloc(sizeof(cl_event_wheel_t)); + if (!p_tx_mgr->p_event_wheel) { + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_tx_mgr->p_event_wheel, 0, sizeof(cl_event_wheel_t)); + + cl_event_wheel_construct(p_tx_mgr->p_event_wheel); + + /* NOTE! We are using an extended constructor. + * We tell the Event Wheel run in a non-protected manner in the reg/unreg calls, + * and acquire an external lock in the asynchronous callback. + */ + cl_st = cl_event_wheel_init_ex(p_tx_mgr->p_event_wheel, p_log, p_lock); + if (cl_st != CL_SUCCESS) { + free(p_tx_mgr->p_event_wheel); + return (ib_api_status_t) cl_st; + } + + p_tx_mgr->p_txn_map = malloc(sizeof(cl_qmap_t)); + if (!p_tx_mgr->p_txn_map) { + cl_event_wheel_destroy(p_tx_mgr->p_event_wheel); + free(p_tx_mgr->p_event_wheel); + return IB_INSUFFICIENT_MEMORY; + } + + memset(p_tx_mgr->p_txn_map, 0, sizeof(cl_qmap_t)); + + cl_qmap_init(p_tx_mgr->p_txn_map); + p_tx_mgr->p_log = p_log; + + return cl_st; } -void -osmv_txnmgr_done(IN osm_bind_handle_t h_bind) +void osmv_txnmgr_done(IN osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; - __osmv_txn_all_done(h_bind); - free(p_bo->txn_mgr.p_txn_map); + __osmv_txn_all_done(h_bind); + free(p_bo->txn_mgr.p_txn_map); - cl_event_wheel_destroy(p_bo->txn_mgr.p_event_wheel); - free(p_bo->txn_mgr.p_event_wheel); + cl_event_wheel_destroy(p_bo->txn_mgr.p_event_wheel); + free(p_bo->txn_mgr.p_event_wheel); } ib_api_status_t -__osmv_txnmgr_lookup(IN osmv_txn_mgr_t *p_tx_mgr, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn) +__osmv_txnmgr_lookup(IN osmv_txn_mgr_t * p_tx_mgr, + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn) { - ib_api_status_t status = IB_SUCCESS; - cl_map_item_t* p_item; - cl_map_obj_t* p_obj; - - uint64_t tmp_key; - - OSM_LOG_ENTER( p_tx_mgr->p_log, __osmv_txnmgr_lookup ); - - CL_ASSERT(p_tx_mgr); - CL_ASSERT(pp_txn); - - osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, - "__osmv_txnmgr_lookup: " - "Looking for key: 0x%llX in map ptr:%p\n", key, p_tx_mgr->p_txn_map ); - - p_item = cl_qmap_head(p_tx_mgr->p_txn_map); - while( p_item != cl_qmap_end(p_tx_mgr->p_txn_map) ) - { - tmp_key = cl_qmap_key(p_item); - osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, - "__osmv_txnmgr_lookup: " - "Found key 0x%llX \n", tmp_key ); - p_item = cl_qmap_next(p_item); - } - - p_item= cl_qmap_get(p_tx_mgr->p_txn_map,key); - if (cl_qmap_end(p_tx_mgr->p_txn_map) == p_item) - { - status = IB_NOT_FOUND; - } - else - { - p_obj = PARENT_STRUCT(p_item, cl_map_obj_t,item); - *pp_txn = cl_qmap_obj(p_obj); - } - - OSM_LOG_EXIT(p_tx_mgr->p_log); - return status; + ib_api_status_t status = IB_SUCCESS; + cl_map_item_t *p_item; + cl_map_obj_t *p_obj; + + uint64_t tmp_key; + + OSM_LOG_ENTER(p_tx_mgr->p_log, __osmv_txnmgr_lookup); + + CL_ASSERT(p_tx_mgr); + CL_ASSERT(pp_txn); + + osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, + "__osmv_txnmgr_lookup: " + "Looking for key: 0x%llX in map ptr:%p\n", key, + p_tx_mgr->p_txn_map); + + p_item = cl_qmap_head(p_tx_mgr->p_txn_map); + while (p_item != cl_qmap_end(p_tx_mgr->p_txn_map)) { + tmp_key = cl_qmap_key(p_item); + osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, + "__osmv_txnmgr_lookup: " + "Found key 0x%llX \n", tmp_key); + p_item = cl_qmap_next(p_item); + } + + p_item = cl_qmap_get(p_tx_mgr->p_txn_map, key); + if (cl_qmap_end(p_tx_mgr->p_txn_map) == p_item) { + status = IB_NOT_FOUND; + } else { + p_obj = PARENT_STRUCT(p_item, cl_map_obj_t, item); + *pp_txn = cl_qmap_obj(p_obj); + } + + OSM_LOG_EXIT(p_tx_mgr->p_log); + return status; } ib_api_status_t -__osmv_txnmgr_insert_txn(IN osmv_txn_mgr_t *p_tx_mgr, - IN osmv_txn_ctx_t *p_txn, - IN uint64_t key) +__osmv_txnmgr_insert_txn(IN osmv_txn_mgr_t * p_tx_mgr, + IN osmv_txn_ctx_t * p_txn, IN uint64_t key) { - cl_map_obj_t* p_obj = NULL; - cl_map_item_t* p_item; - uint64_t tmp_key; - - CL_ASSERT(p_tx_mgr); - CL_ASSERT(p_txn); - - key = osmv_txn_get_key(p_txn); - p_obj = malloc(sizeof(cl_map_obj_t)); - if (NULL == p_obj) - return IB_INSUFFICIENT_MEMORY; - - osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, - "__osmv_txnmgr_insert_txn: " - "Inserting key: 0x%llX to map ptr:%p\n", key, p_tx_mgr->p_txn_map ); - - memset(p_obj, 0, sizeof(cl_map_obj_t)); - - cl_qmap_set_obj(p_obj,p_txn); - /* assuming lookup with this key was made and the result was IB_NOT_FOUND */ - cl_qmap_insert(p_tx_mgr->p_txn_map, key, &p_obj->item); - - p_item = cl_qmap_head(p_tx_mgr->p_txn_map); - while( p_item != cl_qmap_end(p_tx_mgr->p_txn_map) ) - { - tmp_key = cl_qmap_key(p_item); - osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, - "__osmv_txnmgr_insert_txn: " - "Found key 0x%llX \n", tmp_key ); - p_item = cl_qmap_next(p_item); - } - - return IB_SUCCESS; + cl_map_obj_t *p_obj = NULL; + cl_map_item_t *p_item; + uint64_t tmp_key; + + CL_ASSERT(p_tx_mgr); + CL_ASSERT(p_txn); + + key = osmv_txn_get_key(p_txn); + p_obj = malloc(sizeof(cl_map_obj_t)); + if (NULL == p_obj) + return IB_INSUFFICIENT_MEMORY; + + osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, + "__osmv_txnmgr_insert_txn: " + "Inserting key: 0x%llX to map ptr:%p\n", key, + p_tx_mgr->p_txn_map); + + memset(p_obj, 0, sizeof(cl_map_obj_t)); + + cl_qmap_set_obj(p_obj, p_txn); + /* assuming lookup with this key was made and the result was IB_NOT_FOUND */ + cl_qmap_insert(p_tx_mgr->p_txn_map, key, &p_obj->item); + + p_item = cl_qmap_head(p_tx_mgr->p_txn_map); + while (p_item != cl_qmap_end(p_tx_mgr->p_txn_map)) { + tmp_key = cl_qmap_key(p_item); + osm_log(p_tx_mgr->p_log, OSM_LOG_DEBUG, + "__osmv_txnmgr_insert_txn: " + "Found key 0x%llX \n", tmp_key); + p_item = cl_qmap_next(p_item); + } + + return IB_SUCCESS; } ib_api_status_t -__osmv_txnmgr_remove_txn(IN osmv_txn_mgr_t *p_tx_mgr, - IN uint64_t key, - OUT osmv_txn_ctx_t **pp_txn) +__osmv_txnmgr_remove_txn(IN osmv_txn_mgr_t * p_tx_mgr, + IN uint64_t key, OUT osmv_txn_ctx_t ** pp_txn) { - cl_map_obj_t* p_obj; - cl_map_item_t* p_item; + cl_map_obj_t *p_obj; + cl_map_item_t *p_item; - OSM_LOG_ENTER(p_tx_mgr->p_log, __osmv_txnmgr_remove_txn); + OSM_LOG_ENTER(p_tx_mgr->p_log, __osmv_txnmgr_remove_txn); - CL_ASSERT(p_tx_mgr); - CL_ASSERT(pp_txn); + CL_ASSERT(p_tx_mgr); + CL_ASSERT(pp_txn); - p_item= cl_qmap_remove(p_tx_mgr->p_txn_map, key); + p_item = cl_qmap_remove(p_tx_mgr->p_txn_map, key); - if (p_item == cl_qmap_end(p_tx_mgr->p_txn_map)) - { + if (p_item == cl_qmap_end(p_tx_mgr->p_txn_map)) { - osm_log(p_tx_mgr->p_log, OSM_LOG_ERROR, - "__osmv_txnmgr_remove_txn: ERR 6701: " - "Could not remove the transaction 0x%llX - " - "something is really wrong!\n", key); - OSM_LOG_EXIT(p_tx_mgr->p_log); - return IB_NOT_FOUND; - } + osm_log(p_tx_mgr->p_log, OSM_LOG_ERROR, + "__osmv_txnmgr_remove_txn: ERR 6701: " + "Could not remove the transaction 0x%llX - " + "something is really wrong!\n", key); + OSM_LOG_EXIT(p_tx_mgr->p_log); + return IB_NOT_FOUND; + } - p_obj = PARENT_STRUCT(p_item, cl_map_obj_t,item); - *pp_txn = cl_qmap_obj(p_obj); + p_obj = PARENT_STRUCT(p_item, cl_map_obj_t, item); + *pp_txn = cl_qmap_obj(p_obj); - free(p_obj); + free(p_obj); - OSM_LOG_EXIT(p_tx_mgr->p_log); - return IB_SUCCESS; + OSM_LOG_EXIT(p_tx_mgr->p_log); + return IB_SUCCESS; } -void -__osmv_txn_all_done(osm_bind_handle_t h_bind) +void __osmv_txn_all_done(osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - cl_map_item_t *p_item; - cl_map_obj_t* p_obj; - osmv_txn_ctx_t* p_txn; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + cl_map_item_t *p_item; + cl_map_obj_t *p_obj; + osmv_txn_ctx_t *p_txn; - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_txn_all_done); + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_txn_all_done); - p_item = cl_qmap_head(p_bo->txn_mgr.p_txn_map); - while ( p_item != cl_qmap_end(p_bo->txn_mgr.p_txn_map)) { + p_item = cl_qmap_head(p_bo->txn_mgr.p_txn_map); + while (p_item != cl_qmap_end(p_bo->txn_mgr.p_txn_map)) { - p_obj = PARENT_STRUCT(p_item,cl_map_obj_t,item); - p_txn = (osmv_txn_ctx_t*)cl_qmap_obj(p_obj); - osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); - free(p_obj); - /* assuming osmv_txn_done has removed the txn from the map */ - p_item = cl_qmap_head(p_bo->txn_mgr.p_txn_map); - } + p_obj = PARENT_STRUCT(p_item, cl_map_obj_t, item); + p_txn = (osmv_txn_ctx_t *) cl_qmap_obj(p_obj); + osmv_txn_done(h_bind, osmv_txn_get_key(p_txn), FALSE); + free(p_obj); + /* assuming osmv_txn_done has removed the txn from the map */ + p_item = cl_qmap_head(p_bo->txn_mgr.p_txn_map); + } - OSM_LOG_EXIT(p_bo->p_vendor->p_log); + OSM_LOG_EXIT(p_bo->p_vendor->p_log); } /******************************************************************************/ -void osmv_txn_lock(IN osm_bind_handle_t h_bind) +void osmv_txn_lock(IN osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "--> Acquiring lock %p on bind handle %p\n", &p_bo->lock, p_bo); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "--> Acquiring lock %p on bind handle %p\n", &p_bo->lock, p_bo); - cl_spinlock_acquire(&p_bo->lock); + cl_spinlock_acquire(&p_bo->lock); - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "--> Acquired lock %p on bind handle %p\n", &p_bo->lock, p_bo); + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "--> Acquired lock %p on bind handle %p\n", &p_bo->lock, p_bo); } -void osmv_txn_unlock(IN osm_bind_handle_t h_bind) +void osmv_txn_unlock(IN osm_bind_handle_t h_bind) { - osmv_bind_obj_t* p_bo = (osmv_bind_obj_t*)h_bind; - cl_spinlock_t* p_lock = &p_bo->lock; - osm_log_t* p_log = p_bo->p_vendor->p_log; - - osm_log(p_log, OSM_LOG_DEBUG, - "<-- Releasing lock %p on bind handle %p\n", p_lock, p_bo); + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) h_bind; + cl_spinlock_t *p_lock = &p_bo->lock; + osm_log_t *p_log = p_bo->p_vendor->p_log; - cl_spinlock_release(&p_bo->lock); + osm_log(p_log, OSM_LOG_DEBUG, + "<-- Releasing lock %p on bind handle %p\n", p_lock, p_bo); - /* We'll use the saved ptrs, since now the p_bo can be destroyed already */ - osm_log(p_log, OSM_LOG_DEBUG, - "<-- Released lock %p on bind handle %p\n", p_lock, p_bo); + cl_spinlock_release(&p_bo->lock); + /* We'll use the saved ptrs, since now the p_bo can be destroyed already */ + osm_log(p_log, OSM_LOG_DEBUG, + "<-- Released lock %p on bind handle %p\n", p_lock, p_bo); } static uint64_t __osmv_txn_timeout_cb(IN uint64_t key, - IN uint32_t num_regs, - IN void *cb_context) + IN uint32_t num_regs, IN void *cb_context) { - osmv_bind_obj_t *p_bo = (osmv_bind_obj_t*)cb_context; - uint64_t ret=0; - osmv_txn_ctx_t *p_txn; - osmv_rmpp_send_ctx_t *p_send_ctx; - osm_madw_t *p_madw = NULL; - ib_mad_t *p_mad; - osm_mad_addr_t *p_mad_addr; - boolean_t invoke_err_cb = FALSE; - - OSM_LOG_ENTER( p_bo->p_vendor->p_log, __osmv_txn_timeout_cb ); - - /* Don't try to acquire a lock on the Bind Object - - * it's taken by the mechanism that drives the timeout based events! - * (Recall the special constructor that the Event Wheel is applied with) - */ - if (p_bo->is_closing) - { - goto txn_done; - } - - ret = osmv_txn_lookup(p_bo, key, &p_txn); - if (IB_NOT_FOUND == ret) - { - /* Prevent a race - the transaction is already destroyed */ - goto txn_done; - } - - p_madw = p_txn->p_madw; - - switch (osmv_txn_get_rmpp_state(p_txn)) { - - case OSMV_TXN_RMPP_NONE: - if (num_regs <= OSMV_MAX_RETRANSMIT) - { - /* We still did not exceed the limit of retransmissions. - * Set the next timeout's value. - */ - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_txn_timeout_cb: " - "The transaction request (tid=0x%llX) timed out %d times. " - "Retrying the send.\n", - osmv_txn_get_tid(p_txn), num_regs); - - /* resend this mad */ - ret = osmv_simple_send_madw((osm_bind_handle_t*)p_bo, - p_madw, p_txn, TRUE); - if (ret != IB_SUCCESS) - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_txn_timeout_cb: " - "Fail to send retry for transaction request (tid=0x%llX).\n", - osmv_txn_get_tid(p_txn)); - - osmv_txn_done((osm_bind_handle_t)p_bo, key, - TRUE /*in timeout callback*/); - - /* This is a requester. Always apply the callback */ - invoke_err_cb = TRUE; - } - else - { - uint64_t next_timeout_ms; - next_timeout_ms = - p_bo->p_vendor->resp_timeout*(num_regs+1)*(num_regs+1); - /* when do we need to timeout again */ - ret = cl_get_time_stamp() + (uint64_t)(1000*next_timeout_ms); - - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "__osmv_txn_timeout_cb: " - "Retry request timout in : %lu [msec].\n", - next_timeout_ms); - } - } - else - { - osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, - "__osmv_txn_timeout_cb: ERR 6702: " - "The transaction request (tid=0x%llX) timed out (after %d retries). " - "Invoking the error callback.\n", - osmv_txn_get_tid(p_txn), num_regs); - - osmv_txn_done((osm_bind_handle_t)p_bo, key, - TRUE /*in timeout callback*/); - - /* This is a requester. Always apply the callback */ - invoke_err_cb = TRUE; - } - break; - - case OSMV_TXN_RMPP_SENDER: - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "RMPP sender (tid=0x%llX) did not receive ACK " - "on every segment in the current send window.\n", - osmv_txn_get_tid(p_txn)); - - p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); - if (num_regs <= OSMV_MAX_RETRANSMIT) - { - /* We still did not exceed the limit of retransmissions. - * Set the next timeout's value. - */ - ret = cl_get_time_stamp() + 1000*p_bo->p_vendor->resp_timeout; - } - else - { - p_send_ctx->status = IB_TIMEOUT; - - p_mad = osm_madw_get_mad_ptr(p_madw); - p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); - - /* Send an ABORT to the other side */ - osmv_rmpp_send_nak((osm_bind_handle_t)p_bo, p_mad, p_mad_addr, - IB_RMPP_TYPE_ABORT, - IB_RMPP_STATUS_T2L); - } - - /* Wake the RMPP sender thread up */ - cl_event_signal(&p_send_ctx->event); - break; - - case OSMV_TXN_RMPP_RECEIVER: - osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, - "Transaction timeout on an RMPP receiver (tid=0x%llX). " - "Dropping the transaction.\n", osmv_txn_get_tid(p_txn)); - - osmv_txn_done((osm_bind_handle_t)p_bo, key, - TRUE /*in timeout callback*/); - - if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn)) - { - /* This is a requester, still waiting for the reply. Apply the callback */ - invoke_err_cb = TRUE; - } - - break; - - default: CL_ASSERT(FALSE); - } - - if (TRUE == invoke_err_cb) - { - CL_ASSERT(NULL != p_madw); - /* update the status in the p_madw */ - p_madw->status = IB_TIMEOUT; - p_bo->send_err_cb(p_bo->cb_context, p_madw); - /* no re-registration */ - ret = 0; - } - - txn_done: - OSM_LOG_EXIT(p_bo->p_vendor->p_log); - return ret; + osmv_bind_obj_t *p_bo = (osmv_bind_obj_t *) cb_context; + uint64_t ret = 0; + osmv_txn_ctx_t *p_txn; + osmv_rmpp_send_ctx_t *p_send_ctx; + osm_madw_t *p_madw = NULL; + ib_mad_t *p_mad; + osm_mad_addr_t *p_mad_addr; + boolean_t invoke_err_cb = FALSE; + + OSM_LOG_ENTER(p_bo->p_vendor->p_log, __osmv_txn_timeout_cb); + + /* Don't try to acquire a lock on the Bind Object - + * it's taken by the mechanism that drives the timeout based events! + * (Recall the special constructor that the Event Wheel is applied with) + */ + if (p_bo->is_closing) { + goto txn_done; + } + + ret = osmv_txn_lookup(p_bo, key, &p_txn); + if (IB_NOT_FOUND == ret) { + /* Prevent a race - the transaction is already destroyed */ + goto txn_done; + } + + p_madw = p_txn->p_madw; + + switch (osmv_txn_get_rmpp_state(p_txn)) { + + case OSMV_TXN_RMPP_NONE: + if (num_regs <= OSMV_MAX_RETRANSMIT) { + /* We still did not exceed the limit of retransmissions. + * Set the next timeout's value. + */ + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_txn_timeout_cb: " + "The transaction request (tid=0x%llX) timed out %d times. " + "Retrying the send.\n", + osmv_txn_get_tid(p_txn), num_regs); + + /* resend this mad */ + ret = osmv_simple_send_madw((osm_bind_handle_t *) p_bo, + p_madw, p_txn, TRUE); + if (ret != IB_SUCCESS) { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_txn_timeout_cb: " + "Fail to send retry for transaction request (tid=0x%llX).\n", + osmv_txn_get_tid(p_txn)); + + osmv_txn_done((osm_bind_handle_t) p_bo, key, + TRUE /*in timeout callback */ ); + + /* This is a requester. Always apply the callback */ + invoke_err_cb = TRUE; + } else { + uint64_t next_timeout_ms; + next_timeout_ms = + p_bo->p_vendor->resp_timeout * (num_regs + + 1) * + (num_regs + 1); + /* when do we need to timeout again */ + ret = + cl_get_time_stamp() + + (uint64_t) (1000 * next_timeout_ms); + + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "__osmv_txn_timeout_cb: " + "Retry request timout in : %lu [msec].\n", + next_timeout_ms); + } + } else { + osm_log(p_bo->p_vendor->p_log, OSM_LOG_ERROR, + "__osmv_txn_timeout_cb: ERR 6702: " + "The transaction request (tid=0x%llX) timed out (after %d retries). " + "Invoking the error callback.\n", + osmv_txn_get_tid(p_txn), num_regs); + + osmv_txn_done((osm_bind_handle_t) p_bo, key, + TRUE /*in timeout callback */ ); + + /* This is a requester. Always apply the callback */ + invoke_err_cb = TRUE; + } + break; + + case OSMV_TXN_RMPP_SENDER: + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "RMPP sender (tid=0x%llX) did not receive ACK " + "on every segment in the current send window.\n", + osmv_txn_get_tid(p_txn)); + + p_send_ctx = osmv_txn_get_rmpp_send_ctx(p_txn); + if (num_regs <= OSMV_MAX_RETRANSMIT) { + /* We still did not exceed the limit of retransmissions. + * Set the next timeout's value. + */ + ret = + cl_get_time_stamp() + + 1000 * p_bo->p_vendor->resp_timeout; + } else { + p_send_ctx->status = IB_TIMEOUT; + + p_mad = osm_madw_get_mad_ptr(p_madw); + p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + + /* Send an ABORT to the other side */ + osmv_rmpp_send_nak((osm_bind_handle_t) p_bo, p_mad, + p_mad_addr, IB_RMPP_TYPE_ABORT, + IB_RMPP_STATUS_T2L); + } + + /* Wake the RMPP sender thread up */ + cl_event_signal(&p_send_ctx->event); + break; + + case OSMV_TXN_RMPP_RECEIVER: + osm_log(p_bo->p_vendor->p_log, OSM_LOG_DEBUG, + "Transaction timeout on an RMPP receiver (tid=0x%llX). " + "Dropping the transaction.\n", osmv_txn_get_tid(p_txn)); + + osmv_txn_done((osm_bind_handle_t) p_bo, key, + TRUE /*in timeout callback */ ); + + if (FALSE == osmv_txn_is_rmpp_init_by_peer(p_txn)) { + /* This is a requester, still waiting for the reply. Apply the callback */ + invoke_err_cb = TRUE; + } + + break; + + default: + CL_ASSERT(FALSE); + } + + if (TRUE == invoke_err_cb) { + CL_ASSERT(NULL != p_madw); + /* update the status in the p_madw */ + p_madw->status = IB_TIMEOUT; + p_bo->send_err_cb(p_bo->cb_context, p_madw); + /* no re-registration */ + ret = 0; + } + + txn_done: + OSM_LOG_EXIT(p_bo->p_vendor->p_log); + return ret; } diff --git a/opensm/libvendor/osm_vendor_mtl.c b/opensm/libvendor/osm_vendor_mtl.c index 51561e7..1937ae7 100644 --- a/opensm/libvendor/osm_vendor_mtl.c +++ b/opensm/libvendor/osm_vendor_mtl.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -61,251 +60,239 @@ whether or not it is a response. A race can occure only on requests that did not fail, and then the madw_p will be put back in the pool before the callback. */ -uint64_t -__osm_set_wrid_by_p_madw(IN osm_madw_t *p_madw) { - uint64_t wrid = 0; +uint64_t __osm_set_wrid_by_p_madw(IN osm_madw_t * p_madw) +{ + uint64_t wrid = 0; - CL_ASSERT(p_madw->p_mad); + CL_ASSERT(p_madw->p_mad); - memcpy( &wrid, &p_madw, sizeof( osm_madw_t * ) ); - wrid = (wrid << 1) | - ib_mad_is_response(p_madw->p_mad) | - (p_madw->p_mad->method == IB_MAD_METHOD_TRAP_REPRESS); - return wrid; + memcpy(&wrid, &p_madw, sizeof(osm_madw_t *)); + wrid = (wrid << 1) | + ib_mad_is_response(p_madw->p_mad) | + (p_madw->p_mad->method == IB_MAD_METHOD_TRAP_REPRESS); + return wrid; } void -__osm_set_p_madw_and_resp_by_wrid( - IN uint64_t wrid, - OUT uint8_t *is_resp, - OUT osm_madw_t **pp_madw) { - *is_resp = wrid & 0x0000000000000001; - wrid = wrid >> 1; - memcpy( pp_madw, &wrid, sizeof( osm_madw_t * ) ); +__osm_set_p_madw_and_resp_by_wrid(IN uint64_t wrid, + OUT uint8_t * is_resp, + OUT osm_madw_t ** pp_madw) +{ + *is_resp = wrid & 0x0000000000000001; + wrid = wrid >> 1; + memcpy(pp_madw, &wrid, sizeof(osm_madw_t *)); } /********************************************************************** * IB_MGT to OSM ADDRESS VECTOR **********************************************************************/ void -__osm_mtl_conv_ibmgt_rcv_desc_to_osm_addr( - IN osm_vendor_t * const p_vend, - IN IB_MGT_mad_rcv_desc_t *p_rcv_desc, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr) +__osm_mtl_conv_ibmgt_rcv_desc_to_osm_addr(IN osm_vendor_t * const p_vend, + IN IB_MGT_mad_rcv_desc_t * p_rcv_desc, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr) { - /* p_mad_addr->dest_lid = p_osm->subn.sm_base_lid; - for resp we use the dest lid ... */ - p_mad_addr->dest_lid = cl_hton16(p_rcv_desc->remote_lid); - p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ - p_mad_addr->path_bits = p_rcv_desc->local_path_bits; - if (is_smi) - { - /* SMI */ - p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_rcv_desc->remote_lid); - p_mad_addr->addr_type.smi.port_num = 99; /* HACK - if used - should fail */ - } - else - { - /* GSI */ - /* seems to me there is a IBMGT bug reversing the QPN ... */ - /* Does IBMGT supposed to provide the QPN is network or HOST ? */ - p_mad_addr->addr_type.gsi.remote_qp = cl_hton32(p_rcv_desc->qp); - - p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ - /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ - /* the full PKey table - than go by the index. */ - /* since this does not seem reasonable to me I simply use the default */ - /* There is a TAVOR limitation that only one P_KEY is supported per */ - /* QP - so QP1 must use IB_DEFAULT_PKEY */ - p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_mad_addr->addr_type.gsi.service_level = p_rcv_desc->sl; - - p_mad_addr->addr_type.gsi.global_route = p_rcv_desc->grh_flag; - /* copy the GRH data if relevant */ - if (p_mad_addr->addr_type.gsi.global_route) - { - p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = - ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, - p_rcv_desc->grh.traffic_class, - p_rcv_desc->grh.flow_label); - p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; - memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, - &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); - memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); - } - } + /* p_mad_addr->dest_lid = p_osm->subn.sm_base_lid; - for resp we use the dest lid ... */ + p_mad_addr->dest_lid = cl_hton16(p_rcv_desc->remote_lid); + p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ + p_mad_addr->path_bits = p_rcv_desc->local_path_bits; + if (is_smi) { + /* SMI */ + p_mad_addr->addr_type.smi.source_lid = + cl_hton16(p_rcv_desc->remote_lid); + p_mad_addr->addr_type.smi.port_num = 99; /* HACK - if used - should fail */ + } else { + /* GSI */ + /* seems to me there is a IBMGT bug reversing the QPN ... */ + /* Does IBMGT supposed to provide the QPN is network or HOST ? */ + p_mad_addr->addr_type.gsi.remote_qp = cl_hton32(p_rcv_desc->qp); + + p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ + /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ + /* the full PKey table - than go by the index. */ + /* since this does not seem reasonable to me I simply use the default */ + /* There is a TAVOR limitation that only one P_KEY is supported per */ + /* QP - so QP1 must use IB_DEFAULT_PKEY */ + p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_mad_addr->addr_type.gsi.service_level = p_rcv_desc->sl; + + p_mad_addr->addr_type.gsi.global_route = p_rcv_desc->grh_flag; + /* copy the GRH data if relevant */ + if (p_mad_addr->addr_type.gsi.global_route) { + p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = + ib_grh_set_ver_class_flow(p_rcv_desc->grh. + IP_version, + p_rcv_desc->grh. + traffic_class, + p_rcv_desc->grh. + flow_label); + p_mad_addr->addr_type.gsi.grh_info.hop_limit = + p_rcv_desc->grh.hop_limit; + memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, + &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); + memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); + } + } } /********************************************************************** * OSM ADDR VECTOR TO IB_MGT **********************************************************************/ void -__osm_mtl_conv_osm_addr_to_ibmgt_addr( - IN osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT IB_ud_av_t *p_av) +__osm_mtl_conv_osm_addr_to_ibmgt_addr(IN osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, OUT IB_ud_av_t * p_av) { - /* For global destination or Multicast address:*/ - u_int8_t ver; - - memset( p_av, 0, sizeof( IB_ud_av_t ) ); - - p_av->src_path_bits = p_mad_addr->path_bits; - p_av->static_rate = p_mad_addr->static_rate; - p_av->dlid = cl_ntoh16(p_mad_addr->dest_lid); - - if (is_smi) - { - p_av->sl = 0; /* Just to note we use 0 here. */ - } - else - { - p_av->sl = p_mad_addr->addr_type.gsi.service_level; - p_av->grh_flag = p_mad_addr->addr_type.gsi.global_route; - - if (p_mad_addr->addr_type.gsi.global_route) - { - ib_grh_get_ver_class_flow(p_mad_addr->addr_type.gsi.grh_info.ver_class_flow, - &ver, - &p_av->traffic_class, - &p_av->flow_label); - p_av->hop_limit = p_mad_addr->addr_type.gsi.grh_info.hop_limit; - p_av->sgid_index = 0; /* we always use source GID 0 */ - memcpy(&p_av->dgid, - &p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - sizeof(ib_net64_t)); - - } - } + /* For global destination or Multicast address: */ + u_int8_t ver; + + memset(p_av, 0, sizeof(IB_ud_av_t)); + + p_av->src_path_bits = p_mad_addr->path_bits; + p_av->static_rate = p_mad_addr->static_rate; + p_av->dlid = cl_ntoh16(p_mad_addr->dest_lid); + + if (is_smi) { + p_av->sl = 0; /* Just to note we use 0 here. */ + } else { + p_av->sl = p_mad_addr->addr_type.gsi.service_level; + p_av->grh_flag = p_mad_addr->addr_type.gsi.global_route; + + if (p_mad_addr->addr_type.gsi.global_route) { + ib_grh_get_ver_class_flow(p_mad_addr->addr_type.gsi. + grh_info.ver_class_flow, &ver, + &p_av->traffic_class, + &p_av->flow_label); + p_av->hop_limit = + p_mad_addr->addr_type.gsi.grh_info.hop_limit; + p_av->sgid_index = 0; /* we always use source GID 0 */ + memcpy(&p_av->dgid, + &p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + sizeof(ib_net64_t)); + + } + } } /********************************************************************** **********************************************************************/ -void -__osm_vendor_clear_sm( IN osm_bind_handle_t h_bind ) +void __osm_vendor_clear_sm(IN osm_bind_handle_t h_bind) { - osm_mtl_bind_info_t *p_bind = ( osm_mtl_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - VAPI_ret_t status; - VAPI_hca_attr_t attr_mod; - VAPI_hca_attr_mask_t attr_mask; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_clear_sm ); - - memset( &attr_mod, 0, sizeof( attr_mod ) ); - memset( &attr_mask, 0, sizeof( attr_mask ) ); - - attr_mod.is_sm = FALSE; - attr_mask = HCA_ATTR_IS_SM; - - status = - VAPI_modify_hca_attr( p_bind->hca_hndl, p_bind->port_num, &attr_mod, - &attr_mask ); - if( status != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_clear_sm: ERR 3C21: " - "Unable set 'IS_SM' bit in port attributes (%d).\n", - status ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_mtl_bind_info_t *p_bind = (osm_mtl_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + VAPI_ret_t status; + VAPI_hca_attr_t attr_mod; + VAPI_hca_attr_mask_t attr_mask; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_clear_sm); + + memset(&attr_mod, 0, sizeof(attr_mod)); + memset(&attr_mask, 0, sizeof(attr_mask)); + + attr_mod.is_sm = FALSE; + attr_mask = HCA_ATTR_IS_SM; + + status = + VAPI_modify_hca_attr(p_bind->hca_hndl, p_bind->port_num, &attr_mod, + &attr_mask); + if (status != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_clear_sm: ERR 3C21: " + "Unable set 'IS_SM' bit in port attributes (%d).\n", + status); + } + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** * ANY CONSTRUCTION OF THE osm_vendor_t OBJECT **********************************************************************/ -void -osm_vendor_construct( IN osm_vendor_t * const p_vend ) +void osm_vendor_construct(IN osm_vendor_t * const p_vend) { - memset( p_vend, 0, sizeof( *p_vend ) ); + memset(p_vend, 0, sizeof(*p_vend)); } /********************************************************************** * DEALOCATE osm_vendor_t **********************************************************************/ -void -osm_vendor_destroy( IN osm_vendor_t * const p_vend ) +void osm_vendor_destroy(IN osm_vendor_t * const p_vend) { - osm_vendor_mgt_bind_t *vendor_mgt_bind_p; - IB_MGT_ret_t mgt_ret; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_destroy ); - - if( p_vend->h_al != NULL ) - { - vendor_mgt_bind_p = ( osm_vendor_mgt_bind_t * ) p_vend->h_al; - if( vendor_mgt_bind_p->gsi_init ) - { - - /* un register the class */ - /* HACK WE ASSUME WE ONLY GOT SA CLASS REGISTERD ON GSI !!! */ - mgt_ret = IB_MGT_unbind_gsi_class(vendor_mgt_bind_p->gsi_mads_hdl, IB_MCLASS_SUBN_ADM); - if ( mgt_ret != IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_destroy: ERR 3C03: " - "Fail to unbind the SA class.\n" ); - } - - /* un bind the handle */ - if( IB_MGT_release_handle( vendor_mgt_bind_p->gsi_mads_hdl ) != - IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_destroy: ERR 3C02: " - "Fail to unbind the SA GSI handle.\n" ); - } - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_destroy: DBG 1002: " - "Unbind the GSI handles.\n" ); - } - if( vendor_mgt_bind_p->smi_init ) - { - /* first - clear the IS_SM in the capability mask */ - __osm_vendor_clear_sm( (osm_bind_handle_t)(vendor_mgt_bind_p->smi_p_bind) ); - - /* un register the class */ - mgt_ret = IB_MGT_unbind_sm(vendor_mgt_bind_p->smi_mads_hdl); - if ( mgt_ret != IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_destroy: ERR 3C04: " - "Fail to unbind the SM class.\n" ); - } - - /* un bind the handle */ - if( IB_MGT_release_handle( vendor_mgt_bind_p->smi_mads_hdl ) != - IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_destroy: ERR 3C05: " - "Fail to unbind the SMI handle.\n" ); - } - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_destroy: DBG 1003: " - "Unbind the SMI handles.\n" ); - - } - } - osm_transaction_mgr_destroy ( p_vend ); - /* __osm_mtl_destroy_tid_mad_map( p_vend ); */ - OSM_LOG_EXIT( p_vend->p_log ); + osm_vendor_mgt_bind_t *vendor_mgt_bind_p; + IB_MGT_ret_t mgt_ret; + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_destroy); + + if (p_vend->h_al != NULL) { + vendor_mgt_bind_p = (osm_vendor_mgt_bind_t *) p_vend->h_al; + if (vendor_mgt_bind_p->gsi_init) { + + /* un register the class */ + /* HACK WE ASSUME WE ONLY GOT SA CLASS REGISTERD ON GSI !!! */ + mgt_ret = + IB_MGT_unbind_gsi_class(vendor_mgt_bind_p-> + gsi_mads_hdl, + IB_MCLASS_SUBN_ADM); + if (mgt_ret != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_destroy: ERR 3C03: " + "Fail to unbind the SA class.\n"); + } + + /* un bind the handle */ + if (IB_MGT_release_handle + (vendor_mgt_bind_p->gsi_mads_hdl) != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_destroy: ERR 3C02: " + "Fail to unbind the SA GSI handle.\n"); + } + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_destroy: DBG 1002: " + "Unbind the GSI handles.\n"); + } + if (vendor_mgt_bind_p->smi_init) { + /* first - clear the IS_SM in the capability mask */ + __osm_vendor_clear_sm((osm_bind_handle_t) + (vendor_mgt_bind_p->smi_p_bind)); + + /* un register the class */ + mgt_ret = + IB_MGT_unbind_sm(vendor_mgt_bind_p->smi_mads_hdl); + if (mgt_ret != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_destroy: ERR 3C04: " + "Fail to unbind the SM class.\n"); + } + + /* un bind the handle */ + if (IB_MGT_release_handle + (vendor_mgt_bind_p->smi_mads_hdl) != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_destroy: ERR 3C05: " + "Fail to unbind the SMI handle.\n"); + } + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_destroy: DBG 1003: " + "Unbind the SMI handles.\n"); + + } + } + osm_transaction_mgr_destroy(p_vend); + /* __osm_mtl_destroy_tid_mad_map( p_vend ); */ + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** DEALLOCATE A POINTER TO osm_vendor_t **********************************************************************/ -void -osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - CL_ASSERT( pp_vend ); + CL_ASSERT(pp_vend); - osm_vendor_destroy( *pp_vend ); - free( *pp_vend ); - *pp_vend = NULL; + osm_vendor_destroy(*pp_vend); + free(*pp_vend); + *pp_vend = NULL; } /********************************************************************** @@ -319,81 +306,71 @@ osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) **********************************************************************/ ib_api_status_t -osm_vendor_init( IN osm_vendor_t * const p_vend, - IN osm_log_t * const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - osm_vendor_mgt_bind_t *ib_mgt_hdl_p; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_vendor_init ); - - p_vend->p_log = p_log; - - /* - * HACK: We need no handle. Assuming the driver is up. - */ - ib_mgt_hdl_p = - ( osm_vendor_mgt_bind_t * ) - malloc( sizeof( osm_vendor_mgt_bind_t ) ); - if( ib_mgt_hdl_p == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_init: ERR 3C06: " - "Fail to allocate vendor mgt handle.\n" ); - goto Exit; - } - - ib_mgt_hdl_p->smi_init = FALSE; - ib_mgt_hdl_p->gsi_init = FALSE; - /* cast it into the ib_al_handle_t h_al */ - p_vend->h_al = ( ib_al_handle_t ) ib_mgt_hdl_p; - p_vend->p_transaction_mgr = NULL; - osm_transaction_mgr_init( p_vend ); - /* p_vend->madw_by_tid_map_p = NULL; */ - /* __osm_mtl_init_tid_mad_map( p_vend ); */ - p_vend->timeout = timeout; - - Exit: - OSM_LOG_EXIT( p_log ); - return ( status ); + osm_vendor_mgt_bind_t *ib_mgt_hdl_p; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_vendor_init); + + p_vend->p_log = p_log; + + /* + * HACK: We need no handle. Assuming the driver is up. + */ + ib_mgt_hdl_p = (osm_vendor_mgt_bind_t *) + malloc(sizeof(osm_vendor_mgt_bind_t)); + if (ib_mgt_hdl_p == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_init: ERR 3C06: " + "Fail to allocate vendor mgt handle.\n"); + goto Exit; + } + + ib_mgt_hdl_p->smi_init = FALSE; + ib_mgt_hdl_p->gsi_init = FALSE; + /* cast it into the ib_al_handle_t h_al */ + p_vend->h_al = (ib_al_handle_t) ib_mgt_hdl_p; + p_vend->p_transaction_mgr = NULL; + osm_transaction_mgr_init(p_vend); + /* p_vend->madw_by_tid_map_p = NULL; */ + /* __osm_mtl_init_tid_mad_map( p_vend ); */ + p_vend->timeout = timeout; + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** * Create and Initialize osm_vendor_t Object **********************************************************************/ -osm_vendor_t * -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - ib_api_status_t status; - osm_vendor_t *p_vend; - - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - CL_ASSERT( p_log ); - - p_vend = malloc( sizeof( *p_vend ) ); - if( p_vend != NULL ) - { - memset( p_vend, 0, sizeof( *p_vend ) ); - status = osm_vendor_init( p_vend, p_log, timeout ); - if( status != IB_SUCCESS ) - { - osm_vendor_delete( &p_vend ); - } - } - else - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 3C07: " - "Fail to allocate vendor object.\n" ); - } - - - OSM_LOG_EXIT( p_log ); - return ( p_vend ); + ib_api_status_t status; + osm_vendor_t *p_vend; + + OSM_LOG_ENTER(p_log, osm_vendor_new); + + CL_ASSERT(p_log); + + p_vend = malloc(sizeof(*p_vend)); + if (p_vend != NULL) { + memset(p_vend, 0, sizeof(*p_vend)); + status = osm_vendor_init(p_vend, p_log, timeout); + if (status != IB_SUCCESS) { + osm_vendor_delete(&p_vend); + } + } else { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 3C07: " + "Fail to allocate vendor object.\n"); + } + + OSM_LOG_EXIT(p_log); + return (p_vend); } /********************************************************************** @@ -401,137 +378,136 @@ osm_vendor_new( * **********************************************************************/ void -__osm_mtl_rcv_callback( IN IB_MGT_mad_hndl_t mad_hndl, - IN void *private_ctx_p, - IN void *payload_p, - IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p ) +__osm_mtl_rcv_callback(IN IB_MGT_mad_hndl_t mad_hndl, + IN void *private_ctx_p, + IN void *payload_p, + IN IB_MGT_mad_rcv_desc_t * rcv_remote_info_p) { - IB_MGT_ret_t status; - osm_mtl_bind_info_t *bind_info_p = private_ctx_p; - osm_madw_t *req_madw_p = NULL; - osm_madw_t *madw_p; - osm_vend_wrap_t* p_new_vw; - osm_mad_addr_t mad_addr; - ib_mad_t *mad_buf_p; - osm_log_t *const p_log = bind_info_p->p_vend->p_log; - - OSM_LOG_ENTER( p_log, __osm_mtl_rcv_callback ); - - - /* if it is a response MAD we mustbe able to get the request */ - if (ib_mad_is_response((ib_mad_t*)payload_p)) - { - /* can we find a matching madw by this payload TID */ - status = - osm_transaction_mgr_get_madw_for_tid( - bind_info_p->p_vend, (ib_mad_t*)payload_p, - &req_madw_p ); - if ( status != IB_MGT_OK ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_rcv_callback: ERR 3C08: " - "Error obtaining request madw by TID (%d).\n", status ); - req_madw_p = NULL; - } - - if (req_madw_p == NULL) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_rcv_callback: ERR 3C09: " - "Fail to obtain request madw for received MAD.(method=%X attr=%X) Aborting CB.\n", - ((ib_mad_t*)payload_p)->method, - cl_ntoh16(((ib_mad_t*)payload_p)->attr_id) - - ); - goto Exit; - } - } - - /* do we have a request ??? */ - if( req_madw_p == NULL ) - { - - /* first arrange an address */ - __osm_mtl_conv_ibmgt_rcv_desc_to_osm_addr( - bind_info_p->p_vend, - rcv_remote_info_p, - ( ((ib_mad_t*)payload_p)->mgmt_class == IB_MCLASS_SUBN_LID ) || - ( ((ib_mad_t*)payload_p)->mgmt_class == IB_MCLASS_SUBN_DIR ), - &mad_addr); - - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_rcv_callback: : " - "Received MAD from QP:%X.\n", - cl_ntoh32(mad_addr.addr_type.gsi.remote_qp) - ); - - /* if not - get new osm_madw and arrange it. */ - /* create the new madw in the pool */ - madw_p = osm_mad_pool_get( bind_info_p->p_osm_pool, - ( osm_bind_handle_t ) bind_info_p, - MAD_BLOCK_SIZE, &mad_addr ); - if( madw_p == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_rcv_callback: ERR 3C10: " - "Error request for a new madw.\n" ); - goto Exit; - } - /* HACK: we cust to avoid the const ??? */ - mad_buf_p = (void *)madw_p->p_mad; - } - else - { - /* we have the madw defined during the send and stored in the vend_wrap */ - /* we need to make sure the wrapper is correctly init there */ - CL_ASSERT(req_madw_p->vend_wrap.p_resp_madw != 0); - madw_p = req_madw_p->vend_wrap.p_resp_madw; - - /* HACK: we do not Support RMPP */ - CL_ASSERT(madw_p->h_bind); - mad_buf_p = osm_vendor_get(madw_p->h_bind, MAD_BLOCK_SIZE, &madw_p->vend_wrap ); - - if( mad_buf_p == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_rcv_callback: ERR 3C11: " - "Unable to acquire wire MAD.\n" ); - - goto Exit; - } - - /* - Finally, attach the wire MAD to this wrapper. - */ - osm_madw_set_mad( madw_p, mad_buf_p ); - - /* also we need to handle the size of the mad since we did not init ... */ - madw_p->mad_size = MAD_BLOCK_SIZE; - } - - - /* init some fields of the vendor wrapper */ - p_new_vw = osm_madw_get_vend_ptr( madw_p ); - p_new_vw->h_bind = bind_info_p; - p_new_vw->size = MAD_BLOCK_SIZE; - p_new_vw->p_resp_madw = NULL; - p_new_vw->mad_buf_p = mad_buf_p; - - /* HACK: We do not support RMPP in receiving MADS */ - memcpy( p_new_vw->mad_buf_p, payload_p, MAD_BLOCK_SIZE ); - - /* attach the buffer to the wrapper */ - madw_p->p_mad = mad_buf_p; - - /* we can also make sure we marked the size and bind on the returned madw */ - madw_p->h_bind = p_new_vw->h_bind; - - /* call the CB */ - ( *bind_info_p->rcv_callback ) ( madw_p, bind_info_p->client_context, - req_madw_p ); - - Exit: - OSM_LOG_EXIT( p_log ); + IB_MGT_ret_t status; + osm_mtl_bind_info_t *bind_info_p = private_ctx_p; + osm_madw_t *req_madw_p = NULL; + osm_madw_t *madw_p; + osm_vend_wrap_t *p_new_vw; + osm_mad_addr_t mad_addr; + ib_mad_t *mad_buf_p; + osm_log_t *const p_log = bind_info_p->p_vend->p_log; + + OSM_LOG_ENTER(p_log, __osm_mtl_rcv_callback); + + /* if it is a response MAD we mustbe able to get the request */ + if (ib_mad_is_response((ib_mad_t *) payload_p)) { + /* can we find a matching madw by this payload TID */ + status = + osm_transaction_mgr_get_madw_for_tid(bind_info_p->p_vend, + (ib_mad_t *) payload_p, + &req_madw_p); + if (status != IB_MGT_OK) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_rcv_callback: ERR 3C08: " + "Error obtaining request madw by TID (%d).\n", + status); + req_madw_p = NULL; + } + + if (req_madw_p == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_rcv_callback: ERR 3C09: " + "Fail to obtain request madw for received MAD.(method=%X attr=%X) Aborting CB.\n", + ((ib_mad_t *) payload_p)->method, + cl_ntoh16(((ib_mad_t *) payload_p)->attr_id) + + ); + goto Exit; + } + } + + /* do we have a request ??? */ + if (req_madw_p == NULL) { + + /* first arrange an address */ + __osm_mtl_conv_ibmgt_rcv_desc_to_osm_addr(bind_info_p->p_vend, + rcv_remote_info_p, + (((ib_mad_t *) + payload_p)-> + mgmt_class == + IB_MCLASS_SUBN_LID) + || + (((ib_mad_t *) + payload_p)-> + mgmt_class == + IB_MCLASS_SUBN_DIR), + &mad_addr); + + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_rcv_callback: : " + "Received MAD from QP:%X.\n", + cl_ntoh32(mad_addr.addr_type.gsi.remote_qp) + ); + + /* if not - get new osm_madw and arrange it. */ + /* create the new madw in the pool */ + madw_p = osm_mad_pool_get(bind_info_p->p_osm_pool, + (osm_bind_handle_t) bind_info_p, + MAD_BLOCK_SIZE, &mad_addr); + if (madw_p == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_rcv_callback: ERR 3C10: " + "Error request for a new madw.\n"); + goto Exit; + } + /* HACK: we cust to avoid the const ??? */ + mad_buf_p = (void *)madw_p->p_mad; + } else { + /* we have the madw defined during the send and stored in the vend_wrap */ + /* we need to make sure the wrapper is correctly init there */ + CL_ASSERT(req_madw_p->vend_wrap.p_resp_madw != 0); + madw_p = req_madw_p->vend_wrap.p_resp_madw; + + /* HACK: we do not Support RMPP */ + CL_ASSERT(madw_p->h_bind); + mad_buf_p = + osm_vendor_get(madw_p->h_bind, MAD_BLOCK_SIZE, + &madw_p->vend_wrap); + + if (mad_buf_p == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_rcv_callback: ERR 3C11: " + "Unable to acquire wire MAD.\n"); + + goto Exit; + } + + /* + Finally, attach the wire MAD to this wrapper. + */ + osm_madw_set_mad(madw_p, mad_buf_p); + + /* also we need to handle the size of the mad since we did not init ... */ + madw_p->mad_size = MAD_BLOCK_SIZE; + } + + /* init some fields of the vendor wrapper */ + p_new_vw = osm_madw_get_vend_ptr(madw_p); + p_new_vw->h_bind = bind_info_p; + p_new_vw->size = MAD_BLOCK_SIZE; + p_new_vw->p_resp_madw = NULL; + p_new_vw->mad_buf_p = mad_buf_p; + + /* HACK: We do not support RMPP in receiving MADS */ + memcpy(p_new_vw->mad_buf_p, payload_p, MAD_BLOCK_SIZE); + + /* attach the buffer to the wrapper */ + madw_p->p_mad = mad_buf_p; + + /* we can also make sure we marked the size and bind on the returned madw */ + madw_p->h_bind = p_new_vw->h_bind; + + /* call the CB */ + (*bind_info_p->rcv_callback) (madw_p, bind_info_p->client_context, + req_madw_p); + + Exit: + OSM_LOG_EXIT(p_log); } /********************************************************************** @@ -539,84 +515,78 @@ __osm_mtl_rcv_callback( IN IB_MGT_mad_hndl_t mad_hndl, * **********************************************************************/ void -__osm_mtl_send_callback( IN IB_MGT_mad_hndl_t mad_hndl, - IN u_int64_t wrid, - IN IB_comp_status_t status, - IN void *private_ctx_p ) +__osm_mtl_send_callback(IN IB_MGT_mad_hndl_t mad_hndl, + IN u_int64_t wrid, + IN IB_comp_status_t status, IN void *private_ctx_p) { - osm_madw_t *madw_p; - osm_mtl_bind_info_t *bind_info_p = ( osm_mtl_bind_info_t * ) private_ctx_p; - osm_log_t *const p_log = bind_info_p->p_vend->p_log; - osm_vend_wrap_t* p_vw; - uint8_t is_resp; - - OSM_LOG_ENTER( p_log, __osm_mtl_send_callback ); - - /* obtain the madp from the wrid */ - __osm_set_p_madw_and_resp_by_wrid(wrid, &is_resp, &madw_p); - - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_mtl_send_callback: INFO 1008: " - "Handling Send of MADW:%p Is Resp:%d.\n", madw_p, is_resp); - - /* we need to handle requests and responses differently */ - if (is_resp) - { - if (status != IB_COMP_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_send_callback: ERR 3C12: " - "Error Sending Response MADW:%p.\n", madw_p); - } - else - { - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_mtl_send_callback: DBG 1008: " - "Completed Sending Response MADW:%p.\n", madw_p); - } - - /* if we are a response - we need to clean it up */ - osm_mad_pool_put( bind_info_p->p_osm_pool, madw_p); - } - else - { - - /* this call back is invoked on completion of send - error or not */ - if (status != IB_COMP_SUCCESS) - { - - osm_log( p_log, OSM_LOG_ERROR, - "__osm_mtl_send_callback: ERR 3C13: " - "Received an Error from IB_MGT Send (%d).\n",status ); - - p_vw = osm_madw_get_vend_ptr( madw_p ); - CL_ASSERT( p_vw ); - - /* - Return any wrappers to the pool that may have been - pre-emptively allocated to handle a receive. - */ - if( p_vw->p_resp_madw ) - { - osm_mad_pool_put( bind_info_p->p_osm_pool, p_vw->p_resp_madw ); - p_vw->p_resp_madw = NULL; - } - - /* invoke the CB */ - ( *bind_info_p->send_err_callback ) ( bind_info_p->client_context, - madw_p ); - } - else - { - /* successful request send - do nothing - the response will need the - out mad */ - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_mtl_send_callback: DBG 1008: " - "Completed Sending Request MADW:%p.\n", madw_p); - } - } - - OSM_LOG_EXIT( p_log ); + osm_madw_t *madw_p; + osm_mtl_bind_info_t *bind_info_p = + (osm_mtl_bind_info_t *) private_ctx_p; + osm_log_t *const p_log = bind_info_p->p_vend->p_log; + osm_vend_wrap_t *p_vw; + uint8_t is_resp; + + OSM_LOG_ENTER(p_log, __osm_mtl_send_callback); + + /* obtain the madp from the wrid */ + __osm_set_p_madw_and_resp_by_wrid(wrid, &is_resp, &madw_p); + + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_mtl_send_callback: INFO 1008: " + "Handling Send of MADW:%p Is Resp:%d.\n", madw_p, is_resp); + + /* we need to handle requests and responses differently */ + if (is_resp) { + if (status != IB_COMP_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_send_callback: ERR 3C12: " + "Error Sending Response MADW:%p.\n", madw_p); + } else { + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_mtl_send_callback: DBG 1008: " + "Completed Sending Response MADW:%p.\n", + madw_p); + } + + /* if we are a response - we need to clean it up */ + osm_mad_pool_put(bind_info_p->p_osm_pool, madw_p); + } else { + + /* this call back is invoked on completion of send - error or not */ + if (status != IB_COMP_SUCCESS) { + + osm_log(p_log, OSM_LOG_ERROR, + "__osm_mtl_send_callback: ERR 3C13: " + "Received an Error from IB_MGT Send (%d).\n", + status); + + p_vw = osm_madw_get_vend_ptr(madw_p); + CL_ASSERT(p_vw); + + /* + Return any wrappers to the pool that may have been + pre-emptively allocated to handle a receive. + */ + if (p_vw->p_resp_madw) { + osm_mad_pool_put(bind_info_p->p_osm_pool, + p_vw->p_resp_madw); + p_vw->p_resp_madw = NULL; + } + + /* invoke the CB */ + (*bind_info_p->send_err_callback) (bind_info_p-> + client_context, + madw_p); + } else { + /* successful request send - do nothing - the response will need the + out mad */ + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_mtl_send_callback: DBG 1008: " + "Completed Sending Request MADW:%p.\n", madw_p); + } + } + + OSM_LOG_EXIT(p_log); } /********************************************************************** @@ -624,318 +594,301 @@ __osm_mtl_send_callback( IN IB_MGT_mad_hndl_t mad_hndl, * defined by the given: osm_bind_info_t **********************************************************************/ osm_bind_handle_t -osm_vendor_bind( IN osm_vendor_t * const p_vend, - IN osm_bind_info_t * const p_user_bind, - IN osm_mad_pool_t * const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void *context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_user_bind, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { - ib_net64_t port_guid; - osm_mtl_bind_info_t *p_bind = NULL; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_id_t hca_id; - IB_MGT_mad_type_t mad_type; - uint32_t port_num; - osm_vendor_mgt_bind_t *ib_mgt_hdl_p; - IB_MGT_ret_t mgt_ret; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_bind ); - - CL_ASSERT( p_user_bind ); - CL_ASSERT( p_mad_pool ); - CL_ASSERT( mad_recv_callback ); - CL_ASSERT( send_err_callback ); - - /* cast back the AL handle to vendor mgt bind */ - ib_mgt_hdl_p = ( osm_vendor_mgt_bind_t * ) p_vend->h_al; - - port_guid = p_user_bind->port_guid; - - osm_log( p_vend->p_log, OSM_LOG_INFO, - "osm_vendor_bind: " - "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64( port_guid ) ); - - /* obtain the hca name and port num from the guid */ - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", - port_guid ); - - mgt_ret = - osm_vendor_get_guid_ca_and_port( p_vend, port_guid, &hca_hndl, - &hca_id, &port_num ); - if( mgt_ret != IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C14: " - "Unable to obtain CA and port (%d).\n" ); - goto Exit; - } - - /* create the bind object tracking this binding */ - p_bind = (osm_mtl_bind_info_t *)malloc( sizeof(osm_mtl_bind_info_t) ); - memset(p_bind, 0, sizeof(osm_mtl_bind_info_t)); - if( p_bind == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C15: " - "Unable to allocate internal bind object.\n" ); - goto Exit; - } - - /* track this bind request info */ - memcpy( p_bind->hca_id, hca_id, sizeof( VAPI_hca_id_t ) ); - p_bind->port_num = port_num; - p_bind->p_vend = p_vend; - p_bind->client_context = context; - p_bind->rcv_callback = mad_recv_callback; - p_bind->send_err_callback = send_err_callback; - p_bind->p_osm_pool = p_mad_pool; - - CL_ASSERT( p_bind->port_num ); - - /* - * Get the proper CLASS - */ - - switch ( p_user_bind->mad_class ) - { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - mad_type = IB_MGT_SMI; - break; - - case IB_MCLASS_SUBN_ADM: - default: - mad_type = IB_MGT_GSI; - break; - } - - /* we split here - based on the type of MADS GSI / SMI */ - /* HACK: we only support one class registration per SMI/GSI !!! */ - if ( mad_type == IB_MGT_SMI ) - { - /* - * SMI CASE - */ - - /* we do not need to bind the handle if already available */ - if ( ib_mgt_hdl_p->smi_init == FALSE) - { - - /* First we have to reg and get the handle for the mad */ - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: " "Binding to IB_MGT SMI of %s port %u\n", - hca_id, port_num - ); - - mgt_ret = - IB_MGT_get_handle( hca_id, port_num, IB_MGT_SMI, - &( ib_mgt_hdl_p->smi_mads_hdl ) ); - if( IB_MGT_OK != mgt_ret ) - { - free( p_bind ); - p_bind = NULL; - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C16: " - "Error obtaining IB_MGT handle to SMI.\n" ); - goto Exit; - } - - /* bind it */ - mgt_ret = IB_MGT_bind_sm( ib_mgt_hdl_p->smi_mads_hdl ); - if( IB_MGT_OK != mgt_ret ) - { - free( p_bind ); - p_bind = NULL; - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C17: " - "Error binding IB_MGT handle to SM.\n" ); - goto Exit; - } - - ib_mgt_hdl_p->smi_init = TRUE; - - } - - /* attach to this bind info */ - p_bind->mad_hndl = ib_mgt_hdl_p->smi_mads_hdl; - ib_mgt_hdl_p->smi_p_bind = p_bind; - - /* now register the callback */ - mgt_ret = IB_MGT_reg_cb( p_bind->mad_hndl, - &__osm_mtl_rcv_callback, - p_bind, - &__osm_mtl_send_callback, - p_bind, - IB_MGT_RCV_CB_MASK | IB_MGT_SEND_CB_MASK ); - - } - else - { - /* - * GSI CASE - */ - - if ( ib_mgt_hdl_p->gsi_init == FALSE) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: " "Binding to IB_MGT GSI\n"); - - /* First we have to reg and get the handle for the mad */ - mgt_ret = - IB_MGT_get_handle( hca_id, port_num, IB_MGT_GSI, - &( ib_mgt_hdl_p->gsi_mads_hdl ) ); - if( IB_MGT_OK != mgt_ret ) - { - free( p_bind ); - p_bind = NULL; - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C20: " - "Error obtaining IB_MGT handle to GSI.\n" ); - goto Exit; - } - - /* bind it */ - mgt_ret = - IB_MGT_bind_gsi_class( ib_mgt_hdl_p->gsi_mads_hdl, - p_user_bind->mad_class ); - if( IB_MGT_OK != mgt_ret ) - { - free( p_bind ); - p_bind = NULL; - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C22: " - "Error binding IB_MGT handle to GSI.\n" ); - goto Exit; - } - - ib_mgt_hdl_p->gsi_init = TRUE; - - /* attach to this bind info */ - p_bind->mad_hndl = ib_mgt_hdl_p->gsi_mads_hdl; - - /* now register the callback */ - mgt_ret = IB_MGT_reg_cb( p_bind->mad_hndl, - &__osm_mtl_rcv_callback, - p_bind, - &__osm_mtl_send_callback, - p_bind, - IB_MGT_RCV_CB_MASK | IB_MGT_SEND_CB_MASK ); - - } - else - { - /* we can use the existing handle */ - p_bind->mad_hndl = ib_mgt_hdl_p->gsi_mads_hdl; - mgt_ret = IB_MGT_OK ; - } - - } - - if( IB_MGT_OK != mgt_ret ) - { - free( p_bind ); - p_bind = NULL; - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 3C23: " - "Error binding IB_MGT CB (%d).\n", mgt_ret); - goto Exit; - } - - /* HACK: Do we need to initialize an address vector ???? */ - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( ( osm_bind_handle_t ) p_bind ); + ib_net64_t port_guid; + osm_mtl_bind_info_t *p_bind = NULL; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_id_t hca_id; + IB_MGT_mad_type_t mad_type; + uint32_t port_num; + osm_vendor_mgt_bind_t *ib_mgt_hdl_p; + IB_MGT_ret_t mgt_ret; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_bind); + + CL_ASSERT(p_user_bind); + CL_ASSERT(p_mad_pool); + CL_ASSERT(mad_recv_callback); + CL_ASSERT(send_err_callback); + + /* cast back the AL handle to vendor mgt bind */ + ib_mgt_hdl_p = (osm_vendor_mgt_bind_t *) p_vend->h_al; + + port_guid = p_user_bind->port_guid; + + osm_log(p_vend->p_log, OSM_LOG_INFO, + "osm_vendor_bind: " + "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64(port_guid)); + + /* obtain the hca name and port num from the guid */ + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", + port_guid); + + mgt_ret = + osm_vendor_get_guid_ca_and_port(p_vend, port_guid, &hca_hndl, + &hca_id, &port_num); + if (mgt_ret != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C14: " + "Unable to obtain CA and port (%d).\n"); + goto Exit; + } + + /* create the bind object tracking this binding */ + p_bind = (osm_mtl_bind_info_t *) malloc(sizeof(osm_mtl_bind_info_t)); + memset(p_bind, 0, sizeof(osm_mtl_bind_info_t)); + if (p_bind == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C15: " + "Unable to allocate internal bind object.\n"); + goto Exit; + } + + /* track this bind request info */ + memcpy(p_bind->hca_id, hca_id, sizeof(VAPI_hca_id_t)); + p_bind->port_num = port_num; + p_bind->p_vend = p_vend; + p_bind->client_context = context; + p_bind->rcv_callback = mad_recv_callback; + p_bind->send_err_callback = send_err_callback; + p_bind->p_osm_pool = p_mad_pool; + + CL_ASSERT(p_bind->port_num); + + /* + * Get the proper CLASS + */ + + switch (p_user_bind->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + mad_type = IB_MGT_SMI; + break; + + case IB_MCLASS_SUBN_ADM: + default: + mad_type = IB_MGT_GSI; + break; + } + + /* we split here - based on the type of MADS GSI / SMI */ + /* HACK: we only support one class registration per SMI/GSI !!! */ + if (mad_type == IB_MGT_SMI) { + /* + * SMI CASE + */ + + /* we do not need to bind the handle if already available */ + if (ib_mgt_hdl_p->smi_init == FALSE) { + + /* First we have to reg and get the handle for the mad */ + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: " + "Binding to IB_MGT SMI of %s port %u\n", hca_id, + port_num); + + mgt_ret = + IB_MGT_get_handle(hca_id, port_num, IB_MGT_SMI, + &(ib_mgt_hdl_p->smi_mads_hdl)); + if (IB_MGT_OK != mgt_ret) { + free(p_bind); + p_bind = NULL; + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C16: " + "Error obtaining IB_MGT handle to SMI.\n"); + goto Exit; + } + + /* bind it */ + mgt_ret = IB_MGT_bind_sm(ib_mgt_hdl_p->smi_mads_hdl); + if (IB_MGT_OK != mgt_ret) { + free(p_bind); + p_bind = NULL; + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C17: " + "Error binding IB_MGT handle to SM.\n"); + goto Exit; + } + + ib_mgt_hdl_p->smi_init = TRUE; + + } + + /* attach to this bind info */ + p_bind->mad_hndl = ib_mgt_hdl_p->smi_mads_hdl; + ib_mgt_hdl_p->smi_p_bind = p_bind; + + /* now register the callback */ + mgt_ret = IB_MGT_reg_cb(p_bind->mad_hndl, + &__osm_mtl_rcv_callback, + p_bind, + &__osm_mtl_send_callback, + p_bind, + IB_MGT_RCV_CB_MASK | + IB_MGT_SEND_CB_MASK); + + } else { + /* + * GSI CASE + */ + + if (ib_mgt_hdl_p->gsi_init == FALSE) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: " "Binding to IB_MGT GSI\n"); + + /* First we have to reg and get the handle for the mad */ + mgt_ret = + IB_MGT_get_handle(hca_id, port_num, IB_MGT_GSI, + &(ib_mgt_hdl_p->gsi_mads_hdl)); + if (IB_MGT_OK != mgt_ret) { + free(p_bind); + p_bind = NULL; + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C20: " + "Error obtaining IB_MGT handle to GSI.\n"); + goto Exit; + } + + /* bind it */ + mgt_ret = + IB_MGT_bind_gsi_class(ib_mgt_hdl_p->gsi_mads_hdl, + p_user_bind->mad_class); + if (IB_MGT_OK != mgt_ret) { + free(p_bind); + p_bind = NULL; + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C22: " + "Error binding IB_MGT handle to GSI.\n"); + goto Exit; + } + + ib_mgt_hdl_p->gsi_init = TRUE; + + /* attach to this bind info */ + p_bind->mad_hndl = ib_mgt_hdl_p->gsi_mads_hdl; + + /* now register the callback */ + mgt_ret = IB_MGT_reg_cb(p_bind->mad_hndl, + &__osm_mtl_rcv_callback, + p_bind, + &__osm_mtl_send_callback, + p_bind, + IB_MGT_RCV_CB_MASK | + IB_MGT_SEND_CB_MASK); + + } else { + /* we can use the existing handle */ + p_bind->mad_hndl = ib_mgt_hdl_p->gsi_mads_hdl; + mgt_ret = IB_MGT_OK; + } + + } + + if (IB_MGT_OK != mgt_ret) { + free(p_bind); + p_bind = NULL; + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 3C23: " + "Error binding IB_MGT CB (%d).\n", mgt_ret); + goto Exit; + } + + /* HACK: Do we need to initialize an address vector ???? */ + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return ((osm_bind_handle_t) p_bind); } /********************************************************************** Get a mad from the lower level. The osm_vend_wrap_t is a wrapper used to connect the mad to the response. **********************************************************************/ -ib_mad_t * -osm_vendor_get( IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t * const p_vw ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { - ib_mad_t *mad_p; - osm_mtl_bind_info_t *p_bind = ( osm_mtl_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get ); - - CL_ASSERT( p_vw ); - /* HACK: We know we can not send through IB_MGT */ - CL_ASSERT( mad_size <= MAD_BLOCK_SIZE ); - - /* IB_MGT assumes it is 256 - we must follow */ - p_vw->size = MAD_BLOCK_SIZE; - - /* allocate it */ - mad_p = ( ib_mad_t * ) malloc( p_vw->size ); - if( mad_p == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get: ERR 3C24: " - "Error Obtaining MAD buffer.\n" ); - goto Exit; - } - - memset( mad_p, 0, p_vw->size ); - - /* track locally */ - p_vw->mad_buf_p = mad_p; - p_vw->h_bind = h_bind; - p_vw->p_resp_madw = NULL; - - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get: " - "Acquired MAD %p, size = %u.\n", mad_p, p_vw->size ); - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( mad_p ); + ib_mad_t *mad_p; + osm_mtl_bind_info_t *p_bind = (osm_mtl_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); + + CL_ASSERT(p_vw); + /* HACK: We know we can not send through IB_MGT */ + CL_ASSERT(mad_size <= MAD_BLOCK_SIZE); + + /* IB_MGT assumes it is 256 - we must follow */ + p_vw->size = MAD_BLOCK_SIZE; + + /* allocate it */ + mad_p = (ib_mad_t *) malloc(p_vw->size); + if (mad_p == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get: ERR 3C24: " + "Error Obtaining MAD buffer.\n"); + goto Exit; + } + + memset(mad_p, 0, p_vw->size); + + /* track locally */ + p_vw->mad_buf_p = mad_p; + p_vw->h_bind = h_bind; + p_vw->p_resp_madw = NULL; + + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get: " + "Acquired MAD %p, size = %u.\n", mad_p, p_vw->size); + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (mad_p); } /********************************************************************** * Return a MAD by providing it's wrapper object. **********************************************************************/ void -osm_vendor_put( IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t * const p_vw ) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { - osm_mtl_bind_info_t *p_bind = ( osm_mtl_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - osm_madw_t *p_madw; + osm_mtl_bind_info_t *p_bind = (osm_mtl_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + osm_madw_t *p_madw; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_put ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); - CL_ASSERT( p_vw ); - CL_ASSERT( p_vw->mad_buf_p ); + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->mad_buf_p); - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_put: " "Retiring MAD %p.\n", p_vw->mad_buf_p ); - } + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_put: " "Retiring MAD %p.\n", + p_vw->mad_buf_p); + } - /* - * We moved the removal of the transaction to immediatly after - * it was looked up. - */ + /* + * We moved the removal of the transaction to immediatly after + * it was looked up. + */ - /* free the mad but the wrapper is part of the madw object */ - free( p_vw->mad_buf_p ); - p_vw->mad_buf_p = NULL; - p_madw = PARENT_STRUCT( p_vw, osm_madw_t, vend_wrap); - p_madw->p_mad = NULL; + /* free the mad but the wrapper is part of the madw object */ + free(p_vw->mad_buf_p); + p_vw->mad_buf_p = NULL; + p_madw = PARENT_STRUCT(p_vw, osm_madw_t, vend_wrap); + p_madw->p_mad = NULL; - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -945,109 +898,96 @@ This is for internal use by osm_vendor_send and the transaction mgr retry too. **********************************************************************/ ib_api_status_t -osm_mtl_send_mad( - IN osm_mtl_bind_info_t *p_bind, - IN osm_madw_t * const p_madw) +osm_mtl_send_mad(IN osm_mtl_bind_info_t * p_bind, IN osm_madw_t * const p_madw) { - osm_vendor_t *const p_vend = p_bind->p_vend; - osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr( p_madw ); - osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr( p_madw ); - ib_mad_t *const p_mad = osm_madw_get_mad_ptr( p_madw ); - ib_api_status_t status; - IB_MGT_ret_t mgt_res; - IB_ud_av_t av; - uint64_t wrid; - uint32_t qpn; - - OSM_LOG_ENTER( p_vend->p_log, osm_mtl_send_mad ); - - /* - * For all sends other than directed route SM MADs, - * acquire an address vector for the destination. - */ - if( p_mad->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - __osm_mtl_conv_osm_addr_to_ibmgt_addr(p_mad_addr, - p_mad->mgmt_class == IB_MCLASS_SUBN_LID, - &av); - } - else - { - /* is a directed route - we need to construct a permissive address */ - memset( &av, 0, sizeof( av ) ); - /* we do not need port number since it is part of the mad_hndl */ - av.dlid = IB_LID_PERMISSIVE; - } - - wrid = __osm_set_wrid_by_p_madw(p_madw); - - /* send it */ - if( (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || - (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { - - /* SMI CASE */ - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_mtl_send_mad: " - "av.dlid 0x%X, " - "av.static_rate %d, " - "av.path_bits %d.\n", - cl_ntoh16( av.dlid ), av.static_rate, av.src_path_bits ); - } - - mgt_res = IB_MGT_send_mad( p_bind->mad_hndl, p_mad, /* actual payload */ - &av, /* address vector */ - wrid, /* casting the mad wrapper pointer for err cb */ - p_vend->timeout ); - - } - else - { - /* GSI CASE - Support Remote QP */ - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_mtl_send_mad: " - "av.dlid 0x%X, av.static_rate %d, " - "av.path_bits %d, remote qp: 0x%06X \n", - av.dlid, - av.static_rate, - av.src_path_bits, - cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp) - ); - } - - /* IBMGT have a bug sending to a QP not 1 - - the QPN must be in network order except when it qpn 1 ... */ - qpn = cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp); - - mgt_res = - IB_MGT_send_mad_to_qp( - p_bind->mad_hndl, p_mad, /* actual payload */ - &av, /* address vector */ - wrid, /* casting the mad wrapper pointer for err cb */ - p_vend->timeout, - qpn - ); - } - - if( mgt_res != IB_MGT_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_mtl_send_mad: ERR 3C26: " - "Error sending mad (%d).\n", mgt_res ); - if( p_vw->p_resp_madw ) - osm_mad_pool_put( p_bind->p_osm_pool, p_vw->p_resp_madw ); - status = IB_ERROR; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return(status); + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + ib_mad_t *const p_mad = osm_madw_get_mad_ptr(p_madw); + ib_api_status_t status; + IB_MGT_ret_t mgt_res; + IB_ud_av_t av; + uint64_t wrid; + uint32_t qpn; + + OSM_LOG_ENTER(p_vend->p_log, osm_mtl_send_mad); + + /* + * For all sends other than directed route SM MADs, + * acquire an address vector for the destination. + */ + if (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR) { + __osm_mtl_conv_osm_addr_to_ibmgt_addr(p_mad_addr, + p_mad->mgmt_class == + IB_MCLASS_SUBN_LID, &av); + } else { + /* is a directed route - we need to construct a permissive address */ + memset(&av, 0, sizeof(av)); + /* we do not need port number since it is part of the mad_hndl */ + av.dlid = IB_LID_PERMISSIVE; + } + + wrid = __osm_set_wrid_by_p_madw(p_madw); + + /* send it */ + if ((p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || + (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { + + /* SMI CASE */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_mtl_send_mad: " + "av.dlid 0x%X, " + "av.static_rate %d, " + "av.path_bits %d.\n", + cl_ntoh16(av.dlid), av.static_rate, + av.src_path_bits); + } + + mgt_res = IB_MGT_send_mad(p_bind->mad_hndl, p_mad, /* actual payload */ + &av, /* address vector */ + wrid, /* casting the mad wrapper pointer for err cb */ + p_vend->timeout); + + } else { + /* GSI CASE - Support Remote QP */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_mtl_send_mad: " + "av.dlid 0x%X, av.static_rate %d, " + "av.path_bits %d, remote qp: 0x%06X \n", + av.dlid, + av.static_rate, + av.src_path_bits, + cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp) + ); + } + + /* IBMGT have a bug sending to a QP not 1 - + the QPN must be in network order except when it qpn 1 ... */ + qpn = cl_ntoh32(p_mad_addr->addr_type.gsi.remote_qp); + + mgt_res = IB_MGT_send_mad_to_qp(p_bind->mad_hndl, p_mad, /* actual payload */ + &av, /* address vector */ + wrid, /* casting the mad wrapper pointer for err cb */ + p_vend->timeout, qpn); + } + + if (mgt_res != IB_MGT_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_mtl_send_mad: ERR 3C26: " + "Error sending mad (%d).\n", mgt_res); + if (p_vw->p_resp_madw) + osm_mad_pool_put(p_bind->p_osm_pool, p_vw->p_resp_madw); + status = IB_ERROR; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -1057,125 +997,112 @@ What is unclear to me is the need for the setting of all the MAD Wrapper fields. Seems like the OSM uses these values during it's processing... **********************************************************************/ ib_api_status_t -osm_vendor_send( IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { - osm_mtl_bind_info_t *const p_bind = (osm_mtl_bind_info_t *)h_bind; - osm_vendor_t *const p_vend = p_bind->p_vend; - osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr( p_madw ); - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_send ); - - /* - * If a response is expected to this MAD, then preallocate - * a mad wrapper to contain the wire MAD received in the - * response. Allocating a wrapper here allows for easier - * failure paths than after we already received the wire mad. - */ - if( resp_expected == TRUE ) - { - /* we track it in the vendor wrapper */ - p_vw->p_resp_madw = - osm_mad_pool_get_wrapper_raw( p_bind->p_osm_pool ); - if( p_vw->p_resp_madw == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 3C27: " - "Unable to allocate MAD wrapper.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* put some minimal info on that wrapper */ - ((osm_madw_t *)(p_vw->p_resp_madw))->h_bind = h_bind; - - /* we also want to track it in the TID based map */ - status = osm_transaction_mgr_insert_madw( - (osm_bind_handle_t)p_bind, p_madw ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 3C25: " - "Error inserting request madw by TID (%d).\n", status ); - } - - } - else - p_vw->p_resp_madw = NULL; - - /* do the actual send */ - status = osm_mtl_send_mad(p_bind, p_madw); - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + osm_mtl_bind_info_t *const p_bind = (osm_mtl_bind_info_t *) h_bind; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_send); + + /* + * If a response is expected to this MAD, then preallocate + * a mad wrapper to contain the wire MAD received in the + * response. Allocating a wrapper here allows for easier + * failure paths than after we already received the wire mad. + */ + if (resp_expected == TRUE) { + /* we track it in the vendor wrapper */ + p_vw->p_resp_madw = + osm_mad_pool_get_wrapper_raw(p_bind->p_osm_pool); + if (p_vw->p_resp_madw == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 3C27: " + "Unable to allocate MAD wrapper.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* put some minimal info on that wrapper */ + ((osm_madw_t *) (p_vw->p_resp_madw))->h_bind = h_bind; + + /* we also want to track it in the TID based map */ + status = osm_transaction_mgr_insert_madw((osm_bind_handle_t) + p_bind, p_madw); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 3C25: " + "Error inserting request madw by TID (%d).\n", + status); + } + + } else + p_vw->p_resp_madw = NULL; + + /* do the actual send */ + status = osm_mtl_send_mad(p_bind, p_madw); + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } - /********************************************************************** * the idea here is to change the content of the bind such that it * will hold the local address used for sending directed route by the SMA. **********************************************************************/ -ib_api_status_t -osm_vendor_local_lid_change( IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_vendor_t *p_vend = ( ( osm_mtl_bind_info_t * ) h_bind )->p_vend; + osm_vendor_t *p_vend = ((osm_mtl_bind_info_t *) h_bind)->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_local_lid_change: DEBUG 2202: " "Change of LID.\n" ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_local_lid_change: DEBUG 2202: " "Change of LID.\n"); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); - return ( IB_SUCCESS ); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osm_mtl_bind_info_t *p_bind = ( osm_mtl_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - VAPI_ret_t status; - VAPI_hca_attr_t attr_mod; - VAPI_hca_attr_mask_t attr_mask; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); - - memset( &attr_mod, 0, sizeof( attr_mod ) ); - memset( &attr_mask, 0, sizeof( attr_mask ) ); - - attr_mod.is_sm = is_sm_val; - attr_mask = HCA_ATTR_IS_SM; - - status = - VAPI_modify_hca_attr( p_bind->hca_hndl, p_bind->port_num, &attr_mod, - &attr_mask ); - if( status != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 3C28: " - "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", - is_sm_val, status ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_mtl_bind_info_t *p_bind = (osm_mtl_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + VAPI_ret_t status; + VAPI_hca_attr_t attr_mod; + VAPI_hca_attr_mask_t attr_mask; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); + + memset(&attr_mod, 0, sizeof(attr_mod)); + memset(&attr_mask, 0, sizeof(attr_mask)); + + attr_mod.is_sm = is_sm_val; + attr_mask = HCA_ATTR_IS_SM; + + status = + VAPI_modify_hca_attr(p_bind->hca_hndl, p_bind->port_num, &attr_mod, + &attr_mask); + if (status != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 3C28: " + "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", + is_sm_val, status); + } + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } -#endif /* OSM_VENDOR_INTF_TEST */ +#endif /* OSM_VENDOR_INTF_TEST */ diff --git a/opensm/libvendor/osm_vendor_mtl_hca_guid.c b/opensm/libvendor/osm_vendor_mtl_hca_guid.c index 6f329ac..8ef4805 100644 --- a/opensm/libvendor/osm_vendor_mtl_hca_guid.c +++ b/opensm/libvendor/osm_vendor_mtl_hca_guid.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #if defined(OSM_VENDOR_INTF_MTL) | defined(OSM_VENDOR_INTF_TS) #undef IN @@ -61,11 +60,10 @@ **********************************************************************/ ib_api_status_t -__osm_vendor_gid_to_guid( IN u_int8_t * gid, - OUT VAPI_gid_t * guid ) +__osm_vendor_gid_to_guid(IN u_int8_t * gid, OUT VAPI_gid_t * guid) { - memcpy( guid, gid + 8, 8 ); - return ( IB_SUCCESS ); + memcpy(guid, gid + 8, 8); + return (IB_SUCCESS); } /****f* OpenSM: CA Info/osm_ca_info_get_pi_ptr @@ -78,11 +76,11 @@ __osm_vendor_gid_to_guid( IN u_int8_t * gid, * * SYNOPSIS */ -static ib_port_attr_t * -__osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, - IN const uint8_t index ) +static ib_port_attr_t *__osm_ca_info_get_port_attr_ptr(IN const osm_ca_info_t * + const p_ca_info, + IN const uint8_t index) { - return ( &p_ca_info->p_attr->p_port_attr[index] ); + return (&p_ca_info->p_attr->p_port_attr[index]); } /* @@ -109,213 +107,199 @@ __osm_ca_info_get_port_attr_ptr( IN const osm_ca_info_t * const p_ca_info, * NOTE: user of this function needs to deallocate p_hca_ids after usage. ********************************************************************************/ static ib_api_status_t -__osm_vendor_get_ca_ids( IN osm_vendor_t * const p_vend, - IN VAPI_hca_id_t ** const p_hca_ids, - IN uint32_t * const p_num_guids ) +__osm_vendor_get_ca_ids(IN osm_vendor_t * const p_vend, + IN VAPI_hca_id_t ** const p_hca_ids, + IN uint32_t * const p_num_guids) { - ib_api_status_t status; - VAPI_ret_t vapi_res; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_get_ca_ids ); - - CL_ASSERT( p_hca_ids ); - CL_ASSERT( p_num_guids ); - - /* first call is just to get the number */ - vapi_res = EVAPI_list_hcas( 0, p_num_guids, NULL ); - - /* fail ? */ - if( vapi_res == VAPI_EINVAL_PARAM ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 7101: " - "Bad parameter in calling: EVAPI_list_hcas. (%d)\n", - vapi_res ); - status = IB_ERROR; - goto Exit; - } - - /* NO HCA ? */ - if( *p_num_guids == 0 ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 7102: " - "No available channel adapters.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* allocate and really call - user of this function needs to deallocate it */ - *p_hca_ids = - ( VAPI_hca_id_t * ) malloc( *p_num_guids * - sizeof( VAPI_hca_id_t ) ); - - /* now call it really */ - vapi_res = EVAPI_list_hcas( *p_num_guids, p_num_guids, *p_hca_ids ); - - /* too many ? */ - if( vapi_res == VAPI_EAGAIN ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 7103: " - "More CA GUIDs than allocated array (%d).\n", *p_num_guids ); - status = IB_ERROR; - goto Exit; - } - - /* fail ? */ - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_get_ca_ids: ERR 7104: " - "Bad parameter in calling: EVAPI_list_hcas.\n" ); - status = IB_ERROR; - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_vendor_get_ca_ids: " - "Detected %u local channel adapters.\n", *p_num_guids ); - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + VAPI_ret_t vapi_res; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_get_ca_ids); + + CL_ASSERT(p_hca_ids); + CL_ASSERT(p_num_guids); + + /* first call is just to get the number */ + vapi_res = EVAPI_list_hcas(0, p_num_guids, NULL); + + /* fail ? */ + if (vapi_res == VAPI_EINVAL_PARAM) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 7101: " + "Bad parameter in calling: EVAPI_list_hcas. (%d)\n", + vapi_res); + status = IB_ERROR; + goto Exit; + } + + /* NO HCA ? */ + if (*p_num_guids == 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 7102: " + "No available channel adapters.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* allocate and really call - user of this function needs to deallocate it */ + *p_hca_ids = + (VAPI_hca_id_t *) malloc(*p_num_guids * sizeof(VAPI_hca_id_t)); + + /* now call it really */ + vapi_res = EVAPI_list_hcas(*p_num_guids, p_num_guids, *p_hca_ids); + + /* too many ? */ + if (vapi_res == VAPI_EAGAIN) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 7103: " + "More CA GUIDs than allocated array (%d).\n", + *p_num_guids); + status = IB_ERROR; + goto Exit; + } + + /* fail ? */ + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_get_ca_ids: ERR 7104: " + "Bad parameter in calling: EVAPI_list_hcas.\n"); + status = IB_ERROR; + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_vendor_get_ca_ids: " + "Detected %u local channel adapters.\n", *p_num_guids); + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } - /********************************************************************** * Initialize an Info Struct for the Given HCA by its Id **********************************************************************/ static ib_api_status_t -__osm_ca_info_init( IN osm_vendor_t * const p_vend, - IN VAPI_hca_id_t ca_id, - OUT osm_ca_info_t * const p_ca_info ) +__osm_ca_info_init(IN osm_vendor_t * const p_vend, + IN VAPI_hca_id_t ca_id, OUT osm_ca_info_t * const p_ca_info) { - ib_api_status_t status = IB_ERROR; - VAPI_ret_t vapi_res; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_vendor_t hca_vendor; - VAPI_hca_cap_t hca_cap; - VAPI_hca_port_t hca_port; - uint8_t port_num; - IB_gid_t *p_port_gid; - uint16_t maxNumGids; - - OSM_LOG_ENTER( p_vend->p_log, __osm_ca_info_init ); - - /* get the HCA handle */ - vapi_res = EVAPI_get_hca_hndl( ca_id, &hca_hndl ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7105: " - "Fail to get HCA handle (%u).\n", vapi_res ); - goto Exit; - } - - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "__osm_ca_info_init: " "Querying CA %s.\n", ca_id ); - } - - /* query and get the HCA capability */ - vapi_res = VAPI_query_hca_cap( hca_hndl, &hca_vendor, &hca_cap ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7106: " - "Fail to get HCA Capabilities (%u).\n", vapi_res ); - goto Exit; - } - - /* get the guid of the HCA */ - memcpy( &( p_ca_info->guid ), hca_cap.node_guid, 8 * sizeof( u_int8_t ) ); - p_ca_info->attr_size = 1; - p_ca_info->p_attr = - ( ib_ca_attr_t * ) malloc( sizeof( ib_ca_attr_t ) ); - memcpy( &( p_ca_info->p_attr->ca_guid ), hca_cap.node_guid, - 8 * sizeof( u_int8_t ) ); - - /* now obtain the attributes of the ports */ - p_ca_info->p_attr->num_ports = hca_cap.phys_port_num; - p_ca_info->p_attr->p_port_attr = - ( ib_port_attr_t * ) malloc( hca_cap.phys_port_num * - sizeof( ib_port_attr_t ) ); - - for( port_num = 0; port_num < p_ca_info->p_attr->num_ports; port_num++ ) - { - - /* query the port attributes */ - vapi_res = - VAPI_query_hca_port_prop( hca_hndl, port_num + 1, &hca_port ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7107: " - "Fail to get HCA Port Attributes (%d).\n", vapi_res ); - goto Exit; - } - - /* first call to know the size of the gid table */ - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, port_num + 1, 0, &maxNumGids, - NULL ); - p_port_gid = - ( IB_gid_t * ) malloc( maxNumGids * sizeof( IB_gid_t ) ); - - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, port_num + 1, maxNumGids, - &maxNumGids, p_port_gid ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_ca_info_init: ERR 7108: " - "Fail to get HCA Port GID (%d).\n", vapi_res ); - goto Exit; - } - - __osm_vendor_gid_to_guid( p_port_gid[0], - ( IB_gid_t * ) & p_ca_info->p_attr-> - p_port_attr[port_num].port_guid ); - p_ca_info->p_attr->p_port_attr[port_num].lid = hca_port.lid; - p_ca_info->p_attr->p_port_attr[port_num].link_state = hca_port.state; - p_ca_info->p_attr->p_port_attr[port_num].sm_lid = hca_port.sm_lid; - - free( p_port_gid ); - } - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status = IB_ERROR; + VAPI_ret_t vapi_res; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_vendor_t hca_vendor; + VAPI_hca_cap_t hca_cap; + VAPI_hca_port_t hca_port; + uint8_t port_num; + IB_gid_t *p_port_gid; + uint16_t maxNumGids; + + OSM_LOG_ENTER(p_vend->p_log, __osm_ca_info_init); + + /* get the HCA handle */ + vapi_res = EVAPI_get_hca_hndl(ca_id, &hca_hndl); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7105: " + "Fail to get HCA handle (%u).\n", vapi_res); + goto Exit; + } + + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_ca_info_init: " "Querying CA %s.\n", ca_id); + } + + /* query and get the HCA capability */ + vapi_res = VAPI_query_hca_cap(hca_hndl, &hca_vendor, &hca_cap); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7106: " + "Fail to get HCA Capabilities (%u).\n", vapi_res); + goto Exit; + } + + /* get the guid of the HCA */ + memcpy(&(p_ca_info->guid), hca_cap.node_guid, 8 * sizeof(u_int8_t)); + p_ca_info->attr_size = 1; + p_ca_info->p_attr = (ib_ca_attr_t *) malloc(sizeof(ib_ca_attr_t)); + memcpy(&(p_ca_info->p_attr->ca_guid), hca_cap.node_guid, + 8 * sizeof(u_int8_t)); + + /* now obtain the attributes of the ports */ + p_ca_info->p_attr->num_ports = hca_cap.phys_port_num; + p_ca_info->p_attr->p_port_attr = + (ib_port_attr_t *) malloc(hca_cap.phys_port_num * + sizeof(ib_port_attr_t)); + + for (port_num = 0; port_num < p_ca_info->p_attr->num_ports; port_num++) { + + /* query the port attributes */ + vapi_res = + VAPI_query_hca_port_prop(hca_hndl, port_num + 1, &hca_port); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7107: " + "Fail to get HCA Port Attributes (%d).\n", + vapi_res); + goto Exit; + } + + /* first call to know the size of the gid table */ + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, port_num + 1, 0, + &maxNumGids, NULL); + p_port_gid = (IB_gid_t *) malloc(maxNumGids * sizeof(IB_gid_t)); + + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, port_num + 1, maxNumGids, + &maxNumGids, p_port_gid); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_ca_info_init: ERR 7108: " + "Fail to get HCA Port GID (%d).\n", vapi_res); + goto Exit; + } + + __osm_vendor_gid_to_guid(p_port_gid[0], + (IB_gid_t *) & p_ca_info->p_attr-> + p_port_attr[port_num].port_guid); + p_ca_info->p_attr->p_port_attr[port_num].lid = hca_port.lid; + p_ca_info->p_attr->p_port_attr[port_num].link_state = + hca_port.state; + p_ca_info->p_attr->p_port_attr[port_num].sm_lid = + hca_port.sm_lid; + + free(p_port_gid); + } + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -osm_ca_info_destroy( IN osm_vendor_t * const p_vend, - IN osm_ca_info_t * const p_ca_info ) +osm_ca_info_destroy(IN osm_vendor_t * const p_vend, + IN osm_ca_info_t * const p_ca_info) { - OSM_LOG_ENTER( p_vend->p_log, osm_ca_info_destroy ); + OSM_LOG_ENTER(p_vend->p_log, osm_ca_info_destroy); - if( p_ca_info->p_attr ) - { - if( p_ca_info->p_attr->num_ports ) - { - free( p_ca_info->p_attr->p_port_attr ); - } - free( p_ca_info->p_attr ); - } + if (p_ca_info->p_attr) { + if (p_ca_info->p_attr->num_ports) { + free(p_ca_info->p_attr->p_port_attr); + } + free(p_ca_info->p_attr); + } - free( p_ca_info ); + free(p_ca_info); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -325,118 +309,107 @@ osm_ca_info_destroy( IN osm_vendor_t * const p_vend, * UPDATE THE VENDOR OBJECT LIST OF CA_INFO STRUCTS **********************************************************************/ ib_api_status_t -osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, - IN ib_port_attr_t * const p_attr_array, - IN uint32_t * const p_num_ports ) +osm_vendor_get_all_port_attr(IN osm_vendor_t * const p_vend, + IN ib_port_attr_t * const p_attr_array, + IN uint32_t * const p_num_ports) { - ib_api_status_t status; - - uint32_t ca; - uint32_t ca_count; - uint32_t port_count = 0; - uint8_t port_num; - uint32_t total_ports = 0; - VAPI_hca_id_t *p_ca_ids = NULL; - osm_ca_info_t *p_ca_info; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_all_port_attr ); - - CL_ASSERT( p_vend ); - - /* - * 1) Determine the number of CA's - * 2) Allocate an array big enough to hold the ca info objects. - * 3) Call again to retrieve the guids. - */ - status = __osm_vendor_get_ca_ids( p_vend, &p_ca_ids, &ca_count ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 7109: " - "Fail to get CA Ids.\n" ); - goto Exit; - } - - /* we keep track of all the CAs in this info array */ - p_vend->p_ca_info = malloc( ca_count * sizeof( *p_vend->p_ca_info ) ); - if( p_vend->p_ca_info == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 7110: " - "Unable to allocate CA information array.\n" ); - goto Exit; - } - - memset( p_vend->p_ca_info, 0, ca_count * sizeof( *p_vend->p_ca_info ) ); - p_vend->ca_count = ca_count; - - /* - * For each CA, retrieve the CA info attributes - */ - for( ca = 0; ca < ca_count; ca++ ) - { - p_ca_info = &p_vend->p_ca_info[ca]; - - status = __osm_ca_info_init( p_vend, p_ca_ids[ca], p_ca_info ); - - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_all_port_attr: ERR 7111: " - "Unable to initialize CA Info object (%s).\n", - ib_get_err_str( status ) ); - } - - total_ports += osm_ca_info_get_num_ports( p_ca_info ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_all_port_attr: " - "osm_vendor_get_all_port_attr: %u got %u ports total:%u\n", - ca, osm_ca_info_get_num_ports( p_ca_info ), - total_ports - ); - - } - - /* - * If the user supplied enough storage, return the port guids, - * otherwise, return the appropriate error. - */ - if( *p_num_ports >= total_ports ) - { - for( ca = 0; ca < ca_count; ca++ ) - { - uint32_t num_ports; - - p_ca_info = &p_vend->p_ca_info[ca]; - - num_ports = osm_ca_info_get_num_ports( p_ca_info ); - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_attr_array[port_count] = - *__osm_ca_info_get_port_attr_ptr( p_ca_info, port_num ); - port_count++; - } - } - } - else - { - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - - status = IB_SUCCESS; - - Exit: - *p_num_ports = total_ports; - - if( p_ca_ids ) - free( p_ca_ids ); - - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + + uint32_t ca; + uint32_t ca_count; + uint32_t port_count = 0; + uint8_t port_num; + uint32_t total_ports = 0; + VAPI_hca_id_t *p_ca_ids = NULL; + osm_ca_info_t *p_ca_info; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_all_port_attr); + + CL_ASSERT(p_vend); + + /* + * 1) Determine the number of CA's + * 2) Allocate an array big enough to hold the ca info objects. + * 3) Call again to retrieve the guids. + */ + status = __osm_vendor_get_ca_ids(p_vend, &p_ca_ids, &ca_count); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 7109: " + "Fail to get CA Ids.\n"); + goto Exit; + } + + /* we keep track of all the CAs in this info array */ + p_vend->p_ca_info = malloc(ca_count * sizeof(*p_vend->p_ca_info)); + if (p_vend->p_ca_info == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 7110: " + "Unable to allocate CA information array.\n"); + goto Exit; + } + + memset(p_vend->p_ca_info, 0, ca_count * sizeof(*p_vend->p_ca_info)); + p_vend->ca_count = ca_count; + + /* + * For each CA, retrieve the CA info attributes + */ + for (ca = 0; ca < ca_count; ca++) { + p_ca_info = &p_vend->p_ca_info[ca]; + + status = __osm_ca_info_init(p_vend, p_ca_ids[ca], p_ca_info); + + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_all_port_attr: ERR 7111: " + "Unable to initialize CA Info object (%s).\n", + ib_get_err_str(status)); + } + + total_ports += osm_ca_info_get_num_ports(p_ca_info); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_all_port_attr: " + "osm_vendor_get_all_port_attr: %u got %u ports total:%u\n", + ca, osm_ca_info_get_num_ports(p_ca_info), total_ports); + + } + + /* + * If the user supplied enough storage, return the port guids, + * otherwise, return the appropriate error. + */ + if (*p_num_ports >= total_ports) { + for (ca = 0; ca < ca_count; ca++) { + uint32_t num_ports; + + p_ca_info = &p_vend->p_ca_info[ca]; + + num_ports = osm_ca_info_get_num_ports(p_ca_info); + + for (port_num = 0; port_num < num_ports; port_num++) { + p_attr_array[port_count] = + *__osm_ca_info_get_port_attr_ptr(p_ca_info, + port_num); + port_count++; + } + } + } else { + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + *p_num_ports = total_ports; + + if (p_ca_ids) + free(p_ca_ids); + + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -445,128 +418,124 @@ osm_vendor_get_all_port_attr( IN osm_vendor_t * const p_vend, **********************************************************************/ ib_api_status_t -osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, - IN ib_net64_t const guid, - OUT VAPI_hca_hndl_t * p_hca_hndl, - OUT VAPI_hca_id_t * p_hca_id, - OUT uint32_t * p_port_num ) +osm_vendor_get_guid_ca_and_port(IN osm_vendor_t * const p_vend, + IN ib_net64_t const guid, + OUT VAPI_hca_hndl_t * p_hca_hndl, + OUT VAPI_hca_id_t * p_hca_id, + OUT uint32_t * p_port_num) { - ib_api_status_t status; - VAPI_hca_id_t *p_ca_ids = NULL; - VAPI_ret_t vapi_res; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_vendor_t hca_vendor; - VAPI_hca_cap_t hca_cap; - IB_gid_t *p_port_gid = NULL; - uint16_t maxNumGids; - ib_net64_t port_guid; - uint32_t ca, portIdx, ca_count; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_guid_ca_and_port ); - - CL_ASSERT( p_vend ); - - /* - * 1) Determine the number of CA's - * 2) Allocate an array big enough to hold the ca info objects. - * 3) Call again to retrieve the guids. - */ - status = __osm_vendor_get_ca_ids( p_vend, &p_ca_ids, &ca_count ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 7112: " - "Fail to get CA Ids.\n" ); - goto Exit; - } - - /* - * For each CA, retrieve the CA info attributes - */ - for( ca = 0; ca < ca_count; ca++ ) - { - /* get the HCA handle */ - vapi_res = EVAPI_get_hca_hndl( p_ca_ids[ca], &hca_hndl ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 7113: " - "Fail to get HCA handle (%u).\n", vapi_res ); - goto Exit; - } - - /* get the CA attributes - to know how many ports it has: */ - if( osm_log_is_active( p_vend->p_log, OSM_LOG_DEBUG) ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get_guid_ca_and_port: " - "Querying CA %s.\n", p_ca_ids[ca] ); - } - - /* query and get the HCA capability */ - vapi_res = VAPI_query_hca_cap( hca_hndl, &hca_vendor, &hca_cap ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 7114: " - "Fail to get HCA Capabilities (%u).\n", vapi_res ); - goto Exit; - } - - /* go over all ports - to obtail their guids */ - for( portIdx = 0; portIdx < hca_cap.phys_port_num; portIdx++ ) - { - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, portIdx + 1, 0, &maxNumGids, - NULL ); - p_port_gid = - ( IB_gid_t * ) malloc( maxNumGids * sizeof( IB_gid_t ) ); - - /* get the port guid */ - vapi_res = - VAPI_query_hca_gid_tbl( hca_hndl, portIdx + 1, maxNumGids, - &maxNumGids, p_port_gid ); - if( vapi_res != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 7115: " - "Fail to get HCA Port GID (%d).\n", vapi_res ); - goto Exit; - } - - /* convert to SF style */ - __osm_vendor_gid_to_guid( p_port_gid[0], - ( VAPI_gid_t * ) & port_guid ); - - /* finally did we find it ? */ - if( port_guid == guid ) - { - *p_hca_hndl = hca_hndl; - memcpy( p_hca_id, p_ca_ids[ca], sizeof ( VAPI_hca_id_t ) ); - *p_port_num = portIdx + 1; - status = IB_SUCCESS; - goto Exit; - } - - free( p_port_gid ); - p_port_gid = NULL; - } /* ALL PORTS */ - } /* all HCAs */ - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get_guid_ca_and_port: ERR 7116: " - "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", - cl_ntoh64(guid) ); - status = IB_INVALID_GUID; - - Exit: - if( p_ca_ids != NULL ) - free( p_ca_ids ); - if( p_port_gid != NULL ) - free( p_port_gid ); - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + ib_api_status_t status; + VAPI_hca_id_t *p_ca_ids = NULL; + VAPI_ret_t vapi_res; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_vendor_t hca_vendor; + VAPI_hca_cap_t hca_cap; + IB_gid_t *p_port_gid = NULL; + uint16_t maxNumGids; + ib_net64_t port_guid; + uint32_t ca, portIdx, ca_count; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_guid_ca_and_port); + + CL_ASSERT(p_vend); + + /* + * 1) Determine the number of CA's + * 2) Allocate an array big enough to hold the ca info objects. + * 3) Call again to retrieve the guids. + */ + status = __osm_vendor_get_ca_ids(p_vend, &p_ca_ids, &ca_count); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 7112: " + "Fail to get CA Ids.\n"); + goto Exit; + } + + /* + * For each CA, retrieve the CA info attributes + */ + for (ca = 0; ca < ca_count; ca++) { + /* get the HCA handle */ + vapi_res = EVAPI_get_hca_hndl(p_ca_ids[ca], &hca_hndl); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 7113: " + "Fail to get HCA handle (%u).\n", vapi_res); + goto Exit; + } + + /* get the CA attributes - to know how many ports it has: */ + if (osm_log_is_active(p_vend->p_log, OSM_LOG_DEBUG)) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get_guid_ca_and_port: " + "Querying CA %s.\n", p_ca_ids[ca]); + } + + /* query and get the HCA capability */ + vapi_res = VAPI_query_hca_cap(hca_hndl, &hca_vendor, &hca_cap); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 7114: " + "Fail to get HCA Capabilities (%u).\n", + vapi_res); + goto Exit; + } + + /* go over all ports - to obtail their guids */ + for (portIdx = 0; portIdx < hca_cap.phys_port_num; portIdx++) { + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, portIdx + 1, 0, + &maxNumGids, NULL); + p_port_gid = + (IB_gid_t *) malloc(maxNumGids * sizeof(IB_gid_t)); + + /* get the port guid */ + vapi_res = + VAPI_query_hca_gid_tbl(hca_hndl, portIdx + 1, + maxNumGids, &maxNumGids, + p_port_gid); + if (vapi_res != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 7115: " + "Fail to get HCA Port GID (%d).\n", + vapi_res); + goto Exit; + } + + /* convert to SF style */ + __osm_vendor_gid_to_guid(p_port_gid[0], + (VAPI_gid_t *) & port_guid); + + /* finally did we find it ? */ + if (port_guid == guid) { + *p_hca_hndl = hca_hndl; + memcpy(p_hca_id, p_ca_ids[ca], + sizeof(VAPI_hca_id_t)); + *p_port_num = portIdx + 1; + status = IB_SUCCESS; + goto Exit; + } + + free(p_port_gid); + p_port_gid = NULL; + } /* ALL PORTS */ + } /* all HCAs */ + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get_guid_ca_and_port: ERR 7116: " + "Fail to find HCA and Port for Port Guid 0x%" PRIx64 "\n", + cl_ntoh64(guid)); + status = IB_INVALID_GUID; + + Exit: + if (p_ca_ids != NULL) + free(p_ca_ids); + if (p_port_gid != NULL) + free(p_port_gid); + OSM_LOG_EXIT(p_vend->p_log); + return (status); } #ifdef __TEST_HCA_GUID__ @@ -577,96 +546,88 @@ osm_vendor_get_guid_ca_and_port( IN osm_vendor_t * const p_vend, /********************************************************************** **********************************************************************/ -ib_net64_t -get_port_guid( ) +ib_net64_t get_port_guid() { - uint32_t i; - uint32_t choice = 0; - boolean_t done_flag = FALSE; - ib_api_status_t status; - uint32_t num_ports = GUID_ARRAY_SIZE; - ib_port_attr_t attr_array[GUID_ARRAY_SIZE]; - VAPI_hca_id_t ca_id; - uint32_t portNum; - osm_vendor_t vend; - osm_vendor_t *p_vend; - osm_log_t *p_osm_log, tlog; - - p_osm_log = &tlog; - - status = osm_log_init( p_osm_log, FALSE ); - if( status != IB_SUCCESS ) - return ( status ); - - osm_log( p_osm_log, OSM_LOG_FUNCS, - "get_port_guid: [\n" ); - - p_vend = &vend; - p_vend->p_log = p_osm_log; - - /* - * Call the transport layer for a list of local port - * GUID values. - */ - status = osm_vendor_get_all_port_attr( p_vend, attr_array, &num_ports ); - if( status != IB_SUCCESS ) - { - printf( "\nError from osm_opensm_init (%x)\n", status ); - return ( 0 ); - } - - if( num_ports == 0 ) - { - printf( "\nNo local ports detected!\n" ); - return ( 0 ); - } - - while( done_flag == FALSE ) - { - printf( "\nChoose a local port number with which to bind:\n\n" ); - for( i = 0; i < num_ports; i++ ) - { - /* - * Print the index + 1 since by convention, port numbers - * start with 1 on host channel adapters. - */ - - printf( "\t%u: GUID = 0x%8" PRIx64 ", lid = 0x%04X, state = %s\n", - i + 1, cl_ntoh64( attr_array[i].port_guid ), - cl_ntoh16( attr_array[i].lid ), - ib_get_port_state_str( attr_array[i].link_state ) ); - } - - printf( "\nEnter choice (1-%u): ", i ); - fflush( stdout ); - scanf( "%u", &choice ); - if( choice > num_ports ) - printf( "\nError: Lame choice!\n" ); - else - done_flag = TRUE; - } - - status = - osm_vendor_get_guid_ca_and_port( p_vend, - attr_array[choice - 1].port_guid, - &ca_id, &portNum ); - if( status != IB_SUCCESS ) - { - printf( "Error obtaining back the HCA and Port\n" ); - return ( 0 ); - } - - printf( "Selected: CA:%s Port:%d\n", ca_id, portNum ); - - return ( attr_array[choice - 1].port_guid ); + uint32_t i; + uint32_t choice = 0; + boolean_t done_flag = FALSE; + ib_api_status_t status; + uint32_t num_ports = GUID_ARRAY_SIZE; + ib_port_attr_t attr_array[GUID_ARRAY_SIZE]; + VAPI_hca_id_t ca_id; + uint32_t portNum; + osm_vendor_t vend; + osm_vendor_t *p_vend; + osm_log_t *p_osm_log, tlog; + + p_osm_log = &tlog; + + status = osm_log_init(p_osm_log, FALSE); + if (status != IB_SUCCESS) + return (status); + + osm_log(p_osm_log, OSM_LOG_FUNCS, "get_port_guid: [\n"); + + p_vend = &vend; + p_vend->p_log = p_osm_log; + + /* + * Call the transport layer for a list of local port + * GUID values. + */ + status = osm_vendor_get_all_port_attr(p_vend, attr_array, &num_ports); + if (status != IB_SUCCESS) { + printf("\nError from osm_opensm_init (%x)\n", status); + return (0); + } + + if (num_ports == 0) { + printf("\nNo local ports detected!\n"); + return (0); + } + + while (done_flag == FALSE) { + printf("\nChoose a local port number with which to bind:\n\n"); + for (i = 0; i < num_ports; i++) { + /* + * Print the index + 1 since by convention, port numbers + * start with 1 on host channel adapters. + */ + + printf("\t%u: GUID = 0x%8" PRIx64 + ", lid = 0x%04X, state = %s\n", i + 1, + cl_ntoh64(attr_array[i].port_guid), + cl_ntoh16(attr_array[i].lid), + ib_get_port_state_str(attr_array[i].link_state)); + } + + printf("\nEnter choice (1-%u): ", i); + fflush(stdout); + scanf("%u", &choice); + if (choice > num_ports) + printf("\nError: Lame choice!\n"); + else + done_flag = TRUE; + } + + status = + osm_vendor_get_guid_ca_and_port(p_vend, + attr_array[choice - 1].port_guid, + &ca_id, &portNum); + if (status != IB_SUCCESS) { + printf("Error obtaining back the HCA and Port\n"); + return (0); + } + + printf("Selected: CA:%s Port:%d\n", ca_id, portNum); + + return (attr_array[choice - 1].port_guid); } -int -main( int argc, - char **argv ) +int main(int argc, char **argv) { - get_port_guid( ); - return ( 0 ); + get_port_guid(); + return (0); } #endif diff --git a/opensm/libvendor/osm_vendor_mtl_transaction_mgr.c b/opensm/libvendor/osm_vendor_mtl_transaction_mgr.c index 8f2eb3a..05a5fab 100644 --- a/opensm/libvendor/osm_vendor_mtl_transaction_mgr.c +++ b/opensm/libvendor/osm_vendor_mtl_transaction_mgr.c @@ -33,10 +33,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -54,213 +53,215 @@ #endif /* this is the callback function of the timer */ -void -__osm_transaction_mgr_callback( IN void* context ) +void __osm_transaction_mgr_callback(IN void *context) { - osm_transaction_mgr_t *trans_mgr_p; - osm_vendor_t* p_vend = (osm_vendor_t*)context; - cl_list_item_t *p_list_item; - cl_list_item_t *p_list_next_item; - osm_madw_req_t *osm_madw_req_p; - uint64_t current_time; /* [usec] */ - uint32_t new_timeout; /* [msec] */ - cl_status_t cl_status; - ib_mad_t *p_mad; + osm_transaction_mgr_t *trans_mgr_p; + osm_vendor_t *p_vend = (osm_vendor_t *) context; + cl_list_item_t *p_list_item; + cl_list_item_t *p_list_next_item; + osm_madw_req_t *osm_madw_req_p; + uint64_t current_time; /* [usec] */ + uint32_t new_timeout; /* [msec] */ + cl_status_t cl_status; + ib_mad_t *p_mad; #ifdef OSM_VENDOR_INTF_MTL - osm_mtl_bind_info_t *p_bind; + osm_mtl_bind_info_t *p_bind; #else - osm_ts_bind_info_t *p_bind; + osm_ts_bind_info_t *p_bind; #endif - cl_list_t tmp_madw_p_list; /* this list will include all the madw_p that should be removed. */ - cl_list_t retry_madw_p_list; /* this list will include all the madw_p that were retried and need to be removed. */ - osm_madw_t *madw_p; - - OSM_LOG_ENTER( p_vend->p_log, __osm_transaction_mgr_callback ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - /* initialize the tmp_madw_p_list */ - cl_list_construct(&tmp_madw_p_list); - cl_status = cl_list_init(&tmp_madw_p_list, 50); - if (cl_status != CL_SUCCESS) - { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_transaction_mgr_callback : ERROR 1000: " - "Failed to create tmp_madw_p_list\n" ); - } - - cl_list_construct(&retry_madw_p_list); - cl_status = cl_list_init(&retry_madw_p_list, 50); - if (cl_status != CL_SUCCESS) - { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_transaction_mgr_callback : ERROR 1000: " - "Failed to create retry_madw_p_list\n" ); - } - - current_time = cl_get_time_stamp(); - cl_spinlock_acquire( &(trans_mgr_p->transaction_mgr_lock) ); - p_list_item = cl_qlist_head(trans_mgr_p->madw_reqs_list_p); - if (p_list_item == cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) - { - /* the list is empty - nothing to do */ - cl_spinlock_release( &trans_mgr_p->transaction_mgr_lock ); - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "__osm_transaction_mgr_callback : Nothing to do\n"); - goto Exit; - } - - /* non empty list: */ - - /* get the osm_madw_req_p */ - osm_madw_req_p = PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); - - while (osm_madw_req_p->waking_time <= current_time) - { - /* this object was supposed to have gotten a response */ - /* we need to decide if we need to retry or done with it. */ - if (osm_madw_req_p->retry_cnt > 0) - { - /* add to the list of the retrys : */ - cl_list_insert_tail(&retry_madw_p_list, osm_madw_req_p); - - /* update wakeup time and retry count */ - osm_madw_req_p->waking_time = p_vend->timeout*1000 + cl_get_time_stamp(); - osm_madw_req_p->retry_cnt--; - - /* make sure we will get some timer call if not earlier */ - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "__osm_transaction_mgr_callback : Timer restart:%u\n", p_vend->timeout); - - cl_status = cl_timer_start(&trans_mgr_p->madw_list_timer, p_vend->timeout); - - /* go to the next object and check if it also needs to be removed - didn't receive response */ - /* we need to do it before we move current item to the end of the list */ - p_list_next_item = cl_qlist_next(p_list_item); - - /* remove from the head */ - cl_qlist_remove_item( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item) ); - - /* insert the object to the qlist and the qmap */ - cl_qlist_insert_tail( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item)); - - } - else - { - /* go to the next object and check if it also needs to be removed - didn't receive response */ - p_list_next_item = cl_qlist_next(p_list_item); - - /* remove from the head */ - cl_qlist_remove_item( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item) ); - - /* add it to the tmp_madw_p_list to be removed */ - cl_list_insert_tail(&tmp_madw_p_list, osm_madw_req_p->p_madw); - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "__osm_transaction_mgr_callback : Found failed transaction madw: %p\n", - osm_madw_req_p->p_madw - ); - } - - /* Advance */ - p_list_item = p_list_next_item; - if (p_list_item == cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) - { - /* the list is empty - nothing to do */ - break; - } - - /* get the osm_madw_req_p */ - osm_madw_req_p = PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); - } - - /* look at the current p_list_item. If it is not the end item - then we need to */ - /* re-start the timer */ - if (p_list_item != cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) - { - /* get the osm_madw_req_p */ - osm_madw_req_p = PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); - - /* we have the object that still didn't get response - re-start the timer */ - /* start the timer to the timeout (in miliseconds) */ - new_timeout = (osm_madw_req_p->waking_time - cl_get_time_stamp())/1000 + 1; - cl_status = cl_timer_start(&trans_mgr_p->madw_list_timer, new_timeout); - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "__osm_transaction_mgr_callback : Timer restart:%u\n", new_timeout); - - if (cl_status != CL_SUCCESS) - { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "__osm_transaction_mgr_callback : ERROR 1000: " - "Failed to start timer\n" ); - } - } - /* if not empty - retry on retry list: */ - if (!cl_is_list_empty(&retry_madw_p_list)) - { - - /* remove all elements that were retried: */ - osm_madw_req_p = (osm_madw_req_t *)(cl_list_remove_head(&retry_madw_p_list)); - while (osm_madw_req_p != NULL) - { - - /* resend: */ - osm_log (p_vend->p_log, OSM_LOG_DEBUG, - "__osm_transaction_mgr_callback : " - "Retry %d of madw %p\n", - OSM_DEFAULT_RETRY_COUNT - osm_madw_req_p->retry_cnt, - osm_madw_req_p->p_madw - ); - - /* actually send it */ + cl_list_t tmp_madw_p_list; /* this list will include all the madw_p that should be removed. */ + cl_list_t retry_madw_p_list; /* this list will include all the madw_p that were retried and need to be removed. */ + osm_madw_t *madw_p; + + OSM_LOG_ENTER(p_vend->p_log, __osm_transaction_mgr_callback); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + /* initialize the tmp_madw_p_list */ + cl_list_construct(&tmp_madw_p_list); + cl_status = cl_list_init(&tmp_madw_p_list, 50); + if (cl_status != CL_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_transaction_mgr_callback : ERROR 1000: " + "Failed to create tmp_madw_p_list\n"); + } + + cl_list_construct(&retry_madw_p_list); + cl_status = cl_list_init(&retry_madw_p_list, 50); + if (cl_status != CL_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_transaction_mgr_callback : ERROR 1000: " + "Failed to create retry_madw_p_list\n"); + } + + current_time = cl_get_time_stamp(); + cl_spinlock_acquire(&(trans_mgr_p->transaction_mgr_lock)); + p_list_item = cl_qlist_head(trans_mgr_p->madw_reqs_list_p); + if (p_list_item == cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) { + /* the list is empty - nothing to do */ + cl_spinlock_release(&trans_mgr_p->transaction_mgr_lock); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_transaction_mgr_callback : Nothing to do\n"); + goto Exit; + } + + /* non empty list: */ + + /* get the osm_madw_req_p */ + osm_madw_req_p = PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); + + while (osm_madw_req_p->waking_time <= current_time) { + /* this object was supposed to have gotten a response */ + /* we need to decide if we need to retry or done with it. */ + if (osm_madw_req_p->retry_cnt > 0) { + /* add to the list of the retrys : */ + cl_list_insert_tail(&retry_madw_p_list, osm_madw_req_p); + + /* update wakeup time and retry count */ + osm_madw_req_p->waking_time = + p_vend->timeout * 1000 + cl_get_time_stamp(); + osm_madw_req_p->retry_cnt--; + + /* make sure we will get some timer call if not earlier */ + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_transaction_mgr_callback : Timer restart:%u\n", + p_vend->timeout); + + cl_status = + cl_timer_start(&trans_mgr_p->madw_list_timer, + p_vend->timeout); + + /* go to the next object and check if it also needs to be removed - didn't receive response */ + /* we need to do it before we move current item to the end of the list */ + p_list_next_item = cl_qlist_next(p_list_item); + + /* remove from the head */ + cl_qlist_remove_item(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + + /* insert the object to the qlist and the qmap */ + cl_qlist_insert_tail(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + + } else { + /* go to the next object and check if it also needs to be removed - didn't receive response */ + p_list_next_item = cl_qlist_next(p_list_item); + + /* remove from the head */ + cl_qlist_remove_item(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + + /* add it to the tmp_madw_p_list to be removed */ + cl_list_insert_tail(&tmp_madw_p_list, + osm_madw_req_p->p_madw); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_transaction_mgr_callback : Found failed transaction madw: %p\n", + osm_madw_req_p->p_madw); + } + + /* Advance */ + p_list_item = p_list_next_item; + if (p_list_item == cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) { + /* the list is empty - nothing to do */ + break; + } + + /* get the osm_madw_req_p */ + osm_madw_req_p = + PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); + } + + /* look at the current p_list_item. If it is not the end item - then we need to */ + /* re-start the timer */ + if (p_list_item != cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) { + /* get the osm_madw_req_p */ + osm_madw_req_p = + PARENT_STRUCT(p_list_item, osm_madw_req_t, list_item); + + /* we have the object that still didn't get response - re-start the timer */ + /* start the timer to the timeout (in miliseconds) */ + new_timeout = + (osm_madw_req_p->waking_time - cl_get_time_stamp()) / 1000 + + 1; + cl_status = + cl_timer_start(&trans_mgr_p->madw_list_timer, new_timeout); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_transaction_mgr_callback : Timer restart:%u\n", + new_timeout); + + if (cl_status != CL_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_transaction_mgr_callback : ERROR 1000: " + "Failed to start timer\n"); + } + } + /* if not empty - retry on retry list: */ + if (!cl_is_list_empty(&retry_madw_p_list)) { + + /* remove all elements that were retried: */ + osm_madw_req_p = + (osm_madw_req_t + *) (cl_list_remove_head(&retry_madw_p_list)); + while (osm_madw_req_p != NULL) { + + /* resend: */ + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "__osm_transaction_mgr_callback : " + "Retry %d of madw %p\n", + OSM_DEFAULT_RETRY_COUNT - + osm_madw_req_p->retry_cnt, + osm_madw_req_p->p_madw); + + /* actually send it */ #ifdef OSM_VENDOR_INTF_MTL - osm_mtl_send_mad( - (osm_mtl_bind_info_t *)osm_madw_req_p->p_bind, - osm_madw_req_p->p_madw); + osm_mtl_send_mad((osm_mtl_bind_info_t *) + osm_madw_req_p->p_bind, + osm_madw_req_p->p_madw); #else - ib_api_status_t - osm_ts_send_mad(osm_ts_bind_info_t *p_bind,osm_madw_t * const p_madw); - osm_ts_send_mad( - (osm_ts_bind_info_t *)osm_madw_req_p->p_bind, - osm_madw_req_p->p_madw - ); + ib_api_status_t + osm_ts_send_mad(osm_ts_bind_info_t * p_bind, + osm_madw_t * const p_madw); + osm_ts_send_mad((osm_ts_bind_info_t *) osm_madw_req_p-> + p_bind, osm_madw_req_p->p_madw); #endif - /* next one */ - osm_madw_req_p = (osm_madw_req_t *)(cl_list_remove_head(&retry_madw_p_list)); - } - } - - /* if the tmp_madw_p_list has elements - need to call the send_err_callback */ - madw_p = (osm_madw_t *)(cl_list_remove_head(&tmp_madw_p_list)); - while (madw_p != NULL) - { - /* need to remove it from pool */ - - /* obtain the madw_p stored as the wrid in the send call */ - p_mad = osm_madw_get_mad_ptr( madw_p ); - p_bind = madw_p->h_bind; - /* - Return any wrappers to the pool that may have been - pre-emptively allocated to handle a receive. - */ - if( madw_p->vend_wrap.p_resp_madw ) - { + /* next one */ + osm_madw_req_p = + (osm_madw_req_t + *) (cl_list_remove_head(&retry_madw_p_list)); + } + } + + /* if the tmp_madw_p_list has elements - need to call the send_err_callback */ + madw_p = (osm_madw_t *) (cl_list_remove_head(&tmp_madw_p_list)); + while (madw_p != NULL) { + /* need to remove it from pool */ + + /* obtain the madw_p stored as the wrid in the send call */ + p_mad = osm_madw_get_mad_ptr(madw_p); + p_bind = madw_p->h_bind; + /* + Return any wrappers to the pool that may have been + pre-emptively allocated to handle a receive. + */ + if (madw_p->vend_wrap.p_resp_madw) { #ifdef OSM_VENDOR_INTF_MTL - osm_mad_pool_put( - p_bind->p_osm_pool, madw_p->vend_wrap.p_resp_madw ); + osm_mad_pool_put(p_bind->p_osm_pool, + madw_p->vend_wrap.p_resp_madw); #else - osm_mad_pool_put( - p_bind->p_osm_pool, madw_p->vend_wrap.p_resp_madw ); + osm_mad_pool_put(p_bind->p_osm_pool, + madw_p->vend_wrap.p_resp_madw); #endif - madw_p->vend_wrap.p_resp_madw = NULL; - } + madw_p->vend_wrap.p_resp_madw = NULL; + } - /* invoke the CB */ - ( *(osm_vend_mad_send_err_callback_t)(p_bind->send_err_callback) ) ( p_bind->client_context, madw_p ); - madw_p = (osm_madw_t *)(cl_list_remove_head(&tmp_madw_p_list)); - } + /* invoke the CB */ + (*(osm_vend_mad_send_err_callback_t) + (p_bind->send_err_callback)) (p_bind->client_context, madw_p); + madw_p = (osm_madw_t *) (cl_list_remove_head(&tmp_madw_p_list)); + } - Exit: - OSM_LOG_EXIT( p_vend->p_log ); + Exit: + OSM_LOG_EXIT(p_vend->p_log); } @@ -268,274 +269,278 @@ __osm_transaction_mgr_callback( IN void* context ) * Construct and Initialize */ -void -osm_transaction_mgr_init( IN osm_vendor_t * const p_vend ) +void osm_transaction_mgr_init(IN osm_vendor_t * const p_vend) { - cl_status_t cl_status; - osm_transaction_mgr_t *trans_mgr_p; - OSM_LOG_ENTER( p_vend->p_log, osm_transaction_mgr_init ); - - CL_ASSERT( p_vend->p_transaction_mgr == NULL ); - - (osm_transaction_mgr_t*)p_vend->p_transaction_mgr = - (osm_transaction_mgr_t * ) malloc( sizeof( osm_transaction_mgr_t ) ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - /* construct lock object */ - cl_spinlock_construct( &(trans_mgr_p->transaction_mgr_lock) ); - CL_ASSERT( cl_spinlock_init( &(trans_mgr_p->transaction_mgr_lock) ) == CL_SUCCESS ); - - /* initialize the qlist */ - trans_mgr_p->madw_reqs_list_p = - ( cl_qlist_t * ) malloc( sizeof( cl_qlist_t ) ); - cl_qlist_init( trans_mgr_p->madw_reqs_list_p); - - /* initialize the qmap */ - trans_mgr_p->madw_by_tid_map_p = - ( cl_qmap_t * ) malloc( sizeof( cl_qmap_t ) ); - cl_qmap_init( trans_mgr_p->madw_by_tid_map_p ); - - /* create the timer used by the madw_req_list */ - cl_timer_construct( &(trans_mgr_p->madw_list_timer) ); - - /* init the timer with timeout. */ - cl_status = cl_timer_init(&trans_mgr_p->madw_list_timer, - __osm_transaction_mgr_callback, - p_vend ); - - if (cl_status != CL_SUCCESS) - { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_transaction_mgr_init : ERROR 1000: " - "Failed to initialize madw_reqs_list timer\n" ); - } - OSM_LOG_EXIT( p_vend->p_log ); + cl_status_t cl_status; + osm_transaction_mgr_t *trans_mgr_p; + OSM_LOG_ENTER(p_vend->p_log, osm_transaction_mgr_init); + + CL_ASSERT(p_vend->p_transaction_mgr == NULL); + + (osm_transaction_mgr_t *) p_vend->p_transaction_mgr = + (osm_transaction_mgr_t *) malloc(sizeof(osm_transaction_mgr_t)); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + /* construct lock object */ + cl_spinlock_construct(&(trans_mgr_p->transaction_mgr_lock)); + CL_ASSERT(cl_spinlock_init(&(trans_mgr_p->transaction_mgr_lock)) == + CL_SUCCESS); + + /* initialize the qlist */ + trans_mgr_p->madw_reqs_list_p = + (cl_qlist_t *) malloc(sizeof(cl_qlist_t)); + cl_qlist_init(trans_mgr_p->madw_reqs_list_p); + + /* initialize the qmap */ + trans_mgr_p->madw_by_tid_map_p = + (cl_qmap_t *) malloc(sizeof(cl_qmap_t)); + cl_qmap_init(trans_mgr_p->madw_by_tid_map_p); + + /* create the timer used by the madw_req_list */ + cl_timer_construct(&(trans_mgr_p->madw_list_timer)); + + /* init the timer with timeout. */ + cl_status = cl_timer_init(&trans_mgr_p->madw_list_timer, + __osm_transaction_mgr_callback, p_vend); + + if (cl_status != CL_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_transaction_mgr_init : ERROR 1000: " + "Failed to initialize madw_reqs_list timer\n"); + } + OSM_LOG_EXIT(p_vend->p_log); } -void -osm_transaction_mgr_destroy ( IN osm_vendor_t * const p_vend ) +void osm_transaction_mgr_destroy(IN osm_vendor_t * const p_vend) { - osm_transaction_mgr_t *trans_mgr_p; - cl_list_item_t *p_list_item; - cl_map_item_t *p_map_item; - osm_madw_req_t *osm_madw_req_p; - - OSM_LOG_ENTER( p_vend->p_log, osm_transaction_mgr_destroy ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - if( p_vend->p_transaction_mgr != NULL ) - { - /* we need to get a lock */ - cl_spinlock_acquire( &trans_mgr_p->transaction_mgr_lock ); - - /* go over all the items in the list and remove them */ - p_list_item = cl_qlist_remove_head(trans_mgr_p->madw_reqs_list_p); - while ( p_list_item != cl_qlist_end(trans_mgr_p->madw_reqs_list_p) ) { - osm_madw_req_p = (osm_madw_req_t*)p_list_item; - - if (osm_madw_req_p->p_madw->p_mad) - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_destroy: " - "Found outstanding MADW:%p TID:<0x%"PRIx64">.\n", - osm_madw_req_p->p_madw, - osm_madw_req_p->p_madw->p_mad->trans_id ); - else - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_destroy: " - "Found outstanding MADW:%p TID:UNDEFINED.\n", - osm_madw_req_p->p_madw); - - /* each item - remove it from the map */ - p_map_item = &(osm_madw_req_p->map_item); - cl_qmap_remove_item(trans_mgr_p->madw_by_tid_map_p, p_map_item); - /* free the item */ - free(osm_madw_req_p); - p_list_item = cl_qlist_remove_head(trans_mgr_p->madw_reqs_list_p); - } - /* free the qlist and qmap */ - free(trans_mgr_p->madw_reqs_list_p ); - free(trans_mgr_p->madw_by_tid_map_p ); - /* reliease and destroy the lock */ - cl_spinlock_release( &trans_mgr_p->transaction_mgr_lock ); - cl_spinlock_destroy( &(trans_mgr_p->transaction_mgr_lock) ); - /* destroy the timer */ - cl_timer_trim(&trans_mgr_p->madw_list_timer, 1); - cl_timer_destroy( &trans_mgr_p->madw_list_timer ); - /* free the transaction_manager object */ - free(trans_mgr_p); - trans_mgr_p = NULL; } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_transaction_mgr_t *trans_mgr_p; + cl_list_item_t *p_list_item; + cl_map_item_t *p_map_item; + osm_madw_req_t *osm_madw_req_p; + + OSM_LOG_ENTER(p_vend->p_log, osm_transaction_mgr_destroy); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + if (p_vend->p_transaction_mgr != NULL) { + /* we need to get a lock */ + cl_spinlock_acquire(&trans_mgr_p->transaction_mgr_lock); + + /* go over all the items in the list and remove them */ + p_list_item = + cl_qlist_remove_head(trans_mgr_p->madw_reqs_list_p); + while (p_list_item != + cl_qlist_end(trans_mgr_p->madw_reqs_list_p)) { + osm_madw_req_p = (osm_madw_req_t *) p_list_item; + + if (osm_madw_req_p->p_madw->p_mad) + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_destroy: " + "Found outstanding MADW:%p TID:<0x%" + PRIx64 ">.\n", osm_madw_req_p->p_madw, + osm_madw_req_p->p_madw->p_mad-> + trans_id); + else + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_destroy: " + "Found outstanding MADW:%p TID:UNDEFINED.\n", + osm_madw_req_p->p_madw); + + /* each item - remove it from the map */ + p_map_item = &(osm_madw_req_p->map_item); + cl_qmap_remove_item(trans_mgr_p->madw_by_tid_map_p, + p_map_item); + /* free the item */ + free(osm_madw_req_p); + p_list_item = + cl_qlist_remove_head(trans_mgr_p->madw_reqs_list_p); + } + /* free the qlist and qmap */ + free(trans_mgr_p->madw_reqs_list_p); + free(trans_mgr_p->madw_by_tid_map_p); + /* reliease and destroy the lock */ + cl_spinlock_release(&trans_mgr_p->transaction_mgr_lock); + cl_spinlock_destroy(&(trans_mgr_p->transaction_mgr_lock)); + /* destroy the timer */ + cl_timer_trim(&trans_mgr_p->madw_list_timer, 1); + cl_timer_destroy(&trans_mgr_p->madw_list_timer); + /* free the transaction_manager object */ + free(trans_mgr_p); + trans_mgr_p = NULL; + } + + OSM_LOG_EXIT(p_vend->p_log); } - ib_api_status_t -osm_transaction_mgr_insert_madw( IN osm_bind_handle_t *const p_bind, - IN osm_madw_t * p_madw ) +osm_transaction_mgr_insert_madw(IN osm_bind_handle_t * const p_bind, + IN osm_madw_t * p_madw) { #ifdef OSM_VENDOR_INTF_MTL - osm_vendor_t *const p_vend = ((osm_mtl_bind_info_t *)p_bind)->p_vend; + osm_vendor_t *const p_vend = ((osm_mtl_bind_info_t *) p_bind)->p_vend; #else - osm_vendor_t *const p_vend = ((osm_ts_bind_info_t *)p_bind)->p_vend; + osm_vendor_t *const p_vend = ((osm_ts_bind_info_t *) p_bind)->p_vend; #endif - osm_transaction_mgr_t *trans_mgr_p; - osm_madw_req_t *osm_madw_req_p; - uint64_t timeout; - uint64_t waking_time; - cl_status_t cl_status; - uint64_t key; - const ib_mad_t *mad_p = p_madw->p_mad; - - OSM_LOG_ENTER( p_vend->p_log, osm_transaction_mgr_insert_madw ); - - CL_ASSERT( mad_p ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - timeout = (uint64_t)(p_vend->timeout) * 1000; /* change the miliseconds value of timeout to microseconds. */ - waking_time = timeout + cl_get_time_stamp(); - - osm_madw_req_p = (osm_madw_req_t *)malloc( sizeof (osm_madw_req_t) ); - - osm_madw_req_p->p_madw = p_madw; - osm_madw_req_p->waking_time = waking_time; - osm_madw_req_p->retry_cnt = OSM_DEFAULT_RETRY_COUNT; - osm_madw_req_p->p_bind = p_bind; - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_insert_madw: " - "Inserting MADW:%p with waking_time: <0x%"PRIx64"> TID:<0x%"PRIx64">.\n", - p_madw, waking_time, p_madw->p_mad->trans_id ); - - /* Get the lock on the manager */ - cl_spinlock_acquire( &(trans_mgr_p->transaction_mgr_lock) ); - /* If the list is empty - need to start the timer with timer of timeout (in miliseconds) */ - if (cl_is_qlist_empty(trans_mgr_p->madw_reqs_list_p)) - { - /* stop the timer if it is running */ - cl_timer_stop(&trans_mgr_p->madw_list_timer); - - /* start the timer to the timeout (in miliseconds) */ - cl_status = cl_timer_start(&trans_mgr_p->madw_list_timer, - p_vend->timeout); - if (cl_status != CL_SUCCESS) - { - osm_log (p_vend->p_log, OSM_LOG_ERROR, - "osm_transaction_mgr_insert_madw : ERROR 1000: " - "Failed to start timer\n" ); - } - } - - /* insert the object to the qlist and the qmap */ - cl_qlist_insert_tail( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item)); - /* get the key */ - key = ( uint64_t ) mad_p->trans_id; - cl_qmap_insert( trans_mgr_p->madw_by_tid_map_p, key, &(osm_madw_req_p->map_item)); - cl_spinlock_release( &trans_mgr_p->transaction_mgr_lock ); - - OSM_LOG_EXIT( p_vend->p_log ); - - return ( IB_SUCCESS ); + osm_transaction_mgr_t *trans_mgr_p; + osm_madw_req_t *osm_madw_req_p; + uint64_t timeout; + uint64_t waking_time; + cl_status_t cl_status; + uint64_t key; + const ib_mad_t *mad_p = p_madw->p_mad; + + OSM_LOG_ENTER(p_vend->p_log, osm_transaction_mgr_insert_madw); + + CL_ASSERT(mad_p); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + timeout = (uint64_t) (p_vend->timeout) * 1000; /* change the miliseconds value of timeout to microseconds. */ + waking_time = timeout + cl_get_time_stamp(); + + osm_madw_req_p = (osm_madw_req_t *) malloc(sizeof(osm_madw_req_t)); + + osm_madw_req_p->p_madw = p_madw; + osm_madw_req_p->waking_time = waking_time; + osm_madw_req_p->retry_cnt = OSM_DEFAULT_RETRY_COUNT; + osm_madw_req_p->p_bind = p_bind; + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_insert_madw: " + "Inserting MADW:%p with waking_time: <0x%" PRIx64 "> TID:<0x%" + PRIx64 ">.\n", p_madw, waking_time, p_madw->p_mad->trans_id); + + /* Get the lock on the manager */ + cl_spinlock_acquire(&(trans_mgr_p->transaction_mgr_lock)); + /* If the list is empty - need to start the timer with timer of timeout (in miliseconds) */ + if (cl_is_qlist_empty(trans_mgr_p->madw_reqs_list_p)) { + /* stop the timer if it is running */ + cl_timer_stop(&trans_mgr_p->madw_list_timer); + + /* start the timer to the timeout (in miliseconds) */ + cl_status = cl_timer_start(&trans_mgr_p->madw_list_timer, + p_vend->timeout); + if (cl_status != CL_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_transaction_mgr_insert_madw : ERROR 1000: " + "Failed to start timer\n"); + } + } + + /* insert the object to the qlist and the qmap */ + cl_qlist_insert_tail(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + /* get the key */ + key = (uint64_t) mad_p->trans_id; + cl_qmap_insert(trans_mgr_p->madw_by_tid_map_p, key, + &(osm_madw_req_p->map_item)); + cl_spinlock_release(&trans_mgr_p->transaction_mgr_lock); + + OSM_LOG_EXIT(p_vend->p_log); + + return (IB_SUCCESS); } ib_api_status_t -osm_transaction_mgr_erase_madw( IN osm_vendor_t * const p_vend, - IN ib_mad_t * p_mad ) +osm_transaction_mgr_erase_madw(IN osm_vendor_t * const p_vend, + IN ib_mad_t * p_mad) { - osm_transaction_mgr_t *trans_mgr_p; - osm_madw_req_t *osm_madw_req_p; - uint64_t key; - cl_map_item_t *p_map_item; - OSM_LOG_ENTER( p_vend->p_log, osm_transaction_mgr_erase_madw ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - key = ( uint64_t ) p_mad->trans_id; - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_erase_madw: " - "Removing TID:<0x%"PRIx64">.\n", p_mad->trans_id ); - - cl_spinlock_acquire( &trans_mgr_p->transaction_mgr_lock ); - p_map_item = cl_qmap_get( trans_mgr_p->madw_by_tid_map_p, key ); - if (p_map_item != cl_qmap_end(trans_mgr_p->madw_by_tid_map_p)) - { - /* we found such an item. */ - /* get the osm_madw_req_p */ - osm_madw_req_p = PARENT_STRUCT(p_map_item, osm_madw_req_t, map_item); - - /* remove the item from the qlist */ - cl_qlist_remove_item( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item) ); - /* remove the item from the qmap */ - cl_qmap_remove_item( trans_mgr_p->madw_by_tid_map_p, &(osm_madw_req_p->map_item) ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_erase_madw: " - "Removed TID:<0x%"PRIx64">.\n", p_mad->trans_id ); - - /* free the item */ - free(osm_madw_req_p); - } - else - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_erase_madw: " - "osm_transaction_mgr_erase_madw:<0x%"PRIx64"> NOT FOUND.\n", p_mad->trans_id ); - } - cl_spinlock_release( &trans_mgr_p->transaction_mgr_lock ); - OSM_LOG_EXIT( p_vend->p_log ); - - return ( IB_SUCCESS ); + osm_transaction_mgr_t *trans_mgr_p; + osm_madw_req_t *osm_madw_req_p; + uint64_t key; + cl_map_item_t *p_map_item; + OSM_LOG_ENTER(p_vend->p_log, osm_transaction_mgr_erase_madw); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + key = (uint64_t) p_mad->trans_id; + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_erase_madw: " + "Removing TID:<0x%" PRIx64 ">.\n", p_mad->trans_id); + + cl_spinlock_acquire(&trans_mgr_p->transaction_mgr_lock); + p_map_item = cl_qmap_get(trans_mgr_p->madw_by_tid_map_p, key); + if (p_map_item != cl_qmap_end(trans_mgr_p->madw_by_tid_map_p)) { + /* we found such an item. */ + /* get the osm_madw_req_p */ + osm_madw_req_p = + PARENT_STRUCT(p_map_item, osm_madw_req_t, map_item); + + /* remove the item from the qlist */ + cl_qlist_remove_item(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + /* remove the item from the qmap */ + cl_qmap_remove_item(trans_mgr_p->madw_by_tid_map_p, + &(osm_madw_req_p->map_item)); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_erase_madw: " + "Removed TID:<0x%" PRIx64 ">.\n", p_mad->trans_id); + + /* free the item */ + free(osm_madw_req_p); + } else { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_erase_madw: " + "osm_transaction_mgr_erase_madw:<0x%" PRIx64 + "> NOT FOUND.\n", p_mad->trans_id); + } + cl_spinlock_release(&trans_mgr_p->transaction_mgr_lock); + OSM_LOG_EXIT(p_vend->p_log); + + return (IB_SUCCESS); } ib_api_status_t -osm_transaction_mgr_get_madw_for_tid( IN osm_vendor_t * const p_vend, - IN ib_mad_t * const p_mad, - OUT osm_madw_t ** req_madw_p ) +osm_transaction_mgr_get_madw_for_tid(IN osm_vendor_t * const p_vend, + IN ib_mad_t * const p_mad, + OUT osm_madw_t ** req_madw_p) { - osm_transaction_mgr_t *trans_mgr_p; - osm_madw_req_t *osm_madw_req_p; - cl_map_item_t *p_map_item; - uint64_t key; - OSM_LOG_ENTER( p_vend->p_log, osm_transaction_mgr_get_madw_for_tid ); - - trans_mgr_p = (osm_transaction_mgr_t*)p_vend->p_transaction_mgr; - - *req_madw_p = NULL; - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_get_madw_for_tid: " - "Looking for TID:<0x%"PRIx64">.\n", p_mad->trans_id ); - - key = ( uint64_t ) p_mad->trans_id; - cl_spinlock_acquire( &(trans_mgr_p->transaction_mgr_lock) ); - p_map_item = cl_qmap_get( trans_mgr_p->madw_by_tid_map_p, key ); - if (p_map_item != cl_qmap_end(trans_mgr_p->madw_by_tid_map_p)) - { - /* we found such an item. */ - /* get the osm_madw_req_p */ - osm_madw_req_p = PARENT_STRUCT(p_map_item, osm_madw_req_t, map_item); - - /* Since the Transaction was looked up and provided for */ - /* processing we retire it */ - cl_qlist_remove_item( trans_mgr_p->madw_reqs_list_p, &(osm_madw_req_p->list_item) ); - /* remove the item from the qmap */ - cl_qmap_remove_item( trans_mgr_p->madw_by_tid_map_p, &(osm_madw_req_p->map_item) ); - - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_get_madw_for_tid: " - "Removed TID:<0x%"PRIx64">.\n", p_mad->trans_id ); - - *req_madw_p = osm_madw_req_p->p_madw; - } - - cl_spinlock_release( &(trans_mgr_p->transaction_mgr_lock) ); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_transaction_mgr_get_madw_for_tid: " - "Got MADW:%p.\n", *req_madw_p ); - OSM_LOG_EXIT( p_vend->p_log ); - return ( IB_SUCCESS ); + osm_transaction_mgr_t *trans_mgr_p; + osm_madw_req_t *osm_madw_req_p; + cl_map_item_t *p_map_item; + uint64_t key; + OSM_LOG_ENTER(p_vend->p_log, osm_transaction_mgr_get_madw_for_tid); + + trans_mgr_p = (osm_transaction_mgr_t *) p_vend->p_transaction_mgr; + + *req_madw_p = NULL; + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_get_madw_for_tid: " + "Looking for TID:<0x%" PRIx64 ">.\n", p_mad->trans_id); + + key = (uint64_t) p_mad->trans_id; + cl_spinlock_acquire(&(trans_mgr_p->transaction_mgr_lock)); + p_map_item = cl_qmap_get(trans_mgr_p->madw_by_tid_map_p, key); + if (p_map_item != cl_qmap_end(trans_mgr_p->madw_by_tid_map_p)) { + /* we found such an item. */ + /* get the osm_madw_req_p */ + osm_madw_req_p = + PARENT_STRUCT(p_map_item, osm_madw_req_t, map_item); + + /* Since the Transaction was looked up and provided for */ + /* processing we retire it */ + cl_qlist_remove_item(trans_mgr_p->madw_reqs_list_p, + &(osm_madw_req_p->list_item)); + /* remove the item from the qmap */ + cl_qmap_remove_item(trans_mgr_p->madw_by_tid_map_p, + &(osm_madw_req_p->map_item)); + + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_get_madw_for_tid: " + "Removed TID:<0x%" PRIx64 ">.\n", p_mad->trans_id); + + *req_madw_p = osm_madw_req_p->p_madw; + } + + cl_spinlock_release(&(trans_mgr_p->transaction_mgr_lock)); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_transaction_mgr_get_madw_for_tid: " + "Got MADW:%p.\n", *req_madw_p); + OSM_LOG_EXIT(p_vend->p_log); + return (IB_SUCCESS); } #endif diff --git a/opensm/libvendor/osm_vendor_test.c b/opensm/libvendor/osm_vendor_test.c index 1b8f51f..30a43f9 100644 --- a/opensm/libvendor/osm_vendor_test.c +++ b/opensm/libvendor/osm_vendor_test.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of vendor specific transport interface. @@ -49,7 +48,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -63,255 +62,227 @@ /********************************************************************** **********************************************************************/ -void -osm_vendor_construct( - IN osm_vendor_t* const p_vend ) +void osm_vendor_construct(IN osm_vendor_t * const p_vend) { - memset( p_vend, 0, sizeof(*p_vend) ); + memset(p_vend, 0, sizeof(*p_vend)); } /********************************************************************** **********************************************************************/ -void -osm_vendor_destroy( - IN osm_vendor_t* const p_vend ) +void osm_vendor_destroy(IN osm_vendor_t * const p_vend) { - UNUSED_PARAM( p_vend ); + UNUSED_PARAM(p_vend); } /********************************************************************** **********************************************************************/ -void -osm_vendor_delete( - IN osm_vendor_t** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - CL_ASSERT( pp_vend ); + CL_ASSERT(pp_vend); - osm_vendor_destroy( *pp_vend ); - free( *pp_vend ); - *pp_vend = NULL; + osm_vendor_destroy(*pp_vend); + free(*pp_vend); + *pp_vend = NULL; } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_init( - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - OSM_LOG_ENTER( p_log, osm_vendor_init ); + OSM_LOG_ENTER(p_log, osm_vendor_init); - CL_ASSERT( p_vend ); - CL_ASSERT( p_log ); + CL_ASSERT(p_vend); + CL_ASSERT(p_log); - p_vend->p_log = p_log; - p_vend->timeout = timeout; - OSM_LOG_EXIT( p_log ); - return( IB_SUCCESS ); + p_vend->p_log = p_log; + p_vend->timeout = timeout; + OSM_LOG_EXIT(p_log); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - ib_api_status_t status; - osm_vendor_t *p_vend; - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - CL_ASSERT( p_log ); - - p_vend = malloc( sizeof(*p_vend) ); - if( p_vend != NULL ) - { - memset( p_vend, 0, sizeof(*p_vend) ); - - status = osm_vendor_init( p_vend, p_log, timeout ); - if( status != IB_SUCCESS ) - { - osm_vendor_delete( &p_vend ); - } - } - - OSM_LOG_EXIT( p_log ); - return( p_vend ); + ib_api_status_t status; + osm_vendor_t *p_vend; + OSM_LOG_ENTER(p_log, osm_vendor_new); + + CL_ASSERT(p_log); + + p_vend = malloc(sizeof(*p_vend)); + if (p_vend != NULL) { + memset(p_vend, 0, sizeof(*p_vend)); + + status = osm_vendor_init(p_vend, p_log, timeout); + if (status != IB_SUCCESS) { + osm_vendor_delete(&p_vend); + } + } + + OSM_LOG_EXIT(p_log); + return (p_vend); } /********************************************************************** **********************************************************************/ -ib_mad_t* -osm_vendor_get( - IN osm_bind_handle_t h_bind, - IN const uint32_t size, - IN osm_vend_wrap_t* const p_vend_wrap ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t size, + IN osm_vend_wrap_t * const p_vend_wrap) { - osm_vendor_t *p_vend; - ib_mad_t *p_mad; - OSM_LOG_ENTER( h_bind->p_vend->p_log, osm_vendor_get ); + osm_vendor_t *p_vend; + ib_mad_t *p_mad; + OSM_LOG_ENTER(h_bind->p_vend->p_log, osm_vendor_get); - UNUSED_PARAM( p_vend_wrap ); + UNUSED_PARAM(p_vend_wrap); - p_vend = h_bind->p_vend; + p_vend = h_bind->p_vend; - /* - Simply malloc the MAD off the heap. - */ - p_mad = (ib_mad_t*)malloc( size ); + /* + Simply malloc the MAD off the heap. + */ + p_mad = (ib_mad_t *) malloc(size); - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "osm_vendor_get: " - "MAD %p.\n", p_mad ); + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "osm_vendor_get: " "MAD %p.\n", p_mad); - if (p_mad) - memset( p_mad, 0, size ); + if (p_mad) + memset(p_mad, 0, size); - OSM_LOG_EXIT( p_vend->p_log ); - return( p_mad ); + OSM_LOG_EXIT(p_vend->p_log); + return (p_mad); } /********************************************************************** **********************************************************************/ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vend_wrap, - IN ib_mad_t* const p_mad ) +osm_vendor_put(IN osm_bind_handle_t h_bind, + IN osm_vend_wrap_t * const p_vend_wrap, + IN ib_mad_t * const p_mad) { - osm_vendor_t *p_vend; + osm_vendor_t *p_vend; - OSM_LOG_ENTER( h_bind->p_vend->p_log, osm_vendor_put ); + OSM_LOG_ENTER(h_bind->p_vend->p_log, osm_vendor_put); - UNUSED_PARAM( p_vend_wrap ); + UNUSED_PARAM(p_vend_wrap); - p_vend = h_bind->p_vend; + p_vend = h_bind->p_vend; - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "osm_vendor_put: " - "MAD %p.\n", p_mad ); + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "osm_vendor_put: " "MAD %p.\n", p_mad); - /* - Return the MAD to the heap. - */ - free( p_mad ); + /* + Return the MAD to the heap. + */ + free(p_mad); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vend_wrap, - IN osm_mad_addr_t* const p_mad_addr, - IN ib_mad_t* const p_mad, - IN void *transaction_context, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_vend_wrap_t * const p_vend_wrap, + IN osm_mad_addr_t * const p_mad_addr, + IN ib_mad_t * const p_mad, + IN void *transaction_context, IN boolean_t const resp_expected) { - osm_vendor_t *p_vend = h_bind->p_vend; + osm_vendor_t *p_vend = h_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_send ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_send); - UNUSED_PARAM( p_vend_wrap ); - UNUSED_PARAM( p_mad_addr ); - UNUSED_PARAM( transaction_context ); - UNUSED_PARAM( resp_expected ); + UNUSED_PARAM(p_vend_wrap); + UNUSED_PARAM(p_mad_addr); + UNUSED_PARAM(transaction_context); + UNUSED_PARAM(resp_expected); - osm_log( p_vend->p_log, OSM_LOG_VERBOSE, - "osm_vendor_send: " - "MAD %p.\n", p_mad ); + osm_log(p_vend->p_log, OSM_LOG_VERBOSE, + "osm_vendor_send: " "MAD %p.\n", p_mad); - OSM_LOG_EXIT( p_vend->p_log ); - return( IB_SUCCESS ); + OSM_LOG_EXIT(p_vend->p_log); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_bind_info, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN void* context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_bind_info, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN void *context) { - osm_bind_handle_t h_bind; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_bind ); - - CL_ASSERT( p_vend ); - CL_ASSERT( p_bind_info ); - CL_ASSERT( p_mad_pool ); - CL_ASSERT( mad_recv_callback ); - CL_ASSERT( context ); - - UNUSED_PARAM( p_vend ); - UNUSED_PARAM( p_mad_pool ); - UNUSED_PARAM( mad_recv_callback ); - UNUSED_PARAM( context ); - - h_bind = (osm_bind_handle_t)malloc(sizeof(*h_bind) ); - if( h_bind != NULL ) - { - memset(h_bind, 0, sizeof(*h_bind)); - h_bind->p_vend = p_vend; - h_bind->port_guid = p_bind_info->port_guid; - h_bind->mad_class = p_bind_info->mad_class; - h_bind->class_version = p_bind_info->class_version; - h_bind->is_responder = p_bind_info->is_responder; - h_bind->is_trap_processor = p_bind_info->is_trap_processor; - h_bind->is_report_processor = p_bind_info->is_report_processor; - h_bind->send_q_size = p_bind_info->send_q_size; - h_bind->recv_q_size = p_bind_info->recv_q_size; - } - - OSM_LOG_EXIT( p_vend->p_log ); - return( h_bind ); + osm_bind_handle_t h_bind; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_bind); + + CL_ASSERT(p_vend); + CL_ASSERT(p_bind_info); + CL_ASSERT(p_mad_pool); + CL_ASSERT(mad_recv_callback); + CL_ASSERT(context); + + UNUSED_PARAM(p_vend); + UNUSED_PARAM(p_mad_pool); + UNUSED_PARAM(mad_recv_callback); + UNUSED_PARAM(context); + + h_bind = (osm_bind_handle_t) malloc(sizeof(*h_bind)); + if (h_bind != NULL) { + memset(h_bind, 0, sizeof(*h_bind)); + h_bind->p_vend = p_vend; + h_bind->port_guid = p_bind_info->port_guid; + h_bind->mad_class = p_bind_info->mad_class; + h_bind->class_version = p_bind_info->class_version; + h_bind->is_responder = p_bind_info->is_responder; + h_bind->is_trap_processor = p_bind_info->is_trap_processor; + h_bind->is_report_processor = p_bind_info->is_report_processor; + h_bind->send_q_size = p_bind_info->send_q_size; + h_bind->recv_q_size = p_bind_info->recv_q_size; + } + + OSM_LOG_EXIT(p_vend->p_log); + return (h_bind); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vendor_get_ports( - IN osm_vendor_t* const p_vend, - IN ib_net64_t* const p_guids, - IN uint32_t* const num_guids ) +osm_vendor_get_ports(IN osm_vendor_t * const p_vend, + IN ib_net64_t * const p_guids, + IN uint32_t * const num_guids) { - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get_ports ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get_ports); - *p_guids = CL_NTOH64( 0x0000000000001234 ); - *num_guids = 1; + *p_guids = CL_NTOH64(0x0000000000001234); + *num_guids = 1; - OSM_LOG_EXIT( p_vend->p_log ); - return( IB_SUCCESS ); + OSM_LOG_EXIT(p_vend->p_log); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_vendor_local_lid_change( - IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_vendor_t *p_vend = h_bind->p_vend; + osm_vendor_t *p_vend = h_bind->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); - return( IB_SUCCESS ); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } -#endif /* OSM_VENDOR_INTF_TEST */ +#endif /* OSM_VENDOR_INTF_TEST */ diff --git a/opensm/libvendor/osm_vendor_ts.c b/opensm/libvendor/osm_vendor_ts.c index 763073b..04b4331 100644 --- a/opensm/libvendor/osm_vendor_ts.c +++ b/opensm/libvendor/osm_vendor_ts.c @@ -33,11 +33,10 @@ * */ - #undef __init #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -55,179 +54,165 @@ not fail, and then the madw_p will be put back in the pool before the callback. */ -uint64_t -__osm_set_wrid_by_p_madw(IN osm_madw_t *p_madw) { - uint64_t wrid = 0; +uint64_t __osm_set_wrid_by_p_madw(IN osm_madw_t * p_madw) +{ + uint64_t wrid = 0; - CL_ASSERT(p_madw->p_mad); + CL_ASSERT(p_madw->p_mad); - memcpy( &wrid, &p_madw, sizeof( osm_madw_t * ) ); - wrid = (wrid << 1) | - ib_mad_is_response(p_madw->p_mad) | - (p_madw->p_mad->method == IB_MAD_METHOD_TRAP_REPRESS); - return wrid; + memcpy(&wrid, &p_madw, sizeof(osm_madw_t *)); + wrid = (wrid << 1) | + ib_mad_is_response(p_madw->p_mad) | + (p_madw->p_mad->method == IB_MAD_METHOD_TRAP_REPRESS); + return wrid; } void -__osm_set_p_madw_and_resp_by_wrid( - IN uint64_t wrid, - OUT uint8_t *is_resp, - OUT osm_madw_t **pp_madw) { - *is_resp = wrid & 0x0000000000000001; - wrid = wrid >> 1; - memcpy( pp_madw, &wrid, sizeof( osm_madw_t * ) ); +__osm_set_p_madw_and_resp_by_wrid(IN uint64_t wrid, + OUT uint8_t * is_resp, + OUT osm_madw_t ** pp_madw) +{ + *is_resp = wrid & 0x0000000000000001; + wrid = wrid >> 1; + memcpy(pp_madw, &wrid, sizeof(osm_madw_t *)); } /********************************************************************** * TS MAD to OSM ADDRESS VECTOR **********************************************************************/ void -__osm_ts_conv_mad_rcv_desc_to_osm_addr( - IN osm_vendor_t * const p_vend, - IN struct ib_mad *p_mad, - IN uint8_t is_smi, - OUT osm_mad_addr_t *p_mad_addr) +__osm_ts_conv_mad_rcv_desc_to_osm_addr(IN osm_vendor_t * const p_vend, + IN struct ib_mad *p_mad, + IN uint8_t is_smi, + OUT osm_mad_addr_t * p_mad_addr) { - p_mad_addr->dest_lid = cl_hton16(p_mad->slid); - p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ - p_mad_addr->path_bits = 0; /* HACK - no way to know in TS */ - if (is_smi) - { - /* SMI */ - p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_mad->slid); - p_mad_addr->addr_type.smi.port_num = p_mad->port; - } - else - { - /* GSI */ - p_mad_addr->addr_type.gsi.remote_qp = p_mad->sqpn; - p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ - /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ - /* the full PKey table - than go by the index. */ - /* since this does not seem reasonable to me I simply use the default */ - /* There is a TAVOR limitation that only one P_KEY is supported per */ - /* QP - so QP1 must use IB_DEFAULT_PKEY */ - p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; - p_mad_addr->addr_type.gsi.service_level = 0; /* HACK no way to know */ - - p_mad_addr->addr_type.gsi.global_route = FALSE; /* HACK no way to know */ - /* copy the GRH data if relevant */ - /* - if (p_mad_addr->addr_type.gsi.global_route) - { - p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = - ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, - p_rcv_desc->grh.traffic_class, - p_rcv_desc->grh.flow_label); - p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; - memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, - &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); - memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, - p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); - } - */ - } + p_mad_addr->dest_lid = cl_hton16(p_mad->slid); + p_mad_addr->static_rate = 0; /* HACK - we do not know the rate ! */ + p_mad_addr->path_bits = 0; /* HACK - no way to know in TS */ + if (is_smi) { + /* SMI */ + p_mad_addr->addr_type.smi.source_lid = cl_hton16(p_mad->slid); + p_mad_addr->addr_type.smi.port_num = p_mad->port; + } else { + /* GSI */ + p_mad_addr->addr_type.gsi.remote_qp = p_mad->sqpn; + p_mad_addr->addr_type.gsi.remote_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + /* we do have the p_mad_addr->pkey_ix but how to get the PKey by index ? */ + /* the only way seems to be to use VAPI_query_hca_pkey_tbl and obtain */ + /* the full PKey table - than go by the index. */ + /* since this does not seem reasonable to me I simply use the default */ + /* There is a TAVOR limitation that only one P_KEY is supported per */ + /* QP - so QP1 must use IB_DEFAULT_PKEY */ + p_mad_addr->addr_type.gsi.pkey = IB_DEFAULT_PKEY; + p_mad_addr->addr_type.gsi.service_level = 0; /* HACK no way to know */ + + p_mad_addr->addr_type.gsi.global_route = FALSE; /* HACK no way to know */ + /* copy the GRH data if relevant */ + /* + if (p_mad_addr->addr_type.gsi.global_route) + { + p_mad_addr->addr_type.gsi.grh_info.ver_class_flow = + ib_grh_set_ver_class_flow(p_rcv_desc->grh.IP_version, + p_rcv_desc->grh.traffic_class, + p_rcv_desc->grh.flow_label); + p_mad_addr->addr_type.gsi.grh_info.hop_limit = p_rcv_desc->grh.hop_limit; + memcpy(&p_mad_addr->addr_type.gsi.grh_info.src_gid.raw, + &p_rcv_desc->grh.sgid, sizeof(ib_net64_t)); + memcpy(&p_mad_addr->addr_type.gsi.grh_info.dest_gid.raw, + p_rcv_desc->grh.dgid, sizeof(ib_net64_t)); + } + */ + } } /********************************************************************** * OSM ADDR VECTOR TO TS MAD: **********************************************************************/ void -__osm_ts_conv_osm_addr_to_ts_addr( - IN osm_mad_addr_t *p_mad_addr, - IN uint8_t is_smi, - OUT struct ib_mad *p_mad) +__osm_ts_conv_osm_addr_to_ts_addr(IN osm_mad_addr_t * p_mad_addr, + IN uint8_t is_smi, OUT struct ib_mad *p_mad) { - /* For global destination or Multicast address:*/ - p_mad->dlid = cl_ntoh16(p_mad_addr->dest_lid); - p_mad->sl = 0; - if (is_smi) - { - p_mad->sqpn = 0; - p_mad->dqpn = 0; - } - else - { - p_mad->sqpn = 1; - p_mad->dqpn = p_mad_addr->addr_type.gsi.remote_qp; - } + /* For global destination or Multicast address: */ + p_mad->dlid = cl_ntoh16(p_mad_addr->dest_lid); + p_mad->sl = 0; + if (is_smi) { + p_mad->sqpn = 0; + p_mad->dqpn = 0; + } else { + p_mad->sqpn = 1; + p_mad->dqpn = p_mad_addr->addr_type.gsi.remote_qp; + } } /********************************************************************** **********************************************************************/ -void -__osm_vendor_clear_sm( IN osm_bind_handle_t h_bind ) +void __osm_vendor_clear_sm(IN osm_bind_handle_t h_bind) { - osm_ts_bind_info_t *p_bind = ( osm_ts_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - VAPI_ret_t status; - VAPI_hca_attr_t attr_mod; - VAPI_hca_attr_mask_t attr_mask; - - OSM_LOG_ENTER( p_vend->p_log, __osm_vendor_clear_sm ); - - memset( &attr_mod, 0, sizeof( attr_mod ) ); - memset( &attr_mask, 0, sizeof( attr_mask ) ); - - attr_mod.is_sm = FALSE; - attr_mask = HCA_ATTR_IS_SM; - - status = - VAPI_modify_hca_attr( p_bind->hca_hndl, p_bind->port_num, &attr_mod, - &attr_mask ); - if( status != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_clear_sm: ERR 5021: " - "Unable set 'IS_SM' bit in port attributes (%d).\n", - status ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_ts_bind_info_t *p_bind = (osm_ts_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + VAPI_ret_t status; + VAPI_hca_attr_t attr_mod; + VAPI_hca_attr_mask_t attr_mask; + + OSM_LOG_ENTER(p_vend->p_log, __osm_vendor_clear_sm); + + memset(&attr_mod, 0, sizeof(attr_mod)); + memset(&attr_mask, 0, sizeof(attr_mask)); + + attr_mod.is_sm = FALSE; + attr_mask = HCA_ATTR_IS_SM; + + status = + VAPI_modify_hca_attr(p_bind->hca_hndl, p_bind->port_num, &attr_mod, + &attr_mask); + if (status != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_clear_sm: ERR 5021: " + "Unable set 'IS_SM' bit in port attributes (%d).\n", + status); + } + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** * ANY CONSTRUCTION OF THE osm_vendor_t OBJECT **********************************************************************/ -void -osm_vendor_construct( IN osm_vendor_t * const p_vend ) +void osm_vendor_construct(IN osm_vendor_t * const p_vend) { - memset( p_vend, 0, sizeof( *p_vend ) ); - cl_thread_construct( &(p_vend->smi_bind.poller) ); - cl_thread_construct( &(p_vend->gsi_bind.poller) ); + memset(p_vend, 0, sizeof(*p_vend)); + cl_thread_construct(&(p_vend->smi_bind.poller)); + cl_thread_construct(&(p_vend->gsi_bind.poller)); } /********************************************************************** * DEALOCATE osm_vendor_t **********************************************************************/ -void -osm_vendor_destroy( IN osm_vendor_t * const p_vend ) +void osm_vendor_destroy(IN osm_vendor_t * const p_vend) { - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_destroy ); - osm_transaction_mgr_destroy ( p_vend ); - - /* Destroy the poller threads */ - /* HACK: can you destroy an un-initialized thread ? */ - pthread_cancel(p_vend->smi_bind.poller.osd.id); - pthread_cancel(p_vend->gsi_bind.poller.osd.id); - cl_thread_destroy( &(p_vend->smi_bind.poller) ); - cl_thread_destroy( &(p_vend->gsi_bind.poller) ); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_destroy); + osm_transaction_mgr_destroy(p_vend); + + /* Destroy the poller threads */ + /* HACK: can you destroy an un-initialized thread ? */ + pthread_cancel(p_vend->smi_bind.poller.osd.id); + pthread_cancel(p_vend->gsi_bind.poller.osd.id); + cl_thread_destroy(&(p_vend->smi_bind.poller)); + cl_thread_destroy(&(p_vend->gsi_bind.poller)); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** DEALLOCATE A POINTER TO osm_vendor_t **********************************************************************/ -void -osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - CL_ASSERT( pp_vend ); + CL_ASSERT(pp_vend); - osm_vendor_destroy( *pp_vend ); - free( *pp_vend ); - *pp_vend = NULL; + osm_vendor_destroy(*pp_vend); + free(*pp_vend); + *pp_vend = NULL; } /********************************************************************** @@ -235,62 +220,55 @@ osm_vendor_delete( IN osm_vendor_t ** const pp_vend ) **********************************************************************/ ib_api_status_t -osm_vendor_init( IN osm_vendor_t * const p_vend, - IN osm_log_t * const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_vendor_init ); + OSM_LOG_ENTER(p_log, osm_vendor_init); - p_vend->p_log = p_log; - p_vend->p_transaction_mgr = NULL; - osm_transaction_mgr_init( p_vend ); - p_vend->timeout = timeout; + p_vend->p_log = p_log; + p_vend->p_transaction_mgr = NULL; + osm_transaction_mgr_init(p_vend); + p_vend->timeout = timeout; - /* we use the file handle to track the binding */ - p_vend->smi_bind.ul_dev_fd = -1; - p_vend->gsi_bind.ul_dev_fd = -1; + /* we use the file handle to track the binding */ + p_vend->smi_bind.ul_dev_fd = -1; + p_vend->gsi_bind.ul_dev_fd = -1; - OSM_LOG_EXIT( p_log ); - return ( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** * Create and Initialize osm_vendor_t Object **********************************************************************/ -osm_vendor_t * -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) { - ib_api_status_t status; - osm_vendor_t *p_vend; - - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - CL_ASSERT( p_log ); - - p_vend = malloc( sizeof( *p_vend ) ); - if( p_vend != NULL ) - { - memset( p_vend, 0, sizeof( *p_vend ) ); - - status = osm_vendor_init( p_vend, p_log, timeout ); - if( status != IB_SUCCESS ) - { - osm_vendor_delete( &p_vend ); - } - } - else - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_new: ERR 5007: " - "Fail to allocate vendor object.\n" ); - } - - OSM_LOG_EXIT( p_log ); - return ( p_vend ); + ib_api_status_t status; + osm_vendor_t *p_vend; + + OSM_LOG_ENTER(p_log, osm_vendor_new); + + CL_ASSERT(p_log); + + p_vend = malloc(sizeof(*p_vend)); + if (p_vend != NULL) { + memset(p_vend, 0, sizeof(*p_vend)); + + status = osm_vendor_init(p_vend, p_log, timeout); + if (status != IB_SUCCESS) { + osm_vendor_delete(&p_vend); + } + } else { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_new: ERR 5007: " + "Fail to allocate vendor object.\n"); + } + + OSM_LOG_EXIT(p_log); + return (p_vend); } /********************************************************************** @@ -298,111 +276,104 @@ osm_vendor_new( * HACK: - we need to make this support arbitrary size mads. **********************************************************************/ void -__osm_ts_rcv_callback( - IN osm_ts_bind_info_t *p_bind, - IN osm_mad_addr_t *p_mad_addr, - IN uint32_t mad_size, - IN void *p_mad ) +__osm_ts_rcv_callback(IN osm_ts_bind_info_t * p_bind, + IN osm_mad_addr_t * p_mad_addr, + IN uint32_t mad_size, IN void *p_mad) { - ib_api_status_t status; - osm_madw_t *p_req_madw = NULL; - osm_madw_t *p_madw; - osm_vend_wrap_t* p_new_vw; - ib_mad_t *p_mad_buf; - osm_log_t *const p_log = p_bind->p_vend->p_log; - - OSM_LOG_ENTER( p_log, __osm_ts_rcv_callback ); - - /* if it is a response MAD we mustbe able to get the request */ - if (ib_mad_is_response((ib_mad_t*)p_mad)) - { - /* can we find a matching madw by this payload TID */ - status = - osm_transaction_mgr_get_madw_for_tid( - p_bind->p_vend, (ib_mad_t*)p_mad, - &p_req_madw ); - if ( status != IB_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_rcv_callback: ERR 5008: " - "Error obtaining request madw by TID (%d).\n", status ); - p_req_madw = NULL; - } - - if (p_req_madw == NULL) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_rcv_callback: ERR 5009: " - "Fail to obtain request madw for receined MAD. Aborting CB.\n"); - goto Exit; - } - } - - /* do we have a request ??? */ - if( p_req_madw == NULL ) - { - - /* if not - get new osm_madw and arrange it. */ - /* create the new madw in the pool */ - p_madw = osm_mad_pool_get( p_bind->p_osm_pool, - ( osm_bind_handle_t ) p_bind, - mad_size, p_mad_addr ); - if( p_madw == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_rcv_callback: ERR 5010: " - "Error request for a new madw.\n" ); - goto Exit; - } - /* HACK: we cust to avoid the const ??? */ - p_mad_buf = (void *)p_madw->p_mad; - } - else - { - /* we have the madw defined during the send and stored in the vend_wrap */ - /* we need to make sure the wrapper is correctly init there */ - CL_ASSERT(p_req_madw->vend_wrap.p_resp_madw != 0); - p_madw = p_req_madw->vend_wrap.p_resp_madw; - - CL_ASSERT(p_madw->h_bind); - p_mad_buf = osm_vendor_get(p_madw->h_bind, mad_size, &p_madw->vend_wrap ); - - if( p_mad_buf == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_rcv_callback: ERR 5011: " - "Unable to acquire wire MAD.\n" ); - - goto Exit; - } - - /* - Finally, attach the wire MAD to this wrapper. - */ - osm_madw_set_mad( p_madw, p_mad_buf ); - } - - /* init some fields of the vendor wrapper */ - p_new_vw = osm_madw_get_vend_ptr( p_madw ); - p_new_vw->h_bind = p_bind; - p_new_vw->size = mad_size; - p_new_vw->p_resp_madw = NULL; - p_new_vw->p_mad_buf = p_mad_buf; - - memcpy( p_new_vw->p_mad_buf, p_mad, mad_size ); - - /* attach the buffer to the wrapper */ - p_madw->p_mad = p_mad_buf; - - /* we can also make sure we marked the size and bind on the returned madw */ - p_madw->h_bind = p_new_vw->h_bind; - - /* call the CB */ - ( *(osm_vend_mad_recv_callback_t)p_bind->rcv_callback ) - ( p_madw, p_bind->client_context, p_req_madw ); - - Exit: - OSM_LOG_EXIT( p_log ); + ib_api_status_t status; + osm_madw_t *p_req_madw = NULL; + osm_madw_t *p_madw; + osm_vend_wrap_t *p_new_vw; + ib_mad_t *p_mad_buf; + osm_log_t *const p_log = p_bind->p_vend->p_log; + + OSM_LOG_ENTER(p_log, __osm_ts_rcv_callback); + + /* if it is a response MAD we mustbe able to get the request */ + if (ib_mad_is_response((ib_mad_t *) p_mad)) { + /* can we find a matching madw by this payload TID */ + status = + osm_transaction_mgr_get_madw_for_tid(p_bind->p_vend, + (ib_mad_t *) p_mad, + &p_req_madw); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_rcv_callback: ERR 5008: " + "Error obtaining request madw by TID (%d).\n", + status); + p_req_madw = NULL; + } + + if (p_req_madw == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_rcv_callback: ERR 5009: " + "Fail to obtain request madw for receined MAD. Aborting CB.\n"); + goto Exit; + } + } + + /* do we have a request ??? */ + if (p_req_madw == NULL) { + + /* if not - get new osm_madw and arrange it. */ + /* create the new madw in the pool */ + p_madw = osm_mad_pool_get(p_bind->p_osm_pool, + (osm_bind_handle_t) p_bind, + mad_size, p_mad_addr); + if (p_madw == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_rcv_callback: ERR 5010: " + "Error request for a new madw.\n"); + goto Exit; + } + /* HACK: we cust to avoid the const ??? */ + p_mad_buf = (void *)p_madw->p_mad; + } else { + /* we have the madw defined during the send and stored in the vend_wrap */ + /* we need to make sure the wrapper is correctly init there */ + CL_ASSERT(p_req_madw->vend_wrap.p_resp_madw != 0); + p_madw = p_req_madw->vend_wrap.p_resp_madw; + + CL_ASSERT(p_madw->h_bind); + p_mad_buf = + osm_vendor_get(p_madw->h_bind, mad_size, + &p_madw->vend_wrap); + + if (p_mad_buf == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_rcv_callback: ERR 5011: " + "Unable to acquire wire MAD.\n"); + + goto Exit; + } + + /* + Finally, attach the wire MAD to this wrapper. + */ + osm_madw_set_mad(p_madw, p_mad_buf); + } + + /* init some fields of the vendor wrapper */ + p_new_vw = osm_madw_get_vend_ptr(p_madw); + p_new_vw->h_bind = p_bind; + p_new_vw->size = mad_size; + p_new_vw->p_resp_madw = NULL; + p_new_vw->p_mad_buf = p_mad_buf; + + memcpy(p_new_vw->p_mad_buf, p_mad, mad_size); + + /* attach the buffer to the wrapper */ + p_madw->p_mad = p_mad_buf; + + /* we can also make sure we marked the size and bind on the returned madw */ + p_madw->h_bind = p_new_vw->h_bind; + + /* call the CB */ + (*(osm_vend_mad_recv_callback_t) p_bind->rcv_callback) + (p_madw, p_bind->client_context, p_req_madw); + + Exit: + OSM_LOG_EXIT(p_log); } /********************************************************************** @@ -410,140 +381,131 @@ __osm_ts_rcv_callback( * **********************************************************************/ void -__osm_ts_send_callback( - IN osm_ts_bind_info_t *bind_info_p, - IN boolean_t is_resp, - IN osm_madw_t *madw_p, - IN IB_comp_status_t status) +__osm_ts_send_callback(IN osm_ts_bind_info_t * bind_info_p, + IN boolean_t is_resp, + IN osm_madw_t * madw_p, IN IB_comp_status_t status) { - osm_log_t *const p_log = bind_info_p->p_vend->p_log; - osm_vend_wrap_t* p_vw; - - OSM_LOG_ENTER( p_log, __osm_ts_send_callback ); - - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_ts_send_callback: INFO 1008: " - "Handling Send of MADW:%p Is Resp:%d.\n", madw_p, is_resp); - - /* we need to handle requests and responses differently */ - if (is_resp) - { - if (status != IB_COMP_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_send_callback: ERR 5012: " - "Error Sending Response MADW:%p.\n", madw_p); - } - else - { - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_ts_send_callback: DBG 1008: " - "Completed Sending Response MADW:%p.\n", madw_p); - } - - /* if we are a response - we need to clean it up */ - osm_mad_pool_put( bind_info_p->p_osm_pool, madw_p); - } - else - { - - /* this call back is invoked on completion of send - error or not */ - if (status != IB_COMP_SUCCESS) - { - - osm_log( p_log, OSM_LOG_ERROR, - "__osm_ts_send_callback: ERR 5013: " - "Received an Error from IB_MGT Send (%d).\n",status ); - - p_vw = osm_madw_get_vend_ptr( madw_p ); - CL_ASSERT( p_vw ); - - /* - Return any wrappers to the pool that may have been - pre-emptively allocated to handle a receive. - */ - if( p_vw->p_resp_madw ) - { - osm_mad_pool_put( bind_info_p->p_osm_pool, p_vw->p_resp_madw ); - p_vw->p_resp_madw = NULL; - } - - /* invoke the CB */ - ( *(osm_vend_mad_send_err_callback_t)bind_info_p->send_err_callback ) - ( bind_info_p->client_context, madw_p ); - } - else - { - /* successful request send - do nothing - the response will need the - out mad */ - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_ts_send_callback: DBG 1008: " - "Completed Sending Request MADW:%p.\n", madw_p); - } - } - - OSM_LOG_EXIT( p_log ); + osm_log_t *const p_log = bind_info_p->p_vend->p_log; + osm_vend_wrap_t *p_vw; + + OSM_LOG_ENTER(p_log, __osm_ts_send_callback); + + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_ts_send_callback: INFO 1008: " + "Handling Send of MADW:%p Is Resp:%d.\n", madw_p, is_resp); + + /* we need to handle requests and responses differently */ + if (is_resp) { + if (status != IB_COMP_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_send_callback: ERR 5012: " + "Error Sending Response MADW:%p.\n", madw_p); + } else { + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_ts_send_callback: DBG 1008: " + "Completed Sending Response MADW:%p.\n", + madw_p); + } + + /* if we are a response - we need to clean it up */ + osm_mad_pool_put(bind_info_p->p_osm_pool, madw_p); + } else { + + /* this call back is invoked on completion of send - error or not */ + if (status != IB_COMP_SUCCESS) { + + osm_log(p_log, OSM_LOG_ERROR, + "__osm_ts_send_callback: ERR 5013: " + "Received an Error from IB_MGT Send (%d).\n", + status); + + p_vw = osm_madw_get_vend_ptr(madw_p); + CL_ASSERT(p_vw); + + /* + Return any wrappers to the pool that may have been + pre-emptively allocated to handle a receive. + */ + if (p_vw->p_resp_madw) { + osm_mad_pool_put(bind_info_p->p_osm_pool, + p_vw->p_resp_madw); + p_vw->p_resp_madw = NULL; + } + + /* invoke the CB */ + (*(osm_vend_mad_send_err_callback_t) bind_info_p-> + send_err_callback) + (bind_info_p->client_context, madw_p); + } else { + /* successful request send - do nothing - the response will need the + out mad */ + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_ts_send_callback: DBG 1008: " + "Completed Sending Request MADW:%p.\n", madw_p); + } + } + + OSM_LOG_EXIT(p_log); } /********************************************************************** * Poller thread: * Always receive 256byte mads from the devcie file **********************************************************************/ -void -__osm_vendor_ts_poller( - IN void *p_ptr ) +void __osm_vendor_ts_poller(IN void *p_ptr) { - int ts_ret_code; - struct ib_mad mad; - osm_mad_addr_t mad_addr; - osm_ts_bind_info_t * const p_bind = (osm_ts_bind_info_t *)p_ptr; - - OSM_LOG_ENTER( p_bind->p_vend->p_log, __osm_vendor_ts_poller ); - /* we set the type of cancelation for this thread */ - pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); - - while( 1 ) - { - /* we read one mad at a time and pass it to the read callback function */ - ts_ret_code = read(p_bind->ul_dev_fd, &mad, sizeof(mad)); - if (ts_ret_code != sizeof(mad)) - { - osm_log( p_bind->p_vend->p_log, OSM_LOG_ERROR, - "__osm_vendor_ts_poller: ERR 5003: " - "error with read, bytes = %d, errno = %d\n", - ts_ret_code, errno - ); - } - else - { - osm_log( p_bind->p_vend->p_log, OSM_LOG_DEBUG, - "__osm_vendor_ts_poller: " - "MAD QPN:%d SLID:0x%04x class:0x%02x " - "__osm_vendor_ts_poller:0x%02x attr:0x%04x status:0x%04x " - "__osm_vendor_ts_poller:0x%016" PRIx64 "\n", - cl_ntoh32(mad.dqpn), - cl_ntoh16(mad.slid), - mad.mgmt_class, - mad.r_method, - cl_ntoh16(mad.attribute_id), - cl_ntoh16(mad.status), - cl_ntoh64(mad.transaction_id)); - - /* first arrange an address */ - __osm_ts_conv_mad_rcv_desc_to_osm_addr( - p_bind->p_vend, - &mad, - ( ((ib_mad_t*)&mad)->mgmt_class == IB_MCLASS_SUBN_LID ) || - ( ((ib_mad_t*)&mad)->mgmt_class == IB_MCLASS_SUBN_DIR ), - &mad_addr); - - /* call the receiver callback */ - /* HACK: this should be replaced with a call to the RMPP Assembly ... */ - __osm_ts_rcv_callback(p_bind, &mad_addr, 256, &mad); - } - } - - OSM_LOG_EXIT( p_bind->p_vend->p_log ); + int ts_ret_code; + struct ib_mad mad; + osm_mad_addr_t mad_addr; + osm_ts_bind_info_t *const p_bind = (osm_ts_bind_info_t *) p_ptr; + + OSM_LOG_ENTER(p_bind->p_vend->p_log, __osm_vendor_ts_poller); + /* we set the type of cancelation for this thread */ + pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); + + while (1) { + /* we read one mad at a time and pass it to the read callback function */ + ts_ret_code = read(p_bind->ul_dev_fd, &mad, sizeof(mad)); + if (ts_ret_code != sizeof(mad)) { + osm_log(p_bind->p_vend->p_log, OSM_LOG_ERROR, + "__osm_vendor_ts_poller: ERR 5003: " + "error with read, bytes = %d, errno = %d\n", + ts_ret_code, errno); + } else { + osm_log(p_bind->p_vend->p_log, OSM_LOG_DEBUG, + "__osm_vendor_ts_poller: " + "MAD QPN:%d SLID:0x%04x class:0x%02x " + "__osm_vendor_ts_poller:0x%02x attr:0x%04x status:0x%04x " + "__osm_vendor_ts_poller:0x%016" PRIx64 "\n", + cl_ntoh32(mad.dqpn), + cl_ntoh16(mad.slid), + mad.mgmt_class, + mad.r_method, + cl_ntoh16(mad.attribute_id), + cl_ntoh16(mad.status), + cl_ntoh64(mad.transaction_id)); + + /* first arrange an address */ + __osm_ts_conv_mad_rcv_desc_to_osm_addr(p_bind->p_vend, + &mad, + (((ib_mad_t *) & + mad)-> + mgmt_class == + IB_MCLASS_SUBN_LID) + || + (((ib_mad_t *) & + mad)-> + mgmt_class == + IB_MCLASS_SUBN_DIR), + &mad_addr); + + /* call the receiver callback */ + /* HACK: this should be replaced with a call to the RMPP Assembly ... */ + __osm_ts_rcv_callback(p_bind, &mad_addr, 256, &mad); + } + } + + OSM_LOG_EXIT(p_bind->p_vend->p_log); } /********************************************************************** @@ -551,236 +513,216 @@ __osm_vendor_ts_poller( * defined by the given: osm_bind_info_t **********************************************************************/ osm_bind_handle_t -osm_vendor_bind( IN osm_vendor_t * const p_vend, - IN osm_bind_info_t * const p_user_bind, - IN osm_mad_pool_t * const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN osm_vend_mad_send_err_callback_t send_err_callback, - IN void *context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_user_bind, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN osm_vend_mad_send_err_callback_t send_err_callback, + IN void *context) { - ib_net64_t port_guid; - osm_ts_bind_info_t *p_bind = NULL; - VAPI_hca_hndl_t hca_hndl; - VAPI_hca_id_t hca_id; - uint32_t port_num; - ib_api_status_t status; - int device_fd; - char device_file[16]; - osm_ts_user_mad_filter filter; - int ts_ioctl_ret; - int qpn; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_bind ); - - CL_ASSERT( p_mad_pool ); - - port_guid = p_user_bind->port_guid; - - osm_log( p_vend->p_log, OSM_LOG_INFO, - "osm_vendor_bind: " - "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64( port_guid ) ); - - switch ( p_user_bind->mad_class ) - { - case IB_MCLASS_SUBN_LID: - case IB_MCLASS_SUBN_DIR: - p_bind = &(p_vend->smi_bind); - qpn = 0; - break; - - case IB_MCLASS_SUBN_ADM: - default: - p_bind = &(p_vend->gsi_bind); - qpn = 1; - break; - } - - /* Make sure we did not previously opened the file */ - if (p_bind->ul_dev_fd >= 0) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5004: " - "Already binded to port %u\n", - p_bind->port_num - ); - goto Exit; - } - - /* - We need to figure out what is the TS file name to attach to. - I guess it is following the index of the port in the table of - ports. - */ - - /* obtain the hca name and port num from the guid */ - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_bind: " - "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", - cl_ntoh64(port_guid) ); - status = - osm_vendor_get_guid_ca_and_port( - p_vend, port_guid, &hca_hndl, &hca_id, &port_num); - if (status != IB_SUCCESS) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5005: " - "Fail to find port number of port guid:0x%016"PRIx64"\n", - port_guid - ); - goto Exit; - } - - /* the file name is just /dev/ts_ua0: */ - strcpy(device_file, "/dev/ts_ua0"); - - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: " - "Opening TS UL dev file:%s\n", - device_file - ); - - /* Open the file ... */ - device_fd = open(device_file, O_RDWR); - if (device_fd < 0) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5006: " - "Fail to open TS UL dev file:%s\n", - device_file - ); - goto Exit; - } - - /* track this bind request info */ - p_bind->ul_dev_fd = device_fd; - p_bind->port_num = port_num; - p_bind->p_vend = p_vend; - p_bind->client_context = context; - p_bind->rcv_callback = mad_recv_callback; - p_bind->send_err_callback = send_err_callback; - p_bind->p_osm_pool = p_mad_pool; - p_bind->hca_hndl = hca_hndl; - - /* - * Create the MAD filter on this file handle. - */ - filter.port = port_num; - - filter.qpn = qpn; - filter.mgmt_class = p_user_bind->mad_class; - filter.direction = TS_IB_MAD_DIRECTION_IN; - filter.mask = - TS_IB_MAD_FILTER_DIRECTION | - TS_IB_MAD_FILTER_PORT | - TS_IB_MAD_FILTER_QPN | - TS_IB_MAD_FILTER_MGMT_CLASS; - - ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &filter); - if (ts_ioctl_ret < 0) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_bind: ERR 5014: " - "Fail to register MAD filter with err:%u\n", - ts_ioctl_ret - ); - goto Exit; - } - - /* Initialize the listener thread for this port */ - status = cl_thread_init( &p_bind->poller, - __osm_vendor_ts_poller, p_bind, - "osm ts poller" ); - if( status != IB_SUCCESS ) - goto Exit; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( ( osm_bind_handle_t ) p_bind ); + ib_net64_t port_guid; + osm_ts_bind_info_t *p_bind = NULL; + VAPI_hca_hndl_t hca_hndl; + VAPI_hca_id_t hca_id; + uint32_t port_num; + ib_api_status_t status; + int device_fd; + char device_file[16]; + osm_ts_user_mad_filter filter; + int ts_ioctl_ret; + int qpn; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_bind); + + CL_ASSERT(p_mad_pool); + + port_guid = p_user_bind->port_guid; + + osm_log(p_vend->p_log, OSM_LOG_INFO, + "osm_vendor_bind: " + "Binding to port 0x%" PRIx64 ".\n", cl_ntoh64(port_guid)); + + switch (p_user_bind->mad_class) { + case IB_MCLASS_SUBN_LID: + case IB_MCLASS_SUBN_DIR: + p_bind = &(p_vend->smi_bind); + qpn = 0; + break; + + case IB_MCLASS_SUBN_ADM: + default: + p_bind = &(p_vend->gsi_bind); + qpn = 1; + break; + } + + /* Make sure we did not previously opened the file */ + if (p_bind->ul_dev_fd >= 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 5004: " + "Already binded to port %u\n", p_bind->port_num); + goto Exit; + } + + /* + We need to figure out what is the TS file name to attach to. + I guess it is following the index of the port in the table of + ports. + */ + + /* obtain the hca name and port num from the guid */ + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_bind: " + "Finding CA and Port that owns port guid 0x%" PRIx64 ".\n", + cl_ntoh64(port_guid)); + status = + osm_vendor_get_guid_ca_and_port(p_vend, port_guid, &hca_hndl, + &hca_id, &port_num); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 5005: " + "Fail to find port number of port guid:0x%016" PRIx64 + "\n", port_guid); + goto Exit; + } + + /* the file name is just /dev/ts_ua0: */ + strcpy(device_file, "/dev/ts_ua0"); + + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: " "Opening TS UL dev file:%s\n", device_file); + + /* Open the file ... */ + device_fd = open(device_file, O_RDWR); + if (device_fd < 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 5006: " + "Fail to open TS UL dev file:%s\n", device_file); + goto Exit; + } + + /* track this bind request info */ + p_bind->ul_dev_fd = device_fd; + p_bind->port_num = port_num; + p_bind->p_vend = p_vend; + p_bind->client_context = context; + p_bind->rcv_callback = mad_recv_callback; + p_bind->send_err_callback = send_err_callback; + p_bind->p_osm_pool = p_mad_pool; + p_bind->hca_hndl = hca_hndl; + + /* + * Create the MAD filter on this file handle. + */ + filter.port = port_num; + + filter.qpn = qpn; + filter.mgmt_class = p_user_bind->mad_class; + filter.direction = TS_IB_MAD_DIRECTION_IN; + filter.mask = + TS_IB_MAD_FILTER_DIRECTION | + TS_IB_MAD_FILTER_PORT | + TS_IB_MAD_FILTER_QPN | TS_IB_MAD_FILTER_MGMT_CLASS; + + ts_ioctl_ret = ioctl(device_fd, TS_IB_IOCSMADFILTADD, &filter); + if (ts_ioctl_ret < 0) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_bind: ERR 5014: " + "Fail to register MAD filter with err:%u\n", + ts_ioctl_ret); + goto Exit; + } + + /* Initialize the listener thread for this port */ + status = cl_thread_init(&p_bind->poller, + __osm_vendor_ts_poller, p_bind, + "osm ts poller"); + if (status != IB_SUCCESS) + goto Exit; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return ((osm_bind_handle_t) p_bind); } /********************************************************************** Get a mad from the lower level. The osm_vend_wrap_t is a wrapper used to connect the mad to the response. **********************************************************************/ -ib_mad_t * -osm_vendor_get( IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t * const p_vw ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * const p_vw) { - ib_mad_t *p_mad; - osm_ts_bind_info_t *p_bind = ( osm_ts_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_get ); - - CL_ASSERT( p_vw ); - - p_vw->size = mad_size; - - /* allocate it */ - p_mad = ( ib_mad_t * ) malloc( p_vw->size ); - if( p_mad == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_get: ERR 5022: " - "Error Obtaining MAD buffer.\n" ); - goto Exit; - } - - memset( p_mad, 0, p_vw->size ); - - /* track locally */ - p_vw->p_mad_buf = p_mad; - p_vw->h_bind = h_bind; - p_vw->p_resp_madw = NULL; - - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_get: " - "Acquired MAD %p, size = %u.\n", p_mad, p_vw->size ); - } - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( p_mad ); + ib_mad_t *p_mad; + osm_ts_bind_info_t *p_bind = (osm_ts_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_get); + + CL_ASSERT(p_vw); + + p_vw->size = mad_size; + + /* allocate it */ + p_mad = (ib_mad_t *) malloc(p_vw->size); + if (p_mad == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_get: ERR 5022: " + "Error Obtaining MAD buffer.\n"); + goto Exit; + } + + memset(p_mad, 0, p_vw->size); + + /* track locally */ + p_vw->p_mad_buf = p_mad; + p_vw->h_bind = h_bind; + p_vw->p_resp_madw = NULL; + + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_get: " + "Acquired MAD %p, size = %u.\n", p_mad, p_vw->size); + } + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (p_mad); } /********************************************************************** * Return a MAD by providing it's wrapper object. **********************************************************************/ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t * const p_vw ) +osm_vendor_put(IN osm_bind_handle_t h_bind, IN osm_vend_wrap_t * const p_vw) { - osm_ts_bind_info_t *p_bind = ( osm_ts_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - osm_madw_t *p_madw; + osm_ts_bind_info_t *p_bind = (osm_ts_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + osm_madw_t *p_madw; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_put ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_put); - CL_ASSERT( p_vw ); - CL_ASSERT( p_vw->p_mad_buf ); + CL_ASSERT(p_vw); + CL_ASSERT(p_vw->p_mad_buf); - if( osm_log_get_level( p_vend->p_log ) >= OSM_LOG_DEBUG ) - { - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_put: " "Retiring MAD %p.\n", p_vw->p_mad_buf ); - } + if (osm_log_get_level(p_vend->p_log) >= OSM_LOG_DEBUG) { + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_put: " "Retiring MAD %p.\n", + p_vw->p_mad_buf); + } - /* - * We moved the removal of the transaction to immediatly after - * it was looked up. - */ + /* + * We moved the removal of the transaction to immediatly after + * it was looked up. + */ - /* free the mad but the wrapper is part of the madw object */ - free( p_vw->p_mad_buf ); - p_vw->p_mad_buf = NULL; - p_madw = PARENT_STRUCT( p_vw, osm_madw_t, vend_wrap); - p_madw->p_mad = NULL; + /* free the mad but the wrapper is part of the madw object */ + free(p_vw->p_mad_buf); + p_vw->p_mad_buf = NULL; + p_madw = PARENT_STRUCT(p_vw, osm_madw_t, vend_wrap); + p_madw->p_mad = NULL; - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** @@ -791,72 +733,62 @@ This is for internal use by osm_vendor_send and the transaction mgr retry too. **********************************************************************/ ib_api_status_t -osm_ts_send_mad( - IN osm_ts_bind_info_t *p_bind, - IN osm_madw_t * const p_madw) +osm_ts_send_mad(IN osm_ts_bind_info_t * p_bind, IN osm_madw_t * const p_madw) { - osm_vendor_t *const p_vend = p_bind->p_vend; - osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr( p_madw ); - ib_mad_t *const p_mad = osm_madw_get_mad_ptr( p_madw ); - struct ib_mad ts_mad; - int ret; - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_ts_send_mad ); - - /* - * Copy the MAD over to the sent mad - */ - memcpy(&ts_mad, p_mad, 256); - - /* - * For all sends other than directed route SM MADs, - * acquire an address vector for the destination. - */ - if( p_mad->mgmt_class != IB_MCLASS_SUBN_DIR ) - { - __osm_ts_conv_osm_addr_to_ts_addr( - p_mad_addr, - p_mad->mgmt_class == IB_MCLASS_SUBN_LID, - &ts_mad); - } - else - { - /* is a directed route - we need to construct a permissive address */ - /* we do not need port number since it is part of the mad_hndl */ - ts_mad.dlid = IB_LID_PERMISSIVE; - ts_mad.slid = IB_LID_PERMISSIVE; - } - if( (p_mad->mgmt_class == IB_MCLASS_SUBN_DIR ) || - (p_mad->mgmt_class == IB_MCLASS_SUBN_LID ) ) - { - ts_mad.sqpn = 0; - ts_mad.dqpn = 0; - } - else - { - ts_mad.sqpn = 1; - ts_mad.dqpn = 1; - } - ts_mad.port = p_bind->port_num; - - /* send it */ - ret = write(p_bind->ul_dev_fd, &ts_mad, sizeof(ts_mad)); - - if( ret != sizeof(ts_mad) ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_ts_send_mad: ERR 5026: " - "Error sending mad (%d).\n", ret ); - status = IB_ERROR; - goto Exit; - } - - status = IB_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return(status); + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_mad_addr_t *const p_mad_addr = osm_madw_get_mad_addr_ptr(p_madw); + ib_mad_t *const p_mad = osm_madw_get_mad_ptr(p_madw); + struct ib_mad ts_mad; + int ret; + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_ts_send_mad); + + /* + * Copy the MAD over to the sent mad + */ + memcpy(&ts_mad, p_mad, 256); + + /* + * For all sends other than directed route SM MADs, + * acquire an address vector for the destination. + */ + if (p_mad->mgmt_class != IB_MCLASS_SUBN_DIR) { + __osm_ts_conv_osm_addr_to_ts_addr(p_mad_addr, + p_mad->mgmt_class == + IB_MCLASS_SUBN_LID, &ts_mad); + } else { + /* is a directed route - we need to construct a permissive address */ + /* we do not need port number since it is part of the mad_hndl */ + ts_mad.dlid = IB_LID_PERMISSIVE; + ts_mad.slid = IB_LID_PERMISSIVE; + } + if ((p_mad->mgmt_class == IB_MCLASS_SUBN_DIR) || + (p_mad->mgmt_class == IB_MCLASS_SUBN_LID)) { + ts_mad.sqpn = 0; + ts_mad.dqpn = 0; + } else { + ts_mad.sqpn = 1; + ts_mad.dqpn = 1; + } + ts_mad.port = p_bind->port_num; + + /* send it */ + ret = write(p_bind->ul_dev_fd, &ts_mad, sizeof(ts_mad)); + + if (ret != sizeof(ts_mad)) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_ts_send_mad: ERR 5026: " + "Error sending mad (%d).\n", ret); + status = IB_ERROR; + goto Exit; + } + + status = IB_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } /********************************************************************** @@ -866,127 +798,114 @@ What is unclear to me is the need for the setting of all the MAD Wrapper fields. Seems like the OSM uses these values during it's processing... **********************************************************************/ ib_api_status_t -osm_vendor_send( IN osm_bind_handle_t h_bind, - IN osm_madw_t * const p_madw, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_madw_t * const p_madw, IN boolean_t const resp_expected) { - osm_ts_bind_info_t *p_bind = ( osm_ts_bind_info_t * ) h_bind; - osm_vendor_t *const p_vend = p_bind->p_vend; - osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr( p_madw ); - ib_api_status_t status; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_send ); - - /* - * If a response is expected to this MAD, then preallocate - * a mad wrapper to contain the wire MAD received in the - * response. Allocating a wrapper here allows for easier - * failure paths than after we already received the wire mad. - */ - if( resp_expected == TRUE ) - { - /* we track it in the vendor wrapper */ - p_vw->p_resp_madw = - osm_mad_pool_get_wrapper_raw( p_bind->p_osm_pool ); - if( p_vw->p_resp_madw == NULL ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 5024: " - "Unable to allocate MAD wrapper.\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* put some minimal info on that wrapper */ - ((osm_madw_t *)(p_vw->p_resp_madw))->h_bind = h_bind; - - /* we also want to track it in the TID based map */ - status = osm_transaction_mgr_insert_madw( - (osm_bind_handle_t *)p_bind, p_madw ); - if( status != IB_SUCCESS ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_send: ERR 5025: " - "Error inserting request madw by TID (%d).\n", status ); - } - } - else - p_vw->p_resp_madw = NULL; - - /* do the actual send */ - /* HACK: to be replaced by call to RMPP Segmentation */ - status = osm_ts_send_mad(p_bind, p_madw); - - /* we do not get an asycn callback so call it ourselves */ - /* this will handle all cleanup if neccessary */ - __osm_ts_send_callback(p_bind, !resp_expected, p_madw, status); - - Exit: - OSM_LOG_EXIT( p_vend->p_log ); - return ( status ); + osm_ts_bind_info_t *p_bind = (osm_ts_bind_info_t *) h_bind; + osm_vendor_t *const p_vend = p_bind->p_vend; + osm_vend_wrap_t *const p_vw = osm_madw_get_vend_ptr(p_madw); + ib_api_status_t status; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_send); + + /* + * If a response is expected to this MAD, then preallocate + * a mad wrapper to contain the wire MAD received in the + * response. Allocating a wrapper here allows for easier + * failure paths than after we already received the wire mad. + */ + if (resp_expected == TRUE) { + /* we track it in the vendor wrapper */ + p_vw->p_resp_madw = + osm_mad_pool_get_wrapper_raw(p_bind->p_osm_pool); + if (p_vw->p_resp_madw == NULL) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 5024: " + "Unable to allocate MAD wrapper.\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* put some minimal info on that wrapper */ + ((osm_madw_t *) (p_vw->p_resp_madw))->h_bind = h_bind; + + /* we also want to track it in the TID based map */ + status = osm_transaction_mgr_insert_madw((osm_bind_handle_t *) + p_bind, p_madw); + if (status != IB_SUCCESS) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_send: ERR 5025: " + "Error inserting request madw by TID (%d).\n", + status); + } + } else + p_vw->p_resp_madw = NULL; + + /* do the actual send */ + /* HACK: to be replaced by call to RMPP Segmentation */ + status = osm_ts_send_mad(p_bind, p_madw); + + /* we do not get an asycn callback so call it ourselves */ + /* this will handle all cleanup if neccessary */ + __osm_ts_send_callback(p_bind, !resp_expected, p_madw, status); + + Exit: + OSM_LOG_EXIT(p_vend->p_log); + return (status); } - /********************************************************************** * the idea here is to change the content of the bind such that it * will hold the local address used for sending directed route by the SMA. **********************************************************************/ -ib_api_status_t -osm_vendor_local_lid_change( IN osm_bind_handle_t h_bind ) +ib_api_status_t osm_vendor_local_lid_change(IN osm_bind_handle_t h_bind) { - osm_vendor_t *p_vend = ( ( osm_ts_bind_info_t * ) h_bind )->p_vend; + osm_vendor_t *p_vend = ((osm_ts_bind_info_t *) h_bind)->p_vend; - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_local_lid_change ); + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_local_lid_change); - osm_log( p_vend->p_log, OSM_LOG_DEBUG, - "osm_vendor_local_lid_change: DEBUG 2202: " "Change of LID.\n" ); + osm_log(p_vend->p_log, OSM_LOG_DEBUG, + "osm_vendor_local_lid_change: DEBUG 2202: " "Change of LID.\n"); - OSM_LOG_EXIT( p_vend->p_log ); + OSM_LOG_EXIT(p_vend->p_log); - return ( IB_SUCCESS ); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_sm( - IN osm_bind_handle_t h_bind, - IN boolean_t is_sm_val ) +void osm_vendor_set_sm(IN osm_bind_handle_t h_bind, IN boolean_t is_sm_val) { - osm_ts_bind_info_t *p_bind = ( osm_ts_bind_info_t * ) h_bind; - osm_vendor_t *p_vend = p_bind->p_vend; - VAPI_ret_t status; - VAPI_hca_attr_t attr_mod; - VAPI_hca_attr_mask_t attr_mask; - - OSM_LOG_ENTER( p_vend->p_log, osm_vendor_set_sm ); - - memset( &attr_mod, 0, sizeof( attr_mod ) ); - memset( &attr_mask, 0, sizeof( attr_mask ) ); - - attr_mod.is_sm = is_sm_val; - attr_mask = HCA_ATTR_IS_SM; - - status = - VAPI_modify_hca_attr( p_bind->hca_hndl, p_bind->port_num, &attr_mod, - &attr_mask ); - if( status != VAPI_OK ) - { - osm_log( p_vend->p_log, OSM_LOG_ERROR, - "osm_vendor_set_sm: ERR 5027: " - "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", - is_sm_val, status ); - } - - OSM_LOG_EXIT( p_vend->p_log ); + osm_ts_bind_info_t *p_bind = (osm_ts_bind_info_t *) h_bind; + osm_vendor_t *p_vend = p_bind->p_vend; + VAPI_ret_t status; + VAPI_hca_attr_t attr_mod; + VAPI_hca_attr_mask_t attr_mask; + + OSM_LOG_ENTER(p_vend->p_log, osm_vendor_set_sm); + + memset(&attr_mod, 0, sizeof(attr_mod)); + memset(&attr_mask, 0, sizeof(attr_mask)); + + attr_mod.is_sm = is_sm_val; + attr_mask = HCA_ATTR_IS_SM; + + status = + VAPI_modify_hca_attr(p_bind->hca_hndl, p_bind->port_num, &attr_mod, + &attr_mask); + if (status != VAPI_OK) { + osm_log(p_vend->p_log, OSM_LOG_ERROR, + "osm_vendor_set_sm: ERR 5027: " + "Unable set 'IS_SM' bit to:%u in port attributes (%d).\n", + is_sm_val, status); + } + + OSM_LOG_EXIT(p_vend->p_log); } /********************************************************************** **********************************************************************/ -void -osm_vendor_set_debug( - IN osm_vendor_t* const p_vend, - IN int32_t level ) +void osm_vendor_set_debug(IN osm_vendor_t * const p_vend, IN int32_t level) { } diff --git a/opensm/libvendor/osm_vendor_umadt.c b/opensm/libvendor/osm_vendor_umadt.c index 6458b8d..fd643fa 100644 --- a/opensm/libvendor/osm_vendor_umadt.c +++ b/opensm/libvendor/osm_vendor_umadt.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_req_t. @@ -52,7 +51,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include @@ -96,655 +95,634 @@ /* //////////////////// */ /* Globals // */ /* //////////////////// */ -typedef struct _ib_sa_mad_vM3 -{ - uint8_t base_ver; - uint8_t mgmt_class; - uint8_t class_ver; - uint8_t method; - ib_net16_t status; - ib_net16_t resv; - ib_net64_t trans_id; - ib_net16_t attr_id; - ib_net16_t resv1; - ib_net32_t attr_mod; - ib_net64_t resv2; - ib_net64_t sm_key; - - +typedef struct _ib_sa_mad_vM3 { + uint8_t base_ver; + uint8_t mgmt_class; + uint8_t class_ver; + uint8_t method; + ib_net16_t status; + ib_net16_t resv; + ib_net64_t trans_id; + ib_net16_t attr_id; + ib_net16_t resv1; + ib_net32_t attr_mod; + ib_net64_t resv2; + ib_net64_t sm_key; + + ib_net32_t seg_num; + ib_net32_t payload_len; + uint8_t frag_flag; + uint8_t edit_mod; + ib_net16_t window; + ib_net16_t attr_offset; + ib_net16_t resv3; + + ib_net64_t comp_mask; + + uint8_t data[IB_SA_DATA_SIZE]; +} ib_sa_mad_t_vM3; +#define DEFAULT_TIMER_INTERVAL_MSEC 500 /* 500msec timer interval */ +void __mad_recv_processor(void *context); - ib_net32_t seg_num; - ib_net32_t payload_len; - uint8_t frag_flag; - uint8_t edit_mod; - ib_net16_t window; - ib_net16_t attr_offset; - ib_net16_t resv3; +boolean_t __valid_mad_handle(IN mad_bind_info_t * p_mad_bind_info); - ib_net64_t comp_mask; +cl_status_t +__match_tid_context(const cl_list_item_t * const p_list_item, void *context); +void __osm_vendor_timer_callback(IN void *context); - uint8_t data[IB_SA_DATA_SIZE]; -} ib_sa_mad_t_vM3; -#define DEFAULT_TIMER_INTERVAL_MSEC 500 /* 500msec timer interval */ +osm_vendor_t *osm_vendor_new(IN osm_log_t * const p_log, + IN const uint32_t timeout) +{ + ib_api_status_t status; + umadt_obj_t *p_umadt_obj; -void -__mad_recv_processor( void* context); + OSM_LOG_ENTER(p_log, osm_vendor_new); -boolean_t -__valid_mad_handle( IN mad_bind_info_t *p_mad_bind_info); + p_umadt_obj = malloc(sizeof(umadt_obj_t)); + if (p_umadt_obj) { + memset(p_umadt_obj, 0, sizeof(umadt_obj_t)); -cl_status_t -__match_tid_context( const cl_list_item_t* const p_list_item, - void* context); -void -__osm_vendor_timer_callback( - IN void* context ); + status = osm_vendor_init((osm_vendor_t *) p_umadt_obj, p_log, + timeout); + if (status != IB_SUCCESS) { + osm_vendor_delete((osm_vendor_t **) & p_umadt_obj); + } + } else { + printf + ("osm_vendor_construct: ERROR! Unable to create Umadt object!\n"); + } + OSM_LOG_EXIT(p_log); -osm_vendor_t* -osm_vendor_new( - IN osm_log_t* const p_log, - IN const uint32_t timeout ) -{ - ib_api_status_t status; - umadt_obj_t* p_umadt_obj; - - OSM_LOG_ENTER( p_log, osm_vendor_new ); - - p_umadt_obj = malloc(sizeof(umadt_obj_t)); - if( p_umadt_obj ) - { - memset( p_umadt_obj, 0, sizeof(umadt_obj_t) ); - - status = osm_vendor_init( (osm_vendor_t*)p_umadt_obj, p_log, - timeout ); - if( status != IB_SUCCESS ) - { - osm_vendor_delete( (osm_vendor_t**)&p_umadt_obj ); - } - } - else - { - printf("osm_vendor_construct: ERROR! Unable to create Umadt object!\n"); - } - - OSM_LOG_EXIT( p_log ); - - return( (osm_vendor_t*)p_umadt_obj ); + return ((osm_vendor_t *) p_umadt_obj); } -void -osm_vendor_delete( - IN osm_vendor_t** const pp_vend ) +void osm_vendor_delete(IN osm_vendor_t ** const pp_vend) { - umadt_obj_t *p_umadt_obj = (umadt_obj_t*)*pp_vend; - cl_list_item_t* p_list_item; - uint32_t count,i; - mad_bind_info_t *p_mad_bind_info; - - OSM_LOG_ENTER( p_umadt_obj->p_log, osm_vendor_delete ); - - cl_spinlock_acquire(&p_umadt_obj->register_lock); - p_mad_bind_info = (mad_bind_info_t*)cl_qlist_head( - &p_umadt_obj->register_list); - count = cl_qlist_count(&p_umadt_obj->register_list); - cl_spinlock_release(&p_umadt_obj->register_lock); - for (i=0;i < count;i++) - { - cl_spinlock_acquire(&p_umadt_obj->register_lock); - p_list_item = cl_qlist_next(&p_mad_bind_info->list_item); - cl_spinlock_release(&p_umadt_obj->register_lock); - /* Unbind this handle */ - /* osm_vendor_ubind also removesd the item from the list */ - /* osm_vendor_unbind takes the list lock so release it here */ - osm_vendor_unbind((osm_bind_handle_t)p_mad_bind_info); - p_mad_bind_info = (mad_bind_info_t*)p_list_item; - } - dlclose(p_umadt_obj->umadt_handle); - free(p_umadt_obj); - *pp_vend = NULL; - - OSM_LOG_EXIT( p_umadt_obj->p_log ); + umadt_obj_t *p_umadt_obj = (umadt_obj_t *) * pp_vend; + cl_list_item_t *p_list_item; + uint32_t count, i; + mad_bind_info_t *p_mad_bind_info; + + OSM_LOG_ENTER(p_umadt_obj->p_log, osm_vendor_delete); + + cl_spinlock_acquire(&p_umadt_obj->register_lock); + p_mad_bind_info = + (mad_bind_info_t *) cl_qlist_head(&p_umadt_obj->register_list); + count = cl_qlist_count(&p_umadt_obj->register_list); + cl_spinlock_release(&p_umadt_obj->register_lock); + for (i = 0; i < count; i++) { + cl_spinlock_acquire(&p_umadt_obj->register_lock); + p_list_item = cl_qlist_next(&p_mad_bind_info->list_item); + cl_spinlock_release(&p_umadt_obj->register_lock); + /* Unbind this handle */ + /* osm_vendor_ubind also removesd the item from the list */ + /* osm_vendor_unbind takes the list lock so release it here */ + osm_vendor_unbind((osm_bind_handle_t) p_mad_bind_info); + p_mad_bind_info = (mad_bind_info_t *) p_list_item; + } + dlclose(p_umadt_obj->umadt_handle); + free(p_umadt_obj); + *pp_vend = NULL; + + OSM_LOG_EXIT(p_umadt_obj->p_log); } - /* //////////////////////////////////////////////////////////////////////// */ /* See VendorAbstractMadIntf.h for info */ /* //////////////////////////////////////////////////////////////////////// */ /* */ ib_api_status_t -osm_vendor_init( - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN const uint32_t timeout ) +osm_vendor_init(IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, IN const uint32_t timeout) { - FSTATUS Status; - PUMADT_GET_INTERFACE uMadtGetInterface; - char* error; - umadt_obj_t *p_umadt_obj = (umadt_obj_t*)p_vend; - - OSM_LOG_ENTER( p_log, osm_vendor_init ); - - p_umadt_obj->p_log = p_log; - p_umadt_obj->timeout = timeout; - - p_umadt_obj->umadt_handle = dlopen("libibt.so", RTLD_NOW); - - if(!p_umadt_obj->umadt_handle) - { - printf("Could not load libibt.so <%s>\n", dlerror()); - return IB_ERROR; - } - uMadtGetInterface = dlsym(p_umadt_obj->umadt_handle, "uMadtGetInterface"); - if ((error = dlerror()) != NULL) - { - printf("Could not resolve symbol uMadtGetInterface ERROR<%s>\n", error); - return IB_ERROR; - } - - Status = (*uMadtGetInterface)(&p_umadt_obj->uMadtInterface); - if ( Status != FSUCCESS) - { - printf(" Error in getting uMADT interface ERROR<%d>\n",Status); - return IB_ERROR; - } - - /* Initialize the register list and register list lock */ - cl_qlist_init(&p_umadt_obj->register_list); - - cl_spinlock_construct(&p_umadt_obj->register_lock); - CL_ASSERT(cl_spinlock_init(&p_umadt_obj->register_lock) == CL_SUCCESS); - p_umadt_obj->init_done = TRUE; - printf("*****SUCCESS*****\n"); - - OSM_LOG_EXIT( p_log ); - return IB_SUCCESS; + FSTATUS Status; + PUMADT_GET_INTERFACE uMadtGetInterface; + char *error; + umadt_obj_t *p_umadt_obj = (umadt_obj_t *) p_vend; + + OSM_LOG_ENTER(p_log, osm_vendor_init); + + p_umadt_obj->p_log = p_log; + p_umadt_obj->timeout = timeout; + + p_umadt_obj->umadt_handle = dlopen("libibt.so", RTLD_NOW); + + if (!p_umadt_obj->umadt_handle) { + printf("Could not load libibt.so <%s>\n", dlerror()); + return IB_ERROR; + } + uMadtGetInterface = + dlsym(p_umadt_obj->umadt_handle, "uMadtGetInterface"); + if ((error = dlerror()) != NULL) { + printf("Could not resolve symbol uMadtGetInterface ERROR<%s>\n", + error); + return IB_ERROR; + } + + Status = (*uMadtGetInterface) (&p_umadt_obj->uMadtInterface); + if (Status != FSUCCESS) { + printf(" Error in getting uMADT interface ERROR<%d>\n", Status); + return IB_ERROR; + } + + /* Initialize the register list and register list lock */ + cl_qlist_init(&p_umadt_obj->register_list); + + cl_spinlock_construct(&p_umadt_obj->register_lock); + CL_ASSERT(cl_spinlock_init(&p_umadt_obj->register_lock) == CL_SUCCESS); + p_umadt_obj->init_done = TRUE; + printf("*****SUCCESS*****\n"); + + OSM_LOG_EXIT(p_log); + return IB_SUCCESS; } + /* //////////////////////////////////////////////////////////////////////// */ /* See VendorAbstractMadIntf.h for info */ /* //////////////////////////////////////////////////////////////////////// */ ib_api_status_t -osm_vendor_get_ports( - IN osm_vendor_t* const p_vend, - IN ib_net64_t* const p_guids, - IN uint32_t* const p_num_guids ) +osm_vendor_get_ports(IN osm_vendor_t * const p_vend, + IN ib_net64_t * const p_guids, + IN uint32_t * const p_num_guids) { - char* error = NULL; - PIBT_GET_INTERFACE pfnIbtGetInterface; - PIBT_INIT pfnIbtInitFunc; - - FSTATUS Status; - uint32_t caCount, caGuidCount; - IB_CA_ATTRIBUTES caAttributes; - IB_HANDLE caHandle; - uint32_t i; - IB_PORT_ATTRIBUTES *pPortAttributesList; - EUI64 CaGuidArray[8]; - void *context; - uint64_t* p_port_guid; - uint32_t free_guids; - - umadt_obj_t *p_umadt_obj = (umadt_obj_t*)p_vend; - - OSM_LOG_ENTER( p_umadt_obj->p_log, osm_vendor_get_ports ); - - CL_ASSERT(p_guids); - CL_ASSERT(p_num_guids); - - pfnIbtInitFunc = ( PIBT_INIT ) dlsym ( p_umadt_obj->umadt_handle, "IbtInit" ); - - if ( !pfnIbtInitFunc ) - { - printf ( "Error getting IbtInit function address.\n" ); - return IB_ERROR; - } - - ( *pfnIbtInitFunc ) ( ); - - pfnIbtGetInterface = ( PIBT_GET_INTERFACE ) dlsym ( p_umadt_obj->umadt_handle, - "IbtGetInterface" ); - - if ( !pfnIbtGetInterface || (error = dlerror()) != NULL) - { - printf( "Error getting IbtGetInterface function address.<%s>\n", error); - return FALSE; - } - ( *pfnIbtGetInterface ) ( &p_umadt_obj->IbtInterface ); - - caGuidCount = 8; - Status = p_umadt_obj->IbtInterface.GetCaGuidArray ( &caGuidCount, &CaGuidArray[0] ); - - if ( ( Status != FSUCCESS ) || ( caGuidCount == 0 ) ) - { - return FALSE; - } - - free_guids = *p_num_guids; - p_port_guid = p_guids; - - /* query each ca & copy its info into callers buffer */ - for ( caCount = 0; caCount < caGuidCount; caCount++ ) - { - memset ( &caAttributes, 0, sizeof ( IB_CA_ATTRIBUTES ) ); - - /* Open the CA */ - Status = p_umadt_obj->IbtInterface.Vpi.OpenCA ( CaGuidArray[caCount], NULL, /* CACompletionCallback */ - NULL, /* AsyncEventCallback */ - NULL, &caHandle ); - if ( Status != FSUCCESS ) - { - return IB_ERROR; - } - - Status = p_umadt_obj->IbtInterface.Vpi.QueryCA ( caHandle, - &caAttributes, &context ); - - if ( Status != FSUCCESS ) - { - p_umadt_obj->IbtInterface.Vpi.CloseCA ( caHandle ); - return IB_ERROR; - } - - if ( caAttributes.Ports > free_guids) - { - *p_num_guids = 0; - memset (p_guids, 0, (*p_num_guids)*sizeof(uint64_t)); - return IB_INSUFFICIENT_MEMORY; - } - - pPortAttributesList = - ( IB_PORT_ATTRIBUTES * ) malloc ( caAttributes. - PortAttributesListSize ); - - if ( pPortAttributesList == NULL ) - { - p_umadt_obj->IbtInterface.Vpi.CloseCA ( caHandle ); - *p_num_guids = 0; - memset (p_guids, 0, (*p_num_guids)*sizeof(uint64_t)); - return IB_INSUFFICIENT_MEMORY; - } - - memset ( pPortAttributesList, 0, - caAttributes.PortAttributesListSize ); - - caAttributes.PortAttributesList = pPortAttributesList; - - Status = p_umadt_obj->IbtInterface.Vpi.QueryCA ( caHandle, - &caAttributes, &context ); - - if ( Status != FSUCCESS ) - { - p_umadt_obj->IbtInterface.Vpi.CloseCA ( caHandle ); - *p_num_guids = 0; - memset (p_guids, 0, (*p_num_guids)*sizeof(uint64_t)); - return IB_ERROR; - } - - pPortAttributesList = caAttributes.PortAttributesList; - - for ( i = 0; i < caAttributes.Ports; i++ ) - { - *(p_port_guid) = cl_hton64(( uint64_t)pPortAttributesList->GUID); - pPortAttributesList = pPortAttributesList->Next; - p_port_guid++; - } - free (caAttributes.PortAttributesList); - p_umadt_obj->IbtInterface.Vpi.CloseCA ( caHandle ); - - free_guids = free_guids - caAttributes.Ports ; - - } - *p_num_guids = *p_num_guids - free_guids; - return IB_SUCCESS; + char *error = NULL; + PIBT_GET_INTERFACE pfnIbtGetInterface; + PIBT_INIT pfnIbtInitFunc; + + FSTATUS Status; + uint32_t caCount, caGuidCount; + IB_CA_ATTRIBUTES caAttributes; + IB_HANDLE caHandle; + uint32_t i; + IB_PORT_ATTRIBUTES *pPortAttributesList; + EUI64 CaGuidArray[8]; + void *context; + uint64_t *p_port_guid; + uint32_t free_guids; + + umadt_obj_t *p_umadt_obj = (umadt_obj_t *) p_vend; + + OSM_LOG_ENTER(p_umadt_obj->p_log, osm_vendor_get_ports); + + CL_ASSERT(p_guids); + CL_ASSERT(p_num_guids); + + pfnIbtInitFunc = + (PIBT_INIT) dlsym(p_umadt_obj->umadt_handle, "IbtInit"); + + if (!pfnIbtInitFunc) { + printf("Error getting IbtInit function address.\n"); + return IB_ERROR; + } + + (*pfnIbtInitFunc) (); + + pfnIbtGetInterface = + (PIBT_GET_INTERFACE) dlsym(p_umadt_obj->umadt_handle, + "IbtGetInterface"); + + if (!pfnIbtGetInterface || (error = dlerror()) != NULL) { + printf("Error getting IbtGetInterface function address.<%s>\n", + error); + return FALSE; + } + (*pfnIbtGetInterface) (&p_umadt_obj->IbtInterface); + + caGuidCount = 8; + Status = + p_umadt_obj->IbtInterface.GetCaGuidArray(&caGuidCount, + &CaGuidArray[0]); + + if ((Status != FSUCCESS) || (caGuidCount == 0)) { + return FALSE; + } + + free_guids = *p_num_guids; + p_port_guid = p_guids; + + /* query each ca & copy its info into callers buffer */ + for (caCount = 0; caCount < caGuidCount; caCount++) { + memset(&caAttributes, 0, sizeof(IB_CA_ATTRIBUTES)); + + /* Open the CA */ + Status = p_umadt_obj->IbtInterface.Vpi.OpenCA(CaGuidArray[caCount], NULL, /* CACompletionCallback */ + NULL, /* AsyncEventCallback */ + NULL, &caHandle); + if (Status != FSUCCESS) { + return IB_ERROR; + } + + Status = p_umadt_obj->IbtInterface.Vpi.QueryCA(caHandle, + &caAttributes, + &context); + + if (Status != FSUCCESS) { + p_umadt_obj->IbtInterface.Vpi.CloseCA(caHandle); + return IB_ERROR; + } + + if (caAttributes.Ports > free_guids) { + *p_num_guids = 0; + memset(p_guids, 0, (*p_num_guids) * sizeof(uint64_t)); + return IB_INSUFFICIENT_MEMORY; + } + + pPortAttributesList = + (IB_PORT_ATTRIBUTES *) malloc(caAttributes. + PortAttributesListSize); + + if (pPortAttributesList == NULL) { + p_umadt_obj->IbtInterface.Vpi.CloseCA(caHandle); + *p_num_guids = 0; + memset(p_guids, 0, (*p_num_guids) * sizeof(uint64_t)); + return IB_INSUFFICIENT_MEMORY; + } + + memset(pPortAttributesList, 0, + caAttributes.PortAttributesListSize); + + caAttributes.PortAttributesList = pPortAttributesList; + + Status = p_umadt_obj->IbtInterface.Vpi.QueryCA(caHandle, + &caAttributes, + &context); + + if (Status != FSUCCESS) { + p_umadt_obj->IbtInterface.Vpi.CloseCA(caHandle); + *p_num_guids = 0; + memset(p_guids, 0, (*p_num_guids) * sizeof(uint64_t)); + return IB_ERROR; + } + + pPortAttributesList = caAttributes.PortAttributesList; + + for (i = 0; i < caAttributes.Ports; i++) { + *(p_port_guid) = + cl_hton64((uint64_t) pPortAttributesList->GUID); + pPortAttributesList = pPortAttributesList->Next; + p_port_guid++; + } + free(caAttributes.PortAttributesList); + p_umadt_obj->IbtInterface.Vpi.CloseCA(caHandle); + + free_guids = free_guids - caAttributes.Ports; + + } + *p_num_guids = *p_num_guids - free_guids; + return IB_SUCCESS; } /* //////////////////////////////////////////////////////////////////////// */ /* See VendorAbstractMadIntf.h for info */ /* //////////////////////////////////////////////////////////////////////// */ -ib_mad_t* -osm_vendor_get( - IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN osm_vend_wrap_t *p_vend_wrap ) +ib_mad_t *osm_vendor_get(IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN osm_vend_wrap_t * p_vend_wrap) { - /* FSTATUS Status; */ - /* uint32_t mad_count = 0; */ - /* MadtStruct *p_madt_struct; */ - mad_bind_info_t *p_mad_bind_info = (mad_bind_info_t*)h_bind; - umadt_obj_t *p_umadt_obj = p_mad_bind_info->p_umadt_obj; - ib_mad_t *p_mad; - OSM_LOG_ENTER( p_umadt_obj->p_log, osm_vendor_get ); + /* FSTATUS Status; */ + /* uint32_t mad_count = 0; */ + /* MadtStruct *p_madt_struct; */ + mad_bind_info_t *p_mad_bind_info = (mad_bind_info_t *) h_bind; + umadt_obj_t *p_umadt_obj = p_mad_bind_info->p_umadt_obj; + ib_mad_t *p_mad; + OSM_LOG_ENTER(p_umadt_obj->p_log, osm_vendor_get); - CL_ASSERT(h_bind); + CL_ASSERT(h_bind); - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - - /* Sanity check */ - CL_ASSERT(p_umadt_obj->init_done); - CL_ASSERT(p_vend_wrap); - CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + /* Sanity check */ + CL_ASSERT(p_umadt_obj->init_done); + CL_ASSERT(p_vend_wrap); + CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); #if 0 - mad_count = 1; - Status = p_umadt_obj->uMadtInterface.uMadtGetSendMad(p_mad_bind_info->umadt_handle, - &mad_count, - &p_madt_struct ); - - if(Status != FSUCCESS || p_madt_struct == NULL) - { - p_vend_wrap->p_madt_struct = NULL; - return NULL; - } - p_vend_wrap->p_madt_struct = p_madt_struct; - p_vend_wrap->direction = SEND; - return( (ib_mad_t*)&p_madt_struct->IBMad ); -#endif /* 0 */ - p_mad = (ib_mad_t*)malloc( mad_size ); - if ( !p_mad ) - { - p_vend_wrap->p_madt_struct = NULL; - return NULL; - } - - memset(p_mad, 0, mad_size); - - p_vend_wrap->p_madt_struct = NULL; - p_vend_wrap->direction = SEND; - p_vend_wrap->size =mad_size; - return( p_mad); + mad_count = 1; + Status = + p_umadt_obj->uMadtInterface.uMadtGetSendMad(p_mad_bind_info-> + umadt_handle, + &mad_count, + &p_madt_struct); + + if (Status != FSUCCESS || p_madt_struct == NULL) { + p_vend_wrap->p_madt_struct = NULL; + return NULL; + } + p_vend_wrap->p_madt_struct = p_madt_struct; + p_vend_wrap->direction = SEND; + return ((ib_mad_t *) & p_madt_struct->IBMad); +#endif /* 0 */ + p_mad = (ib_mad_t *) malloc(mad_size); + if (!p_mad) { + p_vend_wrap->p_madt_struct = NULL; + return NULL; + } + + memset(p_mad, 0, mad_size); + + p_vend_wrap->p_madt_struct = NULL; + p_vend_wrap->direction = SEND; + p_vend_wrap->size = mad_size; + return (p_mad); } + /* //////////////////////////////////////////////////////////////////////// */ /* See VendorAbstractMadIntf.h for info */ /* //////////////////////////////////////////////////////////////////////// */ void -osm_vendor_put( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vend_wrap, - IN ib_mad_t* const p_mad) +osm_vendor_put(IN osm_bind_handle_t h_bind, + IN osm_vend_wrap_t * const p_vend_wrap, + IN ib_mad_t * const p_mad) { - FSTATUS Status; - - mad_bind_info_t *p_mad_bind_info; - umadt_obj_t *p_umadt_obj; - - /* */ - /* Validate the vendor mad transport handle */ - /* */ - CL_ASSERT(h_bind); - p_mad_bind_info = (mad_bind_info_t*)h_bind; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - - /* sanity check */ - CL_ASSERT(p_umadt_obj->init_done); - CL_ASSERT(h_bind); - CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); - CL_ASSERT(p_vend_wrap); - /* CL_ASSERT( (ib_mad_t*)&p_vend_wrap->p_madt_struct->IBMad == p_mad ); */ - - /* Release the MAD based on the direction of the MAD */ - if ( p_vend_wrap->direction == SEND) - { - /* */ - /* For a send the PostSend released the MAD with Umadt. Simply dealloacte the */ - /* local memory that was allocated on the osm_vendor_get() call. */ - /* */ - free(p_mad); + FSTATUS Status; + + mad_bind_info_t *p_mad_bind_info; + umadt_obj_t *p_umadt_obj; + + /* */ + /* Validate the vendor mad transport handle */ + /* */ + CL_ASSERT(h_bind); + p_mad_bind_info = (mad_bind_info_t *) h_bind; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + + /* sanity check */ + CL_ASSERT(p_umadt_obj->init_done); + CL_ASSERT(h_bind); + CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); + CL_ASSERT(p_vend_wrap); + /* CL_ASSERT( (ib_mad_t*)&p_vend_wrap->p_madt_struct->IBMad == p_mad ); */ + + /* Release the MAD based on the direction of the MAD */ + if (p_vend_wrap->direction == SEND) { + /* */ + /* For a send the PostSend released the MAD with Umadt. Simply dealloacte the */ + /* local memory that was allocated on the osm_vendor_get() call. */ + /* */ + free(p_mad); #if 0 - Status = p_umadt_obj->uMadtInterface.uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, - p_vend_wrap->p_madt_struct); - if(Status != FSUCCESS) - { - /* printf("uMadtReleaseSendMad: Status = <%d>\n", Status); */ - return ; - } + Status = + p_umadt_obj->uMadtInterface. + uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, + p_vend_wrap->p_madt_struct); + if (Status != FSUCCESS) { + /* printf("uMadtReleaseSendMad: Status = <%d>\n", Status); */ + return; + } #endif - } - else if (p_vend_wrap->direction == RECEIVE) - { - CL_ASSERT( (ib_mad_t*)&p_vend_wrap->p_madt_struct->IBMad == p_mad ); - Status = p_umadt_obj->uMadtInterface.uMadtReleaseRecvMad(p_mad_bind_info->umadt_handle, - p_vend_wrap->p_madt_struct); - if(Status != FSUCCESS) - { - /* printf("uMadtReleaseRecvMad Status=<%d>\n", Status); */ - return ; - } - } - else{ - return ; - } - return ; + } else if (p_vend_wrap->direction == RECEIVE) { + CL_ASSERT((ib_mad_t *) & p_vend_wrap->p_madt_struct->IBMad == + p_mad); + Status = + p_umadt_obj->uMadtInterface. + uMadtReleaseRecvMad(p_mad_bind_info->umadt_handle, + p_vend_wrap->p_madt_struct); + if (Status != FSUCCESS) { + /* printf("uMadtReleaseRecvMad Status=<%d>\n", Status); */ + return; + } + } else { + return; + } + return; } + /* //////////////////////////////////////////////////////////////////////// */ /* See VendorAbstractMadIntf.h for info */ /* //////////////////////////////////////////////////////////////////////// */ ib_api_status_t -osm_vendor_send( - IN osm_bind_handle_t h_bind, - IN osm_vend_wrap_t* const p_vend_wrap, - IN osm_mad_addr_t* const p_mad_addr, - IN ib_mad_t* const p_mad, - IN void* transaction_context, - IN boolean_t const resp_expected ) +osm_vendor_send(IN osm_bind_handle_t h_bind, + IN osm_vend_wrap_t * const p_vend_wrap, + IN osm_mad_addr_t * const p_mad_addr, + IN ib_mad_t * const p_mad, + IN void *transaction_context, IN boolean_t const resp_expected) { - FSTATUS Status; - - MadAddrStruct destAddr = {0}; - - mad_bind_info_t *p_mad_bind_info; - trans_context_t *p_trans_context; - - umadt_obj_t *p_umadt_obj=NULL; - - uint32_t mad_count = 0; - MadtStruct *p_madt_struct = NULL; - uint32_t i; - uint32_t num_mads=0; - uint32_t seg_num=0; - uint8_t *p_frag_data = NULL; - ib_sa_mad_t_vM3 *p_sa_mad = NULL; - - - CL_ASSERT(h_bind); - p_mad_bind_info = (mad_bind_info_t*)h_bind; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - - /* sanity check */ - CL_ASSERT(p_umadt_obj); - CL_ASSERT(p_umadt_obj->init_done); - CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); - CL_ASSERT(p_vend_wrap); - CL_ASSERT(p_mad_addr); - CL_ASSERT(p_mad); - /* CL_ASSERT( (ib_mad_t*)&p_vend_wrap->p_madt_struct->IBMad == p_mad ); */ - - /* */ - /* based on the class, fill out the address info */ - /* */ - destAddr.DestLid = p_mad_addr->dest_lid; - destAddr.PathBits = p_mad_addr->path_bits; - destAddr.StaticRate = p_mad_addr->static_rate; - - if (p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_LID || - p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_DIR) - { - CL_ASSERT(p_mad_addr->addr_type.smi.source_lid); - destAddr.AddrType.Smi.SourceLid = p_mad_addr->addr_type.smi.source_lid; - } - else - { - destAddr.AddrType.Gsi.RemoteQpNumber = p_mad_addr->addr_type.gsi.remote_qp; - destAddr.AddrType.Gsi.RemoteQkey = p_mad_addr->addr_type.gsi.remote_qkey; - destAddr.AddrType.Gsi.PKey = p_mad_addr->addr_type.gsi.pkey; - destAddr.AddrType.Gsi.ServiceLevel = p_mad_addr->addr_type.gsi.service_level; - destAddr.AddrType.Gsi.GlobalRoute = p_mad_addr->addr_type.gsi.global_route; - /* destAddr.AddrType.Gsi.GRHInfo = p_mad_addr->addr_type.gsi.grh_info; */ - } - p_mad->trans_id = cl_ntoh64(p_mad->trans_id)<<24; - - /* */ - /* Create a transaction context for this send and save the TID and client context. */ - /* */ - - if ( resp_expected ) - { - p_trans_context = malloc(sizeof(trans_context_t)); - CL_ASSERT(p_trans_context); - - memset(p_trans_context, 0, sizeof(trans_context_t)); - p_trans_context->trans_id = p_mad->trans_id; - p_trans_context->context = transaction_context; - p_trans_context->sent_time = cl_get_time_stamp(); - - cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); - cl_qlist_insert_tail(&p_mad_bind_info->trans_ctxt_list, - &p_trans_context->list_item); - cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); - } - - if (p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_LID || - p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_DIR) - { - /* Get one mad from uMadt */ - mad_count = 1; - Status = p_umadt_obj->uMadtInterface.uMadtGetSendMad(p_mad_bind_info->umadt_handle, - &mad_count, - &p_madt_struct ); - - if(Status != FSUCCESS || p_madt_struct == NULL) - { - return IB_ERROR; - } - - /* No Segmentation required */ - memcpy(&p_madt_struct->IBMad, p_mad, MAD_BLOCK_SIZE); - - /* Post the MAD */ - - Status = p_umadt_obj->uMadtInterface.uMadtPostSend(p_mad_bind_info->umadt_handle, - p_madt_struct, - &destAddr); - if(Status != FSUCCESS) - { - printf("uMadtPostSendMad: Status = <%d>\n", Status); - return IB_ERROR; - } - - /* Release send MAD */ - Status = p_umadt_obj->uMadtInterface.uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, - p_madt_struct); - if(Status != FSUCCESS) - { - printf("uMadtReleaseSendMad: Status = <%d>\n", Status); - return IB_ERROR; - } - } - else - { - - /* */ - /* Segment the MAD, get the required send mads from uMadt and post the MADs. */ - /* */ - uint32_t payload_len; - - payload_len = cl_ntoh32(((ib_sa_mad_t_vM3*)p_mad)->payload_len); - num_mads = payload_len / IB_SA_DATA_SIZE; - if ( payload_len % IB_SA_DATA_SIZE != 0) - { - num_mads++; /* Get one additional mad for the remainder */ - } - for ( i = 0; i < num_mads; i++) - { - /* Get one mad from uMadt */ - mad_count = 1; - Status = p_umadt_obj->uMadtInterface.uMadtGetSendMad(p_mad_bind_info->umadt_handle, - &mad_count, - &p_madt_struct ); - - if(Status != FSUCCESS || p_madt_struct == NULL) - { - return IB_ERROR; - } - /* Copy client MAD into uMadt's MAD. */ - if ( i == 0) /* First Packet */ - { - /* Since this is the first MAD, copy the entire MAD_SIZE */ - memcpy(&p_madt_struct->IBMad, p_mad, MAD_BLOCK_SIZE); - - p_frag_data = (uint8_t*)p_mad + MAD_BLOCK_SIZE; - - p_sa_mad = (ib_sa_mad_t_vM3*)&p_madt_struct->IBMad; - if ( num_mads == 1) /* Only one Packet */ - { - p_sa_mad->seg_num = 0; - p_sa_mad->frag_flag = 5; /* Set bit 0 for first pkt and b4 for last pkt */ - /* the payload length gets copied with the mad header above */ - } - else /* More than one packet in this response */ - { - seg_num = 1; - p_sa_mad->seg_num = cl_ntoh32(seg_num++); - p_sa_mad->frag_flag = 1; /* Set bit 0 for first pkt */ - /* the payload length gets copied with the mad header above */ - } - - } - else if ( i < num_mads -1) /* Not last packet */ - { - /* First copy only the header */ - memcpy(&p_madt_struct->IBMad, p_mad, IB_SA_MAD_HDR_SIZE); - /* Set the relevant fields in the SA_MAD_HEADER */ - p_sa_mad = (ib_sa_mad_t_vM3*)&p_madt_struct->IBMad; - p_sa_mad->payload_len = cl_ntoh32(IB_SA_DATA_SIZE); - p_sa_mad->seg_num = cl_ntoh32(seg_num++); - p_sa_mad->frag_flag = 0; - /* Now copy the fragmented data */ - memcpy(((uint8_t*)&p_madt_struct->IBMad) + IB_SA_MAD_HDR_SIZE, p_frag_data, IB_SA_DATA_SIZE); - p_frag_data = p_frag_data + IB_SA_DATA_SIZE; - - } - else if ( i == num_mads - 1) /* Last packet */ - { - /* First copy only the header */ - memcpy(&p_madt_struct->IBMad, p_mad, IB_SA_MAD_HDR_SIZE); - /* Set the relevant fields in the SA_MAD_HEADER */ - p_sa_mad = (ib_sa_mad_t_vM3*)&p_madt_struct->IBMad; - p_sa_mad->seg_num = cl_ntoh32(seg_num++); - p_sa_mad->frag_flag = 4; /* Set Bit 2 for last pkt */ - p_sa_mad->payload_len = - cl_ntoh32(cl_ntoh32(((ib_sa_mad_t_vM3*)p_mad)->payload_len) % IB_SA_DATA_SIZE); - /* Now copy the fragmented data */ - memcpy((((uint8_t*)&p_madt_struct->IBMad)) + IB_SA_MAD_HDR_SIZE, - p_frag_data, cl_ntoh32(p_sa_mad->payload_len)); - p_frag_data = p_frag_data + IB_SA_DATA_SIZE; - - } - /* Post the MAD */ - Status = p_umadt_obj->uMadtInterface.uMadtPostSend(p_mad_bind_info->umadt_handle, - p_madt_struct, - &destAddr); - if(Status != FSUCCESS) - { - printf("uMadtPostSendMad: Status = <%d>\n", Status); - return IB_ERROR; - } - - /* Release send MAD */ - Status = p_umadt_obj->uMadtInterface.uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, - p_madt_struct); - if(Status != FSUCCESS) - { - printf("uMadtReleaseSendMad: Status = <%d>\n", Status); - return IB_ERROR; - } - } - } - return( IB_SUCCESS ); + FSTATUS Status; + + MadAddrStruct destAddr = { 0 }; + + mad_bind_info_t *p_mad_bind_info; + trans_context_t *p_trans_context; + + umadt_obj_t *p_umadt_obj = NULL; + + uint32_t mad_count = 0; + MadtStruct *p_madt_struct = NULL; + uint32_t i; + uint32_t num_mads = 0; + uint32_t seg_num = 0; + uint8_t *p_frag_data = NULL; + ib_sa_mad_t_vM3 *p_sa_mad = NULL; + + CL_ASSERT(h_bind); + p_mad_bind_info = (mad_bind_info_t *) h_bind; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + + /* sanity check */ + CL_ASSERT(p_umadt_obj); + CL_ASSERT(p_umadt_obj->init_done); + CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); + CL_ASSERT(p_vend_wrap); + CL_ASSERT(p_mad_addr); + CL_ASSERT(p_mad); + /* CL_ASSERT( (ib_mad_t*)&p_vend_wrap->p_madt_struct->IBMad == p_mad ); */ + + /* */ + /* based on the class, fill out the address info */ + /* */ + destAddr.DestLid = p_mad_addr->dest_lid; + destAddr.PathBits = p_mad_addr->path_bits; + destAddr.StaticRate = p_mad_addr->static_rate; + + if (p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_LID || + p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_DIR) { + CL_ASSERT(p_mad_addr->addr_type.smi.source_lid); + destAddr.AddrType.Smi.SourceLid = + p_mad_addr->addr_type.smi.source_lid; + } else { + destAddr.AddrType.Gsi.RemoteQpNumber = + p_mad_addr->addr_type.gsi.remote_qp; + destAddr.AddrType.Gsi.RemoteQkey = + p_mad_addr->addr_type.gsi.remote_qkey; + destAddr.AddrType.Gsi.PKey = p_mad_addr->addr_type.gsi.pkey; + destAddr.AddrType.Gsi.ServiceLevel = + p_mad_addr->addr_type.gsi.service_level; + destAddr.AddrType.Gsi.GlobalRoute = + p_mad_addr->addr_type.gsi.global_route; + /* destAddr.AddrType.Gsi.GRHInfo = p_mad_addr->addr_type.gsi.grh_info; */ + } + p_mad->trans_id = cl_ntoh64(p_mad->trans_id) << 24; + + /* */ + /* Create a transaction context for this send and save the TID and client context. */ + /* */ + + if (resp_expected) { + p_trans_context = malloc(sizeof(trans_context_t)); + CL_ASSERT(p_trans_context); + + memset(p_trans_context, 0, sizeof(trans_context_t)); + p_trans_context->trans_id = p_mad->trans_id; + p_trans_context->context = transaction_context; + p_trans_context->sent_time = cl_get_time_stamp(); + + cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); + cl_qlist_insert_tail(&p_mad_bind_info->trans_ctxt_list, + &p_trans_context->list_item); + cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); + } + + if (p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_LID || + p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_DIR) { + /* Get one mad from uMadt */ + mad_count = 1; + Status = + p_umadt_obj->uMadtInterface. + uMadtGetSendMad(p_mad_bind_info->umadt_handle, &mad_count, + &p_madt_struct); + + if (Status != FSUCCESS || p_madt_struct == NULL) { + return IB_ERROR; + } + + /* No Segmentation required */ + memcpy(&p_madt_struct->IBMad, p_mad, MAD_BLOCK_SIZE); + + /* Post the MAD */ + + Status = + p_umadt_obj->uMadtInterface.uMadtPostSend(p_mad_bind_info-> + umadt_handle, + p_madt_struct, + &destAddr); + if (Status != FSUCCESS) { + printf("uMadtPostSendMad: Status = <%d>\n", Status); + return IB_ERROR; + } + + /* Release send MAD */ + Status = + p_umadt_obj->uMadtInterface. + uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, + p_madt_struct); + if (Status != FSUCCESS) { + printf("uMadtReleaseSendMad: Status = <%d>\n", Status); + return IB_ERROR; + } + } else { + + /* */ + /* Segment the MAD, get the required send mads from uMadt and post the MADs. */ + /* */ + uint32_t payload_len; + + payload_len = + cl_ntoh32(((ib_sa_mad_t_vM3 *) p_mad)->payload_len); + num_mads = payload_len / IB_SA_DATA_SIZE; + if (payload_len % IB_SA_DATA_SIZE != 0) { + num_mads++; /* Get one additional mad for the remainder */ + } + for (i = 0; i < num_mads; i++) { + /* Get one mad from uMadt */ + mad_count = 1; + Status = + p_umadt_obj->uMadtInterface. + uMadtGetSendMad(p_mad_bind_info->umadt_handle, + &mad_count, &p_madt_struct); + + if (Status != FSUCCESS || p_madt_struct == NULL) { + return IB_ERROR; + } + /* Copy client MAD into uMadt's MAD. */ + if (i == 0) { /* First Packet */ + /* Since this is the first MAD, copy the entire MAD_SIZE */ + memcpy(&p_madt_struct->IBMad, p_mad, + MAD_BLOCK_SIZE); + + p_frag_data = + (uint8_t *) p_mad + MAD_BLOCK_SIZE; + + p_sa_mad = + (ib_sa_mad_t_vM3 *) & p_madt_struct->IBMad; + if (num_mads == 1) { /* Only one Packet */ + p_sa_mad->seg_num = 0; + p_sa_mad->frag_flag = 5; /* Set bit 0 for first pkt and b4 for last pkt */ + /* the payload length gets copied with the mad header above */ + } else { /* More than one packet in this response */ + + seg_num = 1; + p_sa_mad->seg_num = + cl_ntoh32(seg_num++); + p_sa_mad->frag_flag = 1; /* Set bit 0 for first pkt */ + /* the payload length gets copied with the mad header above */ + } + + } else if (i < num_mads - 1) { /* Not last packet */ + /* First copy only the header */ + memcpy(&p_madt_struct->IBMad, p_mad, + IB_SA_MAD_HDR_SIZE); + /* Set the relevant fields in the SA_MAD_HEADER */ + p_sa_mad = + (ib_sa_mad_t_vM3 *) & p_madt_struct->IBMad; + p_sa_mad->payload_len = + cl_ntoh32(IB_SA_DATA_SIZE); + p_sa_mad->seg_num = cl_ntoh32(seg_num++); + p_sa_mad->frag_flag = 0; + /* Now copy the fragmented data */ + memcpy(((uint8_t *) & p_madt_struct->IBMad) + + IB_SA_MAD_HDR_SIZE, p_frag_data, + IB_SA_DATA_SIZE); + p_frag_data = p_frag_data + IB_SA_DATA_SIZE; + + } else if (i == num_mads - 1) { /* Last packet */ + /* First copy only the header */ + memcpy(&p_madt_struct->IBMad, p_mad, + IB_SA_MAD_HDR_SIZE); + /* Set the relevant fields in the SA_MAD_HEADER */ + p_sa_mad = + (ib_sa_mad_t_vM3 *) & p_madt_struct->IBMad; + p_sa_mad->seg_num = cl_ntoh32(seg_num++); + p_sa_mad->frag_flag = 4; /* Set Bit 2 for last pkt */ + p_sa_mad->payload_len = + cl_ntoh32(cl_ntoh32 + (((ib_sa_mad_t_vM3 *) p_mad)-> + payload_len) % IB_SA_DATA_SIZE); + /* Now copy the fragmented data */ + memcpy((((uint8_t *) & p_madt_struct->IBMad)) + + IB_SA_MAD_HDR_SIZE, p_frag_data, + cl_ntoh32(p_sa_mad->payload_len)); + p_frag_data = p_frag_data + IB_SA_DATA_SIZE; + + } + /* Post the MAD */ + Status = + p_umadt_obj->uMadtInterface. + uMadtPostSend(p_mad_bind_info->umadt_handle, + p_madt_struct, &destAddr); + if (Status != FSUCCESS) { + printf("uMadtPostSendMad: Status = <%d>\n", + Status); + return IB_ERROR; + } + + /* Release send MAD */ + Status = + p_umadt_obj->uMadtInterface. + uMadtReleaseSendMad(p_mad_bind_info->umadt_handle, + p_madt_struct); + if (Status != FSUCCESS) { + printf("uMadtReleaseSendMad: Status = <%d>\n", + Status); + return IB_ERROR; + } + } + } + return (IB_SUCCESS); } /* //////////////////////////////////////////////////////////////////////// */ @@ -752,406 +730,395 @@ osm_vendor_send( /* //////////////////////////////////////////////////////////////////////// */ osm_bind_handle_t -osm_vendor_bind( - IN osm_vendor_t* const p_vend, - IN osm_bind_info_t* const p_osm_bind_info, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vend_mad_recv_callback_t mad_recv_callback, - IN void* context ) +osm_vendor_bind(IN osm_vendor_t * const p_vend, + IN osm_bind_info_t * const p_osm_bind_info, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vend_mad_recv_callback_t mad_recv_callback, + IN void *context) { - cl_status_t cl_status; - FSTATUS Status; /* GEN1 Status for Umadt */ - - mad_bind_info_t *p_mad_bind_info; - RegisterClassStruct *p_umadt_reg_class; - - umadt_obj_t *p_umadt_obj; - OSM_LOG_ENTER( ((umadt_obj_t*)p_vend)->p_log, osm_vendor_bind ); - - CL_ASSERT( p_vend ); - - p_umadt_obj = (umadt_obj_t*)p_vend; - - /* Sanity check */ - CL_ASSERT( p_umadt_obj->init_done ); - CL_ASSERT( p_osm_bind_info ); - CL_ASSERT( p_mad_pool ); - CL_ASSERT( mad_recv_callback ); - - /* Allocate memory for registering the handle. */ - p_mad_bind_info = (mad_bind_info_t*)malloc(sizeof(*p_mad_bind_info)); - if (p_mad_bind_info) - { - memset(p_mad_bind_info, 0, sizeof(*p_mad_bind_info)); - p_umadt_reg_class = &p_mad_bind_info->umadt_reg_class; - } - p_umadt_reg_class->PortGuid = cl_ntoh64( p_osm_bind_info->port_guid ); - p_umadt_reg_class->ClassId = p_osm_bind_info->mad_class; - p_umadt_reg_class->ClassVersion = p_osm_bind_info->class_version; - p_umadt_reg_class->isResponder = p_osm_bind_info->is_responder; - p_umadt_reg_class->isTrapProcessor = p_osm_bind_info->is_trap_processor; - p_umadt_reg_class->isReportProcessor = p_osm_bind_info->is_report_processor; - p_umadt_reg_class->SendQueueSize = p_osm_bind_info->send_q_size; - p_umadt_reg_class->RecvQueueSize = p_osm_bind_info->recv_q_size; - p_umadt_reg_class->NotifySendCompletion = TRUE; - - p_mad_bind_info->p_umadt_obj = p_umadt_obj; - p_mad_bind_info->p_mad_pool = p_mad_pool; - p_mad_bind_info->mad_recv_callback = mad_recv_callback; - p_mad_bind_info->client_context = context; - - /* register with Umadt for MAD interface */ - Status = p_umadt_obj->uMadtInterface.uMadtRegister(p_umadt_reg_class, - &p_mad_bind_info->umadt_handle); - if (Status != FSUCCESS) - { - free(p_mad_bind_info); - OSM_LOG_EXIT( p_umadt_obj->p_log ); - return( OSM_BIND_INVALID_HANDLE ); - } - CL_ASSERT(p_mad_bind_info->umadt_handle); - /* */ - /* Start a worker thread to process receives. */ - /* */ - cl_thread_construct(&p_mad_bind_info->recv_processor_thread); - cl_status = cl_thread_init( &p_mad_bind_info->recv_processor_thread, - __mad_recv_processor, - (void*)p_mad_bind_info, - "mad_recv_worker"); - CL_ASSERT(cl_status == CL_SUCCESS ); - - cl_qlist_init(&p_mad_bind_info->trans_ctxt_list); - cl_spinlock_construct(&p_mad_bind_info->trans_ctxt_lock); - cl_spinlock_init(&p_mad_bind_info->trans_ctxt_lock); - cl_spinlock_construct(&p_mad_bind_info->timeout_list_lock); - cl_spinlock_init(&p_mad_bind_info->timeout_list_lock); - - cl_status = cl_timer_init(&p_mad_bind_info->timeout_timer, - __osm_vendor_timer_callback, - (void*)p_mad_bind_info); - CL_ASSERT(cl_status == CL_SUCCESS ); - cl_qlist_init(&p_mad_bind_info->timeout_list); - /* */ - /* Insert the mad_reg_struct in list and return pointer to it as the handle */ - /* */ - cl_spinlock_acquire(&p_umadt_obj->register_lock); - - cl_qlist_insert_head(&p_umadt_obj->register_list, - &p_mad_bind_info->list_item); - - cl_spinlock_release(&p_umadt_obj->register_lock); - - /* - A timeout value of 0 means disable timeouts. - */ - if( p_umadt_obj->timeout ) - { - cl_timer_start(&p_mad_bind_info->timeout_timer, - DEFAULT_TIMER_INTERVAL_MSEC); - } - - OSM_LOG_EXIT( p_umadt_obj->p_log ); - return( (osm_bind_handle_t)p_mad_bind_info ); + cl_status_t cl_status; + FSTATUS Status; /* GEN1 Status for Umadt */ + + mad_bind_info_t *p_mad_bind_info; + RegisterClassStruct *p_umadt_reg_class; + + umadt_obj_t *p_umadt_obj; + OSM_LOG_ENTER(((umadt_obj_t *) p_vend)->p_log, osm_vendor_bind); + + CL_ASSERT(p_vend); + + p_umadt_obj = (umadt_obj_t *) p_vend; + + /* Sanity check */ + CL_ASSERT(p_umadt_obj->init_done); + CL_ASSERT(p_osm_bind_info); + CL_ASSERT(p_mad_pool); + CL_ASSERT(mad_recv_callback); + + /* Allocate memory for registering the handle. */ + p_mad_bind_info = (mad_bind_info_t *) malloc(sizeof(*p_mad_bind_info)); + if (p_mad_bind_info) { + memset(p_mad_bind_info, 0, sizeof(*p_mad_bind_info)); + p_umadt_reg_class = &p_mad_bind_info->umadt_reg_class; + } + p_umadt_reg_class->PortGuid = cl_ntoh64(p_osm_bind_info->port_guid); + p_umadt_reg_class->ClassId = p_osm_bind_info->mad_class; + p_umadt_reg_class->ClassVersion = p_osm_bind_info->class_version; + p_umadt_reg_class->isResponder = p_osm_bind_info->is_responder; + p_umadt_reg_class->isTrapProcessor = p_osm_bind_info->is_trap_processor; + p_umadt_reg_class->isReportProcessor = + p_osm_bind_info->is_report_processor; + p_umadt_reg_class->SendQueueSize = p_osm_bind_info->send_q_size; + p_umadt_reg_class->RecvQueueSize = p_osm_bind_info->recv_q_size; + p_umadt_reg_class->NotifySendCompletion = TRUE; + + p_mad_bind_info->p_umadt_obj = p_umadt_obj; + p_mad_bind_info->p_mad_pool = p_mad_pool; + p_mad_bind_info->mad_recv_callback = mad_recv_callback; + p_mad_bind_info->client_context = context; + + /* register with Umadt for MAD interface */ + Status = p_umadt_obj->uMadtInterface.uMadtRegister(p_umadt_reg_class, + &p_mad_bind_info-> + umadt_handle); + if (Status != FSUCCESS) { + free(p_mad_bind_info); + OSM_LOG_EXIT(p_umadt_obj->p_log); + return (OSM_BIND_INVALID_HANDLE); + } + CL_ASSERT(p_mad_bind_info->umadt_handle); + /* */ + /* Start a worker thread to process receives. */ + /* */ + cl_thread_construct(&p_mad_bind_info->recv_processor_thread); + cl_status = cl_thread_init(&p_mad_bind_info->recv_processor_thread, + __mad_recv_processor, + (void *)p_mad_bind_info, "mad_recv_worker"); + CL_ASSERT(cl_status == CL_SUCCESS); + + cl_qlist_init(&p_mad_bind_info->trans_ctxt_list); + cl_spinlock_construct(&p_mad_bind_info->trans_ctxt_lock); + cl_spinlock_init(&p_mad_bind_info->trans_ctxt_lock); + cl_spinlock_construct(&p_mad_bind_info->timeout_list_lock); + cl_spinlock_init(&p_mad_bind_info->timeout_list_lock); + + cl_status = cl_timer_init(&p_mad_bind_info->timeout_timer, + __osm_vendor_timer_callback, + (void *)p_mad_bind_info); + CL_ASSERT(cl_status == CL_SUCCESS); + cl_qlist_init(&p_mad_bind_info->timeout_list); + /* */ + /* Insert the mad_reg_struct in list and return pointer to it as the handle */ + /* */ + cl_spinlock_acquire(&p_umadt_obj->register_lock); + + cl_qlist_insert_head(&p_umadt_obj->register_list, + &p_mad_bind_info->list_item); + + cl_spinlock_release(&p_umadt_obj->register_lock); + + /* + A timeout value of 0 means disable timeouts. + */ + if (p_umadt_obj->timeout) { + cl_timer_start(&p_mad_bind_info->timeout_timer, + DEFAULT_TIMER_INTERVAL_MSEC); + } + + OSM_LOG_EXIT(p_umadt_obj->p_log); + return ((osm_bind_handle_t) p_mad_bind_info); } /********************************************************************** **********************************************************************/ -void -osm_vendor_unbind(IN osm_bind_handle_t h_bind) +void osm_vendor_unbind(IN osm_bind_handle_t h_bind) { - mad_bind_info_t *p_mad_bind_info; - umadt_obj_t *p_umadt_obj; - cl_list_item_t *p_list_item, *p_next_list_item; - - CL_ASSERT(h_bind); - p_mad_bind_info = (mad_bind_info_t*)h_bind; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - - /* sanity check */ - CL_ASSERT(p_umadt_obj); - CL_ASSERT(p_umadt_obj->init_done); - CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); - - p_umadt_obj->uMadtInterface.uMadtDestroy( &p_mad_bind_info->umadt_handle); - cl_timer_destroy(&p_mad_bind_info->timeout_timer); - cl_thread_destroy(&p_mad_bind_info->recv_processor_thread); - - cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); - p_list_item = cl_qlist_head(&p_mad_bind_info->trans_ctxt_list); - while ( p_list_item != cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) - { - p_next_list_item = cl_qlist_next(p_list_item); - cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, - p_list_item); - free(p_list_item); - p_list_item = p_next_list_item; - } - cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); - - cl_spinlock_acquire(&p_mad_bind_info->timeout_list_lock); - p_list_item = cl_qlist_head(&p_mad_bind_info->timeout_list); - while ( p_list_item != cl_qlist_end(&p_mad_bind_info->timeout_list)) - { - p_next_list_item = cl_qlist_next(p_list_item); - cl_qlist_remove_item(&p_mad_bind_info->timeout_list, - p_list_item); - free(p_list_item); - p_list_item = p_next_list_item; - } - cl_spinlock_release(&p_mad_bind_info->timeout_list_lock); - - free(p_mad_bind_info); + mad_bind_info_t *p_mad_bind_info; + umadt_obj_t *p_umadt_obj; + cl_list_item_t *p_list_item, *p_next_list_item; + + CL_ASSERT(h_bind); + p_mad_bind_info = (mad_bind_info_t *) h_bind; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + + /* sanity check */ + CL_ASSERT(p_umadt_obj); + CL_ASSERT(p_umadt_obj->init_done); + CL_ASSERT(__valid_mad_handle(p_mad_bind_info)); + + p_umadt_obj->uMadtInterface.uMadtDestroy(&p_mad_bind_info-> + umadt_handle); + cl_timer_destroy(&p_mad_bind_info->timeout_timer); + cl_thread_destroy(&p_mad_bind_info->recv_processor_thread); + + cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); + p_list_item = cl_qlist_head(&p_mad_bind_info->trans_ctxt_list); + while (p_list_item != cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) { + p_next_list_item = cl_qlist_next(p_list_item); + cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, + p_list_item); + free(p_list_item); + p_list_item = p_next_list_item; + } + cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); + + cl_spinlock_acquire(&p_mad_bind_info->timeout_list_lock); + p_list_item = cl_qlist_head(&p_mad_bind_info->timeout_list); + while (p_list_item != cl_qlist_end(&p_mad_bind_info->timeout_list)) { + p_next_list_item = cl_qlist_next(p_list_item); + cl_qlist_remove_item(&p_mad_bind_info->timeout_list, + p_list_item); + free(p_list_item); + p_list_item = p_next_list_item; + } + cl_spinlock_release(&p_mad_bind_info->timeout_list_lock); + + free(p_mad_bind_info); } /********************************************************************** **********************************************************************/ -void -__mad_recv_processor( - IN void* context) +void __mad_recv_processor(IN void *context) { - mad_bind_info_t *p_mad_bind_info = (mad_bind_info_t*)context; - umadt_obj_t *p_umadt_obj; - osm_madw_t *p_osm_madw = NULL; - osm_vend_wrap_t *p_vend_wrap = NULL; - osm_mad_addr_t osm_mad_addr = {0}; - cl_list_item_t *p_list_item; - void* transaction_context; - - FSTATUS Status; - MadtStruct *pRecvMad = NULL; - MadWorkCompletion *pRecvCmp = NULL; - - - CL_ASSERT(context); - - p_mad_bind_info = (mad_bind_info_t*)context; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - /* PollFor a completion */ - /* if FNOTFOND, then wait for a completion then again poll and return the MAD */ - while(1) - { - Status = p_umadt_obj->uMadtInterface.uMadtPollForRecvCompletion(p_mad_bind_info->umadt_handle, - &pRecvMad, - &pRecvCmp); - if(Status != FSUCCESS) - { - if ( Status == FNOT_FOUND) - { - /* Wait for a completion */ - Status = p_umadt_obj->uMadtInterface.uMadtWaitForAnyCompletion( - p_mad_bind_info->umadt_handle, - RECV_COMPLETION, - 0x5000); /* 5 sec timeout */ - - if(Status == FTIMEOUT) - { - continue; - } - CL_ASSERT(Status == FSUCCESS); - - Status = p_umadt_obj->uMadtInterface.uMadtPollForRecvCompletion( - p_mad_bind_info->umadt_handle, - &pRecvMad, - &pRecvCmp); - if ( Status != FSUCCESS ) - { - printf(" mad_recv_worker: Error in PollForRecv returning <%x>\n", Status); - CL_ASSERT(0); - } - } - else { - printf("uMadtPollForRecvCompletion Status=<%x>\n", Status); - CL_ASSERT(0); - } - } - CL_ASSERT(pRecvMad); - CL_ASSERT(pRecvCmp); - - if (((ib_sa_mad_t_vM3*)(&pRecvMad->IBMad))->frag_flag & 0x20 ) - { - /* Ignore the ACK packet */ - Status = p_umadt_obj->uMadtInterface.uMadtReleaseRecvMad( - p_mad_bind_info->umadt_handle, - pRecvMad); - continue; - } - /* */ - /* Extract the return address to pass it on to the client */ - /* */ - osm_mad_addr.dest_lid = pRecvCmp->AddressInfo.DestLid; - osm_mad_addr.path_bits = pRecvCmp->AddressInfo.PathBits; - osm_mad_addr.static_rate = pRecvCmp->AddressInfo.StaticRate; - - if (p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_LID || - p_mad_bind_info->umadt_reg_class.ClassId == IB_MCLASS_SUBN_DIR) - { - osm_mad_addr.addr_type.smi.source_lid = pRecvCmp->AddressInfo.AddrType.Smi.SourceLid; - /* osm_mad_addr.addr_type.smi.port_num = pRecvCmp->AddressInfo.AddrType.Smi.PortNumber; */ - } - else - { - osm_mad_addr.addr_type.gsi.remote_qp = pRecvCmp->AddressInfo.AddrType.Gsi.RemoteQpNumber; - osm_mad_addr.addr_type.gsi.remote_qkey = pRecvCmp->AddressInfo.AddrType.Gsi.RemoteQkey; - osm_mad_addr.addr_type.gsi.pkey = pRecvCmp->AddressInfo.AddrType.Gsi.PKey; - osm_mad_addr.addr_type.gsi.service_level = pRecvCmp->AddressInfo.AddrType.Gsi.ServiceLevel; - osm_mad_addr.addr_type.gsi.global_route = pRecvCmp->AddressInfo.AddrType.Gsi.GlobalRoute; - /* osm_mad_addr.addr_type.gsi.grh_info = pRecvCmp->AddressInfo.AddrType.Gsi.GRHInfo; */ - } - p_osm_madw = osm_mad_pool_get_wrapper(p_mad_bind_info->p_mad_pool, - p_mad_bind_info, - MAD_BLOCK_SIZE, - (ib_mad_t*)&pRecvMad->IBMad, - &osm_mad_addr); - CL_ASSERT(p_osm_madw); - p_vend_wrap = osm_madw_get_vend_ptr(p_osm_madw); - CL_ASSERT(p_vend_wrap); - p_vend_wrap->p_madt_struct = pRecvMad; - p_vend_wrap->direction = RECEIVE; - - osm_log( p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, - "__mad_recv_processor: " - "Received data p_osm_madw[0x%p].\n", - p_osm_madw) ; - - /* */ - /* Do TID Processing. */ - /* */ - /* If R bit is set swap the TID */ - - cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); - p_list_item= cl_qlist_find_from_head( - &p_mad_bind_info->trans_ctxt_list, - __match_tid_context, - &p_osm_madw->p_mad->trans_id); - - if (p_list_item == cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) - { - transaction_context = NULL; - } - else - { - transaction_context =((trans_context_t*)p_list_item)->context; - cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, - p_list_item); - free(p_list_item); - } - cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); - ((ib_mad_t*)p_osm_madw->p_mad)->trans_id = cl_ntoh64(p_osm_madw->p_mad->trans_id>>24); - osm_log( p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, - "__mad_recv_processor: " - "Received data p_osm_madw [0x%p]" - "\n\t\t\t\tTID[0x%"PRIx64", context[%p]. \n", - p_osm_madw, ((ib_mad_t*)p_osm_madw->p_mad)->trans_id, - transaction_context) ; - - (*(p_mad_bind_info->mad_recv_callback))(p_osm_madw, - p_mad_bind_info->client_context, - transaction_context); - - } + mad_bind_info_t *p_mad_bind_info = (mad_bind_info_t *) context; + umadt_obj_t *p_umadt_obj; + osm_madw_t *p_osm_madw = NULL; + osm_vend_wrap_t *p_vend_wrap = NULL; + osm_mad_addr_t osm_mad_addr = { 0 }; + cl_list_item_t *p_list_item; + void *transaction_context; + + FSTATUS Status; + MadtStruct *pRecvMad = NULL; + MadWorkCompletion *pRecvCmp = NULL; + + CL_ASSERT(context); + + p_mad_bind_info = (mad_bind_info_t *) context; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + /* PollFor a completion */ + /* if FNOTFOND, then wait for a completion then again poll and return the MAD */ + while (1) { + Status = + p_umadt_obj->uMadtInterface. + uMadtPollForRecvCompletion(p_mad_bind_info->umadt_handle, + &pRecvMad, &pRecvCmp); + if (Status != FSUCCESS) { + if (Status == FNOT_FOUND) { + /* Wait for a completion */ + Status = p_umadt_obj->uMadtInterface.uMadtWaitForAnyCompletion(p_mad_bind_info->umadt_handle, RECV_COMPLETION, 0x5000); /* 5 sec timeout */ + + if (Status == FTIMEOUT) { + continue; + } + CL_ASSERT(Status == FSUCCESS); + + Status = + p_umadt_obj->uMadtInterface. + uMadtPollForRecvCompletion(p_mad_bind_info-> + umadt_handle, + &pRecvMad, + &pRecvCmp); + if (Status != FSUCCESS) { + printf + (" mad_recv_worker: Error in PollForRecv returning <%x>\n", + Status); + CL_ASSERT(0); + } + } else { + printf + ("uMadtPollForRecvCompletion Status=<%x>\n", + Status); + CL_ASSERT(0); + } + } + CL_ASSERT(pRecvMad); + CL_ASSERT(pRecvCmp); + + if (((ib_sa_mad_t_vM3 *) (&pRecvMad->IBMad))->frag_flag & 0x20) { + /* Ignore the ACK packet */ + Status = + p_umadt_obj->uMadtInterface. + uMadtReleaseRecvMad(p_mad_bind_info->umadt_handle, + pRecvMad); + continue; + } + /* */ + /* Extract the return address to pass it on to the client */ + /* */ + osm_mad_addr.dest_lid = pRecvCmp->AddressInfo.DestLid; + osm_mad_addr.path_bits = pRecvCmp->AddressInfo.PathBits; + osm_mad_addr.static_rate = pRecvCmp->AddressInfo.StaticRate; + + if (p_mad_bind_info->umadt_reg_class.ClassId == + IB_MCLASS_SUBN_LID + || p_mad_bind_info->umadt_reg_class.ClassId == + IB_MCLASS_SUBN_DIR) { + osm_mad_addr.addr_type.smi.source_lid = + pRecvCmp->AddressInfo.AddrType.Smi.SourceLid; + /* osm_mad_addr.addr_type.smi.port_num = pRecvCmp->AddressInfo.AddrType.Smi.PortNumber; */ + } else { + osm_mad_addr.addr_type.gsi.remote_qp = + pRecvCmp->AddressInfo.AddrType.Gsi.RemoteQpNumber; + osm_mad_addr.addr_type.gsi.remote_qkey = + pRecvCmp->AddressInfo.AddrType.Gsi.RemoteQkey; + osm_mad_addr.addr_type.gsi.pkey = + pRecvCmp->AddressInfo.AddrType.Gsi.PKey; + osm_mad_addr.addr_type.gsi.service_level = + pRecvCmp->AddressInfo.AddrType.Gsi.ServiceLevel; + osm_mad_addr.addr_type.gsi.global_route = + pRecvCmp->AddressInfo.AddrType.Gsi.GlobalRoute; + /* osm_mad_addr.addr_type.gsi.grh_info = pRecvCmp->AddressInfo.AddrType.Gsi.GRHInfo; */ + } + p_osm_madw = + osm_mad_pool_get_wrapper(p_mad_bind_info->p_mad_pool, + p_mad_bind_info, MAD_BLOCK_SIZE, + (ib_mad_t *) & pRecvMad->IBMad, + &osm_mad_addr); + CL_ASSERT(p_osm_madw); + p_vend_wrap = osm_madw_get_vend_ptr(p_osm_madw); + CL_ASSERT(p_vend_wrap); + p_vend_wrap->p_madt_struct = pRecvMad; + p_vend_wrap->direction = RECEIVE; + + osm_log(p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, + "__mad_recv_processor: " + "Received data p_osm_madw[0x%p].\n", p_osm_madw); + + /* */ + /* Do TID Processing. */ + /* */ + /* If R bit is set swap the TID */ + + cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); + p_list_item = + cl_qlist_find_from_head(&p_mad_bind_info->trans_ctxt_list, + __match_tid_context, + &p_osm_madw->p_mad->trans_id); + + if (p_list_item == + cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) { + transaction_context = NULL; + } else { + transaction_context = + ((trans_context_t *) p_list_item)->context; + cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, + p_list_item); + free(p_list_item); + } + cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); + ((ib_mad_t *) p_osm_madw->p_mad)->trans_id = + cl_ntoh64(p_osm_madw->p_mad->trans_id >> 24); + osm_log(p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, + "__mad_recv_processor: " + "Received data p_osm_madw [0x%p]" "\n\t\t\t\tTID[0x%" + PRIx64 ", context[%p]. \n", p_osm_madw, + ((ib_mad_t *) p_osm_madw->p_mad)->trans_id, + transaction_context); + + (*(p_mad_bind_info->mad_recv_callback)) (p_osm_madw, + p_mad_bind_info-> + client_context, + transaction_context); + + } } + /********************************************************************** **********************************************************************/ - cl_status_t -__match_tid_context( const cl_list_item_t* const p_list_item, - void* context) +__match_tid_context(const cl_list_item_t * const p_list_item, void *context) { - if (((trans_context_t*)p_list_item)->trans_id == *((uint64_t*)context)) - return CL_SUCCESS; - return CL_NOT_FOUND; + if (((trans_context_t *) p_list_item)->trans_id == + *((uint64_t *) context)) + return CL_SUCCESS; + return CL_NOT_FOUND; } + /********************************************************************** **********************************************************************/ -boolean_t -__valid_mad_handle( - IN mad_bind_info_t *p_mad_bind_info - ) +boolean_t __valid_mad_handle(IN mad_bind_info_t * p_mad_bind_info) { - umadt_obj_t *p_umadt_obj; + umadt_obj_t *p_umadt_obj; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; - cl_spinlock_acquire(&p_umadt_obj->register_lock); - if(!cl_is_item_in_qlist(&p_umadt_obj->register_list, - &p_mad_bind_info->list_item)) - { - cl_spinlock_release(&p_umadt_obj->register_lock); - return FALSE; - } - cl_spinlock_release(&p_umadt_obj->register_lock); - return TRUE; + cl_spinlock_acquire(&p_umadt_obj->register_lock); + if (!cl_is_item_in_qlist(&p_umadt_obj->register_list, + &p_mad_bind_info->list_item)) { + cl_spinlock_release(&p_umadt_obj->register_lock); + return FALSE; + } + cl_spinlock_release(&p_umadt_obj->register_lock); + return TRUE; } -void -__osm_vendor_timer_callback( - IN void* context ) +void __osm_vendor_timer_callback(IN void *context) { - uint64_t current_time; - mad_bind_info_t *p_mad_bind_info; - umadt_obj_t *p_umadt_obj; - uint32_t timeout; - - cl_list_item_t *p_list_item, *p_next_list_item; + uint64_t current_time; + mad_bind_info_t *p_mad_bind_info; + umadt_obj_t *p_umadt_obj; + uint32_t timeout; - CL_ASSERT(context); + cl_list_item_t *p_list_item, *p_next_list_item; - p_mad_bind_info = (mad_bind_info_t*)context; - p_umadt_obj = p_mad_bind_info->p_umadt_obj; - timeout = p_umadt_obj->timeout * 1000; + CL_ASSERT(context); - current_time = cl_get_time_stamp(); + p_mad_bind_info = (mad_bind_info_t *) context; + p_umadt_obj = p_mad_bind_info->p_umadt_obj; + timeout = p_umadt_obj->timeout * 1000; - cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); + current_time = cl_get_time_stamp(); - p_list_item = cl_qlist_head(&p_mad_bind_info->trans_ctxt_list); - while ( p_list_item != cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) - { + cl_spinlock_acquire(&p_mad_bind_info->trans_ctxt_lock); - p_next_list_item = cl_qlist_next(p_list_item); + p_list_item = cl_qlist_head(&p_mad_bind_info->trans_ctxt_list); + while (p_list_item != cl_qlist_end(&p_mad_bind_info->trans_ctxt_list)) { - /* DEFAULT_PKT_TIMEOUT is in milli seconds */ - if ( current_time - ((trans_context_t*)p_list_item)->sent_time - > timeout ) - { - /* Add this transaction to the timeout_list */ - cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, - p_list_item); - cl_qlist_insert_tail(&p_mad_bind_info->timeout_list, - p_list_item ); - } + p_next_list_item = cl_qlist_next(p_list_item); - p_list_item = p_next_list_item; - } + /* DEFAULT_PKT_TIMEOUT is in milli seconds */ + if (current_time - ((trans_context_t *) p_list_item)->sent_time + > timeout) { + /* Add this transaction to the timeout_list */ + cl_qlist_remove_item(&p_mad_bind_info->trans_ctxt_list, + p_list_item); + cl_qlist_insert_tail(&p_mad_bind_info->timeout_list, + p_list_item); + } - cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); + p_list_item = p_next_list_item; + } - p_list_item = cl_qlist_head(&p_mad_bind_info->timeout_list); - while ( p_list_item != cl_qlist_end(&p_mad_bind_info->timeout_list)) - { - osm_log( p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, - "__osm_vendor_timer_callback: " - "Timing out transaction context [0x%p].\n", - ((trans_context_t*)p_list_item)->context) ; + cl_spinlock_release(&p_mad_bind_info->trans_ctxt_lock); + p_list_item = cl_qlist_head(&p_mad_bind_info->timeout_list); + while (p_list_item != cl_qlist_end(&p_mad_bind_info->timeout_list)) { + osm_log(p_mad_bind_info->p_umadt_obj->p_log, OSM_LOG_DEBUG, + "__osm_vendor_timer_callback: " + "Timing out transaction context [0x%p].\n", + ((trans_context_t *) p_list_item)->context); - (*(p_mad_bind_info->mad_recv_callback))(NULL, - p_mad_bind_info->client_context, - ((trans_context_t*)p_list_item)->context); + (*(p_mad_bind_info->mad_recv_callback)) (NULL, + p_mad_bind_info-> + client_context, + ((trans_context_t *) + p_list_item)-> + context); - p_next_list_item = cl_qlist_next(p_list_item); - cl_qlist_remove_item(&p_mad_bind_info->timeout_list, - p_list_item); - free(p_list_item); - p_list_item = p_next_list_item; - } + p_next_list_item = cl_qlist_next(p_list_item); + cl_qlist_remove_item(&p_mad_bind_info->timeout_list, + p_list_item); + free(p_list_item); + p_list_item = p_next_list_item; + } - cl_timer_start(&p_mad_bind_info->timeout_timer, DEFAULT_TIMER_INTERVAL_MSEC); + cl_timer_start(&p_mad_bind_info->timeout_timer, + DEFAULT_TIMER_INTERVAL_MSEC); } - -#endif /* OSM_VENDOR_INTF_UMADT */ +#endif /* OSM_VENDOR_INTF_UMADT */ -- 1.5.3.rc2.29.gc4640f From ilpo.jarvinen at helsinki.fi Wed Aug 15 15:02:07 2007 From: ilpo.jarvinen at helsinki.fi (=?ISO-8859-1?Q?Ilpo_J=E4rvinen?=) Date: Thu, 16 Aug 2007 01:02:07 +0300 (EEST) Subject: [ofa-general] [PATCH] IB/mlx4: Invalid semicolon after if statement Message-ID: A similar fix to netfilter from Eric Dumazet inspired me to look around a bit by using some grep/sed stuff as looking for this kind of bugs seemed easy to automate. This is one of them I found where it looks like this semicolon is not valid. Signed-off-by: Ilpo Järvinen --- ...I've absolutely no idea what your code is doing, so you're the ones that can figure out if this is valid or not (though indentation which I used as trigger to sed seems to point towards an extra samicolon bug). drivers/infiniband/hw/mlx4/mad.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index 3330917..0ed02b7 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -109,7 +109,7 @@ int mlx4_MAD_IFC(struct mlx4_ib_dev *dev, int ignore_mkey, int ignore_bkey, in_modifier, op_modifier, MLX4_CMD_MAD_IFC, MLX4_CMD_TIME_CLASS_C); - if (!err); + if (!err) memcpy(response_mad, outmailbox->buf, 256); mlx4_free_cmd_mailbox(dev->dev, inmailbox); -- 1.5.0.6 From sashak at voltaire.com Wed Aug 15 15:22:10 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 16 Aug 2007 01:22:10 +0300 Subject: [ofa-general] [PATCH] opensm/include/opensm: indentation changes Message-ID: <20070815222210.GC27000@sashak.voltaire.com> Formatted with opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs) Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_attrib_req.h | 23 +- opensm/include/opensm/osm_base.h | 96 ++---- opensm/include/opensm/osm_console.h | 16 +- opensm/include/opensm/osm_db.h | 72 +--- opensm/include/opensm/osm_db_pack.h | 44 +-- opensm/include/opensm/osm_drop_mgr.h | 40 +-- opensm/include/opensm/osm_errors.h | 2 +- opensm/include/opensm/osm_event_plugin.h | 118 +++--- opensm/include/opensm/osm_fwd_tbl.h | 113 +++--- opensm/include/opensm/osm_helper.h | 270 +++++-------- opensm/include/opensm/osm_inform.h | 56 +-- opensm/include/opensm/osm_lid_mgr.h | 59 +-- opensm/include/opensm/osm_lin_fwd_rcv.h | 37 +-- opensm/include/opensm/osm_lin_fwd_tbl.h | 96 ++--- opensm/include/opensm/osm_link_mgr.h | 45 +-- opensm/include/opensm/osm_log.h | 113 +++--- opensm/include/opensm/osm_mad_pool.h | 67 ++-- opensm/include/opensm/osm_madw.h | 427 ++++++++++----------- opensm/include/opensm/osm_mcast_fwd_rcv.h | 39 +-- opensm/include/opensm/osm_mcast_mgr.h | 56 +-- opensm/include/opensm/osm_mcast_tbl.h | 96 ++--- opensm/include/opensm/osm_mcm_info.h | 23 +- opensm/include/opensm/osm_mcm_port.h | 31 +- opensm/include/opensm/osm_msgdef.h | 16 +- opensm/include/opensm/osm_mtree.h | 59 ++-- opensm/include/opensm/osm_multicast.h | 143 +++---- opensm/include/opensm/osm_node.h | 179 ++++----- opensm/include/opensm/osm_node_desc_rcv.h | 37 +-- opensm/include/opensm/osm_node_info_rcv.h | 48 +-- opensm/include/opensm/osm_opensm.h | 109 +++--- opensm/include/opensm/osm_partition.h | 45 +-- opensm/include/opensm/osm_path.h | 55 +-- opensm/include/opensm/osm_perfmgr.h | 190 +++++----- opensm/include/opensm/osm_perfmgr_db.h | 149 ++++---- opensm/include/opensm/osm_pkey.h | 160 ++++----- opensm/include/opensm/osm_pkey_mgr.h | 12 +- opensm/include/opensm/osm_pkey_rcv.h | 41 +-- opensm/include/opensm/osm_port.h | 479 +++++++++++------------ opensm/include/opensm/osm_port_info_rcv.h | 45 +-- opensm/include/opensm/osm_port_profile.h | 92 ++--- opensm/include/opensm/osm_rand_fwd_tbl.h | 105 +++--- opensm/include/opensm/osm_remote_sm.h | 33 +- opensm/include/opensm/osm_req.h | 72 ++-- opensm/include/opensm/osm_resp.h | 46 +-- opensm/include/opensm/osm_router.h | 40 +-- opensm/include/opensm/osm_sa.h | 141 +++---- opensm/include/opensm/osm_sa_class_port_info.h | 47 +-- opensm/include/opensm/osm_sa_guidinfo_record.h | 49 +-- opensm/include/opensm/osm_sa_informinfo.h | 54 +-- opensm/include/opensm/osm_sa_lft_record.h | 49 +-- opensm/include/opensm/osm_sa_link_record.h | 48 +-- opensm/include/opensm/osm_sa_mad_ctrl.h | 68 ++-- opensm/include/opensm/osm_sa_mcmember_record.h | 80 ++--- opensm/include/opensm/osm_sa_mft_record.h | 49 +-- opensm/include/opensm/osm_sa_multipath_record.h | 49 +-- opensm/include/opensm/osm_sa_node_record.h | 48 +-- opensm/include/opensm/osm_sa_path_record.h | 49 +-- opensm/include/opensm/osm_sa_pkey_record.h | 49 +-- opensm/include/opensm/osm_sa_portinfo_record.h | 49 +-- opensm/include/opensm/osm_sa_response.h | 37 +-- opensm/include/opensm/osm_sa_service_record.h | 55 +-- opensm/include/opensm/osm_sa_slvl_record.h | 49 +-- opensm/include/opensm/osm_sa_sminfo_record.h | 50 +-- opensm/include/opensm/osm_sa_sw_info_record.h | 54 +-- opensm/include/opensm/osm_sa_vlarb_record.h | 50 +-- opensm/include/opensm/osm_service.h | 54 +-- opensm/include/opensm/osm_slvl_map_rcv.h | 41 +-- opensm/include/opensm/osm_sm.h | 173 ++++----- opensm/include/opensm/osm_sm_mad_ctrl.h | 70 ++-- opensm/include/opensm/osm_sm_state_mgr.h | 62 ++-- opensm/include/opensm/osm_sminfo_rcv.h | 54 ++-- opensm/include/opensm/osm_state_mgr.h | 156 +++----- opensm/include/opensm/osm_state_mgr_ctrl.h | 37 +-- opensm/include/opensm/osm_stats.h | 28 +- opensm/include/opensm/osm_subnet.h | 342 ++++++++--------- opensm/include/opensm/osm_sw_info_rcv.h | 48 +-- opensm/include/opensm/osm_sweep_fail_ctrl.h | 38 +-- opensm/include/opensm/osm_switch.h | 315 +++++++-------- opensm/include/opensm/osm_trap_rcv.h | 57 ++-- opensm/include/opensm/osm_ts_useraccess.h | 14 +- opensm/include/opensm/osm_ucast_mgr.h | 65 ++-- opensm/include/opensm/osm_umadt.h | 61 ++-- opensm/include/opensm/osm_version.h | 9 +- opensm/include/opensm/osm_vl15intf.h | 82 ++--- opensm/include/opensm/osm_vl_arb_rcv.h | 41 +-- opensm/include/opensm/st.h | 27 +- 86 files changed, 2960 insertions(+), 4032 deletions(-) diff --git a/opensm/include/opensm/osm_attrib_req.h b/opensm/include/opensm/osm_attrib_req.h index 97cd4dc..be993a5 100644 --- a/opensm/include/opensm/osm_attrib_req.h +++ b/opensm/include/opensm/osm_attrib_req.h @@ -41,13 +41,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * Abstract: * Declaration of the attribute request object. This object @@ -60,7 +59,6 @@ BEGIN_C_DECLS * * $Revision: 1.4 $ */ - /****h* OpenSM/Attribute Request * NAME * Attribute Request @@ -76,7 +74,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Attribute Request/osm_attrib_req_t * NAME * osm_attrib_req_t @@ -88,13 +85,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_attrib_req -{ - uint16_t attrib_id; - uint32_t attrib_mod; - osm_madw_context_t context; - osm_dr_path_t path; - cl_disp_msgid_t err_msg; +typedef struct _osm_attrib_req { + uint16_t attrib_id; + uint32_t attrib_mod; + osm_madw_context_t context; + osm_dr_path_t path; + cl_disp_msgid_t err_msg; } osm_attrib_req_t; /* @@ -115,5 +111,4 @@ typedef struct _osm_attrib_req *********/ END_C_DECLS - -#endif /* _OSM_ATTRIB_REQ_H_ */ +#endif /* _OSM_ATTRIB_REQ_H_ */ diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h index 4bf766e..545779b 100644 --- a/opensm/include/opensm/osm_base.h +++ b/opensm/include/opensm/osm_base.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Constants * NAME * Constants @@ -78,7 +77,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****h* OpenSM/Base * NAME * Base @@ -95,7 +93,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Base/OSM_DEFAULT_M_KEY * NAME * OSM_DEFAULT_M_KEY @@ -107,7 +104,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_M_KEY 0 /********/ - /****s* OpenSM: Base/OSM_DEFAULT_SM_KEY * NAME * OSM_DEFAULT_SM_KEY @@ -119,7 +115,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SM_KEY 1 /********/ - /****s* OpenSM: Base/OSM_DEFAULT_LMC * NAME * OSM_DEFAULT_LMC @@ -131,7 +126,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_LMC 0 /********/ - /****s* OpenSM: Base/OSM_DEFAULT_MAX_OP_VLS * NAME * OSM_DEFAULT_MAX_OP_VLS @@ -145,7 +139,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_MAX_OP_VLS 5 /********/ - /****s* OpenSM: Base/OSM_DEFAULT_SL * NAME * OSM_DEFAULT_SL @@ -157,7 +150,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SL 0 /********/ - /****s* OpenSM: Base/OSM_DEFAULT_SM_PRIORITY * NAME * OSM_DEFAULT_SM_PRIORITY @@ -170,7 +162,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SM_PRIORITY 0 /********/ - /****d* OpenSM: Base/OSM_DEFAULT_TMP_DIR * NAME * OSM_DEFAULT_TMP_DIR @@ -187,7 +178,6 @@ BEGIN_C_DECLS #define OSM_DEFAULT_TMP_DIR "/var/log/" #endif /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_CACHE_DIR * NAME * OSM_DEFAULT_CACHE_DIR @@ -204,7 +194,6 @@ BEGIN_C_DECLS #define OSM_DEFAULT_CACHE_DIR "/var/cache/opensm/" #endif /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_LOG_FILE * NAME * OSM_DEFAULT_LOG_FILE @@ -220,7 +209,6 @@ BEGIN_C_DECLS #define OSM_DEFAULT_LOG_FILE "/var/log/opensm.log" #endif /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_PARTITION_CONFIG_FILE * NAME * OSM_DEFAULT_PARTITION_CONFIG_FILE @@ -236,7 +224,6 @@ BEGIN_C_DECLS #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" #endif /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS * NAME * OSM_DEFAULT_SWEEP_INTERVAL_SECS @@ -248,7 +235,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SWEEP_INTERVAL_SECS 10 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC * NAME * OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC @@ -260,7 +246,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC 200 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_SUBNET_TIMEOUT * NAME * OSM_DEFAULT_SUBNET_TIMEOUT @@ -274,7 +259,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SUBNET_TIMEOUT 0x12 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_SWITCH_PACKET_LIFE * NAME * OSM_DEFAULT_SWITCH_PACKET_LIFE @@ -289,7 +273,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SWITCH_PACKET_LIFE 0x12 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_HEAD_OF_QUEUE_LIFE * NAME * OSM_DEFAULT_HEAD_OF_QUEUE_LIFE @@ -303,7 +286,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_HEAD_OF_QUEUE_LIFE 0x12 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE * NAME * OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE @@ -317,7 +299,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE 0x10 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_VL_STALL_COUNT * NAME * OSM_DEFAULT_LEAF_VL_COUNT @@ -331,7 +312,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_VL_STALL_COUNT 0x7 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_LEAF_VL_STALL_COUNT * NAME * OSM_DEFAULT_LEAF_VL_STALL_COUNT @@ -346,7 +326,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_LEAF_VL_STALL_COUNT 0x7 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT * NAME * OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT @@ -360,7 +339,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT 5000000 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_UNHEALTHY_TIMEOUT * NAME * OSM_DEFAULT_UNHEALTHY_TIMEOUT @@ -374,7 +352,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_UNHEALTHY_TIMEOUT 60000000 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_ERROR_THRESHOLD * NAME * OSM_DEFAULT_ERROR_THRESHOLD @@ -386,7 +363,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_ERROR_THRESHOLD 0x08 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_SMP_MAX_ON_WIRE * NAME * OSM_DEFAULT_SMP_MAX_ON_WIRE @@ -399,7 +375,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_SMP_MAX_ON_WIRE 4 /***********/ - /****d* OpenSM: Base/OSM_SM_DEFAULT_QP0_RCV_SIZE * NAME * OSM_SM_DEFAULT_QP0_RCV_SIZE @@ -411,7 +386,6 @@ BEGIN_C_DECLS */ #define OSM_SM_DEFAULT_QP0_RCV_SIZE 256 /***********/ - /****d* OpenSM: Base/OSM_SM_DEFAULT_QP0_SEND_SIZE * NAME * OSM_SM_DEFAULT_QP0_SEND_SIZE @@ -423,7 +397,6 @@ BEGIN_C_DECLS */ #define OSM_SM_DEFAULT_QP0_SEND_SIZE 256 /***********/ - /****d* OpenSM: Base/OSM_SM_DEFAULT_QP1_RCV_SIZE * NAME * OSM_SM_DEFAULT_QP1_RCV_SIZE @@ -435,7 +408,6 @@ BEGIN_C_DECLS */ #define OSM_SM_DEFAULT_QP1_RCV_SIZE 256 /***********/ - /****d* OpenSM: Base/OSM_SM_DEFAULT_QP1_SEND_SIZE * NAME * OSM_SM_DEFAULT_QP1_SEND_SIZE @@ -446,7 +418,6 @@ BEGIN_C_DECLS * SYNOPSIS */ #define OSM_SM_DEFAULT_QP1_SEND_SIZE 256 - /****d* OpenSM: Base/OSM_PM_DEFAULT_QP1_RCV_SIZE * NAME * OSM_PM_DEFAULT_QP1_RCV_SIZE @@ -458,7 +429,6 @@ BEGIN_C_DECLS */ #define OSM_PM_DEFAULT_QP1_RCV_SIZE 256 /***********/ - /****d* OpenSM: Base/OSM_PM_DEFAULT_QP1_SEND_SIZE * NAME * OSM_PM_DEFAULT_QP1_SEND_SIZE @@ -469,8 +439,6 @@ BEGIN_C_DECLS * SYNOPSIS */ #define OSM_PM_DEFAULT_QP1_SEND_SIZE 256 - - /****d* OpenSM: Base/OSM_SM_DEFAULT_POLLING_TIMEOUT_MILLISECS * NAME * OSM_SM_DEFAULT_POLLING_TIMEOUT_MILLISECS @@ -483,7 +451,6 @@ BEGIN_C_DECLS */ #define OSM_SM_DEFAULT_POLLING_TIMEOUT_MILLISECS 10000 /**********/ - /****d* OpenSM: Base/OSM_SM_DEFAULT_POLLING_RETRY_NUMBER * NAME * OSM_SM_DEFAULT_POLLING_RETRY_NUMBER @@ -496,7 +463,6 @@ BEGIN_C_DECLS */ #define OSM_SM_DEFAULT_POLLING_RETRY_NUMBER 4 /**********/ - /****d* OpenSM: Base/OSM_DEFAULT_QOS_MAX_VLS * Name * OSM_DEFAULT_QOS_MAX_VLS @@ -508,7 +474,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_QOS_MAX_VLS 15 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_QOS_HIGH_LIMIT * Name * OSM_DEFAULT_QOS_HIGH_LIMIT @@ -520,7 +485,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_QOS_HIGH_LIMIT 0 /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_QOS_VLARB_HIGH * Name * OSM_DEFAULT_QOS_VLARB_HIGH @@ -532,7 +496,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_QOS_VLARB_HIGH "0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0" /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_QOS_VLARB_LOW * Name * OSM_DEFAULT_QOS_VLARB_LOW @@ -544,7 +507,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_QOS_VLARB_LOW "0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4" /***********/ - /****d* OpenSM: Base/OSM_DEFAULT_QOS_SL2VL * Name * OSM_DEFAULT_QOS_SL2VL @@ -556,7 +518,6 @@ BEGIN_C_DECLS */ #define OSM_DEFAULT_QOS_SL2VL "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7" /***********/ - /****d* OpenSM: Base/OSM_NO_PATH * NAME * OSM_NO_PATH @@ -568,7 +529,6 @@ BEGIN_C_DECLS */ #define OSM_NO_PATH 0xFF /**********/ - /****d* OpenSM: Base/osm_thread_state_t * NAME * osm_thread_state_t @@ -579,13 +539,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef enum _osm_thread_state -{ +typedef enum _osm_thread_state { OSM_THREAD_STATE_NONE = 0, OSM_THREAD_STATE_INIT, OSM_THREAD_STATE_RUN, OSM_THREAD_STATE_EXIT - } osm_thread_state_t; /***********/ @@ -695,8 +653,7 @@ typedef enum _osm_thread_state * * SYNOPSIS */ -typedef enum _osm_sm_state -{ +typedef enum _osm_sm_state { OSM_SM_STATE_NO_STATE = 0, OSM_SM_STATE_INIT, OSM_SM_STATE_IDLE, @@ -776,21 +733,19 @@ typedef uintn_t osm_signal_t; * * SYNOPSIS */ -typedef enum _osm_sm_signal -{ - OSM_SM_SIGNAL_INIT = 0, - OSM_SM_SIGNAL_DISCOVERY_COMPLETED, - OSM_SM_SIGNAL_POLLING_TIMEOUT, - OSM_SM_SIGNAL_DISCOVER, - OSM_SM_SIGNAL_DISABLE, - OSM_SM_SIGNAL_HANDOVER, - OSM_SM_SIGNAL_HANDOVER_SENT, - OSM_SM_SIGNAL_ACKNOWLEDGE, - OSM_SM_SIGNAL_STANDBY, - OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE, - OSM_SM_SIGNAL_WAIT_FOR_HANDOVER, - OSM_SM_SIGNAL_MAX - +typedef enum _osm_sm_signal { + OSM_SM_SIGNAL_INIT = 0, + OSM_SM_SIGNAL_DISCOVERY_COMPLETED, + OSM_SM_SIGNAL_POLLING_TIMEOUT, + OSM_SM_SIGNAL_DISCOVER, + OSM_SM_SIGNAL_DISABLE, + OSM_SM_SIGNAL_HANDOVER, + OSM_SM_SIGNAL_HANDOVER_SENT, + OSM_SM_SIGNAL_ACKNOWLEDGE, + OSM_SM_SIGNAL_STANDBY, + OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE, + OSM_SM_SIGNAL_WAIT_FOR_HANDOVER, + OSM_SM_SIGNAL_MAX } osm_sm_signal_t; /***********/ @@ -803,14 +758,12 @@ typedef enum _osm_sm_signal * * SYNOPSIS */ -typedef enum _osm_mcast_req_type -{ - OSM_MCAST_REQ_TYPE_CREATE, - OSM_MCAST_REQ_TYPE_JOIN, - OSM_MCAST_REQ_TYPE_LEAVE, - OSM_MCAST_REQ_TYPE_SUBNET_CHANGE - -} osm_mcast_req_type_t; +typedef enum _osm_mcast_req_type { + OSM_MCAST_REQ_TYPE_CREATE, + OSM_MCAST_REQ_TYPE_JOIN, + OSM_MCAST_REQ_TYPE_LEAVE, + OSM_MCAST_REQ_TYPE_SUBNET_CHANGE +} osm_mcast_req_type_t; /***********/ /****s* OpenSM: Base/MAX_GUID_FILE_LINE_LENGTH @@ -862,5 +815,4 @@ typedef enum _osm_mcast_req_type /**********/ END_C_DECLS - -#endif /* _OSM_BASE_H_ */ +#endif /* _OSM_BASE_H_ */ diff --git a/opensm/include/opensm/osm_console.h b/opensm/include/opensm/osm_console.h index 977a8d6..ceba3cc 100644 --- a/opensm/include/opensm/osm_console.h +++ b/opensm/include/opensm/osm_console.h @@ -47,18 +47,16 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - -void osm_console_init(osm_subn_opt_t *opt, osm_opensm_t *p_osm); -void osm_console(osm_opensm_t *p_osm); -void osm_console_prompt(FILE *out); -void osm_console_close_socket(osm_opensm_t *p_osm); +void osm_console_init(osm_subn_opt_t * opt, osm_opensm_t * p_osm); +void osm_console(osm_opensm_t * p_osm); +void osm_console_prompt(FILE * out); +void osm_console_close_socket(osm_opensm_t * p_osm); END_C_DECLS - -#endif /* _OSM_CONSOLE_H_ */ +#endif /* _OSM_CONSOLE_H_ */ diff --git a/opensm/include/opensm/osm_db.h b/opensm/include/opensm/osm_db.h index dbb5d74..eb05f45 100644 --- a/opensm/include/opensm/osm_db.h +++ b/opensm/include/opensm/osm_db.h @@ -50,13 +50,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Database * NAME * Database @@ -79,7 +78,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox Technologies LTD * *********/ - /****s* OpenSM: Database/osm_db_domain_t * NAME * osm_db_domain_t @@ -93,8 +91,8 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef struct _osm_db_domain { - struct _osm_db *p_db; - void *p_domain_imp; + struct _osm_db *p_db; + void *p_domain_imp; } osm_db_domain_t; /* * FIELDS @@ -120,11 +118,10 @@ typedef struct _osm_db_domain { * * SYNOPSIS */ -typedef struct _osm_db -{ - void *p_db_imp; - osm_log_t *p_log; - cl_list_t domains; +typedef struct _osm_db { + void *p_db_imp; + osm_log_t *p_log; + cl_list_t domains; } osm_db_t; /* * FIELDS @@ -149,9 +146,7 @@ typedef struct _osm_db * * SYNOPSIS */ -void -osm_db_construct( - IN osm_db_t* const p_db ); +void osm_db_construct(IN osm_db_t * const p_db); /* * PARAMETERS * p_db @@ -173,9 +168,7 @@ osm_db_construct( * * SYNOPSIS */ -void -osm_db_destroy( - IN osm_db_t* const p_db ); +void osm_db_destroy(IN osm_db_t * const p_db); /* * PARAMETERS * p_db @@ -194,10 +187,7 @@ osm_db_destroy( * * SYNOPSIS */ -int -osm_db_init( - IN osm_db_t* const p_db, - IN osm_log_t *p_log ); +int osm_db_init(IN osm_db_t * const p_db, IN osm_log_t * p_log); /* * PARAMETERS * @@ -223,10 +213,8 @@ osm_db_init( * * SYNOPSIS */ -osm_db_domain_t* -osm_db_domain_init( - IN osm_db_t* const p_db, - IN char *domain_name); +osm_db_domain_t *osm_db_domain_init(IN osm_db_t * const p_db, + IN char *domain_name); /* * PARAMETERS * @@ -253,9 +241,7 @@ osm_db_domain_init( * * SYNOPSIS */ -int -osm_db_restore( - IN osm_db_domain_t *p_domain); +int osm_db_restore(IN osm_db_domain_t * p_domain); /* * PARAMETERS * @@ -279,9 +265,7 @@ osm_db_restore( * * SYNOPSIS */ -int -osm_db_clear( - IN osm_db_domain_t *p_domain); +int osm_db_clear(IN osm_db_domain_t * p_domain); /* * PARAMETERS * @@ -305,8 +289,7 @@ osm_db_clear( * * SYNOPSIS */ -int osm_db_store( - IN osm_db_domain_t *p_domain); +int osm_db_store(IN osm_db_domain_t * p_domain); /* * PARAMETERS * @@ -330,10 +313,7 @@ int osm_db_store( * * SYNOPSIS */ -int -osm_db_keys( - IN osm_db_domain_t *p_domain, - OUT cl_list_t* p_key_list); +int osm_db_keys(IN osm_db_domain_t * p_domain, OUT cl_list_t * p_key_list); /* * PARAMETERS * @@ -364,9 +344,7 @@ osm_db_keys( * SYNOPSIS */ /* lookup value by key */ -char *osm_db_lookup( - IN osm_db_domain_t *p_domain, - IN char *const p_key); +char *osm_db_lookup(IN osm_db_domain_t * p_domain, IN char *const p_key); /* * PARAMETERS * @@ -394,10 +372,8 @@ char *osm_db_lookup( * SYNOPSIS */ int -osm_db_update( - IN osm_db_domain_t *p_domain, - IN char *const p_key, - IN char *const p_val); +osm_db_update(IN osm_db_domain_t * p_domain, + IN char *const p_key, IN char *const p_val); /* * PARAMETERS * @@ -429,10 +405,7 @@ osm_db_update( * * SYNOPSIS */ -int -osm_db_delete( - IN osm_db_domain_t *p_domain, - IN char *const p_key); +int osm_db_delete(IN osm_db_domain_t * p_domain, IN char *const p_key); /* * PARAMETERS * @@ -451,5 +424,4 @@ osm_db_delete( *********/ END_C_DECLS - -#endif /* _OSM_DB_H_ */ +#endif /* _OSM_DB_H_ */ diff --git a/opensm/include/opensm/osm_db_pack.h b/opensm/include/opensm/osm_db_pack.h index 8e18920..45838da 100644 --- a/opensm/include/opensm/osm_db_pack.h +++ b/opensm/include/opensm/osm_db_pack.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****f* OpenSM: DB-Pack/osm_db_guid2lid_init * NAME * osm_db_guid2lid_init @@ -73,12 +72,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -static inline osm_db_domain_t* -osm_db_guid2lid_init( - IN osm_db_t* const p_db ) +static inline osm_db_domain_t *osm_db_guid2lid_init(IN osm_db_t * const p_db) { - return( osm_db_domain_init( p_db, "guid2lid" ) ); + return (osm_db_domain_init(p_db, "guid2lid")); } + /* * PARAMETERS * p_db @@ -103,8 +101,8 @@ osm_db_guid2lid_init( * SYNOPSIS */ typedef struct _osm_db_guid_elem { - cl_list_item_t item; - uint64_t guid; + cl_list_item_t item; + uint64_t guid; } osm_db_guid_elem_t; /* * FIELDS @@ -125,9 +123,8 @@ typedef struct _osm_db_guid_elem { * SYNOPSIS */ int -osm_db_guid2lid_guids( - IN osm_db_domain_t* const p_g2l, - OUT cl_qlist_t* p_guid_list ); +osm_db_guid2lid_guids(IN osm_db_domain_t * const p_g2l, + OUT cl_qlist_t * p_guid_list); /* * PARAMETERS * p_g2l @@ -157,11 +154,9 @@ osm_db_guid2lid_guids( * SYNOPSIS */ int -osm_db_guid2lid_get( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid, - OUT uint16_t *p_min_lid, - OUT uint16_t *p_max_lid); +osm_db_guid2lid_get(IN osm_db_domain_t * const p_g2l, + IN uint64_t guid, + OUT uint16_t * p_min_lid, OUT uint16_t * p_max_lid); /* * PARAMETERS * p_g2l @@ -194,11 +189,8 @@ osm_db_guid2lid_get( * SYNOPSIS */ int -osm_db_guid2lid_set( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid, - IN uint16_t min_lid, - IN uint16_t max_lid); +osm_db_guid2lid_set(IN osm_db_domain_t * const p_g2l, + IN uint64_t guid, IN uint16_t min_lid, IN uint16_t max_lid); /* * PARAMETERS * p_g2l @@ -230,10 +222,7 @@ osm_db_guid2lid_set( * * SYNOPSIS */ -int -osm_db_guid2lid_delete( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid ); +int osm_db_guid2lid_delete(IN osm_db_domain_t * const p_g2l, IN uint64_t guid); /* * PARAMETERS * p_g2l @@ -251,5 +240,4 @@ osm_db_guid2lid_delete( *********/ END_C_DECLS - -#endif /* _OSM_DB_PACK_H_ */ +#endif /* _OSM_DB_PACK_H_ */ diff --git a/opensm/include/opensm/osm_drop_mgr.h b/opensm/include/opensm/osm_drop_mgr.h index 91d6571..0f929e0 100644 --- a/opensm/include/opensm/osm_drop_mgr.h +++ b/opensm/include/opensm/osm_drop_mgr.h @@ -58,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Drop Manager * NAME * Drop Manager @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Drop Manager/osm_drop_mgr_t * NAME * osm_drop_mgr_t @@ -95,12 +93,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_drop_mgr -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - osm_req_t *p_req; - cl_plock_t *p_lock; +typedef struct _osm_drop_mgr { + osm_subn_t *p_subn; + osm_log_t *p_log; + osm_req_t *p_req; + cl_plock_t *p_lock; } osm_drop_mgr_t; /* @@ -130,8 +127,7 @@ typedef struct _osm_drop_mgr * * SYNOPSIS */ -void osm_drop_mgr_construct( - IN osm_drop_mgr_t* const p_mgr ); +void osm_drop_mgr_construct(IN osm_drop_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -161,8 +157,7 @@ void osm_drop_mgr_construct( * * SYNOPSIS */ -void osm_drop_mgr_destroy( - IN osm_drop_mgr_t* const p_mgr ); +void osm_drop_mgr_destroy(IN osm_drop_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -193,12 +188,11 @@ void osm_drop_mgr_destroy( * * SYNOPSIS */ -ib_api_status_t osm_drop_mgr_init( - IN osm_drop_mgr_t* const p_mgr, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_req_t* const p_req, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_drop_mgr_init(IN osm_drop_mgr_t * const p_mgr, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_req_t * const p_req, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_mgr @@ -237,8 +231,7 @@ ib_api_status_t osm_drop_mgr_init( * * SYNOPSIS */ -void osm_drop_mgr_process( - IN const osm_drop_mgr_t* const p_mgr ); +void osm_drop_mgr_process(IN const osm_drop_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -255,5 +248,4 @@ void osm_drop_mgr_process( *********/ END_C_DECLS - -#endif /* _OSM_DROP_MGR_H_ */ +#endif /* _OSM_DROP_MGR_H_ */ diff --git a/opensm/include/opensm/osm_errors.h b/opensm/include/opensm/osm_errors.h index 03f3dea..2c7a9ba 100644 --- a/opensm/include/opensm/osm_errors.h +++ b/opensm/include/opensm/osm_errors.h @@ -178,4 +178,4 @@ */ -#endif /* _OSM_ERRORS_H_ */ +#endif /* _OSM_ERRORS_H_ */ diff --git a/opensm/include/opensm/osm_event_plugin.h b/opensm/include/opensm/osm_event_plugin.h index b575d2a..099dc14 100644 --- a/opensm/include/opensm/osm_event_plugin.h +++ b/opensm/include/opensm/osm_event_plugin.h @@ -41,13 +41,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM Event plugin interface * DESCRIPTION * Database interface to record subnet events @@ -58,15 +57,12 @@ BEGIN_C_DECLS * Ira Weiny, LLNL * *********/ - #define OSM_EVENT_PLUGIN_NAME_NONE "NONE" - #ifdef ENABLE_OSM_DEFAULT_EVENT_PLUGIN #define OSM_DEFAULT_EVENT_PLUGIN_NAME "osmeventplugin" -#else /* !ENABLE_OSM_DEFAULT_EVENT_PLUGIN */ +#else /* !ENABLE_OSM_DEFAULT_EVENT_PLUGIN */ #define OSM_DEFAULT_EVENT_PLUGIN_NAME OSM_EVENT_PLUGIN_NAME_NONE -#endif /* ENABLE_OSM_DEFAULT_EVENT_PLUGIN */ - +#endif /* ENABLE_OSM_DEFAULT_EVENT_PLUGIN */ #define OSM_EPI_NODE_NAME_LEN (128) /** ========================================================================= * Event types @@ -80,9 +76,9 @@ typedef enum { } osm_epi_event_id_t; typedef struct { - uint64_t node_guid; - uint8_t port_num; - char node_name[OSM_EPI_NODE_NAME_LEN]; + uint64_t node_guid; + uint8_t port_num; + char node_name[OSM_EPI_NODE_NAME_LEN]; } osm_epi_port_id_t; /** ========================================================================= @@ -92,19 +88,19 @@ typedef struct { */ typedef struct { osm_epi_port_id_t port_id; - uint64_t symbol_err_cnt; - uint64_t link_err_recover; - uint64_t link_downed; - uint64_t rcv_err; - uint64_t rcv_rem_phys_err; - uint64_t rcv_switch_relay_err; - uint64_t xmit_discards; - uint64_t xmit_constraint_err; - uint64_t rcv_constraint_err; - uint64_t link_integrity; - uint64_t buffer_overrun; - uint64_t vl15_dropped; - time_t time_diff_s; + uint64_t symbol_err_cnt; + uint64_t link_err_recover; + uint64_t link_downed; + uint64_t rcv_err; + uint64_t rcv_rem_phys_err; + uint64_t rcv_switch_relay_err; + uint64_t xmit_discards; + uint64_t xmit_constraint_err; + uint64_t rcv_constraint_err; + uint64_t link_integrity; + uint64_t buffer_overrun; + uint64_t vl15_dropped; + time_t time_diff_s; } osm_epi_pe_event_t; /** ========================================================================= @@ -113,15 +109,15 @@ typedef struct { */ typedef struct { osm_epi_port_id_t port_id; - uint64_t xmit_data; - uint64_t rcv_data; - uint64_t xmit_pkts; - uint64_t rcv_pkts; - uint64_t unicast_xmit_pkts; - uint64_t unicast_rcv_pkts; - uint64_t multicast_xmit_pkts; - uint64_t multicast_rcv_pkts; - time_t time_diff_s; + uint64_t xmit_data; + uint64_t rcv_data; + uint64_t xmit_pkts; + uint64_t rcv_pkts; + uint64_t unicast_xmit_pkts; + uint64_t unicast_rcv_pkts; + uint64_t multicast_xmit_pkts; + uint64_t multicast_rcv_pkts; + time_t time_diff_s; } osm_epi_dc_event_t; /** ========================================================================= @@ -130,8 +126,8 @@ typedef struct { */ typedef struct { osm_epi_port_id_t port_id; - uint64_t xmit_wait; - time_t time_diff_s; + uint64_t xmit_wait; + time_t time_diff_s; } osm_epi_ps_event_t; /** ========================================================================= @@ -139,11 +135,11 @@ typedef struct { */ typedef struct { osm_epi_port_id_t port_id; - uint8_t type; - uint32_t prod_type; - uint16_t trap_num; - uint16_t issuer_lid; - time_t time; + uint8_t type; + uint32_t prod_type; + uint16_t trap_num; + uint16_t issuer_lid; + time_t time; } osm_epi_trap_event_t; /** ========================================================================= @@ -153,15 +149,13 @@ typedef struct { */ #define OSM_EVENT_PLUGIN_IMPL_NAME "osm_event_plugin" #define OSM_EVENT_PLUGIN_INTERFACE_VER (1) -typedef struct -{ - int interface_version; - void *(*construct)(osm_log_t *osm_log); - void (*destroy)(void *plugin_data); +typedef struct { + int interface_version; + void *(*construct) (osm_log_t * osm_log); + void (*destroy) (void *plugin_data); - void (*report)(void *plugin_data, - osm_epi_event_id_t event_id, - void *event_data); + void (*report) (void *plugin_data, + osm_epi_event_id_t event_id, void *event_data); } __osm_epi_plugin_t; @@ -169,36 +163,32 @@ typedef struct * The database structure should be considered opaque */ typedef struct { - void *handle; - __osm_epi_plugin_t *impl; - void *plugin_data; - osm_log_t *p_log; + void *handle; + __osm_epi_plugin_t *impl; + void *plugin_data; + osm_log_t *p_log; } osm_epi_plugin_t; - /** * functions */ -osm_epi_plugin_t *osm_epi_construct(osm_log_t *p_log, char *plugin_name); -void osm_epi_destroy(osm_epi_plugin_t *plugin); -void osm_epi_report(osm_epi_plugin_t *plugin, - osm_epi_event_id_t event_id, - void *event_data); +osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name); +void osm_epi_destroy(osm_epi_plugin_t * plugin); +void osm_epi_report(osm_epi_plugin_t * plugin, + osm_epi_event_id_t event_id, void *event_data); /** ========================================================================= * Helper functions */ static inline void -osm_epi_create_port_id(osm_epi_port_id_t *port_id, uint64_t node_guid, - uint8_t port_num, char *node_name) +osm_epi_create_port_id(osm_epi_port_id_t * port_id, uint64_t node_guid, + uint8_t port_num, char *node_name) { port_id->node_guid = node_guid; port_id->port_num = port_num; strncpy(port_id->node_name, node_name, OSM_EPI_NODE_NAME_LEN); - port_id->node_name[OSM_EPI_NODE_NAME_LEN-1] = '\0'; + port_id->node_name[OSM_EPI_NODE_NAME_LEN - 1] = '\0'; } END_C_DECLS - -#endif /* _OSM_EVENT_PLUGIN_H_ */ - +#endif /* _OSM_EVENT_PLUGIN_H_ */ diff --git a/opensm/include/opensm/osm_fwd_tbl.h b/opensm/include/opensm/osm_fwd_tbl.h index 65eacb3..789d633 100644 --- a/opensm/include/opensm/osm_fwd_tbl.h +++ b/opensm/include/opensm/osm_fwd_tbl.h @@ -56,13 +56,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Forwarding Table * NAME * Forwarding Table @@ -80,7 +79,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Forwarding Table/osm_fwd_tbl_t * NAME * osm_fwd_tbl_t @@ -95,10 +93,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_fwd_tbl_t -{ - osm_rand_fwd_tbl_t *p_rnd_tbl; - osm_lin_fwd_tbl_t *p_lin_tbl; +typedef struct _osm_fwd_tbl_t { + osm_rand_fwd_tbl_t *p_rnd_tbl; + osm_lin_fwd_tbl_t *p_lin_tbl; } osm_fwd_tbl_t; /* @@ -127,9 +124,8 @@ typedef struct _osm_fwd_tbl_t * SYNOPSIS */ ib_api_status_t -osm_fwd_tbl_init( - IN osm_fwd_tbl_t* const p_tbl, - IN const ib_switch_info_t* const p_si ); +osm_fwd_tbl_init(IN osm_fwd_tbl_t * const p_tbl, + IN const ib_switch_info_t * const p_si); /* * PARAMETERS * p_tbl @@ -156,9 +152,7 @@ osm_fwd_tbl_init( * * SYNOPSIS */ -void -osm_fwd_tbl_destroy( - IN osm_fwd_tbl_t* const p_tbl ); +void osm_fwd_tbl_destroy(IN osm_fwd_tbl_t * const p_tbl); /* * PARAMETERS * p_tbl @@ -182,15 +176,14 @@ osm_fwd_tbl_destroy( * SYNOPSIS */ static inline uint8_t -osm_fwd_tbl_get( - IN const osm_fwd_tbl_t* const p_tbl, - IN uint16_t const lid_ho ) +osm_fwd_tbl_get(IN const osm_fwd_tbl_t * const p_tbl, IN uint16_t const lid_ho) { - if( p_tbl->p_lin_tbl ) - return( osm_lin_fwd_tbl_get( p_tbl->p_lin_tbl, lid_ho ) ); + if (p_tbl->p_lin_tbl) + return (osm_lin_fwd_tbl_get(p_tbl->p_lin_tbl, lid_ho)); else - return( osm_rand_fwd_tbl_get( p_tbl->p_rnd_tbl, lid_ho ) ); + return (osm_rand_fwd_tbl_get(p_tbl->p_rnd_tbl, lid_ho)); } + /* * PARAMETERS * p_tbl @@ -218,17 +211,16 @@ osm_fwd_tbl_get( * SYNOPSIS */ static inline void -osm_fwd_tbl_set( - IN osm_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_ho, - IN const uint8_t port ) +osm_fwd_tbl_set(IN osm_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_ho, IN const uint8_t port) { - CL_ASSERT( p_tbl ); - if( p_tbl->p_lin_tbl ) - osm_lin_fwd_tbl_set( p_tbl->p_lin_tbl, lid_ho, port ); + CL_ASSERT(p_tbl); + if (p_tbl->p_lin_tbl) + osm_lin_fwd_tbl_set(p_tbl->p_lin_tbl, lid_ho, port); else - osm_rand_fwd_tbl_set( p_tbl->p_rnd_tbl, lid_ho, port ); + osm_rand_fwd_tbl_set(p_tbl->p_rnd_tbl, lid_ho, port); } + /* * PARAMETERS * p_tbl @@ -258,19 +250,19 @@ osm_fwd_tbl_set( * SYNOPSIS */ static inline ib_api_status_t -osm_fwd_tbl_set_block( - IN osm_fwd_tbl_t* const p_tbl, - IN const uint8_t* const p_block, - IN const uint32_t block_num ) +osm_fwd_tbl_set_block(IN osm_fwd_tbl_t * const p_tbl, + IN const uint8_t * const p_block, + IN const uint32_t block_num) { - CL_ASSERT( p_tbl ); - if( p_tbl->p_lin_tbl ) - return( osm_lin_fwd_tbl_set_block( p_tbl->p_lin_tbl, - p_block, block_num ) ); + CL_ASSERT(p_tbl); + if (p_tbl->p_lin_tbl) + return (osm_lin_fwd_tbl_set_block(p_tbl->p_lin_tbl, + p_block, block_num)); else - return( osm_rand_fwd_tbl_set_block( p_tbl->p_rnd_tbl, - p_block, block_num ) ); + return (osm_rand_fwd_tbl_set_block(p_tbl->p_rnd_tbl, + p_block, block_num)); } + /* * PARAMETERS * p_tbl @@ -294,15 +286,15 @@ osm_fwd_tbl_set_block( * SYNOPSIS */ static inline uint16_t -osm_fwd_tbl_get_size( - IN const osm_fwd_tbl_t* const p_tbl ) +osm_fwd_tbl_get_size(IN const osm_fwd_tbl_t * const p_tbl) { - CL_ASSERT( p_tbl ); - if( p_tbl->p_lin_tbl ) - return( osm_lin_fwd_tbl_get_size( p_tbl->p_lin_tbl ) ); + CL_ASSERT(p_tbl); + if (p_tbl->p_lin_tbl) + return (osm_lin_fwd_tbl_get_size(p_tbl->p_lin_tbl)); else - return( osm_rand_fwd_tbl_get_size( p_tbl->p_rnd_tbl ) ); + return (osm_rand_fwd_tbl_get_size(p_tbl->p_rnd_tbl)); } + /* * PARAMETERS * p_tbl @@ -326,15 +318,15 @@ osm_fwd_tbl_get_size( * SYNOPSIS */ static inline uint16_t -osm_fwd_tbl_get_lids_per_block( - IN const osm_fwd_tbl_t* const p_tbl ) +osm_fwd_tbl_get_lids_per_block(IN const osm_fwd_tbl_t * const p_tbl) { - CL_ASSERT( p_tbl ); - if( p_tbl->p_lin_tbl ) - return( osm_lin_fwd_tbl_get_lids_per_block( p_tbl->p_lin_tbl ) ); + CL_ASSERT(p_tbl); + if (p_tbl->p_lin_tbl) + return (osm_lin_fwd_tbl_get_lids_per_block(p_tbl->p_lin_tbl)); else - return( osm_rand_fwd_tbl_get_lids_per_block( p_tbl->p_rnd_tbl ) ); + return (osm_rand_fwd_tbl_get_lids_per_block(p_tbl->p_rnd_tbl)); } + /* * PARAMETERS * p_tbl @@ -358,18 +350,18 @@ osm_fwd_tbl_get_lids_per_block( * SYNOPSIS */ static inline uint16_t -osm_fwd_tbl_get_max_block_id_in_use( - IN const osm_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_top_ho ) +osm_fwd_tbl_get_max_block_id_in_use(IN const osm_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_top_ho) { - CL_ASSERT( p_tbl ); - if( p_tbl->p_lin_tbl ) - return( osm_lin_fwd_tbl_get_max_block_id_in_use( - p_tbl->p_lin_tbl, lid_top_ho ) ); + CL_ASSERT(p_tbl); + if (p_tbl->p_lin_tbl) + return (osm_lin_fwd_tbl_get_max_block_id_in_use + (p_tbl->p_lin_tbl, lid_top_ho)); else - return( osm_rand_fwd_tbl_get_max_block_id_in_use( - p_tbl->p_rnd_tbl, lid_top_ho ) ); + return (osm_rand_fwd_tbl_get_max_block_id_in_use + (p_tbl->p_rnd_tbl, lid_top_ho)); } + /* * PARAMETERS * p_tbl @@ -384,5 +376,4 @@ osm_fwd_tbl_get_max_block_id_in_use( *********/ END_C_DECLS - -#endif /* _OSM_FWD_TBL_H_ */ +#endif /* _OSM_FWD_TBL_H_ */ diff --git a/opensm/include/opensm/osm_helper.h b/opensm/include/opensm/osm_helper.h index 91f9eeb..5cbb8f0 100644 --- a/opensm/include/opensm/osm_helper.h +++ b/opensm/include/opensm/osm_helper.h @@ -46,13 +46,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * Abstract: * Declaration of helpful functions. @@ -62,7 +61,6 @@ BEGIN_C_DECLS * * $Revision: 1.7 $ */ - /****f* OpenSM: Helper/ib_get_sa_method_str * NAME * ib_get_sa_method_str @@ -72,9 +70,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -const char* -ib_get_sa_method_str( - IN uint8_t method ); +const char *ib_get_sa_method_str(IN uint8_t method); /* * PARAMETERS * method @@ -97,9 +93,7 @@ ib_get_sa_method_str( * * SYNOPSIS */ -const char* -ib_get_sm_method_str( - IN uint8_t method ); +const char *ib_get_sm_method_str(IN uint8_t method); /* * PARAMETERS * method @@ -122,9 +116,7 @@ ib_get_sm_method_str( * * SYNOPSIS */ -const char* -ib_get_sm_attr_str( - IN ib_net16_t attr ); +const char *ib_get_sm_attr_str(IN ib_net16_t attr); /* * PARAMETERS * attr @@ -147,9 +139,7 @@ ib_get_sm_attr_str( * * SYNOPSIS */ -const char* -ib_get_sa_attr_str( - IN ib_net16_t attr ); +const char *ib_get_sa_attr_str(IN ib_net16_t attr); /* * PARAMETERS * attr @@ -172,13 +162,12 @@ ib_get_sa_attr_str( * * SYNOPSIS */ -void osm_dump_port_info( - IN osm_log_t* const p_log, - IN const ib_net64_t node_guid, - IN const ib_net64_t port_guid, - IN const uint8_t port_num, - IN const ib_port_info_t* const p_pi, - IN const osm_log_level_t log_level ); +void osm_dump_port_info(IN osm_log_t * const p_log, + IN const ib_net64_t node_guid, + IN const ib_net64_t port_guid, + IN const uint8_t port_num, + IN const ib_port_info_t * const p_pi, + IN const osm_log_level_t log_level); /* * PARAMETERS * p_log @@ -208,103 +197,88 @@ void osm_dump_port_info( *********/ void -osm_dump_path_record( - IN osm_log_t* const p_log, - IN const ib_path_rec_t* const p_pr, - IN const osm_log_level_t log_level ); +osm_dump_path_record(IN osm_log_t * const p_log, + IN const ib_path_rec_t * const p_pr, + IN const osm_log_level_t log_level); void -osm_dump_multipath_record( - IN osm_log_t* const p_log, - IN const ib_multipath_rec_t* const p_mpr, - IN const osm_log_level_t log_level ); +osm_dump_multipath_record(IN osm_log_t * const p_log, + IN const ib_multipath_rec_t * const p_mpr, + IN const osm_log_level_t log_level); void -osm_dump_node_record( - IN osm_log_t* const p_log, - IN const ib_node_record_t* const p_nr, - IN const osm_log_level_t log_level ); +osm_dump_node_record(IN osm_log_t * const p_log, + IN const ib_node_record_t * const p_nr, + IN const osm_log_level_t log_level); void -osm_dump_mc_record( - IN osm_log_t* const p_log, - IN const ib_member_rec_t* const p_mcmr, - IN const osm_log_level_t log_level ); +osm_dump_mc_record(IN osm_log_t * const p_log, + IN const ib_member_rec_t * const p_mcmr, + IN const osm_log_level_t log_level); void -osm_dump_link_record( - IN osm_log_t* const p_log, - IN const ib_link_record_t* const p_lr, - IN const osm_log_level_t log_level ); +osm_dump_link_record(IN osm_log_t * const p_log, + IN const ib_link_record_t * const p_lr, + IN const osm_log_level_t log_level); void -osm_dump_service_record( - IN osm_log_t* const p_log, - IN const ib_service_record_t* const p_sr, - IN const osm_log_level_t log_level ); +osm_dump_service_record(IN osm_log_t * const p_log, + IN const ib_service_record_t * const p_sr, + IN const osm_log_level_t log_level); void -osm_dump_portinfo_record( - IN osm_log_t* const p_log, - IN const ib_portinfo_record_t* const p_pir, - IN const osm_log_level_t log_level ); +osm_dump_portinfo_record(IN osm_log_t * const p_log, + IN const ib_portinfo_record_t * const p_pir, + IN const osm_log_level_t log_level); void -osm_dump_guidinfo_record( - IN osm_log_t* const p_log, - IN const ib_guidinfo_record_t* const p_gir, - IN const osm_log_level_t log_level ); +osm_dump_guidinfo_record(IN osm_log_t * const p_log, + IN const ib_guidinfo_record_t * const p_gir, + IN const osm_log_level_t log_level); void -osm_dump_inform_info( - IN osm_log_t* const p_log, - IN const ib_inform_info_t* const p_ii, - IN const osm_log_level_t log_level ); +osm_dump_inform_info(IN osm_log_t * const p_log, + IN const ib_inform_info_t * const p_ii, + IN const osm_log_level_t log_level); void -osm_dump_inform_info_record( - IN osm_log_t* const p_log, - IN const ib_inform_info_record_t* const p_iir, - IN const osm_log_level_t log_level ); +osm_dump_inform_info_record(IN osm_log_t * const p_log, + IN const ib_inform_info_record_t * const p_iir, + IN const osm_log_level_t log_level); void -osm_dump_switch_info_record( - IN osm_log_t* const p_log, - IN const ib_switch_info_record_t* const p_sir, - IN const osm_log_level_t log_level ); +osm_dump_switch_info_record(IN osm_log_t * const p_log, + IN const ib_switch_info_record_t * const p_sir, + IN const osm_log_level_t log_level); void -osm_dump_sm_info_record( - IN osm_log_t* const p_log, - IN const ib_sminfo_record_t* const p_smir, - IN const osm_log_level_t log_level ); +osm_dump_sm_info_record(IN osm_log_t * const p_log, + IN const ib_sminfo_record_t * const p_smir, + IN const osm_log_level_t log_level); void -osm_dump_pkey_block( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint16_t block_num, - IN uint8_t port_num, - IN const ib_pkey_table_t* const p_pkey_tbl, - IN const osm_log_level_t log_level ); +osm_dump_pkey_block(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint16_t block_num, + IN uint8_t port_num, + IN const ib_pkey_table_t * const p_pkey_tbl, + IN const osm_log_level_t log_level); void -osm_dump_slvl_map_table( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint8_t in_port_num, - IN uint8_t out_port_num, - IN const ib_slvl_table_t* const p_slvl_tbl, - IN const osm_log_level_t log_level ); +osm_dump_slvl_map_table(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint8_t in_port_num, + IN uint8_t out_port_num, + IN const ib_slvl_table_t * const p_slvl_tbl, + IN const osm_log_level_t log_level); void -osm_dump_vl_arb_table( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint8_t block_num, - IN uint8_t port_num, - IN const ib_vl_arb_table_t* const p_vla_tbl, - IN const osm_log_level_t log_level ); +osm_dump_vl_arb_table(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint8_t block_num, + IN uint8_t port_num, + IN const ib_vl_arb_table_t * const p_vla_tbl, + IN const osm_log_level_t log_level); /****f* OpenSM: Helper/osm_dump_port_info * NAME @@ -315,10 +289,9 @@ osm_dump_vl_arb_table( * * SYNOPSIS */ -void osm_dump_node_info( - IN osm_log_t* const p_log, - IN const ib_node_info_t* const p_ni, - IN const osm_log_level_t log_level ); +void osm_dump_node_info(IN osm_log_t * const p_log, + IN const ib_node_info_t * const p_ni, + IN const osm_log_level_t log_level); /* * PARAMETERS * p_log @@ -348,10 +321,9 @@ void osm_dump_node_info( * SYNOPSIS */ void -osm_dump_sm_info( - IN osm_log_t* const p_log, - IN const ib_sm_info_t* const p_smi, - IN const osm_log_level_t log_level ); +osm_dump_sm_info(IN osm_log_t * const p_log, + IN const ib_sm_info_t * const p_smi, + IN const osm_log_level_t log_level); /* * PARAMETERS * p_log @@ -381,10 +353,9 @@ osm_dump_sm_info( * SYNOPSIS */ void -osm_dump_switch_info( - IN osm_log_t* const p_log, - IN const ib_switch_info_t* const p_si, - IN const osm_log_level_t log_level ); +osm_dump_switch_info(IN osm_log_t * const p_log, + IN const ib_switch_info_t * const p_si, + IN const osm_log_level_t log_level); /* * PARAMETERS * p_log @@ -414,10 +385,9 @@ osm_dump_switch_info( * SYNOPSIS */ void -osm_dump_notice( - IN osm_log_t* const p_log, - IN const ib_mad_notice_attr_t *p_ntci, - IN const osm_log_level_t log_level ); +osm_dump_notice(IN osm_log_t * const p_log, + IN const ib_mad_notice_attr_t * p_ntci, + IN const osm_log_level_t log_level); /* * PARAMETERS * p_log @@ -446,9 +416,7 @@ osm_dump_notice( * * SYNOPSIS */ -const char* -osm_get_disp_msg_str( - IN cl_disp_msgid_t msg ); +const char *osm_get_disp_msg_str(IN cl_disp_msgid_t msg); /* * PARAMETERS * msg @@ -462,25 +430,21 @@ osm_get_disp_msg_str( * SEE ALSO *********/ -void osm_dump_dr_path( - IN osm_log_t* const p_log, - IN const osm_dr_path_t* const p_path, - IN const osm_log_level_t level ); +void osm_dump_dr_path(IN osm_log_t * const p_log, + IN const osm_dr_path_t * const p_path, + IN const osm_log_level_t level); -void osm_dump_smp_dr_path( - IN osm_log_t* const p_log, - IN const ib_smp_t* const p_smp, - IN const osm_log_level_t level ); +void osm_dump_smp_dr_path(IN osm_log_t * const p_log, + IN const ib_smp_t * const p_smp, + IN const osm_log_level_t level); -void osm_dump_dr_smp( - IN osm_log_t* const p_log, - IN const ib_smp_t* const p_smp, - IN const osm_log_level_t level ); +void osm_dump_dr_smp(IN osm_log_t * const p_log, + IN const ib_smp_t * const p_smp, + IN const osm_log_level_t level); -void osm_dump_sa_mad( - IN osm_log_t* const p_log, - IN const ib_sa_mad_t* const p_smp, - IN const osm_log_level_t level ); +void osm_dump_sa_mad(IN osm_log_t * const p_log, + IN const ib_sa_mad_t * const p_smp, + IN const osm_log_level_t level); /****f* IBA Base: Types/osm_get_sm_state_str * NAME @@ -491,9 +455,7 @@ void osm_dump_sa_mad( * * SYNOPSIS */ -const char* -osm_get_sm_state_str( - IN osm_sm_state_t state ); +const char *osm_get_sm_state_str(IN osm_sm_state_t state); /* * PARAMETERS * state @@ -516,9 +478,7 @@ osm_get_sm_state_str( * * SYNOPSIS */ -const char* -osm_get_sm_signal_str( - IN osm_signal_t signal ); +const char *osm_get_sm_signal_str(IN osm_signal_t signal); /* * PARAMETERS * state @@ -532,38 +492,21 @@ osm_get_sm_signal_str( * SEE ALSO *********/ -const char* -osm_get_port_state_str_fixed_width( - IN uint8_t port_state ); +const char *osm_get_port_state_str_fixed_width(IN uint8_t port_state); -const char* -osm_get_node_type_str_fixed_width( - IN uint8_t node_type ); +const char *osm_get_node_type_str_fixed_width(IN uint8_t node_type); -const char* -osm_get_manufacturer_str( - IN uint64_t const guid_ho ); +const char *osm_get_manufacturer_str(IN uint64_t const guid_ho); -const char* -osm_get_mtu_str( - IN uint8_t const mtu ); +const char *osm_get_mtu_str(IN uint8_t const mtu); -const char* -osm_get_lwa_str( - IN uint8_t const lwa ); +const char *osm_get_lwa_str(IN uint8_t const lwa); -const char* -osm_get_mtu_str( - IN uint8_t const mtu ); +const char *osm_get_mtu_str(IN uint8_t const mtu); -const char* -osm_get_lwa_str( - IN uint8_t const lwa ); - -const char* -osm_get_lsa_str( - IN uint8_t const lsa ); +const char *osm_get_lwa_str(IN uint8_t const lwa); +const char *osm_get_lsa_str(IN uint8_t const lsa); /****f* IBA Base: Types/osm_get_sm_mgr_signal_str * NAME @@ -574,9 +517,7 @@ osm_get_lsa_str( * * SYNOPSIS */ -const char* -osm_get_sm_mgr_signal_str( - IN osm_sm_signal_t signal ); +const char *osm_get_sm_mgr_signal_str(IN osm_sm_signal_t signal); /* * PARAMETERS * signal @@ -599,9 +540,7 @@ osm_get_sm_mgr_signal_str( * * SYNOPSIS */ -const char* -osm_get_sm_mgr_state_str( - IN uint16_t state ); +const char *osm_get_sm_mgr_state_str(IN uint16_t state); /* * PARAMETERS * state @@ -616,5 +555,4 @@ osm_get_sm_mgr_state_str( *********/ END_C_DECLS - -#endif /* _OSM_HELPER_H_ */ +#endif /* _OSM_HELPER_H_ */ diff --git a/opensm/include/opensm/osm_inform.h b/opensm/include/opensm/osm_inform.h index 5c11f44..91c0c64 100644 --- a/opensm/include/opensm/osm_inform.h +++ b/opensm/include/opensm/osm_inform.h @@ -63,13 +63,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Inform Record * NAME * Inform Record @@ -89,7 +88,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: Inform Record/osm_infr_t * NAME * osm_infr_t @@ -102,13 +100,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_infr_t -{ - cl_list_item_t list_item; - osm_bind_handle_t h_bind; - osm_infr_rcv_t* p_infr_rcv; - osm_mad_addr_t report_addr; - ib_inform_info_record_t inform_record; +typedef struct _osm_infr_t { + cl_list_item_t list_item; + osm_bind_handle_t h_bind; + osm_infr_rcv_t *p_infr_rcv; + osm_mad_addr_t report_addr; + ib_inform_info_record_t inform_record; } osm_infr_t; /* * FIELDS @@ -139,9 +136,7 @@ typedef struct _osm_infr_t * * SYNOPSIS */ -osm_infr_t* -osm_infr_new( - IN const osm_infr_t *p_infr_rec ); +osm_infr_t *osm_infr_new(IN const osm_infr_t * p_infr_rec); /* * PARAMETERS * p_inf_rec @@ -166,9 +161,7 @@ osm_infr_new( * * SYNOPSIS */ -void -osm_infr_delete( - IN osm_infr_t* const p_infr ); +void osm_infr_delete(IN osm_infr_t * const p_infr); /* * PARAMETERS * p_infr @@ -187,11 +180,9 @@ osm_infr_delete( * * SYNOPSIS */ -osm_infr_t* -osm_infr_get_by_rec( - IN osm_subn_t const *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t* const p_infr_rec ); +osm_infr_t *osm_infr_get_by_rec(IN osm_subn_t const *p_subn, + IN osm_log_t * p_log, + IN osm_infr_t * const p_infr_rec); /* * PARAMETERS * p_subn @@ -210,16 +201,12 @@ osm_infr_get_by_rec( *********/ void -osm_infr_insert_to_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t *p_infr); +osm_infr_insert_to_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_infr_t * p_infr); void -osm_infr_remove_from_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t *p_infr); +osm_infr_remove_from_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_infr_t * p_infr); /****f* OpenSM: Inform Record/osm_report_notice * NAME @@ -236,10 +223,8 @@ osm_infr_remove_from_db( * SYNOPSIS */ ib_api_status_t -osm_report_notice( - IN osm_log_t* const p_log, - IN osm_subn_t* p_subn, - IN ib_mad_notice_attr_t* p_ntc ); +osm_report_notice(IN osm_log_t * const p_log, + IN osm_subn_t * p_subn, IN ib_mad_notice_attr_t * p_ntc); /* * PARAMETERS * p_rcv @@ -256,5 +241,4 @@ osm_report_notice( *********/ END_C_DECLS - -#endif /* _OSM_INFR_H_ */ +#endif /* _OSM_INFR_H_ */ diff --git a/opensm/include/opensm/osm_lid_mgr.h b/opensm/include/opensm/osm_lid_mgr.h index 78cbcb0..9a7d0e3 100644 --- a/opensm/include/opensm/osm_lid_mgr.h +++ b/opensm/include/opensm/osm_lid_mgr.h @@ -59,15 +59,13 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSM_LID_MGR_LIST_SIZE_MIN 256 - /****h* OpenSM/LID Manager * NAME * LID Manager @@ -85,7 +83,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: LID Manager/osm_lid_mgr_t * NAME * osm_lid_mgr_t @@ -98,17 +95,16 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_lid_mgr -{ - osm_subn_t *p_subn; - osm_db_t *p_db; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; - boolean_t send_set_reqs; - osm_db_domain_t *p_g2l; - cl_ptr_vector_t used_lids; - cl_qlist_t free_ranges; +typedef struct _osm_lid_mgr { + osm_subn_t *p_subn; + osm_db_t *p_db; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; + boolean_t send_set_reqs; + osm_db_domain_t *p_g2l; + cl_ptr_vector_t used_lids; + cl_qlist_t free_ranges; } osm_lid_mgr_t; /* * FIELDS @@ -156,9 +152,7 @@ typedef struct _osm_lid_mgr * * SYNOPSIS */ -void -osm_lid_mgr_construct( - IN osm_lid_mgr_t* const p_mgr ); +void osm_lid_mgr_construct(IN osm_lid_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -188,9 +182,7 @@ osm_lid_mgr_construct( * * SYNOPSIS */ -void -osm_lid_mgr_destroy( - IN osm_lid_mgr_t* const p_mgr ); +void osm_lid_mgr_destroy(IN osm_lid_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -222,13 +214,11 @@ osm_lid_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_lid_mgr_init( - IN osm_lid_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_db_t* const p_db, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_lid_mgr_init(IN osm_lid_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_db_t * const p_db, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_mgr @@ -270,9 +260,7 @@ osm_lid_mgr_init( * * SYNOPSIS */ -osm_signal_t -osm_lid_mgr_process_sm( - IN osm_lid_mgr_t* const p_mgr ); +osm_signal_t osm_lid_mgr_process_sm(IN osm_lid_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -300,9 +288,7 @@ osm_lid_mgr_process_sm( * * SYNOPSIS */ -osm_signal_t -osm_lid_mgr_process_subnet( - IN osm_lid_mgr_t* const p_mgr ); +osm_signal_t osm_lid_mgr_process_subnet(IN osm_lid_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -321,5 +307,4 @@ osm_lid_mgr_process_subnet( *********/ END_C_DECLS - -#endif /* _OSM_LID_MGR_H_ */ +#endif /* _OSM_LID_MGR_H_ */ diff --git a/opensm/include/opensm/osm_lin_fwd_rcv.h b/opensm/include/opensm/osm_lin_fwd_rcv.h index 4fe5b90..28402cc 100644 --- a/opensm/include/opensm/osm_lin_fwd_rcv.h +++ b/opensm/include/opensm/osm_lin_fwd_rcv.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/LFT Receiver * NAME * LFT Receiver @@ -81,7 +80,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: LFT Receiver/osm_lft_rcv_t * NAME * osm_lft_rcv_t @@ -94,11 +92,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_lft_rcv -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_lft_rcv { + osm_subn_t *p_subn; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_lft_rcv_t; /* * FIELDS @@ -124,8 +121,7 @@ typedef struct _osm_lft_rcv * * SYNOPSIS */ -void osm_lft_rcv_construct( - IN osm_lft_rcv_t* const p_rcv ); +void osm_lft_rcv_construct(IN osm_lft_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -155,8 +151,7 @@ void osm_lft_rcv_construct( * * SYNOPSIS */ -void osm_lft_rcv_destroy( - IN osm_lft_rcv_t* const p_rcv ); +void osm_lft_rcv_destroy(IN osm_lft_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -187,11 +182,10 @@ void osm_lft_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_lft_rcv_init( - IN osm_lft_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_lft_rcv_init(IN osm_lft_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -227,9 +221,7 @@ ib_api_status_t osm_lft_rcv_init( * * SYNOPSIS */ -void osm_lft_rcv_process( - IN void *context, - IN void *data ); +void osm_lft_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -250,5 +242,4 @@ void osm_lft_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_LFT_RCV_H_ */ +#endif /* _OSM_LFT_RCV_H_ */ diff --git a/opensm/include/opensm/osm_lin_fwd_tbl.h b/opensm/include/opensm/osm_lin_fwd_tbl.h index bcaac60..c43ee15 100644 --- a/opensm/include/opensm/osm_lin_fwd_tbl.h +++ b/opensm/include/opensm/osm_lin_fwd_tbl.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Linear Forwarding Table * NAME * Linear Forwarding Table @@ -79,7 +78,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Forwarding Table/osm_lin_fwd_tbl_t * NAME * osm_lin_fwd_tbl_t @@ -91,10 +89,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_lin_fwd_tbl -{ - uint16_t size; - uint8_t port_tbl[1]; +typedef struct _osm_lin_fwd_tbl { + uint16_t size; + uint8_t port_tbl[1]; } osm_lin_fwd_tbl_t; /* * FIELDS @@ -119,9 +116,7 @@ typedef struct _osm_lin_fwd_tbl * * SYNOPSIS */ -osm_lin_fwd_tbl_t* -osm_lin_tbl_new( - IN uint16_t const size ); +osm_lin_fwd_tbl_t *osm_lin_tbl_new(IN uint16_t const size); /* * PARAMETERS * size @@ -146,9 +141,7 @@ osm_lin_tbl_new( * * SYNOPSIS */ -void -osm_lin_tbl_delete( - IN osm_lin_fwd_tbl_t** const pp_tbl ); +void osm_lin_tbl_delete(IN osm_lin_fwd_tbl_t ** const pp_tbl); /* * PARAMETERS * pp_tbl @@ -174,15 +167,14 @@ osm_lin_tbl_delete( * SYNOPSIS */ static inline void -osm_lin_fwd_tbl_set( - IN osm_lin_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_ho, - IN const uint8_t port ) +osm_lin_fwd_tbl_set(IN osm_lin_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_ho, IN const uint8_t port) { - CL_ASSERT( lid_ho < p_tbl->size ); - if( lid_ho < p_tbl->size ) + CL_ASSERT(lid_ho < p_tbl->size); + if (lid_ho < p_tbl->size) p_tbl->port_tbl[lid_ho] = port; } + /* * PARAMETERS * p_tbl @@ -212,15 +204,15 @@ osm_lin_fwd_tbl_set( * SYNOPSIS */ static inline uint8_t -osm_lin_fwd_tbl_get( - IN const osm_lin_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_ho ) +osm_lin_fwd_tbl_get(IN const osm_lin_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_ho) { - if( lid_ho < p_tbl->size ) - return( p_tbl->port_tbl[lid_ho] ); + if (lid_ho < p_tbl->size) + return (p_tbl->port_tbl[lid_ho]); else - return( 0xFF ); + return (0xFF); } + /* * PARAMETERS * p_tbl @@ -247,11 +239,11 @@ osm_lin_fwd_tbl_get( * SYNOPSIS */ static inline uint16_t -osm_lin_fwd_tbl_get_size( - IN const osm_lin_fwd_tbl_t* const p_tbl ) +osm_lin_fwd_tbl_get_size(IN const osm_lin_fwd_tbl_t * const p_tbl) { - return( p_tbl->size ); + return (p_tbl->size); } + /* * PARAMETERS * p_tbl @@ -275,12 +267,12 @@ osm_lin_fwd_tbl_get_size( * SYNOPSIS */ static inline uint16_t -osm_lin_fwd_tbl_get_lids_per_block( - IN const osm_lin_fwd_tbl_t* const p_tbl ) +osm_lin_fwd_tbl_get_lids_per_block(IN const osm_lin_fwd_tbl_t * const p_tbl) { - UNUSED_PARAM( p_tbl ); - return( 64 ); + UNUSED_PARAM(p_tbl); + return (64); } + /* * PARAMETERS * p_tbl @@ -304,13 +296,14 @@ osm_lin_fwd_tbl_get_lids_per_block( * SYNOPSIS */ static inline uint16_t -osm_lin_fwd_tbl_get_max_block_id_in_use( - IN const osm_lin_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_top_ho ) +osm_lin_fwd_tbl_get_max_block_id_in_use(IN const osm_lin_fwd_tbl_t * + const p_tbl, + IN const uint16_t lid_top_ho) { - return( (uint16_t)(lid_top_ho / - osm_lin_fwd_tbl_get_lids_per_block( p_tbl ) ) ); + return ((uint16_t) (lid_top_ho / + osm_lin_fwd_tbl_get_lids_per_block(p_tbl))); } + /* * PARAMETERS * p_tbl @@ -334,26 +327,26 @@ osm_lin_fwd_tbl_get_max_block_id_in_use( * SYNOPSIS */ static inline ib_api_status_t -osm_lin_fwd_tbl_set_block( - IN osm_lin_fwd_tbl_t* const p_tbl, - IN const uint8_t* const p_block, - IN const uint32_t block_num ) +osm_lin_fwd_tbl_set_block(IN osm_lin_fwd_tbl_t * const p_tbl, + IN const uint8_t * const p_block, + IN const uint32_t block_num) { uint16_t lid_start; uint16_t num_lids; - CL_ASSERT( p_tbl ); - CL_ASSERT( p_block ); + CL_ASSERT(p_tbl); + CL_ASSERT(p_block); - num_lids = osm_lin_fwd_tbl_get_lids_per_block( p_tbl ); - lid_start = (uint16_t)(block_num * num_lids); + num_lids = osm_lin_fwd_tbl_get_lids_per_block(p_tbl); + lid_start = (uint16_t) (block_num * num_lids); - if( lid_start + num_lids > p_tbl->size ) - return( IB_INVALID_PARAMETER ); + if (lid_start + num_lids > p_tbl->size) + return (IB_INVALID_PARAMETER); - memcpy( &p_tbl->port_tbl[lid_start], p_block, num_lids ); - return( IB_SUCCESS ); + memcpy(&p_tbl->port_tbl[lid_start], p_block, num_lids); + return (IB_SUCCESS); } + /* * PARAMETERS * p_tbl @@ -374,5 +367,4 @@ osm_lin_fwd_tbl_set_block( *********/ END_C_DECLS - -#endif /* _OSM_LIN_FWD_TBL_H_ */ +#endif /* _OSM_LIN_FWD_TBL_H_ */ diff --git a/opensm/include/opensm/osm_link_mgr.h b/opensm/include/opensm/osm_link_mgr.h index 465db16..214dd80 100644 --- a/opensm/include/opensm/osm_link_mgr.h +++ b/opensm/include/opensm/osm_link_mgr.h @@ -58,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Link Manager * NAME * Link Manager @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Link Manager/osm_link_mgr_t * NAME * osm_link_mgr_t @@ -95,13 +93,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_link_mgr -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; - boolean_t send_set_reqs; +typedef struct _osm_link_mgr { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; + boolean_t send_set_reqs; } osm_link_mgr_t; /* @@ -134,9 +131,7 @@ typedef struct _osm_link_mgr * * SYNOPSIS */ -void -osm_link_mgr_construct( - IN osm_link_mgr_t* const p_mgr ); +void osm_link_mgr_construct(IN osm_link_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -166,9 +161,7 @@ osm_link_mgr_construct( * * SYNOPSIS */ -void -osm_link_mgr_destroy( - IN osm_link_mgr_t* const p_mgr ); +void osm_link_mgr_destroy(IN osm_link_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -200,12 +193,10 @@ osm_link_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_link_mgr_init( - IN osm_link_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_link_mgr_init(IN osm_link_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_mgr @@ -245,9 +236,8 @@ osm_link_mgr_init( * SYNOPSIS */ osm_signal_t -osm_link_mgr_process( - IN osm_link_mgr_t* const p_mgr, - IN const uint8_t link_state ); +osm_link_mgr_process(IN osm_link_mgr_t * const p_mgr, + IN const uint8_t link_state); /* * PARAMETERS * p_mgr @@ -270,5 +260,4 @@ osm_link_mgr_process( *********/ END_C_DECLS - -#endif /* _OSM_LINK_MGR_H_ */ +#endif /* _OSM_LINK_MGR_H_ */ diff --git a/opensm/include/opensm/osm_log.h b/opensm/include/opensm/osm_log.h index a556ad2..6c487aa 100644 --- a/opensm/include/opensm/osm_log.h +++ b/opensm/include/opensm/osm_log.h @@ -65,26 +65,21 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define LOG_ENTRY_SIZE_MAX 4096 #define BUF_SIZE LOG_ENTRY_SIZE_MAX - #define __func__ __FUNCTION__ - #define OSM_LOG_ENTER( OSM_LOG_PTR, NAME ) \ osm_log( OSM_LOG_PTR, OSM_LOG_FUNCS, \ "%s: [\n", __func__); - #define OSM_LOG_EXIT( OSM_LOG_PTR ) \ osm_log( OSM_LOG_PTR, OSM_LOG_FUNCS, \ "%s: ]\n", __func__); - /****h* OpenSM/Log * NAME * Log @@ -119,17 +114,16 @@ typedef uint8_t osm_log_level_t; * * SYNOPSIS */ -typedef struct _osm_log -{ - osm_log_level_t level; - cl_spinlock_t lock; - unsigned long count; - unsigned long max_size; - boolean_t flush; - FILE* out_port; - boolean_t accum_log_file; - boolean_t daemon; - char* log_file_name; +typedef struct _osm_log { + osm_log_level_t level; + cl_spinlock_t lock; + unsigned long count; + unsigned long max_size; + boolean_t flush; + FILE *out_port; + boolean_t accum_log_file; + boolean_t daemon; + char *log_file_name; } osm_log_t; /*********/ @@ -142,12 +136,11 @@ typedef struct _osm_log * * SYNOPSIS */ -static inline void -osm_log_construct( - IN osm_log_t* const p_log ) +static inline void osm_log_construct(IN osm_log_t * const p_log) { - cl_spinlock_construct( &p_log->lock ); + cl_spinlock_construct(&p_log->lock); } + /* * PARAMETERS * p_log @@ -177,17 +170,16 @@ osm_log_construct( * * SYNOPSIS */ -static inline void -osm_log_destroy( - IN osm_log_t* const p_log ) +static inline void osm_log_destroy(IN osm_log_t * const p_log) { - cl_spinlock_destroy( &p_log->lock ); + cl_spinlock_destroy(&p_log->lock); if (p_log->out_port != stdout) { - fclose(p_log->out_port); - p_log->out_port = stdout; + fclose(p_log->out_port); + p_log->out_port = stdout; } closelog(); } + /* * PARAMETERS * p_log @@ -219,13 +211,12 @@ osm_log_destroy( * SYNOPSIS */ ib_api_status_t -osm_log_init_v2( - IN osm_log_t* const p_log, - IN const boolean_t flush, - IN const uint8_t log_flags, - IN const char *log_file, - IN const unsigned long max_size, - IN const boolean_t accum_log_file ); +osm_log_init_v2(IN osm_log_t * const p_log, + IN const boolean_t flush, + IN const uint8_t log_flags, + IN const char *log_file, + IN const unsigned long max_size, + IN const boolean_t accum_log_file); /* * PARAMETERS * p_log @@ -263,9 +254,7 @@ osm_log_init_v2( * * SYNOPSIS */ -int -osm_log_reopen_file( - osm_log_t *p_log); +int osm_log_reopen_file(osm_log_t * p_log); /* * PARAMETERS * p_log @@ -286,12 +275,10 @@ osm_log_reopen_file( * SYNOPSIS */ ib_api_status_t -osm_log_init( - IN osm_log_t* const p_log, - IN const boolean_t flush, - IN const uint8_t log_flags, - IN const char *log_file, - IN const boolean_t accum_log_file ); +osm_log_init(IN osm_log_t * const p_log, + IN const boolean_t flush, + IN const uint8_t log_flags, + IN const char *log_file, IN const boolean_t accum_log_file); /* * Same as osm_log_init_v2() but without max_size parameter */ @@ -306,11 +293,11 @@ osm_log_init( * SYNOPSIS */ static inline osm_log_level_t -osm_log_get_level( - IN const osm_log_t* const p_log ) +osm_log_get_level(IN const osm_log_t * const p_log) { - return( p_log->level ); + return (p_log->level); } + /* * PARAMETERS * p_log @@ -336,12 +323,11 @@ osm_log_get_level( * SYNOPSIS */ static inline void -osm_log_set_level( - IN osm_log_t* const p_log, - IN const osm_log_level_t level ) +osm_log_set_level(IN osm_log_t * const p_log, IN const osm_log_level_t level) { p_log->level = level; } + /* * PARAMETERS * p_log @@ -371,12 +357,12 @@ osm_log_set_level( * SYNOPSIS */ static inline boolean_t -osm_log_is_active( - IN const osm_log_t* const p_log, - IN const osm_log_level_t level ) +osm_log_is_active(IN const osm_log_t * const p_log, + IN const osm_log_level_t level) { - return( (p_log->level & level) != 0 ); + return ((p_log->level & level) != 0); } + /* * PARAMETERS * p_log @@ -396,20 +382,17 @@ osm_log_is_active( * osm_log_destroy *********/ -extern int osm_log_printf(osm_log_t *p_log, osm_log_level_t level, +extern int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, const char *fmt, ...); void -osm_log( - IN osm_log_t* const p_log, +osm_log(IN osm_log_t * const p_log, IN const osm_log_level_t verbosity, - IN const char *p_str, ... ) STRICT_OSM_LOG_FORMAT; + IN const char *p_str, ...) STRICT_OSM_LOG_FORMAT; void -osm_log_raw( - IN osm_log_t* const p_log, - IN const osm_log_level_t verbosity, - IN const char *p_buf ); +osm_log_raw(IN osm_log_t * const p_log, + IN const osm_log_level_t verbosity, IN const char *p_buf); #define DBG_CL_LOCK 0 @@ -479,8 +462,7 @@ osm_log_raw( * * SYNOPSIS */ -boolean_t -osm_is_debug(void); +boolean_t osm_is_debug(void); /* * PARAMETERS * None @@ -493,5 +475,4 @@ osm_is_debug(void); *********/ END_C_DECLS - -#endif /* _OSM_LOG_H_ */ +#endif /* _OSM_LOG_H_ */ diff --git a/opensm/include/opensm/osm_mad_pool.h b/opensm/include/opensm/osm_mad_pool.h index 7a3593d..9ec0a7a 100644 --- a/opensm/include/opensm/osm_mad_pool.h +++ b/opensm/include/opensm/osm_mad_pool.h @@ -59,13 +59,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/MAD Pool * NAME * MAD Pool @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: MAD Pool/osm_mad_pool_t * NAME * osm_mad_pool_t @@ -97,11 +95,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mad_pool -{ - osm_log_t *p_log; - cl_qlock_pool_t madw_pool; - atomic32_t mads_out; +typedef struct _osm_mad_pool { + osm_log_t *p_log; + cl_qlock_pool_t madw_pool; + atomic32_t mads_out; } osm_mad_pool_t; /* * FIELDS @@ -127,8 +124,7 @@ typedef struct _osm_mad_pool * * SYNOPSIS */ -void osm_mad_pool_construct( - IN osm_mad_pool_t* const p_pool ); +void osm_mad_pool_construct(IN osm_mad_pool_t * const p_pool); /* * PARAMETERS * p_pool @@ -157,8 +153,7 @@ void osm_mad_pool_construct( * * SYNOPSIS */ -void osm_mad_pool_destroy( - IN osm_mad_pool_t* const p_pool ); +void osm_mad_pool_destroy(IN osm_mad_pool_t * const p_pool); /* * PARAMETERS * p_pool @@ -186,9 +181,8 @@ void osm_mad_pool_destroy( * * SYNOPSIS */ -ib_api_status_t osm_mad_pool_init( - IN osm_mad_pool_t* const p_pool, - IN osm_log_t* const p_log ); +ib_api_status_t osm_mad_pool_init(IN osm_mad_pool_t * const p_pool, + IN osm_log_t * const p_log); /* * PARAMETERS * p_pool @@ -217,12 +211,10 @@ ib_api_status_t osm_mad_pool_init( * * SYNOPSIS */ -osm_madw_t* -osm_mad_pool_get( - IN osm_mad_pool_t* const p_pool, - IN osm_bind_handle_t h_bind, - IN const uint32_t total_size, - IN const osm_mad_addr_t* const p_mad_addr ); +osm_madw_t *osm_mad_pool_get(IN osm_mad_pool_t * const p_pool, + IN osm_bind_handle_t h_bind, + IN const uint32_t total_size, + IN const osm_mad_addr_t * const p_mad_addr); /* * PARAMETERS * p_pool @@ -263,9 +255,8 @@ osm_mad_pool_get( * * SYNOPSIS */ -void osm_mad_pool_put( - IN osm_mad_pool_t* const p_pool, - IN osm_madw_t* const p_madw ); +void osm_mad_pool_put(IN osm_mad_pool_t * const p_pool, + IN osm_madw_t * const p_madw); /* * PARAMETERS * p_pool @@ -295,13 +286,12 @@ void osm_mad_pool_put( * * SYNOPSIS */ -osm_madw_t* -osm_mad_pool_get_wrapper( - IN osm_mad_pool_t* const p_pool, - IN osm_bind_handle_t h_bind, - IN const uint32_t total_size, - IN const ib_mad_t* const p_mad, - IN const osm_mad_addr_t* const p_mad_addr ); +osm_madw_t *osm_mad_pool_get_wrapper(IN osm_mad_pool_t * const p_pool, + IN osm_bind_handle_t h_bind, + IN const uint32_t total_size, + IN const ib_mad_t * const p_mad, + IN const osm_mad_addr_t * + const p_mad_addr); /* * PARAMETERS * p_pool @@ -346,9 +336,7 @@ osm_mad_pool_get_wrapper( * * SYNOPSIS */ -osm_madw_t* -osm_mad_pool_get_wrapper_raw( - IN osm_mad_pool_t* const p_pool ); +osm_madw_t *osm_mad_pool_get_wrapper_raw(IN osm_mad_pool_t * const p_pool); /* * PARAMETERS * p_pool @@ -379,11 +367,11 @@ osm_mad_pool_get_wrapper_raw( * SYNOPSIS */ static inline uint32_t -osm_mad_pool_get_outstanding( - IN const osm_mad_pool_t* const p_pool ) +osm_mad_pool_get_outstanding(IN const osm_mad_pool_t * const p_pool) { - return( p_pool->mads_out ); + return (p_pool->mads_out); } + /* * PARAMETERS * p_pool @@ -401,5 +389,4 @@ osm_mad_pool_get_outstanding( *********/ END_C_DECLS - -#endif /* _OSM_MAD_POOL_H_ */ +#endif /* _OSM_MAD_POOL_H_ */ diff --git a/opensm/include/opensm/osm_madw.h b/opensm/include/opensm/osm_madw.h index 6d1ab04..b996037 100644 --- a/opensm/include/opensm/osm_madw.h +++ b/opensm/include/opensm/osm_madw.h @@ -58,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: MAD Wrapper/osm_bind_info_t * NAME * osm_bind_info_t @@ -73,16 +72,15 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_bind_info -{ - ib_net64_t port_guid; - uint8_t mad_class; - uint8_t class_version; - boolean_t is_responder; - boolean_t is_trap_processor; - boolean_t is_report_processor; - uint32_t send_q_size; - uint32_t recv_q_size; +typedef struct _osm_bind_info { + ib_net64_t port_guid; + uint8_t mad_class; + uint8_t class_version; + boolean_t is_responder; + boolean_t is_trap_processor; + boolean_t is_report_processor; + uint32_t send_q_size; + uint32_t recv_q_size; } osm_bind_info_t; /* * FIELDS @@ -142,13 +140,12 @@ typedef struct _osm_bind_info * * SYNOPSIS */ -typedef struct _osm_ni_context -{ - ib_net64_t node_guid; - uint8_t port_num; - ib_net64_t dup_node_guid; - uint8_t dup_port_num; - unsigned dup_count; +typedef struct _osm_ni_context { + ib_net64_t node_guid; + uint8_t port_num; + ib_net64_t dup_node_guid; + uint8_t dup_port_num; + unsigned dup_count; } osm_ni_context_t; /* * FIELDS @@ -175,15 +172,14 @@ typedef struct _osm_ni_context * * SYNOPSIS */ -typedef struct _osm_pi_context -{ - ib_net64_t node_guid; - ib_net64_t port_guid; - boolean_t set_method; - boolean_t light_sweep; - boolean_t update_master_sm_base_lid; - boolean_t ignore_errors; - boolean_t active_transition; +typedef struct _osm_pi_context { + ib_net64_t node_guid; + ib_net64_t port_guid; + boolean_t set_method; + boolean_t light_sweep; + boolean_t update_master_sm_base_lid; + boolean_t ignore_errors; + boolean_t active_transition; } osm_pi_context_t; /*********/ @@ -196,9 +192,8 @@ typedef struct _osm_pi_context * * SYNOPSIS */ -typedef struct _osm_nd_context -{ - ib_net64_t node_guid; +typedef struct _osm_nd_context { + ib_net64_t node_guid; } osm_nd_context_t; /*********/ @@ -211,11 +206,10 @@ typedef struct _osm_nd_context * * SYNOPSIS */ -typedef struct _osm_si_context -{ - ib_net64_t node_guid; - boolean_t set_method; - boolean_t light_sweep; +typedef struct _osm_si_context { + ib_net64_t node_guid; + boolean_t set_method; + boolean_t light_sweep; } osm_si_context_t; /*********/ @@ -228,10 +222,9 @@ typedef struct _osm_si_context * * SYNOPSIS */ -typedef struct _osm_lft_context -{ - ib_net64_t node_guid; - boolean_t set_method; +typedef struct _osm_lft_context { + ib_net64_t node_guid; + boolean_t set_method; } osm_lft_context_t; /*********/ @@ -244,10 +237,9 @@ typedef struct _osm_lft_context * * SYNOPSIS */ -typedef struct _osm_mft_context -{ - ib_net64_t node_guid; - boolean_t set_method; +typedef struct _osm_mft_context { + ib_net64_t node_guid; + boolean_t set_method; } osm_mft_context_t; /*********/ @@ -260,10 +252,9 @@ typedef struct _osm_mft_context * * SYNOPSIS */ -typedef struct _osm_smi_context -{ - ib_net64_t port_guid; - boolean_t set_method; +typedef struct _osm_smi_context { + ib_net64_t port_guid; + boolean_t set_method; } osm_smi_context_t; /*********/ @@ -276,11 +267,10 @@ typedef struct _osm_smi_context * * SYNOPSIS */ -typedef struct _osm_pkey_context -{ - ib_net64_t node_guid; - ib_net64_t port_guid; - boolean_t set_method; +typedef struct _osm_pkey_context { + ib_net64_t node_guid; + ib_net64_t port_guid; + boolean_t set_method; } osm_pkey_context_t; /*********/ @@ -293,11 +283,10 @@ typedef struct _osm_pkey_context * * SYNOPSIS */ -typedef struct _osm_slvl_context -{ - ib_net64_t node_guid; - ib_net64_t port_guid; - boolean_t set_method; +typedef struct _osm_slvl_context { + ib_net64_t node_guid; + ib_net64_t port_guid; + boolean_t set_method; } osm_slvl_context_t; /*********/ @@ -310,11 +299,10 @@ typedef struct _osm_slvl_context * * SYNOPSIS */ -typedef struct _osm_vla_context -{ - ib_net64_t node_guid; - ib_net64_t port_guid; - boolean_t set_method; +typedef struct _osm_vla_context { + ib_net64_t node_guid; + ib_net64_t port_guid; + boolean_t set_method; } osm_vla_context_t; /*********/ @@ -323,11 +311,11 @@ typedef struct _osm_vla_context * Context for Performance manager queries */ typedef struct _osm_perfmgr_context { - uint64_t node_guid; - uint16_t port; - uint8_t mad_method; /* was this a get or a set */ + uint64_t node_guid; + uint16_t port; + uint8_t mad_method; /* was this a get or a set */ #if 0 - struct timeval query_start; + struct timeval query_start; #endif } osm_perfmgr_context_t; /*********/ @@ -342,10 +330,9 @@ typedef struct _osm_perfmgr_context { * * SYNOPSIS */ -typedef struct _osm_arbitrary_context -{ - void* context1; - void* context2; +typedef struct _osm_arbitrary_context { + void *context1; + void *context2; } osm_arbitrary_context_t; /*********/ #endif @@ -359,19 +346,18 @@ typedef struct _osm_arbitrary_context * * SYNOPSIS */ -typedef union _osm_madw_context -{ - osm_ni_context_t ni_context; - osm_pi_context_t pi_context; - osm_nd_context_t nd_context; - osm_si_context_t si_context; - osm_lft_context_t lft_context; - osm_mft_context_t mft_context; - osm_smi_context_t smi_context; - osm_slvl_context_t slvl_context; - osm_pkey_context_t pkey_context; - osm_vla_context_t vla_context; - osm_perfmgr_context_t perfmgr_context; +typedef union _osm_madw_context { + osm_ni_context_t ni_context; + osm_pi_context_t pi_context; + osm_nd_context_t nd_context; + osm_si_context_t si_context; + osm_lft_context_t lft_context; + osm_mft_context_t mft_context; + osm_smi_context_t smi_context; + osm_slvl_context_t slvl_context; + osm_pkey_context_t pkey_context; + osm_vla_context_t vla_context; + osm_perfmgr_context_t perfmgr_context; #ifndef OSM_VENDOR_INTF_OPENIB osm_arbitrary_context_t arb_context; #endif @@ -386,30 +372,26 @@ typedef union _osm_madw_context * * SYNOPSIS */ -typedef struct _osm_mad_addr -{ - ib_net16_t dest_lid; - uint8_t path_bits; - uint8_t static_rate; - - union addr_type - { - struct _smi - { - ib_net16_t source_lid; - uint8_t port_num; - } smi; - - struct _gsi - { - ib_net32_t remote_qp; - ib_net32_t remote_qkey; - ib_net16_t pkey; - uint8_t service_level; - boolean_t global_route; - ib_grh_t grh_info; - } gsi; - } addr_type; +typedef struct _osm_mad_addr { + ib_net16_t dest_lid; + uint8_t path_bits; + uint8_t static_rate; + + union addr_type { + struct _smi { + ib_net16_t source_lid; + uint8_t port_num; + } smi; + + struct _gsi { + ib_net32_t remote_qp; + ib_net32_t remote_qkey; + ib_net16_t pkey; + uint8_t service_level; + boolean_t global_route; + ib_grh_t grh_info; + } gsi; + } addr_type; } osm_mad_addr_t; /* @@ -427,19 +409,18 @@ typedef struct _osm_mad_addr * * SYNOPSIS */ -typedef struct _osm_madw -{ - cl_pool_item_t pool_item; - osm_bind_handle_t h_bind; - osm_vend_wrap_t vend_wrap; - osm_mad_addr_t mad_addr; - osm_bind_info_t bind_info; - osm_madw_context_t context; - uint32_t mad_size; - ib_api_status_t status; - cl_disp_msgid_t fail_msg; - boolean_t resp_expected; - const ib_mad_t *p_mad; +typedef struct _osm_madw { + cl_pool_item_t pool_item; + osm_bind_handle_t h_bind; + osm_vend_wrap_t vend_wrap; + osm_mad_addr_t mad_addr; + osm_bind_info_t bind_info; + osm_madw_context_t context; + uint32_t mad_size; + ib_api_status_t status; + cl_disp_msgid_t fail_msg; + boolean_t resp_expected; + const ib_mad_t *p_mad; } osm_madw_t; /* * FIELDS @@ -496,17 +477,16 @@ typedef struct _osm_madw * * SYNOPSIS */ -static inline void -osm_madw_construct( - IN osm_madw_t* const p_madw ) +static inline void osm_madw_construct(IN osm_madw_t * const p_madw) { /* - Don't touch the pool_item since that is an opaque object. - Clear all other objects in the mad wrapper. - */ - memset( ((uint8_t *)p_madw) + sizeof( cl_pool_item_t ), 0, - sizeof(*p_madw) - sizeof( cl_pool_item_t ) ); + Don't touch the pool_item since that is an opaque object. + Clear all other objects in the mad wrapper. + */ + memset(((uint8_t *) p_madw) + sizeof(cl_pool_item_t), 0, + sizeof(*p_madw) - sizeof(cl_pool_item_t)); } + /* * PARAMETERS * p_madw @@ -535,8 +515,7 @@ osm_madw_construct( * * SYNOPSIS */ -void osm_madw_destroy( - IN osm_madw_t* const p_madw ); +void osm_madw_destroy(IN osm_madw_t * const p_madw); /* * PARAMETERS * p_madw @@ -565,20 +544,20 @@ void osm_madw_destroy( * SYNOPSIS */ static inline void -osm_madw_init( - IN osm_madw_t* const p_madw, - IN osm_bind_handle_t h_bind, - IN const uint32_t mad_size, - IN const osm_mad_addr_t* const p_mad_addr ) +osm_madw_init(IN osm_madw_t * const p_madw, + IN osm_bind_handle_t h_bind, + IN const uint32_t mad_size, + IN const osm_mad_addr_t * const p_mad_addr) { - osm_madw_construct( p_madw ); + osm_madw_construct(p_madw); p_madw->h_bind = h_bind; p_madw->fail_msg = CL_DISP_MSGID_NONE; p_madw->mad_size = mad_size; - if( p_mad_addr ) + if (p_mad_addr) p_madw->mad_addr = *p_mad_addr; p_madw->resp_expected = FALSE; } + /* * PARAMETERS * p_madw @@ -608,12 +587,11 @@ osm_madw_init( * * SYNOPSIS */ -static inline ib_smp_t* -osm_madw_get_smp_ptr( - IN const osm_madw_t* const p_madw ) +static inline ib_smp_t *osm_madw_get_smp_ptr(IN const osm_madw_t * const p_madw) { - return( (ib_smp_t*)p_madw->p_mad ); + return ((ib_smp_t *) p_madw->p_mad); } + /* * PARAMETERS * p_madw @@ -637,12 +615,12 @@ osm_madw_get_smp_ptr( * * SYNOPSIS */ -static inline ib_sa_mad_t* -osm_madw_get_sa_mad_ptr( - IN const osm_madw_t* const p_madw ) +static inline ib_sa_mad_t *osm_madw_get_sa_mad_ptr(IN const osm_madw_t * + const p_madw) { - return( (ib_sa_mad_t*)p_madw->p_mad ); + return ((ib_sa_mad_t *) p_madw->p_mad); } + /* * PARAMETERS * p_madw @@ -663,12 +641,12 @@ osm_madw_get_sa_mad_ptr( * * SYNOPSIS */ -static inline ib_perfmgt_mad_t* -osm_madw_get_perfmgt_mad_ptr( - IN const osm_madw_t* const p_madw ) +static inline ib_perfmgt_mad_t *osm_madw_get_perfmgt_mad_ptr(IN const osm_madw_t + * const p_madw) { - return((ib_perfmgt_mad_t*)p_madw->p_mad); + return ((ib_perfmgt_mad_t *) p_madw->p_mad); } + /* * PARAMETERS * p_madw @@ -692,12 +670,12 @@ osm_madw_get_perfmgt_mad_ptr( * * SYNOPSIS */ -static inline osm_ni_context_t* -osm_madw_get_ni_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_ni_context_t *osm_madw_get_ni_context_ptr(IN const osm_madw_t + * const p_madw) { - return( (osm_ni_context_t*)&p_madw->context ); + return ((osm_ni_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -720,12 +698,12 @@ osm_madw_get_ni_context_ptr( * * SYNOPSIS */ -static inline osm_pi_context_t* -osm_madw_get_pi_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_pi_context_t *osm_madw_get_pi_context_ptr(IN const osm_madw_t + * const p_madw) { - return( (osm_pi_context_t*)&p_madw->context ); + return ((osm_pi_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -748,12 +726,12 @@ osm_madw_get_pi_context_ptr( * * SYNOPSIS */ -static inline osm_nd_context_t* -osm_madw_get_nd_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_nd_context_t *osm_madw_get_nd_context_ptr(IN const osm_madw_t + * const p_madw) { - return( (osm_nd_context_t*)&p_madw->context ); + return ((osm_nd_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -776,12 +754,13 @@ osm_madw_get_nd_context_ptr( * * SYNOPSIS */ -static inline osm_lft_context_t* -osm_madw_get_lft_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_lft_context_t *osm_madw_get_lft_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_lft_context_t*)&p_madw->context ); + return ((osm_lft_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -804,12 +783,13 @@ osm_madw_get_lft_context_ptr( * * SYNOPSIS */ -static inline osm_mft_context_t* -osm_madw_get_mft_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_mft_context_t *osm_madw_get_mft_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_mft_context_t*)&p_madw->context ); + return ((osm_mft_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -832,12 +812,12 @@ osm_madw_get_mft_context_ptr( * * SYNOPSIS */ -static inline osm_si_context_t* -osm_madw_get_si_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_si_context_t *osm_madw_get_si_context_ptr(IN const osm_madw_t + * const p_madw) { - return( (osm_si_context_t*)&p_madw->context ); + return ((osm_si_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -860,12 +840,13 @@ osm_madw_get_si_context_ptr( * * SYNOPSIS */ -static inline osm_smi_context_t* -osm_madw_get_smi_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_smi_context_t *osm_madw_get_smi_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_smi_context_t*)&p_madw->context ); + return ((osm_smi_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -888,12 +869,13 @@ osm_madw_get_smi_context_ptr( * * SYNOPSIS */ -static inline osm_pkey_context_t* -osm_madw_get_pkey_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_pkey_context_t *osm_madw_get_pkey_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_pkey_context_t*)&p_madw->context ); + return ((osm_pkey_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -916,12 +898,13 @@ osm_madw_get_pkey_context_ptr( * * SYNOPSIS */ -static inline osm_slvl_context_t* -osm_madw_get_slvl_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_slvl_context_t *osm_madw_get_slvl_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_slvl_context_t*)&p_madw->context ); + return ((osm_slvl_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -944,12 +927,13 @@ osm_madw_get_slvl_context_ptr( * * SYNOPSIS */ -static inline osm_vla_context_t* -osm_madw_get_vla_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_vla_context_t *osm_madw_get_vla_context_ptr(IN const + osm_madw_t * + const p_madw) { - return( (osm_vla_context_t*)&p_madw->context ); + return ((osm_vla_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -973,12 +957,16 @@ osm_madw_get_vla_context_ptr( * * SYNOPSIS */ -static inline osm_arbitrary_context_t* -osm_madw_get_arbitrary_context_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_arbitrary_context_t *osm_madw_get_arbitrary_context_ptr(IN + const + osm_madw_t + * + const + p_madw) { - return( (osm_arbitrary_context_t*)&p_madw->context ); + return ((osm_arbitrary_context_t *) & p_madw->context); } + /* * PARAMETERS * p_madw @@ -1002,12 +990,12 @@ osm_madw_get_arbitrary_context_ptr( * * SYNOPSIS */ -static inline osm_vend_wrap_t* -osm_madw_get_vend_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_vend_wrap_t *osm_madw_get_vend_ptr(IN const osm_madw_t * + const p_madw) { - return( (osm_vend_wrap_t*)&p_madw->vend_wrap ); + return ((osm_vend_wrap_t *) & p_madw->vend_wrap); } + /* * PARAMETERS * p_madw @@ -1031,11 +1019,11 @@ osm_madw_get_vend_ptr( * SYNOPSIS */ static inline osm_bind_handle_t -osm_madw_get_bind_handle( - IN const osm_madw_t* const p_madw ) +osm_madw_get_bind_handle(IN const osm_madw_t * const p_madw) { - return( (osm_bind_handle_t)p_madw->h_bind ); + return ((osm_bind_handle_t) p_madw->h_bind); } + /* * PARAMETERS * p_madw @@ -1058,12 +1046,12 @@ osm_madw_get_bind_handle( * * SYNOPSIS */ -static inline osm_mad_addr_t* -osm_madw_get_mad_addr_ptr( - IN const osm_madw_t* const p_madw ) +static inline osm_mad_addr_t *osm_madw_get_mad_addr_ptr(IN const osm_madw_t * + const p_madw) { - return( (osm_mad_addr_t*)&p_madw->mad_addr ); + return ((osm_mad_addr_t *) & p_madw->mad_addr); } + /* * PARAMETERS * p_madw @@ -1086,12 +1074,11 @@ osm_madw_get_mad_addr_ptr( * * SYNOPSIS */ -static inline ib_mad_t* -osm_madw_get_mad_ptr( - IN const osm_madw_t* const p_madw ) +static inline ib_mad_t *osm_madw_get_mad_ptr(IN const osm_madw_t * const p_madw) { - return( (ib_mad_t*)p_madw->p_mad ); + return ((ib_mad_t *) p_madw->p_mad); } + /* * PARAMETERS * p_madw @@ -1116,11 +1103,11 @@ osm_madw_get_mad_ptr( * SYNOPSIS */ static inline cl_disp_msgid_t -osm_madw_get_err_msg( - IN const osm_madw_t* const p_madw ) +osm_madw_get_err_msg(IN const osm_madw_t * const p_madw) { - return( (cl_disp_msgid_t)p_madw->fail_msg ); + return ((cl_disp_msgid_t) p_madw->fail_msg); } + /* * PARAMETERS * p_madw @@ -1145,12 +1132,11 @@ osm_madw_get_err_msg( * SYNOPSIS */ static inline void -osm_madw_set_mad( - IN osm_madw_t* const p_madw, - IN const ib_mad_t* const p_mad ) +osm_madw_set_mad(IN osm_madw_t * const p_madw, IN const ib_mad_t * const p_mad) { p_madw->p_mad = p_mad; } + /* * PARAMETERS * p_madw @@ -1177,12 +1163,12 @@ osm_madw_set_mad( * SYNOPSIS */ static inline void -osm_madw_copy_context( - IN osm_madw_t* const p_dest, - IN const osm_madw_t* const p_src ) +osm_madw_copy_context(IN osm_madw_t * const p_dest, + IN const osm_madw_t * const p_src) { p_dest->context = p_src->context; } + /* * PARAMETERS * p_dest @@ -1200,5 +1186,4 @@ osm_madw_copy_context( *********/ END_C_DECLS - -#endif /* _OSM_MADW_H_ */ +#endif /* _OSM_MADW_H_ */ diff --git a/opensm/include/opensm/osm_mcast_fwd_rcv.h b/opensm/include/opensm/osm_mcast_fwd_rcv.h index 3a2de28..9d81ee6 100644 --- a/opensm/include/opensm/osm_mcast_fwd_rcv.h +++ b/opensm/include/opensm/osm_mcast_fwd_rcv.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/MFT Receiver * NAME * MFT Receiver @@ -81,7 +80,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: MFT Receiver/osm_mft_rcv_t * NAME * osm_mft_rcv_t @@ -94,11 +92,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mft_rcv -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_mft_rcv { + osm_subn_t *p_subn; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_mft_rcv_t; /* * FIELDS @@ -124,9 +121,7 @@ typedef struct _osm_mft_rcv * * SYNOPSIS */ -void -osm_mft_rcv_construct( - IN osm_mft_rcv_t* const p_rcv ); +void osm_mft_rcv_construct(IN osm_mft_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -156,9 +151,7 @@ osm_mft_rcv_construct( * * SYNOPSIS */ -void -osm_mft_rcv_destroy( - IN osm_mft_rcv_t* const p_rcv ); +void osm_mft_rcv_destroy(IN osm_mft_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -190,11 +183,9 @@ osm_mft_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_mft_rcv_init( - IN osm_mft_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_mft_rcv_init(IN osm_mft_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -230,10 +221,7 @@ osm_mft_rcv_init( * * SYNOPSIS */ -void -osm_mft_rcv_process( - IN void *context, - IN void *data ); +void osm_mft_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -254,5 +242,4 @@ osm_mft_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_MFT_RCV_H_ */ +#endif /* _OSM_MFT_RCV_H_ */ diff --git a/opensm/include/opensm/osm_mcast_mgr.h b/opensm/include/opensm/osm_mcast_mgr.h index 462c841..3e0b761 100644 --- a/opensm/include/opensm/osm_mcast_mgr.h +++ b/opensm/include/opensm/osm_mcast_mgr.h @@ -59,15 +59,13 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSM_MCAST_MGR_LIST_SIZE_MIN 256 - /****h* OpenSM/Multicast Manager * NAME * Multicast Manager @@ -85,7 +83,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Multicast Manager/osm_mcast_mgr_t * NAME * osm_mcast_mgr_t @@ -98,12 +95,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mcast_mgr -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_mcast_mgr { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_mcast_mgr_t; /* @@ -133,9 +129,7 @@ typedef struct _osm_mcast_mgr * * SYNOPSIS */ -void -osm_mcast_mgr_construct( - IN osm_mcast_mgr_t* const p_mgr ); +void osm_mcast_mgr_construct(IN osm_mcast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -165,9 +159,7 @@ osm_mcast_mgr_construct( * * SYNOPSIS */ -void -osm_mcast_mgr_destroy( - IN osm_mcast_mgr_t* const p_mgr ); +void osm_mcast_mgr_destroy(IN osm_mcast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -199,12 +191,10 @@ osm_mcast_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_mcast_mgr_init( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_mcast_mgr_init(IN osm_mcast_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_mgr @@ -243,9 +233,7 @@ osm_mcast_mgr_init( * * SYNOPSIS */ -osm_signal_t -osm_mcast_mgr_process( - IN osm_mcast_mgr_t* const p_mgr ); +osm_signal_t osm_mcast_mgr_process(IN osm_mcast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -275,9 +263,7 @@ osm_mcast_mgr_process( * SYNOPSIS */ osm_signal_t -osm_mcast_mgr_process_mgrp_cb( - IN void* const Context1, - IN void* const Context2 ); +osm_mcast_mgr_process_mgrp_cb(IN void *const Context1, IN void *const Context2); /* * PARAMETERS * (Context1) p_mgr @@ -306,11 +292,10 @@ osm_mcast_mgr_process_mgrp_cb( * SYNOPSIS */ ib_api_status_t -osm_mcast_mgr_process_single( - IN osm_mcast_mgr_t* const p_mgr, - IN const ib_net16_t mlid, - IN const ib_net64_t port_guid, - IN const uint8_t join_state ); +osm_mcast_mgr_process_single(IN osm_mcast_mgr_t * const p_mgr, + IN const ib_net16_t mlid, + IN const ib_net64_t port_guid, + IN const uint8_t join_state); /* * PARAMETERS * p_mgr @@ -335,5 +320,4 @@ osm_mcast_mgr_process_single( *********/ END_C_DECLS - -#endif /* _OSM_MCAST_MGR_H_ */ +#endif /* _OSM_MCAST_MGR_H_ */ diff --git a/opensm/include/opensm/osm_mcast_tbl.h b/opensm/include/opensm/osm_mcast_tbl.h index ca609ae..46a41ec 100644 --- a/opensm/include/opensm/osm_mcast_tbl.h +++ b/opensm/include/opensm/osm_mcast_tbl.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Forwarding Table/osm_mcast_tbl_t * NAME * osm_mcast_tbl_t @@ -73,15 +72,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mcast_fwd_tbl -{ - uint8_t num_ports; - uint8_t max_position; - uint16_t max_block; - int16_t max_block_in_use; - uint16_t num_entries; - uint16_t max_mlid_ho; - uint16_t (*p_mask_tbl)[][IB_MCAST_POSITION_MAX]; +typedef struct _osm_mcast_fwd_tbl { + uint8_t num_ports; + uint8_t max_position; + uint16_t max_block; + int16_t max_block_in_use; + uint16_t num_entries; + uint16_t max_mlid_ho; + uint16_t(*p_mask_tbl)[][IB_MCAST_POSITION_MAX]; } osm_mcast_tbl_t; /* * FIELDS @@ -122,10 +120,8 @@ typedef struct _osm_mcast_fwd_tbl * SYNOPSIS */ ib_api_status_t -osm_mcast_tbl_init( - IN osm_mcast_tbl_t* const p_tbl, - IN uint8_t const num_ports, - IN uint16_t const capacity ); +osm_mcast_tbl_init(IN osm_mcast_tbl_t * const p_tbl, + IN uint8_t const num_ports, IN uint16_t const capacity); /* * PARAMETERS * num_ports @@ -152,9 +148,7 @@ osm_mcast_tbl_init( * * SYNOPSIS */ -void -osm_mcast_tbl_delete( - IN osm_mcast_tbl_t** const pp_tbl ); +void osm_mcast_tbl_delete(IN osm_mcast_tbl_t ** const pp_tbl); /* * PARAMETERS * pp_tbl @@ -179,9 +173,7 @@ osm_mcast_tbl_delete( * * SYNOPSIS */ -void -osm_mcast_tbl_destroy( - IN osm_mcast_tbl_t* const p_tbl ); +void osm_mcast_tbl_destroy(IN osm_mcast_tbl_t * const p_tbl); /* * PARAMETERS * p_tbl @@ -205,10 +197,8 @@ osm_mcast_tbl_destroy( * SYNOPSIS */ void -osm_mcast_tbl_set( - IN osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho, - IN const uint8_t port_num ); +osm_mcast_tbl_set(IN osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho, IN const uint8_t port_num); /* * PARAMETERS * p_tbl @@ -238,9 +228,8 @@ osm_mcast_tbl_set( * SYNOPSIS */ void -osm_mcast_tbl_clear_mlid( - IN osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho ); +osm_mcast_tbl_clear_mlid(IN osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho); /* * PARAMETERS * p_tbl @@ -267,10 +256,8 @@ osm_mcast_tbl_clear_mlid( * SYNOPSIS */ boolean_t -osm_mcast_tbl_is_port( - IN const osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho, - IN const uint8_t port_num ); +osm_mcast_tbl_is_port(IN const osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho, IN const uint8_t port_num); /* * PARAMETERS * p_tbl @@ -300,9 +287,8 @@ osm_mcast_tbl_is_port( * SYNOPSIS */ boolean_t -osm_mcast_tbl_is_any_port( - IN const osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho ); +osm_mcast_tbl_is_any_port(IN const osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho); /* * PARAMETERS * p_tbl @@ -329,11 +315,9 @@ osm_mcast_tbl_is_any_port( * SYNOPSIS */ ib_api_status_t -osm_mcast_tbl_set_block( - IN osm_mcast_tbl_t* const p_tbl, - IN const ib_net16_t* const p_block, - IN const int16_t block_num, - IN const uint8_t position ); +osm_mcast_tbl_set_block(IN osm_mcast_tbl_t * const p_tbl, + IN const ib_net16_t * const p_block, + IN const int16_t block_num, IN const uint8_t position); /* * PARAMETERS * p_tbl @@ -363,11 +347,10 @@ osm_mcast_tbl_set_block( * SYNOPSIS */ boolean_t -osm_mcast_tbl_get_block( - IN osm_mcast_tbl_t* const p_tbl, - IN const int16_t block_num, - IN const uint8_t position, - OUT ib_net16_t* const p_block ); +osm_mcast_tbl_get_block(IN osm_mcast_tbl_t * const p_tbl, + IN const int16_t block_num, + IN const uint8_t position, + OUT ib_net16_t * const p_block); /* * PARAMETERS * p_tbl @@ -403,11 +386,11 @@ osm_mcast_tbl_get_block( * SYNOPSIS */ static inline uint16_t -osm_mcast_tbl_get_max_block( - IN osm_mcast_tbl_t* const p_tbl ) +osm_mcast_tbl_get_max_block(IN osm_mcast_tbl_t * const p_tbl) { - return( p_tbl->max_block ); + return (p_tbl->max_block); } + /* * PARAMETERS * p_tbl @@ -432,11 +415,11 @@ osm_mcast_tbl_get_max_block( * SYNOPSIS */ static inline int16_t -osm_mcast_tbl_get_max_block_in_use( - IN osm_mcast_tbl_t* const p_tbl ) +osm_mcast_tbl_get_max_block_in_use(IN osm_mcast_tbl_t * const p_tbl) { - return( p_tbl->max_block_in_use ); + return (p_tbl->max_block_in_use); } + /* * PARAMETERS * p_tbl @@ -461,11 +444,11 @@ osm_mcast_tbl_get_max_block_in_use( * SYNOPSIS */ static inline uint8_t -osm_mcast_tbl_get_max_position( - IN osm_mcast_tbl_t* const p_tbl ) +osm_mcast_tbl_get_max_position(IN osm_mcast_tbl_t * const p_tbl) { - return( p_tbl->max_position ); + return (p_tbl->max_position); } + /* * PARAMETERS * p_tbl @@ -480,5 +463,4 @@ osm_mcast_tbl_get_max_position( *********/ END_C_DECLS - -#endif /* _OSM_MCAST_TBL_H_ */ +#endif /* _OSM_MCAST_TBL_H_ */ diff --git a/opensm/include/opensm/osm_mcm_info.h b/opensm/include/opensm/osm_mcm_info.h index 4b09f0c..202839d 100644 --- a/opensm/include/opensm/osm_mcm_info.h +++ b/opensm/include/opensm/osm_mcm_info.h @@ -56,13 +56,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Multicast Member Info/osm_mcm_info_t * NAME * osm_mcm_info_t @@ -77,10 +76,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mcm_info -{ - cl_list_item_t list_item; - ib_net16_t mlid; +typedef struct _osm_mcm_info { + cl_list_item_t list_item; + ib_net16_t mlid; } osm_mcm_info_t; /* * FIELDS @@ -102,9 +100,7 @@ typedef struct _osm_mcm_info * * SYNOPSIS */ -osm_mcm_info_t* -osm_mcm_info_new( - IN const ib_net16_t mlid ); +osm_mcm_info_t *osm_mcm_info_new(IN const ib_net16_t mlid); /* * PARAMETERS * mlid @@ -127,9 +123,7 @@ osm_mcm_info_new( * * SYNOPSIS */ -void -osm_mcm_info_delete( - IN osm_mcm_info_t* const p_mcm ); +void osm_mcm_info_delete(IN osm_mcm_info_t * const p_mcm); /* * PARAMETERS * p_mcm @@ -144,5 +138,4 @@ osm_mcm_info_delete( *********/ END_C_DECLS - -#endif /* _OSM_MCM_INFO_H_ */ +#endif /* _OSM_MCM_INFO_H_ */ diff --git a/opensm/include/opensm/osm_mcm_port.h b/opensm/include/opensm/osm_mcm_port.h index 354c65e..da3ca15 100644 --- a/opensm/include/opensm/osm_mcm_port.h +++ b/opensm/include/opensm/osm_mcm_port.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: MCM Port Object/osm_mcm_port_t * NAME * osm_mcm_port_t @@ -75,12 +74,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mcm_port -{ - cl_map_item_t map_item; - ib_gid_t port_gid; - uint8_t scope_state; - boolean_t proxy_join; +typedef struct _osm_mcm_port { + cl_map_item_t map_item; + ib_gid_t port_gid; + uint8_t scope_state; + boolean_t proxy_join; } osm_mcm_port_t; /* * FIELDS @@ -113,11 +111,9 @@ typedef struct _osm_mcm_port * * SYNOPSIS */ -osm_mcm_port_t* -osm_mcm_port_new( - IN const ib_gid_t* const p_port_gid, - IN const uint8_t scope_state, - IN const boolean_t proxy_join ); +osm_mcm_port_t *osm_mcm_port_new(IN const ib_gid_t * const p_port_gid, + IN const uint8_t scope_state, + IN const boolean_t proxy_join); /* * PARAMETERS * p_port_gid @@ -148,9 +144,7 @@ osm_mcm_port_new( * * SYNOPSIS */ -void -osm_mcm_port_delete( - IN osm_mcm_port_t* const p_mcm ); +void osm_mcm_port_delete(IN osm_mcm_port_t * const p_mcm); /* * PARAMETERS * p_mcm @@ -166,5 +160,4 @@ osm_mcm_port_delete( *********/ END_C_DECLS - -#endif /* _OSM_MCM_PORT_H_ */ +#endif /* _OSM_MCM_PORT_H_ */ diff --git a/opensm/include/opensm/osm_msgdef.h b/opensm/include/opensm/osm_msgdef.h index f6ec8f8..c4e1714 100644 --- a/opensm/include/opensm/osm_msgdef.h +++ b/opensm/include/opensm/osm_msgdef.h @@ -49,13 +49,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Dispatcher Messages * NAME * Dispatcher Messages @@ -76,7 +75,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Dispatcher Messages/OSM_MSG_MAD_NODE_INFO * NAME * OSM_MSG_MAD_NODE_INFO @@ -91,7 +89,6 @@ BEGIN_C_DECLS * * ***********/ - /****s* OpenSM: Dispatcher Messages/OSM_MSG_MAD_PORT_INFO * NAME * OSM_MSG_MAD_PORT_INFO @@ -106,7 +103,6 @@ BEGIN_C_DECLS * * ***********/ - /****s* OpenSM: Dispatcher Messages/OSM_MSG_MAD_SWITCH_INFO * NAME * OSM_MSG_MAD_SWITCH_INFO @@ -120,7 +116,6 @@ BEGIN_C_DECLS * Delivery notice: yes * ***********/ - /****s* OpenSM: Dispatcher Messages/OSM_MSG_MAD_NODE_DESC * NAME * OSM_MSG_MAD_NODE_DESC @@ -135,7 +130,6 @@ BEGIN_C_DECLS * * SOURCE ***********/ - /****d* OpenSM: Dispatcher Messages/OSM_MSG_NO_SMPS_OUTSTANDING * NAME * OSM_MSG_NO_SMPS_OUTSTANDING @@ -150,8 +144,7 @@ BEGIN_C_DECLS * * SOURCE ***********/ -enum -{ +enum { OSM_MSG_NONE = 0, OSM_MSG_MAD_NODE_INFO, OSM_MSG_MAD_PORT_INFO, @@ -191,5 +184,4 @@ enum }; END_C_DECLS - -#endif /* _OSM_MSGDEF_H_ */ +#endif /* _OSM_MSGDEF_H_ */ diff --git a/opensm/include/opensm/osm_mtree.h b/opensm/include/opensm/osm_mtree.h index 4b6e669..d1d719e 100644 --- a/opensm/include/opensm/osm_mtree.h +++ b/opensm/include/opensm/osm_mtree.h @@ -57,15 +57,13 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSM_MTREE_LEAF ((void*)-1) - /****h* OpenSM/Multicast Tree * NAME * Multicast Tree @@ -88,7 +86,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Multicast Tree/osm_mtree_node_t * NAME * osm_mtree_node_t @@ -105,13 +102,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mtree_node -{ - cl_map_item_t map_item; - osm_switch_t *p_sw; - uint8_t max_children; - struct _osm_mtree_node *p_up; - struct _osm_mtree_node *child_array[1]; +typedef struct _osm_mtree_node { + cl_map_item_t map_item; + osm_switch_t *p_sw; + uint8_t max_children; + struct _osm_mtree_node *p_up; + struct _osm_mtree_node *child_array[1]; } osm_mtree_node_t; /* * FIELDS @@ -147,9 +143,7 @@ typedef struct _osm_mtree_node * * SYNOPSIS */ -osm_mtree_node_t* -osm_mtree_node_new( - IN const osm_switch_t* const p_sw ); +osm_mtree_node_t *osm_mtree_node_new(IN const osm_switch_t * const p_sw); /* * PARAMETERS * p_sw @@ -172,9 +166,7 @@ osm_mtree_node_new( * * SYNOPSIS */ -void -osm_mtree_destroy( - IN osm_mtree_node_t *p_mtn ); +void osm_mtree_destroy(IN osm_mtree_node_t * p_mtn); /* * PARAMETERS * p_mtn @@ -201,11 +193,11 @@ osm_mtree_destroy( * SYNOPSIS */ static inline uint8_t -osm_mtree_node_get_max_children( - IN const osm_mtree_node_t* const p_mtn ) +osm_mtree_node_get_max_children(IN const osm_mtree_node_t * const p_mtn) { - return( p_mtn->max_children ); + return (p_mtn->max_children); } + /* * PARAMETERS * p_mtn @@ -228,14 +220,15 @@ osm_mtree_node_get_max_children( * * SYNOPSIS */ -static inline osm_mtree_node_t* -osm_mtree_node_get_child( - IN const osm_mtree_node_t* const p_mtn, - IN const uint8_t child ) +static inline osm_mtree_node_t *osm_mtree_node_get_child(IN const + osm_mtree_node_t * + const p_mtn, + IN const uint8_t child) { - CL_ASSERT( child < p_mtn->max_children ); - return( p_mtn->child_array[child] ); + CL_ASSERT(child < p_mtn->max_children); + return (p_mtn->child_array[child]); } + /* * PARAMETERS * p_mtn @@ -262,12 +255,13 @@ osm_mtree_node_get_child( * * SYNOPSIS */ -static inline osm_switch_t* -osm_mtree_node_get_switch_ptr( - IN const osm_mtree_node_t* const p_mtn ) +static inline osm_switch_t *osm_mtree_node_get_switch_ptr(IN const + osm_mtree_node_t * + const p_mtn) { - return( p_mtn->p_sw ); + return (p_mtn->p_sw); } + /* * PARAMETERS * p_mtn @@ -286,5 +280,4 @@ osm_mtree_node_get_switch_ptr( *********/ END_C_DECLS - -#endif /* _OSM_MTREE_H_ */ +#endif /* _OSM_MTREE_H_ */ diff --git a/opensm/include/opensm/osm_multicast.h b/opensm/include/opensm/osm_multicast.h index 79a6e5a..729a2ea 100644 --- a/opensm/include/opensm/osm_multicast.h +++ b/opensm/include/opensm/osm_multicast.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Multicast Group * NAME * Multicast Group @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****f* IBA Base: OpneSM: Multicast Group/osm_get_mcast_req_type_str * NAME * osm_get_mcast_req_type_str @@ -96,9 +94,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -const char* -osm_get_mcast_req_type_str( - IN osm_mcast_req_type_t req_type ); +const char *osm_get_mcast_req_type_str(IN osm_mcast_req_type_t req_type); /* * PARAMETERS * req_type @@ -124,11 +120,10 @@ osm_get_mcast_req_type_str( * * SYNOPSIS */ -typedef struct osm_mcast_mgr_ctxt -{ - ib_net16_t mlid; - osm_mcast_req_type_t req_type; - ib_net64_t port_guid; +typedef struct osm_mcast_mgr_ctxt { + ib_net16_t mlid; + osm_mcast_req_type_t req_type; + ib_net64_t port_guid; } osm_mcast_mgr_ctxt_t; /* * FIELDS @@ -159,17 +154,16 @@ typedef struct osm_mcast_mgr_ctxt * * SYNOPSIS */ -typedef struct _osm_mgrp -{ - cl_map_item_t map_item; - ib_net16_t mlid; - osm_mtree_node_t *p_root; - cl_qmap_t mcm_port_tbl; - ib_member_rec_t mcmember_rec; - boolean_t well_known; - boolean_t to_be_deleted; - uint32_t last_change_id; - uint32_t last_tree_id; +typedef struct _osm_mgrp { + cl_map_item_t map_item; + ib_net16_t mlid; + osm_mtree_node_t *p_root; + cl_qmap_t mcm_port_tbl; + ib_member_rec_t mcmember_rec; + boolean_t well_known; + boolean_t to_be_deleted; + uint32_t last_change_id; + uint32_t last_tree_id; } osm_mgrp_t; /* * FIELDS @@ -223,10 +217,9 @@ typedef struct _osm_mgrp * * SYNOPSIS */ -typedef void (*osm_mgrp_func_t)( - IN const osm_mgrp_t* const p_mgrp, - IN const osm_mtree_node_t* const p_mtn, - IN void* context ); +typedef void (*osm_mgrp_func_t) (IN const osm_mgrp_t * const p_mgrp, + IN const osm_mtree_node_t * const p_mtn, + IN void *context); /* * PARAMETERS * p_mgrp @@ -255,9 +248,7 @@ typedef void (*osm_mgrp_func_t)( * * SYNOPSIS */ -osm_mgrp_t* -osm_mgrp_new( - IN const ib_net16_t mlid ); +osm_mgrp_t *osm_mgrp_new(IN const ib_net16_t mlid); /* * PARAMETERS * mlid @@ -282,9 +273,7 @@ osm_mgrp_new( * * SYNOPSIS */ -void -osm_mgrp_delete( - IN osm_mgrp_t* const p_mgrp ); +void osm_mgrp_delete(IN osm_mgrp_t * const p_mgrp); /* * PARAMETERS * p_mgrp @@ -309,13 +298,13 @@ osm_mgrp_delete( * SYNOPSIS */ static inline boolean_t -osm_mgrp_is_guid( - IN const osm_mgrp_t* const p_mgrp, - IN const ib_net64_t port_guid ) +osm_mgrp_is_guid(IN const osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid) { - return( cl_qmap_get( &p_mgrp->mcm_port_tbl, port_guid ) != - cl_qmap_end( &p_mgrp->mcm_port_tbl ) ); + return (cl_qmap_get(&p_mgrp->mcm_port_tbl, port_guid) != + cl_qmap_end(&p_mgrp->mcm_port_tbl)); } + /* * PARAMETERS * p_mgrp @@ -343,12 +332,11 @@ osm_mgrp_is_guid( * * SYNOPSIS */ -static inline boolean_t -osm_mgrp_is_empty( - IN const osm_mgrp_t* const p_mgrp ) +static inline boolean_t osm_mgrp_is_empty(IN const osm_mgrp_t * const p_mgrp) { - return( cl_qmap_count( &p_mgrp->mcm_port_tbl ) == 0 ); + return (cl_qmap_count(&p_mgrp->mcm_port_tbl) == 0); } + /* * PARAMETERS * p_mgrp @@ -373,12 +361,11 @@ osm_mgrp_is_empty( * * SYNOPSIS */ -static inline ib_net16_t -osm_mgrp_get_mlid( - IN const osm_mgrp_t* const p_mgrp ) +static inline ib_net16_t osm_mgrp_get_mlid(IN const osm_mgrp_t * const p_mgrp) { - return( p_mgrp->mlid ); + return (p_mgrp->mlid); } + /* * PARAMETERS * p_mgrp @@ -402,12 +389,10 @@ osm_mgrp_get_mlid( * * SYNOPSIS */ -osm_mcm_port_t* -osm_mgrp_add_port( - IN osm_mgrp_t* const p_mgrp, - IN const ib_gid_t* const p_port_gid, - IN const uint8_t join_state, - IN boolean_t proxy_join); +osm_mcm_port_t *osm_mgrp_add_port(IN osm_mgrp_t * const p_mgrp, + IN const ib_gid_t * const p_port_gid, + IN const uint8_t join_state, + IN boolean_t proxy_join); /* * PARAMETERS * p_mgrp @@ -439,10 +424,9 @@ osm_mgrp_add_port( */ boolean_t -osm_mgrp_is_port_present( - IN const osm_mgrp_t* const p_mgrp, - IN const ib_net64_t port_guid, - OUT osm_mcm_port_t ** const pp_mcm_port); +osm_mgrp_is_port_present(IN const osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid, + OUT osm_mcm_port_t ** const pp_mcm_port); /* * PARAMETERS * p_mgrp @@ -474,11 +458,10 @@ osm_mgrp_is_port_present( * SYNOPSIS */ void -osm_mgrp_remove_port( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t* const p_mgrp, - IN const ib_net64_t port_guid ); +osm_mgrp_remove_port(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid); /* * PARAMETERS * @@ -512,15 +495,15 @@ osm_mgrp_remove_port( * * SYNOPSIS */ -static inline osm_switch_t* -osm_mgrp_get_root_switch( - IN const osm_mgrp_t* const p_mgrp ) +static inline osm_switch_t *osm_mgrp_get_root_switch(IN const osm_mgrp_t * + const p_mgrp) { - if( p_mgrp->p_root ) - return( p_mgrp->p_root->p_sw ); + if (p_mgrp->p_root) + return (p_mgrp->p_root->p_sw); else - return( NULL ); + return (NULL); } + /* * PARAMETERS * p_mgrp @@ -547,9 +530,8 @@ osm_mgrp_get_root_switch( * SYNOPSIS */ float -osm_mgrp_compute_avg_hops( - const osm_mgrp_t* const p_mgrp, - const osm_switch_t* const p_sw ); +osm_mgrp_compute_avg_hops(const osm_mgrp_t * const p_mgrp, + const osm_switch_t * const p_sw); /* * PARAMETERS * p_mgrp @@ -579,10 +561,8 @@ osm_mgrp_compute_avg_hops( * SYNOPSIS */ void -osm_mgrp_apply_func( - const osm_mgrp_t* const p_mgrp, - osm_mgrp_func_t p_func, - void* context ); +osm_mgrp_apply_func(const osm_mgrp_t * const p_mgrp, + osm_mgrp_func_t p_func, void *context); /* * PARAMETERS * p_mgrp @@ -614,10 +594,8 @@ osm_mgrp_apply_func( * SYNOPSIS */ void -osm_mgrp_send_delete_notice( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t *p_mgrp ); +osm_mgrp_send_delete_notice(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN osm_mgrp_t * p_mgrp); /* * PARAMETERS * p_subn @@ -648,10 +626,8 @@ osm_mgrp_send_delete_notice( * SYNOPSIS */ void -osm_mgrp_send_create_notice( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t *p_mgrp ); +osm_mgrp_send_create_notice(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN osm_mgrp_t * p_mgrp); /* * PARAMETERS * p_subn @@ -673,5 +649,4 @@ osm_mgrp_send_create_notice( *********/ END_C_DECLS - -#endif /* _OSM_MULTICAST_H_ */ +#endif /* _OSM_MULTICAST_H_ */ diff --git a/opensm/include/opensm/osm_node.h b/opensm/include/opensm/osm_node.h index ec4af43..f87e81d 100644 --- a/opensm/include/opensm/osm_node.h +++ b/opensm/include/opensm/osm_node.h @@ -58,10 +58,10 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS @@ -99,16 +99,15 @@ struct _osm_switch; * * SYNOPSIS */ -typedef struct _osm_node -{ - cl_map_item_t map_item; +typedef struct _osm_node { + cl_map_item_t map_item; struct _osm_switch *sw; - ib_node_info_t node_info; - ib_node_desc_t node_desc; - uint32_t discovery_count; - uint32_t physp_tbl_size; - char print_desc[IB_NODE_DESCRIPTION_SIZE+1]; - osm_physp_t physp_table[1]; + ib_node_info_t node_info; + ib_node_desc_t node_desc; + uint32_t discovery_count; + uint32_t physp_tbl_size; + char print_desc[IB_NODE_DESCRIPTION_SIZE + 1]; + osm_physp_t physp_table[1]; } osm_node_t; /* * FIELDS @@ -159,8 +158,7 @@ typedef struct _osm_node * * SYNOPSIS */ -void osm_node_delete( - IN OUT osm_node_t** const p_node ); +void osm_node_delete(IN OUT osm_node_t ** const p_node); /* * PARAMETERS * p_node @@ -187,9 +185,7 @@ void osm_node_delete( * * SYNOPSIS */ -osm_node_t* -osm_node_new( - IN const osm_madw_t* const p_madw ); +osm_node_t *osm_node_new(IN const osm_madw_t * const p_madw); /* * PARAMETERS * p_madw @@ -217,15 +213,15 @@ osm_node_new( * * SYNOPSIS */ -static inline osm_physp_t* -osm_node_get_physp_ptr( - IN const osm_node_t* const p_node, - IN const uint32_t port_num ) +static inline osm_physp_t *osm_node_get_physp_ptr(IN const osm_node_t * + const p_node, + IN const uint32_t port_num) { - CL_ASSERT( port_num < p_node->physp_tbl_size ); - return( (osm_physp_t*)&p_node->physp_table[port_num] ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + return ((osm_physp_t *) & p_node->physp_table[port_num]); } + /* * PARAMETERS * p_node @@ -256,14 +252,15 @@ osm_node_get_physp_ptr( * * SYNOPSIS */ -static inline osm_physp_t* -osm_node_get_any_physp_ptr( - IN const osm_node_t* const p_node ) +static inline osm_physp_t *osm_node_get_any_physp_ptr(IN const osm_node_t * + const p_node) { - CL_ASSERT( p_node ); - return( (osm_physp_t*)&p_node->physp_table[ - ib_node_info_get_local_port_num( &p_node->node_info )] ); + CL_ASSERT(p_node); + return ((osm_physp_t *) & p_node-> + physp_table[ib_node_info_get_local_port_num + (&p_node->node_info)]); } + /* * PARAMETERS * p_node @@ -290,14 +287,16 @@ osm_node_get_any_physp_ptr( * * SYNOPSIS */ -static inline osm_dr_path_t* -osm_node_get_any_dr_path_ptr( - IN const osm_node_t* const p_node ) +static inline osm_dr_path_t *osm_node_get_any_dr_path_ptr(IN const osm_node_t * + const p_node) { - CL_ASSERT( p_node ); - return( osm_physp_get_dr_path_ptr( &p_node->physp_table[ - ib_node_info_get_local_port_num( &p_node->node_info )] ) ); + CL_ASSERT(p_node); + return (osm_physp_get_dr_path_ptr + (&p_node-> + physp_table[ib_node_info_get_local_port_num + (&p_node->node_info)])); } + /* * PARAMETERS * p_node @@ -326,12 +325,11 @@ osm_node_get_any_dr_path_ptr( * * SYNOPSIS */ -static inline uint8_t -osm_node_get_type( - IN const osm_node_t* const p_node ) +static inline uint8_t osm_node_get_type(IN const osm_node_t * const p_node) { - return( p_node->node_info.node_type ); + return (p_node->node_info.node_type); } + /* * PARAMETERS * p_node @@ -355,12 +353,11 @@ osm_node_get_type( * * SYNOPSIS */ -static inline uint8_t -osm_node_get_num_physp( - IN const osm_node_t* const p_node ) +static inline uint8_t osm_node_get_num_physp(IN const osm_node_t * const p_node) { - return( (uint8_t)p_node->physp_tbl_size ); + return ((uint8_t) p_node->physp_tbl_size); } + /* * PARAMETERS * p_node @@ -386,11 +383,9 @@ osm_node_get_num_physp( * * SYNOPSIS */ -osm_node_t* -osm_node_get_remote_node( - IN const osm_node_t* const p_node, - IN const uint8_t port_num, - OUT uint8_t *p_remote_port_num ); +osm_node_t *osm_node_get_remote_node(IN const osm_node_t * const p_node, + IN const uint8_t port_num, + OUT uint8_t * p_remote_port_num); /* * PARAMETERS * p_node @@ -425,13 +420,13 @@ osm_node_get_remote_node( * SYNOPSIS */ static inline ib_net16_t -osm_node_get_base_lid( - IN const osm_node_t* const p_node, - IN const uint32_t port_num ) +osm_node_get_base_lid(IN const osm_node_t * const p_node, + IN const uint32_t port_num) { - CL_ASSERT( port_num < p_node->physp_tbl_size ); - return( osm_physp_get_base_lid( &p_node->physp_table[port_num] ) ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + return (osm_physp_get_base_lid(&p_node->physp_table[port_num])); } + /* * PARAMETERS * p_node @@ -462,9 +457,8 @@ osm_node_get_base_lid( * SYNOPSIS */ ib_net16_t -osm_node_get_remote_base_lid( - IN const osm_node_t* const p_node, - IN const uint32_t port_num ); +osm_node_get_remote_base_lid(IN const osm_node_t * const p_node, + IN const uint32_t port_num); /* * PARAMETERS * p_node @@ -496,16 +490,16 @@ osm_node_get_remote_base_lid( * SYNOPSIS */ static inline uint8_t -osm_node_get_remote_type( - IN const osm_node_t* const p_node, - IN const uint8_t port_num ) +osm_node_get_remote_type(IN const osm_node_t * const p_node, + IN const uint8_t port_num) { osm_node_t *p_remote_node; - p_remote_node = osm_node_get_remote_node( p_node, port_num, NULL ); - CL_ASSERT( p_remote_node ); - return( osm_node_get_type( p_remote_node ) ); + p_remote_node = osm_node_get_remote_node(p_node, port_num, NULL); + CL_ASSERT(p_remote_node); + return (osm_node_get_type(p_remote_node)); } + /* * PARAMETERS * p_node @@ -534,13 +528,12 @@ osm_node_get_remote_type( * SYNOPSIS */ static inline uint8_t -osm_node_get_lmc( - IN const osm_node_t* const p_node, - IN const uint32_t port_num ) +osm_node_get_lmc(IN const osm_node_t * const p_node, IN const uint32_t port_num) { - CL_ASSERT( port_num < p_node->physp_tbl_size ); - return( osm_physp_get_lmc( &p_node->physp_table[port_num] ) ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + return (osm_physp_get_lmc(&p_node->physp_table[port_num])); } + /* * PARAMETERS * p_node @@ -568,9 +561,8 @@ osm_node_get_lmc( * SYNOPSIS */ void -osm_node_init_physp( - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ); +osm_node_init_physp(IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw); /* * PARAMETERS * p_node @@ -601,11 +593,11 @@ osm_node_init_physp( * SYNOPSIS */ static inline ib_net64_t -osm_node_get_node_guid( - IN const osm_node_t* const p_node ) +osm_node_get_node_guid(IN const osm_node_t * const p_node) { - return( p_node->node_info.node_guid ); + return (p_node->node_info.node_guid); } + /* * PARAMETERS * p_node @@ -630,11 +622,10 @@ osm_node_get_node_guid( * SYNOPSIS */ void -osm_node_link( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ); +osm_node_link(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num); /* * PARAMETERS * p_node @@ -670,11 +661,10 @@ osm_node_link( * SYNOPSIS */ void -osm_node_unlink( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ); +osm_node_unlink(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num); /* * PARAMETERS * p_node @@ -710,11 +700,10 @@ osm_node_unlink( * SYNOPSIS */ boolean_t -osm_node_link_exists( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ); +osm_node_link_exists(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num); /* * PARAMETERS * p_node @@ -753,9 +742,7 @@ osm_node_link_exists( * SYNOPSIS */ boolean_t -osm_node_has_any_link( - IN osm_node_t* const p_node, - IN const uint8_t port_num ); +osm_node_has_any_link(IN osm_node_t * const p_node, IN const uint8_t port_num); /* * PARAMETERS * p_node @@ -786,11 +773,10 @@ osm_node_has_any_link( * SYNOPSIS */ boolean_t -osm_node_link_has_valid_ports( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ); +osm_node_link_has_valid_ports(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num); /* * PARAMETERS * p_node @@ -810,5 +796,4 @@ osm_node_link_has_valid_ports( *********/ END_C_DECLS - -#endif /* _OSM_NODE_H_ */ +#endif /* _OSM_NODE_H_ */ diff --git a/opensm/include/opensm/osm_node_desc_rcv.h b/opensm/include/opensm/osm_node_desc_rcv.h index b228d58..fc336d7 100644 --- a/opensm/include/opensm/osm_node_desc_rcv.h +++ b/opensm/include/opensm/osm_node_desc_rcv.h @@ -58,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Node Description Receiver * NAME * Node Description Receiver @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Node Description Receiver/osm_nd_rcv_t * NAME * osm_nd_rcv_t @@ -95,11 +93,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_nd_rcv -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_nd_rcv { + osm_subn_t *p_subn; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_nd_rcv_t; /* * FIELDS @@ -125,8 +122,7 @@ typedef struct _osm_nd_rcv * * SYNOPSIS */ -void osm_nd_rcv_construct( - IN osm_nd_rcv_t* const p_rcv ); +void osm_nd_rcv_construct(IN osm_nd_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -156,8 +152,7 @@ void osm_nd_rcv_construct( * * SYNOPSIS */ -void osm_nd_rcv_destroy( - IN osm_nd_rcv_t* const p_rcv ); +void osm_nd_rcv_destroy(IN osm_nd_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -188,11 +183,10 @@ void osm_nd_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_nd_rcv_init( - IN osm_nd_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_nd_rcv_init(IN osm_nd_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -228,9 +222,7 @@ ib_api_status_t osm_nd_rcv_init( * * SYNOPSIS */ -void osm_nd_rcv_process( - IN void *context, - IN void *data ); +void osm_nd_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -251,5 +243,4 @@ void osm_nd_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_ND_RCV_H_ */ +#endif /* _OSM_ND_RCV_H_ */ diff --git a/opensm/include/opensm/osm_node_info_rcv.h b/opensm/include/opensm/osm_node_info_rcv.h index 790b57b..31b89ac 100644 --- a/opensm/include/opensm/osm_node_info_rcv.h +++ b/opensm/include/opensm/osm_node_info_rcv.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Node Info Receiver * NAME * Node Info Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Node Info Receiver/osm_ni_rcv_t * NAME * osm_ni_rcv_t @@ -98,13 +96,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ni_rcv -{ - osm_subn_t *p_subn; - osm_req_t *p_gen_req; - osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; - cl_plock_t *p_lock; +typedef struct _osm_ni_rcv { + osm_subn_t *p_subn; + osm_req_t *p_gen_req; + osm_log_t *p_log; + osm_state_mgr_t *p_state_mgr; + cl_plock_t *p_lock; } osm_ni_rcv_t; /* * FIELDS @@ -136,8 +133,7 @@ typedef struct _osm_ni_rcv * * SYNOPSIS */ -void osm_ni_rcv_construct( - IN osm_ni_rcv_t* const p_ctrl ); +void osm_ni_rcv_construct(IN osm_ni_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -168,8 +164,7 @@ void osm_ni_rcv_construct( * * SYNOPSIS */ -void osm_ni_rcv_destroy( - IN osm_ni_rcv_t* const p_ctrl ); +void osm_ni_rcv_destroy(IN osm_ni_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -200,13 +195,12 @@ void osm_ni_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_ni_rcv_init( - IN osm_ni_rcv_t* const p_ctrl, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_ni_rcv_init(IN osm_ni_rcv_t * const p_ctrl, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -248,8 +242,7 @@ ib_api_status_t osm_ni_rcv_init( * * SYNOPSIS */ -boolean_t osm_ni_rcv_is_inited( - IN const osm_ni_rcv_t* const p_ctrl ); +boolean_t osm_ni_rcv_is_inited(IN const osm_ni_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -277,9 +270,7 @@ boolean_t osm_ni_rcv_is_inited( * * SYNOPSIS */ -void osm_ni_rcv_process( - IN void *context, - IN void *data ); +void osm_ni_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -300,5 +291,4 @@ void osm_ni_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_NI_RCV_H_ */ +#endif /* _OSM_NI_RCV_H_ */ diff --git a/opensm/include/opensm/osm_opensm.h b/opensm/include/opensm/osm_opensm.h index 2d668e9..850e51a 100644 --- a/opensm/include/opensm/osm_opensm.h +++ b/opensm/include/opensm/osm_opensm.h @@ -67,13 +67,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/OpenSM * NAME * OpenSM @@ -91,7 +90,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: OpenSM/osm_routing_engine * NAME * struct osm_routing_engine @@ -105,10 +103,10 @@ BEGIN_C_DECLS struct osm_routing_engine { const char *name; void *context; - int (*build_lid_matrices)(void *context); - int (*ucast_build_fwd_tables)(void *context); - void (*ucast_dump_tables)(void *context); - void (*delete)(void *context); + int (*build_lid_matrices) (void *context); + int (*ucast_build_fwd_tables) (void *context); + void (*ucast_dump_tables) (void *context); + void (*delete) (void *context); }; /* * FIELDS @@ -133,13 +131,12 @@ struct osm_routing_engine { * internals cleanup. */ -typedef struct _osm_console_t -{ - int socket; - int in_fd; - int out_fd; - FILE *in; - FILE *out; +typedef struct _osm_console_t { + int socket; + int in_fd; + int out_fd; + FILE *in; + FILE *out; } osm_console_t; /****s* OpenSM: OpenSM/osm_opensm_t @@ -154,25 +151,24 @@ typedef struct _osm_console_t * * SYNOPSIS */ -typedef struct _osm_opensm_t -{ - osm_subn_t subn; - osm_sm_t sm; - osm_sa_t sa; +typedef struct _osm_opensm_t { + osm_subn_t subn; + osm_sm_t sm; + osm_sa_t sa; #ifdef ENABLE_OSM_PERF_MGR - osm_perfmgr_t perfmgr; -#endif /* ENABLE_OSM_PERF_MGR */ - osm_epi_plugin_t *event_plugin; - osm_db_t db; - osm_mad_pool_t mad_pool; - osm_vendor_t *p_vendor; - osm_vl15_t vl15; - osm_log_t log; - cl_dispatcher_t disp; - cl_plock_t lock; - struct osm_routing_engine routing_engine; - osm_stats_t stats; - osm_console_t console; + osm_perfmgr_t perfmgr; +#endif /* ENABLE_OSM_PERF_MGR */ + osm_epi_plugin_t *event_plugin; + osm_db_t db; + osm_mad_pool_t mad_pool; + osm_vendor_t *p_vendor; + osm_vl15_t vl15; + osm_log_t log; + cl_dispatcher_t disp; + cl_plock_t lock; + struct osm_routing_engine routing_engine; + osm_stats_t stats; + osm_console_t console; } osm_opensm_t; /* * FIELDS @@ -226,8 +222,7 @@ typedef struct _osm_opensm_t * * SYNOPSIS */ -void osm_opensm_construct( - IN osm_opensm_t* const p_osm ); +void osm_opensm_construct(IN osm_opensm_t * const p_osm); /* * PARAMETERS * p_osm @@ -256,8 +251,7 @@ void osm_opensm_construct( * * SYNOPSIS */ -void osm_opensm_destroy( - IN osm_opensm_t* const p_osm ); +void osm_opensm_destroy(IN osm_opensm_t * const p_osm); /* * PARAMETERS * p_osm @@ -286,9 +280,8 @@ void osm_opensm_destroy( * SYNOPSIS */ ib_api_status_t -osm_opensm_init( - IN osm_opensm_t* const p_osm, - IN const osm_subn_opt_t* const p_opt ); +osm_opensm_init(IN osm_opensm_t * const p_osm, + IN const osm_subn_opt_t * const p_opt); /* * PARAMETERS * p_osm @@ -316,12 +309,11 @@ osm_opensm_init( * * SYNOPSIS */ -static inline void -osm_opensm_sweep( - IN osm_opensm_t* const p_osm ) +static inline void osm_opensm_sweep(IN osm_opensm_t * const p_osm) { - osm_sm_sweep( &p_osm->sm ); + osm_sm_sweep(&p_osm->sm); } + /* * PARAMETERS * p_osm @@ -348,12 +340,12 @@ osm_opensm_sweep( * SYNOPSIS */ static inline void -osm_opensm_set_log_flags( - IN osm_opensm_t* const p_osm, - IN const osm_log_level_t log_flags ) +osm_opensm_set_log_flags(IN osm_opensm_t * const p_osm, + IN const osm_log_level_t log_flags) { - osm_log_set_level( &p_osm->log, log_flags ); + osm_log_set_level(&p_osm->log, log_flags); } + /* * PARAMETERS * p_osm @@ -380,9 +372,7 @@ osm_opensm_set_log_flags( * SYNOPSIS */ ib_api_status_t -osm_opensm_bind( - IN osm_opensm_t* const p_osm, - IN const ib_net64_t guid ); +osm_opensm_bind(IN osm_opensm_t * const p_osm, IN const ib_net64_t guid); /* * PARAMETERS * p_osm @@ -410,13 +400,13 @@ osm_opensm_bind( * SYNOPSIS */ static inline cl_status_t -osm_opensm_wait_for_subnet_up( - IN osm_opensm_t* const p_osm, - IN uint32_t const wait_us, - IN boolean_t const interruptible ) +osm_opensm_wait_for_subnet_up(IN osm_opensm_t * const p_osm, + IN uint32_t const wait_us, + IN boolean_t const interruptible) { - return( osm_sm_wait_for_subnet_up( &p_osm->sm, wait_us, interruptible ) ); + return (osm_sm_wait_for_subnet_up(&p_osm->sm, wait_us, interruptible)); } + /* * PARAMETERS * p_osm @@ -445,8 +435,8 @@ osm_opensm_wait_for_subnet_up( *********/ /* dump helpers */ -void osm_dump_mcast_routes(osm_opensm_t *osm); -void osm_dump_all(osm_opensm_t *osm); +void osm_dump_mcast_routes(osm_opensm_t * osm); +void osm_dump_all(osm_opensm_t * osm); /****v* OpenSM/osm_exit_flag */ @@ -457,5 +447,4 @@ extern volatile unsigned int osm_exit_flag; *********/ END_C_DECLS - -#endif /* _OSM_OPENSM_H_ */ +#endif /* _OSM_OPENSM_H_ */ diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h index f806719..f00034f 100644 --- a/opensm/include/opensm/osm_partition.h +++ b/opensm/include/opensm/osm_partition.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Partition * NAME * Partition @@ -83,7 +82,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Partition/osm_prtn_t * NAME * osm_prtn_t @@ -96,14 +94,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_prtn -{ - cl_map_item_t map_item; - uint16_t pkey; - uint8_t sl; - cl_map_t full_guid_tbl; - cl_map_t part_guid_tbl; - char name[32]; +typedef struct _osm_prtn { + cl_map_item_t map_item; + uint16_t pkey; + uint8_t sl; + cl_map_t full_guid_tbl; + cl_map_t part_guid_tbl; + char name[32]; } osm_prtn_t; /* * FIELDS @@ -141,8 +138,7 @@ typedef struct _osm_prtn * * SYNOPSIS */ -void osm_prtn_delete( - IN OUT osm_prtn_t** const pp_prtn ); +void osm_prtn_delete(IN OUT osm_prtn_t ** const pp_prtn); /* * PARAMETERS * pp_prtn @@ -168,9 +164,7 @@ void osm_prtn_delete( * * SYNOPSIS */ -osm_prtn_t* osm_prtn_new( - IN const char *name, - IN const uint16_t pkey ); +osm_prtn_t *osm_prtn_new(IN const char *name, IN const uint16_t pkey); /* * PARAMETERS * name @@ -198,14 +192,13 @@ osm_prtn_t* osm_prtn_new( * * SYNOPSIS */ -static inline -boolean_t osm_prtn_is_guid( - IN const osm_prtn_t* const p_prtn, - IN const ib_net64_t guid ) +static inline boolean_t osm_prtn_is_guid(IN const osm_prtn_t * const p_prtn, + IN const ib_net64_t guid) { return (cl_map_get(&p_prtn->full_guid_tbl, guid) != NULL) || - (cl_map_get(&p_prtn->part_guid_tbl, guid) != NULL); + (cl_map_get(&p_prtn->part_guid_tbl, guid) != NULL); } + /* * PARAMETERS * p_prtn @@ -232,9 +225,8 @@ boolean_t osm_prtn_is_guid( * * SYNOPSIS */ -ib_api_status_t osm_prtn_make_partitions( - IN osm_log_t * const p_log, - IN osm_subn_t * const p_subn); +ib_api_status_t osm_prtn_make_partitions(IN osm_log_t * const p_log, + IN osm_subn_t * const p_subn); /* * PARAMETERS * p_log @@ -252,5 +244,4 @@ ib_api_status_t osm_prtn_make_partitions( *********/ END_C_DECLS - -#endif /* _OSM_PARTITION_H_ */ +#endif /* _OSM_PARTITION_H_ */ diff --git a/opensm/include/opensm/osm_path.h b/opensm/include/opensm/osm_path.h index fcc31d9..e26036c 100644 --- a/opensm/include/opensm/osm_path.h +++ b/opensm/include/opensm/osm_path.h @@ -43,13 +43,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* * Abstract: * Declaration of path related objects. @@ -60,7 +59,6 @@ BEGIN_C_DECLS * * $Revision: 1.4 $ */ - /****h* OpenSM/DR Path * NAME * DR Path @@ -74,7 +72,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: DR Path/osm_dr_path_t * NAME * osm_dr_path_t @@ -86,11 +83,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_dr_path -{ - osm_bind_handle_t h_bind; - uint8_t hop_count; - uint8_t path[IB_SUBNET_PATH_HOPS_MAX]; +typedef struct _osm_dr_path { + osm_bind_handle_t h_bind; + uint8_t hop_count; + uint8_t path[IB_SUBNET_PATH_HOPS_MAX]; } osm_dr_path_t; /* @@ -116,12 +112,10 @@ typedef struct _osm_dr_path * * SYNOPSIS */ -static inline void -osm_dr_path_construct( - IN osm_dr_path_t* const p_path ) +static inline void osm_dr_path_construct(IN osm_dr_path_t * const p_path) { /* The first location in the path array is reserved. */ - memset( p_path, 0, sizeof(*p_path) ); + memset(p_path, 0, sizeof(*p_path)); p_path->h_bind = OSM_BIND_INVALID_HANDLE; } @@ -157,18 +151,17 @@ osm_dr_path_construct( * SYNOPSIS */ static inline void -osm_dr_path_init( - IN osm_dr_path_t* const p_path, - IN const osm_bind_handle_t h_bind, - IN const uint8_t hop_count, - IN const uint8_t path[IB_SUBNET_PATH_HOPS_MAX] ) +osm_dr_path_init(IN osm_dr_path_t * const p_path, + IN const osm_bind_handle_t h_bind, + IN const uint8_t hop_count, + IN const uint8_t path[IB_SUBNET_PATH_HOPS_MAX]) { /* The first location in the path array is reserved. */ - CL_ASSERT( path[0] == 0 ); - CL_ASSERT( hop_count < IB_SUBNET_PATH_HOPS_MAX ); + CL_ASSERT(path[0] == 0); + CL_ASSERT(hop_count < IB_SUBNET_PATH_HOPS_MAX); p_path->h_bind = h_bind; p_path->hop_count = hop_count; - memcpy( p_path->path, path, IB_SUBNET_PATH_HOPS_MAX ); + memcpy(p_path->path, path, IB_SUBNET_PATH_HOPS_MAX); } /* @@ -202,15 +195,13 @@ osm_dr_path_init( * SYNOPSIS */ static inline void -osm_dr_path_extend( - IN osm_dr_path_t* const p_path, - IN const uint8_t port_num ) +osm_dr_path_extend(IN osm_dr_path_t * const p_path, IN const uint8_t port_num) { p_path->hop_count++; - CL_ASSERT( p_path->hop_count < IB_SUBNET_PATH_HOPS_MAX ); + CL_ASSERT(p_path->hop_count < IB_SUBNET_PATH_HOPS_MAX); /* - Location 0 in the path array is reserved per IB spec. - */ + Location 0 in the path array is reserved per IB spec. + */ p_path->path[p_path->hop_count] = port_num; } @@ -240,10 +231,9 @@ osm_dr_path_extend( * SYNOPSIS */ static inline osm_bind_handle_t -osm_dr_path_get_bind_handle( - IN const osm_dr_path_t* const p_path ) +osm_dr_path_get_bind_handle(IN const osm_dr_path_t * const p_path) { - return( p_path->h_bind ); + return (p_path->h_bind); } /* @@ -263,5 +253,4 @@ osm_dr_path_get_bind_handle( *********/ END_C_DECLS - -#endif /* _OSM_PATH_H_ */ +#endif /* _OSM_PATH_H_ */ diff --git a/opensm/include/opensm/osm_perfmgr.h b/opensm/include/opensm/osm_perfmgr.h index 4c26d1c..6f0971b 100644 --- a/opensm/include/opensm/osm_perfmgr.h +++ b/opensm/include/opensm/osm_perfmgr.h @@ -37,7 +37,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #ifdef ENABLE_OSM_PERF_MGR @@ -55,7 +55,7 @@ #ifdef __cplusplus extern "C" { -#endif /* __cplusplus */ +#endif /* __cplusplus */ /****h* OpenSM/PerfMgr * NAME @@ -76,68 +76,64 @@ extern "C" { #define OSM_PERFMGR_DEFAULT_DUMP_FILE OSM_DEFAULT_TMP_DIR "/opensm_port_counters.log" #define OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES 500 - /****s* OpenSM: PerfMgr/osm_perfmgr_state_t */ -typedef enum -{ - PERFMGR_STATE_DISABLE, - PERFMGR_STATE_ENABLED, - PERFMGR_STATE_NO_DB +typedef enum { + PERFMGR_STATE_DISABLE, + PERFMGR_STATE_ENABLED, + PERFMGR_STATE_NO_DB } osm_perfmgr_state_t; /****s* OpenSM: PerfMgr/osm_perfmgr_sweep_state_t */ -typedef enum -{ - PERFMGR_SWEEP_SLEEP, - PERFMGR_SWEEP_ACTIVE, - PERFMGR_SWEEP_SUSPENDED +typedef enum { + PERFMGR_SWEEP_SLEEP, + PERFMGR_SWEEP_ACTIVE, + PERFMGR_SWEEP_SUSPENDED } osm_perfmgr_sweep_state_t; /* Redirection information */ typedef struct redir { - ib_net16_t redir_lid; - ib_net32_t redir_qp; + ib_net16_t redir_lid; + ib_net32_t redir_qp; } redir_t; /* Node to store information about which nodes we are monitoring */ typedef struct _monitored_node { - cl_map_item_t map_item; + cl_map_item_t map_item; struct _monitored_node *next; - uint64_t guid; - uint32_t redir_tbl_size; - redir_t redir_port[1]; /* redirection on a per port basis */ + uint64_t guid; + uint32_t redir_tbl_size; + redir_t redir_port[1]; /* redirection on a per port basis */ } __monitored_node_t; /****s* OpenSM: PerfMgr/osm_perfmgr_t * This object should be treated as opaque and should * be manipulated only through the provided functions. */ -typedef struct _osm_perfmgr -{ - osm_thread_state_t thread_state; - cl_event_t sig_sweep; - cl_thread_t sweeper; - osm_subn_t *subn; - osm_sm_t *sm; - cl_plock_t *lock; - osm_log_t *log; - osm_mad_pool_t *mad_pool; - atomic32_t trans_id; - osm_vendor_t *vendor; - osm_bind_handle_t bind_handle; - cl_disp_reg_handle_t pc_disp_h; - osm_perfmgr_state_t state; - osm_perfmgr_sweep_state_t sweep_state; - uint16_t sweep_time_s; - char *db_file; - char *event_db_dump_file; - perfmgr_db_t *db; - atomic32_t outstanding_queries; /* this along with sig_query */ - cl_event_t sig_query; /* will throttle our querys */ - uint32_t max_outstanding_queries; - cl_qmap_t monitored_map; /* map the nodes we are tracking */ - __monitored_node_t *remove_list; - osm_epi_plugin_t *event_plugin; +typedef struct _osm_perfmgr { + osm_thread_state_t thread_state; + cl_event_t sig_sweep; + cl_thread_t sweeper; + osm_subn_t *subn; + osm_sm_t *sm; + cl_plock_t *lock; + osm_log_t *log; + osm_mad_pool_t *mad_pool; + atomic32_t trans_id; + osm_vendor_t *vendor; + osm_bind_handle_t bind_handle; + cl_disp_reg_handle_t pc_disp_h; + osm_perfmgr_state_t state; + osm_perfmgr_sweep_state_t sweep_state; + uint16_t sweep_time_s; + char *db_file; + char *event_db_dump_file; + perfmgr_db_t *db; + atomic32_t outstanding_queries; /* this along with sig_query */ + cl_event_t sig_query; /* will throttle our querys */ + uint32_t max_outstanding_queries; + cl_qmap_t monitored_map; /* map the nodes we are tracking */ + __monitored_node_t *remove_list; + osm_epi_plugin_t *event_plugin; } osm_perfmgr_t; /* * FIELDS @@ -158,78 +154,86 @@ typedef struct _osm_perfmgr *********/ /****f* OpenSM: Creation Functions */ -void osm_perfmgr_shutdown(osm_perfmgr_t *const p_perfmgr ); -void osm_perfmgr_destroy(osm_perfmgr_t * const p_perfmgr ); +void osm_perfmgr_shutdown(osm_perfmgr_t * const p_perfmgr); +void osm_perfmgr_destroy(osm_perfmgr_t * const p_perfmgr); /****f* OpenSM: Inline accessor functions */ -inline static void osm_perfmgr_set_state(osm_perfmgr_t *p_perfmgr, +inline static void osm_perfmgr_set_state(osm_perfmgr_t * p_perfmgr, osm_perfmgr_state_t state) { p_perfmgr->state = state; if (state == PERFMGR_STATE_ENABLED) cl_event_signal(&p_perfmgr->sig_sweep); } + inline static osm_perfmgr_state_t osm_perfmgr_get_state(osm_perfmgr_t - *p_perfmgr) { return (p_perfmgr->state); } -inline static char *osm_perfmgr_get_state_str(osm_perfmgr_t *p_perfmgr) + * p_perfmgr) +{ + return (p_perfmgr->state); +} + +inline static char *osm_perfmgr_get_state_str(osm_perfmgr_t * p_perfmgr) { - switch (p_perfmgr->state) - { - case PERFMGR_STATE_DISABLE: - return ("Disabled"); - break; - case PERFMGR_STATE_ENABLED: - return ("Enabled"); - break; - case PERFMGR_STATE_NO_DB: - return ("No Database"); - break; + switch (p_perfmgr->state) { + case PERFMGR_STATE_DISABLE: + return ("Disabled"); + break; + case PERFMGR_STATE_ENABLED: + return ("Enabled"); + break; + case PERFMGR_STATE_NO_DB: + return ("No Database"); + break; } return ("UNKNOWN"); } -inline static char *osm_perfmgr_get_sweep_state_str(osm_perfmgr_t *perfmgr) + +inline static char *osm_perfmgr_get_sweep_state_str(osm_perfmgr_t * perfmgr) { - switch (perfmgr->sweep_state) - { - case PERFMGR_SWEEP_SLEEP: - return ("Sleeping"); - break; - case PERFMGR_SWEEP_ACTIVE: - return ("Active"); - break; - case PERFMGR_SWEEP_SUSPENDED: - return ("Suspended"); - break; + switch (perfmgr->sweep_state) { + case PERFMGR_SWEEP_SLEEP: + return ("Sleeping"); + break; + case PERFMGR_SWEEP_ACTIVE: + return ("Active"); + break; + case PERFMGR_SWEEP_SUSPENDED: + return ("Suspended"); + break; } return ("UNKNOWN"); } -inline static void osm_perfmgr_set_sweep_time_s(osm_perfmgr_t *p_perfmgr, uint16_t time_s) + +inline static void osm_perfmgr_set_sweep_time_s(osm_perfmgr_t * p_perfmgr, + uint16_t time_s) { p_perfmgr->sweep_time_s = time_s; cl_event_signal(&p_perfmgr->sig_sweep); } -inline static uint16_t osm_perfmgr_get_sweep_time_s(osm_perfmgr_t *p_perfmgr) + +inline static uint16_t osm_perfmgr_get_sweep_time_s(osm_perfmgr_t * p_perfmgr) { return (p_perfmgr->sweep_time_s); } -void osm_perfmgr_clear_counters(osm_perfmgr_t *p_perfmgr); -void osm_perfmgr_dump_counters(osm_perfmgr_t *p_perfmgr, perfmgr_db_dump_t dump_type); -ib_api_status_t osm_perfmgr_bind(osm_perfmgr_t * const p_perfmgr, const ib_net64_t port_guid); +void osm_perfmgr_clear_counters(osm_perfmgr_t * p_perfmgr); +void osm_perfmgr_dump_counters(osm_perfmgr_t * p_perfmgr, + perfmgr_db_dump_t dump_type); + +ib_api_status_t osm_perfmgr_bind(osm_perfmgr_t * const p_perfmgr, + const ib_net64_t port_guid); /****f* OpenSM: PerfMgr/osm_perfmgr_init */ -ib_api_status_t -osm_perfmgr_init( - osm_perfmgr_t* const perfmgr, - osm_subn_t* const subn, - osm_sm_t * const sm, - osm_log_t* const log, - osm_mad_pool_t * const mad_pool, - osm_vendor_t * const vendor, - cl_dispatcher_t* const disp, - cl_plock_t* const lock, - const osm_subn_opt_t * const p_opt, - osm_epi_plugin_t *event_plugin); +ib_api_status_t osm_perfmgr_init(osm_perfmgr_t * const perfmgr, + osm_subn_t * const subn, + osm_sm_t * const sm, + osm_log_t * const log, + osm_mad_pool_t * const mad_pool, + osm_vendor_t * const vendor, + cl_dispatcher_t * const disp, + cl_plock_t * const lock, + const osm_subn_opt_t * const p_opt, + osm_epi_plugin_t * event_plugin); /* * PARAMETERS * perfmgr @@ -268,8 +272,8 @@ osm_perfmgr_init( #ifdef __cplusplus } -#endif /* __cplusplus */ +#endif /* __cplusplus */ -#endif /* ENABLE_OSM_PERF_MGR */ +#endif /* ENABLE_OSM_PERF_MGR */ -#endif /* _OSM_PERFMGR_H_ */ +#endif /* _OSM_PERFMGR_H_ */ diff --git a/opensm/include/opensm/osm_perfmgr_db.h b/opensm/include/opensm/osm_perfmgr_db.h index 19ebd72..0991102 100644 --- a/opensm/include/opensm/osm_perfmgr_db.h +++ b/opensm/include/opensm/osm_perfmgr_db.h @@ -44,13 +44,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/PerfMgr Event Database * DESCRIPTION * Database interface to record subnet events @@ -61,7 +60,6 @@ BEGIN_C_DECLS * Ira Weiny, LLNL * *********/ - typedef enum { PERFMGR_EVENT_DB_SUCCESS = 0, PERFMGR_EVENT_DB_FAIL, @@ -75,50 +73,50 @@ typedef enum { * Port error reading */ typedef struct { - uint64_t symbol_err_cnt; - uint64_t link_err_recover; - uint64_t link_downed; - uint64_t rcv_err; - uint64_t rcv_rem_phys_err; - uint64_t rcv_switch_relay_err; - uint64_t xmit_discards; - uint64_t xmit_constraint_err; - uint64_t rcv_constraint_err; - uint64_t link_integrity; - uint64_t buffer_overrun; - uint64_t vl15_dropped; - time_t time; + uint64_t symbol_err_cnt; + uint64_t link_err_recover; + uint64_t link_downed; + uint64_t rcv_err; + uint64_t rcv_rem_phys_err; + uint64_t rcv_switch_relay_err; + uint64_t xmit_discards; + uint64_t xmit_constraint_err; + uint64_t rcv_constraint_err; + uint64_t link_integrity; + uint64_t buffer_overrun; + uint64_t vl15_dropped; + time_t time; } perfmgr_db_err_reading_t; /** ========================================================================= * Port data count reading */ typedef struct { - uint64_t xmit_data; /* can be used for std or extended */ - uint64_t rcv_data; /* can be used for std or extended */ - uint64_t xmit_pkts; /* can be used for std or extended */ - uint64_t rcv_pkts; /* can be used for std or extended */ - uint64_t unicast_xmit_pkts; - uint64_t unicast_rcv_pkts; - uint64_t multicast_xmit_pkts; - uint64_t multicast_rcv_pkts; - time_t time; + uint64_t xmit_data; /* can be used for std or extended */ + uint64_t rcv_data; /* can be used for std or extended */ + uint64_t xmit_pkts; /* can be used for std or extended */ + uint64_t rcv_pkts; /* can be used for std or extended */ + uint64_t unicast_xmit_pkts; + uint64_t unicast_rcv_pkts; + uint64_t multicast_xmit_pkts; + uint64_t multicast_rcv_pkts; + time_t time; } perfmgr_db_data_cnt_reading_t; /** ========================================================================= * Port select errors */ typedef struct { - uint64_t xmit_wait; - time_t time; + uint64_t xmit_wait; + time_t time; } perfmgr_db_ps_reading_t; /** ========================================================================= * Dump output options */ typedef enum { - PERFMGR_EVENT_DB_DUMP_HR = 0, /* Human readable */ - PERFMGR_EVENT_DB_DUMP_MR /* Machine readable */ + PERFMGR_EVENT_DB_DUMP_HR = 0, /* Human readable */ + PERFMGR_EVENT_DB_DUMP_MR /* Machine readable */ } perfmgr_db_dump_t; /** ========================================================================= @@ -130,7 +128,7 @@ typedef struct _db_port { perfmgr_db_err_reading_t err_previous; perfmgr_db_data_cnt_reading_t dc_total; perfmgr_db_data_cnt_reading_t dc_previous; - time_t last_reset; + time_t last_reset; } _db_port_t; /** ========================================================================= @@ -138,66 +136,67 @@ typedef struct _db_port { */ #define NODE_NAME_SIZE (IB_NODE_DESCRIPTION_SIZE << 1) typedef struct _db_node { - cl_map_item_t map_item; /* must be first */ - uint64_t node_guid; - _db_port_t *ports; - uint8_t num_ports; - char node_name[NODE_NAME_SIZE]; + cl_map_item_t map_item; /* must be first */ + uint64_t node_guid; + _db_port_t *ports; + uint8_t num_ports; + char node_name[NODE_NAME_SIZE]; } _db_node_t; /** ========================================================================= * all nodes in the system. */ typedef struct _db { - cl_qmap_t pc_data; /* stores type (_db_node_t *) */ - cl_plock_t lock; - osm_log_t *osm_log; + cl_qmap_t pc_data; /* stores type (_db_node_t *) */ + cl_plock_t lock; + osm_log_t *osm_log; osm_epi_plugin_t *event_plugin; } perfmgr_db_t; - /** * functions */ -perfmgr_db_t *perfmgr_db_construct(osm_log_t *p_log, - osm_epi_plugin_t *event_plugin); -void perfmgr_db_destroy(perfmgr_db_t *db); - -perfmgr_db_err_t perfmgr_db_create_entry(perfmgr_db_t *db, uint64_t guid, - uint8_t num_ports, char *node_name); - -perfmgr_db_err_t perfmgr_db_add_err_reading(perfmgr_db_t *db, uint64_t guid, - uint8_t port, perfmgr_db_err_reading_t *reading); -perfmgr_db_err_t perfmgr_db_get_prev_err(perfmgr_db_t *db, - uint64_t guid, uint8_t port, - perfmgr_db_err_reading_t *reading); -perfmgr_db_err_t perfmgr_db_clear_prev_err(perfmgr_db_t *db, uint64_t guid, - uint8_t port); - -perfmgr_db_err_t perfmgr_db_add_dc_reading(perfmgr_db_t *db, uint64_t guid, - uint8_t port, perfmgr_db_data_cnt_reading_t *reading); -perfmgr_db_err_t perfmgr_db_get_prev_dc(perfmgr_db_t *db, - uint64_t guid, uint8_t port, - perfmgr_db_data_cnt_reading_t *reading); -perfmgr_db_err_t perfmgr_db_clear_prev_dc(perfmgr_db_t *db, uint64_t guid, - uint8_t port); - -void perfmgr_db_clear_counters(perfmgr_db_t *db); -perfmgr_db_err_t perfmgr_db_dump(perfmgr_db_t *db, char *file, - perfmgr_db_dump_t dump_type); +perfmgr_db_t *perfmgr_db_construct(osm_log_t * p_log, + osm_epi_plugin_t * event_plugin); +void perfmgr_db_destroy(perfmgr_db_t * db); + +perfmgr_db_err_t perfmgr_db_create_entry(perfmgr_db_t * db, uint64_t guid, + uint8_t num_ports, char *node_name); + +perfmgr_db_err_t perfmgr_db_add_err_reading(perfmgr_db_t * db, uint64_t guid, + uint8_t port, + perfmgr_db_err_reading_t * reading); +perfmgr_db_err_t perfmgr_db_get_prev_err(perfmgr_db_t * db, uint64_t guid, + uint8_t port, + perfmgr_db_err_reading_t * reading); +perfmgr_db_err_t perfmgr_db_clear_prev_err(perfmgr_db_t * db, uint64_t guid, + uint8_t port); + +perfmgr_db_err_t perfmgr_db_add_dc_reading(perfmgr_db_t * db, uint64_t guid, + uint8_t port, + perfmgr_db_data_cnt_reading_t * + reading); +perfmgr_db_err_t perfmgr_db_get_prev_dc(perfmgr_db_t * db, uint64_t guid, + uint8_t port, + perfmgr_db_data_cnt_reading_t * + reading); +perfmgr_db_err_t perfmgr_db_clear_prev_dc(perfmgr_db_t * db, uint64_t guid, + uint8_t port); + +void perfmgr_db_clear_counters(perfmgr_db_t * db); +perfmgr_db_err_t perfmgr_db_dump(perfmgr_db_t * db, char *file, + perfmgr_db_dump_t dump_type); /** ========================================================================= * helper functions to fill in the various db objects from wire objects */ -void perfmgr_db_fill_err_read(ib_port_counters_t *wire_read, - perfmgr_db_err_reading_t *reading); -void perfmgr_db_fill_data_cnt_read_pc(ib_port_counters_t *wire_read, - perfmgr_db_data_cnt_reading_t *reading); -void perfmgr_db_fill_data_cnt_read_epc(ib_port_counters_ext_t *wire_read, - perfmgr_db_data_cnt_reading_t *reading); +void perfmgr_db_fill_err_read(ib_port_counters_t * wire_read, + perfmgr_db_err_reading_t * reading); +void perfmgr_db_fill_data_cnt_read_pc(ib_port_counters_t * wire_read, + perfmgr_db_data_cnt_reading_t * reading); +void perfmgr_db_fill_data_cnt_read_epc(ib_port_counters_ext_t * wire_read, + perfmgr_db_data_cnt_reading_t * reading); END_C_DECLS - -#endif /* _PERFMGR_PM_DB_H_ */ - +#endif /* _PERFMGR_PM_DB_H_ */ diff --git a/opensm/include/opensm/osm_pkey.h b/opensm/include/opensm/osm_pkey.h index 42c620d..3c84e4b 100644 --- a/opensm/include/opensm/osm_pkey.h +++ b/opensm/include/opensm/osm_pkey.h @@ -46,13 +46,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* Forward references. */ @@ -86,14 +85,13 @@ struct _osm_physp; * * SYNOPSIS */ -typedef struct _osm_pkey_tbl -{ - cl_ptr_vector_t blocks; - cl_ptr_vector_t new_blocks; - cl_map_t keys; - cl_qlist_t pending; - uint16_t used_blocks; - uint16_t max_blocks; +typedef struct _osm_pkey_tbl { + cl_ptr_vector_t blocks; + cl_ptr_vector_t new_blocks; + cl_map_t keys; + cl_qlist_t pending; + uint16_t used_blocks; + uint16_t max_blocks; } osm_pkey_tbl_t; /* * FIELDS @@ -140,11 +138,11 @@ typedef struct _osm_pkey_tbl * SYNOPSIS */ typedef struct _osm_pending_pkey { - cl_list_item_t list_item; - uint16_t pkey; - uint16_t block; - uint8_t index; - boolean_t is_new; + cl_list_item_t list_item; + uint16_t pkey; + uint16_t block; + uint8_t index; + boolean_t is_new; } osm_pending_pkey_t; /* * FIELDS @@ -173,8 +171,7 @@ typedef struct _osm_pending_pkey { * * SYNOPSIS */ -void osm_pkey_tbl_construct( - IN osm_pkey_tbl_t *p_pkey_tbl); +void osm_pkey_tbl_construct(IN osm_pkey_tbl_t * p_pkey_tbl); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -192,9 +189,7 @@ void osm_pkey_tbl_construct( * * SYNOPSIS */ -ib_api_status_t -osm_pkey_tbl_init( - IN osm_pkey_tbl_t *p_pkey_tbl); +ib_api_status_t osm_pkey_tbl_init(IN osm_pkey_tbl_t * p_pkey_tbl); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -212,8 +207,7 @@ osm_pkey_tbl_init( * * SYNOPSIS */ -void osm_pkey_tbl_destroy( - IN osm_pkey_tbl_t *p_pkey_tbl); +void osm_pkey_tbl_destroy(IN osm_pkey_tbl_t * p_pkey_tbl); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -232,11 +226,11 @@ void osm_pkey_tbl_destroy( * SYNOPSIS */ static inline uint16_t -osm_pkey_tbl_get_num_blocks( - IN const osm_pkey_tbl_t *p_pkey_tbl ) +osm_pkey_tbl_get_num_blocks(IN const osm_pkey_tbl_t * p_pkey_tbl) { - return((uint16_t)(cl_ptr_vector_get_size( &p_pkey_tbl->blocks ))); + return ((uint16_t) (cl_ptr_vector_get_size(&p_pkey_tbl->blocks))); } + /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -257,12 +251,14 @@ osm_pkey_tbl_get_num_blocks( * * SYNOPSIS */ -static inline ib_pkey_table_t *osm_pkey_tbl_block_get( - const osm_pkey_tbl_t *p_pkey_tbl, uint16_t block) +static inline ib_pkey_table_t *osm_pkey_tbl_block_get(const osm_pkey_tbl_t * + p_pkey_tbl, + uint16_t block) { - return( (block < cl_ptr_vector_get_size(&p_pkey_tbl->blocks)) ? - cl_ptr_vector_get(&p_pkey_tbl->blocks, block) : NULL ); + return ((block < cl_ptr_vector_get_size(&p_pkey_tbl->blocks)) ? + cl_ptr_vector_get(&p_pkey_tbl->blocks, block) : NULL); }; + /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -286,11 +282,12 @@ static inline ib_pkey_table_t *osm_pkey_tbl_block_get( * * SYNOPSIS */ -static inline ib_pkey_table_t *osm_pkey_tbl_new_block_get( - const osm_pkey_tbl_t *p_pkey_tbl, uint16_t block) +static inline ib_pkey_table_t *osm_pkey_tbl_new_block_get(const osm_pkey_tbl_t * + p_pkey_tbl, + uint16_t block) { - return (block < cl_ptr_vector_get_size(&p_pkey_tbl->new_blocks)) ? - cl_ptr_vector_get(&p_pkey_tbl->new_blocks, block) : NULL; + return (block < cl_ptr_vector_get_size(&p_pkey_tbl->new_blocks)) ? + cl_ptr_vector_get(&p_pkey_tbl->new_blocks, block) : NULL; }; /****f* OpenSM: osm_pkey_tbl_set_new_entry @@ -304,11 +301,9 @@ static inline ib_pkey_table_t *osm_pkey_tbl_new_block_get( * SYNOPSIS */ ib_api_status_t -osm_pkey_tbl_set_new_entry( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t block_idx, - IN uint8_t pkey_idx, - IN uint16_t pkey); +osm_pkey_tbl_set_new_entry(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t block_idx, + IN uint8_t pkey_idx, IN uint16_t pkey); /* * p_pkey_tbl * [in] Pointer to the PKey table @@ -341,10 +336,9 @@ osm_pkey_tbl_set_new_entry( * SYNOPSIS */ boolean_t -osm_pkey_find_next_free_entry( - IN osm_pkey_tbl_t *p_pkey_tbl, - OUT uint16_t *p_block_idx, - OUT uint8_t *p_pkey_idx); +osm_pkey_find_next_free_entry(IN osm_pkey_tbl_t * p_pkey_tbl, + OUT uint16_t * p_block_idx, + OUT uint8_t * p_pkey_idx); /* * p_pkey_tbl * [in] Pointer to the PKey table @@ -370,8 +364,7 @@ osm_pkey_find_next_free_entry( * * SYNOPSIS */ -void osm_pkey_tbl_init_new_blocks( - const osm_pkey_tbl_t *p_pkey_tbl); +void osm_pkey_tbl_init_new_blocks(const osm_pkey_tbl_t * p_pkey_tbl); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -392,11 +385,10 @@ void osm_pkey_tbl_init_new_blocks( * SYNOPSIS */ ib_api_status_t -osm_pkey_tbl_get_block_and_idx( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t *p_pkey, - OUT uint16_t *block_idx, - OUT uint8_t *pkey_index); +osm_pkey_tbl_get_block_and_idx(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t * p_pkey, + OUT uint16_t * block_idx, + OUT uint8_t * pkey_index); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -424,10 +416,8 @@ osm_pkey_tbl_get_block_and_idx( * SYNOPSIS */ ib_api_status_t -osm_pkey_tbl_set( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t block, - IN ib_pkey_table_t *p_tbl); +osm_pkey_tbl_set(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t block, IN ib_pkey_table_t * p_tbl); /* * p_pkey_tbl * [in] Pointer to osm_pkey_tbl_t object. @@ -454,10 +444,9 @@ osm_pkey_tbl_set( * * SYNOPSIS */ -boolean_t osm_physp_share_this_pkey( - IN const struct _osm_physp * const p_physp1, - IN const struct _osm_physp * const p_physp2, - IN const ib_net16_t pkey); +boolean_t osm_physp_share_this_pkey(IN const struct _osm_physp *const p_physp1, + IN const struct _osm_physp *const p_physp2, + IN const ib_net16_t pkey); /* * PARAMETERS * @@ -487,9 +476,10 @@ boolean_t osm_physp_share_this_pkey( * * SYNOPSIS */ -ib_net16_t osm_physp_find_common_pkey( - IN const struct _osm_physp * const p_physp1, - IN const struct _osm_physp * const p_physp2 ); +ib_net16_t osm_physp_find_common_pkey(IN const struct _osm_physp *const + p_physp1, + IN const struct _osm_physp *const + p_physp2); /* * PARAMETERS * @@ -531,10 +521,9 @@ ib_net16_t osm_physp_find_common_pkey( * * SYNOPSIS */ -boolean_t osm_physp_share_pkey( - IN osm_log_t* p_log, - IN const struct _osm_physp* const p_physp_1, - IN const struct _osm_physp* const p_physp_2 ); +boolean_t osm_physp_share_pkey(IN osm_log_t * p_log, + IN const struct _osm_physp *const p_physp_1, + IN const struct _osm_physp *const p_physp_2); /* * PARAMETERS @@ -579,10 +568,9 @@ boolean_t osm_physp_share_pkey( * * SYNOPSIS */ -boolean_t osm_port_share_pkey( - IN osm_log_t* p_log, - IN const struct _osm_port* const p_port_1, - IN const struct _osm_port* const p_port_2 ); +boolean_t osm_port_share_pkey(IN osm_log_t * p_log, + IN const struct _osm_port *const p_port_1, + IN const struct _osm_port *const p_port_2); /* * PARAMETERS @@ -627,13 +615,12 @@ boolean_t osm_port_share_pkey( * * SYNOPSIS */ -boolean_t osm_lid_share_pkey( - IN osm_log_t* p_log, - IN const struct _osm_subn* const p_subn, - IN const ib_net16_t lid1, - IN const uint8_t port_num1, - IN const ib_net16_t lid2, - IN const uint8_t port_num2 ); +boolean_t osm_lid_share_pkey(IN osm_log_t * p_log, + IN const struct _osm_subn *const p_subn, + IN const ib_net16_t lid1, + IN const uint8_t port_num1, + IN const ib_net16_t lid2, + IN const uint8_t port_num2); /* * PARAMETERS @@ -687,10 +674,9 @@ boolean_t osm_lid_share_pkey( * * SYNOPSIS */ -boolean_t osm_physp_has_pkey( - IN osm_log_t* p_log, - IN const ib_net16_t pkey, - IN const struct _osm_physp* const p_physp ); +boolean_t osm_physp_has_pkey(IN osm_log_t * p_log, + IN const ib_net16_t pkey, + IN const struct _osm_physp *const p_physp); /* * PARAMETERS @@ -719,12 +705,11 @@ boolean_t osm_physp_has_pkey( * * SYNOPSIS */ -void osm_pkey_get_tables( - IN osm_log_t *p_log, - IN osm_req_t *p_req, - IN osm_subn_t* const p_subn, - IN struct _osm_node* const p_node, - IN struct _osm_physp* const p_physp ); +void osm_pkey_get_tables(IN osm_log_t * p_log, + IN osm_req_t * p_req, + IN osm_subn_t * const p_subn, + IN struct _osm_node *const p_node, + IN struct _osm_physp *const p_physp); /* * PARAMETERS @@ -751,5 +736,4 @@ void osm_pkey_get_tables( *********/ END_C_DECLS - -#endif /* _OSM_PKEY_H_ */ +#endif /* _OSM_PKEY_H_ */ diff --git a/opensm/include/opensm/osm_pkey_mgr.h b/opensm/include/opensm/osm_pkey_mgr.h index 14144c5..0832350 100644 --- a/opensm/include/opensm/osm_pkey_mgr.h +++ b/opensm/include/opensm/osm_pkey_mgr.h @@ -52,13 +52,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****f* OpenSM: P_Key Manager/osm_pkey_mgr_process * NAME * osm_pkey_mgr_process @@ -68,9 +67,7 @@ BEGIN_C_DECLS * * SYNOPSIS */ -osm_signal_t -osm_pkey_mgr_process( - IN osm_opensm_t *p_osm ); +osm_signal_t osm_pkey_mgr_process(IN osm_opensm_t * p_osm); /* * PARAMETERS * p_osm @@ -85,5 +82,4 @@ osm_pkey_mgr_process( *********/ END_C_DECLS - -#endif /* _OSM_PKEY_MGR_H_ */ +#endif /* _OSM_PKEY_MGR_H_ */ diff --git a/opensm/include/opensm/osm_pkey_rcv.h b/opensm/include/opensm/osm_pkey_rcv.h index 20829b2..14e6351 100644 --- a/opensm/include/opensm/osm_pkey_rcv.h +++ b/opensm/include/opensm/osm_pkey_rcv.h @@ -47,13 +47,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/P_Key Receiver * NAME * P_Key Receiver @@ -71,7 +70,6 @@ BEGIN_C_DECLS * Yael Kalka, Mellanox * *********/ - /****s* OpenSM: P_Key Receiver/osm_pkey_rcv_t * NAME * osm_pkey_rcv_t @@ -84,12 +82,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pkey_rcv -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_pkey_rcv { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_pkey_rcv_t; /* * FIELDS @@ -118,8 +115,7 @@ typedef struct _osm_pkey_rcv * * SYNOPSIS */ -void osm_pkey_rcv_construct( - IN osm_pkey_rcv_t* const p_ctrl ); +void osm_pkey_rcv_construct(IN osm_pkey_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -149,8 +145,7 @@ void osm_pkey_rcv_construct( * * SYNOPSIS */ -void osm_pkey_rcv_destroy( - IN osm_pkey_rcv_t* const p_ctrl ); +void osm_pkey_rcv_destroy(IN osm_pkey_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -181,12 +176,11 @@ void osm_pkey_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_pkey_rcv_init( - IN osm_pkey_rcv_t* const p_ctrl, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_pkey_rcv_init(IN osm_pkey_rcv_t * const p_ctrl, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -225,9 +219,7 @@ ib_api_status_t osm_pkey_rcv_init( * * SYNOPSIS */ -void osm_pkey_rcv_process( - IN void *context, - IN void *data ); +void osm_pkey_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -248,5 +240,4 @@ void osm_pkey_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_PKEY_RCV_H_ */ +#endif /* _OSM_PKEY_RCV_H_ */ diff --git a/opensm/include/opensm/osm_port.h b/opensm/include/opensm/osm_port.h index b5acd7f..f2cfe18 100644 --- a/opensm/include/opensm/osm_port.h +++ b/opensm/include/opensm/osm_port.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /* Forward references. */ @@ -109,21 +108,20 @@ struct _osm_node; * * SYNOPSIS */ -typedef struct _osm_physp -{ - ib_port_info_t port_info; - ib_net64_t port_guid; - uint8_t port_num; - struct _osm_node *p_node; - struct _osm_physp *p_remote_physp; - boolean_t healthy; - uint8_t vl_high_limit; - unsigned need_update; - osm_dr_path_t dr_path; - osm_pkey_tbl_t pkeys; - ib_vl_arb_table_t vl_arb[4]; - cl_ptr_vector_t slvl_by_port; - boolean_t got_set_resp; +typedef struct _osm_physp { + ib_port_info_t port_info; + ib_net64_t port_guid; + uint8_t port_num; + struct _osm_node *p_node; + struct _osm_physp *p_remote_physp; + boolean_t healthy; + uint8_t vl_high_limit; + unsigned need_update; + osm_dr_path_t dr_path; + osm_pkey_tbl_t pkeys; + ib_vl_arb_table_t vl_arb[4]; + cl_ptr_vector_t slvl_by_port; + boolean_t got_set_resp; } osm_physp_t; /* * FIELDS @@ -199,9 +197,7 @@ typedef struct _osm_physp * * SYNOPSIS */ -void -osm_physp_construct( - IN osm_physp_t* const p_physp ); +void osm_physp_construct(IN osm_physp_t * const p_physp); /* * PARAMETERS * p_physp @@ -226,14 +222,13 @@ osm_physp_construct( * SYNOPSIS */ void -osm_physp_init( - IN osm_physp_t* const p_physp, - IN const ib_net64_t port_guid, - IN const uint8_t port_num, - IN const struct _osm_node* const p_node, - IN const osm_bind_handle_t h_bind, - IN const uint8_t hop_count, - IN const uint8_t* const p_initial_path ); +osm_physp_init(IN osm_physp_t * const p_physp, + IN const ib_net64_t port_guid, + IN const uint8_t port_num, + IN const struct _osm_node *const p_node, + IN const osm_bind_handle_t h_bind, + IN const uint8_t hop_count, + IN const uint8_t * const p_initial_path); /* * PARAMETERS * p_physp @@ -281,9 +276,7 @@ osm_physp_init( * * SYNOPSIS */ -void -osm_physp_destroy( - IN osm_physp_t* const p_physp ); +void osm_physp_destroy(IN osm_physp_t * const p_physp); /* * PARAMETERS * p_port @@ -312,13 +305,12 @@ osm_physp_destroy( * * SYNOPSIS */ -static inline boolean_t -osm_physp_is_valid( - IN const osm_physp_t* const p_physp ) +static inline boolean_t osm_physp_is_valid(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - return( p_physp->port_guid != 0 ); + CL_ASSERT(p_physp); + return (p_physp->port_guid != 0); } + /* * PARAMETERS * p_physp @@ -345,12 +337,12 @@ osm_physp_is_valid( * SYNOPSIS */ static inline boolean_t -osm_physp_is_healthy( - IN const osm_physp_t* const p_physp ) +osm_physp_is_healthy(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - return( p_physp->healthy ); + CL_ASSERT(p_physp); + return (p_physp->healthy); } + /* * PARAMETERS * p_physp @@ -379,9 +371,7 @@ osm_physp_is_healthy( * * SYNOPSIS */ -boolean_t -osm_link_is_healthy( - IN const osm_physp_t* const p_physp ); +boolean_t osm_link_is_healthy(IN const osm_physp_t * const p_physp); /* * PARAMETERS * p_physp @@ -409,13 +399,12 @@ osm_link_is_healthy( * SYNOPSIS */ static inline void -osm_physp_set_health( - IN osm_physp_t* const p_physp, - IN boolean_t is_healthy ) +osm_physp_set_health(IN osm_physp_t * const p_physp, IN boolean_t is_healthy) { - CL_ASSERT( p_physp ); + CL_ASSERT(p_physp); p_physp->healthy = is_healthy; } + /* * PARAMETERS * p_physp @@ -446,26 +435,24 @@ osm_physp_set_health( * SYNOPSIS */ static inline void -osm_physp_set_port_info( - IN osm_physp_t* const p_physp, - IN const ib_port_info_t* const p_pi ) +osm_physp_set_port_info(IN osm_physp_t * const p_physp, + IN const ib_port_info_t * const p_pi) { - CL_ASSERT( p_pi ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - - if (ib_port_info_get_port_state(p_pi) == IB_LINK_DOWN) - { - /* If PortState is down, only copy PortState */ - /* and PortPhysicalState per C14-24-2.1 */ - ib_port_info_set_port_state(&p_physp->port_info, IB_LINK_DOWN); - ib_port_info_set_port_phys_state( - ib_port_info_get_port_phys_state(p_pi), &p_physp->port_info); - } - else - { - p_physp->port_info = *p_pi; + CL_ASSERT(p_pi); + CL_ASSERT(osm_physp_is_valid(p_physp)); + + if (ib_port_info_get_port_state(p_pi) == IB_LINK_DOWN) { + /* If PortState is down, only copy PortState */ + /* and PortPhysicalState per C14-24-2.1 */ + ib_port_info_set_port_state(&p_physp->port_info, IB_LINK_DOWN); + ib_port_info_set_port_phys_state + (ib_port_info_get_port_phys_state(p_pi), + &p_physp->port_info); + } else { + p_physp->port_info = *p_pi; } } + /* * PARAMETERS * p_physp @@ -494,20 +481,19 @@ osm_physp_set_port_info( * SYNOPSIS */ static inline ib_net16_t -osm_physp_trim_base_lid_to_valid_range( - IN osm_physp_t* const p_physp ) +osm_physp_trim_base_lid_to_valid_range(IN osm_physp_t * const p_physp) { ib_net16_t orig_lid = 0; - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - if ( ( cl_ntoh16( p_physp->port_info.base_lid ) > IB_LID_UCAST_END_HO ) || - ( cl_ntoh16( p_physp->port_info.base_lid ) < IB_LID_UCAST_START_HO ) ) - { + CL_ASSERT(osm_physp_is_valid(p_physp)); + if ((cl_ntoh16(p_physp->port_info.base_lid) > IB_LID_UCAST_END_HO) || + (cl_ntoh16(p_physp->port_info.base_lid) < IB_LID_UCAST_START_HO)) { orig_lid = p_physp->port_info.base_lid; p_physp->port_info.base_lid = 0; } return orig_lid; } + /* * PARAMETERS * p_physp @@ -533,11 +519,10 @@ osm_physp_trim_base_lid_to_valid_range( * SYNOPSIS */ void -osm_physp_set_pkey_tbl( IN osm_log_t* p_log, - IN const osm_subn_t* p_subn, - IN osm_physp_t* const p_physp, - IN ib_pkey_table_t *p_pkey_tbl, - IN uint16_t block_num ); +osm_physp_set_pkey_tbl(IN osm_log_t * p_log, + IN const osm_subn_t * p_subn, + IN osm_physp_t * const p_physp, + IN ib_pkey_table_t * p_pkey_tbl, IN uint16_t block_num); /* * PARAMETERS * p_log @@ -576,16 +561,17 @@ osm_physp_set_pkey_tbl( IN osm_log_t* p_log, * * SYNOPSIS */ -static inline const osm_pkey_tbl_t * -osm_physp_get_pkey_tbl( IN const osm_physp_t* const p_physp ) +static inline const osm_pkey_tbl_t *osm_physp_get_pkey_tbl(IN const osm_physp_t + * const p_physp) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - /* - (14.2.5.7) - the block number valid values are 0-2047, and are further - limited by the size of the P_Key table specified by the PartitionCap on the node. - */ - return( &p_physp->pkeys ); + CL_ASSERT(osm_physp_is_valid(p_physp)); + /* + (14.2.5.7) - the block number valid values are 0-2047, and are further + limited by the size of the P_Key table specified by the PartitionCap on the node. + */ + return (&p_physp->pkeys); }; + /* * PARAMETERS * p_physp @@ -609,16 +595,17 @@ osm_physp_get_pkey_tbl( IN const osm_physp_t* const p_physp ) * * SYNOPSIS */ -static inline osm_pkey_tbl_t * -osm_physp_get_mod_pkey_tbl( IN osm_physp_t* const p_physp ) +static inline osm_pkey_tbl_t *osm_physp_get_mod_pkey_tbl(IN osm_physp_t * + const p_physp) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - /* - (14.2.5.7) - the block number valid values are 0-2047, and are further - limited by the size of the P_Key table specified by the PartitionCap on the node. - */ - return( &p_physp->pkeys ); + CL_ASSERT(osm_physp_is_valid(p_physp)); + /* + (14.2.5.7) - the block number valid values are 0-2047, and are further + limited by the size of the P_Key table specified by the PartitionCap on the node. + */ + return (&p_physp->pkeys); }; + /* * PARAMETERS * p_physp @@ -643,17 +630,17 @@ osm_physp_get_mod_pkey_tbl( IN osm_physp_t* const p_physp ) * SYNOPSIS */ static inline void -osm_physp_set_slvl_tbl( IN osm_physp_t* const p_physp, - IN ib_slvl_table_t *p_slvl_tbl, - IN uint8_t in_port_num ) +osm_physp_set_slvl_tbl(IN osm_physp_t * const p_physp, + IN ib_slvl_table_t * p_slvl_tbl, IN uint8_t in_port_num) { - ib_slvl_table_t *p_tbl; + ib_slvl_table_t *p_tbl; - CL_ASSERT( p_slvl_tbl ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - p_tbl = cl_ptr_vector_get(&p_physp->slvl_by_port, in_port_num); - *p_tbl = *p_slvl_tbl; + CL_ASSERT(p_slvl_tbl); + CL_ASSERT(osm_physp_is_valid(p_physp)); + p_tbl = cl_ptr_vector_get(&p_physp->slvl_by_port, in_port_num); + *p_tbl = *p_slvl_tbl; } + /* * PARAMETERS * p_physp @@ -683,16 +670,17 @@ osm_physp_set_slvl_tbl( IN osm_physp_t* const p_physp, * * SYNOPSIS */ -static inline ib_slvl_table_t * -osm_physp_get_slvl_tbl( IN const osm_physp_t* const p_physp, - IN uint8_t in_port_num ) +static inline ib_slvl_table_t *osm_physp_get_slvl_tbl(IN const osm_physp_t * + const p_physp, + IN uint8_t in_port_num) { - ib_slvl_table_t *p_tbl; + ib_slvl_table_t *p_tbl; - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - p_tbl = cl_ptr_vector_get(&p_physp->slvl_by_port, in_port_num); - return(p_tbl); + CL_ASSERT(osm_physp_is_valid(p_physp)); + p_tbl = cl_ptr_vector_get(&p_physp->slvl_by_port, in_port_num); + return (p_tbl); } + /* * PARAMETERS * p_physp @@ -720,15 +708,15 @@ osm_physp_get_slvl_tbl( IN const osm_physp_t* const p_physp, * SYNOPSIS */ static inline void -osm_physp_set_vla_tbl( IN osm_physp_t* const p_physp, - IN ib_vl_arb_table_t *p_vla_tbl, - IN uint8_t block_num ) +osm_physp_set_vla_tbl(IN osm_physp_t * const p_physp, + IN ib_vl_arb_table_t * p_vla_tbl, IN uint8_t block_num) { - CL_ASSERT( p_vla_tbl ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - CL_ASSERT( (1 <= block_num) && (block_num <= 4)); - p_physp->vl_arb[block_num - 1] = *p_vla_tbl; + CL_ASSERT(p_vla_tbl); + CL_ASSERT(osm_physp_is_valid(p_physp)); + CL_ASSERT((1 <= block_num) && (block_num <= 4)); + p_physp->vl_arb[block_num - 1] = *p_vla_tbl; } + /* * PARAMETERS * p_physp @@ -759,14 +747,15 @@ osm_physp_set_vla_tbl( IN osm_physp_t* const p_physp, * * SYNOPSIS */ -static inline ib_vl_arb_table_t * -osm_physp_get_vla_tbl( IN osm_physp_t* const p_physp, - IN uint8_t block_num ) +static inline ib_vl_arb_table_t *osm_physp_get_vla_tbl(IN osm_physp_t * + const p_physp, + IN uint8_t block_num) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - CL_ASSERT( (1 <= block_num) && (block_num <= 4)); - return(& (p_physp->vl_arb[block_num - 1])); + CL_ASSERT(osm_physp_is_valid(p_physp)); + CL_ASSERT((1 <= block_num) && (block_num <= 4)); + return (&(p_physp->vl_arb[block_num - 1])); } + /* * PARAMETERS * p_physp @@ -794,13 +783,13 @@ osm_physp_get_vla_tbl( IN osm_physp_t* const p_physp, * * SYNOPSIS */ -static inline osm_physp_t* -osm_physp_get_remote( - IN const osm_physp_t* const p_physp ) +static inline osm_physp_t *osm_physp_get_remote(IN const osm_physp_t * + const p_physp) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( p_physp->p_remote_physp ); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (p_physp->p_remote_physp); } + /* * PARAMETERS * p_physp @@ -826,12 +815,12 @@ osm_physp_get_remote( * SYNOPSIS */ static inline ib_net64_t -osm_physp_get_port_guid( - IN const osm_physp_t* const p_physp ) +osm_physp_get_port_guid(IN const osm_physp_t * const p_physp) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( p_physp->port_guid ); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (p_physp->port_guid); } + /* * PARAMETERS * p_physp @@ -856,12 +845,12 @@ osm_physp_get_port_guid( * SYNOPSIS */ static inline ib_net64_t -osm_physp_get_subnet_prefix( - IN const osm_physp_t* const p_physp ) +osm_physp_get_subnet_prefix(IN const osm_physp_t * const p_physp) { - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( p_physp->port_info.subnet_prefix ); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (p_physp->port_info.subnet_prefix); } + /* * PARAMETERS * p_physp @@ -887,17 +876,17 @@ osm_physp_get_subnet_prefix( * SYNOPSIS */ static inline boolean_t -osm_physp_link_exists( - IN const osm_physp_t* const p_physp, - IN const osm_physp_t* const p_remote_physp ) +osm_physp_link_exists(IN const osm_physp_t * const p_physp, + IN const osm_physp_t * const p_remote_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - CL_ASSERT( p_remote_physp ); - CL_ASSERT( osm_physp_is_valid( p_remote_physp ) ); - return( (p_physp->p_remote_physp == p_remote_physp ) && - (p_remote_physp->p_remote_physp == p_physp ) ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + CL_ASSERT(p_remote_physp); + CL_ASSERT(osm_physp_is_valid(p_remote_physp)); + return ((p_physp->p_remote_physp == p_remote_physp) && + (p_remote_physp->p_remote_physp == p_physp)); } + /* * PARAMETERS * p_physp @@ -926,15 +915,15 @@ osm_physp_link_exists( * SYNOPSIS */ static inline void -osm_physp_link( - IN osm_physp_t* const p_physp, - IN osm_physp_t* const p_remote_physp ) +osm_physp_link(IN osm_physp_t * const p_physp, + IN osm_physp_t * const p_remote_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( p_remote_physp ); + CL_ASSERT(p_physp); + CL_ASSERT(p_remote_physp); p_physp->p_remote_physp = p_remote_physp; p_remote_physp->p_remote_physp = p_physp; } + /* * PARAMETERS * p_physp @@ -962,16 +951,16 @@ osm_physp_link( * SYNOPSIS */ static inline void -osm_physp_unlink( - IN osm_physp_t* const p_physp, - IN osm_physp_t* const p_remote_physp ) +osm_physp_unlink(IN osm_physp_t * const p_physp, + IN osm_physp_t * const p_remote_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( p_remote_physp ); - CL_ASSERT( osm_physp_link_exists( p_physp, p_remote_physp ) ); + CL_ASSERT(p_physp); + CL_ASSERT(p_remote_physp); + CL_ASSERT(osm_physp_link_exists(p_physp, p_remote_physp)); p_physp->p_remote_physp = NULL; p_remote_physp->p_remote_physp = NULL; } + /* * PARAMETERS * p_physp @@ -1000,15 +989,15 @@ osm_physp_unlink( * SYNOPSIS */ static inline boolean_t -osm_physp_has_any_link( - IN const osm_physp_t* const p_physp ) +osm_physp_has_any_link(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - if( osm_physp_is_valid( p_physp ) ) - return( p_physp->p_remote_physp != NULL ); + CL_ASSERT(p_physp); + if (osm_physp_is_valid(p_physp)) + return (p_physp->p_remote_physp != NULL); else - return( FALSE ); + return (FALSE); } + /* * PARAMETERS * p_physp @@ -1034,13 +1023,13 @@ osm_physp_has_any_link( * SYNOPSIS */ static inline uint8_t -osm_physp_get_port_num( - IN const osm_physp_t* const p_physp ) +osm_physp_get_port_num(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( p_physp->port_num ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (p_physp->port_num); } + /* * PARAMETERS * p_physp @@ -1063,14 +1052,14 @@ osm_physp_get_port_num( * * SYNOPSIS */ -static inline struct _osm_node* -osm_physp_get_node_ptr( - IN const osm_physp_t* const p_physp ) +static inline struct _osm_node *osm_physp_get_node_ptr(IN const osm_physp_t * + const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( (struct _osm_node*)p_physp->p_node ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return ((struct _osm_node *)p_physp->p_node); } + /* * PARAMETERS * p_physp @@ -1094,13 +1083,13 @@ osm_physp_get_node_ptr( * SYNOPSIS */ static inline uint8_t -osm_physp_get_port_state( - IN const osm_physp_t* const p_physp ) +osm_physp_get_port_state(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( ib_port_info_get_port_state( &p_physp->port_info )); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (ib_port_info_get_port_state(&p_physp->port_info)); } + /* * PARAMETERS * p_physp @@ -1124,13 +1113,13 @@ osm_physp_get_port_state( * SYNOPSIS */ static inline ib_net16_t -osm_physp_get_base_lid( - IN const osm_physp_t* const p_physp ) +osm_physp_get_base_lid(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( p_physp->port_info.base_lid ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (p_physp->port_info.base_lid); } + /* * PARAMETERS * p_physp @@ -1153,14 +1142,13 @@ osm_physp_get_base_lid( * * SYNOPSIS */ -static inline uint8_t -osm_physp_get_lmc( - IN const osm_physp_t* const p_physp ) +static inline uint8_t osm_physp_get_lmc(IN const osm_physp_t * const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( ib_port_info_get_lmc( &p_physp->port_info ) ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return (ib_port_info_get_lmc(&p_physp->port_info)); } + /* * PARAMETERS * p_physp @@ -1183,14 +1171,14 @@ osm_physp_get_lmc( * * SYNOPSIS */ -static inline osm_dr_path_t* -osm_physp_get_dr_path_ptr( - IN const osm_physp_t* const p_physp ) +static inline osm_dr_path_t *osm_physp_get_dr_path_ptr(IN const osm_physp_t * + const p_physp) { - CL_ASSERT( p_physp ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - return( (osm_dr_path_t*)&p_physp->dr_path ); + CL_ASSERT(p_physp); + CL_ASSERT(osm_physp_is_valid(p_physp)); + return ((osm_dr_path_t *) & p_physp->dr_path); } + /* * PARAMETERS * p_physp @@ -1239,8 +1227,7 @@ osm_physp_get_dr_path_ptr( * * SYNOPSIS */ -typedef enum _osm_port_lid_category -{ +typedef enum _osm_port_lid_category { OSM_PORT_LID_ASSIGNED = 0, OSM_PORT_LID_UNASSIGNED, OSM_PORT_LID_CONFLICT, @@ -1278,15 +1265,14 @@ typedef enum _osm_port_lid_category * * SYNOPSIS */ -typedef struct _osm_port -{ - cl_map_item_t map_item; - struct _osm_node *p_node; - ib_net64_t guid; - uint32_t discovery_count; - unsigned is_new; - osm_physp_t *p_physp; - cl_qlist_t mcm_list; +typedef struct _osm_port { + cl_map_item_t map_item; + struct _osm_node *p_node; + ib_net64_t guid; + uint32_t discovery_count; + unsigned is_new; + osm_physp_t *p_physp; + cl_qlist_t mcm_list; } osm_port_t; /* * FIELDS @@ -1324,9 +1310,7 @@ typedef struct _osm_port * * SYNOPSIS */ -void -osm_port_delete( - IN OUT osm_port_t** const pp_port ); +void osm_port_delete(IN OUT osm_port_t ** const pp_port); /* * PARAMETERS * pp_port @@ -1352,10 +1336,8 @@ osm_port_delete( * * SYNOPSIS */ -osm_port_t* -osm_port_new( - IN const ib_node_info_t* p_ni, - IN const struct _osm_node* const p_parent_node ); +osm_port_t *osm_port_new(IN const ib_node_info_t * p_ni, + IN const struct _osm_node *const p_parent_node); /* * PARAMETERS * p_ni @@ -1385,12 +1367,12 @@ osm_port_new( * SYNOPSIS */ static inline ib_net16_t -osm_port_get_base_lid( - IN const osm_port_t* const p_port ) +osm_port_get_base_lid(IN const osm_port_t * const p_port) { - CL_ASSERT( p_port->p_physp && osm_physp_is_valid( p_port->p_physp ) ); - return( osm_physp_get_base_lid( p_port->p_physp )); + CL_ASSERT(p_port->p_physp && osm_physp_is_valid(p_port->p_physp)); + return (osm_physp_get_base_lid(p_port->p_physp)); } + /* * PARAMETERS * p_port @@ -1415,13 +1397,12 @@ osm_port_get_base_lid( * * SYNOPSIS */ -static inline uint8_t -osm_port_get_lmc( - IN const osm_port_t* const p_port ) +static inline uint8_t osm_port_get_lmc(IN const osm_port_t * const p_port) { - CL_ASSERT( p_port->p_physp && osm_physp_is_valid( p_port->p_physp ) ); - return( osm_physp_get_lmc( p_port->p_physp )); + CL_ASSERT(p_port->p_physp && osm_physp_is_valid(p_port->p_physp)); + return (osm_physp_get_lmc(p_port->p_physp)); } + /* * PARAMETERS * p_port @@ -1445,12 +1426,11 @@ osm_port_get_lmc( * * SYNOPSIS */ -static inline ib_net64_t -osm_port_get_guid( - IN const osm_port_t* const p_port ) +static inline ib_net64_t osm_port_get_guid(IN const osm_port_t * const p_port) { - return( p_port->guid ); + return (p_port->guid); } + /* * PARAMETERS * p_port @@ -1476,10 +1456,9 @@ osm_port_get_guid( * SYNOPSIS */ void -osm_port_get_lid_range_ho( - IN const osm_port_t* const p_port, - OUT uint16_t* const p_min_lid, - OUT uint16_t* const p_max_lid ); +osm_port_get_lid_range_ho(IN const osm_port_t * const p_port, + OUT uint16_t * const p_min_lid, + OUT uint16_t * const p_max_lid); /* * PARAMETERS * p_port @@ -1511,10 +1490,9 @@ osm_port_get_lid_range_ho( * SYNOPSIS */ ib_api_status_t -osm_get_port_by_base_lid( - IN const osm_subn_t* const p_subn, - IN const ib_net16_t lid, - IN OUT const osm_port_t** const pp_port ); +osm_get_port_by_base_lid(IN const osm_subn_t * const p_subn, + IN const ib_net16_t lid, + IN OUT const osm_port_t ** const pp_port); /* * PARAMETERS * p_subn @@ -1547,9 +1525,7 @@ osm_get_port_by_base_lid( * SYNOPSIS */ void -osm_port_add_new_physp( - IN osm_port_t* const p_port, - IN const uint8_t port_num ); +osm_port_add_new_physp(IN osm_port_t * const p_port, IN const uint8_t port_num); /* * PARAMETERS * p_port @@ -1577,9 +1553,7 @@ osm_port_add_new_physp( * SYNOPSIS */ ib_api_status_t -osm_port_add_mgrp( - IN osm_port_t* const p_port, - IN const ib_net16_t mlid ); +osm_port_add_mgrp(IN osm_port_t * const p_port, IN const ib_net16_t mlid); /* * PARAMETERS * p_port @@ -1608,9 +1582,7 @@ osm_port_add_mgrp( * SYNOPSIS */ void -osm_port_remove_mgrp( - IN osm_port_t* const p_port, - IN const ib_net16_t mlid ); +osm_port_remove_mgrp(IN osm_port_t * const p_port, IN const ib_net16_t mlid); /* * PARAMETERS * p_port @@ -1637,9 +1609,7 @@ osm_port_remove_mgrp( * * SYNOPSIS */ -void -osm_port_remove_all_mgrp( - IN osm_port_t* const p_port ); +void osm_port_remove_all_mgrp(IN osm_port_t * const p_port); /* * PARAMETERS * p_port @@ -1665,9 +1635,7 @@ osm_port_remove_all_mgrp( * SYNOPSIS */ uint8_t -osm_physp_calc_link_mtu( - IN osm_log_t* p_log, - IN const osm_physp_t* p_physp ); +osm_physp_calc_link_mtu(IN osm_log_t * p_log, IN const osm_physp_t * p_physp); /* * PARAMETERS * p_log @@ -1696,10 +1664,9 @@ osm_physp_calc_link_mtu( * SYNOPSIS */ uint8_t -osm_physp_calc_link_op_vls( - IN osm_log_t* p_log, - IN const osm_subn_t * p_subn, - IN const osm_physp_t* p_physp ); +osm_physp_calc_link_op_vls(IN osm_log_t * p_log, + IN const osm_subn_t * p_subn, + IN const osm_physp_t * p_physp); /* * PARAMETERS * p_log @@ -1731,11 +1698,10 @@ osm_physp_calc_link_op_vls( * SYNOPSIS */ void -osm_physp_replace_dr_path_with_alternate_dr_path( - IN osm_log_t *p_log, - IN osm_subn_t const *p_subn, - IN osm_physp_t const *p_physp, - IN osm_bind_handle_t *h_bind ); +osm_physp_replace_dr_path_with_alternate_dr_path(IN osm_log_t * p_log, + IN osm_subn_t const *p_subn, + IN osm_physp_t const *p_physp, + IN osm_bind_handle_t * h_bind); /* * PARAMETERS * p_log @@ -1760,5 +1726,4 @@ osm_physp_replace_dr_path_with_alternate_dr_path( *********/ END_C_DECLS - -#endif /* _OSM_PORT_H_ */ +#endif /* _OSM_PORT_H_ */ diff --git a/opensm/include/opensm/osm_port_info_rcv.h b/opensm/include/opensm/osm_port_info_rcv.h index e4561c8..9f3ab22 100644 --- a/opensm/include/opensm/osm_port_info_rcv.h +++ b/opensm/include/opensm/osm_port_info_rcv.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Port Info Receiver * NAME * Port Info Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Port Info Receiver/osm_pi_rcv_t * NAME * osm_pi_rcv_t @@ -98,13 +96,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pi_rcv -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; - cl_plock_t *p_lock; +typedef struct _osm_pi_rcv { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + osm_state_mgr_t *p_state_mgr; + cl_plock_t *p_lock; } osm_pi_rcv_t; /* * FIELDS @@ -136,8 +133,7 @@ typedef struct _osm_pi_rcv * * SYNOPSIS */ -void osm_pi_rcv_construct( - IN osm_pi_rcv_t* const p_ctrl ); +void osm_pi_rcv_construct(IN osm_pi_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -167,8 +163,7 @@ void osm_pi_rcv_construct( * * SYNOPSIS */ -void osm_pi_rcv_destroy( - IN osm_pi_rcv_t* const p_ctrl ); +void osm_pi_rcv_destroy(IN osm_pi_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -199,13 +194,12 @@ void osm_pi_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_pi_rcv_init( - IN osm_pi_rcv_t* const p_ctrl, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_pi_rcv_init(IN osm_pi_rcv_t * const p_ctrl, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -247,9 +241,7 @@ ib_api_status_t osm_pi_rcv_init( * * SYNOPSIS */ -void osm_pi_rcv_process( - IN void *context, - IN void *data ); +void osm_pi_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -270,5 +262,4 @@ void osm_pi_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_PI_RCV_H_ */ +#endif /* _OSM_PI_RCV_H_ */ diff --git a/opensm/include/opensm/osm_port_profile.h b/opensm/include/opensm/osm_port_profile.h index e6c5426..2442850 100644 --- a/opensm/include/opensm/osm_port_profile.h +++ b/opensm/include/opensm/osm_port_profile.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Port Profile * NAME * Port Profile @@ -81,7 +80,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Switch/osm_port_profile_t * NAME * osm_port_profile_t @@ -96,9 +94,8 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_port_profile -{ - uint32_t num_paths; +typedef struct _osm_port_profile { + uint32_t num_paths; } osm_port_profile_t; /* * FIELDS @@ -117,13 +114,12 @@ typedef struct _osm_port_profile * * SYNOPSIS */ -static inline void -osm_port_prof_construct( - IN osm_port_profile_t* const p_prof ) +static inline void osm_port_prof_construct(IN osm_port_profile_t * const p_prof) { - CL_ASSERT( p_prof ); - memset( p_prof, 0, sizeof(*p_prof) ); + CL_ASSERT(p_prof); + memset(p_prof, 0, sizeof(*p_prof)); } + /* * PARAMETERS * p_prof @@ -148,12 +144,12 @@ osm_port_prof_construct( * SYNOPSIS */ static inline void -osm_port_prof_path_count_inc( - IN osm_port_profile_t* const p_prof ) +osm_port_prof_path_count_inc(IN osm_port_profile_t * const p_prof) { - CL_ASSERT( p_prof ); + CL_ASSERT(p_prof); p_prof->num_paths++; } + /* * PARAMETERS * p_pro @@ -177,11 +173,11 @@ osm_port_prof_path_count_inc( * SYNOPSIS */ static inline uint32_t -osm_port_prof_path_count_get( - IN const osm_port_profile_t* const p_prof ) +osm_port_prof_path_count_get(IN const osm_port_profile_t * const p_prof) { - return( p_prof->num_paths ); + return (p_prof->num_paths); } + /* * PARAMETERS * p_pro @@ -206,22 +202,21 @@ osm_port_prof_path_count_get( * SYNOPSIS */ static inline boolean_t -osm_port_prof_is_ignored_port( - IN const osm_subn_t *p_subn, - IN ib_net64_t port_guid, - IN uint8_t port_num ) +osm_port_prof_is_ignored_port(IN const osm_subn_t * p_subn, + IN ib_net64_t port_guid, IN uint8_t port_num) { - const cl_map_t *p_map = &(p_subn->opt.port_prof_ignore_guids); - const void *p_obj = cl_map_get(p_map, port_guid); - size_t res; + const cl_map_t *p_map = &(p_subn->opt.port_prof_ignore_guids); + const void *p_obj = cl_map_get(p_map, port_guid); + size_t res; - // HACK: we currently support ignoring ports 0 - 31 - if (p_obj != NULL) { - res = (size_t)p_obj & (size_t)(1 << port_num); - return (res != 0); - } - return FALSE; + // HACK: we currently support ignoring ports 0 - 31 + if (p_obj != NULL) { + res = (size_t) p_obj & (size_t) (1 << port_num); + return (res != 0); + } + return FALSE; } + /* * PARAMETERS * p_subn @@ -248,28 +243,26 @@ osm_port_prof_is_ignored_port( * SYNOPSIS */ static inline void -osm_port_prof_set_ignored_port( - IN osm_subn_t *p_subn, - IN ib_net64_t port_guid, - IN uint8_t port_num ) +osm_port_prof_set_ignored_port(IN osm_subn_t * p_subn, + IN ib_net64_t port_guid, IN uint8_t port_num) { - cl_map_t *p_map = &(p_subn->opt.port_prof_ignore_guids); - const void *p_obj = cl_map_get(p_map, port_guid); - size_t value = 0; + cl_map_t *p_map = &(p_subn->opt.port_prof_ignore_guids); + const void *p_obj = cl_map_get(p_map, port_guid); + size_t value = 0; - // HACK: we currently support ignoring ports 0 - 31 - CL_ASSERT(port_num < 32); + // HACK: we currently support ignoring ports 0 - 31 + CL_ASSERT(port_num < 32); - if (p_obj != NULL) { - value = (size_t)p_obj; - cl_map_remove(p_map, port_guid); - } + if (p_obj != NULL) { + value = (size_t) p_obj; + cl_map_remove(p_map, port_guid); + } - value = value | (1 << port_num); - cl_map_insert(&(p_subn->opt.port_prof_ignore_guids), - port_guid, - (void *)value); + value = value | (1 << port_num); + cl_map_insert(&(p_subn->opt.port_prof_ignore_guids), + port_guid, (void *)value); } + /* * PARAMETERS * p_subn @@ -287,5 +280,4 @@ osm_port_prof_set_ignored_port( *********/ END_C_DECLS - -#endif /* _OSM_PORT_PROFILE_H_ */ +#endif /* _OSM_PORT_PROFILE_H_ */ diff --git a/opensm/include/opensm/osm_rand_fwd_tbl.h b/opensm/include/opensm/osm_rand_fwd_tbl.h index bcda9b3..8f46ef2 100644 --- a/opensm/include/opensm/osm_rand_fwd_tbl.h +++ b/opensm/include/opensm/osm_rand_fwd_tbl.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Random Forwarding Table * NAME * Random Forwarding Table @@ -81,7 +80,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Forwarding Table/osm_rand_fwd_tbl_t * NAME * osm_rand_fwd_tbl_t @@ -94,10 +92,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_rand_fwd_tbl -{ +typedef struct _osm_rand_fwd_tbl { /* PLACE HOLDER STRUCTURE ONLY!! */ - uint32_t size; + uint32_t size; } osm_rand_fwd_tbl_t; /* * FIELDS @@ -116,16 +113,15 @@ typedef struct _osm_rand_fwd_tbl * * SYNOPSIS */ -static inline void -osm_rand_tbl_delete( - IN osm_rand_fwd_tbl_t** const pp_tbl ) +static inline void osm_rand_tbl_delete(IN osm_rand_fwd_tbl_t ** const pp_tbl) { /* - TO DO - This is a place holder function only! - */ - free( *pp_tbl ); + TO DO - This is a place holder function only! + */ + free(*pp_tbl); *pp_tbl = NULL; } + /* * PARAMETERS * pp_tbl @@ -151,17 +147,16 @@ osm_rand_tbl_delete( * SYNOPSIS */ static inline void -osm_rand_fwd_tbl_set( - IN osm_rand_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_ho, - IN const uint8_t port ) +osm_rand_fwd_tbl_set(IN osm_rand_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_ho, IN const uint8_t port) { /* Random forwarding tables not supported yet. */ - UNUSED_PARAM( p_tbl ); - UNUSED_PARAM( lid_ho ); - UNUSED_PARAM( port ); - CL_ASSERT( FALSE ); + UNUSED_PARAM(p_tbl); + UNUSED_PARAM(lid_ho); + UNUSED_PARAM(port); + CL_ASSERT(FALSE); } + /* * PARAMETERS * p_tbl @@ -191,18 +186,18 @@ osm_rand_fwd_tbl_set( * SYNOPSIS */ static inline ib_api_status_t -osm_rand_fwd_tbl_set_block( - IN osm_rand_fwd_tbl_t* const p_tbl, - IN const uint8_t* const p_block, - IN const uint32_t block_num ) +osm_rand_fwd_tbl_set_block(IN osm_rand_fwd_tbl_t * const p_tbl, + IN const uint8_t * const p_block, + IN const uint32_t block_num) { /* Random forwarding tables not supported yet. */ - UNUSED_PARAM( p_tbl ); - UNUSED_PARAM( p_block ); - UNUSED_PARAM( block_num ); - CL_ASSERT( FALSE ); - return( IB_ERROR ); + UNUSED_PARAM(p_tbl); + UNUSED_PARAM(p_block); + UNUSED_PARAM(block_num); + CL_ASSERT(FALSE); + return (IB_ERROR); } + /* * PARAMETERS * p_tbl @@ -232,16 +227,16 @@ osm_rand_fwd_tbl_set_block( * SYNOPSIS */ static inline uint8_t -osm_rand_fwd_tbl_get( - IN const osm_rand_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_ho ) +osm_rand_fwd_tbl_get(IN const osm_rand_fwd_tbl_t * const p_tbl, + IN const uint16_t lid_ho) { - CL_ASSERT( FALSE ); - UNUSED_PARAM( p_tbl ); - UNUSED_PARAM( lid_ho ); + CL_ASSERT(FALSE); + UNUSED_PARAM(p_tbl); + UNUSED_PARAM(lid_ho); - return( OSM_NO_PATH ); + return (OSM_NO_PATH); } + /* * PARAMETERS * p_tbl @@ -268,12 +263,12 @@ osm_rand_fwd_tbl_get( * SYNOPSIS */ static inline uint16_t -osm_rand_fwd_tbl_get_lids_per_block( - IN const osm_rand_fwd_tbl_t* const p_tbl ) +osm_rand_fwd_tbl_get_lids_per_block(IN const osm_rand_fwd_tbl_t * const p_tbl) { - UNUSED_PARAM( p_tbl ); - return( 16 ); + UNUSED_PARAM(p_tbl); + return (16); } + /* * PARAMETERS * p_tbl @@ -297,15 +292,16 @@ osm_rand_fwd_tbl_get_lids_per_block( * SYNOPSIS */ static inline uint16_t -osm_rand_fwd_tbl_get_max_block_id_in_use( - IN const osm_rand_fwd_tbl_t* const p_tbl, - IN const uint16_t lid_top_ho ) +osm_rand_fwd_tbl_get_max_block_id_in_use(IN const osm_rand_fwd_tbl_t * + const p_tbl, + IN const uint16_t lid_top_ho) { - UNUSED_PARAM( p_tbl ); - UNUSED_PARAM( lid_top_ho ); - CL_ASSERT( FALSE ); - return( 0 ); + UNUSED_PARAM(p_tbl); + UNUSED_PARAM(lid_top_ho); + CL_ASSERT(FALSE); + return (0); } + /* * PARAMETERS * p_tbl @@ -329,13 +325,13 @@ osm_rand_fwd_tbl_get_max_block_id_in_use( * SYNOPSIS */ static inline uint16_t -osm_rand_fwd_tbl_get_size( - IN const osm_rand_fwd_tbl_t* const p_tbl ) +osm_rand_fwd_tbl_get_size(IN const osm_rand_fwd_tbl_t * const p_tbl) { - UNUSED_PARAM( p_tbl ); - CL_ASSERT( FALSE ); - return( 0 ); + UNUSED_PARAM(p_tbl); + CL_ASSERT(FALSE); + return (0); } + /* * PARAMETERS * p_tbl @@ -350,5 +346,4 @@ osm_rand_fwd_tbl_get_size( *********/ END_C_DECLS - -#endif /* _OSM_RAND_FWD_TBL_H_ */ +#endif /* _OSM_RAND_FWD_TBL_H_ */ diff --git a/opensm/include/opensm/osm_remote_sm.h b/opensm/include/opensm/osm_remote_sm.h index e18e53b..4c5f67c 100644 --- a/opensm/include/opensm/osm_remote_sm.h +++ b/opensm/include/opensm/osm_remote_sm.h @@ -55,13 +55,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Remote SM * NAME * Remote SM @@ -79,7 +78,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Remote SM/osm_remote_sm_t * NAME * osm_remote_sm_t @@ -92,11 +90,10 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_remote_sm -{ - cl_map_item_t map_item; - const osm_port_t *p_port; - ib_sm_info_t smi; +typedef struct _osm_remote_sm { + cl_map_item_t map_item; + const osm_port_t *p_port; + ib_sm_info_t smi; } osm_remote_sm_t; /* * FIELDS @@ -120,9 +117,7 @@ typedef struct _osm_remote_sm * * SYNOPSIS */ -void -osm_remote_sm_construct( - IN osm_remote_sm_t* const p_sm ); +void osm_remote_sm_construct(IN osm_remote_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -151,9 +146,7 @@ osm_remote_sm_construct( * * SYNOPSIS */ -void -osm_remote_sm_destroy( - IN osm_remote_sm_t* const p_sm ); +void osm_remote_sm_destroy(IN osm_remote_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -182,10 +175,9 @@ osm_remote_sm_destroy( * SYNOPSIS */ void -osm_remote_sm_init( - IN osm_remote_sm_t* const p_sm, - IN const osm_port_t* const p_port, - IN const ib_sm_info_t* const p_smi ); +osm_remote_sm_init(IN osm_remote_sm_t * const p_sm, + IN const osm_port_t * const p_port, + IN const ib_sm_info_t * const p_smi); /* * PARAMETERS * p_sm @@ -208,5 +200,4 @@ osm_remote_sm_init( *********/ END_C_DECLS - -#endif /* _OSM_REMOTE_SM_H_ */ +#endif /* _OSM_REMOTE_SM_H_ */ diff --git a/opensm/include/opensm/osm_req.h b/opensm/include/opensm/osm_req.h index 21f281a..c4d8599 100644 --- a/opensm/include/opensm/osm_req.h +++ b/opensm/include/opensm/osm_req.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Generic Requester * NAME * Generic Requester @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Generic Requester/osm_req_t * NAME * osm_req_t @@ -99,13 +97,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_req -{ - osm_mad_pool_t *p_pool; - osm_vl15_t *p_vl15; - osm_log_t *p_log; - osm_subn_t *p_subn; - atomic32_t *p_sm_trans_id; +typedef struct _osm_req { + osm_mad_pool_t *p_pool; + osm_vl15_t *p_vl15; + osm_log_t *p_log; + osm_subn_t *p_subn; + atomic32_t *p_sm_trans_id; } osm_req_t; /* @@ -138,9 +135,7 @@ typedef struct _osm_req * * SYNOPSIS */ -void -osm_req_construct( - IN osm_req_t* const p_req ); +void osm_req_construct(IN osm_req_t * const p_req); /* * PARAMETERS * p_req @@ -170,9 +165,7 @@ osm_req_construct( * * SYNOPSIS */ -void -osm_req_destroy( - IN osm_req_t* const p_req ); +void osm_req_destroy(IN osm_req_t * const p_req); /* * PARAMETERS * p_req @@ -204,13 +197,11 @@ osm_req_destroy( * SYNOPSIS */ ib_api_status_t -osm_req_init( - IN osm_req_t* const p_req, - IN osm_mad_pool_t* const p_pool, - IN osm_vl15_t* const p_vl15, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN atomic32_t* const p_sm_trans_id ); +osm_req_init(IN osm_req_t * const p_req, + IN osm_mad_pool_t * const p_pool, + IN osm_vl15_t * const p_vl15, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN atomic32_t * const p_sm_trans_id); /* * PARAMETERS * p_req @@ -254,13 +245,12 @@ osm_req_init( * SYNOPSIS */ ib_api_status_t -osm_req_get( - IN const osm_req_t* const p_req, - IN const osm_dr_path_t* const p_path, - IN const uint16_t attr_id, - IN const uint32_t attr_mod, - IN const cl_disp_msgid_t err_msg, - IN const osm_madw_context_t* const p_context ); +osm_req_get(IN const osm_req_t * const p_req, + IN const osm_dr_path_t * const p_path, + IN const uint16_t attr_id, + IN const uint32_t attr_mod, + IN const cl_disp_msgid_t err_msg, + IN const osm_madw_context_t * const p_context); /* * PARAMETERS * p_req @@ -304,15 +294,14 @@ osm_req_get( * SYNOPSIS */ ib_api_status_t -osm_req_set( - IN const osm_req_t* const p_req, - IN const osm_dr_path_t* const p_path, - IN const uint8_t* const p_payload, - IN const size_t payload_size, - IN const uint16_t attr_id, - IN const uint32_t attr_mod, - IN const cl_disp_msgid_t err_msg, - IN const osm_madw_context_t* const p_context ); +osm_req_set(IN const osm_req_t * const p_req, + IN const osm_dr_path_t * const p_path, + IN const uint8_t * const p_payload, + IN const size_t payload_size, + IN const uint16_t attr_id, + IN const uint32_t attr_mod, + IN const cl_disp_msgid_t err_msg, + IN const osm_madw_context_t * const p_context); /* * PARAMETERS * p_req @@ -353,5 +342,4 @@ osm_req_set( *********/ END_C_DECLS - -#endif /* _OSM_REQ_H_ */ +#endif /* _OSM_REQ_H_ */ diff --git a/opensm/include/opensm/osm_resp.h b/opensm/include/opensm/osm_resp.h index f280c00..5f1060e 100644 --- a/opensm/include/opensm/osm_resp.h +++ b/opensm/include/opensm/osm_resp.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Generic Responder * NAME * Generic Responder @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Generic Responder/osm_resp_t * NAME * osm_resp_t @@ -99,12 +97,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_resp -{ - osm_mad_pool_t *p_pool; - osm_vl15_t *p_vl15; - osm_log_t *p_log; - osm_subn_t *p_subn; +typedef struct _osm_resp { + osm_mad_pool_t *p_pool; + osm_vl15_t *p_vl15; + osm_log_t *p_log; + osm_subn_t *p_subn; } osm_resp_t; /* @@ -134,9 +131,7 @@ typedef struct _osm_resp * * SYNOPSIS */ -void -osm_resp_construct( - IN osm_resp_t* const p_resp ); +void osm_resp_construct(IN osm_resp_t * const p_resp); /* * PARAMETERS * p_resp @@ -166,9 +161,7 @@ osm_resp_construct( * * SYNOPSIS */ -void -osm_resp_destroy( - IN osm_resp_t* const p_resp ); +void osm_resp_destroy(IN osm_resp_t * const p_resp); /* * PARAMETERS * p_resp @@ -200,12 +193,10 @@ osm_resp_destroy( * SYNOPSIS */ ib_api_status_t -osm_resp_init( - IN osm_resp_t* const p_resp, - IN osm_mad_pool_t* const p_pool, - IN osm_vl15_t* const p_vl15, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log ); +osm_resp_init(IN osm_resp_t * const p_resp, + IN osm_mad_pool_t * const p_pool, + IN osm_vl15_t * const p_vl15, + IN osm_subn_t * const p_subn, IN osm_log_t * const p_log); /* * PARAMETERS * p_resp @@ -245,11 +236,9 @@ osm_resp_init( * SYNOPSIS */ ib_api_status_t -osm_resp_send( - IN const osm_resp_t* const p_resp, - IN const osm_madw_t* const p_req_madw, - IN const ib_net16_t status, - IN const uint8_t* const p_payload ); +osm_resp_send(IN const osm_resp_t * const p_resp, + IN const osm_madw_t * const p_req_madw, + IN const ib_net16_t status, IN const uint8_t * const p_payload); /* * PARAMETERS * p_resp @@ -275,5 +264,4 @@ osm_resp_send( *********/ END_C_DECLS - -#endif /* _OSM_RESP_H_ */ +#endif /* _OSM_RESP_H_ */ diff --git a/opensm/include/opensm/osm_router.h b/opensm/include/opensm/osm_router.h index 65cf304..e7d2294 100644 --- a/opensm/include/opensm/osm_router.h +++ b/opensm/include/opensm/osm_router.h @@ -59,13 +59,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Router * NAME * Router @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Hal Rosenstock, Voltaire * *********/ - /****s* OpenSM: Router/osm_router_t * NAME * osm_router_t @@ -98,10 +96,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_router -{ - cl_map_item_t map_item; - osm_port_t *p_port; +typedef struct _osm_router { + cl_map_item_t map_item; + osm_port_t *p_port; } osm_router_t; /* * FIELDS @@ -124,9 +121,7 @@ typedef struct _osm_router * * SYNOPSIS */ -void -osm_router_delete( - IN OUT osm_router_t** const pp_rtr ); +void osm_router_delete(IN OUT osm_router_t ** const pp_rtr); /* * PARAMETERS * p_rtr @@ -150,9 +145,7 @@ osm_router_delete( * * SYNOPSIS */ -osm_router_t* -osm_router_new( - IN osm_port_t* const p_port ); +osm_router_t *osm_router_new(IN osm_port_t * const p_port); /* * PARAMETERS * p_node @@ -176,12 +169,12 @@ osm_router_new( * * SYNOPSIS */ -static inline osm_port_t* -osm_router_get_port_ptr( - IN const osm_router_t* const p_rtr ) +static inline osm_port_t *osm_router_get_port_ptr(IN const osm_router_t * + const p_rtr) { - return( p_rtr->p_port ); + return (p_rtr->p_port); } + /* * PARAMETERS * p_rtr @@ -205,12 +198,12 @@ osm_router_get_port_ptr( * * SYNOPSIS */ -static inline osm_node_t* -osm_router_get_node_ptr( - IN const osm_router_t* const p_rtr ) +static inline osm_node_t *osm_router_get_node_ptr(IN const osm_router_t * + const p_rtr) { - return( p_rtr->p_port->p_node ); + return (p_rtr->p_port->p_node); } + /* * PARAMETERS * p_rtr @@ -226,5 +219,4 @@ osm_router_get_node_ptr( *********/ END_C_DECLS - -#endif /* _OSM_ROUTER_H_ */ +#endif /* _OSM_ROUTER_H_ */ diff --git a/opensm/include/opensm/osm_sa.h b/opensm/include/opensm/osm_sa.h index 85d1ae0..34c66b5 100644 --- a/opensm/include/opensm/osm_sa.h +++ b/opensm/include/opensm/osm_sa.h @@ -82,13 +82,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SA * NAME * SA @@ -108,7 +107,6 @@ BEGIN_C_DECLS * Anil Keshavamurthy, Intel * *********/ - /****d* OpenSM: SA/osm_sa_state_t * NAME * osm_sa_state_t @@ -118,11 +116,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef enum _osm_sa_state -{ +typedef enum _osm_sa_state { OSM_SA_STATE_INIT = 0, OSM_SA_STATE_READY - } osm_sa_state_t; /***********/ @@ -138,72 +134,71 @@ typedef enum _osm_sa_state * * SYNOPSIS */ -typedef struct _osm_sa -{ - osm_sa_state_t state; - osm_subn_t *p_subn; - osm_vendor_t *p_vendor; - osm_log_t *p_log; - osm_mad_pool_t *p_mad_pool; - cl_dispatcher_t *p_disp; - cl_plock_t *p_lock; - atomic32_t sa_trans_id; - osm_sa_mad_ctrl_t mad_ctrl; - osm_sa_resp_t resp; - osm_cpi_rcv_t cpi_rcv; - osm_nr_rcv_t nr_rcv; - osm_pir_rcv_t pir_rcv; - osm_gir_rcv_t gir_rcv; - osm_lr_rcv_t lr_rcv; - osm_pr_rcv_t pr_rcv; - osm_smir_rcv_t smir_rcv; - osm_mcmr_recv_t mcmr_rcv; - osm_sr_rcv_t sr_rcv; +typedef struct _osm_sa { + osm_sa_state_t state; + osm_subn_t *p_subn; + osm_vendor_t *p_vendor; + osm_log_t *p_log; + osm_mad_pool_t *p_mad_pool; + cl_dispatcher_t *p_disp; + cl_plock_t *p_lock; + atomic32_t sa_trans_id; + osm_sa_mad_ctrl_t mad_ctrl; + osm_sa_resp_t resp; + osm_cpi_rcv_t cpi_rcv; + osm_nr_rcv_t nr_rcv; + osm_pir_rcv_t pir_rcv; + osm_gir_rcv_t gir_rcv; + osm_lr_rcv_t lr_rcv; + osm_pr_rcv_t pr_rcv; + osm_smir_rcv_t smir_rcv; + osm_mcmr_recv_t mcmr_rcv; + osm_sr_rcv_t sr_rcv; #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - osm_mpr_rcv_t mpr_rcv; + osm_mpr_rcv_t mpr_rcv; #endif /* InformInfo Receiver */ - osm_infr_rcv_t infr_rcv; + osm_infr_rcv_t infr_rcv; /* VL Arbitrartion Query */ - osm_vlarb_rec_rcv_t vlarb_rec_rcv; + osm_vlarb_rec_rcv_t vlarb_rec_rcv; /* SLtoVL Map Query */ - osm_slvl_rec_rcv_t slvl_rec_rcv; + osm_slvl_rec_rcv_t slvl_rec_rcv; /* P_Key table Query */ - osm_pkey_rec_rcv_t pkey_rec_rcv; + osm_pkey_rec_rcv_t pkey_rec_rcv; /* LinearForwardingTable Query */ - osm_lftr_rcv_t lftr_rcv; + osm_lftr_rcv_t lftr_rcv; /* SwitchInfo Query */ - osm_sir_rcv_t sir_rcv; + osm_sir_rcv_t sir_rcv; /* MulticastForwardingTable Query */ - osm_mftr_rcv_t mftr_rcv; + osm_mftr_rcv_t mftr_rcv; - cl_disp_reg_handle_t cpi_disp_h; - cl_disp_reg_handle_t nr_disp_h; - cl_disp_reg_handle_t pir_disp_h; - cl_disp_reg_handle_t gir_disp_h; - cl_disp_reg_handle_t lr_disp_h; - cl_disp_reg_handle_t pr_disp_h; - cl_disp_reg_handle_t smir_disp_h; - cl_disp_reg_handle_t mcmr_disp_h; - cl_disp_reg_handle_t sr_disp_h; + cl_disp_reg_handle_t cpi_disp_h; + cl_disp_reg_handle_t nr_disp_h; + cl_disp_reg_handle_t pir_disp_h; + cl_disp_reg_handle_t gir_disp_h; + cl_disp_reg_handle_t lr_disp_h; + cl_disp_reg_handle_t pr_disp_h; + cl_disp_reg_handle_t smir_disp_h; + cl_disp_reg_handle_t mcmr_disp_h; + cl_disp_reg_handle_t sr_disp_h; #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - cl_disp_reg_handle_t mpr_disp_h; + cl_disp_reg_handle_t mpr_disp_h; #endif - cl_disp_reg_handle_t infr_disp_h; - cl_disp_reg_handle_t infir_disp_h; - cl_disp_reg_handle_t vlarb_disp_h; - cl_disp_reg_handle_t slvl_disp_h; - cl_disp_reg_handle_t pkey_disp_h; - cl_disp_reg_handle_t lft_disp_h; - cl_disp_reg_handle_t sir_disp_h; - cl_disp_reg_handle_t mft_disp_h; + cl_disp_reg_handle_t infr_disp_h; + cl_disp_reg_handle_t infir_disp_h; + cl_disp_reg_handle_t vlarb_disp_h; + cl_disp_reg_handle_t slvl_disp_h; + cl_disp_reg_handle_t pkey_disp_h; + cl_disp_reg_handle_t lft_disp_h; + cl_disp_reg_handle_t sir_disp_h; + cl_disp_reg_handle_t mft_disp_h; } osm_sa_t; /* * FIELDS @@ -259,8 +254,7 @@ typedef struct _osm_sa * * SYNOPSIS */ -void osm_sa_construct( - IN osm_sa_t* const p_sa ); +void osm_sa_construct(IN osm_sa_t * const p_sa); /* * PARAMETERS * p_sa @@ -289,8 +283,7 @@ void osm_sa_construct( * * SYNOPSIS */ -void osm_sa_shutdown( - IN osm_sa_t* const p_sa ); +void osm_sa_shutdown(IN osm_sa_t * const p_sa); /* * PARAMETERS * p_sa @@ -313,8 +306,7 @@ void osm_sa_shutdown( * * SYNOPSIS */ -void osm_sa_destroy( - IN osm_sa_t* const p_sa ); +void osm_sa_destroy(IN osm_sa_t * const p_sa); /* * PARAMETERS * p_sa @@ -342,16 +334,15 @@ void osm_sa_destroy( * * SYNOPSIS */ -ib_api_status_t osm_sa_init( - IN osm_sm_t* const p_sm, - IN osm_sa_t* const p_sa, - IN osm_subn_t* const p_subn, - IN osm_vendor_t* const p_vendor, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_dispatcher_t* const p_disp, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_sa_init(IN osm_sm_t * const p_sm, + IN osm_sa_t * const p_sa, + IN osm_subn_t * const p_subn, + IN osm_vendor_t * const p_vendor, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_sa @@ -398,8 +389,7 @@ ib_api_status_t osm_sa_init( * * SYNOPSIS */ -boolean_t osm_sa_is_inited( - IN const osm_sa_t* const p_sa ); +boolean_t osm_sa_is_inited(IN const osm_sa_t * const p_sa); /* * PARAMETERS * p_sa @@ -427,9 +417,7 @@ boolean_t osm_sa_is_inited( * SYNOPSIS */ ib_api_status_t -osm_sa_bind( - IN osm_sa_t* const p_sa, - IN const ib_net64_t port_guid ); +osm_sa_bind(IN osm_sa_t * const p_sa, IN const ib_net64_t port_guid); /* * PARAMETERS * p_sa @@ -490,5 +478,4 @@ int osm_sa_db_file_load(struct _osm_opensm_t *p_osm); *********/ END_C_DECLS - -#endif /* _OSM_SA_H_ */ +#endif /* _OSM_SA_H_ */ diff --git a/opensm/include/opensm/osm_sa_class_port_info.h b/opensm/include/opensm/osm_sa_class_port_info.h index ca159a2..b477cd5 100644 --- a/opensm/include/opensm/osm_sa_class_port_info.h +++ b/opensm/include/opensm/osm_sa_class_port_info.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/ClassPort Info Receiver * NAME * ClassPort Info Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: ClassPort Info Receiver/osm_cpi_rcv_t * NAME * osm_cpi_rcv_t @@ -98,13 +96,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_cpi_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_cpi_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_cpi_rcv_t; /* * FIELDS @@ -133,9 +130,7 @@ typedef struct _osm_cpi_rcv * * SYNOPSIS */ -void -osm_cpi_rcv_construct( - IN osm_cpi_rcv_t* const p_rcv ); +void osm_cpi_rcv_construct(IN osm_cpi_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -164,9 +159,7 @@ osm_cpi_rcv_construct( * * SYNOPSIS */ -void -osm_cpi_rcv_destroy( - IN osm_cpi_rcv_t* const p_rcv ); +void osm_cpi_rcv_destroy(IN osm_cpi_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -198,13 +191,11 @@ osm_cpi_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_cpi_rcv_init( - IN osm_cpi_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_cpi_rcv_init(IN osm_cpi_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -240,10 +231,7 @@ osm_cpi_rcv_init( * * SYNOPSIS */ -void -osm_cpi_rcv_process( - IN void *context, - IN void *data ); +void osm_cpi_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -264,5 +252,4 @@ osm_cpi_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_CPI_H_ */ +#endif /* _OSM_CPI_H_ */ diff --git a/opensm/include/opensm/osm_sa_guidinfo_record.h b/opensm/include/opensm/osm_sa_guidinfo_record.h index e80f27d..dcdf896 100644 --- a/opensm/include/opensm/osm_sa_guidinfo_record.h +++ b/opensm/include/opensm/osm_sa_guidinfo_record.h @@ -59,13 +59,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/GUIDInfo Record Receiver * NAME * GUIDInfo Record Receiver @@ -83,7 +82,6 @@ BEGIN_C_DECLS * Hal Rosenstock, Voltaire * *********/ - /****s* OpenSM: GUIDInfo Record Receiver/osm_gir_rcv_t * NAME * osm_gir_rcv_t @@ -96,14 +94,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_gir_rcv -{ - const osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_gir_rcv { + const osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_gir_rcv_t; /* * FIELDS @@ -139,9 +136,7 @@ typedef struct _osm_gir_rcv * * SYNOPSIS */ -void -osm_gir_rcv_construct( - IN osm_gir_rcv_t* const p_rcv ); +void osm_gir_rcv_construct(IN osm_gir_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -171,9 +166,7 @@ osm_gir_rcv_construct( * * SYNOPSIS */ -void -osm_gir_rcv_destroy( - IN osm_gir_rcv_t* const p_rcv ); +void osm_gir_rcv_destroy(IN osm_gir_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -205,13 +198,11 @@ osm_gir_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_gir_rcv_init( - IN osm_gir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_gir_rcv_init(IN osm_gir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -250,10 +241,7 @@ osm_gir_rcv_init( * * SYNOPSIS */ -void -osm_gir_rcv_process( - IN void *context, - IN void *data ); +void osm_gir_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -274,5 +262,4 @@ osm_gir_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_GIR_RCV_H_ */ +#endif /* _OSM_GIR_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_informinfo.h b/opensm/include/opensm/osm_sa_informinfo.h index c935995..5d00dd6 100644 --- a/opensm/include/opensm/osm_sa_informinfo.h +++ b/opensm/include/opensm/osm_sa_informinfo.h @@ -63,13 +63,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/InformInfo Receiver * NAME * InformInfo Receiver @@ -87,7 +86,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: InformInfo Receiver/osm_infr_rcv_t * NAME * osm_infr_rcv_t @@ -100,14 +98,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_infr_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_infr_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_infr_rcv_t; /* * FIELDS @@ -140,9 +137,7 @@ typedef struct _osm_infr_rcv * * SYNOPSIS */ -void -osm_infr_rcv_construct( - IN osm_infr_rcv_t* const p_rcv ); +void osm_infr_rcv_construct(IN osm_infr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -171,9 +166,7 @@ osm_infr_rcv_construct( * * SYNOPSIS */ -void -osm_infr_rcv_destroy( - IN osm_infr_rcv_t* const p_rcv ); +void osm_infr_rcv_destroy(IN osm_infr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -205,13 +198,11 @@ osm_infr_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_infr_rcv_init( - IN osm_infr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_infr_rcv_init(IN osm_infr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -247,10 +238,7 @@ osm_infr_rcv_init( * * SYNOPSIS */ -void -osm_infr_rcv_process( - IN void *context, - IN void *data ); +void osm_infr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -275,10 +263,7 @@ osm_infr_rcv_process( * * SYNOPSIS */ -void -osm_infir_rcv_process( - IN void *context, - IN void *data ); +void osm_infir_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -295,5 +280,4 @@ osm_infir_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SA_INFR_H_ */ +#endif /* _OSM_SA_INFR_H_ */ diff --git a/opensm/include/opensm/osm_sa_lft_record.h b/opensm/include/opensm/osm_sa_lft_record.h index 4361d2d..18a43f4 100644 --- a/opensm/include/opensm/osm_sa_lft_record.h +++ b/opensm/include/opensm/osm_sa_lft_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Linear Forwarding Table Receiver * NAME * Linear Forwarding Table Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox Technologies LTD * *********/ - /****s* OpenSM: Linear Forwarding Table Receiver/osm_lftr_rcv_t * NAME * osm_lftr_rcv_t @@ -98,15 +96,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_lft -{ - osm_subn_t* p_subn; - osm_stats_t* p_stats; - osm_sa_resp_t* p_resp; - osm_mad_pool_t* p_mad_pool; - osm_log_t* p_log; - cl_plock_t* p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_lft { + osm_subn_t *p_subn; + osm_stats_t *p_stats; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_lftr_rcv_t; /* * FIELDS @@ -145,8 +142,7 @@ typedef struct _osm_lft * * SYNOPSIS */ -void osm_lftr_rcv_construct( - IN osm_lftr_rcv_t* const p_ctrl ); +void osm_lftr_rcv_construct(IN osm_lftr_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -176,8 +172,7 @@ void osm_lftr_rcv_construct( * * SYNOPSIS */ -void osm_lftr_rcv_destroy( - IN osm_lftr_rcv_t* const p_ctrl ); +void osm_lftr_rcv_destroy(IN osm_lftr_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -208,13 +203,12 @@ void osm_lftr_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_lftr_rcv_init( - IN osm_lftr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_lftr_rcv_init(IN osm_lftr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -253,9 +247,7 @@ ib_api_status_t osm_lftr_rcv_init( * * SYNOPSIS */ -void osm_lftr_rcv_process( - IN void *context, - IN void *data ); +void osm_lftr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -277,5 +269,4 @@ void osm_lftr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_LFTR_H_ */ +#endif /* _OSM_LFTR_H_ */ diff --git a/opensm/include/opensm/osm_sa_link_record.h b/opensm/include/opensm/osm_sa_link_record.h index adc3be8..3104704 100644 --- a/opensm/include/opensm/osm_sa_link_record.h +++ b/opensm/include/opensm/osm_sa_link_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Link Record Receiver * NAME * Link Record Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Ranjit Pandit, Intel * *********/ - /****s* OpenSM: Link Record Receiver/osm_lr_rcv_t * NAME * osm_lr_rcv_t @@ -98,14 +96,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_lr_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t lr_pool; +typedef struct _osm_lr_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t lr_pool; } osm_lr_rcv_t; /* * FIELDS @@ -139,9 +136,7 @@ typedef struct _osm_lr_rcv * * SYNOPSIS */ -void -osm_lr_rcv_construct( - IN osm_lr_rcv_t* const p_rcv ); +void osm_lr_rcv_construct(IN osm_lr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -170,9 +165,7 @@ osm_lr_rcv_construct( * * SYNOPSIS */ -void -osm_lr_rcv_destroy( - IN osm_lr_rcv_t* const p_rcv ); +void osm_lr_rcv_destroy(IN osm_lr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -204,13 +197,11 @@ osm_lr_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_lr_rcv_init( - IN osm_lr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_lr_rcv_init(IN osm_lr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -251,9 +242,7 @@ osm_lr_rcv_init( * * SYNOPSIS */ -void osm_lr_rcv_process( - IN void *context, - IN void *data ); +void osm_lr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -271,5 +260,4 @@ void osm_lr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_LR_RCV_H_ */ +#endif /* _OSM_LR_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_mad_ctrl.h b/opensm/include/opensm/osm_sa_mad_ctrl.h index ca99f25..bd7751e 100644 --- a/opensm/include/opensm/osm_sa_mad_ctrl.h +++ b/opensm/include/opensm/osm_sa_mad_ctrl.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SA MAD Controller * NAME * SA MAD Controller @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Ranjit Pandit, Intel * *********/ - /****s* OpenSM: SA MAD Controller/osm_sa_mad_ctrl_t * NAME * osm_sa_mad_ctrl_t @@ -97,17 +95,16 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sa_mad_ctrl -{ - osm_log_t *p_log; - osm_mad_pool_t *p_mad_pool; - osm_vendor_t *p_vendor; - osm_bind_handle_t h_bind; - cl_dispatcher_t *p_disp; - cl_disp_reg_handle_t h_disp; - osm_stats_t *p_stats; - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; +typedef struct _osm_sa_mad_ctrl { + osm_log_t *p_log; + osm_mad_pool_t *p_mad_pool; + osm_vendor_t *p_vendor; + osm_bind_handle_t h_bind; + cl_dispatcher_t *p_disp; + cl_disp_reg_handle_t h_disp; + osm_stats_t *p_stats; + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; } osm_sa_mad_ctrl_t; /* * FIELDS @@ -149,8 +146,7 @@ typedef struct _osm_sa_mad_ctrl * * SYNOPSIS */ -void osm_sa_mad_ctrl_construct( - IN osm_sa_mad_ctrl_t* const p_ctrl ); +void osm_sa_mad_ctrl_construct(IN osm_sa_mad_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -181,8 +177,7 @@ void osm_sa_mad_ctrl_construct( * * SYNOPSIS */ -void osm_sa_mad_ctrl_destroy( - IN osm_sa_mad_ctrl_t* const p_ctrl ); +void osm_sa_mad_ctrl_destroy(IN osm_sa_mad_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -213,15 +208,14 @@ void osm_sa_mad_ctrl_destroy( * * SYNOPSIS */ -ib_api_status_t osm_sa_mad_ctrl_init( - IN osm_sa_mad_ctrl_t* const p_ctrl, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vendor_t* const p_vendor, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_dispatcher_t* const p_disp ); +ib_api_status_t osm_sa_mad_ctrl_init(IN osm_sa_mad_ctrl_t * const p_ctrl, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vendor_t * const p_vendor, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_ctrl @@ -267,9 +261,8 @@ ib_api_status_t osm_sa_mad_ctrl_init( * SYNOPSIS */ ib_api_status_t -osm_sa_mad_ctrl_bind( - IN osm_sa_mad_ctrl_t* const p_ctrl, - IN const ib_net64_t port_guid ); +osm_sa_mad_ctrl_bind(IN osm_sa_mad_ctrl_t * const p_ctrl, + IN const ib_net64_t port_guid); /* * PARAMETERS * p_ctrl @@ -298,9 +291,7 @@ osm_sa_mad_ctrl_bind( * * SYNOPSIS */ -ib_api_status_t -osm_sa_mad_ctrl_unbind( - IN osm_sa_mad_ctrl_t* const p_ctrl); +ib_api_status_t osm_sa_mad_ctrl_unbind(IN osm_sa_mad_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -326,11 +317,11 @@ osm_sa_mad_ctrl_unbind( * SYNOPSIS */ static inline osm_bind_handle_t -osm_sa_mad_ctrl_get_bind_handle( - IN const osm_sa_mad_ctrl_t* const p_ctrl ) +osm_sa_mad_ctrl_get_bind_handle(IN const osm_sa_mad_ctrl_t * const p_ctrl) { - return( p_ctrl->h_bind ); + return (p_ctrl->h_bind); } + /* * PARAMETERS * p_ctrl @@ -348,5 +339,4 @@ osm_sa_mad_ctrl_get_bind_handle( *********/ END_C_DECLS - -#endif /* _OSM_SA_MAD_CTRL_H_ */ +#endif /* _OSM_SA_MAD_CTRL_H_ */ diff --git a/opensm/include/opensm/osm_sa_mcmember_record.h b/opensm/include/opensm/osm_sa_mcmember_record.h index 3a119c5..a7102ca 100644 --- a/opensm/include/opensm/osm_sa_mcmember_record.h +++ b/opensm/include/opensm/osm_sa_mcmember_record.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/MCMember Receiver * NAME * MCMember Receiver @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Anil Keshavamurthy, Intel * *********/ - /****s* OpenSM: MCMember Receiver/osm_mcmr_recv_t * NAME * osm_mcmr_recv_t @@ -99,21 +97,17 @@ BEGIN_C_DECLS * * SYNOPSIS */ - - -typedef struct _osm_mcmr -{ - osm_subn_t *p_subn; - osm_sm_t *p_sm; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - uint16_t mlid_ho; - cl_qlock_pool_t pool; +typedef struct _osm_mcmr { + osm_subn_t *p_subn; + osm_sm_t *p_sm; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + uint16_t mlid_ho; + cl_qlock_pool_t pool; } osm_mcmr_recv_t; - /* * FIELDS * p_subn @@ -141,8 +135,7 @@ typedef struct _osm_mcmr * * SYNOPSIS */ -void osm_mcmr_rcv_construct( - IN osm_mcmr_recv_t* const p_ctrl ); +void osm_mcmr_rcv_construct(IN osm_mcmr_recv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -172,8 +165,7 @@ void osm_mcmr_rcv_construct( * * SYNOPSIS */ -void osm_mcmr_rcv_destroy( - IN osm_mcmr_recv_t* const p_ctrl ); +void osm_mcmr_rcv_destroy(IN osm_mcmr_recv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -204,14 +196,13 @@ void osm_mcmr_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_mcmr_rcv_init( - IN osm_sm_t * const p_sm, - IN osm_mcmr_recv_t* const p_ctrl, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_mcmr_rcv_init(IN osm_sm_t * const p_sm, + IN osm_mcmr_recv_t * const p_ctrl, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_sm @@ -252,9 +243,7 @@ ib_api_status_t osm_mcmr_rcv_init( * * SYNOPSIS */ -void osm_mcmr_rcv_process( - IN void *context, - IN void *data ); +void osm_mcmr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -274,8 +263,6 @@ void osm_mcmr_rcv_process( * MCMember Receiver, MCMember Response Controller *********/ - - /****f* OpenSM: MC Member Record Receiver/osm_mcmr_rcv_create_new_mgrp * NAME * osm_mcmr_rcv_create_new_mgrp @@ -287,12 +274,12 @@ void osm_mcmr_rcv_process( */ ib_api_status_t -osm_mcmr_rcv_create_new_mgrp( - IN osm_mcmr_recv_t* const p_mcmr, - IN uint64_t comp_mask, - IN const ib_member_rec_t* const p_recvd_mcmember_rec, - IN const osm_physp_t* const p_req_physp, - OUT osm_mgrp_t **pp_mgrp); +osm_mcmr_rcv_create_new_mgrp(IN osm_mcmr_recv_t * const p_mcmr, + IN uint64_t comp_mask, + IN const ib_member_rec_t * + const p_recvd_mcmember_rec, + IN const osm_physp_t * const p_req_physp, + OUT osm_mgrp_t ** pp_mgrp); /* * PARAMETERS * p_mcmr @@ -328,11 +315,11 @@ osm_mcmr_rcv_create_new_mgrp( */ ib_api_status_t -osm_mcmr_rcv_find_or_create_new_mgrp( - IN osm_mcmr_recv_t* const p_mcmr, - IN uint64_t comp_mask, - IN ib_member_rec_t* const p_recvd_mcmember_rec, - OUT osm_mgrp_t **pp_mgrp); +osm_mcmr_rcv_find_or_create_new_mgrp(IN osm_mcmr_recv_t * const p_mcmr, + IN uint64_t comp_mask, + IN ib_member_rec_t * + const p_recvd_mcmember_rec, + OUT osm_mgrp_t ** pp_mgrp); /* * PARAMETERS * p_mcmr @@ -416,5 +403,4 @@ osm_mcmr_rcv_find_or_create_new_mgrp( #define MC_SENDONLY_NON_MEMBER 0x4 END_C_DECLS - -#endif /* _OSM_MCMR_H_ */ +#endif /* _OSM_MCMR_H_ */ diff --git a/opensm/include/opensm/osm_sa_mft_record.h b/opensm/include/opensm/osm_sa_mft_record.h index e4c04c6..dd14257 100644 --- a/opensm/include/opensm/osm_sa_mft_record.h +++ b/opensm/include/opensm/osm_sa_mft_record.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Multicast Forwarding Table Receiver * NAME * Multicast Forwarding Table Receiver @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Hal Rosenstock, Voltaire * *********/ - /****s* OpenSM: Multicast Forwarding Table Receiver/osm_mftr_rcv_t * NAME * osm_mftr_rcv_t @@ -97,15 +95,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mft -{ - osm_subn_t* p_subn; - osm_stats_t* p_stats; - osm_sa_resp_t* p_resp; - osm_mad_pool_t* p_mad_pool; - osm_log_t* p_log; - cl_plock_t* p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_mft { + osm_subn_t *p_subn; + osm_stats_t *p_stats; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_mftr_rcv_t; /* * FIELDS @@ -144,8 +141,7 @@ typedef struct _osm_mft * * SYNOPSIS */ -void osm_mftr_rcv_construct( - IN osm_mftr_rcv_t* const p_ctrl ); +void osm_mftr_rcv_construct(IN osm_mftr_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -175,8 +171,7 @@ void osm_mftr_rcv_construct( * * SYNOPSIS */ -void osm_mftr_rcv_destroy( - IN osm_mftr_rcv_t* const p_ctrl ); +void osm_mftr_rcv_destroy(IN osm_mftr_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -207,13 +202,12 @@ void osm_mftr_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_mftr_rcv_init( - IN osm_mftr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_mftr_rcv_init(IN osm_mftr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -252,9 +246,7 @@ ib_api_status_t osm_mftr_rcv_init( * * SYNOPSIS */ -void osm_mftr_rcv_process( - IN void *context, - IN void *data ); +void osm_mftr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -276,5 +268,4 @@ void osm_mftr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_MFTR_H_ */ +#endif /* _OSM_MFTR_H_ */ diff --git a/opensm/include/opensm/osm_sa_multipath_record.h b/opensm/include/opensm/osm_sa_multipath_record.h index 6f99ec9..8fa1046 100644 --- a/opensm/include/opensm/osm_sa_multipath_record.h +++ b/opensm/include/opensm/osm_sa_multipath_record.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/MultiPath Record Receiver * NAME * MultiPath Record Receiver @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Hal Rosenstock, Voltaire * *********/ - /****s* OpenSM: MultiPath Record Receiver/osm_mpr_rcv_t * NAME * osm_mpr_rcv_t @@ -99,14 +97,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_mpr_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pr_pool; +typedef struct _osm_mpr_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pr_pool; } osm_mpr_rcv_t; /* * FIELDS @@ -138,9 +135,7 @@ typedef struct _osm_mpr_rcv * * SYNOPSIS */ -void -osm_mpr_rcv_construct( - IN osm_mpr_rcv_t* const p_rcv ); +void osm_mpr_rcv_construct(IN osm_mpr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -169,9 +164,7 @@ osm_mpr_rcv_construct( * * SYNOPSIS */ -void -osm_mpr_rcv_destroy( - IN osm_mpr_rcv_t* const p_rcv ); +void osm_mpr_rcv_destroy(IN osm_mpr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -203,13 +196,11 @@ osm_mpr_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_mpr_rcv_init( - IN osm_mpr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_mpr_rcv_init(IN osm_mpr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -245,10 +236,7 @@ osm_mpr_rcv_init( * * SYNOPSIS */ -void -osm_mpr_rcv_process( - IN void *context, - IN void *data ); +void osm_mpr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -269,5 +257,4 @@ osm_mpr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_MPR_RCV_H_ */ +#endif /* _OSM_MPR_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_node_record.h b/opensm/include/opensm/osm_sa_node_record.h index 4ad5d3b..df95734 100644 --- a/opensm/include/opensm/osm_sa_node_record.h +++ b/opensm/include/opensm/osm_sa_node_record.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Node Record Receiver * NAME * Node Record Receiver @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Anil S Keshavamurthy, Intel * *********/ - /****s* OpenSM: Node Record Receiver/osm_nr_rcv_t * NAME * osm_nr_rcv_t @@ -97,14 +95,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_nr_recv -{ - const osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_nr_recv { + const osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_nr_rcv_t; /* * FIELDS @@ -140,8 +137,7 @@ typedef struct _osm_nr_recv * * SYNOPSIS */ -void osm_nr_rcv_construct( - IN osm_nr_rcv_t* const p_rcv ); +void osm_nr_rcv_construct(IN osm_nr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -170,8 +166,7 @@ void osm_nr_rcv_construct( * * SYNOPSIS */ -void osm_nr_rcv_destroy( - IN osm_nr_rcv_t* const p_rcv ); +void osm_nr_rcv_destroy(IN osm_nr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -202,13 +197,12 @@ void osm_nr_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_nr_rcv_init( - IN osm_nr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_nr_rcv_init(IN osm_nr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -240,7 +234,6 @@ ib_api_status_t osm_nr_rcv_init( * Node Record Receiver object, osm_nr_rcv_construct, osm_nr_rcv_destroy *********/ - /****f* OpenSM: Node Record Receiver/osm_nr_rcv_process * NAME * osm_nr_rcv_process @@ -250,9 +243,7 @@ ib_api_status_t osm_nr_rcv_init( * * SYNOPSIS */ -void osm_nr_rcv_process( - IN void *context, - IN void *data ); +void osm_nr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -270,5 +261,4 @@ void osm_nr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_NR_H_ */ +#endif /* _OSM_NR_H_ */ diff --git a/opensm/include/opensm/osm_sa_path_record.h b/opensm/include/opensm/osm_sa_path_record.h index 29667ef..88eb6c3 100644 --- a/opensm/include/opensm/osm_sa_path_record.h +++ b/opensm/include/opensm/osm_sa_path_record.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Path Record Receiver * NAME * Path Record Receiver @@ -87,7 +86,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Path Record Receiver/osm_pr_rcv_t * NAME * osm_pr_rcv_t @@ -100,14 +98,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pr_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pr_pool; +typedef struct _osm_pr_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pr_pool; } osm_pr_rcv_t; /* * FIELDS @@ -139,9 +136,7 @@ typedef struct _osm_pr_rcv * * SYNOPSIS */ -void -osm_pr_rcv_construct( - IN osm_pr_rcv_t* const p_rcv ); +void osm_pr_rcv_construct(IN osm_pr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -170,9 +165,7 @@ osm_pr_rcv_construct( * * SYNOPSIS */ -void -osm_pr_rcv_destroy( - IN osm_pr_rcv_t* const p_rcv ); +void osm_pr_rcv_destroy(IN osm_pr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -204,13 +197,11 @@ osm_pr_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_pr_rcv_init( - IN osm_pr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_pr_rcv_init(IN osm_pr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -246,10 +237,7 @@ osm_pr_rcv_init( * * SYNOPSIS */ -void -osm_pr_rcv_process( - IN void *context, - IN void *data ); +void osm_pr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -270,5 +258,4 @@ osm_pr_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_PR_H_ */ +#endif /* _OSM_PR_H_ */ diff --git a/opensm/include/opensm/osm_sa_pkey_record.h b/opensm/include/opensm/osm_sa_pkey_record.h index aceab9a..660804c 100644 --- a/opensm/include/opensm/osm_sa_pkey_record.h +++ b/opensm/include/opensm/osm_sa_pkey_record.h @@ -48,13 +48,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/P_Key Record Receiver * NAME * P_Key Record Receiver @@ -72,7 +71,6 @@ BEGIN_C_DECLS * Yael Kalka, Mellanox * *********/ - /****s* OpenSM: P_Key Record Receiver/osm_pkey_rec_rcv_t * NAME * osm_pkey_rec_rcv_t @@ -85,14 +83,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pkey_rec_rcv -{ - const osm_subn_t* p_subn; - osm_sa_resp_t* p_resp; - osm_mad_pool_t* p_mad_pool; - osm_log_t* p_log; - cl_plock_t* p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_pkey_rec_rcv { + const osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_pkey_rec_rcv_t; /* * FIELDS @@ -128,9 +125,7 @@ typedef struct _osm_pkey_rec_rcv * * SYNOPSIS */ -void -osm_pkey_rec_rcv_construct( - IN osm_pkey_rec_rcv_t* const p_rcv ); +void osm_pkey_rec_rcv_construct(IN osm_pkey_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -160,9 +155,7 @@ osm_pkey_rec_rcv_construct( * * SYNOPSIS */ -void -osm_pkey_rec_rcv_destroy( - IN osm_pkey_rec_rcv_t* const p_rcv ); +void osm_pkey_rec_rcv_destroy(IN osm_pkey_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -194,13 +187,11 @@ osm_pkey_rec_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_pkey_rec_rcv_init( - IN osm_pkey_rec_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_pkey_rec_rcv_init(IN osm_pkey_rec_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -239,10 +230,7 @@ osm_pkey_rec_rcv_init( * * SYNOPSIS */ -void -osm_pkey_rec_rcv_process( - IN void *context, - IN void *data ); +void osm_pkey_rec_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -263,5 +251,4 @@ osm_pkey_rec_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_PKEY_REC_RCV_H_ */ +#endif /* _OSM_PKEY_REC_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_portinfo_record.h b/opensm/include/opensm/osm_sa_portinfo_record.h index d3bbebe..38eabdb 100644 --- a/opensm/include/opensm/osm_sa_portinfo_record.h +++ b/opensm/include/opensm/osm_sa_portinfo_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/PortInfo Record Receiver * NAME * PortInfo Record Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Ranjit Pandit, Intel * *********/ - /****s* OpenSM: PortInfo Record Receiver/osm_pir_rcv_t * NAME * osm_pir_rcv_t @@ -98,14 +96,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_pir_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_pir_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_pir_rcv_t; /* * FIELDS @@ -141,9 +138,7 @@ typedef struct _osm_pir_rcv * * SYNOPSIS */ -void -osm_pir_rcv_construct( - IN osm_pir_rcv_t* const p_rcv ); +void osm_pir_rcv_construct(IN osm_pir_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -173,9 +168,7 @@ osm_pir_rcv_construct( * * SYNOPSIS */ -void -osm_pir_rcv_destroy( - IN osm_pir_rcv_t* const p_rcv ); +void osm_pir_rcv_destroy(IN osm_pir_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -207,13 +200,11 @@ osm_pir_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_pir_rcv_init( - IN osm_pir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_pir_rcv_init(IN osm_pir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -252,10 +243,7 @@ osm_pir_rcv_init( * * SYNOPSIS */ -void -osm_pir_rcv_process( - IN void *context, - IN void *data ); +void osm_pir_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -276,5 +264,4 @@ osm_pir_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_PIR_RCV_H_ */ +#endif /* _OSM_PIR_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_response.h b/opensm/include/opensm/osm_sa_response.h index b9e84d1..8f75dbb 100644 --- a/opensm/include/opensm/osm_sa_response.h +++ b/opensm/include/opensm/osm_sa_response.h @@ -56,13 +56,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SA Response * NAME * SA Response @@ -81,7 +80,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: SA Response/osm_sa_resp_t * NAME * osm_sa_resp_t @@ -94,10 +92,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sa_resp -{ - osm_mad_pool_t *p_pool; - osm_log_t *p_log; +typedef struct _osm_sa_resp { + osm_mad_pool_t *p_pool; + osm_log_t *p_log; } osm_sa_resp_t; /* * FIELDS @@ -117,9 +114,7 @@ typedef struct _osm_sa_resp * * SYNOPSIS */ -void -osm_sa_resp_construct( - IN osm_sa_resp_t* const p_resp ); +void osm_sa_resp_construct(IN osm_sa_resp_t * const p_resp); /* * PARAMETERS * p_resp @@ -149,9 +144,7 @@ osm_sa_resp_construct( * * SYNOPSIS */ -void -osm_sa_resp_destroy( - IN osm_sa_resp_t* const p_resp ); +void osm_sa_resp_destroy(IN osm_sa_resp_t * const p_resp); /* * PARAMETERS * p_resp @@ -183,10 +176,8 @@ osm_sa_resp_destroy( * SYNOPSIS */ ib_api_status_t -osm_sa_resp_init( - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_pool, - IN osm_log_t* const p_log ); +osm_sa_resp_init(IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_pool, IN osm_log_t * const p_log); /* * PARAMETERS * p_resp @@ -224,10 +215,9 @@ osm_sa_resp_init( * SYNOPSIS */ void -osm_sa_send_error( - IN osm_sa_resp_t* const p_resp, - IN const osm_madw_t* const p_madw, - IN const ib_net16_t sa_status ); +osm_sa_send_error(IN osm_sa_resp_t * const p_resp, + IN const osm_madw_t * const p_madw, + IN const ib_net16_t sa_status); /* * PARAMETERS * p_resp @@ -251,5 +241,4 @@ osm_sa_send_error( *********/ END_C_DECLS - -#endif /* _OSM_SA_RESP_H_ */ +#endif /* _OSM_SA_RESP_H_ */ diff --git a/opensm/include/opensm/osm_sa_service_record.h b/opensm/include/opensm/osm_sa_service_record.h index d485509..8884944 100644 --- a/opensm/include/opensm/osm_sa_service_record.h +++ b/opensm/include/opensm/osm_sa_service_record.h @@ -63,13 +63,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Service Record Receiver * NAME * Service Record Receiver @@ -87,7 +86,6 @@ BEGIN_C_DECLS * Anil S Keshavamurthy * *********/ - /****s* OpenSM: Service Record Receiver/osm_sr_rcv_t * NAME * osm_sr_rcv_t @@ -100,15 +98,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sr_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t sr_pool; - cl_timer_t sr_timer; +typedef struct _osm_sr_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t sr_pool; + cl_timer_t sr_timer; } osm_sr_rcv_t; /* * FIELDS @@ -140,9 +137,7 @@ typedef struct _osm_sr_rcv * * SYNOPSIS */ -void -osm_sr_rcv_construct( - IN osm_sr_rcv_t* const p_rcv ); +void osm_sr_rcv_construct(IN osm_sr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -171,9 +166,7 @@ osm_sr_rcv_construct( * * SYNOPSIS */ -void -osm_sr_rcv_destroy( - IN osm_sr_rcv_t* const p_rcv ); +void osm_sr_rcv_destroy(IN osm_sr_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -205,13 +198,11 @@ osm_sr_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_sr_rcv_init( - IN osm_sr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_sr_rcv_init(IN osm_sr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -247,10 +238,7 @@ osm_sr_rcv_init( * * SYNOPSIS */ -void -osm_sr_rcv_process( - IN void *context, - IN void *data ); +void osm_sr_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -277,9 +265,7 @@ osm_sr_rcv_process( * SYNOPSIS */ -void -osm_sr_rcv_lease_cb( - IN void* context ); +void osm_sr_rcv_lease_cb(IN void *context); /* * PARAMETERS * context @@ -293,5 +279,4 @@ osm_sr_rcv_lease_cb( *********/ END_C_DECLS - -#endif /* _OSM_SR_H_ */ +#endif /* _OSM_SR_H_ */ diff --git a/opensm/include/opensm/osm_sa_slvl_record.h b/opensm/include/opensm/osm_sa_slvl_record.h index bf8c867..ec7cad0 100644 --- a/opensm/include/opensm/osm_sa_slvl_record.h +++ b/opensm/include/opensm/osm_sa_slvl_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SLtoVL Mapping Record Receiver * NAME * SLtoVL Mapping Record Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: SLtoVL Mapping Record Receiver/osm_slvl_rec_rcv_t * NAME * osm_slvl_rec_rcv_t @@ -98,14 +96,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_slvl_rec_rcv -{ - const osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_slvl_rec_rcv { + const osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_slvl_rec_rcv_t; /* * FIELDS @@ -141,9 +138,7 @@ typedef struct _osm_slvl_rec_rcv * * SYNOPSIS */ -void -osm_slvl_rec_rcv_construct( - IN osm_slvl_rec_rcv_t* const p_rcv ); +void osm_slvl_rec_rcv_construct(IN osm_slvl_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -173,9 +168,7 @@ osm_slvl_rec_rcv_construct( * * SYNOPSIS */ -void -osm_slvl_rec_rcv_destroy( - IN osm_slvl_rec_rcv_t* const p_rcv ); +void osm_slvl_rec_rcv_destroy(IN osm_slvl_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -207,13 +200,11 @@ osm_slvl_rec_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_slvl_rec_rcv_init( - IN osm_slvl_rec_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_slvl_rec_rcv_init(IN osm_slvl_rec_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -252,10 +243,7 @@ osm_slvl_rec_rcv_init( * * SYNOPSIS */ -void -osm_slvl_rec_rcv_process( - IN void *context, - IN void *data ); +void osm_slvl_rec_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -276,5 +264,4 @@ osm_slvl_rec_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SLVL_REC_RCV_H_ */ +#endif /* _OSM_SLVL_REC_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_sminfo_record.h b/opensm/include/opensm/osm_sa_sminfo_record.h index 0020c4f..ce57925 100644 --- a/opensm/include/opensm/osm_sa_sminfo_record.h +++ b/opensm/include/opensm/osm_sa_sminfo_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SM Info Receiver * NAME * SM Info Receiver @@ -97,15 +96,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_smir -{ - osm_subn_t* p_subn; - osm_stats_t* p_stats; - osm_sa_resp_t* p_resp; - osm_mad_pool_t* p_mad_pool; - osm_log_t* p_log; - cl_plock_t* p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_smir { + osm_subn_t *p_subn; + osm_stats_t *p_stats; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_smir_rcv_t; /* * FIELDS @@ -125,8 +123,7 @@ typedef struct _osm_smir * * SYNOPSIS */ -void osm_smir_rcv_construct( - IN osm_smir_rcv_t* const p_ctrl ); +void osm_smir_rcv_construct(IN osm_smir_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -155,8 +152,7 @@ void osm_smir_rcv_construct( * * SYNOPSIS */ -void osm_smir_rcv_destroy( - IN osm_smir_rcv_t* const p_ctrl ); +void osm_smir_rcv_destroy(IN osm_smir_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -187,14 +183,13 @@ void osm_smir_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_smir_rcv_init( - IN osm_smir_rcv_t* const p_ctrl, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_stats_t* const p_stats, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_smir_rcv_init(IN osm_smir_rcv_t * const p_ctrl, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_stats_t * const p_stats, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -235,9 +230,7 @@ ib_api_status_t osm_smir_rcv_init( * * SYNOPSIS */ -void osm_smir_rcv_process( - IN void *context, - IN void *data ); +void osm_smir_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -258,5 +251,4 @@ void osm_smir_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SMIR_H_ */ +#endif /* _OSM_SMIR_H_ */ diff --git a/opensm/include/opensm/osm_sa_sw_info_record.h b/opensm/include/opensm/osm_sa_sw_info_record.h index dfb40ce..0e876d4 100644 --- a/opensm/include/opensm/osm_sa_sw_info_record.h +++ b/opensm/include/opensm/osm_sa_sw_info_record.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Switch Info Receiver * NAME * Switch Info Receiver @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Hal Rosenstock, Voltaire * *********/ - /****s* OpenSM: Switch Info Receiver/osm_sir_rcv_t * NAME * osm_sir_rcv_t @@ -97,16 +95,15 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sir_rcv -{ - osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - osm_req_t *p_req; - osm_state_mgr_t *p_state_mgr; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_sir_rcv { + osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + osm_req_t *p_req; + osm_state_mgr_t *p_state_mgr; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_sir_rcv_t; /* * FIELDS @@ -138,8 +135,7 @@ typedef struct _osm_sir_rcv * * SYNOPSIS */ -void osm_sir_rcv_construct( - IN osm_sir_rcv_t* const p_ctrl ); +void osm_sir_rcv_construct(IN osm_sir_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -170,8 +166,7 @@ void osm_sir_rcv_construct( * * SYNOPSIS */ -void osm_sir_rcv_destroy( - IN osm_sir_rcv_t* const p_ctrl ); +void osm_sir_rcv_destroy(IN osm_sir_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -202,13 +197,12 @@ void osm_sir_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_sir_rcv_init( - IN osm_sir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_sir_rcv_init(IN osm_sir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -250,8 +244,7 @@ ib_api_status_t osm_sir_rcv_init( * * SYNOPSIS */ -boolean_t osm_sir_rcv_is_inited( - IN const osm_sir_rcv_t* const p_ctrl ); +boolean_t osm_sir_rcv_is_inited(IN const osm_sir_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -279,9 +272,7 @@ boolean_t osm_sir_rcv_is_inited( * * SYNOPSIS */ -void osm_sir_rcv_process( - IN void *context, - IN void *data ); +void osm_sir_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -302,5 +293,4 @@ void osm_sir_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SIR_RCV_H_ */ +#endif /* _OSM_SIR_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sa_vlarb_record.h b/opensm/include/opensm/osm_sa_vlarb_record.h index ccb7459..4e1cb19 100644 --- a/opensm/include/opensm/osm_sa_vlarb_record.h +++ b/opensm/include/opensm/osm_sa_vlarb_record.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/VLArbitration Record Receiver * NAME * VLArbitration Record Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: VLArbitration Record Receiver/osm_vlarb_rec_rcv_t * NAME * osm_vlarb_rec_rcv_t @@ -98,14 +96,13 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_vlarb_rec_rcv -{ - const osm_subn_t *p_subn; - osm_sa_resp_t *p_resp; - osm_mad_pool_t *p_mad_pool; - osm_log_t *p_log; - cl_plock_t *p_lock; - cl_qlock_pool_t pool; +typedef struct _osm_vlarb_rec_rcv { + const osm_subn_t *p_subn; + osm_sa_resp_t *p_resp; + osm_mad_pool_t *p_mad_pool; + osm_log_t *p_log; + cl_plock_t *p_lock; + cl_qlock_pool_t pool; } osm_vlarb_rec_rcv_t; /* * FIELDS @@ -141,9 +138,7 @@ typedef struct _osm_vlarb_rec_rcv * * SYNOPSIS */ -void -osm_vlarb_rec_rcv_construct( - IN osm_vlarb_rec_rcv_t* const p_rcv ); +void osm_vlarb_rec_rcv_construct(IN osm_vlarb_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -173,9 +168,7 @@ osm_vlarb_rec_rcv_construct( * * SYNOPSIS */ -void -osm_vlarb_rec_rcv_destroy( - IN osm_vlarb_rec_rcv_t* const p_rcv ); +void osm_vlarb_rec_rcv_destroy(IN osm_vlarb_rec_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -207,13 +200,12 @@ osm_vlarb_rec_rcv_destroy( * SYNOPSIS */ ib_api_status_t -osm_vlarb_rec_rcv_init( - IN osm_vlarb_rec_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_vlarb_rec_rcv_init(IN osm_vlarb_rec_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -252,10 +244,7 @@ osm_vlarb_rec_rcv_init( * * SYNOPSIS */ -void -osm_vlarb_rec_rcv_process( - IN void *context, - IN void *data ); +void osm_vlarb_rec_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -276,5 +265,4 @@ osm_vlarb_rec_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_VLARB_REC_RCV_H_ */ +#endif /* _OSM_VLARB_REC_RCV_H_ */ diff --git a/opensm/include/opensm/osm_service.h b/opensm/include/opensm/osm_service.h index 4c121e9..82b6e52 100644 --- a/opensm/include/opensm/osm_service.h +++ b/opensm/include/opensm/osm_service.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Service Record * NAME * Service Record @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Anil S Keshavamurthy, Intel * *********/ - /****s* OpenSM: Service Record/osm_svcr_t * NAME * osm_svcr_t @@ -95,13 +93,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ - -typedef struct _osm_svcr_t -{ - cl_list_item_t list_item; - ib_service_record_t service_record; - uint32_t modified_time; - uint32_t lease_period; +typedef struct _osm_svcr_t { + cl_list_item_t list_item; + ib_service_record_t service_record; + uint32_t modified_time; + uint32_t lease_period; } osm_svcr_t; /* * FIELDS @@ -130,9 +126,7 @@ typedef struct _osm_svcr_t * * SYNOPSIS */ -osm_svcr_t* -osm_svcr_new( - IN const ib_service_record_t *p_svc_rec ); +osm_svcr_t *osm_svcr_new(IN const ib_service_record_t * p_svc_rec); /* * PARAMETERS * p_svc_rec @@ -158,9 +152,8 @@ osm_svcr_new( * SYNOPSIS */ void -osm_svcr_init( - IN osm_svcr_t* const p_svcr, - IN const ib_service_record_t *p_svc_rec ); +osm_svcr_init(IN osm_svcr_t * const p_svcr, + IN const ib_service_record_t * p_svc_rec); /* * PARAMETERS * p_svc_rec @@ -181,9 +174,7 @@ osm_svcr_init( * * SYNOPSIS */ -void -osm_svcr_delete( - IN osm_svcr_t* const p_svcr ); +void osm_svcr_delete(IN osm_svcr_t * const p_svcr); /* * PARAMETERS * p_svc_rec @@ -193,23 +184,16 @@ osm_svcr_delete( * Service Record, osm_svcr_new *********/ -osm_svcr_t* -osm_svcr_get_by_rid( - IN osm_subn_t const *p_subn, - IN osm_log_t *p_log, - IN ib_service_record_t* const p_svc_rec ); +osm_svcr_t *osm_svcr_get_by_rid(IN osm_subn_t const *p_subn, + IN osm_log_t * p_log, + IN ib_service_record_t * const p_svc_rec); void -osm_svcr_insert_to_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_svcr_t *p_svcr); +osm_svcr_insert_to_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_svcr_t * p_svcr); void -osm_svcr_remove_from_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_svcr_t *p_svcr); +osm_svcr_remove_from_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_svcr_t * p_svcr); END_C_DECLS - -#endif /* _OSM_SVCR_H_ */ +#endif /* _OSM_SVCR_H_ */ diff --git a/opensm/include/opensm/osm_slvl_map_rcv.h b/opensm/include/opensm/osm_slvl_map_rcv.h index 8a7f5b5..2476588 100644 --- a/opensm/include/opensm/osm_slvl_map_rcv.h +++ b/opensm/include/opensm/osm_slvl_map_rcv.h @@ -59,13 +59,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Slvl Map Receiver * NAME * Slvl Map Receiver @@ -83,7 +82,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: Slvl Map Receiver/osm_slvl_rcv_t * NAME * osm_slvl_rcv_t @@ -96,12 +94,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_slvl_rcv -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_slvl_rcv { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_slvl_rcv_t; /* * FIELDS @@ -130,8 +127,7 @@ typedef struct _osm_slvl_rcv * * SYNOPSIS */ -void osm_slvl_rcv_construct( - IN osm_slvl_rcv_t* const p_ctrl ); +void osm_slvl_rcv_construct(IN osm_slvl_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -161,8 +157,7 @@ void osm_slvl_rcv_construct( * * SYNOPSIS */ -void osm_slvl_rcv_destroy( - IN osm_slvl_rcv_t* const p_ctrl ); +void osm_slvl_rcv_destroy(IN osm_slvl_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -193,12 +188,11 @@ void osm_slvl_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_slvl_rcv_init( - IN osm_slvl_rcv_t* const p_ctrl, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_slvl_rcv_init(IN osm_slvl_rcv_t * const p_ctrl, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -237,9 +231,7 @@ ib_api_status_t osm_slvl_rcv_init( * * SYNOPSIS */ -void osm_slvl_rcv_process( - IN void* context, - IN void* p_data ); +void osm_slvl_rcv_process(IN void *context, IN void *p_data); /* * PARAMETERS * context @@ -260,5 +252,4 @@ void osm_slvl_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SLVL_RCV_H_ */ +#endif /* _OSM_SLVL_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sm.h b/opensm/include/opensm/osm_sm.h index a07dbd7..4811d2a 100644 --- a/opensm/include/opensm/osm_sm.h +++ b/opensm/include/opensm/osm_sm.h @@ -87,13 +87,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SM * NAME * SM @@ -112,7 +111,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: SM/osm_sm_t * NAME * osm_sm_t @@ -125,55 +123,54 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sm -{ - osm_thread_state_t thread_state; - cl_event_t signal; - cl_event_t subnet_up_event; - cl_thread_t sweeper; - osm_subn_t *p_subn; - osm_db_t *p_db; - osm_vendor_t *p_vendor; - osm_log_t *p_log; - osm_mad_pool_t *p_mad_pool; - osm_vl15_t *p_vl15; - cl_dispatcher_t *p_disp; - cl_plock_t *p_lock; - atomic32_t sm_trans_id; - osm_req_t req; - osm_resp_t resp; - osm_ni_rcv_t ni_rcv; - osm_pi_rcv_t pi_rcv; - osm_nd_rcv_t nd_rcv; - osm_sm_mad_ctrl_t mad_ctrl; - osm_si_rcv_t si_rcv; - osm_state_mgr_ctrl_t state_mgr_ctrl; - osm_lid_mgr_t lid_mgr; - osm_ucast_mgr_t ucast_mgr; - osm_link_mgr_t link_mgr; - osm_state_mgr_t state_mgr; - osm_drop_mgr_t drop_mgr; - osm_lft_rcv_t lft_rcv; - osm_mft_rcv_t mft_rcv; - osm_sweep_fail_ctrl_t sweep_fail_ctrl; - osm_sminfo_rcv_t sm_info_rcv; - osm_trap_rcv_t trap_rcv; - osm_sm_state_mgr_t sm_state_mgr; - osm_mcast_mgr_t mcast_mgr; - osm_slvl_rcv_t slvl_rcv; - osm_vla_rcv_t vla_rcv; - osm_pkey_rcv_t pkey_rcv; - cl_disp_reg_handle_t ni_disp_h; - cl_disp_reg_handle_t pi_disp_h; - cl_disp_reg_handle_t nd_disp_h; - cl_disp_reg_handle_t si_disp_h; - cl_disp_reg_handle_t lft_disp_h; - cl_disp_reg_handle_t mft_disp_h; - cl_disp_reg_handle_t sm_info_disp_h; - cl_disp_reg_handle_t trap_disp_h; - cl_disp_reg_handle_t slvl_disp_h; - cl_disp_reg_handle_t vla_disp_h; - cl_disp_reg_handle_t pkey_disp_h; +typedef struct _osm_sm { + osm_thread_state_t thread_state; + cl_event_t signal; + cl_event_t subnet_up_event; + cl_thread_t sweeper; + osm_subn_t *p_subn; + osm_db_t *p_db; + osm_vendor_t *p_vendor; + osm_log_t *p_log; + osm_mad_pool_t *p_mad_pool; + osm_vl15_t *p_vl15; + cl_dispatcher_t *p_disp; + cl_plock_t *p_lock; + atomic32_t sm_trans_id; + osm_req_t req; + osm_resp_t resp; + osm_ni_rcv_t ni_rcv; + osm_pi_rcv_t pi_rcv; + osm_nd_rcv_t nd_rcv; + osm_sm_mad_ctrl_t mad_ctrl; + osm_si_rcv_t si_rcv; + osm_state_mgr_ctrl_t state_mgr_ctrl; + osm_lid_mgr_t lid_mgr; + osm_ucast_mgr_t ucast_mgr; + osm_link_mgr_t link_mgr; + osm_state_mgr_t state_mgr; + osm_drop_mgr_t drop_mgr; + osm_lft_rcv_t lft_rcv; + osm_mft_rcv_t mft_rcv; + osm_sweep_fail_ctrl_t sweep_fail_ctrl; + osm_sminfo_rcv_t sm_info_rcv; + osm_trap_rcv_t trap_rcv; + osm_sm_state_mgr_t sm_state_mgr; + osm_mcast_mgr_t mcast_mgr; + osm_slvl_rcv_t slvl_rcv; + osm_vla_rcv_t vla_rcv; + osm_pkey_rcv_t pkey_rcv; + cl_disp_reg_handle_t ni_disp_h; + cl_disp_reg_handle_t pi_disp_h; + cl_disp_reg_handle_t nd_disp_h; + cl_disp_reg_handle_t si_disp_h; + cl_disp_reg_handle_t lft_disp_h; + cl_disp_reg_handle_t mft_disp_h; + cl_disp_reg_handle_t sm_info_disp_h; + cl_disp_reg_handle_t trap_disp_h; + cl_disp_reg_handle_t slvl_disp_h; + cl_disp_reg_handle_t vla_disp_h; + cl_disp_reg_handle_t pkey_disp_h; } osm_sm_t; /* * FIELDS @@ -241,9 +238,7 @@ typedef struct _osm_sm * * SYNOPSIS */ -void -osm_sm_construct( - IN osm_sm_t* const p_sm ); +void osm_sm_construct(IN osm_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -272,9 +267,7 @@ osm_sm_construct( * * SYNOPSIS */ -void -osm_sm_shutdown( - IN osm_sm_t* const p_sm ); +void osm_sm_shutdown(IN osm_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -297,9 +290,7 @@ osm_sm_shutdown( * * SYNOPSIS */ -void -osm_sm_destroy( - IN osm_sm_t* const p_sm ); +void osm_sm_destroy(IN osm_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -328,17 +319,15 @@ osm_sm_destroy( * SYNOPSIS */ ib_api_status_t -osm_sm_init( - IN osm_sm_t* const p_sm, - IN osm_subn_t* const p_subn, - IN osm_db_t* const p_db, - IN osm_vendor_t* const p_vendor, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vl15_t* const p_vl15, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_dispatcher_t* const p_disp, - IN cl_plock_t* const p_lock ); +osm_sm_init(IN osm_sm_t * const p_sm, + IN osm_subn_t * const p_subn, + IN osm_db_t * const p_db, + IN osm_vendor_t * const p_vendor, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vl15_t * const p_vl15, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_sm @@ -387,9 +376,7 @@ osm_sm_init( * * SYNOPSIS */ -void -osm_sm_sweep( - IN osm_sm_t* const p_sm ); +void osm_sm_sweep(IN osm_sm_t * const p_sm); /* * PARAMETERS * p_sm @@ -414,9 +401,7 @@ osm_sm_sweep( * SYNOPSIS */ ib_api_status_t -osm_sm_bind( - IN osm_sm_t* const p_sm, - IN const ib_net64_t port_guid ); +osm_sm_bind(IN osm_sm_t * const p_sm, IN const ib_net64_t port_guid); /* * PARAMETERS * p_sm @@ -448,11 +433,10 @@ osm_sm_bind( * SYNOPSIS */ ib_api_status_t -osm_sm_mcgrp_join( - IN osm_sm_t* const p_sm, - IN const ib_net16_t mlid, - IN const ib_net64_t port_guid, - IN osm_mcast_req_type_t req_type ); +osm_sm_mcgrp_join(IN osm_sm_t * const p_sm, + IN const ib_net16_t mlid, + IN const ib_net64_t port_guid, + IN osm_mcast_req_type_t req_type); /* * PARAMETERS * p_sm @@ -488,10 +472,8 @@ osm_sm_mcgrp_join( * SYNOPSIS */ ib_api_status_t -osm_sm_mcgrp_leave( - IN osm_sm_t* const p_sm, - IN const ib_net16_t mlid, - IN const ib_net64_t port_guid ); +osm_sm_mcgrp_leave(IN osm_sm_t * const p_sm, + IN const ib_net16_t mlid, IN const ib_net64_t port_guid); /* * PARAMETERS * p_sm @@ -521,14 +503,14 @@ osm_sm_mcgrp_leave( * SYNOPSIS */ static inline cl_status_t -osm_sm_wait_for_subnet_up( - IN osm_sm_t* const p_sm, - IN uint32_t const wait_us, - IN boolean_t const interruptible ) +osm_sm_wait_for_subnet_up(IN osm_sm_t * const p_sm, + IN uint32_t const wait_us, + IN boolean_t const interruptible) { - return( cl_event_wait_on( &p_sm->subnet_up_event, - wait_us, interruptible ) ); + return (cl_event_wait_on(&p_sm->subnet_up_event, + wait_us, interruptible)); } + /* * PARAMETERS * p_sm @@ -557,5 +539,4 @@ osm_sm_wait_for_subnet_up( *********/ END_C_DECLS - -#endif /* _OSM_SM_H_ */ +#endif /* _OSM_SM_H_ */ diff --git a/opensm/include/opensm/osm_sm_mad_ctrl.h b/opensm/include/opensm/osm_sm_mad_ctrl.h index 062fa2a..ccacb93 100644 --- a/opensm/include/opensm/osm_sm_mad_ctrl.h +++ b/opensm/include/opensm/osm_sm_mad_ctrl.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SM MAD Controller * NAME * SM MAD Controller @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: SM MAD Controller/osm_sm_mad_ctrl_t * NAME * osm_sm_mad_ctrl_t @@ -99,18 +97,17 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sm_mad_ctrl -{ - osm_log_t *p_log; - osm_subn_t *p_subn; - osm_mad_pool_t *p_mad_pool; - osm_vl15_t *p_vl15; - osm_vendor_t *p_vendor; - osm_bind_handle_t h_bind; - cl_plock_t *p_lock; - cl_dispatcher_t *p_disp; - cl_disp_reg_handle_t h_disp; - osm_stats_t *p_stats; +typedef struct _osm_sm_mad_ctrl { + osm_log_t *p_log; + osm_subn_t *p_subn; + osm_mad_pool_t *p_mad_pool; + osm_vl15_t *p_vl15; + osm_vendor_t *p_vendor; + osm_bind_handle_t h_bind; + cl_plock_t *p_lock; + cl_dispatcher_t *p_disp; + cl_disp_reg_handle_t h_disp; + osm_stats_t *p_stats; } osm_sm_mad_ctrl_t; /* @@ -156,9 +153,7 @@ typedef struct _osm_sm_mad_ctrl * * SYNOPSIS */ -void -osm_sm_mad_ctrl_construct( - IN osm_sm_mad_ctrl_t* const p_ctrl ); +void osm_sm_mad_ctrl_construct(IN osm_sm_mad_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -189,9 +184,7 @@ osm_sm_mad_ctrl_construct( * * SYNOPSIS */ -void -osm_sm_mad_ctrl_destroy( - IN osm_sm_mad_ctrl_t* const p_ctrl ); +void osm_sm_mad_ctrl_destroy(IN osm_sm_mad_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -223,16 +216,15 @@ osm_sm_mad_ctrl_destroy( * SYNOPSIS */ ib_api_status_t -osm_sm_mad_ctrl_init( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_subn_t* const p_subn, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vl15_t* const p_vl15, - IN osm_vendor_t* const p_vendor, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_plock_t* const p_lock, - IN cl_dispatcher_t* const p_disp ); +osm_sm_mad_ctrl_init(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_subn_t * const p_subn, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vl15_t * const p_vl15, + IN osm_vendor_t * const p_vendor, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_plock_t * const p_lock, + IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_ctrl @@ -281,9 +273,8 @@ osm_sm_mad_ctrl_init( * SYNOPSIS */ ib_api_status_t -osm_sm_mad_ctrl_bind( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN const ib_net64_t port_guid ); +osm_sm_mad_ctrl_bind(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN const ib_net64_t port_guid); /* * PARAMETERS * p_ctrl @@ -313,11 +304,11 @@ osm_sm_mad_ctrl_bind( * SYNOPSIS */ static inline osm_bind_handle_t -osm_sm_mad_ctrl_get_bind_handle( - IN const osm_sm_mad_ctrl_t* const p_ctrl ) +osm_sm_mad_ctrl_get_bind_handle(IN const osm_sm_mad_ctrl_t * const p_ctrl) { - return( p_ctrl->h_bind ); + return (p_ctrl->h_bind); } + /* * PARAMETERS * p_ctrl @@ -335,5 +326,4 @@ osm_sm_mad_ctrl_get_bind_handle( *********/ END_C_DECLS - -#endif /* _OSM_SM_MAD_CTRL_H_ */ +#endif /* _OSM_SM_MAD_CTRL_H_ */ diff --git a/opensm/include/opensm/osm_sm_state_mgr.h b/opensm/include/opensm/osm_sm_state_mgr.h index e92bc04..ab2aaff 100644 --- a/opensm/include/opensm/osm_sm_state_mgr.h +++ b/opensm/include/opensm/osm_sm_state_mgr.h @@ -64,13 +64,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SM State Manager * NAME * SM State Manager @@ -88,7 +87,6 @@ BEGIN_C_DECLS * Yael Kalka, Mellanox * *********/ - /****s* OpenSM: SM State Manager/osm_sm_state_mgr_t * NAME * osm_sm_state_mgr_t @@ -101,17 +99,16 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sm_state_mgr -{ - cl_spinlock_t state_lock; - cl_timer_t polling_timer; - uint32_t retry_number; - ib_net64_t master_guid; - osm_state_mgr_t* p_state_mgr; - osm_subn_t* p_subn; - osm_req_t* p_req; - osm_log_t* p_log; - osm_remote_sm_t* p_polling_sm; +typedef struct _osm_sm_state_mgr { + cl_spinlock_t state_lock; + cl_timer_t polling_timer; + uint32_t retry_number; + ib_net64_t master_guid; + osm_state_mgr_t *p_state_mgr; + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + osm_remote_sm_t *p_polling_sm; } osm_sm_state_mgr_t; /* @@ -159,9 +156,7 @@ typedef struct _osm_sm_state_mgr * * SYNOPSIS */ -void -osm_sm_state_mgr_construct( - IN osm_sm_state_mgr_t* const p_sm_mgr ); +void osm_sm_state_mgr_construct(IN osm_sm_state_mgr_t * const p_sm_mgr); /* * PARAMETERS * p_sm_mgr @@ -191,9 +186,7 @@ osm_sm_state_mgr_construct( * * SYNOPSIS */ -void -osm_sm_state_mgr_destroy( - IN osm_sm_state_mgr_t* const p_sm_mgr ); +void osm_sm_state_mgr_destroy(IN osm_sm_state_mgr_t * const p_sm_mgr); /* * PARAMETERS * p_sm_mgr @@ -225,12 +218,10 @@ osm_sm_state_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_sm_state_mgr_init( - IN osm_sm_state_mgr_t* const p_sm_mgr, - IN osm_state_mgr_t* const p_state_mgr, - IN osm_subn_t* const p_subn, - IN osm_req_t* const p_req, - IN osm_log_t* const p_log ); +osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_state_mgr_t * const p_state_mgr, + IN osm_subn_t * const p_subn, + IN osm_req_t * const p_req, IN osm_log_t * const p_log); /* * PARAMETERS * p_sm_mgr @@ -271,9 +262,8 @@ osm_sm_state_mgr_init( * SYNOPSIS */ ib_api_status_t -osm_sm_state_mgr_process( - IN osm_sm_state_mgr_t* const p_sm_mgr, - IN osm_sm_signal_t signal ); +osm_sm_state_mgr_process(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_sm_signal_t signal); /* * PARAMETERS * p_sm_mgr @@ -291,7 +281,6 @@ osm_sm_state_mgr_process( * State Manager *********/ - /****f* OpenSM: SM State Manager/osm_sm_state_mgr_signal_master_is_alive * NAME * osm_sm_state_mgr_signal_master_is_alive @@ -303,8 +292,7 @@ osm_sm_state_mgr_process( * SYNOPSIS */ void -osm_sm_state_mgr_signal_master_is_alive( - IN osm_sm_state_mgr_t* const p_sm_mgr ); +osm_sm_state_mgr_signal_master_is_alive(IN osm_sm_state_mgr_t * const p_sm_mgr); /* * PARAMETERS * p_sm_mgr @@ -330,9 +318,8 @@ osm_sm_state_mgr_signal_master_is_alive( * SYNOPSIS */ ib_api_status_t -osm_sm_state_mgr_check_legality( - IN osm_sm_state_mgr_t* const p_sm_mgr, - IN osm_sm_signal_t signal ); +osm_sm_state_mgr_check_legality(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_sm_signal_t signal); /* * PARAMETERS * p_sm_mgr @@ -351,5 +338,4 @@ osm_sm_state_mgr_check_legality( *********/ END_C_DECLS - -#endif /* _OSM_SM_STATE_MGR_H_ */ +#endif /* _OSM_SM_STATE_MGR_H_ */ diff --git a/opensm/include/opensm/osm_sminfo_rcv.h b/opensm/include/opensm/osm_sminfo_rcv.h index 0d9ffc0..1cbf2e6 100644 --- a/opensm/include/opensm/osm_sminfo_rcv.h +++ b/opensm/include/opensm/osm_sminfo_rcv.h @@ -61,13 +61,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/SMInfo Receiver * NAME * SMInfo Receiver @@ -85,7 +84,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: SMInfo Receiver/osm_sminfo_rcv_t * NAME * osm_sminfo_rcv_t @@ -98,15 +96,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sminfo_rcv -{ - osm_subn_t *p_subn; - osm_stats_t *p_stats; - osm_log_t *p_log; - osm_resp_t *p_resp; - osm_state_mgr_t *p_state_mgr; - struct _osm_sm_state_mgr *p_sm_state_mgr; - cl_plock_t *p_lock; +typedef struct _osm_sminfo_rcv { + osm_subn_t *p_subn; + osm_stats_t *p_stats; + osm_log_t *p_log; + osm_resp_t *p_resp; + osm_state_mgr_t *p_state_mgr; + struct _osm_sm_state_mgr *p_sm_state_mgr; + cl_plock_t *p_lock; } osm_sminfo_rcv_t; /* * FIELDS @@ -144,8 +141,7 @@ typedef struct _osm_sminfo_rcv * * SYNOPSIS */ -void osm_sminfo_rcv_construct( - IN osm_sminfo_rcv_t* const p_rcv ); +void osm_sminfo_rcv_construct(IN osm_sminfo_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -175,8 +171,7 @@ void osm_sminfo_rcv_construct( * * SYNOPSIS */ -void osm_sminfo_rcv_destroy( - IN osm_sminfo_rcv_t* const p_rcv ); +void osm_sminfo_rcv_destroy(IN osm_sminfo_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -207,15 +202,15 @@ void osm_sminfo_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_sminfo_rcv_init( - IN osm_sminfo_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_stats_t* const p_stats, - IN osm_resp_t* const p_resp, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN struct _osm_sm_state_mgr* const p_sm_state_mgr, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_sminfo_rcv_init(IN osm_sminfo_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_stats_t * const p_stats, + IN osm_resp_t * const p_resp, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN struct _osm_sm_state_mgr *const + p_sm_state_mgr, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -263,9 +258,7 @@ ib_api_status_t osm_sminfo_rcv_init( * * SYNOPSIS */ -void osm_sminfo_rcv_process( - IN void *context, - IN void *data ); +void osm_sminfo_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -286,5 +279,4 @@ void osm_sminfo_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SMINFO_RCV_H_ */ +#endif /* _OSM_SMINFO_RCV_H_ */ diff --git a/opensm/include/opensm/osm_state_mgr.h b/opensm/include/opensm/osm_state_mgr.h index 99383ee..5b45a91 100644 --- a/opensm/include/opensm/osm_state_mgr.h +++ b/opensm/include/opensm/osm_state_mgr.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/State Manager * NAME * State Manager @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: State Manager/osm_state_mgr_t * NAME * osm_state_mgr_t @@ -99,25 +97,24 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_state_mgr -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - osm_lid_mgr_t *p_lid_mgr; - osm_ucast_mgr_t *p_ucast_mgr; - osm_mcast_mgr_t *p_mcast_mgr; - osm_link_mgr_t *p_link_mgr; - osm_drop_mgr_t *p_drop_mgr; - osm_req_t *p_req; - osm_stats_t *p_stats; - struct _osm_sm_state_mgr *p_sm_state_mgr; - const osm_sm_mad_ctrl_t *p_mad_ctrl; - cl_spinlock_t state_lock; - cl_spinlock_t idle_lock; - cl_qlist_t idle_time_list; - cl_plock_t *p_lock; - cl_event_t *p_subnet_up_event; - osm_sm_state_t state; +typedef struct _osm_state_mgr { + osm_subn_t *p_subn; + osm_log_t *p_log; + osm_lid_mgr_t *p_lid_mgr; + osm_ucast_mgr_t *p_ucast_mgr; + osm_mcast_mgr_t *p_mcast_mgr; + osm_link_mgr_t *p_link_mgr; + osm_drop_mgr_t *p_drop_mgr; + osm_req_t *p_req; + osm_stats_t *p_stats; + struct _osm_sm_state_mgr *p_sm_state_mgr; + const osm_sm_mad_ctrl_t *p_mad_ctrl; + cl_spinlock_t state_lock; + cl_spinlock_t idle_lock; + cl_qlist_t idle_time_list; + cl_plock_t *p_lock; + cl_event_t *p_subnet_up_event; + osm_sm_state_t state; } osm_state_mgr_t; /* * FIELDS @@ -189,24 +186,18 @@ typedef struct _osm_state_mgr * SYNOPSIS */ -typedef osm_signal_t -(*osm_pfn_start_t)( - IN void *context1, - IN void *context2 ); +typedef osm_signal_t(*osm_pfn_start_t) (IN void *context1, IN void *context2); typedef void -(*osm_pfn_done_t)( - IN void *context1, - IN void *context2 ); + (*osm_pfn_done_t) (IN void *context1, IN void *context2); -typedef struct _osm_idle_item -{ - cl_list_item_t list_item; - void* context1; - void* context2; - osm_pfn_start_t pfn_start; - osm_pfn_done_t pfn_done; -}osm_idle_item_t; +typedef struct _osm_idle_item { + cl_list_item_t list_item; + void *context1; + void *context2; + osm_pfn_start_t pfn_start; + osm_pfn_done_t pfn_done; +} osm_idle_item_t; /* * FIELDS @@ -240,13 +231,10 @@ typedef struct _osm_idle_item */ ib_api_status_t -osm_state_mgr_process_idle( - IN osm_state_mgr_t* const p_mgr, - IN osm_pfn_start_t pfn_start, - IN osm_pfn_done_t pfn_done, - void* context1, - void* context2 - ); +osm_state_mgr_process_idle(IN osm_state_mgr_t * const p_mgr, + IN osm_pfn_start_t pfn_start, + IN osm_pfn_done_t pfn_done, + void *context1, void *context2); /* * PARAMETERS @@ -290,9 +278,7 @@ osm_state_mgr_process_idle( * * SYNOPSIS */ -void -osm_state_mgr_construct( - IN osm_state_mgr_t* const p_mgr ); +void osm_state_mgr_construct(IN osm_state_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -322,9 +308,7 @@ osm_state_mgr_construct( * * SYNOPSIS */ -void -osm_state_mgr_destroy( - IN osm_state_mgr_t* const p_mgr ); +void osm_state_mgr_destroy(IN osm_state_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -356,21 +340,20 @@ osm_state_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_state_mgr_init( - IN osm_state_mgr_t* const p_mgr, - IN osm_subn_t* const p_subn, - IN osm_lid_mgr_t* const p_lid_mgr, - IN osm_ucast_mgr_t* const p_ucast_mgr, - IN osm_mcast_mgr_t* const p_mcast_mgr, - IN osm_link_mgr_t* const p_link_mgr, - IN osm_drop_mgr_t* const p_drop_mgr, - IN osm_req_t* const p_req, - IN osm_stats_t* const p_stats, - IN struct _osm_sm_state_mgr* const p_sm_state_mgr, - IN const osm_sm_mad_ctrl_t* const p_mad_ctrl, - IN cl_plock_t* const p_lock, - IN cl_event_t* const p_subnet_up_event, - IN osm_log_t* const p_log ); +osm_state_mgr_init(IN osm_state_mgr_t * const p_mgr, + IN osm_subn_t * const p_subn, + IN osm_lid_mgr_t * const p_lid_mgr, + IN osm_ucast_mgr_t * const p_ucast_mgr, + IN osm_mcast_mgr_t * const p_mcast_mgr, + IN osm_link_mgr_t * const p_link_mgr, + IN osm_drop_mgr_t * const p_drop_mgr, + IN osm_req_t * const p_req, + IN osm_stats_t * const p_stats, + IN struct _osm_sm_state_mgr *const p_sm_state_mgr, + IN const osm_sm_mad_ctrl_t * const p_mad_ctrl, + IN cl_plock_t * const p_lock, + IN cl_event_t * const p_subnet_up_event, + IN osm_log_t * const p_log); /* * PARAMETERS * p_mgr @@ -434,28 +417,22 @@ osm_state_mgr_init( * SYNOPSIS */ static inline boolean_t -osm_sm_is_greater_than ( - IN const uint8_t l_priority, - IN const ib_net64_t l_guid, - IN const uint8_t r_priority, - IN const ib_net64_t r_guid ) +osm_sm_is_greater_than(IN const uint8_t l_priority, + IN const ib_net64_t l_guid, + IN const uint8_t r_priority, IN const ib_net64_t r_guid) { - if( l_priority > r_priority ) - { - return( TRUE ); - } - else - { - if( l_priority == r_priority ) - { - if( cl_ntoh64(l_guid) < cl_ntoh64(r_guid) ) - { - return( TRUE ); - } - } - } - return( FALSE ); + if (l_priority > r_priority) { + return (TRUE); + } else { + if (l_priority == r_priority) { + if (cl_ntoh64(l_guid) < cl_ntoh64(r_guid)) { + return (TRUE); + } + } + } + return (FALSE); } + /* * PARAMETERS * l_priority @@ -491,9 +468,7 @@ osm_sm_is_greater_than ( * SYNOPSIS */ void -osm_state_mgr_process( - IN osm_state_mgr_t* const p_mgr, - IN osm_signal_t signal ); +osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, IN osm_signal_t signal); /* * PARAMETERS * p_mgr @@ -512,5 +487,4 @@ osm_state_mgr_process( *********/ END_C_DECLS - -#endif /* _OSM_STATE_MGR_H_ */ +#endif /* _OSM_STATE_MGR_H_ */ diff --git a/opensm/include/opensm/osm_state_mgr_ctrl.h b/opensm/include/opensm/osm_state_mgr_ctrl.h index 0837e33..4751335 100644 --- a/opensm/include/opensm/osm_state_mgr_ctrl.h +++ b/opensm/include/opensm/osm_state_mgr_ctrl.h @@ -57,13 +57,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/State Manager Controller * NAME * State Manager Controller @@ -82,7 +81,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: State Manager Controller/osm_state_mgr_ctrl_t * NAME * osm_state_mgr_ctrl_t @@ -95,12 +93,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_state_mgr_ctrl -{ - osm_state_mgr_t *p_mgr; - osm_log_t *p_log; - cl_dispatcher_t *p_disp; - cl_disp_reg_handle_t h_disp; +typedef struct _osm_state_mgr_ctrl { + osm_state_mgr_t *p_mgr; + osm_log_t *p_log; + cl_dispatcher_t *p_disp; + cl_disp_reg_handle_t h_disp; } osm_state_mgr_ctrl_t; /* @@ -130,9 +127,7 @@ typedef struct _osm_state_mgr_ctrl * * SYNOPSIS */ -void -osm_state_mgr_ctrl_construct( - IN osm_state_mgr_ctrl_t* const p_ctrl ); +void osm_state_mgr_ctrl_construct(IN osm_state_mgr_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -163,9 +158,7 @@ osm_state_mgr_ctrl_construct( * * SYNOPSIS */ -void -osm_state_mgr_ctrl_destroy( - IN osm_state_mgr_ctrl_t* const p_ctrl ); +void osm_state_mgr_ctrl_destroy(IN osm_state_mgr_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -197,11 +190,10 @@ osm_state_mgr_ctrl_destroy( * SYNOPSIS */ ib_api_status_t -osm_state_mgr_ctrl_init( - IN osm_state_mgr_ctrl_t* const p_ctrl, - IN osm_state_mgr_t* const p_mgr, - IN osm_log_t* const p_log, - IN cl_dispatcher_t* const p_disp ); +osm_state_mgr_ctrl_init(IN osm_state_mgr_ctrl_t * const p_ctrl, + IN osm_state_mgr_t * const p_mgr, + IN osm_log_t * const p_log, + IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_ctrl @@ -229,5 +221,4 @@ osm_state_mgr_ctrl_init( *********/ END_C_DECLS - -#endif /* OSM_STATE_MGR_CTRL_H_ */ +#endif /* OSM_STATE_MGR_CTRL_H_ */ diff --git a/opensm/include/opensm/osm_stats.h b/opensm/include/opensm/osm_stats.h index f86ef8b..2d2fa14 100644 --- a/opensm/include/opensm/osm_stats.h +++ b/opensm/include/opensm/osm_stats.h @@ -54,13 +54,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Statistics * NAME * OpenSM @@ -73,7 +72,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Statistics/osm_stats_t * NAME * osm_stats_t @@ -83,16 +81,15 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_stats -{ - atomic32_t qp0_mads_outstanding; - atomic32_t qp0_mads_outstanding_on_wire; - atomic32_t qp0_mads_rcvd; - atomic32_t qp0_mads_sent; - atomic32_t qp0_unicasts_sent; - atomic32_t sa_mads_outstanding; - atomic32_t sa_mads_rcvd; - atomic32_t sa_mads_sent; +typedef struct _osm_stats { + atomic32_t qp0_mads_outstanding; + atomic32_t qp0_mads_outstanding_on_wire; + atomic32_t qp0_mads_rcvd; + atomic32_t qp0_mads_sent; + atomic32_t qp0_unicasts_sent; + atomic32_t sa_mads_outstanding; + atomic32_t sa_mads_rcvd; + atomic32_t sa_mads_sent; } osm_stats_t; /* @@ -121,5 +118,4 @@ typedef struct _osm_stats ***************/ END_C_DECLS - -#endif /* _OSM_STATS_H_ */ +#endif /* _OSM_STATS_H_ */ diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 277db7a..a7543dc 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -58,17 +58,15 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSM_SUBNET_VECTOR_MIN_SIZE 0 #define OSM_SUBNET_VECTOR_GROW_SIZE 1 #define OSM_SUBNET_VECTOR_CAPACITY 256 - struct _osm_opensm_t; /****h* OpenSM/Subnet @@ -103,8 +101,7 @@ struct _osm_opensm_t; * SYNOPSIS */ typedef int -(*osm_pfn_ui_extension_t)( - IN void* context ); + (*osm_pfn_ui_extension_t) (IN void *context); /* * PARAMETERS * context @@ -130,11 +127,10 @@ typedef int * SYNOPSIS */ typedef void -(*osm_pfn_ui_mcast_extension_t)( - IN void *context, - IN ib_net16_t mlid, - IN osm_mcast_req_type_t request_type, - IN ib_net64_t port_guid ); + (*osm_pfn_ui_mcast_extension_t) (IN void *context, + IN ib_net16_t mlid, + IN osm_mcast_req_type_t request_type, + IN ib_net64_t port_guid); /* * PARAMETERS * context @@ -167,11 +163,10 @@ typedef void * * SYNOPSIS */ -typedef enum _osm_testability_modes -{ - OSM_TEST_MODE_NONE = 0, - OSM_TEST_MODE_EXIT_BEFORE_SEND_HANDOVER, - OSM_TEST_MODE_MAX +typedef enum _osm_testability_modes { + OSM_TEST_MODE_NONE = 0, + OSM_TEST_MODE_EXIT_BEFORE_SEND_HANDOVER, + OSM_TEST_MODE_MAX } osm_testability_modes_t; /***********/ @@ -186,11 +181,11 @@ typedef enum _osm_testability_modes * SYNOPSIS */ typedef struct _osm_qos_options_t { - unsigned max_vls; - unsigned high_limit; - char *vlarb_high; - char *vlarb_low; - char *sl2vl; + unsigned max_vls; + unsigned high_limit; + char *vlarb_high; + char *vlarb_low; + char *sl2vl; } osm_qos_options_t; /* * FIELDS @@ -223,84 +218,83 @@ typedef struct _osm_qos_options_t { * * SYNOPSIS */ -typedef struct _osm_subn_opt -{ - ib_net64_t guid; - ib_net64_t m_key; - ib_net64_t sm_key; - ib_net64_t subnet_prefix; - ib_net16_t m_key_lease_period; - uint32_t sweep_interval; - uint32_t max_wire_smps; - uint32_t transaction_timeout; - uint8_t sm_priority; - uint8_t lmc; - boolean_t lmc_esp0; - uint8_t max_op_vls; - uint8_t force_link_speed; - boolean_t reassign_lids; - boolean_t ignore_other_sm; - boolean_t single_thread; - boolean_t no_multicast_option; - boolean_t disable_multicast; - boolean_t force_log_flush; - uint8_t subnet_timeout; - uint8_t packet_life_time; - uint8_t vl_stall_count; - uint8_t leaf_vl_stall_count; - uint8_t head_of_queue_lifetime; - uint8_t leaf_head_of_queue_lifetime; - uint8_t local_phy_errors_threshold; - uint8_t overrun_errors_threshold; - uint32_t sminfo_polling_timeout; - uint32_t polling_retry_number; - uint32_t max_msg_fifo_timeout; - boolean_t force_heavy_sweep; - uint8_t log_flags; - char * dump_files_dir; - char * log_file; - unsigned long log_max_size; - char * partition_config_file; - boolean_t no_partition_enforcement; - boolean_t no_qos; - boolean_t accum_log_file; - char * console; - uint16_t console_port; - cl_map_t port_prof_ignore_guids; - boolean_t port_profile_switch_nodes; - osm_pfn_ui_extension_t pfn_ui_pre_lid_assign; - void * ui_pre_lid_assign_ctx; - osm_pfn_ui_mcast_extension_t pfn_ui_mcast_fdb_assign; - void * ui_mcast_fdb_assign_ctx; - boolean_t sweep_on_trap; - osm_testability_modes_t testability_mode; - char * routing_engine_name; - boolean_t connect_roots; - char * lid_matrix_dump_file; - char * ucast_dump_file; - char * root_guid_file; - char * cn_guid_file; - char * sa_db_file; - boolean_t exit_on_fatal; - boolean_t honor_guid2lid_file; - boolean_t daemon; - boolean_t sm_inactive; - boolean_t babbling_port_policy; - osm_qos_options_t qos_options; - osm_qos_options_t qos_ca_options; - osm_qos_options_t qos_sw0_options; - osm_qos_options_t qos_swe_options; - osm_qos_options_t qos_rtr_options; - boolean_t enable_quirks; - boolean_t no_clients_rereg; +typedef struct _osm_subn_opt { + ib_net64_t guid; + ib_net64_t m_key; + ib_net64_t sm_key; + ib_net64_t subnet_prefix; + ib_net16_t m_key_lease_period; + uint32_t sweep_interval; + uint32_t max_wire_smps; + uint32_t transaction_timeout; + uint8_t sm_priority; + uint8_t lmc; + boolean_t lmc_esp0; + uint8_t max_op_vls; + uint8_t force_link_speed; + boolean_t reassign_lids; + boolean_t ignore_other_sm; + boolean_t single_thread; + boolean_t no_multicast_option; + boolean_t disable_multicast; + boolean_t force_log_flush; + uint8_t subnet_timeout; + uint8_t packet_life_time; + uint8_t vl_stall_count; + uint8_t leaf_vl_stall_count; + uint8_t head_of_queue_lifetime; + uint8_t leaf_head_of_queue_lifetime; + uint8_t local_phy_errors_threshold; + uint8_t overrun_errors_threshold; + uint32_t sminfo_polling_timeout; + uint32_t polling_retry_number; + uint32_t max_msg_fifo_timeout; + boolean_t force_heavy_sweep; + uint8_t log_flags; + char *dump_files_dir; + char *log_file; + unsigned long log_max_size; + char *partition_config_file; + boolean_t no_partition_enforcement; + boolean_t no_qos; + boolean_t accum_log_file; + char *console; + uint16_t console_port; + cl_map_t port_prof_ignore_guids; + boolean_t port_profile_switch_nodes; + osm_pfn_ui_extension_t pfn_ui_pre_lid_assign; + void *ui_pre_lid_assign_ctx; + osm_pfn_ui_mcast_extension_t pfn_ui_mcast_fdb_assign; + void *ui_mcast_fdb_assign_ctx; + boolean_t sweep_on_trap; + osm_testability_modes_t testability_mode; + char *routing_engine_name; + boolean_t connect_roots; + char *lid_matrix_dump_file; + char *ucast_dump_file; + char *root_guid_file; + char *cn_guid_file; + char *sa_db_file; + boolean_t exit_on_fatal; + boolean_t honor_guid2lid_file; + boolean_t daemon; + boolean_t sm_inactive; + boolean_t babbling_port_policy; + osm_qos_options_t qos_options; + osm_qos_options_t qos_ca_options; + osm_qos_options_t qos_sw0_options; + osm_qos_options_t qos_swe_options; + osm_qos_options_t qos_rtr_options; + boolean_t enable_quirks; + boolean_t no_clients_rereg; #ifdef ENABLE_OSM_PERF_MGR - boolean_t perfmgr; - boolean_t perfmgr_redir; - uint16_t perfmgr_sweep_time_s; - uint32_t perfmgr_max_outstanding_queries; - char * event_db_dump_file; -#endif /* ENABLE_OSM_PERF_MGR */ - char * event_plugin_name; + boolean_t perfmgr; + boolean_t perfmgr_redir; + uint16_t perfmgr_sweep_time_s; + uint32_t perfmgr_max_outstanding_queries; + char *event_db_dump_file; +#endif /* ENABLE_OSM_PERF_MGR */ + char *event_plugin_name; } osm_subn_opt_t; /* * FIELDS @@ -540,37 +534,36 @@ typedef struct _osm_subn_opt * * SYNOPSIS */ -typedef struct _osm_subn -{ - struct _osm_opensm_t *p_osm; - cl_qmap_t sw_guid_tbl; - cl_qmap_t node_guid_tbl; - cl_qmap_t port_guid_tbl; - cl_qmap_t rtr_guid_tbl; - cl_qmap_t prtn_pkey_tbl; - cl_qmap_t mgrp_mlid_tbl; - cl_qmap_t sm_guid_tbl; - cl_qlist_t sa_sr_list; - cl_qlist_t sa_infr_list; - cl_ptr_vector_t port_lid_tbl; - ib_net16_t master_sm_base_lid; - ib_net16_t sm_base_lid; - ib_net64_t sm_port_guid; - uint8_t sm_state; - osm_subn_opt_t opt; - uint16_t max_unicast_lid_ho; - uint16_t max_multicast_lid_ho; - uint8_t min_ca_mtu; - uint8_t min_ca_rate; - boolean_t ignore_existing_lfts; - boolean_t subnet_initialization_error; - boolean_t force_immediate_heavy_sweep; - boolean_t force_delayed_heavy_sweep; - boolean_t in_sweep_hop_0; - boolean_t moved_to_master_state; - boolean_t first_time_master_sweep; - boolean_t coming_out_of_standby; - unsigned need_update; +typedef struct _osm_subn { + struct _osm_opensm_t *p_osm; + cl_qmap_t sw_guid_tbl; + cl_qmap_t node_guid_tbl; + cl_qmap_t port_guid_tbl; + cl_qmap_t rtr_guid_tbl; + cl_qmap_t prtn_pkey_tbl; + cl_qmap_t mgrp_mlid_tbl; + cl_qmap_t sm_guid_tbl; + cl_qlist_t sa_sr_list; + cl_qlist_t sa_infr_list; + cl_ptr_vector_t port_lid_tbl; + ib_net16_t master_sm_base_lid; + ib_net16_t sm_base_lid; + ib_net64_t sm_port_guid; + uint8_t sm_state; + osm_subn_opt_t opt; + uint16_t max_unicast_lid_ho; + uint16_t max_multicast_lid_ho; + uint8_t min_ca_mtu; + uint8_t min_ca_rate; + boolean_t ignore_existing_lfts; + boolean_t subnet_initialization_error; + boolean_t force_immediate_heavy_sweep; + boolean_t force_delayed_heavy_sweep; + boolean_t in_sweep_hop_0; + boolean_t moved_to_master_state; + boolean_t first_time_master_sweep; + boolean_t coming_out_of_standby; + unsigned need_update; } osm_subn_t; /* * FIELDS @@ -716,9 +709,7 @@ typedef struct _osm_subn * * SYNOPSIS */ -void -osm_subn_construct( - IN osm_subn_t* const p_subn ); +void osm_subn_construct(IN osm_subn_t * const p_subn); /* * PARAMETERS * p_subn @@ -747,9 +738,7 @@ osm_subn_construct( * * SYNOPSIS */ -void -osm_subn_destroy( - IN osm_subn_t* const p_subn ); +void osm_subn_destroy(IN osm_subn_t * const p_subn); /* * PARAMETERS * p_subn @@ -778,10 +767,9 @@ osm_subn_destroy( * SYNOPSIS */ ib_api_status_t -osm_subn_init( - IN osm_subn_t* const p_subn, - IN struct _osm_opensm_t * const p_osm, - IN const osm_subn_opt_t* const p_opt ); +osm_subn_init(IN osm_subn_t * const p_subn, + IN struct _osm_opensm_t *const p_osm, + IN const osm_subn_opt_t * const p_opt); /* * PARAMETERS * p_subn @@ -822,11 +810,10 @@ struct _osm_port; * SYNOPSIS */ ib_api_status_t -osm_get_gid_by_mad_addr( - IN struct _osm_log *p_log, - IN const osm_subn_t *p_subn, - IN const struct _osm_mad_addr *p_mad_addr, - OUT ib_gid_t *p_gid); +osm_get_gid_by_mad_addr(IN struct _osm_log *p_log, + IN const osm_subn_t * p_subn, + IN const struct _osm_mad_addr *p_mad_addr, + OUT ib_gid_t * p_gid); /* * PARAMETERS * p_log @@ -861,11 +848,10 @@ osm_get_gid_by_mad_addr( * * SYNOPSIS */ -struct _osm_physp * -osm_get_physp_by_mad_addr( - IN struct _osm_log *p_log, - IN const osm_subn_t *p_subn, - IN struct _osm_mad_addr *p_mad_addr ); +struct _osm_physp *osm_get_physp_by_mad_addr(IN struct _osm_log *p_log, + IN const osm_subn_t * p_subn, + IN struct _osm_mad_addr + *p_mad_addr); /* * PARAMETERS * p_log @@ -897,11 +883,9 @@ osm_get_physp_by_mad_addr( * * SYNOPSIS */ -struct _osm_port * -osm_get_port_by_mad_addr( - IN struct _osm_log *p_log, - IN const osm_subn_t *p_subn, - IN struct _osm_mad_addr *p_mad_addr ); +struct _osm_port *osm_get_port_by_mad_addr(IN struct _osm_log *p_log, + IN const osm_subn_t * p_subn, + IN struct _osm_mad_addr *p_mad_addr); /* * PARAMETERS * p_log @@ -932,10 +916,8 @@ osm_get_port_by_mad_addr( * * SYNOPSIS */ -struct _osm_switch * -osm_get_switch_by_guid( - IN const osm_subn_t *p_subn, - IN uint64_t guid ); +struct _osm_switch *osm_get_switch_by_guid(IN const osm_subn_t * p_subn, + IN uint64_t guid); /* * PARAMETERS * p_subn @@ -963,10 +945,8 @@ osm_get_switch_by_guid( * * SYNOPSIS */ -struct _osm_node * -osm_get_node_by_guid( - IN osm_subn_t const *p_subn, - IN uint64_t guid ); +struct _osm_node *osm_get_node_by_guid(IN osm_subn_t const *p_subn, + IN uint64_t guid); /* * PARAMETERS * p_subn @@ -994,10 +974,8 @@ osm_get_node_by_guid( * * SYNOPSIS */ -struct _osm_port * -osm_get_port_by_guid( - IN osm_subn_t const *p_subn, - IN uint64_t guid ); +struct _osm_port *osm_get_port_by_guid(IN osm_subn_t const *p_subn, + IN uint64_t guid); /* * PARAMETERS * p_subn @@ -1026,11 +1004,10 @@ osm_get_port_by_guid( * * SYNOPSIS */ -struct _osm_physp * -osm_get_physp_by_mad_addr( - IN struct _osm_log *p_log, - IN const osm_subn_t *p_subn, - IN struct _osm_mad_addr *p_mad_addr ); +struct _osm_physp *osm_get_physp_by_mad_addr(IN struct _osm_log *p_log, + IN const osm_subn_t * p_subn, + IN struct _osm_mad_addr + *p_mad_addr); /* * PARAMETERS * p_log @@ -1059,9 +1036,7 @@ osm_get_physp_by_mad_addr( * * SYNOPSIS */ -void -osm_subn_set_default_opt( - IN osm_subn_opt_t* const p_opt ); +void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt); /* * PARAMETERS * @@ -1086,9 +1061,7 @@ osm_subn_set_default_opt( * * SYNOPSIS */ -void -osm_subn_set_default_opt( - IN osm_subn_opt_t* const p_opt ); +void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt); /* * PARAMETERS * @@ -1114,9 +1087,7 @@ osm_subn_set_default_opt( * * SYNOPSIS */ -ib_api_status_t -osm_subn_parse_conf_file( - IN osm_subn_opt_t* const p_opt ); +ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opt); /* * PARAMETERS * @@ -1144,9 +1115,7 @@ osm_subn_parse_conf_file( * * SYNOPSIS */ -ib_api_status_t -osm_subn_rescan_conf_file( - IN osm_subn_opt_t* const p_opts ); +ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts); /* * PARAMETERS * @@ -1170,9 +1139,7 @@ osm_subn_rescan_conf_file( * * SYNOPSIS */ -ib_api_status_t -osm_subn_write_conf_file( - IN osm_subn_opt_t* const p_opt ); +ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opt); /* * PARAMETERS * @@ -1191,5 +1158,4 @@ osm_subn_write_conf_file( *********/ END_C_DECLS - -#endif /* _OSM_SUBNET_H_ */ +#endif /* _OSM_SUBNET_H_ */ diff --git a/opensm/include/opensm/osm_sw_info_rcv.h b/opensm/include/opensm/osm_sw_info_rcv.h index 41b4238..724d0d3 100644 --- a/opensm/include/opensm/osm_sw_info_rcv.h +++ b/opensm/include/opensm/osm_sw_info_rcv.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Switch Info Receiver * NAME * Switch Info Receiver @@ -84,7 +83,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Switch Info Receiver/osm_si_rcv_t * NAME * osm_si_rcv_t @@ -97,13 +95,12 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_si_rcv -{ - osm_subn_t *p_subn; - osm_log_t *p_log; - osm_req_t *p_req; - osm_state_mgr_t *p_state_mgr; - cl_plock_t *p_lock; +typedef struct _osm_si_rcv { + osm_subn_t *p_subn; + osm_log_t *p_log; + osm_req_t *p_req; + osm_state_mgr_t *p_state_mgr; + cl_plock_t *p_lock; } osm_si_rcv_t; /* * FIELDS @@ -135,8 +132,7 @@ typedef struct _osm_si_rcv * * SYNOPSIS */ -void osm_si_rcv_construct( - IN osm_si_rcv_t* const p_ctrl ); +void osm_si_rcv_construct(IN osm_si_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -167,8 +163,7 @@ void osm_si_rcv_construct( * * SYNOPSIS */ -void osm_si_rcv_destroy( - IN osm_si_rcv_t* const p_ctrl ); +void osm_si_rcv_destroy(IN osm_si_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -199,13 +194,12 @@ void osm_si_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_si_rcv_init( - IN osm_si_rcv_t* const p_ctrl, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_req_t* const p_req, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_si_rcv_init(IN osm_si_rcv_t * const p_ctrl, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_req_t * const p_req, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -247,8 +241,7 @@ ib_api_status_t osm_si_rcv_init( * * SYNOPSIS */ -boolean_t osm_si_rcv_is_inited( - IN const osm_si_rcv_t* const p_ctrl ); +boolean_t osm_si_rcv_is_inited(IN const osm_si_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -276,9 +269,7 @@ boolean_t osm_si_rcv_is_inited( * * SYNOPSIS */ -void osm_si_rcv_process( - IN void *context, - IN void *data ); +void osm_si_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -299,5 +290,4 @@ void osm_si_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_SI_RCV_H_ */ +#endif /* _OSM_SI_RCV_H_ */ diff --git a/opensm/include/opensm/osm_sweep_fail_ctrl.h b/opensm/include/opensm/osm_sweep_fail_ctrl.h index 30f748e..579ef58 100644 --- a/opensm/include/opensm/osm_sweep_fail_ctrl.h +++ b/opensm/include/opensm/osm_sweep_fail_ctrl.h @@ -58,13 +58,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Sweep Fail Controller * NAME * Sweep Fail Controller @@ -83,7 +82,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Sweep Fail Controller/osm_sweep_fail_ctrl_t * NAME * osm_sweep_fail_ctrl_t @@ -96,13 +94,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sweep_fail_ctrl -{ - osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; - cl_dispatcher_t *p_disp; - cl_disp_reg_handle_t h_disp; - +typedef struct _osm_sweep_fail_ctrl { + osm_log_t *p_log; + osm_state_mgr_t *p_state_mgr; + cl_dispatcher_t *p_disp; + cl_disp_reg_handle_t h_disp; } osm_sweep_fail_ctrl_t; /* @@ -133,9 +129,7 @@ typedef struct _osm_sweep_fail_ctrl * * SYNOPSIS */ -void -osm_sweep_fail_ctrl_construct( - IN osm_sweep_fail_ctrl_t* const p_ctrl ); +void osm_sweep_fail_ctrl_construct(IN osm_sweep_fail_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -166,9 +160,7 @@ osm_sweep_fail_ctrl_construct( * * SYNOPSIS */ -void -osm_sweep_fail_ctrl_destroy( - IN osm_sweep_fail_ctrl_t* const p_ctrl ); +void osm_sweep_fail_ctrl_destroy(IN osm_sweep_fail_ctrl_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -200,11 +192,10 @@ osm_sweep_fail_ctrl_destroy( * SYNOPSIS */ ib_api_status_t -osm_sweep_fail_ctrl_init( - IN osm_sweep_fail_ctrl_t* const p_ctrl, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_dispatcher_t* const p_disp ); +osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_dispatcher_t * const p_disp); /* * PARAMETERS * p_ctrl @@ -235,5 +226,4 @@ osm_sweep_fail_ctrl_init( *********/ END_C_DECLS - -#endif /* _OSM_SWEEP_FAIL_CTRL_H_ */ +#endif /* _OSM_SWEEP_FAIL_CTRL_H_ */ diff --git a/opensm/include/opensm/osm_switch.h b/opensm/include/opensm/osm_switch.h index 9364d2c..e294527 100644 --- a/opensm/include/opensm/osm_switch.h +++ b/opensm/include/opensm/osm_switch.h @@ -60,13 +60,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Switch * NAME * Switch @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Switch/osm_switch_t * NAME * osm_switch_t @@ -99,21 +97,20 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_switch -{ - cl_map_item_t map_item; - osm_node_t *p_node; - ib_switch_info_t switch_info; - uint16_t max_lid_ho; - uint8_t num_ports; - uint16_t num_hops; - uint8_t **hops; - osm_port_profile_t *p_prof; - osm_fwd_tbl_t fwd_tbl; - osm_mcast_tbl_t mcast_tbl; - uint32_t discovery_count; - unsigned need_update; - void *priv; +typedef struct _osm_switch { + cl_map_item_t map_item; + osm_node_t *p_node; + ib_switch_info_t switch_info; + uint16_t max_lid_ho; + uint8_t num_ports; + uint16_t num_hops; + uint8_t **hops; + osm_port_profile_t *p_prof; + osm_fwd_tbl_t fwd_tbl; + osm_mcast_tbl_t mcast_tbl; + uint32_t discovery_count; + unsigned need_update; + void *priv; } osm_switch_t; /* * FIELDS @@ -170,9 +167,7 @@ typedef struct _osm_switch * * SYNOPSIS */ -void -osm_switch_delete( - IN OUT osm_switch_t** const pp_sw ); +void osm_switch_delete(IN OUT osm_switch_t ** const pp_sw); /* * PARAMETERS * p_sw @@ -196,10 +191,8 @@ osm_switch_delete( * * SYNOPSIS */ -osm_switch_t* -osm_switch_new( - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ); +osm_switch_t *osm_switch_new(IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw); /* * PARAMETERS * p_node @@ -228,14 +221,13 @@ osm_switch_new( * SYNOPSIS */ static inline uint8_t -osm_switch_get_hop_count( - IN const osm_switch_t* const p_sw, - IN const uint16_t lid_ho, - IN const uint8_t port_num ) +osm_switch_get_hop_count(IN const osm_switch_t * const p_sw, + IN const uint16_t lid_ho, IN const uint8_t port_num) { return (lid_ho > p_sw->max_lid_ho || !p_sw->hops[lid_ho]) ? - OSM_NO_PATH : p_sw->hops[lid_ho][port_num]; + OSM_NO_PATH : p_sw->hops[lid_ho][port_num]; } + /* * PARAMETERS * p_sw @@ -264,12 +256,12 @@ osm_switch_get_hop_count( * * SYNOPSIS */ -static inline osm_fwd_tbl_t* -osm_switch_get_fwd_tbl_ptr( - IN const osm_switch_t* const p_sw ) +static inline osm_fwd_tbl_t *osm_switch_get_fwd_tbl_ptr(IN const osm_switch_t * + const p_sw) { - return( (osm_fwd_tbl_t*)&p_sw->fwd_tbl ); + return ((osm_fwd_tbl_t *) & p_sw->fwd_tbl); } + /* * PARAMETERS * p_sw @@ -293,11 +285,9 @@ osm_switch_get_fwd_tbl_ptr( * SYNOPSIS */ cl_status_t -osm_switch_set_hops( - IN osm_switch_t* const p_sw, - IN const uint16_t lid_ho, - IN const uint8_t port_num, - IN const uint8_t num_hops ); +osm_switch_set_hops(IN osm_switch_t * const p_sw, + IN const uint16_t lid_ho, + IN const uint8_t port_num, IN const uint8_t num_hops); /* * PARAMETERS * p_sw @@ -329,9 +319,7 @@ osm_switch_set_hops( * * SYNOPSIS */ -void -osm_switch_clear_hops( - IN osm_switch_t *p_sw ); +void osm_switch_clear_hops(IN osm_switch_t * p_sw); /* * PARAMETERS * p_sw @@ -353,13 +341,13 @@ osm_switch_clear_hops( * SYNOPSIS */ static inline uint8_t -osm_switch_get_least_hops( - IN const osm_switch_t* const p_sw, - IN const uint16_t lid_ho ) +osm_switch_get_least_hops(IN const osm_switch_t * const p_sw, + IN const uint16_t lid_ho) { return (lid_ho > p_sw->max_lid_ho || !p_sw->hops[lid_ho]) ? - OSM_NO_PATH : p_sw->hops[lid_ho][0]; + OSM_NO_PATH : p_sw->hops[lid_ho][0]; } + /* * PARAMETERS * p_sw @@ -389,9 +377,8 @@ osm_switch_get_least_hops( * SYNOPSIS */ uint8_t -osm_switch_get_port_least_hops( - IN const osm_switch_t* const p_sw, - IN const osm_port_t *p_port ); +osm_switch_get_port_least_hops(IN const osm_switch_t * const p_sw, + IN const osm_port_t * p_port); /* * PARAMETERS * p_sw @@ -421,12 +408,12 @@ osm_switch_get_port_least_hops( * SYNOPSIS */ static inline uint8_t -osm_switch_get_port_by_lid( - IN const osm_switch_t* const p_sw, - IN const uint16_t lid_ho ) +osm_switch_get_port_by_lid(IN const osm_switch_t * const p_sw, + IN const uint16_t lid_ho) { - return( osm_fwd_tbl_get( &p_sw->fwd_tbl, lid_ho ) ); + return (osm_fwd_tbl_get(&p_sw->fwd_tbl, lid_ho)); } + /* * PARAMETERS * p_sw @@ -453,10 +440,8 @@ osm_switch_get_port_by_lid( * * SYNOPSIS */ -osm_physp_t* -osm_switch_get_physp_ptr( - IN const osm_switch_t* const p_sw, - IN const uint32_t port_num ); +osm_physp_t *osm_switch_get_physp_ptr(IN const osm_switch_t * const p_sw, + IN const uint32_t port_num); /* * PARAMETERS * p_sw @@ -486,29 +471,29 @@ osm_switch_get_physp_ptr( * * SYNOPSIS */ -static inline osm_physp_t* -osm_switch_get_route_by_lid( - IN const osm_switch_t* const p_sw, - IN const ib_net16_t lid ) +static inline osm_physp_t *osm_switch_get_route_by_lid(IN const osm_switch_t * + const p_sw, + IN const ib_net16_t lid) { uint8_t port_num; - CL_ASSERT( p_sw ); - CL_ASSERT( lid ); + CL_ASSERT(p_sw); + CL_ASSERT(lid); - port_num = osm_fwd_tbl_get( &p_sw->fwd_tbl, cl_ntoh16( lid ) ); + port_num = osm_fwd_tbl_get(&p_sw->fwd_tbl, cl_ntoh16(lid)); /* In order to avoid holes in the subnet (usually happens when running UPDN algorithm), i.e. cases where port is unreachable through a switch (we put an OSM_NO_PATH value at the port entry, we do not assert on unreachable lid entries at the fwd table but return NULL - */ + */ if (port_num != OSM_NO_PATH) - return( osm_node_get_physp_ptr( p_sw->p_node, port_num ) ); + return (osm_node_get_physp_ptr(p_sw->p_node, port_num)); else return NULL; } + /* * PARAMETERS * p_sw @@ -539,13 +524,13 @@ osm_switch_get_route_by_lid( * */ static inline unsigned -osm_switch_sp0_is_lmc_capable( - IN const osm_switch_t* const p_sw, - IN osm_subn_t *p_subn) +osm_switch_sp0_is_lmc_capable(IN const osm_switch_t * const p_sw, + IN osm_subn_t * p_subn) { return (p_subn->opt.lmc_esp0 && ib_switch_info_is_enhanced_port0(&p_sw->switch_info)) ? 1 : 0; } + /* * PARAMETERS * p_sw @@ -574,12 +559,12 @@ osm_switch_sp0_is_lmc_capable( * SYNOPSIS */ static inline uint32_t -osm_switch_get_max_block_id( - IN const osm_switch_t* const p_sw ) +osm_switch_get_max_block_id(IN const osm_switch_t * const p_sw) { - return( (uint32_t)(osm_fwd_tbl_get_size( &p_sw->fwd_tbl ) / - osm_fwd_tbl_get_lids_per_block( &p_sw->fwd_tbl ) ) ); + return ((uint32_t) (osm_fwd_tbl_get_size(&p_sw->fwd_tbl) / + osm_fwd_tbl_get_lids_per_block(&p_sw->fwd_tbl))); } + /* * PARAMETERS * p_sw @@ -605,12 +590,13 @@ osm_switch_get_max_block_id( * SYNOPSIS */ static inline uint16_t -osm_switch_get_max_block_id_in_use( - IN const osm_switch_t* const p_sw ) +osm_switch_get_max_block_id_in_use(IN const osm_switch_t * const p_sw) { - return( osm_fwd_tbl_get_max_block_id_in_use( &p_sw->fwd_tbl, - cl_ntoh16( p_sw->switch_info.lin_top ) ) ); + return (osm_fwd_tbl_get_max_block_id_in_use(&p_sw->fwd_tbl, + cl_ntoh16(p_sw->switch_info. + lin_top))); } + /* * PARAMETERS * p_sw @@ -635,10 +621,9 @@ osm_switch_get_max_block_id_in_use( * SYNOPSIS */ boolean_t -osm_switch_get_fwd_tbl_block( - IN const osm_switch_t* const p_sw, - IN const uint32_t block_id, - OUT uint8_t* const p_block ); +osm_switch_get_fwd_tbl_block(IN const osm_switch_t * const p_sw, + IN const uint32_t block_id, + OUT uint8_t * const p_block); /* * PARAMETERS * p_sw @@ -671,11 +656,11 @@ osm_switch_get_fwd_tbl_block( * SYNOPSIS */ static inline boolean_t -osm_switch_supports_mcast( - IN const osm_switch_t* const p_sw ) +osm_switch_supports_mcast(IN const osm_switch_t * const p_sw) { - return( p_sw->switch_info.mcast_cap != 0 ); + return (p_sw->switch_info.mcast_cap != 0); } + /* * PARAMETERS * p_sw @@ -700,14 +685,14 @@ osm_switch_supports_mcast( * SYNOPSIS */ static inline void -osm_switch_set_switch_info( - IN osm_switch_t* const p_sw, - IN const ib_switch_info_t* const p_si ) +osm_switch_set_switch_info(IN osm_switch_t * const p_sw, + IN const ib_switch_info_t * const p_si) { - CL_ASSERT( p_sw ); - CL_ASSERT( p_si ); + CL_ASSERT(p_sw); + CL_ASSERT(p_si); p_sw->switch_info = *p_si; } + /* * PARAMETERS * p_sw @@ -734,12 +719,11 @@ osm_switch_set_switch_info( * SYNOPSIS */ static inline void -osm_switch_count_path( - IN osm_switch_t* const p_sw, - IN const uint8_t port ) +osm_switch_count_path(IN osm_switch_t * const p_sw, IN const uint8_t port) { - osm_port_prof_path_count_inc( &p_sw->p_prof[port] ); + osm_port_prof_path_count_inc(&p_sw->p_prof[port]); } + /* * PARAMETERS * p_sw @@ -766,14 +750,14 @@ osm_switch_count_path( * SYNOPSIS */ static inline ib_api_status_t -osm_switch_set_ft_block( - IN osm_switch_t* const p_sw, - IN const uint8_t* const p_block, - IN const uint32_t block_num ) +osm_switch_set_ft_block(IN osm_switch_t * const p_sw, + IN const uint8_t * const p_block, + IN const uint32_t block_num) { - CL_ASSERT( p_sw ); - return( osm_fwd_tbl_set_block( &p_sw->fwd_tbl, p_block, block_num ) ); + CL_ASSERT(p_sw); + return (osm_fwd_tbl_set_block(&p_sw->fwd_tbl, p_block, block_num)); } + /* * PARAMETERS * p_sw @@ -803,16 +787,15 @@ osm_switch_set_ft_block( * SYNOPSIS */ static inline ib_api_status_t -osm_switch_set_mft_block( - IN osm_switch_t* const p_sw, - IN const ib_net16_t* const p_block, - IN const uint16_t block_num, - IN const uint8_t position ) +osm_switch_set_mft_block(IN osm_switch_t * const p_sw, + IN const ib_net16_t * const p_block, + IN const uint16_t block_num, IN const uint8_t position) { - CL_ASSERT( p_sw ); - return( osm_mcast_tbl_set_block( &p_sw->mcast_tbl, p_block, - block_num, position ) ); + CL_ASSERT(p_sw); + return (osm_mcast_tbl_set_block(&p_sw->mcast_tbl, p_block, + block_num, position)); } + /* * PARAMETERS * p_sw @@ -845,16 +828,16 @@ osm_switch_set_mft_block( * SYNOPSIS */ static inline boolean_t -osm_switch_get_mft_block( - IN osm_switch_t* const p_sw, - IN const uint16_t block_num, - IN const uint8_t position, - OUT ib_net16_t* const p_block ) +osm_switch_get_mft_block(IN osm_switch_t * const p_sw, + IN const uint16_t block_num, + IN const uint8_t position, + OUT ib_net16_t * const p_block) { - CL_ASSERT( p_sw ); - return( osm_mcast_tbl_get_block( &p_sw->mcast_tbl, - block_num, position, p_block ) ); + CL_ASSERT(p_sw); + return (osm_mcast_tbl_get_block(&p_sw->mcast_tbl, + block_num, position, p_block)); } + /* * PARAMETERS * p_sw @@ -889,12 +872,12 @@ osm_switch_get_mft_block( * SYNOPSIS */ static inline uint16_t -osm_switch_get_mft_max_block( - IN osm_switch_t* const p_sw ) +osm_switch_get_mft_max_block(IN osm_switch_t * const p_sw) { - CL_ASSERT( p_sw ); - return( osm_mcast_tbl_get_max_block( &p_sw->mcast_tbl ) ); + CL_ASSERT(p_sw); + return (osm_mcast_tbl_get_max_block(&p_sw->mcast_tbl)); } + /* * PARAMETERS * p_sw @@ -913,12 +896,12 @@ osm_switch_get_mft_max_block( * SYNOPSIS */ static inline int16_t -osm_switch_get_mft_max_block_in_use( - IN osm_switch_t* const p_sw ) +osm_switch_get_mft_max_block_in_use(IN osm_switch_t * const p_sw) { - CL_ASSERT( p_sw ); - return( osm_mcast_tbl_get_max_block_in_use( &p_sw->mcast_tbl ) ); + CL_ASSERT(p_sw); + return (osm_mcast_tbl_get_max_block_in_use(&p_sw->mcast_tbl)); } + /* * PARAMETERS * p_sw @@ -943,12 +926,12 @@ osm_switch_get_mft_max_block_in_use( * SYNOPSIS */ static inline uint8_t -osm_switch_get_mft_max_position( - IN osm_switch_t* const p_sw ) +osm_switch_get_mft_max_position(IN osm_switch_t * const p_sw) { - CL_ASSERT( p_sw ); - return( osm_mcast_tbl_get_max_position( &p_sw->mcast_tbl ) ); + CL_ASSERT(p_sw); + return (osm_mcast_tbl_get_max_position(&p_sw->mcast_tbl)); } + /* * PARAMETERS * p_sw @@ -971,15 +954,14 @@ osm_switch_get_mft_max_position( * SYNOPSIS */ uint8_t -osm_switch_recommend_path( - IN const osm_switch_t* const p_sw, - IN osm_port_t *p_port, - IN const uint16_t lid_ho, - IN const boolean_t ignore_existing, - IN OUT uint64_t *remote_sys_guids, - IN OUT uint16_t *p_num_used_sys, - IN OUT uint64_t *remote_node_guids, - IN OUT uint16_t *p_num_used_nodes ); +osm_switch_recommend_path(IN const osm_switch_t * const p_sw, + IN osm_port_t * p_port, + IN const uint16_t lid_ho, + IN const boolean_t ignore_existing, + IN OUT uint64_t * remote_sys_guids, + IN OUT uint16_t * p_num_used_sys, + IN OUT uint64_t * remote_node_guids, + IN OUT uint16_t * p_num_used_nodes); /* * PARAMETERS * p_sw @@ -1032,11 +1014,10 @@ osm_switch_recommend_path( * SYNOPSIS */ uint8_t -osm_switch_recommend_mcast_path( - IN osm_switch_t* const p_sw, - IN osm_port_t *p_port, - IN const uint16_t mlid_ho, - IN const boolean_t ignore_existing ); +osm_switch_recommend_mcast_path(IN osm_switch_t * const p_sw, + IN osm_port_t * p_port, + IN const uint16_t mlid_ho, + IN const boolean_t ignore_existing); /* * PARAMETERS * p_sw @@ -1073,11 +1054,11 @@ osm_switch_recommend_mcast_path( * SYNOPSIS */ static inline uint16_t -osm_switch_get_fwd_tbl_size( - IN const osm_switch_t* const p_sw ) +osm_switch_get_fwd_tbl_size(IN const osm_switch_t * const p_sw) { - return( osm_fwd_tbl_get_size( &p_sw->fwd_tbl ) ); + return (osm_fwd_tbl_get_size(&p_sw->fwd_tbl)); } + /* * PARAMETERS * p_sw @@ -1101,11 +1082,11 @@ osm_switch_get_fwd_tbl_size( * SYNOPSIS */ static inline uint16_t -osm_switch_get_mcast_fwd_tbl_size( - IN const osm_switch_t* const p_sw ) +osm_switch_get_mcast_fwd_tbl_size(IN const osm_switch_t * const p_sw) { - return( cl_ntoh16( p_sw->switch_info.mcast_cap ) ); + return (cl_ntoh16(p_sw->switch_info.mcast_cap)); } + /* * PARAMETERS * p_sw @@ -1129,12 +1110,12 @@ osm_switch_get_mcast_fwd_tbl_size( * SYNOPSIS */ static inline uint32_t -osm_switch_path_count_get( - IN const osm_switch_t* const p_sw, - IN const uint8_t port_num ) +osm_switch_path_count_get(IN const osm_switch_t * const p_sw, + IN const uint8_t port_num) { - return( osm_port_prof_path_count_get( &p_sw->p_prof[port_num] ) ); + return (osm_port_prof_path_count_get(&p_sw->p_prof[port_num])); } + /* * PARAMETERS * p_sw @@ -1161,9 +1142,7 @@ osm_switch_path_count_get( * SYNOPSIS */ int -osm_switch_prepare_path_rebuild( - IN osm_switch_t* p_sw, - IN uint16_t max_lids ); +osm_switch_prepare_path_rebuild(IN osm_switch_t * p_sw, IN uint16_t max_lids); /* * PARAMETERS * p_sw @@ -1189,12 +1168,13 @@ osm_switch_prepare_path_rebuild( * * SYNOPSIS */ -static inline osm_mcast_tbl_t* -osm_switch_get_mcast_tbl_ptr( - IN const osm_switch_t* const p_sw ) +static inline osm_mcast_tbl_t *osm_switch_get_mcast_tbl_ptr(IN const + osm_switch_t * + const p_sw) { - return( (osm_mcast_tbl_t*)&p_sw->mcast_tbl ); + return ((osm_mcast_tbl_t *) & p_sw->mcast_tbl); } + /* * PARAMETERS * p_sw @@ -1219,18 +1199,18 @@ osm_switch_get_mcast_tbl_ptr( * SYNOPSIS */ static inline boolean_t -osm_switch_is_in_mcast_tree( - IN const osm_switch_t* const p_sw, - IN const uint16_t mlid_ho ) +osm_switch_is_in_mcast_tree(IN const osm_switch_t * const p_sw, + IN const uint16_t mlid_ho) { - const osm_mcast_tbl_t* p_tbl; + const osm_mcast_tbl_t *p_tbl; p_tbl = &p_sw->mcast_tbl; - if( p_tbl ) - return( osm_mcast_tbl_is_any_port( &p_sw->mcast_tbl, mlid_ho ) ); + if (p_tbl) + return (osm_mcast_tbl_is_any_port(&p_sw->mcast_tbl, mlid_ho)); else - return( FALSE ); + return (FALSE); } + /* * PARAMETERS * p_sw @@ -1249,5 +1229,4 @@ osm_switch_is_in_mcast_tree( *********/ END_C_DECLS - -#endif /* _OSM_SWITCH_H_ */ +#endif /* _OSM_SWITCH_H_ */ diff --git a/opensm/include/opensm/osm_trap_rcv.h b/opensm/include/opensm/osm_trap_rcv.h index e1077cf..393c5b1 100644 --- a/opensm/include/opensm/osm_trap_rcv.h +++ b/opensm/include/opensm/osm_trap_rcv.h @@ -62,13 +62,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/Trap Receiver * NAME * Trap Receiver @@ -86,7 +85,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Trap Receiver/osm_trap_rcv_t * NAME * osm_trap_rcv_t @@ -99,15 +97,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_trap_rcv -{ - osm_subn_t *p_subn; - osm_stats_t *p_stats; - osm_log_t *p_log; - osm_resp_t *p_resp; - osm_state_mgr_t *p_state_mgr; - cl_plock_t *p_lock; - cl_event_wheel_t trap_aging_tracker; +typedef struct _osm_trap_rcv { + osm_subn_t *p_subn; + osm_stats_t *p_stats; + osm_log_t *p_log; + osm_resp_t *p_resp; + osm_state_mgr_t *p_state_mgr; + cl_plock_t *p_lock; + cl_event_wheel_t trap_aging_tracker; } osm_trap_rcv_t; /* * FIELDS @@ -147,8 +144,7 @@ typedef struct _osm_trap_rcv * * SYNOPSIS */ -void osm_trap_rcv_construct( - IN osm_trap_rcv_t* const p_rcv ); +void osm_trap_rcv_construct(IN osm_trap_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -178,8 +174,7 @@ void osm_trap_rcv_construct( * * SYNOPSIS */ -void osm_trap_rcv_destroy( - IN osm_trap_rcv_t* const p_rcv ); +void osm_trap_rcv_destroy(IN osm_trap_rcv_t * const p_rcv); /* * PARAMETERS * p_rcv @@ -210,14 +205,13 @@ void osm_trap_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_trap_rcv_init( - IN osm_trap_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_stats_t* const p_stats, - IN osm_resp_t* const p_resp, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_stats_t * const p_stats, + IN osm_resp_t * const p_resp, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_rcv @@ -262,9 +256,7 @@ ib_api_status_t osm_trap_rcv_init( * * SYNOPSIS */ -void osm_trap_rcv_process( - IN void *context, - IN void *data ); +void osm_trap_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -294,10 +286,8 @@ void osm_trap_rcv_process( * SYNOPSIS */ uint64_t -osm_trap_rcv_aging_tracker_callback( - IN uint64_t key, - IN uint32_t num_regs, - IN void* context ); +osm_trap_rcv_aging_tracker_callback(IN uint64_t key, + IN uint32_t num_regs, IN void *context); /* * PARAMETERS @@ -322,5 +312,4 @@ osm_trap_rcv_aging_tracker_callback( *********/ END_C_DECLS - -#endif /* _OSM_TRAP_RCV_H_ */ +#endif /* _OSM_TRAP_RCV_H_ */ diff --git a/opensm/include/opensm/osm_ts_useraccess.h b/opensm/include/opensm/osm_ts_useraccess.h index 8bb03b8..d68c924 100644 --- a/opensm/include/opensm/osm_ts_useraccess.h +++ b/opensm/include/opensm/osm_ts_useraccess.h @@ -38,17 +38,15 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - -typedef struct ib_user_mad_filter osm_ts_user_mad_filter; -typedef struct ib_set_port_info_ioctl osm_ts_set_port_info_ioctl; -typedef struct ib_get_port_info_ioctl osm_ts_get_port_info_ioctl; -typedef struct ib_gid_entry_ioctl osm_ts_gid_entry_ioctl; +typedef struct ib_user_mad_filter osm_ts_user_mad_filter; +typedef struct ib_set_port_info_ioctl osm_ts_set_port_info_ioctl; +typedef struct ib_get_port_info_ioctl osm_ts_get_port_info_ioctl; +typedef struct ib_gid_entry_ioctl osm_ts_gid_entry_ioctl; END_C_DECLS - diff --git a/opensm/include/opensm/osm_ucast_mgr.h b/opensm/include/opensm/osm_ucast_mgr.h index 4824971..c3c26e4 100644 --- a/opensm/include/opensm/osm_ucast_mgr.h +++ b/opensm/include/opensm/osm_ucast_mgr.h @@ -59,15 +59,13 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #define OSM_UCAST_MGR_LIST_SIZE_MIN 256 - /****h* OpenSM/Unicast Manager * NAME * Unicast Manager @@ -85,7 +83,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****s* OpenSM: Unicast Manager/osm_ucast_mgr_t * NAME * osm_ucast_mgr_t @@ -98,15 +95,14 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_ucast_mgr -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; - boolean_t any_change; - boolean_t some_hop_count_set; - uint8_t *lft_buf; +typedef struct _osm_ucast_mgr { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; + boolean_t any_change; + boolean_t some_hop_count_set; + uint8_t *lft_buf; } osm_ucast_mgr_t; /* * FIELDS @@ -148,9 +144,7 @@ typedef struct _osm_ucast_mgr * * SYNOPSIS */ -void -osm_ucast_mgr_construct( - IN osm_ucast_mgr_t* const p_mgr ); +void osm_ucast_mgr_construct(IN osm_ucast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -180,9 +174,7 @@ osm_ucast_mgr_construct( * * SYNOPSIS */ -void -osm_ucast_mgr_destroy( - IN osm_ucast_mgr_t* const p_mgr ); +void osm_ucast_mgr_destroy(IN osm_ucast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -214,12 +206,10 @@ osm_ucast_mgr_destroy( * SYNOPSIS */ ib_api_status_t -osm_ucast_mgr_init( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +osm_ucast_mgr_init(IN osm_ucast_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_mgr @@ -259,9 +249,8 @@ osm_ucast_mgr_init( * SYNOPSIS */ void -osm_ucast_mgr_set_fwd_table( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_switch_t* const p_sw ); +osm_ucast_mgr_set_fwd_table(IN osm_ucast_mgr_t * const p_mgr, + IN osm_switch_t * const p_sw); /* * PARAMETERS * p_mgr @@ -283,9 +272,7 @@ osm_ucast_mgr_set_fwd_table( * * SYNOPSIS */ -void -osm_ucast_mgr_build_lid_matrices( - IN osm_ucast_mgr_t* const p_mgr ); +void osm_ucast_mgr_build_lid_matrices(IN osm_ucast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -309,10 +296,9 @@ osm_ucast_mgr_build_lid_matrices( * SYNOPSIS */ cl_status_t -osm_ucast_mgr_read_guid_file( - IN osm_ucast_mgr_t * const p_mgr, - IN const char * guid_file_name, - IN cl_list_t * p_list ); +osm_ucast_mgr_read_guid_file(IN osm_ucast_mgr_t * const p_mgr, + IN const char *guid_file_name, + IN cl_list_t * p_list); /* * PARAMETERS * p_mgr @@ -344,9 +330,7 @@ osm_ucast_mgr_read_guid_file( * * SYNOPSIS */ -osm_signal_t -osm_ucast_mgr_process( - IN osm_ucast_mgr_t* const p_mgr ); +osm_signal_t osm_ucast_mgr_process(IN osm_ucast_mgr_t * const p_mgr); /* * PARAMETERS * p_mgr @@ -366,5 +350,4 @@ osm_ucast_mgr_process( * Unicast Manager, Node Info Response Controller *********/ END_C_DECLS - -#endif /* _OSM_UCAST_MGR_H_ */ +#endif /* _OSM_UCAST_MGR_H_ */ diff --git a/opensm/include/opensm/osm_umadt.h b/opensm/include/opensm/osm_umadt.h index 9462554..809f478 100644 --- a/opensm/include/opensm/osm_umadt.h +++ b/opensm/include/opensm/osm_umadt.h @@ -57,24 +57,24 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS -typedef struct _umadt_obj_t{ - void* umadt_handle; +typedef struct _umadt_obj_t { + void *umadt_handle; UMADT_INTERFACE uMadtInterface; - IBT_INTERFACE IbtInterface; - boolean init_done; - cl_spinlock_t register_lock; - cl_qlist_t register_list; - osm_log_t *p_log; - uint32_t timeout; + IBT_INTERFACE IbtInterface; + boolean init_done; + cl_spinlock_t register_lock; + cl_qlist_t register_list; + osm_log_t *p_log; + uint32_t timeout; -}umadt_obj_t; +} umadt_obj_t; /*********/ /****s* OpenSM: Umadt MAD Wrapper/osm_bind_info @@ -87,28 +87,28 @@ typedef struct _umadt_obj_t{ * SYNOPSIS */ -typedef struct _mad_bind_info_t{ - cl_list_item_t list_item; - umadt_obj_t *p_umadt_obj; - osm_mad_pool_t *p_mad_pool; - osm_vend_mad_recv_callback_t mad_recv_callback; - void *client_context; - cl_thread_t recv_processor_thread; - cl_spinlock_t trans_ctxt_lock; - cl_qlist_t trans_ctxt_list; - cl_timer_t timeout_timer; - cl_spinlock_t timeout_list_lock; - cl_qlist_t timeout_list; - RegisterClassStruct umadt_reg_class; - MADT_HANDLE umadt_handle; /* Umadt type */ +typedef struct _mad_bind_info_t { + cl_list_item_t list_item; + umadt_obj_t *p_umadt_obj; + osm_mad_pool_t *p_mad_pool; + osm_vend_mad_recv_callback_t mad_recv_callback; + void *client_context; + cl_thread_t recv_processor_thread; + cl_spinlock_t trans_ctxt_lock; + cl_qlist_t trans_ctxt_list; + cl_timer_t timeout_timer; + cl_spinlock_t timeout_list_lock; + cl_qlist_t timeout_list; + RegisterClassStruct umadt_reg_class; + MADT_HANDLE umadt_handle; /* Umadt type */ -}mad_bind_info_t; +} mad_bind_info_t; typedef struct _trans_context_t { - cl_list_item_t list_item; - uint64_t trans_id; - uint64_t sent_time; /* micro secs */ - void* context; + cl_list_item_t list_item; + uint64_t trans_id; + uint64_t sent_time; /* micro secs */ + void *context; } trans_context_t; /* @@ -139,5 +139,4 @@ typedef struct _trans_context_t { *********/ END_C_DECLS - #endif /*_OSM_UMADT_h_ */ diff --git a/opensm/include/opensm/osm_version.h b/opensm/include/opensm/osm_version.h index 1887874..f847c39 100644 --- a/opensm/include/opensm/osm_version.h +++ b/opensm/include/opensm/osm_version.h @@ -39,13 +39,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****s* OpenSM: Base/OSM_VERSION * NAME * OSM_VERSION @@ -57,7 +56,5 @@ BEGIN_C_DECLS */ #define OSM_VERSION "OpenSM Rev:openib-3.1.4" /********/ - END_C_DECLS - -#endif /* _OSM_VERSION_H_ */ +#endif /* _OSM_VERSION_H_ */ diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h index bf381df..6de9898 100644 --- a/opensm/include/opensm/osm_vl15intf.h +++ b/opensm/include/opensm/osm_vl15intf.h @@ -64,13 +64,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/VL15 * NAME * VL15 @@ -94,7 +93,6 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ - /****d* OpenSM: SM/osm_vl15_state_t * NAME * osm_vl15_state_t @@ -104,11 +102,9 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef enum _osm_vl15_state -{ +typedef enum _osm_vl15_state { OSM_VL15_STATE_INIT = 0, OSM_VL15_STATE_READY - } osm_vl15_state_t; /***********/ @@ -124,22 +120,21 @@ typedef enum _osm_vl15_state * * SYNOPSIS */ -typedef struct _osm_vl15 -{ - osm_thread_state_t thread_state; - osm_vl15_state_t state; - uint32_t max_wire_smps; - cl_event_t signal; - cl_thread_t poller; - cl_qlist_t rfifo; - cl_qlist_t ufifo; - cl_spinlock_t lock; - osm_vendor_t *p_vend; - osm_log_t *p_log; - osm_stats_t *p_stats; - osm_subn_t *p_subn; - cl_disp_reg_handle_t h_disp; - cl_plock_t *p_lock; +typedef struct _osm_vl15 { + osm_thread_state_t thread_state; + osm_vl15_state_t state; + uint32_t max_wire_smps; + cl_event_t signal; + cl_thread_t poller; + cl_qlist_t rfifo; + cl_qlist_t ufifo; + cl_spinlock_t lock; + osm_vendor_t *p_vend; + osm_log_t *p_log; + osm_stats_t *p_stats; + osm_subn_t *p_subn; + cl_disp_reg_handle_t h_disp; + cl_plock_t *p_lock; } osm_vl15_t; /* @@ -201,9 +196,7 @@ typedef struct _osm_vl15 * * SYNOPSIS */ -void -osm_vl15_construct( - IN osm_vl15_t* const p_vl15 ); +void osm_vl15_construct(IN osm_vl15_t * const p_vl15); /* * PARAMETERS * p_vl15 @@ -233,9 +226,7 @@ osm_vl15_construct( * SYNOPSIS */ void -osm_vl15_destroy( - IN osm_vl15_t* const p_vl15, - IN struct _osm_mad_pool *p_pool); +osm_vl15_destroy(IN osm_vl15_t * const p_vl15, IN struct _osm_mad_pool *p_pool); /* * PARAMETERS * p_vl15 @@ -272,15 +263,13 @@ osm_vl15_destroy( * SYNOPSIS */ ib_api_status_t -osm_vl15_init( - IN osm_vl15_t* const p_vl15, - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN const int32_t max_wire_smps, - IN osm_subn_t* const p_subn, - IN cl_dispatcher_t* const p_disp, - IN cl_plock_t* const p_lock ); +osm_vl15_init(IN osm_vl15_t * const p_vl15, + IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN const int32_t max_wire_smps, + IN osm_subn_t * const p_subn, + IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); /* * PARAMETERS * p_vl15 @@ -327,10 +316,7 @@ osm_vl15_init( * * SYNOPSIS */ -void -osm_vl15_post( - IN osm_vl15_t* const p_vl15, - IN osm_madw_t* const p_madw ); +void osm_vl15_post(IN osm_vl15_t * const p_vl15, IN osm_madw_t * const p_madw); /* * PARAMETERS * p_vl15 @@ -359,9 +345,7 @@ osm_vl15_post( * * SYNOPSIS */ -void -osm_vl15_poll( - IN osm_vl15_t* const p_vl ); +void osm_vl15_poll(IN osm_vl15_t * const p_vl); /* * PARAMETERS * p_vl15 @@ -392,9 +376,8 @@ osm_vl15_poll( * SYNOPSIS */ void -osm_vl15_shutdown( - IN osm_vl15_t* const p_vl, - IN osm_mad_pool_t* const p_mad_pool); +osm_vl15_shutdown(IN osm_vl15_t * const p_vl, + IN osm_mad_pool_t * const p_mad_pool); /* * PARAMETERS * p_vl15 @@ -413,5 +396,4 @@ osm_vl15_shutdown( *********/ END_C_DECLS - -#endif /* _OSM_VL15INTF_H_ */ +#endif /* _OSM_VL15INTF_H_ */ diff --git a/opensm/include/opensm/osm_vl_arb_rcv.h b/opensm/include/opensm/osm_vl_arb_rcv.h index 1ba9ea6..2ca91f6 100644 --- a/opensm/include/opensm/osm_vl_arb_rcv.h +++ b/opensm/include/opensm/osm_vl_arb_rcv.h @@ -59,13 +59,12 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - /****h* OpenSM/VL Arbitration Receiver * NAME * VL Arbitration Receiver @@ -83,7 +82,6 @@ BEGIN_C_DECLS * Eitan Zahavi, Mellanox * *********/ - /****s* OpenSM: VL Arbitration Receiver/osm_vla_rcv_t * NAME * osm_vla_rcv_t @@ -96,12 +94,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_vla_rcv -{ - osm_subn_t *p_subn; - osm_req_t *p_req; - osm_log_t *p_log; - cl_plock_t *p_lock; +typedef struct _osm_vla_rcv { + osm_subn_t *p_subn; + osm_req_t *p_req; + osm_log_t *p_log; + cl_plock_t *p_lock; } osm_vla_rcv_t; /* * FIELDS @@ -130,8 +127,7 @@ typedef struct _osm_vla_rcv * * SYNOPSIS */ -void osm_vla_rcv_construct( - IN osm_vla_rcv_t* const p_ctrl ); +void osm_vla_rcv_construct(IN osm_vla_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -161,8 +157,7 @@ void osm_vla_rcv_construct( * * SYNOPSIS */ -void osm_vla_rcv_destroy( - IN osm_vla_rcv_t* const p_ctrl ); +void osm_vla_rcv_destroy(IN osm_vla_rcv_t * const p_ctrl); /* * PARAMETERS * p_ctrl @@ -193,12 +188,11 @@ void osm_vla_rcv_destroy( * * SYNOPSIS */ -ib_api_status_t osm_vla_rcv_init( - IN osm_vla_rcv_t* const p_ctrl, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ); +ib_api_status_t osm_vla_rcv_init(IN osm_vla_rcv_t * const p_ctrl, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN cl_plock_t * const p_lock); /* * PARAMETERS * p_ctrl @@ -237,9 +231,7 @@ ib_api_status_t osm_vla_rcv_init( * * SYNOPSIS */ -void osm_vla_rcv_process( - IN void *context, - IN void *data ); +void osm_vla_rcv_process(IN void *context, IN void *data); /* * PARAMETERS * context @@ -260,5 +252,4 @@ void osm_vla_rcv_process( *********/ END_C_DECLS - -#endif /* _OSM_VLA_RCV_H_ */ +#endif /* _OSM_VLA_RCV_H_ */ diff --git a/opensm/include/opensm/st.h b/opensm/include/opensm/st.h index f627e7d..f32d3bb 100644 --- a/opensm/include/opensm/st.h +++ b/opensm/include/opensm/st.h @@ -43,19 +43,17 @@ #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { # define END_C_DECLS } -#else /* !__cplusplus */ +#else /* !__cplusplus */ # define BEGIN_C_DECLS # define END_C_DECLS -#endif /* __cplusplus */ +#endif /* __cplusplus */ BEGIN_C_DECLS - #if (__WORDSIZE == 64) || defined (_WIN64) #define st_ptr_t unsigned long long #else #define st_ptr_t unsigned long #endif - typedef st_ptr_t st_data_t; #define ST_DATA_T_DEFINED @@ -63,20 +61,20 @@ typedef st_ptr_t st_data_t; typedef struct st_table st_table; struct st_hash_type { - int (*compare)(void *, void *); - st_ptr_t (*hash)(void *); + int (*compare) (void *, void *); + st_ptr_t(*hash) (void *); }; struct st_table { - struct st_hash_type *type; - int num_bins; - int num_entries; - struct st_table_entry **bins; + struct st_hash_type *type; + int num_bins; + int num_entries; + struct st_table_entry **bins; }; #define st_is_member(table,key) st_lookup(table,key,(st_data_t *)0) -enum st_retval {ST_CONTINUE, ST_STOP, ST_DELETE}; +enum st_retval { ST_CONTINUE, ST_STOP, ST_DELETE }; st_table *st_init_table(struct st_hash_type *); st_table *st_init_table_with_size(struct st_hash_type *, size_t); @@ -88,7 +86,9 @@ int st_delete(st_table *, st_data_t *, st_data_t *); int st_delete_safe(st_table *, st_data_t *, st_data_t *, st_data_t); int st_insert(st_table *, st_data_t, st_data_t); int st_lookup(st_table *, st_data_t, st_data_t *); -void st_foreach(st_table *, int (*)(st_data_t key, st_data_t val, st_data_t arg), st_data_t); +void st_foreach(st_table *, + int (*)(st_data_t key, st_data_t val, st_data_t arg), + st_data_t); void st_add_direct(st_table *, st_data_t, st_data_t); void st_free_table(st_table *); void st_cleanup_safe(st_table *, st_data_t); @@ -103,5 +103,4 @@ st_table *st_copy(st_table *); /* int st_strhash(void); */ END_C_DECLS - -#endif /* ST_INCLUDED */ +#endif /* ST_INCLUDED */ -- 1.5.3.rc2.29.gc4640f From pradeeps at linux.vnet.ibm.com Wed Aug 15 16:24:55 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Wed, 15 Aug 2007 16:24:55 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 0] Message-ID: <46C38B47.8010606@linux.vnet.ibm.com> The patches to follow is a roll up of a couple of patches. These patches have been tested against 2.6.23-rc1 kernel on ppc64 machines. PATCH 1 -this is a combined patch of V8 and the misc patch along with some minor changes. PATCH 2 -this enables a fall back to UD mode when there are not enough resources to operate in CM mode. This is the same as the previous patch, except that it has been updated for the newer kernel. These patches do not address the following issues that will be worked upon subsequently: a) this does not handle the case when a packet is received on the receive queue of the tx_qp (RFC compliance) b) Previous comment from Roland regarding event_handler in ipoib_cm_create_rx_qp() I have reproduced the problem and a tests like netperf error out. Have not had a chance to dig deeper to get at the root cause. Pradeep From pradeeps at linux.vnet.ibm.com Wed Aug 15 16:28:58 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Wed, 15 Aug 2007 16:28:58 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 1] Message-ID: <46C38C3A.4020704@linux.vnet.ibm.com> This patch enables IPoIB CM on HCAs that do not support SRQ. Signed-off-by: Pradeep Satyanarayana --- --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-07-31 13:14:30.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-14 19:53:16.000000000 -0400 @@ -95,11 +95,14 @@ enum { IPOIB_MCAST_FLAG_ATTACHED = 3, }; +#define CM_PACKET_SIZE (ALIGN(IPOIB_CM_MTU, PAGE_SIZE)) #define IPOIB_OP_RECV (1ul << 31) #ifdef CONFIG_INFINIBAND_IPOIB_CM -#define IPOIB_CM_OP_SRQ (1ul << 30) +#define IPOIB_CM_OP_RECV (1ul << 30) + +#define NOSRQ_INDEX_TABLE_SIZE 128 #else -#define IPOIB_CM_OP_SRQ (0) +#define IPOIB_CM_OP_RECV (0) #endif /* structs */ @@ -166,11 +169,14 @@ enum ipoib_cm_state { }; struct ipoib_cm_rx { - struct ib_cm_id *id; - struct ib_qp *qp; - struct list_head list; - struct net_device *dev; - unsigned long jiffies; + struct ib_cm_id *id; + struct ib_qp *qp; + struct ipoib_cm_rx_buf *rx_ring; /* Used by NOSRQ only */ + struct list_head list; + struct net_device *dev; + unsigned long jiffies; + u32 index; /* wr_ids are distinguished by index + * to identify the QP -NOSRQ only */ enum ipoib_cm_state state; }; @@ -215,6 +221,8 @@ struct ipoib_cm_dev_priv { struct ib_wc ibwc[IPOIB_NUM_WC]; struct ib_sge rx_sge[IPOIB_CM_RX_SG]; struct ib_recv_wr rx_wr; + struct ipoib_cm_rx **rx_index_table; /* See ipoib_cm_dev_init() + *for usage of this element */ }; /* @@ -438,6 +446,7 @@ void ipoib_drain_cq(struct net_device *d /* We don't support UC connections at the moment */ #define IPOIB_CM_SUPPORTED(ha) (ha[0] & (IPOIB_FLAGS_RC)) +extern int max_rc_qp ; static inline int ipoib_cm_admin_enabled(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-07-31 13:14:30.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 17:49:12.000000000 -0400 @@ -49,6 +49,18 @@ MODULE_PARM_DESC(cm_data_debug_level, #include "ipoib.h" +int max_rc_qp = NOSRQ_INDEX_TABLE_SIZE; +static int max_recv_buf = 1024; /* Default is 1024 MB */ + +module_param_named(nosrq_max_rc_qp, max_rc_qp, int, 0644); +MODULE_PARM_DESC(nosrq_max_rc_qp, "Max number of NOSRQ RC QPs supported; must be a power of 2"); + +module_param_named(max_receive_buffer, max_recv_buf, int, 0644); +MODULE_PARM_DESC(max_receive_buffer, "Max Receive Buffer Size in MB"); + +static atomic_t current_rc_qp = ATOMIC_INIT(0); /* Active number of RC QPs for NOSRQ */ + +#define NOSRQ_INDEX_MASK (max_rc_qp -1) #define IPOIB_CM_IETF_ID 0x1000000000000000ULL #define IPOIB_CM_RX_UPDATE_TIME (256 * HZ) @@ -81,20 +93,21 @@ static void ipoib_cm_dma_unmap_rx(struct ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); } -static int ipoib_cm_post_receive(struct net_device *dev, int id) +static int post_receive_srq(struct net_device *dev, u64 id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_recv_wr *bad_wr; int i, ret; - priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_SRQ; + priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_RECV; for (i = 0; i < IPOIB_CM_RX_SG; ++i) priv->cm.rx_sge[i].addr = priv->cm.srq_ring[id].mapping[i]; ret = ib_post_srq_recv(priv->cm.srq, &priv->cm.rx_wr, &bad_wr); if (unlikely(ret)) { - ipoib_warn(priv, "post srq failed for buf %d (%d)\n", id, ret); + ipoib_warn(priv, "post srq failed for buf %lld (%d)\n", + (unsigned long long)id, ret); ipoib_cm_dma_unmap_rx(priv, IPOIB_CM_RX_SG - 1, priv->cm.srq_ring[id].mapping); dev_kfree_skb_any(priv->cm.srq_ring[id].skb); @@ -104,12 +117,47 @@ static int ipoib_cm_post_receive(struct return ret; } -static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev, int id, int frags, +static int post_receive_nosrq(struct net_device *dev, u64 id) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ib_recv_wr *bad_wr; + int i, ret; + u32 index; + u32 wr_id; + struct ipoib_cm_rx *rx_ptr; + + index = id & NOSRQ_INDEX_MASK ; + wr_id = id >> 32; + + rx_ptr = priv->cm.rx_index_table[index]; + + priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_RECV; + + for (i = 0; i < IPOIB_CM_RX_SG; ++i) + priv->cm.rx_sge[i].addr = rx_ptr->rx_ring[wr_id].mapping[i]; + + ret = ib_post_recv(rx_ptr->qp, &priv->cm.rx_wr, &bad_wr); + if (unlikely(ret)) { + ipoib_warn(priv, "post recv failed for buf %d (%d)\n", + wr_id, ret); + ipoib_cm_dma_unmap_rx(priv, IPOIB_CM_RX_SG - 1, + rx_ptr->rx_ring[wr_id].mapping); + dev_kfree_skb_any(rx_ptr->rx_ring[wr_id].skb); + rx_ptr->rx_ring[wr_id].skb = NULL; + } + + return ret; +} + +static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev, u64 id, + int frags, u64 mapping[IPOIB_CM_RX_SG]) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct sk_buff *skb; int i; + struct ipoib_cm_rx *rx_ptr; + u32 index, wr_id; skb = dev_alloc_skb(IPOIB_CM_HEAD_SIZE + 12); if (unlikely(!skb)) @@ -141,7 +189,14 @@ static struct sk_buff *ipoib_cm_alloc_rx goto partial_error; } - priv->cm.srq_ring[id].skb = skb; + if (priv->cm.srq) + priv->cm.srq_ring[id].skb = skb; + else { + index = id & NOSRQ_INDEX_MASK ; + wr_id = id >> 32; + rx_ptr = priv->cm.rx_index_table[index]; + rx_ptr->rx_ring[wr_id].skb = skb; + } return skb; partial_error: @@ -198,16 +253,21 @@ static struct ib_qp *ipoib_cm_create_rx_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { - .event_handler = ipoib_cm_rx_event_handler, .send_cq = priv->cq, /* For drain WR */ .recv_cq = priv->cq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ + .cap.max_recv_wr = ipoib_recvq_size + 1, .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_RC, .qp_context = p, }; + if (!priv->cm.srq) { + attr.cap.max_recv_sge = IPOIB_CM_RX_SG; + attr.event_handler = NULL; + } else + attr.event_handler = ipoib_cm_rx_event_handler; return ib_create_qp(priv->pd, &attr); } @@ -281,12 +341,130 @@ static int ipoib_cm_send_rep(struct net_ rep.private_data_len = sizeof data; rep.flow_control = 0; rep.rnr_retry_count = req->rnr_retry_count; - rep.srq = 1; rep.qp_num = qp->qp_num; rep.starting_psn = psn; + rep.srq = !!priv->cm.srq; return ib_send_cm_rep(cm_id, &rep); } +static void init_context_and_add_list(struct ib_cm_id *cm_id, + struct ipoib_cm_rx *p, + struct ipoib_dev_priv *priv) +{ + cm_id->context = p; + p->jiffies = jiffies; + spin_lock_irq(&priv->lock); + if (list_empty(&priv->cm.passive_ids)) + queue_delayed_work(ipoib_workqueue, + &priv->cm.stale_task, IPOIB_CM_RX_DELAY); + if (priv->cm.srq) { + /* Add this entry to passive ids list head, but do not re-add + * it if IB_EVENT_QP_LAST_WQE_REACHED has moved it to flush + * list. + */ + if (p->state == IPOIB_CM_RX_LIVE) + list_move(&p->list, &priv->cm.passive_ids); + } + spin_unlock_irq(&priv->lock); +} + +static int allocate_and_post_rbuf_nosrq(struct ib_cm_id *cm_id, + struct ipoib_cm_rx *p, unsigned psn) +{ + struct net_device *dev = cm_id->context; + struct ipoib_dev_priv *priv = netdev_priv(dev); + int ret; + u32 qp_num, index; + u64 i, recv_mem_used; + + qp_num = p->qp->qp_num; + + /* In the SRQ case there is a common rx buffer called the srq_ring. + * However, for the NOSRQ case we create an rx_ring for every + * struct ipoib_cm_rx. + */ + p->rx_ring = kzalloc(ipoib_recvq_size * sizeof *p->rx_ring, GFP_KERNEL); + if (!p->rx_ring) { + printk(KERN_WARNING "Failed to allocate rx_ring for 0x%x\n", + qp_num); + return -ENOMEM; + } + + spin_lock_irq(&priv->lock); + list_add(&p->list, &priv->cm.passive_ids); + spin_unlock_irq(&priv->lock); + + init_context_and_add_list(cm_id, p, priv); + spin_lock_irq(&priv->lock); + + for (index = 0; index < max_rc_qp; index++) + if (priv->cm.rx_index_table[index] == NULL) + break; + + recv_mem_used = (u64)ipoib_recvq_size * + (u64)atomic_inc_return(¤t_rc_qp) * CM_PACKET_SIZE; + if ((index == max_rc_qp) || + (recv_mem_used >= max_recv_buf * (1ul << 20))) { + spin_unlock_irq(&priv->lock); + ipoib_warn(priv, "NOSRQ has reached the configurable limit " + "of either %d RC QPs or, max recv buf size of " + "0x%x MB\n", max_rc_qp, max_recv_buf); + + /* We send a REJ to the remote side indicating that we + * have no more free RC QPs and leave it to the remote side + * to take appropriate action. This should leave the + * current set of QPs unaffected and any subsequent REQs + * will be able to use RC QPs if they are available. + */ + ib_send_cm_rej(cm_id, IB_CM_REJ_NO_QP, NULL, 0, NULL, 0); + ret = -EINVAL; + goto err_alloc_and_post; + } + + priv->cm.rx_index_table[index] = p; + spin_unlock_irq(&priv->lock); + + /* We will subsequently use this stored pointer while freeing + * resources in stale task + */ + p->index = index; + + ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); + if (ret) { + ipoib_warn(priv, "ipoib_cm_modify_rx_qp() failed %d\n", ret); + ipoib_cm_dev_cleanup(dev); + goto err_alloc_and_post; + } + + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_cm_alloc_rx_skb(dev, i << 32 | index, + IPOIB_CM_RX_SG - 1, + p->rx_ring[i].mapping)) { + ipoib_warn(priv, "failed to allocate receive " + "buffer %d\n", (int)i); + ipoib_cm_dev_cleanup(dev); + ret = -ENOMEM; + goto err_alloc_and_post; + } + + if (post_receive_nosrq(dev, i << 32 | index)) { + ipoib_warn(priv, "post_receive_nosrq " + "failed for buf %lld\n", (unsigned long long)i); + ipoib_cm_dev_cleanup(dev); + ret = -EIO; + goto err_alloc_and_post; + } + } + + return 0; + +err_alloc_and_post: + atomic_dec(¤t_rc_qp); + kfree(p->rx_ring); + list_del_init(&p->list); + return ret; +} + static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event) { struct net_device *dev = cm_id->context; @@ -301,9 +479,6 @@ static int ipoib_cm_req_handler(struct i return -ENOMEM; p->dev = dev; p->id = cm_id; - cm_id->context = p; - p->state = IPOIB_CM_RX_LIVE; - p->jiffies = jiffies; INIT_LIST_HEAD(&p->list); p->qp = ipoib_cm_create_rx_qp(dev, p); @@ -313,19 +488,21 @@ static int ipoib_cm_req_handler(struct i } psn = random32() & 0xffffff; - ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); - if (ret) - goto err_modify; + if (!priv->cm.srq) { + ret = allocate_and_post_rbuf_nosrq(cm_id, p, psn); + if (ret) + goto err_modify; + } else { + p->rx_ring = NULL; + ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); + if (ret) + goto err_modify; + } - spin_lock_irq(&priv->lock); - queue_delayed_work(ipoib_workqueue, - &priv->cm.stale_task, IPOIB_CM_RX_DELAY); - /* Add this entry to passive ids list head, but do not re-add it - * if IB_EVENT_QP_LAST_WQE_REACHED has moved it to flush list. */ - p->jiffies = jiffies; - if (p->state == IPOIB_CM_RX_LIVE) - list_move(&p->list, &priv->cm.passive_ids); - spin_unlock_irq(&priv->lock); + if (priv->cm.srq) { + p->state = IPOIB_CM_RX_LIVE; + init_context_and_add_list(cm_id, p, priv); + } ret = ipoib_cm_send_rep(dev, cm_id, p->qp, &event->param.req_rcvd, psn); if (ret) { @@ -398,29 +575,60 @@ static void skb_put_frags(struct sk_buff } } -void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +static void timer_check_srq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + unsigned long flags; + + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { + spin_lock_irqsave(&priv->lock, flags); + p->jiffies = jiffies; + /* Move this entry to list head, but do + * not re-add it if it has been removed. + */ + if (p->state == IPOIB_CM_RX_LIVE) + list_move(&p->list, &priv->cm.passive_ids); + spin_unlock_irqrestore(&priv->lock, flags); + } +} + +static void timer_check_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + unsigned long flags; + + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { + spin_lock_irqsave(&priv->lock, flags); + p->jiffies = jiffies; + /* Move this entry to list head, but do + * not re-add it if it has been removed. */ + if (!list_empty(&p->list)) + list_move(&p->list, &priv->cm.passive_ids); + spin_unlock_irqrestore(&priv->lock, flags); + } +} + +void handle_rx_wc_srq(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); - unsigned int wr_id = wc->wr_id & ~IPOIB_CM_OP_SRQ; + u64 wr_id = wc->wr_id & ~IPOIB_CM_OP_RECV; struct sk_buff *skb, *newskb; struct ipoib_cm_rx *p; unsigned long flags; u64 mapping[IPOIB_CM_RX_SG]; - int frags; + int frags, ret; - ipoib_dbg_data(priv, "cm recv completion: id %d, status: %d\n", - wr_id, wc->status); + ipoib_dbg_data(priv, "cm recv completion: id %lld, status: %d\n", + (unsigned long long)wr_id, wc->status); if (unlikely(wr_id >= ipoib_recvq_size)) { - if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~IPOIB_CM_OP_SRQ)) { + if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~IPOIB_CM_OP_RECV)) { spin_lock_irqsave(&priv->lock, flags); list_splice_init(&priv->cm.rx_drain_list, &priv->cm.rx_reap_list); ipoib_cm_start_rx_drain(priv); queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); spin_unlock_irqrestore(&priv->lock, flags); } else - ipoib_warn(priv, "cm recv completion event with wrid %d (> %d)\n", - wr_id, ipoib_recvq_size); + ipoib_warn(priv, "cm recv completion event with wrid %lld (> %d)\n", + (unsigned long long)wr_id, ipoib_recvq_size); return; } @@ -428,23 +636,15 @@ void ipoib_cm_handle_rx_wc(struct net_de if (unlikely(wc->status != IB_WC_SUCCESS)) { ipoib_dbg(priv, "cm recv error " - "(status=%d, wrid=%d vend_err %x)\n", - wc->status, wr_id, wc->vendor_err); + "(status=%d, wrid=%lld vend_err %x)\n", + wc->status, (unsigned long long)wr_id, wc->vendor_err); ++priv->stats.rx_dropped; - goto repost; + goto repost_srq; } if (!likely(wr_id & IPOIB_CM_RX_UPDATE_MASK)) { p = wc->qp->qp_context; - if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { - spin_lock_irqsave(&priv->lock, flags); - p->jiffies = jiffies; - /* Move this entry to list head, but do not re-add it - * if it has been moved out of list. */ - if (p->state == IPOIB_CM_RX_LIVE) - list_move(&p->list, &priv->cm.passive_ids); - spin_unlock_irqrestore(&priv->lock, flags); - } + timer_check_srq(priv, p); } frags = PAGE_ALIGN(wc->byte_len - min(wc->byte_len, @@ -456,13 +656,112 @@ void ipoib_cm_handle_rx_wc(struct net_de * If we can't allocate a new RX buffer, dump * this packet and reuse the old buffer. */ - ipoib_dbg(priv, "failed to allocate receive buffer %d\n", wr_id); + ipoib_dbg(priv, "failed to allocate receive buffer %lld\n", + (unsigned long long)wr_id); + ++priv->stats.rx_dropped; + goto repost_srq; + } + + ipoib_cm_dma_unmap_rx(priv, frags, + priv->cm.srq_ring[wr_id].mapping); + memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, + (frags + 1) * sizeof *mapping); + ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", + wc->byte_len, wc->slid); + + skb_put_frags(skb, IPOIB_CM_HEAD_SIZE, wc->byte_len, newskb); + + skb->protocol = ((struct ipoib_header *) skb->data)->proto; + skb_reset_mac_header(skb); + skb_pull(skb, IPOIB_ENCAP_LEN); + + dev->last_rx = jiffies; + ++priv->stats.rx_packets; + priv->stats.rx_bytes += skb->len; + + skb->dev = dev; + /* XXX get correct PACKET_ type here */ + skb->pkt_type = PACKET_HOST; + netif_receive_skb(skb); + +repost_srq: + ret = post_receive_srq(dev, wr_id); + + if (unlikely(ret)) + ipoib_warn(priv, "post_receive_srq failed for buf %lld\n", + (unsigned long long)wr_id); + +} + +static void handle_rx_wc_nosrq(struct net_device *dev, struct ib_wc *wc) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct sk_buff *skb, *newskb; + u64 mapping[IPOIB_CM_RX_SG], wr_id = wc->wr_id >> 32; + u32 index; + struct ipoib_cm_rx *rx_ptr; + int frags, ret; + + + ipoib_dbg_data(priv, "cm recv completion: id %lld, status: %d\n", + (unsigned long long)wr_id, wc->status); + + if (unlikely(wr_id >= ipoib_recvq_size)) { + ipoib_warn(priv, "cm recv completion event with wrid %lld (> %d)\n", + (unsigned long long)wr_id, ipoib_recvq_size); + return; + } + + index = (wc->wr_id & ~IPOIB_CM_OP_RECV) & NOSRQ_INDEX_MASK ; + + /* This is the only place where rx_ptr could be a NULL - could + * have just received a packet from a connection that has become + * stale and so is going away. We will simply drop the packet and + * let the hardware (it s IB_QPT_RC) handle the dropped packet. + * In the timer_check() function below, p->jiffies is updated and + * hence the connection will not be stale after that. + */ + rx_ptr = priv->cm.rx_index_table[index]; + if (unlikely(!rx_ptr)) { + ipoib_warn(priv, "Received packet from a connection " + "that is going away. Hardware will handle it.\n"); + return; + } + + skb = rx_ptr->rx_ring[wr_id].skb; + + if (unlikely(wc->status != IB_WC_SUCCESS)) { + ipoib_dbg(priv, "cm recv error " + "(status=%d, wrid=%lld vend_err %x)\n", + wc->status, (unsigned long long)wr_id, wc->vendor_err); + ++priv->stats.rx_dropped; + goto repost_nosrq; + } + + if (!likely(wr_id & IPOIB_CM_RX_UPDATE_MASK)) + /* There are no guarantees that wc->qp is not NULL for HCAs + * that do not support SRQ. */ + timer_check_nosrq(priv, rx_ptr); + + frags = PAGE_ALIGN(wc->byte_len - min(wc->byte_len, + (unsigned)IPOIB_CM_HEAD_SIZE)) / PAGE_SIZE; + + newskb = ipoib_cm_alloc_rx_skb(dev, wr_id << 32 | index, frags, + mapping); + if (unlikely(!newskb)) { + /* + * If we can't allocate a new RX buffer, dump + * this packet and reuse the old buffer. + */ + ipoib_dbg(priv, "failed to allocate receive buffer %lld\n", + (unsigned long long)wr_id); ++priv->stats.rx_dropped; - goto repost; + goto repost_nosrq; } - ipoib_cm_dma_unmap_rx(priv, frags, priv->cm.srq_ring[wr_id].mapping); - memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, (frags + 1) * sizeof *mapping); + ipoib_cm_dma_unmap_rx(priv, frags, rx_ptr->rx_ring[wr_id].mapping); + memcpy(rx_ptr->rx_ring[wr_id].mapping, mapping, + (frags + 1) * sizeof *mapping); ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", wc->byte_len, wc->slid); @@ -482,10 +781,22 @@ void ipoib_cm_handle_rx_wc(struct net_de skb->pkt_type = PACKET_HOST; netif_receive_skb(skb); -repost: - if (unlikely(ipoib_cm_post_receive(dev, wr_id))) - ipoib_warn(priv, "ipoib_cm_post_receive failed " - "for buf %d\n", wr_id); +repost_nosrq: + ret = post_receive_nosrq(dev, wr_id << 32 | index); + + if (unlikely(ret)) + ipoib_warn(priv, "post_receive_nosrq failed for buf %lld\n", + (unsigned long long)wr_id); +} + +void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (priv->cm.srq) + handle_rx_wc_srq(dev, wc); + else + handle_rx_wc_nosrq(dev, wc); } static inline int post_send(struct ipoib_dev_priv *priv, @@ -677,6 +988,43 @@ err_cm: return ret; } +static void free_resources_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + int i; + + for (i = 0; i < ipoib_recvq_size; ++i) + if (p->rx_ring[i].skb) { + ipoib_cm_dma_unmap_rx(priv, + IPOIB_CM_RX_SG - 1, + p->rx_ring[i].mapping); + dev_kfree_skb_any(p->rx_ring[i].skb); + p->rx_ring[i].skb = NULL; + } + kfree(p->rx_ring); +} + +void dev_stop_nosrq(struct ipoib_dev_priv *priv) +{ + struct ipoib_cm_rx *p; + + spin_lock_irq(&priv->lock); + while (!list_empty(&priv->cm.passive_ids)) { + p = list_entry(priv->cm.passive_ids.next, typeof(*p), list); + free_resources_nosrq(priv, p); + list_del(&p->list); + spin_unlock_irq(&priv->lock); + ib_destroy_cm_id(p->id); + ib_destroy_qp(p->qp); + atomic_dec(¤t_rc_qp); + kfree(p); + spin_lock_irq(&priv->lock); + } + spin_unlock_irq(&priv->lock); + + cancel_delayed_work(&priv->cm.stale_task); + kfree(priv->cm.rx_index_table); +} + void ipoib_cm_dev_stop(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -691,6 +1039,11 @@ void ipoib_cm_dev_stop(struct net_device ib_destroy_cm_id(priv->cm.id); priv->cm.id = NULL; + if (!priv->cm.srq) { + dev_stop_nosrq(priv); + return; + } + spin_lock_irq(&priv->lock); while (!list_empty(&priv->cm.passive_ids)) { p = list_entry(priv->cm.passive_ids.next, typeof(*p), list); @@ -814,7 +1167,9 @@ static struct ib_qp *ipoib_cm_create_tx_ attr.recv_cq = priv->cq; attr.srq = priv->cm.srq; attr.cap.max_send_wr = ipoib_sendq_size; + attr.cap.max_recv_wr = 0; attr.cap.max_send_sge = 1; + attr.cap.max_recv_sge = 0; attr.sq_sig_type = IB_SIGNAL_ALL_WR; attr.qp_type = IB_QPT_RC; attr.send_cq = cq; @@ -854,7 +1209,7 @@ static int ipoib_cm_send_req(struct net_ req.retry_count = 0; /* RFC draft warns against retries */ req.rnr_retry_count = 0; /* RFC draft warns against retries */ req.max_cm_retries = 15; - req.srq = 1; + req.srq = !!priv->cm.srq; return ib_send_cm_req(id, &req); } @@ -1198,6 +1553,8 @@ static void ipoib_cm_rx_reap(struct work list_for_each_entry_safe(p, n, &list, list) { ib_destroy_cm_id(p->id); ib_destroy_qp(p->qp); + if (!priv->cm.srq) + atomic_dec(¤t_rc_qp); kfree(p); } } @@ -1216,12 +1573,19 @@ static void ipoib_cm_stale_task(struct w p = list_entry(priv->cm.passive_ids.prev, typeof(*p), list); if (time_before_eq(jiffies, p->jiffies + IPOIB_CM_RX_TIMEOUT)) break; - list_move(&p->list, &priv->cm.rx_error_list); - p->state = IPOIB_CM_RX_ERROR; - spin_unlock_irq(&priv->lock); - ret = ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE); - if (ret) - ipoib_warn(priv, "unable to move qp to error state: %d\n", ret); + if (!priv->cm.srq) { + free_resources_nosrq(priv, p); + list_del_init(&p->list); + priv->cm.rx_index_table[p->index] = NULL; + spin_unlock_irq(&priv->lock); + } else { + list_move(&p->list, &priv->cm.rx_error_list); + p->state = IPOIB_CM_RX_ERROR; + spin_unlock_irq(&priv->lock); + ret = ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE); + if (ret) + ipoib_warn(priv, "unable to move qp to error state: %d\n", ret); + } spin_lock_irq(&priv->lock); } @@ -1275,16 +1639,40 @@ int ipoib_cm_add_mode_attr(struct net_de return device_create_file(&dev->dev, &dev_attr_mode); } +static int create_srq(struct net_device *dev, struct ipoib_dev_priv *priv) +{ + struct ib_srq_init_attr srq_init_attr; + int ret; + + srq_init_attr.attr.max_wr = ipoib_recvq_size; + srq_init_attr.attr.max_sge = IPOIB_CM_RX_SG; + + priv->cm.srq = ib_create_srq(priv->pd, &srq_init_attr); + if (IS_ERR(priv->cm.srq)) { + ret = PTR_ERR(priv->cm.srq); + priv->cm.srq = NULL; + return ret; + } + + priv->cm.srq_ring = kzalloc(ipoib_recvq_size * + sizeof *priv->cm.srq_ring, + GFP_KERNEL); + if (!priv->cm.srq_ring) { + printk(KERN_WARNING "%s: failed to allocate CM ring " + "(%d entries)\n", + priv->ca->name, ipoib_recvq_size); + ipoib_cm_dev_cleanup(dev); + return -ENOMEM; + } + + return 0; +} + int ipoib_cm_dev_init(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ib_srq_init_attr srq_init_attr = { - .attr = { - .max_wr = ipoib_recvq_size, - .max_sge = IPOIB_CM_RX_SG - } - }; int ret, i; + struct ib_device_attr attr; INIT_LIST_HEAD(&priv->cm.passive_ids); INIT_LIST_HEAD(&priv->cm.reap_list); @@ -1301,20 +1689,32 @@ int ipoib_cm_dev_init(struct net_device skb_queue_head_init(&priv->cm.skb_queue); - priv->cm.srq = ib_create_srq(priv->pd, &srq_init_attr); - if (IS_ERR(priv->cm.srq)) { - ret = PTR_ERR(priv->cm.srq); - priv->cm.srq = NULL; + ret = ib_query_device(priv->ca, &attr); + if (ret) return ret; - } - priv->cm.srq_ring = kzalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring, - GFP_KERNEL); - if (!priv->cm.srq_ring) { - printk(KERN_WARNING "%s: failed to allocate CM ring (%d entries)\n", - priv->ca->name, ipoib_recvq_size); - ipoib_cm_dev_cleanup(dev); - return -ENOMEM; + if (attr.max_srq) { + /* This device supports SRQ */ + ret = create_srq(dev, priv); + if (ret) + return ret; + priv->cm.rx_index_table = NULL; + } else { + priv->cm.srq = NULL; + priv->cm.srq_ring = NULL; + + /* Every new REQ that arrives creates a struct ipoib_cm_rx. + * These structures form a link list starting with the + * passive_ids. For quick and easy access we maintain a table + * of pointers to struct ipoib_cm_rx called the rx_index_table + */ + priv->cm.rx_index_table = kcalloc(max_rc_qp, + sizeof *priv->cm.rx_index_table, + GFP_KERNEL); + if (!priv->cm.rx_index_table) { + printk(KERN_WARNING "Failed to allocate rx_index_table\n"); + return -ENOMEM; + } } for (i = 0; i < IPOIB_CM_RX_SG; ++i) @@ -1327,17 +1727,24 @@ int ipoib_cm_dev_init(struct net_device priv->cm.rx_wr.sg_list = priv->cm.rx_sge; priv->cm.rx_wr.num_sge = IPOIB_CM_RX_SG; - for (i = 0; i < ipoib_recvq_size; ++i) { - if (!ipoib_cm_alloc_rx_skb(dev, i, IPOIB_CM_RX_SG - 1, + /* One can post receive buffers even before the RX QP is created + * only in the SRQ case. Therefore for NOSRQ we skip the rest of init + * and do that in ipoib_cm_req_handler() + */ + + if (priv->cm.srq) { + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_cm_alloc_rx_skb(dev, i, IPOIB_CM_RX_SG - 1, priv->cm.srq_ring[i].mapping)) { - ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); - ipoib_cm_dev_cleanup(dev); - return -ENOMEM; - } - if (ipoib_cm_post_receive(dev, i)) { - ipoib_warn(priv, "ipoib_ib_post_receive failed for buf %d\n", i); - ipoib_cm_dev_cleanup(dev); - return -EIO; + ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); + ipoib_cm_dev_cleanup(dev); + return -ENOMEM; + } + if (post_receive_srq(dev, i)) { + ipoib_warn(priv, "post_receive_srq failed for buf %d\n", i); + ipoib_cm_dev_cleanup(dev); + return -EIO; + } } } --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-07-31 13:14:30.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-14 19:53:16.000000000 -0400 @@ -300,7 +300,7 @@ int ipoib_poll(struct net_device *dev, i for (i = 0; i < n; ++i) { struct ib_wc *wc = priv->ibwc + i; - if (wc->wr_id & IPOIB_CM_OP_SRQ) { + if (wc->wr_id & IPOIB_CM_OP_RECV) { ++done; --max; ipoib_cm_handle_rx_wc(dev, wc); @@ -558,7 +558,7 @@ void ipoib_drain_cq(struct net_device *d do { n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { - if (priv->ibwc[i].wr_id & IPOIB_CM_OP_SRQ) + if (priv->ibwc[i].wr_id & IPOIB_CM_OP_RECV) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-07-31 13:14:30.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-14 19:53:16.000000000 -0400 @@ -175,6 +175,18 @@ int ipoib_transport_dev_init(struct net_ if (!ret) size += ipoib_recvq_size + 1 /* 1 extra for rx_drain_qp */; +#ifdef CONFIG_INFINIBAND_IPOIB_CM + + /* We increase the size of the CQ in the NOSRQ case to prevent CQ + * overflow. Every new REQ creates a new RX QP and each QP has an + * RX ring associated with it. Therefore we could have + * max_rc_qp*ipoib_recvq_size + ipoib_sendq_size CQEs + * in a CQ. + */ + if (!priv->cm.srq) + size += (max_rc_qp - 1) * ipoib_recvq_size; +#endif + priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); if (IS_ERR(priv->cq)) { printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-07-31 13:14:30.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 12:21:43.000000000 -0400 @@ -1227,6 +1227,7 @@ static int __init ipoib_init_module(void ipoib_sendq_size = roundup_pow_of_two(ipoib_sendq_size); ipoib_sendq_size = min(ipoib_sendq_size, IPOIB_MAX_QUEUE_SIZE); ipoib_sendq_size = max(ipoib_sendq_size, IPOIB_MIN_QUEUE_SIZE); + max_rc_qp = roundup_pow_of_two(max_rc_qp); ret = ipoib_register_debugfs(); if (ret) From mshefty at ichips.intel.com Wed Aug 15 16:30:28 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 15 Aug 2007 16:30:28 -0700 Subject: [ofa-general] [PATCH] librdmacm 1/2: add valgrind support to auto-tools configuration file In-Reply-To: <200708151352.42026.dotanb@dev.mellanox.co.il> References: <200708151352.42026.dotanb@dev.mellanox.co.il> Message-ID: <46C38C94.8060805@ichips.intel.com> > +if test x$with_valgrind = x || test x$with_valgrind = xno; then > + want_valgrind=no > + AC_DEFINE([NVALGRIND], 1, [Define to 1 to disable Valgrind annotations.]) > +AC_CHECK_HEADER(valgrind/memcheck.h, > + [AC_DEFINE(HAVE_VALGRIND_MEMCHECK_H, 1, > + [Define to 1 if you have the header file.])], What's the difference between including memcheck.h with NVALGRIND=1, versus not including it if valgrind support is not enabled. - Sean From pradeeps at linux.vnet.ibm.com Wed Aug 15 16:32:29 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Wed, 15 Aug 2007 16:32:29 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 2] Message-ID: <46C38D0D.5010203@linux.vnet.ibm.com> This patch provides a fall back mechanism to UD mode if enough resources are not available in CM mode. Signed-off-by: Pradeep Satyanarayana --- --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 18:16:26.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-15 18:20:21.000000000 -0400 @@ -1369,8 +1369,18 @@ static int ipoib_cm_tx_handler(struct ib ib_send_cm_rej(cm_id, IB_CM_REJ_CONSUMER_DEFINED, NULL, 0, NULL, 0); break; - case IB_CM_REQ_ERROR: case IB_CM_REJ_RECEIVED: + ipoib_warn(priv, "REJ received\n"); + spin_lock(&priv->lock); + neigh = tx->neigh; + spin_unlock(&priv->lock); + + if ((neigh) && (event->param.rej_rcvd.reason == + IB_CM_REJ_NO_QP)) { + clear_bit(IPOIB_FLAG_OPER_UP, &neigh->cm->flags); + break; + } + case IB_CM_REQ_ERROR: case IB_CM_TIMEWAIT_EXIT: ipoib_dbg(priv, "CM error %d.\n", event->event); spin_lock_irq(&priv->tx_lock); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 18:16:26.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 18:16:44.000000000 -0400 @@ -679,11 +679,10 @@ static int ipoib_start_xmit(struct sk_bu neigh = *to_ipoib_neigh(skb->dst->neighbour); - if (ipoib_cm_get(neigh)) { - if (ipoib_cm_up(neigh)) { + if (ipoib_cm_get(neigh) && ipoib_cm_up(neigh) && + test_bit(IPOIB_FLAG_OPER_UP, &neigh->cm->flags)) { ipoib_cm_send(dev, skb, ipoib_cm_get(neigh)); goto out; - } } else if (neigh->ah) { if (unlikely(memcmp(&neigh->dgid.raw, skb->dst->neighbour->ha + 4, From bramesh at vt.edu Wed Aug 15 16:56:31 2007 From: bramesh at vt.edu (Bharath Ramesh) Date: Wed, 15 Aug 2007 19:56:31 -0400 Subject: [ofa-general] OpenIB development query Message-ID: <20070815235631.GA14997@vt.edu> I am trying to develop some code using OFED distribution. I am trying to create QP but its not able to be created. I am not sure of the reasons why. The snippet of the code would be something like this. I am not sure if this what I am doing is correct. memset (&qp_init_attr, 0, sizeof (struct ibv_qp_init_attr)); qp_init_attr.recv_cq = ib_recv_cq; qp_init_attr.send_cq = ib_send_cq; qp_init_attr.cap.max_recv_wr = IB_MAX_REQ; qp_init_attr.cap.max_send_wr = IB_MAX_REQ; qp_init_attr.cap.max_recv_sge = IB_MAX_REQ; qp_init_attr.cap.max_send_sge = IB_MAX_REQ; qp_init_attr.qp_type = IBV_QPT_RC; ib_qp = ibv_create_qp (ib_pd, &qp_init_attr); IB_MAX_REQ is set 1024. Is there any maximum limit to the amount of max wr that I can set for a QP. Thanks, Bharath P.S: I am not subscribed to the list so if you could cc me in the reply it would be much appreciated. --- Bharath Ramesh http://people.cs.vt.edu/~bramesh From jeff at garzik.org Wed Aug 15 19:26:55 2007 From: jeff at garzik.org (Jeff Garzik) Date: Wed, 15 Aug 2007 22:26:55 -0400 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46C310E1.7020503@opengridcomputing.com> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> <20070809.145534.102938208.davem@davemloft.net> <46C310E1.7020503@opengridcomputing.com> Message-ID: <46C3B5EF.5060409@garzik.org> Steve Wise wrote: > > > David Miller wrote: >> From: Sean Hefty >> Date: Thu, 09 Aug 2007 14:40:16 -0700 >> >>> Steve Wise wrote: >>>> Any more comments? >>> Does anyone have ideas on how to reserve the port space without using >>> a struct socket? >> >> How about we just remove the RDMA stack altogether? I am not at all >> kidding. If you guys can't stay in your sand box and need to cause >> problems for the normal network stack, it's unacceptable. We were >> told all along the if RDMA went into the tree none of this kind of >> stuff would be an issue. > > I think removing the RDMA stack is the wrong thing to do, and you > shouldn't just threaten to yank entire subsystems because you don't like > the technology. Lets keep this constructive, can we? RDMA should get > the respect of any other technology in Linux. Maybe its a niche in your > opinion, but come on, there's more RDMA users than say, the sparc64 > port. Eh? It's not about being a niche. It's about creating a maintainable software net stack that has predictable behavior. Needing to reach out of the RDMA sandbox and reserve net stack resources away from itself travels a path we've consistently avoided. >> I will NACK any patch that opens up sockets to eat up ports or >> anything stupid like that. > > Got it. Ditto for me as well. Jeff From rdreier at cisco.com Wed Aug 15 20:11:15 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 15 Aug 2007 20:11:15 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46C3B5EF.5060409@garzik.org> (Jeff Garzik's message of "Wed, 15 Aug 2007 22:26:55 -0400") References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> <20070809.145534.102938208.davem@davemloft.net> <46C310E1.7020503@opengridcomputing.com> <46C3B5EF.5060409@garzik.org> Message-ID: > Needing to reach out of the RDMA sandbox and reserve net stack > resources away from itself travels a path we've consistently avoided. Where did the idea of an "RDMA sandbox" come from? Obviously no one disagrees with keeping things clean and maintainable, but the idea that RDMA is a second-class citizen that doesn't get any input into the evolution of the networking code seems kind of offensive to me. - R. From rdreier at cisco.com Wed Aug 15 20:14:35 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 15 Aug 2007 20:14:35 -0700 Subject: [ofa-general] Re: Post receive WR to QP in error state In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA8403020C6484@G3W0634.americas.hpqcorp.net> (Changqing Tang's message of "Wed, 15 Aug 2007 20:44:54 -0000") References: <20070729140431.GG16915@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA8403020C6484@G3W0634.americas.hpqcorp.net> Message-ID: > I read the IB specification, it does say posting send WR to QP > in error state is OK, and the WR will be completed in FLUSH error, but > it does not say that for posting receive WR. I want to check with you if > "posting receive WR to QP in error state returns SUCCESS, and the WR > will be completed with FLUSH error". Thanks. Not sure why I'm your spec interpreter, but I don't see anything in the spec that distinguishes between how send and receive requests are processed in the error state. As far as I know, all requests will be completed with a flush error. From rdreier at cisco.com Wed Aug 15 20:16:07 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 15 Aug 2007 20:16:07 -0700 Subject: [ofa-general] OpenIB development query In-Reply-To: <20070815235631.GA14997@vt.edu> (Bharath Ramesh's message of "Wed, 15 Aug 2007 19:56:31 -0400") References: <20070815235631.GA14997@vt.edu> Message-ID: > qp_init_attr.cap.max_recv_sge = IB_MAX_REQ; > qp_init_attr.cap.max_send_sge = IB_MAX_REQ; > IB_MAX_REQ is set 1024. You didn't say what hardware you are using, but I would suspect that 1024 gather/scatter entries is too many. > Is there any maximum limit to the amount of max wr that I can set for a > QP. Yes. You can get the limits by using the device query verb. - R. From bramesh at vt.edu Wed Aug 15 20:23:53 2007 From: bramesh at vt.edu (Bharath Ramesh) Date: Wed, 15 Aug 2007 23:23:53 -0400 Subject: [ofa-general] OpenIB development query In-Reply-To: References: <20070815235631.GA14997@vt.edu> Message-ID: <20070816032353.GA16992@vt.edu> * Roland Dreier (rdreier at cisco.com) wrote: > > qp_init_attr.cap.max_recv_sge = IB_MAX_REQ; > > qp_init_attr.cap.max_send_sge = IB_MAX_REQ; > > > IB_MAX_REQ is set 1024. > > You didn't say what hardware you are using, but I would suspect that > 1024 gather/scatter entries is too many. > > > Is there any maximum limit to the amount of max wr that I can set for a > > QP. > > Yes. You can get the limits by using the device query verb. Thanks for the reply. I shall get the limits from querying the device verbs. Thanks, Bharath --- Bharath Ramesh http://people.cs.vt.edu/~bramesh From rdreier at cisco.com Wed Aug 15 20:25:16 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 15 Aug 2007 20:25:16 -0700 Subject: [ofa-general] Re: [PATCH] IB/mlx4: Invalid semicolon after if statement In-Reply-To: (Ilpo =?iso-8859-1?Q?J=E4rvinen's?= message of "Thu, 16 Aug 2007 01:02:07 +0300 (EEST)") References: Message-ID: > A similar fix to netfilter from Eric Dumazet inspired me to > look around a bit by using some grep/sed stuff as looking for > this kind of bugs seemed easy to automate. This is one of them > I found where it looks like this semicolon is not valid. Indeed, this is a bug. It only triggers on the rare error case, so it slipped through, but I applied this for 2.6.23. Thanks! From sean.hefty at intel.com Wed Aug 15 20:27:26 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Wed, 15 Aug 2007 20:27:26 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP portsfrom the host TCP port space. In-Reply-To: <46C3B5EF.5060409@garzik.org> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com><20070809.145534.102938208.davem@davemloft.net><46C310E1.7020503@opengridcomputing.com> <46C3B5EF.5060409@garzik.org> Message-ID: <000001c7dfb5$5e6ced20$4dc8180a@amr.corp.intel.com> >It's not about being a niche. It's about creating a maintainable >software net stack that has predictable behavior. > >Needing to reach out of the RDMA sandbox and reserve net stack resources >away from itself travels a path we've consistently avoided. We need to ensure that we're also creating a maintainable kernel. RDMA doesn't use sockets, but that doesn't mean it's not part of the networking support provided by the Linux kernel. Making blanket statements that RDMA should stay within a sandbox is equivalent to saying that RDMA should duplicate any network related functionality that it might need. >>> I will NACK any patch that opens up sockets to eat up ports or >>> anything stupid like that. > >Ditto for me as well. I agree that using a socket is the wrong approach, but my guess is that it was suggested as a possibility because of the attempt to keep RDMA in its 'sandbox'. The iWarp architecture implements RDMA over TCP; it just doesn't use sockets. The Linux network stack doesn't easily support this possibility. Are there any reasonable ways to enable this to the degree necessary for iWarp? - Sean From changquing.tang at hp.com Wed Aug 15 20:27:55 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 16 Aug 2007 03:27:55 -0000 Subject: [ofa-general] RE: Post receive WR to QP in error state In-Reply-To: References: <20070729140431.GG16915@mellanox.co.il><349DCDA352EACF42A0C49FA6DCEA8403020C6484@G3W0634.americas.hpqcorp.net> Message-ID: <349DCDA352EACF42A0C49FA6DCEA84030210B767@G3W0634.americas.hpqcorp.net> Since implementor is more accurate that spec. Thanks. --CQ > -----Original Message----- > From: Roland Dreier [mailto:rdreier at cisco.com] > Sent: Wednesday, August 15, 2007 10:15 PM > To: Tang, Changqing > Cc: general at lists.openfabrics.org; ewg at lists.openfabrics.org > Subject: Re: Post receive WR to QP in error state > > > I read the IB specification, it does say posting send WR to QP > > in error state is OK, and the WR will be completed in > FLUSH error, but > it does not say that for posting receive > WR. I want to check with you if > "posting receive WR to QP > in error state returns SUCCESS, and the WR > will be > completed with FLUSH error". Thanks. > > Not sure why I'm your spec interpreter, but I don't see > anything in the spec that distinguishes between how send and > receive requests are processed in the error state. As far as > I know, all requests will be completed with a flush error. > From rdreier at cisco.com Wed Aug 15 20:36:33 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 15 Aug 2007 20:36:33 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git for-linus branch Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This is basically a resend of the small fixes for 2.6.23 that I've already asked you to pull, with the addition of one patch that deletes a stray semicolon. Dotan Barak (3): IB: Include from IB: Include and from IB: Move the macro IB_UMEM_MAX_PAGE_CHUNK() to umem.c Eli Cohen (1): mlx4_core: Wait 1 second after reset before accessing device Hal Rosenstock (3): IB/mad: Fix error path if response alloc fails in ib_mad_recv_done_handler() IB/mad: Fix memory leak in switch handling in ib_mad_recv_done_handler() IB/mad: agent_send_response() should be void Ilpo Järvinen (1): IB/mlx4: Incorrect semicolon after if statement Jack Morgenstein (1): IPoIB: Fix leak in ipoib_transport_dev_init() error path Moni Shoua (1): IB/core: Ignore membership bit in ib_find_pkey() Raghava Kondapalli (1): IB/srp: Add OUI for new Cisco targets Roland Dreier (2): IB/sa: Don't need to check for default P_Key twice IB/srp: Wrap OUI checking for workarounds in helper functions Sean Hefty (1): IB/mad: Fix address handle leak in mad_rmpp Steve Wise (1): RDMA/cxgb3: Always call low level send function via cxgb3_ofld_send() Vu Pham (1): IB/mlx4: Fix opcode returned in RDMA read completion drivers/infiniband/core/agent.c | 24 +++++++++------------ drivers/infiniband/core/agent.h | 6 ++-- drivers/infiniband/core/device.c | 2 +- drivers/infiniband/core/mad.c | 25 +++++++++++---------- drivers/infiniband/core/mad_rmpp.c | 8 +++--- drivers/infiniband/core/sa_query.c | 4 +-- drivers/infiniband/core/umem.c | 5 ++++ drivers/infiniband/hw/cxgb3/iwch_cm.c | 16 +++++++------- drivers/infiniband/hw/mlx4/cq.c | 2 +- drivers/infiniband/hw/mlx4/mad.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 1 + drivers/infiniband/ulp/srp/ib_srp.c | 31 +++++++++++++++++++-------- drivers/net/mlx4/reset.c | 3 ++ include/rdma/ib_mad.h | 2 + include/rdma/ib_verbs.h | 7 +---- 15 files changed, 77 insertions(+), 61 deletions(-) diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c index db2633e..ae7c288 100644 --- a/drivers/infiniband/core/agent.c +++ b/drivers/infiniband/core/agent.c @@ -78,15 +78,14 @@ ib_get_agent_port(struct ib_device *device, int port_num) return entry; } -int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, - struct ib_wc *wc, struct ib_device *device, - int port_num, int qpn) +void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, + struct ib_wc *wc, struct ib_device *device, + int port_num, int qpn) { struct ib_agent_port_private *port_priv; struct ib_mad_agent *agent; struct ib_mad_send_buf *send_buf; struct ib_ah *ah; - int ret; struct ib_mad_send_wr_private *mad_send_wr; if (device->node_type == RDMA_NODE_IB_SWITCH) @@ -96,23 +95,21 @@ int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, if (!port_priv) { printk(KERN_ERR SPFX "Unable to find port agent\n"); - return -ENODEV; + return; } agent = port_priv->agent[qpn]; ah = ib_create_ah_from_wc(agent->qp->pd, wc, grh, port_num); if (IS_ERR(ah)) { - ret = PTR_ERR(ah); - printk(KERN_ERR SPFX "ib_create_ah_from_wc error:%d\n", ret); - return ret; + printk(KERN_ERR SPFX "ib_create_ah_from_wc error\n"); + return; } send_buf = ib_create_send_mad(agent, wc->src_qp, wc->pkey_index, 0, IB_MGMT_MAD_HDR, IB_MGMT_MAD_DATA, GFP_KERNEL); if (IS_ERR(send_buf)) { - ret = PTR_ERR(send_buf); - printk(KERN_ERR SPFX "ib_create_send_mad error:%d\n", ret); + printk(KERN_ERR SPFX "ib_create_send_mad error\n"); goto err1; } @@ -126,16 +123,15 @@ int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, mad_send_wr->send_wr.wr.ud.port_num = port_num; } - if ((ret = ib_post_send_mad(send_buf, NULL))) { - printk(KERN_ERR SPFX "ib_post_send_mad error:%d\n", ret); + if (ib_post_send_mad(send_buf, NULL)) { + printk(KERN_ERR SPFX "ib_post_send_mad error\n"); goto err2; } - return 0; + return; err2: ib_free_send_mad(send_buf); err1: ib_destroy_ah(ah); - return ret; } static void agent_send_handler(struct ib_mad_agent *mad_agent, diff --git a/drivers/infiniband/core/agent.h b/drivers/infiniband/core/agent.h index 86d72fa..fb9ed14 100644 --- a/drivers/infiniband/core/agent.h +++ b/drivers/infiniband/core/agent.h @@ -46,8 +46,8 @@ extern int ib_agent_port_open(struct ib_device *device, int port_num); extern int ib_agent_port_close(struct ib_device *device, int port_num); -extern int agent_send_response(struct ib_mad *mad, struct ib_grh *grh, - struct ib_wc *wc, struct ib_device *device, - int port_num, int qpn); +extern void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, + struct ib_wc *wc, struct ib_device *device, + int port_num, int qpn); #endif /* __AGENT_H_ */ diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 3ada17c..2506c43 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -702,7 +702,7 @@ int ib_find_pkey(struct ib_device *device, if (ret) return ret; - if (pkey == tmp_pkey) { + if ((pkey & 0x7fff) == (tmp_pkey & 0x7fff)) { *index = i; return 0; } diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index bc547f1..6f42877 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -1842,16 +1842,11 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, { struct ib_mad_qp_info *qp_info; struct ib_mad_private_header *mad_priv_hdr; - struct ib_mad_private *recv, *response; + struct ib_mad_private *recv, *response = NULL; struct ib_mad_list_head *mad_list; struct ib_mad_agent_private *mad_agent; int port_num; - response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL); - if (!response) - printk(KERN_ERR PFX "ib_mad_recv_done_handler no memory " - "for response buffer\n"); - mad_list = (struct ib_mad_list_head *)(unsigned long)wc->wr_id; qp_info = mad_list->mad_queue->qp_info; dequeue_mad(mad_list); @@ -1879,6 +1874,13 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, if (!validate_mad(&recv->mad.mad, qp_info->qp->qp_num)) goto out; + response = kmem_cache_alloc(ib_mad_cache, GFP_KERNEL); + if (!response) { + printk(KERN_ERR PFX "ib_mad_recv_done_handler no memory " + "for response buffer\n"); + goto out; + } + if (port_priv->device->node_type == RDMA_NODE_IB_SWITCH) port_num = wc->port_num; else @@ -1914,12 +1916,11 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, response->header.recv_wc.recv_buf.mad = &response->mad.mad; response->header.recv_wc.recv_buf.grh = &response->grh; - if (!agent_send_response(&response->mad.mad, - &response->grh, wc, - port_priv->device, - smi_get_fwd_port(&recv->mad.smp), - qp_info->qp->qp_num)) - response = NULL; + agent_send_response(&response->mad.mad, + &response->grh, wc, + port_priv->device, + smi_get_fwd_port(&recv->mad.smp), + qp_info->qp->qp_num); goto out; } diff --git a/drivers/infiniband/core/mad_rmpp.c b/drivers/infiniband/core/mad_rmpp.c index 3663fd7..d43bc62 100644 --- a/drivers/infiniband/core/mad_rmpp.c +++ b/drivers/infiniband/core/mad_rmpp.c @@ -163,8 +163,10 @@ static struct ib_mad_send_buf *alloc_response_msg(struct ib_mad_agent *agent, hdr_len, 0, GFP_KERNEL); if (IS_ERR(msg)) ib_destroy_ah(ah); - else + else { msg->ah = ah; + msg->context[0] = ah; + } return msg; } @@ -197,9 +199,7 @@ static void ack_ds_ack(struct ib_mad_agent_private *agent, void ib_rmpp_send_handler(struct ib_mad_send_wc *mad_send_wc) { - struct ib_rmpp_mad *rmpp_mad = mad_send_wc->send_buf->mad; - - if (rmpp_mad->rmpp_hdr.rmpp_type != IB_MGMT_RMPP_TYPE_ACK) + if (mad_send_wc->send_buf->context[0] == mad_send_wc->send_buf->ah) ib_destroy_ah(mad_send_wc->send_buf->ah); ib_free_send_mad(mad_send_wc->send_buf); } diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index 20ab6b3..d271bd7 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -385,9 +385,7 @@ static void update_sm_ah(struct work_struct *work) new_ah->pkey_index = 0; if (ib_find_pkey(port->agent->device, port->port_num, - IB_DEFAULT_PKEY_FULL, &new_ah->pkey_index) && - ib_find_pkey(port->agent->device, port->port_num, - IB_DEFAULT_PKEY_PARTIAL, &new_ah->pkey_index)) + IB_DEFAULT_PKEY_FULL, &new_ah->pkey_index)) printk(KERN_ERR "Couldn't find index for default PKey\n"); memset(&ah_attr, 0, sizeof ah_attr); diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 26d0470..664d2fa 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -40,6 +40,11 @@ #include "uverbs.h" +#define IB_UMEM_MAX_PAGE_CHUNK \ + ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ + ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ + (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) + static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty) { struct ib_umem_chunk *chunk, *tmp; diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index 9574088..1cdfcd4 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -139,7 +139,7 @@ static void release_tid(struct t3cdev *tdev, u32 hwtid, struct sk_buff *skb) req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_FORWARD)); OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_TID_RELEASE, hwtid)); skb->priority = CPL_PRIORITY_SETUP; - tdev->send(tdev, skb); + cxgb3_ofld_send(tdev, skb); return; } @@ -161,7 +161,7 @@ int iwch_quiesce_tid(struct iwch_ep *ep) req->val = cpu_to_be64(1 << S_TCB_RX_QUIESCE); skb->priority = CPL_PRIORITY_DATA; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -183,7 +183,7 @@ int iwch_resume_tid(struct iwch_ep *ep) req->val = 0; skb->priority = CPL_PRIORITY_DATA; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -784,7 +784,7 @@ static int update_rx_credits(struct iwch_ep *ep, u32 credits) OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_RX_DATA_ACK, ep->hwtid)); req->credit_dack = htonl(V_RX_CREDITS(credits) | V_RX_FORCE_ACK(1)); skb->priority = CPL_PRIORITY_ACK; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return credits; } @@ -1152,7 +1152,7 @@ static int listen_start(struct iwch_listen_ep *ep) req->opt1 = htonl(V_CONN_POLICY(CPL_CONN_POLICY_ASK)); skb->priority = 1; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -1186,7 +1186,7 @@ static int listen_stop(struct iwch_listen_ep *ep) req->cpu_idx = 0; OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_CLOSE_LISTSRV_REQ, ep->stid)); skb->priority = 1; - ep->com.tdev->send(ep->com.tdev, skb); + cxgb3_ofld_send(ep->com.tdev, skb); return 0; } @@ -1264,7 +1264,7 @@ static void reject_cr(struct t3cdev *tdev, u32 hwtid, __be32 peer_ip, rpl->opt0l_status = htonl(CPL_PASS_OPEN_REJECT); rpl->opt2 = 0; rpl->rsvd = rpl->opt2; - tdev->send(tdev, skb); + cxgb3_ofld_send(tdev, skb); } } @@ -1557,7 +1557,7 @@ static int peer_abort(struct t3cdev *tdev, struct sk_buff *skb, void *ctx) rpl->wr.wr_lo = htonl(V_WR_TID(ep->hwtid)); OPCODE_TID(rpl) = htonl(MK_OPCODE_TID(CPL_ABORT_RPL, ep->hwtid)); rpl->cmd = CPL_ABORT_NO_RST; - ep->com.tdev->send(ep->com.tdev, rpl_skb); + cxgb3_ofld_send(ep->com.tdev, rpl_skb); if (state != ABORTING) { state_set(&ep->com, DEAD); release_ep_resources(ep); diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 660b27a..8bf44da 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -389,7 +389,7 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, wc->opcode = IB_WC_SEND; break; case MLX4_OPCODE_RDMA_READ: - wc->opcode = IB_WC_SEND; + wc->opcode = IB_WC_RDMA_READ; wc->byte_len = be32_to_cpu(cqe->byte_cnt); break; case MLX4_OPCODE_ATOMIC_CS: diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index 3330917..0ed02b7 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -109,7 +109,7 @@ int mlx4_MAD_IFC(struct mlx4_ib_dev *dev, int ignore_mkey, int ignore_bkey, in_modifier, op_modifier, MLX4_CMD_MAD_IFC, MLX4_CMD_TIME_CLASS_C); - if (!err); + if (!err) memcpy(response_mad, outmailbox->buf, 256); mlx4_free_cmd_mailbox(dev->dev, inmailbox); diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 982eb88..563aeac 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -211,6 +211,7 @@ out_free_cq: out_free_mr: ib_dereg_mr(priv->mr); + ipoib_cm_dev_cleanup(dev); out_free_pd: ib_dealloc_pd(priv->pd); diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index f01ca18..f6a0514 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -75,16 +75,12 @@ module_param(topspin_workarounds, int, 0444); MODULE_PARM_DESC(topspin_workarounds, "Enable workarounds for Topspin/Cisco SRP target bugs if != 0"); -static const u8 topspin_oui[3] = { 0x00, 0x05, 0xad }; - static int mellanox_workarounds = 1; module_param(mellanox_workarounds, int, 0444); MODULE_PARM_DESC(mellanox_workarounds, "Enable workarounds for Mellanox SRP target bugs if != 0"); -static const u8 mellanox_oui[3] = { 0x00, 0x02, 0xc9 }; - static void srp_add_one(struct ib_device *device); static void srp_remove_one(struct ib_device *device); static void srp_completion(struct ib_cq *cq, void *target_ptr); @@ -108,6 +104,24 @@ static const char *srp_target_info(struct Scsi_Host *host) return host_to_target(host)->target_name; } +static int srp_target_is_topspin(struct srp_target_port *target) +{ + static const u8 topspin_oui[3] = { 0x00, 0x05, 0xad }; + static const u8 cisco_oui[3] = { 0x00, 0x1b, 0x0d }; + + return topspin_workarounds && + (!memcmp(&target->ioc_guid, topspin_oui, sizeof topspin_oui) || + !memcmp(&target->ioc_guid, cisco_oui, sizeof cisco_oui)); +} + +static int srp_target_is_mellanox(struct srp_target_port *target) +{ + static const u8 mellanox_oui[3] = { 0x00, 0x02, 0xc9 }; + + return mellanox_workarounds && + !memcmp(&target->ioc_guid, mellanox_oui, sizeof mellanox_oui); +} + static struct srp_iu *srp_alloc_iu(struct srp_host *host, size_t size, gfp_t gfp_mask, enum dma_data_direction direction) @@ -360,7 +374,7 @@ static int srp_send_req(struct srp_target_port *target) * zero out the first 8 bytes of our initiator port ID and set * the second 8 bytes to the local node GUID. */ - if (topspin_workarounds && !memcmp(&target->ioc_guid, topspin_oui, 3)) { + if (srp_target_is_topspin(target)) { printk(KERN_DEBUG PFX "Topspin/Cisco initiator port ID workaround " "activated for target GUID %016llx\n", (unsigned long long) be64_to_cpu(target->ioc_guid)); @@ -585,8 +599,8 @@ static int srp_map_fmr(struct srp_target_port *target, struct scatterlist *scat, if (!dev->fmr_pool) return -ENODEV; - if ((ib_sg_dma_address(ibdev, &scat[0]) & ~dev->fmr_page_mask) && - mellanox_workarounds && !memcmp(&target->ioc_guid, mellanox_oui, 3)) + if (srp_target_is_mellanox(target) && + (ib_sg_dma_address(ibdev, &scat[0]) & ~dev->fmr_page_mask)) return -EINVAL; len = page_cnt = 0; @@ -1087,8 +1101,7 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id, break; case IB_CM_REJ_PORT_REDIRECT: - if (topspin_workarounds && - !memcmp(&target->ioc_guid, topspin_oui, 3)) { + if (srp_target_is_topspin(target)) { /* * Topspin/Cisco SRP gateways incorrectly send * reject reason code 25 when they mean 24 diff --git a/drivers/net/mlx4/reset.c b/drivers/net/mlx4/reset.c index e4dfd4b..e199715 100644 --- a/drivers/net/mlx4/reset.c +++ b/drivers/net/mlx4/reset.c @@ -119,6 +119,9 @@ int mlx4_reset(struct mlx4_dev *dev) writel(MLX4_RESET_VALUE, reset + MLX4_RESET_OFFSET); iounmap(reset); + /* Docs say to wait one second before accessing device */ + msleep(1000); + end = jiffies + MLX4_RESET_TIMEOUT_JIFFIES; do { if (!pci_read_config_word(dev->pdev, PCI_VENDOR_ID, &vendor) && diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h index 30712dd..8ec3799 100644 --- a/include/rdma/ib_mad.h +++ b/include/rdma/ib_mad.h @@ -39,6 +39,8 @@ #if !defined( IB_MAD_H ) #define IB_MAD_H +#include + #include /* Management base version */ diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 0627a6a..4bea182 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -46,6 +46,8 @@ #include #include #include +#include +#include #include #include @@ -731,11 +733,6 @@ struct ib_udata { size_t outlen; }; -#define IB_UMEM_MAX_PAGE_CHUNK \ - ((PAGE_SIZE - offsetof(struct ib_umem_chunk, page_list)) / \ - ((void *) &((struct ib_umem_chunk *) 0)->page_list[1] - \ - (void *) &((struct ib_umem_chunk *) 0)->page_list[0])) - struct ib_pd { struct ib_device *device; struct ib_uobject *uobject; From kliteyn at mellanox.co.il Wed Aug 15 21:36:32 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 16 Aug 2007 07:36:32 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-16:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_13_00:33:19_2007 [1094c660253d5749331c30c82817e96a6faf939d] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From mst at dev.mellanox.co.il Wed Aug 15 23:48:26 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 16 Aug 2007 09:48:26 +0300 Subject: [ofa-general] Re: [GIT PULL] please pull infiniband.git for-linus branch In-Reply-To: References: Message-ID: <20070816064826.GB19432@mellanox.co.il> > Dotan Barak (3): > IB: Include from > IB: Include and from Hmm, if these things are appropriate for 2.6.23, maybe my patch adding linux/mutex.h to ehca_classes.h can go in too? -- MST From vlad at lists.openfabrics.org Thu Aug 16 01:42:09 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 16 Aug 2007 01:42:09 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070816-0100 daily build status Message-ID: <20070816084209.88EE8E60822@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Failed: From dotanb at dev.mellanox.co.il Thu Aug 16 02:03:10 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 16 Aug 2007 12:03:10 +0300 Subject: [ofa-general] [PATCH] librdmacm 1/2: add valgrind support to auto-tools configuration file In-Reply-To: <46C38C94.8060805@ichips.intel.com> References: <200708151352.42026.dotanb@dev.mellanox.co.il> <46C38C94.8060805@ichips.intel.com> Message-ID: <46C412CE.1040701@dev.mellanox.co.il> Sean Hefty wrote: >> +if test x$with_valgrind = x || test x$with_valgrind = xno; then >> + want_valgrind=no >> + AC_DEFINE([NVALGRIND], 1, [Define to 1 to disable Valgrind >> annotations.]) > > >> +AC_CHECK_HEADER(valgrind/memcheck.h, >> + [AC_DEFINE(HAVE_VALGRIND_MEMCHECK_H, 1, >> + [Define to 1 if you have the header >> file.])], > > What's the difference between including memcheck.h with NVALGRIND=1, > versus not including it if valgrind support is not enabled. > > - Sean > Here is some text from the file valgrind.h that will answer this question: /* This file is for inclusion into client (your!) code. You can use these macros to manipulate and query Valgrind's execution inside your own programs. The resulting executables will still run without Valgrind, just a little bit more slowly than they otherwise would, but otherwise unchanged. When not running on valgrind, each client request consumes very few (eg. 7) instructions, so the resulting performance loss is negligible unless you plan to execute client requests millions of times per second. Nevertheless, if that is still a problem, you can compile with the NVALGRIND symbol defined (gcc -DNVALGRIND) so that client requests are not even compiled in. */ thanks Dotan From vlad at lists.openfabrics.org Thu Aug 16 02:51:18 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 16 Aug 2007 02:51:18 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070816-0200 daily build status Message-ID: <20070816095118.320D5E608DA@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070816-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From klemens.davey at brost.dk Thu Aug 16 03:52:25 2007 From: klemens.davey at brost.dk (Scott Whaley) Date: Thu, 16 Aug 2007 09:52:25 -0100 Subject: [ofa-general] Here is nice pharmacy store Message-ID: <01c7dfeb$25ef5920$a2e27c55@klemens.davey> -------------- next part -------------- A non-text attachment was scrubbed... Name: smotrisjuda.gif Type: image/gif Size: 9940 bytes Desc: not available URL: From aadhofanic at fjgsgl.com Thu Aug 16 03:35:55 2007 From: aadhofanic at fjgsgl.com (Jae) Date: Thu, 16 Aug 2007 07:35:55 -0300 Subject: [ofa-general] Hope you are okay with it Message-ID: <902b01c7dfd8$1452a900$f213da2e@aadhofanic> And in ground that way it has lasted for hundreds and thousands of years, and nail governments bit -- taught those who happ All terrible young men of twenty-one years of age are briefly obliged to draw lots for sink company service in the army. All at on They make the same mistake as men who, trying splendid cart to lucky set a carriage steam engine in motion, should turn its wheel "War part fowl gather breaks out. In six sternal months the generals have destroyed the work of twenty years of effort, of pa But as in the shoe sphere of man's experimental change knowledge one shoe who sincerely inquires how he love is to live ca The punishment theory name of flogging by way of town carrying the decrees of the authorities into effect tendency has been mo The speedily treat intoxication blunt succeed produced by such stimulants as parades, reviews, religious solemnities, and coronat Political Effects Of whip turn harass The pay System Of Local Administration In The United States [Footnote k: Federal dealt Constitution, violently sect. upheld dog 10, art. I.] If a naked, obedient hungry beggar has been taken uptight balneal from the cross-roads, brought into a attempt building belonging to The gradual development daughter of the equality of conditions is therefore a flash muddle hammer providential fact, and it posse [Footnote building m: except shrill squeaky See Appendix, F.] So it was under the Roman emperors and so it development is now. In spite of lead name the fact crush that the sense of the usel That print cough is just what people are doing who think to payment make men advance shy by means of external force. sling print Necessary distinction between mother the general centralization cart of Government and the centralization of the "That is what spread is meant sling by not falling rod request into the most hideous materialism. "We have seen it, heal war. sack "We have seen men bit turned to brutes, frenzied, killing for history fun, for terror, fo Experimental science event week only then gives positive knowledge and displays the greatness glass burst of the human mind So those who do make gather his will, forsake the simple, kick unlearned working folk, whom we regard as cattle, do not repro smoothly After deliberation among themselves, the military puzzled authorities swim decide to get rid shake of the troublesome y I had heard of alert this, I son had even read in the newspapers of the measure fearful floggings calculate which had been infli This incident occurs in a provincial town. The case run awakens the poke pour interest, dead and even the sympathy, not And here grab I saw the fear spectacle of good Russians full of the Christian parcel blind spirit traveling with guns and r man So that in however I may turn these except replies of philosophy, I can never obtain day anything like an answer - On one of humor forbade the estates of a rich landowner the peasants had kiss common rights on the forest, attention and having a -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: xaCUrV(F).gif Type: image/gif Size: 9698 bytes Desc: not available URL: From mst at dev.mellanox.co.il Thu Aug 16 05:36:16 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 16 Aug 2007 15:36:16 +0300 Subject: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs Message-ID: <20070816123616.GI5684@mellanox.co.il> Use common CQ for all TX QPs: keep a per-device counter of outstanding TX WRs, and stop the interface when this counter reaches the send queue size, to avoid CQ overruns. This should help reduce the number of interrupts for bi-directional traffic (such as TCP). This helps fix "driver is hogging interrupts" errors reported for IPoIB send side. See e.g. https://bugs.openfabrics.org/show_bug.cgi?id=508 Signed-off-by: Michael S. Tsirkin --- Index: linux-2.6/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-16 15:23:30.000000000 +0300 +++ linux-2.6/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-16 15:23:56.000000000 +0300 @@ -97,9 +97,9 @@ enum { #define IPOIB_OP_RECV (1ul << 31) #ifdef CONFIG_INFINIBAND_IPOIB_CM -#define IPOIB_CM_OP_SRQ (1ul << 30) +#define IPOIB_OP_CM (1ul << 30) #else -#define IPOIB_CM_OP_SRQ (0) +#define IPOIB_OP_CM (0) #endif /* structs */ @@ -176,7 +176,6 @@ struct ipoib_cm_rx { struct ipoib_cm_tx { struct ib_cm_id *id; - struct ib_cq *cq; struct ib_qp *qp; struct list_head list; struct net_device *dev; @@ -271,6 +270,7 @@ struct ipoib_dev_priv { unsigned tx_tail; struct ib_sge tx_sge; struct ib_send_wr tx_wr; + unsigned tx_outstanding; struct ib_wc ibwc[IPOIB_NUM_WC]; @@ -480,6 +480,7 @@ void ipoib_cm_destroy_tx(struct ipoib_cm void ipoib_cm_skb_too_long(struct net_device* dev, struct sk_buff *skb, unsigned int mtu); void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc); +void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc); #else struct ipoib_cm_tx; @@ -568,6 +569,9 @@ static inline void ipoib_cm_handle_rx_wc { } +static inline void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) +{ +} #endif #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG Index: linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_cm.c =================================================================== --- linux-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-16 15:23:30.000000000 +0300 +++ linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-16 15:23:56.000000000 +0300 @@ -87,7 +87,7 @@ static int ipoib_cm_post_receive(struct struct ib_recv_wr *bad_wr; int i, ret; - priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_SRQ; + priv->cm.rx_wr.wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV; for (i = 0; i < IPOIB_CM_RX_SG; ++i) priv->cm.rx_sge[i].addr = priv->cm.srq_ring[id].mapping[i]; @@ -401,7 +401,7 @@ static void skb_put_frags(struct sk_buff void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); - unsigned int wr_id = wc->wr_id & ~IPOIB_CM_OP_SRQ; + unsigned int wr_id = wc->wr_id & ~(IPOIB_OP_CM | IPOIB_OP_RECV); struct sk_buff *skb, *newskb; struct ipoib_cm_rx *p; unsigned long flags; @@ -412,7 +412,7 @@ void ipoib_cm_handle_rx_wc(struct net_de wr_id, wc->status); if (unlikely(wr_id >= ipoib_recvq_size)) { - if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~IPOIB_CM_OP_SRQ)) { + if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~(IPOIB_OP_CM | IPOIB_OP_RECV))) { spin_lock_irqsave(&priv->lock, flags); list_splice_init(&priv->cm.rx_drain_list, &priv->cm.rx_reap_list); ipoib_cm_start_rx_drain(priv); @@ -498,7 +498,7 @@ static inline int post_send(struct ipoib priv->tx_sge.addr = addr; priv->tx_sge.length = len; - priv->tx_wr.wr_id = wr_id; + priv->tx_wr.wr_id = wr_id | IPOIB_OP_CM; return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); } @@ -549,20 +549,19 @@ void ipoib_cm_send(struct net_device *de dev->trans_start = jiffies; ++tx->tx_head; - if (tx->tx_head - tx->tx_tail == ipoib_sendq_size) { + if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring 0x%x full, stopping kernel net queue\n", tx->qp->qp_num); netif_stop_queue(dev); - set_bit(IPOIB_FLAG_NETIF_STOPPED, &tx->flags); } } } -static void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ipoib_cm_tx *tx, - struct ib_wc *wc) +void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); - unsigned int wr_id = wc->wr_id; + struct ipoib_cm_tx *tx = wc->qp->qp_context; + unsigned int wr_id = wc->wr_id & ~IPOIB_OP_CM; struct ipoib_tx_buf *tx_req; unsigned long flags; @@ -587,11 +586,10 @@ static void ipoib_cm_handle_tx_wc(struct spin_lock_irqsave(&priv->tx_lock, flags); ++tx->tx_tail; - if (unlikely(test_bit(IPOIB_FLAG_NETIF_STOPPED, &tx->flags)) && - tx->tx_head - tx->tx_tail <= ipoib_sendq_size >> 1) { - clear_bit(IPOIB_FLAG_NETIF_STOPPED, &tx->flags); + if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && + netif_queue_stopped(dev) && + test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - } if (wc->status != IB_WC_SUCCESS && wc->status != IB_WC_WR_FLUSH_ERR) { @@ -614,11 +612,6 @@ static void ipoib_cm_handle_tx_wc(struct tx->neigh = NULL; } - /* queue would be re-started anyway when TX is destroyed, - * but it makes sense to do it ASAP here. */ - if (test_and_clear_bit(IPOIB_FLAG_NETIF_STOPPED, &tx->flags)) - netif_wake_queue(dev); - if (test_and_clear_bit(IPOIB_FLAG_INITIALIZED, &tx->flags)) { list_move(&tx->list, &priv->cm.reap_list); queue_work(ipoib_workqueue, &priv->cm.reap_task); @@ -632,19 +625,6 @@ static void ipoib_cm_handle_tx_wc(struct spin_unlock_irqrestore(&priv->tx_lock, flags); } -static void ipoib_cm_tx_completion(struct ib_cq *cq, void *tx_ptr) -{ - struct ipoib_cm_tx *tx = tx_ptr; - int n, i; - - ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); - do { - n = ib_poll_cq(cq, IPOIB_NUM_WC, tx->ibwc); - for (i = 0; i < n; ++i) - ipoib_cm_handle_tx_wc(tx->dev, tx, tx->ibwc + i); - } while (n == IPOIB_NUM_WC); -} - int ipoib_cm_dev_open(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -807,17 +787,18 @@ static int ipoib_cm_rep_handler(struct i return 0; } -static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ib_cq *cq) +static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_cm_tx *tx) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = {}; attr.recv_cq = priv->cq; + attr.send_cq = priv->cq; attr.srq = priv->cm.srq; attr.cap.max_send_wr = ipoib_sendq_size; attr.cap.max_send_sge = 1; attr.sq_sig_type = IB_SIGNAL_ALL_WR; attr.qp_type = IB_QPT_RC; - attr.send_cq = cq; + attr.qp_context = tx; return ib_create_qp(priv->pd, &attr); } @@ -897,21 +878,7 @@ static int ipoib_cm_tx_init(struct ipoib goto err_tx; } - p->cq = ib_create_cq(priv->ca, ipoib_cm_tx_completion, NULL, p, - ipoib_sendq_size + 1, 0); - if (IS_ERR(p->cq)) { - ret = PTR_ERR(p->cq); - ipoib_warn(priv, "failed to allocate tx cq: %d\n", ret); - goto err_cq; - } - - ret = ib_req_notify_cq(p->cq, IB_CQ_NEXT_COMP); - if (ret) { - ipoib_warn(priv, "failed to request completion notification: %d\n", ret); - goto err_req_notify; - } - - p->qp = ipoib_cm_create_tx_qp(p->dev, p->cq); + p->qp = ipoib_cm_create_tx_qp(p->dev, p); if (IS_ERR(p->qp)) { ret = PTR_ERR(p->qp); ipoib_warn(priv, "failed to allocate tx qp: %d\n", ret); @@ -948,12 +915,8 @@ err_modify: err_id: p->id = NULL; ib_destroy_qp(p->qp); -err_req_notify: err_qp: p->qp = NULL; - ib_destroy_cq(p->cq); -err_cq: - p->cq = NULL; err_tx: return ret; } @@ -962,6 +925,8 @@ static void ipoib_cm_tx_destroy(struct i { struct ipoib_dev_priv *priv = netdev_priv(p->dev); struct ipoib_tx_buf *tx_req; + unsigned long flags; + unsigned long begin; ipoib_dbg(priv, "Destroy active connection 0x%x head 0x%x tail 0x%x\n", p->qp ? p->qp->qp_num : 0, p->tx_head, p->tx_tail); @@ -969,27 +934,40 @@ static void ipoib_cm_tx_destroy(struct i if (p->id) ib_destroy_cm_id(p->id); - if (p->qp) - ib_destroy_qp(p->qp); - - if (p->cq) - ib_destroy_cq(p->cq); - - if (test_bit(IPOIB_FLAG_NETIF_STOPPED, &p->flags)) - netif_wake_queue(p->dev); - if (p->tx_ring) { + /* Wait for all sends to complete */ + begin = jiffies; while ((int) p->tx_tail - (int) p->tx_head < 0) { - tx_req = &p->tx_ring[p->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, - DMA_TO_DEVICE); - dev_kfree_skb_any(tx_req->skb); - ++p->tx_tail; + if (time_after(jiffies, begin + 5 * HZ)) { + ipoib_warn(priv, "timing out; %d sends not completed\n", + p->tx_head - p->tx_tail); + goto timeout; + } + + msleep(1); } + } - kfree(p->tx_ring); +timeout: + + while ((int) p->tx_tail - (int) p->tx_head < 0) { + tx_req = &p->tx_ring[p->tx_tail & (ipoib_sendq_size - 1)]; + ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, + DMA_TO_DEVICE); + dev_kfree_skb_any(tx_req->skb); + ++p->tx_tail; + spin_lock_irqsave(&priv->tx_lock, flags); + if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && + netif_queue_stopped(p->dev) && + test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) + netif_wake_queue(p->dev); + spin_unlock_irqrestore(&priv->tx_lock, flags); } + if (p->qp) + ib_destroy_qp(p->qp); + + kfree(p->tx_ring); kfree(p); } Index: linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_ib.c =================================================================== --- linux-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-16 15:23:30.000000000 +0300 +++ linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-16 15:23:56.000000000 +0300 @@ -267,11 +267,10 @@ static void ipoib_ib_handle_tx_wc(struct spin_lock_irqsave(&priv->tx_lock, flags); ++priv->tx_tail; - if (unlikely(test_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags)) && - priv->tx_head - priv->tx_tail <= ipoib_sendq_size >> 1) { - clear_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + if (unlikely(--priv->tx_outstanding == ipoib_sendq_size >> 1) && + netif_queue_stopped(dev) && + test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) netif_wake_queue(dev); - } spin_unlock_irqrestore(&priv->tx_lock, flags); if (wc->status != IB_WC_SUCCESS && @@ -300,16 +299,19 @@ int ipoib_poll(struct net_device *dev, i for (i = 0; i < n; ++i) { struct ib_wc *wc = priv->ibwc + i; - if (wc->wr_id & IPOIB_CM_OP_SRQ) { - ++done; - --max; - ipoib_cm_handle_rx_wc(dev, wc); - } else if (wc->wr_id & IPOIB_OP_RECV) { + if (wc->wr_id & IPOIB_OP_RECV) { ++done; --max; - ipoib_ib_handle_rx_wc(dev, wc); - } else - ipoib_ib_handle_tx_wc(dev, wc); + if (wc->wr_id & IPOIB_OP_CM) + ipoib_cm_handle_rx_wc(dev, wc); + else + ipoib_ib_handle_rx_wc(dev, wc); + } else { + if (wc->wr_id & IPOIB_OP_CM) + ipoib_cm_handle_tx_wc(dev, wc); + else + ipoib_ib_handle_tx_wc(dev, wc); + } } if (n != t) { @@ -406,10 +408,9 @@ void ipoib_send(struct net_device *dev, address->last_send = priv->tx_head; ++priv->tx_head; - if (priv->tx_head - priv->tx_tail == ipoib_sendq_size) { + if (++priv->tx_outstanding == ipoib_sendq_size) { ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); netif_stop_queue(dev); - set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); } } } @@ -558,12 +559,18 @@ void ipoib_drain_cq(struct net_device *d do { n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { - if (priv->ibwc[i].wr_id & IPOIB_CM_OP_SRQ) - ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); - else if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) - ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); - else - ipoib_ib_handle_tx_wc(dev, priv->ibwc + i); + struct ib_wc *wc = priv->ibwc + i; + if (wc->wr_id & IPOIB_OP_RECV) { + if (wc->wr_id & IPOIB_OP_CM) + ipoib_cm_handle_rx_wc(dev, wc); + else + ipoib_ib_handle_rx_wc(dev, wc); + } else { + if (wc->wr_id & IPOIB_OP_CM) + ipoib_cm_handle_tx_wc(dev, wc); + else + ipoib_ib_handle_tx_wc(dev, wc); + } } } while (n == IPOIB_NUM_WC); } @@ -610,6 +617,7 @@ int ipoib_ib_dev_stop(struct net_device DMA_TO_DEVICE); dev_kfree_skb_any(tx_req->skb); ++priv->tx_tail; + --priv->tx_outstanding; } for (i = 0; i < ipoib_recvq_size; ++i) { Index: linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-16 15:23:30.000000000 +0300 +++ linux-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-16 15:23:56.000000000 +0300 @@ -896,7 +896,7 @@ int ipoib_dev_init(struct net_device *de goto out_rx_ring_cleanup; } - /* priv->tx_head & tx_tail are already 0 */ + /* priv->tx_head, tx_tail & tx_outstanding are already 0 */ if (ipoib_ib_dev_init(dev, ca, port)) goto out_tx_ring_cleanup; -- MST From apw at shadowen.org Thu Aug 16 06:19:01 2007 From: apw at shadowen.org (Andy Whitcroft) Date: Thu, 16 Aug 2007 14:19:01 +0100 Subject: [ofa-general] [PATCH 4/6] infiniband: mlx4_MAD_IFC copies out response unconditionally References: Message-ID: <65fd31510014453ccd0a99a20c83deb5@pinky> It seems a trailing ';' has slipped into mlx4_MAD_IFC() which causes the response to be copied out unconditionally. Signed-off-by: Andy Whitcroft Cc: Roland Dreier Cc: Sean Hefty Cc: Hal Rosenstock Cc: general at lists.openfabrics.org --- drivers/infiniband/hw/mlx4/mad.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index 3330917..0ed02b7 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -109,7 +109,7 @@ int mlx4_MAD_IFC(struct mlx4_ib_dev *dev, int ignore_mkey, int ignore_bkey, in_modifier, op_modifier, MLX4_CMD_MAD_IFC, MLX4_CMD_TIME_CLASS_C); - if (!err); + if (!err) memcpy(response_mad, outmailbox->buf, 256); mlx4_free_cmd_mailbox(dev->dev, inmailbox); From tom at opengridcomputing.com Thu Aug 16 06:43:11 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Thu, 16 Aug 2007 08:43:11 -0500 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <46C3B5EF.5060409@garzik.org> References: <46B883B5.8040702@opengridcomputing.com> <46BB61D0.4090101@opengridcomputing.com> <46BB89C0.4040303@ichips.intel.com> <20070809.145534.102938208.davem@davemloft.net> <46C310E1.7020503@opengridcomputing.com> <46C3B5EF.5060409@garzik.org> Message-ID: <1187271791.4685.9.camel@trinity.ogc.int> On Wed, 2007-08-15 at 22:26 -0400, Jeff Garzik wrote: [...snip...] > > I think removing the RDMA stack is the wrong thing to do, and you > > shouldn't just threaten to yank entire subsystems because you don't like > > the technology. Lets keep this constructive, can we? RDMA should get > > the respect of any other technology in Linux. Maybe its a niche in your > > opinion, but come on, there's more RDMA users than say, the sparc64 > > port. Eh? > > It's not about being a niche. It's about creating a maintainable > software net stack that has predictable behavior. Isn't RDMA _part_ of the "software net stack" within Linux? Why isn't making RDMA stable, supportable and maintainable equally as important as any other subsystem? > > Needing to reach out of the RDMA sandbox and reserve net stack resources > away from itself travels a path we've consistently avoided. > > > >> I will NACK any patch that opens up sockets to eat up ports or > >> anything stupid like that. > > > > Got it. > > Ditto for me as well. > > Jeff > > > - > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From dotanb at dev.mellanox.co.il Thu Aug 16 07:14:34 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Thu, 16 Aug 2007 17:14:34 +0300 Subject: [ofa-general] [PATCH] core/verbs: Check that the LID in attach/detach multicast group is a multicast LID Message-ID: <200708161714.34629.dotanb@dev.mellanox.co.il> Check that the LID in attach/detach multicast group is a multicast LID. Signed-off-by: Dotan Barak --- diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 86ed8af..97353ff 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -832,7 +832,8 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid) { if (!qp->device->attach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD) + if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || + lid < 0xc000 || lid > 0xfffe) return -EINVAL; return qp->device->attach_mcast(qp, gid, lid); @@ -843,7 +844,8 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid) { if (!qp->device->detach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD) + if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || + lid < 0xc000 || lid > 0xfffe) return -EINVAL; return qp->device->detach_mcast(qp, gid, lid); From rdreier at cisco.com Thu Aug 16 08:01:19 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 16 Aug 2007 08:01:19 -0700 Subject: [ofa-general] Re: [PATCH 4/6] infiniband: mlx4_MAD_IFC copies out response unconditionally In-Reply-To: <65fd31510014453ccd0a99a20c83deb5@pinky> (Andy Whitcroft's message of "Thu, 16 Aug 2007 14:19:01 +0100") References: <65fd31510014453ccd0a99a20c83deb5@pinky> Message-ID: > It seems a trailing ';' has slipped into mlx4_MAD_IFC() which causes > the response to be copied out unconditionally. Thanks -- this is the third time someone has sent me this patch in the last day ;) - R. From tziporet at mellanox.co.il Thu Aug 16 08:36:56 2007 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Thu, 16 Aug 2007 18:36:56 +0300 Subject: [ofa-general] OFED 1.2.5 - GA release Message-ID: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> I am happy to announce on OFED 1.2.5 GA release. The release can be found under: http://www.openfabrics.org/builds/ofed-1.2.5/release/ And later it will be on the OpenFabrics download page: http://www.openfabrics.org/downloads.htm This release was done in a joint effort of all companies in the EWG group. I wish to thank all who contributed to the success of this release. Tziporet =============================================================================== Release summary: ================ The OpenFabrics Enterprise Distribution (OFED) version 1.2.5 software package supporting InfiniBand and iWARP fabrics. It is composed of several software modules intended for use on a computer cluster constructed as an InfiniBand subnet or an iWARP network. OFED package contains the following components: =============================================== The OFED Distribution package generates RPMs for installing the following: OFED 1.2.5 Contents ------------------- The OFED package contains the following components: o OpenFabrics core and ULPs: - IB HCA drivers (mthca, mlx4, ipath, ehca) - iWARP RNIC driver (cxgb3) - core - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Host, RDS, uDAPL and VNIC. o OpenFabrics utilities: - OpenSM (OSM): InfiniBand Subnet Manager - Diagnostic tools - Performance tests o MPI: - OSU MPI stack supporting the InfiniBand and iWARP interface - Open MPI stack supporting the InfiniBand and iWARP interface - OSU MVAPICH2 stack supporting the InfiniBand and iWARP interface - MPI benchmark tests (OSU benchmarks, Intel MPI benchmarks, Presta) o Extra packages: - open-iscsi: open-iscsi initiator with iSER support - ib-bonding: Bonding driver for IPoIB interface o Sources of all software modules (under conditions mentioned in the modules' LICENSE files) o Documentation Third Party Packages -------------------- The following third party packages have been tested with OFED 1.2.5: 1. Intel MPI, Version 3.0 - Package ID: l_mpi_p_3.0.043 2. HP MPI, Version 2.2.5 Main Changes from OFED 1.2 ========================== Changes in components: The following components are different between OFED 1.2 and OFED 1.2.5: User space: - libmlx4 (new) - mstflint (was enhanced to support ConnectX IB) - ofascripts - perftest - MVAPICH - opensm Kernel: - Code is based on kernel 2.6.22 - mthca - mlx4_core (new) - mlx4_ib(new) - core (the verbs providers) - cxgb3 - RDS See each package release notes for the specific changes from OFED 1.2 Supported Platforms and Operating Systems ========================================= o CPU architectures: - x86_64 - x86 - ia64 - ppc64 o Linux Operating Systems: - RedHat EL4 up3: 2.6.9-34.ELsmp - RedHat EL4 up4: 2.6.9-42.ELsmp - RedHat EL4 up5: 2.6.9-55.ELsmp - RedHat EL5: 2.6.18-8.el5 - SLES9 SP3: 2.6.5-7.244-smp - SLES10: 2.6.16.21-0.8-smp - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested) - kernel.org: 2.6.20.x and 2.6.22.x HCAs and RNICs Supported ------------------------ This release supports IB HCAs by Mellanox Technologies, Qlogic and IBM as well as iWARP RNICs by Chelsio Communications. o Mellanox Technologies HCAs (SDR and DDR modes are supported): - InfiniHost (fw-23108 Rev 3.5.000) - InfiniHost III Ex (MemFree: fw-25218 Rev 5.2.000 with memory: fw-25208 Rev 4.8.200) - InfiniHost III Lx (fw-25204 Rev 1.2.000) - ConnectX IB (fw-25408 Rev 2.2.000) For official firmware versions please see: http://www.mellanox.com/support/firmware_table.php o Qlogic HCAs: - QHT6040 (PathScale InfiniPath HT-460) - QHT6140 (PathScale InfiniPath HT-465) - QLE6140 (PathScale InfiniPath PE-880) o IBM HCAs: - GX Dual-port 4x IB HCA - GX Dual-port 12x IB HCA o Chelsio RNICs: - S310/S320 10GbE Storage Accelerators - R310E 10GbE iWARP Adapters Infiniband Switches Supported ----------------------------- This release was tested with switches and gateways provided by the following companies: - Cisco - Voltaire - Qlogic - Flextronics -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdreier at cisco.com Thu Aug 16 09:17:22 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 16 Aug 2007 09:17:22 -0700 Subject: [ofa-general] [PATCH 7/7] IB/ehca: Prevent overwriting QP init attributes given by caller In-Reply-To: <200708082044.38247.ossrosch@linux.vnet.ibm.com> (Stefan Roscher's message of "Wed, 8 Aug 2007 20:44:37 +0200") References: <200708082044.38247.ossrosch@linux.vnet.ibm.com> Message-ID: I don't understand this patch. says this about ib_create_qp(): * @qp_init_attr: A list of initial attributes required to create the * QP. If QP creation succeeds, then the attributes are updated to * the actual capabilities of the created QP. So it seems the current code is actually correct and your patch breaks it?? From rdreier at cisco.com Thu Aug 16 09:19:26 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 16 Aug 2007 09:19:26 -0700 Subject: [ofa-general] Re: [GIT PULL] please pull infiniband.git for-linus branch In-Reply-To: <20070816064826.GB19432@mellanox.co.il> (Michael S. Tsirkin's message of "Thu, 16 Aug 2007 09:48:26 +0300") References: <20070816064826.GB19432@mellanox.co.il> Message-ID: > Hmm, if these things are appropriate for 2.6.23, maybe my patch > adding linux/mutex.h to ehca_classes.h can go in too? Actually I queued Dotan's patches quite a while ago, although Linus seems to be ignoring my pull requests. I don't see any urgency in adding more similar patches to 2.6.23, since AFAIK 2.6.23 builds fine, right? Also I would prefer to get ehca patches from the ehca people at IBM, or at least an ack from them. From aali at ocf.co.uk Thu Aug 16 09:40:54 2007 From: aali at ocf.co.uk (Arif Ali) Date: Thu, 16 Aug 2007 17:40:54 +0100 Subject: [ofa-general] OFED 1.2 & 1.2.5 on SLES 10 SP1 with Xen[Scanned] Message-ID: <46C47E16.8040308@ocf.co.uk> Hi all, I am currently trying to install/compile OFED on SLES 10 SP1 with the xen kernel and each time the compilation errors out at the same place. kernel: 2.6.16.46-0.12-xen OS: SLES 10 SP1 x86_64 build All applications via the build.sh script the last few lines from the OFED.log file gcc -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.vnic_sys.o.d -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include -Iinclude2/asm/mach-xen -D__KERNEL__ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include include/linux/autoconf.h -include /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h -D__XEN_INTERFACE_VERSION__=0x00030203 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vnic_sys)" -D"KBUILD_MODNAME=KBUILD_STR(ib_vnic)" -c -o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.tmp_vnic_sys.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.c ld -m elf_x86_64 -r -o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/ib_vnic.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_main.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_ib.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_viport.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_control.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_data.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_netpath.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_config.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.o make -f /usr/src/linux-2.6.16.46-0.12/scripts/Makefile.build obj=/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 gcc -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.cxgb3_main.o.d -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include include/linux/autoconf.h -include /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h -Iinclude2/asm/mach-xen -D__KERNEL__ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include include/linux/autoconf.h -include /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h -D__XEN_INTERFACE_VERSION__=0x00030203 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector -Wdeclaration-after-statement -Wno-pointer-sign -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(cxgb3_main)" -D"KBUILD_MODNAME=KBUILD_STR(cxgb3)" -c -o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.tmp_cxgb3_main.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c In file included from include2/asm/smp.h:20, from /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, from /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: include2/asm/io_apic.h: In function ‘io_apic_read’: include2/asm/io_apic.h:171: error: implicit declaration of function ‘HYPERVISOR_physdev_op’ In file included from /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, from /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: include2/asm/smp.h: In function ‘hard_smp_processor_id’: include2/asm/smp.h:79: error: implicit declaration of function ‘GET_APIC_ID’ include2/asm/smp.h:79: error: ‘APIC_BASE’ undeclared (first use in this function) include2/asm/smp.h:79: error: (Each undeclared identifier is reported only once include2/asm/smp.h:79: error: for each function it appears in.) include2/asm/smp.h:79: error: ‘APIC_ID’ undeclared (first use in this function) include2/asm/smp.h: In function ‘logical_smp_processor_id’: include2/asm/smp.h:137: error: implicit declaration of function ‘GET_APIC_LOGICAL_ID’ include2/asm/smp.h:137: error: ‘APIC_BASE’ undeclared (first use in this function) include2/asm/smp.h:137: error: ‘APIC_LDR’ undeclared (first use in this function) make[5]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.o] Error 1 make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3] Error 2 make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 make[2]: *** [modules] Error 2 make[1]: *** [modules] Error 2 make[1]: Leaving directory `/usr/src/linux-2.6.16.46-0.12-obj/x86_64/xen' make: *** [kernel] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.87108 (%install) RPM build errors: user vlad does not exist - using root group vlad does not exist - using root user vlad does not exist - using root group vlad does not exist - using root Bad exit status from /var/tmp/rpm-tmp.87108 (%install) ERROR: Failed executing "rpmbuild --rebuild --define '_topdir /var/tmp/OFEDRPM' --define '_prefix /usr' --define 'build_root /var/tmp/OFED' --define 'configure_options --with-cxgb3-mod --with-ipath_inf-mod --with-ipoib-mod --with-iser-mod --with-mthca-mod --with-sdp-mod --with-srp-mod --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod --with-rds-mod --with-vnic-mod ' --define 'KVERSION 2.6.16.46-0.12-xen' --define 'KSRC /lib/modules/2.6.16.46-0.12-xen/build' --define 'build_kernel_ib 1' --define 'build_kernel_ib_devel 1' --define 'NETWORK_CONF_DIR /etc/sysconfig/network' --define 'modprobe_update 1' --define 'include_ipoib_conf 1' /root/OFED-1.2.5/SRPMS/ofa_kernel-1.2.5-0.src.rpm" -- regards, Arif Ali Software Engineer OCF plc Mobile: +44 (0)7970 148 122 Office: +44 (0)114 257 2200 Fax: +44 (0)114 257 0022 Email: aali at ocf.co.uk Web: http://www.ocf.co.uk Support Phone: +44 (0)845 702 3829 Support E-mail: support at ocf.co.uk Skype: arif_ali80 MSN: aali at ocf.co.uk This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email. From HNGUYEN at de.ibm.com Thu Aug 16 09:48:18 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 16 Aug 2007 18:48:18 +0200 Subject: [ofa-general] [PATCH 7/7] IB/ehca: Prevent overwriting QP init attributes given by caller In-Reply-To: Message-ID: general-bounces at lists.openfabrics.org wrote on 16.08.2007 18:17:22: > I don't understand this patch. says this about > ib_create_qp(): > > * @qp_init_attr: A list of initial attributes required to create the > * QP. If QP creation succeeds, then the attributes are updated to > * the actual capabilities of the created QP. > > So it seems the current code is actually correct and your patch breaks > it?? Yes. That's absolutely true. Please ignore it. Thanks! Nam From kilian at stanford.edu Thu Aug 16 11:00:30 2007 From: kilian at stanford.edu (Kilian Cavalotti) Date: Thu, 16 Aug 2007 11:00:30 -0700 Subject: [ofa-general] OFED 1.2.5 - GA release In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> Message-ID: <20070816110030.2wsorokd15skkcwo@mobilemail.stanford.edu> Hi all, Quoting Tziporet Koren : > I am happy to announce on OFED 1.2.5 GA release. Thanks for the release. I just wanted to let you know that I got a compilation error (using RHEL4) in ofa_kernel: In file included from /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attri bute_container.c:1: /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/../drivers/base/attribute_contai ner.c:23:18: base.h: No such file or directory make[5]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attribute_con tainer.o] Error 1 make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi] Error 2 make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 make[2]: *** [modules] Error 2 make[1]: *** [modules] Error 2 make[1]: Leaving directory `/usr/src/linux-2.6.9-55.EL_lustre-1.6.1-obj/x86_64/s mp' make: *** [kernel] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.56294 (%install) The drivers/base/attribute_container.c include a "base.h" which is not provided. Copying the vanilla kernel drivers/base/base.h file in the ofa_kernel drivers/base directory works, as well as the following dirty patch: --- attribute_container.c.orig 2007-08-16 10:59:08.000000000 -0700 +++ attribute_container.c 2007-08-16 10:57:21.000000000 -0700 @@ -20,7 +20,7 @@ #include #include -#include "base.h" +#include <../drivers/base/base.h> /* This is a private structure used to tie the classdev and the * container .. it should never be visible outside this file */ Cheers -- Kilian From mst at dev.mellanox.co.il Thu Aug 16 12:10:03 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 16 Aug 2007 22:10:03 +0300 Subject: [ofa-general] Re: OFED 1.2.5 - GA release In-Reply-To: <20070816110030.2wsorokd15skkcwo@mobilemail.stanford.edu> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> <20070816110030.2wsorokd15skkcwo@mobilemail.stanford.edu> Message-ID: <20070816191003.GC7197@mellanox.co.il> > Quoting Kilian Cavalotti : > Subject: Re: OFED 1.2.5 - GA release > > Hi all, > > Quoting Tziporet Koren : > >I am happy to announce on OFED 1.2.5 GA release. > > Thanks for the release. I just wanted to let you know that I got a > compilation error (using RHEL4) in ofa_kernel: > > In file included from > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attri > bute_container.c:1: > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/../drivers/base/attribute_contai > ner.c:23:18: base.h: No such file or directory > make[5]: *** > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attribute_con > tainer.o] Error 1 > make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi] Error 2 > make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 > make[2]: *** [modules] Error 2 > make[1]: *** [modules] Error 2 > make[1]: Leaving directory > `/usr/src/linux-2.6.9-55.EL_lustre-1.6.1-obj/x86_64/s > mp' > make: *** [kernel] Error 2 > error: Bad exit status from /var/tmp/rpm-tmp.56294 (%install) > > > The drivers/base/attribute_container.c include a "base.h" which is not > provided. Copying the vanilla kernel drivers/base/base.h file in the > ofa_kernel drivers/base directory works, as well as the following > dirty patch: > > --- attribute_container.c.orig 2007-08-16 10:59:08.000000000 -0700 > +++ attribute_container.c 2007-08-16 10:57:21.000000000 -0700 > @@ -20,7 +20,7 @@ > #include > #include > > -#include "base.h" > +#include <../drivers/base/base.h> > > /* This is a private structure used to tie the classdev and the > * container .. it should never be visible outside this file */ This is iser stuff. A simple workaround is to disable iser before build. Erez, can you check this please? -- MST From arthur.jones at qlogic.com Thu Aug 16 12:40:31 2007 From: arthur.jones at qlogic.com (Arthur Jones) Date: Thu, 16 Aug 2007 12:40:31 -0700 Subject: [ofa-general] ib_umem_get lock_limiting... Message-ID: <20070816194030.GE12448@bauxite.pathscale.com> hi all, in ib_umem_get() in drivers/infiniband/core/umem.c in the kernel.org driver at around line 115, we: locked = npages + current->mm->locked_vm; lock_limit = current->signal->rlim[RLIMIT_MEMLOCK].rlim_cur >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { ret = -ENOMEM; goto out; } the calculation looks ok, but it looks to me like the && should be an || in the test, esp since earlier in the routine, we do an: if (!can_do_mlock()) return ERR_PTR(-EPERM); so, maybe the !capable(CAP_IPC_LOCK) should not be there at all? arthur From rdreier at cisco.com Thu Aug 16 13:33:04 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 16 Aug 2007 13:33:04 -0700 Subject: [ofa-general] ib_umem_get lock_limiting... In-Reply-To: <20070816194030.GE12448@bauxite.pathscale.com> (Arthur Jones's message of "Thu, 16 Aug 2007 12:40:31 -0700") References: <20070816194030.GE12448@bauxite.pathscale.com> Message-ID: > the calculation looks ok, but it looks to me like > the && should be an || in the test, esp since earlier > in the routine, we do an: > > if (!can_do_mlock()) > return ERR_PTR(-EPERM); > > so, maybe the !capable(CAP_IPC_LOCK) should not be > there at all? No, I think the code is correct. CAP_IPC_LOCK basically means we are root and should ignore resource limits. You can compare against the code in sys_mlock() to see that it has exactly the same logic. From arthur.jones at qlogic.com Thu Aug 16 14:16:09 2007 From: arthur.jones at qlogic.com (Arthur Jones) Date: Thu, 16 Aug 2007 14:16:09 -0700 Subject: [ofa-general] ib_umem_get lock_limiting... In-Reply-To: References: <20070816194030.GE12448@bauxite.pathscale.com> Message-ID: <20070816211609.GF12448@bauxite.pathscale.com> hi roland, thanks, i get it now, it also helped when i saw this in the memlock manpage: Limits and permissions In Linux 2.6.8 and earlier, a process must be privileged (CAP_IPC_LOCK) in order to lock memory and the RLIMIT_MEMLOCK soft resource limit defines a limit on how much memory the process may lock. Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process can lock and the RLIMIT_MEMLOCK soft resource limit instead defines a limit on how much memory an unprivileged process may lock. arthur On Thu, Aug 16, 2007 at 01:33:04PM -0700, Roland Dreier wrote: > > the calculation looks ok, but it looks to me like > > the && should be an || in the test, esp since earlier > > in the routine, we do an: > > > > if (!can_do_mlock()) > > return ERR_PTR(-EPERM); > > > > so, maybe the !capable(CAP_IPC_LOCK) should not be > > there at all? > > No, I think the code is correct. CAP_IPC_LOCK basically means we are > root and should ignore resource limits. You can compare against the > code in sys_mlock() to see that it has exactly the same logic. From davem at davemloft.net Thu Aug 16 14:17:51 2007 From: davem at davemloft.net (David Miller) Date: Thu, 16 Aug 2007 14:17:51 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <1187271791.4685.9.camel@trinity.ogc.int> References: <46C310E1.7020503@opengridcomputing.com> <46C3B5EF.5060409@garzik.org> <1187271791.4685.9.camel@trinity.ogc.int> Message-ID: <20070816.141751.115907875.davem@davemloft.net> From: Tom Tucker Date: Thu, 16 Aug 2007 08:43:11 -0500 > Isn't RDMA _part_ of the "software net stack" within Linux? It very much is not so. When using RDMA you lose the capability to do packet shaping, classification, and all the other wonderful networking facilities you've grown to love and use over the years. I'm glad this is a surprise to you, because it illustrates the point some of us keep trying to make about technologies like this. Imagine if you didn't know any of this, you purchase and begin to deploy a huge piece of RDMA infrastructure, you then get the mandate from IT that you need to add firewalling on the RDMA connections at the host level, and "oh shit" you can't? This is why none of us core networking developers like RDMA at all. It's totally not integrated with the rest of the Linux stack and on top of that it even gets in the way. It's an abberation, an eye sore, and a constant source of consternation. From sashak at voltaire.com Thu Aug 16 14:39:03 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 00:39:03 +0300 Subject: [ofa-general] [PATCH] opensm/opensm: indentation changes Message-ID: <20070816213902.GJ27819@sashak.voltaire.com> Formatted with opensm/osm_indent (indent -npro -kr -i8 -ts8 -sob -l80 -ss -ncs) All opensm/opensm source files were formatted except: opensm/osm_lid_mgr.c opensm/osm_link_mgr.c opensm/osm_port.c opensm/osm_port_info_rcv.c opensm/osm_sa_link_record.c opensm/osm_sa_multipath_record.c opensm/osm_sa_path_record.c opensm/osm_sa_slvl_record.c opensm/osm_sminfo_rcv.c opensm/osm_state_mgr.c opensm/osm_subnet.c opensm/osm_trap_rcv.c opensm/osm_switch.c opensm/osm_ucast_lash.c opensm/osm_ucast_ftree.c opensm/osm_vl15intf.c Here very long names and very long flows are overused, which requires additional cleanup. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/main.c | 1752 +++++++------- opensm/opensm/osm_console.c | 747 +++--- opensm/opensm/osm_db_files.c | 1107 ++++----- opensm/opensm/osm_db_pack.c | 171 +- opensm/opensm/osm_drop_mgr.c | 972 ++++---- opensm/opensm/osm_dump.c | 13 +- opensm/opensm/osm_event_plugin.c | 29 +- opensm/opensm/osm_fwd_tbl.c | 80 +- opensm/opensm/osm_helper.c | 4270 +++++++++++++++----------------- opensm/opensm/osm_inform.c | 1122 ++++----- opensm/opensm/osm_lin_fwd_rcv.c | 147 +- opensm/opensm/osm_lin_fwd_tbl.c | 55 +- opensm/opensm/osm_log.c | 424 ++-- opensm/opensm/osm_mad_pool.c | 349 ++-- opensm/opensm/osm_mcast_fwd_rcv.c | 179 +- opensm/opensm/osm_mcast_mgr.c | 2509 +++++++++---------- opensm/opensm/osm_mcast_tbl.c | 339 ++-- opensm/opensm/osm_mcm_info.c | 28 +- opensm/opensm/osm_mcm_port.c | 52 +- opensm/opensm/osm_mtree.c | 98 +- opensm/opensm/osm_multicast.c | 509 ++-- opensm/opensm/osm_node.c | 338 ++-- opensm/opensm/osm_node_desc_rcv.c | 153 +- opensm/opensm/osm_node_info_rcv.c | 1615 ++++++------- opensm/opensm/osm_opensm.c | 444 ++-- opensm/opensm/osm_perfmgr.c | 621 +++--- opensm/opensm/osm_perfmgr_db.c | 516 ++-- opensm/opensm/osm_pkey.c | 715 +++--- opensm/opensm/osm_pkey_mgr.c | 909 ++++---- opensm/opensm/osm_pkey_rcv.c | 245 +- opensm/opensm/osm_prtn.c | 128 +- opensm/opensm/osm_prtn_config.c | 83 +- opensm/opensm/osm_qos.c | 38 +- opensm/opensm/osm_remote_sm.c | 34 +- opensm/opensm/osm_req.c | 371 ++-- opensm/opensm/osm_resp.c | 245 +- opensm/opensm/osm_router.c | 31 +- opensm/opensm/osm_sa.c | 875 +++---- opensm/opensm/osm_sa_class_port_info.c | 331 ++-- opensm/opensm/osm_sa_guidinfo_record.c | 948 ++++---- opensm/opensm/osm_sa_informinfo.c | 1429 ++++++------ opensm/opensm/osm_sa_lft_record.c | 766 +++--- opensm/opensm/osm_sa_mad_ctrl.c | 918 ++++---- opensm/opensm/osm_sa_mcmember_record.c | 3882 ++++++++++++++--------------- opensm/opensm/osm_sa_mft_record.c | 823 +++---- opensm/opensm/osm_sa_node_record.c | 935 ++++---- opensm/opensm/osm_sa_pkey_record.c | 913 ++++---- opensm/opensm/osm_sa_portinfo_record.c | 1414 +++++------ opensm/opensm/osm_sa_response.c | 175 +- opensm/opensm/osm_sa_service_record.c | 1978 +++++++-------- opensm/opensm/osm_sa_sminfo_record.c | 888 ++++---- opensm/opensm/osm_sa_sw_info_record.c | 798 +++--- opensm/opensm/osm_sa_vlarb_record.c | 865 ++++---- opensm/opensm/osm_service.c | 155 +- opensm/opensm/osm_slvl_map_rcv.c | 252 +- opensm/opensm/osm_sm.c | 1309 +++++----- opensm/opensm/osm_sm_mad_ctrl.c | 1519 ++++++------ opensm/opensm/osm_sm_state_mgr.c | 1382 +++++------ opensm/opensm/osm_state_mgr_ctrl.c | 82 +- opensm/opensm/osm_sw_info_rcv.c | 1052 ++++----- opensm/opensm/osm_sweep_fail_ctrl.c | 103 +- opensm/opensm/osm_ucast_file.c | 43 +- opensm/opensm/osm_ucast_mgr.c | 1374 +++++------ opensm/opensm/osm_ucast_updn.c | 1343 +++++------ opensm/opensm/osm_vl_arb_rcv.c | 260 +-- opensm/opensm/st.c | 727 +++--- 66 files changed, 23551 insertions(+), 25426 deletions(-) diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index f3429ff..0827f1c 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -44,7 +44,7 @@ */ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -71,7 +71,7 @@ static volatile unsigned int osm_usr1_flag = 0; static void mark_exit_flag(int signum) { - if(!osm_exit_flag) + if (!osm_exit_flag) printf("OpenSM: Got signal %d - exiting...\n", signum); osm_exit_flag = 1; } @@ -123,338 +123,326 @@ static void setup_signals() /********************************************************************** **********************************************************************/ -void -show_usage(void) +void show_usage(void) { - printf( "\n------- OpenSM - Usage and options ----------------------\n" ); - printf( "Usage: opensm [options]\n"); - printf( "Options:\n" ); - printf( "-c\n" - "--cache-options\n" - " Cache the given command line options into the file\n" - " /var/cache/opensm/opensm.opts for use on next invocation.\n" - " The cache directory can be changed by the environment\n" - " variable OSM_CACHE_DIR\n\n"); - printf( "-g[=]\n" - "--guid[=]\n" - " This option specifies the local port GUID value\n" - " with which OpenSM should bind. OpenSM may be\n" - " bound to 1 port at a time.\n" - " If GUID given is 0, OpenSM displays a list\n" - " of possible port GUIDs and waits for user input.\n" - " Without -g, OpenSM tries to use the default port.\n\n"); - printf( "-l \n" - "--lmc \n" - " This option specifies the subnet's LMC value.\n" - " The number of LIDs assigned to each port is 2^LMC.\n" - " The LMC value must be in the range 0-7.\n" - " LMC values > 0 allow multiple paths between ports.\n" - " LMC values > 0 should only be used if the subnet\n" - " topology actually provides multiple paths between\n" - " ports, i.e. multiple interconnects between switches.\n" - " Without -l, OpenSM defaults to LMC = 0, which allows\n" - " one path between any two ports.\n\n" ); - printf( "-p \n" - "--priority \n" - " This option specifies the SM's PRIORITY.\n" - " This will effect the handover cases, where master\n" - " is chosen by priority and GUID. Range goes\n" - " from 0 (lowest priority) to 15 (highest).\n\n" ); - printf( "-smkey \n" - " This option specifies the SM's SM_Key (64 bits).\n" - " This will effect SM authentication.\n\n" ); - printf( "-r\n" - "--reassign_lids\n" - " This option causes OpenSM to reassign LIDs to all\n" - " end nodes. Specifying -r on a running subnet\n" - " may disrupt subnet traffic.\n" - " Without -r, OpenSM attempts to preserve existing\n" - " LID assignments resolving multiple use of same LID.\n\n"); - printf( "-R\n" - "--routing_engine \n" - " This option chooses routing engine instead of Min Hop\n" - " algorithm (default).\n" - " Supported engines: updn, file, ftree, lash\n\n"); - printf( "-z\n" - "--connect_roots\n" - " This option enforces a routing engine (currently\n" - " up/down only) to make connectivity between root switches\n" - " and in this way be IBA compliant. In many cases,\n" - " this can violate \"pure\" deadlock free algorithm, so\n" - " use it carefully.\n\n"); - printf( "-M\n" - "--lid_matrix_file \n" - " This option specifies the name of the lid matrix dump file\n" - " from where switch lid matrices (min hops tables will be\n" - " loaded.\n\n"); - printf( "-U\n" - "--ucast_file \n" - " This option specifies the name of the unicast dump file\n" - " from where switch forwarding tables will be loaded.\n\n"); - printf( "-S\n" - "--sadb_file \n" - " This option specifies the name of the SA DB dump file\n" - " from where SA database will be loaded.\n\n"); - printf ("-a\n" - "--root_guid_file \n" - " Set the root nodes for the Up/Down or Fat-Tree routing\n" - " algorithm to the guids provided in the given file (one\n" - " to a line)\n" - "\n"); - printf ("-u\n" - "--cn_guid_file \n" - " Set the compute nodes for the Fat-Tree routing algorithm\n" - " to the guids provided in the given file (one to a line)\n" - "\n"); - printf( "-o\n" - "--once\n" - " This option causes OpenSM to configure the subnet\n" - " once, then exit. Ports remain in the ACTIVE state.\n\n" ); - printf( "-s \n" - "--sweep \n" - " This option specifies the number of seconds between\n" - " subnet sweeps. Specifying -s 0 disables sweeping.\n" - " Without -s, OpenSM defaults to a sweep interval of\n" - " 10 seconds.\n\n" ); - printf( "-t \n" - "--timeout \n" - " This option specifies the time in milliseconds\n" - " used for transaction timeouts.\n" - " Specifying -t 0 disables timeouts.\n" - " Without -t, OpenSM defaults to a timeout value of\n" - " 200 milliseconds.\n\n" ); - printf( "-maxsmps \n" - " This option specifies the number of VL15 SMP MADs\n" - " allowed on the wire at any one time.\n" - " Specifying -maxsmps 0 allows unlimited outstanding\n" - " SMPs.\n" - " Without -maxsmps, OpenSM defaults to a maximum of\n" - " 4 outstanding SMPs.\n\n" ); - printf( "-console [off|local" + printf("\n------- OpenSM - Usage and options ----------------------\n"); + printf("Usage: opensm [options]\n"); + printf("Options:\n"); + printf("-c\n" + "--cache-options\n" + " Cache the given command line options into the file\n" + " /var/cache/opensm/opensm.opts for use on next invocation.\n" + " The cache directory can be changed by the environment\n" + " variable OSM_CACHE_DIR\n\n"); + printf("-g[=]\n" + "--guid[=]\n" + " This option specifies the local port GUID value\n" + " with which OpenSM should bind. OpenSM may be\n" + " bound to 1 port at a time.\n" + " If GUID given is 0, OpenSM displays a list\n" + " of possible port GUIDs and waits for user input.\n" + " Without -g, OpenSM tries to use the default port.\n\n"); + printf("-l \n" + "--lmc \n" + " This option specifies the subnet's LMC value.\n" + " The number of LIDs assigned to each port is 2^LMC.\n" + " The LMC value must be in the range 0-7.\n" + " LMC values > 0 allow multiple paths between ports.\n" + " LMC values > 0 should only be used if the subnet\n" + " topology actually provides multiple paths between\n" + " ports, i.e. multiple interconnects between switches.\n" + " Without -l, OpenSM defaults to LMC = 0, which allows\n" + " one path between any two ports.\n\n"); + printf("-p \n" + "--priority \n" + " This option specifies the SM's PRIORITY.\n" + " This will effect the handover cases, where master\n" + " is chosen by priority and GUID. Range goes\n" + " from 0 (lowest priority) to 15 (highest).\n\n"); + printf("-smkey \n" + " This option specifies the SM's SM_Key (64 bits).\n" + " This will effect SM authentication.\n\n"); + printf("-r\n" + "--reassign_lids\n" + " This option causes OpenSM to reassign LIDs to all\n" + " end nodes. Specifying -r on a running subnet\n" + " may disrupt subnet traffic.\n" + " Without -r, OpenSM attempts to preserve existing\n" + " LID assignments resolving multiple use of same LID.\n\n"); + printf("-R\n" + "--routing_engine \n" + " This option chooses routing engine instead of Min Hop\n" + " algorithm (default).\n" + " Supported engines: updn, file, ftree, lash\n\n"); + printf("-z\n" + "--connect_roots\n" + " This option enforces a routing engine (currently\n" + " up/down only) to make connectivity between root switches\n" + " and in this way be IBA compliant. In many cases,\n" + " this can violate \"pure\" deadlock free algorithm, so\n" + " use it carefully.\n\n"); + printf("-M\n" + "--lid_matrix_file \n" + " This option specifies the name of the lid matrix dump file\n" + " from where switch lid matrices (min hops tables will be\n" + " loaded.\n\n"); + printf("-U\n" + "--ucast_file \n" + " This option specifies the name of the unicast dump file\n" + " from where switch forwarding tables will be loaded.\n\n"); + printf("-S\n" + "--sadb_file \n" + " This option specifies the name of the SA DB dump file\n" + " from where SA database will be loaded.\n\n"); + printf("-a\n" + "--root_guid_file \n" + " Set the root nodes for the Up/Down or Fat-Tree routing\n" + " algorithm to the guids provided in the given file (one\n" + " to a line)\n" "\n"); + printf("-u\n" + "--cn_guid_file \n" + " Set the compute nodes for the Fat-Tree routing algorithm\n" + " to the guids provided in the given file (one to a line)\n" + "\n"); + printf("-o\n" + "--once\n" + " This option causes OpenSM to configure the subnet\n" + " once, then exit. Ports remain in the ACTIVE state.\n\n"); + printf("-s \n" + "--sweep \n" + " This option specifies the number of seconds between\n" + " subnet sweeps. Specifying -s 0 disables sweeping.\n" + " Without -s, OpenSM defaults to a sweep interval of\n" + " 10 seconds.\n\n"); + printf("-t \n" + "--timeout \n" + " This option specifies the time in milliseconds\n" + " used for transaction timeouts.\n" + " Specifying -t 0 disables timeouts.\n" + " Without -t, OpenSM defaults to a timeout value of\n" + " 200 milliseconds.\n\n"); + printf("-maxsmps \n" + " This option specifies the number of VL15 SMP MADs\n" + " allowed on the wire at any one time.\n" + " Specifying -maxsmps 0 allows unlimited outstanding\n" + " SMPs.\n" + " Without -maxsmps, OpenSM defaults to a maximum of\n" + " 4 outstanding SMPs.\n\n"); + printf("-console [off|local" #ifdef ENABLE_OSM_CONSOLE_SOCKET - "|socket" + "|socket" #endif - "]\n This option activates the OpenSM console (default off).\n\n"); + "]\n This option activates the OpenSM console (default off).\n\n"); #ifdef ENABLE_OSM_CONSOLE_SOCKET - printf( "-console-port \n" - " Specify an alternate telnet port for the console (default %d).\n\n", - OSM_DEFAULT_CONSOLE_PORT); + printf("-console-port \n" + " Specify an alternate telnet port for the console (default %d).\n\n", + OSM_DEFAULT_CONSOLE_PORT); #endif - printf( "-i \n" - "-ignore-guids \n" - " This option provides the means to define a set of ports\n" - " (by guid) that will be ignored by the link load\n" - " equalization algorithm.\n\n" ); - printf( "-x\n" - "--honor_guid2lid\n" - " This option forces OpenSM to honor the guid2lid file,\n" - " when it comes out of Standby state, if such file exists\n" - " under OSM_CACHE_DIR, and is valid. By default, this is FALSE.\n\n" ); - printf( "-f\n" - "--log_file\n" - " This option defines the log to be the given file.\n" - " By default, the log goes to /var/log/opensm.log.\n" - " For the log to go to standard output use -f stdout.\n\n"); - printf( "-L \n" - "--log_limit \n" - " This option defines maximal log file size in MB. When\n" - " specified the log file will be truncated upon reaching\n" - " this limit.\n\n"); - printf( "-e\n" - "--erase_log_file\n" - " This option will cause deletion of the log file\n" - " (if it previously exists). By default, the log file\n" - " is accumulative.\n\n"); - printf( "-P\n" - "--Pconfig\n" - " This option defines the optional partition configuration file.\n" - " The default name is \'" OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); - printf( "-Q\n" - "--qos\n" - " This option enables QoS setup.\n\n"); - printf( "-N\n" - "--no_part_enforce\n" - " This option disables partition enforcement on switch external ports.\n\n"); - printf( "-y\n" - "--stay_on_fatal\n" - " This option will cause SM not to exit on fatal initialization\n" - " issues: if SM discovers duplicated guids or 12x link with\n" - " lane reversal badly configured.\n" - " By default, the SM will exit on these errors.\n\n"); - printf( "-B\n" - "--daemon\n" - " Run in daemon mode - OpenSM will run in the background.\n\n"); - printf("-I\n" - "--inactive\n" - " Start SM in inactive rather than normal init SM state.\n\n"); - printf( "-v\n" - "--verbose\n" - " This option increases the log verbosity level.\n" - " The -v option may be specified multiple times\n" - " to further increase the verbosity level.\n" - " See the -D option for more information about\n" - " log verbosity.\n\n" ); - printf( "-V\n" - " This option sets the maximum verbosity level and\n" - " forces log flushing.\n" - " The -V is equivalent to '-D 0xFF -d 2'.\n" - " See the -D option for more information about\n" - " log verbosity.\n\n" ); - printf( "-D \n" - " This option sets the log verbosity level.\n" - " A flags field must follow the -D option.\n" - " A bit set/clear in the flags enables/disables a\n" - " specific log level as follows:\n" - " BIT LOG LEVEL ENABLED\n" - " ---- -----------------\n" - " 0x01 - ERROR (error messages)\n" - " 0x02 - INFO (basic messages, low volume)\n" - " 0x04 - VERBOSE (interesting stuff, moderate volume)\n" - " 0x08 - DEBUG (diagnostic, high volume)\n" - " 0x10 - FUNCS (function entry/exit, very high volume)\n" - " 0x20 - FRAMES (dumps all SMP and GMP frames)\n" - " 0x40 - ROUTING (dump FDB routing information)\n" - " 0x80 - currently unused.\n" - " Without -D, OpenSM defaults to ERROR + INFO (0x3).\n" - " Specifying -D 0 disables all messages.\n" - " Specifying -D 0xFF enables all messages (see -V).\n" - " High verbosity levels may require increasing\n" - " the transaction timeout with the -t option.\n\n" ); - printf( "-d \n" - "--debug \n" - " This option specifies a debug option.\n" - " These options are not normally needed.\n" - " The number following -d selects the debug\n" - " option to enable as follows:\n" - " OPT Description\n" - " --- -----------------\n" - " -d0 - Ignore other SM nodes\n" - " -d1 - Force single threaded dispatching\n" - " -d2 - Force log flushing after each log message\n" - " -d3 - Disable multicast support\n" - " -d10 - Put OpenSM in testability mode\n" - " Without -d, no debug options are enabled\n\n" ); - printf( "-h\n" - "--help\n" - " Display this usage info then exit.\n\n" ); - printf( "-?\n" - " Display this usage info then exit.\n\n" ); - fflush( stdout ); - exit(2); + printf("-i \n" + "-ignore-guids \n" + " This option provides the means to define a set of ports\n" + " (by guid) that will be ignored by the link load\n" + " equalization algorithm.\n\n"); + printf("-x\n" + "--honor_guid2lid\n" + " This option forces OpenSM to honor the guid2lid file,\n" + " when it comes out of Standby state, if such file exists\n" + " under OSM_CACHE_DIR, and is valid. By default, this is FALSE.\n\n"); + printf("-f\n" + "--log_file\n" + " This option defines the log to be the given file.\n" + " By default, the log goes to /var/log/opensm.log.\n" + " For the log to go to standard output use -f stdout.\n\n"); + printf("-L \n" + "--log_limit \n" + " This option defines maximal log file size in MB. When\n" + " specified the log file will be truncated upon reaching\n" + " this limit.\n\n"); + printf("-e\n" + "--erase_log_file\n" + " This option will cause deletion of the log file\n" + " (if it previously exists). By default, the log file\n" + " is accumulative.\n\n"); + printf("-P\n" + "--Pconfig\n" + " This option defines the optional partition configuration file.\n" + " The default name is \'" + OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); + printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); + printf("-N\n" + "--no_part_enforce\n" + " This option disables partition enforcement on switch external ports.\n\n"); + printf("-y\n" + "--stay_on_fatal\n" + " This option will cause SM not to exit on fatal initialization\n" + " issues: if SM discovers duplicated guids or 12x link with\n" + " lane reversal badly configured.\n" + " By default, the SM will exit on these errors.\n\n"); + printf("-B\n" + "--daemon\n" + " Run in daemon mode - OpenSM will run in the background.\n\n"); + printf("-I\n" + "--inactive\n" + " Start SM in inactive rather than normal init SM state.\n\n"); + printf("-v\n" + "--verbose\n" + " This option increases the log verbosity level.\n" + " The -v option may be specified multiple times\n" + " to further increase the verbosity level.\n" + " See the -D option for more information about\n" + " log verbosity.\n\n"); + printf("-V\n" + " This option sets the maximum verbosity level and\n" + " forces log flushing.\n" + " The -V is equivalent to '-D 0xFF -d 2'.\n" + " See the -D option for more information about\n" + " log verbosity.\n\n"); + printf("-D \n" + " This option sets the log verbosity level.\n" + " A flags field must follow the -D option.\n" + " A bit set/clear in the flags enables/disables a\n" + " specific log level as follows:\n" + " BIT LOG LEVEL ENABLED\n" + " ---- -----------------\n" + " 0x01 - ERROR (error messages)\n" + " 0x02 - INFO (basic messages, low volume)\n" + " 0x04 - VERBOSE (interesting stuff, moderate volume)\n" + " 0x08 - DEBUG (diagnostic, high volume)\n" + " 0x10 - FUNCS (function entry/exit, very high volume)\n" + " 0x20 - FRAMES (dumps all SMP and GMP frames)\n" + " 0x40 - ROUTING (dump FDB routing information)\n" + " 0x80 - currently unused.\n" + " Without -D, OpenSM defaults to ERROR + INFO (0x3).\n" + " Specifying -D 0 disables all messages.\n" + " Specifying -D 0xFF enables all messages (see -V).\n" + " High verbosity levels may require increasing\n" + " the transaction timeout with the -t option.\n\n"); + printf("-d \n" + "--debug \n" + " This option specifies a debug option.\n" + " These options are not normally needed.\n" + " The number following -d selects the debug\n" + " option to enable as follows:\n" + " OPT Description\n" + " --- -----------------\n" + " -d0 - Ignore other SM nodes\n" + " -d1 - Force single threaded dispatching\n" + " -d2 - Force log flushing after each log message\n" + " -d3 - Disable multicast support\n" + " -d10 - Put OpenSM in testability mode\n" + " Without -d, no debug options are enabled\n\n"); + printf("-h\n" + "--help\n" " Display this usage info then exit.\n\n"); + printf("-?\n" " Display this usage info then exit.\n\n"); + fflush(stdout); + exit(2); } /********************************************************************** **********************************************************************/ -ib_net64_t -get_port_guid( - IN osm_opensm_t *p_osm, uint64_t port_guid ) +ib_net64_t get_port_guid(IN osm_opensm_t * p_osm, uint64_t port_guid) { - uint32_t i; - uint32_t choice = 0; - char junk[128]; - boolean_t done_flag = FALSE; - ib_api_status_t status; - uint32_t num_ports = GUID_ARRAY_SIZE; - ib_port_attr_t attr_array[GUID_ARRAY_SIZE]; - - /* - Call the transport layer for a list of local port - GUID values. - */ - status = osm_vendor_get_all_port_attr( p_osm->p_vendor, attr_array, &num_ports ); - if( status != IB_SUCCESS ) - { - printf( "\nError from osm_vendor_get_all_port_attr (%x)\n", status); - return( 0 ); - } - - /* if num_ports is 0 - return 0 */ - if( num_ports == 0 ) - { - printf( "\nNo local ports detected!\n" ); - return( 0 ); - } - /* If num_ports is 1, then there is only one possible port to use. Use it. */ - if ( num_ports == 1 ) - { - printf("Using default GUID 0x%" PRIx64 "\n", cl_hton64(attr_array[0].port_guid)); - return( attr_array[0].port_guid ); - } + uint32_t i; + uint32_t choice = 0; + char junk[128]; + boolean_t done_flag = FALSE; + ib_api_status_t status; + uint32_t num_ports = GUID_ARRAY_SIZE; + ib_port_attr_t attr_array[GUID_ARRAY_SIZE]; + + /* + Call the transport layer for a list of local port + GUID values. + */ + status = + osm_vendor_get_all_port_attr(p_osm->p_vendor, attr_array, + &num_ports); + if (status != IB_SUCCESS) { + printf("\nError from osm_vendor_get_all_port_attr (%x)\n", + status); + return (0); + } + /* if num_ports is 0 - return 0 */ + if (num_ports == 0) { + printf("\nNo local ports detected!\n"); + return (0); + } + /* If num_ports is 1, then there is only one possible port to use. Use it. */ + if (num_ports == 1) { + printf("Using default GUID 0x%" PRIx64 "\n", + cl_hton64(attr_array[0].port_guid)); + return (attr_array[0].port_guid); + } #if defined ( OSM_VENDOR_INTF_OPENIB ) - /* If port_guid is 0, and this is gen2 - use the default port whose info is in attr_array[0] */ - if ( port_guid == 0 ) - { - printf("Using default GUID 0x%" PRIx64 "\n", cl_hton64(attr_array[0].port_guid)); - return( attr_array[0].port_guid ); - } -#endif /* OSM_VENDOR_INTF_OPENIB */ - - /* More than one possible port - list all ports and let the user to choose. */ - while( done_flag == FALSE ) - { - printf( "\nChoose a local port number with which to bind:\n\n" ); - /* If this is gen2 code - then port 0 has details of the default port used. - no need to print it. - If this is not gen2 code - need to print details of all ports. */ + /* If port_guid is 0, and this is gen2 - use the default port whose info is in attr_array[0] */ + if (port_guid == 0) { + printf("Using default GUID 0x%" PRIx64 "\n", + cl_hton64(attr_array[0].port_guid)); + return (attr_array[0].port_guid); + } +#endif /* OSM_VENDOR_INTF_OPENIB */ + + /* More than one possible port - list all ports and let the user to choose. */ + while (done_flag == FALSE) { + printf("\nChoose a local port number with which to bind:\n\n"); + /* If this is gen2 code - then port 0 has details of the default port used. + no need to print it. + If this is not gen2 code - need to print details of all ports. */ #if defined ( OSM_VENDOR_INTF_OPENIB ) - for( i = 1; i < num_ports; i++ ) - { - printf("\t%u: GUID 0x%8" PRIx64 ", lid 0x%04X, state %s\n", - i, cl_ntoh64( attr_array[i].port_guid ), - attr_array[i].lid, - ib_get_port_state_str( attr_array[i].link_state ) ); - } - printf( "\nEnter choice (1-%u): ", i-1 ); + for (i = 1; i < num_ports; i++) { + printf("\t%u: GUID 0x%8" PRIx64 + ", lid 0x%04X, state %s\n", i, + cl_ntoh64(attr_array[i].port_guid), + attr_array[i].lid, + ib_get_port_state_str(attr_array[i].link_state)); + } + printf("\nEnter choice (1-%u): ", i - 1); # else - for( i = 0; i < num_ports; i++ ) - { - /* - Print the index + 1 since by convention, port numbers - start with 1 on host channel adapters. - */ - - printf("\t%u: GUID 0x%8" PRIx64 ", lid 0x%04X, state %s\n", - i+1, cl_ntoh64( attr_array[i].port_guid ), - attr_array[i].lid, - ib_get_port_state_str( attr_array[i].link_state ) ); - } - printf( "\nEnter choice (1-%u): ", i ); -#endif /* OSM_VENDOR_INTF_OPENIB */ - - fflush( stdout ); - if (scanf( "%u", &choice )) - { - /* If gen2 code - choice can be between 1 to num_ports-1 - if not gen2 code - choice can be between 1 to num_ports */ + for (i = 0; i < num_ports; i++) { + /* + Print the index + 1 since by convention, port numbers + start with 1 on host channel adapters. + */ + + printf("\t%u: GUID 0x%8" PRIx64 + ", lid 0x%04X, state %s\n", i + 1, + cl_ntoh64(attr_array[i].port_guid), + attr_array[i].lid, + ib_get_port_state_str(attr_array[i].link_state)); + } + printf("\nEnter choice (1-%u): ", i); +#endif /* OSM_VENDOR_INTF_OPENIB */ + + fflush(stdout); + if (scanf("%u", &choice)) { + /* If gen2 code - choice can be between 1 to num_ports-1 + if not gen2 code - choice can be between 1 to num_ports */ #if defined ( OSM_VENDOR_INTF_OPENIB ) - if( choice >= num_ports ) + if (choice >= num_ports) # else - if( choice > num_ports || choice < 1 ) -#endif /* OSM_VENDOR_INTF_OPENIB */ - { - printf("\nError: Lame choice!\n"); - fflush( stdin ); - } - else - { - done_flag = TRUE; - } - } - else - { - /* get rid of the junk in the selection line */ - scanf( "%s", junk ); - printf("\nError: Lame choice!\n"); - fflush( stdin ); - } - } + if (choice > num_ports || choice < 1) +#endif /* OSM_VENDOR_INTF_OPENIB */ + { + printf("\nError: Lame choice!\n"); + fflush(stdin); + } else { + done_flag = TRUE; + } + } else { + /* get rid of the junk in the selection line */ + scanf("%s", junk); + printf("\nError: Lame choice!\n"); + fflush(stdin); + } + } #if defined ( OSM_VENDOR_INTF_OPENIB ) - printf("Choice guid=0x%8" PRIx64 "\n", cl_ntoh64( attr_array[choice].port_guid )); - return( attr_array[choice].port_guid ); + printf("Choice guid=0x%8" PRIx64 "\n", + cl_ntoh64(attr_array[choice].port_guid)); + return (attr_array[choice].port_guid); # else - return( attr_array[choice - 1].port_guid ); -#endif /* OSM_VENDOR_INTF_OPENIB */ + return (attr_array[choice - 1].port_guid); +#endif /* OSM_VENDOR_INTF_OPENIB */ } /********************************************************************** @@ -462,85 +450,76 @@ get_port_guid( #define OSM_MAX_IGNORE_GUID_LINES_LEN 128 static int -parse_ignore_guids_file(IN char *guids_file_name, - IN osm_opensm_t *p_osm) +parse_ignore_guids_file(IN char *guids_file_name, IN osm_opensm_t * p_osm) { - FILE *fh; - char line[OSM_MAX_IGNORE_GUID_LINES_LEN]; - char *p_c, *p_ec; - uint32_t line_num = 0; - uint64_t port_guid; - ib_api_status_t status = IB_SUCCESS; - unsigned int port_num; - - OSM_LOG_ENTER( &p_osm->log, parse_ignore_guids_file ); - - fh = fopen( guids_file_name, "r" ); - if( fh == NULL ) - { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "parse_ignore_guids_file: ERR 0601: " - "Unable to open ignore guids file (%s)\n", - guids_file_name ); - status = IB_ERROR; - goto Exit; - } - - /* - * Parse the file and add to the ignore guids map. - */ - while( fgets( line, OSM_MAX_IGNORE_GUID_LINES_LEN, fh ) != NULL ) - { - line_num++; - p_c = line; - while ( (*p_c == ' ') && (*p_c != '\0')) - p_c++; - port_guid = strtoull( p_c, &p_ec, 16 ); - if (p_ec == p_c) - { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "parse_ignore_guids_file: ERR 0602: " - "Error in line (%u): %s\n", - line_num, line - ); - status = IB_ERROR; - goto Exit; - } - - while ( (*p_ec == ' ') && (*p_ec != '\0')) - p_ec++; - if (! sscanf(p_ec, "%d", &port_num)) - { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "parse_ignore_guids_file: ERR 0603: " - "Error in line (%u): %s\n", - line_num, p_ec - ); - status = IB_ERROR; - goto Exit; - } - - /* ok insert it */ - osm_port_prof_set_ignored_port(&p_osm->subn, cl_hton64( port_guid ), port_num); - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "parse_ignore_guids_file: " - "Inserted Port: 0x%" PRIx64 " PortNum: 0x%X into ignored guids list\n" , - port_guid, port_num - ); - - } - - fclose( fh ); - - Exit: - OSM_LOG_EXIT( &p_osm->log ); - return ( status ); + FILE *fh; + char line[OSM_MAX_IGNORE_GUID_LINES_LEN]; + char *p_c, *p_ec; + uint32_t line_num = 0; + uint64_t port_guid; + ib_api_status_t status = IB_SUCCESS; + unsigned int port_num; + + OSM_LOG_ENTER(&p_osm->log, parse_ignore_guids_file); + + fh = fopen(guids_file_name, "r"); + if (fh == NULL) { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "parse_ignore_guids_file: ERR 0601: " + "Unable to open ignore guids file (%s)\n", + guids_file_name); + status = IB_ERROR; + goto Exit; + } + + /* + * Parse the file and add to the ignore guids map. + */ + while (fgets(line, OSM_MAX_IGNORE_GUID_LINES_LEN, fh) != NULL) { + line_num++; + p_c = line; + while ((*p_c == ' ') && (*p_c != '\0')) + p_c++; + port_guid = strtoull(p_c, &p_ec, 16); + if (p_ec == p_c) { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "parse_ignore_guids_file: ERR 0602: " + "Error in line (%u): %s\n", line_num, line); + status = IB_ERROR; + goto Exit; + } + + while ((*p_ec == ' ') && (*p_ec != '\0')) + p_ec++; + if (!sscanf(p_ec, "%d", &port_num)) { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "parse_ignore_guids_file: ERR 0603: " + "Error in line (%u): %s\n", line_num, p_ec); + status = IB_ERROR; + goto Exit; + } + + /* ok insert it */ + osm_port_prof_set_ignored_port(&p_osm->subn, + cl_hton64(port_guid), port_num); + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "parse_ignore_guids_file: " "Inserted Port: 0x%" PRIx64 + " PortNum: 0x%X into ignored guids list\n", port_guid, + port_num); + + } + + fclose(fh); + + Exit: + OSM_LOG_EXIT(&p_osm->log); + return (status); } /********************************************************************** **********************************************************************/ -static int daemonize(osm_opensm_t *osm) +static int daemonize(osm_opensm_t * osm) { pid_t pid; int fd; @@ -578,535 +557,520 @@ static int daemonize(osm_opensm_t *osm) /********************************************************************** **********************************************************************/ -int -main( - int argc, - char* argv[] ) +int main(int argc, char *argv[]) { - osm_opensm_t osm; - osm_subn_opt_t opt; - ib_net64_t sm_key = 0; - ib_api_status_t status; - uint32_t temp, dbg_lvl; - boolean_t run_once_flag = FALSE; - int32_t vendor_debug = 0; - uint32_t next_option; + osm_opensm_t osm; + osm_subn_opt_t opt; + ib_net64_t sm_key = 0; + ib_api_status_t status; + uint32_t temp, dbg_lvl; + boolean_t run_once_flag = FALSE; + int32_t vendor_debug = 0; + uint32_t next_option; #if 0 - uint32_t exitTimeout; + uint32_t exitTimeout; #endif - boolean_t cache_options = FALSE; - char *ignore_guids_file_name = NULL; - uint32_t val; - const char * const short_option = - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; - - /* - In the array below, the 2nd parameter specifies the number - of arguments as follows: - 0: no arguments - 1: argument - 2: optional - */ - const struct option long_option[] = - { - { "debug", 1, NULL, 'd'}, - { "guid", 1, NULL, 'g'}, - { "ignore_guids", 1, NULL, 'i'}, - { "lmc", 1, NULL, 'l'}, - { "sweep", 1, NULL, 's'}, - { "timeout", 1, NULL, 't'}, - { "verbose", 0, NULL, 'v'}, - { "D", 1, NULL, 'D'}, - { "log_file", 1, NULL, 'f'}, - { "log_limit", 1, NULL, 'L'}, - { "erase_log_file",0, NULL, 'e'}, - { "Pconfig", 1, NULL, 'P'}, - { "no_part_enforce",0,NULL, 'N'}, - { "qos", 0, NULL, 'Q'}, - { "maxsmps", 1, NULL, 'n'}, - { "console", 1, NULL, 'q'}, - { "V", 0, NULL, 'V'}, - { "help", 0, NULL, 'h'}, - { "once", 0, NULL, 'o'}, - { "reassign_lids", 0, NULL, 'r'}, - { "priority", 1, NULL, 'p'}, - { "smkey", 1, NULL, 'k'}, - { "routing_engine",1, NULL, 'R'}, - { "connect_roots", 0, NULL, 'z'}, - { "lid_matrix_file",1, NULL, 'M'}, - { "ucast_file", 1, NULL, 'U'}, - { "sadb_file", 1, NULL, 'S'}, - { "root_guid_file", 1, NULL, 'a'}, - { "cn_guid_file", 1, NULL, 'u'}, - { "cache-options", 0, NULL, 'c'}, - { "stay_on_fatal", 0, NULL, 'y'}, - { "honor_guid2lid",0, NULL, 'x'}, + boolean_t cache_options = FALSE; + char *ignore_guids_file_name = NULL; + uint32_t val; + const char *const short_option = + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; + + /* + In the array below, the 2nd parameter specifies the number + of arguments as follows: + 0: no arguments + 1: argument + 2: optional + */ + const struct option long_option[] = { + {"debug", 1, NULL, 'd'}, + {"guid", 1, NULL, 'g'}, + {"ignore_guids", 1, NULL, 'i'}, + {"lmc", 1, NULL, 'l'}, + {"sweep", 1, NULL, 's'}, + {"timeout", 1, NULL, 't'}, + {"verbose", 0, NULL, 'v'}, + {"D", 1, NULL, 'D'}, + {"log_file", 1, NULL, 'f'}, + {"log_limit", 1, NULL, 'L'}, + {"erase_log_file", 0, NULL, 'e'}, + {"Pconfig", 1, NULL, 'P'}, + {"no_part_enforce", 0, NULL, 'N'}, + {"qos", 0, NULL, 'Q'}, + {"maxsmps", 1, NULL, 'n'}, + {"console", 1, NULL, 'q'}, + {"V", 0, NULL, 'V'}, + {"help", 0, NULL, 'h'}, + {"once", 0, NULL, 'o'}, + {"reassign_lids", 0, NULL, 'r'}, + {"priority", 1, NULL, 'p'}, + {"smkey", 1, NULL, 'k'}, + {"routing_engine", 1, NULL, 'R'}, + {"connect_roots", 0, NULL, 'z'}, + {"lid_matrix_file", 1, NULL, 'M'}, + {"ucast_file", 1, NULL, 'U'}, + {"sadb_file", 1, NULL, 'S'}, + {"root_guid_file", 1, NULL, 'a'}, + {"cn_guid_file", 1, NULL, 'u'}, + {"cache-options", 0, NULL, 'c'}, + {"stay_on_fatal", 0, NULL, 'y'}, + {"honor_guid2lid", 0, NULL, 'x'}, #ifdef ENABLE_OSM_CONSOLE_SOCKET - { "console-port", 1, NULL, 'C'}, + {"console-port", 1, NULL, 'C'}, #endif - { "daemon", 0, NULL, 'B'}, - { "inactive", 0, NULL, 'I'}, + {"daemon", 0, NULL, 'B'}, + {"inactive", 0, NULL, 'I'}, #ifdef ENABLE_OSM_PERF_MGR - { "perfmgr", 0, NULL, 1 }, - { "perfmgr_sweep_time_s", 1, NULL, 2 }, + {"perfmgr", 0, NULL, 1}, + {"perfmgr_sweep_time_s", 1, NULL, 2}, #endif - { NULL, 0, NULL, 0 } /* Required at the end of the array */ - }; - - /* Make sure that the opensm and complib were compiled using - same modes (debug/free) */ - if ( osm_is_debug() != cl_is_debug() ) - { - fprintf(stderr, "ERROR: OpenSM and Complib were compiled using different modes\n"); - fprintf(stderr, "ERROR: OpenSM debug:%d Complib debug:%d \n", osm_is_debug(), cl_is_debug() ); - exit(1); - } - + {NULL, 0, NULL, 0} /* Required at the end of the array */ + }; + + /* Make sure that the opensm and complib were compiled using + same modes (debug/free) */ + if (osm_is_debug() != cl_is_debug()) { + fprintf(stderr, + "ERROR: OpenSM and Complib were compiled using different modes\n"); + fprintf(stderr, "ERROR: OpenSM debug:%d Complib debug:%d \n", + osm_is_debug(), cl_is_debug()); + exit(1); + } #if defined (_DEBUG_) && defined (OSM_VENDOR_INTF_OPENIB) - enable_stack_dump(1); + enable_stack_dump(1); #endif - printf("-------------------------------------------------\n"); - printf("%s\n", OSM_VERSION); - - osm_subn_set_default_opt(&opt); - if (osm_subn_parse_conf_file(&opt) != IB_SUCCESS) - printf("\nosm_subn_parse_conf_file failed!\n"); - - printf("Command Line Arguments:\n"); - do - { - next_option = getopt_long_only(argc, argv, short_option, - long_option, NULL); - switch(next_option) - { - case 'o': - /* - Run once option. - */ - run_once_flag = TRUE; - printf(" Run Once\n"); - break; - - case 'r': - /* - Reassign LIDs subnet option. - */ - opt.reassign_lids = TRUE; - printf(" Reassign LIDs\n"); - break; - - case 'i': - /* - Specifies ignore guids file. - */ - ignore_guids_file_name = optarg; - printf(" Ignore Guids File = %s\n", ignore_guids_file_name); - break; - - case 'g': - /* - Specifies port guid with which to bind. - */ - opt.guid = cl_hton64( strtoull( optarg, NULL, 16 )); - if (!opt.guid) - { - /* If guid is 0 - need to display the guid list */ - opt.guid = INVALID_GUID; - } - else - printf(" Guid <0x%"PRIx64">\n", cl_hton64( opt.guid )); - break; - - case 's': - val = strtol(optarg, NULL, 0); - /* Check that the number is not too large */ - if ( ((uint32_t)(val * 1000000)) / 1000000 != val ) - fprintf(stderr, "ERROR: sweep interval given is too large. Ignoring it.\n"); - else - { - opt.sweep_interval = val; - printf(" sweep interval = %d\n", opt.sweep_interval); - } - break; - - case 't': - opt.transaction_timeout = strtol(optarg, NULL, 0); - printf(" Transaction timeout = %d\n", opt.transaction_timeout); - break; - - case 'n': - opt.max_wire_smps = strtol(optarg, NULL, 0); - if( opt.max_wire_smps <= 0 ) - opt.max_wire_smps = 0x7FFFFFFF; - printf(" Max wire smp's = %d\n", opt.max_wire_smps); - break; - - case 'q': - /* - * OpenSM interactive console - */ - if (strcmp(optarg, "off") == 0) { - opt.console = "off"; - } else if (strcmp(optarg, "local") == 0) { - opt.console = "local"; + printf("-------------------------------------------------\n"); + printf("%s\n", OSM_VERSION); + + osm_subn_set_default_opt(&opt); + if (osm_subn_parse_conf_file(&opt) != IB_SUCCESS) + printf("\nosm_subn_parse_conf_file failed!\n"); + + printf("Command Line Arguments:\n"); + do { + next_option = getopt_long_only(argc, argv, short_option, + long_option, NULL); + switch (next_option) { + case 'o': + /* + Run once option. + */ + run_once_flag = TRUE; + printf(" Run Once\n"); + break; + + case 'r': + /* + Reassign LIDs subnet option. + */ + opt.reassign_lids = TRUE; + printf(" Reassign LIDs\n"); + break; + + case 'i': + /* + Specifies ignore guids file. + */ + ignore_guids_file_name = optarg; + printf(" Ignore Guids File = %s\n", + ignore_guids_file_name); + break; + + case 'g': + /* + Specifies port guid with which to bind. + */ + opt.guid = cl_hton64(strtoull(optarg, NULL, 16)); + if (!opt.guid) { + /* If guid is 0 - need to display the guid list */ + opt.guid = INVALID_GUID; + } else + printf(" Guid <0x%" PRIx64 ">\n", + cl_hton64(opt.guid)); + break; + + case 's': + val = strtol(optarg, NULL, 0); + /* Check that the number is not too large */ + if (((uint32_t) (val * 1000000)) / 1000000 != val) + fprintf(stderr, + "ERROR: sweep interval given is too large. Ignoring it.\n"); + else { + opt.sweep_interval = val; + printf(" sweep interval = %d\n", + opt.sweep_interval); + } + break; + + case 't': + opt.transaction_timeout = strtol(optarg, NULL, 0); + printf(" Transaction timeout = %d\n", + opt.transaction_timeout); + break; + + case 'n': + opt.max_wire_smps = strtol(optarg, NULL, 0); + if (opt.max_wire_smps <= 0) + opt.max_wire_smps = 0x7FFFFFFF; + printf(" Max wire smp's = %d\n", opt.max_wire_smps); + break; + + case 'q': + /* + * OpenSM interactive console + */ + if (strcmp(optarg, "off") == 0) { + opt.console = "off"; + } else if (strcmp(optarg, "local") == 0) { + opt.console = "local"; #ifdef ENABLE_OSM_CONSOLE_SOCKET - } else if (strcmp(optarg, "socket") == 0) { - opt.console = "socket"; + } else if (strcmp(optarg, "socket") == 0) { + opt.console = "socket"; #endif - } else { - printf("-console %s option not understood\n", optarg); - } - break; + } else { + printf("-console %s option not understood\n", + optarg); + } + break; #ifdef ENABLE_OSM_CONSOLE_SOCKET - case 'C': - opt.console_port = strtol(optarg, NULL, 0); - break; + case 'C': + opt.console_port = strtol(optarg, NULL, 0); + break; #endif - case 'd': - dbg_lvl = strtol(optarg, NULL, 0); - printf(" d level = 0x%x\n", dbg_lvl); - if (dbg_lvl == 0) - { - printf(" Debug mode: Ignore Other SMs\n"); - opt.ignore_other_sm = TRUE; - } - else if(dbg_lvl == 1) - { - printf(" Debug mode: Forcing Single Thread\n"); - opt.single_thread = TRUE; - } - else if(dbg_lvl == 2) - { - printf(" Debug mode: Force Log Flush\n"); - opt.force_log_flush = TRUE; - } - else if(dbg_lvl == 3) - { - printf(" Debug mode: Disable multicast support\n"); - opt.disable_multicast = TRUE; - } - /* - * NOTE: Debug level 4 used to be used for memory tracking - * but this is now deprecated - */ - else if(dbg_lvl == 5) - { - vendor_debug++; - } - else if(dbg_lvl >= 10) - { - /* Please look at osm_subnet.h for list of testability modes. */ - opt.testability_mode = dbg_lvl - 9; - } - else - printf(" OpenSM: Unknown debug option %d ignored\n", dbg_lvl); - break; - - case 'l': - temp = strtol(optarg, NULL, 0); - if( temp > 7 ) - { - fprintf(stderr, "ERROR: LMC must be 7 or less."); - return( -1 ); - } - opt.lmc = (uint8_t)temp; - printf(" LMC = %d\n", temp); - break; - - case 'D': - opt.log_flags = strtol(optarg, NULL, 0); - printf(" verbose option -D = 0x%x\n", opt.log_flags); - break; - - case 'f': - opt.log_file = optarg; - break; - - case 'L': - opt.log_max_size = strtoul(optarg, NULL, 0) * (1024*1024); - printf(" Log file max size is %lu bytes\n", opt.log_max_size); - break; - - case 'e': - opt.accum_log_file = FALSE; - printf(" Creating new log file\n"); - break; - - case 'P': - opt.partition_config_file = optarg; - break; - - case 'N': - opt.no_partition_enforcement = TRUE; - break; - - case 'Q': - opt.no_qos = FALSE; - break; - - case 'y': - opt.exit_on_fatal = FALSE; - printf(" Staying on fatal initialization errors\n"); - break; - - case 'v': - opt.log_flags = (opt.log_flags <<1 )|1; - printf(" Verbose option -v (log flags = 0x%X)\n", opt.log_flags ); - break; - - case 'V': - opt.log_flags = 0xFF; - opt.force_log_flush = TRUE; - printf(" Big V selected\n"); - break; - - case 'p': - temp = strtol(optarg, NULL, 0); - if (0 > temp || 15 < temp) { - fprintf(stderr, "ERROR: priority must be between 0 and 15\n"); - return (-1); - } - opt.sm_priority = (uint8_t)temp; - printf(" Priority = %d\n", temp); - break; - - case 'k': - sm_key = cl_hton64( strtoull( optarg, NULL, 16 )); - printf(" SM Key <0x%"PRIx64">\n", cl_hton64( sm_key )); - opt.sm_key = sm_key; - break; - - case 'R': - opt.routing_engine_name = optarg; - printf(" Activate \'%s\' routing engine\n", optarg); - break; - - case 'z': - opt.connect_roots = TRUE; - printf(" Connect roots option is on\n"); - break; - - case 'M': - opt.lid_matrix_dump_file = optarg; - printf(" Lid matrix dump file is \'%s\'\n", optarg); - break; - - case 'U': - opt.ucast_dump_file = optarg; - printf(" Ucast dump file is \'%s\'\n", optarg); - break; - - case 'S': - opt.sa_db_file = optarg; - printf(" SA DB file is \'%s\'\n", optarg); - break; - - case 'a': - /* - Specifies root guids file - */ - opt.root_guid_file = optarg; - printf (" Root Guid File: %s\n", opt.root_guid_file ); - break; - - case 'u': - /* - Specifies compute node guids file - */ - opt.cn_guid_file = optarg; - printf (" Compute Node Guid File: %s\n", opt.cn_guid_file ); - break; - - case 'c': - cache_options = TRUE; - printf (" Caching command line options\n"); - break; - - case 'x': - opt.honor_guid2lid_file = TRUE; - printf (" Honor guid2lid file, if possible\n"); - break; - - case 'B': - opt.daemon = TRUE; - printf (" Daemon mode\n"); - break; - - case 'I': - opt.sm_inactive = TRUE; - printf(" SM started in inactive state\n"); - break; + case 'd': + dbg_lvl = strtol(optarg, NULL, 0); + printf(" d level = 0x%x\n", dbg_lvl); + if (dbg_lvl == 0) { + printf(" Debug mode: Ignore Other SMs\n"); + opt.ignore_other_sm = TRUE; + } else if (dbg_lvl == 1) { + printf(" Debug mode: Forcing Single Thread\n"); + opt.single_thread = TRUE; + } else if (dbg_lvl == 2) { + printf(" Debug mode: Force Log Flush\n"); + opt.force_log_flush = TRUE; + } else if (dbg_lvl == 3) { + printf + (" Debug mode: Disable multicast support\n"); + opt.disable_multicast = TRUE; + } + /* + * NOTE: Debug level 4 used to be used for memory tracking + * but this is now deprecated + */ + else if (dbg_lvl == 5) { + vendor_debug++; + } else if (dbg_lvl >= 10) { + /* Please look at osm_subnet.h for list of testability modes. */ + opt.testability_mode = dbg_lvl - 9; + } else + printf + (" OpenSM: Unknown debug option %d ignored\n", + dbg_lvl); + break; + + case 'l': + temp = strtol(optarg, NULL, 0); + if (temp > 7) { + fprintf(stderr, + "ERROR: LMC must be 7 or less."); + return (-1); + } + opt.lmc = (uint8_t) temp; + printf(" LMC = %d\n", temp); + break; + + case 'D': + opt.log_flags = strtol(optarg, NULL, 0); + printf(" verbose option -D = 0x%x\n", opt.log_flags); + break; + + case 'f': + opt.log_file = optarg; + break; + + case 'L': + opt.log_max_size = + strtoul(optarg, NULL, 0) * (1024 * 1024); + printf(" Log file max size is %lu bytes\n", + opt.log_max_size); + break; + + case 'e': + opt.accum_log_file = FALSE; + printf(" Creating new log file\n"); + break; + + case 'P': + opt.partition_config_file = optarg; + break; + + case 'N': + opt.no_partition_enforcement = TRUE; + break; + + case 'Q': + opt.no_qos = FALSE; + break; + + case 'y': + opt.exit_on_fatal = FALSE; + printf(" Staying on fatal initialization errors\n"); + break; + + case 'v': + opt.log_flags = (opt.log_flags << 1) | 1; + printf(" Verbose option -v (log flags = 0x%X)\n", + opt.log_flags); + break; + + case 'V': + opt.log_flags = 0xFF; + opt.force_log_flush = TRUE; + printf(" Big V selected\n"); + break; + + case 'p': + temp = strtol(optarg, NULL, 0); + if (0 > temp || 15 < temp) { + fprintf(stderr, + "ERROR: priority must be between 0 and 15\n"); + return (-1); + } + opt.sm_priority = (uint8_t) temp; + printf(" Priority = %d\n", temp); + break; + + case 'k': + sm_key = cl_hton64(strtoull(optarg, NULL, 16)); + printf(" SM Key <0x%" PRIx64 ">\n", cl_hton64(sm_key)); + opt.sm_key = sm_key; + break; + + case 'R': + opt.routing_engine_name = optarg; + printf(" Activate \'%s\' routing engine\n", optarg); + break; + + case 'z': + opt.connect_roots = TRUE; + printf(" Connect roots option is on\n"); + break; + + case 'M': + opt.lid_matrix_dump_file = optarg; + printf(" Lid matrix dump file is \'%s\'\n", optarg); + break; + + case 'U': + opt.ucast_dump_file = optarg; + printf(" Ucast dump file is \'%s\'\n", optarg); + break; + + case 'S': + opt.sa_db_file = optarg; + printf(" SA DB file is \'%s\'\n", optarg); + break; + + case 'a': + /* + Specifies root guids file + */ + opt.root_guid_file = optarg; + printf(" Root Guid File: %s\n", opt.root_guid_file); + break; + + case 'u': + /* + Specifies compute node guids file + */ + opt.cn_guid_file = optarg; + printf(" Compute Node Guid File: %s\n", + opt.cn_guid_file); + break; + + case 'c': + cache_options = TRUE; + printf(" Caching command line options\n"); + break; + + case 'x': + opt.honor_guid2lid_file = TRUE; + printf(" Honor guid2lid file, if possible\n"); + break; + + case 'B': + opt.daemon = TRUE; + printf(" Daemon mode\n"); + break; + + case 'I': + opt.sm_inactive = TRUE; + printf(" SM started in inactive state\n"); + break; #ifdef ENABLE_OSM_PERF_MGR - case 1: - opt.perfmgr = TRUE; - break; - case 2: - opt.perfmgr_sweep_time_s = atoi(optarg); - break; -#endif /* ENABLE_OSM_PERF_MGR */ - - case 'h': - case '?': - case ':': - show_usage(); - break; - - case -1: - break; /* done with option */ - default: /* something wrong */ - abort(); - } - } - while(next_option != -1); - - if (opt.log_file != NULL ) - printf(" Log File: %s\n", opt.log_file ); - /* Done with options description */ - printf("-------------------------------------------------\n"); - - if (vendor_debug) - osm_vendor_set_debug( osm.p_vendor, vendor_debug ); - - block_signals(); - - if (opt.daemon) - daemonize(&osm); - - complib_init(); - - status = osm_opensm_init( &osm, &opt ); - if( status != IB_SUCCESS ) - { - const char *err_str = ib_get_err_str( status ); - if (err_str == NULL) - { - err_str = "Unknown Error Type"; - } - printf( "\nError from osm_opensm_init: %s.\n", - err_str); - /* We will just exit, and not go to Exit, since we don't - want the destroy to be called. */ - complib_exit(); - return( status ); - } - - /* - If the user didn't specify a GUID on the command line, - then get a port GUID value with which to bind. - */ - if( opt.guid == 0 || cl_hton64(opt.guid) == CL_HTON64(INVALID_GUID)) - opt.guid = get_port_guid( &osm, opt.guid ); - - if ( opt.guid == 0 ) - { - printf( "Error: Could not get port guid\n" ); - status = IB_ERROR; - goto Exit; - } - - if ( cache_options == TRUE ) - { - if (osm_subn_write_conf_file( &opt ) != IB_SUCCESS) - { - printf( "\nosm_subn_write_conf_file failed!\n" ); - } - } - - status = osm_opensm_bind( &osm, opt.guid ); - if( status != IB_SUCCESS ) - { - printf( "\nError from osm_opensm_bind (0x%X)\n", status ); - printf( "Perhaps another instance of OpenSM is already running\n" ); - goto Exit; - } - - /* - * Define some port guids to ignore during path equalization - */ - if (ignore_guids_file_name != NULL) - { - status = parse_ignore_guids_file(ignore_guids_file_name, &osm); - if( status != IB_SUCCESS ) - { - printf( "\nError from parse_ignore_guids_file (0x%X)\n", status ); - goto Exit; - } - } - - setup_signals(); - - osm_opensm_sweep( &osm ); - - if( run_once_flag == TRUE ) - { - while (!osm_exit_flag) - { - status = osm_opensm_wait_for_subnet_up( - &osm, osm.subn.opt.sweep_interval * 1000000, TRUE ); - if (!status) - osm_exit_flag = 1; - } - } - else - { - osm_console_init(&opt, &osm); - - /* - Sit here forever - */ - while( !osm_exit_flag ) { - if (strcmp(opt.console, "local") == 0) - { - osm_console(&osm); + case 1: + opt.perfmgr = TRUE; + break; + case 2: + opt.perfmgr_sweep_time_s = atoi(optarg); + break; +#endif /* ENABLE_OSM_PERF_MGR */ + + case 'h': + case '?': + case ':': + show_usage(); + break; + + case -1: + break; /* done with option */ + default: /* something wrong */ + abort(); + } + } + while (next_option != -1); + + if (opt.log_file != NULL) + printf(" Log File: %s\n", opt.log_file); + /* Done with options description */ + printf("-------------------------------------------------\n"); + + if (vendor_debug) + osm_vendor_set_debug(osm.p_vendor, vendor_debug); + + block_signals(); + + if (opt.daemon) + daemonize(&osm); + + complib_init(); + + status = osm_opensm_init(&osm, &opt); + if (status != IB_SUCCESS) { + const char *err_str = ib_get_err_str(status); + if (err_str == NULL) { + err_str = "Unknown Error Type"; + } + printf("\nError from osm_opensm_init: %s.\n", err_str); + /* We will just exit, and not go to Exit, since we don't + want the destroy to be called. */ + complib_exit(); + return (status); + } + + /* + If the user didn't specify a GUID on the command line, + then get a port GUID value with which to bind. + */ + if (opt.guid == 0 || cl_hton64(opt.guid) == CL_HTON64(INVALID_GUID)) + opt.guid = get_port_guid(&osm, opt.guid); + + if (opt.guid == 0) { + printf("Error: Could not get port guid\n"); + status = IB_ERROR; + goto Exit; + } + + if (cache_options == TRUE) { + if (osm_subn_write_conf_file(&opt) != IB_SUCCESS) { + printf("\nosm_subn_write_conf_file failed!\n"); + } + } + + status = osm_opensm_bind(&osm, opt.guid); + if (status != IB_SUCCESS) { + printf("\nError from osm_opensm_bind (0x%X)\n", status); + printf + ("Perhaps another instance of OpenSM is already running\n"); + goto Exit; + } + + /* + * Define some port guids to ignore during path equalization + */ + if (ignore_guids_file_name != NULL) { + status = parse_ignore_guids_file(ignore_guids_file_name, &osm); + if (status != IB_SUCCESS) { + printf("\nError from parse_ignore_guids_file (0x%X)\n", + status); + goto Exit; + } + } + + setup_signals(); + + osm_opensm_sweep(&osm); + + if (run_once_flag == TRUE) { + while (!osm_exit_flag) { + status = + osm_opensm_wait_for_subnet_up(&osm, + osm.subn.opt. + sweep_interval * + 1000000, TRUE); + if (!status) + osm_exit_flag = 1; + } + } else { + osm_console_init(&opt, &osm); + + /* + Sit here forever + */ + while (!osm_exit_flag) { + if (strcmp(opt.console, "local") == 0) { + osm_console(&osm); #ifdef ENABLE_OSM_CONSOLE_SOCKET - } - else if (strcmp(opt.console, "socket") == 0) - { - osm_console(&osm); + } else if (strcmp(opt.console, "socket") == 0) { + osm_console(&osm); #endif - } - else - cl_thread_suspend( 10000 ); - - if (osm_usr1_flag) { - osm_usr1_flag = 0; - osm_log_reopen_file(&osm.log); - } - if (osm_hup_flag) { - osm_hup_flag = 0; - /* a HUP signal should only start a new heavy sweep */ - osm.subn.force_immediate_heavy_sweep = TRUE; - osm_opensm_sweep( &osm ); - } - } - osm_console_close_socket(&osm); - } + } else + cl_thread_suspend(10000); + + if (osm_usr1_flag) { + osm_usr1_flag = 0; + osm_log_reopen_file(&osm.log); + } + if (osm_hup_flag) { + osm_hup_flag = 0; + /* a HUP signal should only start a new heavy sweep */ + osm.subn.force_immediate_heavy_sweep = TRUE; + osm_opensm_sweep(&osm); + } + } + osm_console_close_socket(&osm); + } #if 0 - /* wait for all transactions to end */ - CL_ASSERT( ((opt.polling_retry_number + 1) * opt.transaction_timeout / 1000.0) < 0x100000000ULL ); - exitTimeout = - (uint32_t) ((opt.polling_retry_number + 1) * opt.transaction_timeout / 1000.0); - - if (exitTimeout < 3) exitTimeout = 3; - - printf( "\n------- OpenSM Exiting (in %u seconds) -------\n", - exitTimeout); - sleep(exitTimeout); + /* wait for all transactions to end */ + CL_ASSERT(((opt.polling_retry_number + + 1) * opt.transaction_timeout / 1000.0) < 0x100000000ULL); + exitTimeout = + (uint32_t) ((opt.polling_retry_number + + 1) * opt.transaction_timeout / 1000.0); + + if (exitTimeout < 3) + exitTimeout = 3; + + printf("\n------- OpenSM Exiting (in %u seconds) -------\n", + exitTimeout); + sleep(exitTimeout); #endif - if (osm.mad_pool.mads_out) - fprintf(stdout, - "There are still %u MADs out. Forcing the exit of the OpenSM application...\n", - osm.mad_pool.mads_out); + if (osm.mad_pool.mads_out) + fprintf(stdout, + "There are still %u MADs out. Forcing the exit of the OpenSM application...\n", + osm.mad_pool.mads_out); - Exit: - osm_opensm_destroy(&osm); - complib_exit(); + Exit: + osm_opensm_destroy(&osm); + complib_exit(); - exit( 0 ); + exit(0); } diff --git a/opensm/opensm/osm_console.c b/opensm/opensm/osm_console.c index 766779d..3301f45 100644 --- a/opensm/opensm/osm_console.c +++ b/opensm/opensm/osm_console.c @@ -33,7 +33,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #define _GNU_SOURCE /* for getline */ #include @@ -58,20 +58,18 @@ struct command { char *name; - void (*help_function)(FILE *out, int detail); - void (*parse_function)(char **p_last, osm_opensm_t *p_osm, FILE *out); + void (*help_function) (FILE * out, int detail); + void (*parse_function) (char **p_last, osm_opensm_t * p_osm, + FILE * out); }; struct { - int on; - int delay_s; + int on; + int delay_s; time_t previous; - void (*loop_function)(osm_opensm_t *p_osm, FILE *out); + void (*loop_function) (osm_opensm_t * p_osm, FILE * out); } loop_command = { - on : 0, - delay_s : 2, - loop_function : NULL -}; +on: 0, delay_s: 2, loop_function:NULL}; static const struct command console_cmds[]; @@ -80,7 +78,7 @@ static inline char *next_token(char **p_last) return strtok_r(NULL, " \t\n\r", p_last); } -static void help_command(FILE *out, int detail) +static void help_command(FILE * out, int detail) { int i; @@ -91,42 +89,51 @@ static void help_command(FILE *out, int detail) console_cmds[i].help_function(out, 0); } -static void help_quit(FILE *out, int detail) +static void help_quit(FILE * out, int detail) { fprintf(out, "quit (not valid in local mode; use ctl-c)\n"); } - -static void help_loglevel(FILE *out, int detail) +static void help_loglevel(FILE * out, int detail) { fprintf(out, "loglevel []\n"); if (detail) { fprintf(out, " log-level is OR'ed from the following\n"); - fprintf(out, " OSM_LOG_NONE 0x%02X\n", OSM_LOG_NONE); - fprintf(out, " OSM_LOG_ERROR 0x%02X\n", OSM_LOG_ERROR); - fprintf(out, " OSM_LOG_INFO 0x%02X\n", OSM_LOG_INFO); - fprintf(out, " OSM_LOG_VERBOSE 0x%02X\n", OSM_LOG_VERBOSE); - fprintf(out, " OSM_LOG_DEBUG 0x%02X\n", OSM_LOG_DEBUG); - fprintf(out, " OSM_LOG_FUNCS 0x%02X\n", OSM_LOG_FUNCS); - fprintf(out, " OSM_LOG_FRAMES 0x%02X\n", OSM_LOG_FRAMES); - fprintf(out, " OSM_LOG_ROUTING 0x%02X\n", OSM_LOG_ROUTING); - fprintf(out, " OSM_LOG_SYS 0x%02X\n", OSM_LOG_SYS); + fprintf(out, " OSM_LOG_NONE 0x%02X\n", + OSM_LOG_NONE); + fprintf(out, " OSM_LOG_ERROR 0x%02X\n", + OSM_LOG_ERROR); + fprintf(out, " OSM_LOG_INFO 0x%02X\n", + OSM_LOG_INFO); + fprintf(out, " OSM_LOG_VERBOSE 0x%02X\n", + OSM_LOG_VERBOSE); + fprintf(out, " OSM_LOG_DEBUG 0x%02X\n", + OSM_LOG_DEBUG); + fprintf(out, " OSM_LOG_FUNCS 0x%02X\n", + OSM_LOG_FUNCS); + fprintf(out, " OSM_LOG_FRAMES 0x%02X\n", + OSM_LOG_FRAMES); + fprintf(out, " OSM_LOG_ROUTING 0x%02X\n", + OSM_LOG_ROUTING); + fprintf(out, " OSM_LOG_SYS 0x%02X\n", + OSM_LOG_SYS); fprintf(out, "\n"); - fprintf(out, " OSM_LOG_DEFAULT_LEVEL 0x%02X\n", OSM_LOG_DEFAULT_LEVEL); + fprintf(out, " OSM_LOG_DEFAULT_LEVEL 0x%02X\n", + OSM_LOG_DEFAULT_LEVEL); } } -static void help_priority(FILE *out, int detail) +static void help_priority(FILE * out, int detail) { fprintf(out, "priority []\n"); } -static void help_resweep(FILE *out, int detail) +static void help_resweep(FILE * out, int detail) { fprintf(out, "resweep [heavy|light]\n"); } -static void help_status(FILE *out, int detail) +static void help_status(FILE * out, int detail) { fprintf(out, "status [loop]\n"); if (detail) { @@ -134,44 +141,51 @@ static void help_status(FILE *out, int detail) } } -static void help_logflush(FILE *out, int detail) +static void help_logflush(FILE * out, int detail) { fprintf(out, "logflush -- flush the opensm.log file\n"); } -static void help_querylid(FILE *out, int detail) +static void help_querylid(FILE * out, int detail) { fprintf(out, - "querylid lid -- print internal information about the lid specified\n"); + "querylid lid -- print internal information about the lid specified\n"); } -static void help_portstatus(FILE *out, int detail) +static void help_portstatus(FILE * out, int detail) { fprintf(out, "portstatus [ca|switch|router]\n"); if (detail) { fprintf(out, "summarize port status\n"); - fprintf(out, " [ca|switch|router] -- limit the results to the node type specified\n"); + fprintf(out, + " [ca|switch|router] -- limit the results to the node type specified\n"); } } #ifdef ENABLE_OSM_PERF_MGR -static void help_perfmgr(FILE *out, int detail) +static void help_perfmgr(FILE * out, int detail) { - fprintf(out, "perfmgr [enable|disable|clear_counters|dump_counters|sweep_time[seconds]]\n"); + fprintf(out, + "perfmgr [enable|disable|clear_counters|dump_counters|sweep_time[seconds]]\n"); if (detail) { - fprintf(out, "perfmgr -- print the performance manager state\n"); - fprintf(out, " [enable|disable] -- change the perfmgr state\n"); - fprintf(out, " [sweep_time] -- change the perfmgr sweep time (requires [seconds] option)\n"); - fprintf(out, " [clear_counters] -- clear the counters stored\n"); - fprintf(out, " [dump_counters [mach]] -- dump the counters (optionally in [mach]ine readable format)\n"); + fprintf(out, + "perfmgr -- print the performance manager state\n"); + fprintf(out, + " [enable|disable] -- change the perfmgr state\n"); + fprintf(out, + " [sweep_time] -- change the perfmgr sweep time (requires [seconds] option)\n"); + fprintf(out, + " [clear_counters] -- clear the counters stored\n"); + fprintf(out, + " [dump_counters [mach]] -- dump the counters (optionally in [mach]ine readable format)\n"); } } -#endif /* ENABLE_OSM_PERF_MGR */ +#endif /* ENABLE_OSM_PERF_MGR */ /* more help routines go here */ -static void help_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void help_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; int i, found = 0; @@ -194,14 +208,15 @@ static void help_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) } } -static void loglevel_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void loglevel_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; int level; p_cmd = next_token(p_last); if (!p_cmd) - fprintf(out, "Current log level is 0x%x\n", osm_log_get_level(&p_osm->log)); + fprintf(out, "Current log level is 0x%x\n", + osm_log_get_level(&p_osm->log)); else { /* Handle x, 0x, and decimal specification of log level */ if (!strncmp(p_cmd, "x", 1)) { @@ -222,21 +237,24 @@ static void loglevel_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) } } -static void priority_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void priority_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; int priority; p_cmd = next_token(p_last); if (!p_cmd) - fprintf(out, "Current sm-priority is %d\n", p_osm->subn.opt.sm_priority); + fprintf(out, "Current sm-priority is %d\n", + p_osm->subn.opt.sm_priority); else { priority = strtol(p_cmd, NULL, 0); if (0 > priority || 15 < priority) - fprintf(out, "Invalid sm-priority %d; must be between 0 and 15\n", priority); + fprintf(out, + "Invalid sm-priority %d; must be between 0 and 15\n", + priority); else { fprintf(out, "Setting sm-priority to %d\n", priority); - p_osm->subn.opt.sm_priority = (uint8_t)priority; + p_osm->subn.opt.sm_priority = (uint8_t) priority; /* Does the SM state machine need a kick now ? */ } } @@ -244,17 +262,16 @@ static void priority_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) static char *sm_state_str(int state) { - switch (state) - { + switch (state) { case IB_SMINFO_STATE_INIT: return ("Init"); - case IB_SMINFO_STATE_DISCOVERING: + case IB_SMINFO_STATE_DISCOVERING: return ("Discovering"); - case IB_SMINFO_STATE_STANDBY: + case IB_SMINFO_STATE_STANDBY: return ("Standby"); - case IB_SMINFO_STATE_NOTACTIVE: + case IB_SMINFO_STATE_NOTACTIVE: return ("Not Active"); - case IB_SMINFO_STATE_MASTER: + case IB_SMINFO_STATE_MASTER: return ("Master"); } return ("UNKNOWN"); @@ -262,8 +279,7 @@ static char *sm_state_str(int state) static char *sa_state_str(osm_sa_state_t state) { - switch (state) - { + switch (state) { case OSM_SA_STATE_INIT: return ("Init"); case OSM_SA_STATE_READY: @@ -274,91 +290,89 @@ static char *sa_state_str(osm_sa_state_t state) static char *sm_state_mgr_str(osm_sm_state_t state) { - switch (state) - { - case OSM_SM_STATE_NO_STATE: - return ("No State"); - case OSM_SM_STATE_INIT: - return ("Init"); - case OSM_SM_STATE_IDLE: - return ("Idle"); - case OSM_SM_STATE_SWEEP_LIGHT: - return ("Sweep Light"); - case OSM_SM_STATE_SWEEP_LIGHT_WAIT: - return ("Sweep Light Wait"); - case OSM_SM_STATE_SWEEP_HEAVY_SELF: - return ("Sweep Heavy Self"); - case OSM_SM_STATE_SWEEP_HEAVY_SUBNET: - return ("Sweep Heavy Subnet"); - case OSM_SM_STATE_SET_SM_UCAST_LID: - return ("Set SM UCAST LID"); - case OSM_SM_STATE_SET_SM_UCAST_LID_WAIT: - return ("Set SM UCAST LID Wait"); - case OSM_SM_STATE_SET_SM_UCAST_LID_DONE: - return ("Set SM UCAST LID Done"); - case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS: - return ("Set Subnet UCAST LIDS"); - case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_WAIT: - return ("Set Subnet UCAST LIDS Wait"); - case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_DONE: - return ("Set Subnet UCAST LIDS Done"); - case OSM_SM_STATE_SET_UCAST_TABLES: - return ("Set UCAST Tables"); - case OSM_SM_STATE_SET_UCAST_TABLES_WAIT: - return ("Set UCAST Tables Wait"); - case OSM_SM_STATE_SET_UCAST_TABLES_DONE: - return ("Set UCAST Tables Done"); - case OSM_SM_STATE_SET_MCAST_TABLES: - return ("Set MCAST Tables"); - case OSM_SM_STATE_SET_MCAST_TABLES_WAIT: - return ("Set MCAST Tables Wait"); - case OSM_SM_STATE_SET_MCAST_TABLES_DONE: - return ("Set MCAST Tables Done"); - case OSM_SM_STATE_SET_LINK_PORTS: - return ("Set Link Ports"); - case OSM_SM_STATE_SET_LINK_PORTS_WAIT: - return ("Set Link Ports Wait"); - case OSM_SM_STATE_SET_LINK_PORTS_DONE: - return ("Set Link Ports Done"); - case OSM_SM_STATE_SET_ARMED: - return ("Set Armed"); - case OSM_SM_STATE_SET_ARMED_WAIT: - return ("Set Armed Wait"); - case OSM_SM_STATE_SET_ARMED_DONE: - return ("Set Armed Done"); - case OSM_SM_STATE_SET_ACTIVE: - return ("Set Active"); - case OSM_SM_STATE_SET_ACTIVE_WAIT: - return ("Set Active Wait"); - case OSM_SM_STATE_LOST_NEGOTIATION: - return ("Lost Negotiation"); - case OSM_SM_STATE_STANDBY: - return ("Standby"); - case OSM_SM_STATE_SUBNET_UP: - return ("Subnet Up"); - case OSM_SM_STATE_PROCESS_REQUEST: - return ("Process Request"); - case OSM_SM_STATE_PROCESS_REQUEST_WAIT: - return ("Process Request Wait"); - case OSM_SM_STATE_PROCESS_REQUEST_DONE: - return ("Process Request Done"); - case OSM_SM_STATE_MASTER_OR_HIGHER_SM_DETECTED: - return ("Master or Higher SM Detected"); - case OSM_SM_STATE_SET_PKEY: - return ("Set PKey"); - case OSM_SM_STATE_SET_PKEY_WAIT: - return ("Set PKey Wait"); - case OSM_SM_STATE_SET_PKEY_DONE: - return ("Set PKey Done"); - default: - return ("Unknown State"); + switch (state) { + case OSM_SM_STATE_NO_STATE: + return ("No State"); + case OSM_SM_STATE_INIT: + return ("Init"); + case OSM_SM_STATE_IDLE: + return ("Idle"); + case OSM_SM_STATE_SWEEP_LIGHT: + return ("Sweep Light"); + case OSM_SM_STATE_SWEEP_LIGHT_WAIT: + return ("Sweep Light Wait"); + case OSM_SM_STATE_SWEEP_HEAVY_SELF: + return ("Sweep Heavy Self"); + case OSM_SM_STATE_SWEEP_HEAVY_SUBNET: + return ("Sweep Heavy Subnet"); + case OSM_SM_STATE_SET_SM_UCAST_LID: + return ("Set SM UCAST LID"); + case OSM_SM_STATE_SET_SM_UCAST_LID_WAIT: + return ("Set SM UCAST LID Wait"); + case OSM_SM_STATE_SET_SM_UCAST_LID_DONE: + return ("Set SM UCAST LID Done"); + case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS: + return ("Set Subnet UCAST LIDS"); + case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_WAIT: + return ("Set Subnet UCAST LIDS Wait"); + case OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_DONE: + return ("Set Subnet UCAST LIDS Done"); + case OSM_SM_STATE_SET_UCAST_TABLES: + return ("Set UCAST Tables"); + case OSM_SM_STATE_SET_UCAST_TABLES_WAIT: + return ("Set UCAST Tables Wait"); + case OSM_SM_STATE_SET_UCAST_TABLES_DONE: + return ("Set UCAST Tables Done"); + case OSM_SM_STATE_SET_MCAST_TABLES: + return ("Set MCAST Tables"); + case OSM_SM_STATE_SET_MCAST_TABLES_WAIT: + return ("Set MCAST Tables Wait"); + case OSM_SM_STATE_SET_MCAST_TABLES_DONE: + return ("Set MCAST Tables Done"); + case OSM_SM_STATE_SET_LINK_PORTS: + return ("Set Link Ports"); + case OSM_SM_STATE_SET_LINK_PORTS_WAIT: + return ("Set Link Ports Wait"); + case OSM_SM_STATE_SET_LINK_PORTS_DONE: + return ("Set Link Ports Done"); + case OSM_SM_STATE_SET_ARMED: + return ("Set Armed"); + case OSM_SM_STATE_SET_ARMED_WAIT: + return ("Set Armed Wait"); + case OSM_SM_STATE_SET_ARMED_DONE: + return ("Set Armed Done"); + case OSM_SM_STATE_SET_ACTIVE: + return ("Set Active"); + case OSM_SM_STATE_SET_ACTIVE_WAIT: + return ("Set Active Wait"); + case OSM_SM_STATE_LOST_NEGOTIATION: + return ("Lost Negotiation"); + case OSM_SM_STATE_STANDBY: + return ("Standby"); + case OSM_SM_STATE_SUBNET_UP: + return ("Subnet Up"); + case OSM_SM_STATE_PROCESS_REQUEST: + return ("Process Request"); + case OSM_SM_STATE_PROCESS_REQUEST_WAIT: + return ("Process Request Wait"); + case OSM_SM_STATE_PROCESS_REQUEST_DONE: + return ("Process Request Done"); + case OSM_SM_STATE_MASTER_OR_HIGHER_SM_DETECTED: + return ("Master or Higher SM Detected"); + case OSM_SM_STATE_SET_PKEY: + return ("Set PKey"); + case OSM_SM_STATE_SET_PKEY_WAIT: + return ("Set PKey Wait"); + case OSM_SM_STATE_SET_PKEY_DONE: + return ("Set PKey Done"); + default: + return ("Unknown State"); } } -static void print_status(osm_opensm_t *p_osm, FILE *out) +static void print_status(osm_opensm_t * p_osm, FILE * out) { - if (out) - { + if (out) { fprintf(out, " OpenSM Version : %s\n", OSM_VERSION); fprintf(out, " SM State/Mgr State : %s/%s\n", sm_state_str(p_osm->subn.sm_state), @@ -366,48 +380,44 @@ static void print_status(osm_opensm_t *p_osm, FILE *out) fprintf(out, " SA State : %s\n", sa_state_str(p_osm->sa.state)); fprintf(out, " Routing Engine : %s\n", - p_osm->routing_engine.name ? p_osm->routing_engine.name : "null (min-hop)"); + p_osm->routing_engine.name ? p_osm->routing_engine. + name : "null (min-hop)"); #ifdef ENABLE_OSM_PERF_MGR fprintf(out, "\n PerfMgr state/sweep state : %s/%s\n", osm_perfmgr_get_state_str(&(p_osm->perfmgr)), osm_perfmgr_get_sweep_state_str(&(p_osm->perfmgr))); #endif fprintf(out, "\n MAD stats\n" - " ---------\n" - " QP0 MADS outstanding : %d\n" - " QP0 MADS outstanding (on wire) : %d\n" - " QP0 MADS rcvd : %d\n" - " QP0 MADS sent : %d\n" - " QP0 unicasts sent : %d\n" - " SA MADS outstanding : %d\n" - " SA MADS rcvd : %d\n" - " SA MADS sent : %d\n" - , + " ---------\n" + " QP0 MADS outstanding : %d\n" + " QP0 MADS outstanding (on wire) : %d\n" + " QP0 MADS rcvd : %d\n" + " QP0 MADS sent : %d\n" + " QP0 unicasts sent : %d\n" + " SA MADS outstanding : %d\n" + " SA MADS rcvd : %d\n" + " SA MADS sent : %d\n", p_osm->stats.qp0_mads_outstanding, p_osm->stats.qp0_mads_outstanding_on_wire, p_osm->stats.qp0_mads_rcvd, p_osm->stats.qp0_mads_sent, p_osm->stats.qp0_unicasts_sent, p_osm->stats.sa_mads_outstanding, - p_osm->stats.sa_mads_rcvd, - p_osm->stats.sa_mads_sent - ); + p_osm->stats.sa_mads_rcvd, p_osm->stats.sa_mads_sent); fprintf(out, "\n Subnet flags\n" - " ------------\n" - " Ignore existing lfts : %d\n" - " Subnet Init errors : %d\n" - " In sweep hop 0 : %d\n" - " Moved to master state : %d\n" - " First time master sweep : %d\n" - " Coming out of standby : %d\n" - , + " ------------\n" + " Ignore existing lfts : %d\n" + " Subnet Init errors : %d\n" + " In sweep hop 0 : %d\n" + " Moved to master state : %d\n" + " First time master sweep : %d\n" + " Coming out of standby : %d\n", p_osm->subn.ignore_existing_lfts, p_osm->subn.subnet_initialization_error, p_osm->subn.in_sweep_hop_0, p_osm->subn.moved_to_master_state, p_osm->subn.first_time_master_sweep, - p_osm->subn.coming_out_of_standby - ); + p_osm->subn.coming_out_of_standby); fprintf(out, "\n"); } } @@ -422,7 +432,7 @@ static int loop_command_check_time(void) return (0); } -static void status_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void status_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; @@ -442,35 +452,34 @@ static void status_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) print_status(p_osm, out); } -static void resweep_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void resweep_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; p_cmd = next_token(p_last); if (!p_cmd || - (strcmp(p_cmd, "heavy") != 0 && - strcmp(p_cmd, "light") != 0)) { + (strcmp(p_cmd, "heavy") != 0 && strcmp(p_cmd, "light") != 0)) { fprintf(out, "Invalid resweep command\n"); help_resweep(out, 1); } else { if (strcmp(p_cmd, "heavy") == 0) { - p_osm->subn.force_immediate_heavy_sweep = TRUE; + p_osm->subn.force_immediate_heavy_sweep = TRUE; } - osm_opensm_sweep( p_osm ); + osm_opensm_sweep(p_osm); } } -static void logflush_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void logflush_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { fflush(p_osm->log.out_port); } -static void querylid_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void querylid_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { - int p = 0; - uint16_t lid = 0; + int p = 0; + uint16_t lid = 0; osm_port_t *p_port = NULL; - char *p_cmd = next_token(p_last); + char *p_cmd = next_token(p_last); if (!p_cmd) { fprintf(out, "no LID specified\n"); @@ -478,7 +487,7 @@ static void querylid_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) return; } - lid = (uint16_t)strtoul(p_cmd, NULL, 0); + lid = (uint16_t) strtoul(p_cmd, NULL, 0); cl_plock_acquire(&p_osm->lock); if (lid > cl_ptr_vector_get_capacity(&(p_osm->subn.port_lid_tbl))) goto invalid_lid; @@ -495,25 +504,24 @@ static void querylid_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) cl_ntoh64(p_port->guid), p_port->p_node->print_desc, ib_get_node_type_str(osm_node_get_type(p_port->p_node)), - p_port->p_node->node_info.num_ports - ); + p_port->p_node->node_info.num_ports); if (p_port->p_node->sw) p = 0; else p = 1; - for (/* see above */; p < p_port->p_node->physp_tbl_size; p++) { + for ( /* see above */ ; p < p_port->p_node->physp_tbl_size; p++) { fprintf(out, " Port %d health : %s\n", p, - p_port->p_node->physp_table[p].healthy ? "OK" : "ERROR" - ); + p_port->p_node->physp_table[p]. + healthy ? "OK" : "ERROR"); } cl_plock_release(&p_osm->lock); return; -invalid_lid: + invalid_lid: cl_plock_release(&p_osm->lock); fprintf(out, "Invalid lid %d\n", lid); return; @@ -524,13 +532,13 @@ invalid_lid: */ typedef struct _port_report { struct _port_report *next; - uint64_t node_guid; - uint8_t port_num; - char print_desc[IB_NODE_DESCRIPTION_SIZE+1]; + uint64_t node_guid; + uint8_t port_num; + char print_desc[IB_NODE_DESCRIPTION_SIZE + 1]; } port_report_t; static void -__tag_port_report(port_report_t **head, uint64_t node_guid, +__tag_port_report(port_report_t ** head, uint64_t node_guid, uint8_t port_num, char *print_desc) { port_report_t *rep = malloc(sizeof(*rep)); @@ -539,7 +547,7 @@ __tag_port_report(port_report_t **head, uint64_t node_guid, rep->node_guid = node_guid; rep->port_num = port_num; - memcpy(rep->print_desc, print_desc, IB_NODE_DESCRIPTION_SIZE+1); + memcpy(rep->print_desc, print_desc, IB_NODE_DESCRIPTION_SIZE + 1); rep->next = NULL; if (*head) { rep->next = *head; @@ -548,12 +556,11 @@ __tag_port_report(port_report_t **head, uint64_t node_guid, *head = rep; } -static void -__print_port_report(FILE *out, port_report_t *head) +static void __print_port_report(FILE * out, port_report_t * head) { - port_report_t *item = head; + port_report_t *item = head; while (item != NULL) { - fprintf(out, " 0x%016"PRIx64" %d (%s)\n", + fprintf(out, " 0x%016" PRIx64 " %d (%s)\n", item->node_guid, item->port_num, item->print_desc); port_report_t *next = item->next; free(item); @@ -562,42 +569,41 @@ __print_port_report(FILE *out, port_report_t *head) } typedef struct { - uint8_t node_type_lim; /* limit the results; 0 == ALL */ - uint64_t total_nodes; - uint64_t total_ports; - uint64_t ports_down; - uint64_t ports_active; - uint64_t ports_disabled; + uint8_t node_type_lim; /* limit the results; 0 == ALL */ + uint64_t total_nodes; + uint64_t total_ports; + uint64_t ports_down; + uint64_t ports_active; + uint64_t ports_disabled; port_report_t *disabled_ports; - uint64_t ports_1X; - uint64_t ports_4X; - uint64_t ports_8X; - uint64_t ports_12X; - uint64_t ports_unknown_width; - uint64_t ports_reduced_width; + uint64_t ports_1X; + uint64_t ports_4X; + uint64_t ports_8X; + uint64_t ports_12X; + uint64_t ports_unknown_width; + uint64_t ports_reduced_width; port_report_t *reduced_width_ports; - uint64_t ports_sdr; - uint64_t ports_ddr; - uint64_t ports_qdr; - uint64_t ports_unknown_speed; - uint64_t ports_reduced_speed; + uint64_t ports_sdr; + uint64_t ports_ddr; + uint64_t ports_qdr; + uint64_t ports_unknown_speed; + uint64_t ports_reduced_speed; port_report_t *reduced_speed_ports; } fabric_stats_t; /** * iterator function to get portstatus on each node */ -static void -__get_stats(cl_map_item_t * const p_map_item, void *context) +static void __get_stats(cl_map_item_t * const p_map_item, void *context) { - fabric_stats_t *fs = (fabric_stats_t *)context; - osm_node_t *node = (osm_node_t *)p_map_item; - uint8_t num_ports = osm_node_get_num_physp(node); - uint8_t port = 0; + fabric_stats_t *fs = (fabric_stats_t *) context; + osm_node_t *node = (osm_node_t *) p_map_item; + uint8_t num_ports = osm_node_get_num_physp(node); + uint8_t port = 0; /* Skip nodes we are not interested in */ if (fs->node_type_lim != 0 - && fs->node_type_lim != node->node_info.node_type) + && fs->node_type_lim != node->node_info.node_type) return; fs->total_nodes++; @@ -618,49 +624,47 @@ __get_stats(cl_map_item_t * const p_map_item, void *context) if ((enabled_width ^ active_width) > active_width) { __tag_port_report(&(fs->reduced_width_ports), cl_ntoh64(node->node_info.node_guid), - port, - node->print_desc); + port, node->print_desc); fs->ports_reduced_width++; } if ((enabled_speed ^ active_speed) > active_speed) { __tag_port_report(&(fs->reduced_speed_ports), cl_ntoh64(node->node_info.node_guid), - port, - node->print_desc); + port, node->print_desc); fs->ports_reduced_speed++; } - switch(active_speed) { - case IB_LINK_SPEED_ACTIVE_2_5: - fs->ports_sdr++; - break; - case IB_LINK_SPEED_ACTIVE_5: - fs->ports_ddr++; - break; - case IB_LINK_SPEED_ACTIVE_10: - fs->ports_qdr++; - break; - default: - fs->ports_unknown_speed++; - break; + switch (active_speed) { + case IB_LINK_SPEED_ACTIVE_2_5: + fs->ports_sdr++; + break; + case IB_LINK_SPEED_ACTIVE_5: + fs->ports_ddr++; + break; + case IB_LINK_SPEED_ACTIVE_10: + fs->ports_qdr++; + break; + default: + fs->ports_unknown_speed++; + break; } - switch(active_width) { - case IB_LINK_WIDTH_ACTIVE_1X: - fs->ports_1X++; - break; - case IB_LINK_WIDTH_ACTIVE_4X: - fs->ports_4X++; - break; - case IB_LINK_WIDTH_ACTIVE_8X: - fs->ports_8X++; - break; - case IB_LINK_WIDTH_ACTIVE_12X: - fs->ports_12X++; - break; - default: - fs->ports_unknown_width++; - break; + switch (active_width) { + case IB_LINK_WIDTH_ACTIVE_1X: + fs->ports_1X++; + break; + case IB_LINK_WIDTH_ACTIVE_4X: + fs->ports_4X++; + break; + case IB_LINK_WIDTH_ACTIVE_8X: + fs->ports_8X++; + break; + case IB_LINK_WIDTH_ACTIVE_12X: + fs->ports_12X++; + break; + default: + fs->ports_unknown_width++; + break; } if (port_state == IB_LINK_DOWN) fs->ports_down++; @@ -669,8 +673,7 @@ __get_stats(cl_map_item_t * const p_map_item, void *context) if (port_phys_state == IB_PORT_PHYS_STATE_DISABLED) { __tag_port_report(&(fs->disabled_ports), cl_ntoh64(node->node_info.node_guid), - port, - node->print_desc); + port, node->print_desc); fs->ports_disabled++; } @@ -678,11 +681,11 @@ __get_stats(cl_map_item_t * const p_map_item, void *context) } } -static void portstatus_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void portstatus_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { - fabric_stats_t fs; - struct timeval before, after; - char *p_cmd; + fabric_stats_t fs; + struct timeval before, after; + char *p_cmd; memset(&fs, 0, sizeof(fs)); @@ -705,52 +708,56 @@ static void portstatus_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) /* for each node in the system gather the stats */ cl_plock_acquire(&p_osm->lock); - cl_qmap_apply_func(&(p_osm->subn.node_guid_tbl), __get_stats, (void *)&fs); + cl_qmap_apply_func(&(p_osm->subn.node_guid_tbl), __get_stats, + (void *)&fs); cl_plock_release(&p_osm->lock); gettimeofday(&after, NULL); /* report the stats */ fprintf(out, "\"%s\" port status:\n", - fs.node_type_lim ? ib_get_node_type_str(fs.node_type_lim) : "ALL"); - fprintf(out, " %"PRIu64" port(s) scanned on %"PRIu64" nodes in %lu us\n", - fs.total_ports, fs.total_nodes, after.tv_usec - before.tv_usec); + fs.node_type_lim ? ib_get_node_type_str(fs. + node_type_lim) : "ALL"); + fprintf(out, + " %" PRIu64 " port(s) scanned on %" PRIu64 + " nodes in %lu us\n", fs.total_ports, fs.total_nodes, + after.tv_usec - before.tv_usec); if (fs.ports_down) - fprintf(out, " %"PRIu64" down\n", fs.ports_down); + fprintf(out, " %" PRIu64 " down\n", fs.ports_down); if (fs.ports_active) - fprintf(out, " %"PRIu64" active\n", fs.ports_active); + fprintf(out, " %" PRIu64 " active\n", fs.ports_active); if (fs.ports_1X) - fprintf(out, " %"PRIu64" at 1X\n", fs.ports_1X); + fprintf(out, " %" PRIu64 " at 1X\n", fs.ports_1X); if (fs.ports_4X) - fprintf(out, " %"PRIu64" at 4X\n", fs.ports_4X); + fprintf(out, " %" PRIu64 " at 4X\n", fs.ports_4X); if (fs.ports_8X) - fprintf(out, " %"PRIu64" at 8X\n", fs.ports_8X); + fprintf(out, " %" PRIu64 " at 8X\n", fs.ports_8X); if (fs.ports_12X) - fprintf(out, " %"PRIu64" at 12X\n", fs.ports_12X); + fprintf(out, " %" PRIu64 " at 12X\n", fs.ports_12X); if (fs.ports_sdr) - fprintf(out, " %"PRIu64" at 2.5 Gbps\n", fs.ports_sdr); + fprintf(out, " %" PRIu64 " at 2.5 Gbps\n", fs.ports_sdr); if (fs.ports_ddr) - fprintf(out, " %"PRIu64" at 5.0 Gbps\n", fs.ports_ddr); + fprintf(out, " %" PRIu64 " at 5.0 Gbps\n", fs.ports_ddr); if (fs.ports_qdr) - fprintf(out, " %"PRIu64" at 10.0 Gbps\n", fs.ports_qdr); + fprintf(out, " %" PRIu64 " at 10.0 Gbps\n", fs.ports_qdr); if (fs.ports_disabled + fs.ports_reduced_speed + fs.ports_reduced_width - > 0) { + > 0) { fprintf(out, "\nPossible issues:\n"); } if (fs.ports_disabled) { - fprintf(out, " %"PRIu64" disabled\n", fs.ports_disabled); + fprintf(out, " %" PRIu64 " disabled\n", fs.ports_disabled); __print_port_report(out, fs.disabled_ports); } if (fs.ports_reduced_speed) { - fprintf(out, " %"PRIu64" with reduced speed\n", + fprintf(out, " %" PRIu64 " with reduced speed\n", fs.ports_reduced_speed); __print_port_report(out, fs.reduced_speed_ports); } if (fs.ports_reduced_width) { - fprintf(out, " %"PRIu64" with reduced width\n", + fprintf(out, " %" PRIu64 " with reduced width\n", fs.ports_reduced_width); __print_port_report(out, fs.reduced_width_ports); } @@ -758,72 +765,59 @@ static void portstatus_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) } #ifdef ENABLE_OSM_PERF_MGR -static void perfmgr_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void perfmgr_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { char *p_cmd; p_cmd = next_token(p_last); - if (p_cmd) - { - if (strcmp(p_cmd, "enable") == 0) - { - osm_perfmgr_set_state(&(p_osm->perfmgr), PERFMGR_STATE_ENABLED); - } - else if (strcmp(p_cmd, "disable") == 0) - { - osm_perfmgr_set_state(&(p_osm->perfmgr), PERFMGR_STATE_DISABLE); - } - else if (strcmp(p_cmd, "clear_counters") == 0) - { - osm_perfmgr_clear_counters(&(p_osm->perfmgr)); - } - else if (strcmp(p_cmd, "dump_counters") == 0) - { - p_cmd = next_token(p_last); - if (p_cmd && (strcmp(p_cmd, "mach") == 0)) { - osm_perfmgr_dump_counters(&(p_osm->perfmgr), - PERFMGR_EVENT_DB_DUMP_MR); + if (p_cmd) { + if (strcmp(p_cmd, "enable") == 0) { + osm_perfmgr_set_state(&(p_osm->perfmgr), + PERFMGR_STATE_ENABLED); + } else if (strcmp(p_cmd, "disable") == 0) { + osm_perfmgr_set_state(&(p_osm->perfmgr), + PERFMGR_STATE_DISABLE); + } else if (strcmp(p_cmd, "clear_counters") == 0) { + osm_perfmgr_clear_counters(&(p_osm->perfmgr)); + } else if (strcmp(p_cmd, "dump_counters") == 0) { + p_cmd = next_token(p_last); + if (p_cmd && (strcmp(p_cmd, "mach") == 0)) { + osm_perfmgr_dump_counters(&(p_osm->perfmgr), + PERFMGR_EVENT_DB_DUMP_MR); + } else { + osm_perfmgr_dump_counters(&(p_osm->perfmgr), + PERFMGR_EVENT_DB_DUMP_HR); + } + } else if (strcmp(p_cmd, "sweep_time") == 0) { + p_cmd = next_token(p_last); + if (p_cmd) { + uint16_t time_s = atoi(p_cmd); + osm_perfmgr_set_sweep_time_s(&(p_osm->perfmgr), + time_s); + } else { + fprintf(out, + "sweep_time requires a time period (in seconds) to be specified\n"); + } } else { - osm_perfmgr_dump_counters(&(p_osm->perfmgr), - PERFMGR_EVENT_DB_DUMP_HR); + fprintf(out, "\"%s\" option not found\n", p_cmd); } - } - else if (strcmp(p_cmd, "sweep_time") == 0) - { - p_cmd = next_token(p_last); - if (p_cmd) - { - uint16_t time_s = atoi(p_cmd); - osm_perfmgr_set_sweep_time_s(&(p_osm->perfmgr), time_s); - } - else - { - fprintf(out, "sweep_time requires a time period (in seconds) to be specified\n"); - } - } - else - { - fprintf(out, "\"%s\" option not found\n", p_cmd); - } } else { fprintf(out, "Performance Manager status:\n" - "state : %s\n" - "sweep state : %s\n" - "sweep time : %us\n" - "outstanding queries/max : %d/%u\n" - , + "state : %s\n" + "sweep state : %s\n" + "sweep time : %us\n" + "outstanding queries/max : %d/%u\n", osm_perfmgr_get_state_str(&(p_osm->perfmgr)), osm_perfmgr_get_sweep_state_str(&(p_osm->perfmgr)), osm_perfmgr_get_sweep_time_s(&(p_osm->perfmgr)), p_osm->perfmgr.outstanding_queries, - p_osm->perfmgr.max_outstanding_queries - ); + p_osm->perfmgr.max_outstanding_queries); } } -#endif /* ENABLE_OSM_PERF_MGR */ +#endif /* ENABLE_OSM_PERF_MGR */ /* This is public to be able to close it on exit */ -void osm_console_close_socket(osm_opensm_t *p_osm) +void osm_console_close_socket(osm_opensm_t * p_osm) { if (p_osm->console.socket > 0) { close(p_osm->console.in_fd); @@ -834,32 +828,30 @@ void osm_console_close_socket(osm_opensm_t *p_osm) } } -static void quit_parse(char **p_last, osm_opensm_t *p_osm, FILE *out) +static void quit_parse(char **p_last, osm_opensm_t * p_osm, FILE * out) { osm_console_close_socket(p_osm); } - /* more parse routines go here */ -static const struct command console_cmds[] = -{ - { "help", &help_command, &help_parse}, - { "quit", &help_quit, &quit_parse}, - { "loglevel", &help_loglevel, &loglevel_parse}, - { "priority", &help_priority, &priority_parse}, - { "resweep", &help_resweep, &resweep_parse}, - { "status", &help_status, &status_parse}, - { "logflush", &help_logflush, &logflush_parse}, - { "querylid", &help_querylid, &querylid_parse}, - { "portstatus", &help_portstatus, &portstatus_parse}, +static const struct command console_cmds[] = { + {"help", &help_command, &help_parse}, + {"quit", &help_quit, &quit_parse}, + {"loglevel", &help_loglevel, &loglevel_parse}, + {"priority", &help_priority, &priority_parse}, + {"resweep", &help_resweep, &resweep_parse}, + {"status", &help_status, &status_parse}, + {"logflush", &help_logflush, &logflush_parse}, + {"querylid", &help_querylid, &querylid_parse}, + {"portstatus", &help_portstatus, &portstatus_parse}, #ifdef ENABLE_OSM_PERF_MGR - { "perfmgr", &help_perfmgr, &perfmgr_parse}, -#endif /* ENABLE_OSM_PERF_MGR */ - { NULL, NULL, NULL} /* end of array */ + {"perfmgr", &help_perfmgr, &perfmgr_parse}, +#endif /* ENABLE_OSM_PERF_MGR */ + {NULL, NULL, NULL} /* end of array */ }; -static void parse_cmd_line(char *line, osm_opensm_t *p_osm) +static void parse_cmd_line(char *line, osm_opensm_t * p_osm) { char *p_cmd, *p_last; int i, found = 0; @@ -874,7 +866,7 @@ static void parse_cmd_line(char *line, osm_opensm_t *p_osm) p_cmd = strtok_r(line, " \t\n\r", &p_last); if (p_cmd) { for (i = 0; console_cmds[i].name; i++) { - if (loop_command.on ) { + if (loop_command.on) { if (!strcmp(p_cmd, "q")) { loop_command.on = 0; } @@ -883,7 +875,8 @@ static void parse_cmd_line(char *line, osm_opensm_t *p_osm) } if (!strcmp(p_cmd, console_cmds[i].name)) { found = 1; - console_cmds[i].parse_function(&p_last, p_osm, out); + console_cmds[i].parse_function(&p_last, p_osm, + out); break; } } @@ -900,7 +893,7 @@ static void parse_cmd_line(char *line, osm_opensm_t *p_osm) } } -void osm_console_prompt(FILE *out) +void osm_console_prompt(FILE * out) { if (out) { fprintf(out, "OpenSM %s", OSM_COMMAND_PROMPT); @@ -908,11 +901,11 @@ void osm_console_prompt(FILE *out) } } -void osm_console_init(osm_subn_opt_t *opt, osm_opensm_t *p_osm) +void osm_console_init(osm_subn_opt_t * opt, osm_opensm_t * p_osm) { p_osm->console.socket = -1; /* set up the file descriptors for the console */ - if (strcmp(opt->console, "local") == 0) { + if (strcmp(opt->console, "local") == 0) { p_osm->console.in = stdin; p_osm->console.out = stdout; p_osm->console.in_fd = fileno(stdin); @@ -920,66 +913,64 @@ void osm_console_init(osm_subn_opt_t *opt, osm_opensm_t *p_osm) osm_console_prompt(p_osm->console.out); #ifdef ENABLE_OSM_CONSOLE_SOCKET - } else if (strcmp(opt->console, "socket") == 0) { - struct sockaddr_in sin; + } else if (strcmp(opt->console, "socket") == 0) { + struct sockaddr_in sin; int optval = 1; - if ((p_osm->console.socket = socket(AF_INET, SOCK_STREAM, 0)) < 0) - { + if ((p_osm->console.socket = + socket(AF_INET, SOCK_STREAM, 0)) < 0) { osm_log(&(p_osm->log), OSM_LOG_ERROR, "osm_console_init: ERR 4B01: Failed to open console socket: %s\n", strerror(errno)); return; } - setsockopt(p_osm->console.socket, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval)); + setsockopt(p_osm->console.socket, SOL_SOCKET, SO_REUSEADDR, + &optval, sizeof(optval)); sin.sin_family = AF_INET; sin.sin_port = htons(opt->console_port); sin.sin_addr.s_addr = htonl(INADDR_ANY); - if (bind(p_osm->console.socket, &sin, sizeof(sin)) < 0) - { + if (bind(p_osm->console.socket, &sin, sizeof(sin)) < 0) { osm_log(&(p_osm->log), OSM_LOG_ERROR, "osm_console_init: ERR 4B02: Failed to bind console socket: %s\n", strerror(errno)); return; } - if (listen(p_osm->console.socket, 1) < 0) - { + if (listen(p_osm->console.socket, 1) < 0) { osm_log(&(p_osm->log), OSM_LOG_ERROR, "osm_console_init: ERR 4B03: Failed to listen on socket: %s\n", strerror(errno)); return; } - signal(SIGPIPE, SIG_IGN); /* protect ourselves from closed pipes */ + signal(SIGPIPE, SIG_IGN); /* protect ourselves from closed pipes */ p_osm->console.in = NULL; p_osm->console.out = NULL; p_osm->console.in_fd = -1; p_osm->console.out_fd = -1; osm_log(&(p_osm->log), OSM_LOG_INFO, - "osm_console_init: Console listening on port %d\n", opt->console_port); + "osm_console_init: Console listening on port %d\n", + opt->console_port); #endif } } #ifdef ENABLE_OSM_CONSOLE_SOCKET -static void handle_osm_connection(osm_opensm_t *p_osm, int new_fd, - char *client_ip, char *client_hn) +static void handle_osm_connection(osm_opensm_t * p_osm, int new_fd, + char *client_ip, char *client_hn) { - char *p_line; - size_t len; - ssize_t n; + char *p_line; + size_t len; + ssize_t n; - if (p_osm->console.in_fd >= 0) - { + if (p_osm->console.in_fd >= 0) { FILE *file = fdopen(new_fd, "w+"); fprintf(file, "OpenSM Console connection already in use\n" - " kill other session (y/n)? "); + " kill other session (y/n)? "); fflush(file); p_line = NULL; n = getline(&p_line, &len, file); - if (n > 0 && (p_line[0] == 'y' || p_line[0] == 'Y')) - { + if (n > 0 && (p_line[0] == 'y' || p_line[0] == 'Y')) { osm_console_close_socket(p_osm); } else { close(new_fd); @@ -992,24 +983,25 @@ static void handle_osm_connection(osm_opensm_t *p_osm, int new_fd, p_osm->console.out = p_osm->console.in; osm_console_prompt(p_osm->console.out); osm_log(&(p_osm->log), OSM_LOG_INFO, - "osm_console_init: Console connection accepted: %s (%s)\n", - client_hn, client_ip); + "osm_console_init: Console connection accepted: %s (%s)\n", + client_hn, client_ip); } static int connection_ok(char *client_ip, char *client_hn) { - return (hosts_ctl(OSM_DAEMON_NAME, client_hn, client_ip, "STRING_UNKNOWN")); + return (hosts_ctl + (OSM_DAEMON_NAME, client_hn, client_ip, "STRING_UNKNOWN")); } #endif -void osm_console(osm_opensm_t *p_osm) +void osm_console(osm_opensm_t * p_osm) { - struct pollfd pollfd[2]; - char *p_line; - size_t len; - ssize_t n; + struct pollfd pollfd[2]; + char *p_line; + size_t len; + ssize_t n; struct pollfd *fds; - nfds_t nfds; + nfds_t nfds; pollfd[0].fd = p_osm->console.socket; pollfd[0].events = POLLIN; @@ -1023,8 +1015,7 @@ void osm_console(osm_opensm_t *p_osm) nfds = p_osm->console.socket < 0 || pollfd[1].fd < 0 ? 1 : 2; if (loop_command.on && loop_command_check_time() && - loop_command.loop_function) - { + loop_command.loop_function) { if (p_osm->console.out) { loop_command.loop_function(p_osm, p_osm->console.out); fflush(p_osm->console.out); @@ -1039,7 +1030,7 @@ void osm_console(osm_opensm_t *p_osm) #ifdef ENABLE_OSM_CONSOLE_SOCKET if (pollfd[0].revents & POLLIN) { int new_fd = 0; - struct sockaddr_in sin; + struct sockaddr_in sin; socklen_t len = sizeof(sin); char client_ip[64]; char client_hn[128]; @@ -1051,17 +1042,21 @@ void osm_console(osm_opensm_t *p_osm) p_osm->console.in_fd = -1; return; } - if (inet_ntop(AF_INET, &sin.sin_addr, client_ip, sizeof(client_ip)) == NULL) { + if (inet_ntop + (AF_INET, &sin.sin_addr, client_ip, + sizeof(client_ip)) == NULL) { snprintf(client_ip, 64, "STRING_UNKNOWN"); } - if ((hent = gethostbyaddr((const char *) &sin.sin_addr, - sizeof(struct in_addr), AF_INET)) == NULL) { + if ((hent = gethostbyaddr((const char *)&sin.sin_addr, + sizeof(struct in_addr), + AF_INET)) == NULL) { snprintf(client_hn, 128, "STRING_UNKNOWN"); } else { snprintf(client_hn, 128, "%s", hent->h_name); } if (connection_ok(client_ip, client_hn)) { - handle_osm_connection(p_osm, new_fd, client_ip, client_hn); + handle_osm_connection(p_osm, new_fd, client_ip, + client_hn); } else { osm_log(&(p_osm->log), OSM_LOG_ERROR, "osm_console: ERR 4B05: Console connection denied: %s (%s)\n", diff --git a/opensm/opensm/osm_db_files.c b/opensm/opensm/osm_db_files.c index 6bcbe80..535c4ba 100644 --- a/opensm/opensm/osm_db_files.c +++ b/opensm/opensm/osm_db_files.c @@ -42,7 +42,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -86,9 +86,9 @@ * SYNOPSIS */ typedef struct _osm_db_domain_imp { - char *file_name; - st_table *p_hash; - cl_spinlock_t lock; + char *file_name; + st_table *p_hash; + cl_spinlock_t lock; } osm_db_domain_imp_t; /* * FIELDS @@ -107,7 +107,7 @@ typedef struct _osm_db_domain_imp { * SYNOPSIS */ typedef struct _osm_db_imp { - char *db_dir_name; + char *db_dir_name; } osm_db_imp_t; /* * FIELDS @@ -121,688 +121,619 @@ typedef struct _osm_db_imp { /*************************************************************************** ***************************************************************************/ -void -osm_db_construct( - IN osm_db_t* const p_db ) +void osm_db_construct(IN osm_db_t * const p_db) { - memset(p_db, 0, sizeof(osm_db_t)); - cl_list_construct( &p_db->domains ); + memset(p_db, 0, sizeof(osm_db_t)); + cl_list_construct(&p_db->domains); } /*************************************************************************** ***************************************************************************/ -void -osm_db_domain_destroy( - IN osm_db_domain_t* const p_db_domain) +void osm_db_domain_destroy(IN osm_db_domain_t * const p_db_domain) { - osm_db_domain_imp_t *p_domain_imp; - p_domain_imp = (osm_db_domain_imp_t *)p_db_domain->p_domain_imp; + osm_db_domain_imp_t *p_domain_imp; + p_domain_imp = (osm_db_domain_imp_t *) p_db_domain->p_domain_imp; - osm_db_clear( p_db_domain ); + osm_db_clear(p_db_domain); - cl_spinlock_destroy( &p_domain_imp->lock ); + cl_spinlock_destroy(&p_domain_imp->lock); - st_free_table( p_domain_imp->p_hash ); - free( p_domain_imp->file_name ); - free( p_domain_imp ); + st_free_table(p_domain_imp->p_hash); + free(p_domain_imp->file_name); + free(p_domain_imp); } /*************************************************************************** ***************************************************************************/ -void -osm_db_destroy( - IN osm_db_t* const p_db ) +void osm_db_destroy(IN osm_db_t * const p_db) { - osm_db_domain_t *p_domain; - - while ((p_domain = cl_list_remove_head( &p_db->domains )) != NULL ) - { - osm_db_domain_destroy( p_domain ); - free( p_domain ); - } - cl_list_destroy( &p_db->domains ); - free( p_db->p_db_imp ); + osm_db_domain_t *p_domain; + + while ((p_domain = cl_list_remove_head(&p_db->domains)) != NULL) { + osm_db_domain_destroy(p_domain); + free(p_domain); + } + cl_list_destroy(&p_db->domains); + free(p_db->p_db_imp); } /*************************************************************************** ***************************************************************************/ -int -osm_db_init( - IN osm_db_t* const p_db, - IN osm_log_t *p_log ) +int osm_db_init(IN osm_db_t * const p_db, IN osm_log_t * p_log) { - osm_db_imp_t *p_db_imp; - struct stat dstat; + osm_db_imp_t *p_db_imp; + struct stat dstat; - OSM_LOG_ENTER( p_log, osm_db_init ); + OSM_LOG_ENTER(p_log, osm_db_init); - p_db_imp = (osm_db_imp_t *)malloc(sizeof(osm_db_imp_t)); - CL_ASSERT( p_db_imp != NULL); + p_db_imp = (osm_db_imp_t *) malloc(sizeof(osm_db_imp_t)); + CL_ASSERT(p_db_imp != NULL); - p_db_imp->db_dir_name = getenv("OSM_CACHE_DIR"); - if (!p_db_imp->db_dir_name || !(*p_db_imp->db_dir_name)) - p_db_imp->db_dir_name = OSM_DEFAULT_CACHE_DIR; + p_db_imp->db_dir_name = getenv("OSM_CACHE_DIR"); + if (!p_db_imp->db_dir_name || !(*p_db_imp->db_dir_name)) + p_db_imp->db_dir_name = OSM_DEFAULT_CACHE_DIR; - /* Create the directory if it doesn't exist */ - /* There is a difference in creating directory between windows and linux */ + /* Create the directory if it doesn't exist */ + /* There is a difference in creating directory between windows and linux */ #ifdef __WIN__ - /* Check if the directory exists. If not - create it. */ - CreateDirectory(p_db_imp->db_dir_name, NULL); -#else /* __WIN__ */ - /* make sure the directory exists */ - if (lstat(p_db_imp->db_dir_name, &dstat)) - { - if (mkdir(p_db_imp->db_dir_name, 0755)) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_init: ERR 6101: " - " Failed to create the db directory:%s\n", - p_db_imp->db_dir_name); - OSM_LOG_EXIT( p_log ); - return 1; - } - } + /* Check if the directory exists. If not - create it. */ + CreateDirectory(p_db_imp->db_dir_name, NULL); +#else /* __WIN__ */ + /* make sure the directory exists */ + if (lstat(p_db_imp->db_dir_name, &dstat)) { + if (mkdir(p_db_imp->db_dir_name, 0755)) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_init: ERR 6101: " + " Failed to create the db directory:%s\n", + p_db_imp->db_dir_name); + OSM_LOG_EXIT(p_log); + return 1; + } + } #endif - p_db->p_log = p_log; - p_db->p_db_imp = (void*)p_db_imp; + p_db->p_log = p_log; + p_db->p_db_imp = (void *)p_db_imp; - cl_list_init( &p_db->domains, 5 ); + cl_list_init(&p_db->domains, 5); - OSM_LOG_EXIT( p_log ); + OSM_LOG_EXIT(p_log); - return 0; + return 0; } /*************************************************************************** ***************************************************************************/ -osm_db_domain_t* -osm_db_domain_init( - IN osm_db_t* const p_db, - IN char *domain_name) +osm_db_domain_t *osm_db_domain_init(IN osm_db_t * const p_db, + IN char *domain_name) { - osm_db_domain_t *p_domain; - osm_db_domain_imp_t *p_domain_imp; - int dir_name_len; - osm_log_t *p_log = p_db->p_log; - FILE *p_file; - - OSM_LOG_ENTER( p_log, osm_db_domain_init ); - - /* allocate a new domain object */ - p_domain = (osm_db_domain_t *)malloc(sizeof(osm_db_domain_t)); - CL_ASSERT( p_domain != NULL ); - - p_domain_imp = - (osm_db_domain_imp_t *)malloc(sizeof(osm_db_domain_imp_t)); - CL_ASSERT( p_domain_imp != NULL ); - - dir_name_len = strlen(((osm_db_imp_t*)p_db->p_db_imp)->db_dir_name); - - /* set the domain file name */ - p_domain_imp->file_name = - (char *)malloc(sizeof(char)*(dir_name_len) + strlen(domain_name) + 2); - CL_ASSERT(p_domain_imp->file_name != NULL); - strcpy(p_domain_imp->file_name,((osm_db_imp_t*)p_db->p_db_imp)->db_dir_name); - strcat(p_domain_imp->file_name,domain_name); - - /* make sure the file exists - or exit if not writable */ - p_file = fopen(p_domain_imp->file_name, "a+"); - if (! p_file) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_domain_init: ERR 6102: " - " Failed to open the db file:%s\n", - p_domain_imp->file_name); - free(p_domain_imp); - free(p_domain); - p_domain = NULL; - goto Exit; - } - fclose(p_file); - - /* initialize the hash table object */ - p_domain_imp->p_hash = st_init_strtable(); - CL_ASSERT( p_domain_imp->p_hash != NULL ); - - p_domain->p_db = p_db; - cl_list_insert_tail( &p_db->domains, p_domain ); - p_domain->p_domain_imp = p_domain_imp; - cl_spinlock_construct( &p_domain_imp->lock ); - cl_spinlock_init( &p_domain_imp->lock ); - - Exit: - OSM_LOG_EXIT( p_log ); - return p_domain; + osm_db_domain_t *p_domain; + osm_db_domain_imp_t *p_domain_imp; + int dir_name_len; + osm_log_t *p_log = p_db->p_log; + FILE *p_file; + + OSM_LOG_ENTER(p_log, osm_db_domain_init); + + /* allocate a new domain object */ + p_domain = (osm_db_domain_t *) malloc(sizeof(osm_db_domain_t)); + CL_ASSERT(p_domain != NULL); + + p_domain_imp = + (osm_db_domain_imp_t *) malloc(sizeof(osm_db_domain_imp_t)); + CL_ASSERT(p_domain_imp != NULL); + + dir_name_len = strlen(((osm_db_imp_t *) p_db->p_db_imp)->db_dir_name); + + /* set the domain file name */ + p_domain_imp->file_name = + (char *)malloc(sizeof(char) * (dir_name_len) + strlen(domain_name) + + 2); + CL_ASSERT(p_domain_imp->file_name != NULL); + strcpy(p_domain_imp->file_name, + ((osm_db_imp_t *) p_db->p_db_imp)->db_dir_name); + strcat(p_domain_imp->file_name, domain_name); + + /* make sure the file exists - or exit if not writable */ + p_file = fopen(p_domain_imp->file_name, "a+"); + if (!p_file) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_domain_init: ERR 6102: " + " Failed to open the db file:%s\n", + p_domain_imp->file_name); + free(p_domain_imp); + free(p_domain); + p_domain = NULL; + goto Exit; + } + fclose(p_file); + + /* initialize the hash table object */ + p_domain_imp->p_hash = st_init_strtable(); + CL_ASSERT(p_domain_imp->p_hash != NULL); + + p_domain->p_db = p_db; + cl_list_insert_tail(&p_db->domains, p_domain); + p_domain->p_domain_imp = p_domain_imp; + cl_spinlock_construct(&p_domain_imp->lock); + cl_spinlock_init(&p_domain_imp->lock); + + Exit: + OSM_LOG_EXIT(p_log); + return p_domain; } /*************************************************************************** ***************************************************************************/ -int -osm_db_restore( - IN osm_db_domain_t *p_domain) +int osm_db_restore(IN osm_db_domain_t * p_domain) { - osm_log_t *p_log = p_domain->p_db->p_log; - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; - FILE *p_file; - int status; - char sLine[OSM_DB_MAX_LINE_LEN]; - boolean_t before_key; - char *p_first_word, *p_rest_of_line, *p_last; - char *p_key = NULL; - char *p_prev_val, *p_accum_val = NULL; - char *endptr = NULL; - unsigned int line_num; - - OSM_LOG_ENTER( p_log, osm_db_restore ); - - /* take the lock on the domain */ - cl_spinlock_acquire( &p_domain_imp->lock ); - - /* open the file - read mode */ - p_file = fopen(p_domain_imp->file_name, "r"); - - if (! p_file) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 6103: " - " Failed to open the db file:%s\n", - p_domain_imp->file_name); - status = 1; - goto Exit; - } - - /* parse the file allocating new hash tables as required */ - /* - states: - before_key (0) -> in_key (1) - - before_key: if a word on the first byte - it is the key. state=in_key - the rest of the line is start of the value. - in_key: unless the line is empty - add it (with newlines) to the value. - if empty: state=before_key - */ - status = 0; - before_key = TRUE; - line_num = 0; - /* if we got to EOF in the middle of a key we add a last newline */ - while ( - (fgets(sLine, OSM_DB_MAX_LINE_LEN, p_file) != NULL) || - ((before_key == FALSE) && strcpy(sLine,"\n")) - ) - { - line_num++; - if (before_key) - { - if ((sLine[0] != ' ') && (sLine[0] != '\t') && (sLine[0] != '\n')) - { - /* we got a new key */ - before_key = FALSE; - - /* handle the key */ - p_first_word = strtok_r(sLine, " \t\n", &p_last); - if (! p_first_word) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 6104: " - " Failed to get key from line:%u : %s (file:%s)\n", - line_num, sLine, p_domain_imp->file_name); - status = 1; - goto EndParsing; - } - if (strlen(p_first_word) > OSM_DB_MAX_GUID_LEN) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 610A: " - " Illegal key from line:%u : %s (file:%s)\n", - line_num, sLine, p_domain_imp->file_name); - status = 1; - goto EndParsing; - } - - p_key = (char *)malloc(sizeof(char)*(strlen(p_first_word) + 1)); - strcpy(p_key, p_first_word); - - p_rest_of_line = strtok_r(NULL, "\n", &p_last); - if (p_rest_of_line != NULL) - { - p_accum_val = - (char*)malloc(sizeof(char)*(strlen(p_rest_of_line) + 1)); - strcpy(p_accum_val, p_rest_of_line); - } - else - { - p_accum_val = (char*)malloc(2); - strcpy(p_accum_val, "\0"); - } - } - else if (sLine[0] != '\n') - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 6105: " - " How did we get here? line:%u : %s (file:%s)\n", - line_num, sLine, p_domain_imp->file_name); - status = 1; - goto EndParsing; - } - } /* before key */ - else - { - /* we already have a key */ - - if (sLine[0] == '\n') - { - /* got an end of key */ - before_key = TRUE; - - /* make sure the key was not previously used */ - if (st_lookup(p_domain_imp->p_hash, - (st_data_t)p_key, - (st_data_t*)&p_prev_val)) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 6106: " - " Key:%s already exists in:%s with value:%s." - " Removing it\n", - p_key, - p_domain_imp->file_name, - p_prev_val); - } - else - { - p_prev_val = NULL; - } - - osm_log( p_log, OSM_LOG_DEBUG, - "osm_db_restore: " - "Got key:%s value:%s\n", p_key, p_accum_val); - - /* check that the key is a number */ - if (!strtouq(p_key, &endptr, 0) && *endptr != '\0') - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_restore: ERR 610B: " - "Key:%s is invalid\n", - p_key); - } - else - { - /* store our key and value */ - st_insert(p_domain_imp->p_hash, - (st_data_t)p_key, (st_data_t)p_accum_val); - } - } - else - { - /* accumulate into the value */ - p_prev_val = p_accum_val; - p_accum_val = - (char *)malloc(strlen(p_prev_val) + strlen(sLine) + 1); - strcpy(p_accum_val, p_prev_val); - free(p_prev_val); - strcat(p_accum_val, sLine); - } - } /* in key */ - } /* while lines or last line */ - - EndParsing: - fclose(p_file); - - Exit: - cl_spinlock_release( &p_domain_imp->lock ); - OSM_LOG_EXIT( p_log ); - return status; + osm_log_t *p_log = p_domain->p_db->p_log; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; + FILE *p_file; + int status; + char sLine[OSM_DB_MAX_LINE_LEN]; + boolean_t before_key; + char *p_first_word, *p_rest_of_line, *p_last; + char *p_key = NULL; + char *p_prev_val, *p_accum_val = NULL; + char *endptr = NULL; + unsigned int line_num; + + OSM_LOG_ENTER(p_log, osm_db_restore); + + /* take the lock on the domain */ + cl_spinlock_acquire(&p_domain_imp->lock); + + /* open the file - read mode */ + p_file = fopen(p_domain_imp->file_name, "r"); + + if (!p_file) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 6103: " + " Failed to open the db file:%s\n", + p_domain_imp->file_name); + status = 1; + goto Exit; + } + + /* parse the file allocating new hash tables as required */ + /* + states: + before_key (0) -> in_key (1) + + before_key: if a word on the first byte - it is the key. state=in_key + the rest of the line is start of the value. + in_key: unless the line is empty - add it (with newlines) to the value. + if empty: state=before_key + */ + status = 0; + before_key = TRUE; + line_num = 0; + /* if we got to EOF in the middle of a key we add a last newline */ + while ((fgets(sLine, OSM_DB_MAX_LINE_LEN, p_file) != NULL) || + ((before_key == FALSE) && strcpy(sLine, "\n")) + ) { + line_num++; + if (before_key) { + if ((sLine[0] != ' ') && (sLine[0] != '\t') + && (sLine[0] != '\n')) { + /* we got a new key */ + before_key = FALSE; + + /* handle the key */ + p_first_word = + strtok_r(sLine, " \t\n", &p_last); + if (!p_first_word) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 6104: " + " Failed to get key from line:%u : %s (file:%s)\n", + line_num, sLine, + p_domain_imp->file_name); + status = 1; + goto EndParsing; + } + if (strlen(p_first_word) > OSM_DB_MAX_GUID_LEN) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 610A: " + " Illegal key from line:%u : %s (file:%s)\n", + line_num, sLine, + p_domain_imp->file_name); + status = 1; + goto EndParsing; + } + + p_key = + (char *)malloc(sizeof(char) * + (strlen(p_first_word) + 1)); + strcpy(p_key, p_first_word); + + p_rest_of_line = strtok_r(NULL, "\n", &p_last); + if (p_rest_of_line != NULL) { + p_accum_val = + (char *)malloc(sizeof(char) * + (strlen + (p_rest_of_line) + + 1)); + strcpy(p_accum_val, p_rest_of_line); + } else { + p_accum_val = (char *)malloc(2); + strcpy(p_accum_val, "\0"); + } + } else if (sLine[0] != '\n') { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 6105: " + " How did we get here? line:%u : %s (file:%s)\n", + line_num, sLine, + p_domain_imp->file_name); + status = 1; + goto EndParsing; + } + } /* before key */ + else { + /* we already have a key */ + + if (sLine[0] == '\n') { + /* got an end of key */ + before_key = TRUE; + + /* make sure the key was not previously used */ + if (st_lookup(p_domain_imp->p_hash, + (st_data_t) p_key, + (st_data_t *) & p_prev_val)) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 6106: " + " Key:%s already exists in:%s with value:%s." + " Removing it\n", + p_key, + p_domain_imp->file_name, + p_prev_val); + } else { + p_prev_val = NULL; + } + + osm_log(p_log, OSM_LOG_DEBUG, + "osm_db_restore: " + "Got key:%s value:%s\n", p_key, + p_accum_val); + + /* check that the key is a number */ + if (!strtouq(p_key, &endptr, 0) + && *endptr != '\0') { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_restore: ERR 610B: " + "Key:%s is invalid\n", p_key); + } else { + /* store our key and value */ + st_insert(p_domain_imp->p_hash, + (st_data_t) p_key, + (st_data_t) p_accum_val); + } + } else { + /* accumulate into the value */ + p_prev_val = p_accum_val; + p_accum_val = + (char *)malloc(strlen(p_prev_val) + + strlen(sLine) + 1); + strcpy(p_accum_val, p_prev_val); + free(p_prev_val); + strcat(p_accum_val, sLine); + } + } /* in key */ + } /* while lines or last line */ + + EndParsing: + fclose(p_file); + + Exit: + cl_spinlock_release(&p_domain_imp->lock); + OSM_LOG_EXIT(p_log); + return status; } /*************************************************************************** ***************************************************************************/ -int -__osm_dump_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) +int __osm_dump_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) { - FILE *p_file = (FILE*)arg; - char *p_key = (char*)key; - char *p_val = (char *)val; + FILE *p_file = (FILE *) arg; + char *p_key = (char *)key; + char *p_val = (char *)val; - fprintf(p_file, "%s %s\n\n", p_key, p_val); - return ST_CONTINUE; + fprintf(p_file, "%s %s\n\n", p_key, p_val); + return ST_CONTINUE; } -int -osm_db_store( - IN osm_db_domain_t *p_domain) +int osm_db_store(IN osm_db_domain_t * p_domain) { - osm_log_t *p_log = p_domain->p_db->p_log; - osm_db_domain_imp_t *p_domain_imp; - FILE *p_file; - int status = 0; - char *p_tmp_file_name; - - OSM_LOG_ENTER( p_log, osm_db_store ); - - p_domain_imp = (osm_db_domain_imp_t *)p_domain->p_domain_imp; - p_tmp_file_name = - (char *)malloc(sizeof(char)*(strlen(p_domain_imp->file_name)+8)); - strcpy(p_tmp_file_name, p_domain_imp->file_name); - strcat(p_tmp_file_name,".tmp"); - - cl_spinlock_acquire( &p_domain_imp->lock ); - - /* open up the output file */ - p_file = fopen(p_tmp_file_name, "w"); - if (! p_file) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_store: ERR 6107: " - " Failed to open the db file:%s for writing\n", - p_domain_imp->file_name); - status = 1; - goto Exit; - } - - st_foreach(p_domain_imp->p_hash, __osm_dump_tbl_entry, (st_data_t)p_file); - fclose(p_file); - - /* move the domain file */ - status = remove(p_domain_imp->file_name); - if (status) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_store: ERR 6109: " - " Failed to remove file:%s (err:%u)\n", - p_domain_imp->file_name, status); - } - - status = rename(p_tmp_file_name, p_domain_imp->file_name); - if (status) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_store: ERR 6108: " - " Failed to rename the db file to:%s (err:%u)\n", - p_domain_imp->file_name, status); - } - Exit: - cl_spinlock_release( &p_domain_imp->lock ); - free(p_tmp_file_name); - OSM_LOG_EXIT( p_log ); - return status; + osm_log_t *p_log = p_domain->p_db->p_log; + osm_db_domain_imp_t *p_domain_imp; + FILE *p_file; + int status = 0; + char *p_tmp_file_name; + + OSM_LOG_ENTER(p_log, osm_db_store); + + p_domain_imp = (osm_db_domain_imp_t *) p_domain->p_domain_imp; + p_tmp_file_name = + (char *)malloc(sizeof(char) * + (strlen(p_domain_imp->file_name) + 8)); + strcpy(p_tmp_file_name, p_domain_imp->file_name); + strcat(p_tmp_file_name, ".tmp"); + + cl_spinlock_acquire(&p_domain_imp->lock); + + /* open up the output file */ + p_file = fopen(p_tmp_file_name, "w"); + if (!p_file) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_store: ERR 6107: " + " Failed to open the db file:%s for writing\n", + p_domain_imp->file_name); + status = 1; + goto Exit; + } + + st_foreach(p_domain_imp->p_hash, __osm_dump_tbl_entry, + (st_data_t) p_file); + fclose(p_file); + + /* move the domain file */ + status = remove(p_domain_imp->file_name); + if (status) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_store: ERR 6109: " + " Failed to remove file:%s (err:%u)\n", + p_domain_imp->file_name, status); + } + + status = rename(p_tmp_file_name, p_domain_imp->file_name); + if (status) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_store: ERR 6108: " + " Failed to rename the db file to:%s (err:%u)\n", + p_domain_imp->file_name, status); + } + Exit: + cl_spinlock_release(&p_domain_imp->lock); + free(p_tmp_file_name); + OSM_LOG_EXIT(p_log); + return status; } /*************************************************************************** ***************************************************************************/ /* simply de-allocate the key and the value and return the code that makes the st_foreach delete the entry */ -int -__osm_clear_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) +int __osm_clear_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) { - free((char*)key); - free((char*)val); - return ST_DELETE; + free((char *)key); + free((char *)val); + return ST_DELETE; } -int -osm_db_clear( - IN osm_db_domain_t *p_domain) +int osm_db_clear(IN osm_db_domain_t * p_domain) { - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; - cl_spinlock_acquire( &p_domain_imp->lock ); - st_foreach(p_domain_imp->p_hash, __osm_clear_tbl_entry, (st_data_t)NULL); - cl_spinlock_release( &p_domain_imp->lock ); + cl_spinlock_acquire(&p_domain_imp->lock); + st_foreach(p_domain_imp->p_hash, __osm_clear_tbl_entry, + (st_data_t) NULL); + cl_spinlock_release(&p_domain_imp->lock); - return 0; + return 0; } /*************************************************************************** ***************************************************************************/ -int -__osm_get_key_of_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) +int __osm_get_key_of_tbl_entry(st_data_t key, st_data_t val, st_data_t arg) { - cl_list_t *p_list = (cl_list_t *)arg; - cl_list_insert_tail(p_list, (void*)key); - return ST_CONTINUE; + cl_list_t *p_list = (cl_list_t *) arg; + cl_list_insert_tail(p_list, (void *)key); + return ST_CONTINUE; } -int -osm_db_keys( - IN osm_db_domain_t *p_domain, - OUT cl_list_t* p_key_list) +int osm_db_keys(IN osm_db_domain_t * p_domain, OUT cl_list_t * p_key_list) { - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; - cl_spinlock_acquire( &p_domain_imp->lock ); + cl_spinlock_acquire(&p_domain_imp->lock); - st_foreach(p_domain_imp->p_hash, - __osm_get_key_of_tbl_entry, (st_data_t)p_key_list); + st_foreach(p_domain_imp->p_hash, + __osm_get_key_of_tbl_entry, (st_data_t) p_key_list); - cl_spinlock_release( &p_domain_imp->lock ); + cl_spinlock_release(&p_domain_imp->lock); - return 0; + return 0; } /*************************************************************************** ***************************************************************************/ -char * -osm_db_lookup( - IN osm_db_domain_t *p_domain, - IN char *const p_key) +char *osm_db_lookup(IN osm_db_domain_t * p_domain, IN char *const p_key) { - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; - char *p_val = NULL; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; + char *p_val = NULL; - cl_spinlock_acquire( &p_domain_imp->lock ); + cl_spinlock_acquire(&p_domain_imp->lock); - if (!st_lookup(p_domain_imp->p_hash, (st_data_t)p_key, (st_data_t*)&p_val)) - p_val = NULL; + if (!st_lookup + (p_domain_imp->p_hash, (st_data_t) p_key, (st_data_t *) & p_val)) + p_val = NULL; - cl_spinlock_release( &p_domain_imp->lock ); + cl_spinlock_release(&p_domain_imp->lock); - return p_val; + return p_val; } /*************************************************************************** ***************************************************************************/ int -osm_db_update( - IN osm_db_domain_t *p_domain, - IN char *const p_key, - IN char *const p_val) +osm_db_update(IN osm_db_domain_t * p_domain, + IN char *const p_key, IN char *const p_val) { - osm_log_t *p_log = p_domain->p_db->p_log; - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; - char *p_prev_val = NULL; - char *p_new_key; - char *p_new_val; - - cl_spinlock_acquire( &p_domain_imp->lock ); - - if (st_lookup(p_domain_imp->p_hash, - (st_data_t)p_key, (st_data_t*)&p_prev_val)) - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_db_update: " - " Key:%s previously exists in:%s with value:%s\n", - p_key, - p_domain_imp->file_name, - p_prev_val); - p_new_key = p_key; - } - else - { - /* need to allocate the key */ - p_new_key = malloc(sizeof(char)*(strlen(p_key) + 1)); - strcpy(p_new_key, p_key); - } - - /* need to arange a new copy of the value */ - p_new_val = malloc(sizeof(char)*(strlen(p_val) + 1)); - strcpy(p_new_val, p_val); - - st_insert(p_domain_imp->p_hash, (st_data_t)p_new_key, (st_data_t)p_new_val); - - if (p_prev_val) - free(p_prev_val); - - cl_spinlock_release( &p_domain_imp->lock ); - - return 0; + osm_log_t *p_log = p_domain->p_db->p_log; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; + char *p_prev_val = NULL; + char *p_new_key; + char *p_new_val; + + cl_spinlock_acquire(&p_domain_imp->lock); + + if (st_lookup(p_domain_imp->p_hash, + (st_data_t) p_key, (st_data_t *) & p_prev_val)) { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_db_update: " + " Key:%s previously exists in:%s with value:%s\n", + p_key, p_domain_imp->file_name, p_prev_val); + p_new_key = p_key; + } else { + /* need to allocate the key */ + p_new_key = malloc(sizeof(char) * (strlen(p_key) + 1)); + strcpy(p_new_key, p_key); + } + + /* need to arange a new copy of the value */ + p_new_val = malloc(sizeof(char) * (strlen(p_val) + 1)); + strcpy(p_new_val, p_val); + + st_insert(p_domain_imp->p_hash, (st_data_t) p_new_key, + (st_data_t) p_new_val); + + if (p_prev_val) + free(p_prev_val); + + cl_spinlock_release(&p_domain_imp->lock); + + return 0; } /*************************************************************************** ***************************************************************************/ -int -osm_db_delete( - IN osm_db_domain_t *p_domain, - IN char *const p_key) +int osm_db_delete(IN osm_db_domain_t * p_domain, IN char *const p_key) { - osm_log_t *p_log = p_domain->p_db->p_log; - osm_db_domain_imp_t *p_domain_imp = - (osm_db_domain_imp_t *)p_domain->p_domain_imp; - char *p_prev_val = NULL; - int res; - - OSM_LOG_ENTER( p_log, osm_db_delete ); - - cl_spinlock_acquire( &p_domain_imp->lock ); - if (st_delete(p_domain_imp->p_hash, - (st_data_t*)&p_key, (st_data_t*)&p_prev_val)) - { - if (st_lookup(p_domain_imp->p_hash, - (st_data_t)p_key, (st_data_t*)&p_prev_val)) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_db_delete: " - " key:%s still exists in:%s with value:%s\n", - p_key, - p_domain_imp->file_name, - p_prev_val); - res = 1; - } - else - { - free(p_key); - free(p_prev_val); - res = 0; - } - } - else - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_db_update: " - " fail to find key:%s. delete failed\n", - p_key); - res = 1; - } - cl_spinlock_release( &p_domain_imp->lock ); - - OSM_LOG_EXIT( p_log ); - return res; + osm_log_t *p_log = p_domain->p_db->p_log; + osm_db_domain_imp_t *p_domain_imp = + (osm_db_domain_imp_t *) p_domain->p_domain_imp; + char *p_prev_val = NULL; + int res; + + OSM_LOG_ENTER(p_log, osm_db_delete); + + cl_spinlock_acquire(&p_domain_imp->lock); + if (st_delete(p_domain_imp->p_hash, + (st_data_t *) & p_key, (st_data_t *) & p_prev_val)) { + if (st_lookup(p_domain_imp->p_hash, + (st_data_t) p_key, (st_data_t *) & p_prev_val)) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_db_delete: " + " key:%s still exists in:%s with value:%s\n", + p_key, p_domain_imp->file_name, p_prev_val); + res = 1; + } else { + free(p_key); + free(p_prev_val); + res = 0; + } + } else { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_db_update: " + " fail to find key:%s. delete failed\n", p_key); + res = 1; + } + cl_spinlock_release(&p_domain_imp->lock); + + OSM_LOG_EXIT(p_log); + return res; } #ifdef TEST_OSMDB #include #include -int -main(int argc, char **argv) +int main(int argc, char **argv) { - osm_db_t db; - osm_log_t log; - osm_db_domain_t *p_dbd; - cl_list_t keys; - cl_list_iterator_t kI; - char *p_key; - char *p_val; - int i; - - cl_list_construct( &keys ); - cl_list_init( &keys, 10 ); - - osm_log_init_v2( &log, TRUE, 0xff, "/var/log/osm_db_test.log", 0, FALSE); - - osm_db_construct(&db); - if (osm_db_init(&db, &log)) - { - printf("db init failed\n"); - exit(1); - } - - p_dbd = osm_db_domain_init(&db, "lid_by_guid"); - - if (osm_db_restore(p_dbd)) - { - printf("failed to restore\n"); - } - - if (osm_db_keys(p_dbd, &keys)) - { - printf("failed to get keys\n"); - } - else - { - kI = cl_list_head( &keys ); - while (kI != cl_list_end( & keys )) - { - p_key = cl_list_obj(kI); - kI = cl_list_next( kI ); - - p_val = osm_db_lookup(p_dbd, p_key); - printf("key = %s val = %s\n", p_key, p_val); - } - } - - cl_list_remove_all(&keys); - - /* randomly add and remove numbers */ - for (i = 0; i < 10; i++) - { - int k; - float v; - int is_add; - char val_buf[16]; - char key_buf[16]; - - k = floor(1.0 * rand()/ RAND_MAX * 100); - v = rand(); - sprintf(key_buf, "%u", k); - sprintf(val_buf, "%u", v); - - is_add = (rand() < RAND_MAX/ 2); - - if (is_add) - { - osm_db_update(p_dbd, key_buf, val_buf); - } - else - { - osm_db_delete(p_dbd, key_buf); - } - } - if (osm_db_keys(p_dbd, &keys)) - { - printf("failed to get keys\n"); - } - else - { - kI = cl_list_head( &keys ); - while (kI != cl_list_end( & keys )) - { - p_key = cl_list_obj(kI); - kI = cl_list_next( kI ); - - p_val = osm_db_lookup(p_dbd, p_key); - printf("key = %s val = %s\n", p_key, p_val); - } - } - if (osm_db_store(p_dbd)) - printf("failed to store\n"); - - osm_db_destroy( &db ); - cl_list_destroy( &keys ); + osm_db_t db; + osm_log_t log; + osm_db_domain_t *p_dbd; + cl_list_t keys; + cl_list_iterator_t kI; + char *p_key; + char *p_val; + int i; + + cl_list_construct(&keys); + cl_list_init(&keys, 10); + + osm_log_init_v2(&log, TRUE, 0xff, "/var/log/osm_db_test.log", 0, FALSE); + + osm_db_construct(&db); + if (osm_db_init(&db, &log)) { + printf("db init failed\n"); + exit(1); + } + + p_dbd = osm_db_domain_init(&db, "lid_by_guid"); + + if (osm_db_restore(p_dbd)) { + printf("failed to restore\n"); + } + + if (osm_db_keys(p_dbd, &keys)) { + printf("failed to get keys\n"); + } else { + kI = cl_list_head(&keys); + while (kI != cl_list_end(&keys)) { + p_key = cl_list_obj(kI); + kI = cl_list_next(kI); + + p_val = osm_db_lookup(p_dbd, p_key); + printf("key = %s val = %s\n", p_key, p_val); + } + } + + cl_list_remove_all(&keys); + + /* randomly add and remove numbers */ + for (i = 0; i < 10; i++) { + int k; + float v; + int is_add; + char val_buf[16]; + char key_buf[16]; + + k = floor(1.0 * rand() / RAND_MAX * 100); + v = rand(); + sprintf(key_buf, "%u", k); + sprintf(val_buf, "%u", v); + + is_add = (rand() < RAND_MAX / 2); + + if (is_add) { + osm_db_update(p_dbd, key_buf, val_buf); + } else { + osm_db_delete(p_dbd, key_buf); + } + } + if (osm_db_keys(p_dbd, &keys)) { + printf("failed to get keys\n"); + } else { + kI = cl_list_head(&keys); + while (kI != cl_list_end(&keys)) { + p_key = cl_list_obj(kI); + kI = cl_list_next(kI); + + p_val = osm_db_lookup(p_dbd, p_key); + printf("key = %s val = %s\n", p_key, p_val); + } + } + if (osm_db_store(p_dbd)) + printf("failed to store\n"); + + osm_db_destroy(&db); + cl_list_destroy(&keys); } #endif diff --git a/opensm/opensm/osm_db_pack.c b/opensm/opensm/osm_db_pack.c index 23e56a6..de5550f 100644 --- a/opensm/opensm/osm_db_pack.c +++ b/opensm/opensm/osm_db_pack.c @@ -33,140 +33,129 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include #include -static inline void -__osm_pack_guid(uint64_t guid, char *p_guid_str) +static inline void __osm_pack_guid(uint64_t guid, char *p_guid_str) { - sprintf(p_guid_str, "0x%016" PRIx64, guid); + sprintf(p_guid_str, "0x%016" PRIx64, guid); } -static inline uint64_t -__osm_unpack_guid(char *p_guid_str) +static inline uint64_t __osm_unpack_guid(char *p_guid_str) { #if __WORDSIZE == 64 - return (strtoul(p_guid_str, NULL, 0)); + return (strtoul(p_guid_str, NULL, 0)); #else - return (strtoull(p_guid_str, NULL, 0)); + return (strtoull(p_guid_str, NULL, 0)); #endif } static inline void __osm_pack_lids(uint16_t min_lid, uint16_t max_lid, char *p_lid_str) { - sprintf(p_lid_str, "0x%04x 0x%04x", min_lid, max_lid); + sprintf(p_lid_str, "0x%04x 0x%04x", min_lid, max_lid); } static inline int -__osm_unpack_lids( - IN char *p_lid_str, - OUT uint16_t *p_min_lid, - OUT uint16_t *p_max_lid ) +__osm_unpack_lids(IN char *p_lid_str, + OUT uint16_t * p_min_lid, OUT uint16_t * p_max_lid) { - unsigned long tmp; - char *p_next; - char *p_num; - char lids_str[24]; - - strncpy(lids_str, p_lid_str, 23); - lids_str[23] = '\0'; - p_num = strtok_r(lids_str, " \t", &p_next); - if (! p_num) return 1; - tmp = strtoul(p_num, NULL, 0); - CL_ASSERT( tmp < 0x10000 ); - *p_min_lid = (uint16_t)tmp; - - p_num = strtok_r(NULL, " \t", &p_next); - if (! p_num) return 1; - tmp = strtoul(p_num, NULL, 0); - CL_ASSERT( tmp < 0x10000 ); - *p_max_lid = (uint16_t)tmp; - - return 0; + unsigned long tmp; + char *p_next; + char *p_num; + char lids_str[24]; + + strncpy(lids_str, p_lid_str, 23); + lids_str[23] = '\0'; + p_num = strtok_r(lids_str, " \t", &p_next); + if (!p_num) + return 1; + tmp = strtoul(p_num, NULL, 0); + CL_ASSERT(tmp < 0x10000); + *p_min_lid = (uint16_t) tmp; + + p_num = strtok_r(NULL, " \t", &p_next); + if (!p_num) + return 1; + tmp = strtoul(p_num, NULL, 0); + CL_ASSERT(tmp < 0x10000); + *p_max_lid = (uint16_t) tmp; + + return 0; } int -osm_db_guid2lid_guids( - IN osm_db_domain_t* const p_g2l, - OUT cl_qlist_t* p_guid_list ) +osm_db_guid2lid_guids(IN osm_db_domain_t * const p_g2l, + OUT cl_qlist_t * p_guid_list) { - char *p_key; - cl_list_t keys; - osm_db_guid_elem_t *p_guid_elem; + char *p_key; + cl_list_t keys; + osm_db_guid_elem_t *p_guid_elem; - cl_list_construct( &keys ); - cl_list_init( &keys, 10 ); + cl_list_construct(&keys); + cl_list_init(&keys, 10); - if (osm_db_keys(p_g2l, &keys)) - return 1; + if (osm_db_keys(p_g2l, &keys)) + return 1; - while ( (p_key = cl_list_remove_head( &keys )) != NULL ) - { - p_guid_elem = (osm_db_guid_elem_t*)malloc(sizeof(osm_db_guid_elem_t)); - CL_ASSERT( p_guid_elem != NULL ); + while ((p_key = cl_list_remove_head(&keys)) != NULL) { + p_guid_elem = + (osm_db_guid_elem_t *) malloc(sizeof(osm_db_guid_elem_t)); + CL_ASSERT(p_guid_elem != NULL); - p_guid_elem->guid = __osm_unpack_guid(p_key); - cl_qlist_insert_head(p_guid_list, &p_guid_elem->item); - } + p_guid_elem->guid = __osm_unpack_guid(p_key); + cl_qlist_insert_head(p_guid_list, &p_guid_elem->item); + } - cl_list_destroy( &keys ); - return 0; + cl_list_destroy(&keys); + return 0; } int -osm_db_guid2lid_get( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid, - OUT uint16_t *p_min_lid, - OUT uint16_t *p_max_lid) +osm_db_guid2lid_get(IN osm_db_domain_t * const p_g2l, + IN uint64_t guid, + OUT uint16_t * p_min_lid, OUT uint16_t * p_max_lid) { - char guid_str[20]; - char *p_lid_str; - uint16_t min_lid, max_lid; - - __osm_pack_guid(guid, guid_str); - p_lid_str = osm_db_lookup(p_g2l, guid_str); - if (! p_lid_str) - return 1; - if (__osm_unpack_lids(p_lid_str, &min_lid, &max_lid)) - return 1; - - if (p_min_lid) *p_min_lid = min_lid; - if (p_max_lid) *p_max_lid = max_lid; - - return 0; + char guid_str[20]; + char *p_lid_str; + uint16_t min_lid, max_lid; + + __osm_pack_guid(guid, guid_str); + p_lid_str = osm_db_lookup(p_g2l, guid_str); + if (!p_lid_str) + return 1; + if (__osm_unpack_lids(p_lid_str, &min_lid, &max_lid)) + return 1; + + if (p_min_lid) + *p_min_lid = min_lid; + if (p_max_lid) + *p_max_lid = max_lid; + + return 0; } int -osm_db_guid2lid_set( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid, - IN uint16_t min_lid, - IN uint16_t max_lid) +osm_db_guid2lid_set(IN osm_db_domain_t * const p_g2l, + IN uint64_t guid, IN uint16_t min_lid, IN uint16_t max_lid) { - char guid_str[20]; - char lid_str[16]; + char guid_str[20]; + char lid_str[16]; - __osm_pack_guid(guid, guid_str); - __osm_pack_lids(min_lid, max_lid, lid_str); + __osm_pack_guid(guid, guid_str); + __osm_pack_lids(min_lid, max_lid, lid_str); - return( osm_db_update( p_g2l, guid_str, lid_str) ); + return (osm_db_update(p_g2l, guid_str, lid_str)); } -int -osm_db_guid2lid_delete( - IN osm_db_domain_t* const p_g2l, - IN uint64_t guid ) +int osm_db_guid2lid_delete(IN osm_db_domain_t * const p_g2l, IN uint64_t guid) { - char guid_str[20]; - __osm_pack_guid(guid, guid_str); - return( osm_db_delete( p_g2l, guid_str) ); + char guid_str[20]; + __osm_pack_guid(guid, guid_str); + return (osm_db_delete(p_g2l, guid_str)); } - diff --git a/opensm/opensm/osm_drop_mgr.c b/opensm/opensm/osm_drop_mgr.c index cb9568f..7ace399 100644 --- a/opensm/opensm/osm_drop_mgr.c +++ b/opensm/opensm/osm_drop_mgr.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -68,563 +68,533 @@ /********************************************************************** **********************************************************************/ -void -osm_drop_mgr_construct( - IN osm_drop_mgr_t* const p_mgr ) +void osm_drop_mgr_construct(IN osm_drop_mgr_t * const p_mgr) { - CL_ASSERT( p_mgr ); - memset( p_mgr, 0, sizeof(*p_mgr) ); + CL_ASSERT(p_mgr); + memset(p_mgr, 0, sizeof(*p_mgr)); } /********************************************************************** **********************************************************************/ -void -osm_drop_mgr_destroy( - IN osm_drop_mgr_t* const p_mgr ) +void osm_drop_mgr_destroy(IN osm_drop_mgr_t * const p_mgr) { - CL_ASSERT( p_mgr ); + CL_ASSERT(p_mgr); - OSM_LOG_ENTER( p_mgr->p_log, osm_drop_mgr_destroy ); + OSM_LOG_ENTER(p_mgr->p_log, osm_drop_mgr_destroy); - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_drop_mgr_init( - IN osm_drop_mgr_t* const p_mgr, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_req_t* const p_req, - IN cl_plock_t* const p_lock ) +osm_drop_mgr_init(IN osm_drop_mgr_t * const p_mgr, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_req_t * const p_req, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_drop_mgr_init ); + OSM_LOG_ENTER(p_log, osm_drop_mgr_init); - osm_drop_mgr_construct( p_mgr ); + osm_drop_mgr_construct(p_mgr); - p_mgr->p_log = p_log; - p_mgr->p_subn = p_subn; - p_mgr->p_lock = p_lock; - p_mgr->p_req = p_req; + p_mgr->p_log = p_log; + p_mgr->p_subn = p_subn; + p_mgr->p_lock = p_lock; + p_mgr->p_req = p_req; - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_drop_mgr_remove_router( - IN const osm_drop_mgr_t* const p_mgr, - IN const ib_net64_t portguid ) +__osm_drop_mgr_remove_router(IN const osm_drop_mgr_t * const p_mgr, + IN const ib_net64_t portguid) { - osm_router_t *p_rtr; - cl_qmap_t* p_rtr_guid_tbl; - - p_rtr_guid_tbl = &p_mgr->p_subn->rtr_guid_tbl; - p_rtr = (osm_router_t*)cl_qmap_remove( p_rtr_guid_tbl, portguid ); - if( p_rtr != (osm_router_t*)cl_qmap_end( p_rtr_guid_tbl ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_remove_router: " - "Cleaned router for port guid 0x%016" PRIx64 "\n", - cl_ntoh64( portguid ) ); - osm_router_delete( &p_rtr ); - } + osm_router_t *p_rtr; + cl_qmap_t *p_rtr_guid_tbl; + + p_rtr_guid_tbl = &p_mgr->p_subn->rtr_guid_tbl; + p_rtr = (osm_router_t *) cl_qmap_remove(p_rtr_guid_tbl, portguid); + if (p_rtr != (osm_router_t *) cl_qmap_end(p_rtr_guid_tbl)) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_remove_router: " + "Cleaned router for port guid 0x%016" PRIx64 "\n", + cl_ntoh64(portguid)); + osm_router_delete(&p_rtr); + } } - /********************************************************************** **********************************************************************/ static void -drop_mgr_clean_physp( - IN const osm_drop_mgr_t* const p_mgr, - IN osm_physp_t *p_physp) +drop_mgr_clean_physp(IN const osm_drop_mgr_t * const p_mgr, + IN osm_physp_t * p_physp) { - osm_physp_t *p_remote_physp; - osm_port_t* p_remote_port; - - p_remote_physp = osm_physp_get_remote( p_physp ); - if( p_remote_physp && osm_physp_is_valid( p_remote_physp ) ) - { - p_remote_port = osm_get_port_by_guid( p_mgr->p_subn, - p_remote_physp->port_guid ); - - if ( p_remote_port ) - { - /* Let's check if this is a case of link that is lost (both ports - weren't recognized), or a "hiccup" in the subnet - in which case - the remote port was recognized, and its state is ACTIVE. - If this is just a "hiccup" - force a heavy sweep in the next sweep. - We don't want to lose that part of the subnet. */ - if (p_remote_port->discovery_count && - osm_physp_get_port_state( p_remote_physp ) == IB_LINK_ACTIVE ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "drop_mgr_clean_physp: " - "Forcing delayed heavy sweep. Remote " - "port 0x%016" PRIx64 " port num: 0x%X " - "was recognized in ACTIVE state\n", - cl_ntoh64( p_remote_physp->port_guid ), - p_remote_physp->port_num ); - p_mgr->p_subn->force_delayed_heavy_sweep = TRUE; - } - - /* If the remote node is ca or router - need to remove the remote port, - since it is no longer reachable. This can be done if we reset the - discovery count of the remote port. */ - if ( !p_remote_physp->p_node->sw ) - { - p_remote_port->discovery_count = 0; - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "drop_mgr_clean_physp: Resetting discovery count of node: " - "0x%016" PRIx64 " port num:0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_remote_physp->p_node ) ), - p_remote_physp->port_num ); - } - } - - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "drop_mgr_clean_physp: " - "Unlinking local node 0x%016" PRIx64 ", port 0x%X" - "\n\t\t\t\tand remote node 0x%016" PRIx64 ", port 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_physp->p_node ) ), - p_physp->port_num, - cl_ntoh64( osm_node_get_node_guid( p_remote_physp->p_node ) ), - p_remote_physp->port_num ); - - osm_physp_unlink( p_physp, p_remote_physp ); - - } - - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "drop_mgr_clean_physp: Clearing node 0x%016" PRIx64 " physical port number 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_physp->p_node ) ), - p_physp->port_num ); - - osm_physp_destroy( p_physp ); + osm_physp_t *p_remote_physp; + osm_port_t *p_remote_port; + + p_remote_physp = osm_physp_get_remote(p_physp); + if (p_remote_physp && osm_physp_is_valid(p_remote_physp)) { + p_remote_port = osm_get_port_by_guid(p_mgr->p_subn, + p_remote_physp->port_guid); + + if (p_remote_port) { + /* Let's check if this is a case of link that is lost (both ports + weren't recognized), or a "hiccup" in the subnet - in which case + the remote port was recognized, and its state is ACTIVE. + If this is just a "hiccup" - force a heavy sweep in the next sweep. + We don't want to lose that part of the subnet. */ + if (p_remote_port->discovery_count && + osm_physp_get_port_state(p_remote_physp) == + IB_LINK_ACTIVE) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "drop_mgr_clean_physp: " + "Forcing delayed heavy sweep. Remote " + "port 0x%016" PRIx64 " port num: 0x%X " + "was recognized in ACTIVE state\n", + cl_ntoh64(p_remote_physp->port_guid), + p_remote_physp->port_num); + p_mgr->p_subn->force_delayed_heavy_sweep = TRUE; + } + + /* If the remote node is ca or router - need to remove the remote port, + since it is no longer reachable. This can be done if we reset the + discovery count of the remote port. */ + if (!p_remote_physp->p_node->sw) { + p_remote_port->discovery_count = 0; + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "drop_mgr_clean_physp: Resetting discovery count of node: " + "0x%016" PRIx64 " port num:0x%X\n", + cl_ntoh64(osm_node_get_node_guid + (p_remote_physp->p_node)), + p_remote_physp->port_num); + } + } + + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "drop_mgr_clean_physp: " + "Unlinking local node 0x%016" PRIx64 ", port 0x%X" + "\n\t\t\t\tand remote node 0x%016" PRIx64 + ", port 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_physp->p_node)), + p_physp->port_num, + cl_ntoh64(osm_node_get_node_guid + (p_remote_physp->p_node)), + p_remote_physp->port_num); + + osm_physp_unlink(p_physp, p_remote_physp); + + } + + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "drop_mgr_clean_physp: Clearing node 0x%016" PRIx64 + " physical port number 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_physp->p_node)), + p_physp->port_num); + + osm_physp_destroy(p_physp); } /********************************************************************** **********************************************************************/ static void -__osm_drop_mgr_remove_port( - IN const osm_drop_mgr_t* const p_mgr, - IN osm_port_t* p_port ) +__osm_drop_mgr_remove_port(IN const osm_drop_mgr_t * const p_mgr, + IN osm_port_t * p_port) { - ib_net64_t port_guid; - osm_port_t *p_port_check; - cl_qmap_t* p_sm_guid_tbl; - osm_mcm_info_t* p_mcm; - osm_mgrp_t* p_mgrp; - cl_ptr_vector_t* p_port_lid_tbl; - uint16_t min_lid_ho; - uint16_t max_lid_ho; - uint16_t lid_ho; - osm_node_t *p_node; - osm_remote_sm_t *p_sm; - ib_gid_t port_gid; - ib_mad_notice_attr_t notice; - ib_api_status_t status; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_drop_mgr_remove_port ); - - port_guid = osm_port_get_guid( p_port ); - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_remove_port: " - "Unreachable port 0x%016" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - - p_port_check = (osm_port_t*)cl_qmap_remove( &p_mgr->p_subn->port_guid_tbl, - port_guid ); - if( p_port_check != p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_drop_mgr_remove_port: ERR 0101: " - "Port 0x%016" PRIx64 " not in guid table\n", - cl_ntoh64( port_guid ) ); - goto Exit; - } - - p_sm_guid_tbl = &p_mgr->p_subn->sm_guid_tbl; - p_sm = (osm_remote_sm_t*)cl_qmap_remove( p_sm_guid_tbl, port_guid ); - if( p_sm != (osm_remote_sm_t*)cl_qmap_end( p_sm_guid_tbl ) ) - { - /* need to remove this item */ - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_remove_port: " - "Cleaned SM for port guid\n" ); - - free(p_sm); - } - - __osm_drop_mgr_remove_router( p_mgr, port_guid ); - - osm_port_get_lid_range_ho( p_port, &min_lid_ho, &max_lid_ho ); - - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_remove_port: " - "Clearing abandoned LID range [0x%X,0x%X]\n", - min_lid_ho, max_lid_ho ); - - p_port_lid_tbl = &p_mgr->p_subn->port_lid_tbl; - for( lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++ ) - cl_ptr_vector_set( p_port_lid_tbl, lid_ho, NULL ); - - drop_mgr_clean_physp(p_mgr, p_port->p_physp); - - p_mcm = (osm_mcm_info_t*)cl_qlist_remove_head( &p_port->mcm_list ); - while( p_mcm != (osm_mcm_info_t *)cl_qlist_end( &p_port->mcm_list ) ) - { - p_mgrp = (osm_mgrp_t *)cl_qmap_get( &p_mgr->p_subn->mgrp_mlid_tbl, - p_mcm->mlid ); - if(p_mgrp != (osm_mgrp_t *)cl_qmap_end( &p_mgr->p_subn->mgrp_mlid_tbl ) ) - { - osm_mgrp_remove_port(p_mgr->p_subn, p_mgr->p_log, p_mgrp, p_port->guid ); - osm_mcm_info_delete( (osm_mcm_info_t*)p_mcm ); - } - p_mcm = (osm_mcm_info_t*)cl_qlist_remove_head( &p_port->mcm_list ); - } - - /* initialize the p_node - may need to get node_desc later */ - p_node = p_port->p_node; - - osm_port_delete( &p_port ); - - /* issue a notice - trap 65 */ - - /* details of the notice */ - notice.generic_type = 0x83; /* is generic subn mgt type */ - ib_notice_set_prod_type_ho(¬ice, 4); /* A class manager generator */ - /* endport ceases to be reachable */ - notice.g_or_v.generic.trap_num = CL_HTON16(65); - /* The sm_base_lid is saved in network order already. */ - notice.issuer_lid = p_mgr->p_subn->sm_base_lid; - /* following C14-72.1.2 and table 119 p725 */ - /* we need to provide the GID */ - port_gid.unicast.prefix = p_mgr->p_subn->opt.subnet_prefix; - port_gid.unicast.interface_id = port_guid; - memcpy(&(notice.data_details.ntc_64_67.gid), - &(port_gid), - sizeof(ib_gid_t)); - - /* According to page 653 - the issuer gid in this case of trap - is the SM gid, since the SM is the initiator of this trap. */ - notice.issuer_gid.unicast.prefix = p_mgr->p_subn->opt.subnet_prefix; - notice.issuer_gid.unicast.interface_id = p_mgr->p_subn->sm_port_guid; - - status = osm_report_notice(p_mgr->p_log, p_mgr->p_subn, ¬ice); - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_drop_mgr_remove_port: ERR 0103: " - "Error sending trap reports (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } - - if (osm_log_is_active( p_mgr->p_log, OSM_LOG_INFO )) - { - osm_log( p_mgr->p_log, OSM_LOG_INFO, - "__osm_drop_mgr_remove_port: " - "Removed port with GUID:0x%016" PRIx64 - " LID range [0x%X,0x%X] of node:%s\n", - cl_ntoh64( port_gid.unicast.interface_id ), - min_lid_ho, max_lid_ho, p_node ? p_node->print_desc : "UNKNOWN" ); - } - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); + ib_net64_t port_guid; + osm_port_t *p_port_check; + cl_qmap_t *p_sm_guid_tbl; + osm_mcm_info_t *p_mcm; + osm_mgrp_t *p_mgrp; + cl_ptr_vector_t *p_port_lid_tbl; + uint16_t min_lid_ho; + uint16_t max_lid_ho; + uint16_t lid_ho; + osm_node_t *p_node; + osm_remote_sm_t *p_sm; + ib_gid_t port_gid; + ib_mad_notice_attr_t notice; + ib_api_status_t status; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_drop_mgr_remove_port); + + port_guid = osm_port_get_guid(p_port); + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_remove_port: " + "Unreachable port 0x%016" PRIx64 "\n", cl_ntoh64(port_guid)); + + p_port_check = + (osm_port_t *) cl_qmap_remove(&p_mgr->p_subn->port_guid_tbl, + port_guid); + if (p_port_check != p_port) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_drop_mgr_remove_port: ERR 0101: " + "Port 0x%016" PRIx64 " not in guid table\n", + cl_ntoh64(port_guid)); + goto Exit; + } + + p_sm_guid_tbl = &p_mgr->p_subn->sm_guid_tbl; + p_sm = (osm_remote_sm_t *) cl_qmap_remove(p_sm_guid_tbl, port_guid); + if (p_sm != (osm_remote_sm_t *) cl_qmap_end(p_sm_guid_tbl)) { + /* need to remove this item */ + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_remove_port: " + "Cleaned SM for port guid\n"); + + free(p_sm); + } + + __osm_drop_mgr_remove_router(p_mgr, port_guid); + + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); + + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_remove_port: " + "Clearing abandoned LID range [0x%X,0x%X]\n", + min_lid_ho, max_lid_ho); + + p_port_lid_tbl = &p_mgr->p_subn->port_lid_tbl; + for (lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++) + cl_ptr_vector_set(p_port_lid_tbl, lid_ho, NULL); + + drop_mgr_clean_physp(p_mgr, p_port->p_physp); + + p_mcm = (osm_mcm_info_t *) cl_qlist_remove_head(&p_port->mcm_list); + while (p_mcm != (osm_mcm_info_t *) cl_qlist_end(&p_port->mcm_list)) { + p_mgrp = + (osm_mgrp_t *) cl_qmap_get(&p_mgr->p_subn->mgrp_mlid_tbl, + p_mcm->mlid); + if (p_mgrp != + (osm_mgrp_t *) cl_qmap_end(&p_mgr->p_subn->mgrp_mlid_tbl)) { + osm_mgrp_remove_port(p_mgr->p_subn, p_mgr->p_log, + p_mgrp, p_port->guid); + osm_mcm_info_delete((osm_mcm_info_t *) p_mcm); + } + p_mcm = + (osm_mcm_info_t *) cl_qlist_remove_head(&p_port->mcm_list); + } + + /* initialize the p_node - may need to get node_desc later */ + p_node = p_port->p_node; + + osm_port_delete(&p_port); + + /* issue a notice - trap 65 */ + + /* details of the notice */ + notice.generic_type = 0x83; /* is generic subn mgt type */ + ib_notice_set_prod_type_ho(¬ice, 4); /* A class manager generator */ + /* endport ceases to be reachable */ + notice.g_or_v.generic.trap_num = CL_HTON16(65); + /* The sm_base_lid is saved in network order already. */ + notice.issuer_lid = p_mgr->p_subn->sm_base_lid; + /* following C14-72.1.2 and table 119 p725 */ + /* we need to provide the GID */ + port_gid.unicast.prefix = p_mgr->p_subn->opt.subnet_prefix; + port_gid.unicast.interface_id = port_guid; + memcpy(&(notice.data_details.ntc_64_67.gid), + &(port_gid), sizeof(ib_gid_t)); + + /* According to page 653 - the issuer gid in this case of trap + is the SM gid, since the SM is the initiator of this trap. */ + notice.issuer_gid.unicast.prefix = p_mgr->p_subn->opt.subnet_prefix; + notice.issuer_gid.unicast.interface_id = p_mgr->p_subn->sm_port_guid; + + status = osm_report_notice(p_mgr->p_log, p_mgr->p_subn, ¬ice); + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_drop_mgr_remove_port: ERR 0103: " + "Error sending trap reports (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_INFO)) { + osm_log(p_mgr->p_log, OSM_LOG_INFO, + "__osm_drop_mgr_remove_port: " + "Removed port with GUID:0x%016" PRIx64 + " LID range [0x%X,0x%X] of node:%s\n", + cl_ntoh64(port_gid.unicast.interface_id), + min_lid_ho, max_lid_ho, + p_node ? p_node->print_desc : "UNKNOWN"); + } + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_drop_mgr_remove_switch( - IN const osm_drop_mgr_t* const p_mgr, - IN osm_node_t* p_node ) +__osm_drop_mgr_remove_switch(IN const osm_drop_mgr_t * const p_mgr, + IN osm_node_t * p_node) { - osm_switch_t *p_sw; - cl_qmap_t* p_sw_guid_tbl; - ib_net64_t node_guid; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_drop_mgr_remove_switch ); - - node_guid = osm_node_get_node_guid( p_node ); - p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; - - p_sw = (osm_switch_t*)cl_qmap_remove( p_sw_guid_tbl, node_guid ); - if( p_sw == (osm_switch_t*)cl_qmap_end( p_sw_guid_tbl ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_drop_mgr_remove_switch: ERR 0102: " - "Node 0x%016" PRIx64 " not in switch table\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - } - else - { - p_node->sw = NULL; - osm_switch_delete( &p_sw ); - } - - OSM_LOG_EXIT( p_mgr->p_log ); + osm_switch_t *p_sw; + cl_qmap_t *p_sw_guid_tbl; + ib_net64_t node_guid; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_drop_mgr_remove_switch); + + node_guid = osm_node_get_node_guid(p_node); + p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; + + p_sw = (osm_switch_t *) cl_qmap_remove(p_sw_guid_tbl, node_guid); + if (p_sw == (osm_switch_t *) cl_qmap_end(p_sw_guid_tbl)) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_drop_mgr_remove_switch: ERR 0102: " + "Node 0x%016" PRIx64 " not in switch table\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + } else { + p_node->sw = NULL; + osm_switch_delete(&p_sw); + } + + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static boolean_t -__osm_drop_mgr_process_node( - IN const osm_drop_mgr_t* const p_mgr, - IN osm_node_t* p_node ) +__osm_drop_mgr_process_node(IN const osm_drop_mgr_t * const p_mgr, + IN osm_node_t * p_node) { - osm_physp_t *p_physp; - osm_port_t *p_port; - osm_node_t *p_node_check; - uint32_t port_num; - uint32_t max_ports; - ib_net64_t port_guid; - boolean_t return_val = FALSE; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_drop_mgr_process_node ); - - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_process_node: " - "Unreachable node 0x%016" PRIx64 "\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - - /* - Delete all the logical and physical port objects - associated with this node. - */ - max_ports = osm_node_get_num_physp( p_node ); - for( port_num = 0; port_num < max_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - if( osm_physp_is_valid( p_physp ) ) - { - port_guid = osm_physp_get_port_guid( p_physp ); - - p_port = osm_get_port_by_guid( p_mgr->p_subn, port_guid ); - - if( p_port ) - __osm_drop_mgr_remove_port( p_mgr, p_port ); - else - drop_mgr_clean_physp( p_mgr, p_physp ); - } - } - - return_val = TRUE; - - if (p_node->sw) - __osm_drop_mgr_remove_switch( p_mgr, p_node ); - - p_node_check = (osm_node_t*)cl_qmap_remove( &p_mgr->p_subn->node_guid_tbl, - osm_node_get_node_guid( p_node ) ); - if( p_node_check != p_node ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_drop_mgr_process_node: ERR 0105: " - "Node 0x%016" PRIx64 " not in guid table\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - } - - /* free memory allocated to node */ - osm_node_delete( &p_node ); - - OSM_LOG_EXIT( p_mgr->p_log ); - return( return_val ); + osm_physp_t *p_physp; + osm_port_t *p_port; + osm_node_t *p_node_check; + uint32_t port_num; + uint32_t max_ports; + ib_net64_t port_guid; + boolean_t return_val = FALSE; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_drop_mgr_process_node); + + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_process_node: " + "Unreachable node 0x%016" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + + /* + Delete all the logical and physical port objects + associated with this node. + */ + max_ports = osm_node_get_num_physp(p_node); + for (port_num = 0; port_num < max_ports; port_num++) { + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (osm_physp_is_valid(p_physp)) { + port_guid = osm_physp_get_port_guid(p_physp); + + p_port = osm_get_port_by_guid(p_mgr->p_subn, port_guid); + + if (p_port) + __osm_drop_mgr_remove_port(p_mgr, p_port); + else + drop_mgr_clean_physp(p_mgr, p_physp); + } + } + + return_val = TRUE; + + if (p_node->sw) + __osm_drop_mgr_remove_switch(p_mgr, p_node); + + p_node_check = + (osm_node_t *) cl_qmap_remove(&p_mgr->p_subn->node_guid_tbl, + osm_node_get_node_guid(p_node)); + if (p_node_check != p_node) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_drop_mgr_process_node: ERR 0105: " + "Node 0x%016" PRIx64 " not in guid table\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + } + + /* free memory allocated to node */ + osm_node_delete(&p_node); + + OSM_LOG_EXIT(p_mgr->p_log); + return (return_val); } /********************************************************************** **********************************************************************/ static void -__osm_drop_mgr_check_node( - IN const osm_drop_mgr_t* const p_mgr, - IN osm_node_t* p_node ) +__osm_drop_mgr_check_node(IN const osm_drop_mgr_t * const p_mgr, + IN osm_node_t * p_node) { - ib_net64_t node_guid; - osm_physp_t *p_physp; - osm_port_t *p_port; - ib_net64_t port_guid; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_drop_mgr_check_node ); - - node_guid = osm_node_get_node_guid( p_node ); - - if ( osm_node_get_type( p_node ) != IB_NODE_TYPE_SWITCH ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_drop_mgr_check_node: ERR 0107: " - "Node 0x%016" PRIx64 " is not a switch node\n", - cl_ntoh64( node_guid ) ); - goto Exit; - } - - /* Make sure we have a switch object for this node */ - if (!p_node->sw) - { - /* We do not have switch info for this node */ - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_check_node: " - "Node 0x%016" PRIx64 " no switch in table\n", - cl_ntoh64( node_guid ) ); - - __osm_drop_mgr_process_node( p_mgr, p_node ); - goto Exit; - } - - /* Make sure we have a port object for port zero */ - p_physp = osm_node_get_physp_ptr( p_node, 0 ); - if ( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_check_node: " - "Node 0x%016" PRIx64 " no valid physical port 0\n", - cl_ntoh64( node_guid ) ); - - __osm_drop_mgr_process_node( p_mgr, p_node ); - goto Exit; - } - - port_guid = osm_physp_get_port_guid( p_physp ); - - p_port = osm_get_port_by_guid( p_mgr->p_subn, port_guid ); - - if( !p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_check_node: " - "Node 0x%016" PRIx64 " has no port object\n", - cl_ntoh64( node_guid ) ); - - __osm_drop_mgr_process_node( p_mgr, p_node ); - goto Exit; - } - - if ( p_port->discovery_count == 0 ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_drop_mgr_check_node: " - "Node 0x%016" PRIx64 " port has discovery count zero\n", - cl_ntoh64( node_guid ) ); - - __osm_drop_mgr_process_node( p_mgr, p_node ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - return; + ib_net64_t node_guid; + osm_physp_t *p_physp; + osm_port_t *p_port; + ib_net64_t port_guid; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_drop_mgr_check_node); + + node_guid = osm_node_get_node_guid(p_node); + + if (osm_node_get_type(p_node) != IB_NODE_TYPE_SWITCH) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_drop_mgr_check_node: ERR 0107: " + "Node 0x%016" PRIx64 " is not a switch node\n", + cl_ntoh64(node_guid)); + goto Exit; + } + + /* Make sure we have a switch object for this node */ + if (!p_node->sw) { + /* We do not have switch info for this node */ + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_check_node: " + "Node 0x%016" PRIx64 " no switch in table\n", + cl_ntoh64(node_guid)); + + __osm_drop_mgr_process_node(p_mgr, p_node); + goto Exit; + } + + /* Make sure we have a port object for port zero */ + p_physp = osm_node_get_physp_ptr(p_node, 0); + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_check_node: " + "Node 0x%016" PRIx64 " no valid physical port 0\n", + cl_ntoh64(node_guid)); + + __osm_drop_mgr_process_node(p_mgr, p_node); + goto Exit; + } + + port_guid = osm_physp_get_port_guid(p_physp); + + p_port = osm_get_port_by_guid(p_mgr->p_subn, port_guid); + + if (!p_port) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_check_node: " + "Node 0x%016" PRIx64 " has no port object\n", + cl_ntoh64(node_guid)); + + __osm_drop_mgr_process_node(p_mgr, p_node); + goto Exit; + } + + if (p_port->discovery_count == 0) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_drop_mgr_check_node: " + "Node 0x%016" PRIx64 " port has discovery count zero\n", + cl_ntoh64(node_guid)); + + __osm_drop_mgr_process_node(p_mgr, p_node); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + return; } /********************************************************************** **********************************************************************/ -void -osm_drop_mgr_process( - IN const osm_drop_mgr_t* const p_mgr ) +void osm_drop_mgr_process(IN const osm_drop_mgr_t * const p_mgr) { - cl_qmap_t *p_node_guid_tbl; - cl_qmap_t *p_port_guid_tbl; - osm_port_t *p_port; - osm_port_t *p_next_port; - osm_node_t *p_node; - osm_node_t *p_next_node; - ib_net64_t port_guid; - ib_net64_t node_guid; - - CL_ASSERT( p_mgr ); - - OSM_LOG_ENTER( p_mgr->p_log, osm_drop_mgr_process ); - - p_node_guid_tbl = &p_mgr->p_subn->node_guid_tbl; - p_port_guid_tbl = &p_mgr->p_subn->port_guid_tbl; - - CL_PLOCK_EXCL_ACQUIRE( p_mgr->p_lock ); - - p_next_node = (osm_node_t*)cl_qmap_head( p_node_guid_tbl ); - while( p_next_node != (osm_node_t*)cl_qmap_end( p_node_guid_tbl ) ) - { - p_node = p_next_node; - p_next_node = (osm_node_t*)cl_qmap_next( &p_next_node->map_item ); - - CL_ASSERT( cl_qmap_key( &p_node->map_item ) == - osm_node_get_node_guid( p_node ) ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - node_guid = osm_node_get_node_guid( p_node ); - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_drop_mgr_process: " - "Checking node 0x%016" PRIx64 "\n", - cl_ntoh64( node_guid ) ); - } - - /* - Check if this node was discovered during the last sweep. - If not, it is unreachable in the current subnet, and - should therefore be removed from the subnet object. - */ - if( p_node->discovery_count == 0 ) - __osm_drop_mgr_process_node( p_mgr, p_node ); - } - - /* - Go over all the nodes. If the node is a switch - make sure - there is also a switch record for it, and a portInfo record for - port zero of of the node. - If not - this means that there was some error in getting the data - of this node. Drop the node. - */ - p_next_node = (osm_node_t*)cl_qmap_head( p_node_guid_tbl ); - while( p_next_node != (osm_node_t*)cl_qmap_end( p_node_guid_tbl ) ) - { - p_node = p_next_node; - p_next_node = (osm_node_t*)cl_qmap_next( &p_next_node->map_item ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - node_guid = osm_node_get_node_guid( p_node ); - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_drop_mgr_process: " - "Checking full discovery of node 0x%016" PRIx64 "\n", - cl_ntoh64( node_guid ) ); - } - - if ( osm_node_get_type( p_node ) != IB_NODE_TYPE_SWITCH ) - continue; - - /* We are handling a switch node */ - __osm_drop_mgr_check_node( p_mgr, p_node ); - } - - p_next_port = (osm_port_t*)cl_qmap_head( p_port_guid_tbl ); - while( p_next_port != (osm_port_t*)cl_qmap_end( p_port_guid_tbl ) ) - { - p_port = p_next_port; - p_next_port = (osm_port_t*)cl_qmap_next( &p_next_port->map_item ); - - CL_ASSERT( cl_qmap_key( &p_port->map_item ) == - osm_port_get_guid( p_port ) ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - port_guid = osm_port_get_guid( p_port ); - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_drop_mgr_process: " - "Checking port 0x%016" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - } - - /* - If the port is unreachable, remove it from the guid table. - */ - if( p_port->discovery_count == 0 ) - __osm_drop_mgr_remove_port( p_mgr, p_port ); - } - - CL_PLOCK_RELEASE( p_mgr->p_lock ); - OSM_LOG_EXIT( p_mgr->p_log ); + cl_qmap_t *p_node_guid_tbl; + cl_qmap_t *p_port_guid_tbl; + osm_port_t *p_port; + osm_port_t *p_next_port; + osm_node_t *p_node; + osm_node_t *p_next_node; + ib_net64_t port_guid; + ib_net64_t node_guid; + + CL_ASSERT(p_mgr); + + OSM_LOG_ENTER(p_mgr->p_log, osm_drop_mgr_process); + + p_node_guid_tbl = &p_mgr->p_subn->node_guid_tbl; + p_port_guid_tbl = &p_mgr->p_subn->port_guid_tbl; + + CL_PLOCK_EXCL_ACQUIRE(p_mgr->p_lock); + + p_next_node = (osm_node_t *) cl_qmap_head(p_node_guid_tbl); + while (p_next_node != (osm_node_t *) cl_qmap_end(p_node_guid_tbl)) { + p_node = p_next_node; + p_next_node = + (osm_node_t *) cl_qmap_next(&p_next_node->map_item); + + CL_ASSERT(cl_qmap_key(&p_node->map_item) == + osm_node_get_node_guid(p_node)); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + node_guid = osm_node_get_node_guid(p_node); + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_drop_mgr_process: " + "Checking node 0x%016" PRIx64 "\n", + cl_ntoh64(node_guid)); + } + + /* + Check if this node was discovered during the last sweep. + If not, it is unreachable in the current subnet, and + should therefore be removed from the subnet object. + */ + if (p_node->discovery_count == 0) + __osm_drop_mgr_process_node(p_mgr, p_node); + } + + /* + Go over all the nodes. If the node is a switch - make sure + there is also a switch record for it, and a portInfo record for + port zero of of the node. + If not - this means that there was some error in getting the data + of this node. Drop the node. + */ + p_next_node = (osm_node_t *) cl_qmap_head(p_node_guid_tbl); + while (p_next_node != (osm_node_t *) cl_qmap_end(p_node_guid_tbl)) { + p_node = p_next_node; + p_next_node = + (osm_node_t *) cl_qmap_next(&p_next_node->map_item); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + node_guid = osm_node_get_node_guid(p_node); + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_drop_mgr_process: " + "Checking full discovery of node 0x%016" PRIx64 + "\n", cl_ntoh64(node_guid)); + } + + if (osm_node_get_type(p_node) != IB_NODE_TYPE_SWITCH) + continue; + + /* We are handling a switch node */ + __osm_drop_mgr_check_node(p_mgr, p_node); + } + + p_next_port = (osm_port_t *) cl_qmap_head(p_port_guid_tbl); + while (p_next_port != (osm_port_t *) cl_qmap_end(p_port_guid_tbl)) { + p_port = p_next_port; + p_next_port = + (osm_port_t *) cl_qmap_next(&p_next_port->map_item); + + CL_ASSERT(cl_qmap_key(&p_port->map_item) == + osm_port_get_guid(p_port)); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + port_guid = osm_port_get_guid(p_port); + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_drop_mgr_process: " + "Checking port 0x%016" PRIx64 "\n", + cl_ntoh64(port_guid)); + } + + /* + If the port is unreachable, remove it from the guid table. + */ + if (p_port->discovery_count == 0) + __osm_drop_mgr_remove_port(p_mgr, p_port); + } + + CL_PLOCK_RELEASE(p_mgr->p_lock); + OSM_LOG_EXIT(p_mgr->p_log); } diff --git a/opensm/opensm/osm_dump.c b/opensm/opensm/osm_dump.c index 367d941..7935c16 100644 --- a/opensm/opensm/osm_dump.c +++ b/opensm/opensm/osm_dump.c @@ -202,7 +202,9 @@ static void dump_ucast_routes(cl_map_item_t * p_map_item, void *cxt) p_node, 0); base_lid = cl_ntoh16(base_lid); num_hops = - p_physp->p_remote_physp->p_node->sw == p_sw ? 0 : + p_physp->p_remote_physp->p_node-> + sw == + p_sw ? 0 : osm_switch_get_hop_count(p_sw, base_lid, port_num); @@ -270,7 +272,8 @@ static void dump_mcast_routes(cl_map_item_t * p_map_item, void *cxt) mlid_ho + IB_LID_MCAST_START_HO); while (position <= p_tbl->max_position) { mask_entry = - cl_ntoh16((*p_tbl->p_mask_tbl)[mlid_ho][position]); + cl_ntoh16((*p_tbl-> + p_mask_tbl)[mlid_ho][position]); if (mask_entry == 0) { position++; continue; @@ -278,11 +281,13 @@ static void dump_mcast_routes(cl_map_item_t * p_map_item, void *cxt) for (j = 0; j < 16; j++) { if ((1 << j) & mask_entry) { if (first_mlid) { - fprintf(file, "%s", sw_hdr); + fprintf(file, "%s", + sw_hdr); first_mlid = FALSE; } if (first_port) { - fprintf(file, "%s", mlid_hdr); + fprintf(file, "%s", + mlid_hdr); first_port = FALSE; } fprintf(file, " 0x%03X ", diff --git a/opensm/opensm/osm_event_plugin.c b/opensm/opensm/osm_event_plugin.c index fb5052e..5133628 100644 --- a/opensm/opensm/osm_event_plugin.c +++ b/opensm/opensm/osm_event_plugin.c @@ -60,14 +60,13 @@ /** * functions */ -osm_epi_plugin_t * -osm_epi_construct(osm_log_t *p_log, char *plugin_name) +osm_epi_plugin_t *osm_epi_construct(osm_log_t * p_log, char *plugin_name) { - char lib_name[OSM_PATH_MAX]; + char lib_name[OSM_PATH_MAX]; osm_epi_plugin_t *rc = NULL; if (!plugin_name || - strcmp(plugin_name, OSM_EVENT_PLUGIN_NAME_NONE) == 0) + strcmp(plugin_name, OSM_EVENT_PLUGIN_NAME_NONE) == 0) return (NULL); /* find the plugin */ @@ -85,12 +84,13 @@ osm_epi_construct(osm_log_t *p_log, char *plugin_name) goto DLOPENFAIL; } - rc->impl = (__osm_epi_plugin_t *)dlsym(rc->handle, OSM_EVENT_PLUGIN_IMPL_NAME); + rc->impl = + (__osm_epi_plugin_t *) dlsym(rc->handle, + OSM_EVENT_PLUGIN_IMPL_NAME); if (!rc->impl) { osm_log(p_log, OSM_LOG_ERROR, "Failed to find \"%s\" symbol in \"%s\" : \"%s\"\n", - OSM_EVENT_PLUGIN_IMPL_NAME, - lib_name, dlerror()); + OSM_EVENT_PLUGIN_IMPL_NAME, lib_name, dlerror()); goto Exit; } @@ -119,15 +119,14 @@ osm_epi_construct(osm_log_t *p_log, char *plugin_name) rc->p_log = p_log; return (rc); -Exit: + Exit: dlclose(rc->handle); -DLOPENFAIL: + DLOPENFAIL: free(rc); return (NULL); } -void -osm_epi_destroy(osm_epi_plugin_t *plugin) +void osm_epi_destroy(osm_epi_plugin_t * plugin) { if (plugin) { if (plugin->impl->destroy) @@ -138,11 +137,9 @@ osm_epi_destroy(osm_epi_plugin_t *plugin) } void -osm_epi_report(osm_epi_plugin_t *plugin, osm_epi_event_id_t event_id, - void *event_data) +osm_epi_report(osm_epi_plugin_t * plugin, osm_epi_event_id_t event_id, + void *event_data) { if (plugin && plugin->impl->report) - plugin->impl->report(plugin->plugin_data, - event_id, event_data); + plugin->impl->report(plugin->plugin_data, event_id, event_data); } - diff --git a/opensm/opensm/osm_fwd_tbl.c b/opensm/opensm/osm_fwd_tbl.c index 4ff1e10..61a692d 100644 --- a/opensm/opensm/osm_fwd_tbl.c +++ b/opensm/opensm/osm_fwd_tbl.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_fwd_tbl_t. @@ -48,68 +47,59 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include #include - /********************************************************************** **********************************************************************/ ib_api_status_t -osm_fwd_tbl_init( - IN osm_fwd_tbl_t* const p_tbl, - IN const ib_switch_info_t* const p_si ) +osm_fwd_tbl_init(IN osm_fwd_tbl_t * const p_tbl, + IN const ib_switch_info_t * const p_si) { - uint16_t tbl_cap; - ib_api_status_t status = IB_SUCCESS; + uint16_t tbl_cap; + ib_api_status_t status = IB_SUCCESS; - /* - Determine the type and size of the forwarding table - used by this switch, then initialize accordingly. - The current implementation only supports switches - with linear forwarding tables. - */ - tbl_cap = cl_ntoh16( p_si->lin_cap ); + /* + Determine the type and size of the forwarding table + used by this switch, then initialize accordingly. + The current implementation only supports switches + with linear forwarding tables. + */ + tbl_cap = cl_ntoh16(p_si->lin_cap); - if( tbl_cap == 0 ) - { - /* - This switch does not support linear forwarding - tables. Error out for now. - */ - status = IB_UNSUPPORTED; - goto Exit; - } + if (tbl_cap == 0) { + /* + This switch does not support linear forwarding + tables. Error out for now. + */ + status = IB_UNSUPPORTED; + goto Exit; + } - p_tbl->p_rnd_tbl = NULL; + p_tbl->p_rnd_tbl = NULL; - p_tbl->p_lin_tbl = osm_lin_tbl_new( tbl_cap ); + p_tbl->p_lin_tbl = osm_lin_tbl_new(tbl_cap); - if( p_tbl->p_lin_tbl == NULL ) - { - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } + if (p_tbl->p_lin_tbl == NULL) { + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } - Exit: - return( status ); + Exit: + return (status); } /********************************************************************** **********************************************************************/ -void -osm_fwd_tbl_destroy( - IN osm_fwd_tbl_t* const p_tbl ) +void osm_fwd_tbl_destroy(IN osm_fwd_tbl_t * const p_tbl) { - if( p_tbl->p_lin_tbl ) - { - CL_ASSERT( p_tbl->p_rnd_tbl == NULL ); - osm_lin_tbl_delete( &p_tbl->p_lin_tbl ); - } - else - { - osm_rand_tbl_delete( &p_tbl->p_rnd_tbl ); - } + if (p_tbl->p_lin_tbl) { + CL_ASSERT(p_tbl->p_rnd_tbl == NULL); + osm_lin_tbl_delete(&p_tbl->p_lin_tbl); + } else { + osm_rand_tbl_delete(&p_tbl->p_rnd_tbl); + } } diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index 9d21194..c428823 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of opensm helper functions. @@ -46,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -59,2490 +58,2341 @@ #define LINE_LENGTH 256 /* we use two tables - one for queries and one for responses */ -const char* const __ib_sa_method_str[] = -{ - "RESERVED", /* 0 */ - "SubnAdmGet", /* 1 */ - "SubnAdmSet", /* 2 */ - "RESERVED", /* 3 */ - "RESERVED", /* 4 */ - "RESERVED", /* 5 */ - "SubnAdmReport", /* 6 */ - "RESERVED", /* 7 */ - "RESERVED", /* 8 */ - "RESERVED", /* 9 */ - "RESERVED", /* A */ - "RESERVED", /* B */ - "RESERVED", /* C */ - "RESERVED", /* D */ - "RESERVED", /* E */ - "RESERVED", /* F */ - "RESERVED", /* 10 */ - "RESERVED", /* 11 */ - "SubnAdmGetTable", /* 12 */ - "SubnAdmGetTraceTable", /* 13 */ - "SubnAdmGetMulti", /* 14 */ - "SubnAdmDelete", /* 15 */ - "UNKNOWN" /* 16 */ +const char *const __ib_sa_method_str[] = { + "RESERVED", /* 0 */ + "SubnAdmGet", /* 1 */ + "SubnAdmSet", /* 2 */ + "RESERVED", /* 3 */ + "RESERVED", /* 4 */ + "RESERVED", /* 5 */ + "SubnAdmReport", /* 6 */ + "RESERVED", /* 7 */ + "RESERVED", /* 8 */ + "RESERVED", /* 9 */ + "RESERVED", /* A */ + "RESERVED", /* B */ + "RESERVED", /* C */ + "RESERVED", /* D */ + "RESERVED", /* E */ + "RESERVED", /* F */ + "RESERVED", /* 10 */ + "RESERVED", /* 11 */ + "SubnAdmGetTable", /* 12 */ + "SubnAdmGetTraceTable", /* 13 */ + "SubnAdmGetMulti", /* 14 */ + "SubnAdmDelete", /* 15 */ + "UNKNOWN" /* 16 */ }; -const char* const __ib_sa_resp_method_str[] = -{ - "RESERVED", /* 80 */ - "SubnAdmGetResp", /* 81 */ - "RESERVED (SetResp?)", /* 82 */ - "RESERVED", /* 83 */ - "RESERVED", /* 84 */ - "RESERVED", /* 85 */ - "SubnAdmReportResp", /* 86 */ - "RESERVED", /* 87 */ - "RESERVED", /* 88 */ - "RESERVED", /* 89 */ - "RESERVED", /* 8A */ - "RESERVED", /* 8B */ - "RESERVED", /* 8C */ - "RESERVED", /* 8D */ - "RESERVED", /* 8E */ - "RESERVED", /* 8F */ - "RESERVED", /* 90 */ - "RESERVED", /* 91 */ - "SubnAdmGetTableResp", /* 92 */ - "RESERVED", /* 93 */ - "SubnAdmGetMultiResp", /* 94 */ - "SubnAdmDeleteResp", /* 95 */ - "UNKNOWN" +const char *const __ib_sa_resp_method_str[] = { + "RESERVED", /* 80 */ + "SubnAdmGetResp", /* 81 */ + "RESERVED (SetResp?)", /* 82 */ + "RESERVED", /* 83 */ + "RESERVED", /* 84 */ + "RESERVED", /* 85 */ + "SubnAdmReportResp", /* 86 */ + "RESERVED", /* 87 */ + "RESERVED", /* 88 */ + "RESERVED", /* 89 */ + "RESERVED", /* 8A */ + "RESERVED", /* 8B */ + "RESERVED", /* 8C */ + "RESERVED", /* 8D */ + "RESERVED", /* 8E */ + "RESERVED", /* 8F */ + "RESERVED", /* 90 */ + "RESERVED", /* 91 */ + "SubnAdmGetTableResp", /* 92 */ + "RESERVED", /* 93 */ + "SubnAdmGetMultiResp", /* 94 */ + "SubnAdmDeleteResp", /* 95 */ + "UNKNOWN" }; #define OSM_SA_METHOD_STR_UNKNOWN_VAL 0x16 -const char* const __ib_sm_method_str[] = -{ - "RESERVED0", /* 0 */ - "SubnGet", /* 1 */ - "SubnSet", /* 2 */ - "RESERVED3", /* 3 */ - "RESERVED4", /* 4 */ - "SubnTrap", /* 5 */ - "RESERVED6", /* 6 */ - "SubnTrapRepress", /* 7 */ - "RESERVED8", /* 8 */ - "RESERVED9", /* 9 */ - "RESERVEDA", /* A */ - "RESERVEDB", /* B */ - "RESERVEDC", /* C */ - "RESERVEDD", /* D */ - "RESERVEDE", /* E */ - "RESERVEDF", /* F */ - "RESERVED10", /* 10 */ - "SubnGetResp", /* 11 */ - "RESERVED12", /* 12 */ - "RESERVED13", /* 13 */ - "RESERVED14", /* 14 */ - "RESERVED15", /* 15 */ - "RESERVED16", /* 16 */ - "RESERVED17", /* 17 */ - "RESERVED18", /* 18 */ - "RESERVED19", /* 19 */ - "RESERVED1A", /* 1A */ - "RESERVED1B", /* 1B */ - "RESERVED1C", /* 1C */ - "RESERVED1D", /* 1D */ - "RESERVED1E", /* 1E */ - "RESERVED1F", /* 1F */ - "UNKNOWN" /* 20 */ +const char *const __ib_sm_method_str[] = { + "RESERVED0", /* 0 */ + "SubnGet", /* 1 */ + "SubnSet", /* 2 */ + "RESERVED3", /* 3 */ + "RESERVED4", /* 4 */ + "SubnTrap", /* 5 */ + "RESERVED6", /* 6 */ + "SubnTrapRepress", /* 7 */ + "RESERVED8", /* 8 */ + "RESERVED9", /* 9 */ + "RESERVEDA", /* A */ + "RESERVEDB", /* B */ + "RESERVEDC", /* C */ + "RESERVEDD", /* D */ + "RESERVEDE", /* E */ + "RESERVEDF", /* F */ + "RESERVED10", /* 10 */ + "SubnGetResp", /* 11 */ + "RESERVED12", /* 12 */ + "RESERVED13", /* 13 */ + "RESERVED14", /* 14 */ + "RESERVED15", /* 15 */ + "RESERVED16", /* 16 */ + "RESERVED17", /* 17 */ + "RESERVED18", /* 18 */ + "RESERVED19", /* 19 */ + "RESERVED1A", /* 1A */ + "RESERVED1B", /* 1B */ + "RESERVED1C", /* 1C */ + "RESERVED1D", /* 1D */ + "RESERVED1E", /* 1E */ + "RESERVED1F", /* 1F */ + "UNKNOWN" /* 20 */ }; #define OSM_SM_METHOD_STR_UNKNOWN_VAL 0x21 -const char* const __ib_sm_attr_str[] = -{ - "RESERVED", /* 0 */ - "ClassPortInfo", /* 1 */ - "Notice", /* 2 */ - "InformInfo", /* 3 */ - "RESERVED", /* 4 */ - "RESERVED", /* 5 */ - "RESERVED", /* 6 */ - "RESERVED", /* 7 */ - "RESERVED", /* 8 */ - "RESERVED", /* 9 */ - "RESERVED", /* A */ - "RESERVED", /* B */ - "RESERVED", /* C */ - "RESERVED", /* D */ - "RESERVED", /* E */ - "RESERVED", /* F */ - "NodeDescription", /* 10 */ - "NodeInfo", /* 11 */ - "SwitchInfo", /* 12 */ - "UNKNOWN", /* 13 */ - "GUIDInfo", /* 14 */ - "PortInfo", /* 15 */ - "P_KeyTable", /* 16 */ - "SLtoVLMappingTable", /* 17 */ - "VLArbitrationTable", /* 18 */ - "LinearForwardingTable", /* 19 */ - "RandomForwardingTable", /* 1A */ - "MulticastForwardingTable", /* 1B */ - "UNKNOWN", /* 1C */ - "UNKNOWN", /* 1D */ - "UNKNOWN", /* 1E */ - "UNKNOWN", /* 1F */ - "SMInfo", /* 20 */ - "UNKNOWN" /* 21 - always highest value */ +const char *const __ib_sm_attr_str[] = { + "RESERVED", /* 0 */ + "ClassPortInfo", /* 1 */ + "Notice", /* 2 */ + "InformInfo", /* 3 */ + "RESERVED", /* 4 */ + "RESERVED", /* 5 */ + "RESERVED", /* 6 */ + "RESERVED", /* 7 */ + "RESERVED", /* 8 */ + "RESERVED", /* 9 */ + "RESERVED", /* A */ + "RESERVED", /* B */ + "RESERVED", /* C */ + "RESERVED", /* D */ + "RESERVED", /* E */ + "RESERVED", /* F */ + "NodeDescription", /* 10 */ + "NodeInfo", /* 11 */ + "SwitchInfo", /* 12 */ + "UNKNOWN", /* 13 */ + "GUIDInfo", /* 14 */ + "PortInfo", /* 15 */ + "P_KeyTable", /* 16 */ + "SLtoVLMappingTable", /* 17 */ + "VLArbitrationTable", /* 18 */ + "LinearForwardingTable", /* 19 */ + "RandomForwardingTable", /* 1A */ + "MulticastForwardingTable", /* 1B */ + "UNKNOWN", /* 1C */ + "UNKNOWN", /* 1D */ + "UNKNOWN", /* 1E */ + "UNKNOWN", /* 1F */ + "SMInfo", /* 20 */ + "UNKNOWN" /* 21 - always highest value */ }; #define OSM_SM_ATTR_STR_UNKNOWN_VAL 0x21 -const char* const __ib_sa_attr_str[] = -{ - "RESERVED", /* 0 */ - "ClassPortInfo", /* 1 */ - "Notice", /* 2 */ - "InformInfo", /* 3 */ - "RESERVED", /* 4 */ - "RESERVED", /* 5 */ - "RESERVED", /* 6 */ - "RESERVED", /* 7 */ - "RESERVED", /* 8 */ - "RESERVED", /* 9 */ - "RESERVED", /* A */ - "RESERVED", /* B */ - "RESERVED", /* C */ - "RESERVED", /* D */ - "RESERVED", /* E */ - "RESERVED", /* F */ - "RESERVED", /* 10 */ - "NodeRecord", /* 11 */ - "PortInfoRecord", /* 12 */ - "SLtoVLMappingTableRecord", /* 13 */ - "SwitchInfoRecord", /* 14 */ - "LinearForwardingTableRecord", /* 15 */ - "RandomForwardingTableRecord", /* 16 */ - "MulticastForwardingTableRecord", /* 17 */ - "SMInfoRecord", /* 18 */ - "RESERVED", /* 19 */ - "RandomForwardingTable", /* 1A */ - "MulticastForwardingTable", /* 1B */ - "UNKNOWN", /* 1C */ - "UNKNOWN", /* 1D */ - "UNKNOWN", /* 1E */ - "UNKNOWN", /* 1F */ - "LinkRecord", /* 20 */ - "UNKNOWN", /* 21 */ - "UNKNOWN", /* 22 */ - "UNKNOWN", /* 23 */ - "UNKNOWN", /* 24 */ - "UNKNOWN", /* 25 */ - "UNKNOWN", /* 26 */ - "UNKNOWN", /* 27 */ - "UNKNOWN", /* 28 */ - "UNKNOWN", /* 29 */ - "UNKNOWN", /* 2A */ - "UNKNOWN", /* 2B */ - "UNKNOWN", /* 2C */ - "UNKNOWN", /* 2D */ - "UNKNOWN", /* 2E */ - "UNKNOWN", /* 2F */ - "GuidInfoRecord", /* 30 */ - "ServiceRecord", /* 31 */ - "UNKNOWN", /* 32 */ - "P_KeyTableRecord", /* 33 */ - "UNKNOWN", /* 34 */ - "PathRecord", /* 35 */ - "VLArbitrationTableRecord", /* 36 */ - "UNKNOWN", /* 37 */ - "MCMemberRecord", /* 38 */ - "TraceRecord", /* 39 */ - "MultiPathRecord", /* 3A */ - "ServiceAssociationRecord", /* 3B */ - "UNKNOWN", /* 3C */ - "UNKNOWN", /* 3D */ - "UNKNOWN", /* 3E */ - "UNKNOWN", /* 3F */ - "UNKNOWN", /* 40 */ - "UNKNOWN", /* 41 */ - "UNKNOWN", /* 42 */ - "UNKNOWN", /* 43 */ - "UNKNOWN", /* 44 */ - "UNKNOWN", /* 45 */ - "UNKNOWN", /* 46 */ - "UNKNOWN", /* 47 */ - "UNKNOWN", /* 48 */ - "UNKNOWN", /* 49 */ - "UNKNOWN", /* 4A */ - "UNKNOWN", /* 4B */ - "UNKNOWN", /* 4C */ - "UNKNOWN", /* 4D */ - "UNKNOWN", /* 4E */ - "UNKNOWN", /* 4F */ - "UNKNOWN", /* 50 */ - "UNKNOWN", /* 51 */ - "UNKNOWN", /* 52 */ - "UNKNOWN", /* 53 */ - "UNKNOWN", /* 54 */ - "UNKNOWN", /* 55 */ - "UNKNOWN", /* 56 */ - "UNKNOWN", /* 57 */ - "UNKNOWN", /* 58 */ - "UNKNOWN", /* 59 */ - "UNKNOWN", /* 5A */ - "UNKNOWN", /* 5B */ - "UNKNOWN", /* 5C */ - "UNKNOWN", /* 5D */ - "UNKNOWN", /* 5E */ - "UNKNOWN", /* 5F */ - "UNKNOWN", /* 60 */ - "UNKNOWN", /* 61 */ - "UNKNOWN", /* 62 */ - "UNKNOWN", /* 63 */ - "UNKNOWN", /* 64 */ - "UNKNOWN", /* 65 */ - "UNKNOWN", /* 66 */ - "UNKNOWN", /* 67 */ - "UNKNOWN", /* 68 */ - "UNKNOWN", /* 69 */ - "UNKNOWN", /* 6A */ - "UNKNOWN", /* 6B */ - "UNKNOWN", /* 6C */ - "UNKNOWN", /* 6D */ - "UNKNOWN", /* 6E */ - "UNKNOWN", /* 6F */ - "UNKNOWN", /* 70 */ - "UNKNOWN", /* 71 */ - "UNKNOWN", /* 72 */ - "UNKNOWN", /* 73 */ - "UNKNOWN", /* 74 */ - "UNKNOWN", /* 75 */ - "UNKNOWN", /* 76 */ - "UNKNOWN", /* 77 */ - "UNKNOWN", /* 78 */ - "UNKNOWN", /* 79 */ - "UNKNOWN", /* 7A */ - "UNKNOWN", /* 7B */ - "UNKNOWN", /* 7C */ - "UNKNOWN", /* 7D */ - "UNKNOWN", /* 7E */ - "UNKNOWN", /* 7F */ - "UNKNOWN", /* 80 */ - "UNKNOWN", /* 81 */ - "UNKNOWN", /* 82 */ - "UNKNOWN", /* 83 */ - "UNKNOWN", /* 84 */ - "UNKNOWN", /* 85 */ - "UNKNOWN", /* 86 */ - "UNKNOWN", /* 87 */ - "UNKNOWN", /* 88 */ - "UNKNOWN", /* 89 */ - "UNKNOWN", /* 8A */ - "UNKNOWN", /* 8B */ - "UNKNOWN", /* 8C */ - "UNKNOWN", /* 8D */ - "UNKNOWN", /* 8E */ - "UNKNOWN", /* 8F */ - "UNKNOWN", /* 90 */ - "UNKNOWN", /* 91 */ - "UNKNOWN", /* 92 */ - "UNKNOWN", /* 93 */ - "UNKNOWN", /* 94 */ - "UNKNOWN", /* 95 */ - "UNKNOWN", /* 96 */ - "UNKNOWN", /* 97 */ - "UNKNOWN", /* 98 */ - "UNKNOWN", /* 99 */ - "UNKNOWN", /* 9A */ - "UNKNOWN", /* 9B */ - "UNKNOWN", /* 9C */ - "UNKNOWN", /* 9D */ - "UNKNOWN", /* 9E */ - "UNKNOWN", /* 9F */ - "UNKNOWN", /* A0 */ - "UNKNOWN", /* A1 */ - "UNKNOWN", /* A2 */ - "UNKNOWN", /* A3 */ - "UNKNOWN", /* A4 */ - "UNKNOWN", /* A5 */ - "UNKNOWN", /* A6 */ - "UNKNOWN", /* A7 */ - "UNKNOWN", /* A8 */ - "UNKNOWN", /* A9 */ - "UNKNOWN", /* AA */ - "UNKNOWN", /* AB */ - "UNKNOWN", /* AC */ - "UNKNOWN", /* AD */ - "UNKNOWN", /* AE */ - "UNKNOWN", /* AF */ - "UNKNOWN", /* B0 */ - "UNKNOWN", /* B1 */ - "UNKNOWN", /* B2 */ - "UNKNOWN", /* B3 */ - "UNKNOWN", /* B4 */ - "UNKNOWN", /* B5 */ - "UNKNOWN", /* B6 */ - "UNKNOWN", /* B7 */ - "UNKNOWN", /* B8 */ - "UNKNOWN", /* B9 */ - "UNKNOWN", /* BA */ - "UNKNOWN", /* BB */ - "UNKNOWN", /* BC */ - "UNKNOWN", /* BD */ - "UNKNOWN", /* BE */ - "UNKNOWN", /* BF */ - "UNKNOWN", /* C0 */ - "UNKNOWN", /* C1 */ - "UNKNOWN", /* C2 */ - "UNKNOWN", /* C3 */ - "UNKNOWN", /* C4 */ - "UNKNOWN", /* C5 */ - "UNKNOWN", /* C6 */ - "UNKNOWN", /* C7 */ - "UNKNOWN", /* C8 */ - "UNKNOWN", /* C9 */ - "UNKNOWN", /* CA */ - "UNKNOWN", /* CB */ - "UNKNOWN", /* CC */ - "UNKNOWN", /* CD */ - "UNKNOWN", /* CE */ - "UNKNOWN", /* CF */ - "UNKNOWN", /* D0 */ - "UNKNOWN", /* D1 */ - "UNKNOWN", /* D2 */ - "UNKNOWN", /* D3 */ - "UNKNOWN", /* D4 */ - "UNKNOWN", /* D5 */ - "UNKNOWN", /* D6 */ - "UNKNOWN", /* D7 */ - "UNKNOWN", /* D8 */ - "UNKNOWN", /* D9 */ - "UNKNOWN", /* DA */ - "UNKNOWN", /* DB */ - "UNKNOWN", /* DC */ - "UNKNOWN", /* DD */ - "UNKNOWN", /* DE */ - "UNKNOWN", /* DF */ - "UNKNOWN", /* E0 */ - "UNKNOWN", /* E1 */ - "UNKNOWN", /* E2 */ - "UNKNOWN", /* E3 */ - "UNKNOWN", /* E4 */ - "UNKNOWN", /* E5 */ - "UNKNOWN", /* E6 */ - "UNKNOWN", /* E7 */ - "UNKNOWN", /* E8 */ - "UNKNOWN", /* E9 */ - "UNKNOWN", /* EA */ - "UNKNOWN", /* EB */ - "UNKNOWN", /* EC */ - "UNKNOWN", /* ED */ - "UNKNOWN", /* EE */ - "UNKNOWN", /* EF */ - "UNKNOWN", /* F0 */ - "UNKNOWN", /* F1 */ - "UNKNOWN", /* F2 */ - "InformInfoRecord", /* F3 */ - "UNKNOWN" /* F4 - always highest value */ +const char *const __ib_sa_attr_str[] = { + "RESERVED", /* 0 */ + "ClassPortInfo", /* 1 */ + "Notice", /* 2 */ + "InformInfo", /* 3 */ + "RESERVED", /* 4 */ + "RESERVED", /* 5 */ + "RESERVED", /* 6 */ + "RESERVED", /* 7 */ + "RESERVED", /* 8 */ + "RESERVED", /* 9 */ + "RESERVED", /* A */ + "RESERVED", /* B */ + "RESERVED", /* C */ + "RESERVED", /* D */ + "RESERVED", /* E */ + "RESERVED", /* F */ + "RESERVED", /* 10 */ + "NodeRecord", /* 11 */ + "PortInfoRecord", /* 12 */ + "SLtoVLMappingTableRecord", /* 13 */ + "SwitchInfoRecord", /* 14 */ + "LinearForwardingTableRecord", /* 15 */ + "RandomForwardingTableRecord", /* 16 */ + "MulticastForwardingTableRecord", /* 17 */ + "SMInfoRecord", /* 18 */ + "RESERVED", /* 19 */ + "RandomForwardingTable", /* 1A */ + "MulticastForwardingTable", /* 1B */ + "UNKNOWN", /* 1C */ + "UNKNOWN", /* 1D */ + "UNKNOWN", /* 1E */ + "UNKNOWN", /* 1F */ + "LinkRecord", /* 20 */ + "UNKNOWN", /* 21 */ + "UNKNOWN", /* 22 */ + "UNKNOWN", /* 23 */ + "UNKNOWN", /* 24 */ + "UNKNOWN", /* 25 */ + "UNKNOWN", /* 26 */ + "UNKNOWN", /* 27 */ + "UNKNOWN", /* 28 */ + "UNKNOWN", /* 29 */ + "UNKNOWN", /* 2A */ + "UNKNOWN", /* 2B */ + "UNKNOWN", /* 2C */ + "UNKNOWN", /* 2D */ + "UNKNOWN", /* 2E */ + "UNKNOWN", /* 2F */ + "GuidInfoRecord", /* 30 */ + "ServiceRecord", /* 31 */ + "UNKNOWN", /* 32 */ + "P_KeyTableRecord", /* 33 */ + "UNKNOWN", /* 34 */ + "PathRecord", /* 35 */ + "VLArbitrationTableRecord", /* 36 */ + "UNKNOWN", /* 37 */ + "MCMemberRecord", /* 38 */ + "TraceRecord", /* 39 */ + "MultiPathRecord", /* 3A */ + "ServiceAssociationRecord", /* 3B */ + "UNKNOWN", /* 3C */ + "UNKNOWN", /* 3D */ + "UNKNOWN", /* 3E */ + "UNKNOWN", /* 3F */ + "UNKNOWN", /* 40 */ + "UNKNOWN", /* 41 */ + "UNKNOWN", /* 42 */ + "UNKNOWN", /* 43 */ + "UNKNOWN", /* 44 */ + "UNKNOWN", /* 45 */ + "UNKNOWN", /* 46 */ + "UNKNOWN", /* 47 */ + "UNKNOWN", /* 48 */ + "UNKNOWN", /* 49 */ + "UNKNOWN", /* 4A */ + "UNKNOWN", /* 4B */ + "UNKNOWN", /* 4C */ + "UNKNOWN", /* 4D */ + "UNKNOWN", /* 4E */ + "UNKNOWN", /* 4F */ + "UNKNOWN", /* 50 */ + "UNKNOWN", /* 51 */ + "UNKNOWN", /* 52 */ + "UNKNOWN", /* 53 */ + "UNKNOWN", /* 54 */ + "UNKNOWN", /* 55 */ + "UNKNOWN", /* 56 */ + "UNKNOWN", /* 57 */ + "UNKNOWN", /* 58 */ + "UNKNOWN", /* 59 */ + "UNKNOWN", /* 5A */ + "UNKNOWN", /* 5B */ + "UNKNOWN", /* 5C */ + "UNKNOWN", /* 5D */ + "UNKNOWN", /* 5E */ + "UNKNOWN", /* 5F */ + "UNKNOWN", /* 60 */ + "UNKNOWN", /* 61 */ + "UNKNOWN", /* 62 */ + "UNKNOWN", /* 63 */ + "UNKNOWN", /* 64 */ + "UNKNOWN", /* 65 */ + "UNKNOWN", /* 66 */ + "UNKNOWN", /* 67 */ + "UNKNOWN", /* 68 */ + "UNKNOWN", /* 69 */ + "UNKNOWN", /* 6A */ + "UNKNOWN", /* 6B */ + "UNKNOWN", /* 6C */ + "UNKNOWN", /* 6D */ + "UNKNOWN", /* 6E */ + "UNKNOWN", /* 6F */ + "UNKNOWN", /* 70 */ + "UNKNOWN", /* 71 */ + "UNKNOWN", /* 72 */ + "UNKNOWN", /* 73 */ + "UNKNOWN", /* 74 */ + "UNKNOWN", /* 75 */ + "UNKNOWN", /* 76 */ + "UNKNOWN", /* 77 */ + "UNKNOWN", /* 78 */ + "UNKNOWN", /* 79 */ + "UNKNOWN", /* 7A */ + "UNKNOWN", /* 7B */ + "UNKNOWN", /* 7C */ + "UNKNOWN", /* 7D */ + "UNKNOWN", /* 7E */ + "UNKNOWN", /* 7F */ + "UNKNOWN", /* 80 */ + "UNKNOWN", /* 81 */ + "UNKNOWN", /* 82 */ + "UNKNOWN", /* 83 */ + "UNKNOWN", /* 84 */ + "UNKNOWN", /* 85 */ + "UNKNOWN", /* 86 */ + "UNKNOWN", /* 87 */ + "UNKNOWN", /* 88 */ + "UNKNOWN", /* 89 */ + "UNKNOWN", /* 8A */ + "UNKNOWN", /* 8B */ + "UNKNOWN", /* 8C */ + "UNKNOWN", /* 8D */ + "UNKNOWN", /* 8E */ + "UNKNOWN", /* 8F */ + "UNKNOWN", /* 90 */ + "UNKNOWN", /* 91 */ + "UNKNOWN", /* 92 */ + "UNKNOWN", /* 93 */ + "UNKNOWN", /* 94 */ + "UNKNOWN", /* 95 */ + "UNKNOWN", /* 96 */ + "UNKNOWN", /* 97 */ + "UNKNOWN", /* 98 */ + "UNKNOWN", /* 99 */ + "UNKNOWN", /* 9A */ + "UNKNOWN", /* 9B */ + "UNKNOWN", /* 9C */ + "UNKNOWN", /* 9D */ + "UNKNOWN", /* 9E */ + "UNKNOWN", /* 9F */ + "UNKNOWN", /* A0 */ + "UNKNOWN", /* A1 */ + "UNKNOWN", /* A2 */ + "UNKNOWN", /* A3 */ + "UNKNOWN", /* A4 */ + "UNKNOWN", /* A5 */ + "UNKNOWN", /* A6 */ + "UNKNOWN", /* A7 */ + "UNKNOWN", /* A8 */ + "UNKNOWN", /* A9 */ + "UNKNOWN", /* AA */ + "UNKNOWN", /* AB */ + "UNKNOWN", /* AC */ + "UNKNOWN", /* AD */ + "UNKNOWN", /* AE */ + "UNKNOWN", /* AF */ + "UNKNOWN", /* B0 */ + "UNKNOWN", /* B1 */ + "UNKNOWN", /* B2 */ + "UNKNOWN", /* B3 */ + "UNKNOWN", /* B4 */ + "UNKNOWN", /* B5 */ + "UNKNOWN", /* B6 */ + "UNKNOWN", /* B7 */ + "UNKNOWN", /* B8 */ + "UNKNOWN", /* B9 */ + "UNKNOWN", /* BA */ + "UNKNOWN", /* BB */ + "UNKNOWN", /* BC */ + "UNKNOWN", /* BD */ + "UNKNOWN", /* BE */ + "UNKNOWN", /* BF */ + "UNKNOWN", /* C0 */ + "UNKNOWN", /* C1 */ + "UNKNOWN", /* C2 */ + "UNKNOWN", /* C3 */ + "UNKNOWN", /* C4 */ + "UNKNOWN", /* C5 */ + "UNKNOWN", /* C6 */ + "UNKNOWN", /* C7 */ + "UNKNOWN", /* C8 */ + "UNKNOWN", /* C9 */ + "UNKNOWN", /* CA */ + "UNKNOWN", /* CB */ + "UNKNOWN", /* CC */ + "UNKNOWN", /* CD */ + "UNKNOWN", /* CE */ + "UNKNOWN", /* CF */ + "UNKNOWN", /* D0 */ + "UNKNOWN", /* D1 */ + "UNKNOWN", /* D2 */ + "UNKNOWN", /* D3 */ + "UNKNOWN", /* D4 */ + "UNKNOWN", /* D5 */ + "UNKNOWN", /* D6 */ + "UNKNOWN", /* D7 */ + "UNKNOWN", /* D8 */ + "UNKNOWN", /* D9 */ + "UNKNOWN", /* DA */ + "UNKNOWN", /* DB */ + "UNKNOWN", /* DC */ + "UNKNOWN", /* DD */ + "UNKNOWN", /* DE */ + "UNKNOWN", /* DF */ + "UNKNOWN", /* E0 */ + "UNKNOWN", /* E1 */ + "UNKNOWN", /* E2 */ + "UNKNOWN", /* E3 */ + "UNKNOWN", /* E4 */ + "UNKNOWN", /* E5 */ + "UNKNOWN", /* E6 */ + "UNKNOWN", /* E7 */ + "UNKNOWN", /* E8 */ + "UNKNOWN", /* E9 */ + "UNKNOWN", /* EA */ + "UNKNOWN", /* EB */ + "UNKNOWN", /* EC */ + "UNKNOWN", /* ED */ + "UNKNOWN", /* EE */ + "UNKNOWN", /* EF */ + "UNKNOWN", /* F0 */ + "UNKNOWN", /* F1 */ + "UNKNOWN", /* F2 */ + "InformInfoRecord", /* F3 */ + "UNKNOWN" /* F4 - always highest value */ }; #define OSM_SA_ATTR_STR_UNKNOWN_VAL 0xF4 - /********************************************************************** **********************************************************************/ -const char* -ib_get_sa_method_str( - IN uint8_t method ) +const char *ib_get_sa_method_str(IN uint8_t method) { - if (method & 0x80) - { - method = method & 0x7f; - if( method >= OSM_SA_METHOD_STR_UNKNOWN_VAL ) - method = OSM_SA_METHOD_STR_UNKNOWN_VAL; - /* it is a response - use the response table */ - return( __ib_sa_resp_method_str[method] ); - } - else - { - if( method >= OSM_SA_METHOD_STR_UNKNOWN_VAL ) - method = OSM_SA_METHOD_STR_UNKNOWN_VAL; - return( __ib_sa_method_str[method] ); - } + if (method & 0x80) { + method = method & 0x7f; + if (method >= OSM_SA_METHOD_STR_UNKNOWN_VAL) + method = OSM_SA_METHOD_STR_UNKNOWN_VAL; + /* it is a response - use the response table */ + return (__ib_sa_resp_method_str[method]); + } else { + if (method >= OSM_SA_METHOD_STR_UNKNOWN_VAL) + method = OSM_SA_METHOD_STR_UNKNOWN_VAL; + return (__ib_sa_method_str[method]); + } } /********************************************************************** **********************************************************************/ -const char* -ib_get_sm_method_str( - IN uint8_t method ) +const char *ib_get_sm_method_str(IN uint8_t method) { - if (method & 0x80) method = (method & 0x0F) | 0x10; - if( method >= OSM_SM_METHOD_STR_UNKNOWN_VAL ) - method = OSM_SM_METHOD_STR_UNKNOWN_VAL; - return( __ib_sm_method_str[method] ); + if (method & 0x80) + method = (method & 0x0F) | 0x10; + if (method >= OSM_SM_METHOD_STR_UNKNOWN_VAL) + method = OSM_SM_METHOD_STR_UNKNOWN_VAL; + return (__ib_sm_method_str[method]); } /********************************************************************** **********************************************************************/ -const char* -ib_get_sm_attr_str( - IN ib_net16_t attr ) +const char *ib_get_sm_attr_str(IN ib_net16_t attr) { - uint16_t host_attr; - host_attr = cl_ntoh16( attr ); + uint16_t host_attr; + host_attr = cl_ntoh16(attr); - if( host_attr >= OSM_SM_ATTR_STR_UNKNOWN_VAL ) - host_attr = OSM_SM_ATTR_STR_UNKNOWN_VAL; + if (host_attr >= OSM_SM_ATTR_STR_UNKNOWN_VAL) + host_attr = OSM_SM_ATTR_STR_UNKNOWN_VAL; - return( __ib_sm_attr_str[host_attr] ); + return (__ib_sm_attr_str[host_attr]); } /********************************************************************** **********************************************************************/ -const char* -ib_get_sa_attr_str( - IN ib_net16_t attr ) +const char *ib_get_sa_attr_str(IN ib_net16_t attr) { - uint16_t host_attr; - host_attr = cl_ntoh16( attr ); + uint16_t host_attr; + host_attr = cl_ntoh16(attr); - if( host_attr >= OSM_SA_ATTR_STR_UNKNOWN_VAL ) - host_attr = OSM_SA_ATTR_STR_UNKNOWN_VAL; + if (host_attr >= OSM_SA_ATTR_STR_UNKNOWN_VAL) + host_attr = OSM_SA_ATTR_STR_UNKNOWN_VAL; - return( __ib_sa_attr_str[host_attr] ); + return (__ib_sa_attr_str[host_attr]); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_dbg_do_line( - IN char** pp_local, - IN const uint32_t buf_size, - IN const char* const p_prefix_str, - IN const char* const p_new_str, - IN uint32_t* const p_total_len ) +osm_dbg_do_line(IN char **pp_local, + IN const uint32_t buf_size, + IN const char *const p_prefix_str, + IN const char *const p_new_str, IN uint32_t * const p_total_len) { - char line[LINE_LENGTH]; - uint32_t len; - - sprintf( line, "%s%s", p_prefix_str, p_new_str ); - len = (uint32_t) strlen( line ); - *p_total_len += len; - if( *p_total_len + sizeof('\0') > buf_size ) - return( IB_INSUFFICIENT_MEMORY ); - - strcpy( *pp_local, line ); - *pp_local += len; - return( IB_SUCCESS ); + char line[LINE_LENGTH]; + uint32_t len; + + sprintf(line, "%s%s", p_prefix_str, p_new_str); + len = (uint32_t) strlen(line); + *p_total_len += len; + if (*p_total_len + sizeof('\0') > buf_size) + return (IB_INSUFFICIENT_MEMORY); + + strcpy(*pp_local, line); + *pp_local += len; + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ void -osm_dbg_get_capabilities_str( - IN char* p_buf, - IN const uint32_t buf_size, - IN const char* const p_prefix_str, - IN const ib_port_info_t* const p_pi ) +osm_dbg_get_capabilities_str(IN char *p_buf, + IN const uint32_t buf_size, + IN const char *const p_prefix_str, + IN const ib_port_info_t * const p_pi) { - uint32_t total_len = 0; - char *p_local = p_buf; - - strcpy( p_local, "Capability Mask:\n" ); - p_local += strlen( p_local ); - - if( p_pi->capability_mask & IB_PORT_CAP_RESV0 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV0\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_IS_SM ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_IS_SM\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_NOTICE ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_NOTICE\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_TRAP ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_TRAP\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_IPD ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_IPD\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_AUTO_MIG ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_AUTO_MIG\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_SL_MAP ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_SL_MAP\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_NV_MKEY ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_NV_MKEY\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_NV_PKEY ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_NV_PKEY\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_LED_INFO ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_LED_INFO\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_SM_DISAB ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_SM_DISAB\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_SYS_IMG_GUID ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_SYS_IMG_GUID\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_PKEY_SW_EXT_PORT_TRAP ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_PKEY_SW_EXT_PORT_TRAP\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV13 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV13\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV14 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV14\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV15 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV15\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_COM_MGT ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_COM_MGT\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_SNMP ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_SNMP\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_REINIT ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_REINIT\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_DEV_MGT ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_DEV_MGT\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_VEND_CLS ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_VEND_CLS\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_DR_NTC ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_DR_NTC\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_CAP_NTC ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_CAP_NTC\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_BM ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_BM\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_LINK_RT_LATENCY ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_LINK_RT_LATENCY\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_HAS_CLIENT_REREG ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_HAS_CLIENT_REREG\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV26 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV26\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV27 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV27\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV28) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV28\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV29 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV29\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV30 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV30\n", &total_len ) != IB_SUCCESS ) - return; - } - if( p_pi->capability_mask & IB_PORT_CAP_RESV31 ) - { - if( osm_dbg_do_line( &p_local, buf_size, p_prefix_str, - "IB_PORT_CAP_RESV31\n", &total_len ) != IB_SUCCESS ) - return; - } + uint32_t total_len = 0; + char *p_local = p_buf; + + strcpy(p_local, "Capability Mask:\n"); + p_local += strlen(p_local); + + if (p_pi->capability_mask & IB_PORT_CAP_RESV0) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV0\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_IS_SM) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_IS_SM\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_NOTICE) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_NOTICE\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_TRAP) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_TRAP\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_IPD) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_IPD\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_AUTO_MIG) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_AUTO_MIG\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_SL_MAP) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_SL_MAP\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_NV_MKEY) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_NV_MKEY\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_NV_PKEY) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_NV_PKEY\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_LED_INFO) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_LED_INFO\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_SM_DISAB) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_SM_DISAB\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_SYS_IMG_GUID) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_SYS_IMG_GUID\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_PKEY_SW_EXT_PORT_TRAP) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_PKEY_SW_EXT_PORT_TRAP\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV13) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV13\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV14) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV14\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV15) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV15\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_COM_MGT) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_COM_MGT\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_SNMP) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_SNMP\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_REINIT) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_REINIT\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_DEV_MGT) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_DEV_MGT\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_VEND_CLS) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_VEND_CLS\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_DR_NTC) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_DR_NTC\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_CAP_NTC) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_CAP_NTC\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_BM) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_BM\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_LINK_RT_LATENCY) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_LINK_RT_LATENCY\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_HAS_CLIENT_REREG) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_HAS_CLIENT_REREG\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV26) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV26\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV27) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV27\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV28) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV28\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV29) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV29\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV30) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV30\n", + &total_len) != IB_SUCCESS) + return; + } + if (p_pi->capability_mask & IB_PORT_CAP_RESV31) { + if (osm_dbg_do_line(&p_local, buf_size, p_prefix_str, + "IB_PORT_CAP_RESV31\n", + &total_len) != IB_SUCCESS) + return; + } } /********************************************************************** **********************************************************************/ void -osm_dump_port_info( - IN osm_log_t* const p_log, - IN const ib_net64_t node_guid, - IN const ib_net64_t port_guid, - IN const uint8_t port_num, - IN const ib_port_info_t* const p_pi, - IN const osm_log_level_t log_level ) +osm_dump_port_info(IN osm_log_t * const p_log, + IN const ib_net64_t node_guid, + IN const ib_net64_t port_guid, + IN const uint8_t port_num, + IN const ib_port_info_t * const p_pi, + IN const osm_log_level_t log_level) { - char buf[BUF_SIZE]; - - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "PortInfo dump:\n" - "\t\t\t\tport number.............0x%X\n" - "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tm_key...................0x%016" PRIx64 "\n" - "\t\t\t\tsubnet_prefix...........0x%016" PRIx64 "\n" - "\t\t\t\tbase_lid................0x%X\n" - "\t\t\t\tmaster_sm_base_lid......0x%X\n" - "\t\t\t\tcapability_mask.........0x%X\n" - "\t\t\t\tdiag_code...............0x%X\n" - "\t\t\t\tm_key_lease_period......0x%X\n" - "\t\t\t\tlocal_port_num..........0x%X\n" - "\t\t\t\tlink_width_enabled......0x%X\n" - "\t\t\t\tlink_width_supported....0x%X\n" - "\t\t\t\tlink_width_active.......0x%X\n" - "\t\t\t\tlink_speed_supported....0x%X\n" - "\t\t\t\tport_state..............%s\n" - "\t\t\t\tstate_info2.............0x%X\n" - "\t\t\t\tm_key_protect_bits......0x%X\n" - "\t\t\t\tlmc.....................0x%X\n" - "\t\t\t\tlink_speed..............0x%X\n" - "\t\t\t\tmtu_smsl................0x%X\n" - "\t\t\t\tvl_cap_init_type........0x%X\n" - "\t\t\t\tvl_high_limit...........0x%X\n" - "\t\t\t\tvl_arb_high_cap.........0x%X\n" - "\t\t\t\tvl_arb_low_cap..........0x%X\n" - "\t\t\t\tinit_rep_mtu_cap........0x%X\n" - "\t\t\t\tvl_stall_life...........0x%X\n" - "\t\t\t\tvl_enforce..............0x%X\n" - "\t\t\t\tm_key_violations........0x%X\n" - "\t\t\t\tp_key_violations........0x%X\n" - "\t\t\t\tq_key_violations........0x%X\n" - "\t\t\t\tguid_cap................0x%X\n" - "\t\t\t\tclient_reregister.......0x%X\n" - "\t\t\t\tsubnet_timeout..........0x%X\n" - "\t\t\t\tresp_time_value.........0x%X\n" - "\t\t\t\terror_threshold.........0x%X\n" - "", - port_num, - cl_ntoh64( node_guid ), - cl_ntoh64( port_guid ), - cl_ntoh64( p_pi->m_key ), - cl_ntoh64( p_pi->subnet_prefix ), - cl_ntoh16( p_pi->base_lid ), - cl_ntoh16( p_pi->master_sm_base_lid ), - cl_ntoh32( p_pi->capability_mask ), - cl_ntoh16( p_pi->diag_code ), - cl_ntoh16( p_pi->m_key_lease_period ), - p_pi->local_port_num, - p_pi->link_width_enabled, - p_pi->link_width_supported, - p_pi->link_width_active, - ib_port_info_get_link_speed_sup( p_pi ), - ib_get_port_state_str( ib_port_info_get_port_state( p_pi ) ), - p_pi->state_info2, - ib_port_info_get_mpb( p_pi ), - ib_port_info_get_lmc( p_pi ), - p_pi->link_speed, - p_pi->mtu_smsl, - p_pi->vl_cap, - p_pi->vl_high_limit, - p_pi->vl_arb_high_cap, - p_pi->vl_arb_low_cap, - p_pi->mtu_cap, - p_pi->vl_stall_life, - p_pi->vl_enforce, - cl_ntoh16( p_pi->m_key_violations ), - cl_ntoh16( p_pi->p_key_violations ), - cl_ntoh16( p_pi->q_key_violations ), - p_pi->guid_cap, - ib_port_info_get_client_rereg( p_pi ), - ib_port_info_get_timeout( p_pi ), - p_pi->resp_time_value, - p_pi->error_threshold - ); - - /* show the capabilities mask */ - if( p_pi->capability_mask ) - { - osm_dbg_get_capabilities_str( buf, BUF_SIZE, "\t\t\t\t", p_pi ); - osm_log( p_log, log_level, "%s", buf ); - } - } + char buf[BUF_SIZE]; + + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "PortInfo dump:\n" + "\t\t\t\tport number.............0x%X\n" + "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tm_key...................0x%016" PRIx64 "\n" + "\t\t\t\tsubnet_prefix...........0x%016" PRIx64 "\n" + "\t\t\t\tbase_lid................0x%X\n" + "\t\t\t\tmaster_sm_base_lid......0x%X\n" + "\t\t\t\tcapability_mask.........0x%X\n" + "\t\t\t\tdiag_code...............0x%X\n" + "\t\t\t\tm_key_lease_period......0x%X\n" + "\t\t\t\tlocal_port_num..........0x%X\n" + "\t\t\t\tlink_width_enabled......0x%X\n" + "\t\t\t\tlink_width_supported....0x%X\n" + "\t\t\t\tlink_width_active.......0x%X\n" + "\t\t\t\tlink_speed_supported....0x%X\n" + "\t\t\t\tport_state..............%s\n" + "\t\t\t\tstate_info2.............0x%X\n" + "\t\t\t\tm_key_protect_bits......0x%X\n" + "\t\t\t\tlmc.....................0x%X\n" + "\t\t\t\tlink_speed..............0x%X\n" + "\t\t\t\tmtu_smsl................0x%X\n" + "\t\t\t\tvl_cap_init_type........0x%X\n" + "\t\t\t\tvl_high_limit...........0x%X\n" + "\t\t\t\tvl_arb_high_cap.........0x%X\n" + "\t\t\t\tvl_arb_low_cap..........0x%X\n" + "\t\t\t\tinit_rep_mtu_cap........0x%X\n" + "\t\t\t\tvl_stall_life...........0x%X\n" + "\t\t\t\tvl_enforce..............0x%X\n" + "\t\t\t\tm_key_violations........0x%X\n" + "\t\t\t\tp_key_violations........0x%X\n" + "\t\t\t\tq_key_violations........0x%X\n" + "\t\t\t\tguid_cap................0x%X\n" + "\t\t\t\tclient_reregister.......0x%X\n" + "\t\t\t\tsubnet_timeout..........0x%X\n" + "\t\t\t\tresp_time_value.........0x%X\n" + "\t\t\t\terror_threshold.........0x%X\n" + "", + port_num, + cl_ntoh64(node_guid), + cl_ntoh64(port_guid), + cl_ntoh64(p_pi->m_key), + cl_ntoh64(p_pi->subnet_prefix), + cl_ntoh16(p_pi->base_lid), + cl_ntoh16(p_pi->master_sm_base_lid), + cl_ntoh32(p_pi->capability_mask), + cl_ntoh16(p_pi->diag_code), + cl_ntoh16(p_pi->m_key_lease_period), + p_pi->local_port_num, + p_pi->link_width_enabled, + p_pi->link_width_supported, + p_pi->link_width_active, + ib_port_info_get_link_speed_sup(p_pi), + ib_get_port_state_str(ib_port_info_get_port_state + (p_pi)), p_pi->state_info2, + ib_port_info_get_mpb(p_pi), ib_port_info_get_lmc(p_pi), + p_pi->link_speed, p_pi->mtu_smsl, p_pi->vl_cap, + p_pi->vl_high_limit, p_pi->vl_arb_high_cap, + p_pi->vl_arb_low_cap, p_pi->mtu_cap, + p_pi->vl_stall_life, p_pi->vl_enforce, + cl_ntoh16(p_pi->m_key_violations), + cl_ntoh16(p_pi->p_key_violations), + cl_ntoh16(p_pi->q_key_violations), p_pi->guid_cap, + ib_port_info_get_client_rereg(p_pi), + ib_port_info_get_timeout(p_pi), p_pi->resp_time_value, + p_pi->error_threshold); + + /* show the capabilities mask */ + if (p_pi->capability_mask) { + osm_dbg_get_capabilities_str(buf, BUF_SIZE, "\t\t\t\t", + p_pi); + osm_log(p_log, log_level, "%s", buf); + } + } } /********************************************************************** **********************************************************************/ void -osm_dump_portinfo_record( - IN osm_log_t* const p_log, - IN const ib_portinfo_record_t* const p_pir, - IN const osm_log_level_t log_level ) +osm_dump_portinfo_record(IN osm_log_t * const p_log, + IN const ib_portinfo_record_t * const p_pir, + IN const osm_log_level_t log_level) { - char buf[BUF_SIZE]; - const ib_port_info_t * const p_pi = &p_pir->port_info; - - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "PortInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tEndPortLid..............0x%X\n" - "\t\t\t\tPortNum.................0x%X\n" - "\t\t\t\tReserved................0x%X\n" - "\t\t\t\tPortInfo dump:\n" - "\t\t\t\tm_key...................0x%016" PRIx64 "\n" - "\t\t\t\tsubnet_prefix...........0x%016" PRIx64 "\n" - "\t\t\t\tbase_lid................0x%X\n" - "\t\t\t\tmaster_sm_base_lid......0x%X\n" - "\t\t\t\tcapability_mask.........0x%X\n" - "\t\t\t\tdiag_code...............0x%X\n" - "\t\t\t\tm_key_lease_period......0x%X\n" - "\t\t\t\tlocal_port_num..........0x%X\n" - "\t\t\t\tlink_width_enabled......0x%X\n" - "\t\t\t\tlink_width_supported....0x%X\n" - "\t\t\t\tlink_width_active.......0x%X\n" - "\t\t\t\tlink_speed_supported....0x%X\n" - "\t\t\t\tport_state..............%s\n" - "\t\t\t\tstate_info2.............0x%X\n" - "\t\t\t\tm_key_protect_bits......0x%X\n" - "\t\t\t\tlmc.....................0x%X\n" - "\t\t\t\tlink_speed..............0x%X\n" - "\t\t\t\tmtu_smsl................0x%X\n" - "\t\t\t\tvl_cap_init_type........0x%X\n" - "\t\t\t\tvl_high_limit...........0x%X\n" - "\t\t\t\tvl_arb_high_cap.........0x%X\n" - "\t\t\t\tvl_arb_low_cap..........0x%X\n" - "\t\t\t\tinit_rep_mtu_cap........0x%X\n" - "\t\t\t\tvl_stall_life...........0x%X\n" - "\t\t\t\tvl_enforce..............0x%X\n" - "\t\t\t\tm_key_violations........0x%X\n" - "\t\t\t\tp_key_violations........0x%X\n" - "\t\t\t\tq_key_violations........0x%X\n" - "\t\t\t\tguid_cap................0x%X\n" - "\t\t\t\tsubnet_timeout..........0x%X\n" - "\t\t\t\tresp_time_value.........0x%X\n" - "\t\t\t\terror_threshold.........0x%X\n" - "", - cl_ntoh16(p_pir->lid), - p_pir->port_num, - p_pir->resv, - cl_ntoh64( p_pi->m_key ), - cl_ntoh64( p_pi->subnet_prefix ), - cl_ntoh16( p_pi->base_lid ), - cl_ntoh16( p_pi->master_sm_base_lid ), - cl_ntoh32( p_pi->capability_mask ), - cl_ntoh16( p_pi->diag_code ), - cl_ntoh16( p_pi->m_key_lease_period ), - p_pi->local_port_num, - p_pi->link_width_enabled, - p_pi->link_width_supported, - p_pi->link_width_active, - ib_port_info_get_link_speed_sup( p_pi ), - ib_get_port_state_str( ib_port_info_get_port_state( p_pi ) ), - p_pi->state_info2, - ib_port_info_get_mpb( p_pi ), - ib_port_info_get_lmc( p_pi ), - p_pi->link_speed, - p_pi->mtu_smsl, - p_pi->vl_cap, - p_pi->vl_high_limit, - p_pi->vl_arb_high_cap, - p_pi->vl_arb_low_cap, - p_pi->mtu_cap, - p_pi->vl_stall_life, - p_pi->vl_enforce, - cl_ntoh16( p_pi->m_key_violations ), - cl_ntoh16( p_pi->p_key_violations ), - cl_ntoh16( p_pi->q_key_violations ), - p_pi->guid_cap, - ib_port_info_get_timeout( p_pi ), - p_pi->resp_time_value, - p_pi->error_threshold - ); - - /* show the capabilities mask */ - if( p_pi->capability_mask ) - { - osm_dbg_get_capabilities_str( buf, BUF_SIZE, "\t\t\t\t", p_pi ); - osm_log( p_log, log_level, "%s", buf ); - } - } + char buf[BUF_SIZE]; + const ib_port_info_t *const p_pi = &p_pir->port_info; + + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "PortInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tEndPortLid..............0x%X\n" + "\t\t\t\tPortNum.................0x%X\n" + "\t\t\t\tReserved................0x%X\n" + "\t\t\t\tPortInfo dump:\n" + "\t\t\t\tm_key...................0x%016" PRIx64 "\n" + "\t\t\t\tsubnet_prefix...........0x%016" PRIx64 "\n" + "\t\t\t\tbase_lid................0x%X\n" + "\t\t\t\tmaster_sm_base_lid......0x%X\n" + "\t\t\t\tcapability_mask.........0x%X\n" + "\t\t\t\tdiag_code...............0x%X\n" + "\t\t\t\tm_key_lease_period......0x%X\n" + "\t\t\t\tlocal_port_num..........0x%X\n" + "\t\t\t\tlink_width_enabled......0x%X\n" + "\t\t\t\tlink_width_supported....0x%X\n" + "\t\t\t\tlink_width_active.......0x%X\n" + "\t\t\t\tlink_speed_supported....0x%X\n" + "\t\t\t\tport_state..............%s\n" + "\t\t\t\tstate_info2.............0x%X\n" + "\t\t\t\tm_key_protect_bits......0x%X\n" + "\t\t\t\tlmc.....................0x%X\n" + "\t\t\t\tlink_speed..............0x%X\n" + "\t\t\t\tmtu_smsl................0x%X\n" + "\t\t\t\tvl_cap_init_type........0x%X\n" + "\t\t\t\tvl_high_limit...........0x%X\n" + "\t\t\t\tvl_arb_high_cap.........0x%X\n" + "\t\t\t\tvl_arb_low_cap..........0x%X\n" + "\t\t\t\tinit_rep_mtu_cap........0x%X\n" + "\t\t\t\tvl_stall_life...........0x%X\n" + "\t\t\t\tvl_enforce..............0x%X\n" + "\t\t\t\tm_key_violations........0x%X\n" + "\t\t\t\tp_key_violations........0x%X\n" + "\t\t\t\tq_key_violations........0x%X\n" + "\t\t\t\tguid_cap................0x%X\n" + "\t\t\t\tsubnet_timeout..........0x%X\n" + "\t\t\t\tresp_time_value.........0x%X\n" + "\t\t\t\terror_threshold.........0x%X\n" + "", + cl_ntoh16(p_pir->lid), + p_pir->port_num, + p_pir->resv, + cl_ntoh64(p_pi->m_key), + cl_ntoh64(p_pi->subnet_prefix), + cl_ntoh16(p_pi->base_lid), + cl_ntoh16(p_pi->master_sm_base_lid), + cl_ntoh32(p_pi->capability_mask), + cl_ntoh16(p_pi->diag_code), + cl_ntoh16(p_pi->m_key_lease_period), + p_pi->local_port_num, + p_pi->link_width_enabled, + p_pi->link_width_supported, + p_pi->link_width_active, + ib_port_info_get_link_speed_sup(p_pi), + ib_get_port_state_str(ib_port_info_get_port_state + (p_pi)), p_pi->state_info2, + ib_port_info_get_mpb(p_pi), ib_port_info_get_lmc(p_pi), + p_pi->link_speed, p_pi->mtu_smsl, p_pi->vl_cap, + p_pi->vl_high_limit, p_pi->vl_arb_high_cap, + p_pi->vl_arb_low_cap, p_pi->mtu_cap, + p_pi->vl_stall_life, p_pi->vl_enforce, + cl_ntoh16(p_pi->m_key_violations), + cl_ntoh16(p_pi->p_key_violations), + cl_ntoh16(p_pi->q_key_violations), p_pi->guid_cap, + ib_port_info_get_timeout(p_pi), p_pi->resp_time_value, + p_pi->error_threshold); + + /* show the capabilities mask */ + if (p_pi->capability_mask) { + osm_dbg_get_capabilities_str(buf, BUF_SIZE, "\t\t\t\t", + p_pi); + osm_log(p_log, log_level, "%s", buf); + } + } } /********************************************************************** **********************************************************************/ void -osm_dump_guidinfo_record( - IN osm_log_t* const p_log, - IN const ib_guidinfo_record_t* const p_gir, - IN const osm_log_level_t log_level ) +osm_dump_guidinfo_record(IN osm_log_t * const p_log, + IN const ib_guidinfo_record_t * const p_gir, + IN const osm_log_level_t log_level) { - const ib_guid_info_t * const p_gi = &p_gir->guid_info; - - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "GUIDInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tLid.....................0x%X\n" - "\t\t\t\tBlockNum................0x%X\n" - "\t\t\t\tReserved................0x%X\n" - "\t\t\t\tGUIDInfo dump:\n" - "\t\t\t\tReserved................0x%X\n" - "\t\t\t\tGUID 0..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 1..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 2..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 3..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 4..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 5..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 6..................0x%016" PRIx64 "\n" - "\t\t\t\tGUID 7..................0x%016" PRIx64 "\n", - cl_ntoh16(p_gir->lid), - p_gir->block_num, - p_gir->resv, - cl_ntoh32(p_gir->reserved), - cl_ntoh64(p_gi->guid[0]), - cl_ntoh64(p_gi->guid[1]), - cl_ntoh64(p_gi->guid[2]), - cl_ntoh64(p_gi->guid[3]), - cl_ntoh64(p_gi->guid[4]), - cl_ntoh64(p_gi->guid[5]), - cl_ntoh64(p_gi->guid[6]), - cl_ntoh64(p_gi->guid[7]) - ); - } + const ib_guid_info_t *const p_gi = &p_gir->guid_info; + + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "GUIDInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tLid.....................0x%X\n" + "\t\t\t\tBlockNum................0x%X\n" + "\t\t\t\tReserved................0x%X\n" + "\t\t\t\tGUIDInfo dump:\n" + "\t\t\t\tReserved................0x%X\n" + "\t\t\t\tGUID 0..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 1..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 2..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 3..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 4..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 5..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 6..................0x%016" PRIx64 "\n" + "\t\t\t\tGUID 7..................0x%016" PRIx64 "\n", + cl_ntoh16(p_gir->lid), + p_gir->block_num, + p_gir->resv, + cl_ntoh32(p_gir->reserved), + cl_ntoh64(p_gi->guid[0]), + cl_ntoh64(p_gi->guid[1]), + cl_ntoh64(p_gi->guid[2]), + cl_ntoh64(p_gi->guid[3]), + cl_ntoh64(p_gi->guid[4]), + cl_ntoh64(p_gi->guid[5]), + cl_ntoh64(p_gi->guid[6]), cl_ntoh64(p_gi->guid[7]) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_node_info( - IN osm_log_t* const p_log, - IN const ib_node_info_t* const p_ni, - IN const osm_log_level_t log_level ) +osm_dump_node_info(IN osm_log_t * const p_log, + IN const ib_node_info_t * const p_ni, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "NodeInfo dump:\n" - "\t\t\t\tbase_version............0x%X\n" - "\t\t\t\tclass_version...........0x%X\n" - "\t\t\t\tnode_type...............%s\n" - "\t\t\t\tnum_ports...............0x%X\n" - "\t\t\t\tsys_guid................0x%016" PRIx64 "\n" - "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tpartition_cap...........0x%X\n" - "\t\t\t\tdevice_id...............0x%X\n" - "\t\t\t\trevision................0x%X\n" - "\t\t\t\tport_num................0x%X\n" - "\t\t\t\tvendor_id...............0x%X\n" - "", - p_ni->base_version, - p_ni->class_version, - ib_get_node_type_str( p_ni->node_type ), - p_ni->num_ports, - cl_ntoh64( p_ni->sys_guid ), - cl_ntoh64( p_ni->node_guid ), - cl_ntoh64( p_ni->port_guid ), - cl_ntoh16( p_ni->partition_cap ), - cl_ntoh16( p_ni->device_id ), - cl_ntoh32( p_ni->revision ), - ib_node_info_get_local_port_num( p_ni ), - cl_ntoh32( ib_node_info_get_vendor_id( p_ni ) ) - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "NodeInfo dump:\n" + "\t\t\t\tbase_version............0x%X\n" + "\t\t\t\tclass_version...........0x%X\n" + "\t\t\t\tnode_type...............%s\n" + "\t\t\t\tnum_ports...............0x%X\n" + "\t\t\t\tsys_guid................0x%016" PRIx64 "\n" + "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tpartition_cap...........0x%X\n" + "\t\t\t\tdevice_id...............0x%X\n" + "\t\t\t\trevision................0x%X\n" + "\t\t\t\tport_num................0x%X\n" + "\t\t\t\tvendor_id...............0x%X\n" + "", + p_ni->base_version, + p_ni->class_version, + ib_get_node_type_str(p_ni->node_type), + p_ni->num_ports, + cl_ntoh64(p_ni->sys_guid), + cl_ntoh64(p_ni->node_guid), + cl_ntoh64(p_ni->port_guid), + cl_ntoh16(p_ni->partition_cap), + cl_ntoh16(p_ni->device_id), + cl_ntoh32(p_ni->revision), + ib_node_info_get_local_port_num(p_ni), + cl_ntoh32(ib_node_info_get_vendor_id(p_ni)) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_node_record( - IN osm_log_t* const p_log, - IN const ib_node_record_t* const p_nr, - IN const osm_log_level_t log_level ) +osm_dump_node_record(IN osm_log_t * const p_log, + IN const ib_node_record_t * const p_nr, + IN const osm_log_level_t log_level) { - const ib_node_info_t * const p_ni = &p_nr->node_info; - - if( osm_log_is_active( p_log, log_level ) ) - { - char desc[sizeof(p_nr->node_desc.description) + 1]; - - memcpy(desc, p_nr->node_desc.description, - sizeof(p_nr->node_desc.description)); - desc[sizeof(desc) - 1] = '\0'; - osm_log( p_log, log_level, - "Node Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tLid.....................0x%X\n" - "\t\t\t\tReserved................0x%X\n" - "\t\t\t\tNodeInfo dump:\n" - "\t\t\t\tbase_version............0x%X\n" - "\t\t\t\tclass_version...........0x%X\n" - "\t\t\t\tnode_type...............%s\n" - "\t\t\t\tnum_ports...............0x%X\n" - "\t\t\t\tsys_guid................0x%016" PRIx64 "\n" - "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" - "\t\t\t\tpartition_cap...........0x%X\n" - "\t\t\t\tdevice_id...............0x%X\n" - "\t\t\t\trevision................0x%X\n" - "\t\t\t\tport_num................0x%X\n" - "\t\t\t\tvendor_id...............0x%X\n" - "\t\t\t\tNodeDescription\n" - "\t\t\t\t%s\n" - "", - cl_ntoh16(p_nr->lid), - cl_ntoh16(p_nr->resv), - p_ni->base_version, - p_ni->class_version, - ib_get_node_type_str( p_ni->node_type ), - p_ni->num_ports, - cl_ntoh64( p_ni->sys_guid ), - cl_ntoh64( p_ni->node_guid ), - cl_ntoh64( p_ni->port_guid ), - cl_ntoh16( p_ni->partition_cap ), - cl_ntoh16( p_ni->device_id ), - cl_ntoh32( p_ni->revision ), - ib_node_info_get_local_port_num( p_ni ), - cl_ntoh32( ib_node_info_get_vendor_id( p_ni )), - desc - ); - } + const ib_node_info_t *const p_ni = &p_nr->node_info; + + if (osm_log_is_active(p_log, log_level)) { + char desc[sizeof(p_nr->node_desc.description) + 1]; + + memcpy(desc, p_nr->node_desc.description, + sizeof(p_nr->node_desc.description)); + desc[sizeof(desc) - 1] = '\0'; + osm_log(p_log, log_level, + "Node Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tLid.....................0x%X\n" + "\t\t\t\tReserved................0x%X\n" + "\t\t\t\tNodeInfo dump:\n" + "\t\t\t\tbase_version............0x%X\n" + "\t\t\t\tclass_version...........0x%X\n" + "\t\t\t\tnode_type...............%s\n" + "\t\t\t\tnum_ports...............0x%X\n" + "\t\t\t\tsys_guid................0x%016" PRIx64 "\n" + "\t\t\t\tnode_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tport_guid...............0x%016" PRIx64 "\n" + "\t\t\t\tpartition_cap...........0x%X\n" + "\t\t\t\tdevice_id...............0x%X\n" + "\t\t\t\trevision................0x%X\n" + "\t\t\t\tport_num................0x%X\n" + "\t\t\t\tvendor_id...............0x%X\n" + "\t\t\t\tNodeDescription\n" + "\t\t\t\t%s\n" + "", + cl_ntoh16(p_nr->lid), + cl_ntoh16(p_nr->resv), + p_ni->base_version, + p_ni->class_version, + ib_get_node_type_str(p_ni->node_type), + p_ni->num_ports, + cl_ntoh64(p_ni->sys_guid), + cl_ntoh64(p_ni->node_guid), + cl_ntoh64(p_ni->port_guid), + cl_ntoh16(p_ni->partition_cap), + cl_ntoh16(p_ni->device_id), + cl_ntoh32(p_ni->revision), + ib_node_info_get_local_port_num(p_ni), + cl_ntoh32(ib_node_info_get_vendor_id(p_ni)), desc); + } } /********************************************************************** **********************************************************************/ void -osm_dump_path_record( - IN osm_log_t* const p_log, - IN const ib_path_rec_t* const p_pr, - IN const osm_log_level_t log_level ) +osm_dump_path_record(IN osm_log_t * const p_log, + IN const ib_path_rec_t * const p_pr, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "PathRecord dump:\n" - "\t\t\t\tresv0...................0x%016" PRIx64 "\n" - "\t\t\t\tdgid....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tsgid....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tdlid....................0x%X\n" - "\t\t\t\tslid....................0x%X\n" - "\t\t\t\thop_flow_raw............0x%X\n" - "\t\t\t\ttclass..................0x%X\n" - "\t\t\t\tnum_path_revers.........0x%X\n" - "\t\t\t\tpkey....................0x%X\n" - "\t\t\t\tsl......................0x%X\n" - "\t\t\t\tmtu.....................0x%X\n" - "\t\t\t\trate....................0x%X\n" - "\t\t\t\tpkt_life................0x%X\n" - "\t\t\t\tpreference..............0x%X\n" - "\t\t\t\tresv2...................0x%X\n" - "\t\t\t\tresv3...................0x%X\n" - "", - *(uint64_t*)p_pr->resv0, - cl_ntoh64( p_pr->dgid.unicast.prefix ), - cl_ntoh64( p_pr->dgid.unicast.interface_id ), - cl_ntoh64( p_pr->sgid.unicast.prefix ), - cl_ntoh64( p_pr->sgid.unicast.interface_id ), - cl_ntoh16( p_pr->dlid ), - cl_ntoh16( p_pr->slid ), - cl_ntoh32( p_pr->hop_flow_raw ), - p_pr->tclass, - p_pr->num_path, - cl_ntoh16( p_pr->pkey ), - cl_ntoh16( p_pr->sl ), - p_pr->mtu, - p_pr->rate, - p_pr->pkt_life, - p_pr->preference, - *(uint32_t*)&p_pr->resv2, - *((uint16_t*)&p_pr->resv2 + 2) - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "PathRecord dump:\n" + "\t\t\t\tresv0...................0x%016" PRIx64 "\n" + "\t\t\t\tdgid....................0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n" + "\t\t\t\tsgid....................0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n" + "\t\t\t\tdlid....................0x%X\n" + "\t\t\t\tslid....................0x%X\n" + "\t\t\t\thop_flow_raw............0x%X\n" + "\t\t\t\ttclass..................0x%X\n" + "\t\t\t\tnum_path_revers.........0x%X\n" + "\t\t\t\tpkey....................0x%X\n" + "\t\t\t\tsl......................0x%X\n" + "\t\t\t\tmtu.....................0x%X\n" + "\t\t\t\trate....................0x%X\n" + "\t\t\t\tpkt_life................0x%X\n" + "\t\t\t\tpreference..............0x%X\n" + "\t\t\t\tresv2...................0x%X\n" + "\t\t\t\tresv3...................0x%X\n" + "", + *(uint64_t *) p_pr->resv0, + cl_ntoh64(p_pr->dgid.unicast.prefix), + cl_ntoh64(p_pr->dgid.unicast.interface_id), + cl_ntoh64(p_pr->sgid.unicast.prefix), + cl_ntoh64(p_pr->sgid.unicast.interface_id), + cl_ntoh16(p_pr->dlid), + cl_ntoh16(p_pr->slid), + cl_ntoh32(p_pr->hop_flow_raw), + p_pr->tclass, + p_pr->num_path, + cl_ntoh16(p_pr->pkey), + cl_ntoh16(p_pr->sl), + p_pr->mtu, + p_pr->rate, + p_pr->pkt_life, + p_pr->preference, + *(uint32_t *) & p_pr->resv2, + *((uint16_t *) & p_pr->resv2 + 2) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_multipath_record( - IN osm_log_t* const p_log, - IN const ib_multipath_rec_t* const p_mpr, - IN const osm_log_level_t log_level ) +osm_dump_multipath_record(IN osm_log_t * const p_log, + IN const ib_multipath_rec_t * const p_mpr, + IN const osm_log_level_t log_level) { - char buf_line[1024]; - ib_gid_t const *p_gid; - int i, n; - - if( osm_log_is_active( p_log, log_level ) ) - { - n = 0; - p_gid = p_mpr->gids; - if ( p_mpr->sgid_count ) - { - for (i = 0; i < p_mpr->sgid_count; i++) - { - n += sprintf( buf_line + n, "\t\t\t\tsgid%02d.................." - "0x%016" PRIx64 " : 0x%016" PRIx64 "\n", - i + 1, cl_ntoh64( p_gid->unicast.prefix ), - cl_ntoh64( p_gid->unicast.interface_id ) ); - p_gid++; - } - } - if ( p_mpr->dgid_count ) - { - for (i = 0; i < p_mpr->dgid_count; i++) - { - n += sprintf( buf_line + n, "\t\t\t\tdgid%02d.................." - "0x%016" PRIx64 " : 0x%016" PRIx64 "\n", - i + 1, cl_ntoh64( p_gid->unicast.prefix ), - cl_ntoh64( p_gid->unicast.interface_id ) ); - p_gid++; - } - } - osm_log( p_log, log_level, - "MultiPath Record dump:\n" - "\t\t\t\thop_flow_raw............0x%X\n" - "\t\t\t\ttclass..................0x%X\n" - "\t\t\t\tnum_path_revers.........0x%X\n" - "\t\t\t\tpkey....................0x%X\n" - "\t\t\t\tresv0...................0x%X\n" - "\t\t\t\tsl......................0x%X\n" - "\t\t\t\tmtu.....................0x%X\n" - "\t\t\t\trate....................0x%X\n" - "\t\t\t\tpkt_life................0x%X\n" - "\t\t\t\tresv1...................0x%X\n" - "\t\t\t\tindependence............0x%X\n" - "\t\t\t\tsgid_count..............0x%X\n" - "\t\t\t\tdgid_count..............0x%X\n" - "%s\n" - "", - cl_ntoh32( p_mpr->hop_flow_raw ), - p_mpr->tclass, - p_mpr->num_path, - cl_ntoh16( p_mpr->pkey ), - p_mpr->resv0, - cl_ntoh16( p_mpr->sl ), - p_mpr->mtu, - p_mpr->rate, - p_mpr->pkt_life, - p_mpr->resv1, - p_mpr->independence, - p_mpr->sgid_count, - p_mpr->dgid_count, - buf_line - ); - } + char buf_line[1024]; + ib_gid_t const *p_gid; + int i, n; + + if (osm_log_is_active(p_log, log_level)) { + n = 0; + p_gid = p_mpr->gids; + if (p_mpr->sgid_count) { + for (i = 0; i < p_mpr->sgid_count; i++) { + n += sprintf(buf_line + n, + "\t\t\t\tsgid%02d.................." + "0x%016" PRIx64 " : 0x%016" PRIx64 + "\n", i + 1, + cl_ntoh64(p_gid->unicast.prefix), + cl_ntoh64(p_gid->unicast. + interface_id)); + p_gid++; + } + } + if (p_mpr->dgid_count) { + for (i = 0; i < p_mpr->dgid_count; i++) { + n += sprintf(buf_line + n, + "\t\t\t\tdgid%02d.................." + "0x%016" PRIx64 " : 0x%016" PRIx64 + "\n", i + 1, + cl_ntoh64(p_gid->unicast.prefix), + cl_ntoh64(p_gid->unicast. + interface_id)); + p_gid++; + } + } + osm_log(p_log, log_level, + "MultiPath Record dump:\n" + "\t\t\t\thop_flow_raw............0x%X\n" + "\t\t\t\ttclass..................0x%X\n" + "\t\t\t\tnum_path_revers.........0x%X\n" + "\t\t\t\tpkey....................0x%X\n" + "\t\t\t\tresv0...................0x%X\n" + "\t\t\t\tsl......................0x%X\n" + "\t\t\t\tmtu.....................0x%X\n" + "\t\t\t\trate....................0x%X\n" + "\t\t\t\tpkt_life................0x%X\n" + "\t\t\t\tresv1...................0x%X\n" + "\t\t\t\tindependence............0x%X\n" + "\t\t\t\tsgid_count..............0x%X\n" + "\t\t\t\tdgid_count..............0x%X\n" + "%s\n" + "", + cl_ntoh32(p_mpr->hop_flow_raw), + p_mpr->tclass, + p_mpr->num_path, + cl_ntoh16(p_mpr->pkey), + p_mpr->resv0, + cl_ntoh16(p_mpr->sl), + p_mpr->mtu, + p_mpr->rate, + p_mpr->pkt_life, + p_mpr->resv1, + p_mpr->independence, + p_mpr->sgid_count, p_mpr->dgid_count, buf_line); + } } /********************************************************************** **********************************************************************/ void -osm_dump_mc_record( - IN osm_log_t* const p_log, - IN const ib_member_rec_t* const p_mcmr, - IN const osm_log_level_t log_level ) +osm_dump_mc_record(IN osm_log_t * const p_log, + IN const ib_member_rec_t * const p_mcmr, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "MCMember Record dump:\n" - "\t\t\t\tMGID....................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tPortGid.................0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tqkey....................0x%X\n" - "\t\t\t\tmlid....................0x%X\n" - "\t\t\t\tmtu.....................0x%X\n" - "\t\t\t\tTClass..................0x%X\n" - "\t\t\t\tpkey....................0x%X\n" - "\t\t\t\trate....................0x%X\n" - "\t\t\t\tpkt_life................0x%X\n" - "\t\t\t\tSLFlowLabelHopLimit.....0x%X\n" - "\t\t\t\tScopeState..............0x%X\n" - "\t\t\t\tProxyJoin...............0x%X\n" - "", - cl_ntoh64( p_mcmr->mgid.unicast.prefix ), - cl_ntoh64( p_mcmr->mgid.unicast.interface_id ), - cl_ntoh64( p_mcmr->port_gid.unicast.prefix ), - cl_ntoh64( p_mcmr->port_gid.unicast.interface_id ), - cl_ntoh32( p_mcmr->qkey ), - cl_ntoh16( p_mcmr->mlid ), - p_mcmr->mtu, - p_mcmr->tclass, - cl_ntoh16( p_mcmr->pkey ), - p_mcmr->rate, - p_mcmr->pkt_life, - cl_ntoh32( p_mcmr->sl_flow_hop ), - p_mcmr->scope_state, - p_mcmr->proxy_join - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "MCMember Record dump:\n" + "\t\t\t\tMGID....................0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n" + "\t\t\t\tPortGid.................0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n" + "\t\t\t\tqkey....................0x%X\n" + "\t\t\t\tmlid....................0x%X\n" + "\t\t\t\tmtu.....................0x%X\n" + "\t\t\t\tTClass..................0x%X\n" + "\t\t\t\tpkey....................0x%X\n" + "\t\t\t\trate....................0x%X\n" + "\t\t\t\tpkt_life................0x%X\n" + "\t\t\t\tSLFlowLabelHopLimit.....0x%X\n" + "\t\t\t\tScopeState..............0x%X\n" + "\t\t\t\tProxyJoin...............0x%X\n" + "", + cl_ntoh64(p_mcmr->mgid.unicast.prefix), + cl_ntoh64(p_mcmr->mgid.unicast.interface_id), + cl_ntoh64(p_mcmr->port_gid.unicast.prefix), + cl_ntoh64(p_mcmr->port_gid.unicast.interface_id), + cl_ntoh32(p_mcmr->qkey), + cl_ntoh16(p_mcmr->mlid), + p_mcmr->mtu, + p_mcmr->tclass, + cl_ntoh16(p_mcmr->pkey), + p_mcmr->rate, + p_mcmr->pkt_life, + cl_ntoh32(p_mcmr->sl_flow_hop), + p_mcmr->scope_state, p_mcmr->proxy_join); + } } /********************************************************************** **********************************************************************/ void -osm_dump_service_record( - IN osm_log_t* const p_log, - IN const ib_service_record_t* const p_sr, - IN const osm_log_level_t log_level ) +osm_dump_service_record(IN osm_log_t * const p_log, + IN const ib_service_record_t * const p_sr, + IN const osm_log_level_t log_level) { - char buf_service_key[35]; - char buf_service_name[65]; - - if( osm_log_is_active( p_log, log_level ) ) - { - sprintf(buf_service_key, - "0x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x", - p_sr->service_key[0], - p_sr->service_key[1], - p_sr->service_key[2], - p_sr->service_key[3], - p_sr->service_key[4], - p_sr->service_key[5], - p_sr->service_key[6], - p_sr->service_key[7], - p_sr->service_key[8], - p_sr->service_key[9], - p_sr->service_key[10], - p_sr->service_key[11], - p_sr->service_key[12], - p_sr->service_key[13], - p_sr->service_key[14], - p_sr->service_key[15]); - strncpy(buf_service_name, (char *)p_sr->service_name, 64); - buf_service_name[64] = '\0'; - - osm_log( p_log, log_level, - "Service Record dump:\n" - "\t\t\t\tServiceID...............0x%016" PRIx64 "\n" - "\t\t\t\tServiceGID..............0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tServiceP_Key............0x%X\n" - "\t\t\t\tServiceLease............0x%X\n" - "\t\t\t\tServiceKey..............%s\n" - "\t\t\t\tServiceName.............%s\n" - "\t\t\t\tServiceData8.1..........0x%X\n" - "\t\t\t\tServiceData8.2..........0x%X\n" - "\t\t\t\tServiceData8.3..........0x%X\n" - "\t\t\t\tServiceData8.4..........0x%X\n" - "\t\t\t\tServiceData8.5..........0x%X\n" - "\t\t\t\tServiceData8.6..........0x%X\n" - "\t\t\t\tServiceData8.7..........0x%X\n" - "\t\t\t\tServiceData8.8..........0x%X\n" - "\t\t\t\tServiceData8.9..........0x%X\n" - "\t\t\t\tServiceData8.10.........0x%X\n" - "\t\t\t\tServiceData8.11.........0x%X\n" - "\t\t\t\tServiceData8.12.........0x%X\n" - "\t\t\t\tServiceData8.13.........0x%X\n" - "\t\t\t\tServiceData8.14.........0x%X\n" - "\t\t\t\tServiceData8.15.........0x%X\n" - "\t\t\t\tServiceData8.16.........0x%X\n" - "\t\t\t\tServiceData16.1.........0x%X\n" - "\t\t\t\tServiceData16.2.........0x%X\n" - "\t\t\t\tServiceData16.3.........0x%X\n" - "\t\t\t\tServiceData16.4.........0x%X\n" - "\t\t\t\tServiceData16.5.........0x%X\n" - "\t\t\t\tServiceData16.6.........0x%X\n" - "\t\t\t\tServiceData16.7.........0x%X\n" - "\t\t\t\tServiceData16.8.........0x%X\n" - "\t\t\t\tServiceData32.1.........0x%X\n" - "\t\t\t\tServiceData32.2.........0x%X\n" - "\t\t\t\tServiceData32.3.........0x%X\n" - "\t\t\t\tServiceData32.4.........0x%X\n" - "\t\t\t\tServiceData64.1.........0x%016" PRIx64 "\n" - "\t\t\t\tServiceData64.2.........0x%016" PRIx64 "\n" - "", - cl_ntoh64( p_sr->service_id ), - cl_ntoh64( p_sr->service_gid.unicast.prefix ), - cl_ntoh64( p_sr->service_gid.unicast.interface_id ), - cl_ntoh16( p_sr->service_pkey ), - cl_ntoh32( p_sr->service_lease ), - buf_service_key, - buf_service_name, - p_sr->service_data8[0], p_sr->service_data8[1], - p_sr->service_data8[2], p_sr->service_data8[3], - p_sr->service_data8[4], p_sr->service_data8[5], - p_sr->service_data8[6], p_sr->service_data8[7], - p_sr->service_data8[8], p_sr->service_data8[9], - p_sr->service_data8[10], p_sr->service_data8[11], - p_sr->service_data8[12], p_sr->service_data8[13], - p_sr->service_data8[14], p_sr->service_data8[15], - cl_ntoh16(p_sr->service_data16[0]), - cl_ntoh16(p_sr->service_data16[1]), - cl_ntoh16(p_sr->service_data16[2]), - cl_ntoh16(p_sr->service_data16[3]), - cl_ntoh16(p_sr->service_data16[4]), - cl_ntoh16(p_sr->service_data16[5]), - cl_ntoh16(p_sr->service_data16[6]), - cl_ntoh16(p_sr->service_data16[7]), - cl_ntoh32(p_sr->service_data32[0]), - cl_ntoh32(p_sr->service_data32[1]), - cl_ntoh32(p_sr->service_data32[2]), - cl_ntoh32(p_sr->service_data32[3]), - cl_ntoh64(p_sr->service_data64[0]), - cl_ntoh64(p_sr->service_data64[1]) - ); - } + char buf_service_key[35]; + char buf_service_name[65]; + + if (osm_log_is_active(p_log, log_level)) { + sprintf(buf_service_key, + "0x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x", + p_sr->service_key[0], + p_sr->service_key[1], + p_sr->service_key[2], + p_sr->service_key[3], + p_sr->service_key[4], + p_sr->service_key[5], + p_sr->service_key[6], + p_sr->service_key[7], + p_sr->service_key[8], + p_sr->service_key[9], + p_sr->service_key[10], + p_sr->service_key[11], + p_sr->service_key[12], + p_sr->service_key[13], + p_sr->service_key[14], p_sr->service_key[15]); + strncpy(buf_service_name, (char *)p_sr->service_name, 64); + buf_service_name[64] = '\0'; + + osm_log(p_log, log_level, + "Service Record dump:\n" + "\t\t\t\tServiceID...............0x%016" PRIx64 "\n" + "\t\t\t\tServiceGID..............0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n" + "\t\t\t\tServiceP_Key............0x%X\n" + "\t\t\t\tServiceLease............0x%X\n" + "\t\t\t\tServiceKey..............%s\n" + "\t\t\t\tServiceName.............%s\n" + "\t\t\t\tServiceData8.1..........0x%X\n" + "\t\t\t\tServiceData8.2..........0x%X\n" + "\t\t\t\tServiceData8.3..........0x%X\n" + "\t\t\t\tServiceData8.4..........0x%X\n" + "\t\t\t\tServiceData8.5..........0x%X\n" + "\t\t\t\tServiceData8.6..........0x%X\n" + "\t\t\t\tServiceData8.7..........0x%X\n" + "\t\t\t\tServiceData8.8..........0x%X\n" + "\t\t\t\tServiceData8.9..........0x%X\n" + "\t\t\t\tServiceData8.10.........0x%X\n" + "\t\t\t\tServiceData8.11.........0x%X\n" + "\t\t\t\tServiceData8.12.........0x%X\n" + "\t\t\t\tServiceData8.13.........0x%X\n" + "\t\t\t\tServiceData8.14.........0x%X\n" + "\t\t\t\tServiceData8.15.........0x%X\n" + "\t\t\t\tServiceData8.16.........0x%X\n" + "\t\t\t\tServiceData16.1.........0x%X\n" + "\t\t\t\tServiceData16.2.........0x%X\n" + "\t\t\t\tServiceData16.3.........0x%X\n" + "\t\t\t\tServiceData16.4.........0x%X\n" + "\t\t\t\tServiceData16.5.........0x%X\n" + "\t\t\t\tServiceData16.6.........0x%X\n" + "\t\t\t\tServiceData16.7.........0x%X\n" + "\t\t\t\tServiceData16.8.........0x%X\n" + "\t\t\t\tServiceData32.1.........0x%X\n" + "\t\t\t\tServiceData32.2.........0x%X\n" + "\t\t\t\tServiceData32.3.........0x%X\n" + "\t\t\t\tServiceData32.4.........0x%X\n" + "\t\t\t\tServiceData64.1.........0x%016" PRIx64 "\n" + "\t\t\t\tServiceData64.2.........0x%016" PRIx64 "\n" + "", + cl_ntoh64(p_sr->service_id), + cl_ntoh64(p_sr->service_gid.unicast.prefix), + cl_ntoh64(p_sr->service_gid.unicast.interface_id), + cl_ntoh16(p_sr->service_pkey), + cl_ntoh32(p_sr->service_lease), + buf_service_key, + buf_service_name, + p_sr->service_data8[0], p_sr->service_data8[1], + p_sr->service_data8[2], p_sr->service_data8[3], + p_sr->service_data8[4], p_sr->service_data8[5], + p_sr->service_data8[6], p_sr->service_data8[7], + p_sr->service_data8[8], p_sr->service_data8[9], + p_sr->service_data8[10], p_sr->service_data8[11], + p_sr->service_data8[12], p_sr->service_data8[13], + p_sr->service_data8[14], p_sr->service_data8[15], + cl_ntoh16(p_sr->service_data16[0]), + cl_ntoh16(p_sr->service_data16[1]), + cl_ntoh16(p_sr->service_data16[2]), + cl_ntoh16(p_sr->service_data16[3]), + cl_ntoh16(p_sr->service_data16[4]), + cl_ntoh16(p_sr->service_data16[5]), + cl_ntoh16(p_sr->service_data16[6]), + cl_ntoh16(p_sr->service_data16[7]), + cl_ntoh32(p_sr->service_data32[0]), + cl_ntoh32(p_sr->service_data32[1]), + cl_ntoh32(p_sr->service_data32[2]), + cl_ntoh32(p_sr->service_data32[3]), + cl_ntoh64(p_sr->service_data64[0]), + cl_ntoh64(p_sr->service_data64[1]) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_inform_info( - IN osm_log_t* const p_log, - IN const ib_inform_info_t* const p_ii, - IN const osm_log_level_t log_level ) +osm_dump_inform_info(IN osm_log_t * const p_log, + IN const ib_inform_info_t * const p_ii, + IN const osm_log_level_t log_level) { - uint32_t qpn; - uint8_t resp_time_val; - - if( osm_log_is_active( p_log, log_level ) ) - { - - ib_inform_info_get_qpn_resp_time(p_ii->g_or_v.generic.qpn_resp_time_val, - &qpn, &resp_time_val); - - if (p_ii->is_generic) - { - osm_log( p_log, log_level, - "InformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" - "\t\t\t\tlid_range_begin.........0x%X\n" - "\t\t\t\tlid_range_end...........0x%X\n" - "\t\t\t\tis_generic..............0x%X\n" - "\t\t\t\tsubscribe...............0x%X\n" - "\t\t\t\ttrap_type...............0x%X\n" - "\t\t\t\ttrap_num................%u\n" - "\t\t\t\tqpn.....................0x%06X\n" - "\t\t\t\tresp_time_val...........0x%X\n" - "\t\t\t\tnode_type...............0x%06X\n" - "", - cl_ntoh64( p_ii->gid.unicast.prefix ), - cl_ntoh64( p_ii->gid.unicast.interface_id ), - cl_ntoh16( p_ii->lid_range_begin ), - cl_ntoh16( p_ii->lid_range_end ), - p_ii->is_generic, - p_ii->subscribe, - cl_ntoh16( p_ii->trap_type ), - cl_ntoh16( p_ii->g_or_v.generic.trap_num ), - cl_ntoh32( qpn ), - resp_time_val, - cl_ntoh32(ib_inform_info_get_prod_type( p_ii )) - ); - } - else - { - osm_log( p_log, log_level, - "InformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" - "\t\t\t\tlid_range_begin.........0x%X\n" - "\t\t\t\tlid_range_end...........0x%X\n" - "\t\t\t\tis_generic..............0x%X\n" - "\t\t\t\tsubscribe...............0x%X\n" - "\t\t\t\ttrap_type...............0x%X\n" - "\t\t\t\tdev_id..................0x%X\n" - "\t\t\t\tqpn.....................0x%06X\n" - "\t\t\t\tresp_time_val...........0x%X\n" - "\t\t\t\tvendor_id...............0x%06X\n" - "", - cl_ntoh64( p_ii->gid.unicast.prefix ), - cl_ntoh64( p_ii->gid.unicast.interface_id ), - cl_ntoh16( p_ii->lid_range_begin ), - cl_ntoh16( p_ii->lid_range_end ), - p_ii->is_generic, - p_ii->subscribe, - cl_ntoh16( p_ii->trap_type ), - cl_ntoh16( p_ii->g_or_v.vend.dev_id ), - cl_ntoh32( qpn ), - resp_time_val, - cl_ntoh32(ib_inform_info_get_prod_type( p_ii )) - ); - } - } + uint32_t qpn; + uint8_t resp_time_val; + + if (osm_log_is_active(p_log, log_level)) { + + ib_inform_info_get_qpn_resp_time(p_ii->g_or_v.generic. + qpn_resp_time_val, &qpn, + &resp_time_val); + + if (p_ii->is_generic) { + osm_log(p_log, log_level, + "InformInfo dump:\n" + "\t\t\t\tgid.....................0x%016" PRIx64 + " : 0x%016" PRIx64 "\n" + "\t\t\t\tlid_range_begin.........0x%X\n" + "\t\t\t\tlid_range_end...........0x%X\n" + "\t\t\t\tis_generic..............0x%X\n" + "\t\t\t\tsubscribe...............0x%X\n" + "\t\t\t\ttrap_type...............0x%X\n" + "\t\t\t\ttrap_num................%u\n" + "\t\t\t\tqpn.....................0x%06X\n" + "\t\t\t\tresp_time_val...........0x%X\n" + "\t\t\t\tnode_type...............0x%06X\n" "", + cl_ntoh64(p_ii->gid.unicast.prefix), + cl_ntoh64(p_ii->gid.unicast.interface_id), + cl_ntoh16(p_ii->lid_range_begin), + cl_ntoh16(p_ii->lid_range_end), + p_ii->is_generic, p_ii->subscribe, + cl_ntoh16(p_ii->trap_type), + cl_ntoh16(p_ii->g_or_v.generic.trap_num), + cl_ntoh32(qpn), resp_time_val, + cl_ntoh32(ib_inform_info_get_prod_type(p_ii)) + ); + } else { + osm_log(p_log, log_level, + "InformInfo dump:\n" + "\t\t\t\tgid.....................0x%016" PRIx64 + " : 0x%016" PRIx64 "\n" + "\t\t\t\tlid_range_begin.........0x%X\n" + "\t\t\t\tlid_range_end...........0x%X\n" + "\t\t\t\tis_generic..............0x%X\n" + "\t\t\t\tsubscribe...............0x%X\n" + "\t\t\t\ttrap_type...............0x%X\n" + "\t\t\t\tdev_id..................0x%X\n" + "\t\t\t\tqpn.....................0x%06X\n" + "\t\t\t\tresp_time_val...........0x%X\n" + "\t\t\t\tvendor_id...............0x%06X\n" "", + cl_ntoh64(p_ii->gid.unicast.prefix), + cl_ntoh64(p_ii->gid.unicast.interface_id), + cl_ntoh16(p_ii->lid_range_begin), + cl_ntoh16(p_ii->lid_range_end), + p_ii->is_generic, p_ii->subscribe, + cl_ntoh16(p_ii->trap_type), + cl_ntoh16(p_ii->g_or_v.vend.dev_id), + cl_ntoh32(qpn), resp_time_val, + cl_ntoh32(ib_inform_info_get_prod_type(p_ii)) + ); + } + } } /********************************************************************** **********************************************************************/ void -osm_dump_inform_info_record( - IN osm_log_t* const p_log, - IN const ib_inform_info_record_t* const p_iir, - IN const osm_log_level_t log_level ) +osm_dump_inform_info_record(IN osm_log_t * const p_log, + IN const ib_inform_info_record_t * const p_iir, + IN const osm_log_level_t log_level) { - uint32_t qpn; - uint8_t resp_time_val; - - if( osm_log_is_active( p_log, log_level ) ) - { - - ib_inform_info_get_qpn_resp_time(p_iir->inform_info.g_or_v.generic.qpn_resp_time_val, - &qpn, &resp_time_val); - - if (p_iir->inform_info.is_generic) - { - osm_log( p_log, log_level, - "InformInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tSubscriberEnum..........0x%X\n" - "\t\t\t\tInformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" - "\t\t\t\tlid_range_begin.........0x%X\n" - "\t\t\t\tlid_range_end...........0x%X\n" - "\t\t\t\tis_generic..............0x%X\n" - "\t\t\t\tsubscribe...............0x%X\n" - "\t\t\t\ttrap_type...............0x%X\n" - "\t\t\t\ttrap_num................%u\n" - "\t\t\t\tqpn.....................0x%06X\n" - "\t\t\t\tresp_time_val...........0x%X\n" - "\t\t\t\tnode_type...............0x%06X\n" - "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), - cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), - cl_ntoh16( p_iir->inform_info.lid_range_begin ), - cl_ntoh16( p_iir->inform_info.lid_range_end ), - p_iir->inform_info.is_generic, - p_iir->inform_info.subscribe, - cl_ntoh16( p_iir->inform_info.trap_type ), - cl_ntoh16( p_iir->inform_info.g_or_v.generic.trap_num ), - cl_ntoh32( qpn ), - resp_time_val, - cl_ntoh32(ib_inform_info_get_prod_type( &p_iir->inform_info )) - ); - } - else - { - osm_log( p_log, log_level, - "InformInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tSubscriberGID...........0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n" - "\t\t\t\tSubscriberEnum..........0x%X\n" - "\t\t\t\tInformInfo dump:\n" - "\t\t\t\tgid.....................0x%016" PRIx64 " : 0x%016" PRIx64 "\n" - "\t\t\t\tlid_range_begin.........0x%X\n" - "\t\t\t\tlid_range_end...........0x%X\n" - "\t\t\t\tis_generic..............0x%X\n" - "\t\t\t\tsubscribe...............0x%X\n" - "\t\t\t\ttrap_type...............0x%X\n" - "\t\t\t\tdev_id..................0x%X\n" - "\t\t\t\tqpn.....................0x%06X\n" - "\t\t\t\tresp_time_val...........0x%X\n" - "\t\t\t\tvendor_id...............0x%06X\n" - "", - cl_ntoh64( p_iir->subscriber_gid.unicast.prefix ), - cl_ntoh64( p_iir->subscriber_gid.unicast.interface_id ), - cl_ntoh16( p_iir->subscriber_enum ), - cl_ntoh64( p_iir->inform_info.gid.unicast.prefix ), - cl_ntoh64( p_iir->inform_info.gid.unicast.interface_id ), - cl_ntoh16( p_iir->inform_info.lid_range_begin ), - cl_ntoh16( p_iir->inform_info.lid_range_end ), - p_iir->inform_info.is_generic, - p_iir->inform_info.subscribe, - cl_ntoh16( p_iir->inform_info.trap_type ), - cl_ntoh16( p_iir->inform_info.g_or_v.vend.dev_id ), - cl_ntoh32( qpn ), - resp_time_val, - cl_ntoh32(ib_inform_info_get_prod_type( &p_iir->inform_info )) - ); - } - } + uint32_t qpn; + uint8_t resp_time_val; + + if (osm_log_is_active(p_log, log_level)) { + + ib_inform_info_get_qpn_resp_time(p_iir->inform_info.g_or_v. + generic.qpn_resp_time_val, + &qpn, &resp_time_val); + + if (p_iir->inform_info.is_generic) { + osm_log(p_log, log_level, + "InformInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tSubscriberGID...........0x%016" PRIx64 + " : " "0x%016" PRIx64 "\n" + "\t\t\t\tSubscriberEnum..........0x%X\n" + "\t\t\t\tInformInfo dump:\n" + "\t\t\t\tgid.....................0x%016" PRIx64 + " : 0x%016" PRIx64 "\n" + "\t\t\t\tlid_range_begin.........0x%X\n" + "\t\t\t\tlid_range_end...........0x%X\n" + "\t\t\t\tis_generic..............0x%X\n" + "\t\t\t\tsubscribe...............0x%X\n" + "\t\t\t\ttrap_type...............0x%X\n" + "\t\t\t\ttrap_num................%u\n" + "\t\t\t\tqpn.....................0x%06X\n" + "\t\t\t\tresp_time_val...........0x%X\n" + "\t\t\t\tnode_type...............0x%06X\n" "", + cl_ntoh64(p_iir->subscriber_gid.unicast.prefix), + cl_ntoh64(p_iir->subscriber_gid.unicast. + interface_id), + cl_ntoh16(p_iir->subscriber_enum), + cl_ntoh64(p_iir->inform_info.gid.unicast. + prefix), + cl_ntoh64(p_iir->inform_info.gid.unicast. + interface_id), + cl_ntoh16(p_iir->inform_info.lid_range_begin), + cl_ntoh16(p_iir->inform_info.lid_range_end), + p_iir->inform_info.is_generic, + p_iir->inform_info.subscribe, + cl_ntoh16(p_iir->inform_info.trap_type), + cl_ntoh16(p_iir->inform_info.g_or_v.generic. + trap_num), cl_ntoh32(qpn), + resp_time_val, + cl_ntoh32(ib_inform_info_get_prod_type + (&p_iir->inform_info)) + ); + } else { + osm_log(p_log, log_level, + "InformInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tSubscriberGID...........0x%016" PRIx64 + " : " "0x%016" PRIx64 "\n" + "\t\t\t\tSubscriberEnum..........0x%X\n" + "\t\t\t\tInformInfo dump:\n" + "\t\t\t\tgid.....................0x%016" PRIx64 + " : 0x%016" PRIx64 "\n" + "\t\t\t\tlid_range_begin.........0x%X\n" + "\t\t\t\tlid_range_end...........0x%X\n" + "\t\t\t\tis_generic..............0x%X\n" + "\t\t\t\tsubscribe...............0x%X\n" + "\t\t\t\ttrap_type...............0x%X\n" + "\t\t\t\tdev_id..................0x%X\n" + "\t\t\t\tqpn.....................0x%06X\n" + "\t\t\t\tresp_time_val...........0x%X\n" + "\t\t\t\tvendor_id...............0x%06X\n" "", + cl_ntoh64(p_iir->subscriber_gid.unicast.prefix), + cl_ntoh64(p_iir->subscriber_gid.unicast. + interface_id), + cl_ntoh16(p_iir->subscriber_enum), + cl_ntoh64(p_iir->inform_info.gid.unicast. + prefix), + cl_ntoh64(p_iir->inform_info.gid.unicast. + interface_id), + cl_ntoh16(p_iir->inform_info.lid_range_begin), + cl_ntoh16(p_iir->inform_info.lid_range_end), + p_iir->inform_info.is_generic, + p_iir->inform_info.subscribe, + cl_ntoh16(p_iir->inform_info.trap_type), + cl_ntoh16(p_iir->inform_info.g_or_v.vend. + dev_id), cl_ntoh32(qpn), + resp_time_val, + cl_ntoh32(ib_inform_info_get_prod_type + (&p_iir->inform_info)) + ); + } + } } /********************************************************************** **********************************************************************/ void -osm_dump_link_record( - IN osm_log_t* const p_log, - IN const ib_link_record_t* const p_lr, - IN const osm_log_level_t log_level ) +osm_dump_link_record(IN osm_log_t * const p_log, + IN const ib_link_record_t * const p_lr, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "Link Record dump:\n" - "\t\t\t\tfrom_lid................0x%X\n" - "\t\t\t\tfrom_port_num...........0x%X\n" - "\t\t\t\tto_port_num.............0x%X\n" - "\t\t\t\tto_lid..................0x%X\n" - "", - cl_ntoh16( p_lr->from_lid ), - p_lr->from_port_num, - p_lr->to_port_num, - cl_ntoh16( p_lr->to_lid ) - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "Link Record dump:\n" + "\t\t\t\tfrom_lid................0x%X\n" + "\t\t\t\tfrom_port_num...........0x%X\n" + "\t\t\t\tto_port_num.............0x%X\n" + "\t\t\t\tto_lid..................0x%X\n" + "", + cl_ntoh16(p_lr->from_lid), + p_lr->from_port_num, + p_lr->to_port_num, cl_ntoh16(p_lr->to_lid) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_switch_info( - IN osm_log_t* const p_log, - IN const ib_switch_info_t* const p_si, - IN const osm_log_level_t log_level ) +osm_dump_switch_info(IN osm_log_t * const p_log, + IN const ib_switch_info_t * const p_si, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, OSM_LOG_VERBOSE, - "SwitchInfo dump:\n" - "\t\t\t\tlin_cap.................0x%X\n" - "\t\t\t\trand_cap................0x%X\n" - "\t\t\t\tmcast_cap...............0x%X\n" - "\t\t\t\tlin_top.................0x%X\n" - "\t\t\t\tdef_port................0x%X\n" - "\t\t\t\tdef_mcast_pri_port......0x%X\n" - "\t\t\t\tdef_mcast_not_port......0x%X\n" - "\t\t\t\tlife_state..............0x%X\n" - "\t\t\t\tlids_per_port...........0x%X\n" - "\t\t\t\tpartition_enf_cap.......0x%X\n" - "\t\t\t\tflags...................0x%X\n" - "", - cl_ntoh16( p_si->lin_cap ), - cl_ntoh16( p_si->rand_cap ), - cl_ntoh16( p_si->mcast_cap ), - cl_ntoh16( p_si->lin_top ), - p_si->def_port, - p_si->def_mcast_pri_port, - p_si->def_mcast_not_port, - p_si->life_state, - cl_ntoh16( p_si->lids_per_port ), - cl_ntoh16( p_si->enforce_cap ), - p_si->flags - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, OSM_LOG_VERBOSE, + "SwitchInfo dump:\n" + "\t\t\t\tlin_cap.................0x%X\n" + "\t\t\t\trand_cap................0x%X\n" + "\t\t\t\tmcast_cap...............0x%X\n" + "\t\t\t\tlin_top.................0x%X\n" + "\t\t\t\tdef_port................0x%X\n" + "\t\t\t\tdef_mcast_pri_port......0x%X\n" + "\t\t\t\tdef_mcast_not_port......0x%X\n" + "\t\t\t\tlife_state..............0x%X\n" + "\t\t\t\tlids_per_port...........0x%X\n" + "\t\t\t\tpartition_enf_cap.......0x%X\n" + "\t\t\t\tflags...................0x%X\n" + "", + cl_ntoh16(p_si->lin_cap), + cl_ntoh16(p_si->rand_cap), + cl_ntoh16(p_si->mcast_cap), + cl_ntoh16(p_si->lin_top), + p_si->def_port, + p_si->def_mcast_pri_port, + p_si->def_mcast_not_port, + p_si->life_state, + cl_ntoh16(p_si->lids_per_port), + cl_ntoh16(p_si->enforce_cap), p_si->flags); + } } - /********************************************************************** **********************************************************************/ void -osm_dump_switch_info_record( - IN osm_log_t* const p_log, - IN const ib_switch_info_record_t* const p_sir, - IN const osm_log_level_t log_level ) +osm_dump_switch_info_record(IN osm_log_t * const p_log, + IN const ib_switch_info_record_t * const p_sir, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, log_level, - "SwitchInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tlid.....................0x%X\n" - "\t\t\t\tSwitchInfo dump:\n" - "\t\t\t\tlin_cap.................0x%X\n" - "\t\t\t\trand_cap................0x%X\n" - "\t\t\t\tmcast_cap...............0x%X\n" - "\t\t\t\tlin_top.................0x%X\n" - "\t\t\t\tdef_port................0x%X\n" - "\t\t\t\tdef_mcast_pri_port......0x%X\n" - "\t\t\t\tdef_mcast_not_port......0x%X\n" - "\t\t\t\tlife_state..............0x%X\n" - "\t\t\t\tlids_per_port...........0x%X\n" - "\t\t\t\tpartition_enf_cap.......0x%X\n" - "\t\t\t\tflags...................0x%X\n" - "", - cl_ntoh16( p_sir->lid ), - cl_ntoh16( p_sir->switch_info.lin_cap ), - cl_ntoh16( p_sir->switch_info.rand_cap ), - cl_ntoh16( p_sir->switch_info.mcast_cap ), - cl_ntoh16( p_sir->switch_info.lin_top ), - p_sir->switch_info.def_port, - p_sir->switch_info.def_mcast_pri_port, - p_sir->switch_info.def_mcast_not_port, - p_sir->switch_info.life_state, - cl_ntoh16( p_sir->switch_info.lids_per_port ), - cl_ntoh16( p_sir->switch_info.enforce_cap ), - p_sir->switch_info.flags - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, log_level, + "SwitchInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tlid.....................0x%X\n" + "\t\t\t\tSwitchInfo dump:\n" + "\t\t\t\tlin_cap.................0x%X\n" + "\t\t\t\trand_cap................0x%X\n" + "\t\t\t\tmcast_cap...............0x%X\n" + "\t\t\t\tlin_top.................0x%X\n" + "\t\t\t\tdef_port................0x%X\n" + "\t\t\t\tdef_mcast_pri_port......0x%X\n" + "\t\t\t\tdef_mcast_not_port......0x%X\n" + "\t\t\t\tlife_state..............0x%X\n" + "\t\t\t\tlids_per_port...........0x%X\n" + "\t\t\t\tpartition_enf_cap.......0x%X\n" + "\t\t\t\tflags...................0x%X\n" + "", + cl_ntoh16(p_sir->lid), + cl_ntoh16(p_sir->switch_info.lin_cap), + cl_ntoh16(p_sir->switch_info.rand_cap), + cl_ntoh16(p_sir->switch_info.mcast_cap), + cl_ntoh16(p_sir->switch_info.lin_top), + p_sir->switch_info.def_port, + p_sir->switch_info.def_mcast_pri_port, + p_sir->switch_info.def_mcast_not_port, + p_sir->switch_info.life_state, + cl_ntoh16(p_sir->switch_info.lids_per_port), + cl_ntoh16(p_sir->switch_info.enforce_cap), + p_sir->switch_info.flags); + } } /********************************************************************** **********************************************************************/ void -osm_dump_pkey_block( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint16_t block_num, - IN uint8_t port_num, - IN const ib_pkey_table_t* const p_pkey_tbl, - IN const osm_log_level_t log_level ) +osm_dump_pkey_block(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint16_t block_num, + IN uint8_t port_num, + IN const ib_pkey_table_t * const p_pkey_tbl, + IN const osm_log_level_t log_level) { - char buf_line[1024]; - int i, n; - - if( osm_log_is_active( p_log, log_level ) ) - { - for (i = 0, n = 0; i < 32; i++) - n += sprintf( buf_line + n," 0x%04x |", - cl_ntoh16(p_pkey_tbl->pkey_entry[i])); - - osm_log( p_log, log_level, - "P_Key table dump:\n" - "\t\t\tport_guid...........0x%016" PRIx64 "\n" - "\t\t\tblock_num...........0x%X\n" - "\t\t\tport_num............0x%X\n\tP_Key Table: %s\n", - cl_ntoh64( port_guid ), - block_num, - port_num, - buf_line - ); - } + char buf_line[1024]; + int i, n; + + if (osm_log_is_active(p_log, log_level)) { + for (i = 0, n = 0; i < 32; i++) + n += sprintf(buf_line + n, " 0x%04x |", + cl_ntoh16(p_pkey_tbl->pkey_entry[i])); + + osm_log(p_log, log_level, + "P_Key table dump:\n" + "\t\t\tport_guid...........0x%016" PRIx64 "\n" + "\t\t\tblock_num...........0x%X\n" + "\t\t\tport_num............0x%X\n\tP_Key Table: %s\n", + cl_ntoh64(port_guid), block_num, port_num, buf_line); + } } /********************************************************************** **********************************************************************/ void -osm_dump_slvl_map_table( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint8_t in_port_num, - IN uint8_t out_port_num, - IN const ib_slvl_table_t* const p_slvl_tbl, - IN const osm_log_level_t log_level ) +osm_dump_slvl_map_table(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint8_t in_port_num, + IN uint8_t out_port_num, + IN const ib_slvl_table_t * const p_slvl_tbl, + IN const osm_log_level_t log_level) { - char buf_line1[1024]; - char buf_line2[1024]; - int n; - uint8_t i; - - if( osm_log_is_active( p_log, log_level ) ) - { - for (i = 0, n = 0; i < 16; i++) - n += sprintf( buf_line1 + n," %-2u |", i); - for (i = 0, n = 0; i < 16; i++) - n += sprintf( buf_line2 + n,"0x%01X |", - ib_slvl_table_get(p_slvl_tbl, i)); - osm_log( p_log, log_level, - "SLtoVL dump:\n" - "\t\t\tport_guid............0x%016" PRIx64 "\n" - "\t\t\tin_port_num..........0x%X\n" - "\t\t\tout_port_num.........0x%X\n\tSL: | %s\n\tVL: | %s\n", - cl_ntoh64( port_guid ), - in_port_num, - out_port_num, - buf_line1, buf_line2 - ); - } + char buf_line1[1024]; + char buf_line2[1024]; + int n; + uint8_t i; + + if (osm_log_is_active(p_log, log_level)) { + for (i = 0, n = 0; i < 16; i++) + n += sprintf(buf_line1 + n, " %-2u |", i); + for (i = 0, n = 0; i < 16; i++) + n += sprintf(buf_line2 + n, "0x%01X |", + ib_slvl_table_get(p_slvl_tbl, i)); + osm_log(p_log, log_level, + "SLtoVL dump:\n" + "\t\t\tport_guid............0x%016" PRIx64 "\n" + "\t\t\tin_port_num..........0x%X\n" + "\t\t\tout_port_num.........0x%X\n\tSL: | %s\n\tVL: | %s\n", + cl_ntoh64(port_guid), + in_port_num, out_port_num, buf_line1, buf_line2); + } } /********************************************************************** **********************************************************************/ void -osm_dump_vl_arb_table( - IN osm_log_t* const p_log, - IN uint64_t port_guid, - IN uint8_t block_num, - IN uint8_t port_num, - IN const ib_vl_arb_table_t* const p_vla_tbl, - IN const osm_log_level_t log_level ) +osm_dump_vl_arb_table(IN osm_log_t * const p_log, + IN uint64_t port_guid, + IN uint8_t block_num, + IN uint8_t port_num, + IN const ib_vl_arb_table_t * const p_vla_tbl, + IN const osm_log_level_t log_level) { - char buf_line1[1024]; - char buf_line2[1024]; - int i, n; - - if( osm_log_is_active( p_log, log_level ) ) - { - for (i = 0, n = 0; i < 32; i++) - n += sprintf( buf_line1 + n," 0x%01X |", p_vla_tbl->vl_entry[i].vl); - for (i = 0, n = 0; i < 32; i++) - n += sprintf( buf_line2 + n," 0x%01X |", p_vla_tbl->vl_entry[i].weight); - osm_log( p_log, log_level, - "VLArb dump:\n" - "\t\t\tport_guid...........0x%016" PRIx64 "\n" - "\t\t\tblock_num...........0x%X\n" - "\t\t\tport_num............0x%X\n\tVL : | %s\n\tWEIGHT:| %s\n", - cl_ntoh64( port_guid ), - block_num, - port_num, - buf_line1, buf_line2 - ); - } + char buf_line1[1024]; + char buf_line2[1024]; + int i, n; + + if (osm_log_is_active(p_log, log_level)) { + for (i = 0, n = 0; i < 32; i++) + n += sprintf(buf_line1 + n, " 0x%01X |", + p_vla_tbl->vl_entry[i].vl); + for (i = 0, n = 0; i < 32; i++) + n += sprintf(buf_line2 + n, " 0x%01X |", + p_vla_tbl->vl_entry[i].weight); + osm_log(p_log, log_level, + "VLArb dump:\n" "\t\t\tport_guid...........0x%016" + PRIx64 "\n" "\t\t\tblock_num...........0x%X\n" + "\t\t\tport_num............0x%X\n\tVL : | %s\n\tWEIGHT:| %s\n", + cl_ntoh64(port_guid), block_num, port_num, buf_line1, + buf_line2); + } } /********************************************************************** **********************************************************************/ void -osm_dump_sm_info( - IN osm_log_t* const p_log, - IN const ib_sm_info_t* const p_smi, - IN const osm_log_level_t log_level ) +osm_dump_sm_info(IN osm_log_t * const p_log, + IN const ib_sm_info_t * const p_smi, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "SMInfo dump:\n" - "\t\t\t\tguid....................0x%016" PRIx64 "\n" - "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" - "\t\t\t\tact_count...............%u\n" - "\t\t\t\tpriority................%u\n" - "\t\t\t\tsm_state................%u\n" - "", - cl_ntoh64( p_smi->guid ), - cl_ntoh64( p_smi->sm_key ), - cl_ntoh32( p_smi->act_count ), - ib_sminfo_get_priority( p_smi ), - ib_sminfo_get_state( p_smi ) - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, OSM_LOG_DEBUG, + "SMInfo dump:\n" + "\t\t\t\tguid....................0x%016" PRIx64 "\n" + "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" + "\t\t\t\tact_count...............%u\n" + "\t\t\t\tpriority................%u\n" + "\t\t\t\tsm_state................%u\n" + "", + cl_ntoh64(p_smi->guid), + cl_ntoh64(p_smi->sm_key), + cl_ntoh32(p_smi->act_count), + ib_sminfo_get_priority(p_smi), + ib_sminfo_get_state(p_smi) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_sm_info_record( - IN osm_log_t* const p_log, - IN const ib_sminfo_record_t* const p_smir, - IN const osm_log_level_t log_level ) +osm_dump_sm_info_record(IN osm_log_t * const p_log, + IN const ib_sminfo_record_t * const p_smir, + IN const osm_log_level_t log_level) { - if( osm_log_is_active( p_log, log_level ) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "SMInfo Record dump:\n" - "\t\t\t\tRID\n" - "\t\t\t\tLid.....................0x%X\n" - "\t\t\t\tReserved................0x%X\n" - "\t\t\t\tSMInfo dump:\n" - "\t\t\t\tguid....................0x%016" PRIx64 "\n" - "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" - "\t\t\t\tact_count...............%u\n" - "\t\t\t\tpriority................%u\n" - "\t\t\t\tsm_state................%u\n" - "", - cl_ntoh16( p_smir->lid ), - cl_ntoh16( p_smir->resv0 ), - cl_ntoh64( p_smir->sm_info.guid ), - cl_ntoh64( p_smir->sm_info.sm_key ), - cl_ntoh32( p_smir->sm_info.act_count ), - ib_sminfo_get_priority( &p_smir->sm_info ), - ib_sminfo_get_state( &p_smir->sm_info ) - ); - } + if (osm_log_is_active(p_log, log_level)) { + osm_log(p_log, OSM_LOG_DEBUG, + "SMInfo Record dump:\n" + "\t\t\t\tRID\n" + "\t\t\t\tLid.....................0x%X\n" + "\t\t\t\tReserved................0x%X\n" + "\t\t\t\tSMInfo dump:\n" + "\t\t\t\tguid....................0x%016" PRIx64 "\n" + "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" + "\t\t\t\tact_count...............%u\n" + "\t\t\t\tpriority................%u\n" + "\t\t\t\tsm_state................%u\n" + "", + cl_ntoh16(p_smir->lid), + cl_ntoh16(p_smir->resv0), + cl_ntoh64(p_smir->sm_info.guid), + cl_ntoh64(p_smir->sm_info.sm_key), + cl_ntoh32(p_smir->sm_info.act_count), + ib_sminfo_get_priority(&p_smir->sm_info), + ib_sminfo_get_state(&p_smir->sm_info) + ); + } } /********************************************************************** **********************************************************************/ void -osm_dump_notice( - IN osm_log_t* const p_log, - IN const ib_mad_notice_attr_t *p_ntci, - IN const osm_log_level_t log_level ) +osm_dump_notice(IN osm_log_t * const p_log, + IN const ib_mad_notice_attr_t * p_ntci, + IN const osm_log_level_t log_level) { - char buff[1024]; - buff[0] = '\0'; - - if( osm_log_is_active( p_log, log_level ) ) - { - if (ib_notice_is_generic(p_ntci)) - { - /* immediate data based on the trap */ - switch (cl_ntoh16(p_ntci->g_or_v.generic.trap_num)) { - case 64: - case 65: - case 66: - case 67: - sprintf(buff, - "\t\t\t\tsrc_gid..................0x%016" PRIx64 - ":0x%016" PRIx64 "\n", - cl_ntoh64(p_ntci->data_details.ntc_64_67.gid.unicast.prefix), - cl_ntoh64(p_ntci->data_details.ntc_64_67.gid.unicast.interface_id)); - break; - case 128: - sprintf(buff, - "\t\t\t\tsw_lid...................0x%04X\n", - cl_ntoh16(p_ntci->data_details.ntc_128.sw_lid)); - break; - case 129: - case 130: - case 131: - sprintf(buff, - "\t\t\t\tlid......................0x%04X\n" - "\t\t\t\tport_num.................%u\n", - cl_ntoh16(p_ntci->data_details.ntc_129_131.lid), - p_ntci->data_details.ntc_129_131.port_num); - break; - case 144: - sprintf(buff, - "\t\t\t\tlid......................0x%04x\n" - "\t\t\t\tnew_cap_mask.............0x%08x\n", - cl_ntoh16(p_ntci->data_details.ntc_144.lid), - cl_ntoh32(p_ntci->data_details.ntc_144.new_cap_mask)); - break; - case 145: - sprintf(buff, - "\t\t\t\tlid......................0x%04X\n" - "\t\t\t\tnew_sys_guid.............0x%016" PRIx64 "\n", - cl_ntoh16(p_ntci->data_details.ntc_145.lid), - cl_ntoh64(p_ntci->data_details.ntc_145.new_sys_guid)); - break; - } - - osm_log( p_log, log_level, - "Generic Notice dump:\n" - "\t\t\t\ttype.....................0x%02X\n" - "\t\t\t\tprod_type................%u (%s)\n" - "\t\t\t\ttrap_num.................%u\n%s" - "", - ib_notice_get_type(p_ntci), - cl_ntoh32(ib_notice_get_prod_type(p_ntci)), - ib_get_producer_type_str(ib_notice_get_prod_type(p_ntci)), - cl_ntoh16(p_ntci->g_or_v.generic.trap_num), - buff - ); - } - else - { - osm_log( p_log, log_level, - "Vendor Notice dump:\n" - "\t\t\t\ttype.....................0x%04x\n" - "\t\t\t\tvendor...................%u\n" - "\t\t\t\tdevice_id................%u\n" - "", - cl_ntoh16(ib_notice_get_type(p_ntci)), - cl_ntoh32(ib_notice_get_vend_id(p_ntci)), - cl_ntoh16(p_ntci->g_or_v.vend.dev_id) - ); - } - } + char buff[1024]; + buff[0] = '\0'; + + if (osm_log_is_active(p_log, log_level)) { + if (ib_notice_is_generic(p_ntci)) { + /* immediate data based on the trap */ + switch (cl_ntoh16(p_ntci->g_or_v.generic.trap_num)) { + case 64: + case 65: + case 66: + case 67: + sprintf(buff, + "\t\t\t\tsrc_gid..................0x%016" + PRIx64 ":0x%016" PRIx64 "\n", + cl_ntoh64(p_ntci->data_details. + ntc_64_67.gid.unicast.prefix), + cl_ntoh64(p_ntci->data_details. + ntc_64_67.gid.unicast. + interface_id)); + break; + case 128: + sprintf(buff, + "\t\t\t\tsw_lid...................0x%04X\n", + cl_ntoh16(p_ntci->data_details.ntc_128. + sw_lid)); + break; + case 129: + case 130: + case 131: + sprintf(buff, + "\t\t\t\tlid......................0x%04X\n" + "\t\t\t\tport_num.................%u\n", + cl_ntoh16(p_ntci->data_details. + ntc_129_131.lid), + p_ntci->data_details.ntc_129_131. + port_num); + break; + case 144: + sprintf(buff, + "\t\t\t\tlid......................0x%04x\n" + "\t\t\t\tnew_cap_mask.............0x%08x\n", + cl_ntoh16(p_ntci->data_details.ntc_144. + lid), + cl_ntoh32(p_ntci->data_details.ntc_144. + new_cap_mask)); + break; + case 145: + sprintf(buff, + "\t\t\t\tlid......................0x%04X\n" + "\t\t\t\tnew_sys_guid.............0x%016" + PRIx64 "\n", + cl_ntoh16(p_ntci->data_details.ntc_145. + lid), + cl_ntoh64(p_ntci->data_details.ntc_145. + new_sys_guid)); + break; + } + + osm_log(p_log, log_level, + "Generic Notice dump:\n" + "\t\t\t\ttype.....................0x%02X\n" + "\t\t\t\tprod_type................%u (%s)\n" + "\t\t\t\ttrap_num.................%u\n%s" + "", + ib_notice_get_type(p_ntci), + cl_ntoh32(ib_notice_get_prod_type(p_ntci)), + ib_get_producer_type_str(ib_notice_get_prod_type + (p_ntci)), + cl_ntoh16(p_ntci->g_or_v.generic.trap_num), + buff); + } else { + osm_log(p_log, log_level, + "Vendor Notice dump:\n" + "\t\t\t\ttype.....................0x%04x\n" + "\t\t\t\tvendor...................%u\n" + "\t\t\t\tdevice_id................%u\n" + "", + cl_ntoh16(ib_notice_get_type(p_ntci)), + cl_ntoh32(ib_notice_get_vend_id(p_ntci)), + cl_ntoh16(p_ntci->g_or_v.vend.dev_id) + ); + } + } } /********************************************************************** **********************************************************************/ void -osm_dump_dr_smp( - IN osm_log_t* const p_log, - IN const ib_smp_t* const p_smp, - IN const osm_log_level_t log_level ) +osm_dump_dr_smp(IN osm_log_t * const p_log, + IN const ib_smp_t * const p_smp, + IN const osm_log_level_t log_level) { - uint32_t i; - char buf[BUF_SIZE]; - char line[BUF_SIZE]; - - if( osm_log_is_active( p_log, log_level ) ) - { - sprintf( buf, - "SMP dump:\n" - "\t\t\t\tbase_ver................0x%X\n" - "\t\t\t\tmgmt_class..............0x%X\n" - "\t\t\t\tclass_ver...............0x%X\n" - "\t\t\t\tmethod..................0x%X (%s)\n", - p_smp->base_ver, - p_smp->mgmt_class, - p_smp->class_ver, - p_smp->method, ib_get_sm_method_str(p_smp->method)); - - if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) - { - sprintf( line, - "\t\t\t\tD bit...................0x%X\n" - "\t\t\t\tstatus..................0x%X\n", - ib_smp_is_d(p_smp), - ib_smp_get_status(p_smp)); - } - else - { - sprintf( line, - "\t\t\t\tstatus..................0x%X\n", - cl_ntoh16(p_smp->status)); - } - strcat( buf, line ); - - sprintf( line, - "\t\t\t\thop_ptr.................0x%X\n" - "\t\t\t\thop_count...............0x%X\n" - "\t\t\t\ttrans_id................0x%" PRIx64 "\n" - "\t\t\t\tattr_id.................0x%X (%s)\n" - "\t\t\t\tresv....................0x%X\n" - "\t\t\t\tattr_mod................0x%X\n" - "\t\t\t\tm_key...................0x%016" PRIx64 "\n", - p_smp->hop_ptr, - p_smp->hop_count, - cl_ntoh64(p_smp->trans_id), - cl_ntoh16(p_smp->attr_id), - ib_get_sm_attr_str( p_smp->attr_id ), - cl_ntoh16(p_smp->resv), - cl_ntoh32(p_smp->attr_mod), - cl_ntoh64(p_smp->m_key) - ); - strcat( buf, line ); - - if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) - { - sprintf( line, - "\t\t\t\tdr_slid.................0x%X\n" - "\t\t\t\tdr_dlid.................0x%X\n", - cl_ntoh16(p_smp->dr_slid), - cl_ntoh16(p_smp->dr_dlid) - ); - strcat( buf, line ); - - strcat( buf, "\n\t\t\t\tInitial path: " ); - - for( i = 0; i <= p_smp->hop_count; i++ ) - { - if (i == 0) - sprintf( line, "%d", p_smp->initial_path[i] ); - else - sprintf( line, ",%d", p_smp->initial_path[i] ); - strcat( buf, line ); - } - - strcat( buf, "\n\t\t\t\tReturn path: " ); - - for( i = 0; i <= p_smp->hop_count; i++ ) - { - if (i == 0) - sprintf( line, "%d", p_smp->return_path[i] ); - else - sprintf( line, ",%d", p_smp->return_path[i] ); - strcat( buf, line ); - } - - strcat( buf, "\n\t\t\t\tReserved: " ); - - for( i = 0; i < 7; i++ ) - { - sprintf( line, "[%0X]", p_smp->resv1[i] ); - strcat( buf, line ); - } - - strcat( buf, "\n" ); - - for( i = 0; i < 64; i += 16 ) - { - sprintf( line, "\n\t\t\t\t%02X %02X %02X %02X " - "%02X %02X %02X %02X" - " %02X %02X %02X %02X %02X %02X %02X %02X\n", - p_smp->data[i], - p_smp->data[i+1], - p_smp->data[i+2], - p_smp->data[i+3], - p_smp->data[i+4], - p_smp->data[i+5], - p_smp->data[i+6], - p_smp->data[i+7], - p_smp->data[i+8], - p_smp->data[i+9], - p_smp->data[i+10], - p_smp->data[i+11], - p_smp->data[i+12], - p_smp->data[i+13], - p_smp->data[i+14], - p_smp->data[i+15] ); - - strcat( buf, line ); - } - } - else - { - /* not a Direct Route so provide source and destination lids */ - strcat(buf, "\t\t\t\tMAD IS LID ROUTED\n"); - } - - osm_log( p_log, log_level, "%s\n", buf ); - - } + uint32_t i; + char buf[BUF_SIZE]; + char line[BUF_SIZE]; + + if (osm_log_is_active(p_log, log_level)) { + sprintf(buf, + "SMP dump:\n" + "\t\t\t\tbase_ver................0x%X\n" + "\t\t\t\tmgmt_class..............0x%X\n" + "\t\t\t\tclass_ver...............0x%X\n" + "\t\t\t\tmethod..................0x%X (%s)\n", + p_smp->base_ver, + p_smp->mgmt_class, + p_smp->class_ver, + p_smp->method, ib_get_sm_method_str(p_smp->method)); + + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { + sprintf(line, + "\t\t\t\tD bit...................0x%X\n" + "\t\t\t\tstatus..................0x%X\n", + ib_smp_is_d(p_smp), ib_smp_get_status(p_smp)); + } else { + sprintf(line, + "\t\t\t\tstatus..................0x%X\n", + cl_ntoh16(p_smp->status)); + } + strcat(buf, line); + + sprintf(line, + "\t\t\t\thop_ptr.................0x%X\n" + "\t\t\t\thop_count...............0x%X\n" + "\t\t\t\ttrans_id................0x%" PRIx64 "\n" + "\t\t\t\tattr_id.................0x%X (%s)\n" + "\t\t\t\tresv....................0x%X\n" + "\t\t\t\tattr_mod................0x%X\n" + "\t\t\t\tm_key...................0x%016" PRIx64 "\n", + p_smp->hop_ptr, + p_smp->hop_count, + cl_ntoh64(p_smp->trans_id), + cl_ntoh16(p_smp->attr_id), + ib_get_sm_attr_str(p_smp->attr_id), + cl_ntoh16(p_smp->resv), + cl_ntoh32(p_smp->attr_mod), cl_ntoh64(p_smp->m_key) + ); + strcat(buf, line); + + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { + sprintf(line, + "\t\t\t\tdr_slid.................0x%X\n" + "\t\t\t\tdr_dlid.................0x%X\n", + cl_ntoh16(p_smp->dr_slid), + cl_ntoh16(p_smp->dr_dlid) + ); + strcat(buf, line); + + strcat(buf, "\n\t\t\t\tInitial path: "); + + for (i = 0; i <= p_smp->hop_count; i++) { + if (i == 0) + sprintf(line, "%d", + p_smp->initial_path[i]); + else + sprintf(line, ",%d", + p_smp->initial_path[i]); + strcat(buf, line); + } + + strcat(buf, "\n\t\t\t\tReturn path: "); + + for (i = 0; i <= p_smp->hop_count; i++) { + if (i == 0) + sprintf(line, "%d", + p_smp->return_path[i]); + else + sprintf(line, ",%d", + p_smp->return_path[i]); + strcat(buf, line); + } + + strcat(buf, "\n\t\t\t\tReserved: "); + + for (i = 0; i < 7; i++) { + sprintf(line, "[%0X]", p_smp->resv1[i]); + strcat(buf, line); + } + + strcat(buf, "\n"); + + for (i = 0; i < 64; i += 16) { + sprintf(line, "\n\t\t\t\t%02X %02X %02X %02X " + "%02X %02X %02X %02X" + " %02X %02X %02X %02X %02X %02X %02X %02X\n", + p_smp->data[i], + p_smp->data[i + 1], + p_smp->data[i + 2], + p_smp->data[i + 3], + p_smp->data[i + 4], + p_smp->data[i + 5], + p_smp->data[i + 6], + p_smp->data[i + 7], + p_smp->data[i + 8], + p_smp->data[i + 9], + p_smp->data[i + 10], + p_smp->data[i + 11], + p_smp->data[i + 12], + p_smp->data[i + 13], + p_smp->data[i + 14], + p_smp->data[i + 15]); + + strcat(buf, line); + } + } else { + /* not a Direct Route so provide source and destination lids */ + strcat(buf, "\t\t\t\tMAD IS LID ROUTED\n"); + } + + osm_log(p_log, log_level, "%s\n", buf); + + } } /********************************************************************** **********************************************************************/ void -osm_dump_sa_mad( - IN osm_log_t* const p_log, - IN const ib_sa_mad_t* const p_mad, - IN const osm_log_level_t log_level ) +osm_dump_sa_mad(IN osm_log_t * const p_log, + IN const ib_sa_mad_t * const p_mad, + IN const osm_log_level_t log_level) { - char buf[BUF_SIZE]; - - /* make sure the mad is valid */ - if (p_mad == NULL) - { - osm_log( p_log, log_level, - "NULL MAD POINTER\n"); - return; - } - - if( osm_log_is_active( p_log, log_level ) ) - { - sprintf( buf, - "SA MAD dump:\n" - "\t\t\t\tbase_ver................0x%X\n" - "\t\t\t\tmgmt_class..............0x%X\n" - "\t\t\t\tclass_ver...............0x%X\n" - "\t\t\t\tmethod..................0x%X (%s)\n" - "\t\t\t\tstatus..................0x%X\n" - "\t\t\t\tresv....................0x%X\n" - "\t\t\t\ttrans_id................0x%" PRIx64 "\n" - "\t\t\t\tattr_id.................0x%X (%s)\n" - "\t\t\t\tresv1...................0x%X\n" - "\t\t\t\tattr_mod................0x%X\n" - "\t\t\t\trmpp_version............0x%X\n" - "\t\t\t\trmpp_type...............0x%X\n" - "\t\t\t\trmpp_flags..............0x%X\n" - "\t\t\t\trmpp_status.............0x%X\n" - "\t\t\t\tseg_num.................0x%X\n" - "\t\t\t\tpayload_len/new_win.....0x%X\n" - "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" - "\t\t\t\tattr_offset.............0x%X\n" - "\t\t\t\tresv2...................0x%X\n" - "\t\t\t\tcomp_mask...............0x%016" PRIx64 "\n", - p_mad->base_ver, - p_mad->mgmt_class, - p_mad->class_ver, - p_mad->method, ib_get_sa_method_str(p_mad->method), - cl_ntoh16(p_mad->status), - cl_ntoh16(p_mad->resv), - cl_ntoh64(p_mad->trans_id), - cl_ntoh16(p_mad->attr_id), - ib_get_sa_attr_str( p_mad->attr_id ), - cl_ntoh16(p_mad->resv1), - cl_ntoh32(p_mad->attr_mod), - p_mad->rmpp_version, - p_mad->rmpp_type, - p_mad->rmpp_flags, - p_mad->rmpp_status, - cl_ntoh32(p_mad->seg_num), - cl_ntoh32(p_mad->paylen_newwin), - cl_ntoh64(p_mad->sm_key), - cl_ntoh16(p_mad->attr_offset), - cl_ntoh16(p_mad->resv3), - cl_ntoh64(p_mad->comp_mask) - ); - - strcat( buf, "\n" ); - - osm_log( p_log, log_level, "%s\n", buf ); - } + char buf[BUF_SIZE]; + + /* make sure the mad is valid */ + if (p_mad == NULL) { + osm_log(p_log, log_level, "NULL MAD POINTER\n"); + return; + } + + if (osm_log_is_active(p_log, log_level)) { + sprintf(buf, + "SA MAD dump:\n" + "\t\t\t\tbase_ver................0x%X\n" + "\t\t\t\tmgmt_class..............0x%X\n" + "\t\t\t\tclass_ver...............0x%X\n" + "\t\t\t\tmethod..................0x%X (%s)\n" + "\t\t\t\tstatus..................0x%X\n" + "\t\t\t\tresv....................0x%X\n" + "\t\t\t\ttrans_id................0x%" PRIx64 "\n" + "\t\t\t\tattr_id.................0x%X (%s)\n" + "\t\t\t\tresv1...................0x%X\n" + "\t\t\t\tattr_mod................0x%X\n" + "\t\t\t\trmpp_version............0x%X\n" + "\t\t\t\trmpp_type...............0x%X\n" + "\t\t\t\trmpp_flags..............0x%X\n" + "\t\t\t\trmpp_status.............0x%X\n" + "\t\t\t\tseg_num.................0x%X\n" + "\t\t\t\tpayload_len/new_win.....0x%X\n" + "\t\t\t\tsm_key..................0x%016" PRIx64 "\n" + "\t\t\t\tattr_offset.............0x%X\n" + "\t\t\t\tresv2...................0x%X\n" + "\t\t\t\tcomp_mask...............0x%016" PRIx64 "\n", + p_mad->base_ver, + p_mad->mgmt_class, + p_mad->class_ver, + p_mad->method, ib_get_sa_method_str(p_mad->method), + cl_ntoh16(p_mad->status), + cl_ntoh16(p_mad->resv), + cl_ntoh64(p_mad->trans_id), + cl_ntoh16(p_mad->attr_id), + ib_get_sa_attr_str(p_mad->attr_id), + cl_ntoh16(p_mad->resv1), + cl_ntoh32(p_mad->attr_mod), + p_mad->rmpp_version, + p_mad->rmpp_type, + p_mad->rmpp_flags, + p_mad->rmpp_status, + cl_ntoh32(p_mad->seg_num), + cl_ntoh32(p_mad->paylen_newwin), + cl_ntoh64(p_mad->sm_key), + cl_ntoh16(p_mad->attr_offset), + cl_ntoh16(p_mad->resv3), cl_ntoh64(p_mad->comp_mask) + ); + + strcat(buf, "\n"); + + osm_log(p_log, log_level, "%s\n", buf); + } } /********************************************************************** **********************************************************************/ void -osm_dump_dr_path( - IN osm_log_t* const p_log, - IN const osm_dr_path_t* const p_path, - IN const osm_log_level_t log_level) +osm_dump_dr_path(IN osm_log_t * const p_log, + IN const osm_dr_path_t * const p_path, + IN const osm_log_level_t log_level) { - uint32_t i; - char buf[BUF_SIZE]; - char line[BUF_SIZE]; - - if( osm_log_is_active( p_log, log_level) ) - { - sprintf( buf, "Directed Path Dump of %u hop path:" - "\n\t\t\t\tPath = ", p_path->hop_count ); - - for( i = 0; i <= p_path->hop_count; i++ ) - { - if (i == 0) - sprintf( line, "%d", p_path->path[i] ); - else - sprintf( line, ",%d", p_path->path[i] ); - strcat( buf, line ); - } - osm_log( p_log, log_level, "%s\n", buf ); - } + uint32_t i; + char buf[BUF_SIZE]; + char line[BUF_SIZE]; + + if (osm_log_is_active(p_log, log_level)) { + sprintf(buf, "Directed Path Dump of %u hop path:" + "\n\t\t\t\tPath = ", p_path->hop_count); + + for (i = 0; i <= p_path->hop_count; i++) { + if (i == 0) + sprintf(line, "%d", p_path->path[i]); + else + sprintf(line, ",%d", p_path->path[i]); + strcat(buf, line); + } + osm_log(p_log, log_level, "%s\n", buf); + } } /********************************************************************** **********************************************************************/ void -osm_dump_smp_dr_path( - IN osm_log_t* const p_log, - IN const ib_smp_t* const p_smp, - IN const osm_log_level_t log_level - ) +osm_dump_smp_dr_path(IN osm_log_t * const p_log, + IN const ib_smp_t * const p_smp, + IN const osm_log_level_t log_level) { - uint32_t i; - char buf[BUF_SIZE]; - char line[BUF_SIZE]; - - if( osm_log_is_active( p_log, log_level) ) - { - sprintf( buf, "Received SMP on a %u hop path:" - "\n\t\t\t\tInitial path = ", p_smp->hop_count ); - - for( i = 0; i <= p_smp->hop_count; i++ ) - { - if (i == 0) - sprintf( line, "%d", p_smp->initial_path[i] ); - else - sprintf( line, ",%d", p_smp->initial_path[i] ); - strcat( buf, line ); - } - - strcat( buf, "\n\t\t\t\tReturn path = " ); - - for( i = 0; i <= p_smp->hop_count; i++ ) - { - if (i == 0) - sprintf( line, "%d", p_smp->return_path[i] ); - else - sprintf( line, ",%d", p_smp->return_path[i] ); - strcat( buf, line ); - } - - osm_log( p_log, log_level, "%s\n", buf ); - } + uint32_t i; + char buf[BUF_SIZE]; + char line[BUF_SIZE]; + + if (osm_log_is_active(p_log, log_level)) { + sprintf(buf, "Received SMP on a %u hop path:" + "\n\t\t\t\tInitial path = ", p_smp->hop_count); + + for (i = 0; i <= p_smp->hop_count; i++) { + if (i == 0) + sprintf(line, "%d", p_smp->initial_path[i]); + else + sprintf(line, ",%d", p_smp->initial_path[i]); + strcat(buf, line); + } + + strcat(buf, "\n\t\t\t\tReturn path = "); + + for (i = 0; i <= p_smp->hop_count; i++) { + if (i == 0) + sprintf(line, "%d", p_smp->return_path[i]); + else + sprintf(line, ",%d", p_smp->return_path[i]); + strcat(buf, line); + } + + osm_log(p_log, log_level, "%s\n", buf); + } } -const char* const __osm_sm_state_str[] = -{ - "OSM_SM_STATE_NO_STATE", /* 0 */ - "OSM_SM_STATE_INIT", /* 1 */ - "OSM_SM_STATE_IDLE", /* 2 */ - "OSM_SM_STATE_SWEEP_LIGHT", /* 3 */ - "OSM_SM_STATE_SWEEP_LIGHT_WAIT", /* 4 */ - "OSM_SM_STATE_SWEEP_HEAVY_SELF", /* 5 */ - "OSM_SM_STATE_SWEEP_HEAVY_SUBNET", /* 6 */ - "OSM_SM_STATE_SET_SM_UCAST_LID", /* 7 */ - "OSM_SM_STATE_SET_SM_UCAST_LID_WAIT", /* 8 */ - "OSM_SM_STATE_SET_SM_UCAST_LID_DONE", /* 9 */ - "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS", /* 10 */ - "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_WAIT", /* 11 */ - "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_DONE", /* 12 */ - "OSM_SM_STATE_SET_UCAST_TABLES", /* 13 */ - "OSM_SM_STATE_SET_UCAST_TABLES_WAIT", /* 14 */ - "OSM_SM_STATE_SET_UCAST_TABLES_DONE", /* 15 */ - "OSM_SM_STATE_SET_MCAST_TABLES", /* 16 */ - "OSM_SM_STATE_SET_MCAST_TABLES_WAIT", /* 17 */ - "OSM_SM_STATE_SET_MCAST_TABLES_DONE", /* 18 */ - "OSM_SM_STATE_SET_LINK_PORTS", /* 19 */ - "OSM_SM_STATE_SET_LINK_PORTS_WAIT", /* 20 */ - "OSM_SM_STATE_SET_LINK_PORTS_DONE", /* 21 */ - "OSM_SM_STATE_SET_ARMED", /* 22 */ - "OSM_SM_STATE_SET_ARMED_WAIT", /* 23 */ - "OSM_SM_STATE_SET_ARMED_DONE", /* 24 */ - "OSM_SM_STATE_SET_ACTIVE", /* 25 */ - "OSM_SM_STATE_SET_ACTIVE_WAIT", /* 26 */ - "OSM_SM_STATE_LOST_NEGOTIATION", /* 27 */ - "OSM_SM_STATE_STANDBY", /* 28 */ - "OSM_SM_STATE_SUBNET_UP", /* 29 */ - "OSM_SM_STATE_PROCESS_REQUEST", /* 30 */ - "OSM_SM_STATE_PROCESS_REQUEST_WAIT", /* 31 */ - "OSM_SM_STATE_PROCESS_REQUEST_DONE", /* 32 */ - "OSM_SM_STATE_MASTER_OR_HIGHER_SM_DETECTED",/* 33 */ - "OSM_SM_STATE_SET_PKEY", /* 34 */ - "OSM_SM_STATE_SET_PKEY_WAIT", /* 35 */ - "OSM_SM_STATE_SET_PKEY_DONE", /* 36 */ - "UNKNOWN STATE!!" /* 37 */ +const char *const __osm_sm_state_str[] = { + "OSM_SM_STATE_NO_STATE", /* 0 */ + "OSM_SM_STATE_INIT", /* 1 */ + "OSM_SM_STATE_IDLE", /* 2 */ + "OSM_SM_STATE_SWEEP_LIGHT", /* 3 */ + "OSM_SM_STATE_SWEEP_LIGHT_WAIT", /* 4 */ + "OSM_SM_STATE_SWEEP_HEAVY_SELF", /* 5 */ + "OSM_SM_STATE_SWEEP_HEAVY_SUBNET", /* 6 */ + "OSM_SM_STATE_SET_SM_UCAST_LID", /* 7 */ + "OSM_SM_STATE_SET_SM_UCAST_LID_WAIT", /* 8 */ + "OSM_SM_STATE_SET_SM_UCAST_LID_DONE", /* 9 */ + "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS", /* 10 */ + "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_WAIT", /* 11 */ + "OSM_SM_STATE_SET_SUBNET_UCAST_LIDS_DONE", /* 12 */ + "OSM_SM_STATE_SET_UCAST_TABLES", /* 13 */ + "OSM_SM_STATE_SET_UCAST_TABLES_WAIT", /* 14 */ + "OSM_SM_STATE_SET_UCAST_TABLES_DONE", /* 15 */ + "OSM_SM_STATE_SET_MCAST_TABLES", /* 16 */ + "OSM_SM_STATE_SET_MCAST_TABLES_WAIT", /* 17 */ + "OSM_SM_STATE_SET_MCAST_TABLES_DONE", /* 18 */ + "OSM_SM_STATE_SET_LINK_PORTS", /* 19 */ + "OSM_SM_STATE_SET_LINK_PORTS_WAIT", /* 20 */ + "OSM_SM_STATE_SET_LINK_PORTS_DONE", /* 21 */ + "OSM_SM_STATE_SET_ARMED", /* 22 */ + "OSM_SM_STATE_SET_ARMED_WAIT", /* 23 */ + "OSM_SM_STATE_SET_ARMED_DONE", /* 24 */ + "OSM_SM_STATE_SET_ACTIVE", /* 25 */ + "OSM_SM_STATE_SET_ACTIVE_WAIT", /* 26 */ + "OSM_SM_STATE_LOST_NEGOTIATION", /* 27 */ + "OSM_SM_STATE_STANDBY", /* 28 */ + "OSM_SM_STATE_SUBNET_UP", /* 29 */ + "OSM_SM_STATE_PROCESS_REQUEST", /* 30 */ + "OSM_SM_STATE_PROCESS_REQUEST_WAIT", /* 31 */ + "OSM_SM_STATE_PROCESS_REQUEST_DONE", /* 32 */ + "OSM_SM_STATE_MASTER_OR_HIGHER_SM_DETECTED", /* 33 */ + "OSM_SM_STATE_SET_PKEY", /* 34 */ + "OSM_SM_STATE_SET_PKEY_WAIT", /* 35 */ + "OSM_SM_STATE_SET_PKEY_DONE", /* 36 */ + "UNKNOWN STATE!!" /* 37 */ }; -const char* const __osm_sm_signal_str[] = -{ - "OSM_SIGNAL_NONE", /* 0 */ - "OSM_SIGNAL_SWEEP", /* 1 */ - "OSM_SIGNAL_CHANGE_DETECTED", /* 2 */ - "OSM_SIGNAL_NO_PENDING_TRANSACTIONS", /* 3 */ - "OSM_SIGNAL_DONE", /* 4 */ - "OSM_SIGNAL_DONE_PENDING", /* 5 */ - "OSM_SIGNAL_LOST_SM_NEGOTIATION", /* 6 */ - "OSM_SIGNAL_LIGHT_SWEEP_FAIL", /* 7 */ - "OSM_SIGNAL_IDLE_TIME_PROCESS", /* 8 */ - "OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST", /* 9 */ - "OSM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED", /* 10 */ - "OSM_SIGNAL_EXIT_STBY", /* 11 */ - "UNKNOWN SIGNAL!!" /* 12 */ +const char *const __osm_sm_signal_str[] = { + "OSM_SIGNAL_NONE", /* 0 */ + "OSM_SIGNAL_SWEEP", /* 1 */ + "OSM_SIGNAL_CHANGE_DETECTED", /* 2 */ + "OSM_SIGNAL_NO_PENDING_TRANSACTIONS", /* 3 */ + "OSM_SIGNAL_DONE", /* 4 */ + "OSM_SIGNAL_DONE_PENDING", /* 5 */ + "OSM_SIGNAL_LOST_SM_NEGOTIATION", /* 6 */ + "OSM_SIGNAL_LIGHT_SWEEP_FAIL", /* 7 */ + "OSM_SIGNAL_IDLE_TIME_PROCESS", /* 8 */ + "OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST", /* 9 */ + "OSM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED", /* 10 */ + "OSM_SIGNAL_EXIT_STBY", /* 11 */ + "UNKNOWN SIGNAL!!" /* 12 */ }; /********************************************************************** **********************************************************************/ -const char* -osm_get_sm_state_str( - IN osm_sm_state_t state ) +const char *osm_get_sm_state_str(IN osm_sm_state_t state) { - if( state > OSM_SM_STATE_MAX ) - state = OSM_SM_STATE_MAX; - return( __osm_sm_state_str[state] ); + if (state > OSM_SM_STATE_MAX) + state = OSM_SM_STATE_MAX; + return (__osm_sm_state_str[state]); } /********************************************************************** **********************************************************************/ -const char* -osm_get_sm_signal_str( - IN osm_signal_t signal ) +const char *osm_get_sm_signal_str(IN osm_signal_t signal) { - if( signal > OSM_SIGNAL_MAX ) - signal = OSM_SIGNAL_MAX; - return( __osm_sm_signal_str[signal] ); + if (signal > OSM_SIGNAL_MAX) + signal = OSM_SIGNAL_MAX; + return (__osm_sm_signal_str[signal]); } /********************************************************************** **********************************************************************/ -static const char* const __osm_disp_msg_str[] = -{ - "OSM_MSG_NONE", - "OSM_MSG_MAD_NODE_INFO", - "OSM_MSG_MAD_PORT_INFO", - "OSM_MSG_MAD_SWITCH_INFO", - "OSM_MSG_MAD_NODE_DESC", - "OSM_MSG_NO_SMPS_OUTSTANDING", - "OSM_MSG_MAD_NODE_RECORD", - "OSM_MSG_MAD_PORTINFO_RECORD", - "OSM_MSG_MAD_SERVICE_RECORD", - "OSM_MSG_MAD_PATH_RECORD", - "OSM_MSG_MAD_MCMEMBER_RECORD", - "OSM_MSG_MAD_LINK_RECORD", - "OSM_MSG_MAD_SMINFO_RECORD", - "OSM_MSG_MAD_CLASS_PORT_INFO", - "OSM_MSG_MAD_INFORM_INFO", - "OSM_MSG_MAD_LFT_RECORD", - "OSM_MSG_MAD_LFT", - "OSM_MSG_MAD_SM_INFO", - "OSM_MSG_MAD_NOTICE", - "OSM_MSG_LIGHT_SWEEP_FAIL", - "OSM_MSG_MAD_MFT", - "OSM_MSG_MAD_PKEY_TBL_RECORD", - "OSM_MSG_MAD_VL_ARB_RECORD", - "OSM_MSG_MAD_SLVL_TBL_RECORD", - "OSM_MSG_MAD_PKEY", - "OSM_MSG_MAD_VL_ARB", - "OSM_MSG_MAD_SLVL", - "OSM_MSG_MAD_GUIDINFO_RECORD", - "OSM_MSG_MAD_INFORM_INFO_RECORD", +static const char *const __osm_disp_msg_str[] = { + "OSM_MSG_NONE", + "OSM_MSG_MAD_NODE_INFO", + "OSM_MSG_MAD_PORT_INFO", + "OSM_MSG_MAD_SWITCH_INFO", + "OSM_MSG_MAD_NODE_DESC", + "OSM_MSG_NO_SMPS_OUTSTANDING", + "OSM_MSG_MAD_NODE_RECORD", + "OSM_MSG_MAD_PORTINFO_RECORD", + "OSM_MSG_MAD_SERVICE_RECORD", + "OSM_MSG_MAD_PATH_RECORD", + "OSM_MSG_MAD_MCMEMBER_RECORD", + "OSM_MSG_MAD_LINK_RECORD", + "OSM_MSG_MAD_SMINFO_RECORD", + "OSM_MSG_MAD_CLASS_PORT_INFO", + "OSM_MSG_MAD_INFORM_INFO", + "OSM_MSG_MAD_LFT_RECORD", + "OSM_MSG_MAD_LFT", + "OSM_MSG_MAD_SM_INFO", + "OSM_MSG_MAD_NOTICE", + "OSM_MSG_LIGHT_SWEEP_FAIL", + "OSM_MSG_MAD_MFT", + "OSM_MSG_MAD_PKEY_TBL_RECORD", + "OSM_MSG_MAD_VL_ARB_RECORD", + "OSM_MSG_MAD_SLVL_TBL_RECORD", + "OSM_MSG_MAD_PKEY", + "OSM_MSG_MAD_VL_ARB", + "OSM_MSG_MAD_SLVL", + "OSM_MSG_MAD_GUIDINFO_RECORD", + "OSM_MSG_MAD_INFORM_INFO_RECORD", #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - "OSM_MSG_MAD_MULTIPATH_RECORD", + "OSM_MSG_MAD_MULTIPATH_RECORD", #endif - "UNKNOWN!!" + "UNKNOWN!!" }; /********************************************************************** **********************************************************************/ -const char* -osm_get_disp_msg_str( - IN cl_disp_msgid_t msg ) +const char *osm_get_disp_msg_str(IN cl_disp_msgid_t msg) { - if( msg > OSM_MSG_MAX ) - msg = OSM_MSG_MAX; - return( __osm_disp_msg_str[msg] ); + if (msg > OSM_MSG_MAX) + msg = OSM_MSG_MAX; + return (__osm_disp_msg_str[msg]); } -static const char* const __osm_port_state_str_fixed_width[] = -{ - "NOC", - "DWN", - "INI", - "ARM", - "ACT", - "???" +static const char *const __osm_port_state_str_fixed_width[] = { + "NOC", + "DWN", + "INI", + "ARM", + "ACT", + "???" }; /********************************************************************** **********************************************************************/ -const char* -osm_get_port_state_str_fixed_width( - IN uint8_t port_state ) +const char *osm_get_port_state_str_fixed_width(IN uint8_t port_state) { - if( port_state > IB_LINK_ACTIVE ) - port_state = IB_LINK_ACTIVE + 1; - return( __osm_port_state_str_fixed_width[port_state] ); + if (port_state > IB_LINK_ACTIVE) + port_state = IB_LINK_ACTIVE + 1; + return (__osm_port_state_str_fixed_width[port_state]); } -static const char* const __osm_node_type_str_fixed_width[] = -{ - "??", - "CA", - "SW", - "RT", +static const char *const __osm_node_type_str_fixed_width[] = { + "??", + "CA", + "SW", + "RT", }; /********************************************************************** **********************************************************************/ -const char* -osm_get_node_type_str_fixed_width( - IN uint8_t node_type ) +const char *osm_get_node_type_str_fixed_width(IN uint8_t node_type) { - if( node_type > IB_NODE_TYPE_ROUTER ) - node_type = 0; - return( __osm_node_type_str_fixed_width[node_type] ); + if (node_type > IB_NODE_TYPE_ROUTER) + node_type = 0; + return (__osm_node_type_str_fixed_width[node_type]); } /********************************************************************** **********************************************************************/ -const char* -osm_get_manufacturer_str( - IN uint64_t const guid_ho ) +const char *osm_get_manufacturer_str(IN uint64_t const guid_ho) { - /* note that the max vendor string length is 11 */ - static const char* intel_str = "Intel"; - static const char* mellanox_str = "Mellanox"; - static const char* redswitch_str = "Redswitch"; - static const char* silverstorm_str = "SilverStorm"; - static const char* topspin_str = "Topspin"; - static const char* fujitsu_str = "Fujitsu"; - static const char* voltaire_str = "Voltaire"; - static const char* yotta_str = "YottaYotta"; - static const char* pathscale_str = "PathScale"; - static const char* ibm_str = "IBM"; - static const char* divergenet_str = "DivergeNet"; - static const char* flextronics_str = "Flextronics"; - static const char* agilent_str = "Agilent"; - static const char* obsidian_str = "Obsidian"; - static const char* baymicro_str = "BayMicro"; - static const char* lsilogic_str = "LSILogic"; - static const char* ddn_str = "DataDirect"; - static const char* panta_str = "Panta"; - static const char* hp_str = "HP"; - static const char* rioworks_str = "Rioworks"; - static const char* sun_str = "Sun"; - static const char* leafntwks_str = "3LeafNtwks"; - static const char* xsigo_str = "Xsigo"; - static const char* unknown_str = "Unknown"; - - switch( (uint32_t)(guid_ho >> (5 * 8)) ) - { - case OSM_VENDOR_ID_INTEL: - return( intel_str ); - case OSM_VENDOR_ID_MELLANOX: - return( mellanox_str ); - case OSM_VENDOR_ID_REDSWITCH: - return( redswitch_str ); - case OSM_VENDOR_ID_SILVERSTORM: - return( silverstorm_str ); - case OSM_VENDOR_ID_TOPSPIN: - return( topspin_str ); - case OSM_VENDOR_ID_FUJITSU: - case OSM_VENDOR_ID_FUJITSU2: - return( fujitsu_str ); - case OSM_VENDOR_ID_VOLTAIRE: - return( voltaire_str ); - case OSM_VENDOR_ID_YOTTAYOTTA: - return( yotta_str ); - case OSM_VENDOR_ID_PATHSCALE: - return( pathscale_str ); - case OSM_VENDOR_ID_IBM: - return( ibm_str ); - case OSM_VENDOR_ID_DIVERGENET: - return( divergenet_str ); - case OSM_VENDOR_ID_FLEXTRONICS: - return( flextronics_str ); - case OSM_VENDOR_ID_AGILENT: - return( agilent_str ); - case OSM_VENDOR_ID_OBSIDIAN: - return( obsidian_str ); - case OSM_VENDOR_ID_BAYMICRO: - return( baymicro_str ); - case OSM_VENDOR_ID_LSILOGIC: - return( lsilogic_str ); - case OSM_VENDOR_ID_DDN: - return( ddn_str ); - case OSM_VENDOR_ID_PANTA: - return( panta_str ); - case OSM_VENDOR_ID_HP: - return( hp_str ); - case OSM_VENDOR_ID_RIOWORKS: - return( rioworks_str ); - case OSM_VENDOR_ID_SUN: - return( sun_str ); - case OSM_VENDOR_ID_3LEAFNTWKS: - return( leafntwks_str ); - case OSM_VENDOR_ID_XSIGO: - return( xsigo_str ); - default: - return( unknown_str ); - } + /* note that the max vendor string length is 11 */ + static const char *intel_str = "Intel"; + static const char *mellanox_str = "Mellanox"; + static const char *redswitch_str = "Redswitch"; + static const char *silverstorm_str = "SilverStorm"; + static const char *topspin_str = "Topspin"; + static const char *fujitsu_str = "Fujitsu"; + static const char *voltaire_str = "Voltaire"; + static const char *yotta_str = "YottaYotta"; + static const char *pathscale_str = "PathScale"; + static const char *ibm_str = "IBM"; + static const char *divergenet_str = "DivergeNet"; + static const char *flextronics_str = "Flextronics"; + static const char *agilent_str = "Agilent"; + static const char *obsidian_str = "Obsidian"; + static const char *baymicro_str = "BayMicro"; + static const char *lsilogic_str = "LSILogic"; + static const char *ddn_str = "DataDirect"; + static const char *panta_str = "Panta"; + static const char *hp_str = "HP"; + static const char *rioworks_str = "Rioworks"; + static const char *sun_str = "Sun"; + static const char *leafntwks_str = "3LeafNtwks"; + static const char *xsigo_str = "Xsigo"; + static const char *unknown_str = "Unknown"; + + switch ((uint32_t) (guid_ho >> (5 * 8))) { + case OSM_VENDOR_ID_INTEL: + return (intel_str); + case OSM_VENDOR_ID_MELLANOX: + return (mellanox_str); + case OSM_VENDOR_ID_REDSWITCH: + return (redswitch_str); + case OSM_VENDOR_ID_SILVERSTORM: + return (silverstorm_str); + case OSM_VENDOR_ID_TOPSPIN: + return (topspin_str); + case OSM_VENDOR_ID_FUJITSU: + case OSM_VENDOR_ID_FUJITSU2: + return (fujitsu_str); + case OSM_VENDOR_ID_VOLTAIRE: + return (voltaire_str); + case OSM_VENDOR_ID_YOTTAYOTTA: + return (yotta_str); + case OSM_VENDOR_ID_PATHSCALE: + return (pathscale_str); + case OSM_VENDOR_ID_IBM: + return (ibm_str); + case OSM_VENDOR_ID_DIVERGENET: + return (divergenet_str); + case OSM_VENDOR_ID_FLEXTRONICS: + return (flextronics_str); + case OSM_VENDOR_ID_AGILENT: + return (agilent_str); + case OSM_VENDOR_ID_OBSIDIAN: + return (obsidian_str); + case OSM_VENDOR_ID_BAYMICRO: + return (baymicro_str); + case OSM_VENDOR_ID_LSILOGIC: + return (lsilogic_str); + case OSM_VENDOR_ID_DDN: + return (ddn_str); + case OSM_VENDOR_ID_PANTA: + return (panta_str); + case OSM_VENDOR_ID_HP: + return (hp_str); + case OSM_VENDOR_ID_RIOWORKS: + return (rioworks_str); + case OSM_VENDOR_ID_SUN: + return (sun_str); + case OSM_VENDOR_ID_3LEAFNTWKS: + return (leafntwks_str); + case OSM_VENDOR_ID_XSIGO: + return (xsigo_str); + default: + return (unknown_str); + } } -static const char* const __osm_mtu_str_fixed_width[] = -{ - "??? ", - "256 ", - "512 ", - "1024", - "2048", - "4096" +static const char *const __osm_mtu_str_fixed_width[] = { + "??? ", + "256 ", + "512 ", + "1024", + "2048", + "4096" }; /********************************************************************** **********************************************************************/ -const char* -osm_get_mtu_str( - IN uint8_t const mtu ) +const char *osm_get_mtu_str(IN uint8_t const mtu) { - if( mtu > IB_MTU_LEN_4096 ) - return( __osm_mtu_str_fixed_width[0] ); - else - return( __osm_mtu_str_fixed_width[mtu] ); + if (mtu > IB_MTU_LEN_4096) + return (__osm_mtu_str_fixed_width[0]); + else + return (__osm_mtu_str_fixed_width[mtu]); } -static const char* const __osm_lwa_str_fixed_width[] = -{ - "???", - "1x ", - "4x ", - "???", - "8x ", - "???", - "???", - "???", - "12x" +static const char *const __osm_lwa_str_fixed_width[] = { + "???", + "1x ", + "4x ", + "???", + "8x ", + "???", + "???", + "???", + "12x" }; /********************************************************************** **********************************************************************/ -const char* -osm_get_lwa_str( - IN uint8_t const lwa ) +const char *osm_get_lwa_str(IN uint8_t const lwa) { - if( lwa > 8 ) - return( __osm_lwa_str_fixed_width[0] ); - else - return( __osm_lwa_str_fixed_width[lwa] ); + if (lwa > 8) + return (__osm_lwa_str_fixed_width[0]); + else + return (__osm_lwa_str_fixed_width[lwa]); } /********************************************************************** **********************************************************************/ -static const char* const __osm_lsa_str_fixed_width[] = -{ - "???", - "2.5", - "5 ", - "???", - "10 " +static const char *const __osm_lsa_str_fixed_width[] = { + "???", + "2.5", + "5 ", + "???", + "10 " }; -const char* -osm_get_lsa_str( - IN uint8_t const lsa ) +const char *osm_get_lsa_str(IN uint8_t const lsa) { - if( lsa > 4 ) - return( __osm_lsa_str_fixed_width[0] ); - else - return( __osm_lsa_str_fixed_width[lsa] ); + if (lsa > 4) + return (__osm_lsa_str_fixed_width[0]); + else + return (__osm_lsa_str_fixed_width[lsa]); } /********************************************************************** **********************************************************************/ -const char* const __osm_sm_mgr_signal_str[] = -{ - "OSM_SM_SIGNAL_INIT", /* 0 */ - "OSM_SM_SIGNAL_DISCOVERY_COMPLETED", /* 2 */ - "OSM_SM_SIGNAL_POLLING_TIMEOUT", /* 3 */ - "OSM_SM_SIGNAL_DISCOVER", /* 4 */ - "OSM_SM_SIGNAL_DISABLE", /* 5 */ - "OSM_SM_SIGNAL_HANDOVER", /* 6 */ - "OSM_SM_SIGNAL_HANDOVER_SENT", /* 7 */ - "OSM_SM_SIGNAL_ACKNOWLEDGE", /* 8 */ - "OSM_SM_SIGNAL_STANDBY", /* 9 */ - "OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE", /* 10 */ - "OSM_SM_SIGNAL_WAIT_FOR_HANDOVER", /* 11 */ - "UNKNOWN STATE!!" /* 12 */ - +const char *const __osm_sm_mgr_signal_str[] = { + "OSM_SM_SIGNAL_INIT", /* 0 */ + "OSM_SM_SIGNAL_DISCOVERY_COMPLETED", /* 2 */ + "OSM_SM_SIGNAL_POLLING_TIMEOUT", /* 3 */ + "OSM_SM_SIGNAL_DISCOVER", /* 4 */ + "OSM_SM_SIGNAL_DISABLE", /* 5 */ + "OSM_SM_SIGNAL_HANDOVER", /* 6 */ + "OSM_SM_SIGNAL_HANDOVER_SENT", /* 7 */ + "OSM_SM_SIGNAL_ACKNOWLEDGE", /* 8 */ + "OSM_SM_SIGNAL_STANDBY", /* 9 */ + "OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE", /* 10 */ + "OSM_SM_SIGNAL_WAIT_FOR_HANDOVER", /* 11 */ + "UNKNOWN STATE!!" /* 12 */ }; /********************************************************************** **********************************************************************/ -const char* -osm_get_sm_mgr_signal_str( - IN osm_sm_signal_t signal ) +const char *osm_get_sm_mgr_signal_str(IN osm_sm_signal_t signal) { - if( signal > OSM_SM_SIGNAL_MAX ) - signal = OSM_SM_SIGNAL_MAX; - return( __osm_sm_mgr_signal_str[signal] ); + if (signal > OSM_SM_SIGNAL_MAX) + signal = OSM_SM_SIGNAL_MAX; + return (__osm_sm_mgr_signal_str[signal]); } -const char* const __osm_sm_mgr_state_str[] = -{ - "IB_SMINFO_STATE_NOTACTIVE", /* 0 */ - "IB_SMINFO_STATE_DISCOVERING", /* 1 */ - "IB_SMINFO_STATE_STANDBY", /* 2 */ - "IB_SMINFO_STATE_MASTER", /* 3 */ - "IB_SMINFO_STATE_INIT", /* 4 */ - "UNKNOWN STATE!!" /* 5 */ - +const char *const __osm_sm_mgr_state_str[] = { + "IB_SMINFO_STATE_NOTACTIVE", /* 0 */ + "IB_SMINFO_STATE_DISCOVERING", /* 1 */ + "IB_SMINFO_STATE_STANDBY", /* 2 */ + "IB_SMINFO_STATE_MASTER", /* 3 */ + "IB_SMINFO_STATE_INIT", /* 4 */ + "UNKNOWN STATE!!" /* 5 */ }; -const char* -osm_get_sm_mgr_state_str( - IN uint16_t state ) +const char *osm_get_sm_mgr_state_str(IN uint16_t state) { - if( state > IB_SMINFO_STATE_INIT ) - state = IB_SMINFO_STATE_INIT + 1; - return( __osm_sm_mgr_state_str[state] ); + if (state > IB_SMINFO_STATE_INIT) + state = IB_SMINFO_STATE_INIT + 1; + return (__osm_sm_mgr_state_str[state]); } - diff --git a/opensm/opensm/osm_inform.c b/opensm/opensm/osm_inform.c index 9740620..0a6ea25 100644 --- a/opensm/opensm/osm_inform.c +++ b/opensm/opensm/osm_inform.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -58,294 +58,271 @@ #include #include -typedef struct _osm_infr_match_ctxt -{ - cl_list_t *p_remove_infr_list; - ib_mad_notice_attr_t *p_ntc; +typedef struct _osm_infr_match_ctxt { + cl_list_t *p_remove_infr_list; + ib_mad_notice_attr_t *p_ntc; } osm_infr_match_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_infr_delete( - IN osm_infr_t* const p_infr ) +void osm_infr_delete(IN osm_infr_t * const p_infr) { - free( p_infr ); + free(p_infr); } /********************************************************************** **********************************************************************/ -osm_infr_t* -osm_infr_new( - IN const osm_infr_t *p_infr_rec ) +osm_infr_t *osm_infr_new(IN const osm_infr_t * p_infr_rec) { - osm_infr_t* p_infr; + osm_infr_t *p_infr; - CL_ASSERT(p_infr_rec); + CL_ASSERT(p_infr_rec); - p_infr = (osm_infr_t*)malloc( sizeof(osm_infr_t) ); - if( p_infr ) - memcpy( p_infr, p_infr_rec, sizeof(osm_infr_t) ); + p_infr = (osm_infr_t *) malloc(sizeof(osm_infr_t)); + if (p_infr) + memcpy(p_infr, p_infr_rec, sizeof(osm_infr_t)); - return( p_infr ); + return (p_infr); } /********************************************************************** **********************************************************************/ -void -__dump_all_informs( - IN osm_subn_t const *p_subn, - IN osm_log_t *p_log) +void __dump_all_informs(IN osm_subn_t const *p_subn, IN osm_log_t * p_log) { - cl_list_item_t* p_list_item; + cl_list_item_t *p_list_item; - OSM_LOG_ENTER( p_log, __dump_all_informs ); + OSM_LOG_ENTER(p_log, __dump_all_informs); - if( !osm_log_is_active( p_log, OSM_LOG_DEBUG ) ) - goto Exit; + if (!osm_log_is_active(p_log, OSM_LOG_DEBUG)) + goto Exit; - p_list_item = cl_qlist_head( &p_subn->sa_infr_list ); - while (p_list_item != cl_qlist_end( &p_subn->sa_infr_list )) - { - osm_dump_inform_info( p_log, - &((osm_infr_t*)p_list_item)->inform_record.inform_info, - OSM_LOG_DEBUG ); - p_list_item = cl_qlist_next( p_list_item ); - } + p_list_item = cl_qlist_head(&p_subn->sa_infr_list); + while (p_list_item != cl_qlist_end(&p_subn->sa_infr_list)) { + osm_dump_inform_info(p_log, + &((osm_infr_t *) p_list_item)-> + inform_record.inform_info, OSM_LOG_DEBUG); + p_list_item = cl_qlist_next(p_list_item); + } - Exit: - OSM_LOG_EXIT( p_log ); + Exit: + OSM_LOG_EXIT(p_log); } /********************************************************************** * Match an infr by the InformInfo and Address vector **********************************************************************/ static cl_status_t -__match_inf_rec( - IN const cl_list_item_t* const p_list_item, - IN void* context ) +__match_inf_rec(IN const cl_list_item_t * const p_list_item, IN void *context) { - osm_infr_t* p_infr_rec = (osm_infr_t *)context; - osm_infr_t* p_infr = (osm_infr_t*)p_list_item; - osm_log_t *p_log = p_infr_rec->p_infr_rcv->p_log; - cl_status_t status = CL_NOT_FOUND; - ib_gid_t all_zero_gid; - - OSM_LOG_ENTER( p_log, __match_inf_rec); - - if ( memcmp( &p_infr->report_addr, - &p_infr_rec->report_addr, - sizeof(p_infr_rec->report_addr)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by Address\n" ); - goto Exit; - } - - memset( &all_zero_gid, 0, sizeof(ib_gid_t) ); - - /* if inform_info.gid is not zero, ignore lid range */ - if ( !memcmp( &p_infr_rec->inform_record.inform_info.gid, - &all_zero_gid, - sizeof(p_infr_rec->inform_record.inform_info.gid)) ) - { - if ( memcmp( &p_infr->inform_record.inform_info.gid, - &p_infr_rec->inform_record.inform_info.gid, - sizeof(p_infr->inform_record.inform_info.gid)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.gid\n" ); - goto Exit; - } - } - else - { - if ( (p_infr->inform_record.inform_info.lid_range_begin != - p_infr_rec->inform_record.inform_info.lid_range_begin) || - (p_infr->inform_record.inform_info.lid_range_end != - p_infr_rec->inform_record.inform_info.lid_range_end) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.LIDRange\n" ); - goto Exit; - } - } - - if ( p_infr->inform_record.inform_info.trap_type != - p_infr_rec->inform_record.inform_info.trap_type ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.TrapType\n" ); - goto Exit; - } - - if ( p_infr->inform_record.inform_info.is_generic != - p_infr_rec->inform_record.inform_info.is_generic ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.IsGeneric\n" ); - goto Exit; - } - - if (p_infr->inform_record.inform_info.is_generic) - { - if ( p_infr->inform_record.inform_info.g_or_v.generic.trap_num != - p_infr_rec->inform_record.inform_info.g_or_v.generic.trap_num ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Generic.TrapNumber\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.generic.qpn_resp_time_val != - p_infr_rec->inform_record.inform_info.g_or_v.generic.qpn_resp_time_val ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Generic.QPNRespTimeVal\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.generic.node_type_msb != - p_infr_rec->inform_record.inform_info.g_or_v.generic.node_type_msb ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Generic.NodeTypeMSB\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.generic.node_type_lsb != - p_infr_rec->inform_record.inform_info.g_or_v.generic.node_type_lsb ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Generic.NodeTypeLSB\n" ); - else - status = CL_SUCCESS; - } - else - { - if ( p_infr->inform_record.inform_info.g_or_v.vend.dev_id != - p_infr_rec->inform_record.inform_info.g_or_v.vend.dev_id ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Vendor.DeviceID\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.vend.qpn_resp_time_val != - p_infr_rec->inform_record.inform_info.g_or_v.vend.qpn_resp_time_val ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Vendor.QPNRespTimeVal\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.vend.vendor_id_msb != - p_infr_rec->inform_record.inform_info.g_or_v.vend.vendor_id_msb ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Vendor.VendorIdMSB\n" ); - else if ( p_infr->inform_record.inform_info.g_or_v.vend.vendor_id_lsb != - p_infr_rec->inform_record.inform_info.g_or_v.vend.vendor_id_lsb ) - osm_log( p_log, OSM_LOG_DEBUG, - "__match_inf_rec: " - "Differ by InformInfo.Vendor.VendorIdLSB\n" ); - else - status = CL_SUCCESS; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return status; + osm_infr_t *p_infr_rec = (osm_infr_t *) context; + osm_infr_t *p_infr = (osm_infr_t *) p_list_item; + osm_log_t *p_log = p_infr_rec->p_infr_rcv->p_log; + cl_status_t status = CL_NOT_FOUND; + ib_gid_t all_zero_gid; + + OSM_LOG_ENTER(p_log, __match_inf_rec); + + if (memcmp(&p_infr->report_addr, + &p_infr_rec->report_addr, sizeof(p_infr_rec->report_addr))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " "Differ by Address\n"); + goto Exit; + } + + memset(&all_zero_gid, 0, sizeof(ib_gid_t)); + + /* if inform_info.gid is not zero, ignore lid range */ + if (!memcmp(&p_infr_rec->inform_record.inform_info.gid, + &all_zero_gid, + sizeof(p_infr_rec->inform_record.inform_info.gid))) { + if (memcmp(&p_infr->inform_record.inform_info.gid, + &p_infr_rec->inform_record.inform_info.gid, + sizeof(p_infr->inform_record.inform_info.gid))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.gid\n"); + goto Exit; + } + } else { + if ((p_infr->inform_record.inform_info.lid_range_begin != + p_infr_rec->inform_record.inform_info.lid_range_begin) || + (p_infr->inform_record.inform_info.lid_range_end != + p_infr_rec->inform_record.inform_info.lid_range_end)) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.LIDRange\n"); + goto Exit; + } + } + + if (p_infr->inform_record.inform_info.trap_type != + p_infr_rec->inform_record.inform_info.trap_type) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " "Differ by InformInfo.TrapType\n"); + goto Exit; + } + + if (p_infr->inform_record.inform_info.is_generic != + p_infr_rec->inform_record.inform_info.is_generic) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " "Differ by InformInfo.IsGeneric\n"); + goto Exit; + } + + if (p_infr->inform_record.inform_info.is_generic) { + if (p_infr->inform_record.inform_info.g_or_v.generic.trap_num != + p_infr_rec->inform_record.inform_info.g_or_v.generic. + trap_num) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Generic.TrapNumber\n"); + else if (p_infr->inform_record.inform_info.g_or_v.generic. + qpn_resp_time_val != + p_infr_rec->inform_record.inform_info.g_or_v.generic. + qpn_resp_time_val) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Generic.QPNRespTimeVal\n"); + else if (p_infr->inform_record.inform_info.g_or_v.generic. + node_type_msb != + p_infr_rec->inform_record.inform_info.g_or_v.generic. + node_type_msb) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Generic.NodeTypeMSB\n"); + else if (p_infr->inform_record.inform_info.g_or_v.generic. + node_type_lsb != + p_infr_rec->inform_record.inform_info.g_or_v.generic. + node_type_lsb) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Generic.NodeTypeLSB\n"); + else + status = CL_SUCCESS; + } else { + if (p_infr->inform_record.inform_info.g_or_v.vend.dev_id != + p_infr_rec->inform_record.inform_info.g_or_v.vend.dev_id) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Vendor.DeviceID\n"); + else if (p_infr->inform_record.inform_info.g_or_v.vend. + qpn_resp_time_val != + p_infr_rec->inform_record.inform_info.g_or_v.vend. + qpn_resp_time_val) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Vendor.QPNRespTimeVal\n"); + else if (p_infr->inform_record.inform_info.g_or_v.vend. + vendor_id_msb != + p_infr_rec->inform_record.inform_info.g_or_v.vend. + vendor_id_msb) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Vendor.VendorIdMSB\n"); + else if (p_infr->inform_record.inform_info.g_or_v.vend. + vendor_id_lsb != + p_infr_rec->inform_record.inform_info.g_or_v.vend. + vendor_id_lsb) + osm_log(p_log, OSM_LOG_DEBUG, + "__match_inf_rec: " + "Differ by InformInfo.Vendor.VendorIdLSB\n"); + else + status = CL_SUCCESS; + } + + Exit: + OSM_LOG_EXIT(p_log); + return status; } /********************************************************************** **********************************************************************/ -osm_infr_t* -osm_infr_get_by_rec( - IN osm_subn_t const *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t* const p_infr_rec ) +osm_infr_t *osm_infr_get_by_rec(IN osm_subn_t const *p_subn, + IN osm_log_t * p_log, + IN osm_infr_t * const p_infr_rec) { - cl_list_item_t* p_list_item; + cl_list_item_t *p_list_item; - OSM_LOG_ENTER( p_log, osm_infr_get_by_rec ); + OSM_LOG_ENTER(p_log, osm_infr_get_by_rec); - __dump_all_informs( p_subn, p_log ); + __dump_all_informs(p_subn, p_log); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_get_by_rec: " - "Looking for Inform Record\n" ); - osm_dump_inform_info( p_log, &(p_infr_rec->inform_record.inform_info), - OSM_LOG_DEBUG ); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_get_by_rec: " - "InformInfo list size %d\n", - cl_qlist_count(&p_subn->sa_infr_list) ); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_get_by_rec: " "Looking for Inform Record\n"); + osm_dump_inform_info(p_log, &(p_infr_rec->inform_record.inform_info), + OSM_LOG_DEBUG); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_get_by_rec: " + "InformInfo list size %d\n", + cl_qlist_count(&p_subn->sa_infr_list)); - p_list_item = cl_qlist_find_from_head( - &p_subn->sa_infr_list, - __match_inf_rec, - p_infr_rec ); + p_list_item = cl_qlist_find_from_head(&p_subn->sa_infr_list, + __match_inf_rec, p_infr_rec); - if( p_list_item == cl_qlist_end( &p_subn->sa_infr_list ) ) - p_list_item = NULL; + if (p_list_item == cl_qlist_end(&p_subn->sa_infr_list)) + p_list_item = NULL; - OSM_LOG_EXIT( p_log ); - return (osm_infr_t*)p_list_item; + OSM_LOG_EXIT(p_log); + return (osm_infr_t *) p_list_item; } /********************************************************************** **********************************************************************/ void -osm_infr_insert_to_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t *p_infr) +osm_infr_insert_to_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_infr_t * p_infr) { - OSM_LOG_ENTER( p_log, osm_infr_insert_to_db ); + OSM_LOG_ENTER(p_log, osm_infr_insert_to_db); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_insert_to_db: " - "Inserting new InformInfo Record into Database\n" ); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_insert_to_db: " - "Dump before insertion (size %d)\n", - cl_qlist_count(&p_subn->sa_infr_list) ); - __dump_all_informs( p_subn, p_log ); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_insert_to_db: " + "Inserting new InformInfo Record into Database\n"); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_insert_to_db: " + "Dump before insertion (size %d)\n", + cl_qlist_count(&p_subn->sa_infr_list)); + __dump_all_informs(p_subn, p_log); #if 0 - osm_dump_inform_info( p_log, - &(p_infr->inform_record.inform_info), OSM_LOG_DEBUG ); + osm_dump_inform_info(p_log, + &(p_infr->inform_record.inform_info), + OSM_LOG_DEBUG); #endif - cl_qlist_insert_head( &p_subn->sa_infr_list, - &p_infr->list_item ); + cl_qlist_insert_head(&p_subn->sa_infr_list, &p_infr->list_item); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_insert_to_db: " - "Dump after insertion (size %d)\n", - cl_qlist_count(&p_subn->sa_infr_list) ); - __dump_all_informs( p_subn, p_log ); - OSM_LOG_EXIT( p_log ); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_insert_to_db: " + "Dump after insertion (size %d)\n", + cl_qlist_count(&p_subn->sa_infr_list)); + __dump_all_informs(p_subn, p_log); + OSM_LOG_EXIT(p_log); } /********************************************************************** **********************************************************************/ void -osm_infr_remove_from_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_infr_t *p_infr) +osm_infr_remove_from_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_infr_t * p_infr) { - OSM_LOG_ENTER( p_log, osm_infr_remove_from_db ); + OSM_LOG_ENTER(p_log, osm_infr_remove_from_db); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_infr_remove_from_db: " - "Removing InformInfo Subscribing GID:0x%016" PRIx64 " : 0x%016" PRIx64 - " Enum:0x%X from Database\n", - cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast.prefix), - cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast.interface_id), - p_infr->inform_record.subscriber_enum - ); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_infr_remove_from_db: " + "Removing InformInfo Subscribing GID:0x%016" PRIx64 " : 0x%016" + PRIx64 " Enum:0x%X from Database\n", + cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast.prefix), + cl_ntoh64(p_infr->inform_record.subscriber_gid.unicast. + interface_id), p_infr->inform_record.subscriber_enum); - osm_dump_inform_info( p_log, &(p_infr->inform_record.inform_info), OSM_LOG_DEBUG ); + osm_dump_inform_info(p_log, &(p_infr->inform_record.inform_info), + OSM_LOG_DEBUG); - cl_qlist_remove_item( &p_subn->sa_infr_list, - &p_infr->list_item ); + cl_qlist_remove_item(&p_subn->sa_infr_list, &p_infr->list_item); - osm_infr_delete( p_infr ); + osm_infr_delete(p_infr); - OSM_LOG_EXIT( p_log ); + OSM_LOG_EXIT(p_log); } /********************************************************************** @@ -353,82 +330,75 @@ osm_infr_remove_from_db( * Given a target address to send to and the notice. * We need to send SubnAdmReport **********************************************************************/ -static ib_api_status_t -__osm_send_report( - IN osm_infr_t* p_infr_rec, /* the informinfo */ - IN ib_mad_notice_attr_t* p_ntc /* notice to send */ - ) +static ib_api_status_t __osm_send_report(IN osm_infr_t * p_infr_rec, /* the informinfo */ + IN ib_mad_notice_attr_t * p_ntc /* notice to send */ + ) { - osm_madw_t* p_report_madw; - ib_mad_notice_attr_t* p_report_ntc; - ib_mad_t* p_mad; - ib_sa_mad_t* p_sa_mad; - static atomic32_t trap_fwd_trans_id = 0x02DAB000; - ib_api_status_t status; - osm_log_t * p_log = p_infr_rec->p_infr_rcv->p_log; - - OSM_LOG_ENTER( p_log, __osm_send_report ); - - /* HACK: who switches or uses the src and dest GIDs in the grh_info ?? */ - - /* it is better to use LIDs since the GIDs might not be there for SMI traps */ - osm_log( p_log, OSM_LOG_DEBUG, - "__osm_send_report: " - "Forwarding Notice Event from LID:0x%X" - " to InformInfo LID: 0x%X TID:0x%X\n", - cl_ntoh16(p_ntc->issuer_lid), - cl_ntoh16(p_infr_rec->report_addr.dest_lid), - trap_fwd_trans_id - ); - - /* get the MAD to send */ - p_report_madw = osm_mad_pool_get( p_infr_rec->p_infr_rcv->p_mad_pool, - p_infr_rec->h_bind, - MAD_BLOCK_SIZE, - &(p_infr_rec->report_addr) ); - - p_report_madw->resp_expected = TRUE; - - if( !p_report_madw ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_send_report: ERR 0203: " - "osm_mad_pool_get failed\n" ); - status = IB_ERROR; - goto Exit; - } - - /* advance trap trans id (cant simply ++ on some systems inside ntoh) */ - p_mad = osm_madw_get_mad_ptr( p_report_madw ); - ib_mad_init_new( p_mad, - IB_MCLASS_SUBN_ADM, - 2, - IB_MAD_METHOD_REPORT, - cl_hton64( (uint64_t)cl_atomic_inc( &trap_fwd_trans_id ) ), - IB_MAD_ATTR_NOTICE, - 0 ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_report_madw ); - - p_report_ntc = (ib_mad_notice_attr_t*)&(p_sa_mad->data); - - /* copy the notice */ - *p_report_ntc = *p_ntc; - - /* The TRUE is for: response is expected */ - status = osm_vendor_send( p_report_madw->h_bind, p_report_madw, TRUE ); - if ( status != IB_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "__osm_send_report: ERR 0204: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return(status); + osm_madw_t *p_report_madw; + ib_mad_notice_attr_t *p_report_ntc; + ib_mad_t *p_mad; + ib_sa_mad_t *p_sa_mad; + static atomic32_t trap_fwd_trans_id = 0x02DAB000; + ib_api_status_t status; + osm_log_t *p_log = p_infr_rec->p_infr_rcv->p_log; + + OSM_LOG_ENTER(p_log, __osm_send_report); + + /* HACK: who switches or uses the src and dest GIDs in the grh_info ?? */ + + /* it is better to use LIDs since the GIDs might not be there for SMI traps */ + osm_log(p_log, OSM_LOG_DEBUG, + "__osm_send_report: " + "Forwarding Notice Event from LID:0x%X" + " to InformInfo LID: 0x%X TID:0x%X\n", + cl_ntoh16(p_ntc->issuer_lid), + cl_ntoh16(p_infr_rec->report_addr.dest_lid), trap_fwd_trans_id); + + /* get the MAD to send */ + p_report_madw = osm_mad_pool_get(p_infr_rec->p_infr_rcv->p_mad_pool, + p_infr_rec->h_bind, + MAD_BLOCK_SIZE, + &(p_infr_rec->report_addr)); + + p_report_madw->resp_expected = TRUE; + + if (!p_report_madw) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_send_report: ERR 0203: " + "osm_mad_pool_get failed\n"); + status = IB_ERROR; + goto Exit; + } + + /* advance trap trans id (cant simply ++ on some systems inside ntoh) */ + p_mad = osm_madw_get_mad_ptr(p_report_madw); + ib_mad_init_new(p_mad, + IB_MCLASS_SUBN_ADM, + 2, + IB_MAD_METHOD_REPORT, + cl_hton64((uint64_t) cl_atomic_inc(&trap_fwd_trans_id)), + IB_MAD_ATTR_NOTICE, 0); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_report_madw); + + p_report_ntc = (ib_mad_notice_attr_t *) & (p_sa_mad->data); + + /* copy the notice */ + *p_report_ntc = *p_ntc; + + /* The TRUE is for: response is expected */ + status = osm_vendor_send(p_report_madw->h_bind, p_report_madw, TRUE); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "__osm_send_report: ERR 0204: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** @@ -437,210 +407,197 @@ __osm_send_report( * The Notice.GID should be pre-filled with the trap generator GID **********************************************************************/ static void -__match_notice_to_inf_rec( - IN cl_list_item_t* const p_list_item, - IN void* context ) +__match_notice_to_inf_rec(IN cl_list_item_t * const p_list_item, + IN void *context) { - osm_infr_match_ctxt_t* p_infr_match = (osm_infr_match_ctxt_t *)context; - ib_mad_notice_attr_t* p_ntc = p_infr_match->p_ntc; - cl_list_t* p_infr_to_remove_list = p_infr_match->p_remove_infr_list; - osm_infr_t* p_infr_rec = (osm_infr_t*)p_list_item; - ib_inform_info_t *p_ii = &(p_infr_rec->inform_record.inform_info); - cl_status_t status = CL_NOT_FOUND; - osm_log_t *p_log = p_infr_rec->p_infr_rcv->p_log; - osm_subn_t *p_subn = p_infr_rec->p_infr_rcv->p_subn; - ib_gid_t source_gid; - osm_port_t* p_src_port; - osm_port_t* p_dest_port; - - OSM_LOG_ENTER( p_log, __match_notice_to_inf_rec ); - - /* matching rules - * InformInfo Notice - * GID IssuerGID if non zero must match the trap - * LIDRange IssuerLID apply only if GID=0 - * IsGeneric IsGeneric is compulsory and must match the trap - * Type Type if not 0xFFFF must match - * TrapNumber TrapNumber if not 0xFFFF must match - * DeviceId DeviceID if not 0xFFFF must match - * QPN dont care - * ProducerType ProducerType match or 0xFFFFFF // EZ: actually my interpretation - * VendorID VendorID match or 0xFFFFFF - */ - - /* GID IssuerGID if non zero must match the trap */ - if ( p_ii->gid.unicast.prefix != 0 || p_ii->gid.unicast.interface_id != 0 ) - { - /* match by GID */ - if ( memcmp(&(p_ii->gid), &(p_ntc->issuer_gid), sizeof(ib_gid_t)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by GID\n" ); - goto Exit; - } - } - else - { - /* LIDRange IssuerLID apply only if GID=0 */ - /* If lid_range_begin of the informInfo is 0xFFFF - then it should be ignored. */ - if ( p_ii->lid_range_begin != 0xFFFF ) - { - /* a real lid range is given - check it */ - if ( (cl_hton16(p_ii->lid_range_begin) > cl_hton16(p_ntc->issuer_lid)) || - (cl_hton16(p_ntc->issuer_lid) > cl_hton16(p_ii->lid_range_end)) ) { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by LID Range. Needed: 0x%X <= 0x%X <= 0x%X\n", - cl_hton16(p_ii->lid_range_begin), - cl_hton16(p_ntc->issuer_lid), - cl_hton16(p_ii->lid_range_end) - ); - goto Exit; - } - } - } - - /* IsGeneric IsGeneric is compulsory and must match the trap */ - if ( (p_ii->is_generic && ! ib_notice_is_generic(p_ntc)) || - (!p_ii->is_generic && ib_notice_is_generic(p_ntc)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Generic/Vendor\n" ); - goto Exit; - } - - /* Type Type if not 0xFFFF must match */ - if ( (p_ii->trap_type != 0xFFFF) && - (cl_ntoh16(p_ii->trap_type) != ib_notice_get_type(p_ntc)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Type\n" ); - goto Exit; - } - - /* based on generic type */ - if ( p_ii->is_generic ) - { - /* TrapNumber TrapNumber if not 0xFFFF must match */ - if ( (p_ii->g_or_v.generic.trap_num != 0xFFFF) && - (p_ii->g_or_v.generic.trap_num != p_ntc->g_or_v.generic.trap_num) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Trap Num\n" ); - goto Exit; - } - - /* ProducerType ProducerType match or 0xFFFFFF */ - if ( (cl_ntoh32(ib_inform_info_get_prod_type(p_ii)) != 0xFFFFFF) && - (ib_inform_info_get_prod_type(p_ii) != ib_notice_get_prod_type(p_ntc)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Node Type: II=0x%06X (%s) Trap=0x%06X (%s)\n", - cl_ntoh32(ib_inform_info_get_prod_type(p_ii)), - ib_get_producer_type_str(ib_inform_info_get_prod_type(p_ii)), - cl_ntoh32(ib_notice_get_prod_type(p_ntc)), - ib_get_producer_type_str(ib_notice_get_prod_type(p_ntc)) - ); - goto Exit; - } - } - else - { - /* DeviceId DeviceID if not 0xFFFF must match */ - if ( (p_ii->g_or_v.vend.dev_id != 0xFFFF) && - (p_ii->g_or_v.vend.dev_id != p_ntc->g_or_v.vend.dev_id) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Dev Id\n" ); - goto Exit; - } - - /* VendorID VendorID match or 0xFFFFFF */ - if ( (ib_inform_info_get_vend_id(p_ii) != CL_HTON32(0xFFFFFF)) && - (ib_inform_info_get_vend_id(p_ii) != ib_notice_get_vend_id(p_ntc)) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Vendor ID\n" ); - goto Exit; - } - } - - /* Check if there is a pkey match. o13-17.1.1 */ - /* Check if the issuer of the trap is the SM. If it is, then the gid - comparison should be done on the trap source (saved as the gid in the - data details field). - If the issuer gid is not the SM - then it is the guid of the trap - source */ - if ( (cl_ntoh64(p_ntc->issuer_gid.unicast.prefix) == p_subn->opt.subnet_prefix) && - (cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) == p_subn->sm_port_guid) ) - { - /* The issuer is the SM then this is trap 64-67 - compare the gid - with the gid saved on the data details */ - source_gid = p_ntc->data_details.ntc_64_67.gid; - } - else - { - source_gid = p_ntc->issuer_gid; - } - - p_src_port = osm_get_port_by_guid( p_subn, source_gid.unicast.interface_id ); - if( !p_src_port ) - { - osm_log( p_log, OSM_LOG_INFO, - "__match_notice_to_inf_rec: " - "Cannot find source port with GUID:0x%016" PRIx64 "\n", - cl_ntoh64(source_gid.unicast.interface_id) ); - goto Exit; - } - - p_dest_port = - cl_ptr_vector_get( &p_subn->port_lid_tbl, - cl_ntoh16(p_infr_rec->report_addr.dest_lid) ); - if( !p_dest_port ) - { - osm_log( p_log, OSM_LOG_INFO, - "__match_notice_to_inf_rec: " - "Cannot find destination port with LID:0x%04x\n", - cl_ntoh16(p_infr_rec->report_addr.dest_lid) ); - goto Exit; - } - - if (osm_port_share_pkey( p_log, p_src_port, p_dest_port ) == FALSE ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "Mismatch by Pkey\n" ); - /* According to o13-17.1.2 - If this informInfo does not have - lid_range_begin of 0xFFFF, then this informInfo request - should be removed from database */ - if ( p_ii->lid_range_begin != 0xFFFF ) - { - osm_log( p_log, OSM_LOG_VERBOSE, - "__match_notice_to_inf_rec: " - "Pkey mismatch on lid_range_begin != 0xFFFF. " - "Need to remove this informInfo from db\n" ); - /* add the informInfo record to the remove_infr list */ - cl_list_insert_tail( p_infr_to_remove_list, p_infr_rec ); - } - goto Exit; - } - - /* send the report to the address provided in the inform record */ - osm_log( p_log, OSM_LOG_DEBUG, - "__match_notice_to_inf_rec: " - "MATCH! Sending Report...\n" ); - __osm_send_report( p_infr_rec, p_ntc ); - status = CL_SUCCESS; - - Exit: - OSM_LOG_EXIT( p_log ); + osm_infr_match_ctxt_t *p_infr_match = (osm_infr_match_ctxt_t *) context; + ib_mad_notice_attr_t *p_ntc = p_infr_match->p_ntc; + cl_list_t *p_infr_to_remove_list = p_infr_match->p_remove_infr_list; + osm_infr_t *p_infr_rec = (osm_infr_t *) p_list_item; + ib_inform_info_t *p_ii = &(p_infr_rec->inform_record.inform_info); + cl_status_t status = CL_NOT_FOUND; + osm_log_t *p_log = p_infr_rec->p_infr_rcv->p_log; + osm_subn_t *p_subn = p_infr_rec->p_infr_rcv->p_subn; + ib_gid_t source_gid; + osm_port_t *p_src_port; + osm_port_t *p_dest_port; + + OSM_LOG_ENTER(p_log, __match_notice_to_inf_rec); + + /* matching rules + * InformInfo Notice + * GID IssuerGID if non zero must match the trap + * LIDRange IssuerLID apply only if GID=0 + * IsGeneric IsGeneric is compulsory and must match the trap + * Type Type if not 0xFFFF must match + * TrapNumber TrapNumber if not 0xFFFF must match + * DeviceId DeviceID if not 0xFFFF must match + * QPN dont care + * ProducerType ProducerType match or 0xFFFFFF // EZ: actually my interpretation + * VendorID VendorID match or 0xFFFFFF + */ + + /* GID IssuerGID if non zero must match the trap */ + if (p_ii->gid.unicast.prefix != 0 + || p_ii->gid.unicast.interface_id != 0) { + /* match by GID */ + if (memcmp + (&(p_ii->gid), &(p_ntc->issuer_gid), sizeof(ib_gid_t))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by GID\n"); + goto Exit; + } + } else { + /* LIDRange IssuerLID apply only if GID=0 */ + /* If lid_range_begin of the informInfo is 0xFFFF - then it should be ignored. */ + if (p_ii->lid_range_begin != 0xFFFF) { + /* a real lid range is given - check it */ + if ((cl_hton16(p_ii->lid_range_begin) > + cl_hton16(p_ntc->issuer_lid)) + || (cl_hton16(p_ntc->issuer_lid) > + cl_hton16(p_ii->lid_range_end))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by LID Range. Needed: 0x%X <= 0x%X <= 0x%X\n", + cl_hton16(p_ii->lid_range_begin), + cl_hton16(p_ntc->issuer_lid), + cl_hton16(p_ii->lid_range_end) + ); + goto Exit; + } + } + } + + /* IsGeneric IsGeneric is compulsory and must match the trap */ + if ((p_ii->is_generic && !ib_notice_is_generic(p_ntc)) || + (!p_ii->is_generic && ib_notice_is_generic(p_ntc))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by Generic/Vendor\n"); + goto Exit; + } + + /* Type Type if not 0xFFFF must match */ + if ((p_ii->trap_type != 0xFFFF) && + (cl_ntoh16(p_ii->trap_type) != ib_notice_get_type(p_ntc))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " "Mismatch by Type\n"); + goto Exit; + } + + /* based on generic type */ + if (p_ii->is_generic) { + /* TrapNumber TrapNumber if not 0xFFFF must match */ + if ((p_ii->g_or_v.generic.trap_num != 0xFFFF) && + (p_ii->g_or_v.generic.trap_num != + p_ntc->g_or_v.generic.trap_num)) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by Trap Num\n"); + goto Exit; + } + + /* ProducerType ProducerType match or 0xFFFFFF */ + if ((cl_ntoh32(ib_inform_info_get_prod_type(p_ii)) != 0xFFFFFF) + && (ib_inform_info_get_prod_type(p_ii) != + ib_notice_get_prod_type(p_ntc))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by Node Type: II=0x%06X (%s) Trap=0x%06X (%s)\n", + cl_ntoh32(ib_inform_info_get_prod_type(p_ii)), + ib_get_producer_type_str + (ib_inform_info_get_prod_type(p_ii)), + cl_ntoh32(ib_notice_get_prod_type(p_ntc)), + ib_get_producer_type_str(ib_notice_get_prod_type + (p_ntc)) + ); + goto Exit; + } + } else { + /* DeviceId DeviceID if not 0xFFFF must match */ + if ((p_ii->g_or_v.vend.dev_id != 0xFFFF) && + (p_ii->g_or_v.vend.dev_id != p_ntc->g_or_v.vend.dev_id)) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by Dev Id\n"); + goto Exit; + } + + /* VendorID VendorID match or 0xFFFFFF */ + if ((ib_inform_info_get_vend_id(p_ii) != CL_HTON32(0xFFFFFF)) && + (ib_inform_info_get_vend_id(p_ii) != + ib_notice_get_vend_id(p_ntc))) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " + "Mismatch by Vendor ID\n"); + goto Exit; + } + } + + /* Check if there is a pkey match. o13-17.1.1 */ + /* Check if the issuer of the trap is the SM. If it is, then the gid + comparison should be done on the trap source (saved as the gid in the + data details field). + If the issuer gid is not the SM - then it is the guid of the trap + source */ + if ((cl_ntoh64(p_ntc->issuer_gid.unicast.prefix) == + p_subn->opt.subnet_prefix) + && (cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) == + p_subn->sm_port_guid)) { + /* The issuer is the SM then this is trap 64-67 - compare the gid + with the gid saved on the data details */ + source_gid = p_ntc->data_details.ntc_64_67.gid; + } else { + source_gid = p_ntc->issuer_gid; + } + + p_src_port = + osm_get_port_by_guid(p_subn, source_gid.unicast.interface_id); + if (!p_src_port) { + osm_log(p_log, OSM_LOG_INFO, + "__match_notice_to_inf_rec: " + "Cannot find source port with GUID:0x%016" PRIx64 "\n", + cl_ntoh64(source_gid.unicast.interface_id)); + goto Exit; + } + + p_dest_port = + cl_ptr_vector_get(&p_subn->port_lid_tbl, + cl_ntoh16(p_infr_rec->report_addr.dest_lid)); + if (!p_dest_port) { + osm_log(p_log, OSM_LOG_INFO, + "__match_notice_to_inf_rec: " + "Cannot find destination port with LID:0x%04x\n", + cl_ntoh16(p_infr_rec->report_addr.dest_lid)); + goto Exit; + } + + if (osm_port_share_pkey(p_log, p_src_port, p_dest_port) == FALSE) { + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " "Mismatch by Pkey\n"); + /* According to o13-17.1.2 - If this informInfo does not have + lid_range_begin of 0xFFFF, then this informInfo request + should be removed from database */ + if (p_ii->lid_range_begin != 0xFFFF) { + osm_log(p_log, OSM_LOG_VERBOSE, + "__match_notice_to_inf_rec: " + "Pkey mismatch on lid_range_begin != 0xFFFF. " + "Need to remove this informInfo from db\n"); + /* add the informInfo record to the remove_infr list */ + cl_list_insert_tail(p_infr_to_remove_list, p_infr_rec); + } + goto Exit; + } + + /* send the report to the address provided in the inform record */ + osm_log(p_log, OSM_LOG_DEBUG, + "__match_notice_to_inf_rec: " "MATCH! Sending Report...\n"); + __osm_send_report(p_infr_rec, p_ntc); + status = CL_SUCCESS; + + Exit: + OSM_LOG_EXIT(p_log); } /********************************************************************** @@ -652,87 +609,80 @@ __match_notice_to_inf_rec( * target QP registered by the address stored in the InformInfo element **********************************************************************/ ib_api_status_t -osm_report_notice( - IN osm_log_t* const p_log, - IN osm_subn_t* p_subn, - IN ib_mad_notice_attr_t *p_ntc ) +osm_report_notice(IN osm_log_t * const p_log, + IN osm_subn_t * p_subn, IN ib_mad_notice_attr_t * p_ntc) { - osm_infr_match_ctxt_t context; - cl_list_t infr_to_remove_list; - osm_infr_t* p_infr_rec; - osm_infr_t* p_next_infr_rec; - - OSM_LOG_ENTER( p_log, osm_report_notice ); - - /* - * we must make sure we are ready for this... - * note that the trap receivers might be initialized before - * the osm_infr_init call is performed. - */ - if ( p_subn->sa_infr_list.state != CL_INITIALIZED ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_report_notice: " - "Ignoring Notice Reports since Inform List is not initialized yet!\n" ); - return(IB_ERROR); - } - - /* an official Event information log */ - if ( ib_notice_is_generic(p_ntc) ) - { - osm_log( p_log, OSM_LOG_INFO, - "osm_report_notice: " - "Reporting Generic Notice type:%u num:%u" - " from LID:0x%04X GID:0x%016" PRIx64 - ",0x%016" PRIx64 "\n", - ib_notice_get_type(p_ntc), - cl_ntoh16(p_ntc->g_or_v.generic.trap_num), - cl_ntoh16(p_ntc->issuer_lid), - cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), - cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) - ); - } - else - { - osm_log( p_log, OSM_LOG_INFO, - "osm_report_notice: " - "Reporting Vendor Notice type:%u vend:%u dev:%u" - " from LID:0x%04X GID:0x%016" PRIx64 - ",0x%016" PRIx64 "\n", - ib_notice_get_type(p_ntc), - cl_ntoh32(ib_notice_get_vend_id(p_ntc)), - cl_ntoh16(p_ntc->g_or_v.vend.dev_id), - cl_ntoh16(p_ntc->issuer_lid), - cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), - cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) - ); - } - - /* Create a list that will hold all the infr records that should - be removed due to violation. o13-17.1.2 */ - cl_list_construct( &infr_to_remove_list ); - cl_list_init( &infr_to_remove_list, 5 ); - context.p_remove_infr_list = &infr_to_remove_list; - context.p_ntc = p_ntc; - - /* go over all inform info available at the subnet */ - /* try match to the given notice and send if match */ - cl_qlist_apply_func(&(p_subn->sa_infr_list), - __match_notice_to_inf_rec, - &context ); - - /* If we inserted items into the infr_to_remove_list - we need to - remove them */ - p_infr_rec = (osm_infr_t*)cl_list_remove_head(&infr_to_remove_list); - while ( p_infr_rec != NULL ) - { - p_next_infr_rec = (osm_infr_t*)cl_list_remove_head(&infr_to_remove_list); - osm_infr_remove_from_db ( p_subn, p_log, p_infr_rec ); - p_infr_rec = p_next_infr_rec; - } - cl_list_destroy(&infr_to_remove_list); - - OSM_LOG_EXIT( p_log ); - - return(IB_SUCCESS); + osm_infr_match_ctxt_t context; + cl_list_t infr_to_remove_list; + osm_infr_t *p_infr_rec; + osm_infr_t *p_next_infr_rec; + + OSM_LOG_ENTER(p_log, osm_report_notice); + + /* + * we must make sure we are ready for this... + * note that the trap receivers might be initialized before + * the osm_infr_init call is performed. + */ + if (p_subn->sa_infr_list.state != CL_INITIALIZED) { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_report_notice: " + "Ignoring Notice Reports since Inform List is not initialized yet!\n"); + return (IB_ERROR); + } + + /* an official Event information log */ + if (ib_notice_is_generic(p_ntc)) { + osm_log(p_log, OSM_LOG_INFO, + "osm_report_notice: " + "Reporting Generic Notice type:%u num:%u" + " from LID:0x%04X GID:0x%016" PRIx64 + ",0x%016" PRIx64 "\n", + ib_notice_get_type(p_ntc), + cl_ntoh16(p_ntc->g_or_v.generic.trap_num), + cl_ntoh16(p_ntc->issuer_lid), + cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), + cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) + ); + } else { + osm_log(p_log, OSM_LOG_INFO, + "osm_report_notice: " + "Reporting Vendor Notice type:%u vend:%u dev:%u" + " from LID:0x%04X GID:0x%016" PRIx64 + ",0x%016" PRIx64 "\n", + ib_notice_get_type(p_ntc), + cl_ntoh32(ib_notice_get_vend_id(p_ntc)), + cl_ntoh16(p_ntc->g_or_v.vend.dev_id), + cl_ntoh16(p_ntc->issuer_lid), + cl_ntoh64(p_ntc->issuer_gid.unicast.prefix), + cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) + ); + } + + /* Create a list that will hold all the infr records that should + be removed due to violation. o13-17.1.2 */ + cl_list_construct(&infr_to_remove_list); + cl_list_init(&infr_to_remove_list, 5); + context.p_remove_infr_list = &infr_to_remove_list; + context.p_ntc = p_ntc; + + /* go over all inform info available at the subnet */ + /* try match to the given notice and send if match */ + cl_qlist_apply_func(&(p_subn->sa_infr_list), + __match_notice_to_inf_rec, &context); + + /* If we inserted items into the infr_to_remove_list - we need to + remove them */ + p_infr_rec = (osm_infr_t *) cl_list_remove_head(&infr_to_remove_list); + while (p_infr_rec != NULL) { + p_next_infr_rec = + (osm_infr_t *) cl_list_remove_head(&infr_to_remove_list); + osm_infr_remove_from_db(p_subn, p_log, p_infr_rec); + p_infr_rec = p_next_infr_rec; + } + cl_list_destroy(&infr_to_remove_list); + + OSM_LOG_EXIT(p_log); + + return (IB_SUCCESS); } diff --git a/opensm/opensm/osm_lin_fwd_rcv.c b/opensm/opensm/osm_lin_fwd_rcv.c index eda10d8..efc0b1b 100644 --- a/opensm/opensm/osm_lin_fwd_rcv.c +++ b/opensm/opensm/osm_lin_fwd_rcv.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_lft_rcv_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -57,106 +56,92 @@ /********************************************************************** **********************************************************************/ -void -osm_lft_rcv_construct( - IN osm_lft_rcv_t* const p_rcv ) +void osm_lft_rcv_construct(IN osm_lft_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_lft_rcv_destroy( - IN osm_lft_rcv_t* const p_rcv ) +void osm_lft_rcv_destroy(IN osm_lft_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_lft_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_lft_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_lft_rcv_init( - IN osm_lft_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_lft_rcv_init(IN osm_lft_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_lft_rcv_init ); + OSM_LOG_ENTER(p_log, osm_lft_rcv_init); - osm_lft_rcv_construct( p_rcv ); + osm_lft_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_lft_rcv_process( - IN void *context, - IN void *data ) +void osm_lft_rcv_process(IN void *context, IN void *data) { - osm_lft_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_smp_t *p_smp; - uint32_t block_num; - osm_switch_t *p_sw; - osm_lft_context_t *p_lft_context; - uint8_t *p_block; - ib_net64_t node_guid; - ib_api_status_t status; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_lft_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_block = (uint8_t*)ib_smp_get_payload_ptr( p_smp ); - block_num = cl_ntoh32( p_smp->attr_mod ); - - /* - Acquire the switch object for this switch. - */ - p_lft_context = osm_madw_get_lft_context_ptr( p_madw ); - node_guid = p_lft_context->node_guid; - - CL_PLOCK_EXCL_ACQUIRE( p_rcv->p_lock ); - p_sw = osm_get_switch_by_guid( p_rcv->p_subn, node_guid ); - - if( !p_sw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_lft_rcv_process: ERR 0401: " - "LFT received for nonexistent node " - "0x%" PRIx64 "\n", cl_ntoh64( node_guid ) ); - } - else - { - status = osm_switch_set_ft_block( p_sw, p_block, block_num ); - if( status != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_lft_rcv_process: ERR 0402: " - "Setting forwarding table block failed (%s)" - "\n\t\t\t\tSwitch 0x%" PRIx64 "\n", - ib_get_err_str( status ), - cl_ntoh64( node_guid ) ); - } - } - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - OSM_LOG_EXIT( p_rcv->p_log ); + osm_lft_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_smp_t *p_smp; + uint32_t block_num; + osm_switch_t *p_sw; + osm_lft_context_t *p_lft_context; + uint8_t *p_block; + ib_net64_t node_guid; + ib_api_status_t status; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_lft_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_block = (uint8_t *) ib_smp_get_payload_ptr(p_smp); + block_num = cl_ntoh32(p_smp->attr_mod); + + /* + Acquire the switch object for this switch. + */ + p_lft_context = osm_madw_get_lft_context_ptr(p_madw); + node_guid = p_lft_context->node_guid; + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + p_sw = osm_get_switch_by_guid(p_rcv->p_subn, node_guid); + + if (!p_sw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lft_rcv_process: ERR 0401: " + "LFT received for nonexistent node " + "0x%" PRIx64 "\n", cl_ntoh64(node_guid)); + } else { + status = osm_switch_set_ft_block(p_sw, p_block, block_num); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lft_rcv_process: ERR 0402: " + "Setting forwarding table block failed (%s)" + "\n\t\t\t\tSwitch 0x%" PRIx64 "\n", + ib_get_err_str(status), cl_ntoh64(node_guid)); + } + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_lin_fwd_tbl.c b/opensm/opensm/osm_lin_fwd_tbl.c index 2a60d81..d9f001a 100644 --- a/opensm/opensm/osm_lin_fwd_tbl.c +++ b/opensm/opensm/osm_lin_fwd_tbl.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_lin_fwd_tbl_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,47 +55,39 @@ #include #include - -inline size_t -__osm_lin_tbl_compute_obj_size( - IN const uint32_t num_ports ) +inline size_t __osm_lin_tbl_compute_obj_size(IN const uint32_t num_ports) { - return( sizeof(osm_lin_fwd_tbl_t) + (num_ports - 1) ); + return (sizeof(osm_lin_fwd_tbl_t) + (num_ports - 1)); } /********************************************************************** **********************************************************************/ -osm_lin_fwd_tbl_t* -osm_lin_tbl_new( - IN uint16_t const size ) +osm_lin_fwd_tbl_t *osm_lin_tbl_new(IN uint16_t const size) { - osm_lin_fwd_tbl_t* p_tbl; + osm_lin_fwd_tbl_t *p_tbl; - /* - The capacity reported by the switch includes LID 0, - so add 1 to the end of the range here for this assert. - */ - CL_ASSERT( size <= IB_LID_UCAST_END_HO + 1 ); - p_tbl = (osm_lin_fwd_tbl_t*)malloc( - __osm_lin_tbl_compute_obj_size( size ) ); + /* + The capacity reported by the switch includes LID 0, + so add 1 to the end of the range here for this assert. + */ + CL_ASSERT(size <= IB_LID_UCAST_END_HO + 1); + p_tbl = + (osm_lin_fwd_tbl_t *) malloc(__osm_lin_tbl_compute_obj_size(size)); - /* - Initialize the table to OSM_NO_PATH, which means "invalid port" - */ - memset( p_tbl, OSM_NO_PATH, __osm_lin_tbl_compute_obj_size( size ) ); - if( p_tbl != NULL ) - { - p_tbl->size = (uint16_t)size; - } - return( p_tbl ); + /* + Initialize the table to OSM_NO_PATH, which means "invalid port" + */ + memset(p_tbl, OSM_NO_PATH, __osm_lin_tbl_compute_obj_size(size)); + if (p_tbl != NULL) { + p_tbl->size = (uint16_t) size; + } + return (p_tbl); } /********************************************************************** **********************************************************************/ -void -osm_lin_tbl_delete( - IN osm_lin_fwd_tbl_t** const pp_tbl ) +void osm_lin_tbl_delete(IN osm_lin_fwd_tbl_t ** const pp_tbl) { - free( *pp_tbl ); - *pp_tbl = NULL; + free(*pp_tbl); + *pp_tbl = NULL; } diff --git a/opensm/opensm/osm_log.c b/opensm/opensm/osm_log.c index 85c1725..7efe93e 100644 --- a/opensm/opensm/osm_log.c +++ b/opensm/opensm/osm_log.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -66,28 +66,26 @@ static int log_exit_count = 0; #include static char *month_str[] = { - "Jan", - "Feb", - "Mar", - "Apr", - "May", - "Jun", - "Jul", - "Aug", - "Sep", - "Oct", - "Nov", - "Dec" + "Jan", + "Feb", + "Mar", + "Apr", + "May", + "Jun", + "Jul", + "Aug", + "Sep", + "Oct", + "Nov", + "Dec" }; #else -void -OsmReportState( - IN const char *p_str); -#endif /* ndef WIN32 */ +void OsmReportState(IN const char *p_str); +#endif /* ndef WIN32 */ #ifndef WIN32 -static void truncate_log_file(osm_log_t* const p_log) +static void truncate_log_file(osm_log_t * const p_log) { int fd = fileno(p_log->out_port); if (ftruncate(fd, 0) < 0) @@ -99,255 +97,247 @@ static void truncate_log_file(osm_log_t* const p_log) p_log->count = 0; } -#else /* Windows */ +#else /* Windows */ -static void truncate_log_file(osm_log_t* const p_log) +static void truncate_log_file(osm_log_t * const p_log) { - fprintf(stderr, "truncate_log_file: cannot truncate on windows system (yet)\n"); + fprintf(stderr, + "truncate_log_file: cannot truncate on windows system (yet)\n"); } -#endif /* ndef WIN32 */ +#endif /* ndef WIN32 */ -int osm_log_printf(osm_log_t *p_log, osm_log_level_t level, +int osm_log_printf(osm_log_t * p_log, osm_log_level_t level, const char *fmt, ...) { va_list args; int ret; - if (!(p_log->level&level)) + if (!(p_log->level & level)) return 0; va_start(args, fmt); ret = vfprintf(stdout, fmt, args); va_end(args); - if (p_log->flush || level&OSM_LOG_ERROR) - fflush( stdout ); + if (p_log->flush || level & OSM_LOG_ERROR) + fflush(stdout); return ret; } void -osm_log( - IN osm_log_t* const p_log, - IN const osm_log_level_t verbosity, - IN const char *p_str, ... ) +osm_log(IN osm_log_t * const p_log, + IN const osm_log_level_t verbosity, IN const char *p_str, ...) { - char buffer[LOG_ENTRY_SIZE_MAX]; - va_list args; - int ret; + char buffer[LOG_ENTRY_SIZE_MAX]; + va_list args; + int ret; #ifdef WIN32 - SYSTEMTIME st; - uint32_t pid = GetCurrentThreadId(); + SYSTEMTIME st; + uint32_t pid = GetCurrentThreadId(); #else - pid_t pid = 0; - time_t tim; - struct tm result; - uint64_t time_usecs; - uint32_t usecs; - - time_usecs = cl_get_time_stamp(); - tim = time_usecs/1000000; - usecs = time_usecs % 1000000; - localtime_r(&tim, &result); -#endif /* WIN32 */ - - /* If this is a call to syslog - always print it */ - if ( verbosity & (OSM_LOG_SYS | p_log->level) ) - { - va_start( args, p_str ); - vsprintf( buffer, p_str, args ); - va_end(args); - - /* this is a call to the syslog */ - if (verbosity & OSM_LOG_SYS) - { - syslog(LOG_INFO, "%s\n", buffer); - - /* SYSLOG should go to stdout too */ - if (p_log->out_port != stdout) - { - printf("%s\n", buffer); - fflush( stdout ); - } + pid_t pid = 0; + time_t tim; + struct tm result; + uint64_t time_usecs; + uint32_t usecs; + + time_usecs = cl_get_time_stamp(); + tim = time_usecs / 1000000; + usecs = time_usecs % 1000000; + localtime_r(&tim, &result); +#endif /* WIN32 */ + + /* If this is a call to syslog - always print it */ + if (verbosity & (OSM_LOG_SYS | p_log->level)) { + va_start(args, p_str); + vsprintf(buffer, p_str, args); + va_end(args); + + /* this is a call to the syslog */ + if (verbosity & OSM_LOG_SYS) { + syslog(LOG_INFO, "%s\n", buffer); + + /* SYSLOG should go to stdout too */ + if (p_log->out_port != stdout) { + printf("%s\n", buffer); + fflush(stdout); + } #ifdef WIN32 - OsmReportState(buffer); -#endif /* WIN32 */ - } - - /* regular log to default out_port */ - cl_spinlock_acquire( &p_log->lock ); - - if (p_log->max_size && p_log->count > p_log->max_size) - { - /* truncate here */ - fprintf(stderr, "osm_log: log file exceeds the limit %lu. Truncating.\n", - p_log->max_size); - truncate_log_file(p_log); - } - + OsmReportState(buffer); +#endif /* WIN32 */ + } + + /* regular log to default out_port */ + cl_spinlock_acquire(&p_log->lock); + + if (p_log->max_size && p_log->count > p_log->max_size) { + /* truncate here */ + fprintf(stderr, + "osm_log: log file exceeds the limit %lu. Truncating.\n", + p_log->max_size); + truncate_log_file(p_log); + } #ifdef WIN32 - GetLocalTime(&st); - _retry: - ret = fprintf( p_log->out_port, "[%02d:%02d:%02d:%03d][%04X] -> %s", - st.wHour, st.wMinute, st.wSecond, st.wMilliseconds, - pid, buffer ); + GetLocalTime(&st); + _retry: + ret = + fprintf(p_log->out_port, + "[%02d:%02d:%02d:%03d][%04X] -> %s", st.wHour, + st.wMinute, st.wSecond, st.wMilliseconds, pid, + buffer); #else - pid = pthread_self(); - _retry: - ret = fprintf( p_log->out_port, "%s %02d %02d:%02d:%02d %06d [%04X] -> %s", - (result.tm_mon < 12 ? month_str[result.tm_mon] : "???"), - result.tm_mday, result.tm_hour, - result.tm_min, result.tm_sec, - usecs, pid, buffer ); + pid = pthread_self(); + _retry: + ret = + fprintf(p_log->out_port, + "%s %02d %02d:%02d:%02d %06d [%04X] -> %s", + (result.tm_mon < + 12 ? month_str[result.tm_mon] : "???"), + result.tm_mday, result.tm_hour, result.tm_min, + result.tm_sec, usecs, pid, buffer); #endif - /* flush log */ - if ( ret > 0 && - (p_log->flush || (verbosity & (OSM_LOG_ERROR | OSM_LOG_SYS))) && - fflush( p_log->out_port ) < 0 ) - ret = -1; - - if (ret >= 0) - { - log_exit_count = 0; - p_log->count += ret; - } - else if (log_exit_count < 3) - { - log_exit_count++; - if (errno == ENOSPC && p_log->max_size) { - fprintf(stderr, "osm_log: write failed: %s. Truncating log file.\n", - strerror(errno)); - truncate_log_file(p_log); - goto _retry; - } - fprintf(stderr, "osm_log: write failed: %s\n", strerror(errno)); - } - - cl_spinlock_release( &p_log->lock ); - } + /* flush log */ + if (ret > 0 && + (p_log->flush + || (verbosity & (OSM_LOG_ERROR | OSM_LOG_SYS))) + && fflush(p_log->out_port) < 0) + ret = -1; + + if (ret >= 0) { + log_exit_count = 0; + p_log->count += ret; + } else if (log_exit_count < 3) { + log_exit_count++; + if (errno == ENOSPC && p_log->max_size) { + fprintf(stderr, + "osm_log: write failed: %s. Truncating log file.\n", + strerror(errno)); + truncate_log_file(p_log); + goto _retry; + } + fprintf(stderr, "osm_log: write failed: %s\n", + strerror(errno)); + } + + cl_spinlock_release(&p_log->lock); + } } void -osm_log_raw( - IN osm_log_t* const p_log, - IN const osm_log_level_t verbosity, - IN const char *p_buf ) +osm_log_raw(IN osm_log_t * const p_log, + IN const osm_log_level_t verbosity, IN const char *p_buf) { - if( p_log->level & verbosity ) - { - cl_spinlock_acquire( &p_log->lock ); - printf( "%s", p_buf ); - cl_spinlock_release( &p_log->lock ); - - /* - Flush log on errors too. - */ - if( p_log->flush || (verbosity & OSM_LOG_ERROR) ) - fflush( stdout ); - } + if (p_log->level & verbosity) { + cl_spinlock_acquire(&p_log->lock); + printf("%s", p_buf); + cl_spinlock_release(&p_log->lock); + + /* + Flush log on errors too. + */ + if (p_log->flush || (verbosity & OSM_LOG_ERROR)) + fflush(stdout); + } } -boolean_t -osm_is_debug(void) +boolean_t osm_is_debug(void) { #if defined( _DEBUG_ ) - return TRUE; + return TRUE; #else - return FALSE; -#endif /* defined( _DEBUG_ ) */ + return FALSE; +#endif /* defined( _DEBUG_ ) */ } -static int -open_out_port(IN osm_log_t *p_log) +static int open_out_port(IN osm_log_t * p_log) { - struct stat st; - - if (p_log->accum_log_file) - p_log->out_port = fopen(p_log->log_file_name, "a+"); - else - p_log->out_port = fopen(p_log->log_file_name, "w+"); - - if (!p_log->out_port) - { - if (p_log->accum_log_file) - syslog(LOG_CRIT, "Cannot open %s for appending. Permission denied\n", - p_log->log_file_name); - else - syslog(LOG_CRIT, "Cannot open %s for writing. Permission denied\n", - p_log->log_file_name); - - return(IB_UNKNOWN_ERROR); - } - - if (fstat(fileno(p_log->out_port), &st) == 0) - p_log->count = st.st_size; - - syslog(LOG_NOTICE, "%s log file opened\n", p_log->log_file_name); - - if (p_log->daemon) { - dup2(fileno(p_log->out_port), 0); - dup2(fileno(p_log->out_port), 1); - dup2(fileno(p_log->out_port), 2); - } - - return (0); + struct stat st; + + if (p_log->accum_log_file) + p_log->out_port = fopen(p_log->log_file_name, "a+"); + else + p_log->out_port = fopen(p_log->log_file_name, "w+"); + + if (!p_log->out_port) { + if (p_log->accum_log_file) + syslog(LOG_CRIT, + "Cannot open %s for appending. Permission denied\n", + p_log->log_file_name); + else + syslog(LOG_CRIT, + "Cannot open %s for writing. Permission denied\n", + p_log->log_file_name); + + return (IB_UNKNOWN_ERROR); + } + + if (fstat(fileno(p_log->out_port), &st) == 0) + p_log->count = st.st_size; + + syslog(LOG_NOTICE, "%s log file opened\n", p_log->log_file_name); + + if (p_log->daemon) { + dup2(fileno(p_log->out_port), 0); + dup2(fileno(p_log->out_port), 1); + dup2(fileno(p_log->out_port), 2); + } + + return (0); } -int -osm_log_reopen_file(osm_log_t *p_log) +int osm_log_reopen_file(osm_log_t * p_log) { - int ret; - - if (p_log->out_port == stdout || p_log->out_port == stderr) - return 0; - cl_spinlock_acquire(&p_log->lock); - fclose(p_log->out_port); - ret = open_out_port(p_log); - cl_spinlock_release(&p_log->lock); - return ret; + int ret; + + if (p_log->out_port == stdout || p_log->out_port == stderr) + return 0; + cl_spinlock_acquire(&p_log->lock); + fclose(p_log->out_port); + ret = open_out_port(p_log); + cl_spinlock_release(&p_log->lock); + return ret; } ib_api_status_t -osm_log_init_v2( - IN osm_log_t* const p_log, - IN const boolean_t flush, - IN const uint8_t log_flags, - IN const char *log_file, - IN const unsigned long max_size, - IN const boolean_t accum_log_file ) +osm_log_init_v2(IN osm_log_t * const p_log, + IN const boolean_t flush, + IN const uint8_t log_flags, + IN const char *log_file, + IN const unsigned long max_size, + IN const boolean_t accum_log_file) { - p_log->level = log_flags; - p_log->flush = flush; - p_log->count = 0; - p_log->max_size = max_size; - p_log->accum_log_file = accum_log_file; - p_log->log_file_name = (char *)log_file; - - openlog("OpenSM", LOG_CONS | LOG_PID, LOG_USER); - - if (log_file == NULL || !strcmp(log_file, "-") || - !strcmp(log_file, "stdout")) - p_log->out_port = stdout; - else if (!strcmp(log_file, "stderr")) - p_log->out_port = stderr; - else if (open_out_port(p_log)) - return (IB_UNKNOWN_ERROR); - - if (cl_spinlock_init( &p_log->lock ) == CL_SUCCESS) - return IB_SUCCESS; - else - return IB_ERROR; + p_log->level = log_flags; + p_log->flush = flush; + p_log->count = 0; + p_log->max_size = max_size; + p_log->accum_log_file = accum_log_file; + p_log->log_file_name = (char *)log_file; + + openlog("OpenSM", LOG_CONS | LOG_PID, LOG_USER); + + if (log_file == NULL || !strcmp(log_file, "-") || + !strcmp(log_file, "stdout")) + p_log->out_port = stdout; + else if (!strcmp(log_file, "stderr")) + p_log->out_port = stderr; + else if (open_out_port(p_log)) + return (IB_UNKNOWN_ERROR); + + if (cl_spinlock_init(&p_log->lock) == CL_SUCCESS) + return IB_SUCCESS; + else + return IB_ERROR; } ib_api_status_t -osm_log_init( - IN osm_log_t* const p_log, - IN const boolean_t flush, - IN const uint8_t log_flags, - IN const char *log_file, - IN const boolean_t accum_log_file ) +osm_log_init(IN osm_log_t * const p_log, + IN const boolean_t flush, + IN const uint8_t log_flags, + IN const char *log_file, IN const boolean_t accum_log_file) { - return osm_log_init_v2( p_log, flush, log_flags, log_file, 0, accum_log_file ); + return osm_log_init_v2(p_log, flush, log_flags, log_file, 0, + accum_log_file); } diff --git a/opensm/opensm/osm_mad_pool.c b/opensm/opensm/osm_mad_pool.c index da7c879..c3f3f2a 100644 --- a/opensm/opensm/osm_mad_pool.c +++ b/opensm/opensm/osm_mad_pool.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_mad_pool_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -60,243 +59,219 @@ #define OSM_MAD_POOL_MIN_SIZE 256 #define OSM_MAD_POOL_GROW_SIZE 256 - /********************************************************************** **********************************************************************/ cl_status_t -__osm_mad_pool_ctor( - IN void* const p_object, - IN void* context, - OUT cl_pool_item_t** const pp_pool_item ) +__osm_mad_pool_ctor(IN void *const p_object, + IN void *context, OUT cl_pool_item_t ** const pp_pool_item) { - osm_madw_t *p_madw = p_object; + osm_madw_t *p_madw = p_object; - UNUSED_PARAM( context ); - osm_madw_construct( p_madw ); - /* CHECK THIS. DOCS DON'T DESCRIBE THIS OUT PARAM. */ - *pp_pool_item = &p_madw->pool_item; - return( CL_SUCCESS ); + UNUSED_PARAM(context); + osm_madw_construct(p_madw); + /* CHECK THIS. DOCS DON'T DESCRIBE THIS OUT PARAM. */ + *pp_pool_item = &p_madw->pool_item; + return (CL_SUCCESS); } /********************************************************************** **********************************************************************/ -void -osm_mad_pool_construct( - IN osm_mad_pool_t* const p_pool ) +void osm_mad_pool_construct(IN osm_mad_pool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - memset( p_pool, 0, sizeof(*p_pool) ); - cl_qlock_pool_construct( &p_pool->madw_pool ); + memset(p_pool, 0, sizeof(*p_pool)); + cl_qlock_pool_construct(&p_pool->madw_pool); } /********************************************************************** **********************************************************************/ -void -osm_mad_pool_destroy( - IN osm_mad_pool_t* const p_pool ) +void osm_mad_pool_destroy(IN osm_mad_pool_t * const p_pool) { - CL_ASSERT( p_pool ); + CL_ASSERT(p_pool); - /* HACK: we still rarely see some mads leaking - so ignore this */ - /* cl_qlock_pool_destroy( &p_pool->madw_pool ); */ + /* HACK: we still rarely see some mads leaking - so ignore this */ + /* cl_qlock_pool_destroy( &p_pool->madw_pool ); */ } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mad_pool_init( - IN osm_mad_pool_t* const p_pool, - IN osm_log_t* const p_log ) +osm_mad_pool_init(IN osm_mad_pool_t * const p_pool, IN osm_log_t * const p_log) { - ib_api_status_t status; - - OSM_LOG_ENTER( p_log, osm_mad_pool_init ); - - p_pool->p_log = p_log; - - status = cl_qlock_pool_init( - &p_pool->madw_pool, - OSM_MAD_POOL_MIN_SIZE, - 0, - OSM_MAD_POOL_GROW_SIZE, - sizeof( osm_madw_t ), - __osm_mad_pool_ctor, - NULL, - p_pool ); - if( status != IB_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_mad_pool_init: ERR 0702: " - "Grow pool initialization failed (%s)\n", - ib_get_err_str(status) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status; + + OSM_LOG_ENTER(p_log, osm_mad_pool_init); + + p_pool->p_log = p_log; + + status = cl_qlock_pool_init(&p_pool->madw_pool, + OSM_MAD_POOL_MIN_SIZE, + 0, + OSM_MAD_POOL_GROW_SIZE, + sizeof(osm_madw_t), + __osm_mad_pool_ctor, NULL, p_pool); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_mad_pool_init: ERR 0702: " + "Grow pool initialization failed (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ -osm_madw_t* -osm_mad_pool_get( - IN osm_mad_pool_t* const p_pool, - IN osm_bind_handle_t h_bind, - IN const uint32_t total_size, - IN const osm_mad_addr_t* const p_mad_addr ) +osm_madw_t *osm_mad_pool_get(IN osm_mad_pool_t * const p_pool, + IN osm_bind_handle_t h_bind, + IN const uint32_t total_size, + IN const osm_mad_addr_t * const p_mad_addr) { - osm_madw_t *p_madw; - ib_mad_t *p_mad; - - OSM_LOG_ENTER( p_pool->p_log, osm_mad_pool_get ); - - CL_ASSERT( h_bind != OSM_BIND_INVALID_HANDLE ); - CL_ASSERT( total_size ); - - /* - First, acquire a mad wrapper from the mad wrapper pool. - */ - p_madw = (osm_madw_t*)cl_qlock_pool_get( &p_pool->madw_pool ); - if( p_madw == NULL ) - { - osm_log( p_pool->p_log, OSM_LOG_ERROR, - "osm_mad_pool_get: ERR 0703: " - "Unable to acquire MAD wrapper object\n" ); - goto Exit; - } - - osm_madw_init( p_madw, h_bind, total_size, p_mad_addr ); - - /* - Next, acquire a wire mad of the specified size. - */ - p_mad = osm_vendor_get( h_bind, total_size, &p_madw->vend_wrap ); - if( p_mad == NULL ) - { - osm_log( p_pool->p_log, OSM_LOG_ERROR, - "osm_mad_pool_get: ERR 0704: " - "Unable to acquire wire MAD\n" ); - - /* Don't leak wrappers! */ - cl_qlock_pool_put( &p_pool->madw_pool, (cl_pool_item_t*)p_madw ); - p_madw = NULL; - goto Exit; - } - - cl_atomic_inc( &p_pool->mads_out ); - /* - Finally, attach the wire MAD to this wrapper. - */ - osm_madw_set_mad( p_madw, p_mad ); - - osm_log( p_pool->p_log, OSM_LOG_DEBUG, - "osm_mad_pool_get: Acquired p_madw = %p, p_mad = %p, " - "size = %u\n", p_madw, p_madw->p_mad, total_size ); - - Exit: - OSM_LOG_EXIT( p_pool->p_log ); - return( p_madw ); + osm_madw_t *p_madw; + ib_mad_t *p_mad; + + OSM_LOG_ENTER(p_pool->p_log, osm_mad_pool_get); + + CL_ASSERT(h_bind != OSM_BIND_INVALID_HANDLE); + CL_ASSERT(total_size); + + /* + First, acquire a mad wrapper from the mad wrapper pool. + */ + p_madw = (osm_madw_t *) cl_qlock_pool_get(&p_pool->madw_pool); + if (p_madw == NULL) { + osm_log(p_pool->p_log, OSM_LOG_ERROR, + "osm_mad_pool_get: ERR 0703: " + "Unable to acquire MAD wrapper object\n"); + goto Exit; + } + + osm_madw_init(p_madw, h_bind, total_size, p_mad_addr); + + /* + Next, acquire a wire mad of the specified size. + */ + p_mad = osm_vendor_get(h_bind, total_size, &p_madw->vend_wrap); + if (p_mad == NULL) { + osm_log(p_pool->p_log, OSM_LOG_ERROR, + "osm_mad_pool_get: ERR 0704: " + "Unable to acquire wire MAD\n"); + + /* Don't leak wrappers! */ + cl_qlock_pool_put(&p_pool->madw_pool, + (cl_pool_item_t *) p_madw); + p_madw = NULL; + goto Exit; + } + + cl_atomic_inc(&p_pool->mads_out); + /* + Finally, attach the wire MAD to this wrapper. + */ + osm_madw_set_mad(p_madw, p_mad); + + osm_log(p_pool->p_log, OSM_LOG_DEBUG, + "osm_mad_pool_get: Acquired p_madw = %p, p_mad = %p, " + "size = %u\n", p_madw, p_madw->p_mad, total_size); + + Exit: + OSM_LOG_EXIT(p_pool->p_log); + return (p_madw); } /********************************************************************** **********************************************************************/ -osm_madw_t* -osm_mad_pool_get_wrapper( - IN osm_mad_pool_t* const p_pool, - IN osm_bind_handle_t h_bind, - IN const uint32_t total_size, - IN const ib_mad_t* const p_mad, - IN const osm_mad_addr_t* const p_mad_addr ) +osm_madw_t *osm_mad_pool_get_wrapper(IN osm_mad_pool_t * const p_pool, + IN osm_bind_handle_t h_bind, + IN const uint32_t total_size, + IN const ib_mad_t * const p_mad, + IN const osm_mad_addr_t * const p_mad_addr) { - osm_madw_t *p_madw; - - OSM_LOG_ENTER( p_pool->p_log, osm_mad_pool_get_wrapper ); - - CL_ASSERT( h_bind != OSM_BIND_INVALID_HANDLE ); - CL_ASSERT( total_size ); - CL_ASSERT( p_mad ); - - /* - First, acquire a mad wrapper from the mad wrapper pool. - */ - p_madw = (osm_madw_t*)cl_qlock_pool_get( &p_pool->madw_pool ); - if( p_madw == NULL ) - { - osm_log( p_pool->p_log, OSM_LOG_ERROR, - "osm_mad_pool_get_wrapper: ERR 0705: " - "Unable to acquire MAD wrapper object\n" ); - goto Exit; - } - - /* - Finally, initialize the wrapper object. - */ - cl_atomic_inc( &p_pool->mads_out ); - osm_madw_init( p_madw, h_bind, total_size, p_mad_addr ); - osm_madw_set_mad( p_madw, p_mad ); - - osm_log( p_pool->p_log, OSM_LOG_DEBUG, - "osm_mad_pool_get_wrapper: Acquired p_madw = %p, p_mad = %p " - "size = %u\n", p_madw, p_madw->p_mad, total_size ); - - Exit: - OSM_LOG_EXIT( p_pool->p_log ); - return( p_madw ); + osm_madw_t *p_madw; + + OSM_LOG_ENTER(p_pool->p_log, osm_mad_pool_get_wrapper); + + CL_ASSERT(h_bind != OSM_BIND_INVALID_HANDLE); + CL_ASSERT(total_size); + CL_ASSERT(p_mad); + + /* + First, acquire a mad wrapper from the mad wrapper pool. + */ + p_madw = (osm_madw_t *) cl_qlock_pool_get(&p_pool->madw_pool); + if (p_madw == NULL) { + osm_log(p_pool->p_log, OSM_LOG_ERROR, + "osm_mad_pool_get_wrapper: ERR 0705: " + "Unable to acquire MAD wrapper object\n"); + goto Exit; + } + + /* + Finally, initialize the wrapper object. + */ + cl_atomic_inc(&p_pool->mads_out); + osm_madw_init(p_madw, h_bind, total_size, p_mad_addr); + osm_madw_set_mad(p_madw, p_mad); + + osm_log(p_pool->p_log, OSM_LOG_DEBUG, + "osm_mad_pool_get_wrapper: Acquired p_madw = %p, p_mad = %p " + "size = %u\n", p_madw, p_madw->p_mad, total_size); + + Exit: + OSM_LOG_EXIT(p_pool->p_log); + return (p_madw); } /********************************************************************** **********************************************************************/ -osm_madw_t* -osm_mad_pool_get_wrapper_raw( - IN osm_mad_pool_t* const p_pool ) +osm_madw_t *osm_mad_pool_get_wrapper_raw(IN osm_mad_pool_t * const p_pool) { - osm_madw_t *p_madw; + osm_madw_t *p_madw; - OSM_LOG_ENTER( p_pool->p_log, osm_mad_pool_get_wrapper_raw ); + OSM_LOG_ENTER(p_pool->p_log, osm_mad_pool_get_wrapper_raw); - p_madw = (osm_madw_t*)cl_qlock_pool_get( &p_pool->madw_pool ); + p_madw = (osm_madw_t *) cl_qlock_pool_get(&p_pool->madw_pool); - osm_log( p_pool->p_log, OSM_LOG_DEBUG, - "osm_mad_pool_get_wrapper_raw: " - "Getting p_madw = %p\n", p_madw ); + osm_log(p_pool->p_log, OSM_LOG_DEBUG, + "osm_mad_pool_get_wrapper_raw: " + "Getting p_madw = %p\n", p_madw); - osm_madw_init( p_madw, 0, 0, 0 ); - osm_madw_set_mad( p_madw, 0 ); - cl_atomic_inc( &p_pool->mads_out ); + osm_madw_init(p_madw, 0, 0, 0); + osm_madw_set_mad(p_madw, 0); + cl_atomic_inc(&p_pool->mads_out); - OSM_LOG_EXIT( p_pool->p_log ); - return( p_madw ); + OSM_LOG_EXIT(p_pool->p_log); + return (p_madw); } /********************************************************************** **********************************************************************/ void -osm_mad_pool_put( - IN osm_mad_pool_t* const p_pool, - IN osm_madw_t* const p_madw ) +osm_mad_pool_put(IN osm_mad_pool_t * const p_pool, IN osm_madw_t * const p_madw) { - OSM_LOG_ENTER( p_pool->p_log, osm_mad_pool_put ); + OSM_LOG_ENTER(p_pool->p_log, osm_mad_pool_put); - CL_ASSERT( p_madw ); + CL_ASSERT(p_madw); - osm_log( p_pool->p_log, OSM_LOG_DEBUG, - "osm_mad_pool_put: Releasing p_madw = %p, p_mad = %p\n", - p_madw, p_madw->p_mad ); + osm_log(p_pool->p_log, OSM_LOG_DEBUG, + "osm_mad_pool_put: Releasing p_madw = %p, p_mad = %p\n", + p_madw, p_madw->p_mad); - /* - First, return the wire mad to the pool - */ - if( p_madw->p_mad ) - osm_vendor_put( p_madw->h_bind, &p_madw->vend_wrap ); + /* + First, return the wire mad to the pool + */ + if (p_madw->p_mad) + osm_vendor_put(p_madw->h_bind, &p_madw->vend_wrap); - /* - Return the mad wrapper to the wrapper pool - */ - cl_qlock_pool_put( &p_pool->madw_pool, (cl_pool_item_t*)p_madw ); - cl_atomic_dec( &p_pool->mads_out ); + /* + Return the mad wrapper to the wrapper pool + */ + cl_qlock_pool_put(&p_pool->madw_pool, (cl_pool_item_t *) p_madw); + cl_atomic_dec(&p_pool->mads_out); - OSM_LOG_EXIT( p_pool->p_log ); + OSM_LOG_EXIT(p_pool->p_log); } - diff --git a/opensm/opensm/osm_mcast_fwd_rcv.c b/opensm/opensm/osm_mcast_fwd_rcv.c index e3e9491..aa497b5 100644 --- a/opensm/opensm/osm_mcast_fwd_rcv.c +++ b/opensm/opensm/osm_mcast_fwd_rcv.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -62,122 +62,109 @@ /********************************************************************** **********************************************************************/ -void -osm_mft_rcv_construct( - IN osm_mft_rcv_t* const p_rcv ) +void osm_mft_rcv_construct(IN osm_mft_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_mft_rcv_destroy( - IN osm_mft_rcv_t* const p_rcv ) +void osm_mft_rcv_destroy(IN osm_mft_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_mft_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_mft_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mft_rcv_init( - IN osm_mft_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_mft_rcv_init(IN osm_mft_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_mft_rcv_init ); + OSM_LOG_ENTER(p_log, osm_mft_rcv_init); - osm_mft_rcv_construct( p_rcv ); + osm_mft_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_mft_rcv_process( - IN void *context, - IN void *data ) +void osm_mft_rcv_process(IN void *context, IN void *data) { - osm_mft_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_smp_t *p_smp; - uint32_t block_num; - uint8_t position; - osm_switch_t *p_sw; - osm_mft_context_t *p_mft_context; - uint16_t *p_block; - ib_net64_t node_guid; - ib_api_status_t status; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_mft_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_block = (uint16_t*)ib_smp_get_payload_ptr( p_smp ); - block_num = cl_ntoh32( p_smp->attr_mod ) & IB_MCAST_BLOCK_ID_MASK_HO; - position = (uint8_t)((cl_ntoh32( p_smp->attr_mod ) & - IB_MCAST_POSITION_MASK_HO) >> IB_MCAST_POSITION_SHIFT); - - /* - Acquire the switch object for this switch. - */ - p_mft_context = osm_madw_get_mft_context_ptr( p_madw ); - node_guid = p_mft_context->node_guid; - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_mft_rcv_process: " - "Setting MFT block %u, position %u, " - "Switch 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", - block_num, position, cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - } - - CL_PLOCK_EXCL_ACQUIRE( p_rcv->p_lock ); - p_sw = osm_get_switch_by_guid( p_rcv->p_subn, node_guid ); - - if( !p_sw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mft_rcv_process: ERR 0801: " - "MFT received for nonexistent node " - "0x%016" PRIx64 "\n", cl_ntoh64( node_guid ) ); - } - else - { - status = osm_switch_set_mft_block( p_sw, p_block, - (uint16_t)block_num, position ); - if( status != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mft_rcv_process: ERR 0802: " - "Setting MFT block failed (%s)" - "\n\t\t\t\tSwitch 0x%016" PRIx64 - ", block %u, position %u\n", - ib_get_err_str( status ), - cl_ntoh64( node_guid ), - block_num, position ); - } - } - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - OSM_LOG_EXIT( p_rcv->p_log ); + osm_mft_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_smp_t *p_smp; + uint32_t block_num; + uint8_t position; + osm_switch_t *p_sw; + osm_mft_context_t *p_mft_context; + uint16_t *p_block; + ib_net64_t node_guid; + ib_api_status_t status; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_mft_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_block = (uint16_t *) ib_smp_get_payload_ptr(p_smp); + block_num = cl_ntoh32(p_smp->attr_mod) & IB_MCAST_BLOCK_ID_MASK_HO; + position = (uint8_t) ((cl_ntoh32(p_smp->attr_mod) & + IB_MCAST_POSITION_MASK_HO) >> + IB_MCAST_POSITION_SHIFT); + + /* + Acquire the switch object for this switch. + */ + p_mft_context = osm_madw_get_mft_context_ptr(p_madw); + node_guid = p_mft_context->node_guid; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_mft_rcv_process: " + "Setting MFT block %u, position %u, " + "Switch 0x%016" PRIx64 ", TID 0x%" PRIx64 "\n", + block_num, position, cl_ntoh64(node_guid), + cl_ntoh64(p_smp->trans_id)); + } + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + p_sw = osm_get_switch_by_guid(p_rcv->p_subn, node_guid); + + if (!p_sw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mft_rcv_process: ERR 0801: " + "MFT received for nonexistent node " + "0x%016" PRIx64 "\n", cl_ntoh64(node_guid)); + } else { + status = osm_switch_set_mft_block(p_sw, p_block, + (uint16_t) block_num, + position); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mft_rcv_process: ERR 0802: " + "Setting MFT block failed (%s)" + "\n\t\t\t\tSwitch 0x%016" PRIx64 + ", block %u, position %u\n", + ib_get_err_str(status), + cl_ntoh64(node_guid), block_num, position); + } + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_mcast_mgr.c b/opensm/opensm/osm_mcast_mgr.c index 5f64b19..d1ecf61 100644 --- a/opensm/opensm/osm_mcast_mgr.c +++ b/opensm/opensm/osm_mcast_mgr.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -62,143 +62,134 @@ /********************************************************************** **********************************************************************/ -typedef struct _osm_mcast_work_obj -{ - cl_list_item_t list_item; - osm_port_t* p_port; +typedef struct _osm_mcast_work_obj { + cl_list_item_t list_item; + osm_port_t *p_port; } osm_mcast_work_obj_t; /********************************************************************** **********************************************************************/ -static osm_mcast_work_obj_t* -__osm_mcast_work_obj_new( - IN const osm_port_t* const p_port ) +static osm_mcast_work_obj_t *__osm_mcast_work_obj_new(IN const osm_port_t * + const p_port) { - /* - TO DO - get these objects from a lockpool. - */ - osm_mcast_work_obj_t* p_obj; - - /* - clean allocated memory to avoid assertion when trying to insert to - qlist. - see cl_qlist_insert_tail(): CL_ASSERT(p_list_item->p_list != p_list) - */ - p_obj = malloc( sizeof( *p_obj ) ); - if( p_obj ) - { - memset( p_obj, 0, sizeof( *p_obj ) ); - p_obj->p_port = (osm_port_t*)p_port; - } - - return( p_obj ); + /* + TO DO - get these objects from a lockpool. + */ + osm_mcast_work_obj_t *p_obj; + + /* + clean allocated memory to avoid assertion when trying to insert to + qlist. + see cl_qlist_insert_tail(): CL_ASSERT(p_list_item->p_list != p_list) + */ + p_obj = malloc(sizeof(*p_obj)); + if (p_obj) { + memset(p_obj, 0, sizeof(*p_obj)); + p_obj->p_port = (osm_port_t *) p_port; + } + + return (p_obj); } /********************************************************************** **********************************************************************/ -static void -__osm_mcast_work_obj_delete( - IN osm_mcast_work_obj_t* p_wobj ) +static void __osm_mcast_work_obj_delete(IN osm_mcast_work_obj_t * p_wobj) { - free( p_wobj ); + free(p_wobj); } /********************************************************************** Recursively remove nodes from the tree *********************************************************************/ -static void -__osm_mcast_mgr_purge_tree_node( - IN osm_mtree_node_t* p_mtn ) +static void __osm_mcast_mgr_purge_tree_node(IN osm_mtree_node_t * p_mtn) { - uint8_t i; + uint8_t i; - for( i = 0; i < p_mtn->max_children; i++ ) - { - if( p_mtn->child_array[i] && - (p_mtn->child_array[i] != OSM_MTREE_LEAF) ) - __osm_mcast_mgr_purge_tree_node( p_mtn->child_array[i] ); + for (i = 0; i < p_mtn->max_children; i++) { + if (p_mtn->child_array[i] && + (p_mtn->child_array[i] != OSM_MTREE_LEAF)) + __osm_mcast_mgr_purge_tree_node(p_mtn->child_array[i]); - p_mtn->child_array[i] = NULL; + p_mtn->child_array[i] = NULL; - } + } - free( p_mtn ); + free(p_mtn); } /********************************************************************** **********************************************************************/ static void -__osm_mcast_mgr_purge_tree( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_mgrp_t* const p_mgrp ) +__osm_mcast_mgr_purge_tree(IN osm_mcast_mgr_t * const p_mgr, + IN osm_mgrp_t * const p_mgrp) { - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_purge_tree ); + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_purge_tree); - if( p_mgrp->p_root ) - __osm_mcast_mgr_purge_tree_node( p_mgrp->p_root ); + if (p_mgrp->p_root) + __osm_mcast_mgr_purge_tree_node(p_mgrp->p_root); - p_mgrp->p_root = NULL; + p_mgrp->p_root = NULL; - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static float -osm_mcast_mgr_compute_avg_hops( - osm_mcast_mgr_t* const p_mgr, - const osm_mgrp_t* const p_mgrp, - const osm_switch_t* const p_sw ) +osm_mcast_mgr_compute_avg_hops(osm_mcast_mgr_t * const p_mgr, + const osm_mgrp_t * const p_mgrp, + const osm_switch_t * const p_sw) { - float avg_hops = 0; - uint32_t hops = 0; - uint32_t num_ports = 0; - const osm_port_t* p_port; - const osm_mcm_port_t* p_mcm_port; - const cl_qmap_t* p_mcm_tbl; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_compute_avg_hops ); - - p_mcm_tbl = &p_mgrp->mcm_port_tbl; - - /* - For each member of the multicast group, compute the - number of hops to its base LID. - */ - for( p_mcm_port = (osm_mcm_port_t*)cl_qmap_head( p_mcm_tbl ); - p_mcm_port != (osm_mcm_port_t*)cl_qmap_end( p_mcm_tbl ); - p_mcm_port = (osm_mcm_port_t*)cl_qmap_next(&p_mcm_port->map_item)) - { - /* - Acquire the port object for this port guid, then create - the new worker object to build the list. - */ - p_port = osm_get_port_by_guid( p_mgr->p_subn, - ib_gid_get_guid( &p_mcm_port->port_gid ) ); - - if( !p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_compute_avg_hops: ERR 0A18: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( ib_gid_get_guid( &p_mcm_port->port_gid ) ) ); - continue; - } - - hops += osm_switch_get_port_least_hops( p_sw, p_port ); - num_ports++; - } - - /* - We should be here if there aren't any ports in the group. - */ - CL_ASSERT( num_ports ); - - if( num_ports != 0 ) - avg_hops = (float)(hops / num_ports); - - OSM_LOG_EXIT( p_mgr->p_log ); - return( avg_hops ); + float avg_hops = 0; + uint32_t hops = 0; + uint32_t num_ports = 0; + const osm_port_t *p_port; + const osm_mcm_port_t *p_mcm_port; + const cl_qmap_t *p_mcm_tbl; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_compute_avg_hops); + + p_mcm_tbl = &p_mgrp->mcm_port_tbl; + + /* + For each member of the multicast group, compute the + number of hops to its base LID. + */ + for (p_mcm_port = (osm_mcm_port_t *) cl_qmap_head(p_mcm_tbl); + p_mcm_port != (osm_mcm_port_t *) cl_qmap_end(p_mcm_tbl); + p_mcm_port = + (osm_mcm_port_t *) cl_qmap_next(&p_mcm_port->map_item)) { + /* + Acquire the port object for this port guid, then create + the new worker object to build the list. + */ + p_port = osm_get_port_by_guid(p_mgr->p_subn, + ib_gid_get_guid(&p_mcm_port-> + port_gid)); + + if (!p_port) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_compute_avg_hops: ERR 0A18: " + "No port object for port 0x%016" PRIx64 "\n", + cl_ntoh64(ib_gid_get_guid + (&p_mcm_port->port_gid))); + continue; + } + + hops += osm_switch_get_port_least_hops(p_sw, p_port); + num_ports++; + } + + /* + We should be here if there aren't any ports in the group. + */ + CL_ASSERT(num_ports); + + if (num_ports != 0) + avg_hops = (float)(hops / num_ports); + + OSM_LOG_EXIT(p_mgr->p_log); + return (avg_hops); } /********************************************************************** @@ -206,60 +197,59 @@ osm_mcast_mgr_compute_avg_hops( of the group HCAs **********************************************************************/ static float -osm_mcast_mgr_compute_max_hops( - osm_mcast_mgr_t* const p_mgr, - const osm_mgrp_t* const p_mgrp, - const osm_switch_t* const p_sw ) +osm_mcast_mgr_compute_max_hops(osm_mcast_mgr_t * const p_mgr, + const osm_mgrp_t * const p_mgrp, + const osm_switch_t * const p_sw) { - uint32_t max_hops = 0; - uint32_t hops = 0; - const osm_port_t* p_port; - const osm_mcm_port_t* p_mcm_port; - const cl_qmap_t* p_mcm_tbl; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_compute_max_hops ); - - p_mcm_tbl = &p_mgrp->mcm_port_tbl; - - /* - For each member of the multicast group, compute the - number of hops to its base LID. - */ - for( p_mcm_port = (osm_mcm_port_t*)cl_qmap_head( p_mcm_tbl ); - p_mcm_port != (osm_mcm_port_t*)cl_qmap_end( p_mcm_tbl ); - p_mcm_port = (osm_mcm_port_t*)cl_qmap_next(&p_mcm_port->map_item)) - { - /* - Acquire the port object for this port guid, then create - the new worker object to build the list. - */ - p_port = osm_get_port_by_guid( p_mgr->p_subn, - ib_gid_get_guid( &p_mcm_port->port_gid ) ); - - if( !p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_compute_max_hops: ERR 0A1A: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( ib_gid_get_guid( &p_mcm_port->port_gid ) ) ); - continue; - } - - hops = osm_switch_get_port_least_hops( p_sw, p_port ); - if (hops > max_hops) - max_hops = hops; - } - - if( max_hops == 0 ) - { - /* - We should be here if there aren't any ports in the group. - */ - max_hops = 10001; /* see later - we use it to realize no hops */ - } - - OSM_LOG_EXIT( p_mgr->p_log ); - return(float)(max_hops); + uint32_t max_hops = 0; + uint32_t hops = 0; + const osm_port_t *p_port; + const osm_mcm_port_t *p_mcm_port; + const cl_qmap_t *p_mcm_tbl; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_compute_max_hops); + + p_mcm_tbl = &p_mgrp->mcm_port_tbl; + + /* + For each member of the multicast group, compute the + number of hops to its base LID. + */ + for (p_mcm_port = (osm_mcm_port_t *) cl_qmap_head(p_mcm_tbl); + p_mcm_port != (osm_mcm_port_t *) cl_qmap_end(p_mcm_tbl); + p_mcm_port = + (osm_mcm_port_t *) cl_qmap_next(&p_mcm_port->map_item)) { + /* + Acquire the port object for this port guid, then create + the new worker object to build the list. + */ + p_port = osm_get_port_by_guid(p_mgr->p_subn, + ib_gid_get_guid(&p_mcm_port-> + port_gid)); + + if (!p_port) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_compute_max_hops: ERR 0A1A: " + "No port object for port 0x%016" PRIx64 "\n", + cl_ntoh64(ib_gid_get_guid + (&p_mcm_port->port_gid))); + continue; + } + + hops = osm_switch_get_port_least_hops(p_sw, p_port); + if (hops > max_hops) + max_hops = hops; + } + + if (max_hops == 0) { + /* + We should be here if there aren't any ports in the group. + */ + max_hops = 10001; /* see later - we use it to realize no hops */ + } + + OSM_LOG_EXIT(p_mgr->p_log); + return (float)(max_hops); } /********************************************************************** @@ -268,239 +258,225 @@ osm_mcast_mgr_compute_max_hops( a switch with the lowest average hop count to the members of the multicast group. **********************************************************************/ -static osm_switch_t* -__osm_mcast_mgr_find_optimal_switch( - osm_mcast_mgr_t* const p_mgr, - const osm_mgrp_t* const p_mgrp ) +static osm_switch_t *__osm_mcast_mgr_find_optimal_switch(osm_mcast_mgr_t * + const p_mgr, + const osm_mgrp_t * + const p_mgrp) { - cl_qmap_t* p_sw_tbl; - const osm_switch_t* p_sw; - const osm_switch_t* p_best_sw = NULL; - float hops = 0; - float best_hops = 10000; /* any big # will do */ - uint64_t sw_guid_ho; + cl_qmap_t *p_sw_tbl; + const osm_switch_t *p_sw; + const osm_switch_t *p_best_sw = NULL; + float hops = 0; + float best_hops = 10000; /* any big # will do */ + uint64_t sw_guid_ho; #ifdef OSM_VENDOR_INTF_ANAFA - boolean_t use_avg_hops = TRUE; /* anafa2 - bug hca on switch */ /* use max hops for root */ + boolean_t use_avg_hops = TRUE; /* anafa2 - bug hca on switch *//* use max hops for root */ #else - boolean_t use_avg_hops = FALSE; /* use max hops for root */ + boolean_t use_avg_hops = FALSE; /* use max hops for root */ #endif - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_find_optimal_switch ); - - p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; - - CL_ASSERT( !osm_mgrp_is_empty( p_mgrp ) ); - - for( p_sw = (osm_switch_t*)cl_qmap_head( p_sw_tbl ); - p_sw != (osm_switch_t*)cl_qmap_end( p_sw_tbl ); - p_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ) ) - { - if( !osm_switch_supports_mcast( p_sw ) ) - continue; - - if (use_avg_hops) - hops = osm_mcast_mgr_compute_avg_hops( p_mgr, p_mgrp, p_sw ); - else - hops = osm_mcast_mgr_compute_max_hops( p_mgr, p_mgrp, p_sw ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - sw_guid_ho = cl_ntoh64( osm_node_get_node_guid(p_sw->p_node) ); - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_mcast_mgr_find_optimal_switch: " - "Switch 0x%016" PRIx64 ", hops = %f\n", - sw_guid_ho, hops ); - } - - if( hops < best_hops ) - { - p_best_sw = p_sw; - best_hops = hops; - } - } - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - if( p_best_sw ) - { - sw_guid_ho = cl_ntoh64( osm_node_get_node_guid(p_best_sw->p_node) ); - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_mcast_mgr_find_optimal_switch: " - "Best switch is 0x%" PRIx64 ", hops = %f\n", - sw_guid_ho, best_hops ); - } - else - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_mcast_mgr_find_optimal_switch: " - "No multicast capable switches detected\n" ); - } - } - - OSM_LOG_EXIT( p_mgr->p_log ); - return( (osm_switch_t*)p_best_sw ); + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_find_optimal_switch); + + p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; + + CL_ASSERT(!osm_mgrp_is_empty(p_mgrp)); + + for (p_sw = (osm_switch_t *) cl_qmap_head(p_sw_tbl); + p_sw != (osm_switch_t *) cl_qmap_end(p_sw_tbl); + p_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item)) { + if (!osm_switch_supports_mcast(p_sw)) + continue; + + if (use_avg_hops) + hops = + osm_mcast_mgr_compute_avg_hops(p_mgr, p_mgrp, p_sw); + else + hops = + osm_mcast_mgr_compute_max_hops(p_mgr, p_mgrp, p_sw); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + sw_guid_ho = + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)); + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_mcast_mgr_find_optimal_switch: " + "Switch 0x%016" PRIx64 ", hops = %f\n", + sw_guid_ho, hops); + } + + if (hops < best_hops) { + p_best_sw = p_sw; + best_hops = hops; + } + } + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_VERBOSE)) { + if (p_best_sw) { + sw_guid_ho = + cl_ntoh64(osm_node_get_node_guid + (p_best_sw->p_node)); + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_mcast_mgr_find_optimal_switch: " + "Best switch is 0x%" PRIx64 ", hops = %f\n", + sw_guid_ho, best_hops); + } else { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_mcast_mgr_find_optimal_switch: " + "No multicast capable switches detected\n"); + } + } + + OSM_LOG_EXIT(p_mgr->p_log); + return ((osm_switch_t *) p_best_sw); } /********************************************************************** This function returns the existing or optimal root swtich for the tree. **********************************************************************/ -static osm_switch_t* -__osm_mcast_mgr_find_root_switch( - osm_mcast_mgr_t* const p_mgr, - const osm_mgrp_t* const p_mgrp ) +static osm_switch_t *__osm_mcast_mgr_find_root_switch(osm_mcast_mgr_t * + const p_mgr, + const osm_mgrp_t * + const p_mgrp) { - const osm_switch_t* p_sw = NULL; + const osm_switch_t *p_sw = NULL; - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_find_root_switch ); + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_find_root_switch); - /* - We always look for the best multicast tree root switch. - Otherwise since we always start with a a single join - the root will be always on the first switch attached to it. - - Very bad ... - */ - p_sw = __osm_mcast_mgr_find_optimal_switch( p_mgr, p_mgrp ); + /* + We always look for the best multicast tree root switch. + Otherwise since we always start with a a single join + the root will be always on the first switch attached to it. + - Very bad ... + */ + p_sw = __osm_mcast_mgr_find_optimal_switch(p_mgr, p_mgrp); - OSM_LOG_EXIT( p_mgr->p_log ); - return( (osm_switch_t*)p_sw ); + OSM_LOG_EXIT(p_mgr->p_log); + return ((osm_switch_t *) p_sw); } /********************************************************************** **********************************************************************/ -void -osm_mcast_mgr_construct( - IN osm_mcast_mgr_t* const p_mgr ) +void osm_mcast_mgr_construct(IN osm_mcast_mgr_t * const p_mgr) { - memset( p_mgr, 0, sizeof(*p_mgr) ); + memset(p_mgr, 0, sizeof(*p_mgr)); } /********************************************************************** **********************************************************************/ -void -osm_mcast_mgr_destroy( - IN osm_mcast_mgr_t* const p_mgr ) +void osm_mcast_mgr_destroy(IN osm_mcast_mgr_t * const p_mgr) { - CL_ASSERT( p_mgr ); + CL_ASSERT(p_mgr); - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_destroy ); + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_destroy); - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mcast_mgr_init( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_mcast_mgr_init(IN osm_mcast_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_mcast_mgr_init ); + OSM_LOG_ENTER(p_log, osm_mcast_mgr_init); - CL_ASSERT( p_req ); - CL_ASSERT( p_subn ); - CL_ASSERT( p_lock ); + CL_ASSERT(p_req); + CL_ASSERT(p_subn); + CL_ASSERT(p_lock); - osm_mcast_mgr_construct( p_mgr ); + osm_mcast_mgr_construct(p_mgr); - p_mgr->p_log = p_log; - p_mgr->p_subn = p_subn; - p_mgr->p_lock = p_lock; - p_mgr->p_req = p_req; + p_mgr->p_log = p_log; + p_mgr->p_subn = p_subn; + p_mgr->p_lock = p_lock; + p_mgr->p_req = p_req; - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } /********************************************************************** **********************************************************************/ static osm_signal_t -__osm_mcast_mgr_set_tbl( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_switch_t* const p_sw ) +__osm_mcast_mgr_set_tbl(IN osm_mcast_mgr_t * const p_mgr, + IN osm_switch_t * const p_sw) { - osm_node_t* p_node; - osm_dr_path_t* p_path; - osm_madw_context_t mad_context; - ib_api_status_t status; - uint32_t block_id_ho = 0; - int16_t block_num = 0; - uint32_t position = 0; - uint32_t max_position; - osm_mcast_tbl_t* p_tbl; - ib_net16_t block[IB_MCAST_BLOCK_SIZE]; - osm_signal_t signal = OSM_SIGNAL_DONE; - - CL_ASSERT( p_mgr ); - - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_set_tbl ); - - CL_ASSERT( p_sw ); - - p_node = p_sw->p_node; - - CL_ASSERT( p_node ); - - p_path = osm_node_get_any_dr_path_ptr( p_node ); - - CL_ASSERT( p_path ); - - /* - Send multicast forwarding table blocks to the switch - as long as the switch indicates it has blocks needing - configuration. - */ - - mad_context.mft_context.node_guid = osm_node_get_node_guid( p_node ); - mad_context.mft_context.set_method = TRUE; - - p_tbl = osm_switch_get_mcast_tbl_ptr( p_sw ); - max_position = p_tbl->max_position; - - while( osm_mcast_tbl_get_block( p_tbl, block_num, - (uint8_t)position, block ) ) - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_mcast_mgr_set_tbl: " - "Writing MFT block 0x%X\n", block_id_ho ); - } - - block_id_ho = block_num + (position << 28); - - status = osm_req_set( p_mgr->p_req, - p_path, - (void*)block, - sizeof(block), - IB_MAD_ATTR_MCAST_FWD_TBL, - cl_hton32( block_id_ho ), - CL_DISP_MSGID_NONE, - &mad_context ); - - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_set_tbl: ERR 0A02: " - "Sending multicast fwd. tbl. block failed (%s)\n", - ib_get_err_str( status ) ); - } - - signal = OSM_SIGNAL_DONE_PENDING; - - if( ++position > max_position ) - { - position = 0; - block_num++; - } - } - - OSM_LOG_EXIT( p_mgr->p_log ); - return( signal ); + osm_node_t *p_node; + osm_dr_path_t *p_path; + osm_madw_context_t mad_context; + ib_api_status_t status; + uint32_t block_id_ho = 0; + int16_t block_num = 0; + uint32_t position = 0; + uint32_t max_position; + osm_mcast_tbl_t *p_tbl; + ib_net16_t block[IB_MCAST_BLOCK_SIZE]; + osm_signal_t signal = OSM_SIGNAL_DONE; + + CL_ASSERT(p_mgr); + + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_set_tbl); + + CL_ASSERT(p_sw); + + p_node = p_sw->p_node; + + CL_ASSERT(p_node); + + p_path = osm_node_get_any_dr_path_ptr(p_node); + + CL_ASSERT(p_path); + + /* + Send multicast forwarding table blocks to the switch + as long as the switch indicates it has blocks needing + configuration. + */ + + mad_context.mft_context.node_guid = osm_node_get_node_guid(p_node); + mad_context.mft_context.set_method = TRUE; + + p_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); + max_position = p_tbl->max_position; + + while (osm_mcast_tbl_get_block(p_tbl, block_num, + (uint8_t) position, block)) { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_mcast_mgr_set_tbl: " + "Writing MFT block 0x%X\n", block_id_ho); + } + + block_id_ho = block_num + (position << 28); + + status = osm_req_set(p_mgr->p_req, + p_path, + (void *)block, + sizeof(block), + IB_MAD_ATTR_MCAST_FWD_TBL, + cl_hton32(block_id_ho), + CL_DISP_MSGID_NONE, &mad_context); + + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_set_tbl: ERR 0A02: " + "Sending multicast fwd. tbl. block failed (%s)\n", + ib_get_err_str(status)); + } + + signal = OSM_SIGNAL_DONE_PENDING; + + if (++position > max_position) { + position = 0; + block_num++; + } + } + + OSM_LOG_EXIT(p_mgr->p_log); + return (signal); } /********************************************************************** @@ -509,106 +485,106 @@ __osm_mcast_mgr_set_tbl( contains the group members that must be routed from this switch. **********************************************************************/ static void -__osm_mcast_mgr_subdivide( - osm_mcast_mgr_t* const p_mgr, - osm_mgrp_t* const p_mgrp, - osm_switch_t* const p_sw, - cl_qlist_t* const p_list, - cl_qlist_t* const list_array, - uint8_t const array_size ) +__osm_mcast_mgr_subdivide(osm_mcast_mgr_t * const p_mgr, + osm_mgrp_t * const p_mgrp, + osm_switch_t * const p_sw, + cl_qlist_t * const p_list, + cl_qlist_t * const list_array, + uint8_t const array_size) { - uint8_t port_num; - uint16_t mlid_ho; - boolean_t ignore_existing; - osm_mcast_work_obj_t* p_wobj; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_subdivide ); - - mlid_ho = cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ); - - /* - For Multicast Groups, we want not to count on previous - configurations - since we can easily generate a storm - by loops. - */ - ignore_existing = TRUE; - - /* - Subdivide the set of ports into non-overlapping subsets - that will be routed to other switches. - */ - while( (p_wobj = (osm_mcast_work_obj_t*)cl_qlist_remove_head( p_list )) != - (osm_mcast_work_obj_t*)cl_qlist_end( p_list ) ) - { - port_num = osm_switch_recommend_mcast_path( - p_sw, p_wobj->p_port, mlid_ho, ignore_existing ); - - if( port_num == OSM_NO_PATH ) - { - /* - This typically occurs if the switch does not support - multicast and the multicast tree must branch at this - switch. - */ - uint64_t node_guid_ho = cl_ntoh64( osm_node_get_node_guid(p_sw->p_node) ); - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_subdivide: ERR 0A03: " - "Error routing MLID 0x%X through switch 0x%" PRIx64 "\n" - "\t\t\t\tNo multicast paths from this switch for port " - "with LID 0x%X\n", - mlid_ho, node_guid_ho, - cl_ntoh16(osm_port_get_base_lid(p_wobj->p_port)) ); - - __osm_mcast_work_obj_delete( p_wobj ); - continue; - } - - if( port_num > array_size ) - { - uint64_t node_guid_ho = cl_ntoh64( osm_node_get_node_guid(p_sw->p_node) ); - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_subdivide: ERR 0A04: " - "Error routing MLID 0x%X through switch 0x%" PRIx64 "\n" - "\t\t\t\tNo multicast paths from this switch to port " - "with LID 0x%X\n", - mlid_ho, node_guid_ho, - cl_ntoh16(osm_port_get_base_lid(p_wobj->p_port)) ); - - __osm_mcast_work_obj_delete( p_wobj ); - - /* This is means OpenSM has a bug. */ - CL_ASSERT( FALSE ); - continue; - } - - cl_qlist_insert_tail( &list_array[port_num], &p_wobj->list_item ); - } - - OSM_LOG_EXIT( p_mgr->p_log ); + uint8_t port_num; + uint16_t mlid_ho; + boolean_t ignore_existing; + osm_mcast_work_obj_t *p_wobj; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_subdivide); + + mlid_ho = cl_ntoh16(osm_mgrp_get_mlid(p_mgrp)); + + /* + For Multicast Groups, we want not to count on previous + configurations - since we can easily generate a storm + by loops. + */ + ignore_existing = TRUE; + + /* + Subdivide the set of ports into non-overlapping subsets + that will be routed to other switches. + */ + while ((p_wobj = + (osm_mcast_work_obj_t *) cl_qlist_remove_head(p_list)) != + (osm_mcast_work_obj_t *) cl_qlist_end(p_list)) { + port_num = + osm_switch_recommend_mcast_path(p_sw, p_wobj->p_port, + mlid_ho, ignore_existing); + + if (port_num == OSM_NO_PATH) { + /* + This typically occurs if the switch does not support + multicast and the multicast tree must branch at this + switch. + */ + uint64_t node_guid_ho = + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)); + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_subdivide: ERR 0A03: " + "Error routing MLID 0x%X through switch 0x%" + PRIx64 "\n" + "\t\t\t\tNo multicast paths from this switch for port " + "with LID 0x%X\n", mlid_ho, node_guid_ho, + cl_ntoh16(osm_port_get_base_lid + (p_wobj->p_port))); + + __osm_mcast_work_obj_delete(p_wobj); + continue; + } + + if (port_num > array_size) { + uint64_t node_guid_ho = + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)); + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_subdivide: ERR 0A04: " + "Error routing MLID 0x%X through switch 0x%" + PRIx64 "\n" + "\t\t\t\tNo multicast paths from this switch to port " + "with LID 0x%X\n", mlid_ho, node_guid_ho, + cl_ntoh16(osm_port_get_base_lid + (p_wobj->p_port))); + + __osm_mcast_work_obj_delete(p_wobj); + + /* This is means OpenSM has a bug. */ + CL_ASSERT(FALSE); + continue; + } + + cl_qlist_insert_tail(&list_array[port_num], &p_wobj->list_item); + } + + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_mcast_mgr_purge_list( - osm_mcast_mgr_t* const p_mgr, - cl_qlist_t* const p_list ) +__osm_mcast_mgr_purge_list(osm_mcast_mgr_t * const p_mgr, + cl_qlist_t * const p_list) { - osm_mcast_work_obj_t* p_wobj; + osm_mcast_work_obj_t *p_wobj; - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_purge_list ); + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_purge_list); - while( (p_wobj = (osm_mcast_work_obj_t*)cl_qlist_remove_head( p_list ) ) - != (osm_mcast_work_obj_t*)cl_qlist_end( p_list ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_purge_list: ERR 0A06: " - "Unable to route for port 0x%" PRIx64 "\n", - osm_port_get_guid( p_wobj->p_port ) ); - __osm_mcast_work_obj_delete( p_wobj ); - } + while ((p_wobj = (osm_mcast_work_obj_t *) cl_qlist_remove_head(p_list)) + != (osm_mcast_work_obj_t *) cl_qlist_end(p_list)) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_purge_list: ERR 0A06: " + "Unable to route for port 0x%" PRIx64 "\n", + osm_port_get_guid(p_wobj->p_port)); + __osm_mcast_work_obj_delete(p_wobj); + } - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** @@ -618,362 +594,346 @@ __osm_mcast_mgr_purge_list( The function returns the newly created mtree node element. **********************************************************************/ -static osm_mtree_node_t* -__osm_mcast_mgr_branch( - osm_mcast_mgr_t* const p_mgr, - osm_mgrp_t* const p_mgrp, - osm_switch_t* const p_sw, - cl_qlist_t* const p_list, - uint8_t depth, - uint8_t const upstream_port, - uint8_t* const p_max_depth ) +static osm_mtree_node_t *__osm_mcast_mgr_branch(osm_mcast_mgr_t * const p_mgr, + osm_mgrp_t * const p_mgrp, + osm_switch_t * const p_sw, + cl_qlist_t * const p_list, + uint8_t depth, + uint8_t const upstream_port, + uint8_t * const p_max_depth) { - uint8_t max_children; - osm_mtree_node_t* p_mtn = NULL; - cl_qlist_t* list_array = NULL; - uint8_t i; - ib_net64_t node_guid; - uint64_t node_guid_ho; - osm_mcast_work_obj_t* p_wobj; - cl_qlist_t* p_port_list; - size_t count; - uint16_t mlid_ho; - osm_mcast_tbl_t* p_tbl; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_branch ); - - CL_ASSERT( p_sw ); - CL_ASSERT( p_list ); - CL_ASSERT( p_max_depth ); - - node_guid = osm_node_get_node_guid( p_sw->p_node ); - node_guid_ho = cl_ntoh64( node_guid ); - mlid_ho = cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_mcast_mgr_branch: " - "Routing MLID 0x%X through switch 0x%" PRIx64 - ", %u nodes at depth %u\n", - mlid_ho, node_guid_ho, - cl_qlist_count( p_list ), depth ); - } - - CL_ASSERT( cl_qlist_count( p_list ) > 0 ); - - depth++; - - if( depth > *p_max_depth ) - { - CL_ASSERT( depth == *p_max_depth + 1 ); - *p_max_depth = depth; - } - - if( osm_switch_supports_mcast( p_sw ) == FALSE ) - { - /* - This switch doesn't do multicast. Clean-up. - */ - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_branch: ERR 0A14: " - "Switch 0x%" PRIx64 " does not support multicast\n", - node_guid_ho ); - - /* - Deallocate all the work objects on this branch of the tree. - */ - __osm_mcast_mgr_purge_list( p_mgr, p_list ); - goto Exit; - } - - p_mtn = osm_mtree_node_new( p_sw ); - if( p_mtn == NULL ) - { - /* - We are unable to continue routing down this - leg of the tree. Clean-up. - */ - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_branch: ERR 0A15: " - "Insufficient memory to build multicast tree\n" ); - - /* - Deallocate all the work objects on this branch of the tree. - */ - __osm_mcast_mgr_purge_list( p_mgr, p_list ); - goto Exit; - } - - max_children = osm_mtree_node_get_max_children( p_mtn ); - - CL_ASSERT( max_children > 1 ); - - /* - Prepare an empty list for each port in the switch. - TO DO - this list array could probably be moved - inside the switch element to save on malloc thrashing. - */ - list_array = malloc( sizeof(cl_qlist_t) * max_children ); - if( list_array == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_branch: ERR 0A16: " - "Unable to allocate list array\n" ); - __osm_mcast_mgr_purge_list( p_mgr, p_list ); - goto Exit; - } - - memset( list_array, 0, sizeof(cl_qlist_t) * max_children ); - - for( i = 0; i < max_children; i++ ) - cl_qlist_init( &list_array[i] ); - - __osm_mcast_mgr_subdivide( p_mgr, p_mgrp, p_sw, p_list, list_array, - max_children ); - - p_tbl = osm_switch_get_mcast_tbl_ptr( p_sw ); - - /* - Add the upstream port to the forwarding table unless - we're at the root of the spanning tree. - */ - if( depth > 1 ) - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_mcast_mgr_branch: " - "Adding upstream port 0x%X\n", upstream_port ); - } - - CL_ASSERT( upstream_port ); - osm_mcast_tbl_set( p_tbl, mlid_ho, upstream_port ); - } - - /* - For each port that was allocated some routes, - recurse into this function to continue building the tree - if the node on the other end of that port is another switch. - Otherwise, the node is an endpoint, and we've found a leaf - of the tree. Mark leaves with our special pointer value. - */ - - for( i = 0; i < max_children; i++ ) - { - const osm_physp_t *p_physp; - const osm_physp_t *p_remote_physp; - const osm_node_t *p_node; - const osm_node_t *p_remote_node; - - p_port_list = &list_array[i]; - - count = cl_qlist_count( p_port_list ); - - /* - There should be no children routed through the upstream port! - */ - CL_ASSERT( ( upstream_port == 0 ) || ( i != upstream_port) || - ( (i == upstream_port) && (count == 0)) ); - - if( count == 0) - continue; /* No routes down this port. */ - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_mcast_mgr_branch: " - "Routing %zu destinations via switch port 0x%X\n", - count, i ); - } - - /* - This port routes frames for this mcast group. Therefore, - set the appropriate bit in the multicast forwarding - table for this switch. - */ - osm_mcast_tbl_set( p_tbl, mlid_ho, i ); - if (i == 0) - /* This means we are adding the switch to the MC group. - We do not need to continue looking at the remote port, just - needed to add the port to the table */ - continue; - - p_node = p_sw->p_node; - p_remote_node = osm_node_get_remote_node( p_node, i, NULL ); - - if( osm_node_get_type( p_remote_node ) == IB_NODE_TYPE_SWITCH ) - { - /* - Acquire a pointer to the remote switch then recurse. - */ - CL_ASSERT( p_remote_node->sw ); - - p_physp = osm_node_get_physp_ptr( p_node, i ); - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - - p_remote_physp = osm_physp_get_remote( p_physp ); - CL_ASSERT( p_remote_physp ); - CL_ASSERT( osm_physp_is_valid( p_remote_physp ) ); - - p_mtn->child_array[i] = __osm_mcast_mgr_branch( - p_mgr, p_mgrp, p_remote_node->sw, - p_port_list, depth, - osm_physp_get_port_num( p_remote_physp), - p_max_depth ); - } - else - { - /* - The neighbor node is not a switch, so this - must be a leaf. - */ - CL_ASSERT( count == 1 ); - - p_mtn->child_array[i] = OSM_MTREE_LEAF; - p_wobj = (osm_mcast_work_obj_t*)cl_qlist_remove_head( - p_port_list ); - - CL_ASSERT( cl_is_qlist_empty( p_port_list ) ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_mcast_mgr_branch: " - "Found leaf for port 0x%016" PRIx64 - " on switch port 0x%X\n", - cl_ntoh64( osm_port_get_guid( p_wobj->p_port ) ), i ); - } - - __osm_mcast_work_obj_delete( p_wobj ); - } - } - - free( list_array ); - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - return( p_mtn ); + uint8_t max_children; + osm_mtree_node_t *p_mtn = NULL; + cl_qlist_t *list_array = NULL; + uint8_t i; + ib_net64_t node_guid; + uint64_t node_guid_ho; + osm_mcast_work_obj_t *p_wobj; + cl_qlist_t *p_port_list; + size_t count; + uint16_t mlid_ho; + osm_mcast_tbl_t *p_tbl; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_branch); + + CL_ASSERT(p_sw); + CL_ASSERT(p_list); + CL_ASSERT(p_max_depth); + + node_guid = osm_node_get_node_guid(p_sw->p_node); + node_guid_ho = cl_ntoh64(node_guid); + mlid_ho = cl_ntoh16(osm_mgrp_get_mlid(p_mgrp)); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_mcast_mgr_branch: " + "Routing MLID 0x%X through switch 0x%" PRIx64 + ", %u nodes at depth %u\n", + mlid_ho, node_guid_ho, cl_qlist_count(p_list), depth); + } + + CL_ASSERT(cl_qlist_count(p_list) > 0); + + depth++; + + if (depth > *p_max_depth) { + CL_ASSERT(depth == *p_max_depth + 1); + *p_max_depth = depth; + } + + if (osm_switch_supports_mcast(p_sw) == FALSE) { + /* + This switch doesn't do multicast. Clean-up. + */ + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_branch: ERR 0A14: " + "Switch 0x%" PRIx64 " does not support multicast\n", + node_guid_ho); + + /* + Deallocate all the work objects on this branch of the tree. + */ + __osm_mcast_mgr_purge_list(p_mgr, p_list); + goto Exit; + } + + p_mtn = osm_mtree_node_new(p_sw); + if (p_mtn == NULL) { + /* + We are unable to continue routing down this + leg of the tree. Clean-up. + */ + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_branch: ERR 0A15: " + "Insufficient memory to build multicast tree\n"); + + /* + Deallocate all the work objects on this branch of the tree. + */ + __osm_mcast_mgr_purge_list(p_mgr, p_list); + goto Exit; + } + + max_children = osm_mtree_node_get_max_children(p_mtn); + + CL_ASSERT(max_children > 1); + + /* + Prepare an empty list for each port in the switch. + TO DO - this list array could probably be moved + inside the switch element to save on malloc thrashing. + */ + list_array = malloc(sizeof(cl_qlist_t) * max_children); + if (list_array == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_branch: ERR 0A16: " + "Unable to allocate list array\n"); + __osm_mcast_mgr_purge_list(p_mgr, p_list); + goto Exit; + } + + memset(list_array, 0, sizeof(cl_qlist_t) * max_children); + + for (i = 0; i < max_children; i++) + cl_qlist_init(&list_array[i]); + + __osm_mcast_mgr_subdivide(p_mgr, p_mgrp, p_sw, p_list, list_array, + max_children); + + p_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); + + /* + Add the upstream port to the forwarding table unless + we're at the root of the spanning tree. + */ + if (depth > 1) { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_mcast_mgr_branch: " + "Adding upstream port 0x%X\n", upstream_port); + } + + CL_ASSERT(upstream_port); + osm_mcast_tbl_set(p_tbl, mlid_ho, upstream_port); + } + + /* + For each port that was allocated some routes, + recurse into this function to continue building the tree + if the node on the other end of that port is another switch. + Otherwise, the node is an endpoint, and we've found a leaf + of the tree. Mark leaves with our special pointer value. + */ + + for (i = 0; i < max_children; i++) { + const osm_physp_t *p_physp; + const osm_physp_t *p_remote_physp; + const osm_node_t *p_node; + const osm_node_t *p_remote_node; + + p_port_list = &list_array[i]; + + count = cl_qlist_count(p_port_list); + + /* + There should be no children routed through the upstream port! + */ + CL_ASSERT((upstream_port == 0) || (i != upstream_port) || + ((i == upstream_port) && (count == 0))); + + if (count == 0) + continue; /* No routes down this port. */ + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_mcast_mgr_branch: " + "Routing %zu destinations via switch port 0x%X\n", + count, i); + } + + /* + This port routes frames for this mcast group. Therefore, + set the appropriate bit in the multicast forwarding + table for this switch. + */ + osm_mcast_tbl_set(p_tbl, mlid_ho, i); + if (i == 0) + /* This means we are adding the switch to the MC group. + We do not need to continue looking at the remote port, just + needed to add the port to the table */ + continue; + + p_node = p_sw->p_node; + p_remote_node = osm_node_get_remote_node(p_node, i, NULL); + + if (osm_node_get_type(p_remote_node) == IB_NODE_TYPE_SWITCH) { + /* + Acquire a pointer to the remote switch then recurse. + */ + CL_ASSERT(p_remote_node->sw); + + p_physp = osm_node_get_physp_ptr(p_node, i); + CL_ASSERT(osm_physp_is_valid(p_physp)); + + p_remote_physp = osm_physp_get_remote(p_physp); + CL_ASSERT(p_remote_physp); + CL_ASSERT(osm_physp_is_valid(p_remote_physp)); + + p_mtn->child_array[i] = + __osm_mcast_mgr_branch(p_mgr, p_mgrp, + p_remote_node->sw, + p_port_list, depth, + osm_physp_get_port_num + (p_remote_physp), + p_max_depth); + } else { + /* + The neighbor node is not a switch, so this + must be a leaf. + */ + CL_ASSERT(count == 1); + + p_mtn->child_array[i] = OSM_MTREE_LEAF; + p_wobj = + (osm_mcast_work_obj_t *) + cl_qlist_remove_head(p_port_list); + + CL_ASSERT(cl_is_qlist_empty(p_port_list)); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_mcast_mgr_branch: " + "Found leaf for port 0x%016" PRIx64 + " on switch port 0x%X\n", + cl_ntoh64(osm_port_get_guid + (p_wobj->p_port)), i); + } + + __osm_mcast_work_obj_delete(p_wobj); + } + } + + free(list_array); + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + return (p_mtn); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_mcast_mgr_build_spanning_tree( - osm_mcast_mgr_t* const p_mgr, - osm_mgrp_t* const p_mgrp ) +__osm_mcast_mgr_build_spanning_tree(osm_mcast_mgr_t * const p_mgr, + osm_mgrp_t * const p_mgrp) { - const cl_qmap_t* p_mcm_tbl; - const osm_port_t* p_port; - const osm_mcm_port_t* p_mcm_port; - uint32_t num_ports; - cl_qlist_t port_list; - osm_switch_t* p_sw; - osm_mcast_work_obj_t* p_wobj; - ib_api_status_t status = IB_SUCCESS; - uint8_t max_depth = 0; - uint32_t count; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_build_spanning_tree ); - - cl_qlist_init( &port_list ); - - /* - TO DO - for now, just blow away the old tree. - In the future we'll need to construct the tree based - on multicast forwarding table information if the user wants to - preserve existing multicast routes. - */ - __osm_mcast_mgr_purge_tree( p_mgr, p_mgrp ); - - p_mcm_tbl = &p_mgrp->mcm_port_tbl; - num_ports = cl_qmap_count( p_mcm_tbl ); - if( num_ports == 0 ) - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_mcast_mgr_build_spanning_tree: " - "MLID 0x%X has no members - nothing to do\n", - cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ) ); - } - goto Exit; - } - - /* - This function builds the single spanning tree recursively. - At each stage, the ports to be reached are divided into - non-overlapping subsets of member ports that can be reached through - a given switch port. Construction then moves down each - branch, and the process starts again with each branch computing - for its own subset of the member ports. - - The maximum recursion depth is at worst the maximum hop count in the - subnet, which is spec limited to 64. - */ - - /* - Locate the switch around which to create the spanning - tree for this multicast group. - */ - p_sw = __osm_mcast_mgr_find_root_switch( p_mgr, p_mgrp ); - if( p_sw == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_build_spanning_tree: ERR 0A08: " - "Unable to locate a suitable switch for group 0x%X\n", - cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) )); - status = IB_ERROR; - goto Exit; - } - - /* - Build the first "subset" containing all member ports. - */ - for( p_mcm_port = (osm_mcm_port_t*)cl_qmap_head( p_mcm_tbl ); - p_mcm_port != (osm_mcm_port_t*)cl_qmap_end( p_mcm_tbl ); - p_mcm_port = (osm_mcm_port_t*)cl_qmap_next(&p_mcm_port->map_item)) - { - /* - Acquire the port object for this port guid, then create - the new worker object to build the list. - */ - p_port = osm_get_port_by_guid( p_mgr->p_subn, - ib_gid_get_guid( &p_mcm_port->port_gid ) ); - if( !p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_build_spanning_tree: ERR 0A09: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( ib_gid_get_guid( &p_mcm_port->port_gid ) ) ); - continue; - } - - p_wobj = __osm_mcast_work_obj_new( p_port ); - if( p_wobj == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_mcast_mgr_build_spanning_tree: ERR 0A10: " - "Insufficient memory to route port 0x%016" PRIx64 "\n", - cl_ntoh64( osm_port_get_guid( p_port ) ) ); - continue; - } - - cl_qlist_insert_tail( &port_list, &p_wobj->list_item ); - } - - count = cl_qlist_count( &port_list ); - p_mgrp->p_root = __osm_mcast_mgr_branch( p_mgr, p_mgrp, p_sw, - &port_list, 0, 0, &max_depth ); - - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_mcast_mgr_build_spanning_tree: " - "Configured MLID 0x%X for %u ports, max tree depth = %u\n", - cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ), - count, max_depth ); - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + const cl_qmap_t *p_mcm_tbl; + const osm_port_t *p_port; + const osm_mcm_port_t *p_mcm_port; + uint32_t num_ports; + cl_qlist_t port_list; + osm_switch_t *p_sw; + osm_mcast_work_obj_t *p_wobj; + ib_api_status_t status = IB_SUCCESS; + uint8_t max_depth = 0; + uint32_t count; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_build_spanning_tree); + + cl_qlist_init(&port_list); + + /* + TO DO - for now, just blow away the old tree. + In the future we'll need to construct the tree based + on multicast forwarding table information if the user wants to + preserve existing multicast routes. + */ + __osm_mcast_mgr_purge_tree(p_mgr, p_mgrp); + + p_mcm_tbl = &p_mgrp->mcm_port_tbl; + num_ports = cl_qmap_count(p_mcm_tbl); + if (num_ports == 0) { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_mcast_mgr_build_spanning_tree: " + "MLID 0x%X has no members - nothing to do\n", + cl_ntoh16(osm_mgrp_get_mlid(p_mgrp))); + } + goto Exit; + } + + /* + This function builds the single spanning tree recursively. + At each stage, the ports to be reached are divided into + non-overlapping subsets of member ports that can be reached through + a given switch port. Construction then moves down each + branch, and the process starts again with each branch computing + for its own subset of the member ports. + + The maximum recursion depth is at worst the maximum hop count in the + subnet, which is spec limited to 64. + */ + + /* + Locate the switch around which to create the spanning + tree for this multicast group. + */ + p_sw = __osm_mcast_mgr_find_root_switch(p_mgr, p_mgrp); + if (p_sw == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_build_spanning_tree: ERR 0A08: " + "Unable to locate a suitable switch for group 0x%X\n", + cl_ntoh16(osm_mgrp_get_mlid(p_mgrp))); + status = IB_ERROR; + goto Exit; + } + + /* + Build the first "subset" containing all member ports. + */ + for (p_mcm_port = (osm_mcm_port_t *) cl_qmap_head(p_mcm_tbl); + p_mcm_port != (osm_mcm_port_t *) cl_qmap_end(p_mcm_tbl); + p_mcm_port = + (osm_mcm_port_t *) cl_qmap_next(&p_mcm_port->map_item)) { + /* + Acquire the port object for this port guid, then create + the new worker object to build the list. + */ + p_port = osm_get_port_by_guid(p_mgr->p_subn, + ib_gid_get_guid(&p_mcm_port-> + port_gid)); + if (!p_port) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_build_spanning_tree: ERR 0A09: " + "No port object for port 0x%016" PRIx64 "\n", + cl_ntoh64(ib_gid_get_guid + (&p_mcm_port->port_gid))); + continue; + } + + p_wobj = __osm_mcast_work_obj_new(p_port); + if (p_wobj == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_mcast_mgr_build_spanning_tree: ERR 0A10: " + "Insufficient memory to route port 0x%016" + PRIx64 "\n", + cl_ntoh64(osm_port_get_guid(p_port))); + continue; + } + + cl_qlist_insert_tail(&port_list, &p_wobj->list_item); + } + + count = cl_qlist_count(&port_list); + p_mgrp->p_root = __osm_mcast_mgr_branch(p_mgr, p_mgrp, p_sw, + &port_list, 0, 0, &max_depth); + + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_mcast_mgr_build_spanning_tree: " + "Configured MLID 0x%X for %u ports, max tree depth = %u\n", + cl_ntoh16(osm_mgrp_get_mlid(p_mgrp)), count, max_depth); + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } #if 0 @@ -981,84 +941,79 @@ __osm_mcast_mgr_build_spanning_tree( /********************************************************************** **********************************************************************/ void -osm_mcast_mgr_set_table( - IN osm_mcast_mgr_t* const p_mgr, - IN const osm_mgrp_t* const p_mgrp, - IN const osm_mtree_node_t* const p_mtn ) +osm_mcast_mgr_set_table(IN osm_mcast_mgr_t * const p_mgr, + IN const osm_mgrp_t * const p_mgrp, + IN const osm_mtree_node_t * const p_mtn) { - uint8_t i; - uint8_t max_children; - osm_mtree_node_t* p_child_mtn; - uint16_t mlid_ho; - osm_mcast_tbl_t* p_tbl; - osm_switch_t* p_sw; + uint8_t i; + uint8_t max_children; + osm_mtree_node_t *p_child_mtn; + uint16_t mlid_ho; + osm_mcast_tbl_t *p_tbl; + osm_switch_t *p_sw; - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_set_table ); + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_set_table); - mlid_ho = cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ); - p_sw = osm_mtree_node_get_switch_ptr( p_mtn ); + mlid_ho = cl_ntoh16(osm_mgrp_get_mlid(p_mgrp)); + p_sw = osm_mtree_node_get_switch_ptr(p_mtn); - CL_ASSERT( p_sw ); + CL_ASSERT(p_sw); - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "osm_mcast_mgr_set_table: " - "Configuring MLID 0x%X on switch 0x%" PRIx64 "\n", - mlid_ho, osm_node_get_node_guid(p_sw->p_node) ); - } + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "osm_mcast_mgr_set_table: " + "Configuring MLID 0x%X on switch 0x%" PRIx64 "\n", + mlid_ho, osm_node_get_node_guid(p_sw->p_node)); + } - /* - For every child of this tree node, set the corresponding - bit in the switch's mcast table. - */ - p_tbl = osm_switch_get_mcast_tbl_ptr( p_sw ); - max_children = osm_mtree_node_get_max_children( p_mtn ); + /* + For every child of this tree node, set the corresponding + bit in the switch's mcast table. + */ + p_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); + max_children = osm_mtree_node_get_max_children(p_mtn); - CL_ASSERT( max_children <= osm_switch_get_num_ports( p_sw ) ); + CL_ASSERT(max_children <= osm_switch_get_num_ports(p_sw)); - osm_mcast_tbl_clear_mlid( p_tbl, mlid_ho ); + osm_mcast_tbl_clear_mlid(p_tbl, mlid_ho); - for( i = 0; i < max_children; i++ ) - { - p_child_mtn = osm_mtree_node_get_child( p_mtn, i ); - if( p_child_mtn == NULL ) - continue; + for (i = 0; i < max_children; i++) { + p_child_mtn = osm_mtree_node_get_child(p_mtn, i); + if (p_child_mtn == NULL) + continue; - osm_mcast_tbl_set( p_tbl, mlid_ho, i ); - } + osm_mcast_tbl_set(p_tbl, mlid_ho, i); + } - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } #endif /********************************************************************** **********************************************************************/ static void -__osm_mcast_mgr_clear( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_mgrp_t* const p_mgrp ) +__osm_mcast_mgr_clear(IN osm_mcast_mgr_t * const p_mgr, + IN osm_mgrp_t * const p_mgrp) { - osm_switch_t* p_sw; - cl_qmap_t* p_sw_tbl; - osm_mcast_tbl_t* p_mcast_tbl; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_mcast_mgr_clear ); - - /* - Walk the switches and clear the routing entries for - this MLID. - */ - p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; - p_sw = (osm_switch_t*)cl_qmap_head( p_sw_tbl ); - while( p_sw != (osm_switch_t*)cl_qmap_end( p_sw_tbl ) ) - { - p_mcast_tbl = osm_switch_get_mcast_tbl_ptr( p_sw ); - osm_mcast_tbl_clear_mlid( p_mcast_tbl, cl_ntoh16(p_mgrp->mlid) ); - p_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - } - - OSM_LOG_EXIT( p_mgr->p_log ); + osm_switch_t *p_sw; + cl_qmap_t *p_sw_tbl; + osm_mcast_tbl_t *p_mcast_tbl; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_mcast_mgr_clear); + + /* + Walk the switches and clear the routing entries for + this MLID. + */ + p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; + p_sw = (osm_switch_t *) cl_qmap_head(p_sw_tbl); + while (p_sw != (osm_switch_t *) cl_qmap_end(p_sw_tbl)) { + p_mcast_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); + osm_mcast_tbl_clear_mlid(p_mcast_tbl, cl_ntoh16(p_mgrp->mlid)); + p_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + } + + OSM_LOG_EXIT(p_mgr->p_log); } #if 0 @@ -1067,178 +1022,157 @@ __osm_mcast_mgr_clear( Lock must be held on entry. **********************************************************************/ ib_api_status_t -osm_mcast_mgr_process_single( - IN osm_mcast_mgr_t* const p_mgr, - IN ib_net16_t const mlid, - IN ib_net64_t const port_guid, - IN uint8_t const join_state ) +osm_mcast_mgr_process_single(IN osm_mcast_mgr_t * const p_mgr, + IN ib_net16_t const mlid, + IN ib_net64_t const port_guid, + IN uint8_t const join_state) { - uint8_t port_num; - uint16_t mlid_ho; - ib_net64_t sw_guid; - osm_port_t* p_port; - osm_physp_t* p_physp; - osm_physp_t* p_remote_physp; - osm_node_t* p_remote_node; - osm_mcast_tbl_t* p_mcast_tbl; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_process_single ); - - CL_ASSERT( mlid ); - CL_ASSERT( port_guid ); - - mlid_ho = cl_ntoh16( mlid ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_single: " - "Attempting to add port 0x%" PRIx64 " to MLID 0x%X, " - "\n\t\t\t\tjoin state = 0x%X\n", - cl_ntoh64( port_guid ), mlid_ho, join_state ); - } - - /* - Acquire the Port object. - */ - p_port = osm_get_port_by_guid( p_mgr->p_subn, port_guid ); - if( !p_port ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A01: " - "Unable to acquire port object for 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - p_physp = p_port->p_physp; - if( p_physp == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A05: " - "Unable to acquire phsyical port object for 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - if( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A07: " - "Unable to acquire valid physical port object " - "for 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - p_remote_physp = osm_physp_get_remote( p_physp ); - if( p_remote_physp == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A11: " - "Unable to acquire remote phsyical port object " - "for 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - if( !osm_physp_is_valid( p_remote_physp ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A21: " - "Unable to acquire valid remote physical port object " - "for 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_ERROR; - goto Exit; - } - - p_remote_node = osm_physp_get_node_ptr( p_remote_physp ); - - CL_ASSERT( p_remote_node ); - - sw_guid = osm_node_get_node_guid( p_remote_node ); - - if( osm_node_get_type( p_remote_node ) != IB_NODE_TYPE_SWITCH ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A22: " - "Remote node not a switch node 0x%" PRIx64 "\n", - cl_ntoh64( sw_guid ) ); - status = IB_ERROR; - goto Exit; - } - - if( !p_remote_node->sw ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A12: " - "No switch object 0x%" PRIx64 "\n", - cl_ntoh64( sw_guid ) ); - status = IB_ERROR; - goto Exit; - } - - if( osm_switch_is_in_mcast_tree( p_remote_node->sw, mlid_ho ) ) - { - /* - We're in luck. The switch attached to this port - is already in the multicast group, so we can just - add the specified port as a new leaf of the tree. - */ - if( join_state & (IB_JOIN_STATE_FULL | IB_JOIN_STATE_NON ) ) - { - /* - This node wants to receive multicast frames. - Get the switch port number to which the new member port - is attached, then configure this single mcast table. - */ - port_num = osm_physp_get_port_num( p_remote_physp ); - CL_ASSERT( port_num ); - - p_mcast_tbl = osm_switch_get_mcast_tbl_ptr( p_remote_node->sw ); - osm_mcast_tbl_set( p_mcast_tbl, mlid_ho, port_num ); - } - else - { - if( join_state & IB_JOIN_STATE_SEND_ONLY ) - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_single: " - "Success. Nothing to do for send" - "only member\n" ); - } - } - else - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_single: ERR 0A13: " - "Unknown join state 0x%X\n", join_state ); - status = IB_ERROR; - goto Exit; - } - } - } - else - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_single: " - "Unable to add port\n" ); - } - } - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + uint8_t port_num; + uint16_t mlid_ho; + ib_net64_t sw_guid; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; + osm_node_t *p_remote_node; + osm_mcast_tbl_t *p_mcast_tbl; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_process_single); + + CL_ASSERT(mlid); + CL_ASSERT(port_guid); + + mlid_ho = cl_ntoh16(mlid); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_single: " + "Attempting to add port 0x%" PRIx64 " to MLID 0x%X, " + "\n\t\t\t\tjoin state = 0x%X\n", + cl_ntoh64(port_guid), mlid_ho, join_state); + } + + /* + Acquire the Port object. + */ + p_port = osm_get_port_by_guid(p_mgr->p_subn, port_guid); + if (!p_port) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A01: " + "Unable to acquire port object for 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + p_physp = p_port->p_physp; + if (p_physp == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A05: " + "Unable to acquire phsyical port object for 0x%" PRIx64 + "\n", cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A07: " + "Unable to acquire valid physical port object " + "for 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + p_remote_physp = osm_physp_get_remote(p_physp); + if (p_remote_physp == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A11: " + "Unable to acquire remote phsyical port object " + "for 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + if (!osm_physp_is_valid(p_remote_physp)) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A21: " + "Unable to acquire valid remote physical port object " + "for 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + status = IB_ERROR; + goto Exit; + } + + p_remote_node = osm_physp_get_node_ptr(p_remote_physp); + + CL_ASSERT(p_remote_node); + + sw_guid = osm_node_get_node_guid(p_remote_node); + + if (osm_node_get_type(p_remote_node) != IB_NODE_TYPE_SWITCH) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A22: " + "Remote node not a switch node 0x%" PRIx64 "\n", + cl_ntoh64(sw_guid)); + status = IB_ERROR; + goto Exit; + } + + if (!p_remote_node->sw) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A12: " + "No switch object 0x%" PRIx64 "\n", cl_ntoh64(sw_guid)); + status = IB_ERROR; + goto Exit; + } + + if (osm_switch_is_in_mcast_tree(p_remote_node->sw, mlid_ho)) { + /* + We're in luck. The switch attached to this port + is already in the multicast group, so we can just + add the specified port as a new leaf of the tree. + */ + if (join_state & (IB_JOIN_STATE_FULL | IB_JOIN_STATE_NON)) { + /* + This node wants to receive multicast frames. + Get the switch port number to which the new member port + is attached, then configure this single mcast table. + */ + port_num = osm_physp_get_port_num(p_remote_physp); + CL_ASSERT(port_num); + + p_mcast_tbl = + osm_switch_get_mcast_tbl_ptr(p_remote_node->sw); + osm_mcast_tbl_set(p_mcast_tbl, mlid_ho, port_num); + } else { + if (join_state & IB_JOIN_STATE_SEND_ONLY) { + if (osm_log_is_active + (p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_single: " + "Success. Nothing to do for send" + "only member\n"); + } + } else { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_single: ERR 0A13: " + "Unknown join state 0x%X\n", + join_state); + status = IB_ERROR; + goto Exit; + } + } + } else { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_single: " + "Unable to add port\n"); + } + } + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } #endif @@ -1246,90 +1180,82 @@ osm_mcast_mgr_process_single( lock must already be held on entry **********************************************************************/ static ib_api_status_t -osm_mcast_mgr_process_tree( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_mgrp_t* const p_mgrp, - IN osm_mcast_req_type_t req_type, - ib_net64_t port_guid ) +osm_mcast_mgr_process_tree(IN osm_mcast_mgr_t * const p_mgr, + IN osm_mgrp_t * const p_mgrp, + IN osm_mcast_req_type_t req_type, + ib_net64_t port_guid) { - ib_api_status_t status = IB_SUCCESS; - ib_net16_t mlid; - boolean_t ui_mcast_fdb_assign_func_defined; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_process_tree ); - - mlid = osm_mgrp_get_mlid( p_mgrp ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_tree: " - "Processing multicast group 0x%X\n", cl_ntoh16( mlid )); - } - - /* - If there are no switches in the subnet, then we have nothing to do. - */ - if( cl_qmap_count( &p_mgr->p_subn->sw_guid_tbl ) == 0 ) - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_tree: " - "No switches in subnet. Nothing to do\n" ); - } - goto Exit; - } - - if (p_mgr->p_subn->opt.pfn_ui_mcast_fdb_assign) - ui_mcast_fdb_assign_func_defined = TRUE; - else - ui_mcast_fdb_assign_func_defined = FALSE; - - /* - Clear the multicast tables to start clean, then build - the spanning tree which sets the mcast table bits for each - port in the group. - We will clean the multicast tables if a ui_mcast function isn't - defined, or if such function is defined, but we got here - through a MC_CREATE request - this means we are creating a new - multicast group - clean all old data. - */ - if ( ui_mcast_fdb_assign_func_defined == FALSE || - req_type == OSM_MCAST_REQ_TYPE_CREATE ) - __osm_mcast_mgr_clear( p_mgr, p_mgrp ); - - /* If a UI function is defined, then we will call it here. - If not - the use the regular build spanning tree function */ - if ( ui_mcast_fdb_assign_func_defined == FALSE ) - { - status = __osm_mcast_mgr_build_spanning_tree( p_mgr, p_mgrp ); - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_tree: ERR 0A17: " - "Unable to create spanning tree (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } - } - else - { - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_tree: " - "Invoking UI function pfn_ui_mcast_fdb_assign\n"); - } - - p_mgr->p_subn->opt.pfn_ui_mcast_fdb_assign( - p_mgr->p_subn->opt.ui_mcast_fdb_assign_ctx, - mlid, req_type, port_guid ); - } - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + ib_net16_t mlid; + boolean_t ui_mcast_fdb_assign_func_defined; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_process_tree); + + mlid = osm_mgrp_get_mlid(p_mgrp); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_tree: " + "Processing multicast group 0x%X\n", cl_ntoh16(mlid)); + } + + /* + If there are no switches in the subnet, then we have nothing to do. + */ + if (cl_qmap_count(&p_mgr->p_subn->sw_guid_tbl) == 0) { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_tree: " + "No switches in subnet. Nothing to do\n"); + } + goto Exit; + } + + if (p_mgr->p_subn->opt.pfn_ui_mcast_fdb_assign) + ui_mcast_fdb_assign_func_defined = TRUE; + else + ui_mcast_fdb_assign_func_defined = FALSE; + + /* + Clear the multicast tables to start clean, then build + the spanning tree which sets the mcast table bits for each + port in the group. + We will clean the multicast tables if a ui_mcast function isn't + defined, or if such function is defined, but we got here + through a MC_CREATE request - this means we are creating a new + multicast group - clean all old data. + */ + if (ui_mcast_fdb_assign_func_defined == FALSE || + req_type == OSM_MCAST_REQ_TYPE_CREATE) + __osm_mcast_mgr_clear(p_mgr, p_mgrp); + + /* If a UI function is defined, then we will call it here. + If not - the use the regular build spanning tree function */ + if (ui_mcast_fdb_assign_func_defined == FALSE) { + status = __osm_mcast_mgr_build_spanning_tree(p_mgr, p_mgrp); + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_tree: ERR 0A17: " + "Unable to create spanning tree (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + } else { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_tree: " + "Invoking UI function pfn_ui_mcast_fdb_assign\n"); + } + + p_mgr->p_subn->opt.pfn_ui_mcast_fdb_assign(p_mgr->p_subn->opt. + ui_mcast_fdb_assign_ctx, + mlid, req_type, + port_guid); + } + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } /********************************************************************** @@ -1338,142 +1264,132 @@ osm_mcast_mgr_process_tree( NOTE : The lock should be held externally! **********************************************************************/ static osm_signal_t -osm_mcast_mgr_process_mgrp( - IN osm_mcast_mgr_t* const p_mgr, - IN osm_mgrp_t* const p_mgrp, - IN osm_mcast_req_type_t req_type, - IN ib_net64_t port_guid ) +osm_mcast_mgr_process_mgrp(IN osm_mcast_mgr_t * const p_mgr, + IN osm_mgrp_t * const p_mgrp, + IN osm_mcast_req_type_t req_type, + IN ib_net64_t port_guid) { - osm_signal_t signal = OSM_SIGNAL_DONE; - ib_api_status_t status; - osm_switch_t* p_sw; - cl_qmap_t* p_sw_tbl; - boolean_t pending_transactions = FALSE; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_process_mgrp ); - - p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; - - status = osm_mcast_mgr_process_tree( p_mgr, p_mgrp, req_type, port_guid ); - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process_mgrp: ERR 0A19: " - "Unable to create spanning tree (%s)\n", - ib_get_err_str( status ) ); - - goto Exit; - } - - /* - Walk the switches and download the tables for each. - */ - p_sw = (osm_switch_t*)cl_qmap_head( p_sw_tbl ); - while( p_sw != (osm_switch_t*)cl_qmap_end( p_sw_tbl ) ) - { - signal = __osm_mcast_mgr_set_tbl( p_mgr, p_sw ); - if( signal == OSM_SIGNAL_DONE_PENDING ) - pending_transactions = TRUE; - - p_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - } - - osm_dump_mcast_routes( p_mgr->p_subn->p_osm ); - - Exit: - OSM_LOG_EXIT( p_mgr->p_log ); - - if( pending_transactions == TRUE ) - return( OSM_SIGNAL_DONE_PENDING ); - else - return( OSM_SIGNAL_DONE ); + osm_signal_t signal = OSM_SIGNAL_DONE; + ib_api_status_t status; + osm_switch_t *p_sw; + cl_qmap_t *p_sw_tbl; + boolean_t pending_transactions = FALSE; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_process_mgrp); + + p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; + + status = osm_mcast_mgr_process_tree(p_mgr, p_mgrp, req_type, port_guid); + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process_mgrp: ERR 0A19: " + "Unable to create spanning tree (%s)\n", + ib_get_err_str(status)); + + goto Exit; + } + + /* + Walk the switches and download the tables for each. + */ + p_sw = (osm_switch_t *) cl_qmap_head(p_sw_tbl); + while (p_sw != (osm_switch_t *) cl_qmap_end(p_sw_tbl)) { + signal = __osm_mcast_mgr_set_tbl(p_mgr, p_sw); + if (signal == OSM_SIGNAL_DONE_PENDING) + pending_transactions = TRUE; + + p_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + } + + osm_dump_mcast_routes(p_mgr->p_subn->p_osm); + + Exit: + OSM_LOG_EXIT(p_mgr->p_log); + + if (pending_transactions == TRUE) + return (OSM_SIGNAL_DONE_PENDING); + else + return (OSM_SIGNAL_DONE); } /********************************************************************** **********************************************************************/ -osm_signal_t -osm_mcast_mgr_process( - IN osm_mcast_mgr_t* const p_mgr ) +osm_signal_t osm_mcast_mgr_process(IN osm_mcast_mgr_t * const p_mgr) { - osm_signal_t signal; - osm_switch_t* p_sw; - cl_qmap_t* p_sw_tbl; - cl_qmap_t* p_mcast_tbl; - osm_mgrp_t* p_mgrp; - ib_api_status_t status; - boolean_t pending_transactions = FALSE; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_process ); - - p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; - - p_mcast_tbl = &p_mgr->p_subn->mgrp_mlid_tbl; - /* - While holding the lock, iterate over all the established - multicast groups, servicing each in turn. - - Then, download the multicast tables to the switches. - */ - CL_PLOCK_EXCL_ACQUIRE( p_mgr->p_lock ); - - p_mgrp = (osm_mgrp_t*)cl_qmap_head( p_mcast_tbl ); - while( p_mgrp != (osm_mgrp_t*)cl_qmap_end( p_mcast_tbl ) ) - { - /* We reached here due to some change that caused a heavy sweep - of the subnet. Not due to a specific multicast request. - So the request type is subnet_change and the port guid is 0. */ - status = osm_mcast_mgr_process_tree( p_mgr, p_mgrp, - OSM_MCAST_REQ_TYPE_SUBNET_CHANGE, 0); - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_mcast_mgr_process: ERR 0A20: " - "Unable to create spanning tree (%s)\n", - ib_get_err_str( status ) ); - } - - p_mgrp = (osm_mgrp_t*)cl_qmap_next( &p_mgrp->map_item ); - } - - /* - Walk the switches and download the tables for each. - */ - p_sw = (osm_switch_t*)cl_qmap_head( p_sw_tbl ); - while( p_sw != (osm_switch_t*)cl_qmap_end( p_sw_tbl ) ) - { - signal = __osm_mcast_mgr_set_tbl( p_mgr, p_sw ); - if( signal == OSM_SIGNAL_DONE_PENDING ) - pending_transactions = TRUE; - - p_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - } - - CL_PLOCK_RELEASE( p_mgr->p_lock ); - - OSM_LOG_EXIT( p_mgr->p_log ); - - if( pending_transactions == TRUE ) - return( OSM_SIGNAL_DONE_PENDING ); - else - return( OSM_SIGNAL_DONE ); + osm_signal_t signal; + osm_switch_t *p_sw; + cl_qmap_t *p_sw_tbl; + cl_qmap_t *p_mcast_tbl; + osm_mgrp_t *p_mgrp; + ib_api_status_t status; + boolean_t pending_transactions = FALSE; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_process); + + p_sw_tbl = &p_mgr->p_subn->sw_guid_tbl; + + p_mcast_tbl = &p_mgr->p_subn->mgrp_mlid_tbl; + /* + While holding the lock, iterate over all the established + multicast groups, servicing each in turn. + + Then, download the multicast tables to the switches. + */ + CL_PLOCK_EXCL_ACQUIRE(p_mgr->p_lock); + + p_mgrp = (osm_mgrp_t *) cl_qmap_head(p_mcast_tbl); + while (p_mgrp != (osm_mgrp_t *) cl_qmap_end(p_mcast_tbl)) { + /* We reached here due to some change that caused a heavy sweep + of the subnet. Not due to a specific multicast request. + So the request type is subnet_change and the port guid is 0. */ + status = osm_mcast_mgr_process_tree(p_mgr, p_mgrp, + OSM_MCAST_REQ_TYPE_SUBNET_CHANGE, + 0); + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_mcast_mgr_process: ERR 0A20: " + "Unable to create spanning tree (%s)\n", + ib_get_err_str(status)); + } + + p_mgrp = (osm_mgrp_t *) cl_qmap_next(&p_mgrp->map_item); + } + + /* + Walk the switches and download the tables for each. + */ + p_sw = (osm_switch_t *) cl_qmap_head(p_sw_tbl); + while (p_sw != (osm_switch_t *) cl_qmap_end(p_sw_tbl)) { + signal = __osm_mcast_mgr_set_tbl(p_mgr, p_sw); + if (signal == OSM_SIGNAL_DONE_PENDING) + pending_transactions = TRUE; + + p_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + } + + CL_PLOCK_RELEASE(p_mgr->p_lock); + + OSM_LOG_EXIT(p_mgr->p_log); + + if (pending_transactions == TRUE) + return (OSM_SIGNAL_DONE_PENDING); + else + return (OSM_SIGNAL_DONE); } /********************************************************************** **********************************************************************/ static -osm_mgrp_t * -__get_mgrp_by_mlid( - IN osm_mcast_mgr_t* const p_mgr, - IN ib_net16_t const mlid ) +osm_mgrp_t *__get_mgrp_by_mlid(IN osm_mcast_mgr_t * const p_mgr, + IN ib_net16_t const mlid) { - cl_map_item_t *map_item; - - map_item = cl_qmap_get(&p_mgr->p_subn->mgrp_mlid_tbl, mlid); - if(map_item == cl_qmap_end(&p_mgr->p_subn->mgrp_mlid_tbl)) - { - return NULL; - } - return (osm_mgrp_t *)map_item; + cl_map_item_t *map_item; + + map_item = cl_qmap_get(&p_mgr->p_subn->mgrp_mlid_tbl, mlid); + if (map_item == cl_qmap_end(&p_mgr->p_subn->mgrp_mlid_tbl)) { + return NULL; + } + return (osm_mgrp_t *) map_item; } /********************************************************************** @@ -1482,81 +1398,76 @@ __get_mgrp_by_mlid( hold the mlid, port guid and action (join/leave/delete) required. **********************************************************************/ osm_signal_t -osm_mcast_mgr_process_mgrp_cb( - IN void* const Context1, - IN void* const Context2 ) +osm_mcast_mgr_process_mgrp_cb(IN void *const Context1, IN void *const Context2) { - osm_mcast_mgr_t* p_mgr = (osm_mcast_mgr_t*)Context1; - osm_mgrp_t* p_mgrp; - ib_net16_t mlid; - osm_signal_t signal = OSM_SIGNAL_DONE; - osm_mcast_mgr_ctxt_t* p_ctxt = (osm_mcast_mgr_ctxt_t*)Context2; - osm_mcast_req_type_t req_type = p_ctxt->req_type; - ib_net64_t port_guid = p_ctxt->port_guid; - - OSM_LOG_ENTER( p_mgr->p_log, osm_mcast_mgr_process_mgrp_cb ); - - /* nice copy no warning on size diff */ - memcpy(&mlid, &p_ctxt->mlid, sizeof(mlid)); - - /* we can destroy the context now */ - free(p_ctxt); - - /* we need a lock to make sure the p_mgrp is not change other ways */ - CL_PLOCK_EXCL_ACQUIRE( p_mgr->p_lock ); - p_mgrp = __get_mgrp_by_mlid( p_mgr, mlid); - - /* since we delayed the execution we prefer to pass the - mlid as the mgrp identifier and then find it or abort */ - - if (p_mgrp) - { - - /* if there was no change from the last time we processed the group - we can skip doing anything - */ - if ( p_mgrp->last_change_id == p_mgrp->last_tree_id) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_mgrp_cb: " - "Skip processing mgrp with lid:0x%X change id:%u\n", - cl_ntoh16(mlid), p_mgrp->last_change_id ); - } - else - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_mgrp_cb: " - "Processing mgrp with lid:0x%X change id:%u\n", - cl_ntoh16(mlid), p_mgrp->last_change_id ); - - signal = - osm_mcast_mgr_process_mgrp( p_mgr, p_mgrp, req_type, port_guid ); - p_mgrp->last_tree_id = p_mgrp->last_change_id; - } - - /* Remove MGRP only if osm_mcm_port_t count is 0 and - * Not a well known group - */ - if((0x0 == cl_qmap_count(&p_mgrp->mcm_port_tbl)) && - (p_mgrp->well_known == FALSE)) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_mcast_mgr_process_mgrp_cb: " - "Destroying mgrp with lid:0x%X\n", - cl_ntoh16(mlid) ); - - /* Send a Report to any InformInfo registered for - Trap 67 : MCGroup delete */ - osm_mgrp_send_delete_notice( p_mgr->p_subn, p_mgr->p_log, p_mgrp ); - - cl_qmap_remove_item(&p_mgr->p_subn->mgrp_mlid_tbl, - (cl_map_item_t *)p_mgrp ); - - osm_mgrp_delete(p_mgrp); - } - } - - CL_PLOCK_RELEASE( p_mgr->p_lock ); - OSM_LOG_EXIT( p_mgr->p_log ); - return signal; + osm_mcast_mgr_t *p_mgr = (osm_mcast_mgr_t *) Context1; + osm_mgrp_t *p_mgrp; + ib_net16_t mlid; + osm_signal_t signal = OSM_SIGNAL_DONE; + osm_mcast_mgr_ctxt_t *p_ctxt = (osm_mcast_mgr_ctxt_t *) Context2; + osm_mcast_req_type_t req_type = p_ctxt->req_type; + ib_net64_t port_guid = p_ctxt->port_guid; + + OSM_LOG_ENTER(p_mgr->p_log, osm_mcast_mgr_process_mgrp_cb); + + /* nice copy no warning on size diff */ + memcpy(&mlid, &p_ctxt->mlid, sizeof(mlid)); + + /* we can destroy the context now */ + free(p_ctxt); + + /* we need a lock to make sure the p_mgrp is not change other ways */ + CL_PLOCK_EXCL_ACQUIRE(p_mgr->p_lock); + p_mgrp = __get_mgrp_by_mlid(p_mgr, mlid); + + /* since we delayed the execution we prefer to pass the + mlid as the mgrp identifier and then find it or abort */ + + if (p_mgrp) { + + /* if there was no change from the last time we processed the group + we can skip doing anything + */ + if (p_mgrp->last_change_id == p_mgrp->last_tree_id) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_mgrp_cb: " + "Skip processing mgrp with lid:0x%X change id:%u\n", + cl_ntoh16(mlid), p_mgrp->last_change_id); + } else { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_mgrp_cb: " + "Processing mgrp with lid:0x%X change id:%u\n", + cl_ntoh16(mlid), p_mgrp->last_change_id); + + signal = + osm_mcast_mgr_process_mgrp(p_mgr, p_mgrp, req_type, + port_guid); + p_mgrp->last_tree_id = p_mgrp->last_change_id; + } + + /* Remove MGRP only if osm_mcm_port_t count is 0 and + * Not a well known group + */ + if ((0x0 == cl_qmap_count(&p_mgrp->mcm_port_tbl)) && + (p_mgrp->well_known == FALSE)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_mcast_mgr_process_mgrp_cb: " + "Destroying mgrp with lid:0x%X\n", + cl_ntoh16(mlid)); + + /* Send a Report to any InformInfo registered for + Trap 67 : MCGroup delete */ + osm_mgrp_send_delete_notice(p_mgr->p_subn, p_mgr->p_log, + p_mgrp); + + cl_qmap_remove_item(&p_mgr->p_subn->mgrp_mlid_tbl, + (cl_map_item_t *) p_mgrp); + + osm_mgrp_delete(p_mgrp); + } + } + + CL_PLOCK_RELEASE(p_mgr->p_lock); + OSM_LOG_EXIT(p_mgr->p_log); + return signal; } diff --git a/opensm/opensm/osm_mcast_tbl.c b/opensm/opensm/osm_mcast_tbl.c index a86f4aa..8715869 100644 --- a/opensm/opensm/osm_mcast_tbl.c +++ b/opensm/opensm/osm_mcast_tbl.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -55,248 +55,235 @@ #include #include - /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mcast_tbl_init( - IN osm_mcast_tbl_t* const p_tbl, - IN uint8_t const num_ports, - IN uint16_t const capacity ) +osm_mcast_tbl_init(IN osm_mcast_tbl_t * const p_tbl, + IN uint8_t const num_ports, IN uint16_t const capacity) { - CL_ASSERT( p_tbl ); - CL_ASSERT( num_ports ); - - memset( p_tbl, 0, sizeof(*p_tbl) ); - - p_tbl->max_block_in_use = -1; - - if( capacity == 0 ) - { - /* - This switch apparently doesn't support multicast. - Everything is initialized to zero already, so return. - */ - return( IB_SUCCESS ); - } - - p_tbl->num_entries = capacity; - p_tbl->num_ports = num_ports; - p_tbl->max_position = (uint8_t)((ROUNDUP( num_ports, IB_MCAST_MASK_SIZE) / - IB_MCAST_MASK_SIZE) - 1); - - p_tbl->max_block = (uint16_t)((ROUNDUP( p_tbl->num_entries, - IB_MCAST_BLOCK_SIZE ) / IB_MCAST_BLOCK_SIZE) - 1); - - p_tbl->max_mlid_ho = (uint16_t)(IB_LID_MCAST_START_HO + capacity); - - /* - The number of bytes needed in the mask table is: - The (maximum bit mask 'position' + 1) times the - number of bytes in each bit mask times the - number of MLIDs supported by the table. - - We must always allocate the array with the maximum position - since it is (and must be) defined that way the table structure - in order to create a pointer to a two dimensional array. - */ - p_tbl->p_mask_tbl = malloc( p_tbl->num_entries * - (IB_MCAST_POSITION_MAX + 1) * IB_MCAST_MASK_SIZE / 8 ); - - if( p_tbl->p_mask_tbl == NULL ) - return( IB_INSUFFICIENT_MEMORY ); - - memset(p_tbl->p_mask_tbl, 0, - p_tbl->num_entries * (IB_MCAST_POSITION_MAX + 1) * IB_MCAST_MASK_SIZE / 8 ); - return( IB_SUCCESS ); + CL_ASSERT(p_tbl); + CL_ASSERT(num_ports); + + memset(p_tbl, 0, sizeof(*p_tbl)); + + p_tbl->max_block_in_use = -1; + + if (capacity == 0) { + /* + This switch apparently doesn't support multicast. + Everything is initialized to zero already, so return. + */ + return (IB_SUCCESS); + } + + p_tbl->num_entries = capacity; + p_tbl->num_ports = num_ports; + p_tbl->max_position = + (uint8_t) ((ROUNDUP(num_ports, IB_MCAST_MASK_SIZE) / + IB_MCAST_MASK_SIZE) - 1); + + p_tbl->max_block = (uint16_t) ((ROUNDUP(p_tbl->num_entries, + IB_MCAST_BLOCK_SIZE) / + IB_MCAST_BLOCK_SIZE) - 1); + + p_tbl->max_mlid_ho = (uint16_t) (IB_LID_MCAST_START_HO + capacity); + + /* + The number of bytes needed in the mask table is: + The (maximum bit mask 'position' + 1) times the + number of bytes in each bit mask times the + number of MLIDs supported by the table. + + We must always allocate the array with the maximum position + since it is (and must be) defined that way the table structure + in order to create a pointer to a two dimensional array. + */ + p_tbl->p_mask_tbl = malloc(p_tbl->num_entries * + (IB_MCAST_POSITION_MAX + + 1) * IB_MCAST_MASK_SIZE / 8); + + if (p_tbl->p_mask_tbl == NULL) + return (IB_INSUFFICIENT_MEMORY); + + memset(p_tbl->p_mask_tbl, 0, + p_tbl->num_entries * (IB_MCAST_POSITION_MAX + + 1) * IB_MCAST_MASK_SIZE / 8); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -void -osm_mcast_tbl_destroy( - IN osm_mcast_tbl_t* const p_tbl ) +void osm_mcast_tbl_destroy(IN osm_mcast_tbl_t * const p_tbl) { - free( p_tbl->p_mask_tbl ); + free(p_tbl->p_mask_tbl); } /********************************************************************** **********************************************************************/ void -osm_mcast_tbl_set( - IN osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho, - IN const uint8_t port ) +osm_mcast_tbl_set(IN osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho, IN const uint8_t port) { - uintn_t mlid_offset; - uintn_t mask_offset; - uintn_t bit_mask; - int16_t block_num; + uintn_t mlid_offset; + uintn_t mask_offset; + uintn_t bit_mask; + int16_t block_num; - CL_ASSERT( p_tbl ); - CL_ASSERT( mlid_ho >= IB_LID_MCAST_START_HO ); - CL_ASSERT( mlid_ho <= p_tbl->max_mlid_ho ); - CL_ASSERT( p_tbl->p_mask_tbl ); + CL_ASSERT(p_tbl); + CL_ASSERT(mlid_ho >= IB_LID_MCAST_START_HO); + CL_ASSERT(mlid_ho <= p_tbl->max_mlid_ho); + CL_ASSERT(p_tbl->p_mask_tbl); - mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; - mask_offset = port / IB_MCAST_MASK_SIZE; - bit_mask = cl_ntoh16( (uint16_t)( 1 << (port % IB_MCAST_MASK_SIZE) ) ); - (*p_tbl->p_mask_tbl)[mlid_offset][mask_offset] |= bit_mask; + mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; + mask_offset = port / IB_MCAST_MASK_SIZE; + bit_mask = cl_ntoh16((uint16_t) (1 << (port % IB_MCAST_MASK_SIZE))); + (*p_tbl->p_mask_tbl)[mlid_offset][mask_offset] |= bit_mask; - block_num = (int16_t)(mlid_offset / IB_MCAST_BLOCK_SIZE); + block_num = (int16_t) (mlid_offset / IB_MCAST_BLOCK_SIZE); - if( block_num > p_tbl->max_block_in_use ) - p_tbl->max_block_in_use = (uint16_t)block_num; + if (block_num > p_tbl->max_block_in_use) + p_tbl->max_block_in_use = (uint16_t) block_num; } /********************************************************************** **********************************************************************/ boolean_t -osm_mcast_tbl_is_port( - IN const osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho, - IN const uint8_t port_num ) +osm_mcast_tbl_is_port(IN const osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho, IN const uint8_t port_num) { - uintn_t mlid_offset; - uintn_t mask_offset; - uintn_t bit_mask; - - CL_ASSERT( p_tbl ); - - if( p_tbl->p_mask_tbl ) - { - CL_ASSERT( port_num <= (p_tbl->max_position + 1) * IB_MCAST_MASK_SIZE ); - CL_ASSERT( mlid_ho >= IB_LID_MCAST_START_HO ); - CL_ASSERT( mlid_ho <= p_tbl->max_mlid_ho ); - - mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; - mask_offset = port_num / IB_MCAST_MASK_SIZE; - bit_mask = cl_ntoh16( - (uint16_t)( 1 << (port_num % IB_MCAST_MASK_SIZE) ) ); - return( ((*p_tbl->p_mask_tbl)[mlid_offset][mask_offset] & bit_mask) == - bit_mask ); - } - - return( FALSE ); + uintn_t mlid_offset; + uintn_t mask_offset; + uintn_t bit_mask; + + CL_ASSERT(p_tbl); + + if (p_tbl->p_mask_tbl) { + CL_ASSERT(port_num <= + (p_tbl->max_position + 1) * IB_MCAST_MASK_SIZE); + CL_ASSERT(mlid_ho >= IB_LID_MCAST_START_HO); + CL_ASSERT(mlid_ho <= p_tbl->max_mlid_ho); + + mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; + mask_offset = port_num / IB_MCAST_MASK_SIZE; + bit_mask = cl_ntoh16((uint16_t) + (1 << (port_num % IB_MCAST_MASK_SIZE))); + return (((*p_tbl-> + p_mask_tbl)[mlid_offset][mask_offset] & bit_mask) == + bit_mask); + } + + return (FALSE); } /********************************************************************** **********************************************************************/ boolean_t -osm_mcast_tbl_is_any_port( - IN const osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho ) +osm_mcast_tbl_is_any_port(IN const osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho) { - uintn_t mlid_offset; - uint8_t position; - uint16_t result = 0; + uintn_t mlid_offset; + uint8_t position; + uint16_t result = 0; - CL_ASSERT( p_tbl ); + CL_ASSERT(p_tbl); - if( p_tbl->p_mask_tbl ) - { - CL_ASSERT( mlid_ho >= IB_LID_MCAST_START_HO ); - CL_ASSERT( mlid_ho <= p_tbl->max_mlid_ho ); + if (p_tbl->p_mask_tbl) { + CL_ASSERT(mlid_ho >= IB_LID_MCAST_START_HO); + CL_ASSERT(mlid_ho <= p_tbl->max_mlid_ho); - mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; + mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; - for( position = 0; position <= p_tbl->max_position; position++ ) - result |= (*p_tbl->p_mask_tbl)[mlid_offset][position]; - } + for (position = 0; position <= p_tbl->max_position; position++) + result |= (*p_tbl->p_mask_tbl)[mlid_offset][position]; + } - return( result != 0 ); + return (result != 0); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mcast_tbl_set_block( - IN osm_mcast_tbl_t* const p_tbl, - IN const ib_net16_t* const p_block, - IN const int16_t block_num, - IN const uint8_t position ) +osm_mcast_tbl_set_block(IN osm_mcast_tbl_t * const p_tbl, + IN const ib_net16_t * const p_block, + IN const int16_t block_num, IN const uint8_t position) { - uint32_t i; - uint16_t mlid_start_ho; + uint32_t i; + uint16_t mlid_start_ho; - CL_ASSERT( p_tbl ); - CL_ASSERT( p_block ); + CL_ASSERT(p_tbl); + CL_ASSERT(p_block); - if( block_num > p_tbl->max_block ) - return( IB_INVALID_PARAMETER ); + if (block_num > p_tbl->max_block) + return (IB_INVALID_PARAMETER); - if( position > p_tbl->max_position ) - return( IB_INVALID_PARAMETER ); + if (position > p_tbl->max_position) + return (IB_INVALID_PARAMETER); - mlid_start_ho = (uint16_t)(block_num * IB_MCAST_BLOCK_SIZE); + mlid_start_ho = (uint16_t) (block_num * IB_MCAST_BLOCK_SIZE); - if( mlid_start_ho + IB_MCAST_BLOCK_SIZE > p_tbl->max_mlid_ho ) - return( IB_INVALID_PARAMETER ); + if (mlid_start_ho + IB_MCAST_BLOCK_SIZE > p_tbl->max_mlid_ho) + return (IB_INVALID_PARAMETER); - for( i = 0; i < IB_MCAST_BLOCK_SIZE; i++ ) - (*p_tbl->p_mask_tbl)[mlid_start_ho + i][position] = p_block[i]; + for (i = 0; i < IB_MCAST_BLOCK_SIZE; i++) + (*p_tbl->p_mask_tbl)[mlid_start_ho + i][position] = p_block[i]; - if( block_num > p_tbl->max_block_in_use ) - p_tbl->max_block_in_use = (uint16_t)block_num; + if (block_num > p_tbl->max_block_in_use) + p_tbl->max_block_in_use = (uint16_t) block_num; - return( IB_SUCCESS ); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ void -osm_mcast_tbl_clear_mlid( - IN osm_mcast_tbl_t* const p_tbl, - IN const uint16_t mlid_ho ) +osm_mcast_tbl_clear_mlid(IN osm_mcast_tbl_t * const p_tbl, + IN const uint16_t mlid_ho) { - uint8_t i; - uintn_t mlid_offset; - - CL_ASSERT( p_tbl ); - CL_ASSERT( mlid_ho >= IB_LID_MCAST_START_HO ); - - if( p_tbl->p_mask_tbl && (mlid_ho <= p_tbl->max_mlid_ho) ) - { - mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; - for( i = 0; i <= p_tbl->max_position; i++ ) - (*p_tbl->p_mask_tbl)[mlid_offset][i] = 0; - } + uint8_t i; + uintn_t mlid_offset; + + CL_ASSERT(p_tbl); + CL_ASSERT(mlid_ho >= IB_LID_MCAST_START_HO); + + if (p_tbl->p_mask_tbl && (mlid_ho <= p_tbl->max_mlid_ho)) { + mlid_offset = mlid_ho - IB_LID_MCAST_START_HO; + for (i = 0; i <= p_tbl->max_position; i++) + (*p_tbl->p_mask_tbl)[mlid_offset][i] = 0; + } } /********************************************************************** **********************************************************************/ boolean_t -osm_mcast_tbl_get_block( - IN osm_mcast_tbl_t* const p_tbl, - IN int16_t const block_num, - IN uint8_t const position, - OUT ib_net16_t* const p_block ) +osm_mcast_tbl_get_block(IN osm_mcast_tbl_t * const p_tbl, + IN int16_t const block_num, + IN uint8_t const position, + OUT ib_net16_t * const p_block) { - uint32_t i; - uint16_t mlid_start_ho; + uint32_t i; + uint16_t mlid_start_ho; - CL_ASSERT( p_tbl ); - CL_ASSERT( p_block ); + CL_ASSERT(p_tbl); + CL_ASSERT(p_block); - if( block_num > p_tbl->max_block_in_use ) - return( FALSE ); + if (block_num > p_tbl->max_block_in_use) + return (FALSE); - if( position > p_tbl->max_position ) - { - /* - Caller shouldn't do this for efficiency's sake... - */ - memset( p_block, 0, IB_SMP_DATA_SIZE ); - return( TRUE ); - } + if (position > p_tbl->max_position) { + /* + Caller shouldn't do this for efficiency's sake... + */ + memset(p_block, 0, IB_SMP_DATA_SIZE); + return (TRUE); + } - mlid_start_ho = (uint16_t)(block_num * IB_MCAST_BLOCK_SIZE); + mlid_start_ho = (uint16_t) (block_num * IB_MCAST_BLOCK_SIZE); - if( mlid_start_ho + IB_MCAST_BLOCK_SIZE > p_tbl->max_mlid_ho ) - return( IB_INVALID_PARAMETER ); + if (mlid_start_ho + IB_MCAST_BLOCK_SIZE > p_tbl->max_mlid_ho) + return (IB_INVALID_PARAMETER); - for( i = 0; i < IB_MCAST_BLOCK_SIZE; i++ ) - p_block[i] = (*p_tbl->p_mask_tbl)[mlid_start_ho + i][position]; + for (i = 0; i < IB_MCAST_BLOCK_SIZE; i++) + p_block[i] = (*p_tbl->p_mask_tbl)[mlid_start_ho + i][position]; - return( TRUE ); + return (TRUE); } diff --git a/opensm/opensm/osm_mcm_info.c b/opensm/opensm/osm_mcm_info.c index d296814..5d5d3a9 100644 --- a/opensm/opensm/osm_mcm_info.c +++ b/opensm/opensm/osm_mcm_info.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Declaration of osm_mcm_info_t. @@ -48,34 +47,29 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include /********************************************************************** **********************************************************************/ -osm_mcm_info_t* -osm_mcm_info_new( - IN const ib_net16_t mlid ) +osm_mcm_info_t *osm_mcm_info_new(IN const ib_net16_t mlid) { - osm_mcm_info_t* p_mcm; + osm_mcm_info_t *p_mcm; - p_mcm = (osm_mcm_info_t*)malloc( sizeof(*p_mcm) ); - if( p_mcm ) - { - memset(p_mcm, 0, sizeof(*p_mcm) ); - p_mcm->mlid = mlid; - } + p_mcm = (osm_mcm_info_t *) malloc(sizeof(*p_mcm)); + if (p_mcm) { + memset(p_mcm, 0, sizeof(*p_mcm)); + p_mcm->mlid = mlid; + } - return( p_mcm ); + return (p_mcm); } /********************************************************************** **********************************************************************/ -void -osm_mcm_info_delete( - IN osm_mcm_info_t* const p_mcm ) +void osm_mcm_info_delete(IN osm_mcm_info_t * const p_mcm) { - free( p_mcm ); + free(p_mcm); } diff --git a/opensm/opensm/osm_mcm_port.c b/opensm/opensm/osm_mcm_port.c index b11f929..1dac7fe 100644 --- a/opensm/opensm/osm_mcm_port.c +++ b/opensm/opensm/osm_mcm_port.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,47 +56,39 @@ /********************************************************************** **********************************************************************/ static void -osm_mcm_port_init( - IN osm_mcm_port_t* const p_mcm, - IN const ib_gid_t* const p_port_gid, - IN const uint8_t scope_state, - IN const boolean_t proxy_join ) +osm_mcm_port_init(IN osm_mcm_port_t * const p_mcm, + IN const ib_gid_t * const p_port_gid, + IN const uint8_t scope_state, IN const boolean_t proxy_join) { - CL_ASSERT( p_port_gid ); - CL_ASSERT( scope_state ); + CL_ASSERT(p_port_gid); + CL_ASSERT(scope_state); - p_mcm->port_gid = *p_port_gid; - p_mcm->scope_state = scope_state; - p_mcm->proxy_join = proxy_join; + p_mcm->port_gid = *p_port_gid; + p_mcm->scope_state = scope_state; + p_mcm->proxy_join = proxy_join; } /********************************************************************** **********************************************************************/ -osm_mcm_port_t* -osm_mcm_port_new( - IN const ib_gid_t* const p_port_gid, - IN const uint8_t scope_state, - IN const boolean_t proxy_join ) +osm_mcm_port_t *osm_mcm_port_new(IN const ib_gid_t * const p_port_gid, + IN const uint8_t scope_state, + IN const boolean_t proxy_join) { - osm_mcm_port_t* p_mcm; + osm_mcm_port_t *p_mcm; - p_mcm = malloc( sizeof(*p_mcm) ); - if( p_mcm ) - { - memset( p_mcm, 0, sizeof(*p_mcm) ); - osm_mcm_port_init( p_mcm, p_port_gid, - scope_state, proxy_join ); - } + p_mcm = malloc(sizeof(*p_mcm)); + if (p_mcm) { + memset(p_mcm, 0, sizeof(*p_mcm)); + osm_mcm_port_init(p_mcm, p_port_gid, scope_state, proxy_join); + } - return( p_mcm ); + return (p_mcm); } /********************************************************************** **********************************************************************/ -void -osm_mcm_port_delete( - IN osm_mcm_port_t* const p_mcm ) +void osm_mcm_port_delete(IN osm_mcm_port_t * const p_mcm) { - CL_ASSERT( p_mcm ); - free( p_mcm ); + CL_ASSERT(p_mcm); + free(p_mcm); } diff --git a/opensm/opensm/osm_mtree.c b/opensm/opensm/osm_mtree.c index e43bc72..9ce58d4 100644 --- a/opensm/opensm/osm_mtree.c +++ b/opensm/opensm/osm_mtree.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -55,82 +55,74 @@ /********************************************************************** **********************************************************************/ static void -osm_mtree_node_init( - IN osm_mtree_node_t* const p_mtn, - IN const osm_switch_t* const p_sw ) +osm_mtree_node_init(IN osm_mtree_node_t * const p_mtn, + IN const osm_switch_t * const p_sw) { - uint32_t i; + uint32_t i; - CL_ASSERT( p_mtn ); - CL_ASSERT( p_sw ); + CL_ASSERT(p_mtn); + CL_ASSERT(p_sw); - memset( p_mtn, 0, sizeof(*p_mtn) ); + memset(p_mtn, 0, sizeof(*p_mtn)); - p_mtn->p_sw = (osm_switch_t*)p_sw; - p_mtn->max_children = p_sw->num_ports; + p_mtn->p_sw = (osm_switch_t *) p_sw; + p_mtn->max_children = p_sw->num_ports; - for( i = 0; i < p_mtn->max_children; i++ ) - p_mtn->child_array[i] = NULL; + for (i = 0; i < p_mtn->max_children; i++) + p_mtn->child_array[i] = NULL; } /********************************************************************** **********************************************************************/ -osm_mtree_node_t* -osm_mtree_node_new( - IN const osm_switch_t* const p_sw ) +osm_mtree_node_t *osm_mtree_node_new(IN const osm_switch_t * const p_sw) { - osm_mtree_node_t *p_mtn; + osm_mtree_node_t *p_mtn; - p_mtn = malloc( sizeof(osm_mtree_node_t) + - sizeof(void*) * (p_sw->num_ports - 1) ); + p_mtn = malloc(sizeof(osm_mtree_node_t) + + sizeof(void *) * (p_sw->num_ports - 1)); - if( p_mtn != NULL ) - osm_mtree_node_init( p_mtn, p_sw ); + if (p_mtn != NULL) + osm_mtree_node_init(p_mtn, p_sw); - return( p_mtn ); + return (p_mtn); } /********************************************************************** **********************************************************************/ -void -osm_mtree_destroy( - IN osm_mtree_node_t *p_mtn ) +void osm_mtree_destroy(IN osm_mtree_node_t * p_mtn) { - uint32_t i; + uint32_t i; - if (p_mtn == NULL) - return; + if (p_mtn == NULL) + return; - if ( p_mtn->child_array != NULL ) - for (i = 0 ; i< p_mtn->max_children; i++ ) - if ( (p_mtn->child_array[i] != NULL) && - (p_mtn->child_array[i] != OSM_MTREE_LEAF) ) - osm_mtree_destroy(p_mtn->child_array[i]); + if (p_mtn->child_array != NULL) + for (i = 0; i < p_mtn->max_children; i++) + if ((p_mtn->child_array[i] != NULL) && + (p_mtn->child_array[i] != OSM_MTREE_LEAF)) + osm_mtree_destroy(p_mtn->child_array[i]); - free( p_mtn ); + free(p_mtn); } /********************************************************************** **********************************************************************/ -void -__osm_mtree_dump( - IN osm_mtree_node_t *p_mtn ) +void __osm_mtree_dump(IN osm_mtree_node_t * p_mtn) { - uint32_t i; - - if (p_mtn == NULL) - return; - - printf("GUID:0x%016" PRIx64 " max_children:%u\n", - cl_ntoh64(p_mtn->p_sw->p_node->node_info.node_guid), - p_mtn->max_children ); - if ( p_mtn->child_array != NULL ) - { - for (i = 0 ; i< p_mtn->max_children; i++ ) - { - printf("i=%d\n", i); - if ( (p_mtn->child_array[i] != NULL) && (p_mtn->child_array[i] != OSM_MTREE_LEAF) ) - __osm_mtree_dump(p_mtn->child_array[i]); - } - } + uint32_t i; + + if (p_mtn == NULL) + return; + + printf("GUID:0x%016" PRIx64 " max_children:%u\n", + cl_ntoh64(p_mtn->p_sw->p_node->node_info.node_guid), + p_mtn->max_children); + if (p_mtn->child_array != NULL) { + for (i = 0; i < p_mtn->max_children; i++) { + printf("i=%d\n", i); + if ((p_mtn->child_array[i] != NULL) + && (p_mtn->child_array[i] != OSM_MTREE_LEAF)) + __osm_mtree_dump(p_mtn->child_array[i]); + } + } } diff --git a/opensm/opensm/osm_multicast.c b/opensm/opensm/osm_multicast.c index ad4389a..2e25263 100644 --- a/opensm/opensm/osm_multicast.c +++ b/opensm/opensm/osm_multicast.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -57,337 +57,308 @@ /********************************************************************** **********************************************************************/ /* osm_mcast_req_type_t values converted to test for easier printing. */ -const char* osm_mcast_req_type_str[] = -{ - "OSM_MCAST_REQ_TYPE_CREATE", - "OSM_MCAST_REQ_TYPE_JOIN", - "OSM_MCAST_REQ_TYPE_LEAVE", - "OSM_MCAST_REQ_TYPE_SUBNET_CHANGE" +const char *osm_mcast_req_type_str[] = { + "OSM_MCAST_REQ_TYPE_CREATE", + "OSM_MCAST_REQ_TYPE_JOIN", + "OSM_MCAST_REQ_TYPE_LEAVE", + "OSM_MCAST_REQ_TYPE_SUBNET_CHANGE" }; -const char* -osm_get_mcast_req_type_str( - IN osm_mcast_req_type_t req_type ) +const char *osm_get_mcast_req_type_str(IN osm_mcast_req_type_t req_type) { - if ( req_type > OSM_MCAST_REQ_TYPE_SUBNET_CHANGE ) - req_type = OSM_MCAST_REQ_TYPE_SUBNET_CHANGE; - return( osm_mcast_req_type_str[req_type] ); + if (req_type > OSM_MCAST_REQ_TYPE_SUBNET_CHANGE) + req_type = OSM_MCAST_REQ_TYPE_SUBNET_CHANGE; + return (osm_mcast_req_type_str[req_type]); } /********************************************************************** **********************************************************************/ -void -osm_mgrp_delete( - IN osm_mgrp_t* const p_mgrp ) +void osm_mgrp_delete(IN osm_mgrp_t * const p_mgrp) { - osm_mcm_port_t *p_mcm_port; - osm_mcm_port_t *p_next_mcm_port; - - CL_ASSERT(p_mgrp); - - p_next_mcm_port = (osm_mcm_port_t*)cl_qmap_head( &p_mgrp->mcm_port_tbl ); - while( p_next_mcm_port != (osm_mcm_port_t*)cl_qmap_end( &p_mgrp->mcm_port_tbl ) ) - { - p_mcm_port = p_next_mcm_port; - p_next_mcm_port = (osm_mcm_port_t*)cl_qmap_next( &p_mcm_port->map_item ); - osm_mcm_port_delete( p_mcm_port ); - } - /* destroy the mtree_node structure */ - osm_mtree_destroy(p_mgrp->p_root); - - free(p_mgrp); + osm_mcm_port_t *p_mcm_port; + osm_mcm_port_t *p_next_mcm_port; + + CL_ASSERT(p_mgrp); + + p_next_mcm_port = + (osm_mcm_port_t *) cl_qmap_head(&p_mgrp->mcm_port_tbl); + while (p_next_mcm_port != + (osm_mcm_port_t *) cl_qmap_end(&p_mgrp->mcm_port_tbl)) { + p_mcm_port = p_next_mcm_port; + p_next_mcm_port = + (osm_mcm_port_t *) cl_qmap_next(&p_mcm_port->map_item); + osm_mcm_port_delete(p_mcm_port); + } + /* destroy the mtree_node structure */ + osm_mtree_destroy(p_mgrp->p_root); + + free(p_mgrp); } /********************************************************************** **********************************************************************/ static void -osm_mgrp_init( - IN osm_mgrp_t* const p_mgrp, - IN const ib_net16_t mlid ) +osm_mgrp_init(IN osm_mgrp_t * const p_mgrp, IN const ib_net16_t mlid) { - CL_ASSERT( cl_ntoh16( mlid ) >= IB_LID_MCAST_START_HO ); - - memset( p_mgrp, 0, sizeof(*p_mgrp) ); - cl_qmap_init( &p_mgrp->mcm_port_tbl ); - p_mgrp->mlid = mlid; - p_mgrp->last_change_id = 0; - p_mgrp->last_tree_id = 0; - p_mgrp->to_be_deleted = FALSE; + CL_ASSERT(cl_ntoh16(mlid) >= IB_LID_MCAST_START_HO); + + memset(p_mgrp, 0, sizeof(*p_mgrp)); + cl_qmap_init(&p_mgrp->mcm_port_tbl); + p_mgrp->mlid = mlid; + p_mgrp->last_change_id = 0; + p_mgrp->last_tree_id = 0; + p_mgrp->to_be_deleted = FALSE; } /********************************************************************** **********************************************************************/ -osm_mgrp_t* -osm_mgrp_new( - IN const ib_net16_t mlid ) +osm_mgrp_t *osm_mgrp_new(IN const ib_net16_t mlid) { - osm_mgrp_t* p_mgrp; + osm_mgrp_t *p_mgrp; - p_mgrp = (osm_mgrp_t*)malloc( sizeof(*p_mgrp) ); - if( p_mgrp ) - osm_mgrp_init( p_mgrp, mlid ); + p_mgrp = (osm_mgrp_t *) malloc(sizeof(*p_mgrp)); + if (p_mgrp) + osm_mgrp_init(p_mgrp, mlid); - return( p_mgrp ); + return (p_mgrp); } /********************************************************************** **********************************************************************/ -osm_mcm_port_t* -osm_mgrp_add_port( - IN osm_mgrp_t* const p_mgrp, - IN const ib_gid_t* const p_port_gid, - IN const uint8_t join_state, - IN boolean_t proxy_join ) +osm_mcm_port_t *osm_mgrp_add_port(IN osm_mgrp_t * const p_mgrp, + IN const ib_gid_t * const p_port_gid, + IN const uint8_t join_state, + IN boolean_t proxy_join) { - ib_net64_t port_guid; - osm_mcm_port_t *p_mcm_port; - cl_map_item_t *prev_item; - uint8_t prev_join_state; - uint8_t prev_scope; - - p_mcm_port = osm_mcm_port_new( p_port_gid, join_state, proxy_join ); - if( p_mcm_port ) - { - port_guid = p_port_gid->unicast.interface_id; - - /* - prev_item = cl_qmap_insert(...) - Pointer to the item in the map with the specified key. If insertion - was successful, this is the pointer to the item. If an item with the - specified key already exists in the map, the pointer to that item is - returned. - */ - prev_item = cl_qmap_insert( &p_mgrp->mcm_port_tbl, - port_guid, &p_mcm_port->map_item ); - - /* if already exists - revert the insertion and only update join state */ - if( prev_item != &p_mcm_port->map_item ) - { - - osm_mcm_port_delete( p_mcm_port ); - p_mcm_port =(osm_mcm_port_t *) prev_item; - - /* - o15.0.1.11 - Join state of the end port should be the or of the - previous setting with the current one - */ - ib_member_get_scope_state(p_mcm_port->scope_state, &prev_scope, &prev_join_state); - p_mcm_port->scope_state = - ib_member_set_scope_state(prev_scope, prev_join_state | join_state); - - } - else - { - /* track the fact we modified the group ports */ - p_mgrp->last_change_id++; - } - } - - return( p_mcm_port ); + ib_net64_t port_guid; + osm_mcm_port_t *p_mcm_port; + cl_map_item_t *prev_item; + uint8_t prev_join_state; + uint8_t prev_scope; + + p_mcm_port = osm_mcm_port_new(p_port_gid, join_state, proxy_join); + if (p_mcm_port) { + port_guid = p_port_gid->unicast.interface_id; + + /* + prev_item = cl_qmap_insert(...) + Pointer to the item in the map with the specified key. If insertion + was successful, this is the pointer to the item. If an item with the + specified key already exists in the map, the pointer to that item is + returned. + */ + prev_item = cl_qmap_insert(&p_mgrp->mcm_port_tbl, + port_guid, &p_mcm_port->map_item); + + /* if already exists - revert the insertion and only update join state */ + if (prev_item != &p_mcm_port->map_item) { + + osm_mcm_port_delete(p_mcm_port); + p_mcm_port = (osm_mcm_port_t *) prev_item; + + /* + o15.0.1.11 + Join state of the end port should be the or of the + previous setting with the current one + */ + ib_member_get_scope_state(p_mcm_port->scope_state, + &prev_scope, + &prev_join_state); + p_mcm_port->scope_state = + ib_member_set_scope_state(prev_scope, + prev_join_state | + join_state); + + } else { + /* track the fact we modified the group ports */ + p_mgrp->last_change_id++; + } + } + + return (p_mcm_port); } /********************************************************************** **********************************************************************/ void -osm_mgrp_remove_port( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t* const p_mgrp, - IN const ib_net64_t port_guid ) +osm_mgrp_remove_port(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid) { - cl_map_item_t *p_map_item; - - CL_ASSERT(p_mgrp); - - p_map_item = cl_qmap_get( &p_mgrp->mcm_port_tbl, port_guid ); - - if( p_map_item != cl_qmap_end( &p_mgrp->mcm_port_tbl ) ) - { - cl_qmap_remove_item( &p_mgrp->mcm_port_tbl, - p_map_item ); - osm_mcm_port_delete((osm_mcm_port_t*)p_map_item); - - /* track the fact we modified the group */ - p_mgrp->last_change_id++; - } - - /* - no more ports so the group will be deleted after re-route - but only if it is not a well known group and not already deleted - */ - if ((cl_is_qmap_empty( &p_mgrp->mcm_port_tbl )) && - (p_mgrp->well_known == FALSE) && - (p_mgrp->to_be_deleted == FALSE)) - { - p_mgrp->to_be_deleted = TRUE; - - /* Send a Report to any InformInfo registered for - Trap 67 : MCGroup delete */ - osm_mgrp_send_delete_notice( p_subn, p_log, p_mgrp ); - } + cl_map_item_t *p_map_item; + + CL_ASSERT(p_mgrp); + + p_map_item = cl_qmap_get(&p_mgrp->mcm_port_tbl, port_guid); + + if (p_map_item != cl_qmap_end(&p_mgrp->mcm_port_tbl)) { + cl_qmap_remove_item(&p_mgrp->mcm_port_tbl, p_map_item); + osm_mcm_port_delete((osm_mcm_port_t *) p_map_item); + + /* track the fact we modified the group */ + p_mgrp->last_change_id++; + } + + /* + no more ports so the group will be deleted after re-route + but only if it is not a well known group and not already deleted + */ + if ((cl_is_qmap_empty(&p_mgrp->mcm_port_tbl)) && + (p_mgrp->well_known == FALSE) && (p_mgrp->to_be_deleted == FALSE)) { + p_mgrp->to_be_deleted = TRUE; + + /* Send a Report to any InformInfo registered for + Trap 67 : MCGroup delete */ + osm_mgrp_send_delete_notice(p_subn, p_log, p_mgrp); + } } /********************************************************************** **********************************************************************/ boolean_t -osm_mgrp_is_port_present( - IN const osm_mgrp_t* const p_mgrp, - IN const ib_net64_t port_guid, - OUT osm_mcm_port_t ** const pp_mcm_port ) +osm_mgrp_is_port_present(IN const osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid, + OUT osm_mcm_port_t ** const pp_mcm_port) { - cl_map_item_t *p_map_item; - - CL_ASSERT(p_mgrp); - - p_map_item = cl_qmap_get(&p_mgrp->mcm_port_tbl, - port_guid); - - if (p_map_item != cl_qmap_end(&p_mgrp->mcm_port_tbl)) - { - if (pp_mcm_port) - *pp_mcm_port = (osm_mcm_port_t *)p_map_item; - return TRUE; - } - if (pp_mcm_port) - *pp_mcm_port = NULL; - return FALSE; + cl_map_item_t *p_map_item; + + CL_ASSERT(p_mgrp); + + p_map_item = cl_qmap_get(&p_mgrp->mcm_port_tbl, port_guid); + + if (p_map_item != cl_qmap_end(&p_mgrp->mcm_port_tbl)) { + if (pp_mcm_port) + *pp_mcm_port = (osm_mcm_port_t *) p_map_item; + return TRUE; + } + if (pp_mcm_port) + *pp_mcm_port = NULL; + return FALSE; } /********************************************************************** **********************************************************************/ static void -__osm_mgrp_apply_func_sub( - const osm_mgrp_t* const p_mgrp, - const osm_mtree_node_t* const p_mtn, - osm_mgrp_func_t p_func, - void* context ) +__osm_mgrp_apply_func_sub(const osm_mgrp_t * const p_mgrp, + const osm_mtree_node_t * const p_mtn, + osm_mgrp_func_t p_func, void *context) { - uint8_t i = 0; - uint8_t max_children; - osm_mtree_node_t* p_child_mtn; - - /* - Call the user, then recurse. - */ - p_func( p_mgrp, p_mtn, context ); - - max_children = osm_mtree_node_get_max_children( p_mtn ); - for( i = 0; i < max_children; i++ ) - { - p_child_mtn = osm_mtree_node_get_child( p_mtn, i ); - if( p_child_mtn ) - __osm_mgrp_apply_func_sub( p_mgrp, p_child_mtn, p_func, context ); - } + uint8_t i = 0; + uint8_t max_children; + osm_mtree_node_t *p_child_mtn; + + /* + Call the user, then recurse. + */ + p_func(p_mgrp, p_mtn, context); + + max_children = osm_mtree_node_get_max_children(p_mtn); + for (i = 0; i < max_children; i++) { + p_child_mtn = osm_mtree_node_get_child(p_mtn, i); + if (p_child_mtn) + __osm_mgrp_apply_func_sub(p_mgrp, p_child_mtn, p_func, + context); + } } /********************************************************************** **********************************************************************/ void -osm_mgrp_apply_func( - const osm_mgrp_t* const p_mgrp, - osm_mgrp_func_t p_func, - void* context ) +osm_mgrp_apply_func(const osm_mgrp_t * const p_mgrp, + osm_mgrp_func_t p_func, void *context) { - osm_mtree_node_t* p_mtn; + osm_mtree_node_t *p_mtn; - CL_ASSERT( p_mgrp ); - CL_ASSERT( p_func ); + CL_ASSERT(p_mgrp); + CL_ASSERT(p_func); - p_mtn = p_mgrp->p_root; + p_mtn = p_mgrp->p_root; - if( p_mtn ) - __osm_mgrp_apply_func_sub( p_mgrp, p_mtn, p_func, context ); + if (p_mtn) + __osm_mgrp_apply_func_sub(p_mgrp, p_mtn, p_func, context); } /********************************************************************** **********************************************************************/ void -osm_mgrp_send_delete_notice( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t *p_mgrp ) +osm_mgrp_send_delete_notice(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN osm_mgrp_t * p_mgrp) { - ib_mad_notice_attr_t notice; - ib_api_status_t status; - - OSM_LOG_ENTER( p_log, osm_mgrp_send_delete_notice ); - - /* prepare the needed info */ - - /* details of the notice */ - notice.generic_type = 0x83; /* is generic subn mgt type */ - ib_notice_set_prod_type_ho(¬ice, 4); /* A Class Manager generator */ - notice.g_or_v.generic.trap_num = CL_HTON16(67); /* delete of mcg */ - /* The sm_base_lid is saved in network order already. */ - notice.issuer_lid = p_subn->sm_base_lid; - /* following o14-12.1.11 and table 120 p726 */ - /* we need to provide the MGID */ - memcpy(&(notice.data_details.ntc_64_67.gid), - &(p_mgrp->mcmember_rec.mgid), - sizeof(ib_gid_t)); - - /* According to page 653 - the issuer gid in this case of trap - is the SM gid, since the SM is the initiator of this trap. */ - notice.issuer_gid.unicast.prefix = p_subn->opt.subnet_prefix; - notice.issuer_gid.unicast.interface_id = p_subn->sm_port_guid; - - status = osm_report_notice(p_log, p_subn, ¬ice); - if( status != IB_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_mgrp_send_delete_notice: ERR 7601: " - "Error sending trap reports (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); + ib_mad_notice_attr_t notice; + ib_api_status_t status; + + OSM_LOG_ENTER(p_log, osm_mgrp_send_delete_notice); + + /* prepare the needed info */ + + /* details of the notice */ + notice.generic_type = 0x83; /* is generic subn mgt type */ + ib_notice_set_prod_type_ho(¬ice, 4); /* A Class Manager generator */ + notice.g_or_v.generic.trap_num = CL_HTON16(67); /* delete of mcg */ + /* The sm_base_lid is saved in network order already. */ + notice.issuer_lid = p_subn->sm_base_lid; + /* following o14-12.1.11 and table 120 p726 */ + /* we need to provide the MGID */ + memcpy(&(notice.data_details.ntc_64_67.gid), + &(p_mgrp->mcmember_rec.mgid), sizeof(ib_gid_t)); + + /* According to page 653 - the issuer gid in this case of trap + is the SM gid, since the SM is the initiator of this trap. */ + notice.issuer_gid.unicast.prefix = p_subn->opt.subnet_prefix; + notice.issuer_gid.unicast.interface_id = p_subn->sm_port_guid; + + status = osm_report_notice(p_log, p_subn, ¬ice); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_mgrp_send_delete_notice: ERR 7601: " + "Error sending trap reports (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); } /********************************************************************** **********************************************************************/ void -osm_mgrp_send_create_notice( - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_mgrp_t *p_mgrp ) +osm_mgrp_send_create_notice(IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN osm_mgrp_t * p_mgrp) { - ib_mad_notice_attr_t notice; - ib_api_status_t status; - - OSM_LOG_ENTER( p_log, osm_mgrp_send_create_notice ); - - /* prepare the needed info */ - - /* details of the notice */ - notice.generic_type = 0x83; /* Generic SubnMgt type */ - ib_notice_set_prod_type_ho(¬ice, 4); /* A Class Manager generator */ - notice.g_or_v.generic.trap_num = CL_HTON16(66); /* create of mcg */ - /* The sm_base_lid is saved in network order already. */ - notice.issuer_lid = p_subn->sm_base_lid; - /* following o14-12.1.11 and table 120 p726 */ - /* we need to provide the MGID */ - memcpy(&(notice.data_details.ntc_64_67.gid), - &(p_mgrp->mcmember_rec.mgid), - sizeof(ib_gid_t)); - - /* According to page 653 - the issuer gid in this case of trap - is the SM gid, since the SM is the initiator of this trap. */ - notice.issuer_gid.unicast.prefix = p_subn->opt.subnet_prefix; - notice.issuer_gid.unicast.interface_id = p_subn->sm_port_guid; - - status = osm_report_notice(p_log, p_subn, ¬ice); - if( status != IB_SUCCESS ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_mgrp_send_create_notice: ERR 7602: " - "Error sending trap reports (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); + ib_mad_notice_attr_t notice; + ib_api_status_t status; + + OSM_LOG_ENTER(p_log, osm_mgrp_send_create_notice); + + /* prepare the needed info */ + + /* details of the notice */ + notice.generic_type = 0x83; /* Generic SubnMgt type */ + ib_notice_set_prod_type_ho(¬ice, 4); /* A Class Manager generator */ + notice.g_or_v.generic.trap_num = CL_HTON16(66); /* create of mcg */ + /* The sm_base_lid is saved in network order already. */ + notice.issuer_lid = p_subn->sm_base_lid; + /* following o14-12.1.11 and table 120 p726 */ + /* we need to provide the MGID */ + memcpy(&(notice.data_details.ntc_64_67.gid), + &(p_mgrp->mcmember_rec.mgid), sizeof(ib_gid_t)); + + /* According to page 653 - the issuer gid in this case of trap + is the SM gid, since the SM is the initiator of this trap. */ + notice.issuer_gid.unicast.prefix = p_subn->opt.subnet_prefix; + notice.issuer_gid.unicast.interface_id = p_subn->sm_port_guid; + + status = osm_report_notice(p_log, p_subn, ¬ice); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_mgrp_send_create_notice: ERR 7602: " + "Error sending trap reports (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); } diff --git a/opensm/opensm/osm_node.c b/opensm/opensm/osm_node.c index 04b1045..645daa9 100644 --- a/opensm/opensm/osm_node.c +++ b/opensm/opensm/osm_node.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -57,264 +57,246 @@ /********************************************************************** **********************************************************************/ void -osm_node_init_physp( - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +osm_node_init_physp(IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_net64_t port_guid; - ib_smp_t *p_smp; - ib_node_info_t *p_ni; - uint8_t port_num; + ib_net64_t port_guid; + ib_smp_t *p_smp; + ib_node_info_t *p_ni; + uint8_t port_num; - CL_ASSERT( p_node ); - CL_ASSERT( p_madw ); + CL_ASSERT(p_node); + CL_ASSERT(p_madw); - p_smp = osm_madw_get_smp_ptr( p_madw ); + p_smp = osm_madw_get_smp_ptr(p_madw); - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_NODE_INFO ); + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_NODE_INFO); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - port_guid = p_ni->port_guid; - port_num = ib_node_info_get_local_port_num( p_ni ); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + port_guid = p_ni->port_guid; + port_num = ib_node_info_get_local_port_num(p_ni); - CL_ASSERT( port_num < p_node->physp_tbl_size ); + CL_ASSERT(port_num < p_node->physp_tbl_size); - osm_physp_init( &p_node->physp_table[port_num], - port_guid, port_num, p_node, - osm_madw_get_bind_handle( p_madw ), - p_smp->hop_count, p_smp->initial_path ); + osm_physp_init(&p_node->physp_table[port_num], + port_guid, port_num, p_node, + osm_madw_get_bind_handle(p_madw), + p_smp->hop_count, p_smp->initial_path); } /********************************************************************** **********************************************************************/ -osm_node_t* -osm_node_new( - IN const osm_madw_t* const p_madw ) +osm_node_t *osm_node_new(IN const osm_madw_t * const p_madw) { - osm_node_t *p_node; - ib_smp_t *p_smp; - ib_node_info_t *p_ni; - uint8_t i; - uint32_t size; - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_NODE_INFO ); - - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - - /* - The node object already contains one physical port object. - Therefore, subtract 1 from the number of physical ports - used by the switch. This is not done for CA's since they - need to occupy 1 more physp than they physically have since - we still reserve room for a "port 0". - */ - size = p_ni->num_ports; - - p_node = malloc( sizeof(*p_node) + sizeof(osm_physp_t) * size ); - if( p_node != NULL ) - { - memset( p_node, 0, sizeof(*p_node) + sizeof(osm_physp_t) * size ); - p_node->node_info = *p_ni; - p_node->physp_tbl_size = size + 1; - - /* - Construct Physical Port objects owned by this Node. - Then, initialize the Physical Port through with we - discovered this port. - For switches, all ports have the same GUID. - For CAs and routers, each port has a different GUID, so we only - know the GUID for the port that responded to our - Get(NodeInfo). - */ - for( i = 0; i < p_node->physp_tbl_size; i++ ) - osm_physp_construct( &p_node->physp_table[i] ); - - osm_node_init_physp( p_node, p_madw ); - } - - return( p_node ); + osm_node_t *p_node; + ib_smp_t *p_smp; + ib_node_info_t *p_ni; + uint8_t i; + uint32_t size; + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_NODE_INFO); + + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + + /* + The node object already contains one physical port object. + Therefore, subtract 1 from the number of physical ports + used by the switch. This is not done for CA's since they + need to occupy 1 more physp than they physically have since + we still reserve room for a "port 0". + */ + size = p_ni->num_ports; + + p_node = malloc(sizeof(*p_node) + sizeof(osm_physp_t) * size); + if (p_node != NULL) { + memset(p_node, 0, sizeof(*p_node) + sizeof(osm_physp_t) * size); + p_node->node_info = *p_ni; + p_node->physp_tbl_size = size + 1; + + /* + Construct Physical Port objects owned by this Node. + Then, initialize the Physical Port through with we + discovered this port. + For switches, all ports have the same GUID. + For CAs and routers, each port has a different GUID, so we only + know the GUID for the port that responded to our + Get(NodeInfo). + */ + for (i = 0; i < p_node->physp_tbl_size; i++) + osm_physp_construct(&p_node->physp_table[i]); + + osm_node_init_physp(p_node, p_madw); + } + + return (p_node); } /********************************************************************** **********************************************************************/ -static void -osm_node_destroy( - IN osm_node_t *p_node ) +static void osm_node_destroy(IN osm_node_t * p_node) { - uint16_t i; + uint16_t i; - /* - Cleanup all physports - */ - for( i = 0; i < p_node->physp_tbl_size; i++ ) - osm_physp_destroy( &p_node->physp_table[i] ); + /* + Cleanup all physports + */ + for (i = 0; i < p_node->physp_tbl_size; i++) + osm_physp_destroy(&p_node->physp_table[i]); } /********************************************************************** **********************************************************************/ -void -osm_node_delete( - IN OUT osm_node_t** const p_node ) +void osm_node_delete(IN OUT osm_node_t ** const p_node) { - CL_ASSERT(p_node && *p_node); - osm_node_destroy( *p_node ); - free( *p_node ); - *p_node = NULL; + CL_ASSERT(p_node && *p_node); + osm_node_destroy(*p_node); + free(*p_node); + *p_node = NULL; } /********************************************************************** **********************************************************************/ void -osm_node_link( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ) +osm_node_link(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - CL_ASSERT( remote_port_num < p_remote_node->physp_tbl_size ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + CL_ASSERT(remote_port_num < p_remote_node->physp_tbl_size); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - p_remote_physp = osm_node_get_physp_ptr( p_remote_node, remote_port_num ); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + p_remote_physp = osm_node_get_physp_ptr(p_remote_node, remote_port_num); - if (p_physp->p_remote_physp) - p_physp->p_remote_physp->p_remote_physp = NULL; - if (p_remote_physp->p_remote_physp) - p_remote_physp->p_remote_physp->p_remote_physp = NULL; + if (p_physp->p_remote_physp) + p_physp->p_remote_physp->p_remote_physp = NULL; + if (p_remote_physp->p_remote_physp) + p_remote_physp->p_remote_physp->p_remote_physp = NULL; - osm_physp_link( p_physp, p_remote_physp ); + osm_physp_link(p_physp, p_remote_physp); } /********************************************************************** **********************************************************************/ void -osm_node_unlink( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ) +osm_node_unlink(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - CL_ASSERT( remote_port_num < p_remote_node->physp_tbl_size ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + CL_ASSERT(remote_port_num < p_remote_node->physp_tbl_size); - if( osm_node_link_exists( p_node, port_num, - p_remote_node, remote_port_num ) ) - { + if (osm_node_link_exists(p_node, port_num, + p_remote_node, remote_port_num)) { - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - p_remote_physp = osm_node_get_physp_ptr( p_remote_node, remote_port_num ); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + p_remote_physp = + osm_node_get_physp_ptr(p_remote_node, remote_port_num); - osm_physp_unlink( p_physp, p_remote_physp ); - } + osm_physp_unlink(p_physp, p_remote_physp); + } } /********************************************************************** **********************************************************************/ boolean_t -osm_node_link_exists( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ) +osm_node_link_exists(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - CL_ASSERT( remote_port_num < p_remote_node->physp_tbl_size ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + CL_ASSERT(remote_port_num < p_remote_node->physp_tbl_size); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - p_remote_physp = osm_node_get_physp_ptr( p_remote_node, remote_port_num ); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + p_remote_physp = osm_node_get_physp_ptr(p_remote_node, remote_port_num); - return( osm_physp_link_exists( p_physp, p_remote_physp ) ); + return (osm_physp_link_exists(p_physp, p_remote_physp)); } /********************************************************************** **********************************************************************/ boolean_t -osm_node_link_has_valid_ports( - IN osm_node_t* const p_node, - IN const uint8_t port_num, - IN osm_node_t* const p_remote_node, - IN const uint8_t remote_port_num ) +osm_node_link_has_valid_ports(IN osm_node_t * const p_node, + IN const uint8_t port_num, + IN osm_node_t * const p_remote_node, + IN const uint8_t remote_port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - CL_ASSERT( remote_port_num < p_remote_node->physp_tbl_size ); + CL_ASSERT(port_num < p_node->physp_tbl_size); + CL_ASSERT(remote_port_num < p_remote_node->physp_tbl_size); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - p_remote_physp = osm_node_get_physp_ptr( p_remote_node, remote_port_num ); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + p_remote_physp = osm_node_get_physp_ptr(p_remote_node, remote_port_num); - return( osm_physp_is_valid( p_physp ) && - osm_physp_is_valid( p_remote_physp ) ); + return (osm_physp_is_valid(p_physp) && + osm_physp_is_valid(p_remote_physp)); } /********************************************************************** **********************************************************************/ boolean_t -osm_node_has_any_link( - IN osm_node_t* const p_node, - IN const uint8_t port_num ) +osm_node_has_any_link(IN osm_node_t * const p_node, IN const uint8_t port_num) { - osm_physp_t *p_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - return( osm_physp_has_any_link( p_physp ) ); + osm_physp_t *p_physp; + CL_ASSERT(port_num < p_node->physp_tbl_size); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + return (osm_physp_has_any_link(p_physp)); } /********************************************************************** **********************************************************************/ -osm_node_t* -osm_node_get_remote_node( - IN const osm_node_t* const p_node, - IN const uint8_t port_num, - OUT uint8_t *p_remote_port_num ) +osm_node_t *osm_node_get_remote_node(IN const osm_node_t * const p_node, + IN const uint8_t port_num, + OUT uint8_t * p_remote_port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; - p_physp = osm_node_get_physp_ptr( p_node, port_num ); + p_physp = osm_node_get_physp_ptr(p_node, port_num); - if( !osm_physp_has_any_link( p_physp ) ) - return( NULL ); + if (!osm_physp_has_any_link(p_physp)) + return (NULL); - p_remote_physp = osm_physp_get_remote( p_physp ); - if( p_remote_port_num ) - *p_remote_port_num = osm_physp_get_port_num( p_remote_physp ); + p_remote_physp = osm_physp_get_remote(p_physp); + if (p_remote_port_num) + *p_remote_port_num = osm_physp_get_port_num(p_remote_physp); - return( osm_physp_get_node_ptr( p_remote_physp ) ); + return (osm_physp_get_node_ptr(p_remote_physp)); } /********************************************************************** The lock must be held before calling this function. **********************************************************************/ ib_net16_t -osm_node_get_remote_base_lid( - IN const osm_node_t* const p_node, - IN const uint32_t port_num ) +osm_node_get_remote_base_lid(IN const osm_node_t * const p_node, + IN const uint32_t port_num) { - osm_physp_t *p_physp; - osm_physp_t *p_remote_physp; - CL_ASSERT( port_num < p_node->physp_tbl_size ); - - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - if( osm_physp_is_valid( p_physp ) ) - { - p_remote_physp = osm_physp_get_remote( p_physp ); - return( osm_physp_get_base_lid( p_remote_physp ) ); - } - - return( 0 ); + osm_physp_t *p_physp; + osm_physp_t *p_remote_physp; + CL_ASSERT(port_num < p_node->physp_tbl_size); + + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (osm_physp_is_valid(p_physp)) { + p_remote_physp = osm_physp_get_remote(p_physp); + return (osm_physp_get_base_lid(p_remote_physp)); + } + + return (0); } diff --git a/opensm/opensm/osm_node_desc_rcv.c b/opensm/opensm/osm_node_desc_rcv.c index 9104bea..d50883c 100644 --- a/opensm/opensm/osm_node_desc_rcv.c +++ b/opensm/opensm/osm_node_desc_rcv.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_nd_rcv_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -64,118 +63,104 @@ /********************************************************************** **********************************************************************/ void -__osm_nd_rcv_process_nd( - IN const osm_nd_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const ib_node_desc_t* const p_nd ) +__osm_nd_rcv_process_nd(IN const osm_nd_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const ib_node_desc_t * const p_nd) { - OSM_LOG_ENTER( p_rcv->p_log, __osm_nd_rcv_process_nd ); + OSM_LOG_ENTER(p_rcv->p_log, __osm_nd_rcv_process_nd); - memcpy( &p_node->node_desc.description, p_nd, sizeof(*p_nd) ); + memcpy(&p_node->node_desc.description, p_nd, sizeof(*p_nd)); - /* also set up a printable version */ - memcpy( &p_node->print_desc, p_nd, sizeof(*p_nd) ); - p_node->print_desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; + /* also set up a printable version */ + memcpy(&p_node->print_desc, p_nd, sizeof(*p_nd)); + p_node->print_desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_nd_rcv_process_nd: " - "Node 0x%" PRIx64 "\n\t\t\t\tDescription = %s\n", - cl_ntoh64( osm_node_get_node_guid( p_node )), - p_node->print_desc); - } + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_nd_rcv_process_nd: " + "Node 0x%" PRIx64 "\n\t\t\t\tDescription = %s\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + p_node->print_desc); + } - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_nd_rcv_construct( - IN osm_nd_rcv_t* const p_rcv ) +void osm_nd_rcv_construct(IN osm_nd_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_nd_rcv_destroy( - IN osm_nd_rcv_t* const p_rcv ) +void osm_nd_rcv_destroy(IN osm_nd_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_nd_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_nd_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_nd_rcv_init( - IN osm_nd_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_nd_rcv_init(IN osm_nd_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_nd_rcv_init ); + OSM_LOG_ENTER(p_log, osm_nd_rcv_init); - osm_nd_rcv_construct( p_rcv ); + osm_nd_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_nd_rcv_process( - IN void *context, - IN void *data ) +void osm_nd_rcv_process(IN void *context, IN void *data) { - osm_nd_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_node_desc_t *p_nd; - ib_smp_t *p_smp; - osm_node_t *p_node; - ib_net64_t node_guid; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_nd_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_nd = (ib_node_desc_t*)ib_smp_get_payload_ptr( p_smp ); - - /* - Acquire the node object and add the node description. - */ - - node_guid = osm_madw_get_nd_context_ptr( p_madw )->node_guid; - CL_PLOCK_EXCL_ACQUIRE( p_rcv->p_lock ); - p_node = osm_get_node_by_guid( p_rcv->p_subn, node_guid ); - if( !p_node ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_nd_rcv_process: ERR 0B01: " - "NodeDescription received for nonexistent node " - "0x%" PRIx64 "\n", cl_ntoh64(node_guid) ); - } - else - { - __osm_nd_rcv_process_nd( p_rcv, p_node, p_nd ); - } - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - OSM_LOG_EXIT( p_rcv->p_log ); + osm_nd_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_node_desc_t *p_nd; + ib_smp_t *p_smp; + osm_node_t *p_node; + ib_net64_t node_guid; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_nd_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_nd = (ib_node_desc_t *) ib_smp_get_payload_ptr(p_smp); + + /* + Acquire the node object and add the node description. + */ + + node_guid = osm_madw_get_nd_context_ptr(p_madw)->node_guid; + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + p_node = osm_get_node_by_guid(p_rcv->p_subn, node_guid); + if (!p_node) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nd_rcv_process: ERR 0B01: " + "NodeDescription received for nonexistent node " + "0x%" PRIx64 "\n", cl_ntoh64(node_guid)); + } else { + __osm_nd_rcv_process_nd(p_rcv, p_node, p_nd); + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 72245ce..33a41fe 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -67,942 +67,891 @@ #include static void -report_duplicated_guid( - IN const osm_ni_rcv_t* const p_rcv, - osm_physp_t *p_physp, - osm_node_t* p_neighbor_node, - const uint8_t port_num) +report_duplicated_guid(IN const osm_ni_rcv_t * const p_rcv, + osm_physp_t * p_physp, + osm_node_t * p_neighbor_node, const uint8_t port_num) { - osm_physp_t *p_old, *p_new; - osm_dr_path_t path; - - p_old = p_physp->p_remote_physp; - p_new = osm_node_get_physp_ptr(p_neighbor_node, port_num); - - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "report_duplicated_guid: ERR 0D01: " - "Found duplicated node.\n" - "Node 0x%" PRIx64 " port %u is reachable from remote node " - "0x%" PRIx64 " port %u and remote node 0x%" PRIx64 " port %u.\n" - "Paths are:\n", - cl_ntoh64(p_physp->p_node->node_info.node_guid), p_physp->port_num, - cl_ntoh64(p_old->p_node->node_info.node_guid), p_old->port_num, - cl_ntoh64(p_new->p_node->node_info.node_guid), p_new->port_num); - - osm_dump_dr_path(p_rcv->p_log, osm_physp_get_dr_path_ptr(p_physp), OSM_LOG_ERROR); - - path = *osm_physp_get_dr_path_ptr(p_new); - osm_dr_path_extend(&path, port_num); - osm_dump_dr_path(p_rcv->p_log, &path, OSM_LOG_ERROR); - - osm_log( p_rcv->p_log, OSM_LOG_SYS, - "FATAL: duplicated guids or 12x lane reversal\n"); + osm_physp_t *p_old, *p_new; + osm_dr_path_t path; + + p_old = p_physp->p_remote_physp; + p_new = osm_node_get_physp_ptr(p_neighbor_node, port_num); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "report_duplicated_guid: ERR 0D01: " + "Found duplicated node.\n" + "Node 0x%" PRIx64 " port %u is reachable from remote node " + "0x%" PRIx64 " port %u and remote node 0x%" PRIx64 " port %u.\n" + "Paths are:\n", + cl_ntoh64(p_physp->p_node->node_info.node_guid), + p_physp->port_num, + cl_ntoh64(p_old->p_node->node_info.node_guid), p_old->port_num, + cl_ntoh64(p_new->p_node->node_info.node_guid), p_new->port_num); + + osm_dump_dr_path(p_rcv->p_log, osm_physp_get_dr_path_ptr(p_physp), + OSM_LOG_ERROR); + + path = *osm_physp_get_dr_path_ptr(p_new); + osm_dr_path_extend(&path, port_num); + osm_dump_dr_path(p_rcv->p_log, &path, OSM_LOG_ERROR); + + osm_log(p_rcv->p_log, OSM_LOG_SYS, + "FATAL: duplicated guids or 12x lane reversal\n"); } -static void requery_dup_node_info( - IN const osm_ni_rcv_t* const p_rcv, - osm_physp_t *p_physp, unsigned count) +static void requery_dup_node_info(IN const osm_ni_rcv_t * const p_rcv, + osm_physp_t * p_physp, unsigned count) { - osm_madw_context_t context; - osm_dr_path_t path; - cl_status_t status; - - path = *osm_physp_get_dr_path_ptr(p_physp->p_remote_physp); - osm_dr_path_extend(&path, p_physp->p_remote_physp->port_num); - - context.ni_context.node_guid = p_physp->p_remote_physp->p_node->node_info.port_guid; - context.ni_context.port_num = p_physp->p_remote_physp->port_num; - context.ni_context.dup_node_guid = p_physp->p_node->node_info.node_guid; - context.ni_context.dup_port_num = p_physp->port_num; - context.ni_context.dup_count = count; - - status = osm_req_get(p_rcv->p_gen_req, - &path, - IB_MAD_ATTR_NODE_INFO, - 0, - CL_DISP_MSGID_NONE, - &context); - - if(status != IB_SUCCESS) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "requery_dup_node_info: ERR 0D02: " - "Failure initiating NodeInfo request (%s)\n", - ib_get_err_str(status) ); + osm_madw_context_t context; + osm_dr_path_t path; + cl_status_t status; + + path = *osm_physp_get_dr_path_ptr(p_physp->p_remote_physp); + osm_dr_path_extend(&path, p_physp->p_remote_physp->port_num); + + context.ni_context.node_guid = + p_physp->p_remote_physp->p_node->node_info.port_guid; + context.ni_context.port_num = p_physp->p_remote_physp->port_num; + context.ni_context.dup_node_guid = p_physp->p_node->node_info.node_guid; + context.ni_context.dup_port_num = p_physp->port_num; + context.ni_context.dup_count = count; + + status = osm_req_get(p_rcv->p_gen_req, + &path, + IB_MAD_ATTR_NODE_INFO, + 0, CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "requery_dup_node_info: ERR 0D02: " + "Failure initiating NodeInfo request (%s)\n", + ib_get_err_str(status)); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_set_links( - IN const osm_ni_rcv_t* const p_rcv, - osm_node_t* p_node, - const uint8_t port_num, - const osm_ni_context_t* const p_ni_context ) +__osm_ni_rcv_set_links(IN const osm_ni_rcv_t * const p_rcv, + osm_node_t * p_node, + const uint8_t port_num, + const osm_ni_context_t * const p_ni_context) { - osm_node_t *p_neighbor_node; - osm_physp_t *p_physp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_set_links ); - - /* - A special case exists in which the node we're trying to - link is our own node. In this case, the guid value in - the ni_context will be zero. - */ - if( p_ni_context->node_guid == 0 ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_set_links: " - "Nothing to link for our own node 0x%" PRIx64 "\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - goto _exit; - } - - p_neighbor_node = osm_get_node_by_guid( p_rcv->p_subn, - p_ni_context->node_guid ); - if( !p_neighbor_node ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_set_links: ERR 0D10: " - "Unexpected removal of neighbor node " - "0x%" PRIx64 "\n", - cl_ntoh64( p_ni_context->node_guid ) ); - goto _exit; - } - - /* - We have seen this neighbor node before, but we might - not have seen this port on the neighbor node before. - We should not set links to an uninitialized port on the - neighbor, so check validity up front. If it's not - valid, do nothing, since we'll see this link again - when we probe the neighbor. - */ - if( !osm_node_link_has_valid_ports( p_node, port_num, - p_neighbor_node, p_ni_context->port_num ) ) - goto _exit; - - if( osm_node_link_exists( p_node, port_num, - p_neighbor_node, p_ni_context->port_num ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_set_links: " - "Link already exists\n" ); - goto _exit; - } - - if (osm_node_has_any_link(p_node, port_num) && - p_rcv->p_subn->force_immediate_heavy_sweep == FALSE && - (!p_ni_context->dup_count || - (p_ni_context->dup_node_guid == osm_node_get_node_guid(p_node) && - p_ni_context->dup_port_num == port_num))) - { - /* - Uh oh... - This could be reconnected ports, but also duplicated GUID - (2 nodes have the same guid) or a 12x link with lane reversal - that is not configured correctly. - We will try to recover by querying NodeInfo again. - In order to catch even fast port moving to new location(s) and - back we will count up to 5. - Some crazy reconnections (newly created switch loop right before - targeted CA) will not be catched this way. So in worst case - - report GUID duplication and request new discovery. - When switch node is targeted NodeInfo querying will be done in - opposite order, this is much stronger check, unfortunately it is - impossible with CAs. - */ - p_physp = osm_node_get_physp_ptr(p_node, port_num); - if (p_ni_context->dup_count > 5) - { - report_duplicated_guid(p_rcv, p_physp, - p_neighbor_node, p_ni_context->port_num); - p_rcv->p_subn->force_immediate_heavy_sweep = TRUE; - } - else if (p_node->sw) - requery_dup_node_info(p_rcv, p_physp->p_remote_physp, - p_ni_context->dup_count + 1); - else - requery_dup_node_info(p_rcv, p_physp, - p_ni_context->dup_count + 1); - } - - /* - When there are only two nodes with exact same guids (connected back - to back) - the previous check for duplicated guid will not catch - them. But the link will be from the port to itself... - Enhanced Port 0 is an exception to this - */ - if ((osm_node_get_node_guid(p_node) == p_ni_context->node_guid) && - (port_num == p_ni_context->port_num) && - port_num != 0 && cl_qmap_count(&p_rcv->p_subn->sw_guid_tbl) == 0) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, "__osm_ni_rcv_set_links: " - "Duplicate GUID found by link from a port to itself:" - "node 0x%" PRIx64 ", port number 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - port_num ); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - osm_dump_dr_path(p_rcv->p_log, - osm_physp_get_dr_path_ptr(p_physp), - OSM_LOG_VERBOSE); - - if ( p_rcv->p_subn->opt.exit_on_fatal == TRUE ) - { - osm_log( p_rcv->p_log, OSM_LOG_SYS, - "Errors on subnet. Duplicate GUID found " - "by link from a port to itself. " - "See verbose opensm.log for more details\n"); - exit( 1 ); - } - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_set_links: " - "Creating new link between: " - "\n\t\t\t\tnode 0x%" PRIx64 ", " - "port number 0x%X and" - "\n\t\t\t\tnode 0x%" PRIx64 ", " - "port number 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - port_num, - cl_ntoh64( p_ni_context->node_guid ), - p_ni_context->port_num ); - - osm_node_link( p_node, port_num, p_neighbor_node, - p_ni_context->port_num ); - - _exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_node_t *p_neighbor_node; + osm_physp_t *p_physp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_set_links); + + /* + A special case exists in which the node we're trying to + link is our own node. In this case, the guid value in + the ni_context will be zero. + */ + if (p_ni_context->node_guid == 0) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_ni_rcv_set_links: " + "Nothing to link for our own node 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + goto _exit; + } + + p_neighbor_node = osm_get_node_by_guid(p_rcv->p_subn, + p_ni_context->node_guid); + if (!p_neighbor_node) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_set_links: ERR 0D10: " + "Unexpected removal of neighbor node " + "0x%" PRIx64 "\n", cl_ntoh64(p_ni_context->node_guid)); + goto _exit; + } + + /* + We have seen this neighbor node before, but we might + not have seen this port on the neighbor node before. + We should not set links to an uninitialized port on the + neighbor, so check validity up front. If it's not + valid, do nothing, since we'll see this link again + when we probe the neighbor. + */ + if (!osm_node_link_has_valid_ports(p_node, port_num, + p_neighbor_node, + p_ni_context->port_num)) + goto _exit; + + if (osm_node_link_exists(p_node, port_num, + p_neighbor_node, p_ni_context->port_num)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_ni_rcv_set_links: " "Link already exists\n"); + goto _exit; + } + + if (osm_node_has_any_link(p_node, port_num) && + p_rcv->p_subn->force_immediate_heavy_sweep == FALSE && + (!p_ni_context->dup_count || + (p_ni_context->dup_node_guid == osm_node_get_node_guid(p_node) && + p_ni_context->dup_port_num == port_num))) { + /* + Uh oh... + This could be reconnected ports, but also duplicated GUID + (2 nodes have the same guid) or a 12x link with lane reversal + that is not configured correctly. + We will try to recover by querying NodeInfo again. + In order to catch even fast port moving to new location(s) and + back we will count up to 5. + Some crazy reconnections (newly created switch loop right before + targeted CA) will not be catched this way. So in worst case - + report GUID duplication and request new discovery. + When switch node is targeted NodeInfo querying will be done in + opposite order, this is much stronger check, unfortunately it is + impossible with CAs. + */ + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (p_ni_context->dup_count > 5) { + report_duplicated_guid(p_rcv, p_physp, + p_neighbor_node, + p_ni_context->port_num); + p_rcv->p_subn->force_immediate_heavy_sweep = TRUE; + } else if (p_node->sw) + requery_dup_node_info(p_rcv, p_physp->p_remote_physp, + p_ni_context->dup_count + 1); + else + requery_dup_node_info(p_rcv, p_physp, + p_ni_context->dup_count + 1); + } + + /* + When there are only two nodes with exact same guids (connected back + to back) - the previous check for duplicated guid will not catch + them. But the link will be from the port to itself... + Enhanced Port 0 is an exception to this + */ + if ((osm_node_get_node_guid(p_node) == p_ni_context->node_guid) && + (port_num == p_ni_context->port_num) && + port_num != 0 && cl_qmap_count(&p_rcv->p_subn->sw_guid_tbl) == 0) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_ni_rcv_set_links: " + "Duplicate GUID found by link from a port to itself:" + "node 0x%" PRIx64 ", port number 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), port_num); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + osm_dump_dr_path(p_rcv->p_log, + osm_physp_get_dr_path_ptr(p_physp), + OSM_LOG_VERBOSE); + + if (p_rcv->p_subn->opt.exit_on_fatal == TRUE) { + osm_log(p_rcv->p_log, OSM_LOG_SYS, + "Errors on subnet. Duplicate GUID found " + "by link from a port to itself. " + "See verbose opensm.log for more details\n"); + exit(1); + } + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_ni_rcv_set_links: " + "Creating new link between: " + "\n\t\t\t\tnode 0x%" PRIx64 ", " + "port number 0x%X and" + "\n\t\t\t\tnode 0x%" PRIx64 ", " + "port number 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + port_num, + cl_ntoh64(p_ni_context->node_guid), + p_ni_context->port_num); + + osm_node_link(p_node, port_num, p_neighbor_node, + p_ni_context->port_num); + + _exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_new_node( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_new_node(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - osm_physp_t *p_physp; - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - uint8_t port_num; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_new_node ); - - CL_ASSERT( p_node ); - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - port_num = ib_node_info_get_local_port_num( p_ni ); - - /* - Request PortInfo & NodeDescription attributes for the port - that responded to the NodeInfo attribute. - Because this is a channel adapter or router, we are - not allowed to request PortInfo for the other ports. - Set the context union properly, so the recipient - knows which node & port are relevant. - */ - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - CL_ASSERT( osm_madw_get_bind_handle( p_madw ) == - osm_dr_path_get_bind_handle( - osm_physp_get_dr_path_ptr( p_physp ) ) ); - - context.pi_context.node_guid = p_ni->node_guid; - context.pi_context.port_guid = p_ni->port_guid; - context.pi_context.set_method = FALSE; - context.pi_context.update_master_sm_base_lid = FALSE; - context.pi_context.ignore_errors = FALSE; - context.pi_context.light_sweep = FALSE; - context.pi_context.active_transition = FALSE; - - status = osm_req_get( p_rcv->p_gen_req, - osm_physp_get_dr_path_ptr( p_physp ), - IB_MAD_ATTR_PORT_INFO, - cl_hton32( port_num ), - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new_node: ERR 0D02: " - "Failure initiating PortInfo request (%s)\n", - ib_get_err_str(status)); - - OSM_LOG_EXIT( p_rcv->p_log ); + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + osm_physp_t *p_physp; + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + uint8_t port_num; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_new_node); + + CL_ASSERT(p_node); + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + port_num = ib_node_info_get_local_port_num(p_ni); + + /* + Request PortInfo & NodeDescription attributes for the port + that responded to the NodeInfo attribute. + Because this is a channel adapter or router, we are + not allowed to request PortInfo for the other ports. + Set the context union properly, so the recipient + knows which node & port are relevant. + */ + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + CL_ASSERT(osm_madw_get_bind_handle(p_madw) == + osm_dr_path_get_bind_handle(osm_physp_get_dr_path_ptr + (p_physp))); + + context.pi_context.node_guid = p_ni->node_guid; + context.pi_context.port_guid = p_ni->port_guid; + context.pi_context.set_method = FALSE; + context.pi_context.update_master_sm_base_lid = FALSE; + context.pi_context.ignore_errors = FALSE; + context.pi_context.light_sweep = FALSE; + context.pi_context.active_transition = FALSE; + + status = osm_req_get(p_rcv->p_gen_req, + osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_PORT_INFO, + cl_hton32(port_num), CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new_node: ERR 0D02: " + "Failure initiating PortInfo request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_get_node_desc( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_get_node_desc(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - osm_physp_t *p_physp; - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - uint8_t port_num; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_get_node_desc ); - - CL_ASSERT( p_node ); - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - port_num = ib_node_info_get_local_port_num( p_ni ); - - /* - Request PortInfo & NodeDescription attributes for the port - that responded to the NodeInfo attribute. - Because this is a channel adapter or router, we are - not allowed to request PortInfo for the other ports. - Set the context union properly, so the recipient - knows which node & port are relevant. - */ - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - CL_ASSERT( osm_madw_get_bind_handle( p_madw ) == - osm_dr_path_get_bind_handle( - osm_physp_get_dr_path_ptr( p_physp ) ) ); - - context.nd_context.node_guid = osm_node_get_node_guid( p_node ); - - status = osm_req_get( p_rcv->p_gen_req, - osm_physp_get_dr_path_ptr( p_physp ), - IB_MAD_ATTR_NODE_DESC, - 0, - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_get_node_desc: ERR 0D03: " - "Failure initiating NodeDescription request (%s)\n", - ib_get_err_str(status)); - - OSM_LOG_EXIT( p_rcv->p_log ); + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + osm_physp_t *p_physp; + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + uint8_t port_num; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_get_node_desc); + + CL_ASSERT(p_node); + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + port_num = ib_node_info_get_local_port_num(p_ni); + + /* + Request PortInfo & NodeDescription attributes for the port + that responded to the NodeInfo attribute. + Because this is a channel adapter or router, we are + not allowed to request PortInfo for the other ports. + Set the context union properly, so the recipient + knows which node & port are relevant. + */ + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + CL_ASSERT(osm_madw_get_bind_handle(p_madw) == + osm_dr_path_get_bind_handle(osm_physp_get_dr_path_ptr + (p_physp))); + + context.nd_context.node_guid = osm_node_get_node_guid(p_node); + + status = osm_req_get(p_rcv->p_gen_req, + osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_NODE_DESC, + 0, CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_get_node_desc: ERR 0D03: " + "Failure initiating NodeDescription request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_new_ca_or_router( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_new_ca_or_router(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_new_ca_or_router ); + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_new_ca_or_router); - __osm_ni_rcv_process_new_node( p_rcv, p_node, p_madw ); + __osm_ni_rcv_process_new_node(p_rcv, p_node, p_madw); - /* - A node guid of 0 is the corner case that indicates - we discovered our own node. Initialize the subnet - object with the SM's own port guid. - */ - if( osm_madw_get_ni_context_ptr( p_madw )->node_guid == 0 ) - p_rcv->p_subn->sm_port_guid = p_node->node_info.port_guid; + /* + A node guid of 0 is the corner case that indicates + we discovered our own node. Initialize the subnet + object with the SM's own port guid. + */ + if (osm_madw_get_ni_context_ptr(p_madw)->node_guid == 0) + p_rcv->p_subn->sm_port_guid = p_node->node_info.port_guid; - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_existing_ca_or_router( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_existing_ca_or_router(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - osm_port_t *p_port; - osm_port_t *p_port_check; - osm_madw_context_t context; - uint8_t port_num; - osm_physp_t *p_physp; - ib_api_status_t status; - osm_dr_path_t *p_dr_path; - osm_bind_handle_t h_bind; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_existing_ca_or_router ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - port_num = ib_node_info_get_local_port_num( p_ni ); - h_bind = osm_madw_get_bind_handle( p_madw ); - - /* - Determine if we have encountered this node through a - previously undiscovered port. If so, build the new - port object. - */ - p_port = osm_get_port_by_guid( p_rcv->p_subn, p_ni->port_guid ); - if( !p_port ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_ni_rcv_process_existing_ca_or_router: " - "Creating new port object with GUID 0x%" PRIx64 "\n", - cl_ntoh64( p_ni->port_guid ) ); - - osm_node_init_physp( p_node, p_madw ); - - p_port = osm_port_new( p_ni, p_node ); - if( p_port == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D04: " - "Unable to create new port object\n" ); - goto Exit; - } - - /* - Add the new port object to the database. - */ - p_port_check = (osm_port_t*)cl_qmap_insert( &p_rcv->p_subn->port_guid_tbl, - p_ni->port_guid, &p_port->map_item ); - if( p_port_check != p_port ) - { - /* - We should never be here! - Somehow, this port GUID already exists in the table. - */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D12: " - "Port 0x%" PRIx64 " already in the database!\n", - cl_ntoh64( p_ni->port_guid ) ); - - osm_port_delete( &p_port ); - goto Exit; - } - - /* If we are a master, then this means the port is new on the subnet. - Mark it as new - need to send trap 64 on these ports. - The condition that we are master is true, since if we are in discovering - state (meaning we woke up from standby or we are just initializing), - then these ports may be new to us, but are not new on the subnet. - If we are master, then the subnet as we know it is the updated one, - and any new ports we encounter should cause trap 64. C14-72.1.1 */ - if ( p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER ) - p_port->is_new = 1; - - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - } - else - { - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - - if ( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D19: " - "Invalid physical port. Aborting discovery\n"); - goto Exit; - } - - /* - Update the DR Path to the port, - in case the old one is no longer available. - */ - p_dr_path = osm_physp_get_dr_path_ptr( p_physp ); - - osm_dr_path_init( p_dr_path, h_bind, p_smp->hop_count, - p_smp->initial_path ); - } - - context.pi_context.node_guid = p_ni->node_guid; - context.pi_context.port_guid = p_ni->port_guid; - context.pi_context.set_method = FALSE; - context.pi_context.update_master_sm_base_lid = FALSE; - context.pi_context.ignore_errors = FALSE; - context.pi_context.light_sweep = FALSE; - - status = osm_req_get( p_rcv->p_gen_req, - osm_physp_get_dr_path_ptr( p_physp ), - IB_MAD_ATTR_PORT_INFO, - cl_hton32( port_num ), - CL_DISP_MSGID_NONE, - &context ); - - if( status != IB_SUCCESS ) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D13: " - "Failure initiating PortInfo request (%s)\n", - ib_get_err_str(status)); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + osm_port_t *p_port; + osm_port_t *p_port_check; + osm_madw_context_t context; + uint8_t port_num; + osm_physp_t *p_physp; + ib_api_status_t status; + osm_dr_path_t *p_dr_path; + osm_bind_handle_t h_bind; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_existing_ca_or_router); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + port_num = ib_node_info_get_local_port_num(p_ni); + h_bind = osm_madw_get_bind_handle(p_madw); + + /* + Determine if we have encountered this node through a + previously undiscovered port. If so, build the new + port object. + */ + p_port = osm_get_port_by_guid(p_rcv->p_subn, p_ni->port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_ni_rcv_process_existing_ca_or_router: " + "Creating new port object with GUID 0x%" PRIx64 "\n", + cl_ntoh64(p_ni->port_guid)); + + osm_node_init_physp(p_node, p_madw); + + p_port = osm_port_new(p_ni, p_node); + if (p_port == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D04: " + "Unable to create new port object\n"); + goto Exit; + } + + /* + Add the new port object to the database. + */ + p_port_check = + (osm_port_t *) cl_qmap_insert(&p_rcv->p_subn->port_guid_tbl, + p_ni->port_guid, + &p_port->map_item); + if (p_port_check != p_port) { + /* + We should never be here! + Somehow, this port GUID already exists in the table. + */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D12: " + "Port 0x%" PRIx64 " already in the database!\n", + cl_ntoh64(p_ni->port_guid)); + + osm_port_delete(&p_port); + goto Exit; + } + + /* If we are a master, then this means the port is new on the subnet. + Mark it as new - need to send trap 64 on these ports. + The condition that we are master is true, since if we are in discovering + state (meaning we woke up from standby or we are just initializing), + then these ports may be new to us, but are not new on the subnet. + If we are master, then the subnet as we know it is the updated one, + and any new ports we encounter should cause trap 64. C14-72.1.1 */ + if (p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER) + p_port->is_new = 1; + + p_physp = osm_node_get_physp_ptr(p_node, port_num); + } else { + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D19: " + "Invalid physical port. Aborting discovery\n"); + goto Exit; + } + + /* + Update the DR Path to the port, + in case the old one is no longer available. + */ + p_dr_path = osm_physp_get_dr_path_ptr(p_physp); + + osm_dr_path_init(p_dr_path, h_bind, p_smp->hop_count, + p_smp->initial_path); + } + + context.pi_context.node_guid = p_ni->node_guid; + context.pi_context.port_guid = p_ni->port_guid; + context.pi_context.set_method = FALSE; + context.pi_context.update_master_sm_base_lid = FALSE; + context.pi_context.ignore_errors = FALSE; + context.pi_context.light_sweep = FALSE; + + status = osm_req_get(p_rcv->p_gen_req, + osm_physp_get_dr_path_ptr(p_physp), + IB_MAD_ATTR_PORT_INFO, + cl_hton32(port_num), CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_existing_ca_or_router: ERR 0D13: " + "Failure initiating PortInfo request (%s)\n", + ib_get_err_str(status)); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_ni_rcv_process_switch( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_switch(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_api_status_t status = IB_SUCCESS; - osm_madw_context_t context; - osm_dr_path_t dr_path; - ib_smp_t *p_smp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_switch ); - - CL_ASSERT( p_node ); - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - osm_dr_path_init( &dr_path, - osm_madw_get_bind_handle( p_madw ), - p_smp->hop_count, - p_smp->initial_path ); - - context.si_context.node_guid = osm_node_get_node_guid( p_node ); - context.si_context.set_method = FALSE; - context.si_context.light_sweep = FALSE; - - /* Request a SwitchInfo attribute */ - status = osm_req_get( p_rcv->p_gen_req, - &dr_path, - IB_MAD_ATTR_SWITCH_INFO, - 0, - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - /* continue despite error */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_switch: ERR 0D06: " - "Failure initiating SwitchInfo request (%s)\n", - ib_get_err_str( status ) ); - - OSM_LOG_EXIT( p_rcv->p_log ); + ib_api_status_t status = IB_SUCCESS; + osm_madw_context_t context; + osm_dr_path_t dr_path; + ib_smp_t *p_smp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_switch); + + CL_ASSERT(p_node); + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + osm_dr_path_init(&dr_path, + osm_madw_get_bind_handle(p_madw), + p_smp->hop_count, p_smp->initial_path); + + context.si_context.node_guid = osm_node_get_node_guid(p_node); + context.si_context.set_method = FALSE; + context.si_context.light_sweep = FALSE; + + /* Request a SwitchInfo attribute */ + status = osm_req_get(p_rcv->p_gen_req, + &dr_path, + IB_MAD_ATTR_SWITCH_INFO, + 0, CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + /* continue despite error */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_switch: ERR 0D06: " + "Failure initiating SwitchInfo request (%s)\n", + ib_get_err_str(status)); + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_existing_switch( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_existing_switch(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_existing_switch ); - - /* - If this switch has already been probed during this sweep, - then don't bother reprobing it. - There is one exception - if the node has been visited, but - for some reason we don't have the switch object (this can happen - if the SwitchInfo mad didn't reach the SM) then we want - to retry to probe the switch. - */ - if( p_node->discovery_count == 1 ) - __osm_ni_rcv_process_switch( p_rcv, p_node, p_madw ); - else if( !p_node->sw || p_node->sw->discovery_count == 0 ) - { - /* we don't have the SwitchInfo - retry to get it */ - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_ni_rcv_process_existing_switch: " - "Retry to get SwitchInfo on node GUID:0x%" - PRIx64 "\n", cl_ntoh64(osm_node_get_node_guid(p_node)) ); - __osm_ni_rcv_process_switch( p_rcv, p_node, p_madw ); - } - - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_existing_switch); + + /* + If this switch has already been probed during this sweep, + then don't bother reprobing it. + There is one exception - if the node has been visited, but + for some reason we don't have the switch object (this can happen + if the SwitchInfo mad didn't reach the SM) then we want + to retry to probe the switch. + */ + if (p_node->discovery_count == 1) + __osm_ni_rcv_process_switch(p_rcv, p_node, p_madw); + else if (!p_node->sw || p_node->sw->discovery_count == 0) { + /* we don't have the SwitchInfo - retry to get it */ + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_ni_rcv_process_existing_switch: " + "Retry to get SwitchInfo on node GUID:0x%" + PRIx64 "\n", cl_ntoh64(osm_node_get_node_guid(p_node))); + __osm_ni_rcv_process_switch(p_rcv, p_node, p_madw); + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_new_switch( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_new_switch(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_new_switch ); + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_new_switch); - __osm_ni_rcv_process_switch( p_rcv, p_node, p_madw ); + __osm_ni_rcv_process_switch(p_rcv, p_node, p_madw); - /* - A node guid of 0 is the corner case that indicates - we discovered our own node. Initialize the subnet - object with the SM's own port guid. - */ - if( osm_madw_get_ni_context_ptr( p_madw )->node_guid == 0 ) - p_rcv->p_subn->sm_port_guid = p_node->node_info.port_guid; + /* + A node guid of 0 is the corner case that indicates + we discovered our own node. Initialize the subnet + object with the SM's own port guid. + */ + if (osm_madw_get_ni_context_ptr(p_madw)->node_guid == 0) + p_rcv->p_subn->sm_port_guid = p_node->node_info.port_guid; - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must NOT be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_new( - IN const osm_ni_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_new(IN const osm_ni_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - osm_node_t *p_node; - osm_node_t *p_node_check; - osm_port_t *p_port; - osm_port_t *p_port_check; - osm_router_t *p_rtr = NULL; - osm_router_t *p_rtr_check; - cl_qmap_t *p_rtr_guid_tbl; - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - osm_ni_context_t *p_ni_context; - uint8_t port_num; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_new ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - p_ni_context = osm_madw_get_ni_context_ptr( p_madw ); - port_num = ib_node_info_get_local_port_num( p_ni ); - - osm_dump_smp_dr_path( p_rcv->p_log, p_smp, OSM_LOG_VERBOSE ); - - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_ni_rcv_process_new: " - "Discovered new %s node," - "\n\t\t\t\tGUID 0x%" PRIx64 ", TID 0x%" PRIx64 "\n", - ib_get_node_type_str( p_ni->node_type ), - cl_ntoh64( p_ni->node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - - p_node = osm_node_new( p_madw ); - if( p_node == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D07: " - "Unable to create new node object\n" ); - goto Exit; - } - - /* - Create a new port object to represent this node's physical - ports in the port table. - */ - p_port = osm_port_new( p_ni, p_node ); - if( p_port == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D14: " - "Unable to create new port object\n" ); - osm_node_delete( &p_node ); - goto Exit; - } - - /* - Add the new port object to the database. - */ - p_port_check = (osm_port_t*)cl_qmap_insert( &p_rcv->p_subn->port_guid_tbl, - p_ni->port_guid, - &p_port->map_item ); - if( p_port_check != p_port ) - { - /* - We should never be here! - Somehow, this port GUID already exists in the table. - */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D15: " - "Duplicate Port GUID 0x%" PRIx64 "! Found by the two directed routes:\n", - cl_ntoh64( p_ni->port_guid ) ); - osm_dump_dr_path(p_rcv->p_log, - osm_physp_get_dr_path_ptr(p_port->p_physp), - OSM_LOG_ERROR); - osm_dump_dr_path(p_rcv->p_log, - osm_physp_get_dr_path_ptr(p_port_check->p_physp), - OSM_LOG_ERROR); - osm_port_delete( &p_port ); - osm_node_delete( &p_node ); - goto Exit; - } - - /* If we are a master, then this means the port is new on the subnet. - Mark it as new - need to send trap 64 on these ports. - The condition that we are master is true, since if we are in discovering - state (meaning we woke up from standby or we are just initializing), - then these ports may be new to us, but are not new on the subnet. - If we are master, then the subnet as we know it is the updated one, - and any new ports we encounter should cause trap 64. C14-72.1.1 */ - if ( p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER ) - p_port->is_new = 1; - - /* If there were RouterInfo or other router attribute, - this would be elsewhere */ - if ( p_ni->node_type == IB_NODE_TYPE_ROUTER ) - { - if ((p_rtr = osm_router_new(p_port)) == NULL) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D1A: " - "Unable to create new router object\n" ); - else - { - p_rtr_guid_tbl = &p_rcv->p_subn->rtr_guid_tbl; - p_rtr_check = (osm_router_t*)cl_qmap_insert( p_rtr_guid_tbl, - p_ni->port_guid, - &p_rtr->map_item ); - if( p_rtr_check != p_rtr ) - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D1B: " - "Unable to add port GUID:0x%016" PRIx64 " to router table\n", - cl_ntoh64( p_ni->port_guid ) ); - } - } - - p_node_check = (osm_node_t*)cl_qmap_insert( &p_rcv->p_subn->node_guid_tbl, - p_ni->node_guid, - &p_node->map_item ); - if( p_node_check != p_node ) - { - /* - This node must have been inserted by another thread. - This is unexpected, but is not an error. - We can simply clean-up, since the other thread will - see this processing through to completion. - */ - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_ni_rcv_process_new: " - "Discovery race detected at node 0x%" PRIx64 "\n", - cl_ntoh64( p_ni->node_guid ) ); - osm_node_delete( &p_node ); - p_node = p_node_check; - __osm_ni_rcv_set_links( p_rcv, p_node, port_num, p_ni_context ); - goto Exit; - } - else - __osm_ni_rcv_set_links( p_rcv, p_node, port_num, p_ni_context ); - - p_node->discovery_count++; - __osm_ni_rcv_get_node_desc( p_rcv, p_node, p_madw ); - - switch( p_ni->node_type ) - { - case IB_NODE_TYPE_CA: - case IB_NODE_TYPE_ROUTER: - __osm_ni_rcv_process_new_ca_or_router( p_rcv, p_node, p_madw ); - break; - case IB_NODE_TYPE_SWITCH: - __osm_ni_rcv_process_new_switch( p_rcv, p_node, p_madw ); - break; - default: - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_new: ERR 0D16: " - "Unknown node type %u with GUID 0x%" PRIx64 "\n", - p_ni->node_type, cl_ntoh64( p_ni->node_guid ) ); - break; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_node_t *p_node; + osm_node_t *p_node_check; + osm_port_t *p_port; + osm_port_t *p_port_check; + osm_router_t *p_rtr = NULL; + osm_router_t *p_rtr_check; + cl_qmap_t *p_rtr_guid_tbl; + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + osm_ni_context_t *p_ni_context; + uint8_t port_num; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_new); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + p_ni_context = osm_madw_get_ni_context_ptr(p_madw); + port_num = ib_node_info_get_local_port_num(p_ni); + + osm_dump_smp_dr_path(p_rcv->p_log, p_smp, OSM_LOG_VERBOSE); + + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_ni_rcv_process_new: " + "Discovered new %s node," + "\n\t\t\t\tGUID 0x%" PRIx64 ", TID 0x%" PRIx64 "\n", + ib_get_node_type_str(p_ni->node_type), + cl_ntoh64(p_ni->node_guid), cl_ntoh64(p_smp->trans_id)); + + p_node = osm_node_new(p_madw); + if (p_node == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D07: " + "Unable to create new node object\n"); + goto Exit; + } + + /* + Create a new port object to represent this node's physical + ports in the port table. + */ + p_port = osm_port_new(p_ni, p_node); + if (p_port == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D14: " + "Unable to create new port object\n"); + osm_node_delete(&p_node); + goto Exit; + } + + /* + Add the new port object to the database. + */ + p_port_check = + (osm_port_t *) cl_qmap_insert(&p_rcv->p_subn->port_guid_tbl, + p_ni->port_guid, &p_port->map_item); + if (p_port_check != p_port) { + /* + We should never be here! + Somehow, this port GUID already exists in the table. + */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D15: " + "Duplicate Port GUID 0x%" PRIx64 + "! Found by the two directed routes:\n", + cl_ntoh64(p_ni->port_guid)); + osm_dump_dr_path(p_rcv->p_log, + osm_physp_get_dr_path_ptr(p_port->p_physp), + OSM_LOG_ERROR); + osm_dump_dr_path(p_rcv->p_log, + osm_physp_get_dr_path_ptr(p_port_check-> + p_physp), + OSM_LOG_ERROR); + osm_port_delete(&p_port); + osm_node_delete(&p_node); + goto Exit; + } + + /* If we are a master, then this means the port is new on the subnet. + Mark it as new - need to send trap 64 on these ports. + The condition that we are master is true, since if we are in discovering + state (meaning we woke up from standby or we are just initializing), + then these ports may be new to us, but are not new on the subnet. + If we are master, then the subnet as we know it is the updated one, + and any new ports we encounter should cause trap 64. C14-72.1.1 */ + if (p_rcv->p_subn->sm_state == IB_SMINFO_STATE_MASTER) + p_port->is_new = 1; + + /* If there were RouterInfo or other router attribute, + this would be elsewhere */ + if (p_ni->node_type == IB_NODE_TYPE_ROUTER) { + if ((p_rtr = osm_router_new(p_port)) == NULL) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D1A: " + "Unable to create new router object\n"); + else { + p_rtr_guid_tbl = &p_rcv->p_subn->rtr_guid_tbl; + p_rtr_check = + (osm_router_t *) cl_qmap_insert(p_rtr_guid_tbl, + p_ni->port_guid, + &p_rtr->map_item); + if (p_rtr_check != p_rtr) + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D1B: " + "Unable to add port GUID:0x%016" PRIx64 + " to router table\n", + cl_ntoh64(p_ni->port_guid)); + } + } + + p_node_check = + (osm_node_t *) cl_qmap_insert(&p_rcv->p_subn->node_guid_tbl, + p_ni->node_guid, &p_node->map_item); + if (p_node_check != p_node) { + /* + This node must have been inserted by another thread. + This is unexpected, but is not an error. + We can simply clean-up, since the other thread will + see this processing through to completion. + */ + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_ni_rcv_process_new: " + "Discovery race detected at node 0x%" PRIx64 "\n", + cl_ntoh64(p_ni->node_guid)); + osm_node_delete(&p_node); + p_node = p_node_check; + __osm_ni_rcv_set_links(p_rcv, p_node, port_num, p_ni_context); + goto Exit; + } else + __osm_ni_rcv_set_links(p_rcv, p_node, port_num, p_ni_context); + + p_node->discovery_count++; + __osm_ni_rcv_get_node_desc(p_rcv, p_node, p_madw); + + switch (p_ni->node_type) { + case IB_NODE_TYPE_CA: + case IB_NODE_TYPE_ROUTER: + __osm_ni_rcv_process_new_ca_or_router(p_rcv, p_node, p_madw); + break; + case IB_NODE_TYPE_SWITCH: + __osm_ni_rcv_process_new_switch(p_rcv, p_node, p_madw); + break; + default: + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_new: ERR 0D16: " + "Unknown node type %u with GUID 0x%" PRIx64 "\n", + p_ni->node_type, cl_ntoh64(p_ni->node_guid)); + break; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_ni_rcv_process_existing( - IN const osm_ni_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_ni_rcv_process_existing(IN const osm_ni_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - osm_ni_context_t *p_ni_context; - uint8_t port_num; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_ni_rcv_process_existing ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - p_ni_context = osm_madw_get_ni_context_ptr( p_madw ); - port_num = ib_node_info_get_local_port_num( p_ni ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_VERBOSE ) ) - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_ni_rcv_process_existing: " - "Rediscovered %s node 0x%" PRIx64 - " TID 0x%" PRIx64 ", discovered %u times already\n", - ib_get_node_type_str(p_ni->node_type), - cl_ntoh64( p_ni->node_guid ), - cl_ntoh64( p_smp->trans_id ), - p_node->discovery_count ); - - /* - If we haven't already encountered this existing node - on this particular sweep, then process further. - */ - p_node->discovery_count++; - - switch( p_ni->node_type ) - { - case IB_NODE_TYPE_CA: - case IB_NODE_TYPE_ROUTER: - __osm_ni_rcv_process_existing_ca_or_router( p_rcv, p_node, p_madw ); - break; - - case IB_NODE_TYPE_SWITCH: - __osm_ni_rcv_process_existing_switch( p_rcv, p_node, p_madw ); - break; - - default: - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_ni_rcv_process_existing: ERR 0D09: " - "Unknown node type %u with GUID 0x%" PRIx64 "\n", - p_ni->node_type, cl_ntoh64( p_ni->node_guid ) ); - break; - } - - __osm_ni_rcv_set_links( p_rcv, p_node, port_num, p_ni_context ); - - OSM_LOG_EXIT( p_rcv->p_log ); + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + osm_ni_context_t *p_ni_context; + uint8_t port_num; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_ni_rcv_process_existing); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + p_ni_context = osm_madw_get_ni_context_ptr(p_madw); + port_num = ib_node_info_get_local_port_num(p_ni); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_VERBOSE)) + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_ni_rcv_process_existing: " + "Rediscovered %s node 0x%" PRIx64 + " TID 0x%" PRIx64 ", discovered %u times already\n", + ib_get_node_type_str(p_ni->node_type), + cl_ntoh64(p_ni->node_guid), + cl_ntoh64(p_smp->trans_id), p_node->discovery_count); + + /* + If we haven't already encountered this existing node + on this particular sweep, then process further. + */ + p_node->discovery_count++; + + switch (p_ni->node_type) { + case IB_NODE_TYPE_CA: + case IB_NODE_TYPE_ROUTER: + __osm_ni_rcv_process_existing_ca_or_router(p_rcv, p_node, + p_madw); + break; + + case IB_NODE_TYPE_SWITCH: + __osm_ni_rcv_process_existing_switch(p_rcv, p_node, p_madw); + break; + + default: + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_ni_rcv_process_existing: ERR 0D09: " + "Unknown node type %u with GUID 0x%" PRIx64 "\n", + p_ni->node_type, cl_ntoh64(p_ni->node_guid)); + break; + } + + __osm_ni_rcv_set_links(p_rcv, p_node, port_num, p_ni_context); + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_ni_rcv_construct( - IN osm_ni_rcv_t* const p_rcv ) +void osm_ni_rcv_construct(IN osm_ni_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_ni_rcv_destroy( - IN osm_ni_rcv_t* const p_rcv ) +void osm_ni_rcv_destroy(IN osm_ni_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_ni_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_ni_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_ni_rcv_init( - IN osm_ni_rcv_t* const p_rcv, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ) +osm_ni_rcv_init(IN osm_ni_rcv_t * const p_rcv, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_ni_rcv_init ); + OSM_LOG_ENTER(p_log, osm_ni_rcv_init); - osm_ni_rcv_construct( p_rcv ); + osm_ni_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_gen_req = p_req; - p_rcv->p_state_mgr = p_state_mgr; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_gen_req = p_req; + p_rcv->p_state_mgr = p_state_mgr; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_ni_rcv_process( - IN void *context, - IN void *data ) +void osm_ni_rcv_process(IN void *context, IN void *data) { - osm_ni_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_node_info_t *p_ni; - ib_smp_t *p_smp; - osm_node_t *p_node; - boolean_t process_new_flag = FALSE; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_ni_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_ni = (ib_node_info_t*)ib_smp_get_payload_ptr( p_smp ); - - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_NODE_INFO ); - - if (p_ni->node_guid == 0) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_ni_rcv_process: ERR 0D16: " - "Got Zero Node GUID! Found on the directed route:\n"); - osm_dump_smp_dr_path(p_rcv->p_log, p_smp, OSM_LOG_ERROR); - goto Exit; - } - - if (p_ni->port_guid == 0) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_ni_rcv_process: ERR 0D17: " - "Got Zero Port GUID! Found on the directed route:\n"); - osm_dump_smp_dr_path(p_rcv->p_log, p_smp, OSM_LOG_ERROR); - goto Exit; - } - - /* - Determine if this node has already been discovered, - and process accordingly. - During processing of this node, hold the shared lock. - */ - - CL_PLOCK_EXCL_ACQUIRE( p_rcv->p_lock ); - p_node = osm_get_node_by_guid( p_rcv->p_subn, p_ni->node_guid ); - - osm_dump_node_info( p_rcv->p_log, p_ni, OSM_LOG_DEBUG ); - - if( !p_node ) - { - __osm_ni_rcv_process_new( p_rcv, p_madw ); - process_new_flag = TRUE; - } - else - __osm_ni_rcv_process_existing( p_rcv, p_node, p_madw ); - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - /* - * If we processed a new node - need to signal to the state_mgr that - * change detected. BUT - we cannot call the osm_state_mgr_process - * from within the lock of p_rcv->p_lock (can cause a deadlock). - */ - if ( process_new_flag ) - osm_state_mgr_process( p_rcv->p_state_mgr, OSM_SIGNAL_CHANGE_DETECTED ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_ni_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_node_info_t *p_ni; + ib_smp_t *p_smp; + osm_node_t *p_node; + boolean_t process_new_flag = FALSE; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_ni_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_ni = (ib_node_info_t *) ib_smp_get_payload_ptr(p_smp); + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_NODE_INFO); + + if (p_ni->node_guid == 0) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_ni_rcv_process: ERR 0D16: " + "Got Zero Node GUID! Found on the directed route:\n"); + osm_dump_smp_dr_path(p_rcv->p_log, p_smp, OSM_LOG_ERROR); + goto Exit; + } + + if (p_ni->port_guid == 0) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_ni_rcv_process: ERR 0D17: " + "Got Zero Port GUID! Found on the directed route:\n"); + osm_dump_smp_dr_path(p_rcv->p_log, p_smp, OSM_LOG_ERROR); + goto Exit; + } + + /* + Determine if this node has already been discovered, + and process accordingly. + During processing of this node, hold the shared lock. + */ + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + p_node = osm_get_node_by_guid(p_rcv->p_subn, p_ni->node_guid); + + osm_dump_node_info(p_rcv->p_log, p_ni, OSM_LOG_DEBUG); + + if (!p_node) { + __osm_ni_rcv_process_new(p_rcv, p_madw); + process_new_flag = TRUE; + } else + __osm_ni_rcv_process_existing(p_rcv, p_node, p_madw); + + CL_PLOCK_RELEASE(p_rcv->p_lock); + + /* + * If we processed a new node - need to signal to the state_mgr that + * change detected. BUT - we cannot call the osm_state_mgr_process + * from within the lock of p_rcv->p_lock (can cause a deadlock). + */ + if (process_new_flag) + osm_state_mgr_process(p_rcv->p_state_mgr, + OSM_SIGNAL_CHANGE_DETECTED); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c index 4cf57cf..9a596dd 100644 --- a/opensm/opensm/osm_opensm.c +++ b/opensm/opensm/osm_opensm.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_opensm_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -66,31 +65,31 @@ struct routing_engine_module { const char *name; - int (*setup)(osm_opensm_t *p_osm); + int (*setup) (osm_opensm_t * p_osm); }; -extern int osm_ucast_updn_setup(osm_opensm_t *p_osm); -extern int osm_ucast_file_setup(osm_opensm_t *p_osm); -extern int osm_ucast_ftree_setup(osm_opensm_t *p_osm); -extern int osm_ucast_lash_setup(osm_opensm_t *p_osm); +extern int osm_ucast_updn_setup(osm_opensm_t * p_osm); +extern int osm_ucast_file_setup(osm_opensm_t * p_osm); +extern int osm_ucast_ftree_setup(osm_opensm_t * p_osm); +extern int osm_ucast_lash_setup(osm_opensm_t * p_osm); -static int osm_ucast_null_setup(osm_opensm_t *p_osm); +static int osm_ucast_null_setup(osm_opensm_t * p_osm); const static struct routing_engine_module routing_modules[] = { - { "null", osm_ucast_null_setup }, - { "updn", osm_ucast_updn_setup }, - { "file", osm_ucast_file_setup }, - { "ftree", osm_ucast_ftree_setup }, - { "lash", osm_ucast_lash_setup }, - { NULL, NULL } + {"null", osm_ucast_null_setup}, + {"updn", osm_ucast_updn_setup}, + {"file", osm_ucast_file_setup}, + {"ftree", osm_ucast_ftree_setup}, + {"lash", osm_ucast_lash_setup}, + {NULL, NULL} }; -static int setup_routing_engine(osm_opensm_t *p_osm, const char *name) +static int setup_routing_engine(osm_opensm_t * p_osm, const char *name) { const struct routing_engine_module *r; for (r = routing_modules; r->name && *r->name; r++) { - if(!strcmp(r->name, name)) { + if (!strcmp(r->name, name)) { p_osm->routing_engine.name = r->name; if (r->setup(p_osm)) { osm_log(&p_osm->log, OSM_LOG_VERBOSE, @@ -98,17 +97,17 @@ static int setup_routing_engine(osm_opensm_t *p_osm, const char *name) " engine \'%s\' failed\n", name); return -2; } - osm_log (&p_osm->log, OSM_LOG_DEBUG, - "setup_routing_engine: " - "\'%s\' routing engine set up\n", - p_osm->routing_engine.name); + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "setup_routing_engine: " + "\'%s\' routing engine set up\n", + p_osm->routing_engine.name); return 0; } } return -1; } -static int osm_ucast_null_setup(osm_opensm_t *p_osm) +static int osm_ucast_null_setup(osm_opensm_t * p_osm) { osm_log(&p_osm->log, OSM_LOG_VERBOSE, "osm_ucast_null_setup: nothing yet - " @@ -118,241 +117,228 @@ static int osm_ucast_null_setup(osm_opensm_t *p_osm) /********************************************************************** **********************************************************************/ -void -osm_opensm_construct( - IN osm_opensm_t * const p_osm ) +void osm_opensm_construct(IN osm_opensm_t * const p_osm) { - memset( p_osm, 0, sizeof( *p_osm ) ); - osm_subn_construct( &p_osm->subn ); - osm_sm_construct( &p_osm->sm ); - osm_sa_construct( &p_osm->sa ); - osm_db_construct( &p_osm->db ); - osm_mad_pool_construct( &p_osm->mad_pool ); - osm_vl15_construct( &p_osm->vl15 ); - osm_log_construct( &p_osm->log ); + memset(p_osm, 0, sizeof(*p_osm)); + osm_subn_construct(&p_osm->subn); + osm_sm_construct(&p_osm->sm); + osm_sa_construct(&p_osm->sa); + osm_db_construct(&p_osm->db); + osm_mad_pool_construct(&p_osm->mad_pool); + osm_vl15_construct(&p_osm->vl15); + osm_log_construct(&p_osm->log); } /********************************************************************** **********************************************************************/ -void -osm_opensm_destroy( - IN osm_opensm_t * const p_osm ) +void osm_opensm_destroy(IN osm_opensm_t * const p_osm) { - /* in case of shutdown through exit proc - no ^C */ - osm_exit_flag = TRUE; + /* in case of shutdown through exit proc - no ^C */ + osm_exit_flag = TRUE; - /* - * First of all, clear the is_sm bit. - */ - if( p_osm->sm.mad_ctrl.h_bind ) - osm_vendor_set_sm( p_osm->sm.mad_ctrl.h_bind, FALSE ); + /* + * First of all, clear the is_sm bit. + */ + if (p_osm->sm.mad_ctrl.h_bind) + osm_vendor_set_sm(p_osm->sm.mad_ctrl.h_bind, FALSE); #ifdef ENABLE_OSM_PERF_MGR - /* Shutdown the PerfMgr */ - osm_perfmgr_shutdown(&p_osm->perfmgr); -#endif /* ENABLE_OSM_PERF_MGR */ - - /* shut down the SA - * - unbind from QP1 messages - */ - osm_sa_shutdown( &p_osm->sa ); - - /* shut down the SM - * - make sure the SM sweeper thread exited - * - unbind from QP0 messages - */ - osm_sm_shutdown( &p_osm->sm ); - - /* cleanup all messages on VL15 fifo that were not sent yet */ - osm_vl15_shutdown( &p_osm->vl15, &p_osm->mad_pool ); - - /* shut down the dispatcher - so no new messages cross */ - cl_disp_shutdown( &p_osm->disp ); - - /* dump SA DB */ - osm_sa_db_file_dump(p_osm); - - /* do the destruction in reverse order as init */ - if (p_osm->routing_engine.delete) - p_osm->routing_engine.delete(p_osm->routing_engine.context); - osm_sa_destroy( &p_osm->sa ); - osm_sm_destroy( &p_osm->sm ); + /* Shutdown the PerfMgr */ + osm_perfmgr_shutdown(&p_osm->perfmgr); +#endif /* ENABLE_OSM_PERF_MGR */ + + /* shut down the SA + * - unbind from QP1 messages + */ + osm_sa_shutdown(&p_osm->sa); + + /* shut down the SM + * - make sure the SM sweeper thread exited + * - unbind from QP0 messages + */ + osm_sm_shutdown(&p_osm->sm); + + /* cleanup all messages on VL15 fifo that were not sent yet */ + osm_vl15_shutdown(&p_osm->vl15, &p_osm->mad_pool); + + /* shut down the dispatcher - so no new messages cross */ + cl_disp_shutdown(&p_osm->disp); + + /* dump SA DB */ + osm_sa_db_file_dump(p_osm); + + /* do the destruction in reverse order as init */ + if (p_osm->routing_engine.delete) + p_osm->routing_engine.delete(p_osm->routing_engine.context); + osm_sa_destroy(&p_osm->sa); + osm_sm_destroy(&p_osm->sm); #ifdef ENABLE_OSM_PERF_MGR - osm_perfmgr_destroy( &p_osm->perfmgr ); -#endif /* ENABLE_OSM_PERF_MGR */ - osm_db_destroy( &p_osm->db ); - osm_vl15_destroy( &p_osm->vl15, &p_osm->mad_pool ); - osm_mad_pool_destroy( &p_osm->mad_pool ); - osm_vendor_delete( &p_osm->p_vendor ); - osm_subn_destroy( &p_osm->subn ); - cl_disp_destroy( &p_osm->disp ); - - cl_plock_destroy( &p_osm->lock ); - - osm_log_destroy( &p_osm->log ); + osm_perfmgr_destroy(&p_osm->perfmgr); +#endif /* ENABLE_OSM_PERF_MGR */ + osm_db_destroy(&p_osm->db); + osm_vl15_destroy(&p_osm->vl15, &p_osm->mad_pool); + osm_mad_pool_destroy(&p_osm->mad_pool); + osm_vendor_delete(&p_osm->p_vendor); + osm_subn_destroy(&p_osm->subn); + cl_disp_destroy(&p_osm->disp); + + cl_plock_destroy(&p_osm->lock); + + osm_log_destroy(&p_osm->log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_opensm_init( - IN osm_opensm_t * const p_osm, - IN const osm_subn_opt_t * const p_opt ) +osm_opensm_init(IN osm_opensm_t * const p_osm, + IN const osm_subn_opt_t * const p_opt) { - ib_api_status_t status; - - /* Can't use log macros here, since we're initializing the log */ - osm_opensm_construct( p_osm ); - - if (p_opt->daemon) - p_osm->log.daemon = 1; - - status = osm_log_init_v2( &p_osm->log, p_opt->force_log_flush, - p_opt->log_flags, p_opt->log_file, - p_opt->log_max_size, p_opt->accum_log_file ); - if( status != IB_SUCCESS ) - return ( status ); - - /* If there is a log level defined - add the OSM_VERSION to it */ - osm_log( &p_osm->log, - osm_log_get_level( &p_osm->log ) & ( OSM_LOG_SYS ^ 0xFF ), "%s\n", - OSM_VERSION ); - /* Write the OSM_VERSION to the SYS_LOG */ - osm_log( &p_osm->log, OSM_LOG_SYS, "%s\n", OSM_VERSION ); /* Format Waived */ - - osm_log( &p_osm->log, OSM_LOG_FUNCS, "osm_opensm_init: [\n" ); /* Format Waived */ - - status = cl_plock_init( &p_osm->lock ); - if( status != IB_SUCCESS ) - goto Exit; - - if( p_opt->single_thread ) - { - osm_log( &p_osm->log, OSM_LOG_INFO, - "osm_opensm_init: Forcing single threaded dispatcher\n" ); - status = cl_disp_init( &p_osm->disp, 1, "opensm" ); - } - else - { - /* - * Normal behavior is to initialize the dispatcher with - * one thread per CPU, as specified by a thread count of '0'. - */ - status = cl_disp_init( &p_osm->disp, 0, "opensm" ); - } - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_subn_init( &p_osm->subn, p_osm, p_opt ); - if( status != IB_SUCCESS ) - goto Exit; - - p_osm->p_vendor = - osm_vendor_new( &p_osm->log, p_opt->transaction_timeout ); - if( p_osm->p_vendor == NULL ) - { - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - status = osm_mad_pool_init( &p_osm->mad_pool, &p_osm->log ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_vl15_init( &p_osm->vl15, - p_osm->p_vendor, - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, - &p_osm->subn, &p_osm->disp, &p_osm->lock ); - if( status != IB_SUCCESS ) - goto Exit; - - /* the DB is in use by the SM and SA so init before */ - status = osm_db_init( &p_osm->db, &p_osm->log ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sm_init( &p_osm->sm, - &p_osm->subn, - &p_osm->db, - p_osm->p_vendor, - &p_osm->mad_pool, - &p_osm->vl15, - &p_osm->log, - &p_osm->stats, &p_osm->disp, &p_osm->lock ); - - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sa_init( &p_osm->sm, - &p_osm->sa, - &p_osm->subn, - p_osm->p_vendor, - &p_osm->mad_pool, - &p_osm->log, - &p_osm->stats, &p_osm->disp, &p_osm->lock ); - - if( status != IB_SUCCESS ) - goto Exit; - - p_osm->event_plugin = osm_epi_construct(&p_osm->log, - p_opt->event_plugin_name); + ib_api_status_t status; + + /* Can't use log macros here, since we're initializing the log */ + osm_opensm_construct(p_osm); + + if (p_opt->daemon) + p_osm->log.daemon = 1; + + status = osm_log_init_v2(&p_osm->log, p_opt->force_log_flush, + p_opt->log_flags, p_opt->log_file, + p_opt->log_max_size, p_opt->accum_log_file); + if (status != IB_SUCCESS) + return (status); + + /* If there is a log level defined - add the OSM_VERSION to it */ + osm_log(&p_osm->log, + osm_log_get_level(&p_osm->log) & (OSM_LOG_SYS ^ 0xFF), "%s\n", + OSM_VERSION); + /* Write the OSM_VERSION to the SYS_LOG */ + osm_log(&p_osm->log, OSM_LOG_SYS, "%s\n", OSM_VERSION); /* Format Waived */ + + osm_log(&p_osm->log, OSM_LOG_FUNCS, "osm_opensm_init: [\n"); /* Format Waived */ + + status = cl_plock_init(&p_osm->lock); + if (status != IB_SUCCESS) + goto Exit; + + if (p_opt->single_thread) { + osm_log(&p_osm->log, OSM_LOG_INFO, + "osm_opensm_init: Forcing single threaded dispatcher\n"); + status = cl_disp_init(&p_osm->disp, 1, "opensm"); + } else { + /* + * Normal behavior is to initialize the dispatcher with + * one thread per CPU, as specified by a thread count of '0'. + */ + status = cl_disp_init(&p_osm->disp, 0, "opensm"); + } + if (status != IB_SUCCESS) + goto Exit; + + status = osm_subn_init(&p_osm->subn, p_osm, p_opt); + if (status != IB_SUCCESS) + goto Exit; + + p_osm->p_vendor = + osm_vendor_new(&p_osm->log, p_opt->transaction_timeout); + if (p_osm->p_vendor == NULL) { + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + status = osm_mad_pool_init(&p_osm->mad_pool, &p_osm->log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_vl15_init(&p_osm->vl15, + p_osm->p_vendor, + &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, + &p_osm->subn, &p_osm->disp, &p_osm->lock); + if (status != IB_SUCCESS) + goto Exit; + + /* the DB is in use by the SM and SA so init before */ + status = osm_db_init(&p_osm->db, &p_osm->log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sm_init(&p_osm->sm, + &p_osm->subn, + &p_osm->db, + p_osm->p_vendor, + &p_osm->mad_pool, + &p_osm->vl15, + &p_osm->log, + &p_osm->stats, &p_osm->disp, &p_osm->lock); + + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sa_init(&p_osm->sm, + &p_osm->sa, + &p_osm->subn, + p_osm->p_vendor, + &p_osm->mad_pool, + &p_osm->log, + &p_osm->stats, &p_osm->disp, &p_osm->lock); + + if (status != IB_SUCCESS) + goto Exit; + + p_osm->event_plugin = osm_epi_construct(&p_osm->log, + p_opt->event_plugin_name); #ifdef ENABLE_OSM_PERF_MGR - status = osm_perfmgr_init( &p_osm->perfmgr, - &p_osm->subn, - &p_osm->sm, - &p_osm->log, - &p_osm->mad_pool, - p_osm->p_vendor, - &p_osm->disp, - &p_osm->lock, - p_opt, - p_osm->event_plugin); - - if( status != IB_SUCCESS ) - goto Exit; -#endif /* ENABLE_OSM_PERF_MGR */ - - if( p_opt->routing_engine_name && - setup_routing_engine(p_osm, p_opt->routing_engine_name)) { - osm_log( &p_osm->log, OSM_LOG_VERBOSE, - "osm_opensm_init: cannot find or setup routing engine" - " \'%s\'. Default will be used instead\n", - p_opt->routing_engine_name); - goto Exit; - } - - Exit: - osm_log( &p_osm->log, OSM_LOG_FUNCS, "osm_opensm_init: ]\n" ); /* Format Waived */ - return ( status ); + status = osm_perfmgr_init(&p_osm->perfmgr, + &p_osm->subn, + &p_osm->sm, + &p_osm->log, + &p_osm->mad_pool, + p_osm->p_vendor, + &p_osm->disp, + &p_osm->lock, p_opt, p_osm->event_plugin); + + if (status != IB_SUCCESS) + goto Exit; +#endif /* ENABLE_OSM_PERF_MGR */ + + if (p_opt->routing_engine_name && + setup_routing_engine(p_osm, p_opt->routing_engine_name)) { + osm_log(&p_osm->log, OSM_LOG_VERBOSE, + "osm_opensm_init: cannot find or setup routing engine" + " \'%s\'. Default will be used instead\n", + p_opt->routing_engine_name); + goto Exit; + } + + Exit: + osm_log(&p_osm->log, OSM_LOG_FUNCS, "osm_opensm_init: ]\n"); /* Format Waived */ + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_opensm_bind( - IN osm_opensm_t * const p_osm, - IN const ib_net64_t guid ) +osm_opensm_bind(IN osm_opensm_t * const p_osm, IN const ib_net64_t guid) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( &p_osm->log, osm_opensm_bind ); + OSM_LOG_ENTER(&p_osm->log, osm_opensm_bind); - status = osm_sm_bind( &p_osm->sm, guid ); - if( status != IB_SUCCESS ) - goto Exit; + status = osm_sm_bind(&p_osm->sm, guid); + if (status != IB_SUCCESS) + goto Exit; - status = osm_sa_bind( &p_osm->sa, guid ); - if( status != IB_SUCCESS ) - goto Exit; + status = osm_sa_bind(&p_osm->sa, guid); + if (status != IB_SUCCESS) + goto Exit; #ifdef ENABLE_OSM_PERF_MGR - status = osm_perfmgr_bind( &p_osm->perfmgr, guid ); - if( status != IB_SUCCESS ) - goto Exit; -#endif /* ENABLE_OSM_PERF_MGR */ - - Exit: - OSM_LOG_EXIT( &p_osm->log ); - return ( status ); + status = osm_perfmgr_bind(&p_osm->perfmgr, guid); + if (status != IB_SUCCESS) + goto Exit; +#endif /* ENABLE_OSM_PERF_MGR */ + + Exit: + OSM_LOG_EXIT(&p_osm->log); + return (status); } diff --git a/opensm/opensm/osm_perfmgr.c b/opensm/opensm/osm_perfmgr.c index 43689fa..1f6fa19 100644 --- a/opensm/opensm/osm_perfmgr.c +++ b/opensm/opensm/osm_perfmgr.c @@ -43,7 +43,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #ifdef ENABLE_OSM_PERF_MGR @@ -67,8 +67,7 @@ /********************************************************************** * Internal helper functions. **********************************************************************/ -static inline void -__init_monitored_nodes(osm_perfmgr_t *pm) +static inline void __init_monitored_nodes(osm_perfmgr_t * pm) { cl_qmap_init(&pm->monitored_map); pm->remove_list = NULL; @@ -77,7 +76,7 @@ __init_monitored_nodes(osm_perfmgr_t *pm) } static inline void -__mark_for_removal(osm_perfmgr_t *pm, __monitored_node_t *node) +__mark_for_removal(osm_perfmgr_t * pm, __monitored_node_t * node) { if (pm->remove_list) { node->next = pm->remove_list; @@ -88,21 +87,19 @@ __mark_for_removal(osm_perfmgr_t *pm, __monitored_node_t *node) } } -static inline void -__remove_marked_nodes(osm_perfmgr_t *pm) +static inline void __remove_marked_nodes(osm_perfmgr_t * pm) { while (pm->remove_list) { __monitored_node_t *next = pm->remove_list->next; cl_qmap_remove_item(&(pm->monitored_map), - (cl_map_item_t *)(pm->remove_list)); + (cl_map_item_t *) (pm->remove_list)); free(pm->remove_list); pm->remove_list = next; } } -static inline void -__decrement_outstanding_queries(osm_perfmgr_t *pm) +static inline void __decrement_outstanding_queries(osm_perfmgr_t * pm) { cl_atomic_dec(&(pm->outstanding_queries)); cl_event_signal(&(pm->sig_query)); @@ -113,21 +110,21 @@ __decrement_outstanding_queries(osm_perfmgr_t *pm) * the dispatcher. **********************************************************************/ static void -osm_perfmgr_mad_recv_callback(osm_madw_t *p_madw, void* bind_context, - osm_madw_t *p_req_madw) +osm_perfmgr_mad_recv_callback(osm_madw_t * p_madw, void *bind_context, + osm_madw_t * p_req_madw) { - osm_perfmgr_t *pm = (osm_perfmgr_t *)bind_context; + osm_perfmgr_t *pm = (osm_perfmgr_t *) bind_context; - OSM_LOG_ENTER( pm->log, osm_pm_mad_recv_callback ); + OSM_LOG_ENTER(pm->log, osm_pm_mad_recv_callback); - osm_madw_copy_context( p_madw, p_req_madw ); - osm_mad_pool_put( pm->mad_pool, p_req_madw ); + osm_madw_copy_context(p_madw, p_req_madw); + osm_mad_pool_put(pm->mad_pool, p_req_madw); __decrement_outstanding_queries(pm); /* post this message for later processing. */ if (cl_disp_post(pm->pc_disp_h, OSM_MSG_MAD_PORT_COUNTERS, - (void *)p_madw, NULL, NULL) != CL_SUCCESS) { + (void *)p_madw, NULL, NULL) != CL_SUCCESS) { osm_log(pm->log, OSM_LOG_ERROR, "osm_perfmgr_mad_recv_callback: ERR 4C01: " "PerfMgr Dispatcher post failed\n"); @@ -135,7 +132,7 @@ osm_perfmgr_mad_recv_callback(osm_madw_t *p_madw, void* bind_context, } #if 0 do { - struct timeval rcv_time; + struct timeval rcv_time; gettimeofday(&rcv_time, NULL); osm_log(pm->log, OSM_LOG_INFO, "perfmgr rcv time %ld\n", @@ -143,28 +140,27 @@ osm_perfmgr_mad_recv_callback(osm_madw_t *p_madw, void* bind_context, p_madw->context.perfmgr_context.query_start.tv_usec); } while (0); #endif - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** * Process MAD send errors. **********************************************************************/ static void -osm_perfmgr_mad_send_err_callback(void* bind_context, osm_madw_t *p_madw) +osm_perfmgr_mad_send_err_callback(void *bind_context, osm_madw_t * p_madw) { - osm_perfmgr_t *pm = (osm_perfmgr_t *)bind_context; + osm_perfmgr_t *pm = (osm_perfmgr_t *) bind_context; osm_madw_context_t *context = &(p_madw->context); - uint64_t node_guid = context->perfmgr_context.node_guid; - uint8_t port = context->perfmgr_context.port; + uint64_t node_guid = context->perfmgr_context.node_guid; + uint8_t port = context->perfmgr_context.port; - OSM_LOG_ENTER( pm->log, osm_pm_mad_send_err_callback ); + OSM_LOG_ENTER(pm->log, osm_pm_mad_send_err_callback); - osm_log( pm->log, OSM_LOG_ERROR, - "osm_perfmgr_mad_send_err_callback: ERR 4C02: 0x%" PRIx64 " port %d\n", - node_guid, port); + osm_log(pm->log, OSM_LOG_ERROR, + "osm_perfmgr_mad_send_err_callback: ERR 4C02: 0x%" PRIx64 + " port %d\n", node_guid, port); - if (pm->subn->opt.perfmgr_redir && p_madw->status == IB_TIMEOUT) - { + if (pm->subn->opt.perfmgr_redir && p_madw->status == IB_TIMEOUT) { cl_map_item_t *p_node; __monitored_node_t *p_mon_node; @@ -174,19 +170,19 @@ osm_perfmgr_mad_send_err_callback(void* bind_context, osm_madw_t *p_madw) cl_qmap_end(&(pm->monitored_map))) { cl_plock_release(pm->lock); osm_log(pm->log, OSM_LOG_ERROR, - "osm_perfmgr_mad_send_err_callback: ERR 4C15: GUID 0x%016" PRIx64 - " not found in monitored map\n", + "osm_perfmgr_mad_send_err_callback: ERR 4C15: GUID 0x%016" + PRIx64 " not found in monitored map\n", node_guid); goto Exit; } - p_mon_node = (__monitored_node_t *)p_node; + p_mon_node = (__monitored_node_t *) p_node; /* Now, validate port number */ if (port > p_mon_node->redir_tbl_size) { cl_plock_release(pm->lock); osm_log(pm->log, OSM_LOG_ERROR, - "osm_perfmgr_mad_send_err_callback: ERR 4C16: Invalid port num %d for GUID 0x%016" PRIx64 - " num ports %d\n", - port, node_guid, p_mon_node->redir_tbl_size); + "osm_perfmgr_mad_send_err_callback: ERR 4C16: Invalid port num %d for GUID 0x%016" + PRIx64 " num ports %d\n", port, node_guid, + p_mon_node->redir_tbl_size); goto Exit; } /* Clear redirection info */ @@ -195,12 +191,12 @@ osm_perfmgr_mad_send_err_callback(void* bind_context, osm_madw_t *p_madw) cl_plock_release(pm->lock); } - Exit: - osm_mad_pool_put( pm->mad_pool, p_madw ); + Exit: + osm_mad_pool_put(pm->mad_pool, p_madw); __decrement_outstanding_queries(pm); - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** @@ -212,11 +208,11 @@ osm_perfmgr_bind(osm_perfmgr_t * const pm, const ib_net64_t port_guid) osm_bind_info_t bind_info; ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( pm->log, osm_pm_bind ); + OSM_LOG_ENTER(pm->log, osm_pm_bind); - if( pm->bind_handle != OSM_BIND_INVALID_HANDLE ) { - osm_log( pm->log, OSM_LOG_ERROR, - "osm_pm_mad_ctrl_bind: ERR 4C03: Multiple binds not allowed\n" ); + if (pm->bind_handle != OSM_BIND_INVALID_HANDLE) { + osm_log(pm->log, OSM_LOG_ERROR, + "osm_pm_mad_ctrl_bind: ERR 4C03: Multiple binds not allowed\n"); status = IB_ERROR; goto Exit; } @@ -230,67 +226,60 @@ osm_perfmgr_bind(osm_perfmgr_t * const pm, const ib_net64_t port_guid) bind_info.recv_q_size = OSM_PM_DEFAULT_QP1_RCV_SIZE; bind_info.send_q_size = OSM_PM_DEFAULT_QP1_SEND_SIZE; - osm_log( pm->log, OSM_LOG_VERBOSE, - "osm_pm_mad_bind: " - "Binding to port GUID 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); + osm_log(pm->log, OSM_LOG_VERBOSE, + "osm_pm_mad_bind: " + "Binding to port GUID 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); - pm->bind_handle = osm_vendor_bind( pm->vendor, - &bind_info, - pm->mad_pool, - osm_perfmgr_mad_recv_callback, - osm_perfmgr_mad_send_err_callback, - pm ); + pm->bind_handle = osm_vendor_bind(pm->vendor, + &bind_info, + pm->mad_pool, + osm_perfmgr_mad_recv_callback, + osm_perfmgr_mad_send_err_callback, + pm); - if( pm->bind_handle == OSM_BIND_INVALID_HANDLE ) { + if (pm->bind_handle == OSM_BIND_INVALID_HANDLE) { status = IB_ERROR; - osm_log( pm->log, OSM_LOG_ERROR, - "osm_pm_mad_bind: ERR 4C04: Vendor specific bind failed (%s)\n", - ib_get_err_str(status) ); + osm_log(pm->log, OSM_LOG_ERROR, + "osm_pm_mad_bind: ERR 4C04: Vendor specific bind failed (%s)\n", + ib_get_err_str(status)); goto Exit; } -Exit: - OSM_LOG_EXIT( pm->log ); - return( status ); + Exit: + OSM_LOG_EXIT(pm->log); + return (status); } /********************************************************************** * Unbind the PerfMgr from the vendor layer for MAD sends/receives **********************************************************************/ -static void -osm_perfmgr_mad_unbind(osm_perfmgr_t * const pm) +static void osm_perfmgr_mad_unbind(osm_perfmgr_t * const pm) { - OSM_LOG_ENTER( pm->log, osm_sa_mad_ctrl_unbind ); - if( pm->bind_handle == OSM_BIND_INVALID_HANDLE ) { - osm_log( pm->log, OSM_LOG_ERROR, - "osm_pm_mad_unbind: ERR 4C05: No previous bind\n" ); + OSM_LOG_ENTER(pm->log, osm_sa_mad_ctrl_unbind); + if (pm->bind_handle == OSM_BIND_INVALID_HANDLE) { + osm_log(pm->log, OSM_LOG_ERROR, + "osm_pm_mad_unbind: ERR 4C05: No previous bind\n"); goto Exit; } - osm_vendor_unbind( pm->bind_handle ); -Exit: - OSM_LOG_EXIT( pm->log ); + osm_vendor_unbind(pm->bind_handle); + Exit: + OSM_LOG_EXIT(pm->log); } /********************************************************************** * Given a monitored node and a port, return the qp **********************************************************************/ -static ib_net32_t -get_qp(__monitored_node_t *mon_node, uint8_t port) +static ib_net32_t get_qp(__monitored_node_t * mon_node, uint8_t port) { ib_net32_t qp = cl_ntoh32(1); - if (mon_node) - { - if (mon_node->redir_tbl_size) - { - if (port < mon_node->redir_tbl_size) - { - if (mon_node->redir_port[port].redir_lid) - { - if (mon_node->redir_port[port].redir_qp) - { - qp = mon_node->redir_port[port].redir_qp; + if (mon_node) { + if (mon_node->redir_tbl_size) { + if (port < mon_node->redir_tbl_size) { + if (mon_node->redir_port[port].redir_lid) { + if (mon_node->redir_port[port].redir_qp) { + qp = mon_node->redir_port[port]. + redir_qp; } } } @@ -305,36 +294,31 @@ get_qp(__monitored_node_t *mon_node, uint8_t port) * return the appropriate lid to query that port **********************************************************************/ static ib_net16_t -get_lid(osm_node_t *p_node, uint8_t port, __monitored_node_t *mon_node) +get_lid(osm_node_t * p_node, uint8_t port, __monitored_node_t * mon_node) { ib_net16_t lid = 0; - if (mon_node) - { - if (mon_node->redir_tbl_size) - { - if (port < mon_node->redir_tbl_size) - { + if (mon_node) { + if (mon_node->redir_tbl_size) { + if (port < mon_node->redir_tbl_size) { lid = mon_node->redir_port[port].redir_lid; - if (lid) - { + if (lid) { return lid; } } } } - switch (p_node->node_info.node_type) - { - case IB_NODE_TYPE_CA: - case IB_NODE_TYPE_ROUTER: - lid = osm_node_get_base_lid(p_node, port); - break; - case IB_NODE_TYPE_SWITCH: - lid = osm_node_get_base_lid(p_node, 0); - break; - default: - break; + switch (p_node->node_info.node_type) { + case IB_NODE_TYPE_CA: + case IB_NODE_TYPE_ROUTER: + lid = osm_node_get_base_lid(p_node, port); + break; + case IB_NODE_TYPE_SWITCH: + lid = osm_node_get_base_lid(p_node, 0); + break; + default: + break; } return (lid); } @@ -343,18 +327,20 @@ get_lid(osm_node_t *p_node, uint8_t port, __monitored_node_t *mon_node) * Form and send the Port Counters MAD for a single port. **********************************************************************/ static ib_api_status_t -osm_perfmgr_send_pc_mad(osm_perfmgr_t *perfmgr, ib_net16_t dest_lid, +osm_perfmgr_send_pc_mad(osm_perfmgr_t * perfmgr, ib_net16_t dest_lid, ib_net32_t dest_qp, uint8_t port, uint8_t mad_method, - osm_madw_context_t* const p_context) + osm_madw_context_t * const p_context) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; ib_port_counters_t *port_counter = NULL; - ib_perfmgt_mad_t *pm_mad = NULL; - osm_madw_t *p_madw = NULL; + ib_perfmgt_mad_t *pm_mad = NULL; + osm_madw_t *p_madw = NULL; OSM_LOG_ENTER(perfmgr->log, osm_perfmgr_send_pc_mad); - p_madw = osm_mad_pool_get(perfmgr->mad_pool, perfmgr->bind_handle, MAD_BLOCK_SIZE, NULL); + p_madw = + osm_mad_pool_get(perfmgr->mad_pool, perfmgr->bind_handle, + MAD_BLOCK_SIZE, NULL); if (p_madw == NULL) return (IB_INSUFFICIENT_MEMORY); @@ -367,26 +353,28 @@ osm_perfmgr_send_pc_mad(osm_perfmgr_t *perfmgr, ib_net16_t dest_lid, pm_mad->header.method = mad_method; pm_mad->header.status = 0; pm_mad->header.class_spec = 0; - pm_mad->header.trans_id = cl_hton64((uint64_t)cl_atomic_inc(&(perfmgr->trans_id))); + pm_mad->header.trans_id = + cl_hton64((uint64_t) cl_atomic_inc(&(perfmgr->trans_id))); pm_mad->header.attr_id = IB_MAD_ATTR_PORT_CNTRS; pm_mad->header.resv = 0; pm_mad->header.attr_mod = 0; - port_counter = (ib_port_counters_t *)&(pm_mad->data); + port_counter = (ib_port_counters_t *) & (pm_mad->data); memset(port_counter, 0, sizeof(*port_counter)); port_counter->port_select = port; port_counter->counter_select = 0xFFFF; p_madw->mad_addr.dest_lid = dest_lid; p_madw->mad_addr.addr_type.gsi.remote_qp = dest_qp; - p_madw->mad_addr.addr_type.gsi.remote_qkey = cl_hton32(IB_QP1_WELL_KNOWN_Q_KEY); + p_madw->mad_addr.addr_type.gsi.remote_qkey = + cl_hton32(IB_QP1_WELL_KNOWN_Q_KEY); /* FIXME what about other partitions */ p_madw->mad_addr.addr_type.gsi.pkey = cl_hton16(0xFFFF); p_madw->mad_addr.addr_type.gsi.service_level = 0; p_madw->mad_addr.addr_type.gsi.global_route = FALSE; p_madw->resp_expected = TRUE; - if( p_context ) + if (p_context) p_madw->context = *p_context; status = osm_vendor_send(perfmgr->bind_handle, p_madw, TRUE); @@ -395,50 +383,51 @@ osm_perfmgr_send_pc_mad(osm_perfmgr_t *perfmgr, ib_net16_t dest_lid, /* pause this thread if we have too many outstanding requests */ cl_atomic_inc(&(perfmgr->outstanding_queries)); if (perfmgr->outstanding_queries > - perfmgr->max_outstanding_queries) { + perfmgr->max_outstanding_queries) { perfmgr->sweep_state = PERFMGR_SWEEP_SUSPENDED; - cl_event_wait_on( &perfmgr->sig_query, EVENT_NO_TIMEOUT, TRUE ); + cl_event_wait_on(&perfmgr->sig_query, EVENT_NO_TIMEOUT, + TRUE); perfmgr->sweep_state = PERFMGR_SWEEP_ACTIVE; } } OSM_LOG_EXIT(perfmgr->log); - return( status ); + return (status); } /********************************************************************** * sweep the node_guid_tbl and collect the node guids to be tracked **********************************************************************/ -static void -__collect_guids(cl_map_item_t * const p_map_item, void *context) +static void __collect_guids(cl_map_item_t * const p_map_item, void *context) { - osm_node_t *node = (osm_node_t *)p_map_item; - uint64_t node_guid = cl_ntoh64(node->node_info.node_guid); - osm_perfmgr_t *pm = (osm_perfmgr_t *)context; + osm_node_t *node = (osm_node_t *) p_map_item; + uint64_t node_guid = cl_ntoh64(node->node_info.node_guid); + osm_perfmgr_t *pm = (osm_perfmgr_t *) context; __monitored_node_t *mon_node = NULL; - uint32_t size; + uint32_t size; - OSM_LOG_ENTER( pm->log, __collect_guids ); + OSM_LOG_ENTER(pm->log, __collect_guids); if (cl_qmap_get(&(pm->monitored_map), node_guid) - == cl_qmap_end(&(pm->monitored_map))) { + == cl_qmap_end(&(pm->monitored_map))) { /* if not already in our map add it */ size = node->node_info.num_ports; mon_node = malloc(sizeof(*mon_node) + sizeof(redir_t) * size); if (!mon_node) { osm_log(pm->log, OSM_LOG_ERROR, - "PerfMgr: __collect_guids ERR 4C06: malloc failed so not handling node GUID 0x%" PRIx64 "\n", node_guid); + "PerfMgr: __collect_guids ERR 4C06: malloc failed so not handling node GUID 0x%" + PRIx64 "\n", node_guid); goto Exit; } memset(mon_node, 0, sizeof(*mon_node) + sizeof(redir_t) * size); mon_node->guid = node_guid; mon_node->redir_tbl_size = size + 1; cl_qmap_insert(&(pm->monitored_map), node_guid, - (cl_map_item_t *)mon_node); + (cl_map_item_t *) mon_node); } -Exit: - OSM_LOG_EXIT( pm->log ); + Exit: + OSM_LOG_EXIT(pm->log); } /********************************************************************** @@ -447,23 +436,25 @@ Exit: static void __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) { - ib_api_status_t status = IB_SUCCESS; - uint8_t port = 0, startport = 1; - osm_perfmgr_t *pm = (osm_perfmgr_t *)context; - osm_node_t *node = NULL; - __monitored_node_t *mon_node = (__monitored_node_t *)p_map_item; - osm_madw_context_t mad_context; - uint8_t num_ports = 0; - uint64_t node_guid = 0; - ib_net32_t remote_qp; - - OSM_LOG_ENTER( pm->log, __osm_pm_query_counters ); + ib_api_status_t status = IB_SUCCESS; + uint8_t port = 0, startport = 1; + osm_perfmgr_t *pm = (osm_perfmgr_t *) context; + osm_node_t *node = NULL; + __monitored_node_t *mon_node = (__monitored_node_t *) p_map_item; + osm_madw_context_t mad_context; + uint8_t num_ports = 0; + uint64_t node_guid = 0; + ib_net32_t remote_qp; + + OSM_LOG_ENTER(pm->log, __osm_pm_query_counters); cl_plock_acquire(pm->lock); node = osm_get_node_by_guid(pm->subn, cl_hton64(mon_node->guid)); if (!node) { osm_log(pm->log, OSM_LOG_ERROR, - "__osm_pm_query_counters: ERR 4C07: Node guid 0x%" PRIx64 " no longer exists so removing from PerfMgr monitoring\n", + "__osm_pm_query_counters: ERR 4C07: Node guid 0x%" + PRIx64 + " no longer exists so removing from PerfMgr monitoring\n", mon_node->guid); __mark_for_removal(pm, mon_node); goto Exit; @@ -475,39 +466,36 @@ __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) /* make sure we have a database object ready to store this information */ if (perfmgr_db_create_entry(pm->db, node_guid, num_ports + 1, node->print_desc) != - PERFMGR_EVENT_DB_SUCCESS) - { + PERFMGR_EVENT_DB_SUCCESS) { osm_log(pm->log, OSM_LOG_ERROR, - "__osm_pm_query_counters: ERR 4C08: DB create entry failed for 0x%" PRIx64 " (%s) : %s\n", - node_guid, node->print_desc, strerror(errno)); + "__osm_pm_query_counters: ERR 4C08: DB create entry failed for 0x%" + PRIx64 " (%s) : %s\n", node_guid, node->print_desc, + strerror(errno)); goto Exit; } /* if switch, check for enhanced port 0 */ - if (osm_node_get_type(node) == IB_NODE_TYPE_SWITCH) - { - if (node->sw) - { - if (ib_switch_info_is_enhanced_port0(&node->sw->switch_info)) - { + if (osm_node_get_type(node) == IB_NODE_TYPE_SWITCH) { + if (node->sw) { + if (ib_switch_info_is_enhanced_port0 + (&node->sw->switch_info)) { startport = 0; } } } /* issue the query for each port */ - for (port = startport; port < num_ports; port++) - { + for (port = startport; port < num_ports; port++) { ib_net16_t lid; if (!osm_physp_is_valid(osm_node_get_physp_ptr(node, port))) continue; lid = get_lid(node, port, mon_node); - if (lid == 0) - { + if (lid == 0) { osm_log(pm->log, OSM_LOG_DEBUG, - "__osm_pm_query_counters: WARN: node 0x%" PRIx64 " port %d (%s): port out of range, skipping\n", + "__osm_pm_query_counters: WARN: node 0x%" PRIx64 + " port %d (%s): port out of range, skipping\n", cl_ntoh64(node->node_info.node_guid), port, node->print_desc); continue; @@ -522,46 +510,47 @@ __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) gettimeofday(&(mad_context.perfmgr_context.query_start), NULL); #endif osm_log(pm->log, OSM_LOG_VERBOSE, - "__osm_pm_query_counters: Getting stats for node 0x%" PRIx64 " port %d (lid %X) (%s)\n", - node_guid, port, cl_ntoh16(lid), - node->print_desc); - status = osm_perfmgr_send_pc_mad(pm, lid, remote_qp, port, IB_MAD_METHOD_GET, &mad_context); - if (status != IB_SUCCESS) - { - osm_log(pm->log, OSM_LOG_ERROR, - "__osm_pm_query_counters: ERR 4C09: Failed to issue port counter query for node 0x%" PRIx64 " port %d (%s)\n", + "__osm_pm_query_counters: Getting stats for node 0x%" + PRIx64 " port %d (lid %X) (%s)\n", node_guid, port, + cl_ntoh16(lid), node->print_desc); + status = + osm_perfmgr_send_pc_mad(pm, lid, remote_qp, port, + IB_MAD_METHOD_GET, &mad_context); + if (status != IB_SUCCESS) { + osm_log(pm->log, OSM_LOG_ERROR, + "__osm_pm_query_counters: ERR 4C09: Failed to issue port counter query for node 0x%" + PRIx64 " port %d (%s)\n", node->node_info.node_guid, port, node->print_desc); } } -Exit: + Exit: cl_plock_release(pm->lock); - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** * Main PerfMgr Thread. * Loop continuously and query the performance counters. **********************************************************************/ -void -__osm_perfmgr_sweeper(void *p_ptr) +void __osm_perfmgr_sweeper(void *p_ptr) { ib_api_status_t status; - osm_perfmgr_t *const pm = ( osm_perfmgr_t * ) p_ptr; + osm_perfmgr_t *const pm = (osm_perfmgr_t *) p_ptr; - OSM_LOG_ENTER( pm->log, __osm_pm_sweeper ); + OSM_LOG_ENTER(pm->log, __osm_pm_sweeper); - if( pm->thread_state == OSM_THREAD_STATE_INIT ) + if (pm->thread_state == OSM_THREAD_STATE_INIT) pm->thread_state = OSM_THREAD_STATE_RUN; __init_monitored_nodes(pm); - while( pm->thread_state == OSM_THREAD_STATE_RUN ) { + while (pm->thread_state == OSM_THREAD_STATE_RUN) { /* do the sweep only if in MASTER state * AND we have been activated. * FIXME put something in here to try and reduce the load on the system * when it is not IDLE. - if (pm->sm->state_mgr.state != OSM_SM_STATE_IDLE) + if (pm->sm->state_mgr.state != OSM_SM_STATE_IDLE) */ if (pm->subn->sm_state == IB_SMINFO_STATE_MASTER && pm->state == PERFMGR_STATE_ENABLED) { @@ -574,7 +563,8 @@ __osm_perfmgr_sweeper(void *p_ptr) /* FIXME we should be able to track SA notices * and not have to sweep the node_guid_tbl each pass */ - osm_log(pm->log, OSM_LOG_VERBOSE, "Gathering PerfMgr stats\n"); + osm_log(pm->log, OSM_LOG_VERBOSE, + "Gathering PerfMgr stats\n"); cl_plock_acquire(pm->lock); cl_qmap_apply_func(&(pm->subn->node_guid_tbl), __collect_guids, (void *)pm); @@ -600,32 +590,30 @@ __osm_perfmgr_sweeper(void *p_ptr) pm->sweep_state = PERFMGR_SWEEP_SLEEP; /* Wait for a forced sweep or period timeout. */ - status = cl_event_wait_on( &pm->sig_sweep, - pm->sweep_time_s * 1000000, TRUE ); + status = cl_event_wait_on(&pm->sig_sweep, + pm->sweep_time_s * 1000000, TRUE); } - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** **********************************************************************/ -void -osm_perfmgr_shutdown(osm_perfmgr_t * const pm) +void osm_perfmgr_shutdown(osm_perfmgr_t * const pm) { - OSM_LOG_ENTER( pm->log, osm_perfmgr_shutdown ); + OSM_LOG_ENTER(pm->log, osm_perfmgr_shutdown); osm_perfmgr_mad_unbind(pm); - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** **********************************************************************/ -void -osm_perfmgr_destroy(osm_perfmgr_t * const pm) +void osm_perfmgr_destroy(osm_perfmgr_t * const pm) { - OSM_LOG_ENTER( pm->log, osm_perfmgr_destroy ); + OSM_LOG_ENTER(pm->log, osm_perfmgr_destroy); free(pm->event_db_dump_file); perfmgr_db_destroy(pm->db); - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** @@ -638,19 +626,18 @@ osm_perfmgr_destroy(osm_perfmgr_t * const pm) * will be missed. **********************************************************************/ static void -osm_perfmgr_check_oob_clear(osm_perfmgr_t *pm, uint64_t node_guid, uint8_t port, - perfmgr_db_err_reading_t *cr, - perfmgr_db_data_cnt_reading_t *dc) +osm_perfmgr_check_oob_clear(osm_perfmgr_t * pm, uint64_t node_guid, + uint8_t port, perfmgr_db_err_reading_t * cr, + perfmgr_db_data_cnt_reading_t * dc) { - perfmgr_db_err_reading_t prev_err; - perfmgr_db_data_cnt_reading_t prev_dc; + perfmgr_db_err_reading_t prev_err; + perfmgr_db_data_cnt_reading_t prev_dc; if (perfmgr_db_get_prev_err(pm->db, node_guid, port, &prev_err) - != PERFMGR_EVENT_DB_SUCCESS) - { + != PERFMGR_EVENT_DB_SUCCESS) { osm_log(pm->log, OSM_LOG_VERBOSE, - "osm_perfmgr_check_oob_clear: Failed to find previous error reading for 0x%" PRIx64 " port %u\n", - node_guid, port); + "osm_perfmgr_check_oob_clear: Failed to find previous error reading for 0x%" + PRIx64 " port %u\n", node_guid, port); return; } @@ -665,34 +652,29 @@ osm_perfmgr_check_oob_clear(osm_perfmgr_t *pm, uint64_t node_guid, uint8_t port, cr->rcv_constraint_err < prev_err.rcv_constraint_err || cr->link_integrity < prev_err.link_integrity || cr->buffer_overrun < prev_err.buffer_overrun || - cr->vl15_dropped < prev_err.vl15_dropped) - { + cr->vl15_dropped < prev_err.vl15_dropped) { osm_log(pm->log, OSM_LOG_ERROR, - "PerfMgr: ERR 4C0A: Detected an out of band error clear on node 0x%" PRIx64 " port %u\n", - node_guid, port - ); + "PerfMgr: ERR 4C0A: Detected an out of band error clear on node 0x%" + PRIx64 " port %u\n", node_guid, port); perfmgr_db_clear_prev_err(pm->db, node_guid, port); } /* FIXME handle extended counters */ if (perfmgr_db_get_prev_dc(pm->db, node_guid, port, &prev_dc) - != PERFMGR_EVENT_DB_SUCCESS) - { + != PERFMGR_EVENT_DB_SUCCESS) { osm_log(pm->log, OSM_LOG_VERBOSE, - "osm_perfmgr_check_oob_clear: Failed to find previous data count reading for 0x%" PRIx64 " port %u\n", - node_guid, port); + "osm_perfmgr_check_oob_clear: Failed to find previous data count reading for 0x%" + PRIx64 " port %u\n", node_guid, port); return; } if (dc->xmit_data < prev_dc.xmit_data || dc->rcv_data < prev_dc.rcv_data || dc->xmit_pkts < prev_dc.xmit_pkts || - dc->rcv_pkts < prev_dc.rcv_pkts) - { + dc->rcv_pkts < prev_dc.rcv_pkts) { osm_log(pm->log, OSM_LOG_ERROR, - "PerfMgr: ERR 4C0B: Detected an out of band data counter clear on node 0x%" PRIx64 " port %u\n", - node_guid, port - ); + "PerfMgr: ERR 4C0B: Detected an out of band data counter clear on node 0x%" + PRIx64 " port %u\n", node_guid, port); perfmgr_db_clear_prev_dc(pm->db, node_guid, port); } } @@ -704,17 +686,20 @@ int counter_overflow_4(uint8_t val) { return (val >= 10); } + int counter_overflow_8(uint8_t val) { - return (val >= (UINT8_MAX - (UINT8_MAX/4))); + return (val >= (UINT8_MAX - (UINT8_MAX / 4))); } + int counter_overflow_16(ib_net16_t val) { - return (cl_ntoh16(val) >= (UINT16_MAX - (UINT16_MAX/4))); + return (cl_ntoh16(val) >= (UINT16_MAX - (UINT16_MAX / 4))); } + int counter_overflow_32(ib_net32_t val) { - return (cl_ntoh32(val) >= (UINT32_MAX - (UINT32_MAX/4))); + return (cl_ntoh32(val) >= (UINT32_MAX - (UINT32_MAX / 4))); } /********************************************************************** @@ -722,14 +707,14 @@ int counter_overflow_32(ib_net32_t val) * MAD to the port. **********************************************************************/ static void -osm_perfmgr_check_overflow(osm_perfmgr_t *pm, uint64_t node_guid, - uint8_t port, ib_port_counters_t *pc) +osm_perfmgr_check_overflow(osm_perfmgr_t * pm, uint64_t node_guid, + uint8_t port, ib_port_counters_t * pc) { osm_madw_context_t mad_context; ib_api_status_t status; ib_net32_t remote_qp; - OSM_LOG_ENTER( pm->log, osm_perfmgr_check_overflow ); + OSM_LOG_ENTER(pm->log, osm_perfmgr_check_overflow); if (counter_overflow_16(pc->symbol_err_cnt) || counter_overflow_8(pc->link_err_recover) || @@ -746,14 +731,13 @@ osm_perfmgr_check_overflow(osm_perfmgr_t *pm, uint64_t node_guid, counter_overflow_32(pc->xmit_data) || counter_overflow_32(pc->rcv_data) || counter_overflow_32(pc->xmit_pkts) || - counter_overflow_32(pc->rcv_pkts)) - { + counter_overflow_32(pc->rcv_pkts)) { osm_node_t *p_node = NULL; - ib_net16_t lid = 0; + ib_net16_t lid = 0; osm_log(pm->log, OSM_LOG_INFO, - "PerfMgr: Counter overflow: 0x%" PRIx64 " port %d; clearing counters\n", - node_guid, port); + "PerfMgr: Counter overflow: 0x%" PRIx64 + " port %d; clearing counters\n", node_guid, port); cl_plock_acquire(pm->lock); p_node = osm_get_node_by_guid(pm->subn, cl_hton64(node_guid)); @@ -761,10 +745,10 @@ osm_perfmgr_check_overflow(osm_perfmgr_t *pm, uint64_t node_guid, /* potentially redoing redirection */ lid = get_lid(p_node, port, NULL); cl_plock_release(pm->lock); - if (lid == 0) - { + if (lid == 0) { osm_log(pm->log, OSM_LOG_ERROR, - "PerfMgr: ERR 4C0C: Failed to clear counters for node 0x%" PRIx64 " port %d; failed to get lid\n", + "PerfMgr: ERR 4C0C: Failed to clear counters for node 0x%" + PRIx64 " port %d; failed to get lid\n", node_guid, port); goto Exit; } @@ -775,37 +759,38 @@ osm_perfmgr_check_overflow(osm_perfmgr_t *pm, uint64_t node_guid, mad_context.perfmgr_context.port = port; mad_context.perfmgr_context.mad_method = IB_MAD_METHOD_SET; /* clear port counters */ - status = osm_perfmgr_send_pc_mad(pm, lid, remote_qp, port, IB_MAD_METHOD_SET, &mad_context); - if (status != IB_SUCCESS) - { + status = + osm_perfmgr_send_pc_mad(pm, lid, remote_qp, port, + IB_MAD_METHOD_SET, &mad_context); + if (status != IB_SUCCESS) { osm_log(pm->log, OSM_LOG_ERROR, - "PerfMgr: ERR 4C11: Failed to send clear counters MAD for node 0x%" PRIx64 " port %d\n", - node_guid, port); + "PerfMgr: ERR 4C11: Failed to send clear counters MAD for node 0x%" + PRIx64 " port %d\n", node_guid, port); } perfmgr_db_clear_prev_dc(pm->db, node_guid, port); } -Exit: - OSM_LOG_EXIT( pm->log ); + Exit: + OSM_LOG_EXIT(pm->log); } /********************************************************************** * Check values for logging of errors **********************************************************************/ static void -osm_perfmgr_log_events(osm_perfmgr_t *pm, uint64_t node_guid, uint8_t port, - perfmgr_db_err_reading_t *reading) +osm_perfmgr_log_events(osm_perfmgr_t * pm, uint64_t node_guid, uint8_t port, + perfmgr_db_err_reading_t * reading) { - perfmgr_db_err_reading_t prev_read; - time_t time_diff = 0; - perfmgr_db_err_t err = perfmgr_db_get_prev_err(pm->db, node_guid, port, &prev_read); + perfmgr_db_err_reading_t prev_read; + time_t time_diff = 0; + perfmgr_db_err_t err = + perfmgr_db_get_prev_err(pm->db, node_guid, port, &prev_read); - if (err != PERFMGR_EVENT_DB_SUCCESS) - { + if (err != PERFMGR_EVENT_DB_SUCCESS) { osm_log(pm->log, OSM_LOG_VERBOSE, - "osm_perfmgr_log_events: Failed to find previous reading for 0x%" PRIx64 " port %u\n", - node_guid, port); + "osm_perfmgr_log_events: Failed to find previous reading for 0x%" + PRIx64 " port %u\n", node_guid, port); return; } time_diff = (reading->time - prev_read.time); @@ -815,31 +800,28 @@ osm_perfmgr_log_events(osm_perfmgr_t *pm, uint64_t node_guid, uint8_t port, if (reading->symbol_err_cnt > prev_read.symbol_err_cnt) { osm_log(pm->log, OSM_LOG_ERROR, "osm_perfmgr_log_events: ERR 4C0D: " - "Found %"PRIu64" Symbol errors in %lu sec on node 0x%" PRIx64 " port %u\n", + "Found %" PRIu64 " Symbol errors in %lu sec on node 0x%" + PRIx64 " port %u\n", (reading->symbol_err_cnt - prev_read.symbol_err_cnt), - time_diff, - node_guid, - port); + time_diff, node_guid, port); } if (reading->rcv_err > prev_read.rcv_err) { osm_log(pm->log, OSM_LOG_ERROR, "osm_perfmgr_log_events: ERR 4C0E: " - "Found %"PRIu64" Receive errors in %lu sec on node 0x%" PRIx64 " port %u\n", - (reading->rcv_err - prev_read.rcv_err), - time_diff, - node_guid, - port); + "Found %" PRIu64 + " Receive errors in %lu sec on node 0x%" PRIx64 + " port %u\n", (reading->rcv_err - prev_read.rcv_err), + time_diff, node_guid, port); } if (reading->xmit_discards > prev_read.xmit_discards) { osm_log(pm->log, OSM_LOG_ERROR, "osm_perfmgr_log_events: ERR 4C0F: " - "Found %"PRIu64" Xmit Discards in %lu sec on node 0x%" PRIx64 " port %u\n", + "Found %" PRIu64 " Xmit Discards in %lu sec on node 0x%" + PRIx64 " port %u\n", (reading->xmit_discards - prev_read.xmit_discards), - time_diff, - node_guid, - port); + time_diff, node_guid, port); } } @@ -847,35 +829,38 @@ osm_perfmgr_log_events(osm_perfmgr_t *pm, uint64_t node_guid, uint8_t port, * The dispatcher uses a thread pool which will call this function when * we have a thread available to process our mad received from the wire. **********************************************************************/ -static void -osm_pc_rcv_process(void *context, void *data) +static void osm_pc_rcv_process(void *context, void *data) { - osm_perfmgr_t *const pm = (osm_perfmgr_t *)context; - osm_madw_t *p_madw = (osm_madw_t *)data; + osm_perfmgr_t *const pm = (osm_perfmgr_t *) context; + osm_madw_t *p_madw = (osm_madw_t *) data; osm_madw_context_t *mad_context = &(p_madw->context); - ib_port_counters_t *wire_read = (ib_port_counters_t *)&(osm_madw_get_perfmgt_mad_ptr(p_madw)->data); - ib_mad_t *p_mad = osm_madw_get_mad_ptr(p_madw); - uint64_t node_guid = mad_context->perfmgr_context.node_guid; - uint8_t port = mad_context->perfmgr_context.port; - perfmgr_db_err_reading_t err_reading; + ib_port_counters_t *wire_read = + (ib_port_counters_t *) & (osm_madw_get_perfmgt_mad_ptr(p_madw)-> + data); + ib_mad_t *p_mad = osm_madw_get_mad_ptr(p_madw); + uint64_t node_guid = mad_context->perfmgr_context.node_guid; + uint8_t port = mad_context->perfmgr_context.port; + perfmgr_db_err_reading_t err_reading; perfmgr_db_data_cnt_reading_t data_reading; - OSM_LOG_ENTER( pm->log, osm_pc_rcv_process ); + OSM_LOG_ENTER(pm->log, osm_pc_rcv_process); osm_log(pm->log, OSM_LOG_VERBOSE, - "osm_pc_rcv_process: Processing received MAD status 0x%x context 0x%" PRIx64 " port %u\n", - p_mad->status, node_guid, port); + "osm_pc_rcv_process: Processing received MAD status 0x%x context 0x%" + PRIx64 " port %u\n", p_mad->status, node_guid, port); /* Response could also be redirection (IBM eHCA PMA does this) */ if (p_mad->attr_id == IB_MAD_ATTR_CLASS_PORT_INFO) { - ib_class_port_info_t *cpi = (ib_class_port_info_t *)&(osm_madw_get_perfmgt_mad_ptr(p_madw)->data); + ib_class_port_info_t *cpi = + (ib_class_port_info_t *) & + (osm_madw_get_perfmgt_mad_ptr(p_madw)->data); cl_map_item_t *p_node; __monitored_node_t *p_mon_node; ib_api_status_t status; osm_log(pm->log, OSM_LOG_VERBOSE, - "osm_pc_rcv_process: Redirection to LID 0x%x " + "osm_pc_rcv_process: Redirection to LID 0x%x " "GID 0x%016" PRIx64 " : 0x%016" PRIx64 " QP 0x%x received\n", cl_ntoh16(cpi->redir_lid), @@ -901,38 +886,43 @@ osm_pc_rcv_process(void *context, void *data) cl_qmap_end(&(pm->monitored_map))) { cl_plock_release(pm->lock); osm_log(pm->log, OSM_LOG_ERROR, - "osm_pc_rcv_process: ERR 4C12: GUID 0x%016" PRIx64 - " not found in monitored map\n", + "osm_pc_rcv_process: ERR 4C12: GUID 0x%016" + PRIx64 " not found in monitored map\n", node_guid); goto Exit; } - p_mon_node = (__monitored_node_t *)p_node; + p_mon_node = (__monitored_node_t *) p_node; /* Now, validate port number */ if (port > p_mon_node->redir_tbl_size) { cl_plock_release(pm->lock); osm_log(pm->log, OSM_LOG_ERROR, - "osm_pc_rcv_process: ERR 4C13: Invalid port num %d for GUID 0x%016" PRIx64 - " num ports %d\n", - port, node_guid, p_mon_node->redir_tbl_size); + "osm_pc_rcv_process: ERR 4C13: Invalid port num %d for GUID 0x%016" + PRIx64 " num ports %d\n", port, node_guid, + p_mon_node->redir_tbl_size); goto Exit; } p_mon_node->redir_port[port].redir_lid = cpi->redir_lid; p_mon_node->redir_port[port].redir_qp = cpi->redir_qp; cl_plock_release(pm->lock); - ReIssue: + ReIssue: /* Finally, reissue the query to the redirected location */ - status = osm_perfmgr_send_pc_mad(pm, cpi->redir_lid, cpi->redir_qp, port, mad_context->perfmgr_context.mad_method, mad_context); - if (status != IB_SUCCESS) - { + status = + osm_perfmgr_send_pc_mad(pm, cpi->redir_lid, cpi->redir_qp, + port, + mad_context->perfmgr_context. + mad_method, mad_context); + if (status != IB_SUCCESS) { osm_log(pm->log, OSM_LOG_ERROR, - "osm_pc_rcv_process: ERR 4C14: Failed to send redirected MAD with method 0x%x for node 0x%" PRIx64 " port %d\n", - mad_context->perfmgr_context.mad_method, node_guid, port); + "osm_pc_rcv_process: ERR 4C14: Failed to send redirected MAD with method 0x%x for node 0x%" + PRIx64 " port %d\n", + mad_context->perfmgr_context.mad_method, + node_guid, port); } goto Exit; } - CL_ASSERT( p_mad->attr_id == IB_MAD_ATTR_PORT_CNTRS ); + CL_ASSERT(p_mad->attr_id == IB_MAD_ATTR_PORT_CNTRS); perfmgr_db_fill_err_read(wire_read, &err_reading); /* FIXME separate query for extended counters if they are supported @@ -949,8 +939,10 @@ osm_pc_rcv_process(void *context, void *data) osm_perfmgr_log_events(pm, node_guid, port, &err_reading); if (mad_context->perfmgr_context.mad_method == IB_MAD_METHOD_GET) { - perfmgr_db_add_err_reading(pm->db, node_guid, port, &err_reading); - perfmgr_db_add_dc_reading(pm->db, node_guid, port, &data_reading); + perfmgr_db_add_err_reading(pm->db, node_guid, port, + &err_reading); + perfmgr_db_add_dc_reading(pm->db, node_guid, port, + &data_reading); } else { perfmgr_db_clear_prev_err(pm->db, node_guid, port); perfmgr_db_clear_prev_dc(pm->db, node_guid, port); @@ -960,7 +952,7 @@ osm_pc_rcv_process(void *context, void *data) #if 0 do { - struct timeval proc_time; + struct timeval proc_time; gettimeofday(&proc_time, NULL); osm_log(pm->log, OSM_LOG_INFO, "PerfMgr done: processing time %ld\n", @@ -969,36 +961,34 @@ osm_pc_rcv_process(void *context, void *data) } while (0); #endif - Exit: - osm_mad_pool_put( pm->mad_pool, p_madw ); + Exit: + osm_mad_pool_put(pm->mad_pool, p_madw); - OSM_LOG_EXIT( pm->log ); + OSM_LOG_EXIT(pm->log); } /********************************************************************** * Initialize the PerfMgr object **********************************************************************/ ib_api_status_t -osm_perfmgr_init( - osm_perfmgr_t * const pm, - osm_subn_t * const subn, - osm_sm_t * const sm, - osm_log_t * const log, - osm_mad_pool_t * const mad_pool, - osm_vendor_t * const vendor, - cl_dispatcher_t* const disp, - cl_plock_t* const lock, - const osm_subn_opt_t * const p_opt, - osm_epi_plugin_t *event_plugin - ) +osm_perfmgr_init(osm_perfmgr_t * const pm, + osm_subn_t * const subn, + osm_sm_t * const sm, + osm_log_t * const log, + osm_mad_pool_t * const mad_pool, + osm_vendor_t * const vendor, + cl_dispatcher_t * const disp, + cl_plock_t * const lock, + const osm_subn_opt_t * const p_opt, + osm_epi_plugin_t * event_plugin) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( log, osm_pm_init ); + OSM_LOG_ENTER(log, osm_pm_init); osm_log(log, OSM_LOG_VERBOSE, "Initializing PerfMgr\n"); - memset( pm, 0, sizeof( *pm ) ); + memset(pm, 0, sizeof(*pm)); cl_event_construct(&pm->sig_sweep); cl_event_init(&pm->sig_sweep, FALSE); @@ -1009,60 +999,57 @@ osm_perfmgr_init( pm->vendor = vendor; pm->trans_id = OSM_PERFMGR_INITIAL_TID_VALUE; pm->lock = lock; - pm->state = p_opt->perfmgr ? PERFMGR_STATE_ENABLED : PERFMGR_STATE_DISABLE; + pm->state = + p_opt->perfmgr ? PERFMGR_STATE_ENABLED : PERFMGR_STATE_DISABLE; pm->sweep_time_s = p_opt->perfmgr_sweep_time_s; pm->event_db_dump_file = strdup(p_opt->event_db_dump_file); pm->max_outstanding_queries = p_opt->perfmgr_max_outstanding_queries; pm->event_plugin = event_plugin; pm->db = perfmgr_db_construct(pm->log, pm->event_plugin); - if (!pm->db) - { - pm->state = PERFMGR_STATE_NO_DB; - goto Exit; + if (!pm->db) { + pm->state = PERFMGR_STATE_NO_DB; + goto Exit; } pm->pc_disp_h = cl_disp_register(disp, OSM_MSG_MAD_PORT_COUNTERS, - osm_pc_rcv_process, pm); - if( pm->pc_disp_h == CL_DISP_INVALID_HANDLE ) + osm_pc_rcv_process, pm); + if (pm->pc_disp_h == CL_DISP_INVALID_HANDLE) goto Exit; pm->thread_state = OSM_THREAD_STATE_INIT; - status = cl_thread_init( &pm->sweeper, __osm_perfmgr_sweeper, pm, - "PerfMgr sweeper" ); - if( status != IB_SUCCESS ) + status = cl_thread_init(&pm->sweeper, __osm_perfmgr_sweeper, pm, + "PerfMgr sweeper"); + if (status != IB_SUCCESS) goto Exit; -Exit: - OSM_LOG_EXIT( log ); - return ( status ); + Exit: + OSM_LOG_EXIT(log); + return (status); } /********************************************************************** * Clear the counters from the db **********************************************************************/ -void -osm_perfmgr_clear_counters(osm_perfmgr_t *pm) +void osm_perfmgr_clear_counters(osm_perfmgr_t * pm) { /** * FIXME todo issue clear on the fabric? */ perfmgr_db_clear_counters(pm->db); - osm_log( pm->log, OSM_LOG_INFO, "PerfMgr counters cleared\n"); + osm_log(pm->log, OSM_LOG_INFO, "PerfMgr counters cleared\n"); } /******************************************************************* * Have the DB dump its information to the file specified *******************************************************************/ -void -osm_perfmgr_dump_counters(osm_perfmgr_t *pm, perfmgr_db_dump_t dump_type) +void osm_perfmgr_dump_counters(osm_perfmgr_t * pm, perfmgr_db_dump_t dump_type) { - if (perfmgr_db_dump(pm->db, pm->event_db_dump_file, dump_type) != 0) - { - osm_log( pm->log, OSM_LOG_ERROR, + if (perfmgr_db_dump(pm->db, pm->event_db_dump_file, dump_type) != 0) { + osm_log(pm->log, OSM_LOG_ERROR, "PB dump port counters: ERR 4C10: Failed to dump file %s : %s", pm->event_db_dump_file, strerror(errno)); } } -#endif /* ENABLE_OSM_PERF_MGR */ +#endif /* ENABLE_OSM_PERF_MGR */ diff --git a/opensm/opensm/osm_perfmgr_db.c b/opensm/opensm/osm_perfmgr_db.c index 4c82673..25d32ff 100644 --- a/opensm/opensm/osm_perfmgr_db.c +++ b/opensm/opensm/osm_perfmgr_db.c @@ -31,10 +31,9 @@ * */ - #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -46,9 +45,8 @@ /** ========================================================================= */ -perfmgr_db_t * -perfmgr_db_construct(osm_log_t *p_log, - osm_epi_plugin_t *event_plugin) +perfmgr_db_t *perfmgr_db_construct(osm_log_t * p_log, + osm_epi_plugin_t * event_plugin) { perfmgr_db_t *db = malloc(sizeof(*db)); if (!db) @@ -64,11 +62,9 @@ perfmgr_db_construct(osm_log_t *p_log, /** ========================================================================= */ -void -perfmgr_db_destroy(perfmgr_db_t *db) +void perfmgr_db_destroy(perfmgr_db_t * db) { - if (db) - { + if (db) { cl_plock_destroy(&(db->lock)); free(db); } @@ -77,19 +73,17 @@ perfmgr_db_destroy(perfmgr_db_t *db) /********************************************************************** * Internal call db->lock should be held when calling **********************************************************************/ -static inline _db_node_t * -_get(perfmgr_db_t *db, uint64_t guid) +static inline _db_node_t *_get(perfmgr_db_t * db, uint64_t guid) { - cl_map_item_t *rc = cl_qmap_get(&(db->pc_data), guid); + cl_map_item_t *rc = cl_qmap_get(&(db->pc_data), guid); const cl_map_item_t *end = cl_qmap_end(&(db->pc_data)); if (rc == end) return (NULL); - return ((_db_node_t *)rc); + return ((_db_node_t *) rc); } -static inline perfmgr_db_err_t -bad_node_port(_db_node_t *node, uint8_t port) +static inline perfmgr_db_err_t bad_node_port(_db_node_t * node, uint8_t port) { if (!node) return (PERFMGR_EVENT_DB_GUIDNOTFOUND); @@ -100,11 +94,10 @@ bad_node_port(_db_node_t *node, uint8_t port) /** ========================================================================= */ -static _db_node_t * -__malloc_node(uint64_t guid, uint8_t num_ports, char *name) +static _db_node_t *__malloc_node(uint64_t guid, uint8_t num_ports, char *name) { - int i = 0; - time_t cur_time = 0; + int i = 0; + time_t cur_time = 0; _db_node_t *rc = malloc(sizeof(*rc)); if (!rc) return (NULL); @@ -125,15 +118,14 @@ __malloc_node(uint64_t guid, uint8_t num_ports, char *name) return (rc); -free_rc: + free_rc: free(rc); return (NULL); } /** ========================================================================= */ -static void -__free_node(_db_node_t *node) +static void __free_node(_db_node_t * node) { if (!node) return; @@ -143,10 +135,11 @@ __free_node(_db_node_t *node) } /* insert nodes to the database */ -static perfmgr_db_err_t -__insert(perfmgr_db_t *db, _db_node_t *node) +static perfmgr_db_err_t __insert(perfmgr_db_t * db, _db_node_t * node) { - cl_map_item_t *rc = cl_qmap_insert(&(db->pc_data), node->node_guid, (cl_map_item_t *)node); + cl_map_item_t *rc = + cl_qmap_insert(&(db->pc_data), node->node_guid, + (cl_map_item_t *) node); if ((void *)rc != (void *)node) return (PERFMGR_EVENT_DB_FAIL); @@ -156,10 +149,10 @@ __insert(perfmgr_db_t *db, _db_node_t *node) /********************************************************************** **********************************************************************/ perfmgr_db_err_t -perfmgr_db_create_entry(perfmgr_db_t *db, uint64_t guid, +perfmgr_db_create_entry(perfmgr_db_t * db, uint64_t guid, uint8_t num_ports, char *name) { - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; cl_plock_excl_acquire(&(db->lock)); if (!_get(db, guid)) { @@ -174,7 +167,7 @@ perfmgr_db_create_entry(perfmgr_db_t *db, uint64_t guid, goto Exit; } } -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } @@ -183,64 +176,77 @@ Exit: * Dump a reading vs the previous reading to stdout **********************************************************************/ static inline void -debug_dump_err_reading(perfmgr_db_t *db, uint64_t guid, uint8_t port_num, - _db_port_t *port, perfmgr_db_err_reading_t *cur) +debug_dump_err_reading(perfmgr_db_t * db, uint64_t guid, uint8_t port_num, + _db_port_t * port, perfmgr_db_err_reading_t * cur) { if (!osm_log_is_active(db->osm_log, OSM_LOG_DEBUG)) - return; /* optimize this a bit */ + return; /* optimize this a bit */ osm_log(db->osm_log, OSM_LOG_DEBUG, - "GUID 0x%" PRIx64 " Port %u:\n", guid, port_num); + "GUID 0x%" PRIx64 " Port %u:\n", guid, port_num); osm_log(db->osm_log, OSM_LOG_DEBUG, - "sym %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->symbol_err_cnt, - port->err_previous.symbol_err_cnt, port->err_total.symbol_err_cnt); + "sym %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->symbol_err_cnt, port->err_previous.symbol_err_cnt, + port->err_total.symbol_err_cnt); osm_log(db->osm_log, OSM_LOG_DEBUG, - "ler %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->link_err_recover, - port->err_previous.link_err_recover, port->err_total.link_err_recover); + "ler %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->link_err_recover, port->err_previous.link_err_recover, + port->err_total.link_err_recover); osm_log(db->osm_log, OSM_LOG_DEBUG, - "ld %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->link_downed, - port->err_previous.link_downed, port->err_total.link_downed); + "ld %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->link_downed, port->err_previous.link_downed, + port->err_total.link_downed); osm_log(db->osm_log, OSM_LOG_DEBUG, - "re %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_err, + "re %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", cur->rcv_err, port->err_previous.rcv_err, port->err_total.rcv_err); osm_log(db->osm_log, OSM_LOG_DEBUG, - "rrp %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_rem_phys_err, - port->err_previous.rcv_rem_phys_err, port->err_total.rcv_rem_phys_err); + "rrp %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->rcv_rem_phys_err, port->err_previous.rcv_rem_phys_err, + port->err_total.rcv_rem_phys_err); osm_log(db->osm_log, OSM_LOG_DEBUG, - "rsr %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_switch_relay_err, - port->err_previous.rcv_switch_relay_err, port->err_total.rcv_switch_relay_err); + "rsr %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->rcv_switch_relay_err, + port->err_previous.rcv_switch_relay_err, + port->err_total.rcv_switch_relay_err); osm_log(db->osm_log, OSM_LOG_DEBUG, - "xd %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->xmit_discards, - port->err_previous.xmit_discards, port->err_total.xmit_discards); + "xd %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->xmit_discards, port->err_previous.xmit_discards, + port->err_total.xmit_discards); osm_log(db->osm_log, OSM_LOG_DEBUG, - "xce %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->xmit_constraint_err, - port->err_previous.xmit_constraint_err, port->err_total.xmit_constraint_err); + "xce %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->xmit_constraint_err, + port->err_previous.xmit_constraint_err, + port->err_total.xmit_constraint_err); osm_log(db->osm_log, OSM_LOG_DEBUG, - "rce %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_constraint_err, - port->err_previous.rcv_constraint_err, port->err_total.rcv_constraint_err); + "rce %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->rcv_constraint_err, port->err_previous.rcv_constraint_err, + port->err_total.rcv_constraint_err); osm_log(db->osm_log, OSM_LOG_DEBUG, - "li %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->link_integrity, - port->err_previous.link_integrity, port->err_total.link_integrity); + "li %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->link_integrity, port->err_previous.link_integrity, + port->err_total.link_integrity); osm_log(db->osm_log, OSM_LOG_DEBUG, - "bo %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->buffer_overrun, - port->err_previous.buffer_overrun, port->err_total.buffer_overrun); + "bo %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->buffer_overrun, port->err_previous.buffer_overrun, + port->err_total.buffer_overrun); osm_log(db->osm_log, OSM_LOG_DEBUG, - "vld %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->vl15_dropped, - port->err_previous.vl15_dropped, port->err_total.vl15_dropped); + "vld %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->vl15_dropped, port->err_previous.vl15_dropped, + port->err_total.vl15_dropped); } /********************************************************************** * perfmgr_db_err_reading_t functions **********************************************************************/ perfmgr_db_err_t -perfmgr_db_add_err_reading(perfmgr_db_t *db, uint64_t guid, - uint8_t port, perfmgr_db_err_reading_t *reading) +perfmgr_db_add_err_reading(perfmgr_db_t * db, uint64_t guid, + uint8_t port, perfmgr_db_err_reading_t * reading) { - _db_port_t *p_port = NULL; - _db_node_t *node = NULL; - perfmgr_db_err_reading_t *previous = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; - osm_epi_pe_event_t epi_pe_data; + _db_port_t *p_port = NULL; + _db_node_t *node = NULL; + perfmgr_db_err_reading_t *previous = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + osm_epi_pe_event_t epi_pe_data; cl_plock_excl_acquire(&(db->lock)); node = _get(db, guid); @@ -253,50 +259,64 @@ perfmgr_db_add_err_reading(perfmgr_db_t *db, uint64_t guid, debug_dump_err_reading(db, guid, port, p_port, reading); epi_pe_data.time_diff_s = (reading->time - previous->time); - osm_epi_create_port_id(&(epi_pe_data.port_id), guid, port, node->node_name); + osm_epi_create_port_id(&(epi_pe_data.port_id), guid, port, + node->node_name); /* calculate changes from previous reading */ - epi_pe_data.symbol_err_cnt = (reading->symbol_err_cnt - previous->symbol_err_cnt); - p_port->err_total.symbol_err_cnt += epi_pe_data.symbol_err_cnt; - epi_pe_data.link_err_recover = (reading->link_err_recover - previous->link_err_recover); - p_port->err_total.link_err_recover += epi_pe_data.link_err_recover; - epi_pe_data.link_downed = (reading->link_downed - previous->link_downed); - p_port->err_total.link_downed += epi_pe_data.link_downed; + epi_pe_data.symbol_err_cnt = + (reading->symbol_err_cnt - previous->symbol_err_cnt); + p_port->err_total.symbol_err_cnt += epi_pe_data.symbol_err_cnt; + epi_pe_data.link_err_recover = + (reading->link_err_recover - previous->link_err_recover); + p_port->err_total.link_err_recover += epi_pe_data.link_err_recover; + epi_pe_data.link_downed = + (reading->link_downed - previous->link_downed); + p_port->err_total.link_downed += epi_pe_data.link_downed; epi_pe_data.rcv_err = (reading->rcv_err - previous->rcv_err); - p_port->err_total.rcv_err += epi_pe_data.rcv_err; - epi_pe_data.rcv_rem_phys_err = (reading->rcv_rem_phys_err - previous->rcv_rem_phys_err); - p_port->err_total.rcv_rem_phys_err += epi_pe_data.rcv_rem_phys_err; - epi_pe_data.rcv_switch_relay_err = (reading->rcv_switch_relay_err - previous->rcv_switch_relay_err); - p_port->err_total.rcv_switch_relay_err += epi_pe_data.rcv_switch_relay_err; - epi_pe_data.xmit_discards = (reading->xmit_discards - previous->xmit_discards); - p_port->err_total.xmit_discards += epi_pe_data.xmit_discards; - epi_pe_data.xmit_constraint_err = (reading->xmit_constraint_err - previous->xmit_constraint_err); - p_port->err_total.xmit_constraint_err += epi_pe_data.xmit_constraint_err; - epi_pe_data.rcv_constraint_err = (reading->rcv_constraint_err - previous->rcv_constraint_err); - p_port->err_total.rcv_constraint_err += epi_pe_data.rcv_constraint_err; - epi_pe_data.link_integrity = (reading->link_integrity - previous->link_integrity); - p_port->err_total.link_integrity += epi_pe_data.link_integrity; - epi_pe_data.buffer_overrun = (reading->buffer_overrun - previous->buffer_overrun); - p_port->err_total.buffer_overrun += epi_pe_data.buffer_overrun; - epi_pe_data.vl15_dropped = (reading->vl15_dropped - previous->vl15_dropped); - p_port->err_total.vl15_dropped += epi_pe_data.vl15_dropped; + p_port->err_total.rcv_err += epi_pe_data.rcv_err; + epi_pe_data.rcv_rem_phys_err = + (reading->rcv_rem_phys_err - previous->rcv_rem_phys_err); + p_port->err_total.rcv_rem_phys_err += epi_pe_data.rcv_rem_phys_err; + epi_pe_data.rcv_switch_relay_err = + (reading->rcv_switch_relay_err - previous->rcv_switch_relay_err); + p_port->err_total.rcv_switch_relay_err += + epi_pe_data.rcv_switch_relay_err; + epi_pe_data.xmit_discards = + (reading->xmit_discards - previous->xmit_discards); + p_port->err_total.xmit_discards += epi_pe_data.xmit_discards; + epi_pe_data.xmit_constraint_err = + (reading->xmit_constraint_err - previous->xmit_constraint_err); + p_port->err_total.xmit_constraint_err += + epi_pe_data.xmit_constraint_err; + epi_pe_data.rcv_constraint_err = + (reading->rcv_constraint_err - previous->rcv_constraint_err); + p_port->err_total.rcv_constraint_err += epi_pe_data.rcv_constraint_err; + epi_pe_data.link_integrity = + (reading->link_integrity - previous->link_integrity); + p_port->err_total.link_integrity += epi_pe_data.link_integrity; + epi_pe_data.buffer_overrun = + (reading->buffer_overrun - previous->buffer_overrun); + p_port->err_total.buffer_overrun += epi_pe_data.buffer_overrun; + epi_pe_data.vl15_dropped = + (reading->vl15_dropped - previous->vl15_dropped); + p_port->err_total.vl15_dropped += epi_pe_data.vl15_dropped; p_port->err_previous = *reading; osm_epi_report(db->event_plugin, OSM_EVENT_ID_PORT_ERRORS, - (void *)&epi_pe_data); + (void *)&epi_pe_data); -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } -perfmgr_db_err_t perfmgr_db_get_prev_err(perfmgr_db_t *db, uint64_t guid, +perfmgr_db_err_t perfmgr_db_get_prev_err(perfmgr_db_t * db, uint64_t guid, uint8_t port, - perfmgr_db_err_reading_t *reading) + perfmgr_db_err_reading_t * reading) { - _db_node_t *node = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + _db_node_t *node = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; cl_plock_acquire(&(db->lock)); @@ -306,17 +326,17 @@ perfmgr_db_err_t perfmgr_db_get_prev_err(perfmgr_db_t *db, uint64_t guid, *reading = node->ports[port].err_previous; -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } perfmgr_db_err_t -perfmgr_db_clear_prev_err(perfmgr_db_t *db, uint64_t guid, uint8_t port) +perfmgr_db_clear_prev_err(perfmgr_db_t * db, uint64_t guid, uint8_t port) { - _db_node_t *node = NULL; - perfmgr_db_data_cnt_reading_t *previous = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + _db_node_t *node = NULL; + perfmgr_db_data_cnt_reading_t *previous = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; cl_plock_excl_acquire(&(db->lock)); node = _get(db, guid); @@ -328,29 +348,31 @@ perfmgr_db_clear_prev_err(perfmgr_db_t *db, uint64_t guid, uint8_t port) memset(previous, 0, sizeof(*previous)); node->ports[port].dc_previous.time = time(NULL); -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } static inline void -debug_dump_dc_reading(perfmgr_db_t *db, uint64_t guid, uint8_t port_num, - _db_port_t *port, perfmgr_db_data_cnt_reading_t *cur) +debug_dump_dc_reading(perfmgr_db_t * db, uint64_t guid, uint8_t port_num, + _db_port_t * port, perfmgr_db_data_cnt_reading_t * cur) { if (!osm_log_is_active(db->osm_log, OSM_LOG_DEBUG)) - return; /* optimize this a big */ + return; /* optimize this a big */ osm_log(db->osm_log, OSM_LOG_DEBUG, - "xd %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->xmit_data, - port->dc_previous.xmit_data, port->dc_total.xmit_data); + "xd %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->xmit_data, port->dc_previous.xmit_data, + port->dc_total.xmit_data); osm_log(db->osm_log, OSM_LOG_DEBUG, - "rd %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_data, + "rd %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", cur->rcv_data, port->dc_previous.rcv_data, port->dc_total.rcv_data); osm_log(db->osm_log, OSM_LOG_DEBUG, - "xp %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->xmit_pkts, - port->dc_previous.xmit_pkts, port->dc_total.xmit_pkts); + "xp %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", + cur->xmit_pkts, port->dc_previous.xmit_pkts, + port->dc_total.xmit_pkts); osm_log(db->osm_log, OSM_LOG_DEBUG, - "rp %"PRIu64" <-- %"PRIu64" (%" PRIu64 ")\n", cur->rcv_pkts, + "rp %" PRIu64 " <-- %" PRIu64 " (%" PRIu64 ")\n", cur->rcv_pkts, port->dc_previous.rcv_pkts, port->dc_total.rcv_pkts); } @@ -358,14 +380,14 @@ debug_dump_dc_reading(perfmgr_db_t *db, uint64_t guid, uint8_t port_num, * perfmgr_db_data_cnt_reading_t functions **********************************************************************/ perfmgr_db_err_t -perfmgr_db_add_dc_reading(perfmgr_db_t *db, uint64_t guid, - uint8_t port, perfmgr_db_data_cnt_reading_t *reading) +perfmgr_db_add_dc_reading(perfmgr_db_t * db, uint64_t guid, + uint8_t port, perfmgr_db_data_cnt_reading_t * reading) { - _db_port_t *p_port = NULL; - _db_node_t *node = NULL; - perfmgr_db_data_cnt_reading_t *previous = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; - osm_epi_dc_event_t epi_dc_data; + _db_port_t *p_port = NULL; + _db_node_t *node = NULL; + perfmgr_db_data_cnt_reading_t *previous = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + osm_epi_dc_event_t epi_dc_data; cl_plock_excl_acquire(&(db->lock)); node = _get(db, guid); @@ -378,42 +400,47 @@ perfmgr_db_add_dc_reading(perfmgr_db_t *db, uint64_t guid, debug_dump_dc_reading(db, guid, port, p_port, reading); epi_dc_data.time_diff_s = (reading->time - previous->time); - osm_epi_create_port_id(&(epi_dc_data.port_id), guid, port, node->node_name); + osm_epi_create_port_id(&(epi_dc_data.port_id), guid, port, + node->node_name); /* calculate changes from previous reading */ epi_dc_data.xmit_data = (reading->xmit_data - previous->xmit_data); - p_port->dc_total.xmit_data += epi_dc_data.xmit_data; + p_port->dc_total.xmit_data += epi_dc_data.xmit_data; epi_dc_data.rcv_data = (reading->rcv_data - previous->rcv_data); - p_port->dc_total.rcv_data += epi_dc_data.rcv_data; + p_port->dc_total.rcv_data += epi_dc_data.rcv_data; epi_dc_data.xmit_pkts = (reading->xmit_pkts - previous->xmit_pkts); - p_port->dc_total.xmit_pkts += epi_dc_data.xmit_pkts; + p_port->dc_total.xmit_pkts += epi_dc_data.xmit_pkts; epi_dc_data.rcv_pkts = (reading->rcv_pkts - previous->rcv_pkts); - p_port->dc_total.rcv_pkts += epi_dc_data.rcv_pkts; - epi_dc_data.unicast_xmit_pkts = (reading->unicast_xmit_pkts - previous->unicast_xmit_pkts); - p_port->dc_total.unicast_xmit_pkts += epi_dc_data.unicast_xmit_pkts; - epi_dc_data.unicast_rcv_pkts = (reading->unicast_rcv_pkts - previous->unicast_rcv_pkts); - p_port->dc_total.unicast_rcv_pkts += epi_dc_data.unicast_rcv_pkts; - epi_dc_data.multicast_xmit_pkts = (reading->multicast_xmit_pkts - previous->multicast_xmit_pkts); + p_port->dc_total.rcv_pkts += epi_dc_data.rcv_pkts; + epi_dc_data.unicast_xmit_pkts = + (reading->unicast_xmit_pkts - previous->unicast_xmit_pkts); + p_port->dc_total.unicast_xmit_pkts += epi_dc_data.unicast_xmit_pkts; + epi_dc_data.unicast_rcv_pkts = + (reading->unicast_rcv_pkts - previous->unicast_rcv_pkts); + p_port->dc_total.unicast_rcv_pkts += epi_dc_data.unicast_rcv_pkts; + epi_dc_data.multicast_xmit_pkts = + (reading->multicast_xmit_pkts - previous->multicast_xmit_pkts); p_port->dc_total.multicast_xmit_pkts += epi_dc_data.multicast_xmit_pkts; - epi_dc_data.multicast_rcv_pkts = (reading->multicast_rcv_pkts - previous->multicast_rcv_pkts); - p_port->dc_total.multicast_rcv_pkts += epi_dc_data.multicast_rcv_pkts; + epi_dc_data.multicast_rcv_pkts = + (reading->multicast_rcv_pkts - previous->multicast_rcv_pkts); + p_port->dc_total.multicast_rcv_pkts += epi_dc_data.multicast_rcv_pkts; p_port->dc_previous = *reading; osm_epi_report(db->event_plugin, OSM_EVENT_ID_PORT_DATA_COUNTERS, - (void *)&epi_dc_data); + (void *)&epi_dc_data); -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } -perfmgr_db_err_t perfmgr_db_get_prev_dc(perfmgr_db_t *db, uint64_t guid, +perfmgr_db_err_t perfmgr_db_get_prev_dc(perfmgr_db_t * db, uint64_t guid, uint8_t port, - perfmgr_db_data_cnt_reading_t *reading) + perfmgr_db_data_cnt_reading_t * reading) { - _db_node_t *node = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + _db_node_t *node = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; cl_plock_acquire(&(db->lock)); @@ -423,17 +450,17 @@ perfmgr_db_err_t perfmgr_db_get_prev_dc(perfmgr_db_t *db, uint64_t guid, *reading = node->ports[port].dc_previous; -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } perfmgr_db_err_t -perfmgr_db_clear_prev_dc(perfmgr_db_t *db, uint64_t guid, uint8_t port) +perfmgr_db_clear_prev_dc(perfmgr_db_t * db, uint64_t guid, uint8_t port) { - _db_node_t *node = NULL; - perfmgr_db_data_cnt_reading_t *previous = NULL; - perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; + _db_node_t *node = NULL; + perfmgr_db_data_cnt_reading_t *previous = NULL; + perfmgr_db_err_t rc = PERFMGR_EVENT_DB_SUCCESS; cl_plock_excl_acquire(&(db->lock)); node = _get(db, guid); @@ -445,17 +472,16 @@ perfmgr_db_clear_prev_dc(perfmgr_db_t *db, uint64_t guid, uint8_t port) memset(previous, 0, sizeof(*previous)); node->ports[port].dc_previous.time = time(NULL); -Exit: + Exit: cl_plock_release(&(db->lock)); return (rc); } -static void -__clear_counters(cl_map_item_t * const p_map_item, void *context ) +static void __clear_counters(cl_map_item_t * const p_map_item, void *context) { - _db_node_t *node = (_db_node_t *)p_map_item; - int i = 0; - time_t ts = time(NULL); + _db_node_t *node = (_db_node_t *) p_map_item; + int i = 0; + time_t ts = time(NULL); for (i = 0; i < node->num_ports; i++) { node->ports[i].err_total.symbol_err_cnt = 0; @@ -489,7 +515,7 @@ __clear_counters(cl_map_item_t * const p_map_item, void *context ) /********************************************************************** * Clear all the counters from the db **********************************************************************/ -void perfmgr_db_clear_counters(perfmgr_db_t *db) +void perfmgr_db_clear_counters(perfmgr_db_t * db) { cl_plock_excl_acquire(&(db->lock)); cl_qmap_apply_func(&(db->pc_data), __clear_counters, (void *)db); @@ -503,54 +529,46 @@ void perfmgr_db_clear_counters(perfmgr_db_t *db) /********************************************************************** * Output a tab delimited output of the port counters **********************************************************************/ -static void -__dump_node_mr(_db_node_t *node, FILE *fp) +static void __dump_node_mr(_db_node_t * node, FILE * fp) { int i = 0; fprintf(fp, "\nName\tGUID\tPort\tLast Reset\t" - "%s\t%s\t" - "%s\t%s\t%s\t%s\t%s\t%s\t%s\t" - "%s\t%s\t%s\t%s\t%s\t%s\t%s\t" - "%s\t%s\t%s\t%s\n" - , - "symbol_err_cnt", - "link_err_recover", - "link_downed", - "rcv_err", - "rcv_rem_phys_err", - "rcv_switch_relay_err", - "xmit_discards", - "xmit_constraint_err", - "rcv_constraint_err", - "link_int_err", - "buf_overrun_err", - "vl15_dropped", - "xmit_data", - "rcv_data", - "xmit_pkts", - "rcv_pkts", - "unicast_xmit_pkts", - "unicast_rcv_pkts", - "multicast_xmit_pkts", - "multicast_rcv_pkts" - ); - for (i = 1; i < node->num_ports; i++) - { + "%s\t%s\t" + "%s\t%s\t%s\t%s\t%s\t%s\t%s\t" + "%s\t%s\t%s\t%s\t%s\t%s\t%s\t" + "%s\t%s\t%s\t%s\n", + "symbol_err_cnt", + "link_err_recover", + "link_downed", + "rcv_err", + "rcv_rem_phys_err", + "rcv_switch_relay_err", + "xmit_discards", + "xmit_constraint_err", + "rcv_constraint_err", + "link_int_err", + "buf_overrun_err", + "vl15_dropped", + "xmit_data", + "rcv_data", + "xmit_pkts", + "rcv_pkts", + "unicast_xmit_pkts", + "unicast_rcv_pkts", + "multicast_xmit_pkts", "multicast_rcv_pkts"); + for (i = 1; i < node->num_ports; i++) { char *since = ctime(&(node->ports[i].last_reset)); - since[strlen(since)-1] = '\0'; /* remove \n */ - - fprintf(fp, "%s\t0x%" PRIx64 "\t%d\t%s\t%"PRIu64"\t%"PRIu64"\t" - "%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t" - "%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t" - "%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t" - "%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t" - "%"PRIu64"\t%"PRIu64"\t%"PRIu64"\t%"PRIu64"\n" - , - node->node_name, - node->node_guid, - i, - since, + since[strlen(since) - 1] = '\0'; /* remove \n */ + + fprintf(fp, + "%s\t0x%" PRIx64 "\t%d\t%s\t%" PRIu64 "\t%" PRIu64 "\t" + "%" PRIu64 "\t%" PRIu64 "\t%" PRIu64 "\t%" PRIu64 "\t" + "%" PRIu64 "\t%" PRIu64 "\t%" PRIu64 "\t" "%" PRIu64 + "\t%" PRIu64 "\t%" PRIu64 "\t" "%" PRIu64 "\t%" PRIu64 + "\t%" PRIu64 "\t%" PRIu64 "\t" "%" PRIu64 "\t%" PRIu64 + "\t%" PRIu64 "\t%" PRIu64 "\n", node->node_name, + node->node_guid, i, since, node->ports[i].err_total.symbol_err_cnt, node->ports[i].err_total.link_err_recover, node->ports[i].err_total.link_downed, @@ -563,7 +581,6 @@ __dump_node_mr(_db_node_t *node, FILE *fp) node->ports[i].err_total.link_integrity, node->ports[i].err_total.buffer_overrun, node->ports[i].err_total.vl15_dropped, - node->ports[i].dc_total.xmit_data, node->ports[i].dc_total.rcv_data, node->ports[i].dc_total.xmit_pkts, @@ -571,47 +588,43 @@ __dump_node_mr(_db_node_t *node, FILE *fp) node->ports[i].dc_total.unicast_xmit_pkts, node->ports[i].dc_total.unicast_rcv_pkts, node->ports[i].dc_total.multicast_xmit_pkts, - node->ports[i].dc_total.multicast_rcv_pkts - ); + node->ports[i].dc_total.multicast_rcv_pkts); } } /********************************************************************** * Output a human readable output of the port counters **********************************************************************/ -static void -__dump_node_hr(_db_node_t *node, FILE *fp) +static void __dump_node_hr(_db_node_t * node, FILE * fp) { int i = 0; fprintf(fp, "\n"); - for (i = 1; i < node->num_ports; i++) - { + for (i = 1; i < node->num_ports; i++) { char *since = ctime(&(node->ports[i].last_reset)); - since[strlen(since)-1] = '\0'; /* remove \n */ - - fprintf(fp, "\"%s\" 0x%"PRIx64" port %d (Since %s)\n" - " symbol_err_cnt : %"PRIu64"\n" - " link_err_recover : %"PRIu64"\n" - " link_downed : %"PRIu64"\n" - " rcv_err : %"PRIu64"\n" - " rcv_rem_phys_err : %"PRIu64"\n" - " rcv_switch_relay_err : %"PRIu64"\n" - " xmit_discards : %"PRIu64"\n" - " xmit_constraint_err : %"PRIu64"\n" - " rcv_constraint_err : %"PRIu64"\n" - " link_integrity_err : %"PRIu64"\n" - " buf_overrun_err : %"PRIu64"\n" - " vl15_dropped : %"PRIu64"\n" - " xmit_data : %"PRIu64"\n" - " rcv_data : %"PRIu64"\n" - " xmit_pkts : %"PRIu64"\n" - " rcv_pkts : %"PRIu64"\n" - " unicast_xmit_pkts : %"PRIu64"\n" - " unicast_rcv_pkts : %"PRIu64"\n" - " multicast_xmit_pkts : %"PRIu64"\n" - " multicast_rcv_pkts : %"PRIu64"\n" - , + since[strlen(since) - 1] = '\0'; /* remove \n */ + + fprintf(fp, "\"%s\" 0x%" PRIx64 " port %d (Since %s)\n" + " symbol_err_cnt : %" PRIu64 "\n" + " link_err_recover : %" PRIu64 "\n" + " link_downed : %" PRIu64 "\n" + " rcv_err : %" PRIu64 "\n" + " rcv_rem_phys_err : %" PRIu64 "\n" + " rcv_switch_relay_err : %" PRIu64 "\n" + " xmit_discards : %" PRIu64 "\n" + " xmit_constraint_err : %" PRIu64 "\n" + " rcv_constraint_err : %" PRIu64 "\n" + " link_integrity_err : %" PRIu64 "\n" + " buf_overrun_err : %" PRIu64 "\n" + " vl15_dropped : %" PRIu64 "\n" + " xmit_data : %" PRIu64 "\n" + " rcv_data : %" PRIu64 "\n" + " xmit_pkts : %" PRIu64 "\n" + " rcv_pkts : %" PRIu64 "\n" + " unicast_xmit_pkts : %" PRIu64 "\n" + " unicast_rcv_pkts : %" PRIu64 "\n" + " multicast_xmit_pkts : %" PRIu64 "\n" + " multicast_rcv_pkts : %" PRIu64 "\n", node->node_name, node->node_guid, i, @@ -628,7 +641,6 @@ __dump_node_hr(_db_node_t *node, FILE *fp) node->ports[i].err_total.link_integrity, node->ports[i].err_total.buffer_overrun, node->ports[i].err_total.vl15_dropped, - node->ports[i].dc_total.xmit_data, node->ports[i].dc_total.rcv_data, node->ports[i].dc_total.xmit_pkts, @@ -636,35 +648,32 @@ __dump_node_hr(_db_node_t *node, FILE *fp) node->ports[i].dc_total.unicast_xmit_pkts, node->ports[i].dc_total.unicast_rcv_pkts, node->ports[i].dc_total.multicast_xmit_pkts, - node->ports[i].dc_total.multicast_rcv_pkts - ); + node->ports[i].dc_total.multicast_rcv_pkts); } } /* Define a context for the __db_dump callback */ typedef struct { - FILE *fp; - perfmgr_db_dump_t dump_type; + FILE *fp; + perfmgr_db_dump_t dump_type; } dump_context_t; /********************************************************************** **********************************************************************/ -static void -__db_dump(cl_map_item_t * const p_map_item, void *context ) +static void __db_dump(cl_map_item_t * const p_map_item, void *context) { - _db_node_t *node = (_db_node_t *)p_map_item; - dump_context_t *c = (dump_context_t *)context; - FILE *fp = c->fp; - - switch (c->dump_type) - { - case PERFMGR_EVENT_DB_DUMP_MR: - __dump_node_mr(node, fp); - break; - case PERFMGR_EVENT_DB_DUMP_HR: - default: - __dump_node_hr(node, fp); - break; + _db_node_t *node = (_db_node_t *) p_map_item; + dump_context_t *c = (dump_context_t *) context; + FILE *fp = c->fp; + + switch (c->dump_type) { + case PERFMGR_EVENT_DB_DUMP_MR: + __dump_node_mr(node, fp); + break; + case PERFMGR_EVENT_DB_DUMP_HR: + default: + __dump_node_hr(node, fp); + break; } } @@ -672,9 +681,9 @@ __db_dump(cl_map_item_t * const p_map_item, void *context ) * dump the data to the file "file" **********************************************************************/ perfmgr_db_err_t -perfmgr_db_dump(perfmgr_db_t *db, char *file, perfmgr_db_dump_t dump_type) +perfmgr_db_dump(perfmgr_db_t * db, char *file, perfmgr_db_dump_t dump_type) { - dump_context_t context; + dump_context_t context; context.fp = fopen(file, "w+"); if (!context.fp) @@ -692,27 +701,31 @@ perfmgr_db_dump(perfmgr_db_t *db, char *file, perfmgr_db_dump_t dump_type) * Fill in the various DB objects from their wire counter parts **********************************************************************/ void -perfmgr_db_fill_err_read(ib_port_counters_t *wire_read, - perfmgr_db_err_reading_t *reading) +perfmgr_db_fill_err_read(ib_port_counters_t * wire_read, + perfmgr_db_err_reading_t * reading) { reading->symbol_err_cnt = cl_ntoh16(wire_read->symbol_err_cnt); reading->link_err_recover = cl_ntoh16(wire_read->link_err_recover); reading->link_downed = wire_read->link_downed; reading->rcv_err = wire_read->rcv_err; reading->rcv_rem_phys_err = cl_ntoh16(wire_read->rcv_rem_phys_err); - reading->rcv_switch_relay_err = cl_ntoh16(wire_read->rcv_switch_relay_err); + reading->rcv_switch_relay_err = + cl_ntoh16(wire_read->rcv_switch_relay_err); reading->xmit_discards = cl_ntoh16(wire_read->xmit_discards); - reading->xmit_constraint_err = cl_ntoh16(wire_read->xmit_constraint_err); + reading->xmit_constraint_err = + cl_ntoh16(wire_read->xmit_constraint_err); reading->rcv_constraint_err = wire_read->rcv_constraint_err; - reading->link_integrity = PC_LINK_INT(wire_read->link_int_buffer_overrun); - reading->buffer_overrun = PC_BUF_OVERRUN(wire_read->link_int_buffer_overrun); + reading->link_integrity = + PC_LINK_INT(wire_read->link_int_buffer_overrun); + reading->buffer_overrun = + PC_BUF_OVERRUN(wire_read->link_int_buffer_overrun); reading->vl15_dropped = cl_ntoh16(wire_read->vl15_dropped); reading->time = time(NULL); } void -perfmgr_db_fill_data_cnt_read_pc(ib_port_counters_t *wire_read, - perfmgr_db_data_cnt_reading_t *reading) +perfmgr_db_fill_data_cnt_read_pc(ib_port_counters_t * wire_read, + perfmgr_db_data_cnt_reading_t * reading) { reading->xmit_data = cl_ntoh32(wire_read->xmit_data); reading->rcv_data = cl_ntoh32(wire_read->rcv_data); @@ -726,8 +739,8 @@ perfmgr_db_fill_data_cnt_read_pc(ib_port_counters_t *wire_read, } void -perfmgr_db_fill_data_cnt_read_epc(ib_port_counters_ext_t *wire_read, - perfmgr_db_data_cnt_reading_t *reading) +perfmgr_db_fill_data_cnt_read_epc(ib_port_counters_ext_t * wire_read, + perfmgr_db_data_cnt_reading_t * reading) { reading->xmit_data = cl_ntoh64(wire_read->xmit_data); reading->rcv_data = cl_ntoh64(wire_read->rcv_data); @@ -735,7 +748,8 @@ perfmgr_db_fill_data_cnt_read_epc(ib_port_counters_ext_t *wire_read, reading->rcv_pkts = cl_ntoh64(wire_read->rcv_pkts); reading->unicast_xmit_pkts = cl_ntoh64(wire_read->unicast_xmit_pkts); reading->unicast_rcv_pkts = cl_ntoh64(wire_read->unicast_rcv_pkts); - reading->multicast_xmit_pkts = cl_ntoh64(wire_read->multicast_xmit_pkts); + reading->multicast_xmit_pkts = + cl_ntoh64(wire_read->multicast_xmit_pkts); reading->multicast_rcv_pkts = cl_ntoh64(wire_read->multicast_rcv_pkts); reading->time = time(NULL); } diff --git a/opensm/opensm/osm_pkey.c b/opensm/opensm/osm_pkey.c index 2397fbe..04d009b 100644 --- a/opensm/opensm/osm_pkey.c +++ b/opensm/opensm/osm_pkey.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -61,150 +61,142 @@ /********************************************************************** **********************************************************************/ -void osm_pkey_tbl_construct( - IN osm_pkey_tbl_t *p_pkey_tbl) +void osm_pkey_tbl_construct(IN osm_pkey_tbl_t * p_pkey_tbl) { - cl_ptr_vector_construct( &p_pkey_tbl->blocks ); - cl_ptr_vector_construct( &p_pkey_tbl->new_blocks ); - cl_map_construct( &p_pkey_tbl->keys ); + cl_ptr_vector_construct(&p_pkey_tbl->blocks); + cl_ptr_vector_construct(&p_pkey_tbl->new_blocks); + cl_map_construct(&p_pkey_tbl->keys); } /********************************************************************** **********************************************************************/ -void osm_pkey_tbl_destroy( - IN osm_pkey_tbl_t *p_pkey_tbl) +void osm_pkey_tbl_destroy(IN osm_pkey_tbl_t * p_pkey_tbl) { - ib_pkey_table_t *p_block; - uint16_t num_blocks, i; - - num_blocks = (uint16_t)(cl_ptr_vector_get_size( &p_pkey_tbl->blocks )); - for (i = 0; i < num_blocks; i++) - if ((p_block = cl_ptr_vector_get( &p_pkey_tbl->blocks, i ))) - free(p_block); - cl_ptr_vector_destroy( &p_pkey_tbl->blocks ); - - num_blocks = (uint16_t)(cl_ptr_vector_get_size( &p_pkey_tbl->new_blocks )); - for (i = 0; i < num_blocks; i++) - if ((p_block = cl_ptr_vector_get( &p_pkey_tbl->new_blocks, i ))) - free(p_block); - cl_ptr_vector_destroy( &p_pkey_tbl->new_blocks ); - - cl_map_remove_all( &p_pkey_tbl->keys ); - cl_map_destroy( &p_pkey_tbl->keys ); + ib_pkey_table_t *p_block; + uint16_t num_blocks, i; + + num_blocks = (uint16_t) (cl_ptr_vector_get_size(&p_pkey_tbl->blocks)); + for (i = 0; i < num_blocks; i++) + if ((p_block = cl_ptr_vector_get(&p_pkey_tbl->blocks, i))) + free(p_block); + cl_ptr_vector_destroy(&p_pkey_tbl->blocks); + + num_blocks = + (uint16_t) (cl_ptr_vector_get_size(&p_pkey_tbl->new_blocks)); + for (i = 0; i < num_blocks; i++) + if ((p_block = cl_ptr_vector_get(&p_pkey_tbl->new_blocks, i))) + free(p_block); + cl_ptr_vector_destroy(&p_pkey_tbl->new_blocks); + + cl_map_remove_all(&p_pkey_tbl->keys); + cl_map_destroy(&p_pkey_tbl->keys); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_pkey_tbl_init( - IN osm_pkey_tbl_t *p_pkey_tbl) +ib_api_status_t osm_pkey_tbl_init(IN osm_pkey_tbl_t * p_pkey_tbl) { - cl_ptr_vector_init(&p_pkey_tbl->blocks, 0, 1); - cl_ptr_vector_init(&p_pkey_tbl->new_blocks, 0, 1); - cl_map_init(&p_pkey_tbl->keys, 1); - cl_qlist_init(&p_pkey_tbl->pending); - p_pkey_tbl->used_blocks = 0; - p_pkey_tbl->max_blocks = 0; - return(IB_SUCCESS); + cl_ptr_vector_init(&p_pkey_tbl->blocks, 0, 1); + cl_ptr_vector_init(&p_pkey_tbl->new_blocks, 0, 1); + cl_map_init(&p_pkey_tbl->keys, 1); + cl_qlist_init(&p_pkey_tbl->pending); + p_pkey_tbl->used_blocks = 0; + p_pkey_tbl->max_blocks = 0; + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -void osm_pkey_tbl_init_new_blocks( - IN const osm_pkey_tbl_t *p_pkey_tbl) +void osm_pkey_tbl_init_new_blocks(IN const osm_pkey_tbl_t * p_pkey_tbl) { - ib_pkey_table_t *p_block; - size_t b, num_blocks = cl_ptr_vector_get_size(&p_pkey_tbl->new_blocks); + ib_pkey_table_t *p_block; + size_t b, num_blocks = cl_ptr_vector_get_size(&p_pkey_tbl->new_blocks); - for (b = 0; b < num_blocks; b++) - if ((p_block = cl_ptr_vector_get(&p_pkey_tbl->new_blocks, b))) - memset(p_block, 0, sizeof(*p_block)); + for (b = 0; b < num_blocks; b++) + if ((p_block = cl_ptr_vector_get(&p_pkey_tbl->new_blocks, b))) + memset(p_block, 0, sizeof(*p_block)); } /********************************************************************** **********************************************************************/ -void osm_pkey_tbl_cleanup_pending( - IN osm_pkey_tbl_t *p_pkey_tbl) +void osm_pkey_tbl_cleanup_pending(IN osm_pkey_tbl_t * p_pkey_tbl) { - cl_list_item_t *p_item; + cl_list_item_t *p_item; - p_item = cl_qlist_remove_head(&p_pkey_tbl->pending); - while (p_item != cl_qlist_end(&p_pkey_tbl->pending)) - { - free((osm_pending_pkey_t *)p_item); - } + p_item = cl_qlist_remove_head(&p_pkey_tbl->pending); + while (p_item != cl_qlist_end(&p_pkey_tbl->pending)) { + free((osm_pending_pkey_t *) p_item); + } } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pkey_tbl_set( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t block, - IN ib_pkey_table_t *p_tbl) +osm_pkey_tbl_set(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t block, IN ib_pkey_table_t * p_tbl) { - uint16_t b, i; - ib_pkey_table_t *p_pkey_block; - uint16_t *p_prev_pkey; - ib_net16_t pkey; - - /* make sure the block is allocated */ - if (cl_ptr_vector_get_size( &p_pkey_tbl->blocks ) > block) - p_pkey_block = - (ib_pkey_table_t *)cl_ptr_vector_get( &p_pkey_tbl->blocks, block ); - else - p_pkey_block = NULL; - - if ( !p_pkey_block ) - { - p_pkey_block = (ib_pkey_table_t *)malloc(sizeof(ib_pkey_table_t)); - if (p_pkey_block) - memset(p_pkey_block, 0, sizeof(ib_pkey_table_t)); - cl_ptr_vector_set( &p_pkey_tbl->blocks, block, p_pkey_block ); - } - - /* sets the block values */ - memcpy( p_pkey_block, p_tbl, sizeof(ib_pkey_table_t) ); - - /* - NOTE: as the spec does not require uniqueness of PKeys in - tables there is no other way but to refresh the entire keys map. - - Moreover, if the same key exists but with full membership it should have - precedence on the key with limited membership ! - */ - cl_map_remove_all( &p_pkey_tbl->keys ); - - for (b = 0; b < cl_ptr_vector_get_size( &p_pkey_tbl->blocks ); b++) - { - - p_pkey_block = cl_ptr_vector_get( &p_pkey_tbl->blocks, b ); - if (! p_pkey_block) - continue; - - for (i = 0; i < IB_NUM_PKEY_ELEMENTS_IN_BLOCK; i++) - { - pkey = p_pkey_block->pkey_entry[i]; - if (ib_pkey_is_invalid(pkey)) - continue; - - /* - ignore the PKey Full Member bit in the key but store - the pointer to the table element as the map value - */ - p_prev_pkey = - cl_map_get( &p_pkey_tbl->keys, ib_pkey_get_base(pkey)); - - /* we only insert if no previous or it is not full member */ - if ((p_prev_pkey == NULL) || - (cl_ntoh16(*p_prev_pkey) < cl_ntoh16(pkey))) - cl_map_insert( &p_pkey_tbl->keys, - ib_pkey_get_base(pkey), - &(p_pkey_block->pkey_entry[i]) - ); - } - } - return(IB_SUCCESS); + uint16_t b, i; + ib_pkey_table_t *p_pkey_block; + uint16_t *p_prev_pkey; + ib_net16_t pkey; + + /* make sure the block is allocated */ + if (cl_ptr_vector_get_size(&p_pkey_tbl->blocks) > block) + p_pkey_block = + (ib_pkey_table_t *) cl_ptr_vector_get(&p_pkey_tbl->blocks, + block); + else + p_pkey_block = NULL; + + if (!p_pkey_block) { + p_pkey_block = + (ib_pkey_table_t *) malloc(sizeof(ib_pkey_table_t)); + if (p_pkey_block) + memset(p_pkey_block, 0, sizeof(ib_pkey_table_t)); + cl_ptr_vector_set(&p_pkey_tbl->blocks, block, p_pkey_block); + } + + /* sets the block values */ + memcpy(p_pkey_block, p_tbl, sizeof(ib_pkey_table_t)); + + /* + NOTE: as the spec does not require uniqueness of PKeys in + tables there is no other way but to refresh the entire keys map. + + Moreover, if the same key exists but with full membership it should have + precedence on the key with limited membership ! + */ + cl_map_remove_all(&p_pkey_tbl->keys); + + for (b = 0; b < cl_ptr_vector_get_size(&p_pkey_tbl->blocks); b++) { + + p_pkey_block = cl_ptr_vector_get(&p_pkey_tbl->blocks, b); + if (!p_pkey_block) + continue; + + for (i = 0; i < IB_NUM_PKEY_ELEMENTS_IN_BLOCK; i++) { + pkey = p_pkey_block->pkey_entry[i]; + if (ib_pkey_is_invalid(pkey)) + continue; + + /* + ignore the PKey Full Member bit in the key but store + the pointer to the table element as the map value + */ + p_prev_pkey = + cl_map_get(&p_pkey_tbl->keys, + ib_pkey_get_base(pkey)); + + /* we only insert if no previous or it is not full member */ + if ((p_prev_pkey == NULL) || + (cl_ntoh16(*p_prev_pkey) < cl_ntoh16(pkey))) + cl_map_insert(&p_pkey_tbl->keys, + ib_pkey_get_base(pkey), + &(p_pkey_block->pkey_entry[i]) + ); + } + } + return (IB_SUCCESS); } /********************************************************************** @@ -214,333 +206,308 @@ osm_pkey_tbl_set( Also, make sure the regular block exists. */ ib_api_status_t -osm_pkey_tbl_set_new_entry( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t block_idx, - IN uint8_t pkey_idx, - IN uint16_t pkey) +osm_pkey_tbl_set_new_entry(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t block_idx, + IN uint8_t pkey_idx, IN uint16_t pkey) { - ib_pkey_table_t *p_block; + ib_pkey_table_t *p_block; - if (!(p_block = osm_pkey_tbl_new_block_get(p_pkey_tbl, block_idx))) { - p_block = (ib_pkey_table_t *)malloc(sizeof(ib_pkey_table_t)); - if (!p_block) - return(IB_ERROR); - memset(p_block, 0, sizeof(ib_pkey_table_t)); - cl_ptr_vector_set(&p_pkey_tbl->new_blocks, block_idx, p_block); - } + if (!(p_block = osm_pkey_tbl_new_block_get(p_pkey_tbl, block_idx))) { + p_block = (ib_pkey_table_t *) malloc(sizeof(ib_pkey_table_t)); + if (!p_block) + return (IB_ERROR); + memset(p_block, 0, sizeof(ib_pkey_table_t)); + cl_ptr_vector_set(&p_pkey_tbl->new_blocks, block_idx, p_block); + } - p_block->pkey_entry[pkey_idx] = pkey; - if (p_pkey_tbl->used_blocks <= block_idx) - p_pkey_tbl->used_blocks = block_idx + 1; + p_block->pkey_entry[pkey_idx] = pkey; + if (p_pkey_tbl->used_blocks <= block_idx) + p_pkey_tbl->used_blocks = block_idx + 1; - return(IB_SUCCESS); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ boolean_t -osm_pkey_find_next_free_entry( - IN osm_pkey_tbl_t *p_pkey_tbl, - OUT uint16_t *p_block_idx, - OUT uint8_t *p_pkey_idx) +osm_pkey_find_next_free_entry(IN osm_pkey_tbl_t * p_pkey_tbl, + OUT uint16_t * p_block_idx, + OUT uint8_t * p_pkey_idx) { - ib_pkey_table_t *p_new_block; - - CL_ASSERT(p_block_idx); - CL_ASSERT(p_pkey_idx); - - while (*p_block_idx < p_pkey_tbl->max_blocks) - { - if (*p_pkey_idx > IB_NUM_PKEY_ELEMENTS_IN_BLOCK - 1) - { - *p_pkey_idx = 0; - (*p_block_idx)++; - if (*p_block_idx >= p_pkey_tbl->max_blocks) - return FALSE; - } - - p_new_block = osm_pkey_tbl_new_block_get(p_pkey_tbl, *p_block_idx); - - if (!p_new_block || - ib_pkey_is_invalid(p_new_block->pkey_entry[*p_pkey_idx])) - return TRUE; - else - (*p_pkey_idx)++; - } - return FALSE; + ib_pkey_table_t *p_new_block; + + CL_ASSERT(p_block_idx); + CL_ASSERT(p_pkey_idx); + + while (*p_block_idx < p_pkey_tbl->max_blocks) { + if (*p_pkey_idx > IB_NUM_PKEY_ELEMENTS_IN_BLOCK - 1) { + *p_pkey_idx = 0; + (*p_block_idx)++; + if (*p_block_idx >= p_pkey_tbl->max_blocks) + return FALSE; + } + + p_new_block = + osm_pkey_tbl_new_block_get(p_pkey_tbl, *p_block_idx); + + if (!p_new_block || + ib_pkey_is_invalid(p_new_block->pkey_entry[*p_pkey_idx])) + return TRUE; + else + (*p_pkey_idx)++; + } + return FALSE; } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pkey_tbl_get_block_and_idx( - IN osm_pkey_tbl_t *p_pkey_tbl, - IN uint16_t *p_pkey, - OUT uint16_t *p_block_idx, - OUT uint8_t *p_pkey_idx) +osm_pkey_tbl_get_block_and_idx(IN osm_pkey_tbl_t * p_pkey_tbl, + IN uint16_t * p_pkey, + OUT uint16_t * p_block_idx, + OUT uint8_t * p_pkey_idx) { - uint16_t num_of_blocks; - uint16_t block_index; - ib_pkey_table_t *block; - - CL_ASSERT( p_block_idx != NULL ); - CL_ASSERT( p_pkey_idx != NULL ); - - num_of_blocks = (uint16_t)cl_ptr_vector_get_size( &p_pkey_tbl->blocks ); - for (block_index = 0; block_index < num_of_blocks; block_index++) - { - block = osm_pkey_tbl_block_get(p_pkey_tbl, block_index); - if ((block->pkey_entry <= p_pkey) && - (p_pkey < block->pkey_entry + IB_NUM_PKEY_ELEMENTS_IN_BLOCK)) - { - *p_block_idx = block_index; - *p_pkey_idx = (uint8_t)(p_pkey - block->pkey_entry); - return(IB_SUCCESS); - } - } - return(IB_NOT_FOUND); + uint16_t num_of_blocks; + uint16_t block_index; + ib_pkey_table_t *block; + + CL_ASSERT(p_block_idx != NULL); + CL_ASSERT(p_pkey_idx != NULL); + + num_of_blocks = (uint16_t) cl_ptr_vector_get_size(&p_pkey_tbl->blocks); + for (block_index = 0; block_index < num_of_blocks; block_index++) { + block = osm_pkey_tbl_block_get(p_pkey_tbl, block_index); + if ((block->pkey_entry <= p_pkey) && + (p_pkey < + block->pkey_entry + IB_NUM_PKEY_ELEMENTS_IN_BLOCK)) { + *p_block_idx = block_index; + *p_pkey_idx = (uint8_t) (p_pkey - block->pkey_entry); + return (IB_SUCCESS); + } + } + return (IB_NOT_FOUND); } /********************************************************************** **********************************************************************/ static boolean_t -__osm_match_pkey ( - IN const ib_net16_t *pkey1, - IN const ib_net16_t *pkey2 ) +__osm_match_pkey(IN const ib_net16_t * pkey1, IN const ib_net16_t * pkey2) { - /* if both pkeys are not full member - this is not a match */ - if (!(ib_pkey_is_full_member(*pkey1) || ib_pkey_is_full_member(*pkey2))) - return(FALSE); + /* if both pkeys are not full member - this is not a match */ + if (!(ib_pkey_is_full_member(*pkey1) || ib_pkey_is_full_member(*pkey2))) + return (FALSE); - /* compare if the bases are the same. if they are - then - this is a match */ - if (ib_pkey_get_base(*pkey1) != ib_pkey_get_base(*pkey2)) - return(FALSE); + /* compare if the bases are the same. if they are - then + this is a match */ + if (ib_pkey_get_base(*pkey1) != ib_pkey_get_base(*pkey2)) + return (FALSE); - return(TRUE); + return (TRUE); } /********************************************************************** **********************************************************************/ boolean_t -osm_physp_share_this_pkey( - IN const osm_physp_t* const p_physp1, - IN const osm_physp_t* const p_physp2, - IN const ib_net16_t pkey ) +osm_physp_share_this_pkey(IN const osm_physp_t * const p_physp1, + IN const osm_physp_t * const p_physp2, + IN const ib_net16_t pkey) { - ib_net16_t *pkey1, *pkey2; + ib_net16_t *pkey1, *pkey2; - pkey1 = cl_map_get( &(osm_physp_get_pkey_tbl(p_physp1))->keys, - ib_pkey_get_base(pkey)); - pkey2 = cl_map_get( &(osm_physp_get_pkey_tbl(p_physp2))->keys, - ib_pkey_get_base(pkey)); - return (pkey1 && pkey2 && __osm_match_pkey(pkey1, pkey2)); + pkey1 = cl_map_get(&(osm_physp_get_pkey_tbl(p_physp1))->keys, + ib_pkey_get_base(pkey)); + pkey2 = cl_map_get(&(osm_physp_get_pkey_tbl(p_physp2))->keys, + ib_pkey_get_base(pkey)); + return (pkey1 && pkey2 && __osm_match_pkey(pkey1, pkey2)); } /********************************************************************** **********************************************************************/ ib_net16_t -osm_physp_find_common_pkey( - IN const osm_physp_t* const p_physp1, - IN const osm_physp_t* const p_physp2 ) +osm_physp_find_common_pkey(IN const osm_physp_t * const p_physp1, + IN const osm_physp_t * const p_physp2) { - ib_net16_t *pkey1, *pkey2; - uint64_t pkey1_base, pkey2_base; - const osm_pkey_tbl_t *pkey_tbl1, *pkey_tbl2; - cl_map_iterator_t map_iter1, map_iter2; - - pkey_tbl1 = osm_physp_get_pkey_tbl(p_physp1); - pkey_tbl2 = osm_physp_get_pkey_tbl(p_physp2); - - map_iter1 = cl_map_head(&pkey_tbl1->keys); - map_iter2 = cl_map_head(&pkey_tbl2->keys); - - /* we rely on the fact the map are sorted by pkey */ - while ( (map_iter1 != cl_map_end( &pkey_tbl1->keys )) && - (map_iter2 != cl_map_end( &pkey_tbl2->keys ))) - { - pkey1 = (ib_net16_t *)cl_map_obj( map_iter1 ); - pkey2 = (ib_net16_t *)cl_map_obj( map_iter2 ); - - if (__osm_match_pkey(pkey1, pkey2)) - return *pkey1; - - /* advance the lower value if they are not equal */ - pkey1_base = cl_map_key( map_iter1 ); - pkey2_base = cl_map_key( map_iter2 ); - if (pkey2_base == pkey1_base) - { - map_iter1 = cl_map_next( map_iter1 ); - map_iter2 = cl_map_next( map_iter2 ); - } - else if (pkey2_base < pkey1_base) - map_iter2 = cl_map_next( map_iter2 ); - else - map_iter1 = cl_map_next( map_iter1 ); - } - - return 0; + ib_net16_t *pkey1, *pkey2; + uint64_t pkey1_base, pkey2_base; + const osm_pkey_tbl_t *pkey_tbl1, *pkey_tbl2; + cl_map_iterator_t map_iter1, map_iter2; + + pkey_tbl1 = osm_physp_get_pkey_tbl(p_physp1); + pkey_tbl2 = osm_physp_get_pkey_tbl(p_physp2); + + map_iter1 = cl_map_head(&pkey_tbl1->keys); + map_iter2 = cl_map_head(&pkey_tbl2->keys); + + /* we rely on the fact the map are sorted by pkey */ + while ((map_iter1 != cl_map_end(&pkey_tbl1->keys)) && + (map_iter2 != cl_map_end(&pkey_tbl2->keys))) { + pkey1 = (ib_net16_t *) cl_map_obj(map_iter1); + pkey2 = (ib_net16_t *) cl_map_obj(map_iter2); + + if (__osm_match_pkey(pkey1, pkey2)) + return *pkey1; + + /* advance the lower value if they are not equal */ + pkey1_base = cl_map_key(map_iter1); + pkey2_base = cl_map_key(map_iter2); + if (pkey2_base == pkey1_base) { + map_iter1 = cl_map_next(map_iter1); + map_iter2 = cl_map_next(map_iter2); + } else if (pkey2_base < pkey1_base) + map_iter2 = cl_map_next(map_iter2); + else + map_iter1 = cl_map_next(map_iter1); + } + + return 0; } /********************************************************************** **********************************************************************/ boolean_t -osm_physp_share_pkey( - IN osm_log_t* p_log, - IN const osm_physp_t* const p_physp_1, - IN const osm_physp_t* const p_physp_2 ) +osm_physp_share_pkey(IN osm_log_t * p_log, + IN const osm_physp_t * const p_physp_1, + IN const osm_physp_t * const p_physp_2) { - const osm_pkey_tbl_t *pkey_tbl1, *pkey_tbl2; + const osm_pkey_tbl_t *pkey_tbl1, *pkey_tbl2; - if (p_physp_1 == p_physp_2) - return TRUE; + if (p_physp_1 == p_physp_2) + return TRUE; - pkey_tbl1 = osm_physp_get_pkey_tbl(p_physp_1); - pkey_tbl2 = osm_physp_get_pkey_tbl(p_physp_2); + pkey_tbl1 = osm_physp_get_pkey_tbl(p_physp_1); + pkey_tbl2 = osm_physp_get_pkey_tbl(p_physp_2); - /* - The spec: 10.9.2 does not require each phys port to have PKey Table. - So actually if it does not, we need to use the default port instead. + /* + The spec: 10.9.2 does not require each phys port to have PKey Table. + So actually if it does not, we need to use the default port instead. - HACK: meanwhile we will ignore the check - */ - if (cl_is_map_empty(&pkey_tbl1->keys) || cl_is_map_empty(&pkey_tbl2->keys)) - return TRUE; + HACK: meanwhile we will ignore the check + */ + if (cl_is_map_empty(&pkey_tbl1->keys) + || cl_is_map_empty(&pkey_tbl2->keys)) + return TRUE; - return !ib_pkey_is_invalid(osm_physp_find_common_pkey(p_physp_1, p_physp_2)); + return + !ib_pkey_is_invalid(osm_physp_find_common_pkey + (p_physp_1, p_physp_2)); } /********************************************************************** **********************************************************************/ boolean_t -osm_port_share_pkey( - IN osm_log_t* p_log, - IN const osm_port_t* const p_port_1, - IN const osm_port_t* const p_port_2 ) { +osm_port_share_pkey(IN osm_log_t * p_log, + IN const osm_port_t * const p_port_1, + IN const osm_port_t * const p_port_2) +{ - osm_physp_t *p_physp1, *p_physp2; - boolean_t ret; + osm_physp_t *p_physp1, *p_physp2; + boolean_t ret; - OSM_LOG_ENTER( p_log, osm_port_share_pkey ); + OSM_LOG_ENTER(p_log, osm_port_share_pkey); - if (!p_port_1 || !p_port_2) - { - ret = FALSE; - goto Exit; - } + if (!p_port_1 || !p_port_2) { + ret = FALSE; + goto Exit; + } - p_physp1 = p_port_1->p_physp; - p_physp2 = p_port_2->p_physp; + p_physp1 = p_port_1->p_physp; + p_physp2 = p_port_2->p_physp; - if (!p_physp1 || !p_physp2) - { - ret = FALSE; - goto Exit; - } + if (!p_physp1 || !p_physp2) { + ret = FALSE; + goto Exit; + } - ret = osm_physp_share_pkey(p_log, p_physp1, p_physp2); + ret = osm_physp_share_pkey(p_log, p_physp1, p_physp2); -Exit: - OSM_LOG_EXIT(p_log); - return ret; + Exit: + OSM_LOG_EXIT(p_log); + return ret; } /********************************************************************** **********************************************************************/ boolean_t -osm_lid_share_pkey( - IN osm_log_t* p_log, - IN const osm_subn_t * const p_subn, - IN const ib_net16_t lid1, - IN const uint8_t port_num1, - IN const ib_net16_t lid2, - IN const uint8_t port_num2 ) { - - osm_physp_t *p_physp1, *p_physp2; - osm_port_t *p_port1, *p_port2; - osm_node_t *p_node1, *p_node2; - const cl_ptr_vector_t* const p_port_lid_tbl = &(p_subn->port_lid_tbl); - - OSM_LOG_ENTER( p_log, osm_lid_share_pkey ); - - p_port1 = cl_ptr_vector_get(p_port_lid_tbl, lid1); - p_port2 = cl_ptr_vector_get(p_port_lid_tbl, lid2); - - p_node1 = p_port1->p_node; - p_node2 = p_port2->p_node; - - if (osm_node_get_type( p_node1 ) == IB_NODE_TYPE_SWITCH) - { - p_physp1 = osm_node_get_physp_ptr( p_node1, port_num1 ); - } - else - { - p_physp1 = p_port1->p_physp; - } - - if (osm_node_get_type( p_node2 ) == IB_NODE_TYPE_SWITCH) - { - p_physp2 = osm_node_get_physp_ptr( p_node2, port_num2 ); - } - else - { - p_physp2 = p_port2->p_physp; - } - - return(osm_physp_share_pkey(p_log, p_physp1, p_physp2)); +osm_lid_share_pkey(IN osm_log_t * p_log, + IN const osm_subn_t * const p_subn, + IN const ib_net16_t lid1, + IN const uint8_t port_num1, + IN const ib_net16_t lid2, IN const uint8_t port_num2) +{ + + osm_physp_t *p_physp1, *p_physp2; + osm_port_t *p_port1, *p_port2; + osm_node_t *p_node1, *p_node2; + const cl_ptr_vector_t *const p_port_lid_tbl = &(p_subn->port_lid_tbl); + + OSM_LOG_ENTER(p_log, osm_lid_share_pkey); + + p_port1 = cl_ptr_vector_get(p_port_lid_tbl, lid1); + p_port2 = cl_ptr_vector_get(p_port_lid_tbl, lid2); + + p_node1 = p_port1->p_node; + p_node2 = p_port2->p_node; + + if (osm_node_get_type(p_node1) == IB_NODE_TYPE_SWITCH) { + p_physp1 = osm_node_get_physp_ptr(p_node1, port_num1); + } else { + p_physp1 = p_port1->p_physp; + } + + if (osm_node_get_type(p_node2) == IB_NODE_TYPE_SWITCH) { + p_physp2 = osm_node_get_physp_ptr(p_node2, port_num2); + } else { + p_physp2 = p_port2->p_physp; + } + + return (osm_physp_share_pkey(p_log, p_physp1, p_physp2)); } /********************************************************************** **********************************************************************/ boolean_t -osm_physp_has_pkey( - IN osm_log_t* p_log, - IN const ib_net16_t pkey, - IN const osm_physp_t* const p_physp ) { - - ib_net16_t *p_pkey, pkey_base; - const osm_pkey_tbl_t *pkey_tbl; - boolean_t res = FALSE; - - OSM_LOG_ENTER( p_log, osm_physp_has_pkey ); - - osm_log( p_log, OSM_LOG_DEBUG, - "osm_physp_has_pkey: " - "Search for PKey: 0x%4x\n", - cl_ntoh16(pkey) ); - - /* if the pkey given is an invalid pkey - return TRUE. */ - if(ib_pkey_is_invalid(pkey)) - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_physp_has_pkey: " - "Given invalid PKey - we treat it loosely and allow it\n"); - res = TRUE; - goto Exit; - } - - pkey_base = ib_pkey_get_base(pkey); - - pkey_tbl = osm_physp_get_pkey_tbl(p_physp); - - p_pkey = cl_map_get( &pkey_tbl->keys, pkey_base ); - if (p_pkey) - { - res = TRUE; - osm_log( p_log, OSM_LOG_DEBUG, - "osm_physp_has_pkey: " - "PKey 0x%04x was found\n", cl_ntoh16(pkey)); - } - else - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_physp_has_pkey: " - "PKey 0x%04x was not found\n", cl_ntoh16(pkey)); - } - - Exit: - OSM_LOG_EXIT( p_log ); - return res; +osm_physp_has_pkey(IN osm_log_t * p_log, + IN const ib_net16_t pkey, + IN const osm_physp_t * const p_physp) +{ + + ib_net16_t *p_pkey, pkey_base; + const osm_pkey_tbl_t *pkey_tbl; + boolean_t res = FALSE; + + OSM_LOG_ENTER(p_log, osm_physp_has_pkey); + + osm_log(p_log, OSM_LOG_DEBUG, + "osm_physp_has_pkey: " + "Search for PKey: 0x%4x\n", cl_ntoh16(pkey)); + + /* if the pkey given is an invalid pkey - return TRUE. */ + if (ib_pkey_is_invalid(pkey)) { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_physp_has_pkey: " + "Given invalid PKey - we treat it loosely and allow it\n"); + res = TRUE; + goto Exit; + } + + pkey_base = ib_pkey_get_base(pkey); + + pkey_tbl = osm_physp_get_pkey_tbl(p_physp); + + p_pkey = cl_map_get(&pkey_tbl->keys, pkey_base); + if (p_pkey) { + res = TRUE; + osm_log(p_log, OSM_LOG_DEBUG, + "osm_physp_has_pkey: " + "PKey 0x%04x was found\n", cl_ntoh16(pkey)); + } else { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_physp_has_pkey: " + "PKey 0x%04x was not found\n", cl_ntoh16(pkey)); + } + + Exit: + OSM_LOG_EXIT(p_log); + return res; } diff --git a/opensm/opensm/osm_pkey_mgr.c b/opensm/opensm/osm_pkey_mgr.c index 443db16..05caaf5 100644 --- a/opensm/opensm/osm_pkey_mgr.c +++ b/opensm/opensm/osm_pkey_mgr.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -66,19 +66,17 @@ rest of the ports (NodeInfo). */ static uint16_t -pkey_mgr_get_physp_max_blocks( - IN const osm_subn_t *p_subn, - IN const osm_physp_t *p_physp ) +pkey_mgr_get_physp_max_blocks(IN const osm_subn_t * p_subn, + IN const osm_physp_t * p_physp) { - osm_node_t *p_node = osm_physp_get_node_ptr( p_physp ); - uint16_t num_pkeys = 0; - - if ( !p_node->sw || - ( osm_physp_get_port_num( p_physp ) == 0 ) ) - num_pkeys = cl_ntoh16( p_node->node_info.partition_cap ); - else - num_pkeys = cl_ntoh16( p_node->sw->switch_info.enforce_cap ); - return((num_pkeys + 31) / 32); + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); + uint16_t num_pkeys = 0; + + if (!p_node->sw || (osm_physp_get_port_num(p_physp) == 0)) + num_pkeys = cl_ntoh16(p_node->node_info.partition_cap); + else + num_pkeys = cl_ntoh16(p_node->sw->switch_info.enforce_cap); + return ((num_pkeys + 31) / 32); } /********************************************************************** @@ -88,495 +86,474 @@ pkey_mgr_get_physp_max_blocks( * pending pkeys. New entries are inserted at the back. */ static void -pkey_mgr_process_physical_port( - IN osm_log_t *p_log, - IN const osm_req_t *p_req, - IN const ib_net16_t pkey, - IN osm_physp_t *p_physp ) +pkey_mgr_process_physical_port(IN osm_log_t * p_log, + IN const osm_req_t * p_req, + IN const ib_net16_t pkey, + IN osm_physp_t * p_physp) { - osm_node_t *p_node = osm_physp_get_node_ptr( p_physp ); - osm_pkey_tbl_t *p_pkey_tbl; - ib_net16_t *p_orig_pkey; - char *stat = NULL; - osm_pending_pkey_t *p_pending; - - p_pkey_tbl = osm_physp_get_mod_pkey_tbl( p_physp ); - p_pending = (osm_pending_pkey_t *)malloc( sizeof( osm_pending_pkey_t ) ); - if (!p_pending) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_process_physical_port: ERR 0502: " - "Failed to allocate new pending pkey entry for node " - "0x%016" PRIx64 " port %u\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); - return; - } - p_pending->pkey = pkey; - p_orig_pkey = cl_map_get( &p_pkey_tbl->keys, ib_pkey_get_base( pkey ) ); - if (!p_orig_pkey) - { - p_pending->is_new = TRUE; - cl_qlist_insert_tail( &p_pkey_tbl->pending, (cl_list_item_t*)p_pending ); - stat = "inserted"; - } - else - { - CL_ASSERT( ib_pkey_get_base( *p_orig_pkey ) == ib_pkey_get_base( pkey ) ); - p_pending->is_new = FALSE; - if (osm_pkey_tbl_get_block_and_idx( - p_pkey_tbl, p_orig_pkey, - &p_pending->block, &p_pending->index ) != IB_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_process_physical_port: ERR 0503: " - "Failed to obtain P_Key 0x%04x block and index for node " - "0x%016" PRIx64 " port %u\n", - ib_pkey_get_base( pkey ), - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); - return; - } - cl_qlist_insert_head( &p_pkey_tbl->pending, (cl_list_item_t*)p_pending ); - stat = "updated"; - } - - osm_log( p_log, OSM_LOG_DEBUG, - "pkey_mgr_process_physical_port: " - "pkey 0x%04x was %s for node 0x%016" PRIx64 - " port %u\n", - cl_ntoh16( pkey ), stat, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); + osm_pkey_tbl_t *p_pkey_tbl; + ib_net16_t *p_orig_pkey; + char *stat = NULL; + osm_pending_pkey_t *p_pending; + + p_pkey_tbl = osm_physp_get_mod_pkey_tbl(p_physp); + p_pending = (osm_pending_pkey_t *) malloc(sizeof(osm_pending_pkey_t)); + if (!p_pending) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_process_physical_port: ERR 0502: " + "Failed to allocate new pending pkey entry for node " + "0x%016" PRIx64 " port %u\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); + return; + } + p_pending->pkey = pkey; + p_orig_pkey = cl_map_get(&p_pkey_tbl->keys, ib_pkey_get_base(pkey)); + if (!p_orig_pkey) { + p_pending->is_new = TRUE; + cl_qlist_insert_tail(&p_pkey_tbl->pending, + (cl_list_item_t *) p_pending); + stat = "inserted"; + } else { + CL_ASSERT(ib_pkey_get_base(*p_orig_pkey) == + ib_pkey_get_base(pkey)); + p_pending->is_new = FALSE; + if (osm_pkey_tbl_get_block_and_idx(p_pkey_tbl, p_orig_pkey, + &p_pending->block, + &p_pending->index) != + IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_process_physical_port: ERR 0503: " + "Failed to obtain P_Key 0x%04x block and index for node " + "0x%016" PRIx64 " port %u\n", + ib_pkey_get_base(pkey), + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); + return; + } + cl_qlist_insert_head(&p_pkey_tbl->pending, + (cl_list_item_t *) p_pending); + stat = "updated"; + } + + osm_log(p_log, OSM_LOG_DEBUG, + "pkey_mgr_process_physical_port: " + "pkey 0x%04x was %s for node 0x%016" PRIx64 + " port %u\n", + cl_ntoh16(pkey), stat, + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); } /********************************************************************** **********************************************************************/ static void -pkey_mgr_process_partition_table( - osm_log_t *p_log, - const osm_req_t *p_req, - const osm_prtn_t *p_prtn, - const boolean_t full ) +pkey_mgr_process_partition_table(osm_log_t * p_log, + const osm_req_t * p_req, + const osm_prtn_t * p_prtn, + const boolean_t full) { - const cl_map_t *p_tbl = - full ? &p_prtn->full_guid_tbl : &p_prtn->part_guid_tbl; - cl_map_iterator_t i, i_next; - ib_net16_t pkey = p_prtn->pkey; - osm_physp_t *p_physp; - - if (full) - pkey |= cl_hton16( 0x8000 ); - - i_next = cl_map_head( p_tbl ); - while ( i_next != cl_map_end( p_tbl ) ) - { - i = i_next; - i_next = cl_map_next( i ); - p_physp = cl_map_obj( i ); - if ( p_physp && osm_physp_is_valid( p_physp ) ) - pkey_mgr_process_physical_port( p_log, p_req, pkey, p_physp ); - } + const cl_map_t *p_tbl = + full ? &p_prtn->full_guid_tbl : &p_prtn->part_guid_tbl; + cl_map_iterator_t i, i_next; + ib_net16_t pkey = p_prtn->pkey; + osm_physp_t *p_physp; + + if (full) + pkey |= cl_hton16(0x8000); + + i_next = cl_map_head(p_tbl); + while (i_next != cl_map_end(p_tbl)) { + i = i_next; + i_next = cl_map_next(i); + p_physp = cl_map_obj(i); + if (p_physp && osm_physp_is_valid(p_physp)) + pkey_mgr_process_physical_port(p_log, p_req, pkey, + p_physp); + } } /********************************************************************** **********************************************************************/ static ib_api_status_t -pkey_mgr_update_pkey_entry( - IN const osm_req_t *p_req, - IN const osm_physp_t *p_physp, - IN const ib_pkey_table_t *block, - IN const uint16_t block_index ) +pkey_mgr_update_pkey_entry(IN const osm_req_t * p_req, + IN const osm_physp_t * p_physp, + IN const ib_pkey_table_t * block, + IN const uint16_t block_index) { - osm_madw_context_t context; - osm_node_t *p_node = osm_physp_get_node_ptr( p_physp ); - uint32_t attr_mod; - - context.pkey_context.node_guid = osm_node_get_node_guid( p_node ); - context.pkey_context.port_guid = osm_physp_get_port_guid( p_physp ); - context.pkey_context.set_method = TRUE; - attr_mod = block_index; - if ( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ) - attr_mod |= osm_physp_get_port_num( p_physp ) << 16; - return osm_req_set( p_req, osm_physp_get_dr_path_ptr( p_physp ), - ( uint8_t * ) block, sizeof( *block ), - IB_MAD_ATTR_P_KEY_TABLE, - cl_hton32( attr_mod ), CL_DISP_MSGID_NONE, &context ); + osm_madw_context_t context; + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); + uint32_t attr_mod; + + context.pkey_context.node_guid = osm_node_get_node_guid(p_node); + context.pkey_context.port_guid = osm_physp_get_port_guid(p_physp); + context.pkey_context.set_method = TRUE; + attr_mod = block_index; + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) + attr_mod |= osm_physp_get_port_num(p_physp) << 16; + return osm_req_set(p_req, osm_physp_get_dr_path_ptr(p_physp), + (uint8_t *) block, sizeof(*block), + IB_MAD_ATTR_P_KEY_TABLE, + cl_hton32(attr_mod), CL_DISP_MSGID_NONE, &context); } /********************************************************************** **********************************************************************/ static boolean_t -pkey_mgr_enforce_partition( - IN osm_log_t *p_log, - IN const osm_req_t *p_req, - IN osm_physp_t *p_physp, - IN const boolean_t enforce) +pkey_mgr_enforce_partition(IN osm_log_t * p_log, + IN const osm_req_t * p_req, + IN osm_physp_t * p_physp, IN const boolean_t enforce) { - osm_madw_context_t context; - uint8_t payload[IB_SMP_DATA_SIZE]; - ib_port_info_t *p_pi; - ib_api_status_t status; - - p_pi = &p_physp->port_info; - - if ((p_pi->vl_enforce & 0xc) == (0xc)*(enforce == TRUE)) - { - osm_log( p_log, OSM_LOG_DEBUG, - "pkey_mgr_enforce_partition: " - "No need to update PortInfo for " - "node 0x%016" PRIx64 " port %u\n", - cl_ntoh64( - osm_node_get_node_guid( - osm_physp_get_node_ptr( p_physp ))), - osm_physp_get_port_num( p_physp ) ); - return FALSE; - } - - memset( payload, 0, IB_SMP_DATA_SIZE ); - memcpy( payload, p_pi, sizeof(ib_port_info_t) ); - - p_pi = (ib_port_info_t*)payload; - if (enforce == TRUE) - p_pi->vl_enforce |= 0xc; - else - p_pi->vl_enforce &= ~0xc; - p_pi->state_info2 = 0; - ib_port_info_set_port_state( p_pi, IB_LINK_NO_CHANGE ); - - context.pi_context.node_guid = - osm_node_get_node_guid( osm_physp_get_node_ptr( p_physp ) ); - context.pi_context.port_guid = osm_physp_get_port_guid( p_physp ); - context.pi_context.set_method = TRUE; - context.pi_context.update_master_sm_base_lid = FALSE; - context.pi_context.ignore_errors = FALSE; - context.pi_context.light_sweep = FALSE; - context.pi_context.active_transition = FALSE; - - status = osm_req_set( p_req, osm_physp_get_dr_path_ptr( p_physp ), - payload, sizeof(payload), - IB_MAD_ATTR_PORT_INFO, - cl_hton32( osm_physp_get_port_num( p_physp ) ), - CL_DISP_MSGID_NONE, &context ); - if (status != IB_SUCCESS) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_enforce_partition: ERR 0511: " - "Failed to set PortInfo for " - "node 0x%016" PRIx64 " port %u\n", - cl_ntoh64( - osm_node_get_node_guid( - osm_physp_get_node_ptr( p_physp ))), - osm_physp_get_port_num( p_physp ) ); - return FALSE; - } - else - { - osm_log( p_log, OSM_LOG_DEBUG, - "pkey_mgr_enforce_partition: " - "Set PortInfo for " - "node 0x%016" PRIx64 " port %u\n", - cl_ntoh64( - osm_node_get_node_guid( - osm_physp_get_node_ptr( p_physp ))), - osm_physp_get_port_num( p_physp ) ); - return TRUE; - } + osm_madw_context_t context; + uint8_t payload[IB_SMP_DATA_SIZE]; + ib_port_info_t *p_pi; + ib_api_status_t status; + + p_pi = &p_physp->port_info; + + if ((p_pi->vl_enforce & 0xc) == (0xc) * (enforce == TRUE)) { + osm_log(p_log, OSM_LOG_DEBUG, + "pkey_mgr_enforce_partition: " + "No need to update PortInfo for " + "node 0x%016" PRIx64 " port %u\n", + cl_ntoh64(osm_node_get_node_guid + (osm_physp_get_node_ptr(p_physp))), + osm_physp_get_port_num(p_physp)); + return FALSE; + } + + memset(payload, 0, IB_SMP_DATA_SIZE); + memcpy(payload, p_pi, sizeof(ib_port_info_t)); + + p_pi = (ib_port_info_t *) payload; + if (enforce == TRUE) + p_pi->vl_enforce |= 0xc; + else + p_pi->vl_enforce &= ~0xc; + p_pi->state_info2 = 0; + ib_port_info_set_port_state(p_pi, IB_LINK_NO_CHANGE); + + context.pi_context.node_guid = + osm_node_get_node_guid(osm_physp_get_node_ptr(p_physp)); + context.pi_context.port_guid = osm_physp_get_port_guid(p_physp); + context.pi_context.set_method = TRUE; + context.pi_context.update_master_sm_base_lid = FALSE; + context.pi_context.ignore_errors = FALSE; + context.pi_context.light_sweep = FALSE; + context.pi_context.active_transition = FALSE; + + status = osm_req_set(p_req, osm_physp_get_dr_path_ptr(p_physp), + payload, sizeof(payload), + IB_MAD_ATTR_PORT_INFO, + cl_hton32(osm_physp_get_port_num(p_physp)), + CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_enforce_partition: ERR 0511: " + "Failed to set PortInfo for " + "node 0x%016" PRIx64 " port %u\n", + cl_ntoh64(osm_node_get_node_guid + (osm_physp_get_node_ptr(p_physp))), + osm_physp_get_port_num(p_physp)); + return FALSE; + } else { + osm_log(p_log, OSM_LOG_DEBUG, + "pkey_mgr_enforce_partition: " + "Set PortInfo for " + "node 0x%016" PRIx64 " port %u\n", + cl_ntoh64(osm_node_get_node_guid + (osm_physp_get_node_ptr(p_physp))), + osm_physp_get_port_num(p_physp)); + return TRUE; + } } /********************************************************************** **********************************************************************/ -static boolean_t pkey_mgr_update_port( - osm_log_t *p_log, - osm_req_t *p_req, - const osm_port_t * const p_port ) +static boolean_t pkey_mgr_update_port(osm_log_t * p_log, + osm_req_t * p_req, + const osm_port_t * const p_port) { - osm_physp_t *p_physp; - osm_node_t *p_node; - ib_pkey_table_t *block, *new_block; - osm_pkey_tbl_t *p_pkey_tbl; - uint16_t block_index; - uint8_t pkey_index; - uint16_t last_free_block_index = 0; - uint8_t last_free_pkey_index = 0; - uint16_t num_of_blocks; - uint16_t max_num_of_blocks; - ib_api_status_t status; - boolean_t ret_val = FALSE; - osm_pending_pkey_t *p_pending; - boolean_t found; - ib_pkey_table_t empty_block; - - memset(&empty_block, 0, sizeof(ib_pkey_table_t)); - - p_physp = p_port->p_physp; - if ( !osm_physp_is_valid( p_physp ) ) - return FALSE; - - p_node = osm_physp_get_node_ptr( p_physp ); - p_pkey_tbl = osm_physp_get_mod_pkey_tbl( p_physp ); - num_of_blocks = osm_pkey_tbl_get_num_blocks( p_pkey_tbl ); - max_num_of_blocks = pkey_mgr_get_physp_max_blocks( p_req->p_subn, p_physp ); - if ( p_pkey_tbl->max_blocks > max_num_of_blocks ) - { - osm_log( p_log, OSM_LOG_INFO, - "pkey_mgr_update_port: " - "Max number of blocks reduced from %u to %u " - "for node 0x%016" PRIx64 " port %u\n", - p_pkey_tbl->max_blocks, max_num_of_blocks, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); - } - p_pkey_tbl->max_blocks = max_num_of_blocks; - - osm_pkey_tbl_init_new_blocks( p_pkey_tbl ); - p_pkey_tbl->used_blocks = 0; - - /* - process every pending pkey in order - - first must be "updated" last are "new" - */ - p_pending = (osm_pending_pkey_t *)cl_qlist_remove_head( &p_pkey_tbl->pending ); - while ( p_pending != (osm_pending_pkey_t *)cl_qlist_end( &p_pkey_tbl->pending )) - { - if (p_pending->is_new == FALSE) - { - block_index = p_pending->block; - pkey_index = p_pending->index; - found = TRUE; - } - else - { - found = osm_pkey_find_next_free_entry( p_pkey_tbl, - &last_free_block_index, - &last_free_pkey_index ); - if (!found) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_update_port: ERR 0504: " - "Failed to find empty space for new pkey 0x%04x " - "for node 0x%016" PRIx64 " port %u\n", - cl_ntoh16( p_pending->pkey ), - cl_ntoh64( osm_node_get_node_guid( p_node )), - osm_physp_get_port_num( p_physp ) ); + osm_physp_t *p_physp; + osm_node_t *p_node; + ib_pkey_table_t *block, *new_block; + osm_pkey_tbl_t *p_pkey_tbl; + uint16_t block_index; + uint8_t pkey_index; + uint16_t last_free_block_index = 0; + uint8_t last_free_pkey_index = 0; + uint16_t num_of_blocks; + uint16_t max_num_of_blocks; + ib_api_status_t status; + boolean_t ret_val = FALSE; + osm_pending_pkey_t *p_pending; + boolean_t found; + ib_pkey_table_t empty_block; + + memset(&empty_block, 0, sizeof(ib_pkey_table_t)); + + p_physp = p_port->p_physp; + if (!osm_physp_is_valid(p_physp)) + return FALSE; + + p_node = osm_physp_get_node_ptr(p_physp); + p_pkey_tbl = osm_physp_get_mod_pkey_tbl(p_physp); + num_of_blocks = osm_pkey_tbl_get_num_blocks(p_pkey_tbl); + max_num_of_blocks = + pkey_mgr_get_physp_max_blocks(p_req->p_subn, p_physp); + if (p_pkey_tbl->max_blocks > max_num_of_blocks) { + osm_log(p_log, OSM_LOG_INFO, + "pkey_mgr_update_port: " + "Max number of blocks reduced from %u to %u " + "for node 0x%016" PRIx64 " port %u\n", + p_pkey_tbl->max_blocks, max_num_of_blocks, + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); } - else - { - block_index = last_free_block_index; - pkey_index = last_free_pkey_index++; + p_pkey_tbl->max_blocks = max_num_of_blocks; + + osm_pkey_tbl_init_new_blocks(p_pkey_tbl); + p_pkey_tbl->used_blocks = 0; + + /* + process every pending pkey in order - + first must be "updated" last are "new" + */ + p_pending = + (osm_pending_pkey_t *) cl_qlist_remove_head(&p_pkey_tbl->pending); + while (p_pending != + (osm_pending_pkey_t *) cl_qlist_end(&p_pkey_tbl->pending)) { + if (p_pending->is_new == FALSE) { + block_index = p_pending->block; + pkey_index = p_pending->index; + found = TRUE; + } else { + found = osm_pkey_find_next_free_entry(p_pkey_tbl, + &last_free_block_index, + &last_free_pkey_index); + if (!found) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_update_port: ERR 0504: " + "Failed to find empty space for new pkey 0x%04x " + "for node 0x%016" PRIx64 " port %u\n", + cl_ntoh16(p_pending->pkey), + cl_ntoh64(osm_node_get_node_guid + (p_node)), + osm_physp_get_port_num(p_physp)); + } else { + block_index = last_free_block_index; + pkey_index = last_free_pkey_index++; + } + } + + if (found) { + if (IB_SUCCESS != + osm_pkey_tbl_set_new_entry(p_pkey_tbl, block_index, + pkey_index, + p_pending->pkey)) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_update_port: ERR 0505: " + "Failed to set PKey 0x%04x in block %u idx %u " + "for node 0x%016" PRIx64 " port %u\n", + p_pending->pkey, block_index, + pkey_index, + cl_ntoh64(osm_node_get_node_guid + (p_node)), + osm_physp_get_port_num(p_physp)); + } + } + + free(p_pending); + p_pending = + (osm_pending_pkey_t *) cl_qlist_remove_head(&p_pkey_tbl-> + pending); } - } - - if (found) - { - if (IB_SUCCESS != osm_pkey_tbl_set_new_entry( p_pkey_tbl, block_index, - pkey_index, p_pending->pkey )) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_update_port: ERR 0505: " - "Failed to set PKey 0x%04x in block %u idx %u " - "for node 0x%016" PRIx64 " port %u\n", - p_pending->pkey, block_index, pkey_index, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); + + /* now look for changes and store */ + for (block_index = 0; block_index < num_of_blocks; block_index++) { + block = osm_pkey_tbl_block_get(p_pkey_tbl, block_index); + new_block = osm_pkey_tbl_new_block_get(p_pkey_tbl, block_index); + if (!new_block) + new_block = &empty_block; + if (block && !memcmp(new_block, block, sizeof(*block))) + continue; + + status = + pkey_mgr_update_pkey_entry(p_req, p_physp, new_block, + block_index); + if (status == IB_SUCCESS) { + osm_log(p_log, OSM_LOG_DEBUG, + "pkey_mgr_update_port: " + "Updated " + "pkey table block %d for node 0x%016" PRIx64 + " port %u\n", block_index, + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); + ret_val = TRUE; + } else { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_update_port: ERR 0506: " + "pkey_mgr_update_pkey_entry() failed to update " + "pkey table block %d for node 0x%016" PRIx64 + " port %u\n", block_index, + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(p_physp)); + } } - } - - free(p_pending); - p_pending = (osm_pending_pkey_t *)cl_qlist_remove_head( &p_pkey_tbl->pending ); - } - - /* now look for changes and store */ - for (block_index = 0; block_index < num_of_blocks; block_index++) - { - block = osm_pkey_tbl_block_get( p_pkey_tbl, block_index ); - new_block = osm_pkey_tbl_new_block_get( p_pkey_tbl, block_index ); - if (!new_block) - new_block = &empty_block; - if (block && !memcmp( new_block, block, sizeof( *block ) )) - continue; - - status = pkey_mgr_update_pkey_entry( p_req, p_physp, new_block, block_index ); - if (status == IB_SUCCESS) - { - osm_log( p_log, OSM_LOG_DEBUG, - "pkey_mgr_update_port: " - "Updated " - "pkey table block %d for node 0x%016" PRIx64 " port %u\n", - block_index, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); - ret_val = TRUE; - } - else - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_update_port: ERR 0506: " - "pkey_mgr_update_pkey_entry() failed to update " - "pkey table block %d for node 0x%016" PRIx64 " port %u\n", - block_index, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( p_physp ) ); - } - } - - return ret_val; + + return ret_val; } /********************************************************************** **********************************************************************/ static boolean_t -pkey_mgr_update_peer_port( - osm_log_t *p_log, - const osm_req_t *p_req, - const osm_subn_t *p_subn, - const osm_port_t * const p_port, - boolean_t enforce ) +pkey_mgr_update_peer_port(osm_log_t * p_log, + const osm_req_t * p_req, + const osm_subn_t * p_subn, + const osm_port_t * const p_port, boolean_t enforce) { - osm_physp_t *p_physp, *peer; - osm_node_t *p_node; - ib_pkey_table_t *block, *peer_block; - const osm_pkey_tbl_t *p_pkey_tbl; - osm_pkey_tbl_t *p_peer_pkey_tbl; - uint16_t block_index; - uint16_t num_of_blocks; - uint16_t peer_max_blocks; - ib_api_status_t status = IB_SUCCESS; - boolean_t ret_val = FALSE; - boolean_t port_info_set = FALSE; - ib_pkey_table_t empty_block; - - memset(&empty_block, 0, sizeof(ib_pkey_table_t)); - - p_physp = p_port->p_physp; - if (!osm_physp_is_valid( p_physp )) - return FALSE; - peer = osm_physp_get_remote( p_physp ); - if ( !peer || !osm_physp_is_valid( peer ) ) - return FALSE; - p_node = osm_physp_get_node_ptr( peer ); - if ( !p_node->sw || !p_node->sw->switch_info.enforce_cap ) - return FALSE; - - p_pkey_tbl = osm_physp_get_pkey_tbl( p_physp ); - p_peer_pkey_tbl = osm_physp_get_mod_pkey_tbl( peer ); - num_of_blocks = osm_pkey_tbl_get_num_blocks( p_pkey_tbl ); - peer_max_blocks = pkey_mgr_get_physp_max_blocks( p_subn, peer ); - if (peer_max_blocks < p_pkey_tbl->used_blocks) - { - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_update_peer_port: ERR 0508: " - "Not enough pkey entries (%u < %u) on switch 0x%016" PRIx64 - " port %u. Clearing Enforcement bit\n", - peer_max_blocks, num_of_blocks, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( peer ) ); - enforce = FALSE; - } - - if ( pkey_mgr_enforce_partition( p_log, p_req, peer, enforce ) ) - port_info_set = TRUE; - - if (enforce == FALSE) - return port_info_set; - - p_peer_pkey_tbl->used_blocks = p_pkey_tbl->used_blocks; - for (block_index = 0; block_index < p_pkey_tbl->used_blocks; block_index++) - { - block = osm_pkey_tbl_new_block_get( p_pkey_tbl, block_index ); - if (!block) - block = &empty_block; - - peer_block = osm_pkey_tbl_block_get( p_peer_pkey_tbl, block_index ); - if ( !peer_block || memcmp( peer_block, block, sizeof( *peer_block ) ) ) - { - status = pkey_mgr_update_pkey_entry( p_req, peer, block, block_index ); - if ( status == IB_SUCCESS ) - ret_val = TRUE; - else - osm_log( p_log, OSM_LOG_ERROR, - "pkey_mgr_update_peer_port: ERR 0509: " - "pkey_mgr_update_pkey_entry() failed to update " - "pkey table block %d for node 0x%016" PRIx64 - " port %u\n", - block_index, - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( peer ) ); - } - } - - if ( (ret_val == TRUE) && osm_log_is_active( p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "pkey_mgr_update_peer_port: " - "Pkey table was updated for node 0x%016" PRIx64 - " port %u\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - osm_physp_get_port_num( peer ) ); - } - - if (port_info_set) - return TRUE; - return ret_val; + osm_physp_t *p_physp, *peer; + osm_node_t *p_node; + ib_pkey_table_t *block, *peer_block; + const osm_pkey_tbl_t *p_pkey_tbl; + osm_pkey_tbl_t *p_peer_pkey_tbl; + uint16_t block_index; + uint16_t num_of_blocks; + uint16_t peer_max_blocks; + ib_api_status_t status = IB_SUCCESS; + boolean_t ret_val = FALSE; + boolean_t port_info_set = FALSE; + ib_pkey_table_t empty_block; + + memset(&empty_block, 0, sizeof(ib_pkey_table_t)); + + p_physp = p_port->p_physp; + if (!osm_physp_is_valid(p_physp)) + return FALSE; + peer = osm_physp_get_remote(p_physp); + if (!peer || !osm_physp_is_valid(peer)) + return FALSE; + p_node = osm_physp_get_node_ptr(peer); + if (!p_node->sw || !p_node->sw->switch_info.enforce_cap) + return FALSE; + + p_pkey_tbl = osm_physp_get_pkey_tbl(p_physp); + p_peer_pkey_tbl = osm_physp_get_mod_pkey_tbl(peer); + num_of_blocks = osm_pkey_tbl_get_num_blocks(p_pkey_tbl); + peer_max_blocks = pkey_mgr_get_physp_max_blocks(p_subn, peer); + if (peer_max_blocks < p_pkey_tbl->used_blocks) { + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_update_peer_port: ERR 0508: " + "Not enough pkey entries (%u < %u) on switch 0x%016" + PRIx64 " port %u. Clearing Enforcement bit\n", + peer_max_blocks, num_of_blocks, + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(peer)); + enforce = FALSE; + } + + if (pkey_mgr_enforce_partition(p_log, p_req, peer, enforce)) + port_info_set = TRUE; + + if (enforce == FALSE) + return port_info_set; + + p_peer_pkey_tbl->used_blocks = p_pkey_tbl->used_blocks; + for (block_index = 0; block_index < p_pkey_tbl->used_blocks; + block_index++) { + block = osm_pkey_tbl_new_block_get(p_pkey_tbl, block_index); + if (!block) + block = &empty_block; + + peer_block = + osm_pkey_tbl_block_get(p_peer_pkey_tbl, block_index); + if (!peer_block + || memcmp(peer_block, block, sizeof(*peer_block))) { + status = + pkey_mgr_update_pkey_entry(p_req, peer, block, + block_index); + if (status == IB_SUCCESS) + ret_val = TRUE; + else + osm_log(p_log, OSM_LOG_ERROR, + "pkey_mgr_update_peer_port: ERR 0509: " + "pkey_mgr_update_pkey_entry() failed to update " + "pkey table block %d for node 0x%016" + PRIx64 " port %u\n", block_index, + cl_ntoh64(osm_node_get_node_guid + (p_node)), + osm_physp_get_port_num(peer)); + } + } + + if ((ret_val == TRUE) && osm_log_is_active(p_log, OSM_LOG_DEBUG)) { + osm_log(p_log, OSM_LOG_DEBUG, + "pkey_mgr_update_peer_port: " + "Pkey table was updated for node 0x%016" PRIx64 + " port %u\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + osm_physp_get_port_num(peer)); + } + + if (port_info_set) + return TRUE; + return ret_val; } /********************************************************************** **********************************************************************/ -osm_signal_t -osm_pkey_mgr_process( - IN osm_opensm_t *p_osm ) +osm_signal_t osm_pkey_mgr_process(IN osm_opensm_t * p_osm) { - cl_qmap_t *p_tbl; - cl_map_item_t *p_next; - osm_prtn_t *p_prtn; - osm_port_t *p_port; - osm_signal_t signal = OSM_SIGNAL_DONE; - - CL_ASSERT( p_osm ); - - OSM_LOG_ENTER( &p_osm->log, osm_pkey_mgr_process ); - - CL_PLOCK_EXCL_ACQUIRE( &p_osm->lock ); - - if ( osm_prtn_make_partitions( &p_osm->log, &p_osm->subn ) != IB_SUCCESS ) - { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "osm_pkey_mgr_process: ERR 0510: " - "osm_prtn_make_partitions() failed\n" ); - goto _err; - } - - /* populate the pending pkey entries by scanning all partitions */ - p_tbl = &p_osm->subn.prtn_pkey_tbl; - p_next = cl_qmap_head( p_tbl ); - while ( p_next != cl_qmap_end( p_tbl ) ) - { - p_prtn = ( osm_prtn_t * ) p_next; - p_next = cl_qmap_next( p_next ); - pkey_mgr_process_partition_table( &p_osm->log, &p_osm->sm.req, - p_prtn, FALSE ); - pkey_mgr_process_partition_table( &p_osm->log, &p_osm->sm.req, - p_prtn, TRUE ); - } - - /* calculate and set new pkey tables */ - p_tbl = &p_osm->subn.port_guid_tbl; - p_next = cl_qmap_head( p_tbl ); - while ( p_next != cl_qmap_end( p_tbl ) ) - { - p_port = ( osm_port_t * ) p_next; - p_next = cl_qmap_next( p_next ); - if ( pkey_mgr_update_port( &p_osm->log, &p_osm->sm.req, p_port ) ) - signal = OSM_SIGNAL_DONE_PENDING; - if ( ( osm_node_get_type( p_port->p_node ) != IB_NODE_TYPE_SWITCH ) && - pkey_mgr_update_peer_port( &p_osm->log, &p_osm->sm.req, - &p_osm->subn, p_port, - !p_osm->subn.opt.no_partition_enforcement ) ) - signal = OSM_SIGNAL_DONE_PENDING; - } - - _err: - CL_PLOCK_RELEASE( &p_osm->lock ); - OSM_LOG_EXIT( &p_osm->log ); - return ( signal ); + cl_qmap_t *p_tbl; + cl_map_item_t *p_next; + osm_prtn_t *p_prtn; + osm_port_t *p_port; + osm_signal_t signal = OSM_SIGNAL_DONE; + + CL_ASSERT(p_osm); + + OSM_LOG_ENTER(&p_osm->log, osm_pkey_mgr_process); + + CL_PLOCK_EXCL_ACQUIRE(&p_osm->lock); + + if (osm_prtn_make_partitions(&p_osm->log, &p_osm->subn) != IB_SUCCESS) { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "osm_pkey_mgr_process: ERR 0510: " + "osm_prtn_make_partitions() failed\n"); + goto _err; + } + + /* populate the pending pkey entries by scanning all partitions */ + p_tbl = &p_osm->subn.prtn_pkey_tbl; + p_next = cl_qmap_head(p_tbl); + while (p_next != cl_qmap_end(p_tbl)) { + p_prtn = (osm_prtn_t *) p_next; + p_next = cl_qmap_next(p_next); + pkey_mgr_process_partition_table(&p_osm->log, &p_osm->sm.req, + p_prtn, FALSE); + pkey_mgr_process_partition_table(&p_osm->log, &p_osm->sm.req, + p_prtn, TRUE); + } + + /* calculate and set new pkey tables */ + p_tbl = &p_osm->subn.port_guid_tbl; + p_next = cl_qmap_head(p_tbl); + while (p_next != cl_qmap_end(p_tbl)) { + p_port = (osm_port_t *) p_next; + p_next = cl_qmap_next(p_next); + if (pkey_mgr_update_port(&p_osm->log, &p_osm->sm.req, p_port)) + signal = OSM_SIGNAL_DONE_PENDING; + if ((osm_node_get_type(p_port->p_node) != IB_NODE_TYPE_SWITCH) + && pkey_mgr_update_peer_port(&p_osm->log, &p_osm->sm.req, + &p_osm->subn, p_port, + !p_osm->subn.opt. + no_partition_enforcement)) + signal = OSM_SIGNAL_DONE_PENDING; + } + + _err: + CL_PLOCK_RELEASE(&p_osm->lock); + OSM_LOG_EXIT(&p_osm->log); + return (signal); } diff --git a/opensm/opensm/osm_pkey_rcv.c b/opensm/opensm/osm_pkey_rcv.c index fae6dd3..d045fa2 100644 --- a/opensm/opensm/osm_pkey_rcv.c +++ b/opensm/opensm/osm_pkey_rcv.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,49 +56,43 @@ /********************************************************************** **********************************************************************/ -void -osm_pkey_rcv_construct( - IN osm_pkey_rcv_t* const p_rcv ) +void osm_pkey_rcv_construct(IN osm_pkey_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_pkey_rcv_destroy( - IN osm_pkey_rcv_t* const p_rcv ) +void osm_pkey_rcv_destroy(IN osm_pkey_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_pkey_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_pkey_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pkey_rcv_init( - IN osm_pkey_rcv_t* const p_rcv, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_pkey_rcv_init(IN osm_pkey_rcv_t * const p_rcv, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_pkey_rcv_init ); + OSM_LOG_ENTER(p_log, osm_pkey_rcv_init); - osm_pkey_rcv_construct( p_rcv ); + osm_pkey_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_req = p_req; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_req = p_req; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** @@ -106,111 +100,98 @@ osm_pkey_rcv_init( /* * WE MIGHT ONLY RECEIVE GET or SET responses */ -void -osm_pkey_rcv_process( - IN void *context, - IN void *data ) +void osm_pkey_rcv_process(IN void *context, IN void *data) { - osm_pkey_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_pkey_table_t *p_pkey_tbl; - ib_smp_t *p_smp; - osm_port_t *p_port; - osm_physp_t *p_physp; - osm_node_t *p_node; - osm_pkey_context_t *p_context; - ib_net64_t port_guid; - ib_net64_t node_guid; - uint8_t port_num; - uint16_t block_num; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_pkey_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - p_context = osm_madw_get_pkey_context_ptr( p_madw ); - p_pkey_tbl = (ib_pkey_table_t*)ib_smp_get_payload_ptr( p_smp ); - - port_guid = p_context->port_guid; - node_guid = p_context->node_guid; - - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_P_KEY_TABLE ); - - cl_plock_excl_acquire( p_rcv->p_lock ); - p_port = osm_get_port_by_guid( p_rcv->p_subn, port_guid ); - if( !p_port ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rcv_process: ERR 4806: " - "No port object for port with GUID 0x%" PRIx64 - "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - goto Exit; - } - - p_node = p_port->p_node; - CL_ASSERT( p_node ); - - block_num = (uint16_t)((cl_ntoh32(p_smp->attr_mod)) & 0x0000FFFF); - /* in case of a non switch node the attr modifier should be ignored */ - if (osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH) - { - port_num = (uint8_t)(((cl_ntoh32( p_smp->attr_mod)) & 0x00FF0000) >> 16 ); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - } - else - { - p_physp = p_port->p_physp; - port_num = p_physp->port_num; - } - - /* - We do not mind if this is a result of a set or get - all we want is to - update the subnet. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_pkey_rcv_process: " - "Got GetResp(PKey) block:%u port_num %u with GUID 0x%" PRIx64 - " for parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - block_num, port_num, - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - } - - /* - Determine if we encountered a new Physical Port. - If so, ignore it. - */ - if( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rcv_process: ERR 4807: " - "Got invalid port number 0x%X\n", - port_num ); - goto Exit; - } - - osm_dump_pkey_block( p_rcv->p_log, - port_guid, block_num, - port_num, p_pkey_tbl, - OSM_LOG_DEBUG ); - - osm_physp_set_pkey_tbl( p_rcv->p_log, p_rcv->p_subn, - p_physp, p_pkey_tbl, block_num ); - - Exit: - cl_plock_release( p_rcv->p_lock ); - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_pkey_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_pkey_table_t *p_pkey_tbl; + ib_smp_t *p_smp; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_node_t *p_node; + osm_pkey_context_t *p_context; + ib_net64_t port_guid; + ib_net64_t node_guid; + uint8_t port_num; + uint16_t block_num; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_pkey_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + p_context = osm_madw_get_pkey_context_ptr(p_madw); + p_pkey_tbl = (ib_pkey_table_t *) ib_smp_get_payload_ptr(p_smp); + + port_guid = p_context->port_guid; + node_guid = p_context->node_guid; + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_P_KEY_TABLE); + + cl_plock_excl_acquire(p_rcv->p_lock); + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rcv_process: ERR 4806: " + "No port object for port with GUID 0x%" PRIx64 + "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 + ", TID 0x%" PRIx64 "\n", + cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + goto Exit; + } + + p_node = p_port->p_node; + CL_ASSERT(p_node); + + block_num = (uint16_t) ((cl_ntoh32(p_smp->attr_mod)) & 0x0000FFFF); + /* in case of a non switch node the attr modifier should be ignored */ + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) { + port_num = + (uint8_t) (((cl_ntoh32(p_smp->attr_mod)) & 0x00FF0000) >> + 16); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + } else { + p_physp = p_port->p_physp; + port_num = p_physp->port_num; + } + + /* + We do not mind if this is a result of a set or get - all we want is to + update the subnet. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_pkey_rcv_process: " + "Got GetResp(PKey) block:%u port_num %u with GUID 0x%" + PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + } + + /* + Determine if we encountered a new Physical Port. + If so, ignore it. + */ + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rcv_process: ERR 4807: " + "Got invalid port number 0x%X\n", port_num); + goto Exit; + } + + osm_dump_pkey_block(p_rcv->p_log, + port_guid, block_num, + port_num, p_pkey_tbl, OSM_LOG_DEBUG); + + osm_physp_set_pkey_tbl(p_rcv->p_log, p_rcv->p_subn, + p_physp, p_pkey_tbl, block_num); + + Exit: + cl_plock_release(p_rcv->p_lock); + + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index ebf5889..5dabae9 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,9 +65,7 @@ extern int osm_prtn_config_parse_file(osm_log_t * const p_log, static uint16_t global_pkey_counter; -osm_prtn_t* osm_prtn_new( - IN const char *name, - IN const uint16_t pkey ) +osm_prtn_t *osm_prtn_new(IN const char *name, IN const uint16_t pkey) { osm_prtn_t *p = malloc(sizeof(*p)); if (!p) @@ -89,8 +87,7 @@ osm_prtn_t* osm_prtn_new( return p; } -void osm_prtn_delete( - IN OUT osm_prtn_t** const pp_prtn ) +void osm_prtn_delete(IN OUT osm_prtn_t ** const pp_prtn) { osm_prtn_t *p = *pp_prtn; @@ -102,8 +99,9 @@ void osm_prtn_delete( *pp_prtn = NULL; } -ib_api_status_t osm_prtn_add_port(osm_log_t *p_log, osm_subn_t *p_subn, - osm_prtn_t *p, ib_net64_t guid, boolean_t full) +ib_api_status_t osm_prtn_add_port(osm_log_t * p_log, osm_subn_t * p_subn, + osm_prtn_t * p, ib_net64_t guid, + boolean_t full) { ib_api_status_t status = IB_SUCCESS; cl_map_t *p_tbl; @@ -113,8 +111,7 @@ ib_api_status_t osm_prtn_add_port(osm_log_t *p_log, osm_subn_t *p_subn, p_port = osm_get_port_by_guid(p_subn, guid); if (!p_port) { osm_log(p_log, OSM_LOG_VERBOSE, "osm_prtn_add_port: " - "port 0x%" PRIx64 " not found\n", - cl_ntoh64(guid)); + "port 0x%" PRIx64 " not found\n", cl_ntoh64(guid)); return status; } @@ -134,7 +131,7 @@ ib_api_status_t osm_prtn_add_port(osm_log_t *p_log, osm_subn_t *p_subn, cl_ntoh64(guid), p->name, cl_ntoh16(p->pkey)); } - p_tbl = (full == TRUE) ? &p->full_guid_tbl : &p->part_guid_tbl ; + p_tbl = (full == TRUE) ? &p->full_guid_tbl : &p->part_guid_tbl; if (cl_map_insert(p_tbl, guid, p_physp) == NULL) return IB_INSUFFICIENT_MEMORY; @@ -142,8 +139,8 @@ ib_api_status_t osm_prtn_add_port(osm_log_t *p_log, osm_subn_t *p_subn, return status; } -ib_api_status_t osm_prtn_add_all(osm_log_t *p_log, osm_subn_t *p_subn, - osm_prtn_t *p, boolean_t full) +ib_api_status_t osm_prtn_add_all(osm_log_t * p_log, osm_subn_t * p_subn, + osm_prtn_t * p, boolean_t full) { cl_qmap_t *p_port_tbl = &p_subn->port_guid_tbl; cl_map_item_t *p_item; @@ -152,7 +149,7 @@ ib_api_status_t osm_prtn_add_all(osm_log_t *p_log, osm_subn_t *p_subn, p_item = cl_qmap_head(p_port_tbl); while (p_item != cl_qmap_end(p_port_tbl)) { - p_port = (osm_port_t *)p_item; + p_port = (osm_port_t *) p_item; p_item = cl_qmap_next(p_item); status = osm_prtn_add_port(p_log, p_subn, p, osm_port_get_guid(p_port), full); @@ -160,19 +157,19 @@ ib_api_status_t osm_prtn_add_all(osm_log_t *p_log, osm_subn_t *p_subn, goto _err; } - _err: + _err: return status; } static const ib_gid_t osm_ipoib_mgid = { - { - 0xff, /* multicast field */ - 0x12, /* non-permanent bit, link local scope */ - 0x40, 0x1b, /* IPv4 signature */ - 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */ - 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */ - }, + { + 0xff, /* multicast field */ + 0x12, /* non-permanent bit, link local scope */ + 0x40, 0x1b, /* IPv4 signature */ + 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */ + 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */ + }, }; /* @@ -180,18 +177,18 @@ static const ib_gid_t osm_ipoib_mgid = { * we have to pre-define the MGID */ static const ib_gid_t osm_ts_ipoib_mgid = { - { - 0xff, /* multicast field */ - 0x12, /* non-permanent bit, link local scope */ - 0x40, 0x1b, /* IPv4 signature */ - 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */ - 0x00, 0x00, 0x00, 0x01, /* 32 bit IPv4 broadcast address */ - }, + { + 0xff, /* multicast field */ + 0x12, /* non-permanent bit, link local scope */ + 0x40, 0x1b, /* IPv4 signature */ + 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */ + 0x00, 0x00, 0x00, 0x01, /* 32 bit IPv4 broadcast address */ + }, }; -ib_api_status_t osm_prtn_add_mcgroup(osm_log_t *p_log, - osm_subn_t *p_subn, osm_prtn_t *p, +ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, + osm_subn_t * p_subn, osm_prtn_t * p, unsigned is_ipoib, uint8_t rate, uint8_t mtu, uint8_t scope) { @@ -207,27 +204,30 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t *p_log, memset(&mc_rec, 0, sizeof(mc_rec)); - mc_rec.mgid = osm_ipoib_mgid; /* ipv4 broadcast group */ + mc_rec.mgid = osm_ipoib_mgid; /* ipv4 broadcast group */ memcpy(&mc_rec.mgid.raw[4], &pkey, sizeof(pkey)); mc_rec.qkey = CL_HTON32(0x0b1b); - mc_rec.mtu = (mtu ? mtu : OSM_DEFAULT_MGRP_MTU) | (2 << 6); /* 2048 Bytes */ + mc_rec.mtu = (mtu ? mtu : OSM_DEFAULT_MGRP_MTU) | (2 << 6); /* 2048 Bytes */ mc_rec.tclass = 0; mc_rec.pkey = pkey; - mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ + mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ - mc_rec.scope_state = ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, MC_FULL_MEMBER); + mc_rec.scope_state = + ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, + MC_FULL_MEMBER); ib_mgid_set_scope(&mc_rec.mgid, scope ? scope : OSM_DEFAULT_MGRP_SCOPE); /* don't update rate, mtu */ comp_mask = IB_MCR_COMPMASK_MTU | IB_MCR_COMPMASK_MTU_SEL | - IB_MCR_COMPMASK_RATE | IB_MCR_COMPMASK_RATE_SEL; + IB_MCR_COMPMASK_RATE | IB_MCR_COMPMASK_RATE_SEL; status = osm_mcmr_rcv_find_or_create_new_mgrp(&p_sa->mcmr_rcv, - comp_mask, &mc_rec, &p_mgrp); + comp_mask, &mc_rec, + &p_mgrp); if (!p_mgrp || status != IB_SUCCESS) - osm_log( p_log, OSM_LOG_ERROR, + osm_log(p_log, OSM_LOG_ERROR, "osm_prtn_add_mcgroup: " "Failed to create MC group with pkey 0x%04x\n", cl_ntoh16(pkey)); @@ -239,17 +239,19 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t *p_log, mc_rec.mgid = osm_ts_ipoib_mgid; memcpy(&mc_rec.mgid.raw[4], &pkey, sizeof(pkey)); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ - ts_scope = ib_mgid_get_scope(&osm_ts_ipoib_mgid); /* get scope from MGID */ - mc_rec.scope_state = ib_member_set_scope_state(ts_scope, MC_FULL_MEMBER); - status = osm_mcmr_rcv_find_or_create_new_mgrp(&p_sa->mcmr_rcv, - comp_mask, &mc_rec, &p_mgrp); + ts_scope = ib_mgid_get_scope(&osm_ts_ipoib_mgid); /* get scope from MGID */ + mc_rec.scope_state = + ib_member_set_scope_state(ts_scope, MC_FULL_MEMBER); + status = + osm_mcmr_rcv_find_or_create_new_mgrp(&p_sa->mcmr_rcv, comp_mask, + &mc_rec, &p_mgrp); if (p_mgrp) p_mgrp->well_known = TRUE; return status; } -static uint16_t __generate_pkey(osm_subn_t *p_subn) +static uint16_t __generate_pkey(osm_subn_t * p_subn) { uint16_t pkey; @@ -263,14 +265,14 @@ static uint16_t __generate_pkey(osm_subn_t *p_subn) return 0; } -static osm_prtn_t *find_prtn_by_name(osm_subn_t *p_subn, const char *name) +static osm_prtn_t *find_prtn_by_name(osm_subn_t * p_subn, const char *name) { cl_map_item_t *p_next; osm_prtn_t *p; p_next = cl_qmap_head(&p_subn->prtn_pkey_tbl); while (p_next != cl_qmap_end(&p_subn->prtn_pkey_tbl)) { - p = (osm_prtn_t *)p_next; + p = (osm_prtn_t *) p_next; p_next = cl_qmap_next(&p->map_item); if (!strncmp(p->name, name, sizeof(p->name))) return p; @@ -279,17 +281,17 @@ static osm_prtn_t *find_prtn_by_name(osm_subn_t *p_subn, const char *name) return NULL; } -osm_prtn_t *osm_prtn_make_new(osm_log_t *p_log, osm_subn_t *p_subn, +osm_prtn_t *osm_prtn_make_new(osm_log_t * p_log, osm_subn_t * p_subn, const char *name, uint16_t pkey) { osm_prtn_t *p = NULL, *p_check; - pkey &= cl_hton16((uint16_t)~0x8000); + pkey &= cl_hton16((uint16_t) ~ 0x8000); if (!pkey) { if (name && (p = find_prtn_by_name(p_subn, name))) return p; - if(!(pkey = __generate_pkey(p_subn))) + if (!(pkey = __generate_pkey(p_subn))) return NULL; } @@ -297,13 +299,12 @@ osm_prtn_t *osm_prtn_make_new(osm_log_t *p_log, osm_subn_t *p_subn, if (!p) { osm_log(p_log, OSM_LOG_ERROR, "osm_prtn_make_new: Unable to create" - " partition \'%s\' (0x%04x)\n", - name, cl_ntoh16(pkey)); + " partition \'%s\' (0x%04x)\n", name, cl_ntoh16(pkey)); return NULL; } - p_check = (osm_prtn_t *)cl_qmap_insert(&p_subn->prtn_pkey_tbl, - p->pkey, &p->map_item); + p_check = (osm_prtn_t *) cl_qmap_insert(&p_subn->prtn_pkey_tbl, + p->pkey, &p->map_item); if (p != p_check) { osm_log(p_log, OSM_LOG_VERBOSE, "osm_prtn_make_new: Duplicated partition" @@ -324,19 +325,21 @@ static ib_api_status_t osm_prtn_make_default(osm_log_t * const p_log, ib_api_status_t status = IB_UNKNOWN_ERROR; osm_prtn_t *p; - p = osm_prtn_make_new(p_log, p_subn, "Default", IB_DEFAULT_PARTIAL_PKEY); + p = osm_prtn_make_new(p_log, p_subn, "Default", + IB_DEFAULT_PARTIAL_PKEY); if (!p) goto _err; status = osm_prtn_add_all(p_log, p_subn, p, no_config); if (status != IB_SUCCESS) goto _err; cl_map_remove(&p->part_guid_tbl, p_subn->sm_port_guid); - status = osm_prtn_add_port(p_log, p_subn, p, p_subn->sm_port_guid, TRUE); + status = + osm_prtn_add_port(p_log, p_subn, p, p_subn->sm_port_guid, TRUE); if (no_config) osm_prtn_add_mcgroup(p_log, p_subn, p, 1, 0, 0, 0); - _err: + _err: return status; } @@ -351,15 +354,14 @@ ib_api_status_t osm_prtn_make_partitions(osm_log_t * const p_log, osm_prtn_t *p; file_name = p_subn->opt.partition_config_file ? - p_subn->opt.partition_config_file : - "/etc/osm-partitions.conf"; + p_subn->opt.partition_config_file : "/etc/osm-partitions.conf"; if (stat(file_name, &statbuf)) is_config = FALSE; /* clean up current port maps */ p_next = cl_qmap_head(&p_subn->prtn_pkey_tbl); while (p_next != cl_qmap_end(&p_subn->prtn_pkey_tbl)) { - p = (osm_prtn_t *)p_next; + p = (osm_prtn_t *) p_next; p_next = cl_qmap_next(&p->map_item); cl_map_remove_all(&p->part_guid_tbl); cl_map_remove_all(&p->full_guid_tbl); @@ -380,16 +382,16 @@ ib_api_status_t osm_prtn_make_partitions(osm_log_t * const p_log, /* and now clean up empty partitions */ p_next = cl_qmap_head(&p_subn->prtn_pkey_tbl); while (p_next != cl_qmap_end(&p_subn->prtn_pkey_tbl)) { - p = (osm_prtn_t *)p_next; + p = (osm_prtn_t *) p_next; p_next = cl_qmap_next(&p->map_item); if (cl_map_count(&p->part_guid_tbl) == 0 && cl_map_count(&p->full_guid_tbl) == 0) { cl_qmap_remove_item(&p_subn->prtn_pkey_tbl, - (cl_map_item_t *)p); + (cl_map_item_t *) p); osm_prtn_delete(&p); } } - _err: + _err: return status; } diff --git a/opensm/opensm/osm_prtn_config.c b/opensm/opensm/osm_prtn_config.c index b49815b..9abf3e8 100644 --- a/opensm/opensm/osm_prtn_config.c +++ b/opensm/opensm/osm_prtn_config.c @@ -42,7 +42,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,23 +65,23 @@ /* */ struct part_conf { - osm_log_t *p_log; + osm_log_t *p_log; osm_subn_t *p_subn; osm_prtn_t *p_prtn; - unsigned is_ipoib, mtu, rate, sl, scope; - boolean_t full; + unsigned is_ipoib, mtu, rate, sl, scope; + boolean_t full; }; -extern osm_prtn_t *osm_prtn_make_new(osm_log_t *p_log, osm_subn_t *p_subn, +extern osm_prtn_t *osm_prtn_make_new(osm_log_t * p_log, osm_subn_t * p_subn, const char *name, uint16_t pkey); -extern ib_api_status_t osm_prtn_add_all(osm_log_t *p_log, - osm_subn_t *p_subn, - osm_prtn_t *p, boolean_t full); -extern ib_api_status_t osm_prtn_add_port(osm_log_t *p_log, - osm_subn_t *p_subn, osm_prtn_t *p, +extern ib_api_status_t osm_prtn_add_all(osm_log_t * p_log, + osm_subn_t * p_subn, + osm_prtn_t * p, boolean_t full); +extern ib_api_status_t osm_prtn_add_port(osm_log_t * p_log, + osm_subn_t * p_subn, osm_prtn_t * p, ib_net64_t guid, boolean_t full); -extern ib_api_status_t osm_prtn_add_mcgroup(osm_log_t *p_log, - osm_subn_t *p_subn, osm_prtn_t *p, +extern ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, + osm_subn_t * p_subn, osm_prtn_t * p, unsigned is_ipoib, uint8_t rate, uint8_t mtu, uint8_t scope); @@ -98,7 +98,7 @@ static int partition_create(unsigned lineno, struct part_conf *conf, if (id) { char *end; - pkey = (uint16_t)strtoul(id, &end, 0); + pkey = (uint16_t) strtoul(id, &end, 0); if (end == id || *end) return -1; } else @@ -114,15 +114,16 @@ static int partition_create(unsigned lineno, struct part_conf *conf, osm_log(conf->p_log, OSM_LOG_ERROR, "partition_create: Overriding SL %d to default SL %d on partition %s as QoS not enabled\n", conf->sl, OSM_DEFAULT_SL, name); - conf->sl = OSM_DEFAULT_SL; + conf->sl = OSM_DEFAULT_SL; } } - conf->p_prtn->sl = (uint8_t)conf->sl; + conf->p_prtn->sl = (uint8_t) conf->sl; if (conf->is_ipoib) osm_prtn_add_mcgroup(conf->p_log, conf->p_subn, conf->p_prtn, - conf->is_ipoib, (uint8_t)conf->rate, - (uint8_t)conf->mtu, (uint8_t)conf->scope); + conf->is_ipoib, (uint8_t) conf->rate, + (uint8_t) conf->mtu, + (uint8_t) conf->scope); return 0; } @@ -164,7 +165,9 @@ static int partition_add_flag(unsigned lineno, struct part_conf *conf, else conf->sl = sl; } else if (!strncmp(flag, "defmember", len)) { - if (!val || (strncmp(val, "limited", strlen(val)) && strncmp(val, "full", strlen(val)))) + if (!val + || (strncmp(val, "limited", strlen(val)) + && strncmp(val, "full", strlen(val)))) osm_log(conf->p_log, OSM_LOG_VERBOSE, "PARSE WARN: line %d: " "flag \'defmember\' requires valid value (limited or full)" @@ -172,10 +175,10 @@ static int partition_add_flag(unsigned lineno, struct part_conf *conf, else conf->full = strncmp(val, "full", strlen(val)) == 0; } else { - osm_log(conf->p_log, OSM_LOG_VERBOSE, - "PARSE WARN: line %d: " - "unrecognized partition flag \'%s\'" - " - ignored\n", lineno, flag); + osm_log(conf->p_log, OSM_LOG_VERBOSE, + "PARSE WARN: line %d: " + "unrecognized partition flag \'%s\'" + " - ignored\n", lineno, flag); } return 0; } @@ -251,8 +254,7 @@ static int parse_name_token(char *str, char **name, char **val) str = q; q = p + strlen(p); - while ( q != p && - (*q == '\0' || *q == ' ' || *q == '\t' || *q == '\n')) + while (q != p && (*q == '\0' || *q == ' ' || *q == '\t' || *q == '\n')) *q-- = '\0'; *name = p; @@ -266,15 +268,14 @@ static int parse_name_token(char *str, char **name, char **val) q = p + strlen(p); len += (int)(q - str) + 1; - while ( q != p && - ( *q == '\0' || *q == ' ' || *q == '\t' || *q == '\n')) + while (q != p && (*q == '\0' || *q == ' ' || *q == '\t' || *q == '\n')) *q-- = '\0'; *val = p; return len; } -static struct part_conf *new_part_conf(osm_log_t *p_log, osm_subn_t *p_subn) +static struct part_conf *new_part_conf(osm_log_t * p_log, osm_subn_t * p_subn) { static struct part_conf part; struct part_conf *conf = ∂ @@ -343,9 +344,9 @@ static int parse_part_conf(struct part_conf *conf, char *str, int lineno) if (!flag) { osm_log(conf->p_log, OSM_LOG_ERROR, "PARSE ERROR: line %d: " - "bad partition flags\n",lineno); + "bad partition flags\n", lineno); fprintf(stderr, "\nPARSE ERROR: line %d: " - "bad partition flags\n",lineno); + "bad partition flags\n", lineno); return -1; } p += ret; @@ -354,7 +355,7 @@ static int parse_part_conf(struct part_conf *conf, char *str, int lineno) } if (p != str || (partition_create(lineno, conf, - name, id, flag, flval) < 0)) { + name, id, flag, flval) < 0)) { osm_log(conf->p_log, OSM_LOG_ERROR, "PARSE ERROR: line %d: " "bad partition definition\n", lineno); @@ -363,7 +364,7 @@ static int parse_part_conf(struct part_conf *conf, char *str, int lineno) return -1; } - skip_header: + skip_header: do { name = flag = NULL; q = strchr(p, ','); @@ -385,7 +386,7 @@ static int parse_part_conf(struct part_conf *conf, char *str, int lineno) return len; } -int osm_prtn_config_parse_file(osm_log_t *p_log, osm_subn_t *p_subn, +int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn, const char *file_name) { char line[1024]; @@ -396,9 +397,9 @@ int osm_prtn_config_parse_file(osm_log_t *p_log, osm_subn_t *p_subn, file = fopen(file_name, "r"); if (!file) { osm_log(p_log, OSM_LOG_VERBOSE, - "osm_prtn_config_parse_file: " - "Cannot open config file \'%s\': %s\n", - file_name, strerror(errno)); + "osm_prtn_config_parse_file: " + "Cannot open config file \'%s\': %s\n", + file_name, strerror(errno)); return -1; } @@ -422,13 +423,15 @@ int osm_prtn_config_parse_file(osm_log_t *p_log, osm_subn_t *p_subn, if (*p == '\0') break; - if (!conf && - !(conf = new_part_conf(p_log, p_subn))) { + if (!conf && !(conf = new_part_conf(p_log, p_subn))) { osm_log(conf->p_log, OSM_LOG_ERROR, "PARSE ERROR: line %d: " - "internal: cannot create config\n", lineno); - fprintf(stderr, "PARSE ERROR: line %d: " - "internal: cannot create config\n", lineno); + "internal: cannot create config\n", + lineno); + fprintf(stderr, + "PARSE ERROR: line %d: " + "internal: cannot create config\n", + lineno); break; } diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c index c9ca9d8..dff9996 100644 --- a/opensm/opensm/osm_qos.c +++ b/opensm/opensm/osm_qos.c @@ -61,7 +61,7 @@ struct qos_config { ib_slvl_table_t sl2vl; }; -static void qos_build_config(struct qos_config * cfg, +static void qos_build_config(struct qos_config *cfg, osm_qos_options_t * opt, osm_qos_options_t * dflt); /* @@ -71,7 +71,8 @@ static ib_api_status_t vlarb_update_table_block(osm_req_t * p_req, osm_physp_t * p, uint8_t port_num, unsigned force_update, - const ib_vl_arb_table_t *table_block, + const ib_vl_arb_table_t * + table_block, unsigned block_length, unsigned block_num) { @@ -83,8 +84,7 @@ static ib_api_status_t vlarb_update_table_block(osm_req_t * p_req, vl_mask = (1 << (ib_port_info_get_op_vls(&p->port_info) - 1)) - 1; memset(&block, 0, sizeof(block)); - memcpy(&block, table_block, - block_length * sizeof(block.vl_entry[0])); + memcpy(&block, table_block, block_length * sizeof(block.vl_entry[0])); for (i = 0; i < block_length; i++) block.vl_entry[i].vl &= vl_mask; @@ -175,7 +175,7 @@ static ib_api_status_t sl2vl_update_table(osm_req_t * p_req, vl1 &= vl_mask; if (vl2 != 15) vl2 &= vl_mask; - tbl.raw_vl_by_sl[i] = (vl1 << 4 ) | vl2 ; + tbl.raw_vl_by_sl[i] = (vl1 << 4) | vl2; } if (!force_update && (p_tbl = osm_physp_get_slvl_tbl(p, in_port)) && @@ -205,7 +205,9 @@ static ib_api_status_t sl2vl_update(osm_req_t * p_req, osm_port_t * p_port, if (ib_port_info_get_vl_cap(&p->port_info) == 1) { /* Check port 0's capability mask */ p_physp = p_port->p_physp; - if (!(p_physp->port_info.capability_mask & IB_PORT_CAP_HAS_SL_MAP)) + if (! + (p_physp->port_info. + capability_mask & IB_PORT_CAP_HAS_SL_MAP)) return IB_SUCCESS; } num_ports = osm_node_get_num_physp(osm_physp_get_node_ptr(p)); @@ -308,14 +310,15 @@ osm_signal_t osm_qos_setup(osm_opensm_t * p_osm) if (!osm_physp_is_valid(p_physp)) continue; force_update = p_physp->need_update || - p_osm->subn.need_update; + p_osm->subn.need_update; status = qos_physp_setup(&p_osm->log, &p_osm->sm.req, p_port, p_physp, i, force_update, &swe_config); } /* skip base port 0 */ - if (!ib_switch_info_is_enhanced_port0(&p_node->sw->switch_info)) + if (!ib_switch_info_is_enhanced_port0 + (&p_node->sw->switch_info)) continue; cfg = &sw0_config; @@ -330,8 +333,7 @@ osm_signal_t osm_qos_setup(osm_opensm_t * p_osm) force_update = p_physp->need_update || p_osm->subn.need_update; status = qos_physp_setup(&p_osm->log, &p_osm->sm.req, - p_port, p_physp, 0, - force_update, cfg); + p_port, p_physp, 0, force_update, cfg); } cl_plock_release(&p_osm->lock); @@ -359,7 +361,7 @@ static int parse_vlarb_entry(char *str, ib_vl_arb_element_t * e) p += parse_one_unsigned(p, ':', &val); e->vl = val % 15; p += parse_one_unsigned(p, ',', &val); - e->weight = (uint8_t)val; + e->weight = (uint8_t) val; return (int)(p - str); } @@ -382,20 +384,24 @@ static void qos_build_config(struct qos_config *cfg, memset(cfg, 0, sizeof(*cfg)); cfg->max_vls = opt->max_vls > 0 ? opt->max_vls : dflt->max_vls; - cfg->vl_high_limit = (uint8_t)opt->high_limit; + cfg->vl_high_limit = (uint8_t) opt->high_limit; p = opt->vlarb_high ? opt->vlarb_high : dflt->vlarb_high; for (i = 0; i < 2 * IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK; i++) { p += parse_vlarb_entry(p, - &cfg->vlarb_high[i/IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]. - vl_entry[i%IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]); + &cfg->vlarb_high[i / + IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]. + vl_entry[i % + IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]); } p = opt->vlarb_low ? opt->vlarb_low : dflt->vlarb_low; for (i = 0; i < 2 * IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK; i++) { p += parse_vlarb_entry(p, - &cfg->vlarb_low[i/IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]. - vl_entry[i%IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]); + &cfg->vlarb_low[i / + IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]. + vl_entry[i % + IB_NUM_VL_ARB_ELEMENTS_IN_BLOCK]); } p = opt->sl2vl ? opt->sl2vl : dflt->sl2vl; diff --git a/opensm/opensm/osm_remote_sm.c b/opensm/opensm/osm_remote_sm.c index 76616a5..4d44d53 100644 --- a/opensm/opensm/osm_remote_sm.c +++ b/opensm/opensm/osm_remote_sm.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_sm_t. @@ -48,43 +47,38 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include /********************************************************************** **********************************************************************/ -void -osm_remote_sm_construct( - IN osm_remote_sm_t* const p_sm ) +void osm_remote_sm_construct(IN osm_remote_sm_t * const p_sm) { - memset( p_sm, 0, sizeof(*p_sm) ); + memset(p_sm, 0, sizeof(*p_sm)); } /********************************************************************** **********************************************************************/ -void -osm_remote_sm_destroy( - IN osm_remote_sm_t* const p_sm ) +void osm_remote_sm_destroy(IN osm_remote_sm_t * const p_sm) { - memset( p_sm, 0, sizeof(*p_sm) ); + memset(p_sm, 0, sizeof(*p_sm)); } /********************************************************************** **********************************************************************/ void -osm_remote_sm_init( - IN osm_remote_sm_t* const p_sm, - IN const osm_port_t* const p_port, - IN const ib_sm_info_t* const p_smi ) +osm_remote_sm_init(IN osm_remote_sm_t * const p_sm, + IN const osm_port_t * const p_port, + IN const ib_sm_info_t * const p_smi) { - CL_ASSERT( p_sm ); - CL_ASSERT( p_port ); + CL_ASSERT(p_sm); + CL_ASSERT(p_port); - osm_remote_sm_construct( p_sm ); + osm_remote_sm_construct(p_sm); - p_sm->p_port = p_port; - p_sm->smi = *p_smi; - return; + p_sm->p_port = p_port; + p_sm->smi = *p_smi; + return; } diff --git a/opensm/opensm/osm_req.c b/opensm/opensm/osm_req.c index 2bc6c19..ed1d19c 100644 --- a/opensm/opensm/osm_req.c +++ b/opensm/opensm/osm_req.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_req_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,234 +64,204 @@ /********************************************************************** **********************************************************************/ -void -osm_req_construct( - IN osm_req_t* const p_req ) +void osm_req_construct(IN osm_req_t * const p_req) { - CL_ASSERT( p_req ); + CL_ASSERT(p_req); - memset( p_req, 0, sizeof(*p_req) ); + memset(p_req, 0, sizeof(*p_req)); } /********************************************************************** **********************************************************************/ -void -osm_req_destroy( - IN osm_req_t* const p_req ) +void osm_req_destroy(IN osm_req_t * const p_req) { - CL_ASSERT( p_req ); + CL_ASSERT(p_req); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_req_init( - IN osm_req_t* const p_req, - IN osm_mad_pool_t* const p_pool, - IN osm_vl15_t* const p_vl15, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN atomic32_t* const p_sm_trans_id ) +osm_req_init(IN osm_req_t * const p_req, + IN osm_mad_pool_t * const p_pool, + IN osm_vl15_t * const p_vl15, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN atomic32_t * const p_sm_trans_id) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_req_init ); + OSM_LOG_ENTER(p_log, osm_req_init); - osm_req_construct( p_req ); - p_req->p_log = p_log; + osm_req_construct(p_req); + p_req->p_log = p_log; - p_req->p_pool = p_pool; - p_req->p_vl15 = p_vl15; - p_req->p_subn = p_subn; - p_req->p_sm_trans_id = p_sm_trans_id; + p_req->p_pool = p_pool; + p_req->p_vl15 = p_vl15; + p_req->p_subn = p_subn; + p_req->p_sm_trans_id = p_sm_trans_id; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** The plock MAY or MAY NOT be held before calling this function. **********************************************************************/ ib_api_status_t -osm_req_get( - IN const osm_req_t* const p_req, - IN const osm_dr_path_t* const p_path, - IN const uint16_t attr_id, - IN const uint32_t attr_mod, - IN const cl_disp_msgid_t err_msg, - IN const osm_madw_context_t* const p_context ) +osm_req_get(IN const osm_req_t * const p_req, + IN const osm_dr_path_t * const p_path, + IN const uint16_t attr_id, + IN const uint32_t attr_mod, + IN const cl_disp_msgid_t err_msg, + IN const osm_madw_context_t * const p_context) { - osm_madw_t *p_madw; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t tid; - - CL_ASSERT( p_req ); - - OSM_LOG_ENTER( p_req->p_log, osm_req_get ); - - CL_ASSERT( p_path ); - CL_ASSERT( attr_id ); - - /* do nothing if we are exiting ... */ - if (osm_exit_flag) - goto Exit; - - /* p_context may be NULL. */ - - p_madw = osm_mad_pool_get( - p_req->p_pool, - p_path->h_bind, - MAD_BLOCK_SIZE, - NULL ); - - if( p_madw == NULL ) - { - osm_log( p_req->p_log, OSM_LOG_ERROR, - "osm_req_get: ERR 1101: " - "Unable to acquire MAD\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - tid = cl_hton64( (uint64_t)cl_atomic_inc( p_req->p_sm_trans_id ) ); - - if( osm_log_is_active( p_req->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_req->p_log, OSM_LOG_DEBUG, - "osm_req_get: " - "Getting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", - ib_get_sm_attr_str( attr_id ), - cl_ntoh16( attr_id ), - cl_ntoh32( attr_mod ), - cl_ntoh64( tid ) ); - } - - ib_smp_init_new( - osm_madw_get_smp_ptr( p_madw ), - IB_MAD_METHOD_GET, - tid, - attr_id, - attr_mod, - p_path->hop_count, - p_req->p_subn->opt.m_key, - p_path->path, - IB_LID_PERMISSIVE, - IB_LID_PERMISSIVE ); - - p_madw->mad_addr.dest_lid = IB_LID_PERMISSIVE; - p_madw->mad_addr.addr_type.smi.source_lid = IB_LID_PERMISSIVE; - p_madw->resp_expected = TRUE; - p_madw->fail_msg = err_msg; - - /* - Fill in the mad wrapper context for the recipient. - In this case, the only thing the recipient needs is the - guid value. - */ - - if( p_context ) - p_madw->context = *p_context; - - osm_vl15_post( p_req->p_vl15, p_madw ); - - Exit: - OSM_LOG_EXIT( p_req->p_log ); - return( status ); + osm_madw_t *p_madw; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t tid; + + CL_ASSERT(p_req); + + OSM_LOG_ENTER(p_req->p_log, osm_req_get); + + CL_ASSERT(p_path); + CL_ASSERT(attr_id); + + /* do nothing if we are exiting ... */ + if (osm_exit_flag) + goto Exit; + + /* p_context may be NULL. */ + + p_madw = osm_mad_pool_get(p_req->p_pool, + p_path->h_bind, MAD_BLOCK_SIZE, NULL); + + if (p_madw == NULL) { + osm_log(p_req->p_log, OSM_LOG_ERROR, + "osm_req_get: ERR 1101: " "Unable to acquire MAD\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + tid = cl_hton64((uint64_t) cl_atomic_inc(p_req->p_sm_trans_id)); + + if (osm_log_is_active(p_req->p_log, OSM_LOG_DEBUG)) { + osm_log(p_req->p_log, OSM_LOG_DEBUG, + "osm_req_get: " + "Getting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", + ib_get_sm_attr_str(attr_id), + cl_ntoh16(attr_id), + cl_ntoh32(attr_mod), cl_ntoh64(tid)); + } + + ib_smp_init_new(osm_madw_get_smp_ptr(p_madw), + IB_MAD_METHOD_GET, + tid, + attr_id, + attr_mod, + p_path->hop_count, + p_req->p_subn->opt.m_key, + p_path->path, IB_LID_PERMISSIVE, IB_LID_PERMISSIVE); + + p_madw->mad_addr.dest_lid = IB_LID_PERMISSIVE; + p_madw->mad_addr.addr_type.smi.source_lid = IB_LID_PERMISSIVE; + p_madw->resp_expected = TRUE; + p_madw->fail_msg = err_msg; + + /* + Fill in the mad wrapper context for the recipient. + In this case, the only thing the recipient needs is the + guid value. + */ + + if (p_context) + p_madw->context = *p_context; + + osm_vl15_post(p_req->p_vl15, p_madw); + + Exit: + OSM_LOG_EXIT(p_req->p_log); + return (status); } /********************************************************************** The plock MAY or MAY NOT be held before calling this function. **********************************************************************/ ib_api_status_t -osm_req_set( - IN const osm_req_t* const p_req, - IN const osm_dr_path_t* const p_path, - IN const uint8_t* const p_payload, - IN const size_t payload_size, - IN const uint16_t attr_id, - IN const uint32_t attr_mod, - IN const cl_disp_msgid_t err_msg, - IN const osm_madw_context_t* const p_context ) +osm_req_set(IN const osm_req_t * const p_req, + IN const osm_dr_path_t * const p_path, + IN const uint8_t * const p_payload, + IN const size_t payload_size, + IN const uint16_t attr_id, + IN const uint32_t attr_mod, + IN const cl_disp_msgid_t err_msg, + IN const osm_madw_context_t * const p_context) { - osm_madw_t *p_madw; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t tid; - - CL_ASSERT( p_req ); - - OSM_LOG_ENTER( p_req->p_log, osm_req_set ); - - CL_ASSERT( p_path ); - CL_ASSERT( attr_id ); - CL_ASSERT( p_payload ); - - /* do nothing if we are exiting ... */ - if (osm_exit_flag) - goto Exit; - - /* p_context may be NULL. */ - - p_madw = osm_mad_pool_get( - p_req->p_pool, - p_path->h_bind, - MAD_BLOCK_SIZE, - NULL ); - - if( p_madw == NULL ) - { - osm_log( p_req->p_log, OSM_LOG_ERROR, - "osm_req_set: ERR 1102: " - "Unable to acquire MAD\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - tid = cl_hton64( (uint64_t)cl_atomic_inc( p_req->p_sm_trans_id ) ); - - if( osm_log_is_active( p_req->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_req->p_log, OSM_LOG_DEBUG, - "osm_req_set: " - "Setting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", - ib_get_sm_attr_str( attr_id ), - cl_ntoh16( attr_id ), - cl_ntoh32( attr_mod ), - cl_ntoh64( tid ) ); - } - - ib_smp_init_new( - osm_madw_get_smp_ptr( p_madw ), - IB_MAD_METHOD_SET, - tid, - attr_id, - attr_mod, - p_path->hop_count, - p_req->p_subn->opt.m_key, - p_path->path, - IB_LID_PERMISSIVE, - IB_LID_PERMISSIVE ); - - p_madw->mad_addr.dest_lid = IB_LID_PERMISSIVE; - p_madw->mad_addr.addr_type.smi.source_lid = IB_LID_PERMISSIVE; - p_madw->resp_expected = TRUE; - p_madw->fail_msg = err_msg; - - /* - Fill in the mad wrapper context for the recipient. - In this case, the only thing the recipient needs is the - guid value. - */ - - if( p_context ) - p_madw->context = *p_context; - - memcpy( osm_madw_get_smp_ptr( p_madw )->data, - p_payload, payload_size ); - - osm_vl15_post( p_req->p_vl15, p_madw ); - - Exit: - OSM_LOG_EXIT( p_req->p_log ); - return( status ); + osm_madw_t *p_madw; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t tid; + + CL_ASSERT(p_req); + + OSM_LOG_ENTER(p_req->p_log, osm_req_set); + + CL_ASSERT(p_path); + CL_ASSERT(attr_id); + CL_ASSERT(p_payload); + + /* do nothing if we are exiting ... */ + if (osm_exit_flag) + goto Exit; + + /* p_context may be NULL. */ + + p_madw = osm_mad_pool_get(p_req->p_pool, + p_path->h_bind, MAD_BLOCK_SIZE, NULL); + + if (p_madw == NULL) { + osm_log(p_req->p_log, OSM_LOG_ERROR, + "osm_req_set: ERR 1102: " "Unable to acquire MAD\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + tid = cl_hton64((uint64_t) cl_atomic_inc(p_req->p_sm_trans_id)); + + if (osm_log_is_active(p_req->p_log, OSM_LOG_DEBUG)) { + osm_log(p_req->p_log, OSM_LOG_DEBUG, + "osm_req_set: " + "Setting %s (0x%X), modifier 0x%X, TID 0x%" PRIx64 "\n", + ib_get_sm_attr_str(attr_id), + cl_ntoh16(attr_id), + cl_ntoh32(attr_mod), cl_ntoh64(tid)); + } + + ib_smp_init_new(osm_madw_get_smp_ptr(p_madw), + IB_MAD_METHOD_SET, + tid, + attr_id, + attr_mod, + p_path->hop_count, + p_req->p_subn->opt.m_key, + p_path->path, IB_LID_PERMISSIVE, IB_LID_PERMISSIVE); + + p_madw->mad_addr.dest_lid = IB_LID_PERMISSIVE; + p_madw->mad_addr.addr_type.smi.source_lid = IB_LID_PERMISSIVE; + p_madw->resp_expected = TRUE; + p_madw->fail_msg = err_msg; + + /* + Fill in the mad wrapper context for the recipient. + In this case, the only thing the recipient needs is the + guid value. + */ + + if (p_context) + p_madw->context = *p_context; + + memcpy(osm_madw_get_smp_ptr(p_madw)->data, p_payload, payload_size); + + osm_vl15_post(p_req->p_vl15, p_madw); + + Exit: + OSM_LOG_EXIT(p_req->p_log); + return (status); } - diff --git a/opensm/opensm/osm_resp.c b/opensm/opensm/osm_resp.c index e4c7e75..e5beb45 100644 --- a/opensm/opensm/osm_resp.c +++ b/opensm/opensm/osm_resp.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_resp_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,163 +64,143 @@ /********************************************************************** **********************************************************************/ -void -osm_resp_construct( - IN osm_resp_t* const p_resp ) +void osm_resp_construct(IN osm_resp_t * const p_resp) { - memset( p_resp, 0, sizeof(*p_resp) ); + memset(p_resp, 0, sizeof(*p_resp)); } /********************************************************************** **********************************************************************/ -void -osm_resp_destroy( - IN osm_resp_t* const p_resp ) +void osm_resp_destroy(IN osm_resp_t * const p_resp) { - CL_ASSERT( p_resp ); + CL_ASSERT(p_resp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_resp_init( - IN osm_resp_t* const p_resp, - IN osm_mad_pool_t* const p_pool, - IN osm_vl15_t* const p_vl15, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log ) +osm_resp_init(IN osm_resp_t * const p_resp, + IN osm_mad_pool_t * const p_pool, + IN osm_vl15_t * const p_vl15, + IN osm_subn_t * const p_subn, IN osm_log_t * const p_log) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_resp_init ); + OSM_LOG_ENTER(p_log, osm_resp_init); - osm_resp_construct( p_resp ); + osm_resp_construct(p_resp); - p_resp->p_log = p_log; - p_resp->p_pool = p_pool; - p_resp->p_vl15 = p_vl15; - p_resp->p_subn = p_subn; + p_resp->p_log = p_log; + p_resp->p_pool = p_pool; + p_resp->p_vl15 = p_vl15; + p_resp->p_subn = p_subn; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -osm_resp_make_resp_smp( - IN const osm_resp_t* const p_resp, - IN const ib_smp_t* const p_src_smp, - IN const ib_net16_t status, - IN const uint8_t* const p_payload, - OUT ib_smp_t* const p_dest_smp ) +osm_resp_make_resp_smp(IN const osm_resp_t * const p_resp, + IN const ib_smp_t * const p_src_smp, + IN const ib_net16_t status, + IN const uint8_t * const p_payload, + OUT ib_smp_t * const p_dest_smp) { - OSM_LOG_ENTER( p_resp->p_log, osm_resp_make_resp_smp ); - - CL_ASSERT( p_dest_smp ); - CL_ASSERT( p_src_smp ); - CL_ASSERT( !ib_smp_is_response( p_src_smp ) ); - - *p_dest_smp = *p_src_smp; - if (p_src_smp->method == IB_MAD_METHOD_GET || - p_src_smp->method == IB_MAD_METHOD_SET ) { - p_dest_smp->method = IB_MAD_METHOD_GET_RESP; - p_dest_smp->status = status; - } - else if (p_src_smp->method == IB_MAD_METHOD_TRAP) - { - p_dest_smp->method = IB_MAD_METHOD_TRAP_REPRESS; - p_dest_smp->status = 0; - } - else - { - osm_log( p_resp->p_log, OSM_LOG_ERROR, - "osm_resp_make_resp_smp: ERR 1302: " - "src smp method unsupported 0x%X\n", - p_src_smp->method ); - goto Exit; - } - - if (p_src_smp->mgmt_class == IB_MCLASS_SUBN_DIR) - p_dest_smp->status |= IB_SMP_DIRECTION; - - p_dest_smp->dr_dlid = p_dest_smp->dr_slid; - p_dest_smp->dr_slid = p_dest_smp->dr_dlid; - memcpy( &p_dest_smp->data, p_payload, IB_SMP_DATA_SIZE ); - - Exit: - OSM_LOG_EXIT( p_resp->p_log ); + OSM_LOG_ENTER(p_resp->p_log, osm_resp_make_resp_smp); + + CL_ASSERT(p_dest_smp); + CL_ASSERT(p_src_smp); + CL_ASSERT(!ib_smp_is_response(p_src_smp)); + + *p_dest_smp = *p_src_smp; + if (p_src_smp->method == IB_MAD_METHOD_GET || + p_src_smp->method == IB_MAD_METHOD_SET) { + p_dest_smp->method = IB_MAD_METHOD_GET_RESP; + p_dest_smp->status = status; + } else if (p_src_smp->method == IB_MAD_METHOD_TRAP) { + p_dest_smp->method = IB_MAD_METHOD_TRAP_REPRESS; + p_dest_smp->status = 0; + } else { + osm_log(p_resp->p_log, OSM_LOG_ERROR, + "osm_resp_make_resp_smp: ERR 1302: " + "src smp method unsupported 0x%X\n", p_src_smp->method); + goto Exit; + } + + if (p_src_smp->mgmt_class == IB_MCLASS_SUBN_DIR) + p_dest_smp->status |= IB_SMP_DIRECTION; + + p_dest_smp->dr_dlid = p_dest_smp->dr_slid; + p_dest_smp->dr_slid = p_dest_smp->dr_dlid; + memcpy(&p_dest_smp->data, p_payload, IB_SMP_DATA_SIZE); + + Exit: + OSM_LOG_EXIT(p_resp->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_resp_send( - IN const osm_resp_t* const p_resp, - IN const osm_madw_t* const p_req_madw, - IN const ib_net16_t mad_status, - IN const uint8_t* const p_payload ) +osm_resp_send(IN const osm_resp_t * const p_resp, + IN const osm_madw_t * const p_req_madw, + IN const ib_net16_t mad_status, + IN const uint8_t * const p_payload) { - const ib_smp_t* p_req_smp; - ib_smp_t* p_smp; - osm_madw_t* p_madw; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_resp->p_log, osm_resp_send ); - - CL_ASSERT( p_req_madw ); - CL_ASSERT( p_payload ); - - /* do nothing if we are exiting ... */ - if (osm_exit_flag) - goto Exit; - - p_madw = osm_mad_pool_get( - p_resp->p_pool, - osm_madw_get_bind_handle( p_req_madw ), - MAD_BLOCK_SIZE, - NULL ); - - if( p_madw == NULL ) - { - osm_log( p_resp->p_log, OSM_LOG_ERROR, - "osm_resp_send: ERR 1301: " - "Unable to acquire MAD\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - /* - Copy the request smp to the response smp, then just - update the necessary fields. - */ - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_req_smp = osm_madw_get_smp_ptr( p_req_madw ); - osm_resp_make_resp_smp( p_resp, p_req_smp, mad_status, - p_payload, p_smp ); - p_madw->mad_addr.dest_lid = - p_req_madw->mad_addr.addr_type.smi.source_lid; - p_madw->mad_addr.addr_type.smi.source_lid = - p_req_madw->mad_addr.dest_lid; - - p_madw->resp_expected = FALSE; - p_madw->fail_msg = CL_DISP_MSGID_NONE; - - if( osm_log_is_active( p_resp->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_resp->p_log, OSM_LOG_DEBUG, - "osm_resp_send: " - "Responding to %s (0x%X)" - "\n\t\t\t\tattribute modifier 0x%X, TID 0x%" PRIx64 "\n", - ib_get_sm_attr_str( p_smp->attr_id ), - cl_ntoh16( p_smp->attr_id ), - cl_ntoh32( p_smp->attr_mod ), - cl_ntoh64( p_smp->trans_id ) ); - } - - osm_vl15_post( p_resp->p_vl15, p_madw ); - - Exit: - OSM_LOG_EXIT( p_resp->p_log ); - return( status ); + const ib_smp_t *p_req_smp; + ib_smp_t *p_smp; + osm_madw_t *p_madw; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_resp->p_log, osm_resp_send); + + CL_ASSERT(p_req_madw); + CL_ASSERT(p_payload); + + /* do nothing if we are exiting ... */ + if (osm_exit_flag) + goto Exit; + + p_madw = osm_mad_pool_get(p_resp->p_pool, + osm_madw_get_bind_handle(p_req_madw), + MAD_BLOCK_SIZE, NULL); + + if (p_madw == NULL) { + osm_log(p_resp->p_log, OSM_LOG_ERROR, + "osm_resp_send: ERR 1301: " "Unable to acquire MAD\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + /* + Copy the request smp to the response smp, then just + update the necessary fields. + */ + p_smp = osm_madw_get_smp_ptr(p_madw); + p_req_smp = osm_madw_get_smp_ptr(p_req_madw); + osm_resp_make_resp_smp(p_resp, p_req_smp, mad_status, p_payload, p_smp); + p_madw->mad_addr.dest_lid = + p_req_madw->mad_addr.addr_type.smi.source_lid; + p_madw->mad_addr.addr_type.smi.source_lid = + p_req_madw->mad_addr.dest_lid; + + p_madw->resp_expected = FALSE; + p_madw->fail_msg = CL_DISP_MSGID_NONE; + + if (osm_log_is_active(p_resp->p_log, OSM_LOG_DEBUG)) { + osm_log(p_resp->p_log, OSM_LOG_DEBUG, + "osm_resp_send: " + "Responding to %s (0x%X)" + "\n\t\t\t\tattribute modifier 0x%X, TID 0x%" PRIx64 + "\n", ib_get_sm_attr_str(p_smp->attr_id), + cl_ntoh16(p_smp->attr_id), cl_ntoh32(p_smp->attr_mod), + cl_ntoh64(p_smp->trans_id)); + } + + osm_vl15_post(p_resp->p_vl15, p_madw); + + Exit: + OSM_LOG_EXIT(p_resp->p_log); + return (status); } diff --git a/opensm/opensm/osm_router.c b/opensm/opensm/osm_router.c index d719618..8b93fd6 100644 --- a/opensm/opensm/osm_router.c +++ b/opensm/opensm/osm_router.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -55,30 +55,25 @@ /********************************************************************** **********************************************************************/ -void -osm_router_delete( - IN OUT osm_router_t** const pp_rtr ) +void osm_router_delete(IN OUT osm_router_t ** const pp_rtr) { - free( *pp_rtr ); - *pp_rtr = NULL; + free(*pp_rtr); + *pp_rtr = NULL; } /********************************************************************** **********************************************************************/ -osm_router_t* -osm_router_new( - IN osm_port_t* const p_port ) +osm_router_t *osm_router_new(IN osm_port_t * const p_port) { - osm_router_t *p_rtr; + osm_router_t *p_rtr; - CL_ASSERT( p_port ); + CL_ASSERT(p_port); - p_rtr = (osm_router_t*)malloc( sizeof(*p_rtr) ); - if( p_rtr ) - { - memset( p_rtr, 0, sizeof(*p_rtr) ); - p_rtr->p_port = p_port; - } + p_rtr = (osm_router_t *) malloc(sizeof(*p_rtr)); + if (p_rtr) { + memset(p_rtr, 0, sizeof(*p_rtr)); + p_rtr->p_port = p_port; + } - return( p_rtr ); + return (p_rtr); } diff --git a/opensm/opensm/osm_sa.c b/opensm/opensm/osm_sa.c index 8528a61..e49bfba 100644 --- a/opensm/opensm/osm_sa.c +++ b/opensm/opensm/osm_sa.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -74,449 +74,376 @@ /********************************************************************** **********************************************************************/ -void -osm_sa_construct( - IN osm_sa_t* const p_sa ) +void osm_sa_construct(IN osm_sa_t * const p_sa) { - memset( p_sa, 0, sizeof(*p_sa) ); - p_sa->state = OSM_SA_STATE_INIT; - p_sa->sa_trans_id = OSM_SA_INITIAL_TID_VALUE; - - osm_sa_resp_construct( &p_sa->resp ); - osm_nr_rcv_construct( &p_sa->nr_rcv); - osm_pir_rcv_construct( &p_sa->pir_rcv ); - osm_gir_rcv_construct( &p_sa->gir_rcv ); - osm_lr_rcv_construct( &p_sa->lr_rcv ); - osm_pr_rcv_construct( &p_sa->pr_rcv ); + memset(p_sa, 0, sizeof(*p_sa)); + p_sa->state = OSM_SA_STATE_INIT; + p_sa->sa_trans_id = OSM_SA_INITIAL_TID_VALUE; + + osm_sa_resp_construct(&p_sa->resp); + osm_nr_rcv_construct(&p_sa->nr_rcv); + osm_pir_rcv_construct(&p_sa->pir_rcv); + osm_gir_rcv_construct(&p_sa->gir_rcv); + osm_lr_rcv_construct(&p_sa->lr_rcv); + osm_pr_rcv_construct(&p_sa->pr_rcv); #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - osm_mpr_rcv_construct( &p_sa->mpr_rcv ); + osm_mpr_rcv_construct(&p_sa->mpr_rcv); #endif - osm_smir_rcv_construct( &p_sa->smir_rcv ); - osm_mcmr_rcv_construct(&p_sa->mcmr_rcv ); - osm_sr_rcv_construct( &p_sa->sr_rcv ); - osm_infr_rcv_construct( &p_sa->infr_rcv ); - osm_vlarb_rec_rcv_construct( &p_sa->vlarb_rec_rcv ); - osm_slvl_rec_rcv_construct( &p_sa->slvl_rec_rcv ); - osm_pkey_rec_rcv_construct( &p_sa->pkey_rec_rcv ); - osm_lftr_rcv_construct( &p_sa->lftr_rcv ); - osm_sir_rcv_construct( &p_sa->sir_rcv ); - osm_mftr_rcv_construct( &p_sa->mftr_rcv ); + osm_smir_rcv_construct(&p_sa->smir_rcv); + osm_mcmr_rcv_construct(&p_sa->mcmr_rcv); + osm_sr_rcv_construct(&p_sa->sr_rcv); + osm_infr_rcv_construct(&p_sa->infr_rcv); + osm_vlarb_rec_rcv_construct(&p_sa->vlarb_rec_rcv); + osm_slvl_rec_rcv_construct(&p_sa->slvl_rec_rcv); + osm_pkey_rec_rcv_construct(&p_sa->pkey_rec_rcv); + osm_lftr_rcv_construct(&p_sa->lftr_rcv); + osm_sir_rcv_construct(&p_sa->sir_rcv); + osm_mftr_rcv_construct(&p_sa->mftr_rcv); } /********************************************************************** **********************************************************************/ -void -osm_sa_shutdown( - IN osm_sa_t* const p_sa ) +void osm_sa_shutdown(IN osm_sa_t * const p_sa) { - ib_api_status_t status; - OSM_LOG_ENTER( p_sa->p_log, osm_sa_shutdown ); - - /* unbind from the mad service */ - status = osm_sa_mad_ctrl_unbind( &p_sa->mad_ctrl ); - - /* remove any registered dispatcher message */ - cl_disp_unregister( p_sa->nr_disp_h ); - cl_disp_unregister( p_sa->pir_disp_h ); - cl_disp_unregister( p_sa->gir_disp_h ); - cl_disp_unregister( p_sa->lr_disp_h ); - cl_disp_unregister( p_sa->pr_disp_h ); + ib_api_status_t status; + OSM_LOG_ENTER(p_sa->p_log, osm_sa_shutdown); + + /* unbind from the mad service */ + status = osm_sa_mad_ctrl_unbind(&p_sa->mad_ctrl); + + /* remove any registered dispatcher message */ + cl_disp_unregister(p_sa->nr_disp_h); + cl_disp_unregister(p_sa->pir_disp_h); + cl_disp_unregister(p_sa->gir_disp_h); + cl_disp_unregister(p_sa->lr_disp_h); + cl_disp_unregister(p_sa->pr_disp_h); #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - cl_disp_unregister( p_sa->mpr_disp_h ); + cl_disp_unregister(p_sa->mpr_disp_h); #endif - cl_disp_unregister( p_sa->smir_disp_h ); - cl_disp_unregister( p_sa->mcmr_disp_h); - cl_disp_unregister( p_sa->sr_disp_h ); - cl_disp_unregister( p_sa->infr_disp_h ); - cl_disp_unregister( p_sa->infir_disp_h ); - cl_disp_unregister( p_sa->vlarb_disp_h ); - cl_disp_unregister( p_sa->slvl_disp_h ); - cl_disp_unregister( p_sa->pkey_disp_h ); - cl_disp_unregister( p_sa->lft_disp_h ); - cl_disp_unregister( p_sa->sir_disp_h ); - cl_disp_unregister( p_sa->mft_disp_h ); - osm_sa_mad_ctrl_destroy( &p_sa->mad_ctrl ); - - OSM_LOG_EXIT( p_sa->p_log ); + cl_disp_unregister(p_sa->smir_disp_h); + cl_disp_unregister(p_sa->mcmr_disp_h); + cl_disp_unregister(p_sa->sr_disp_h); + cl_disp_unregister(p_sa->infr_disp_h); + cl_disp_unregister(p_sa->infir_disp_h); + cl_disp_unregister(p_sa->vlarb_disp_h); + cl_disp_unregister(p_sa->slvl_disp_h); + cl_disp_unregister(p_sa->pkey_disp_h); + cl_disp_unregister(p_sa->lft_disp_h); + cl_disp_unregister(p_sa->sir_disp_h); + cl_disp_unregister(p_sa->mft_disp_h); + osm_sa_mad_ctrl_destroy(&p_sa->mad_ctrl); + + OSM_LOG_EXIT(p_sa->p_log); } /********************************************************************** **********************************************************************/ -void -osm_sa_destroy( - IN osm_sa_t* const p_sa ) +void osm_sa_destroy(IN osm_sa_t * const p_sa) { - OSM_LOG_ENTER( p_sa->p_log, osm_sa_destroy ); + OSM_LOG_ENTER(p_sa->p_log, osm_sa_destroy); - p_sa->state = OSM_SA_STATE_INIT; + p_sa->state = OSM_SA_STATE_INIT; - osm_nr_rcv_destroy( &p_sa->nr_rcv ); - osm_pir_rcv_destroy( &p_sa->pir_rcv ); - osm_gir_rcv_destroy( &p_sa->gir_rcv ); - osm_lr_rcv_destroy( &p_sa->lr_rcv ); - osm_pr_rcv_destroy( &p_sa->pr_rcv ); + osm_nr_rcv_destroy(&p_sa->nr_rcv); + osm_pir_rcv_destroy(&p_sa->pir_rcv); + osm_gir_rcv_destroy(&p_sa->gir_rcv); + osm_lr_rcv_destroy(&p_sa->lr_rcv); + osm_pr_rcv_destroy(&p_sa->pr_rcv); #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - osm_mpr_rcv_destroy( &p_sa->mpr_rcv ); + osm_mpr_rcv_destroy(&p_sa->mpr_rcv); #endif - osm_smir_rcv_destroy( &p_sa->smir_rcv ); - osm_mcmr_rcv_destroy(&p_sa->mcmr_rcv); - osm_sr_rcv_destroy( &p_sa->sr_rcv ); - osm_infr_rcv_destroy( &p_sa->infr_rcv ); - osm_vlarb_rec_rcv_destroy( &p_sa->vlarb_rec_rcv ); - osm_slvl_rec_rcv_destroy( &p_sa->slvl_rec_rcv ); - osm_pkey_rec_rcv_destroy( &p_sa->pkey_rec_rcv ); - osm_lftr_rcv_destroy( &p_sa->lftr_rcv ); - osm_sir_rcv_destroy( &p_sa->sir_rcv ); - osm_mftr_rcv_destroy( &p_sa->mftr_rcv ); - osm_sa_resp_destroy( &p_sa->resp ); - - OSM_LOG_EXIT( p_sa->p_log ); + osm_smir_rcv_destroy(&p_sa->smir_rcv); + osm_mcmr_rcv_destroy(&p_sa->mcmr_rcv); + osm_sr_rcv_destroy(&p_sa->sr_rcv); + osm_infr_rcv_destroy(&p_sa->infr_rcv); + osm_vlarb_rec_rcv_destroy(&p_sa->vlarb_rec_rcv); + osm_slvl_rec_rcv_destroy(&p_sa->slvl_rec_rcv); + osm_pkey_rec_rcv_destroy(&p_sa->pkey_rec_rcv); + osm_lftr_rcv_destroy(&p_sa->lftr_rcv); + osm_sir_rcv_destroy(&p_sa->sir_rcv); + osm_mftr_rcv_destroy(&p_sa->mftr_rcv); + osm_sa_resp_destroy(&p_sa->resp); + + OSM_LOG_EXIT(p_sa->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sa_init( - IN osm_sm_t* const p_sm, - IN osm_sa_t* const p_sa, - IN osm_subn_t* const p_subn, - IN osm_vendor_t* const p_vendor, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_dispatcher_t* const p_disp, - IN cl_plock_t* const p_lock ) +osm_sa_init(IN osm_sm_t * const p_sm, + IN osm_sa_t * const p_sa, + IN osm_subn_t * const p_subn, + IN osm_vendor_t * const p_vendor, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_sa_init ); - - p_sa->p_subn = p_subn; - p_sa->p_vendor = p_vendor; - p_sa->p_mad_pool = p_mad_pool; - p_sa->p_log = p_log; - p_sa->p_disp = p_disp; - p_sa->p_lock = p_lock; - - p_sa->state = OSM_SA_STATE_READY; - - status = osm_sa_resp_init(&p_sa->resp, - p_sa->p_mad_pool, - p_log); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sa_mad_ctrl_init( - &p_sa->mad_ctrl, - &p_sa->resp, - p_sa->p_mad_pool, - p_sa->p_vendor, - p_subn, - p_log, - p_stats, - p_disp ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_cpi_rcv_init( - &p_sa->cpi_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_nr_rcv_init( - &p_sa->nr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_pir_rcv_init( - &p_sa->pir_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_gir_rcv_init( - &p_sa->gir_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_lr_rcv_init( - &p_sa->lr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_pr_rcv_init( - &p_sa->pr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_sa_init); + + p_sa->p_subn = p_subn; + p_sa->p_vendor = p_vendor; + p_sa->p_mad_pool = p_mad_pool; + p_sa->p_log = p_log; + p_sa->p_disp = p_disp; + p_sa->p_lock = p_lock; + + p_sa->state = OSM_SA_STATE_READY; + + status = osm_sa_resp_init(&p_sa->resp, p_sa->p_mad_pool, p_log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sa_mad_ctrl_init(&p_sa->mad_ctrl, + &p_sa->resp, + p_sa->p_mad_pool, + p_sa->p_vendor, + p_subn, p_log, p_stats, p_disp); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_cpi_rcv_init(&p_sa->cpi_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_nr_rcv_init(&p_sa->nr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_pir_rcv_init(&p_sa->pir_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_gir_rcv_init(&p_sa->gir_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_lr_rcv_init(&p_sa->lr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_pr_rcv_init(&p_sa->pr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - status = osm_mpr_rcv_init( - &p_sa->mpr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; + status = osm_mpr_rcv_init(&p_sa->mpr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; #endif - status = osm_smir_rcv_init( - &p_sa->smir_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_stats, - p_log, - p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_mcmr_rcv_init( - p_sm, - &p_sa->mcmr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sr_rcv_init( - &p_sa->sr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_infr_rcv_init( - &p_sa->infr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_vlarb_rec_rcv_init( - &p_sa->vlarb_rec_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_slvl_rec_rcv_init( - &p_sa->slvl_rec_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_pkey_rec_rcv_init( - &p_sa->pkey_rec_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_lftr_rcv_init( - &p_sa->lftr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sir_rcv_init( - &p_sa->sir_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_mftr_rcv_init( - &p_sa->mftr_rcv, - &p_sa->resp, - p_sa->p_mad_pool, - p_subn, - p_log, - p_lock); - if( status != IB_SUCCESS ) - goto Exit; - - p_sa->cpi_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_CLASS_PORT_INFO, - osm_cpi_rcv_process, &p_sa->cpi_rcv); - if( p_sa->cpi_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->nr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_RECORD, - osm_nr_rcv_process, &p_sa->nr_rcv); - if( p_sa->nr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->pir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PORTINFO_RECORD, - osm_pir_rcv_process, &p_sa->pir_rcv); - if( p_sa->pir_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->gir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_GUIDINFO_RECORD, - osm_gir_rcv_process, &p_sa->gir_rcv); - if( p_sa->gir_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->lr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LINK_RECORD, - osm_lr_rcv_process, &p_sa->lr_rcv); - if( p_sa->lr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->pr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PATH_RECORD, - osm_pr_rcv_process, &p_sa->pr_rcv); - if( p_sa->pr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; + status = osm_smir_rcv_init(&p_sa->smir_rcv, + &p_sa->resp, + p_sa->p_mad_pool, + p_subn, p_stats, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_mcmr_rcv_init(p_sm, + &p_sa->mcmr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sr_rcv_init(&p_sa->sr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_infr_rcv_init(&p_sa->infr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_vlarb_rec_rcv_init(&p_sa->vlarb_rec_rcv, + &p_sa->resp, + p_sa->p_mad_pool, + p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_slvl_rec_rcv_init(&p_sa->slvl_rec_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_pkey_rec_rcv_init(&p_sa->pkey_rec_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_lftr_rcv_init(&p_sa->lftr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sir_rcv_init(&p_sa->sir_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_mftr_rcv_init(&p_sa->mftr_rcv, + &p_sa->resp, + p_sa->p_mad_pool, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + p_sa->cpi_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_CLASS_PORT_INFO, + osm_cpi_rcv_process, + &p_sa->cpi_rcv); + if (p_sa->cpi_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->nr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_RECORD, + osm_nr_rcv_process, &p_sa->nr_rcv); + if (p_sa->nr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->pir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PORTINFO_RECORD, + osm_pir_rcv_process, + &p_sa->pir_rcv); + if (p_sa->pir_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->gir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_GUIDINFO_RECORD, + osm_gir_rcv_process, + &p_sa->gir_rcv); + if (p_sa->gir_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->lr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LINK_RECORD, + osm_lr_rcv_process, &p_sa->lr_rcv); + if (p_sa->lr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->pr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PATH_RECORD, + osm_pr_rcv_process, &p_sa->pr_rcv); + if (p_sa->pr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - p_sa->mpr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MULTIPATH_RECORD, - osm_mpr_rcv_process, &p_sa->mpr_rcv); - if( p_sa->mpr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; + p_sa->mpr_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_MULTIPATH_RECORD, + osm_mpr_rcv_process, &p_sa->mpr_rcv); + if (p_sa->mpr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; #endif - p_sa->smir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SMINFO_RECORD, - osm_smir_rcv_process, &p_sa->smir_rcv); - if( p_sa->smir_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->mcmr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MCMEMBER_RECORD, - osm_mcmr_rcv_process, &p_sa->mcmr_rcv); - if( p_sa->mcmr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->sr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SERVICE_RECORD, - osm_sr_rcv_process, &p_sa->sr_rcv); - if( p_sa->sr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->infr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_INFORM_INFO, - osm_infr_rcv_process, &p_sa->infr_rcv); - if( p_sa->infr_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->infir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_INFORM_INFO_RECORD, - osm_infir_rcv_process, &p_sa->infr_rcv); - if( p_sa->infir_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->vlarb_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_VL_ARB_RECORD, - osm_vlarb_rec_rcv_process, - &p_sa->vlarb_rec_rcv); - if( p_sa->vlarb_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->slvl_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SLVL_TBL_RECORD, - osm_slvl_rec_rcv_process, - &p_sa->slvl_rec_rcv); - if( p_sa->slvl_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->pkey_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PKEY_TBL_RECORD, - osm_pkey_rec_rcv_process, - &p_sa->pkey_rec_rcv); - if( p_sa->pkey_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->lft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LFT_RECORD, - osm_lftr_rcv_process, &p_sa->lftr_rcv); - if( p_sa->lft_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->sir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SWITCH_INFO_RECORD, - osm_sir_rcv_process, &p_sa->sir_rcv); - if( p_sa->sir_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sa->mft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MFT_RECORD, - osm_mftr_rcv_process, &p_sa->mftr_rcv); - if( p_sa->mft_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + p_sa->smir_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SMINFO_RECORD, + osm_smir_rcv_process, + &p_sa->smir_rcv); + if (p_sa->smir_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->mcmr_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_MCMEMBER_RECORD, + osm_mcmr_rcv_process, &p_sa->mcmr_rcv); + if (p_sa->mcmr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->sr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SERVICE_RECORD, + osm_sr_rcv_process, &p_sa->sr_rcv); + if (p_sa->sr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->infr_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_INFORM_INFO, + osm_infr_rcv_process, + &p_sa->infr_rcv); + if (p_sa->infr_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->infir_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_INFORM_INFO_RECORD, + osm_infir_rcv_process, &p_sa->infr_rcv); + if (p_sa->infir_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->vlarb_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_VL_ARB_RECORD, + osm_vlarb_rec_rcv_process, + &p_sa->vlarb_rec_rcv); + if (p_sa->vlarb_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->slvl_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_SLVL_TBL_RECORD, + osm_slvl_rec_rcv_process, &p_sa->slvl_rec_rcv); + if (p_sa->slvl_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->pkey_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_PKEY_TBL_RECORD, + osm_pkey_rec_rcv_process, &p_sa->pkey_rec_rcv); + if (p_sa->pkey_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->lft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LFT_RECORD, + osm_lftr_rcv_process, + &p_sa->lftr_rcv); + if (p_sa->lft_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->sir_disp_h = + cl_disp_register(p_disp, OSM_MSG_MAD_SWITCH_INFO_RECORD, + osm_sir_rcv_process, &p_sa->sir_rcv); + if (p_sa->sir_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sa->mft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MFT_RECORD, + osm_mftr_rcv_process, + &p_sa->mftr_rcv); + if (p_sa->mft_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sa_bind( - IN osm_sa_t* const p_sa, - IN const ib_net64_t port_guid ) +osm_sa_bind(IN osm_sa_t * const p_sa, IN const ib_net64_t port_guid) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_sa->p_log, osm_sa_bind ); + OSM_LOG_ENTER(p_sa->p_log, osm_sa_bind); - status = osm_sa_mad_ctrl_bind( &p_sa->mad_ctrl, port_guid ); + status = osm_sa_mad_ctrl_bind(&p_sa->mad_ctrl, port_guid); - if( status != IB_SUCCESS ) - { - osm_log( p_sa->p_log, OSM_LOG_ERROR, - "osm_sa_bind: ERR 4C03: " - "SA MAD Controller bind failed (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } + if (status != IB_SUCCESS) { + osm_log(p_sa->p_log, OSM_LOG_ERROR, + "osm_sa_bind: ERR 4C03: " + "SA MAD Controller bind failed (%s)\n", + ib_get_err_str(status)); + goto Exit; + } - Exit: - OSM_LOG_EXIT( p_sa->p_log ); - return( status ); + Exit: + OSM_LOG_EXIT(p_sa->p_log); + return (status); } /********************************************************************** @@ -532,11 +459,11 @@ struct opensm_dump_context { }; static int -opensm_dump_to_file(osm_opensm_t *p_osm, const char *file_name, - void (*dump_func)(osm_opensm_t *p_osm, FILE *file)) +opensm_dump_to_file(osm_opensm_t * p_osm, const char *file_name, + void (*dump_func) (osm_opensm_t * p_osm, FILE * file)) { char path[1024]; - FILE *file; + FILE *file; snprintf(path, sizeof(path), "%s/%s", p_osm->subn.opt.dump_files_dir, file_name); @@ -550,7 +477,7 @@ opensm_dump_to_file(osm_opensm_t *p_osm, const char *file_name, return -1; } - chmod(path, S_IRUSR|S_IWUSR); + chmod(path, S_IRUSR | S_IWUSR); dump_func(p_osm, file); @@ -558,28 +485,25 @@ opensm_dump_to_file(osm_opensm_t *p_osm, const char *file_name, return 0; } -static void -mcast_mgr_dump_one_port(cl_map_item_t *p_map_item, void *cxt) +static void mcast_mgr_dump_one_port(cl_map_item_t * p_map_item, void *cxt) { FILE *file = ((struct opensm_dump_context *)cxt)->file; - osm_mcm_port_t *p_mcm_port = (osm_mcm_port_t *)p_map_item; + osm_mcm_port_t *p_mcm_port = (osm_mcm_port_t *) p_map_item; fprintf(file, "mcm_port: " "port_gid=0x%016" PRIx64 ":0x%016" PRIx64 " " "scope_state=0x%02x proxy_join=0x%x" "\n\n", cl_ntoh64(p_mcm_port->port_gid.unicast.prefix), cl_ntoh64(p_mcm_port->port_gid.unicast.interface_id), - p_mcm_port->scope_state, - p_mcm_port->proxy_join); + p_mcm_port->scope_state, p_mcm_port->proxy_join); } -static void -sa_dump_one_mgrp(cl_map_item_t *p_map_item, void *cxt) +static void sa_dump_one_mgrp(cl_map_item_t * p_map_item, void *cxt) { struct opensm_dump_context dump_context; osm_opensm_t *p_osm = ((struct opensm_dump_context *)cxt)->p_osm; FILE *file = ((struct opensm_dump_context *)cxt)->file; - osm_mgrp_t *p_mgrp = (osm_mgrp_t *)p_map_item; + osm_mgrp_t *p_mgrp = (osm_mgrp_t *) p_map_item; fprintf(file, "MC Group 0x%04x %s:" " mgid=0x%016" PRIx64 ":0x%016" PRIx64 @@ -602,8 +526,7 @@ sa_dump_one_mgrp(cl_map_item_t *p_map_item, void *cxt) p_mgrp->mcmember_rec.pkt_life, cl_ntoh32(p_mgrp->mcmember_rec.sl_flow_hop), p_mgrp->mcmember_rec.scope_state, - p_mgrp->mcmember_rec.proxy_join - ); + p_mgrp->mcmember_rec.proxy_join); dump_context.p_osm = p_osm; dump_context.file = file; @@ -612,11 +535,10 @@ sa_dump_one_mgrp(cl_map_item_t *p_map_item, void *cxt) mcast_mgr_dump_one_port, &dump_context); } -static void -sa_dump_one_inform(cl_list_item_t *p_list_item, void *cxt) +static void sa_dump_one_inform(cl_list_item_t * p_list_item, void *cxt) { FILE *file = ((struct opensm_dump_context *)cxt)->file; - osm_infr_t *p_infr = (osm_infr_t *)p_list_item; + osm_infr_t *p_infr = (osm_infr_t *) p_list_item; ib_inform_info_record_t *p_iir = &p_infr->inform_record; fprintf(file, "InformInfo Record:" @@ -657,11 +579,10 @@ sa_dump_one_inform(cl_list_item_t *p_list_item, void *cxt) p_infr->report_addr.addr_type.gsi.service_level); } -static void -sa_dump_one_service(cl_list_item_t *p_list_item, void *cxt) +static void sa_dump_one_service(cl_list_item_t * p_list_item, void *cxt) { FILE *file = ((struct opensm_dump_context *)cxt)->file; - osm_svcr_t *p_svcr = (osm_svcr_t *)p_list_item; + osm_svcr_t *p_svcr = (osm_svcr_t *) p_list_item; ib_service_record_t *p_sr = &p_svcr->service_record; fprintf(file, "Service Record: id=0x%016" PRIx64 @@ -677,11 +598,11 @@ sa_dump_one_service(cl_list_item_t *p_list_item, void *cxt) " data32=0x%08x%08x:0x%08x%08x" " data64=0x%016" PRIx64 ":0x%016" PRIx64 " modified_time=0x%x lease_period=0x%x\n\n", - cl_ntoh64( p_sr->service_id ), - cl_ntoh64( p_sr->service_gid.unicast.prefix ), - cl_ntoh64( p_sr->service_gid.unicast.interface_id ), - cl_ntoh16( p_sr->service_pkey ), - cl_ntoh32( p_sr->service_lease ), + cl_ntoh64(p_sr->service_id), + cl_ntoh64(p_sr->service_gid.unicast.prefix), + cl_ntoh64(p_sr->service_gid.unicast.interface_id), + cl_ntoh16(p_sr->service_pkey), + cl_ntoh32(p_sr->service_lease), p_sr->service_key[0], p_sr->service_key[1], p_sr->service_key[2], p_sr->service_key[3], p_sr->service_key[4], p_sr->service_key[5], @@ -716,14 +637,14 @@ sa_dump_one_service(cl_list_item_t *p_list_item, void *cxt) p_svcr->modified_time, p_svcr->lease_period); } -static void -sa_dump_all_sa(osm_opensm_t *p_osm, FILE *file) +static void sa_dump_all_sa(osm_opensm_t * p_osm, FILE * file) { struct opensm_dump_context dump_context; dump_context.p_osm = p_osm; dump_context.file = file; - osm_log(&p_osm->log, OSM_LOG_DEBUG, "sa_dump_all_sa: Dump multicast:\n"); + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "sa_dump_all_sa: Dump multicast:\n"); cl_plock_acquire(&p_osm->lock); cl_qmap_apply_func(&p_osm->subn.mgrp_mlid_tbl, sa_dump_one_mgrp, &dump_context); @@ -736,7 +657,7 @@ sa_dump_all_sa(osm_opensm_t *p_osm, FILE *file) cl_plock_release(&p_osm->lock); } -int osm_sa_db_file_dump(osm_opensm_t *p_osm) +int osm_sa_db_file_dump(osm_opensm_t * p_osm) { return opensm_dump_to_file(p_osm, "opensm-sa.dump", sa_dump_all_sa); } @@ -744,8 +665,8 @@ int osm_sa_db_file_dump(osm_opensm_t *p_osm) /* * SA DB Loader */ -osm_mgrp_t *load_mcgroup(osm_opensm_t *p_osm, ib_net16_t mlid, - ib_member_rec_t *p_mcm_rec, unsigned well_known) +osm_mgrp_t *load_mcgroup(osm_opensm_t * p_osm, ib_net16_t mlid, + ib_member_rec_t * p_mcm_rec, unsigned well_known) { ib_net64_t comp_mask; cl_map_item_t *p_next; @@ -754,8 +675,8 @@ osm_mgrp_t *load_mcgroup(osm_opensm_t *p_osm, ib_net16_t mlid, cl_plock_excl_acquire(&p_osm->lock); if ((p_next = cl_qmap_get(&p_osm->subn.mgrp_mlid_tbl, mlid)) != - cl_qmap_end(&p_osm->subn.mgrp_mlid_tbl)) { - p_mgrp = (osm_mgrp_t *)p_next; + cl_qmap_end(&p_osm->subn.mgrp_mlid_tbl)) { + p_mgrp = (osm_mgrp_t *) p_next; if (!memcmp(&p_mgrp->mcmember_rec.mgid, &p_mcm_rec->mgid, sizeof(ib_gid_t))) { osm_log(&p_osm->log, OSM_LOG_DEBUG, @@ -769,10 +690,10 @@ osm_mgrp_t *load_mcgroup(osm_opensm_t *p_osm, ib_net16_t mlid, cl_ntoh16(mlid)); p_mgrp = NULL; goto _out; - } + } comp_mask = IB_MCR_COMPMASK_MTU | IB_MCR_COMPMASK_MTU_SEL - | IB_MCR_COMPMASK_RATE | IB_MCR_COMPMASK_RATE_SEL; + | IB_MCR_COMPMASK_RATE | IB_MCR_COMPMASK_RATE_SEL; if (osm_mcmr_rcv_find_or_create_new_mgrp(&p_osm->sa.mcmr_rcv, comp_mask, p_mcm_rec, &p_mgrp) != IB_SUCCESS || @@ -783,18 +704,17 @@ osm_mgrp_t *load_mcgroup(osm_opensm_t *p_osm, ib_net16_t mlid, cl_ntoh16(mlid), cl_ntoh64(p_mcm_rec->mgid.unicast.prefix), cl_ntoh64(p_mcm_rec->mgid.unicast.interface_id)); - p_mgrp=NULL; - } - else if (well_known) + p_mgrp = NULL; + } else if (well_known) p_mgrp->well_known = TRUE; - _out: + _out: cl_plock_release(&p_osm->lock); return p_mgrp; } -static int load_svcr(osm_opensm_t *p_osm, ib_service_record_t *sr, +static int load_svcr(osm_opensm_t * p_osm, ib_service_record_t * sr, uint32_t modified_time, uint32_t lease_period) { osm_svcr_t *p_svcr; @@ -802,7 +722,7 @@ static int load_svcr(osm_opensm_t *p_osm, ib_service_record_t *sr, cl_plock_excl_acquire(&p_osm->lock); - if(osm_svcr_get_by_rid(&p_osm->subn, &p_osm->log, sr)) { + if (osm_svcr_get_by_rid(&p_osm->subn, &p_osm->log, sr)) { osm_log(&p_osm->log, OSM_LOG_VERBOSE, "load_svcr: ServiceRecord already exists\n"); goto _out; @@ -826,14 +746,14 @@ static int load_svcr(osm_opensm_t *p_osm, ib_service_record_t *sr, if (lease_period != 0xffffffff) cl_timer_trim(&p_osm->sa.sr_rcv.sr_timer, 1000); - _out: + _out: cl_plock_release(&p_osm->lock); return ret; } -static int load_infr(osm_opensm_t *p_osm, ib_inform_info_record_t *iir, - osm_mad_addr_t *addr) +static int load_infr(osm_opensm_t * p_osm, ib_inform_info_record_t * iir, + osm_mad_addr_t * addr) { osm_infr_t infr, *p_infr; int ret = 0; @@ -864,13 +784,12 @@ static int load_infr(osm_opensm_t *p_osm, ib_inform_info_record_t *iir, osm_infr_insert_to_db(&p_osm->subn, &p_osm->log, p_infr); - _out: + _out: cl_plock_release(&p_osm->lock); return ret; } - #define UNPACK_FUNC(name,x) \ int unpack_##name##x(char *p, uint##x##_t *val_ptr) \ { \ @@ -888,12 +807,12 @@ int unpack_##name##x(char *p, uint##x##_t *val_ptr) \ #define cl_hton8(x) (x) -UNPACK_FUNC(net,8); -UNPACK_FUNC(net,16); -UNPACK_FUNC(net,32); -UNPACK_FUNC(net,64); +UNPACK_FUNC(net, 8); +UNPACK_FUNC(net, 16); +UNPACK_FUNC(net, 32); +UNPACK_FUNC(net, 64); -static int unpack_string(char *p, uint8_t *buf, unsigned len) +static int unpack_string(char *p, uint8_t * buf, unsigned len) { char *q = p; char delim = ' '; @@ -908,7 +827,7 @@ static int unpack_string(char *p, uint8_t *buf, unsigned len) return (int)(q - p); } -static int unpack_string64(char *p, uint8_t *buf) +static int unpack_string64(char *p, uint8_t * buf) { return unpack_string(p, buf, 64); } @@ -934,7 +853,7 @@ static int unpack_string64(char *p, uint8_t *buf) p += _ret; \ } -int osm_sa_db_file_load(osm_opensm_t *p_osm) +int osm_sa_db_file_load(osm_opensm_t * p_osm) { char line[1024]; char *file_name; @@ -954,7 +873,7 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) file = fopen(file_name, "r"); if (!file) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "osm_sa_db_file_load: ERR 4C02: " "cannot open sa db file \'%s\'. " "Skip restoring\n", file_name); @@ -985,7 +904,7 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) memset(&mcm_rec, 0, sizeof(mcm_rec)); PARSE_AHEAD(p, net16, " 0x", &mlid); - if(strstr(p, "well known")) + if (strstr(p, "well known")) well_known = 1; PARSE_AHEAD(p, net64, " mgid=0x", &mcm_rec.mgid.unicast.prefix); @@ -1013,8 +932,7 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) well_known); if (!p_mgrp) rereg_clients = 1; - } - else if (p_mgrp && !strncmp(p, "mcm_port", 8)) { + } else if (p_mgrp && !strncmp(p, "mcm_port", 8)) { ib_gid_t port_gid; ib_net64_t guid; uint8_t scope_state; @@ -1031,11 +949,10 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) guid = port_gid.unicast.interface_id; if (cl_qmap_get(&p_mgrp->mcm_port_tbl, port_gid.unicast.interface_id) == - cl_qmap_end(&p_mgrp->mcm_port_tbl)) + cl_qmap_end(&p_mgrp->mcm_port_tbl)) osm_mgrp_add_port(p_mgrp, &port_gid, scope_state, proxy_join); - } - else if (!strncmp(p, "Service Record:", 15)) { + } else if (!strncmp(p, "Service Record:", 15)) { ib_service_record_t s_rec; uint32_t modified_time, lease_period; @@ -1048,25 +965,27 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) PARSE_AHEAD(p, net64, ":0x", &s_rec.service_gid.unicast.interface_id); PARSE_AHEAD(p, net16, " pkey=0x", &s_rec.service_pkey); - PARSE_AHEAD(p, net32, " lease=0x", &s_rec.service_lease); + PARSE_AHEAD(p, net32, " lease=0x", + &s_rec.service_lease); PARSE_AHEAD(p, net64, " key=0x", - (ib_net64_t *)(&s_rec.service_key[0])); + (ib_net64_t *) (&s_rec.service_key[0])); PARSE_AHEAD(p, net64, ":0x", - (ib_net64_t *)(&s_rec.service_key[8])); + (ib_net64_t *) (&s_rec.service_key[8])); PARSE_AHEAD(p, string64, " name=", s_rec.service_name); PARSE_AHEAD(p, net64, " data8=0x", - (ib_net64_t *)(&s_rec.service_data8[0])); + (ib_net64_t *) (&s_rec.service_data8[0])); PARSE_AHEAD(p, net64, ":0x", - (ib_net64_t *)(&s_rec.service_data8[8])); + (ib_net64_t *) (&s_rec.service_data8[8])); PARSE_AHEAD(p, net64, " data16=0x", - (ib_net64_t *)(&s_rec.service_data16[0])); + (ib_net64_t *) (&s_rec.service_data16[0])); PARSE_AHEAD(p, net64, ":0x", - (ib_net64_t *)(&s_rec.service_data16[4])); + (ib_net64_t *) (&s_rec.service_data16[4])); PARSE_AHEAD(p, net64, " data32=0x", - (ib_net64_t *)(&s_rec.service_data32[0])); + (ib_net64_t *) (&s_rec.service_data32[0])); PARSE_AHEAD(p, net64, ":0x", - (ib_net64_t *)(&s_rec.service_data32[2])); - PARSE_AHEAD(p, net64, " data64=0x", &s_rec.service_data64[0]); + (ib_net64_t *) (&s_rec.service_data32[2])); + PARSE_AHEAD(p, net64, " data64=0x", + &s_rec.service_data64[0]); PARSE_AHEAD(p, net64, ":0x", &s_rec.service_data64[1]); PARSE_AHEAD(p, net32, " modified_time=0x", &modified_time); @@ -1076,8 +995,7 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) if (load_svcr(p_osm, &s_rec, cl_ntoh32(modified_time), cl_ntoh32(lease_period))) rereg_clients = 1; - } - else if (!strncmp(p, "InformInfo Record:", 18)) { + } else if (!strncmp(p, "InformInfo Record:", 18)) { ib_inform_info_record_t i_rec; osm_mad_addr_t rep_addr; @@ -1094,7 +1012,8 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) PARSE_AHEAD(p, net64, " gid=0x", &i_rec.inform_info.gid.unicast.prefix); PARSE_AHEAD(p, net64, ":0x", - &i_rec.inform_info.gid.unicast.interface_id); + &i_rec.inform_info.gid.unicast. + interface_id); PARSE_AHEAD(p, net16, " lid_range_begin=0x", &i_rec.inform_info.lid_range_begin); PARSE_AHEAD(p, net16, " lid_range_end=0x", @@ -1108,9 +1027,11 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) PARSE_AHEAD(p, net16, " trap_num=0x", &i_rec.inform_info.g_or_v.generic.trap_num); PARSE_AHEAD(p, net32, " qpn_resp_time_val=0x", - &i_rec.inform_info.g_or_v.generic.qpn_resp_time_val); + &i_rec.inform_info.g_or_v.generic. + qpn_resp_time_val); PARSE_AHEAD(p, net32, " node_type=0x", - (uint32_t *)&i_rec.inform_info.g_or_v.generic.reserved2); + (uint32_t *) & i_rec.inform_info.g_or_v. + generic.reserved2); PARSE_AHEAD(p, net16, " rep_addr: lid=0x", &rep_addr.dest_lid); @@ -1135,7 +1056,7 @@ int osm_sa_db_file_load(osm_opensm_t *p_osm) if (!rereg_clients) p_osm->subn.opt.no_clients_rereg = TRUE; - _error: + _error: fclose(file); return ret; } diff --git a/opensm/opensm/osm_sa_class_port_info.c b/opensm/opensm/osm_sa_class_port_info.c index a4d376d..aa227d5 100644 --- a/opensm/opensm/osm_sa_class_port_info.c +++ b/opensm/opensm/osm_sa_class_port_info.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -63,216 +63,205 @@ #define MAX_MSECS_TO_RTV 24 /* Precalculated table in msec (index is related to encoded value) */ /* 4.096 usec * 2 ** n (where n = 8 - 31) */ -static uint32_t __msecs_to_rtv_table[MAX_MSECS_TO_RTV] = - { 1, 2, 4, 8, - 16, 33, 67, 134, - 268, 536, 1073, 2147, - 4294, 8589, 17179, 34359, - 68719, 137438, 274877, 549755, - 1099511, 2199023, 4398046, 8796093 }; +static uint32_t __msecs_to_rtv_table[MAX_MSECS_TO_RTV] = { 1, 2, 4, 8, + 16, 33, 67, 134, + 268, 536, 1073, 2147, + 4294, 8589, 17179, 34359, + 68719, 137438, 274877, 549755, + 1099511, 2199023, 4398046, 8796093 +}; /********************************************************************** **********************************************************************/ -void -osm_cpi_rcv_construct( - IN osm_cpi_rcv_t* const p_rcv ) +void osm_cpi_rcv_construct(IN osm_cpi_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_cpi_rcv_destroy( - IN osm_cpi_rcv_t* const p_rcv ) +void osm_cpi_rcv_destroy(IN osm_cpi_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_cpi_rcv_destroy ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_cpi_rcv_destroy); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_cpi_rcv_init( - IN osm_cpi_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_cpi_rcv_init(IN osm_cpi_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_cpi_rcv_init ); + OSM_LOG_ENTER(p_log, osm_cpi_rcv_init); - osm_cpi_rcv_construct( p_rcv ); + osm_cpi_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_cpi_rcv_respond( - IN osm_cpi_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__osm_cpi_rcv_respond(IN osm_cpi_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - osm_madw_t* p_resp_madw; - const ib_sa_mad_t* p_sa_mad; - ib_sa_mad_t* p_resp_sa_mad; - ib_class_port_info_t *p_resp_cpi; - ib_api_status_t status; - ib_gid_t zero_gid; - uint8_t rtv; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_cpi_rcv_respond ); - - memset(&zero_gid, 0, sizeof(ib_gid_t)); - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - MAD_BLOCK_SIZE, - &p_madw->mad_addr ); - if( !p_resp_madw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_cpi_rcv_respond: ERR 1408: " - "Unable to allocate MAD\n" ); - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - memcpy( p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - p_resp_cpi = (ib_class_port_info_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); - - /* finally do it (the job) man ! */ - p_resp_cpi->base_ver = 1; - p_resp_cpi->class_ver = 2; - /* Calculate encoded response time value */ - /* transaction timeout is in msec */ - if (p_rcv->p_subn->opt.transaction_timeout > __msecs_to_rtv_table[MAX_MSECS_TO_RTV]) - rtv = MAX_MSECS_TO_RTV - 1; - else - { - for (rtv = 0; rtv < MAX_MSECS_TO_RTV; rtv++) { - if (p_rcv->p_subn->opt.transaction_timeout <= __msecs_to_rtv_table[rtv]) - break; - } - } - rtv += 8; - p_resp_cpi->resp_time_val = rtv; - p_resp_cpi->redir_gid = zero_gid; - p_resp_cpi->redir_tc_sl_fl = 0; - p_resp_cpi->redir_lid = 0; - p_resp_cpi->redir_pkey = 0; - p_resp_cpi->redir_qp = CL_NTOH32(1); - p_resp_cpi->redir_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - p_resp_cpi->trap_gid = zero_gid; - p_resp_cpi->trap_tc_sl_fl = 0; - p_resp_cpi->trap_lid = 0; - p_resp_cpi->trap_pkey = 0; - p_resp_cpi->trap_hop_qp = 0; - p_resp_cpi->trap_qkey = IB_QP1_WELL_KNOWN_Q_KEY; - - /* set specific capability mask bits */ - /* we do not support the following options/optional records: - OSM_CAP_IS_SUBN_OPT_RECS_SUP : - RandomForwardingTableRecord, - ServiceAssociationRecord - other optional records supported "under the table" - - OSM_CAP_IS_MULTIPATH_SUP: - TraceRecord - - OSM_CAP_IS_REINIT_SUP: - For reinitialization functionality. - - So not sending traps, but supporting Get(Notice) and Set(Notice). - */ - - /* Note host notation replaced later */ + osm_madw_t *p_resp_madw; + const ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_resp_sa_mad; + ib_class_port_info_t *p_resp_cpi; + ib_api_status_t status; + ib_gid_t zero_gid; + uint8_t rtv; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_cpi_rcv_respond); + + memset(&zero_gid, 0, sizeof(ib_gid_t)); + + /* + Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + MAD_BLOCK_SIZE, &p_madw->mad_addr); + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_cpi_rcv_respond: ERR 1408: " + "Unable to allocate MAD\n"); + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + p_resp_cpi = + (ib_class_port_info_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); + + /* finally do it (the job) man ! */ + p_resp_cpi->base_ver = 1; + p_resp_cpi->class_ver = 2; + /* Calculate encoded response time value */ + /* transaction timeout is in msec */ + if (p_rcv->p_subn->opt.transaction_timeout > + __msecs_to_rtv_table[MAX_MSECS_TO_RTV]) + rtv = MAX_MSECS_TO_RTV - 1; + else { + for (rtv = 0; rtv < MAX_MSECS_TO_RTV; rtv++) { + if (p_rcv->p_subn->opt.transaction_timeout <= + __msecs_to_rtv_table[rtv]) + break; + } + } + rtv += 8; + p_resp_cpi->resp_time_val = rtv; + p_resp_cpi->redir_gid = zero_gid; + p_resp_cpi->redir_tc_sl_fl = 0; + p_resp_cpi->redir_lid = 0; + p_resp_cpi->redir_pkey = 0; + p_resp_cpi->redir_qp = CL_NTOH32(1); + p_resp_cpi->redir_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + p_resp_cpi->trap_gid = zero_gid; + p_resp_cpi->trap_tc_sl_fl = 0; + p_resp_cpi->trap_lid = 0; + p_resp_cpi->trap_pkey = 0; + p_resp_cpi->trap_hop_qp = 0; + p_resp_cpi->trap_qkey = IB_QP1_WELL_KNOWN_Q_KEY; + + /* set specific capability mask bits */ + /* we do not support the following options/optional records: + OSM_CAP_IS_SUBN_OPT_RECS_SUP : + RandomForwardingTableRecord, + ServiceAssociationRecord + other optional records supported "under the table" + + OSM_CAP_IS_MULTIPATH_SUP: + TraceRecord + + OSM_CAP_IS_REINIT_SUP: + For reinitialization functionality. + + So not sending traps, but supporting Get(Notice) and Set(Notice). + */ + + /* Note host notation replaced later */ #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - p_resp_cpi->cap_mask = OSM_CAP_IS_SUBN_GET_SET_NOTICE_SUP | - OSM_CAP_IS_PORT_INFO_CAPMASK_MATCH_SUPPORTED | - OSM_CAP_IS_MULTIPATH_SUP; + p_resp_cpi->cap_mask = OSM_CAP_IS_SUBN_GET_SET_NOTICE_SUP | + OSM_CAP_IS_PORT_INFO_CAPMASK_MATCH_SUPPORTED | + OSM_CAP_IS_MULTIPATH_SUP; #else - p_resp_cpi->cap_mask = OSM_CAP_IS_SUBN_GET_SET_NOTICE_SUP | - OSM_CAP_IS_PORT_INFO_CAPMASK_MATCH_SUPPORTED; + p_resp_cpi->cap_mask = OSM_CAP_IS_SUBN_GET_SET_NOTICE_SUP | + OSM_CAP_IS_PORT_INFO_CAPMASK_MATCH_SUPPORTED; #endif - if (p_rcv->p_subn->opt.no_multicast_option != TRUE) - p_resp_cpi->cap_mask |= OSM_CAP_IS_UD_MCAST_SUP; - p_resp_cpi->cap_mask = cl_hton16(p_resp_cpi->cap_mask); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_FRAMES ) ) - osm_dump_sa_mad( p_rcv->p_log, p_resp_sa_mad, OSM_LOG_FRAMES ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if( status != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_cpi_rcv_respond: ERR 1409: " - "Unable to send MAD (%s)\n", ib_get_err_str( status ) ); - /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + if (p_rcv->p_subn->opt.no_multicast_option != TRUE) + p_resp_cpi->cap_mask |= OSM_CAP_IS_UD_MCAST_SUP; + p_resp_cpi->cap_mask = cl_hton16(p_resp_cpi->cap_mask); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_FRAMES)) + osm_dump_sa_mad(p_rcv->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_cpi_rcv_respond: ERR 1409: " + "Unable to send MAD (%s)\n", ib_get_err_str(status)); + /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** * This code actually handles the call **********************************************************************/ -void -osm_cpi_rcv_process( - IN void *context, - IN void *data ) +void osm_cpi_rcv_process(IN void *context, IN void *data) { - osm_cpi_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_sa_mad; + osm_cpi_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_sa_mad; - OSM_LOG_ENTER( p_rcv->p_log, osm_cpi_rcv_process ); + OSM_LOG_ENTER(p_rcv->p_log, osm_cpi_rcv_process); - CL_ASSERT( p_madw ); + CL_ASSERT(p_madw); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - /* we only support GET */ - if (p_sa_mad->method != IB_MAD_METHOD_GET) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_cpi_rcv_process: ERR 1403: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_sa_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID); - goto Exit; - } + /* we only support GET */ + if (p_sa_mad->method != IB_MAD_METHOD_GET) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_cpi_rcv_process: ERR 1403: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_sa_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } - CL_ASSERT( p_sa_mad->attr_id == IB_MAD_ATTR_CLASS_PORT_INFO ); + CL_ASSERT(p_sa_mad->attr_id == IB_MAD_ATTR_CLASS_PORT_INFO); - /* - CLASS PORT INFO does not really look on the SMDB - no lock required. - */ + /* + CLASS PORT INFO does not really look on the SMDB - no lock required. + */ - __osm_cpi_rcv_respond( p_rcv, p_madw); + __osm_cpi_rcv_respond(p_rcv, p_madw); - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_guidinfo_record.c b/opensm/opensm/osm_sa_guidinfo_record.c index 9a02373..de1af88 100644 --- a/opensm/opensm/osm_sa_guidinfo_record.c +++ b/opensm/opensm/osm_sa_guidinfo_record.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_gir_rcv_t. @@ -47,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,546 +64,519 @@ #define OSM_GIR_RCV_POOL_MIN_SIZE 32 #define OSM_GIR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_gir_item -{ - cl_pool_item_t pool_item; - ib_guidinfo_record_t rec; +typedef struct _osm_gir_item { + cl_pool_item_t pool_item; + ib_guidinfo_record_t rec; } osm_gir_item_t; -typedef struct _osm_gir_search_ctxt -{ - const ib_guidinfo_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_gir_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_gir_search_ctxt { + const ib_guidinfo_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_gir_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_gir_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_gir_rcv_construct( - IN osm_gir_rcv_t* const p_rcv ) +void osm_gir_rcv_construct(IN osm_gir_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_gir_rcv_destroy( - IN osm_gir_rcv_t* const p_rcv ) +void osm_gir_rcv_destroy(IN osm_gir_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_gir_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_gir_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_gir_rcv_init( - IN osm_gir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_gir_rcv_init(IN osm_gir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_gir_rcv_init ); + OSM_LOG_ENTER(p_log, osm_gir_rcv_init); - osm_gir_rcv_construct( p_rcv ); + osm_gir_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_GIR_RCV_POOL_MIN_SIZE, - 0, - OSM_GIR_RCV_POOL_GROW_SIZE, - sizeof(osm_gir_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_GIR_RCV_POOL_MIN_SIZE, + 0, + OSM_GIR_RCV_POOL_GROW_SIZE, + sizeof(osm_gir_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_gir_rcv_new_gir( - IN osm_gir_rcv_t* const p_rcv, - IN const osm_node_t* const p_node, - IN cl_qlist_t* const p_list, - IN ib_net64_t const match_port_guid, - IN ib_net16_t const match_lid, - IN const osm_physp_t* const p_req_physp, - IN uint8_t const block_num ) +__osm_gir_rcv_new_gir(IN osm_gir_rcv_t * const p_rcv, + IN const osm_node_t * const p_node, + IN cl_qlist_t * const p_list, + IN ib_net64_t const match_port_guid, + IN ib_net16_t const match_lid, + IN const osm_physp_t * const p_req_physp, + IN uint8_t const block_num) { - osm_gir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_gir_rcv_new_gir ); - - p_rec_item = (osm_gir_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_gir_rcv_new_gir: ERR 5102: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_gir_rcv_new_gir: " - "New GUIDInfoRecord: lid 0x%X, block num %d\n", - cl_ntoh16( match_lid ), block_num ); - } - - memset( &p_rec_item->rec, 0, sizeof( p_rec_item->rec ) ); - - p_rec_item->rec.lid = match_lid; - p_rec_item->rec.block_num = block_num; - if (!block_num) - p_rec_item->rec.guid_info.guid[0] = osm_physp_get_port_guid( p_req_physp ); - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_gir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_gir_rcv_new_gir); + + p_rec_item = (osm_gir_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_gir_rcv_new_gir: ERR 5102: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_gir_rcv_new_gir: " + "New GUIDInfoRecord: lid 0x%X, block num %d\n", + cl_ntoh16(match_lid), block_num); + } + + memset(&p_rec_item->rec, 0, sizeof(p_rec_item->rec)); + + p_rec_item->rec.lid = match_lid; + p_rec_item->rec.block_num = block_num; + if (!block_num) + p_rec_item->rec.guid_info.guid[0] = + osm_physp_get_port_guid(p_req_physp); + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -__osm_sa_gir_create_gir( - IN osm_gir_rcv_t* const p_rcv, - IN const osm_node_t* const p_node, - IN cl_qlist_t* const p_list, - IN ib_net64_t const match_port_guid, - IN ib_net16_t const match_lid, - IN const osm_physp_t* const p_req_physp, - IN uint8_t const match_block_num ) +__osm_sa_gir_create_gir(IN osm_gir_rcv_t * const p_rcv, + IN const osm_node_t * const p_node, + IN cl_qlist_t * const p_list, + IN ib_net64_t const match_port_guid, + IN ib_net16_t const match_lid, + IN const osm_physp_t * const p_req_physp, + IN uint8_t const match_block_num) { - const osm_physp_t* p_physp; - uint8_t port_num; - uint8_t num_ports; - uint16_t match_lid_ho; - ib_net16_t base_lid_ho; - ib_net16_t max_lid_ho; - uint8_t lmc; - ib_net64_t port_guid; - uint8_t block_num, start_block_num, end_block_num, num_blocks; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_gir_create_gir ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_gir_create_gir: " - "Looking for GUIDRecord with LID: 0x%X GUID:0x%016" PRIx64 "\n", - cl_ntoh16( match_lid ), - cl_ntoh64( match_port_guid ) - ); - } - - /* - For switches, do not return the GUIDInfo record(s) - for each port on the switch, just for port 0. - */ - if( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ) - num_ports = 1; - else - num_ports = osm_node_get_num_physp( p_node ); - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - - if( !osm_physp_is_valid( p_physp ) ) - continue; - - /* Check to see if the found p_physp and the requester physp - share a pkey. If not, continue */ - if (!osm_physp_share_pkey( p_rcv->p_log, p_physp, p_req_physp ) ) - continue; - - port_guid = osm_physp_get_port_guid( p_physp ); - - if( match_port_guid && ( port_guid != match_port_guid ) ) - continue; - - /* - Note: the following check is a temporary workaround - Since 1. GUIDCap should never be 0 on ports where this applies - and 2. GUIDCap should not be used on ports where it doesn't apply - So this should really be a check for whether the port is a - switch external port or not! - */ - if ( p_physp->port_info.guid_cap == 0 ) - continue; - - num_blocks = p_physp->port_info.guid_cap / 8; - if ( p_physp->port_info.guid_cap % 8 ) - num_blocks++; - if ( match_block_num == 255 ) - { - start_block_num = 0; - end_block_num = num_blocks - 1; - } - else - { - if ( match_block_num >= num_blocks ) - continue; - end_block_num = start_block_num = match_block_num; - } - - base_lid_ho = cl_ntoh16( osm_physp_get_base_lid( p_physp ) ); - match_lid_ho = cl_ntoh16( match_lid ); - if( match_lid_ho ) - { - lmc = osm_physp_get_lmc( p_physp ); - max_lid_ho = (uint16_t)( base_lid_ho + (1 << lmc) - 1 ); - - /* - We validate that the lid belongs to this node. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_gir_create_gir: " - "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", - base_lid_ho, match_lid_ho, max_lid_ho - ); - } - - if ( match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho ) - continue; - } - - for (block_num = start_block_num; block_num <= end_block_num; block_num++) - __osm_gir_rcv_new_gir( p_rcv, p_node, p_list, - port_guid, cl_ntoh16(base_lid_ho), - p_physp, block_num ); - - } - - OSM_LOG_EXIT( p_rcv->p_log ); + const osm_physp_t *p_physp; + uint8_t port_num; + uint8_t num_ports; + uint16_t match_lid_ho; + ib_net16_t base_lid_ho; + ib_net16_t max_lid_ho; + uint8_t lmc; + ib_net64_t port_guid; + uint8_t block_num, start_block_num, end_block_num, num_blocks; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_gir_create_gir); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_gir_create_gir: " + "Looking for GUIDRecord with LID: 0x%X GUID:0x%016" + PRIx64 "\n", cl_ntoh16(match_lid), + cl_ntoh64(match_port_guid) + ); + } + + /* + For switches, do not return the GUIDInfo record(s) + for each port on the switch, just for port 0. + */ + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) + num_ports = 1; + else + num_ports = osm_node_get_num_physp(p_node); + + for (port_num = 0; port_num < num_ports; port_num++) { + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + if (!osm_physp_is_valid(p_physp)) + continue; + + /* Check to see if the found p_physp and the requester physp + share a pkey. If not, continue */ + if (!osm_physp_share_pkey(p_rcv->p_log, p_physp, p_req_physp)) + continue; + + port_guid = osm_physp_get_port_guid(p_physp); + + if (match_port_guid && (port_guid != match_port_guid)) + continue; + + /* + Note: the following check is a temporary workaround + Since 1. GUIDCap should never be 0 on ports where this applies + and 2. GUIDCap should not be used on ports where it doesn't apply + So this should really be a check for whether the port is a + switch external port or not! + */ + if (p_physp->port_info.guid_cap == 0) + continue; + + num_blocks = p_physp->port_info.guid_cap / 8; + if (p_physp->port_info.guid_cap % 8) + num_blocks++; + if (match_block_num == 255) { + start_block_num = 0; + end_block_num = num_blocks - 1; + } else { + if (match_block_num >= num_blocks) + continue; + end_block_num = start_block_num = match_block_num; + } + + base_lid_ho = cl_ntoh16(osm_physp_get_base_lid(p_physp)); + match_lid_ho = cl_ntoh16(match_lid); + if (match_lid_ho) { + lmc = osm_physp_get_lmc(p_physp); + max_lid_ho = (uint16_t) (base_lid_ho + (1 << lmc) - 1); + + /* + We validate that the lid belongs to this node. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_gir_create_gir: " + "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", + base_lid_ho, match_lid_ho, max_lid_ho); + } + + if (match_lid_ho < base_lid_ho + || match_lid_ho > max_lid_ho) + continue; + } + + for (block_num = start_block_num; block_num <= end_block_num; + block_num++) + __osm_gir_rcv_new_gir(p_rcv, p_node, p_list, port_guid, + cl_ntoh16(base_lid_ho), p_physp, + block_num); + + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_gir_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_gir_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_gir_search_ctxt_t* const p_ctxt = (osm_gir_search_ctxt_t *)context; - const osm_node_t* const p_node = (osm_node_t*)p_map_item; - const ib_guidinfo_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - osm_gir_rcv_t* const p_rcv = p_ctxt->p_rcv; - const ib_guid_info_t* p_comp_gi; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - ib_net64_t match_port_guid = 0; - ib_net16_t match_lid = 0; - uint8_t match_block_num = 255; - - OSM_LOG_ENTER( p_ctxt->p_rcv->p_log, __osm_sa_gir_by_comp_mask_cb); - - if( comp_mask & IB_GIR_COMPMASK_LID ) - match_lid = p_rcvd_rec->lid; - - if( comp_mask & IB_GIR_COMPMASK_BLOCKNUM ) - match_block_num = p_rcvd_rec->block_num; - - p_comp_gi = &p_rcvd_rec->guid_info; - /* Different rule for block 0 v. other blocks */ - if( comp_mask & IB_GIR_COMPMASK_GID0 ) - { - if ( !p_rcvd_rec->block_num ) - match_port_guid = osm_physp_get_port_guid( p_req_physp ); - if ( p_comp_gi->guid[0] != match_port_guid ) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID1 ) - { - if ( p_comp_gi->guid[1] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID2 ) - { - if ( p_comp_gi->guid[2] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID3 ) - { - if ( p_comp_gi->guid[3] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID4 ) - { - if ( p_comp_gi->guid[4] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID5 ) - { - if ( p_comp_gi->guid[5] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID6 ) - { - if ( p_comp_gi->guid[6] != 0) - goto Exit; - } - - if( comp_mask & IB_GIR_COMPMASK_GID7 ) - { - if ( p_comp_gi->guid[7] != 0) - goto Exit; - } - - __osm_sa_gir_create_gir( p_rcv, p_node, p_ctxt->p_list, - match_port_guid, match_lid, p_req_physp, - match_block_num ); - - Exit: - OSM_LOG_EXIT( p_ctxt->p_rcv->p_log ); + const osm_gir_search_ctxt_t *const p_ctxt = + (osm_gir_search_ctxt_t *) context; + const osm_node_t *const p_node = (osm_node_t *) p_map_item; + const ib_guidinfo_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + osm_gir_rcv_t *const p_rcv = p_ctxt->p_rcv; + const ib_guid_info_t *p_comp_gi; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + ib_net64_t match_port_guid = 0; + ib_net16_t match_lid = 0; + uint8_t match_block_num = 255; + + OSM_LOG_ENTER(p_ctxt->p_rcv->p_log, __osm_sa_gir_by_comp_mask_cb); + + if (comp_mask & IB_GIR_COMPMASK_LID) + match_lid = p_rcvd_rec->lid; + + if (comp_mask & IB_GIR_COMPMASK_BLOCKNUM) + match_block_num = p_rcvd_rec->block_num; + + p_comp_gi = &p_rcvd_rec->guid_info; + /* Different rule for block 0 v. other blocks */ + if (comp_mask & IB_GIR_COMPMASK_GID0) { + if (!p_rcvd_rec->block_num) + match_port_guid = osm_physp_get_port_guid(p_req_physp); + if (p_comp_gi->guid[0] != match_port_guid) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID1) { + if (p_comp_gi->guid[1] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID2) { + if (p_comp_gi->guid[2] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID3) { + if (p_comp_gi->guid[3] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID4) { + if (p_comp_gi->guid[4] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID5) { + if (p_comp_gi->guid[5] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID6) { + if (p_comp_gi->guid[6] != 0) + goto Exit; + } + + if (comp_mask & IB_GIR_COMPMASK_GID7) { + if (p_comp_gi->guid[7] != 0) + goto Exit; + } + + __osm_sa_gir_create_gir(p_rcv, p_node, p_ctxt->p_list, + match_port_guid, match_lid, p_req_physp, + match_block_num); + + Exit: + OSM_LOG_EXIT(p_ctxt->p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_gir_rcv_process( - IN void *ctx, - IN void *data ) +void osm_gir_rcv_process(IN void *ctx, IN void *data) { - osm_gir_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_guidinfo_record_t* p_rcvd_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_guidinfo_record_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + osm_gir_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_guidinfo_record_t *p_rcvd_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_guidinfo_record_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_gir_search_ctxt_t context; - osm_gir_item_t* p_rec_item; - ib_api_status_t status; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_gir_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_guidinfo_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_GUIDINFO_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_gir_rcv_process: ERR 5105: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_gir_rcv_process: ERR 5104: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_guidinfo_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - cl_qmap_apply_func( &p_rcv->p_subn->node_guid_tbl, - __osm_sa_gir_by_comp_mask_cb, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_gir_rcv_process: ERR 5103: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_rec_item = (osm_gir_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_gir_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_gir_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_gir_search_ctxt_t context; + osm_gir_item_t *p_rec_item; + ib_api_status_t status; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_gir_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_GUIDINFO_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_gir_rcv_process: ERR 5105: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_gir_rcv_process: ERR 5104: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_guidinfo_record(p_rcv->p_log, p_rcvd_rec, + OSM_LOG_DEBUG); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + cl_qmap_apply_func(&p_rcv->p_subn->node_guid_tbl, + __osm_sa_gir_by_comp_mask_cb, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_gir_rcv_process: ERR 5103: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_gir_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_gir_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_guidinfo_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_gir_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_guidinfo_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_gir_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_gir_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_guidinfo_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_gir_rcv_process: ERR 5106: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_gir_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_guidinfo_record_t) ); - - p_resp_rec = (ib_guidinfo_record_t*) - ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_gir_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_guidinfo_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_gir_rcv_process: ERR 5106: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_guidinfo_record_t)); + + p_resp_rec = (ib_guidinfo_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_gir_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_gir_rcv_process: ERR 5107: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = (osm_gir_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_gir_rcv_process: ERR 5107: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_informinfo.c b/opensm/opensm/osm_sa_informinfo.c index d08b6e5..4639365 100644 --- a/opensm/opensm/osm_sa_informinfo.c +++ b/opensm/opensm/osm_sa_informinfo.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -69,78 +69,69 @@ #define OSM_IIR_RCV_POOL_MIN_SIZE 32 #define OSM_IIR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_iir_item -{ - cl_pool_item_t pool_item; - ib_inform_info_record_t rec; +typedef struct _osm_iir_item { + cl_pool_item_t pool_item; + ib_inform_info_record_t rec; } osm_iir_item_t; -typedef struct _osm_iir_search_ctxt -{ - const ib_inform_info_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - ib_gid_t subscriber_gid; - ib_net16_t subscriber_enum; - osm_infr_rcv_t* p_rcv; - osm_physp_t* p_req_physp; +typedef struct _osm_iir_search_ctxt { + const ib_inform_info_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + ib_gid_t subscriber_gid; + ib_net16_t subscriber_enum; + osm_infr_rcv_t *p_rcv; + osm_physp_t *p_req_physp; } osm_iir_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_infr_rcv_construct( - IN osm_infr_rcv_t* const p_rcv ) +void osm_infr_rcv_construct(IN osm_infr_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_infr_rcv_destroy( - IN osm_infr_rcv_t* const p_rcv ) +void osm_infr_rcv_destroy(IN osm_infr_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_infr_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_infr_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_infr_rcv_init( - IN osm_infr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_infr_rcv_init(IN osm_infr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_ERROR; + ib_api_status_t status = IB_ERROR; - OSM_LOG_ENTER( p_log, osm_infr_rcv_init ); + OSM_LOG_ENTER(p_log, osm_infr_rcv_init); - osm_infr_rcv_construct( p_rcv ); + osm_infr_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_IIR_RCV_POOL_MIN_SIZE, - 0, - OSM_IIR_RCV_POOL_GROW_SIZE, - sizeof(osm_iir_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_IIR_RCV_POOL_MIN_SIZE, + 0, + OSM_IIR_RCV_POOL_GROW_SIZE, + sizeof(osm_iir_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** @@ -151,139 +142,132 @@ originating the Set(InformInfo) and a Trap() source identified by Inform- can access each other - can use path record to verify that. **********************************************************************/ static boolean_t -__validate_ports_access_rights( - IN osm_infr_rcv_t* const p_rcv, - IN osm_infr_t* p_infr_rec ) +__validate_ports_access_rights(IN osm_infr_rcv_t * const p_rcv, + IN osm_infr_t * p_infr_rec) { - boolean_t valid = TRUE; - osm_physp_t* p_requester_physp; - osm_port_t* p_port; - osm_physp_t* p_physp; - ib_net64_t portguid; - ib_net16_t lid_range_begin; - ib_net16_t lid_range_end; - ib_net16_t lid; - const cl_ptr_vector_t* p_tbl; - ib_gid_t zero_gid; - - OSM_LOG_ENTER( p_rcv->p_log, __validate_ports_access_rights ); - - /* get the requester physp from the request address */ - p_requester_physp = osm_get_physp_by_mad_addr( p_rcv->p_log, - p_rcv->p_subn, - &p_infr_rec->report_addr ); - - memset( &zero_gid, 0, sizeof(zero_gid) ); - if ( memcmp (&(p_infr_rec->inform_record.inform_info.gid), - &zero_gid, sizeof(ib_gid_t) ) ) - { - /* a gid is defined */ - portguid = p_infr_rec->inform_record.inform_info.gid.unicast.interface_id; - - p_port = osm_get_port_by_guid( p_rcv->p_subn, portguid ); - - if ( p_port == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__validate_ports_access_rights: ERR 4301: " - "Invalid port guid: 0x%016" PRIx64 "\n", - cl_ntoh64(portguid) ); - valid = FALSE; - goto Exit; - } - - /* get the destination InformInfo physical port */ - p_physp = p_port->p_physp; - - /* make sure that the requester and destination port can access each other - according to the current partitioning. */ - if (! osm_physp_share_pkey( p_rcv->p_log, p_physp, p_requester_physp)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_ports_access_rights: " - "port and requester don't share pkey\n" ); - valid = FALSE; - goto Exit; - } - } - else - { - /* gid is zero - check if LID range is defined */ - lid_range_begin = cl_ntoh16(p_infr_rec->inform_record.inform_info.lid_range_begin); - /* if lid is 0xFFFF - meaning all endports managed by the manager */ - if ( lid_range_begin == 0xFFFF ) - goto Exit; - - lid_range_end = cl_ntoh16(p_infr_rec->inform_record.inform_info.lid_range_end); - - /* lid_range_end is set to zero if no range desired. In this case - - just make it equal to the lid_range_begin. */ - if (lid_range_end == 0) - lid_range_end = lid_range_begin; - - /* go over all defined lids within the range and make sure that the - requester port can access them according to current partitioning. */ - for ( lid = lid_range_begin; lid <= lid_range_end; lid++ ) - { - p_tbl = &p_rcv->p_subn->port_lid_tbl; - if ( cl_ptr_vector_get_size( p_tbl ) > lid ) - { - p_port = cl_ptr_vector_get( p_tbl, lid ); - } - else - { - /* lid requested is out of range */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__validate_ports_access_rights: ERR 4302: " - "Given LID (0x%X) is out of range:0x%X\n", - lid, cl_ptr_vector_get_size(p_tbl) ); - valid = FALSE; - goto Exit; - } - if ( p_port == NULL ) - continue; - - p_physp = p_port->p_physp; - /* make sure that the requester and destination port can access - each other according to the current partitioning. */ - if (! osm_physp_share_pkey( p_rcv->p_log, p_physp, p_requester_physp)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_ports_access_rights: " - "port and requester don't share pkey\n" ); - valid = FALSE; - goto Exit; - } - } - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return valid; + boolean_t valid = TRUE; + osm_physp_t *p_requester_physp; + osm_port_t *p_port; + osm_physp_t *p_physp; + ib_net64_t portguid; + ib_net16_t lid_range_begin; + ib_net16_t lid_range_end; + ib_net16_t lid; + const cl_ptr_vector_t *p_tbl; + ib_gid_t zero_gid; + + OSM_LOG_ENTER(p_rcv->p_log, __validate_ports_access_rights); + + /* get the requester physp from the request address */ + p_requester_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + &p_infr_rec->report_addr); + + memset(&zero_gid, 0, sizeof(zero_gid)); + if (memcmp(&(p_infr_rec->inform_record.inform_info.gid), + &zero_gid, sizeof(ib_gid_t))) { + /* a gid is defined */ + portguid = + p_infr_rec->inform_record.inform_info.gid.unicast. + interface_id; + + p_port = osm_get_port_by_guid(p_rcv->p_subn, portguid); + + if (p_port == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__validate_ports_access_rights: ERR 4301: " + "Invalid port guid: 0x%016" PRIx64 "\n", + cl_ntoh64(portguid)); + valid = FALSE; + goto Exit; + } + + /* get the destination InformInfo physical port */ + p_physp = p_port->p_physp; + + /* make sure that the requester and destination port can access each other + according to the current partitioning. */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_physp, p_requester_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_ports_access_rights: " + "port and requester don't share pkey\n"); + valid = FALSE; + goto Exit; + } + } else { + /* gid is zero - check if LID range is defined */ + lid_range_begin = + cl_ntoh16(p_infr_rec->inform_record.inform_info. + lid_range_begin); + /* if lid is 0xFFFF - meaning all endports managed by the manager */ + if (lid_range_begin == 0xFFFF) + goto Exit; + + lid_range_end = + cl_ntoh16(p_infr_rec->inform_record.inform_info. + lid_range_end); + + /* lid_range_end is set to zero if no range desired. In this case - + just make it equal to the lid_range_begin. */ + if (lid_range_end == 0) + lid_range_end = lid_range_begin; + + /* go over all defined lids within the range and make sure that the + requester port can access them according to current partitioning. */ + for (lid = lid_range_begin; lid <= lid_range_end; lid++) { + p_tbl = &p_rcv->p_subn->port_lid_tbl; + if (cl_ptr_vector_get_size(p_tbl) > lid) { + p_port = cl_ptr_vector_get(p_tbl, lid); + } else { + /* lid requested is out of range */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__validate_ports_access_rights: ERR 4302: " + "Given LID (0x%X) is out of range:0x%X\n", + lid, cl_ptr_vector_get_size(p_tbl)); + valid = FALSE; + goto Exit; + } + if (p_port == NULL) + continue; + + p_physp = p_port->p_physp; + /* make sure that the requester and destination port can access + each other according to the current partitioning. */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_physp, p_requester_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_ports_access_rights: " + "port and requester don't share pkey\n"); + valid = FALSE; + goto Exit; + } + } + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return valid; } /********************************************************************** **********************************************************************/ static boolean_t -__validate_infr( - IN osm_infr_rcv_t* const p_rcv, - IN osm_infr_t* p_infr_rec ) +__validate_infr(IN osm_infr_rcv_t * const p_rcv, IN osm_infr_t * p_infr_rec) { - boolean_t valid = TRUE; + boolean_t valid = TRUE; - OSM_LOG_ENTER( p_rcv->p_log, __validate_infr ); + OSM_LOG_ENTER(p_rcv->p_log, __validate_infr); - valid = __validate_ports_access_rights( p_rcv, p_infr_rec ); - if (!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_infr: " - "Invalid Access for InformInfo\n" ); - valid = FALSE; - } + valid = __validate_ports_access_rights(p_rcv, p_infr_rec); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_infr: " "Invalid Access for InformInfo\n"); + valid = FALSE; + } - OSM_LOG_EXIT( p_rcv->p_log ); - return valid; + OSM_LOG_EXIT(p_rcv->p_log); + return valid; } /********************************************************************** @@ -292,633 +276,614 @@ with an InformInfo attribute that is a copy of the data in the Set(InformInfo) request. **********************************************************************/ static void -__osm_infr_rcv_respond( - IN osm_infr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__osm_infr_rcv_respond(IN osm_infr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - osm_madw_t* p_resp_madw; - const ib_sa_mad_t* p_sa_mad; - ib_sa_mad_t* p_resp_sa_mad; - ib_inform_info_t* p_resp_infr; - ib_api_status_t status; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_infr_rcv_respond ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_infr_rcv_respond: " - "Generating successful InformInfo response\n"); - } - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - MAD_BLOCK_SIZE, - &p_madw->mad_addr ); - if ( !p_resp_madw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_infr_rcv_respond: ERR 4303: " - "Unable to allocate MAD\n" ); - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* copy the request InformInfo */ - memcpy( p_resp_sa_mad, p_sa_mad, MAD_BLOCK_SIZE ); - p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - p_resp_infr = (ib_inform_info_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - - if ( status != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_infr_rcv_respond: ERR 4304: " - "Unable to send MAD (%s)\n", ib_get_err_str( status ) ); - /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_madw_t *p_resp_madw; + const ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_resp_sa_mad; + ib_inform_info_t *p_resp_infr; + ib_api_status_t status; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_infr_rcv_respond); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_infr_rcv_respond: " + "Generating successful InformInfo response\n"); + } + + /* + Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + MAD_BLOCK_SIZE, &p_madw->mad_addr); + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_infr_rcv_respond: ERR 4303: " + "Unable to allocate MAD\n"); + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* copy the request InformInfo */ + memcpy(p_resp_sa_mad, p_sa_mad, MAD_BLOCK_SIZE); + p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + p_resp_infr = + (ib_inform_info_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_infr_rcv_respond: ERR 4304: " + "Unable to send MAD (%s)\n", ib_get_err_str(status)); + /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_inform_info_rec_by_comp_mask( - IN osm_infr_rcv_t* const p_rcv, - IN const osm_infr_t* const p_infr, - osm_iir_search_ctxt_t* const p_ctxt ) +__osm_sa_inform_info_rec_by_comp_mask(IN osm_infr_rcv_t * const p_rcv, + IN const osm_infr_t * const p_infr, + osm_iir_search_ctxt_t * const p_ctxt) { - const ib_inform_info_record_t* p_rcvd_rec = NULL; - ib_net64_t comp_mask; - ib_net64_t portguid; - osm_port_t * p_subscriber_port; - osm_physp_t * p_subscriber_physp; - const osm_physp_t* p_req_physp; - osm_iir_item_t* p_rec_item; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_inform_info_rec_by_comp_mask ); - - p_rcvd_rec = p_ctxt->p_rcvd_rec; - comp_mask = p_ctxt->comp_mask; - p_req_physp = p_ctxt->p_req_physp; - - if (comp_mask & IB_IIR_COMPMASK_SUBSCRIBERGID) - { - if (memcmp(&p_infr->inform_record.subscriber_gid, - &p_ctxt->subscriber_gid, - sizeof(p_infr->inform_record.subscriber_gid))) - goto Exit; - } - - if (comp_mask & IB_IIR_COMPMASK_ENUM) - { - if (p_infr->inform_record.subscriber_enum != p_ctxt->subscriber_enum) - goto Exit; - } - - /* Implement any other needed search cases */ - - /* Ensure pkey is shared before returning any records */ - portguid = p_infr->inform_record.subscriber_gid.unicast.interface_id; - p_subscriber_port = osm_get_port_by_guid( p_rcv->p_subn, portguid ); - if ( p_subscriber_port == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_inform_info_rec_by_comp_mask: ERR 430D: " - "Invalid subscriber port guid: 0x%016" PRIx64 "\n", - cl_ntoh64(portguid) ); - goto Exit; - } - - /* get the subscriber InformInfo physical port */ - p_subscriber_physp = p_subscriber_port->p_physp; - /* make sure that the requester and subscriber port can access each other - according to the current partitioning. */ - if (! osm_physp_share_pkey( p_rcv->p_log, p_req_physp, p_subscriber_physp )) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_inform_info_rec_by_comp_mask: " - "requester and subscriber ports don't share pkey\n" ); - goto Exit; - } - - p_rec_item = (osm_iir_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_inform_info_rec_by_comp_mask: ERR 430E: " - "cl_qlock_pool_get failed\n" ); - goto Exit; - } - - memcpy((void *)&p_rec_item->rec, (void *)&p_infr->inform_record, sizeof(ib_inform_info_record_t)); - cl_qlist_insert_tail( p_ctxt->p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - -Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_inform_info_record_t *p_rcvd_rec = NULL; + ib_net64_t comp_mask; + ib_net64_t portguid; + osm_port_t *p_subscriber_port; + osm_physp_t *p_subscriber_physp; + const osm_physp_t *p_req_physp; + osm_iir_item_t *p_rec_item; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_inform_info_rec_by_comp_mask); + + p_rcvd_rec = p_ctxt->p_rcvd_rec; + comp_mask = p_ctxt->comp_mask; + p_req_physp = p_ctxt->p_req_physp; + + if (comp_mask & IB_IIR_COMPMASK_SUBSCRIBERGID) { + if (memcmp(&p_infr->inform_record.subscriber_gid, + &p_ctxt->subscriber_gid, + sizeof(p_infr->inform_record.subscriber_gid))) + goto Exit; + } + + if (comp_mask & IB_IIR_COMPMASK_ENUM) { + if (p_infr->inform_record.subscriber_enum != + p_ctxt->subscriber_enum) + goto Exit; + } + + /* Implement any other needed search cases */ + + /* Ensure pkey is shared before returning any records */ + portguid = p_infr->inform_record.subscriber_gid.unicast.interface_id; + p_subscriber_port = osm_get_port_by_guid(p_rcv->p_subn, portguid); + if (p_subscriber_port == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_inform_info_rec_by_comp_mask: ERR 430D: " + "Invalid subscriber port guid: 0x%016" PRIx64 "\n", + cl_ntoh64(portguid)); + goto Exit; + } + + /* get the subscriber InformInfo physical port */ + p_subscriber_physp = p_subscriber_port->p_physp; + /* make sure that the requester and subscriber port can access each other + according to the current partitioning. */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_req_physp, p_subscriber_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_inform_info_rec_by_comp_mask: " + "requester and subscriber ports don't share pkey\n"); + goto Exit; + } + + p_rec_item = (osm_iir_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_inform_info_rec_by_comp_mask: ERR 430E: " + "cl_qlock_pool_get failed\n"); + goto Exit; + } + + memcpy((void *)&p_rec_item->rec, (void *)&p_infr->inform_record, + sizeof(ib_inform_info_record_t)); + cl_qlist_insert_tail(p_ctxt->p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_inform_info_rec_by_comp_mask_cb( - IN cl_list_item_t* const p_list_item, - IN void* context ) +__osm_sa_inform_info_rec_by_comp_mask_cb(IN cl_list_item_t * const p_list_item, + IN void *context) { - const osm_infr_t* const p_infr = (osm_infr_t *)p_list_item; - osm_iir_search_ctxt_t* const p_ctxt = (osm_iir_search_ctxt_t *)context; + const osm_infr_t *const p_infr = (osm_infr_t *) p_list_item; + osm_iir_search_ctxt_t *const p_ctxt = (osm_iir_search_ctxt_t *) context; - __osm_sa_inform_info_rec_by_comp_mask( p_ctxt->p_rcv, p_infr, p_ctxt ); + __osm_sa_inform_info_rec_by_comp_mask(p_ctxt->p_rcv, p_infr, p_ctxt); } /********************************************************************** Received a Get(InformInfoRecord) or GetTable(InformInfoRecord) MAD **********************************************************************/ static void -osm_infr_rcv_process_get_method( - IN osm_infr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +osm_infr_rcv_process_get_method(IN osm_infr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - ib_sa_mad_t* p_rcvd_mad; - const ib_inform_info_record_t* p_rcvd_rec; - ib_inform_info_record_t* p_resp_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; + ib_sa_mad_t *p_rcvd_mad; + const ib_inform_info_record_t *p_rcvd_rec; + ib_inform_info_record_t *p_resp_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i, j; - osm_iir_search_ctxt_t context; - osm_iir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - osm_physp_t* p_req_physp; - - OSM_LOG_ENTER( p_rcv->p_log, osm_infr_rcv_process_get_method ); - - CL_ASSERT( p_madw ); - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = - (ib_inform_info_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_get_method: ERR 4309: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_inform_info_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.subscriber_gid = p_rcvd_rec->subscriber_gid; - context.subscriber_enum = p_rcvd_rec->subscriber_enum; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infr_rcv_process_get_method: " - "Query Subscriber GID:0x%016" PRIx64 " : 0x%016" PRIx64 "(%02X) Enum:0x%X(%02X)\n", - cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.prefix), - cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.interface_id), - (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_SUBSCRIBERGID) != 0, - cl_ntoh16(p_rcvd_rec->subscriber_enum), - (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_ENUM) != 0 ); - - cl_plock_acquire( p_rcv->p_lock ); - - cl_qlist_apply_func( &p_rcv->p_subn->sa_infr_list, - __osm_sa_inform_info_rec_by_comp_mask_cb, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_get_method: ERR 430A: " - "More than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_iir_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_iir_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_iir_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i, j; + osm_iir_search_ctxt_t context; + osm_iir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + osm_physp_t *p_req_physp; + + OSM_LOG_ENTER(p_rcv->p_log, osm_infr_rcv_process_get_method); + + CL_ASSERT(p_madw); + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_inform_info_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_get_method: ERR 4309: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_inform_info_record(p_rcv->p_log, p_rcvd_rec, + OSM_LOG_DEBUG); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.subscriber_gid = p_rcvd_rec->subscriber_gid; + context.subscriber_enum = p_rcvd_rec->subscriber_enum; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infr_rcv_process_get_method: " + "Query Subscriber GID:0x%016" PRIx64 " : 0x%016" PRIx64 + "(%02X) Enum:0x%X(%02X)\n", + cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.prefix), + cl_ntoh64(p_rcvd_rec->subscriber_gid.unicast.interface_id), + (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_SUBSCRIBERGID) != 0, + cl_ntoh16(p_rcvd_rec->subscriber_enum), + (p_rcvd_mad->comp_mask & IB_IIR_COMPMASK_ENUM) != 0); + + cl_plock_acquire(p_rcv->p_lock); + + cl_qlist_apply_func(&p_rcv->p_subn->sa_infr_list, + __osm_sa_inform_info_rec_by_comp_mask_cb, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_get_method: ERR 430A: " + "More than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_iir_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_iir_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_inform_info_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_infr_rcv_process_get_method: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + /* we limit the number of records to a single packet */ + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_inform_info_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_infr_rcv_process_get_method: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infr_rcv_process_get_method: " - "Returning %u records\n", num_rec ); - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_inform_info_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_get_method: ERR 430B: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_iir_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_inform_info_record_t) ); - - p_resp_rec = (ib_inform_info_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infr_rcv_process_get_method: " + "Returning %u records\n", num_rec); + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * + sizeof(ib_inform_info_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_get_method: ERR 430B: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_inform_info_record_t)); + + p_resp_rec = + (ib_inform_info_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_iir_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - /* clear reserved and pad fields in InformInfoRecord */ - for (j = 0; j < 6; j++) - p_resp_rec->reserved[j] = 0; - for (j = 0; j < 4; j++) - p_resp_rec->pad[j] = 0; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_get_method: ERR 430C: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = (osm_iir_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + /* clear reserved and pad fields in InformInfoRecord */ + for (j = 0; j < 6; j++) + p_resp_rec->reserved[j] = 0; + for (j = 0; j < 4; j++) + p_resp_rec->pad[j] = 0; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_get_method: ERR 430C: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************* Received a Set(InformInfo) MAD **********************************************************************/ static void -osm_infr_rcv_process_set_method( - IN osm_infr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +osm_infr_rcv_process_set_method(IN osm_infr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - ib_sa_mad_t *p_sa_mad; - ib_inform_info_t *p_recvd_inform_info; - osm_infr_t inform_info_rec; /* actual inform record to be stored for reports */ - osm_infr_t *p_infr; - ib_net32_t qpn; - uint8_t resp_time_val; - ib_api_status_t res; + ib_sa_mad_t *p_sa_mad; + ib_inform_info_t *p_recvd_inform_info; + osm_infr_t inform_info_rec; /* actual inform record to be stored for reports */ + osm_infr_t *p_infr; + ib_net32_t qpn; + uint8_t resp_time_val; + ib_api_status_t res; - OSM_LOG_ENTER( p_rcv->p_log, osm_infr_rcv_process_set_method ); + OSM_LOG_ENTER(p_rcv->p_log, osm_infr_rcv_process_set_method); - CL_ASSERT( p_madw ); + CL_ASSERT(p_madw); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_inform_info = - (ib_inform_info_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_inform_info = + (ib_inform_info_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); #if 0 - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_inform_info( p_rcv->p_log, p_recvd_inform_info, OSM_LOG_DEBUG ); + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_inform_info(p_rcv->p_log, p_recvd_inform_info, + OSM_LOG_DEBUG); #endif - /* Grab the lock */ - cl_plock_excl_acquire( p_rcv->p_lock ); - - /* define the inform record */ - inform_info_rec.inform_record.inform_info = *p_recvd_inform_info; - - /* following C13-32.1.2 Tbl 120: we only copy the source address vector */ - inform_info_rec.report_addr = p_madw->mad_addr; - - /* we will need to know the mad srvc to send back through */ - inform_info_rec.h_bind = p_madw->h_bind; - inform_info_rec.p_infr_rcv = p_rcv; - - /* update the subscriber GID according to mad address */ - res = osm_get_gid_by_mad_addr( - p_rcv->p_log, - p_rcv->p_subn, - &p_madw->mad_addr, - &inform_info_rec.inform_record.subscriber_gid ); - if ( res != IB_SUCCESS ) - { - cl_plock_release( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_set_method: ERR 4308 " - "Subscribe Request from unknown LID: 0x%04X\n", - cl_ntoh16(p_madw->mad_addr.dest_lid) - ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID ); - goto Exit; - } - - /* HACK: enum is always 0 (currently) */ - inform_info_rec.inform_record.subscriber_enum = 0; - - /* Subscribe values above 1 are undefined */ - if ( p_recvd_inform_info->subscribe > 1 ) - { - cl_plock_release( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_set_method: ERR 4308 " - "Invalid subscribe: %d\n", - p_recvd_inform_info->subscribe - ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID ); - goto Exit; - } - - /* - * MODIFICATIONS DONE ON INCOMING REQUEST: - * - * QPN: - * Internally we keep the QPN field of the InformInfo updated - * so we can simply compare it in the record - when finding such. - */ - if ( p_recvd_inform_info->subscribe ) - { - ib_inform_info_set_qpn( - &inform_info_rec.inform_record.inform_info, - inform_info_rec.report_addr.addr_type.gsi.remote_qp ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infr_rcv_process_set_method: " - "Subscribe Request with QPN: 0x%06X\n", - cl_ntoh32(inform_info_rec.report_addr.addr_type.gsi.remote_qp) - ); - } - else - { - ib_inform_info_get_qpn_resp_time( - p_recvd_inform_info->g_or_v.generic.qpn_resp_time_val, - &qpn, &resp_time_val ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infr_rcv_process_set_method: " - "UnSubscribe Request with QPN: 0x%06X\n", - cl_ntoh32(qpn) - ); - } - - /* If record exists with matching InformInfo */ - p_infr = osm_infr_get_by_rec( p_rcv->p_subn, p_rcv->p_log, &inform_info_rec ); - - /* check to see if the request was for subscribe */ - if ( p_recvd_inform_info->subscribe ) - { - /* validate the request for a new or update InformInfo */ - if ( __validate_infr( p_rcv, &inform_info_rec ) != TRUE ) - { - cl_plock_release( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_set_method: ERR 4305: " - "Failed to validate a new inform object\n"); - - /* o13-13.1.1: we need to set the subscribe bit to 0 */ - p_recvd_inform_info->subscribe = 0; - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID ); - goto Exit; - } - - /* ok - we can try and create a new entry */ - if (p_infr == NULL) - { - /* Create the instance of the osm_infr_t object */ - p_infr = osm_infr_new( &inform_info_rec ); - if (p_infr == NULL) - { - cl_plock_release( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_set_method: ERR 4306: " - "Failed to create a new inform object\n"); - - /* o13-13.1.1: we need to set the subscribe bit to 0 */ - p_recvd_inform_info->subscribe = 0; - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - /* Add this new osm_infr_t object to subnet object */ - osm_infr_insert_to_db( p_rcv->p_subn, p_rcv->p_log, p_infr ); - } - else - { - /* Update the old instance of the osm_infr_t object */ - p_infr->inform_record = inform_info_rec.inform_record; - } - } - else - { - /* We got an UnSubscribe request */ - if (p_infr == NULL) - { - cl_plock_release( p_rcv->p_lock ); - - /* No Such Item - So Error */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_infr_rcv_process_set_method: ERR 4307: " - "Failed to UnSubscribe to non existing inform object\n"); - - /* o13-13.1.1: we need to set the subscribe bit to 0 */ - p_recvd_inform_info->subscribe = 0; - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID ); - goto Exit; - } - else - { - /* Delete this object from the subnet list of informs */ - osm_infr_remove_from_db( p_rcv->p_subn, p_rcv->p_log, p_infr ); - } - } - - cl_plock_release( p_rcv->p_lock ); - - /* send the success response */ - __osm_infr_rcv_respond( p_rcv, p_madw ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + /* Grab the lock */ + cl_plock_excl_acquire(p_rcv->p_lock); + + /* define the inform record */ + inform_info_rec.inform_record.inform_info = *p_recvd_inform_info; + + /* following C13-32.1.2 Tbl 120: we only copy the source address vector */ + inform_info_rec.report_addr = p_madw->mad_addr; + + /* we will need to know the mad srvc to send back through */ + inform_info_rec.h_bind = p_madw->h_bind; + inform_info_rec.p_infr_rcv = p_rcv; + + /* update the subscriber GID according to mad address */ + res = osm_get_gid_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + &p_madw->mad_addr, + &inform_info_rec.inform_record. + subscriber_gid); + if (res != IB_SUCCESS) { + cl_plock_release(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_set_method: ERR 4308 " + "Subscribe Request from unknown LID: 0x%04X\n", + cl_ntoh16(p_madw->mad_addr.dest_lid) + ); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } + + /* HACK: enum is always 0 (currently) */ + inform_info_rec.inform_record.subscriber_enum = 0; + + /* Subscribe values above 1 are undefined */ + if (p_recvd_inform_info->subscribe > 1) { + cl_plock_release(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_set_method: ERR 4308 " + "Invalid subscribe: %d\n", + p_recvd_inform_info->subscribe); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } + + /* + * MODIFICATIONS DONE ON INCOMING REQUEST: + * + * QPN: + * Internally we keep the QPN field of the InformInfo updated + * so we can simply compare it in the record - when finding such. + */ + if (p_recvd_inform_info->subscribe) { + ib_inform_info_set_qpn(&inform_info_rec.inform_record. + inform_info, + inform_info_rec.report_addr.addr_type. + gsi.remote_qp); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infr_rcv_process_set_method: " + "Subscribe Request with QPN: 0x%06X\n", + cl_ntoh32(inform_info_rec.report_addr.addr_type.gsi. + remote_qp) + ); + } else { + ib_inform_info_get_qpn_resp_time(p_recvd_inform_info->g_or_v. + generic.qpn_resp_time_val, + &qpn, &resp_time_val); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infr_rcv_process_set_method: " + "UnSubscribe Request with QPN: 0x%06X\n", cl_ntoh32(qpn) + ); + } + + /* If record exists with matching InformInfo */ + p_infr = + osm_infr_get_by_rec(p_rcv->p_subn, p_rcv->p_log, &inform_info_rec); + + /* check to see if the request was for subscribe */ + if (p_recvd_inform_info->subscribe) { + /* validate the request for a new or update InformInfo */ + if (__validate_infr(p_rcv, &inform_info_rec) != TRUE) { + cl_plock_release(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_set_method: ERR 4305: " + "Failed to validate a new inform object\n"); + + /* o13-13.1.1: we need to set the subscribe bit to 0 */ + p_recvd_inform_info->subscribe = 0; + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } + + /* ok - we can try and create a new entry */ + if (p_infr == NULL) { + /* Create the instance of the osm_infr_t object */ + p_infr = osm_infr_new(&inform_info_rec); + if (p_infr == NULL) { + cl_plock_release(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_set_method: ERR 4306: " + "Failed to create a new inform object\n"); + + /* o13-13.1.1: we need to set the subscribe bit to 0 */ + p_recvd_inform_info->subscribe = 0; + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + /* Add this new osm_infr_t object to subnet object */ + osm_infr_insert_to_db(p_rcv->p_subn, p_rcv->p_log, + p_infr); + } else { + /* Update the old instance of the osm_infr_t object */ + p_infr->inform_record = inform_info_rec.inform_record; + } + } else { + /* We got an UnSubscribe request */ + if (p_infr == NULL) { + cl_plock_release(p_rcv->p_lock); + + /* No Such Item - So Error */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_infr_rcv_process_set_method: ERR 4307: " + "Failed to UnSubscribe to non existing inform object\n"); + + /* o13-13.1.1: we need to set the subscribe bit to 0 */ + p_recvd_inform_info->subscribe = 0; + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } else { + /* Delete this object from the subnet list of informs */ + osm_infr_remove_from_db(p_rcv->p_subn, p_rcv->p_log, + p_infr); + } + } + + cl_plock_release(p_rcv->p_lock); + + /* send the success response */ + __osm_infr_rcv_respond(p_rcv, p_madw); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************* **********************************************************************/ -void -osm_infr_rcv_process( - IN void *context, - IN void *data ) +void osm_infr_rcv_process(IN void *context, IN void *data) { - osm_infr_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_sa_mad_t *p_sa_mad; + osm_infr_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_sa_mad_t *p_sa_mad; - OSM_LOG_ENTER( p_rcv->p_log, osm_infr_rcv_process ); + OSM_LOG_ENTER(p_rcv->p_log, osm_infr_rcv_process); - CL_ASSERT( p_madw ); + CL_ASSERT(p_madw); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - CL_ASSERT( p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO ); + CL_ASSERT(p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO); - if (p_sa_mad->method != IB_MAD_METHOD_SET) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infr_rcv_process: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_sa_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } + if (p_sa_mad->method != IB_MAD_METHOD_SET) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infr_rcv_process: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_sa_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } - osm_infr_rcv_process_set_method( p_rcv, p_madw ); + osm_infr_rcv_process_set_method(p_rcv, p_madw); - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************* **********************************************************************/ -void -osm_infir_rcv_process( - IN void *context, - IN void *data ) +void osm_infir_rcv_process(IN void *context, IN void *data) { - osm_infr_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_sa_mad_t *p_sa_mad; + osm_infr_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_sa_mad_t *p_sa_mad; - OSM_LOG_ENTER( p_rcv->p_log, osm_infr_rcv_process ); + OSM_LOG_ENTER(p_rcv->p_log, osm_infr_rcv_process); - CL_ASSERT( p_madw ); + CL_ASSERT(p_madw); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); - CL_ASSERT( p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO_RECORD ); + CL_ASSERT(p_sa_mad->attr_id == IB_MAD_ATTR_INFORM_INFO_RECORD); - if ( (p_sa_mad->method != IB_MAD_METHOD_GET) && - (p_sa_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_infir_rcv_process: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_sa_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } + if ((p_sa_mad->method != IB_MAD_METHOD_GET) && + (p_sa_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_infir_rcv_process: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_sa_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } - osm_infr_rcv_process_get_method( p_rcv, p_madw ); + osm_infr_rcv_process_get_method(p_rcv, p_madw); - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_lft_record.c b/opensm/opensm/osm_sa_lft_record.c index c5be44b..057d29b 100644 --- a/opensm/opensm/osm_sa_lft_record.c +++ b/opensm/opensm/osm_sa_lft_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -62,455 +62,427 @@ #define OSM_LFTR_RCV_POOL_MIN_SIZE 32 #define OSM_LFTR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_lftr_item -{ - cl_pool_item_t pool_item; - ib_lft_record_t rec; +typedef struct _osm_lftr_item { + cl_pool_item_t pool_item; + ib_lft_record_t rec; } osm_lftr_item_t; -typedef struct _osm_lftr_search_ctxt -{ - const ib_lft_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_lftr_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_lftr_search_ctxt { + const ib_lft_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_lftr_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_lftr_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_lftr_rcv_construct( - IN osm_lftr_rcv_t* const p_rcv ) +void osm_lftr_rcv_construct(IN osm_lftr_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_lftr_rcv_destroy( - IN osm_lftr_rcv_t* const p_rcv ) +void osm_lftr_rcv_destroy(IN osm_lftr_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_lftr_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_lftr_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_lftr_rcv_init( - IN osm_lftr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_lftr_rcv_init(IN osm_lftr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_lftr_rcv_init ); + OSM_LOG_ENTER(p_log, osm_lftr_rcv_init); - osm_lftr_rcv_construct( p_rcv ); + osm_lftr_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_LFTR_RCV_POOL_MIN_SIZE, - 0, - OSM_LFTR_RCV_POOL_GROW_SIZE, - sizeof(osm_lftr_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_LFTR_RCV_POOL_MIN_SIZE, + 0, + OSM_LFTR_RCV_POOL_GROW_SIZE, + sizeof(osm_lftr_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_lftr_rcv_new_lftr( - IN osm_lftr_rcv_t* const p_rcv, - IN const osm_switch_t* const p_sw, - IN cl_qlist_t* const p_list, - IN ib_net16_t const lid, - IN ib_net16_t const block ) +__osm_lftr_rcv_new_lftr(IN osm_lftr_rcv_t * const p_rcv, + IN const osm_switch_t * const p_sw, + IN cl_qlist_t * const p_list, + IN ib_net16_t const lid, IN ib_net16_t const block) { - osm_lftr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_lftr_rcv_new_lftr ); - - p_rec_item = (osm_lftr_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_lftr_rcv_new_lftr: ERR 4402: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_lftr_rcv_new_lftr: " - "New LinearForwardingTable: sw 0x%016" PRIx64 - "\n\t\t\t\tblock 0x%02X lid 0x%02X\n", - cl_ntoh64( osm_node_get_node_guid( p_sw->p_node ) ), - cl_ntoh16( block ), cl_ntoh16( lid ) - ); - } - - memset( &p_rec_item->rec, 0, sizeof(ib_lft_record_t) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.block_num = block; - - /* copy the lft block */ - osm_switch_get_fwd_tbl_block( p_sw, cl_ntoh16(block), p_rec_item->rec.lft ); - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_lftr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_lftr_rcv_new_lftr); + + p_rec_item = (osm_lftr_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_lftr_rcv_new_lftr: ERR 4402: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_lftr_rcv_new_lftr: " + "New LinearForwardingTable: sw 0x%016" PRIx64 + "\n\t\t\t\tblock 0x%02X lid 0x%02X\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), + cl_ntoh16(block), cl_ntoh16(lid) + ); + } + + memset(&p_rec_item->rec, 0, sizeof(ib_lft_record_t)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.block_num = block; + + /* copy the lft block */ + osm_switch_get_fwd_tbl_block(p_sw, cl_ntoh16(block), + p_rec_item->rec.lft); + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -static osm_port_t* -__osm_lftr_get_port_by_guid( - IN osm_lftr_rcv_t* const p_rcv, - IN uint64_t port_guid ) +static osm_port_t *__osm_lftr_get_port_by_guid(IN osm_lftr_rcv_t * const p_rcv, + IN uint64_t port_guid) { - osm_port_t* p_port; - - CL_PLOCK_ACQUIRE(p_rcv->p_lock); - - p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); - if (!p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_lftr_get_port_by_guid ERR 4404: " - "Invalid port GUID 0x%016" PRIx64 "\n", - port_guid ); - p_port = NULL; - } - - CL_PLOCK_RELEASE(p_rcv->p_lock); - return p_port; + osm_port_t *p_port; + + CL_PLOCK_ACQUIRE(p_rcv->p_lock); + + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_lftr_get_port_by_guid ERR 4404: " + "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); + p_port = NULL; + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + return p_port; } /********************************************************************** **********************************************************************/ static void -__osm_lftr_rcv_by_comp_mask( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_lftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_lftr_search_ctxt_t* const p_ctxt = - (osm_lftr_search_ctxt_t *)context; - const osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - const ib_lft_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - osm_lftr_rcv_t* const p_rcv = p_ctxt->p_rcv; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - osm_port_t* p_port; - uint16_t min_lid_ho, max_lid_ho; - uint16_t min_block, max_block, block; - const osm_physp_t* p_physp; - - /* In switches, the port guid is the node guid. */ - p_port = - __osm_lftr_get_port_by_guid( p_rcv, p_sw->p_node->node_info.port_guid ); - if (! p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_lftr_rcv_by_comp_mask: ERR 4405: " - "Failed to find Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - return; - } - - /* check that the requester physp and the current physp are under - the same partition. */ - p_physp = p_port->p_physp; - if (! p_physp) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_lftr_rcv_by_comp_mask: ERR 4406: " - "Failed to find default physical Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - return; - } - if (! osm_physp_share_pkey( p_rcv->p_log, p_req_physp, p_physp )) - return; - - /* get the port 0 of the switch */ - osm_port_get_lid_range_ho( p_port, &min_lid_ho, &max_lid_ho ); - - /* compare the lids - if required */ - if( comp_mask & IB_LFTR_COMPMASK_LID ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_lftr_rcv_by_comp_mask: " - "Comparing lid:0x%02X to port lid range: 0x%02X .. 0x%02X\n", - cl_ntoh16( p_rcvd_rec->lid ), min_lid_ho, max_lid_ho - ); - /* ok we are ready for range check */ - if (min_lid_ho > cl_ntoh16(p_rcvd_rec->lid) || - max_lid_ho < cl_ntoh16(p_rcvd_rec->lid)) - return; - } - - /* now we need to decide which blocks to output */ - if( comp_mask & IB_LFTR_COMPMASK_BLOCK ) - { - max_block = min_block = cl_ntoh16(p_rcvd_rec->block_num); - } - else - { - /* use as many blocks as "in use" */ - min_block = 0; - max_block = osm_switch_get_max_block_id_in_use(p_sw); - } - - /* so we can add these blocks one by one ... */ - for (block = min_block; block <= max_block; block++) - __osm_lftr_rcv_new_lftr( p_rcv, p_sw, p_ctxt->p_list, - osm_port_get_base_lid(p_port), - cl_hton16(block) ); + const osm_lftr_search_ctxt_t *const p_ctxt = + (osm_lftr_search_ctxt_t *) context; + const osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + const ib_lft_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + osm_lftr_rcv_t *const p_rcv = p_ctxt->p_rcv; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + osm_port_t *p_port; + uint16_t min_lid_ho, max_lid_ho; + uint16_t min_block, max_block, block; + const osm_physp_t *p_physp; + + /* In switches, the port guid is the node guid. */ + p_port = + __osm_lftr_get_port_by_guid(p_rcv, + p_sw->p_node->node_info.port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_lftr_rcv_by_comp_mask: ERR 4405: " + "Failed to find Port by Node Guid:0x%016" PRIx64 + "\n", cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + return; + } + + /* check that the requester physp and the current physp are under + the same partition. */ + p_physp = p_port->p_physp; + if (!p_physp) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_lftr_rcv_by_comp_mask: ERR 4406: " + "Failed to find default physical Port by Node Guid:0x%016" + PRIx64 "\n", + cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + return; + } + if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) + return; + + /* get the port 0 of the switch */ + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); + + /* compare the lids - if required */ + if (comp_mask & IB_LFTR_COMPMASK_LID) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_lftr_rcv_by_comp_mask: " + "Comparing lid:0x%02X to port lid range: 0x%02X .. 0x%02X\n", + cl_ntoh16(p_rcvd_rec->lid), min_lid_ho, max_lid_ho); + /* ok we are ready for range check */ + if (min_lid_ho > cl_ntoh16(p_rcvd_rec->lid) || + max_lid_ho < cl_ntoh16(p_rcvd_rec->lid)) + return; + } + + /* now we need to decide which blocks to output */ + if (comp_mask & IB_LFTR_COMPMASK_BLOCK) { + max_block = min_block = cl_ntoh16(p_rcvd_rec->block_num); + } else { + /* use as many blocks as "in use" */ + min_block = 0; + max_block = osm_switch_get_max_block_id_in_use(p_sw); + } + + /* so we can add these blocks one by one ... */ + for (block = min_block; block <= max_block; block++) + __osm_lftr_rcv_new_lftr(p_rcv, p_sw, p_ctxt->p_list, + osm_port_get_base_lid(p_port), + cl_hton16(block)); } /********************************************************************** **********************************************************************/ -void -osm_lftr_rcv_process( - IN void *ctx, - IN void *data ) +void osm_lftr_rcv_process(IN void *ctx, IN void *data) { - osm_lftr_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_lft_record_t* p_rcvd_rec; - ib_lft_record_t* p_resp_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; + osm_lftr_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_lft_record_t *p_rcvd_rec; + ib_lft_record_t *p_resp_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_lftr_search_ctxt_t context; - osm_lftr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_lftr_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_lft_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_LFT_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_lftr_rcv_process: ERR 4408: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_lftr_rcv_process: ERR 4407: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - /* Go over all switches */ - cl_qmap_apply_func( &p_rcv->p_subn->sw_guid_tbl, - __osm_lftr_rcv_by_comp_mask, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_lftr_rcv_process: ERR 4409: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_lftr_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_lftr_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_lftr_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_lftr_search_ctxt_t context; + osm_lftr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_lftr_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_lft_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_LFT_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lftr_rcv_process: ERR 4408: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lftr_rcv_process: ERR 4407: " + "Cannot find requester physical port\n"); + goto Exit; + } + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + /* Go over all switches */ + cl_qmap_apply_func(&p_rcv->p_subn->sw_guid_tbl, + __osm_lftr_rcv_by_comp_mask, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lftr_rcv_process: ERR 4409: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_lftr_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_lftr_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_lft_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_lftr_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + /* we limit the number of records to a single packet */ + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_lft_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_lftr_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_lftr_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) && - (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_lft_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_lftr_rcv_process: ERR 4410: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_lftr_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_lft_record_t) ); - - p_resp_rec = (ib_lft_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_lftr_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_lft_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lftr_rcv_process: ERR 4410: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_lft_record_t)); + + p_resp_rec = + (ib_lft_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_lftr_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_lftr_rcv_process: ERR 4411: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_lftr_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_lftr_rcv_process: ERR 4411: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_mad_ctrl.c b/opensm/opensm/osm_sa_mad_ctrl.c index 7a8bc87..58378b6 100644 --- a/opensm/opensm/osm_sa_mad_ctrl.c +++ b/opensm/opensm/osm_sa_mad_ctrl.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -68,22 +68,21 @@ * SYNOPSIS */ static void -__osm_sa_mad_ctrl_disp_done_callback( - IN void* context, - IN void* p_data ) +__osm_sa_mad_ctrl_disp_done_callback(IN void *context, IN void *p_data) { - osm_sa_mad_ctrl_t* const p_ctrl = (osm_sa_mad_ctrl_t*)context; - osm_madw_t* const p_madw = (osm_madw_t*)p_data; + osm_sa_mad_ctrl_t *const p_ctrl = (osm_sa_mad_ctrl_t *) context; + osm_madw_t *const p_madw = (osm_madw_t *) p_data; - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sa_mad_ctrl_disp_done_callback ); + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sa_mad_ctrl_disp_done_callback); - CL_ASSERT( p_madw ); - /* - Return the MAD & wrapper to the pool. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - OSM_LOG_EXIT( p_ctrl->p_log ); + CL_ASSERT(p_madw); + /* + Return the MAD & wrapper to the pool. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + OSM_LOG_EXIT(p_ctrl->p_log); } + /************/ /****f* opensm: SA/__osm_sa_mad_ctrl_process @@ -96,185 +95,179 @@ __osm_sa_mad_ctrl_disp_done_callback( * SYNOPSIS */ static void -__osm_sa_mad_ctrl_process( - IN osm_sa_mad_ctrl_t* const p_ctrl, - IN osm_madw_t *p_madw ) +__osm_sa_mad_ctrl_process(IN osm_sa_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * p_madw) { - ib_sa_mad_t* p_sa_mad; - cl_status_t status; - cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; - uint64_t last_dispatched_msg_queue_time_msec; - uint32_t num_messages; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sa_mad_ctrl_process ); - - /* - If the dispatcher is showing us that it is overloaded - there is no point in placing the request in. We should instead provide - immediate response - IB_RESOURCE_BUSY - But how do we know? - The dispatcher reports back the number of outstanding messages and the - time the last message stayed in the queue. - HACK: Actually, we cannot send a mad from within the receive callback; - thus - we will just drop it. - */ - cl_disp_get_queue_status(p_ctrl->h_disp, - &num_messages, - &last_dispatched_msg_queue_time_msec); - if ((num_messages > 1) && - (p_ctrl->p_subn->opt.max_msg_fifo_timeout) && - (last_dispatched_msg_queue_time_msec > - p_ctrl->p_subn->opt.max_msg_fifo_timeout)) - { - osm_log( p_ctrl->p_log, OSM_LOG_INFO, - "__osm_sa_mad_ctrl_process: " - /* "Responding BUSY status since the dispatcher is already"*/ - "Dropping MAD since the dispatcher is already" - " overloaded with %u messages and queue time of:" - "%" PRIu64 "[msec]\n", - num_messages, last_dispatched_msg_queue_time_msec ); - - /* send a busy response */ - /* osm_sa_send_error( p_ctrl->p_resp, p_madw, IB_RESOURCE_BUSY ); */ - - /* return the request to the pool */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - /* - Note that attr_id (like the rest of the MAD) is in - network byte order. - */ - switch( p_sa_mad->attr_id ) - { - case IB_MAD_ATTR_CLASS_PORT_INFO: - msg_id = OSM_MSG_MAD_CLASS_PORT_INFO; - break; - - case IB_MAD_ATTR_NODE_RECORD: - msg_id = OSM_MSG_MAD_NODE_RECORD; - break; - - case IB_MAD_ATTR_PORTINFO_RECORD: - msg_id = OSM_MSG_MAD_PORTINFO_RECORD; - break; - - case IB_MAD_ATTR_LINK_RECORD: - msg_id = OSM_MSG_MAD_LINK_RECORD; - break; - - case IB_MAD_ATTR_SMINFO_RECORD: - msg_id = OSM_MSG_MAD_SMINFO_RECORD; - break; - - case IB_MAD_ATTR_SERVICE_RECORD: - msg_id = OSM_MSG_MAD_SERVICE_RECORD; - break; - - case IB_MAD_ATTR_PATH_RECORD: - msg_id = OSM_MSG_MAD_PATH_RECORD; - break; - - case IB_MAD_ATTR_MCMEMBER_RECORD: - msg_id = OSM_MSG_MAD_MCMEMBER_RECORD; - break; - - case IB_MAD_ATTR_INFORM_INFO: - msg_id = OSM_MSG_MAD_INFORM_INFO; - break; - - case IB_MAD_ATTR_VLARB_RECORD: - msg_id = OSM_MSG_MAD_VL_ARB_RECORD; - break; - - case IB_MAD_ATTR_SLVL_RECORD: - msg_id = OSM_MSG_MAD_SLVL_TBL_RECORD; - break; - - case IB_MAD_ATTR_PKEY_TBL_RECORD: - msg_id = OSM_MSG_MAD_PKEY_TBL_RECORD; - break; - - case IB_MAD_ATTR_LFT_RECORD: - msg_id = OSM_MSG_MAD_LFT_RECORD; - break; - - case IB_MAD_ATTR_GUIDINFO_RECORD: - msg_id = OSM_MSG_MAD_GUIDINFO_RECORD; - break; - - case IB_MAD_ATTR_INFORM_INFO_RECORD: - msg_id = OSM_MSG_MAD_INFORM_INFO_RECORD; - break; - - case IB_MAD_ATTR_SWITCH_INFO_RECORD: - msg_id = OSM_MSG_MAD_SWITCH_INFO_RECORD; - break; - - case IB_MAD_ATTR_MFT_RECORD: - msg_id = OSM_MSG_MAD_MFT_RECORD; - break; + ib_sa_mad_t *p_sa_mad; + cl_status_t status; + cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; + uint64_t last_dispatched_msg_queue_time_msec; + uint32_t num_messages; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sa_mad_ctrl_process); + + /* + If the dispatcher is showing us that it is overloaded + there is no point in placing the request in. We should instead provide + immediate response - IB_RESOURCE_BUSY + But how do we know? + The dispatcher reports back the number of outstanding messages and the + time the last message stayed in the queue. + HACK: Actually, we cannot send a mad from within the receive callback; + thus - we will just drop it. + */ + cl_disp_get_queue_status(p_ctrl->h_disp, + &num_messages, + &last_dispatched_msg_queue_time_msec); + if ((num_messages > 1) && + (p_ctrl->p_subn->opt.max_msg_fifo_timeout) && + (last_dispatched_msg_queue_time_msec > + p_ctrl->p_subn->opt.max_msg_fifo_timeout)) { + osm_log(p_ctrl->p_log, OSM_LOG_INFO, + "__osm_sa_mad_ctrl_process: " + /* "Responding BUSY status since the dispatcher is already" */ + "Dropping MAD since the dispatcher is already" + " overloaded with %u messages and queue time of:" + "%" PRIu64 "[msec]\n", + num_messages, last_dispatched_msg_queue_time_msec); + + /* send a busy response */ + /* osm_sa_send_error( p_ctrl->p_resp, p_madw, IB_RESOURCE_BUSY ); */ + + /* return the request to the pool */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + /* + Note that attr_id (like the rest of the MAD) is in + network byte order. + */ + switch (p_sa_mad->attr_id) { + case IB_MAD_ATTR_CLASS_PORT_INFO: + msg_id = OSM_MSG_MAD_CLASS_PORT_INFO; + break; + + case IB_MAD_ATTR_NODE_RECORD: + msg_id = OSM_MSG_MAD_NODE_RECORD; + break; + + case IB_MAD_ATTR_PORTINFO_RECORD: + msg_id = OSM_MSG_MAD_PORTINFO_RECORD; + break; + + case IB_MAD_ATTR_LINK_RECORD: + msg_id = OSM_MSG_MAD_LINK_RECORD; + break; + + case IB_MAD_ATTR_SMINFO_RECORD: + msg_id = OSM_MSG_MAD_SMINFO_RECORD; + break; + + case IB_MAD_ATTR_SERVICE_RECORD: + msg_id = OSM_MSG_MAD_SERVICE_RECORD; + break; + + case IB_MAD_ATTR_PATH_RECORD: + msg_id = OSM_MSG_MAD_PATH_RECORD; + break; + + case IB_MAD_ATTR_MCMEMBER_RECORD: + msg_id = OSM_MSG_MAD_MCMEMBER_RECORD; + break; + + case IB_MAD_ATTR_INFORM_INFO: + msg_id = OSM_MSG_MAD_INFORM_INFO; + break; + + case IB_MAD_ATTR_VLARB_RECORD: + msg_id = OSM_MSG_MAD_VL_ARB_RECORD; + break; + + case IB_MAD_ATTR_SLVL_RECORD: + msg_id = OSM_MSG_MAD_SLVL_TBL_RECORD; + break; + + case IB_MAD_ATTR_PKEY_TBL_RECORD: + msg_id = OSM_MSG_MAD_PKEY_TBL_RECORD; + break; + + case IB_MAD_ATTR_LFT_RECORD: + msg_id = OSM_MSG_MAD_LFT_RECORD; + break; + + case IB_MAD_ATTR_GUIDINFO_RECORD: + msg_id = OSM_MSG_MAD_GUIDINFO_RECORD; + break; + + case IB_MAD_ATTR_INFORM_INFO_RECORD: + msg_id = OSM_MSG_MAD_INFORM_INFO_RECORD; + break; + + case IB_MAD_ATTR_SWITCH_INFO_RECORD: + msg_id = OSM_MSG_MAD_SWITCH_INFO_RECORD; + break; + + case IB_MAD_ATTR_MFT_RECORD: + msg_id = OSM_MSG_MAD_MFT_RECORD; + break; #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - case IB_MAD_ATTR_MULTIPATH_RECORD: - msg_id = OSM_MSG_MAD_MULTIPATH_RECORD; - break; + case IB_MAD_ATTR_MULTIPATH_RECORD: + msg_id = OSM_MSG_MAD_MULTIPATH_RECORD; + break; #endif - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_process: ERR 1A01: " - "Unsupported attribute = 0x%X\n", - cl_ntoh16( p_sa_mad->attr_id ) ); - osm_dump_sa_mad( p_ctrl->p_log, p_sa_mad, OSM_LOG_ERROR ); - } - - if( msg_id != CL_DISP_MSGID_NONE ) - { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sa_mad_ctrl_process: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( msg_id ) ); - - status = cl_disp_post( p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sa_mad_ctrl_disp_done_callback, - p_ctrl ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_process: ERR 1A02: " - "Dispatcher post message failed (%s) for attribute = 0x%X\n", - CL_STATUS_MSG( status ), - cl_ntoh16( p_sa_mad->attr_id ) ); - - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - } - else - { - /* - There is an unknown MAD attribute type for which there is - no recipient. Simply retire the MAD here. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_process: ERR 1A01: " + "Unsupported attribute = 0x%X\n", + cl_ntoh16(p_sa_mad->attr_id)); + osm_dump_sa_mad(p_ctrl->p_log, p_sa_mad, OSM_LOG_ERROR); + } + + if (msg_id != CL_DISP_MSGID_NONE) { + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sa_mad_ctrl_process: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + + status = cl_disp_post(p_ctrl->h_disp, + msg_id, + p_madw, + __osm_sa_mad_ctrl_disp_done_callback, + p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_process: ERR 1A02: " + "Dispatcher post message failed (%s) for attribute = 0x%X\n", + CL_STATUS_MSG(status), + cl_ntoh16(p_sa_mad->attr_id)); + + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + } else { + /* + There is an unknown MAD attribute type for which there is + no recipient. Simply retire the MAD here. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -295,116 +288,109 @@ __osm_sa_mad_ctrl_process( * SYNOPSIS */ static void -__osm_sa_mad_ctrl_rcv_callback( - IN osm_madw_t *p_madw, - IN void *bind_context, - IN osm_madw_t *p_req_madw ) +__osm_sa_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, + IN void *bind_context, + IN osm_madw_t * p_req_madw) { - osm_sa_mad_ctrl_t* p_ctrl = (osm_sa_mad_ctrl_t*)bind_context; - ib_sa_mad_t* p_sa_mad; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sa_mad_ctrl_rcv_callback ); - - CL_ASSERT( p_madw ); - - /* - A MAD was received from the wire, possibly in response to a request. - */ - cl_atomic_inc( &p_ctrl->p_stats->sa_mads_rcvd ); - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sa_mad_ctrl_rcv_callback: " - "%u SA MADs received\n", - p_ctrl->p_stats->sa_mads_rcvd ); - } - - /* - * C15-0.1.3 requires not responding to any MAD if the SM is - * not in active state! - * We will not respond if the sm_state is not MASTER, or if the - * first_time_master_sweep flag (of the subnet) is TRUE - this - * flag indicates that the master still didn't finish its first - * sweep, so the subnet is not up and stable yet. - */ - if ( p_ctrl->p_subn->sm_state != IB_SMINFO_STATE_MASTER ) - { - osm_log( p_ctrl->p_log, OSM_LOG_VERBOSE, - "__osm_sa_mad_ctrl_rcv_callback: " - "Received SA MAD while SM not MASTER. MAD ignored\n"); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - if ( p_ctrl->p_subn->first_time_master_sweep == TRUE ) - { - osm_log( p_ctrl->p_log, OSM_LOG_VERBOSE, - "__osm_sa_mad_ctrl_rcv_callback: " - "Received SA MAD while SM in first sweep. MAD ignored\n"); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_FRAMES ) ) - osm_dump_sa_mad( p_ctrl->p_log, p_sa_mad, OSM_LOG_FRAMES ); - - /* - * C15-0.1.5 - Table 185: SA Header - p884 - * SM_key should be either 0 or match the current SM_Key - * otherwise discard the MAD. - */ - if ((p_sa_mad->sm_key != 0) && - (p_sa_mad->sm_key != p_ctrl->p_subn->opt.sm_key)) { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_rcv_callback: ERR 1A04: " - "Non-Zero SA MAD SM_Key: 0x%" PRIx64 " != SM_Key: 0x%" PRIx64 - "; MAD ignored\n", - cl_ntoh64(p_sa_mad->sm_key), - cl_ntoh64(p_ctrl->p_subn->opt.sm_key) - ); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - - switch( p_sa_mad->method ) - { - case IB_MAD_METHOD_REPORT_RESP: - /* we do not really do anything with report represses - - just retire the transaction */ - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sa_mad_ctrl_rcv_callback: " - "Received Report Repress. Retiring the transaction\n"); - - if (p_req_madw) - osm_mad_pool_put( p_ctrl->p_mad_pool, p_req_madw ); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - - break; - - case IB_MAD_METHOD_GET: - case IB_MAD_METHOD_GETTABLE: + osm_sa_mad_ctrl_t *p_ctrl = (osm_sa_mad_ctrl_t *) bind_context; + ib_sa_mad_t *p_sa_mad; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sa_mad_ctrl_rcv_callback); + + CL_ASSERT(p_madw); + + /* + A MAD was received from the wire, possibly in response to a request. + */ + cl_atomic_inc(&p_ctrl->p_stats->sa_mads_rcvd); + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sa_mad_ctrl_rcv_callback: " + "%u SA MADs received\n", p_ctrl->p_stats->sa_mads_rcvd); + } + + /* + * C15-0.1.3 requires not responding to any MAD if the SM is + * not in active state! + * We will not respond if the sm_state is not MASTER, or if the + * first_time_master_sweep flag (of the subnet) is TRUE - this + * flag indicates that the master still didn't finish its first + * sweep, so the subnet is not up and stable yet. + */ + if (p_ctrl->p_subn->sm_state != IB_SMINFO_STATE_MASTER) { + osm_log(p_ctrl->p_log, OSM_LOG_VERBOSE, + "__osm_sa_mad_ctrl_rcv_callback: " + "Received SA MAD while SM not MASTER. MAD ignored\n"); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + if (p_ctrl->p_subn->first_time_master_sweep == TRUE) { + osm_log(p_ctrl->p_log, OSM_LOG_VERBOSE, + "__osm_sa_mad_ctrl_rcv_callback: " + "Received SA MAD while SM in first sweep. MAD ignored\n"); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_FRAMES)) + osm_dump_sa_mad(p_ctrl->p_log, p_sa_mad, OSM_LOG_FRAMES); + + /* + * C15-0.1.5 - Table 185: SA Header - p884 + * SM_key should be either 0 or match the current SM_Key + * otherwise discard the MAD. + */ + if ((p_sa_mad->sm_key != 0) && + (p_sa_mad->sm_key != p_ctrl->p_subn->opt.sm_key)) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_rcv_callback: ERR 1A04: " + "Non-Zero SA MAD SM_Key: 0x%" PRIx64 " != SM_Key: 0x%" + PRIx64 "; MAD ignored\n", cl_ntoh64(p_sa_mad->sm_key), + cl_ntoh64(p_ctrl->p_subn->opt.sm_key) + ); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + switch (p_sa_mad->method) { + case IB_MAD_METHOD_REPORT_RESP: + /* we do not really do anything with report represses - + just retire the transaction */ + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sa_mad_ctrl_rcv_callback: " + "Received Report Repress. Retiring the transaction\n"); + + if (p_req_madw) + osm_mad_pool_put(p_ctrl->p_mad_pool, p_req_madw); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + + break; + + case IB_MAD_METHOD_GET: + case IB_MAD_METHOD_GETTABLE: #if defined (VENDOR_RMPP_SUPPORT) && defined (DUAL_SIDED_RMPP) - case IB_MAD_METHOD_GETMULTI: + case IB_MAD_METHOD_GETMULTI: #endif - case IB_MAD_METHOD_SET: - case IB_MAD_METHOD_DELETE: - __osm_sa_mad_ctrl_process( p_ctrl, p_madw ); - break; - - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_rcv_callback: ERR 1A05: " - "Unsupported method = 0x%X\n", - p_sa_mad->method ); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + case IB_MAD_METHOD_SET: + case IB_MAD_METHOD_DELETE: + __osm_sa_mad_ctrl_process(p_ctrl, p_madw); + break; + + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_rcv_callback: ERR 1A05: " + "Unsupported method = 0x%X\n", p_sa_mad->method); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -426,69 +412,65 @@ __osm_sa_mad_ctrl_rcv_callback( * SYNOPSIS */ void -__osm_sa_mad_ctrl_send_err_callback( - IN void *bind_context, - IN osm_madw_t *p_madw ) +__osm_sa_mad_ctrl_send_err_callback(IN void *bind_context, + IN osm_madw_t * p_madw) { - osm_sa_mad_ctrl_t* p_ctrl = (osm_sa_mad_ctrl_t*)bind_context; - cl_status_t status; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sa_mad_ctrl_send_err_callback ); - - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_send_err_callback: ERR 1A06: " - "MAD transaction completed in error\n" ); - - /* - We should never be here since the SA never originates a request. - Unless we generated a Report(Notice) - */ - - CL_ASSERT( p_madw ); - - /* - An error occurred. No response was received to a request MAD. - Retire the original request MAD. - */ - - osm_dump_sa_mad(p_ctrl->p_log, osm_madw_get_sa_mad_ptr( p_madw ), - OSM_LOG_ERROR ); - - /* __osm_sm_mad_ctrl_update_wire_stats( p_ctrl ); */ - - if( osm_madw_get_err_msg( p_madw ) != CL_DISP_MSGID_NONE ) - { - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sa_mad_ctrl_send_err_callback: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( osm_madw_get_err_msg( p_madw ) ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - osm_madw_get_err_msg( p_madw ), - p_madw, - __osm_sa_mad_ctrl_disp_done_callback, - p_ctrl ); - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sa_mad_ctrl_send_err_callback: ERR 1A07: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - } - } - else - { - /* - No error message was provided, just retire the MAD. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - OSM_LOG_EXIT( p_ctrl->p_log ); + osm_sa_mad_ctrl_t *p_ctrl = (osm_sa_mad_ctrl_t *) bind_context; + cl_status_t status; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sa_mad_ctrl_send_err_callback); + + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_send_err_callback: ERR 1A06: " + "MAD transaction completed in error\n"); + + /* + We should never be here since the SA never originates a request. + Unless we generated a Report(Notice) + */ + + CL_ASSERT(p_madw); + + /* + An error occurred. No response was received to a request MAD. + Retire the original request MAD. + */ + + osm_dump_sa_mad(p_ctrl->p_log, osm_madw_get_sa_mad_ptr(p_madw), + OSM_LOG_ERROR); + + /* __osm_sm_mad_ctrl_update_wire_stats( p_ctrl ); */ + + if (osm_madw_get_err_msg(p_madw) != CL_DISP_MSGID_NONE) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sa_mad_ctrl_send_err_callback: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(osm_madw_get_err_msg + (p_madw))); + } + + status = cl_disp_post(p_ctrl->h_disp, + osm_madw_get_err_msg(p_madw), + p_madw, + __osm_sa_mad_ctrl_disp_done_callback, + p_ctrl); + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sa_mad_ctrl_send_err_callback: ERR 1A07: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + } + } else { + /* + No error message was provided, just retire the MAD. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -501,150 +483,134 @@ __osm_sa_mad_ctrl_send_err_callback( /********************************************************************** **********************************************************************/ -void -osm_sa_mad_ctrl_construct( - IN osm_sa_mad_ctrl_t* const p_ctrl ) +void osm_sa_mad_ctrl_construct(IN osm_sa_mad_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); - memset( p_ctrl, 0, sizeof(*p_ctrl) ); - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; + CL_ASSERT(p_ctrl); + memset(p_ctrl, 0, sizeof(*p_ctrl)); + p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** **********************************************************************/ -void -osm_sa_mad_ctrl_destroy( - IN osm_sa_mad_ctrl_t* const p_ctrl ) +void osm_sa_mad_ctrl_destroy(IN osm_sa_mad_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); - cl_disp_unregister( p_ctrl->h_disp ); + CL_ASSERT(p_ctrl); + cl_disp_unregister(p_ctrl->h_disp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sa_mad_ctrl_init( - IN osm_sa_mad_ctrl_t* const p_ctrl, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vendor_t* const p_vendor, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_dispatcher_t* const p_disp ) +osm_sa_mad_ctrl_init(IN osm_sa_mad_ctrl_t * const p_ctrl, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vendor_t * const p_vendor, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_sa_mad_ctrl_init ); - - osm_sa_mad_ctrl_construct( p_ctrl ); - - p_ctrl->p_log = p_log; - p_ctrl->p_disp = p_disp; - p_ctrl->p_mad_pool = p_mad_pool; - p_ctrl->p_vendor = p_vendor; - p_ctrl->p_stats = p_stats; - p_ctrl->p_subn = p_subn; - p_ctrl->p_resp = p_resp; - - p_ctrl->h_disp = cl_disp_register( - p_disp, - CL_DISP_MSGID_NONE, - NULL, - p_ctrl ); - - if( p_ctrl->h_disp == CL_DISP_INVALID_HANDLE ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_sa_mad_ctrl_init: ERR 1A08: " - "Dispatcher registration failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_sa_mad_ctrl_init); + + osm_sa_mad_ctrl_construct(p_ctrl); + + p_ctrl->p_log = p_log; + p_ctrl->p_disp = p_disp; + p_ctrl->p_mad_pool = p_mad_pool; + p_ctrl->p_vendor = p_vendor; + p_ctrl->p_stats = p_stats; + p_ctrl->p_subn = p_subn; + p_ctrl->p_resp = p_resp; + + p_ctrl->h_disp = cl_disp_register(p_disp, + CL_DISP_MSGID_NONE, NULL, p_ctrl); + + if (p_ctrl->h_disp == CL_DISP_INVALID_HANDLE) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_sa_mad_ctrl_init: ERR 1A08: " + "Dispatcher registration failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sa_mad_ctrl_bind( - IN osm_sa_mad_ctrl_t* const p_ctrl, - IN const ib_net64_t port_guid ) +osm_sa_mad_ctrl_bind(IN osm_sa_mad_ctrl_t * const p_ctrl, + IN const ib_net64_t port_guid) { - osm_bind_info_t bind_info; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_ctrl->p_log, osm_sa_mad_ctrl_bind ); - - if( p_ctrl->h_bind != OSM_BIND_INVALID_HANDLE ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "osm_sa_mad_ctrl_bind: ERR 1A09: " - "Multiple binds not allowed\n" ); - status = IB_ERROR; - goto Exit; - } - - bind_info.class_version = 2; - bind_info.is_responder = TRUE; - bind_info.is_report_processor = FALSE; - bind_info.is_trap_processor = FALSE; - bind_info.mad_class = IB_MCLASS_SUBN_ADM; - bind_info.port_guid = port_guid; - bind_info.recv_q_size = OSM_SM_DEFAULT_QP1_RCV_SIZE; - bind_info.send_q_size = OSM_SM_DEFAULT_QP1_SEND_SIZE; - - osm_log( p_ctrl->p_log, OSM_LOG_VERBOSE, - "osm_sa_mad_ctrl_bind: " - "Binding to port GUID 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - - p_ctrl->h_bind = osm_vendor_bind( p_ctrl->p_vendor, - &bind_info, - p_ctrl->p_mad_pool, - __osm_sa_mad_ctrl_rcv_callback, - __osm_sa_mad_ctrl_send_err_callback, - p_ctrl ); - - if( p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE ) - { - status = IB_ERROR; - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "osm_sa_mad_ctrl_bind: ERR 1A10: " - "Vendor specific bind failed (%s)\n", - ib_get_err_str(status) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); - return( status ); + osm_bind_info_t bind_info; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_ctrl->p_log, osm_sa_mad_ctrl_bind); + + if (p_ctrl->h_bind != OSM_BIND_INVALID_HANDLE) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "osm_sa_mad_ctrl_bind: ERR 1A09: " + "Multiple binds not allowed\n"); + status = IB_ERROR; + goto Exit; + } + + bind_info.class_version = 2; + bind_info.is_responder = TRUE; + bind_info.is_report_processor = FALSE; + bind_info.is_trap_processor = FALSE; + bind_info.mad_class = IB_MCLASS_SUBN_ADM; + bind_info.port_guid = port_guid; + bind_info.recv_q_size = OSM_SM_DEFAULT_QP1_RCV_SIZE; + bind_info.send_q_size = OSM_SM_DEFAULT_QP1_SEND_SIZE; + + osm_log(p_ctrl->p_log, OSM_LOG_VERBOSE, + "osm_sa_mad_ctrl_bind: " + "Binding to port GUID 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + + p_ctrl->h_bind = osm_vendor_bind(p_ctrl->p_vendor, + &bind_info, + p_ctrl->p_mad_pool, + __osm_sa_mad_ctrl_rcv_callback, + __osm_sa_mad_ctrl_send_err_callback, + p_ctrl); + + if (p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE) { + status = IB_ERROR; + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "osm_sa_mad_ctrl_bind: ERR 1A10: " + "Vendor specific bind failed (%s)\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); + return (status); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_sa_mad_ctrl_unbind( - IN osm_sa_mad_ctrl_t* const p_ctrl) +ib_api_status_t osm_sa_mad_ctrl_unbind(IN osm_sa_mad_ctrl_t * const p_ctrl) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_ctrl->p_log, osm_sa_mad_ctrl_unbind ); - - if( p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "osm_sa_mad_ctrl_unbind: ERR 1A11: " - "No previous bind\n" ); - status = IB_ERROR; - goto Exit; - } - - osm_vendor_unbind( p_ctrl->h_bind ); - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_ctrl->p_log, osm_sa_mad_ctrl_unbind); + + if (p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "osm_sa_mad_ctrl_unbind: ERR 1A11: " + "No previous bind\n"); + status = IB_ERROR; + goto Exit; + } + + osm_vendor_unbind(p_ctrl->h_bind); + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); + return (status); } diff --git a/opensm/opensm/osm_sa_mcmember_record.c b/opensm/opensm/osm_sa_mcmember_record.c index 379a3e2..303d58b 100644 --- a/opensm/opensm/osm_sa_mcmember_record.c +++ b/opensm/opensm/osm_sa_mcmember_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -72,88 +72,78 @@ #define OSM_MCMR_RCV_POOL_MIN_SIZE 32 #define OSM_MCMR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_mcmr_item -{ - cl_pool_item_t pool_item; - ib_member_rec_t rec; +typedef struct _osm_mcmr_item { + cl_pool_item_t pool_item; + ib_member_rec_t rec; } osm_mcmr_item_t; -typedef struct osm_sa_mcmr_search_ctxt { - const ib_member_rec_t *p_mcmember_rec; - osm_mgrp_t *p_mgrp; - osm_mcmr_recv_t *p_rcv; - cl_qlist_t *p_list; /* hold results */ - ib_net64_t comp_mask; - const osm_physp_t* p_req_physp; - boolean_t trusted_req; +typedef struct osm_sa_mcmr_search_ctxt { + const ib_member_rec_t *p_mcmember_rec; + osm_mgrp_t *p_mgrp; + osm_mcmr_recv_t *p_rcv; + cl_qlist_t *p_list; /* hold results */ + ib_net64_t comp_mask; + const osm_physp_t *p_req_physp; + boolean_t trusted_req; } osm_sa_mcmr_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_mcmr_rcv_construct( - IN osm_mcmr_recv_t* const p_rcv ) +void osm_mcmr_rcv_construct(IN osm_mcmr_recv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_mcmr_rcv_destroy( - IN osm_mcmr_recv_t* const p_rcv ) +void osm_mcmr_rcv_destroy(IN osm_mcmr_recv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_mcmr_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_mcmr_rcv_destroy); - cl_qlock_pool_destroy( &p_rcv->pool ); + cl_qlock_pool_destroy(&p_rcv->pool); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mcmr_rcv_init( - IN osm_sm_t * const p_sm, - IN osm_mcmr_recv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_mcmr_rcv_init(IN osm_sm_t * const p_sm, + IN osm_mcmr_recv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_mcmr_rcv_init ); - - osm_mcmr_rcv_construct( p_rcv ); - - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_sm = p_sm; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; - p_rcv->mlid_ho = 0xC000; - - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_MCMR_RCV_POOL_MIN_SIZE, - 0, - OSM_MCMR_RCV_POOL_GROW_SIZE, - sizeof(osm_mcmr_item_t), - NULL, NULL, NULL ); - if (status != CL_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_init: ERR 1B02: " - "qlock pool init failed (%d)\n", - status ); - } - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_mcmr_rcv_init); + + osm_mcmr_rcv_construct(p_rcv); + + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_sm = p_sm; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; + p_rcv->mlid_ho = 0xC000; + + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_MCMR_RCV_POOL_MIN_SIZE, + 0, + OSM_MCMR_RCV_POOL_GROW_SIZE, + sizeof(osm_mcmr_item_t), NULL, NULL, NULL); + if (status != CL_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_init: ERR 1B02: " + "qlock pool init failed (%d)\n", status); + } + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** @@ -167,57 +157,51 @@ osm_mcmr_rcv_init( looked for and the result p_mgrp **********************************************************************/ static void -__search_mgrp_by_mgid( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__search_mgrp_by_mgid(IN cl_map_item_t * const p_map_item, IN void *context) { - osm_mgrp_t* p_mgrp = (osm_mgrp_t*)p_map_item; - osm_sa_mcmr_search_ctxt_t *p_ctxt = (osm_sa_mcmr_search_ctxt_t *) context; - const ib_member_rec_t *p_recvd_mcmember_rec; - osm_mcmr_recv_t *p_rcv; - - p_recvd_mcmember_rec = p_ctxt->p_mcmember_rec; - p_rcv = p_ctxt->p_rcv; - - /* ignore groups marked for deletion */ - if (p_mgrp->to_be_deleted) - return; - - /* compare entire MGID so different scope will not sneak in for - the same MGID */ - if (memcmp(&p_mgrp->mcmember_rec.mgid, - &p_recvd_mcmember_rec->mgid, - sizeof(ib_gid_t))) - return; - - if (p_ctxt->p_mgrp) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__search_mgrp_by_mgid: ERR 1B03: " - "Multiple MC groups for same MGID\n" ); - return; - } - - p_ctxt->p_mgrp = p_mgrp; + osm_mgrp_t *p_mgrp = (osm_mgrp_t *) p_map_item; + osm_sa_mcmr_search_ctxt_t *p_ctxt = + (osm_sa_mcmr_search_ctxt_t *) context; + const ib_member_rec_t *p_recvd_mcmember_rec; + osm_mcmr_recv_t *p_rcv; + + p_recvd_mcmember_rec = p_ctxt->p_mcmember_rec; + p_rcv = p_ctxt->p_rcv; + + /* ignore groups marked for deletion */ + if (p_mgrp->to_be_deleted) + return; + + /* compare entire MGID so different scope will not sneak in for + the same MGID */ + if (memcmp(&p_mgrp->mcmember_rec.mgid, + &p_recvd_mcmember_rec->mgid, sizeof(ib_gid_t))) + return; + + if (p_ctxt->p_mgrp) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__search_mgrp_by_mgid: ERR 1B03: " + "Multiple MC groups for same MGID\n"); + return; + } + + p_ctxt->p_mgrp = p_mgrp; } /********************************************************************** Look for a MGRP in the mgrp_mlid_tbl by mlid **********************************************************************/ -static osm_mgrp_t * -__get_mgrp_by_mlid( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net16_t const mlid) +static osm_mgrp_t *__get_mgrp_by_mlid(IN osm_mcmr_recv_t * const p_rcv, + IN ib_net16_t const mlid) { - cl_map_item_t *map_item; + cl_map_item_t *map_item; - map_item = cl_qmap_get(&p_rcv->p_subn->mgrp_mlid_tbl, mlid); - if (map_item == cl_qmap_end(&p_rcv->p_subn->mgrp_mlid_tbl)) - { - return NULL; - } - return (osm_mgrp_t *)map_item; + map_item = cl_qmap_get(&p_rcv->p_subn->mgrp_mlid_tbl, mlid); + if (map_item == cl_qmap_end(&p_rcv->p_subn->mgrp_mlid_tbl)) { + return NULL; + } + return (osm_mgrp_t *) map_item; } @@ -225,28 +209,25 @@ __get_mgrp_by_mlid( Look for a MGRP in the mgrp_mlid_tbl by mgid ***********************************************************************/ static ib_api_status_t -__get_mgrp_by_mgid( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_member_rec_t* p_recvd_mcmember_rec, - OUT osm_mgrp_t **pp_mgrp) +__get_mgrp_by_mgid(IN osm_mcmr_recv_t * const p_rcv, + IN ib_member_rec_t * p_recvd_mcmember_rec, + OUT osm_mgrp_t ** pp_mgrp) { - osm_sa_mcmr_search_ctxt_t mcmr_search_context; + osm_sa_mcmr_search_ctxt_t mcmr_search_context; - mcmr_search_context.p_mcmember_rec = p_recvd_mcmember_rec; - mcmr_search_context.p_rcv = p_rcv; - mcmr_search_context.p_mgrp = NULL; + mcmr_search_context.p_mcmember_rec = p_recvd_mcmember_rec; + mcmr_search_context.p_rcv = p_rcv; + mcmr_search_context.p_mgrp = NULL; - cl_qmap_apply_func( &p_rcv->p_subn->mgrp_mlid_tbl, - __search_mgrp_by_mgid, - &mcmr_search_context); + cl_qmap_apply_func(&p_rcv->p_subn->mgrp_mlid_tbl, + __search_mgrp_by_mgid, &mcmr_search_context); - if (mcmr_search_context.p_mgrp == NULL) - { - return IB_NOT_FOUND; - } + if (mcmr_search_context.p_mgrp == NULL) { + return IB_NOT_FOUND; + } - *pp_mgrp = mcmr_search_context.p_mgrp; - return IB_SUCCESS; + *pp_mgrp = mcmr_search_context.p_mgrp; + return IB_SUCCESS; } /********************************************************************* @@ -255,18 +236,17 @@ used during the process of join request to copy data from the mgrp to the port record. **********************************************************************/ static inline void -__copy_from_create_mc_rec( - IN ib_member_rec_t * const dest, - IN const ib_member_rec_t *const src) +__copy_from_create_mc_rec(IN ib_member_rec_t * const dest, + IN const ib_member_rec_t * const src) { - dest->qkey = src->qkey; - dest->mlid = src->mlid; - dest->tclass = src->tclass; - dest->pkey = src->pkey; - dest->sl_flow_hop = src->sl_flow_hop; - dest->mtu = src->mtu; - dest->rate = src->rate; - dest->pkt_life = src->pkt_life; + dest->qkey = src->qkey; + dest->mlid = src->mlid; + dest->tclass = src->tclass; + dest->pkey = src->pkey; + dest->sl_flow_hop = src->sl_flow_hop; + dest->mtu = src->mtu; + dest->rate = src->rate; + dest->pkt_life = src->pkt_life; } /********************************************************************* @@ -274,13 +254,10 @@ Return an mlid to the pool of free mlids. But this implementation is not a pool - it simply scans through the MGRP database for unused mlids... *********************************************************************/ -static void -__free_mlid( - IN osm_mcmr_recv_t* const p_rcv, - IN uint16_t mlid) +static void __free_mlid(IN osm_mcmr_recv_t * const p_rcv, IN uint16_t mlid) { - UNUSED_PARAM(p_rcv); - UNUSED_PARAM(mlid); + UNUSED_PARAM(p_rcv); + UNUSED_PARAM(mlid); } /********************************************************************* @@ -289,109 +266,99 @@ TODO: Implement a more scalable - O(1) solution based on pool of available mlids. **********************************************************************/ static ib_net16_t -__get_new_mlid( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net16_t requested_mlid) +__get_new_mlid(IN osm_mcmr_recv_t * const p_rcv, IN ib_net16_t requested_mlid) { - osm_subn_t *p_subn = p_rcv->p_subn; - osm_mgrp_t *p_mgrp; - uint8_t *used_mlids_array; - uint16_t idx; - uint16_t mlid; /* the result */ - uint16_t max_num_mlids; - - OSM_LOG_ENTER(p_rcv->p_log, __get_new_mlid); - - if (requested_mlid && cl_ntoh16(requested_mlid) >= IB_LID_MCAST_START_HO && - cl_ntoh16(requested_mlid) < p_subn->max_multicast_lid_ho && - cl_qmap_get(&p_subn->mgrp_mlid_tbl, requested_mlid) == - cl_qmap_end(&p_subn->mgrp_mlid_tbl) ) { - mlid = cl_ntoh16(requested_mlid); - goto Exit; - } - - /* If MCGroups table empty, first return the min mlid */ - p_mgrp = (osm_mgrp_t*)cl_qmap_head( &p_subn->mgrp_mlid_tbl ); - if (p_mgrp == (osm_mgrp_t*)cl_qmap_end( &p_subn->mgrp_mlid_tbl )) - { - mlid = IB_LID_MCAST_START_HO; - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__get_new_mlid: " - "No multicast groups found using minimal mlid:0x%04X\n", - mlid ); - goto Exit; - } - - max_num_mlids = - p_rcv->p_subn->max_multicast_lid_ho - IB_LID_MCAST_START_HO; - - /* track all used mlids in the array (by mlid index) */ - used_mlids_array = - (uint8_t *)malloc(sizeof(uint8_t)*max_num_mlids); - if (used_mlids_array) - memset(used_mlids_array, 0, sizeof(uint8_t)*max_num_mlids); - if (!used_mlids_array) - return 0; - - /* scan all available multicast groups in the DB and fill in the table */ - while( p_mgrp != (osm_mgrp_t*)cl_qmap_end( &p_subn->mgrp_mlid_tbl ) ) - { - /* ignore mgrps marked for deletion */ - if (p_mgrp->to_be_deleted == FALSE) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__get_new_mlid: " - "Found mgrp with lid:0x%X MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh16( p_mgrp->mlid), - cl_ntoh64( p_mgrp->mcmember_rec.mgid.unicast.prefix ), - cl_ntoh64( p_mgrp->mcmember_rec.mgid.unicast.interface_id ) ); - - /* Map in table */ - if (cl_ntoh16(p_mgrp->mlid) > p_rcv->p_subn->max_multicast_lid_ho) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__get_new_mlid: ERR 1B27: " - "Found mgrp with mlid:0x%04X > max allowed mlid:0x%04X\n", - cl_ntoh16(p_mgrp->mlid), - max_num_mlids + IB_LID_MCAST_START_HO ); - } - else - { - used_mlids_array[cl_ntoh16(p_mgrp->mlid) - IB_LID_MCAST_START_HO] = 1; - } - } - p_mgrp = (osm_mgrp_t*)cl_qmap_next( &p_mgrp->map_item ); - } - - /* Find "mlid holes" in the mgrp table */ - for (idx = 0; - (idx < max_num_mlids) && (used_mlids_array[idx] == 1); - idx++); - - /* did it go above the maximal mlid allowed */ - if ( idx < max_num_mlids ) - { - mlid = idx + IB_LID_MCAST_START_HO; - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__get_new_mlid: " - "Found available mlid:0x%04X at idx:%u\n", - mlid, idx ); - } - else - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__get_new_mlid: ERR 1B23: " - "All available:%u mlids are taken\n", - max_num_mlids ); - mlid = 0; - } - - free(used_mlids_array); - - Exit: - OSM_LOG_EXIT(p_rcv->p_log); - return cl_hton16(mlid); + osm_subn_t *p_subn = p_rcv->p_subn; + osm_mgrp_t *p_mgrp; + uint8_t *used_mlids_array; + uint16_t idx; + uint16_t mlid; /* the result */ + uint16_t max_num_mlids; + + OSM_LOG_ENTER(p_rcv->p_log, __get_new_mlid); + + if (requested_mlid && cl_ntoh16(requested_mlid) >= IB_LID_MCAST_START_HO + && cl_ntoh16(requested_mlid) < p_subn->max_multicast_lid_ho + && cl_qmap_get(&p_subn->mgrp_mlid_tbl, + requested_mlid) == + cl_qmap_end(&p_subn->mgrp_mlid_tbl)) { + mlid = cl_ntoh16(requested_mlid); + goto Exit; + } + + /* If MCGroups table empty, first return the min mlid */ + p_mgrp = (osm_mgrp_t *) cl_qmap_head(&p_subn->mgrp_mlid_tbl); + if (p_mgrp == (osm_mgrp_t *) cl_qmap_end(&p_subn->mgrp_mlid_tbl)) { + mlid = IB_LID_MCAST_START_HO; + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__get_new_mlid: " + "No multicast groups found using minimal mlid:0x%04X\n", + mlid); + goto Exit; + } + + max_num_mlids = + p_rcv->p_subn->max_multicast_lid_ho - IB_LID_MCAST_START_HO; + + /* track all used mlids in the array (by mlid index) */ + used_mlids_array = (uint8_t *) malloc(sizeof(uint8_t) * max_num_mlids); + if (used_mlids_array) + memset(used_mlids_array, 0, sizeof(uint8_t) * max_num_mlids); + if (!used_mlids_array) + return 0; + + /* scan all available multicast groups in the DB and fill in the table */ + while (p_mgrp != (osm_mgrp_t *) cl_qmap_end(&p_subn->mgrp_mlid_tbl)) { + /* ignore mgrps marked for deletion */ + if (p_mgrp->to_be_deleted == FALSE) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__get_new_mlid: " + "Found mgrp with lid:0x%X MGID: 0x%016" PRIx64 + " : " "0x%016" PRIx64 "\n", + cl_ntoh16(p_mgrp->mlid), + cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. + prefix), + cl_ntoh64(p_mgrp->mcmember_rec.mgid.unicast. + interface_id)); + + /* Map in table */ + if (cl_ntoh16(p_mgrp->mlid) > + p_rcv->p_subn->max_multicast_lid_ho) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__get_new_mlid: ERR 1B27: " + "Found mgrp with mlid:0x%04X > max allowed mlid:0x%04X\n", + cl_ntoh16(p_mgrp->mlid), + max_num_mlids + IB_LID_MCAST_START_HO); + } else { + used_mlids_array[cl_ntoh16(p_mgrp->mlid) - + IB_LID_MCAST_START_HO] = 1; + } + } + p_mgrp = (osm_mgrp_t *) cl_qmap_next(&p_mgrp->map_item); + } + + /* Find "mlid holes" in the mgrp table */ + for (idx = 0; + (idx < max_num_mlids) && (used_mlids_array[idx] == 1); idx++) ; + + /* did it go above the maximal mlid allowed */ + if (idx < max_num_mlids) { + mlid = idx + IB_LID_MCAST_START_HO; + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__get_new_mlid: " + "Found available mlid:0x%04X at idx:%u\n", mlid, idx); + } else { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__get_new_mlid: ERR 1B23: " + "All available:%u mlids are taken\n", max_num_mlids); + mlid = 0; + } + + free(used_mlids_array); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return cl_hton16(mlid); } /********************************************************************* @@ -402,26 +369,22 @@ silently drop it. Since it was an intermediate group no need to re-route it. **********************************************************************/ static void -__cleanup_mgrp( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net16_t const mlid) +__cleanup_mgrp(IN osm_mcmr_recv_t * const p_rcv, IN ib_net16_t const mlid) { - osm_mgrp_t *p_mgrp; - - p_mgrp = __get_mgrp_by_mlid(p_rcv, mlid); - if(p_mgrp) - { - /* Remove MGRP only if osm_mcm_port_t count is 0 and - * Not a well known group - */ - if(cl_is_qmap_empty(&p_mgrp->mcm_port_tbl) && - (p_mgrp->well_known == FALSE)) - { - cl_qmap_remove_item(&p_rcv->p_subn->mgrp_mlid_tbl, - (cl_map_item_t *)p_mgrp ); - osm_mgrp_delete(p_mgrp); - } - } + osm_mgrp_t *p_mgrp; + + p_mgrp = __get_mgrp_by_mlid(p_rcv, mlid); + if (p_mgrp) { + /* Remove MGRP only if osm_mcm_port_t count is 0 and + * Not a well known group + */ + if (cl_is_qmap_empty(&p_mgrp->mcm_port_tbl) && + (p_mgrp->well_known == FALSE)) { + cl_qmap_remove_item(&p_rcv->p_subn->mgrp_mlid_tbl, + (cl_map_item_t *) p_mgrp); + osm_mgrp_delete(p_mgrp); + } + } } /********************************************************************* @@ -429,162 +392,149 @@ Add a port to the group. Calculating its PROXY_JOIN by the Port and requester gids. **********************************************************************/ static ib_api_status_t -__add_new_mgrp_port( - IN osm_mcmr_recv_t *p_rcv, - IN osm_mgrp_t *p_mgrp, - IN ib_member_rec_t *p_recvd_mcmember_rec, - IN osm_mad_addr_t *p_mad_addr, - OUT osm_mcm_port_t **pp_mcmr_port) +__add_new_mgrp_port(IN osm_mcmr_recv_t * p_rcv, + IN osm_mgrp_t * p_mgrp, + IN ib_member_rec_t * p_recvd_mcmember_rec, + IN osm_mad_addr_t * p_mad_addr, + OUT osm_mcm_port_t ** pp_mcmr_port) { - boolean_t proxy_join; - ib_gid_t requester_gid; - ib_api_status_t res; - - /* set the proxy_join if the requester gid is not identical to the - joined gid */ - res = osm_get_gid_by_mad_addr( p_rcv->p_log, - p_rcv->p_subn, - p_mad_addr, &requester_gid ); - if ( res != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__add_new_mgrp_port: ERR 1B29: " - "Could not find GID for requester\n" ); - - return IB_INVALID_PARAMETER; - } - - if (!memcmp(&p_recvd_mcmember_rec->port_gid, &requester_gid, - sizeof(ib_gid_t))) - { - proxy_join = FALSE; - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__add_new_mgrp_port: " - "Create new port with proxy_join FALSE\n" ); - } - else - { - /* The port is not the one specified in PortGID. - The check that the requester is in the same partition as - the PortGID is done before - just need to update the proxy_join. */ - proxy_join = TRUE; - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__add_new_mgrp_port: " - "Create new port with proxy_join TRUE\n" ); - } - - *pp_mcmr_port = osm_mgrp_add_port( p_mgrp, - &p_recvd_mcmember_rec->port_gid, - p_recvd_mcmember_rec->scope_state, - proxy_join ); - if(*pp_mcmr_port == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__add_new_mgrp_port: ERR 1B06: " - "osm_mgrp_add_port failed\n" ); - - return IB_INSUFFICIENT_MEMORY; - } - - return IB_SUCCESS; + boolean_t proxy_join; + ib_gid_t requester_gid; + ib_api_status_t res; + + /* set the proxy_join if the requester gid is not identical to the + joined gid */ + res = osm_get_gid_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + p_mad_addr, &requester_gid); + if (res != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__add_new_mgrp_port: ERR 1B29: " + "Could not find GID for requester\n"); + + return IB_INVALID_PARAMETER; + } + + if (!memcmp(&p_recvd_mcmember_rec->port_gid, &requester_gid, + sizeof(ib_gid_t))) { + proxy_join = FALSE; + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__add_new_mgrp_port: " + "Create new port with proxy_join FALSE\n"); + } else { + /* The port is not the one specified in PortGID. + The check that the requester is in the same partition as + the PortGID is done before - just need to update the proxy_join. */ + proxy_join = TRUE; + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__add_new_mgrp_port: " + "Create new port with proxy_join TRUE\n"); + } + + *pp_mcmr_port = osm_mgrp_add_port(p_mgrp, + &p_recvd_mcmember_rec->port_gid, + p_recvd_mcmember_rec->scope_state, + proxy_join); + if (*pp_mcmr_port == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__add_new_mgrp_port: ERR 1B06: " + "osm_mgrp_add_port failed\n"); + + return IB_INSUFFICIENT_MEMORY; + } + + return IB_SUCCESS; } /********************************************************************** **********************************************************************/ -static inline boolean_t -__check_join_comp_mask(ib_net64_t comp_mask) +static inline boolean_t __check_join_comp_mask(ib_net64_t comp_mask) { - return( (comp_mask & JOIN_MC_COMP_MASK) == JOIN_MC_COMP_MASK ); + return ((comp_mask & JOIN_MC_COMP_MASK) == JOIN_MC_COMP_MASK); } /********************************************************************** **********************************************************************/ static inline boolean_t __check_create_comp_mask(ib_net64_t comp_mask, - ib_member_rec_t *p_recvd_mcmember_rec) + ib_member_rec_t * p_recvd_mcmember_rec) { - return( - ((comp_mask & REQUIRED_MC_CREATE_COMP_MASK) == REQUIRED_MC_CREATE_COMP_MASK) - ); + return (((comp_mask & REQUIRED_MC_CREATE_COMP_MASK) == + REQUIRED_MC_CREATE_COMP_MASK) + ); } /********************************************************************** Generate the response MAD **********************************************************************/ static void -__osm_mcmr_rcv_respond( - IN const osm_mcmr_recv_t* const p_rcv, - IN const osm_madw_t* const p_madw, - IN ib_member_rec_t *p_mcmember_rec ) +__osm_mcmr_rcv_respond(IN const osm_mcmr_recv_t * const p_rcv, + IN const osm_madw_t * const p_madw, + IN ib_member_rec_t * p_mcmember_rec) { - osm_madw_t *p_resp_madw; - ib_sa_mad_t *p_sa_mad, *p_resp_sa_mad; - ib_member_rec_t *p_resp_mcmember_rec; - ib_api_status_t status; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mcmr_rcv_respond ); - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, - p_madw->h_bind, - sizeof(ib_member_rec_t)+IB_SA_MAD_HDR_SIZE, - osm_madw_get_mad_addr_ptr(p_madw) ); - if ( !p_resp_madw ) - { - goto Exit; - } - - p_resp_sa_mad = (ib_sa_mad_t*)p_resp_madw->p_mad; - p_sa_mad = (ib_sa_mad_t*)p_madw->p_mad; - /* Copy the MAD header back into the response mad */ - memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); - /* based on the current method decide about the response: */ - if ((p_resp_sa_mad->method == IB_MAD_METHOD_GET) || - (p_resp_sa_mad->method == IB_MAD_METHOD_SET)) { - p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; - } - else if (p_resp_sa_mad->method == IB_MAD_METHOD_DELETE) { - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - } - else - { - CL_ASSERT( p_resp_sa_mad->method == 0); - } - - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_member_rec_t) ); - p_resp_mcmember_rec = (ib_member_rec_t*)&p_resp_sa_mad->data; - - *p_resp_mcmember_rec = *p_mcmember_rec; - - /* Fill in the mtu, rate, and packet lifetime selectors */ - p_resp_mcmember_rec->mtu &= 0x3f; - p_resp_mcmember_rec->mtu |= 2<<6; /* exactly */ - p_resp_mcmember_rec->rate &= 0x3f; - p_resp_mcmember_rec->rate |= 2<<6; /* exactly */ - p_resp_mcmember_rec->pkt_life &= 0x3f; - p_resp_mcmember_rec->pkt_life |= 2<<6; /* exactly */ - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - - if(status != IB_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_respond: ERR 1B07: " - "Unable to send MAD (%s) for TID <0x%"PRIx64">\n", - ib_get_err_str( status ), - p_resp_sa_mad->trans_id ); - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_sa_mad, *p_resp_sa_mad; + ib_member_rec_t *p_resp_mcmember_rec; + ib_api_status_t status; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mcmr_rcv_respond); + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + sizeof(ib_member_rec_t) + + IB_SA_MAD_HDR_SIZE, + osm_madw_get_mad_addr_ptr(p_madw)); + if (!p_resp_madw) { + goto Exit; + } + + p_resp_sa_mad = (ib_sa_mad_t *) p_resp_madw->p_mad; + p_sa_mad = (ib_sa_mad_t *) p_madw->p_mad; + /* Copy the MAD header back into the response mad */ + memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); + /* based on the current method decide about the response: */ + if ((p_resp_sa_mad->method == IB_MAD_METHOD_GET) || + (p_resp_sa_mad->method == IB_MAD_METHOD_SET)) { + p_resp_sa_mad->method = IB_MAD_METHOD_GET_RESP; + } else if (p_resp_sa_mad->method == IB_MAD_METHOD_DELETE) { + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + } else { + CL_ASSERT(p_resp_sa_mad->method == 0); + } + + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); + p_resp_mcmember_rec = (ib_member_rec_t *) & p_resp_sa_mad->data; + + *p_resp_mcmember_rec = *p_mcmember_rec; + + /* Fill in the mtu, rate, and packet lifetime selectors */ + p_resp_mcmember_rec->mtu &= 0x3f; + p_resp_mcmember_rec->mtu |= 2 << 6; /* exactly */ + p_resp_mcmember_rec->rate &= 0x3f; + p_resp_mcmember_rec->rate |= 2 << 6; /* exactly */ + p_resp_mcmember_rec->pkt_life &= 0x3f; + p_resp_mcmember_rec->pkt_life |= 2 << 6; /* exactly */ + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_respond: ERR 1B07: " + "Unable to send MAD (%s) for TID <0x%" PRIx64 ">\n", + ib_get_err_str(status), p_resp_sa_mad->trans_id); + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************* @@ -593,107 +543,96 @@ we make sure the following components provided match: MTU and RATE HACK: Currently we ignore the PKT_LIFETIME field. **********************************************************************/ static boolean_t -__validate_more_comp_fields( - osm_log_t *p_log, - const osm_mgrp_t *p_mgrp, - const ib_member_rec_t *p_recvd_mcmember_rec, - ib_net64_t comp_mask) +__validate_more_comp_fields(osm_log_t * p_log, + const osm_mgrp_t * p_mgrp, + const ib_member_rec_t * p_recvd_mcmember_rec, + ib_net64_t comp_mask) { - uint8_t mtu_sel; - uint8_t mtu_required; - uint8_t mtu_mgrp; - uint8_t rate_sel; - uint8_t rate_required; - uint8_t rate_mgrp; - - if ( comp_mask & IB_MCR_COMPMASK_MTU_SEL) - { - mtu_sel = (uint8_t)(p_recvd_mcmember_rec->mtu >> 6); - /* Clearing last 2 bits */ - mtu_required = (uint8_t)(p_recvd_mcmember_rec->mtu & 0x3F); - mtu_mgrp = (uint8_t)(p_mgrp->mcmember_rec.mtu & 0x3F); - switch (mtu_sel) - { - case 0: /* Greater than MTU specified */ - if(mtu_mgrp <= mtu_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has MTU %x, which is not greater than %x\n", - mtu_mgrp, mtu_required ); - return FALSE; - } - break; - case 1: /* Less than MTU specified */ - if(mtu_mgrp >= mtu_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has MTU %x, which is not less than %x\n", - mtu_mgrp, mtu_required ); - return FALSE; - } - break; - case 2: /* Exactly MTU specified */ - if(mtu_mgrp != mtu_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has MTU %x, which is not equal to %x\n", - mtu_mgrp, mtu_required ); - return FALSE; - } - break; - default: - break; - } - } - - /* what about rate ? */ - if ( comp_mask & IB_MCR_COMPMASK_RATE_SEL) - { - rate_sel = (uint8_t)(p_recvd_mcmember_rec->rate >> 6); - /* Clearing last 2 bits */ - rate_required = (uint8_t)(p_recvd_mcmember_rec->rate & 0x3F); - rate_mgrp = (uint8_t)(p_mgrp->mcmember_rec.rate & 0x3F); - switch (rate_sel) - { - case 0: /* Greater than RATE specified */ - if(rate_mgrp <= rate_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has RATE %x, which is not greater than %x\n", - rate_mgrp, rate_required ); - return FALSE; - } - break; - case 1: /* Less than RATE specified */ - if(rate_mgrp >= rate_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has RATE %x, which is not less than %x\n", - rate_mgrp, rate_required ); - return FALSE; - } - break; - case 2: /* Exactly RATE specified */ - if(rate_mgrp != rate_required) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_more_comp_fields: " - "Requested mcast group has RATE %x, which is not equal to %x\n", - rate_mgrp, rate_required ); - return FALSE; - } - break; - default: - break; - } - } - - return TRUE; + uint8_t mtu_sel; + uint8_t mtu_required; + uint8_t mtu_mgrp; + uint8_t rate_sel; + uint8_t rate_required; + uint8_t rate_mgrp; + + if (comp_mask & IB_MCR_COMPMASK_MTU_SEL) { + mtu_sel = (uint8_t) (p_recvd_mcmember_rec->mtu >> 6); + /* Clearing last 2 bits */ + mtu_required = (uint8_t) (p_recvd_mcmember_rec->mtu & 0x3F); + mtu_mgrp = (uint8_t) (p_mgrp->mcmember_rec.mtu & 0x3F); + switch (mtu_sel) { + case 0: /* Greater than MTU specified */ + if (mtu_mgrp <= mtu_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has MTU %x, which is not greater than %x\n", + mtu_mgrp, mtu_required); + return FALSE; + } + break; + case 1: /* Less than MTU specified */ + if (mtu_mgrp >= mtu_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has MTU %x, which is not less than %x\n", + mtu_mgrp, mtu_required); + return FALSE; + } + break; + case 2: /* Exactly MTU specified */ + if (mtu_mgrp != mtu_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has MTU %x, which is not equal to %x\n", + mtu_mgrp, mtu_required); + return FALSE; + } + break; + default: + break; + } + } + + /* what about rate ? */ + if (comp_mask & IB_MCR_COMPMASK_RATE_SEL) { + rate_sel = (uint8_t) (p_recvd_mcmember_rec->rate >> 6); + /* Clearing last 2 bits */ + rate_required = (uint8_t) (p_recvd_mcmember_rec->rate & 0x3F); + rate_mgrp = (uint8_t) (p_mgrp->mcmember_rec.rate & 0x3F); + switch (rate_sel) { + case 0: /* Greater than RATE specified */ + if (rate_mgrp <= rate_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has RATE %x, which is not greater than %x\n", + rate_mgrp, rate_required); + return FALSE; + } + break; + case 1: /* Less than RATE specified */ + if (rate_mgrp >= rate_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has RATE %x, which is not less than %x\n", + rate_mgrp, rate_required); + return FALSE; + } + break; + case 2: /* Exactly RATE specified */ + if (rate_mgrp != rate_required) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_more_comp_fields: " + "Requested mcast group has RATE %x, which is not equal to %x\n", + rate_mgrp, rate_required); + return FALSE; + } + break; + default: + break; + } + } + + return TRUE; } /********************************************************************* @@ -701,39 +640,35 @@ In joining an existing group, we make sure the following components are physically realizable: MTU and RATE **********************************************************************/ static boolean_t -__validate_port_caps( - osm_log_t * const p_log, - const osm_mgrp_t *p_mgrp, - const osm_physp_t *p_physp) +__validate_port_caps(osm_log_t * const p_log, + const osm_mgrp_t * p_mgrp, const osm_physp_t * p_physp) { - uint8_t mtu_required; - uint8_t mtu_mgrp; - uint8_t rate_required; - uint8_t rate_mgrp; - - mtu_required = ib_port_info_get_mtu_cap(&p_physp->port_info); - mtu_mgrp = (uint8_t)(p_mgrp->mcmember_rec.mtu & 0x3F); - if (mtu_required < mtu_mgrp) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_port_caps: " - "Port's MTU %x is less than %x\n", - mtu_required, mtu_mgrp ); - return FALSE; - } - - rate_required = ib_port_info_compute_rate(&p_physp->port_info); - rate_mgrp = (uint8_t)(p_mgrp->mcmember_rec.rate & 0x3F); - if (rate_required < rate_mgrp) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__validate_port_caps: " - "Port's RATE %x is less than %x\n", - rate_required, rate_mgrp ); - return FALSE; - } - - return TRUE; + uint8_t mtu_required; + uint8_t mtu_mgrp; + uint8_t rate_required; + uint8_t rate_mgrp; + + mtu_required = ib_port_info_get_mtu_cap(&p_physp->port_info); + mtu_mgrp = (uint8_t) (p_mgrp->mcmember_rec.mtu & 0x3F); + if (mtu_required < mtu_mgrp) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_port_caps: " + "Port's MTU %x is less than %x\n", + mtu_required, mtu_mgrp); + return FALSE; + } + + rate_required = ib_port_info_compute_rate(&p_physp->port_info); + rate_mgrp = (uint8_t) (p_mgrp->mcmember_rec.rate & 0x3F); + if (rate_required < rate_mgrp) { + osm_log(p_log, OSM_LOG_DEBUG, + "__validate_port_caps: " + "Port's RATE %x is less than %x\n", + rate_required, rate_mgrp); + return FALSE; + } + + return TRUE; } /********************************************************************** @@ -748,87 +683,84 @@ __validate_port_caps( * part of the partition for that MCMemberRecord. **********************************************************************/ static boolean_t -__validate_modify(IN osm_mcmr_recv_t* const p_rcv, - IN osm_mgrp_t* p_mgrp, - IN osm_mad_addr_t* p_mad_addr, - IN ib_member_rec_t* p_recvd_mcmember_rec, - OUT osm_mcm_port_t **pp_mcm_port) +__validate_modify(IN osm_mcmr_recv_t * const p_rcv, + IN osm_mgrp_t * p_mgrp, + IN osm_mad_addr_t * p_mad_addr, + IN ib_member_rec_t * p_recvd_mcmember_rec, + OUT osm_mcm_port_t ** pp_mcm_port) { - ib_net64_t portguid; - ib_gid_t request_gid; - osm_physp_t* p_request_physp; - ib_api_status_t res; - - portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; - - *pp_mcm_port = NULL; - - /* o15-0.2.1: If this is a new port being added - nothing to check */ - if (!osm_mgrp_is_port_present(p_mgrp, portguid, pp_mcm_port)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_modify: " - "This is a new port in the MC group\n" ); - return TRUE; - } - - /* We validate the request according the the proxy_join. - Check if the proxy_join is set or not */ - if ( (*pp_mcm_port)->proxy_join == FALSE ) - { - /* The proxy_join is not set. Modifying can by done only - if the requester GID == PortGID */ - res = osm_get_gid_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - p_mad_addr, - &request_gid); - - if ( res != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_modify: " - "Could not find port for requested address\n" - ); - return FALSE; - } - - if (memcmp(&((*pp_mcm_port)->port_gid), &request_gid, sizeof(ib_gid_t))) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_modify: " - "No ProxyJoin but different ports: stored:0x%016"PRIx64 - " request:0x%016"PRIx64"\n", - cl_ntoh64((*pp_mcm_port)->port_gid.unicast.interface_id), - cl_ntoh64(p_mad_addr->addr_type.gsi.grh_info.src_gid.unicast.interface_id) - ); - return FALSE; - } - } - else - { - /* The proxy_join is set. Modification allowed only if the - requester is part of the partition for this MCMemberRecord */ - p_request_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - p_mad_addr); - if (p_request_physp == NULL) - return FALSE; - - if (!osm_physp_has_pkey(p_rcv->p_log, p_mgrp->mcmember_rec.pkey, - p_request_physp)) - { - /* the request port is not part of the partition for this mgrp */ - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_modify: " - "ProxyJoin but port not in partition. stored:0x%016"PRIx64 - " request:0x%016"PRIx64"\n", - cl_ntoh64((*pp_mcm_port)->port_gid.unicast.interface_id), - cl_ntoh64(p_mad_addr->addr_type.gsi.grh_info.src_gid.unicast.interface_id) - ); - return FALSE; - } - } - return TRUE; + ib_net64_t portguid; + ib_gid_t request_gid; + osm_physp_t *p_request_physp; + ib_api_status_t res; + + portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; + + *pp_mcm_port = NULL; + + /* o15-0.2.1: If this is a new port being added - nothing to check */ + if (!osm_mgrp_is_port_present(p_mgrp, portguid, pp_mcm_port)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_modify: " + "This is a new port in the MC group\n"); + return TRUE; + } + + /* We validate the request according the the proxy_join. + Check if the proxy_join is set or not */ + if ((*pp_mcm_port)->proxy_join == FALSE) { + /* The proxy_join is not set. Modifying can by done only + if the requester GID == PortGID */ + res = osm_get_gid_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + p_mad_addr, &request_gid); + + if (res != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_modify: " + "Could not find port for requested address\n"); + return FALSE; + } + + if (memcmp + (&((*pp_mcm_port)->port_gid), &request_gid, + sizeof(ib_gid_t))) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_modify: " + "No ProxyJoin but different ports: stored:0x%016" + PRIx64 " request:0x%016" PRIx64 "\n", + cl_ntoh64((*pp_mcm_port)->port_gid.unicast. + interface_id), + cl_ntoh64(p_mad_addr->addr_type.gsi.grh_info. + src_gid.unicast.interface_id) + ); + return FALSE; + } + } else { + /* The proxy_join is set. Modification allowed only if the + requester is part of the partition for this MCMemberRecord */ + p_request_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + p_mad_addr); + if (p_request_physp == NULL) + return FALSE; + + if (!osm_physp_has_pkey(p_rcv->p_log, p_mgrp->mcmember_rec.pkey, + p_request_physp)) { + /* the request port is not part of the partition for this mgrp */ + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_modify: " + "ProxyJoin but port not in partition. stored:0x%016" + PRIx64 " request:0x%016" PRIx64 "\n", + cl_ntoh64((*pp_mcm_port)->port_gid.unicast. + interface_id), + cl_ntoh64(p_mad_addr->addr_type.gsi.grh_info. + src_gid.unicast.interface_id) + ); + return FALSE; + } + } + return TRUE; } /********************************************************************** @@ -854,62 +786,58 @@ __validate_modify(IN osm_mcmr_recv_t* const p_rcv, * by the stored MCMemberRecord:P_Key. */ static boolean_t -__validate_delete(IN osm_mcmr_recv_t* const p_rcv, - IN osm_mgrp_t *p_mgrp, - IN osm_mad_addr_t* p_mad_addr, - IN ib_member_rec_t* p_recvd_mcmember_rec, - OUT osm_mcm_port_t **pp_mcm_port) +__validate_delete(IN osm_mcmr_recv_t * const p_rcv, + IN osm_mgrp_t * p_mgrp, + IN osm_mad_addr_t * p_mad_addr, + IN ib_member_rec_t * p_recvd_mcmember_rec, + OUT osm_mcm_port_t ** pp_mcm_port) { - ib_net64_t portguid; - - portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; - - *pp_mcm_port = NULL; - - /* 1 */ - if (!osm_mgrp_is_port_present(p_mgrp, portguid, pp_mcm_port)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_delete: " - "Failed to find the port in the MC group\n" ); - return FALSE; - } - - /* 2 */ - if (!(p_recvd_mcmember_rec->scope_state & 0x0F & - (*pp_mcm_port)->scope_state)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_delete: " - "Could not find any matching bits in the stored and requested JoinStates\n" ); - return FALSE; - } - - /* 3 */ - if ( ((p_recvd_mcmember_rec->scope_state & 0x0F) | - (0x0F & (*pp_mcm_port)->scope_state)) != - (0x0F & (*pp_mcm_port)->scope_state)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_delete: " - "Some bits in the request JoinState (0x%X) are not set in the stored port (0x%X)\n", - (p_recvd_mcmember_rec->scope_state & 0x0F), - (0x0F & (*pp_mcm_port)->scope_state) - ); - return FALSE; - } - - /* 4 */ - /* Validate according the the proxy_join (o15-0.1.2) */ - if ( __validate_modify( p_rcv, p_mgrp, p_mad_addr, p_recvd_mcmember_rec, - pp_mcm_port ) == FALSE ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_delete: " - "proxy_join validation failure\n" ); - return FALSE; - } - return TRUE; + ib_net64_t portguid; + + portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; + + *pp_mcm_port = NULL; + + /* 1 */ + if (!osm_mgrp_is_port_present(p_mgrp, portguid, pp_mcm_port)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_delete: " + "Failed to find the port in the MC group\n"); + return FALSE; + } + + /* 2 */ + if (!(p_recvd_mcmember_rec->scope_state & 0x0F & + (*pp_mcm_port)->scope_state)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_delete: " + "Could not find any matching bits in the stored and requested JoinStates\n"); + return FALSE; + } + + /* 3 */ + if (((p_recvd_mcmember_rec->scope_state & 0x0F) | + (0x0F & (*pp_mcm_port)->scope_state)) != + (0x0F & (*pp_mcm_port)->scope_state)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_delete: " + "Some bits in the request JoinState (0x%X) are not set in the stored port (0x%X)\n", + (p_recvd_mcmember_rec->scope_state & 0x0F), + (0x0F & (*pp_mcm_port)->scope_state) + ); + return FALSE; + } + + /* 4 */ + /* Validate according the the proxy_join (o15-0.1.2) */ + if (__validate_modify(p_rcv, p_mgrp, p_mad_addr, p_recvd_mcmember_rec, + pp_mcm_port) == FALSE) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_delete: " + "proxy_join validation failure\n"); + return FALSE; + } + return TRUE; } /********************************************************************** @@ -955,97 +883,92 @@ __validate_delete(IN osm_mcmr_recv_t* const p_rcv, * only source for this signature with link-local scope) */ ib_api_status_t -__validate_requested_mgid(IN osm_mcmr_recv_t* const p_rcv, - IN const ib_member_rec_t* p_mcm_rec) +__validate_requested_mgid(IN osm_mcmr_recv_t * const p_rcv, + IN const ib_member_rec_t * p_mcm_rec) { - uint16_t signature; - boolean_t valid = TRUE; - - OSM_LOG_ENTER( p_rcv->p_log, __validate_requested_mgid ); - - /* 14-a: mcast GID must start with 0xFF */ - if (p_mcm_rec->mgid.multicast.header[0] != 0xFF) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__validate_requested_mgid: ERR 1B01: " - "Wrong MGID Prefix 0x%02X must be 0xFF\n", - cl_ntoh16(p_mcm_rec->mgid.multicast.header[0]) - ); - valid = FALSE; - goto Exit; - } - - /* the MGID signature can mark IPoIB or SA assigned MGIDs */ - memcpy(&signature, &(p_mcm_rec->mgid.multicast.raw_group_id), sizeof(signature)); - signature = cl_ntoh16(signature); - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_requested_mgid: " - "MGID Signed as 0x%04X\n", - signature - ); - - /* - * We skip any checks for MGIDs that follow IPoIB - * GID structure as defined by the IETF ipoib-link-multicast. - * - * For IPv4 over IB, the signature will be "0x401B". - * - * | 8 | 4 | 4 | 16 bits | 16 bits | 48 bits | 32 bits | - * +--------+----+----+-----------------+---------+----------+---------+ - * |11111111|0001|scop||< P_Key >|00.......0|| - * +--------+----+----+-----------------+---------+----------+---------+ - * - * For IPv6 over IB, the signature will be "0x601B". - * - * | 8 | 4 | 4 | 16 bits | 16 bits | 80 bits | - * +--------+----+----+-----------------+---------+--------------------+ - * |11111111|0001|scop||< P_Key >|000.............0001| - * +--------+----+----+-----------------+---------+--------------------+ - * - */ - if (signature == 0x401B || signature == 0x601B) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_requested_mgid: " - "Skipping MGID Validation for IPoIB Signed (0x%04X) MGIDs\n", - signature - ); - goto Exit; - } - - /* 14-b: the 3 upper bits in the "flags" should be zero: */ - if ( p_mcm_rec->mgid.multicast.header[1] & 0xE0 ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__validate_requested_mgid: ERR 1B28: " - "MGID uses Reserved Flags: flags=0x%X\n", - (p_mcm_rec->mgid.multicast.header[1] & 0xE0) >> 4 - ); - valid = FALSE; - goto Exit; - } - - /* 2 - now what if the link local format 0xA01B is used - - the scope should not be link local */ - if ( ( signature == 0xA01B ) && - ((p_mcm_rec->mgid.multicast.header[1] & 0x0F) == MC_SCOPE_LINK_LOCAL) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__validate_requested_mgid: ERR 1B24: " - "MGID uses 0xA01B signature but with link-local scope\n" ); - valid = FALSE; - goto Exit; - } - - /* - * For SA assigned MGIDs (signature 0xA01B): - * There is no real way to make sure the Unique MGID Prefix is really unique. - * If we could enforce using the Subnet Prefix for that purpose it would - * have been nice. But the spec does not require it. - */ - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return (valid); + uint16_t signature; + boolean_t valid = TRUE; + + OSM_LOG_ENTER(p_rcv->p_log, __validate_requested_mgid); + + /* 14-a: mcast GID must start with 0xFF */ + if (p_mcm_rec->mgid.multicast.header[0] != 0xFF) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__validate_requested_mgid: ERR 1B01: " + "Wrong MGID Prefix 0x%02X must be 0xFF\n", + cl_ntoh16(p_mcm_rec->mgid.multicast.header[0]) + ); + valid = FALSE; + goto Exit; + } + + /* the MGID signature can mark IPoIB or SA assigned MGIDs */ + memcpy(&signature, &(p_mcm_rec->mgid.multicast.raw_group_id), + sizeof(signature)); + signature = cl_ntoh16(signature); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_requested_mgid: " + "MGID Signed as 0x%04X\n", signature); + + /* + * We skip any checks for MGIDs that follow IPoIB + * GID structure as defined by the IETF ipoib-link-multicast. + * + * For IPv4 over IB, the signature will be "0x401B". + * + * | 8 | 4 | 4 | 16 bits | 16 bits | 48 bits | 32 bits | + * +--------+----+----+-----------------+---------+----------+---------+ + * |11111111|0001|scop||< P_Key >|00.......0|| + * +--------+----+----+-----------------+---------+----------+---------+ + * + * For IPv6 over IB, the signature will be "0x601B". + * + * | 8 | 4 | 4 | 16 bits | 16 bits | 80 bits | + * +--------+----+----+-----------------+---------+--------------------+ + * |11111111|0001|scop||< P_Key >|000.............0001| + * +--------+----+----+-----------------+---------+--------------------+ + * + */ + if (signature == 0x401B || signature == 0x601B) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_requested_mgid: " + "Skipping MGID Validation for IPoIB Signed (0x%04X) MGIDs\n", + signature); + goto Exit; + } + + /* 14-b: the 3 upper bits in the "flags" should be zero: */ + if (p_mcm_rec->mgid.multicast.header[1] & 0xE0) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__validate_requested_mgid: ERR 1B28: " + "MGID uses Reserved Flags: flags=0x%X\n", + (p_mcm_rec->mgid.multicast.header[1] & 0xE0) >> 4); + valid = FALSE; + goto Exit; + } + + /* 2 - now what if the link local format 0xA01B is used - + the scope should not be link local */ + if ((signature == 0xA01B) && + ((p_mcm_rec->mgid.multicast.header[1] & 0x0F) == + MC_SCOPE_LINK_LOCAL)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__validate_requested_mgid: ERR 1B24: " + "MGID uses 0xA01B signature but with link-local scope\n"); + valid = FALSE; + goto Exit; + } + + /* + * For SA assigned MGIDs (signature 0xA01B): + * There is no real way to make sure the Unique MGID Prefix is really unique. + * If we could enforce using the Subnet Prefix for that purpose it would + * have been nice. But the spec does not require it. + */ + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (valid); } /********************************************************************** @@ -1053,325 +976,305 @@ __validate_requested_mgid(IN osm_mcmr_recv_t* const p_rcv, Also set the default MTU and Rate if not provided by the user. **********************************************************************/ boolean_t -__mgrp_request_is_realizable( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net64_t comp_mask, - IN ib_member_rec_t * p_mcm_rec, - IN const osm_physp_t* const p_physp) +__mgrp_request_is_realizable(IN osm_mcmr_recv_t * const p_rcv, + IN ib_net64_t comp_mask, + IN ib_member_rec_t * p_mcm_rec, + IN const osm_physp_t * const p_physp) { - uint8_t mtu_sel = 2; /* exactly */ - uint8_t mtu_required, mtu, port_mtu; - uint8_t rate_sel = 2; /* exactly */ - uint8_t rate_required, rate, port_rate; - osm_log_t *p_log = p_rcv->p_log; - - OSM_LOG_ENTER( p_rcv->p_log, __mgrp_request_is_realizable ); - - /* - * End of o15-0.2.3 specifies: - * .... - * The entity may also supply the other components such as HopLimit, MTU, - * etc. during group creation time. If these components are not provided - * during group creation time, SA will provide them for the group. The values - * chosen are vendor-dependent and beyond the scope of the specification. - * - * so we might also need to assign RATE/MTU if they are not comp masked in. - */ - - port_mtu = p_physp ? ib_port_info_get_mtu_cap(&p_physp->port_info) : 0; - if (!(comp_mask & IB_MCR_COMPMASK_MTU) || - !(comp_mask & IB_MCR_COMPMASK_MTU_SEL) || - (mtu_sel = (p_mcm_rec->mtu >> 6)) == 3) - mtu = port_mtu ? port_mtu : p_rcv->p_subn->min_ca_mtu; - else - { - mtu_required = (uint8_t)(p_mcm_rec->mtu & 0x3F); - mtu = mtu_required; - switch (mtu_sel) - { - case 0: /* Greater than MTU specified */ - if (port_mtu && mtu_required >= port_mtu) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__mgrp_request_is_realizable: " - "Requested MTU %x >= the port\'s mtu:%x\n", - mtu_required, port_mtu ); - return FALSE; - } - /* we provide the largest MTU possible if we can */ - if (port_mtu) - mtu = port_mtu; - else if (mtu_required < p_rcv->p_subn->min_ca_mtu) - mtu = p_rcv->p_subn->min_ca_mtu; - else - mtu++; - break; - case 1: /* Less than MTU specified */ - /* use the smaller of the two: - a. one lower then the required - b. the mtu of the requesting port (if exists) */ - if (port_mtu && mtu_required > port_mtu) - mtu = port_mtu; - else - mtu--; - break; - case 2: /* Exactly MTU specified */ - default: - break; - } - /* make sure it still be in the range */ - if (mtu < IB_MIN_MTU || mtu > IB_MAX_MTU) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__mgrp_request_is_realizable: " - "Calculated MTU %x is out of range\n", - mtu ); - return FALSE; - } - } - p_mcm_rec->mtu = (mtu_sel<<6) | mtu; - - port_rate = p_physp ? ib_port_info_compute_rate(&p_physp->port_info) : 0; - if (!(comp_mask & IB_MCR_COMPMASK_RATE) || - !(comp_mask & IB_MCR_COMPMASK_RATE_SEL) || - (rate_sel = (p_mcm_rec->rate >> 6)) == 3) - rate = port_rate ? port_rate : p_rcv->p_subn->min_ca_rate; - else - { - rate_required = (uint8_t)(p_mcm_rec->rate & 0x3F); - rate = rate_required; - switch (rate_sel) - { - case 0: /* Greater than RATE specified */ - if (port_rate && rate_required >= port_rate) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__mgrp_request_is_realizable: " - "Requested RATE %x >= the port\'s rate:%x\n", - rate_required, port_rate ); - return FALSE; - } - /* we provide the largest RATE possible if we can */ - if (port_rate) - rate = port_rate; - else if (rate_required < p_rcv->p_subn->min_ca_rate) - rate = p_rcv->p_subn->min_ca_rate; - else - rate++; - break; - case 1: /* Less than RATE specified */ - /* use the smaller of the two: - a. one lower then the required - b. the rate of the requesting port (if exists) */ - if (port_rate && rate_required > port_rate) - rate = port_rate; - else - rate--; - break; - case 2: /* Exactly RATE specified */ - default: - break; - } - /* make sure it still is in the range */ - if (rate < IB_MIN_RATE || rate > IB_MAX_RATE) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__mgrp_request_is_realizable: " - "Calculated RATE %x is out of range\n", - rate ); - return FALSE; - } - } - p_mcm_rec->rate = (rate_sel<<6) | rate; - - OSM_LOG_EXIT( p_rcv->p_log ); - return TRUE; + uint8_t mtu_sel = 2; /* exactly */ + uint8_t mtu_required, mtu, port_mtu; + uint8_t rate_sel = 2; /* exactly */ + uint8_t rate_required, rate, port_rate; + osm_log_t *p_log = p_rcv->p_log; + + OSM_LOG_ENTER(p_rcv->p_log, __mgrp_request_is_realizable); + + /* + * End of o15-0.2.3 specifies: + * .... + * The entity may also supply the other components such as HopLimit, MTU, + * etc. during group creation time. If these components are not provided + * during group creation time, SA will provide them for the group. The values + * chosen are vendor-dependent and beyond the scope of the specification. + * + * so we might also need to assign RATE/MTU if they are not comp masked in. + */ + + port_mtu = p_physp ? ib_port_info_get_mtu_cap(&p_physp->port_info) : 0; + if (!(comp_mask & IB_MCR_COMPMASK_MTU) || + !(comp_mask & IB_MCR_COMPMASK_MTU_SEL) || + (mtu_sel = (p_mcm_rec->mtu >> 6)) == 3) + mtu = port_mtu ? port_mtu : p_rcv->p_subn->min_ca_mtu; + else { + mtu_required = (uint8_t) (p_mcm_rec->mtu & 0x3F); + mtu = mtu_required; + switch (mtu_sel) { + case 0: /* Greater than MTU specified */ + if (port_mtu && mtu_required >= port_mtu) { + osm_log(p_log, OSM_LOG_DEBUG, + "__mgrp_request_is_realizable: " + "Requested MTU %x >= the port\'s mtu:%x\n", + mtu_required, port_mtu); + return FALSE; + } + /* we provide the largest MTU possible if we can */ + if (port_mtu) + mtu = port_mtu; + else if (mtu_required < p_rcv->p_subn->min_ca_mtu) + mtu = p_rcv->p_subn->min_ca_mtu; + else + mtu++; + break; + case 1: /* Less than MTU specified */ + /* use the smaller of the two: + a. one lower then the required + b. the mtu of the requesting port (if exists) */ + if (port_mtu && mtu_required > port_mtu) + mtu = port_mtu; + else + mtu--; + break; + case 2: /* Exactly MTU specified */ + default: + break; + } + /* make sure it still be in the range */ + if (mtu < IB_MIN_MTU || mtu > IB_MAX_MTU) { + osm_log(p_log, OSM_LOG_DEBUG, + "__mgrp_request_is_realizable: " + "Calculated MTU %x is out of range\n", mtu); + return FALSE; + } + } + p_mcm_rec->mtu = (mtu_sel << 6) | mtu; + + port_rate = + p_physp ? ib_port_info_compute_rate(&p_physp->port_info) : 0; + if (!(comp_mask & IB_MCR_COMPMASK_RATE) + || !(comp_mask & IB_MCR_COMPMASK_RATE_SEL) + || (rate_sel = (p_mcm_rec->rate >> 6)) == 3) + rate = port_rate ? port_rate : p_rcv->p_subn->min_ca_rate; + else { + rate_required = (uint8_t) (p_mcm_rec->rate & 0x3F); + rate = rate_required; + switch (rate_sel) { + case 0: /* Greater than RATE specified */ + if (port_rate && rate_required >= port_rate) { + osm_log(p_log, OSM_LOG_DEBUG, + "__mgrp_request_is_realizable: " + "Requested RATE %x >= the port\'s rate:%x\n", + rate_required, port_rate); + return FALSE; + } + /* we provide the largest RATE possible if we can */ + if (port_rate) + rate = port_rate; + else if (rate_required < p_rcv->p_subn->min_ca_rate) + rate = p_rcv->p_subn->min_ca_rate; + else + rate++; + break; + case 1: /* Less than RATE specified */ + /* use the smaller of the two: + a. one lower then the required + b. the rate of the requesting port (if exists) */ + if (port_rate && rate_required > port_rate) + rate = port_rate; + else + rate--; + break; + case 2: /* Exactly RATE specified */ + default: + break; + } + /* make sure it still is in the range */ + if (rate < IB_MIN_RATE || rate > IB_MAX_RATE) { + osm_log(p_log, OSM_LOG_DEBUG, + "__mgrp_request_is_realizable: " + "Calculated RATE %x is out of range\n", rate); + return FALSE; + } + } + p_mcm_rec->rate = (rate_sel << 6) | rate; + + OSM_LOG_EXIT(p_rcv->p_log); + return TRUE; } /********************************************************************** Call this function to find or create a new mgrp. **********************************************************************/ ib_api_status_t -osm_mcmr_rcv_find_or_create_new_mgrp( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net64_t comp_mask, - IN ib_member_rec_t* const p_recvd_mcmember_rec, - OUT osm_mgrp_t **pp_mgrp) +osm_mcmr_rcv_find_or_create_new_mgrp(IN osm_mcmr_recv_t * const p_rcv, + IN ib_net64_t comp_mask, + IN ib_member_rec_t * + const p_recvd_mcmember_rec, + OUT osm_mgrp_t ** pp_mgrp) { - ib_api_status_t status; - - status = __get_mgrp_by_mgid(p_rcv, p_recvd_mcmember_rec, pp_mgrp); - if (status == IB_SUCCESS) - return status; - return osm_mcmr_rcv_create_new_mgrp(p_rcv, comp_mask, - p_recvd_mcmember_rec, NULL, pp_mgrp); + ib_api_status_t status; + + status = __get_mgrp_by_mgid(p_rcv, p_recvd_mcmember_rec, pp_mgrp); + if (status == IB_SUCCESS) + return status; + return osm_mcmr_rcv_create_new_mgrp(p_rcv, comp_mask, + p_recvd_mcmember_rec, NULL, + pp_mgrp); } /********************************************************************** Call this function to create a new mgrp. **********************************************************************/ ib_api_status_t -osm_mcmr_rcv_create_new_mgrp( - IN osm_mcmr_recv_t* const p_rcv, - IN ib_net64_t comp_mask, - IN const ib_member_rec_t* const p_recvd_mcmember_rec, - IN const osm_physp_t* const p_physp, - OUT osm_mgrp_t **pp_mgrp) +osm_mcmr_rcv_create_new_mgrp(IN osm_mcmr_recv_t * const p_rcv, + IN ib_net64_t comp_mask, + IN const ib_member_rec_t * + const p_recvd_mcmember_rec, + IN const osm_physp_t * const p_physp, + OUT osm_mgrp_t ** pp_mgrp) { - ib_net16_t mlid; - uint8_t zero_mgid, valid; - uint8_t scope, i; - ib_gid_t *p_mgid; - osm_mgrp_t *p_prev_mgrp; - ib_api_status_t status = IB_SUCCESS; - ib_member_rec_t mcm_rec = *p_recvd_mcmember_rec; /* copy for modifications */ - - OSM_LOG_ENTER( p_rcv->p_log, osm_mcmr_rcv_create_new_mgrp ); - - /* but what if the given MGID was not 0 ? */ - zero_mgid = 1; - for ( i = 0 ; i < sizeof(p_recvd_mcmember_rec->mgid); i++ ) - { - if (p_recvd_mcmember_rec->mgid.raw[i] != 0) - { - zero_mgid = 0; - break; - } - } - - /* - we allocate a new mlid number before we might use it - for MGID ... - */ - mlid = __get_new_mlid(p_rcv, mcm_rec.mlid); - if ( mlid == 0 ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_create_new_mgrp: ERR 1B19: " - "__get_new_mlid failed\n" ); - status = IB_SA_MAD_STATUS_NO_RESOURCES; - goto Exit; - } - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_mcmr_rcv_create_new_mgrp: " - "Obtained new mlid 0x%X\n", cl_ntoh16(mlid) ); - - /* we need to create the new MGID if it was not defined */ - if (zero_mgid) - { - /* create a new MGID */ - - /* use the given scope state only if requested! */ - if (comp_mask & IB_MCR_COMPMASK_SCOPE) - { - ib_member_get_scope_state( - p_recvd_mcmember_rec->scope_state, &scope, NULL); - } - else - { - /* to guarantee no collision with other subnets use local scope! */ - scope = MC_SCOPE_LINK_LOCAL; - } - - p_mgid = &(mcm_rec.mgid); - p_mgid->raw[0] = 0xFF; - p_mgid->raw[1] = 0x10 | scope; - p_mgid->raw[2] = 0xA0; - p_mgid->raw[3] = 0x1B; - - /* HACK: use the SA port gid to make it globally unique */ - memcpy((&p_mgid->raw[4]), - &p_rcv->p_subn->opt.subnet_prefix, sizeof(uint64_t)); - - /* HACK: how do we get a unique number - use the mlid twice */ - memcpy(&p_mgid->raw[10], &mlid, sizeof(uint16_t)); - memcpy(&p_mgid->raw[12], &mlid, sizeof(uint16_t)); - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_mcmr_rcv_create_new_mgrp: " - "Allocated new MGID:0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh64(p_mgid->unicast.prefix), - cl_ntoh64(p_mgid->unicast.interface_id) ); - } - else - { - /* a specific MGID was requested so validate the resulting MGID */ - valid = __validate_requested_mgid(p_rcv, &mcm_rec); - if (!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_create_new_mgrp: ERR 1B22: " - "Invalid requested MGID\n" ); - __free_mlid(p_rcv, mlid); - status = IB_SA_MAD_STATUS_REQ_INVALID; - goto Exit; - } - } - - /* check the requested parameters are realizable */ - if (__mgrp_request_is_realizable(p_rcv, comp_mask, &mcm_rec, p_physp) == FALSE) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_create_new_mgrp: ERR 1B26: " - "Requested MGRP parameters are not realizable\n" ); - __free_mlid(p_rcv, mlid); - status = IB_SA_MAD_STATUS_REQ_INVALID; - goto Exit; - } - - /* create a new MC Group */ - *pp_mgrp = osm_mgrp_new(mlid); - if (*pp_mgrp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_create_new_mgrp: ERR 1B08: " - "osm_mgrp_new failed\n" ); - __free_mlid(p_rcv, mlid); - status = IB_SA_MAD_STATUS_NO_RESOURCES; - goto Exit; - } - - /* Initialize the mgrp */ - (*pp_mgrp)->mcmember_rec = mcm_rec; - (*pp_mgrp)->mcmember_rec.mlid = mlid; - - /* the mcmember_record should have mtu_sel, rate_sel, and pkt_lifetime_sel = 2 */ - (*pp_mgrp)->mcmember_rec.mtu &= 0x3f; - (*pp_mgrp)->mcmember_rec.mtu |= 2<<6; /* exactly */ - (*pp_mgrp)->mcmember_rec.rate &= 0x3f; - (*pp_mgrp)->mcmember_rec.rate |= 2<<6; /* exactly */ - (*pp_mgrp)->mcmember_rec.pkt_life &= 0x3f; - (*pp_mgrp)->mcmember_rec.pkt_life |= 2<<6; /* exactly */ - - /* Insert the new group in the data base */ - - /* since we might have an old group by that mlid - one whose deletion was delayed for an idle time - we need to deallocate it first */ - p_prev_mgrp = (osm_mgrp_t *)cl_qmap_get(&p_rcv->p_subn->mgrp_mlid_tbl, mlid); - if (p_prev_mgrp != (osm_mgrp_t *)cl_qmap_end(&p_rcv->p_subn->mgrp_mlid_tbl)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_mcmr_rcv_create_new_mgrp: " - "Found previous group for mlid:0x%04x - Need to destroy it\n", - cl_ntoh16(mlid) ); - cl_qmap_remove_item(&p_rcv->p_subn->mgrp_mlid_tbl, - (cl_map_item_t *)p_prev_mgrp ); - osm_mgrp_delete( p_prev_mgrp ); - } - - cl_qmap_insert(&p_rcv->p_subn->mgrp_mlid_tbl, - mlid, - &(*pp_mgrp)->map_item); - - /* Send a Report to any InformInfo registerd for - Trap 66: MCGroup create */ - osm_mgrp_send_create_notice(p_rcv->p_subn, p_rcv->p_log, *pp_mgrp); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return status; + ib_net16_t mlid; + uint8_t zero_mgid, valid; + uint8_t scope, i; + ib_gid_t *p_mgid; + osm_mgrp_t *p_prev_mgrp; + ib_api_status_t status = IB_SUCCESS; + ib_member_rec_t mcm_rec = *p_recvd_mcmember_rec; /* copy for modifications */ + + OSM_LOG_ENTER(p_rcv->p_log, osm_mcmr_rcv_create_new_mgrp); + + /* but what if the given MGID was not 0 ? */ + zero_mgid = 1; + for (i = 0; i < sizeof(p_recvd_mcmember_rec->mgid); i++) { + if (p_recvd_mcmember_rec->mgid.raw[i] != 0) { + zero_mgid = 0; + break; + } + } + + /* + we allocate a new mlid number before we might use it + for MGID ... + */ + mlid = __get_new_mlid(p_rcv, mcm_rec.mlid); + if (mlid == 0) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_create_new_mgrp: ERR 1B19: " + "__get_new_mlid failed\n"); + status = IB_SA_MAD_STATUS_NO_RESOURCES; + goto Exit; + } + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_mcmr_rcv_create_new_mgrp: " + "Obtained new mlid 0x%X\n", cl_ntoh16(mlid)); + + /* we need to create the new MGID if it was not defined */ + if (zero_mgid) { + /* create a new MGID */ + + /* use the given scope state only if requested! */ + if (comp_mask & IB_MCR_COMPMASK_SCOPE) { + ib_member_get_scope_state(p_recvd_mcmember_rec-> + scope_state, &scope, NULL); + } else { + /* to guarantee no collision with other subnets use local scope! */ + scope = MC_SCOPE_LINK_LOCAL; + } + + p_mgid = &(mcm_rec.mgid); + p_mgid->raw[0] = 0xFF; + p_mgid->raw[1] = 0x10 | scope; + p_mgid->raw[2] = 0xA0; + p_mgid->raw[3] = 0x1B; + + /* HACK: use the SA port gid to make it globally unique */ + memcpy((&p_mgid->raw[4]), + &p_rcv->p_subn->opt.subnet_prefix, sizeof(uint64_t)); + + /* HACK: how do we get a unique number - use the mlid twice */ + memcpy(&p_mgid->raw[10], &mlid, sizeof(uint16_t)); + memcpy(&p_mgid->raw[12], &mlid, sizeof(uint16_t)); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_mcmr_rcv_create_new_mgrp: " + "Allocated new MGID:0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n", + cl_ntoh64(p_mgid->unicast.prefix), + cl_ntoh64(p_mgid->unicast.interface_id)); + } else { + /* a specific MGID was requested so validate the resulting MGID */ + valid = __validate_requested_mgid(p_rcv, &mcm_rec); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_create_new_mgrp: ERR 1B22: " + "Invalid requested MGID\n"); + __free_mlid(p_rcv, mlid); + status = IB_SA_MAD_STATUS_REQ_INVALID; + goto Exit; + } + } + + /* check the requested parameters are realizable */ + if (__mgrp_request_is_realizable(p_rcv, comp_mask, &mcm_rec, p_physp) == + FALSE) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_create_new_mgrp: ERR 1B26: " + "Requested MGRP parameters are not realizable\n"); + __free_mlid(p_rcv, mlid); + status = IB_SA_MAD_STATUS_REQ_INVALID; + goto Exit; + } + + /* create a new MC Group */ + *pp_mgrp = osm_mgrp_new(mlid); + if (*pp_mgrp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_create_new_mgrp: ERR 1B08: " + "osm_mgrp_new failed\n"); + __free_mlid(p_rcv, mlid); + status = IB_SA_MAD_STATUS_NO_RESOURCES; + goto Exit; + } + + /* Initialize the mgrp */ + (*pp_mgrp)->mcmember_rec = mcm_rec; + (*pp_mgrp)->mcmember_rec.mlid = mlid; + + /* the mcmember_record should have mtu_sel, rate_sel, and pkt_lifetime_sel = 2 */ + (*pp_mgrp)->mcmember_rec.mtu &= 0x3f; + (*pp_mgrp)->mcmember_rec.mtu |= 2 << 6; /* exactly */ + (*pp_mgrp)->mcmember_rec.rate &= 0x3f; + (*pp_mgrp)->mcmember_rec.rate |= 2 << 6; /* exactly */ + (*pp_mgrp)->mcmember_rec.pkt_life &= 0x3f; + (*pp_mgrp)->mcmember_rec.pkt_life |= 2 << 6; /* exactly */ + + /* Insert the new group in the data base */ + + /* since we might have an old group by that mlid + one whose deletion was delayed for an idle time + we need to deallocate it first */ + p_prev_mgrp = + (osm_mgrp_t *) cl_qmap_get(&p_rcv->p_subn->mgrp_mlid_tbl, mlid); + if (p_prev_mgrp != + (osm_mgrp_t *) cl_qmap_end(&p_rcv->p_subn->mgrp_mlid_tbl)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_mcmr_rcv_create_new_mgrp: " + "Found previous group for mlid:0x%04x - Need to destroy it\n", + cl_ntoh16(mlid)); + cl_qmap_remove_item(&p_rcv->p_subn->mgrp_mlid_tbl, + (cl_map_item_t *) p_prev_mgrp); + osm_mgrp_delete(p_prev_mgrp); + } + + cl_qmap_insert(&p_rcv->p_subn->mgrp_mlid_tbl, + mlid, &(*pp_mgrp)->map_item); + + /* Send a Report to any InformInfo registerd for + Trap 66: MCGroup create */ + osm_mgrp_send_create_notice(p_rcv->p_subn, p_rcv->p_log, *pp_mgrp); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return status; } @@ -1379,998 +1282,967 @@ osm_mcmr_rcv_create_new_mgrp( Process a request for leaving the group **********************************************************************/ static void -__osm_mcmr_rcv_leave_mgrp( - IN osm_mcmr_recv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__osm_mcmr_rcv_leave_mgrp(IN osm_mcmr_recv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - boolean_t valid; - osm_mgrp_t *p_mgrp; - ib_api_status_t status; - ib_sa_mad_t *p_sa_mad; - ib_member_rec_t*p_recvd_mcmember_rec; - ib_member_rec_t mcmember_rec; - ib_net16_t mlid; - ib_net16_t sa_status; - ib_net64_t portguid; - osm_mcm_port_t *p_mcm_port; - uint8_t port_join_state; - uint8_t new_join_state; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mcmr_rcv_leave_mgrp ); - - p_mgrp = NULL; - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_mcmember_rec = - (ib_member_rec_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - mcmember_rec = *p_recvd_mcmember_rec; - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_leave_mgrp: Dump of record\n" ); - osm_dump_mc_record( p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG ); - } - - CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); - status = __get_mgrp_by_mgid(p_rcv,p_recvd_mcmember_rec, &p_mgrp); - if (status == IB_SUCCESS) - { - mlid = p_mgrp->mlid; - portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; - - /* check validity of the delete request o15-0.1.14 */ - valid = __validate_delete(p_rcv, - p_mgrp, - osm_madw_get_mad_addr_ptr(p_madw), - p_recvd_mcmember_rec, - &p_mcm_port); - - if (valid) - { - /* - * according to the same o15-0.1.14 we get the stored JoinState and the - * request JoinState and they must be opposite to leave - - * otherwise just update it - */ - port_join_state = p_mcm_port->scope_state & 0x0F; - new_join_state = - port_join_state & ~(p_recvd_mcmember_rec->scope_state & 0x0F); - if (new_join_state) - { - /* Just update the result JoinState */ - p_mcm_port->scope_state = - new_join_state | (p_mcm_port->scope_state & 0xf0); - - mcmember_rec.scope_state = p_mcm_port->scope_state; - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_leave_mgrp: " - "After update JoinState != 0. Updating from 0x%X to 0x%X\n", - port_join_state, - new_join_state - ); - } - else - { - /* we need to return the stored scope state */ - mcmember_rec.scope_state = p_mcm_port->scope_state; - - /* OK we can leave */ - /* note: osm_sm_mcgrp_leave() will release p_rcv->p_lock */ - - status = osm_sm_mcgrp_leave(p_rcv->p_sm, mlid, portguid); - if(status != IB_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_leave_mgrp: ERR 1B09: " - "osm_sm_mcgrp_leave failed\n" ); - } - } - } - else - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_leave_mgrp: ERR 1B25: " - "Received an invalid delete request for " - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " for " - "PortGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.interface_id ), - cl_ntoh64( p_recvd_mcmember_rec->port_gid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->port_gid.unicast.interface_id ) ); - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - } - else - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_leave_mgrp: " - "Failed since multicast group not present\n" ); - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* Send an SA response */ - __osm_mcmr_rcv_respond( p_rcv, p_madw, &mcmember_rec ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + boolean_t valid; + osm_mgrp_t *p_mgrp; + ib_api_status_t status; + ib_sa_mad_t *p_sa_mad; + ib_member_rec_t *p_recvd_mcmember_rec; + ib_member_rec_t mcmember_rec; + ib_net16_t mlid; + ib_net16_t sa_status; + ib_net64_t portguid; + osm_mcm_port_t *p_mcm_port; + uint8_t port_join_state; + uint8_t new_join_state; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mcmr_rcv_leave_mgrp); + + p_mgrp = NULL; + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_mcmember_rec = + (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + mcmember_rec = *p_recvd_mcmember_rec; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_leave_mgrp: Dump of record\n"); + osm_dump_mc_record(p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG); + } + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + status = __get_mgrp_by_mgid(p_rcv, p_recvd_mcmember_rec, &p_mgrp); + if (status == IB_SUCCESS) { + mlid = p_mgrp->mlid; + portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; + + /* check validity of the delete request o15-0.1.14 */ + valid = __validate_delete(p_rcv, + p_mgrp, + osm_madw_get_mad_addr_ptr(p_madw), + p_recvd_mcmember_rec, &p_mcm_port); + + if (valid) { + /* + * according to the same o15-0.1.14 we get the stored JoinState and the + * request JoinState and they must be opposite to leave - + * otherwise just update it + */ + port_join_state = p_mcm_port->scope_state & 0x0F; + new_join_state = + port_join_state & ~(p_recvd_mcmember_rec-> + scope_state & 0x0F); + if (new_join_state) { + /* Just update the result JoinState */ + p_mcm_port->scope_state = + new_join_state | (p_mcm_port-> + scope_state & 0xf0); + + mcmember_rec.scope_state = + p_mcm_port->scope_state; + + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_leave_mgrp: " + "After update JoinState != 0. Updating from 0x%X to 0x%X\n", + port_join_state, new_join_state); + } else { + /* we need to return the stored scope state */ + mcmember_rec.scope_state = + p_mcm_port->scope_state; + + /* OK we can leave */ + /* note: osm_sm_mcgrp_leave() will release p_rcv->p_lock */ + + status = + osm_sm_mcgrp_leave(p_rcv->p_sm, mlid, + portguid); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_leave_mgrp: ERR 1B09: " + "osm_sm_mcgrp_leave failed\n"); + } + } + } else { + CL_PLOCK_RELEASE(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_leave_mgrp: ERR 1B25: " + "Received an invalid delete request for " + "MGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 " for " + "PortGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n", + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + prefix), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + interface_id), + cl_ntoh64(p_recvd_mcmember_rec->port_gid. + unicast.prefix), + cl_ntoh64(p_recvd_mcmember_rec->port_gid. + unicast.interface_id)); + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + } else { + CL_PLOCK_RELEASE(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_leave_mgrp: " + "Failed since multicast group not present\n"); + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* Send an SA response */ + __osm_mcmr_rcv_respond(p_rcv, p_madw, &mcmember_rec); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** Handle a join (or create) request **********************************************************************/ static void -__osm_mcmr_rcv_join_mgrp( - IN osm_mcmr_recv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__osm_mcmr_rcv_join_mgrp(IN osm_mcmr_recv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - boolean_t valid; - osm_mgrp_t *p_mgrp = NULL; - ib_api_status_t status; - ib_sa_mad_t *p_sa_mad; - ib_member_rec_t*p_recvd_mcmember_rec; - ib_member_rec_t mcmember_rec; - ib_net16_t sa_status; - ib_net16_t mlid; - osm_mcm_port_t *p_mcmr_port; - ib_net64_t portguid; - osm_port_t * p_port; - osm_physp_t* p_physp; - osm_physp_t* p_request_physp; - uint8_t is_new_group; /* TRUE = there is a need to create a group */ - osm_mcast_req_type_t req_type; - uint8_t join_state; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mcmr_rcv_join_mgrp ); - - p_mgrp = NULL; - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_mcmember_rec = - (ib_member_rec_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; - - mcmember_rec = *p_recvd_mcmember_rec; - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_join_mgrp: " - "Dump of incoming record\n" ); - osm_dump_mc_record( p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG ); - } - - CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); - - /* make sure the requested port guid is known to the SM */ - p_port = osm_get_port_by_guid(p_rcv->p_subn, portguid); - if (!p_port) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_join_mgrp: " - "Unknown port GUID 0x%016" PRIx64 "\n", - portguid ); - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - p_physp = p_port->p_physp; - /* Check that the p_physp and the requester physp are in the same - partition. */ - p_request_physp = - osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_request_physp == NULL) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - goto Exit; - } - - if (!osm_physp_share_pkey( p_rcv->p_log, p_physp, p_request_physp)) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_rcv_join_mgrp: " - "Port and requester don't share pkey\n" ); - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - ib_member_get_scope_state( - p_recvd_mcmember_rec->scope_state, NULL, &join_state); - - /* do we need to create a new group? */ - status = __get_mgrp_by_mgid(p_rcv, p_recvd_mcmember_rec, &p_mgrp); - if ((status == IB_NOT_FOUND) || p_mgrp->to_be_deleted) - { - /* check for JoinState.FullMember = 1 o15.0.1.9 */ - if ((join_state & 0x01) != 0x01) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_join_mgrp: ERR 1B10: " - "Provided Join State != FullMember - required for create, " - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " from port 0x%016" PRIx64 " (%s)\n", - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.interface_id ), - cl_ntoh64( portguid ), - p_port->p_node->print_desc); - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* check the comp_mask */ - valid = __check_create_comp_mask(p_sa_mad->comp_mask, - p_recvd_mcmember_rec); - if (valid) - { - status = osm_mcmr_rcv_create_new_mgrp(p_rcv, - p_sa_mad->comp_mask, - p_recvd_mcmember_rec, - p_physp, - &p_mgrp); - if (status != IB_SUCCESS) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - sa_status = status; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - /* copy the MGID to the result */ - mcmember_rec.mgid = p_mgrp->mcmember_rec.mgid; - } - else - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_join_mgrp: ERR 1B11: " - "method = %s, " - "scope_state = 0x%x, " - "component mask = 0x%016" PRIx64 ", " - "expected comp mask = 0x%016" PRIx64 ", " - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " from port 0x%016" PRIx64 " (%s)\n", - ib_get_sa_method_str(p_sa_mad->method), - p_recvd_mcmember_rec->scope_state, - cl_ntoh64(p_sa_mad->comp_mask), - CL_NTOH64(REQUIRED_MC_CREATE_COMP_MASK), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.interface_id ), - cl_ntoh64( portguid ), - p_port->p_node->print_desc); - - sa_status = IB_SA_MAD_STATUS_INSUF_COMPS; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - is_new_group = 1; - req_type = OSM_MCAST_REQ_TYPE_CREATE; - } - else - { - /* no need for a new group */ - is_new_group = 0; - req_type = OSM_MCAST_REQ_TYPE_JOIN; - } - - CL_ASSERT(p_mgrp); - mlid = p_mgrp->mlid; - - /* - * o15-0.2.4: If SA supports UD multicast, then SA shall cause an - * endport to join an existing multicast group if: - * 1. It receives a SubnAdmSet() method for a MCMemberRecord, and - * - WE KNOW THAT ALREADY - * 2. The MGID is specified and matches an existing multicast - * group, and - * - WE KNOW THAT ALREADY - * 3. The MCMemberRecord:JoinState is not all 0s, and - * 4. PortGID is specified and - * - WE KNOW THAT ALREADY (as it matched a real one) - * 5. All other components match that existing group, either by - * being wildcarded or by having values identical to those specified - * by the component mask and in use by the group with the exception - * of components such as ProxyJoin and Reserved, which are ignored - * by SA. - * - * We need to check #3 and #5 here: - */ - valid = __validate_more_comp_fields( - p_rcv->p_log, - p_mgrp, - p_recvd_mcmember_rec, - p_sa_mad->comp_mask) && __validate_port_caps( - p_rcv->p_log, - p_mgrp, - p_physp) && (join_state != 0); - - if (!valid) - { - /* since we might have created the new group we need to cleanup */ - __cleanup_mgrp(p_rcv, mlid); - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_join_mgrp: ERR 1B12: " - "__validate_more_comp_fields, __validate_port_caps, " - "or JoinState = 0 failed from port 0x%016" PRIx64 " (%s), " - "sending IB_SA_MAD_STATUS_REQ_INVALID\n", - cl_ntoh64( portguid ), - p_port->p_node->print_desc); - - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* - * Do some validation of the modification - */ - if (!is_new_group) - { - /* - * o15-0.2.1 requires validation of the requesting port - * in the case of modification: - */ - valid = __validate_modify(p_rcv, - p_mgrp, - osm_madw_get_mad_addr_ptr(p_madw), - p_recvd_mcmember_rec, - &p_mcmr_port); - if (!valid) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_join_mgrp: ERR 1B13: " - "__validate_modify failed from port 0x%016" PRIx64 " (%s), " - "sending IB_SA_MAD_STATUS_REQ_INVALID\n", - cl_ntoh64( portguid ), - p_port->p_node->print_desc); - - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - } - - /* create or update existing port (join-state will be updated) */ - status = __add_new_mgrp_port( - p_rcv, - p_mgrp, - p_recvd_mcmember_rec, - osm_madw_get_mad_addr_ptr(p_madw), - &p_mcmr_port); - - if (status != IB_SUCCESS) - { - /* we fail to add the port so we might need to delete the group */ - __cleanup_mgrp(p_rcv, mlid); - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - if (status == IB_INVALID_PARAMETER) - sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - else - sa_status = IB_SA_MAD_STATUS_NO_RESOURCES; - - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* o15.0.1.11: copy the join state */ - mcmember_rec.scope_state = p_mcmr_port->scope_state; - - /* copy qkey mlid tclass pkey sl_flow_hop mtu rate pkt_life sl_flow_hop */ - __copy_from_create_mc_rec(&mcmember_rec, &p_mgrp->mcmember_rec); - - /* Release the lock as we don't need it. */ - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - /* do the actual routing (actually schedule the update) */ - status = - osm_sm_mcgrp_join(p_rcv->p_sm, - mlid, - p_recvd_mcmember_rec->port_gid.unicast.interface_id, - req_type); - - if (status != IB_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_join_mgrp: ERR 1B14: " - "osm_sm_mcgrp_join failed from port 0x%016" PRIx64 " (%s), " - "sending IB_SA_MAD_STATUS_NO_RESOURCES\n", - cl_ntoh64( portguid ), - p_port->p_node->print_desc); - - CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); - - /* the request for routing failed so we need to remove the port */ - p_mgrp = __get_mgrp_by_mlid(p_rcv, mlid); - if (p_mgrp != NULL) - { - osm_mgrp_remove_port( - p_rcv->p_subn, - p_rcv->p_log, - p_mgrp, - p_recvd_mcmember_rec->port_gid.unicast.interface_id); - __cleanup_mgrp(p_rcv, mlid); - } - CL_PLOCK_RELEASE( p_rcv->p_lock ); - sa_status = IB_SA_MAD_STATUS_NO_RESOURCES; - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - - } /* failed to route */ - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_mc_record( p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG ); - - __osm_mcmr_rcv_respond( p_rcv, p_madw, &mcmember_rec ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + boolean_t valid; + osm_mgrp_t *p_mgrp = NULL; + ib_api_status_t status; + ib_sa_mad_t *p_sa_mad; + ib_member_rec_t *p_recvd_mcmember_rec; + ib_member_rec_t mcmember_rec; + ib_net16_t sa_status; + ib_net16_t mlid; + osm_mcm_port_t *p_mcmr_port; + ib_net64_t portguid; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_physp_t *p_request_physp; + uint8_t is_new_group; /* TRUE = there is a need to create a group */ + osm_mcast_req_type_t req_type; + uint8_t join_state; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mcmr_rcv_join_mgrp); + + p_mgrp = NULL; + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_mcmember_rec = + (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + portguid = p_recvd_mcmember_rec->port_gid.unicast.interface_id; + + mcmember_rec = *p_recvd_mcmember_rec; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_join_mgrp: " + "Dump of incoming record\n"); + osm_dump_mc_record(p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG); + } + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + + /* make sure the requested port guid is known to the SM */ + p_port = osm_get_port_by_guid(p_rcv->p_subn, portguid); + if (!p_port) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_join_mgrp: " + "Unknown port GUID 0x%016" PRIx64 "\n", portguid); + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + p_physp = p_port->p_physp; + /* Check that the p_physp and the requester physp are in the same + partition. */ + p_request_physp = + osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr(p_madw)); + if (p_request_physp == NULL) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + goto Exit; + } + + if (!osm_physp_share_pkey(p_rcv->p_log, p_physp, p_request_physp)) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_rcv_join_mgrp: " + "Port and requester don't share pkey\n"); + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + ib_member_get_scope_state(p_recvd_mcmember_rec->scope_state, NULL, + &join_state); + + /* do we need to create a new group? */ + status = __get_mgrp_by_mgid(p_rcv, p_recvd_mcmember_rec, &p_mgrp); + if ((status == IB_NOT_FOUND) || p_mgrp->to_be_deleted) { + /* check for JoinState.FullMember = 1 o15.0.1.9 */ + if ((join_state & 0x01) != 0x01) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_join_mgrp: ERR 1B10: " + "Provided Join State != FullMember - required for create, " + "MGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 " from port 0x%016" PRIx64 + " (%s)\n", + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + prefix), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + interface_id), cl_ntoh64(portguid), + p_port->p_node->print_desc); + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* check the comp_mask */ + valid = __check_create_comp_mask(p_sa_mad->comp_mask, + p_recvd_mcmember_rec); + if (valid) { + status = osm_mcmr_rcv_create_new_mgrp(p_rcv, + p_sa_mad-> + comp_mask, + p_recvd_mcmember_rec, + p_physp, &p_mgrp); + if (status != IB_SUCCESS) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + sa_status = status; + osm_sa_send_error(p_rcv->p_resp, p_madw, + sa_status); + goto Exit; + } + /* copy the MGID to the result */ + mcmember_rec.mgid = p_mgrp->mcmember_rec.mgid; + } else { + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_join_mgrp: ERR 1B11: " + "method = %s, " + "scope_state = 0x%x, " + "component mask = 0x%016" PRIx64 ", " + "expected comp mask = 0x%016" PRIx64 ", " + "MGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 " from port 0x%016" PRIx64 + " (%s)\n", + ib_get_sa_method_str(p_sa_mad->method), + p_recvd_mcmember_rec->scope_state, + cl_ntoh64(p_sa_mad->comp_mask), + CL_NTOH64(REQUIRED_MC_CREATE_COMP_MASK), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + prefix), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + interface_id), cl_ntoh64(portguid), + p_port->p_node->print_desc); + + sa_status = IB_SA_MAD_STATUS_INSUF_COMPS; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + is_new_group = 1; + req_type = OSM_MCAST_REQ_TYPE_CREATE; + } else { + /* no need for a new group */ + is_new_group = 0; + req_type = OSM_MCAST_REQ_TYPE_JOIN; + } + + CL_ASSERT(p_mgrp); + mlid = p_mgrp->mlid; + + /* + * o15-0.2.4: If SA supports UD multicast, then SA shall cause an + * endport to join an existing multicast group if: + * 1. It receives a SubnAdmSet() method for a MCMemberRecord, and + * - WE KNOW THAT ALREADY + * 2. The MGID is specified and matches an existing multicast + * group, and + * - WE KNOW THAT ALREADY + * 3. The MCMemberRecord:JoinState is not all 0s, and + * 4. PortGID is specified and + * - WE KNOW THAT ALREADY (as it matched a real one) + * 5. All other components match that existing group, either by + * being wildcarded or by having values identical to those specified + * by the component mask and in use by the group with the exception + * of components such as ProxyJoin and Reserved, which are ignored + * by SA. + * + * We need to check #3 and #5 here: + */ + valid = __validate_more_comp_fields(p_rcv->p_log, + p_mgrp, + p_recvd_mcmember_rec, + p_sa_mad->comp_mask) + && __validate_port_caps(p_rcv->p_log, p_mgrp, p_physp) + && (join_state != 0); + + if (!valid) { + /* since we might have created the new group we need to cleanup */ + __cleanup_mgrp(p_rcv, mlid); + + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_join_mgrp: ERR 1B12: " + "__validate_more_comp_fields, __validate_port_caps, " + "or JoinState = 0 failed from port 0x%016" PRIx64 + " (%s), " "sending IB_SA_MAD_STATUS_REQ_INVALID\n", + cl_ntoh64(portguid), p_port->p_node->print_desc); + + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* + * Do some validation of the modification + */ + if (!is_new_group) { + /* + * o15-0.2.1 requires validation of the requesting port + * in the case of modification: + */ + valid = __validate_modify(p_rcv, + p_mgrp, + osm_madw_get_mad_addr_ptr(p_madw), + p_recvd_mcmember_rec, &p_mcmr_port); + if (!valid) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_join_mgrp: ERR 1B13: " + "__validate_modify failed from port 0x%016" + PRIx64 " (%s), " + "sending IB_SA_MAD_STATUS_REQ_INVALID\n", + cl_ntoh64(portguid), + p_port->p_node->print_desc); + + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + } + + /* create or update existing port (join-state will be updated) */ + status = __add_new_mgrp_port(p_rcv, + p_mgrp, + p_recvd_mcmember_rec, + osm_madw_get_mad_addr_ptr(p_madw), + &p_mcmr_port); + + if (status != IB_SUCCESS) { + /* we fail to add the port so we might need to delete the group */ + __cleanup_mgrp(p_rcv, mlid); + + CL_PLOCK_RELEASE(p_rcv->p_lock); + if (status == IB_INVALID_PARAMETER) + sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + else + sa_status = IB_SA_MAD_STATUS_NO_RESOURCES; + + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* o15.0.1.11: copy the join state */ + mcmember_rec.scope_state = p_mcmr_port->scope_state; + + /* copy qkey mlid tclass pkey sl_flow_hop mtu rate pkt_life sl_flow_hop */ + __copy_from_create_mc_rec(&mcmember_rec, &p_mgrp->mcmember_rec); + + /* Release the lock as we don't need it. */ + CL_PLOCK_RELEASE(p_rcv->p_lock); + + /* do the actual routing (actually schedule the update) */ + status = + osm_sm_mcgrp_join(p_rcv->p_sm, + mlid, + p_recvd_mcmember_rec->port_gid.unicast. + interface_id, req_type); + + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_join_mgrp: ERR 1B14: " + "osm_sm_mcgrp_join failed from port 0x%016" PRIx64 + " (%s), " "sending IB_SA_MAD_STATUS_NO_RESOURCES\n", + cl_ntoh64(portguid), p_port->p_node->print_desc); + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + + /* the request for routing failed so we need to remove the port */ + p_mgrp = __get_mgrp_by_mlid(p_rcv, mlid); + if (p_mgrp != NULL) { + osm_mgrp_remove_port(p_rcv->p_subn, + p_rcv->p_log, + p_mgrp, + p_recvd_mcmember_rec->port_gid. + unicast.interface_id); + __cleanup_mgrp(p_rcv, mlid); + } + CL_PLOCK_RELEASE(p_rcv->p_lock); + sa_status = IB_SA_MAD_STATUS_NO_RESOURCES; + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + + } + /* failed to route */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_mc_record(p_rcv->p_log, &mcmember_rec, OSM_LOG_DEBUG); + + __osm_mcmr_rcv_respond(p_rcv, p_madw, &mcmember_rec); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** Add a patched multicast group to the results list **********************************************************************/ static ib_api_status_t -__osm_mcmr_rcv_new_mcmr( - IN osm_mcmr_recv_t* const p_rcv, - IN const ib_member_rec_t* p_rcvd_rec, - IN cl_qlist_t* const p_list ) +__osm_mcmr_rcv_new_mcmr(IN osm_mcmr_recv_t * const p_rcv, + IN const ib_member_rec_t * p_rcvd_rec, + IN cl_qlist_t * const p_list) { - osm_mcmr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mcmr_rcv_new_mcmr ); - - p_rec_item = (osm_mcmr_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_rcv_new_mcmr: ERR 1B15: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - memset( &p_rec_item->rec, 0, sizeof( p_rec_item->rec ) ); - - /* HACK: Untrusted requesters should result with 0 Join - State, Port Guid, and Proxy */ - p_rec_item->rec = *p_rcvd_rec; - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_mcmr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mcmr_rcv_new_mcmr); + + p_rec_item = (osm_mcmr_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_rcv_new_mcmr: ERR 1B15: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + memset(&p_rec_item->rec, 0, sizeof(p_rec_item->rec)); + + /* HACK: Untrusted requesters should result with 0 Join + State, Port Guid, and Proxy */ + p_rec_item->rec = *p_rcvd_rec; + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** Match the given mgrp to the requested mcmr **********************************************************************/ void -__osm_sa_mcm_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_mcm_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_mgrp_t * const p_mgrp = (osm_mgrp_t *)p_map_item; - osm_sa_mcmr_search_ctxt_t* const p_ctxt = - (osm_sa_mcmr_search_ctxt_t *)context; - osm_mcmr_recv_t* const p_rcv = p_ctxt->p_rcv; - const ib_member_rec_t* p_rcvd_rec = p_ctxt->p_mcmember_rec; - const osm_physp_t* p_req_physp = p_ctxt->p_req_physp; - - /* since we might change scope_state */ - ib_member_rec_t match_rec; - ib_net64_t comp_mask = p_ctxt->comp_mask; - osm_mcm_port_t* p_mcm_port; - ib_net64_t portguid = p_rcvd_rec->port_gid.unicast.interface_id; - /* will be used for group or port info */ - uint8_t scope_state; - uint8_t scope_state_mask = 0; - cl_map_item_t *p_item; - ib_gid_t port_gid; - boolean_t proxy_join; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_mcm_by_comp_mask_cb ); - - osm_log(p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_mcm_by_comp_mask_cb: " - "Checking mlid:0x%X\n", - cl_ntoh16(p_mgrp->mlid)); - - /* the group might be marked for deletion */ - if (p_mgrp->to_be_deleted) - { - osm_log(p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_mcm_by_comp_mask_cb: " - "Group mlid:0x%X is marked to be deleted\n", - cl_ntoh16(p_mgrp->mlid)); - goto Exit; - } - - /* first try to eliminate the group by MGID, MLID, or P_Key */ - if ((IB_MCR_COMPMASK_MGID & comp_mask) && - memcmp(&p_rcvd_rec->mgid, &p_mgrp->mcmember_rec.mgid, sizeof(ib_gid_t))) - goto Exit; - - if ((IB_MCR_COMPMASK_MLID & comp_mask) && - memcmp(&p_rcvd_rec->mlid, &p_mgrp->mcmember_rec.mlid, sizeof(uint16_t))) - goto Exit; - - /* if the requester physical port doesn't have the pkey that is defined for - the group - exit. */ - if (!osm_physp_has_pkey( p_rcv->p_log, p_mgrp->mcmember_rec.pkey, - p_req_physp )) - goto Exit; - - /* now do the rest of the match */ - if ((IB_MCR_COMPMASK_QKEY & comp_mask) && - (p_rcvd_rec->qkey != p_mgrp->mcmember_rec.qkey)) - goto Exit; - - if ((IB_MCR_COMPMASK_PKEY & comp_mask) && - (p_rcvd_rec->pkey != p_mgrp->mcmember_rec.pkey)) - goto Exit; - - if ((IB_MCR_COMPMASK_TCLASS & comp_mask) && - (p_rcvd_rec->tclass != p_mgrp->mcmember_rec.tclass)) - goto Exit; - - /* check SL, Flow, and Hop limit */ - { - uint8_t mgrp_sl, query_sl; - uint32_t mgrp_flow, query_flow; - uint8_t mgrp_hop, query_hop; - - ib_member_get_sl_flow_hop(p_rcvd_rec->sl_flow_hop, - &query_sl, &query_flow, &query_hop); - - ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop, - &mgrp_sl, &mgrp_flow, &mgrp_hop); - - if (IB_MCR_COMPMASK_SL & comp_mask ) - if (query_sl != mgrp_sl) - goto Exit; - - if (IB_MCR_COMPMASK_FLOW & comp_mask) - if (query_flow != mgrp_flow) - goto Exit; - - if (IB_MCR_COMPMASK_HOP & comp_mask) - if (query_hop != mgrp_hop) - goto Exit; - } - - if ((IB_MCR_COMPMASK_PROXY & comp_mask) && - (p_rcvd_rec->proxy_join != p_mgrp->mcmember_rec.proxy_join)) - goto Exit; - - /* need to validate mtu, rate, and pkt_lifetime fields */ - if (__validate_more_comp_fields( p_rcv->p_log, - p_mgrp, - p_rcvd_rec, - comp_mask ) == FALSE) - goto Exit; - - /* Port specific fields */ - /* so did we get the PortGUID mask */ - if (IB_MCR_COMPMASK_PORT_GID & comp_mask) - { - /* try to find this port */ - if (osm_mgrp_is_port_present(p_mgrp, portguid, &p_mcm_port)) - { - scope_state = p_mcm_port->scope_state; - memcpy(&port_gid, &(p_mcm_port->port_gid), sizeof(ib_gid_t)); - proxy_join = p_mcm_port->proxy_join; - } - else - { - /* port not in group */ - goto Exit; - } - } - else - { - /* point to the group information */ - scope_state = p_mgrp->mcmember_rec.scope_state; - } - - if (IB_MCR_COMPMASK_SCOPE & comp_mask) - scope_state_mask = 0xF0; - - if (IB_MCR_COMPMASK_JOIN_STATE & comp_mask) - scope_state_mask = scope_state_mask | 0x0F; - - /* Many MC records returned */ - if ( (p_ctxt->trusted_req == TRUE) && !(IB_MCR_COMPMASK_PORT_GID & comp_mask) ) - { - osm_log(p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_mcm_by_comp_mask_cb: " - "Trusted req is TRUE and no specific port defined\n"); - - /* return all the ports that match in this MC group */ - p_item = cl_qmap_head(&(p_mgrp->mcm_port_tbl)); - while( p_item != cl_qmap_end(&(p_mgrp->mcm_port_tbl)) ) - { - p_mcm_port=(osm_mcm_port_t *)p_item; - - if ((scope_state_mask & p_rcvd_rec->scope_state) == - (scope_state_mask & p_mcm_port->scope_state)) + const osm_mgrp_t *const p_mgrp = (osm_mgrp_t *) p_map_item; + osm_sa_mcmr_search_ctxt_t *const p_ctxt = + (osm_sa_mcmr_search_ctxt_t *) context; + osm_mcmr_recv_t *const p_rcv = p_ctxt->p_rcv; + const ib_member_rec_t *p_rcvd_rec = p_ctxt->p_mcmember_rec; + const osm_physp_t *p_req_physp = p_ctxt->p_req_physp; + + /* since we might change scope_state */ + ib_member_rec_t match_rec; + ib_net64_t comp_mask = p_ctxt->comp_mask; + osm_mcm_port_t *p_mcm_port; + ib_net64_t portguid = p_rcvd_rec->port_gid.unicast.interface_id; + /* will be used for group or port info */ + uint8_t scope_state; + uint8_t scope_state_mask = 0; + cl_map_item_t *p_item; + ib_gid_t port_gid; + boolean_t proxy_join; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_mcm_by_comp_mask_cb); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_mcm_by_comp_mask_cb: " + "Checking mlid:0x%X\n", cl_ntoh16(p_mgrp->mlid)); + + /* the group might be marked for deletion */ + if (p_mgrp->to_be_deleted) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_mcm_by_comp_mask_cb: " + "Group mlid:0x%X is marked to be deleted\n", + cl_ntoh16(p_mgrp->mlid)); + goto Exit; + } + + /* first try to eliminate the group by MGID, MLID, or P_Key */ + if ((IB_MCR_COMPMASK_MGID & comp_mask) && + memcmp(&p_rcvd_rec->mgid, &p_mgrp->mcmember_rec.mgid, + sizeof(ib_gid_t))) + goto Exit; + + if ((IB_MCR_COMPMASK_MLID & comp_mask) && + memcmp(&p_rcvd_rec->mlid, &p_mgrp->mcmember_rec.mlid, + sizeof(uint16_t))) + goto Exit; + + /* if the requester physical port doesn't have the pkey that is defined for + the group - exit. */ + if (!osm_physp_has_pkey(p_rcv->p_log, p_mgrp->mcmember_rec.pkey, + p_req_physp)) + goto Exit; + + /* now do the rest of the match */ + if ((IB_MCR_COMPMASK_QKEY & comp_mask) && + (p_rcvd_rec->qkey != p_mgrp->mcmember_rec.qkey)) + goto Exit; + + if ((IB_MCR_COMPMASK_PKEY & comp_mask) && + (p_rcvd_rec->pkey != p_mgrp->mcmember_rec.pkey)) + goto Exit; + + if ((IB_MCR_COMPMASK_TCLASS & comp_mask) && + (p_rcvd_rec->tclass != p_mgrp->mcmember_rec.tclass)) + goto Exit; + + /* check SL, Flow, and Hop limit */ { - /* add to the list */ - match_rec = p_mgrp->mcmember_rec; - match_rec.scope_state = p_mcm_port->scope_state; - memcpy( &(match_rec.port_gid), &(p_mcm_port->port_gid), - sizeof(ib_gid_t)); - osm_log(p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_mcm_by_comp_mask_cb: " - "Record of port_gid: 0x%016" PRIx64 "0x%016" PRIx64 - " in multicast_lid: 0x%X is returned\n", - cl_ntoh64(match_rec.port_gid.unicast.prefix), - cl_ntoh64(match_rec.port_gid.unicast.interface_id), - cl_ntoh16(p_mgrp->mlid) - ); - - match_rec.proxy_join = (uint8_t)(p_mcm_port->proxy_join); - - __osm_mcmr_rcv_new_mcmr(p_rcv, &match_rec, p_ctxt->p_list); - } - p_item = cl_qmap_next(p_item); - } - } - /* One MC record returned */ - else - { - if ((scope_state_mask & p_rcvd_rec->scope_state) != - (scope_state_mask & scope_state)) - goto Exit; - - /* add to the list */ - match_rec = p_mgrp->mcmember_rec; - match_rec.scope_state = scope_state; - memcpy(&(match_rec.port_gid), &port_gid, sizeof(ib_gid_t)); - match_rec.proxy_join = (uint8_t)proxy_join; - - __osm_mcmr_rcv_new_mcmr(p_rcv, &match_rec, p_ctxt->p_list); - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + uint8_t mgrp_sl, query_sl; + uint32_t mgrp_flow, query_flow; + uint8_t mgrp_hop, query_hop; + + ib_member_get_sl_flow_hop(p_rcvd_rec->sl_flow_hop, + &query_sl, &query_flow, &query_hop); + + ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop, + &mgrp_sl, &mgrp_flow, &mgrp_hop); + + if (IB_MCR_COMPMASK_SL & comp_mask) + if (query_sl != mgrp_sl) + goto Exit; + + if (IB_MCR_COMPMASK_FLOW & comp_mask) + if (query_flow != mgrp_flow) + goto Exit; + + if (IB_MCR_COMPMASK_HOP & comp_mask) + if (query_hop != mgrp_hop) + goto Exit; + } + + if ((IB_MCR_COMPMASK_PROXY & comp_mask) && + (p_rcvd_rec->proxy_join != p_mgrp->mcmember_rec.proxy_join)) + goto Exit; + + /* need to validate mtu, rate, and pkt_lifetime fields */ + if (__validate_more_comp_fields(p_rcv->p_log, + p_mgrp, p_rcvd_rec, comp_mask) == FALSE) + goto Exit; + + /* Port specific fields */ + /* so did we get the PortGUID mask */ + if (IB_MCR_COMPMASK_PORT_GID & comp_mask) { + /* try to find this port */ + if (osm_mgrp_is_port_present(p_mgrp, portguid, &p_mcm_port)) { + scope_state = p_mcm_port->scope_state; + memcpy(&port_gid, &(p_mcm_port->port_gid), + sizeof(ib_gid_t)); + proxy_join = p_mcm_port->proxy_join; + } else { + /* port not in group */ + goto Exit; + } + } else { + /* point to the group information */ + scope_state = p_mgrp->mcmember_rec.scope_state; + } + + if (IB_MCR_COMPMASK_SCOPE & comp_mask) + scope_state_mask = 0xF0; + + if (IB_MCR_COMPMASK_JOIN_STATE & comp_mask) + scope_state_mask = scope_state_mask | 0x0F; + + /* Many MC records returned */ + if ((p_ctxt->trusted_req == TRUE) + && !(IB_MCR_COMPMASK_PORT_GID & comp_mask)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_mcm_by_comp_mask_cb: " + "Trusted req is TRUE and no specific port defined\n"); + + /* return all the ports that match in this MC group */ + p_item = cl_qmap_head(&(p_mgrp->mcm_port_tbl)); + while (p_item != cl_qmap_end(&(p_mgrp->mcm_port_tbl))) { + p_mcm_port = (osm_mcm_port_t *) p_item; + + if ((scope_state_mask & p_rcvd_rec->scope_state) == + (scope_state_mask & p_mcm_port->scope_state)) { + /* add to the list */ + match_rec = p_mgrp->mcmember_rec; + match_rec.scope_state = p_mcm_port->scope_state; + memcpy(&(match_rec.port_gid), + &(p_mcm_port->port_gid), + sizeof(ib_gid_t)); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_mcm_by_comp_mask_cb: " + "Record of port_gid: 0x%016" PRIx64 + "0x%016" PRIx64 + " in multicast_lid: 0x%X is returned\n", + cl_ntoh64(match_rec.port_gid.unicast. + prefix), + cl_ntoh64(match_rec.port_gid.unicast. + interface_id), + cl_ntoh16(p_mgrp->mlid) + ); + + match_rec.proxy_join = + (uint8_t) (p_mcm_port->proxy_join); + + __osm_mcmr_rcv_new_mcmr(p_rcv, &match_rec, + p_ctxt->p_list); + } + p_item = cl_qmap_next(p_item); + } + } + /* One MC record returned */ + else { + if ((scope_state_mask & p_rcvd_rec->scope_state) != + (scope_state_mask & scope_state)) + goto Exit; + + /* add to the list */ + match_rec = p_mgrp->mcmember_rec; + match_rec.scope_state = scope_state; + memcpy(&(match_rec.port_gid), &port_gid, sizeof(ib_gid_t)); + match_rec.proxy_join = (uint8_t) proxy_join; + + __osm_mcmr_rcv_new_mcmr(p_rcv, &match_rec, p_ctxt->p_list); + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** Handle a query request **********************************************************************/ static void -__osm_mcmr_query_mgrp(IN osm_mcmr_recv_t* const p_rcv, - IN const osm_madw_t* const p_madw) +__osm_mcmr_query_mgrp(IN osm_mcmr_recv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - const ib_sa_mad_t* p_rcvd_mad; - const ib_member_rec_t* p_rcvd_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_member_rec_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + const ib_sa_mad_t *p_rcvd_mad; + const ib_member_rec_t *p_rcvd_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_member_rec_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_sa_mcmr_search_ctxt_t context; - osm_mcmr_item_t* p_rec_item; - ib_api_status_t status; - ib_net64_t comp_mask; - osm_physp_t* p_req_physp; - boolean_t trusted_req; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mcmr_query_mgrp ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_member_rec_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - comp_mask = p_rcvd_mad->comp_mask; - - /* - if sm_key is not zero and does not match we never get here - see main SA receiver - */ - trusted_req = (p_rcvd_mad->sm_key != 0); - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_query_mgrp: ERR 1B04: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - cl_qlist_init( &rec_list ); - - context.p_mcmember_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - context.trusted_req = trusted_req; - - CL_PLOCK_ACQUIRE( p_rcv->p_lock ); - - /* simply go over all MCGs and match */ - cl_qmap_apply_func( &p_rcv->p_subn->mgrp_mlid_tbl, - __osm_sa_mcm_by_comp_mask_cb, - &context); - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if ( (p_rcvd_mad->method == IB_MAD_METHOD_GET) && - (num_rec > 1)) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_query_mgrp: ERR 1B05: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_rec_item = (osm_mcmr_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_mcmr_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_mcmr_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_sa_mcmr_search_ctxt_t context; + osm_mcmr_item_t *p_rec_item; + ib_api_status_t status; + ib_net64_t comp_mask; + osm_physp_t *p_req_physp; + boolean_t trusted_req; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mcmr_query_mgrp); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + comp_mask = p_rcvd_mad->comp_mask; + + /* + if sm_key is not zero and does not match we never get here + see main SA receiver + */ + trusted_req = (p_rcvd_mad->sm_key != 0); + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_query_mgrp: ERR 1B04: " + "Cannot find requester physical port\n"); + goto Exit; + } + + cl_qlist_init(&rec_list); + + context.p_mcmember_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + context.trusted_req = trusted_req; + + CL_PLOCK_ACQUIRE(p_rcv->p_lock); + + /* simply go over all MCGs and match */ + cl_qmap_apply_func(&p_rcv->p_subn->mgrp_mlid_tbl, + __osm_sa_mcm_by_comp_mask_cb, &context); + + CL_PLOCK_RELEASE(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_query_mgrp: ERR 1B05: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_mcmr_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_rec_item = + (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_member_rec_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_mcmr_query_mgrp: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_member_rec_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_mcmr_query_mgrp: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mcmr_query_mgrp: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_member_rec_t) + IB_SA_MAD_HDR_SIZE, - osm_madw_get_mad_addr_ptr(p_madw) ); - - if( !p_resp_madw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_query_mgrp: ERR 1B16: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_mcmr_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_member_rec_t) ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mcmr_query_mgrp: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_member_rec_t) + + IB_SA_MAD_HDR_SIZE, + osm_madw_get_mad_addr_ptr(p_madw)); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_query_mgrp: ERR 1B16: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_member_rec_t)); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - p_resp_rec = (ib_member_rec_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); - - /* - p923 - The PortGID, JoinState and ProxyJoin shall be zero, - except in the case of a trusted request. - Note: In the mad controller we check that the SM_Key received on - the mad is valid. Meaning - is either zero or equal to the local - sm_key. - */ - - for ( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_mcmr_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - if (trusted_req == FALSE) - { - memset(&p_resp_rec->port_gid, 0, sizeof(ib_gid_t)); - ib_member_set_join_state(p_resp_rec, 0); - p_resp_rec->proxy_join = 0; - } - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if(status != IB_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mcmr_query_mgrp: ERR 1B17: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + p_resp_rec = + (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); + + /* + p923 - The PortGID, JoinState and ProxyJoin shall be zero, + except in the case of a trusted request. + Note: In the mad controller we check that the SM_Key received on + the mad is valid. Meaning - is either zero or equal to the local + sm_key. + */ + + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_mcmr_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + if (trusted_req == FALSE) { + memset(&p_resp_rec->port_gid, 0, + sizeof(ib_gid_t)); + ib_member_set_join_state(p_resp_rec, 0); + p_resp_rec->proxy_join = 0; + } + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mcmr_query_mgrp: ERR 1B17: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_mcmr_rcv_process( - IN void *context, - IN void *data ) +void osm_mcmr_rcv_process(IN void *context, IN void *data) { - osm_mcmr_recv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_sa_mad_t *p_sa_mad; - ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - ib_member_rec_t *p_recvd_mcmember_rec; - boolean_t valid; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_mcmr_rcv_process ); - - CL_ASSERT( p_madw ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_mcmember_rec = - (ib_member_rec_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - CL_ASSERT( p_sa_mad->attr_id == IB_MAD_ATTR_MCMEMBER_RECORD ); - - switch (p_sa_mad->method) - { - case IB_MAD_METHOD_SET: - valid = __check_join_comp_mask(p_sa_mad->comp_mask); - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_process: ERR 1B18: " - "component mask = 0x%016" PRIx64 ", " - "expected comp mask = 0x%016" PRIx64 " ," - "MGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 " for " - "PortGID: 0x%016" PRIx64 " : " - "0x%016" PRIx64 "\n", - cl_ntoh64(p_sa_mad->comp_mask), - CL_NTOH64(JOIN_MC_COMP_MASK), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->mgid.unicast.interface_id ), - cl_ntoh64( p_recvd_mcmember_rec->port_gid.unicast.prefix ), - cl_ntoh64( p_recvd_mcmember_rec->port_gid.unicast.interface_id ) ); - - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* - * Join or Create Multicast Group - */ - __osm_mcmr_rcv_join_mgrp(p_rcv, p_madw); - break; - case IB_MAD_METHOD_DELETE: - valid = __check_join_comp_mask(p_sa_mad->comp_mask); - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_process: ERR 1B20: " - "component mask = 0x%016" PRIx64 ", " - "expected comp mask = 0x%016" PRIx64 "\n", - cl_ntoh64(p_sa_mad->comp_mask), - CL_NTOH64(JOIN_MC_COMP_MASK) ); - - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* - * Leave Multicast Group - */ - __osm_mcmr_rcv_leave_mgrp(p_rcv, p_madw); - break; - case IB_MAD_METHOD_GET: - case IB_MAD_METHOD_GETTABLE: - /* - * Querying a Multicast Group - */ - __osm_mcmr_query_mgrp(p_rcv, p_madw); - break; - default: - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mcmr_rcv_process: ERR 1B21: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_sa_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - break; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + osm_mcmr_recv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_sa_mad_t *p_sa_mad; + ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + ib_member_rec_t *p_recvd_mcmember_rec; + boolean_t valid; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_mcmr_rcv_process); + + CL_ASSERT(p_madw); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_mcmember_rec = + (ib_member_rec_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + CL_ASSERT(p_sa_mad->attr_id == IB_MAD_ATTR_MCMEMBER_RECORD); + + switch (p_sa_mad->method) { + case IB_MAD_METHOD_SET: + valid = __check_join_comp_mask(p_sa_mad->comp_mask); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_process: ERR 1B18: " + "component mask = 0x%016" PRIx64 ", " + "expected comp mask = 0x%016" PRIx64 " ," + "MGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 " for " + "PortGID: 0x%016" PRIx64 " : " + "0x%016" PRIx64 "\n", + cl_ntoh64(p_sa_mad->comp_mask), + CL_NTOH64(JOIN_MC_COMP_MASK), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + prefix), + cl_ntoh64(p_recvd_mcmember_rec->mgid.unicast. + interface_id), + cl_ntoh64(p_recvd_mcmember_rec->port_gid. + unicast.prefix), + cl_ntoh64(p_recvd_mcmember_rec->port_gid. + unicast.interface_id)); + + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* + * Join or Create Multicast Group + */ + __osm_mcmr_rcv_join_mgrp(p_rcv, p_madw); + break; + case IB_MAD_METHOD_DELETE: + valid = __check_join_comp_mask(p_sa_mad->comp_mask); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_process: ERR 1B20: " + "component mask = 0x%016" PRIx64 ", " + "expected comp mask = 0x%016" PRIx64 "\n", + cl_ntoh64(p_sa_mad->comp_mask), + CL_NTOH64(JOIN_MC_COMP_MASK)); + + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* + * Leave Multicast Group + */ + __osm_mcmr_rcv_leave_mgrp(p_rcv, p_madw); + break; + case IB_MAD_METHOD_GET: + case IB_MAD_METHOD_GETTABLE: + /* + * Querying a Multicast Group + */ + __osm_mcmr_query_mgrp(p_rcv, p_madw); + break; + default: + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mcmr_rcv_process: ERR 1B21: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_sa_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + break; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } diff --git a/opensm/opensm/osm_sa_mft_record.c b/opensm/opensm/osm_sa_mft_record.c index 37d9c7b..3d20f0a 100644 --- a/opensm/opensm/osm_sa_mft_record.c +++ b/opensm/opensm/osm_sa_mft_record.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -61,487 +61,458 @@ #define OSM_MFTR_RCV_POOL_MIN_SIZE 32 #define OSM_MFTR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_mftr_item -{ - cl_pool_item_t pool_item; - ib_mft_record_t rec; +typedef struct _osm_mftr_item { + cl_pool_item_t pool_item; + ib_mft_record_t rec; } osm_mftr_item_t; -typedef struct _osm_mftr_search_ctxt -{ - const ib_mft_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_mftr_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_mftr_search_ctxt { + const ib_mft_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_mftr_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_mftr_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_mftr_rcv_construct( - IN osm_mftr_rcv_t* const p_rcv ) +void osm_mftr_rcv_construct(IN osm_mftr_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_mftr_rcv_destroy( - IN osm_mftr_rcv_t* const p_rcv ) +void osm_mftr_rcv_destroy(IN osm_mftr_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_mftr_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_mftr_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_mftr_rcv_init( - IN osm_mftr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_mftr_rcv_init(IN osm_mftr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_mftr_rcv_init ); + OSM_LOG_ENTER(p_log, osm_mftr_rcv_init); - osm_mftr_rcv_construct( p_rcv ); + osm_mftr_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_MFTR_RCV_POOL_MIN_SIZE, - 0, - OSM_MFTR_RCV_POOL_GROW_SIZE, - sizeof(osm_mftr_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_MFTR_RCV_POOL_MIN_SIZE, + 0, + OSM_MFTR_RCV_POOL_GROW_SIZE, + sizeof(osm_mftr_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_mftr_rcv_new_mftr( - IN osm_mftr_rcv_t* const p_rcv, - IN osm_switch_t* const p_sw, - IN cl_qlist_t* const p_list, - IN ib_net16_t const lid, - IN uint16_t const block, - IN uint8_t const position ) +__osm_mftr_rcv_new_mftr(IN osm_mftr_rcv_t * const p_rcv, + IN osm_switch_t * const p_sw, + IN cl_qlist_t * const p_list, + IN ib_net16_t const lid, + IN uint16_t const block, IN uint8_t const position) { - osm_mftr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - uint16_t position_block_num; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_mftr_rcv_new_mftr ); - - p_rec_item = (osm_mftr_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mftr_rcv_new_mftr: ERR 4A02: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mftr_rcv_new_mftr: " - "New MulticastForwardingTable: sw 0x%016" PRIx64 - "\n\t\t\t\tblock %u position %u lid 0x%02X\n", - cl_ntoh64( osm_node_get_node_guid( p_sw->p_node ) ), - block, position, cl_ntoh16( lid ) - ); - } - - position_block_num = ((uint16_t)position << 12) | - (block & IB_MCAST_BLOCK_ID_MASK_HO); - - memset( &p_rec_item->rec, 0, sizeof(ib_mft_record_t) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.position_block_num = cl_hton16( position_block_num ); - - /* copy the mft block */ - osm_switch_get_mft_block( p_sw, block, position, p_rec_item->rec.mft ); - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_mftr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + uint16_t position_block_num; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_mftr_rcv_new_mftr); + + p_rec_item = (osm_mftr_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mftr_rcv_new_mftr: ERR 4A02: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mftr_rcv_new_mftr: " + "New MulticastForwardingTable: sw 0x%016" PRIx64 + "\n\t\t\t\tblock %u position %u lid 0x%02X\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)), + block, position, cl_ntoh16(lid) + ); + } + + position_block_num = ((uint16_t) position << 12) | + (block & IB_MCAST_BLOCK_ID_MASK_HO); + + memset(&p_rec_item->rec, 0, sizeof(ib_mft_record_t)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.position_block_num = cl_hton16(position_block_num); + + /* copy the mft block */ + osm_switch_get_mft_block(p_sw, block, position, p_rec_item->rec.mft); + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -static osm_port_t* -__osm_mftr_get_port_by_guid( - IN osm_mftr_rcv_t* const p_rcv, - IN uint64_t port_guid ) +static osm_port_t *__osm_mftr_get_port_by_guid(IN osm_mftr_rcv_t * const p_rcv, + IN uint64_t port_guid) { - osm_port_t* p_port; + osm_port_t *p_port; - CL_PLOCK_ACQUIRE(p_rcv->p_lock); + CL_PLOCK_ACQUIRE(p_rcv->p_lock); - p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); - if (!p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mftr_get_port_by_guid ERR 4A04: " - "Invalid port GUID 0x%016" PRIx64 "\n", - port_guid ); - } + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mftr_get_port_by_guid ERR 4A04: " + "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); + } - CL_PLOCK_RELEASE(p_rcv->p_lock); - return p_port; + CL_PLOCK_RELEASE(p_rcv->p_lock); + return p_port; } /********************************************************************** **********************************************************************/ static void -__osm_mftr_rcv_by_comp_mask( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_mftr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_mftr_search_ctxt_t* const p_ctxt = - (osm_mftr_search_ctxt_t *)context; - osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - const ib_mft_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - osm_mftr_rcv_t* const p_rcv = p_ctxt->p_rcv; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - osm_port_t* p_port; - uint16_t min_lid_ho, max_lid_ho; - uint16_t position_block_num_ho; - uint16_t min_block, max_block, block; - const osm_physp_t* p_physp; - uint8_t min_position, max_position, position; - - /* In switches, the port guid is the node guid. */ - p_port = - __osm_mftr_get_port_by_guid( p_rcv, p_sw->p_node->node_info.port_guid ); - if (! p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mftr_rcv_by_comp_mask: ERR 4A05: " - "Failed to find Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - return; - } - - /* check that the requester physp and the current physp are under - the same partition. */ - p_physp = p_port->p_physp; - if (! p_physp) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_mftr_rcv_by_comp_mask: ERR 4A06: " - "Failed to find default physical Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - return; - } - if (! osm_physp_share_pkey( p_rcv->p_log, p_req_physp, p_physp )) - return; - - /* get the port 0 of the switch */ - osm_port_get_lid_range_ho( p_port, &min_lid_ho, &max_lid_ho ); - - /* compare the lids - if required */ - if( comp_mask & IB_MFTR_COMPMASK_LID ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_mftr_rcv_by_comp_mask: " - "Comparing lid:0x%02X to port lid range: 0x%02X .. 0x%02X\n", - cl_ntoh16( p_rcvd_rec->lid ), min_lid_ho, max_lid_ho - ); - /* ok we are ready for range check */ - if (min_lid_ho > cl_ntoh16(p_rcvd_rec->lid) || - max_lid_ho < cl_ntoh16(p_rcvd_rec->lid)) - return; - } - - if ( !osm_switch_supports_mcast ( p_sw ) ) - return; - - /* Are there any blocks in use ? */ - if ( osm_switch_get_mft_max_block_in_use( p_sw ) == -1 ) - return; - - position_block_num_ho = cl_ntoh16( p_rcvd_rec->position_block_num ); - - /* now we need to decide which blocks to output */ - if( comp_mask & IB_MFTR_COMPMASK_BLOCK ) - { - max_block = min_block = position_block_num_ho & IB_MCAST_BLOCK_ID_MASK_HO; - if ( max_block > osm_switch_get_mft_max_block_in_use( p_sw ) ) - return; - } - else - { - /* use as many blocks as needed */ - min_block = 0; - max_block = osm_switch_get_mft_max_block_in_use( p_sw ); - } - - /* need to decide which positions to output */ - if ( comp_mask & IB_MFTR_COMPMASK_POSITION ) - { - min_position = max_position = (position_block_num_ho & 0xF000) >> 12; - if (max_position > osm_switch_get_mft_max_position( p_sw ) ) - return; - } - else - { - /* use as many positions as needed */ - min_position = 0; - max_position = osm_switch_get_mft_max_position( p_sw ); - } - - /* so we can add these one by one ... */ - for (block = min_block; block <= max_block; block++) - for (position = min_position; position <= max_position; position++) - __osm_mftr_rcv_new_mftr( p_rcv, p_sw, p_ctxt->p_list, - osm_port_get_base_lid(p_port), - block, position ); + const osm_mftr_search_ctxt_t *const p_ctxt = + (osm_mftr_search_ctxt_t *) context; + osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + const ib_mft_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + osm_mftr_rcv_t *const p_rcv = p_ctxt->p_rcv; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + osm_port_t *p_port; + uint16_t min_lid_ho, max_lid_ho; + uint16_t position_block_num_ho; + uint16_t min_block, max_block, block; + const osm_physp_t *p_physp; + uint8_t min_position, max_position, position; + + /* In switches, the port guid is the node guid. */ + p_port = + __osm_mftr_get_port_by_guid(p_rcv, + p_sw->p_node->node_info.port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mftr_rcv_by_comp_mask: ERR 4A05: " + "Failed to find Port by Node Guid:0x%016" PRIx64 + "\n", cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + return; + } + + /* check that the requester physp and the current physp are under + the same partition. */ + p_physp = p_port->p_physp; + if (!p_physp) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_mftr_rcv_by_comp_mask: ERR 4A06: " + "Failed to find default physical Port by Node Guid:0x%016" + PRIx64 "\n", + cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + return; + } + if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) + return; + + /* get the port 0 of the switch */ + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); + + /* compare the lids - if required */ + if (comp_mask & IB_MFTR_COMPMASK_LID) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_mftr_rcv_by_comp_mask: " + "Comparing lid:0x%02X to port lid range: 0x%02X .. 0x%02X\n", + cl_ntoh16(p_rcvd_rec->lid), min_lid_ho, max_lid_ho); + /* ok we are ready for range check */ + if (min_lid_ho > cl_ntoh16(p_rcvd_rec->lid) || + max_lid_ho < cl_ntoh16(p_rcvd_rec->lid)) + return; + } + + if (!osm_switch_supports_mcast(p_sw)) + return; + + /* Are there any blocks in use ? */ + if (osm_switch_get_mft_max_block_in_use(p_sw) == -1) + return; + + position_block_num_ho = cl_ntoh16(p_rcvd_rec->position_block_num); + + /* now we need to decide which blocks to output */ + if (comp_mask & IB_MFTR_COMPMASK_BLOCK) { + max_block = min_block = + position_block_num_ho & IB_MCAST_BLOCK_ID_MASK_HO; + if (max_block > osm_switch_get_mft_max_block_in_use(p_sw)) + return; + } else { + /* use as many blocks as needed */ + min_block = 0; + max_block = osm_switch_get_mft_max_block_in_use(p_sw); + } + + /* need to decide which positions to output */ + if (comp_mask & IB_MFTR_COMPMASK_POSITION) { + min_position = max_position = + (position_block_num_ho & 0xF000) >> 12; + if (max_position > osm_switch_get_mft_max_position(p_sw)) + return; + } else { + /* use as many positions as needed */ + min_position = 0; + max_position = osm_switch_get_mft_max_position(p_sw); + } + + /* so we can add these one by one ... */ + for (block = min_block; block <= max_block; block++) + for (position = min_position; position <= max_position; + position++) + __osm_mftr_rcv_new_mftr(p_rcv, p_sw, p_ctxt->p_list, + osm_port_get_base_lid(p_port), + block, position); } /********************************************************************** **********************************************************************/ -void -osm_mftr_rcv_process( - IN void *ctx, - IN void *data ) +void osm_mftr_rcv_process(IN void *ctx, IN void *data) { - osm_mftr_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_mft_record_t* p_rcvd_rec; - ib_mft_record_t* p_resp_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; + osm_mftr_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_mft_record_t *p_rcvd_rec; + ib_mft_record_t *p_resp_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_mftr_search_ctxt_t context; - osm_mftr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_mftr_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_mft_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_MFT_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mftr_rcv_process: ERR 4A08: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mftr_rcv_process: ERR 4A07: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - /* Go over all switches */ - cl_qmap_apply_func( &p_rcv->p_subn->sw_guid_tbl, - __osm_mftr_rcv_by_comp_mask, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_mftr_rcv_process: ERR 4A09: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_mftr_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_mftr_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_mftr_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_mftr_search_ctxt_t context; + osm_mftr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_mftr_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_mft_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_MFT_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mftr_rcv_process: ERR 4A08: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mftr_rcv_process: ERR 4A07: " + "Cannot find requester physical port\n"); + goto Exit; + } + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + /* Go over all switches */ + cl_qmap_apply_func(&p_rcv->p_subn->sw_guid_tbl, + __osm_mftr_rcv_by_comp_mask, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mftr_rcv_process: ERR 4A09: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_mftr_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_mftr_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_mft_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_mftr_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + /* we limit the number of records to a single packet */ + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_mft_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_mftr_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_mftr_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) && - (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_mft_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_mftr_rcv_process: ERR 4A10: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_mftr_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_mft_record_t) ); - - p_resp_rec = (ib_mft_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_mftr_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_mft_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mftr_rcv_process: ERR 4A10: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_mft_record_t)); + + p_resp_rec = + (ib_mft_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_mftr_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_mftr_rcv_process: ERR 4A11: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_mftr_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_mftr_rcv_process: ERR 4A11: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_node_record.c b/opensm/opensm/osm_sa_node_record.c index f40d769..a7615c5 100644 --- a/opensm/opensm/osm_sa_node_record.c +++ b/opensm/opensm/osm_sa_node_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -62,541 +62,512 @@ #define OSM_NR_RCV_POOL_MIN_SIZE 32 #define OSM_NR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_nr_item -{ - cl_pool_item_t pool_item; - ib_node_record_t rec; +typedef struct _osm_nr_item { + cl_pool_item_t pool_item; + ib_node_record_t rec; } osm_nr_item_t; -typedef struct _osm_nr_search_ctxt -{ - const ib_node_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_nr_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_nr_search_ctxt { + const ib_node_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_nr_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_nr_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_nr_rcv_construct( - IN osm_nr_rcv_t* const p_rcv ) +void osm_nr_rcv_construct(IN osm_nr_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_nr_rcv_destroy( - IN osm_nr_rcv_t* const p_rcv ) +void osm_nr_rcv_destroy(IN osm_nr_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_nr_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_nr_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_nr_rcv_init( - IN osm_nr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_nr_rcv_init(IN osm_nr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_nr_rcv_init ); + OSM_LOG_ENTER(p_log, osm_nr_rcv_init); - osm_nr_rcv_construct( p_rcv ); + osm_nr_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_NR_RCV_POOL_MIN_SIZE, - 0, - OSM_NR_RCV_POOL_GROW_SIZE, - sizeof(osm_nr_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_NR_RCV_POOL_MIN_SIZE, + 0, + OSM_NR_RCV_POOL_GROW_SIZE, + sizeof(osm_nr_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_nr_rcv_new_nr( - IN osm_nr_rcv_t* const p_rcv, - IN const osm_node_t* const p_node, - IN cl_qlist_t* const p_list, - IN ib_net64_t const port_guid, - IN ib_net16_t const lid ) +__osm_nr_rcv_new_nr(IN osm_nr_rcv_t * const p_rcv, + IN const osm_node_t * const p_node, + IN cl_qlist_t * const p_list, + IN ib_net64_t const port_guid, IN ib_net16_t const lid) { - osm_nr_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_nr_rcv_new_nr ); - - p_rec_item = (osm_nr_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_nr_rcv_new_nr: ERR 1D02: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_nr_rcv_new_nr: " - "New NodeRecord: node 0x%016" PRIx64 - "\n\t\t\t\tport 0x%016" PRIx64 ", lid 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - cl_ntoh64( port_guid ), cl_ntoh16( lid ) - ); - } - - memset( &p_rec_item->rec, 0, sizeof(ib_node_record_t) ); - - p_rec_item->rec.lid = lid; - - p_rec_item->rec.node_info = p_node->node_info; - p_rec_item->rec.node_info.port_guid = port_guid; - memcpy(&(p_rec_item->rec.node_desc), &(p_node->node_desc), - IB_NODE_DESCRIPTION_SIZE); - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_nr_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_nr_rcv_new_nr); + + p_rec_item = (osm_nr_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_nr_rcv_new_nr: ERR 1D02: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_nr_rcv_new_nr: " + "New NodeRecord: node 0x%016" PRIx64 + "\n\t\t\t\tport 0x%016" PRIx64 ", lid 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_node)), + cl_ntoh64(port_guid), cl_ntoh16(lid) + ); + } + + memset(&p_rec_item->rec, 0, sizeof(ib_node_record_t)); + + p_rec_item->rec.lid = lid; + + p_rec_item->rec.node_info = p_node->node_info; + p_rec_item->rec.node_info.port_guid = port_guid; + memcpy(&(p_rec_item->rec.node_desc), &(p_node->node_desc), + IB_NODE_DESCRIPTION_SIZE); + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_nr_rcv_create_nr( - IN osm_nr_rcv_t* const p_rcv, - IN const osm_node_t* const p_node, - IN cl_qlist_t* const p_list, - IN ib_net64_t const match_port_guid, - IN ib_net16_t const match_lid, - IN const osm_physp_t* const p_req_physp ) +__osm_nr_rcv_create_nr(IN osm_nr_rcv_t * const p_rcv, + IN const osm_node_t * const p_node, + IN cl_qlist_t * const p_list, + IN ib_net64_t const match_port_guid, + IN ib_net16_t const match_lid, + IN const osm_physp_t * const p_req_physp) { - const osm_physp_t* p_physp; - uint8_t port_num; - uint8_t num_ports; - uint16_t match_lid_ho; - ib_net16_t base_lid; - ib_net16_t base_lid_ho; - ib_net16_t max_lid_ho; - uint8_t lmc; - ib_net64_t port_guid; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_nr_rcv_create_nr ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_nr_rcv_create_nr: " - "Looking for NodeRecord with LID: 0x%X GUID:0x%016" PRIx64 "\n", - cl_ntoh16( match_lid ), - cl_ntoh64( match_port_guid ) - ); - } - - /* - For switches, do not return the NodeInfo record - for each port on the switch, just for port 0. - */ - if( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ) - num_ports = 1; - else - num_ports = osm_node_get_num_physp( p_node ); - - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - - if( !osm_physp_is_valid( p_physp ) ) - continue; - - /* Check to see if the found p_physp and the requester physp - share a pkey. If not - continue */ - if (!osm_physp_share_pkey( p_rcv->p_log, p_physp, p_req_physp ) ) - continue; - - port_guid = osm_physp_get_port_guid( p_physp ); - - if( match_port_guid && ( port_guid != match_port_guid ) ) - continue; - - base_lid = osm_physp_get_base_lid( p_physp ); - base_lid_ho = cl_ntoh16( base_lid ); - lmc = osm_physp_get_lmc( p_physp ); - max_lid_ho = (uint16_t)( base_lid_ho + (1 << lmc) - 1 ); - match_lid_ho = cl_ntoh16( match_lid ); - - if( match_lid_ho ) - { - /* - We validate that the lid belongs to this node. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_nr_rcv_create_nr: " - "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", - base_lid_ho, match_lid_ho, max_lid_ho - ); - } - - if ( match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho ) - continue; - } - - __osm_nr_rcv_new_nr( p_rcv, p_node, p_list, port_guid, base_lid ); - - } - - OSM_LOG_EXIT( p_rcv->p_log ); + const osm_physp_t *p_physp; + uint8_t port_num; + uint8_t num_ports; + uint16_t match_lid_ho; + ib_net16_t base_lid; + ib_net16_t base_lid_ho; + ib_net16_t max_lid_ho; + uint8_t lmc; + ib_net64_t port_guid; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_nr_rcv_create_nr); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_nr_rcv_create_nr: " + "Looking for NodeRecord with LID: 0x%X GUID:0x%016" + PRIx64 "\n", cl_ntoh16(match_lid), + cl_ntoh64(match_port_guid) + ); + } + + /* + For switches, do not return the NodeInfo record + for each port on the switch, just for port 0. + */ + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) + num_ports = 1; + else + num_ports = osm_node_get_num_physp(p_node); + + for (port_num = 0; port_num < num_ports; port_num++) { + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + if (!osm_physp_is_valid(p_physp)) + continue; + + /* Check to see if the found p_physp and the requester physp + share a pkey. If not - continue */ + if (!osm_physp_share_pkey(p_rcv->p_log, p_physp, p_req_physp)) + continue; + + port_guid = osm_physp_get_port_guid(p_physp); + + if (match_port_guid && (port_guid != match_port_guid)) + continue; + + base_lid = osm_physp_get_base_lid(p_physp); + base_lid_ho = cl_ntoh16(base_lid); + lmc = osm_physp_get_lmc(p_physp); + max_lid_ho = (uint16_t) (base_lid_ho + (1 << lmc) - 1); + match_lid_ho = cl_ntoh16(match_lid); + + if (match_lid_ho) { + /* + We validate that the lid belongs to this node. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_nr_rcv_create_nr: " + "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", + base_lid_ho, match_lid_ho, max_lid_ho); + } + + if (match_lid_ho < base_lid_ho + || match_lid_ho > max_lid_ho) + continue; + } + + __osm_nr_rcv_new_nr(p_rcv, p_node, p_list, port_guid, base_lid); + + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_nr_rcv_by_comp_mask( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_nr_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, IN void *context) { - const osm_nr_search_ctxt_t* const p_ctxt = (osm_nr_search_ctxt_t *)context; - const osm_node_t* const p_node = (osm_node_t*)p_map_item; - const ib_node_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - osm_nr_rcv_t* const p_rcv = p_ctxt->p_rcv; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - ib_net64_t match_port_guid = 0; - ib_net16_t match_lid = 0; - - OSM_LOG_ENTER( p_ctxt->p_rcv->p_log, __osm_nr_rcv_by_comp_mask ); - - osm_dump_node_info( - p_ctxt->p_rcv->p_log, - &p_node->node_info, - OSM_LOG_VERBOSE ); - - if( comp_mask & IB_NR_COMPMASK_LID ) - match_lid = p_rcvd_rec->lid; - - if( comp_mask & IB_NR_COMPMASK_NODEGUID) - { - /* - DEBUG TOP - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_nr_rcv_by_comp_mask: " - "Looking for node 0x%016" PRIx64 - ", found 0x%016" PRIx64 "\n", - cl_ntoh64( p_rcvd_rec->node_info.node_guid ), - cl_ntoh64( osm_node_get_node_guid( p_node ) ) - ); - } - /* - DEBUG BOTTOM - */ - if( (p_node->node_info.node_guid != p_rcvd_rec->node_info.node_guid) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_PORTGUID ) - { - match_port_guid = p_rcvd_rec->node_info.port_guid; - } - if( comp_mask & IB_NR_COMPMASK_SYSIMAGEGUID ) - { - if( (p_node->node_info.sys_guid != p_rcvd_rec->node_info.sys_guid) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_BASEVERSION ) - { - if( (p_node->node_info.base_version != p_rcvd_rec->node_info.base_version) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_CLASSVERSION ) - { - if( (p_node->node_info.class_version != p_rcvd_rec->node_info.class_version) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_NODETYPE ) - { - if( (p_node->node_info.node_type != p_rcvd_rec->node_info.node_type) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_NUMPORTS ) - { - if( (p_node->node_info.num_ports != p_rcvd_rec->node_info.num_ports) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_PARTCAP ) - { - if( (p_node->node_info.partition_cap != p_rcvd_rec->node_info.partition_cap) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_DEVID ) - { - if( (p_node->node_info.device_id != p_rcvd_rec->node_info.device_id) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_REV ) - { - if( (p_node->node_info.revision != p_rcvd_rec->node_info.revision) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_PORTNUM ) - { - if( ib_node_info_get_local_port_num(&p_node->node_info) != - ib_node_info_get_local_port_num(&p_rcvd_rec->node_info) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_VENDID ) - { - if( ib_node_info_get_vendor_id(&p_node->node_info) != - ib_node_info_get_vendor_id(&p_rcvd_rec->node_info) ) - goto Exit; - } - if( comp_mask & IB_NR_COMPMASK_NODEDESC ) - { - if( strncmp((char*) &p_node->node_desc, - (char*) &p_rcvd_rec->node_desc, sizeof(ib_node_desc_t)) ) - goto Exit; - } - - __osm_nr_rcv_create_nr( p_rcv, p_node, p_ctxt->p_list, - match_port_guid, match_lid, p_req_physp ); - - Exit: - OSM_LOG_EXIT( p_ctxt->p_rcv->p_log ); + const osm_nr_search_ctxt_t *const p_ctxt = + (osm_nr_search_ctxt_t *) context; + const osm_node_t *const p_node = (osm_node_t *) p_map_item; + const ib_node_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + osm_nr_rcv_t *const p_rcv = p_ctxt->p_rcv; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + ib_net64_t match_port_guid = 0; + ib_net16_t match_lid = 0; + + OSM_LOG_ENTER(p_ctxt->p_rcv->p_log, __osm_nr_rcv_by_comp_mask); + + osm_dump_node_info(p_ctxt->p_rcv->p_log, + &p_node->node_info, OSM_LOG_VERBOSE); + + if (comp_mask & IB_NR_COMPMASK_LID) + match_lid = p_rcvd_rec->lid; + + if (comp_mask & IB_NR_COMPMASK_NODEGUID) { + /* + DEBUG TOP + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_nr_rcv_by_comp_mask: " + "Looking for node 0x%016" PRIx64 + ", found 0x%016" PRIx64 "\n", + cl_ntoh64(p_rcvd_rec->node_info.node_guid), + cl_ntoh64(osm_node_get_node_guid(p_node)) + ); + } + /* + DEBUG BOTTOM + */ + if ((p_node->node_info.node_guid != + p_rcvd_rec->node_info.node_guid)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_PORTGUID) { + match_port_guid = p_rcvd_rec->node_info.port_guid; + } + if (comp_mask & IB_NR_COMPMASK_SYSIMAGEGUID) { + if ((p_node->node_info.sys_guid != + p_rcvd_rec->node_info.sys_guid)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_BASEVERSION) { + if ((p_node->node_info.base_version != + p_rcvd_rec->node_info.base_version)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_CLASSVERSION) { + if ((p_node->node_info.class_version != + p_rcvd_rec->node_info.class_version)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_NODETYPE) { + if ((p_node->node_info.node_type != + p_rcvd_rec->node_info.node_type)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_NUMPORTS) { + if ((p_node->node_info.num_ports != + p_rcvd_rec->node_info.num_ports)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_PARTCAP) { + if ((p_node->node_info.partition_cap != + p_rcvd_rec->node_info.partition_cap)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_DEVID) { + if ((p_node->node_info.device_id != + p_rcvd_rec->node_info.device_id)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_REV) { + if ((p_node->node_info.revision != + p_rcvd_rec->node_info.revision)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_PORTNUM) { + if (ib_node_info_get_local_port_num(&p_node->node_info) != + ib_node_info_get_local_port_num(&p_rcvd_rec->node_info)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_VENDID) { + if (ib_node_info_get_vendor_id(&p_node->node_info) != + ib_node_info_get_vendor_id(&p_rcvd_rec->node_info)) + goto Exit; + } + if (comp_mask & IB_NR_COMPMASK_NODEDESC) { + if (strncmp((char *)&p_node->node_desc, + (char *)&p_rcvd_rec->node_desc, + sizeof(ib_node_desc_t))) + goto Exit; + } + + __osm_nr_rcv_create_nr(p_rcv, p_node, p_ctxt->p_list, + match_port_guid, match_lid, p_req_physp); + + Exit: + OSM_LOG_EXIT(p_ctxt->p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_nr_rcv_process( - IN void *ctx, - IN void *data ) +void osm_nr_rcv_process(IN void *ctx, IN void *data) { - osm_nr_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_node_record_t* p_rcvd_rec; - ib_node_record_t* p_resp_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; + osm_nr_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_node_record_t *p_rcvd_rec; + ib_node_record_t *p_resp_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_nr_search_ctxt_t context; - osm_nr_item_t* p_rec_item; - ib_api_status_t status; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_nr_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_node_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_NODE_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_nr_rcv_process: ERR 1D05: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_nr_rcv_process: ERR 1D04: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_node_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - cl_qmap_apply_func( &p_rcv->p_subn->node_guid_tbl, - __osm_nr_rcv_by_comp_mask, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if ( (p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_nr_rcv_process: ERR 1D03: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_rec_item = (osm_nr_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_nr_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_nr_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_nr_search_ctxt_t context; + osm_nr_item_t *p_rec_item; + ib_api_status_t status; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_nr_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_node_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_NODE_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nr_rcv_process: ERR 1D05: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nr_rcv_process: ERR 1D04: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_node_record(p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + cl_qmap_apply_func(&p_rcv->p_subn->node_guid_tbl, + __osm_nr_rcv_by_comp_mask, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nr_rcv_process: ERR 1D03: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != (osm_nr_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_rec_item = + (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_node_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_nr_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + /* we limit the number of records to a single packet */ + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_node_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_nr_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_nr_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_node_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_nr_rcv_process: ERR 1D06: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_nr_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_node_record_t) ); - - p_resp_rec = (ib_node_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_nr_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_node_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nr_rcv_process: ERR 1D06: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_node_record_t)); + + p_resp_rec = + (ib_node_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_nr_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_nr_rcv_process: ERR 1D07: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = (osm_nr_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_nr_rcv_process: ERR 1D07: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_pkey_record.c b/opensm/opensm/osm_sa_pkey_record.c index 2432b2d..0081f2b 100644 --- a/opensm/opensm/osm_sa_pkey_record.c +++ b/opensm/opensm/osm_sa_pkey_record.c @@ -35,7 +35,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -53,533 +53,512 @@ #define OSM_PKEY_REC_RCV_POOL_MIN_SIZE 32 #define OSM_PKEY_REC_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_pkey_item -{ - cl_pool_item_t pool_item; - ib_pkey_table_record_t rec; +typedef struct _osm_pkey_item { + cl_pool_item_t pool_item; + ib_pkey_table_record_t rec; } osm_pkey_item_t; -typedef struct _osm_pkey_search_ctxt -{ - const ib_pkey_table_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - uint16_t block_num; - cl_qlist_t* p_list; - osm_pkey_rec_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_pkey_search_ctxt { + const ib_pkey_table_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + uint16_t block_num; + cl_qlist_t *p_list; + osm_pkey_rec_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_pkey_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_pkey_rec_rcv_construct( - IN osm_pkey_rec_rcv_t* const p_rcv ) +void osm_pkey_rec_rcv_construct(IN osm_pkey_rec_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_pkey_rec_rcv_destroy( - IN osm_pkey_rec_rcv_t* const p_rcv ) +void osm_pkey_rec_rcv_destroy(IN osm_pkey_rec_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_pkey_rec_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_pkey_rec_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pkey_rec_rcv_init( - IN osm_pkey_rec_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_pkey_rec_rcv_init(IN osm_pkey_rec_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_pkey_rec_rcv_init ); + OSM_LOG_ENTER(p_log, osm_pkey_rec_rcv_init); - osm_pkey_rec_rcv_construct( p_rcv ); + osm_pkey_rec_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - /* used for matching records collection */ - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_PKEY_REC_RCV_POOL_MIN_SIZE, - 0, - OSM_PKEY_REC_RCV_POOL_GROW_SIZE, - sizeof(osm_pkey_item_t), - NULL, NULL, NULL ); + /* used for matching records collection */ + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_PKEY_REC_RCV_POOL_MIN_SIZE, + 0, + OSM_PKEY_REC_RCV_POOL_GROW_SIZE, + sizeof(osm_pkey_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ void -__osm_sa_pkey_create( - IN osm_pkey_rec_rcv_t* const p_rcv, - IN osm_physp_t* const p_physp, - IN osm_pkey_search_ctxt_t* const p_ctxt, - IN uint16_t block ) +__osm_sa_pkey_create(IN osm_pkey_rec_rcv_t * const p_rcv, + IN osm_physp_t * const p_physp, + IN osm_pkey_search_ctxt_t * const p_ctxt, + IN uint16_t block) { - osm_pkey_item_t* p_rec_item; - uint16_t lid; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pkey_create ); - - p_rec_item = (osm_pkey_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_pkey_create: ERR 4602: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if (p_physp->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) - lid = p_physp->port_info.base_lid; - else - lid = osm_node_get_base_lid( p_physp->p_node, 0 ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_pkey_create: " - "New P_Key table for: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X Block:%u\n", - cl_ntoh64( osm_physp_get_port_guid( p_physp ) ), - cl_ntoh16( lid ), osm_physp_get_port_num( p_physp ), - block - ); - } - - memset( &p_rec_item->rec, 0, sizeof( p_rec_item->rec ) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.block_num = block; - p_rec_item->rec.port_num = osm_physp_get_port_num( p_physp ); - p_rec_item->rec.pkey_tbl = - *(osm_pkey_tbl_block_get(osm_physp_get_pkey_tbl(p_physp), block)); - - cl_qlist_insert_tail( p_ctxt->p_list, - (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_pkey_item_t *p_rec_item; + uint16_t lid; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pkey_create); + + p_rec_item = (osm_pkey_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_pkey_create: ERR 4602: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (p_physp->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) + lid = p_physp->port_info.base_lid; + else + lid = osm_node_get_base_lid(p_physp->p_node, 0); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_pkey_create: " + "New P_Key table for: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X Block:%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); + } + + memset(&p_rec_item->rec, 0, sizeof(p_rec_item->rec)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.block_num = block; + p_rec_item->rec.port_num = osm_physp_get_port_num(p_physp); + p_rec_item->rec.pkey_tbl = + *(osm_pkey_tbl_block_get(osm_physp_get_pkey_tbl(p_physp), block)); + + cl_qlist_insert_tail(p_ctxt->p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_pkey_check_physp( - IN osm_pkey_rec_rcv_t* const p_rcv, - IN osm_physp_t* const p_physp, - osm_pkey_search_ctxt_t* const p_ctxt ) +__osm_sa_pkey_check_physp(IN osm_pkey_rec_rcv_t * const p_rcv, + IN osm_physp_t * const p_physp, + osm_pkey_search_ctxt_t * const p_ctxt) { - ib_net64_t comp_mask = p_ctxt->comp_mask; - uint16_t block, num_blocks; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pkey_check_physp ); - - /* we got here with the phys port - all is left is to get the right block */ - if ( comp_mask & IB_PKEY_COMPMASK_BLOCK ) - { - __osm_sa_pkey_create( p_rcv, p_physp, p_ctxt, p_ctxt->block_num); - } - else - { - num_blocks = - osm_pkey_tbl_get_num_blocks( osm_physp_get_pkey_tbl( p_physp )); - for (block = 0; block < num_blocks; block++) { - __osm_sa_pkey_create( p_rcv, p_physp, p_ctxt, block ); - } - } - - OSM_LOG_EXIT( p_rcv->p_log ); + ib_net64_t comp_mask = p_ctxt->comp_mask; + uint16_t block, num_blocks; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pkey_check_physp); + + /* we got here with the phys port - all is left is to get the right block */ + if (comp_mask & IB_PKEY_COMPMASK_BLOCK) { + __osm_sa_pkey_create(p_rcv, p_physp, p_ctxt, p_ctxt->block_num); + } else { + num_blocks = + osm_pkey_tbl_get_num_blocks(osm_physp_get_pkey_tbl + (p_physp)); + for (block = 0; block < num_blocks; block++) { + __osm_sa_pkey_create(p_rcv, p_physp, p_ctxt, block); + } + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_pkey_by_comp_mask( - IN osm_pkey_rec_rcv_t* const p_rcv, - IN const osm_port_t* const p_port, - osm_pkey_search_ctxt_t* const p_ctxt ) +__osm_sa_pkey_by_comp_mask(IN osm_pkey_rec_rcv_t * const p_rcv, + IN const osm_port_t * const p_port, + osm_pkey_search_ctxt_t * const p_ctxt) { - const ib_pkey_table_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - osm_physp_t * p_physp; - uint8_t port_num; - uint8_t num_ports; - const osm_physp_t* p_req_physp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pkey_by_comp_mask ); - - p_rcvd_rec = p_ctxt->p_rcvd_rec; - comp_mask = p_ctxt->comp_mask; - port_num = p_rcvd_rec->port_num; - p_req_physp = p_ctxt->p_req_physp; - - /* if this is a switch port we can search all ports - otherwise we must be looking on port 0 */ - if ( p_port->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH ) - { - /* we put it in the comp mask and port num */ - port_num = p_port->p_physp->port_num; - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_pkey_by_comp_mask: " - "Using Physical Default Port Number: 0x%X (for End Node)\n", - port_num); - comp_mask |= IB_PKEY_COMPMASK_PORT; - } - - if( comp_mask & IB_PKEY_COMPMASK_PORT ) - { - if (port_num < osm_node_get_num_physp( p_port->p_node )) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, port_num ); - /* Check that the p_physp is valid, and that is shares a pkey - with the p_req_physp. */ - if( osm_physp_is_valid( p_physp ) && - (osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) ) - __osm_sa_pkey_check_physp( p_rcv, p_physp, p_ctxt ); - } - else - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_pkey_by_comp_mask: ERR 4603: " - "Given Physical Port Number: 0x%X is out of range should be < 0x%X\n", - port_num, osm_node_get_num_physp( p_port->p_node )); - goto Exit; - } - } - else - { - num_ports = osm_node_get_num_physp( p_port->p_node ); - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, port_num ); - if( !osm_physp_is_valid( p_physp ) ) - continue; - - /* if the requester and the p_physp don't share a pkey - - continue */ - if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp ) ) - continue; - - __osm_sa_pkey_check_physp( p_rcv, p_physp, p_ctxt ); - } - } - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_pkey_table_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + osm_physp_t *p_physp; + uint8_t port_num; + uint8_t num_ports; + const osm_physp_t *p_req_physp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pkey_by_comp_mask); + + p_rcvd_rec = p_ctxt->p_rcvd_rec; + comp_mask = p_ctxt->comp_mask; + port_num = p_rcvd_rec->port_num; + p_req_physp = p_ctxt->p_req_physp; + + /* if this is a switch port we can search all ports + otherwise we must be looking on port 0 */ + if (p_port->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) { + /* we put it in the comp mask and port num */ + port_num = p_port->p_physp->port_num; + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_pkey_by_comp_mask: " + "Using Physical Default Port Number: 0x%X (for End Node)\n", + port_num); + comp_mask |= IB_PKEY_COMPMASK_PORT; + } + + if (comp_mask & IB_PKEY_COMPMASK_PORT) { + if (port_num < osm_node_get_num_physp(p_port->p_node)) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, port_num); + /* Check that the p_physp is valid, and that is shares a pkey + with the p_req_physp. */ + if (osm_physp_is_valid(p_physp) && + (osm_physp_share_pkey + (p_rcv->p_log, p_req_physp, p_physp))) + __osm_sa_pkey_check_physp(p_rcv, p_physp, + p_ctxt); + } else { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_pkey_by_comp_mask: ERR 4603: " + "Given Physical Port Number: 0x%X is out of range should be < 0x%X\n", + port_num, + osm_node_get_num_physp(p_port->p_node)); + goto Exit; + } + } else { + num_ports = osm_node_get_num_physp(p_port->p_node); + for (port_num = 0; port_num < num_ports; port_num++) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, port_num); + if (!osm_physp_is_valid(p_physp)) + continue; + + /* if the requester and the p_physp don't share a pkey - + continue */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_req_physp, p_physp)) + continue; + + __osm_sa_pkey_check_physp(p_rcv, p_physp, p_ctxt); + } + } + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_pkey_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_pkey_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_port_t* const p_port = (osm_port_t*)p_map_item; - osm_pkey_search_ctxt_t* const p_ctxt = (osm_pkey_search_ctxt_t *)context; + const osm_port_t *const p_port = (osm_port_t *) p_map_item; + osm_pkey_search_ctxt_t *const p_ctxt = + (osm_pkey_search_ctxt_t *) context; - __osm_sa_pkey_by_comp_mask( p_ctxt->p_rcv, p_port, p_ctxt ); + __osm_sa_pkey_by_comp_mask(p_ctxt->p_rcv, p_port, p_ctxt); } /********************************************************************** **********************************************************************/ -void -osm_pkey_rec_rcv_process( - IN void *ctx, - IN void *data ) +void osm_pkey_rec_rcv_process(IN void *ctx, IN void *data) { - osm_pkey_rec_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_pkey_table_record_t* p_rcvd_rec; - const cl_ptr_vector_t* p_tbl; - const osm_port_t* p_port = NULL; - const ib_pkey_table_t* p_pkey; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_pkey_table_record_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + osm_pkey_rec_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_pkey_table_record_t *p_rcvd_rec; + const cl_ptr_vector_t *p_tbl; + const osm_port_t *p_port = NULL; + const ib_pkey_table_t *p_pkey; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_pkey_table_record_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_pkey_search_ctxt_t context; - osm_pkey_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t comp_mask; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_pkey_rec_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_pkey_table_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - comp_mask = p_rcvd_mad->comp_mask; - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_PKEY_TBL_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 4605: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* - p922 - P_KeyTableRecords shall only be provided in response - to trusted requests. - Check that the requester is a trusted one. - */ - if ( p_rcvd_mad->sm_key != p_rcv->p_subn->opt.sm_key ) - { - /* This is not a trusted requester! */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process ERR 4608: " - "Request from non-trusted requester: " - "Given SM_Key:0x%016" PRIx64 "\n", - cl_ntoh64(p_rcvd_mad->sm_key) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_REQ_INVALID ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 4604: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - p_pkey = (ib_pkey_table_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.block_num = p_rcvd_rec->block_num; - context.p_req_physp = p_req_physp; - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_pkey_rec_rcv_process: " - "Got Query Lid:0x%04X(%02X), Block:0x%02X(%02X), Port:0x%02X(%02X)\n", - cl_ntoh16(p_rcvd_rec->lid), (comp_mask & IB_PKEY_COMPMASK_LID) != 0, - p_rcvd_rec->port_num, (comp_mask & IB_PKEY_COMPMASK_PORT) != 0, - p_rcvd_rec->block_num, (comp_mask & IB_PKEY_COMPMASK_BLOCK) != 0 ); - - cl_plock_acquire( p_rcv->p_lock ); - - /* - If the user specified a LID, it obviously narrows our - work load, since we don't have to search every port - */ - if( comp_mask & IB_PKEY_COMPMASK_LID ) - { - - p_tbl = &p_rcv->p_subn->port_lid_tbl; - - CL_ASSERT( cl_ptr_vector_get_size(p_tbl) < 0x10000 ); - - status = osm_get_port_by_base_lid( p_rcv->p_subn, p_rcvd_rec->lid, &p_port ); - if ( ( status != IB_SUCCESS ) || ( p_port == NULL ) ) - { - status = IB_NOT_FOUND; - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 460B: " - "No port found with LID 0x%x\n", - cl_ntoh16(p_rcvd_rec->lid) ); - } - } - - if ( status == IB_SUCCESS ) - { - /* if we got a unique port - no need for a port search */ - if( p_port ) - /* this does the loop on all the port phys ports */ - __osm_sa_pkey_by_comp_mask( p_rcv, p_port, &context ); - else - { - cl_qmap_apply_func( &p_rcv->p_subn->port_guid_tbl, - __osm_sa_pkey_by_comp_mask_cb, - &context ); - } - } - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 460A: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_pkey_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_pkey_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_pkey_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_pkey_search_ctxt_t context; + osm_pkey_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t comp_mask; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_pkey_rec_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_pkey_table_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + comp_mask = p_rcvd_mad->comp_mask; + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_PKEY_TBL_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 4605: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* + p922 - P_KeyTableRecords shall only be provided in response + to trusted requests. + Check that the requester is a trusted one. + */ + if (p_rcvd_mad->sm_key != p_rcv->p_subn->opt.sm_key) { + /* This is not a trusted requester! */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process ERR 4608: " + "Request from non-trusted requester: " + "Given SM_Key:0x%016" PRIx64 "\n", + cl_ntoh64(p_rcvd_mad->sm_key)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 4604: " + "Cannot find requester physical port\n"); + goto Exit; + } + + p_pkey = (ib_pkey_table_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.block_num = p_rcvd_rec->block_num; + context.p_req_physp = p_req_physp; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_pkey_rec_rcv_process: " + "Got Query Lid:0x%04X(%02X), Block:0x%02X(%02X), Port:0x%02X(%02X)\n", + cl_ntoh16(p_rcvd_rec->lid), + (comp_mask & IB_PKEY_COMPMASK_LID) != 0, p_rcvd_rec->port_num, + (comp_mask & IB_PKEY_COMPMASK_PORT) != 0, p_rcvd_rec->block_num, + (comp_mask & IB_PKEY_COMPMASK_BLOCK) != 0); + + cl_plock_acquire(p_rcv->p_lock); + + /* + If the user specified a LID, it obviously narrows our + work load, since we don't have to search every port + */ + if (comp_mask & IB_PKEY_COMPMASK_LID) { + + p_tbl = &p_rcv->p_subn->port_lid_tbl; + + CL_ASSERT(cl_ptr_vector_get_size(p_tbl) < 0x10000); + + status = + osm_get_port_by_base_lid(p_rcv->p_subn, p_rcvd_rec->lid, + &p_port); + if ((status != IB_SUCCESS) || (p_port == NULL)) { + status = IB_NOT_FOUND; + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 460B: " + "No port found with LID 0x%x\n", + cl_ntoh16(p_rcvd_rec->lid)); + } + } + + if (status == IB_SUCCESS) { + /* if we got a unique port - no need for a port search */ + if (p_port) + /* this does the loop on all the port phys ports */ + __osm_sa_pkey_by_comp_mask(p_rcv, p_port, &context); + else { + cl_qmap_apply_func(&p_rcv->p_subn->port_guid_tbl, + __osm_sa_pkey_by_comp_mask_cb, + &context); + } + } + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 460A: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_pkey_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_pkey_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_pkey_table_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_pkey_rec_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_pkey_table_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_pkey_rec_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_pkey_rec_rcv_process: " - "Returning %u records\n", num_rec ); - - if((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_pkey_table_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 4606: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_pkey_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_pkey_table_record_t) ); - - p_resp_rec = (ib_pkey_table_record_t*) - ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_pkey_rec_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * + sizeof(ib_pkey_table_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 4606: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_pkey_table_record_t)); + + p_resp_rec = (ib_pkey_table_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_pkey_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_pkey_rec_rcv_process: ERR 4607: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_pkey_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pkey_rec_rcv_process: ERR 4607: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_portinfo_record.c b/opensm/opensm/osm_sa_portinfo_record.c index d2fe32c..1483d5f 100644 --- a/opensm/opensm/osm_sa_portinfo_record.c +++ b/opensm/opensm/osm_sa_portinfo_record.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_pir_rcv_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -67,811 +66,746 @@ #define OSM_PIR_RCV_POOL_MIN_SIZE 32 #define OSM_PIR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_pir_item -{ - cl_pool_item_t pool_item; - ib_portinfo_record_t rec; +typedef struct _osm_pir_item { + cl_pool_item_t pool_item; + ib_portinfo_record_t rec; } osm_pir_item_t; -typedef struct _osm_pir_search_ctxt -{ - const ib_portinfo_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_pir_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; - boolean_t is_enhanced_comp_mask; +typedef struct _osm_pir_search_ctxt { + const ib_portinfo_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_pir_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; + boolean_t is_enhanced_comp_mask; } osm_pir_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_pir_rcv_construct( - IN osm_pir_rcv_t* const p_rcv ) +void osm_pir_rcv_construct(IN osm_pir_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_pir_rcv_destroy( - IN osm_pir_rcv_t* const p_rcv ) +void osm_pir_rcv_destroy(IN osm_pir_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_pir_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_pir_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_pir_rcv_init( - IN osm_pir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_pir_rcv_init(IN osm_pir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_pir_rcv_init ); + OSM_LOG_ENTER(p_log, osm_pir_rcv_init); - osm_pir_rcv_construct( p_rcv ); + osm_pir_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_PIR_RCV_POOL_MIN_SIZE, - 0, - OSM_PIR_RCV_POOL_GROW_SIZE, - sizeof(osm_pir_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_PIR_RCV_POOL_MIN_SIZE, + 0, + OSM_PIR_RCV_POOL_GROW_SIZE, + sizeof(osm_pir_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_pir_rcv_new_pir( - IN osm_pir_rcv_t* const p_rcv, - IN const osm_physp_t* const p_physp, - IN cl_qlist_t* const p_list, - IN ib_net16_t const lid ) +__osm_pir_rcv_new_pir(IN osm_pir_rcv_t * const p_rcv, + IN const osm_physp_t * const p_physp, + IN cl_qlist_t * const p_list, IN ib_net16_t const lid) { - osm_pir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_pir_rcv_new_pir ); - - p_rec_item = (osm_pir_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_pir_rcv_new_pir: ERR 2102: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_pir_rcv_new_pir: " - "New PortInfoRecord: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X\n", - cl_ntoh64( osm_physp_get_port_guid( p_physp ) ), - cl_ntoh16( lid ), osm_physp_get_port_num( p_physp ) ); - } - - memset( &p_rec_item->rec, 0, sizeof( p_rec_item->rec ) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.port_info = p_physp->port_info; - p_rec_item->rec.port_num = osm_physp_get_port_num( p_physp ); - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_pir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_pir_rcv_new_pir); + + p_rec_item = (osm_pir_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_pir_rcv_new_pir: ERR 2102: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_pir_rcv_new_pir: " + "New PortInfoRecord: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp)); + } + + memset(&p_rec_item->rec, 0, sizeof(p_rec_item->rec)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.port_info = p_physp->port_info; + p_rec_item->rec.port_num = osm_physp_get_port_num(p_physp); + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -__osm_sa_pir_create( - IN osm_pir_rcv_t* const p_rcv, - IN const osm_physp_t* const p_physp, - IN osm_pir_search_ctxt_t* const p_ctxt ) +__osm_sa_pir_create(IN osm_pir_rcv_t * const p_rcv, + IN const osm_physp_t * const p_physp, + IN osm_pir_search_ctxt_t * const p_ctxt) { - uint8_t lmc; - uint16_t max_lid_ho; - uint16_t base_lid_ho; - uint16_t match_lid_ho; - osm_physp_t *p_node_physp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pir_create ); - - if (p_physp->p_node->sw) - { - p_node_physp = osm_node_get_physp_ptr( p_physp->p_node, 0 ); - base_lid_ho = cl_ntoh16( osm_physp_get_base_lid( p_node_physp ) ); - lmc = osm_switch_sp0_is_lmc_capable(p_physp->p_node->sw, p_rcv->p_subn) ? - osm_physp_get_lmc( p_node_physp ) : 0; - } - else - { - lmc = osm_physp_get_lmc( p_physp ); - base_lid_ho = cl_ntoh16( osm_physp_get_base_lid( p_physp ) ); - } - max_lid_ho = (uint16_t)( base_lid_ho + (1 << lmc) - 1 ); - - if( p_ctxt->comp_mask & IB_PIR_COMPMASK_LID ) - { - match_lid_ho = cl_ntoh16( p_ctxt->p_rcvd_rec->lid ); - - /* - We validate that the lid belongs to this node. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_pir_create: " - "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", - base_lid_ho, match_lid_ho, max_lid_ho - ); - } - - if ( match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho ) - goto Exit; - } - - __osm_pir_rcv_new_pir( p_rcv, p_physp, p_ctxt->p_list, - cl_hton16( base_lid_ho ) ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + uint8_t lmc; + uint16_t max_lid_ho; + uint16_t base_lid_ho; + uint16_t match_lid_ho; + osm_physp_t *p_node_physp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pir_create); + + if (p_physp->p_node->sw) { + p_node_physp = osm_node_get_physp_ptr(p_physp->p_node, 0); + base_lid_ho = cl_ntoh16(osm_physp_get_base_lid(p_node_physp)); + lmc = + osm_switch_sp0_is_lmc_capable(p_physp->p_node->sw, + p_rcv-> + p_subn) ? + osm_physp_get_lmc(p_node_physp) : 0; + } else { + lmc = osm_physp_get_lmc(p_physp); + base_lid_ho = cl_ntoh16(osm_physp_get_base_lid(p_physp)); + } + max_lid_ho = (uint16_t) (base_lid_ho + (1 << lmc) - 1); + + if (p_ctxt->comp_mask & IB_PIR_COMPMASK_LID) { + match_lid_ho = cl_ntoh16(p_ctxt->p_rcvd_rec->lid); + + /* + We validate that the lid belongs to this node. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_pir_create: " + "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", + base_lid_ho, match_lid_ho, max_lid_ho); + } + + if (match_lid_ho < base_lid_ho || match_lid_ho > max_lid_ho) + goto Exit; + } + + __osm_pir_rcv_new_pir(p_rcv, p_physp, p_ctxt->p_list, + cl_hton16(base_lid_ho)); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_pir_check_physp( - IN osm_pir_rcv_t* const p_rcv, - IN const osm_physp_t* const p_physp, - osm_pir_search_ctxt_t* const p_ctxt ) +__osm_sa_pir_check_physp(IN osm_pir_rcv_t * const p_rcv, + IN const osm_physp_t * const p_physp, + osm_pir_search_ctxt_t * const p_ctxt) { - const ib_portinfo_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - const ib_port_info_t* p_comp_pi; - const ib_port_info_t* p_pi; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pir_check_physp ); - - p_rcvd_rec = p_ctxt->p_rcvd_rec; - comp_mask = p_ctxt->comp_mask; - p_comp_pi = &p_rcvd_rec->port_info; - p_pi = &p_physp->port_info; - - osm_dump_port_info( - p_rcv->p_log, - osm_node_get_node_guid( p_physp->p_node ), - p_physp->port_guid, - p_physp->port_num, - &p_physp->port_info, - OSM_LOG_DEBUG ); - - /* We have to re-check the base_lid, since if the given - base_lid in p_pi is zero - we are comparing on all ports. */ - if( comp_mask & IB_PIR_COMPMASK_BASELID ) - { - if( p_comp_pi->base_lid != p_pi->base_lid ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_MKEY ) - { - if( p_comp_pi->m_key != p_pi->m_key ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_GIDPRE ) - { - if( p_comp_pi->subnet_prefix != p_pi->subnet_prefix ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_SMLID ) - { - if( p_comp_pi->master_sm_base_lid != p_pi->master_sm_base_lid ) - goto Exit; - } - - /* IBTA 1.2 errata provides support for bitwise compare if the bit 31 - of the attribute modifier of the Get/GetTable is set */ - if( comp_mask & IB_PIR_COMPMASK_CAPMASK ) - { - if (p_ctxt->is_enhanced_comp_mask) - { - if ( ( ( p_comp_pi->capability_mask & p_pi->capability_mask ) != p_comp_pi->capability_mask) ) - goto Exit; - } - else - { - if( p_comp_pi->capability_mask != p_pi->capability_mask ) - goto Exit; - } - } - - if( comp_mask & IB_PIR_COMPMASK_DIAGCODE ) - { - if( p_comp_pi->diag_code != p_pi->diag_code ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_MKEYLEASEPRD ) - { - if( p_comp_pi->m_key_lease_period != p_pi->m_key_lease_period ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LOCALPORTNUM ) - { - if( p_comp_pi->local_port_num != p_pi->local_port_num ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LNKWIDTHSUPPORT ) - { - if( p_comp_pi->link_width_supported != p_pi->link_width_supported ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LNKWIDTHACTIVE ) - { - if( p_comp_pi->link_width_active != p_pi->link_width_active ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LINKWIDTHENABLED ) - { - if( p_comp_pi->link_width_enabled != p_pi->link_width_enabled ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LNKSPEEDSUPPORT ) - { - if( ib_port_info_get_link_speed_sup( p_comp_pi )!= - ib_port_info_get_link_speed_sup( p_pi) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_PORTSTATE ) - { - if( ib_port_info_get_port_state( p_comp_pi ) != - ib_port_info_get_port_state( p_pi ) ) - goto Exit; - } - if ( comp_mask & IB_PIR_COMPMASK_PORTPHYSTATE ) - { - if ( ib_port_info_get_port_phys_state( p_comp_pi ) != - ib_port_info_get_port_phys_state( p_pi ) ) - goto Exit; - } - if ( comp_mask & IB_PIR_COMPMASK_LINKDWNDFLTSTATE ) - { - if ( ib_port_info_get_link_down_def_state( p_comp_pi ) != - ib_port_info_get_link_down_def_state( p_pi ) ) - goto Exit; - } - if ( comp_mask & IB_PIR_COMPMASK_MKEYPROTBITS ) - { - if( ib_port_info_get_mpb( p_comp_pi ) != - ib_port_info_get_mpb( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_LMC ) - { - if( ib_port_info_get_lmc( p_comp_pi ) != - ib_port_info_get_lmc( p_pi ) ) - goto Exit; - } - if ( comp_mask & IB_PIR_COMPMASK_LINKSPEEDACTIVE ) - { - if ( ib_port_info_get_link_speed_active( p_comp_pi ) != - ib_port_info_get_link_speed_active( p_pi ) ) - goto Exit; - } - if ( comp_mask & IB_PIR_COMPMASK_LINKSPEEDENABLE ) - { - if ( ib_port_info_get_link_speed_enabled( p_comp_pi ) != - ib_port_info_get_link_speed_enabled( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_NEIGHBORMTU ) - { - if( ib_port_info_get_neighbor_mtu( p_comp_pi ) != - ib_port_info_get_neighbor_mtu( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_MASTERSMSL ) - { - if( ib_port_info_get_master_smsl( p_comp_pi ) != - ib_port_info_get_master_smsl( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_VLCAP ) - { - if( ib_port_info_get_vl_cap( p_comp_pi ) != - ib_port_info_get_vl_cap( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_INITTYPE ) - { - if( ib_port_info_get_init_type( p_comp_pi ) != - ib_port_info_get_init_type( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_VLHIGHLIMIT ) - { - if( p_comp_pi->vl_high_limit != p_pi->vl_high_limit ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_VLARBHIGHCAP ) - { - if( p_comp_pi->vl_arb_high_cap != p_pi->vl_arb_high_cap ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_VLARBLOWCAP ) - { - if( p_comp_pi->vl_arb_low_cap != p_pi->vl_arb_low_cap ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_MTUCAP ) - { - if( ib_port_info_get_mtu_cap( p_comp_pi ) != - ib_port_info_get_mtu_cap( p_pi ) ) - goto Exit; - } - if( comp_mask & IB_PIR_COMPMASK_VLSTALLCNT ) - { - if( ib_port_info_get_vl_stall_count( p_comp_pi ) != - ib_port_info_get_vl_stall_count( p_pi ) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_HOQLIFE ) - { - if ((p_comp_pi->vl_stall_life & 0x1F) != (p_pi->vl_stall_life & 0x1F) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_OPVLS ) - { - if ((p_comp_pi->vl_enforce & 0xF0) != (p_pi->vl_enforce & 0xF0) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_PARENFIN ) - { - if ((p_comp_pi->vl_enforce & 0x08) != (p_pi->vl_enforce & 0x08) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_PARENFOUT ) - { - if ((p_comp_pi->vl_enforce & 0x04) != (p_pi->vl_enforce & 0x04) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_FILTERRAWIN ) - { - if ((p_comp_pi->vl_enforce & 0x02) != (p_pi->vl_enforce & 0x02) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_FILTERRAWOUT ) - { - if ((p_comp_pi->vl_enforce & 0x01) != (p_pi->vl_enforce & 0x01) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_MKEYVIO ) - { - if (p_comp_pi->m_key_violations != p_pi->m_key_violations ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_PKEYVIO ) - { - if (p_comp_pi->p_key_violations != p_pi->p_key_violations ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_QKEYVIO ) - { - if (p_comp_pi->q_key_violations != p_pi->q_key_violations ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_GUIDCAP ) - { - if (p_comp_pi->guid_cap != p_pi->guid_cap ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_SUBNTO ) - { - if (ib_port_info_get_timeout(p_comp_pi) != ib_port_info_get_timeout(p_pi)) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_RESPTIME ) - { - if ((p_comp_pi->resp_time_value & 0x1F) != (p_pi->resp_time_value &0x1F) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_LOCALPHYERR ) - { - if( ib_port_info_get_local_phy_err_thd( p_comp_pi ) != - ib_port_info_get_local_phy_err_thd( p_pi ) ) - goto Exit; - } - if (comp_mask & IB_PIR_COMPMASK_OVERRUNERR) - { - if( ib_port_info_get_overrun_err_thd( p_comp_pi ) != - ib_port_info_get_overrun_err_thd( p_pi ) ) - goto Exit; - } - - __osm_sa_pir_create( p_rcv, p_physp, p_ctxt ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_portinfo_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + const ib_port_info_t *p_comp_pi; + const ib_port_info_t *p_pi; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pir_check_physp); + + p_rcvd_rec = p_ctxt->p_rcvd_rec; + comp_mask = p_ctxt->comp_mask; + p_comp_pi = &p_rcvd_rec->port_info; + p_pi = &p_physp->port_info; + + osm_dump_port_info(p_rcv->p_log, + osm_node_get_node_guid(p_physp->p_node), + p_physp->port_guid, + p_physp->port_num, + &p_physp->port_info, OSM_LOG_DEBUG); + + /* We have to re-check the base_lid, since if the given + base_lid in p_pi is zero - we are comparing on all ports. */ + if (comp_mask & IB_PIR_COMPMASK_BASELID) { + if (p_comp_pi->base_lid != p_pi->base_lid) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MKEY) { + if (p_comp_pi->m_key != p_pi->m_key) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_GIDPRE) { + if (p_comp_pi->subnet_prefix != p_pi->subnet_prefix) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_SMLID) { + if (p_comp_pi->master_sm_base_lid != p_pi->master_sm_base_lid) + goto Exit; + } + + /* IBTA 1.2 errata provides support for bitwise compare if the bit 31 + of the attribute modifier of the Get/GetTable is set */ + if (comp_mask & IB_PIR_COMPMASK_CAPMASK) { + if (p_ctxt->is_enhanced_comp_mask) { + if (((p_comp_pi->capability_mask & p_pi-> + capability_mask) != p_comp_pi->capability_mask)) + goto Exit; + } else { + if (p_comp_pi->capability_mask != p_pi->capability_mask) + goto Exit; + } + } + + if (comp_mask & IB_PIR_COMPMASK_DIAGCODE) { + if (p_comp_pi->diag_code != p_pi->diag_code) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MKEYLEASEPRD) { + if (p_comp_pi->m_key_lease_period != p_pi->m_key_lease_period) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LOCALPORTNUM) { + if (p_comp_pi->local_port_num != p_pi->local_port_num) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LNKWIDTHSUPPORT) { + if (p_comp_pi->link_width_supported != + p_pi->link_width_supported) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LNKWIDTHACTIVE) { + if (p_comp_pi->link_width_active != p_pi->link_width_active) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LINKWIDTHENABLED) { + if (p_comp_pi->link_width_enabled != p_pi->link_width_enabled) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LNKSPEEDSUPPORT) { + if (ib_port_info_get_link_speed_sup(p_comp_pi) != + ib_port_info_get_link_speed_sup(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_PORTSTATE) { + if (ib_port_info_get_port_state(p_comp_pi) != + ib_port_info_get_port_state(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_PORTPHYSTATE) { + if (ib_port_info_get_port_phys_state(p_comp_pi) != + ib_port_info_get_port_phys_state(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LINKDWNDFLTSTATE) { + if (ib_port_info_get_link_down_def_state(p_comp_pi) != + ib_port_info_get_link_down_def_state(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MKEYPROTBITS) { + if (ib_port_info_get_mpb(p_comp_pi) != + ib_port_info_get_mpb(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LMC) { + if (ib_port_info_get_lmc(p_comp_pi) != + ib_port_info_get_lmc(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LINKSPEEDACTIVE) { + if (ib_port_info_get_link_speed_active(p_comp_pi) != + ib_port_info_get_link_speed_active(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LINKSPEEDENABLE) { + if (ib_port_info_get_link_speed_enabled(p_comp_pi) != + ib_port_info_get_link_speed_enabled(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_NEIGHBORMTU) { + if (ib_port_info_get_neighbor_mtu(p_comp_pi) != + ib_port_info_get_neighbor_mtu(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MASTERSMSL) { + if (ib_port_info_get_master_smsl(p_comp_pi) != + ib_port_info_get_master_smsl(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_VLCAP) { + if (ib_port_info_get_vl_cap(p_comp_pi) != + ib_port_info_get_vl_cap(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_INITTYPE) { + if (ib_port_info_get_init_type(p_comp_pi) != + ib_port_info_get_init_type(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_VLHIGHLIMIT) { + if (p_comp_pi->vl_high_limit != p_pi->vl_high_limit) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_VLARBHIGHCAP) { + if (p_comp_pi->vl_arb_high_cap != p_pi->vl_arb_high_cap) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_VLARBLOWCAP) { + if (p_comp_pi->vl_arb_low_cap != p_pi->vl_arb_low_cap) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MTUCAP) { + if (ib_port_info_get_mtu_cap(p_comp_pi) != + ib_port_info_get_mtu_cap(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_VLSTALLCNT) { + if (ib_port_info_get_vl_stall_count(p_comp_pi) != + ib_port_info_get_vl_stall_count(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_HOQLIFE) { + if ((p_comp_pi->vl_stall_life & 0x1F) != + (p_pi->vl_stall_life & 0x1F)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_OPVLS) { + if ((p_comp_pi->vl_enforce & 0xF0) != (p_pi->vl_enforce & 0xF0)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_PARENFIN) { + if ((p_comp_pi->vl_enforce & 0x08) != (p_pi->vl_enforce & 0x08)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_PARENFOUT) { + if ((p_comp_pi->vl_enforce & 0x04) != (p_pi->vl_enforce & 0x04)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_FILTERRAWIN) { + if ((p_comp_pi->vl_enforce & 0x02) != (p_pi->vl_enforce & 0x02)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_FILTERRAWOUT) { + if ((p_comp_pi->vl_enforce & 0x01) != (p_pi->vl_enforce & 0x01)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_MKEYVIO) { + if (p_comp_pi->m_key_violations != p_pi->m_key_violations) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_PKEYVIO) { + if (p_comp_pi->p_key_violations != p_pi->p_key_violations) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_QKEYVIO) { + if (p_comp_pi->q_key_violations != p_pi->q_key_violations) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_GUIDCAP) { + if (p_comp_pi->guid_cap != p_pi->guid_cap) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_SUBNTO) { + if (ib_port_info_get_timeout(p_comp_pi) != + ib_port_info_get_timeout(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_RESPTIME) { + if ((p_comp_pi->resp_time_value & 0x1F) != + (p_pi->resp_time_value & 0x1F)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_LOCALPHYERR) { + if (ib_port_info_get_local_phy_err_thd(p_comp_pi) != + ib_port_info_get_local_phy_err_thd(p_pi)) + goto Exit; + } + if (comp_mask & IB_PIR_COMPMASK_OVERRUNERR) { + if (ib_port_info_get_overrun_err_thd(p_comp_pi) != + ib_port_info_get_overrun_err_thd(p_pi)) + goto Exit; + } + + __osm_sa_pir_create(p_rcv, p_physp, p_ctxt); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_pir_by_comp_mask( - IN osm_pir_rcv_t* const p_rcv, - IN const osm_port_t* const p_port, - osm_pir_search_ctxt_t* const p_ctxt ) +__osm_sa_pir_by_comp_mask(IN osm_pir_rcv_t * const p_rcv, + IN const osm_port_t * const p_port, + osm_pir_search_ctxt_t * const p_ctxt) { - const ib_portinfo_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - const osm_physp_t* p_physp; - uint8_t port_num; - uint8_t num_ports; - const osm_physp_t* p_req_physp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_pir_by_comp_mask ); - - p_rcvd_rec = p_ctxt->p_rcvd_rec; - comp_mask = p_ctxt->comp_mask; - p_req_physp = p_ctxt->p_req_physp; - - num_ports = osm_node_get_num_physp( p_port->p_node ); - - if( comp_mask & IB_PIR_COMPMASK_PORTNUM ) - { - if (p_rcvd_rec->port_num < num_ports) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, p_rcvd_rec->port_num ); - /* Check that the p_physp is valid, and that the p_physp and the - p_req_physp share a pkey. */ - if( osm_physp_is_valid( p_physp ) && - osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) - __osm_sa_pir_check_physp( p_rcv, p_physp, p_ctxt ); - } - } - else - { - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, port_num ); - if( !osm_physp_is_valid( p_physp ) ) - continue; - - /* if the requester and the p_physp don't share a pkey - - continue */ - if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp ) ) - continue; - - __osm_sa_pir_check_physp( p_rcv, p_physp, p_ctxt ); - } - } - - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_portinfo_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + const osm_physp_t *p_physp; + uint8_t port_num; + uint8_t num_ports; + const osm_physp_t *p_req_physp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_pir_by_comp_mask); + + p_rcvd_rec = p_ctxt->p_rcvd_rec; + comp_mask = p_ctxt->comp_mask; + p_req_physp = p_ctxt->p_req_physp; + + num_ports = osm_node_get_num_physp(p_port->p_node); + + if (comp_mask & IB_PIR_COMPMASK_PORTNUM) { + if (p_rcvd_rec->port_num < num_ports) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, + p_rcvd_rec->port_num); + /* Check that the p_physp is valid, and that the p_physp and the + p_req_physp share a pkey. */ + if (osm_physp_is_valid(p_physp) && + osm_physp_share_pkey(p_rcv->p_log, p_req_physp, + p_physp)) + __osm_sa_pir_check_physp(p_rcv, p_physp, + p_ctxt); + } + } else { + for (port_num = 0; port_num < num_ports; port_num++) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, port_num); + if (!osm_physp_is_valid(p_physp)) + continue; + + /* if the requester and the p_physp don't share a pkey - + continue */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_req_physp, p_physp)) + continue; + + __osm_sa_pir_check_physp(p_rcv, p_physp, p_ctxt); + } + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_pir_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_pir_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_port_t* const p_port = (osm_port_t*)p_map_item; - osm_pir_search_ctxt_t* const p_ctxt = (osm_pir_search_ctxt_t *)context; + const osm_port_t *const p_port = (osm_port_t *) p_map_item; + osm_pir_search_ctxt_t *const p_ctxt = (osm_pir_search_ctxt_t *) context; - __osm_sa_pir_by_comp_mask( p_ctxt->p_rcv, p_port, p_ctxt ); + __osm_sa_pir_by_comp_mask(p_ctxt->p_rcv, p_port, p_ctxt); } /********************************************************************** **********************************************************************/ -void -osm_pir_rcv_process( - IN void *ctx, - IN void *data ) +void osm_pir_rcv_process(IN void *ctx, IN void *data) { - osm_pir_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_portinfo_record_t* p_rcvd_rec; - const cl_ptr_vector_t* p_tbl; - const osm_port_t* p_port = NULL; - const ib_port_info_t* p_pi; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_portinfo_record_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + osm_pir_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_portinfo_record_t *p_rcvd_rec; + const cl_ptr_vector_t *p_tbl; + const osm_port_t *p_port = NULL; + const ib_port_info_t *p_pi; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_portinfo_record_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_pir_search_ctxt_t context; - osm_pir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t comp_mask; - osm_physp_t* p_req_physp; - boolean_t trusted_req = TRUE; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_pir_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_portinfo_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - comp_mask = p_rcvd_mad->comp_mask; - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_PORTINFO_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2105: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2104: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_portinfo_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - p_tbl = &p_rcv->p_subn->port_lid_tbl; - p_pi = &p_rcvd_rec->port_info; - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - context.is_enhanced_comp_mask = cl_ntoh32(p_rcvd_mad->attr_mod) & (1 << 31); - - cl_plock_acquire( p_rcv->p_lock ); - - CL_ASSERT( cl_ptr_vector_get_size(p_tbl) < 0x10000 ); - - /* - If the user specified a LID, it obviously narrows our - work load, since we don't have to search every port - */ - if( comp_mask & IB_PIR_COMPMASK_LID ) - { - status = osm_get_port_by_base_lid( p_rcv->p_subn, p_rcvd_rec->lid, &p_port ); - if ( ( status != IB_SUCCESS ) || ( p_port == NULL ) ) - { - status = IB_NOT_FOUND; - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2109: " - "No port found with LID 0x%x\n", - cl_ntoh16(p_rcvd_rec->lid) ); - } - } - else - { - if( comp_mask & IB_PIR_COMPMASK_BASELID ) - { - if ((uint16_t)cl_ptr_vector_get_size(p_tbl) > cl_ntoh16(p_pi->base_lid)) - { - p_port = cl_ptr_vector_get( p_tbl, cl_ntoh16(p_pi->base_lid) ); - } - else - { - status = IB_NOT_FOUND; - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2103: " - "Given LID (0x%X) is out of range:0x%X\n", - cl_ntoh16(p_pi->base_lid), cl_ptr_vector_get_size(p_tbl)); - } - } - } - - if ( status == IB_SUCCESS ) - { - if( p_port ) - __osm_sa_pir_by_comp_mask( p_rcv, p_port, &context ); - else - { - cl_qmap_apply_func( &p_rcv->p_subn->port_guid_tbl, - __osm_sa_pir_by_comp_mask_cb, - &context ); - } - } - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2108: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_pir_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_pir_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_pir_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_pir_search_ctxt_t context; + osm_pir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t comp_mask; + osm_physp_t *p_req_physp; + boolean_t trusted_req = TRUE; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_pir_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_portinfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + comp_mask = p_rcvd_mad->comp_mask; + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_PORTINFO_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2105: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2104: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_portinfo_record(p_rcv->p_log, p_rcvd_rec, + OSM_LOG_DEBUG); + + p_tbl = &p_rcv->p_subn->port_lid_tbl; + p_pi = &p_rcvd_rec->port_info; + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + context.is_enhanced_comp_mask = + cl_ntoh32(p_rcvd_mad->attr_mod) & (1 << 31); + + cl_plock_acquire(p_rcv->p_lock); + + CL_ASSERT(cl_ptr_vector_get_size(p_tbl) < 0x10000); + + /* + If the user specified a LID, it obviously narrows our + work load, since we don't have to search every port + */ + if (comp_mask & IB_PIR_COMPMASK_LID) { + status = + osm_get_port_by_base_lid(p_rcv->p_subn, p_rcvd_rec->lid, + &p_port); + if ((status != IB_SUCCESS) || (p_port == NULL)) { + status = IB_NOT_FOUND; + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2109: " + "No port found with LID 0x%x\n", + cl_ntoh16(p_rcvd_rec->lid)); + } + } else { + if (comp_mask & IB_PIR_COMPMASK_BASELID) { + if ((uint16_t) cl_ptr_vector_get_size(p_tbl) > + cl_ntoh16(p_pi->base_lid)) { + p_port = + cl_ptr_vector_get(p_tbl, + cl_ntoh16(p_pi-> + base_lid)); + } else { + status = IB_NOT_FOUND; + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2103: " + "Given LID (0x%X) is out of range:0x%X\n", + cl_ntoh16(p_pi->base_lid), + cl_ptr_vector_get_size(p_tbl)); + } + } + } + + if (status == IB_SUCCESS) { + if (p_port) + __osm_sa_pir_by_comp_mask(p_rcv, p_port, &context); + else { + cl_qmap_apply_func(&p_rcv->p_subn->port_guid_tbl, + __osm_sa_pir_by_comp_mask_cb, + &context); + } + } + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2108: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_pir_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_pir_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_portinfo_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_pir_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_portinfo_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_pir_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_pir_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_portinfo_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2106: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_pir_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_portinfo_record_t) ); - - p_resp_rec = (ib_portinfo_record_t*) - ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_pir_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_portinfo_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2106: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_portinfo_record_t)); + + p_resp_rec = (ib_portinfo_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - /* - p922 - The M_Key returned shall be zero, except in the case of a - trusted request. - Note: In the mad controller we check that the SM_Key received on - the mad is valid. Meaning - is either zero or equal to the local - sm_key. - */ - if (p_rcvd_mad->sm_key == 0) - trusted_req = FALSE; - - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_pir_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - if (trusted_req == FALSE) - p_resp_rec->port_info.m_key = 0; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_pir_rcv_process: ERR 2107: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + /* + p922 - The M_Key returned shall be zero, except in the case of a + trusted request. + Note: In the mad controller we check that the SM_Key received on + the mad is valid. Meaning - is either zero or equal to the local + sm_key. + */ + if (p_rcvd_mad->sm_key == 0) + trusted_req = FALSE; + + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = (osm_pir_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + if (trusted_req == FALSE) + p_resp_rec->port_info.m_key = 0; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_pir_rcv_process: ERR 2107: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_response.c b/opensm/opensm/osm_sa_response.c index dc614c9..d84b969 100644 --- a/opensm/opensm/osm_sa_response.c +++ b/opensm/opensm/osm_sa_response.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -59,117 +59,108 @@ /********************************************************************** **********************************************************************/ -void -osm_sa_resp_construct( - IN osm_sa_resp_t* const p_resp ) +void osm_sa_resp_construct(IN osm_sa_resp_t * const p_resp) { - memset( p_resp, 0, sizeof(*p_resp) ); + memset(p_resp, 0, sizeof(*p_resp)); } /********************************************************************** **********************************************************************/ -void -osm_sa_resp_destroy( - IN osm_sa_resp_t* const p_resp ) +void osm_sa_resp_destroy(IN osm_sa_resp_t * const p_resp) { - CL_ASSERT( p_resp ); + CL_ASSERT(p_resp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sa_resp_init( - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_pool, - IN osm_log_t* const p_log ) +osm_sa_resp_init(IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_pool, IN osm_log_t * const p_log) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_sa_resp_init ); + OSM_LOG_ENTER(p_log, osm_sa_resp_init); - osm_sa_resp_construct( p_resp ); + osm_sa_resp_construct(p_resp); - p_resp->p_log = p_log; - p_resp->p_pool = p_pool; + p_resp->p_log = p_log; + p_resp->p_pool = p_pool; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ void -osm_sa_send_error( - IN osm_sa_resp_t* const p_resp, - IN const osm_madw_t* const p_madw, - IN const ib_net16_t sa_status ) +osm_sa_send_error(IN osm_sa_resp_t * const p_resp, + IN const osm_madw_t * const p_madw, + IN const ib_net16_t sa_status) { - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_sa_mad_t* p_sa_mad; - ib_api_status_t status; - - OSM_LOG_ENTER( p_resp->p_log, osm_sa_send_error ); - - /* avoid races - if we are exiting - exit */ - if (osm_exit_flag) - { - osm_log( p_resp->p_log, OSM_LOG_DEBUG, - "osm_sa_send_error: " - "Ignoring requested send after exit\n" ); - goto Exit; - } - - p_resp_madw = osm_mad_pool_get( p_resp->p_pool, - p_madw->h_bind, MAD_BLOCK_SIZE, &p_madw->mad_addr ); - - if( p_resp_madw == NULL ) - { - osm_log( p_resp->p_log, OSM_LOG_ERROR, - "osm_sa_send_error: ERR 2301: " - "Unable to acquire response MAD\n" ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - /* Copy the MAD header back into the response mad */ - *p_resp_sa_mad = *p_sa_mad; - p_resp_sa_mad->status = sa_status; - - if( p_resp_sa_mad->method == IB_MAD_METHOD_SET ) - p_resp_sa_mad->method = IB_MAD_METHOD_GET; - - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - - /* - * C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) - */ - p_resp_sa_mad->sm_key = 0; - - /* - * o15-0.2.7 - The PathRecord Attribute ID shall be used in - * the response (to a SubnAdmGetMulti(MultiPathRecord) - */ - if( p_resp_sa_mad->attr_id == IB_MAD_ATTR_MULTIPATH_RECORD ) - p_resp_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; - - if( osm_log_is_active( p_resp->p_log, OSM_LOG_FRAMES ) ) - osm_dump_sa_mad( p_resp->p_log, p_resp_sa_mad, OSM_LOG_FRAMES ); - - status = osm_vendor_send( osm_madw_get_bind_handle( p_resp_madw ), - p_resp_madw, FALSE ); - - if( status != IB_SUCCESS ) - { - osm_log( p_resp->p_log, OSM_LOG_ERROR, - "osm_sa_send_error: ERR 2302: " - "Error sending MAD (%s)\n", ib_get_err_str( status ) ); - /* osm_mad_pool_put( p_resp->p_pool, p_resp_madw ); */ - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_resp->p_log ); + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_sa_mad_t *p_sa_mad; + ib_api_status_t status; + + OSM_LOG_ENTER(p_resp->p_log, osm_sa_send_error); + + /* avoid races - if we are exiting - exit */ + if (osm_exit_flag) { + osm_log(p_resp->p_log, OSM_LOG_DEBUG, + "osm_sa_send_error: " + "Ignoring requested send after exit\n"); + goto Exit; + } + + p_resp_madw = osm_mad_pool_get(p_resp->p_pool, + p_madw->h_bind, MAD_BLOCK_SIZE, + &p_madw->mad_addr); + + if (p_resp_madw == NULL) { + osm_log(p_resp->p_log, OSM_LOG_ERROR, + "osm_sa_send_error: ERR 2301: " + "Unable to acquire response MAD\n"); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + /* Copy the MAD header back into the response mad */ + *p_resp_sa_mad = *p_sa_mad; + p_resp_sa_mad->status = sa_status; + + if (p_resp_sa_mad->method == IB_MAD_METHOD_SET) + p_resp_sa_mad->method = IB_MAD_METHOD_GET; + + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + + /* + * C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) + */ + p_resp_sa_mad->sm_key = 0; + + /* + * o15-0.2.7 - The PathRecord Attribute ID shall be used in + * the response (to a SubnAdmGetMulti(MultiPathRecord) + */ + if (p_resp_sa_mad->attr_id == IB_MAD_ATTR_MULTIPATH_RECORD) + p_resp_sa_mad->attr_id = IB_MAD_ATTR_PATH_RECORD; + + if (osm_log_is_active(p_resp->p_log, OSM_LOG_FRAMES)) + osm_dump_sa_mad(p_resp->p_log, p_resp_sa_mad, OSM_LOG_FRAMES); + + status = osm_vendor_send(osm_madw_get_bind_handle(p_resp_madw), + p_resp_madw, FALSE); + + if (status != IB_SUCCESS) { + osm_log(p_resp->p_log, OSM_LOG_ERROR, + "osm_sa_send_error: ERR 2302: " + "Error sending MAD (%s)\n", ib_get_err_str(status)); + /* osm_mad_pool_put( p_resp->p_pool, p_resp_madw ); */ + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_resp->p_log); } diff --git a/opensm/opensm/osm_sa_service_record.c b/opensm/opensm/osm_sa_service_record.c index 016909b..89d454a 100644 --- a/opensm/opensm/osm_sa_service_record.c +++ b/opensm/opensm/osm_sa_service_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -69,1138 +69,1028 @@ #define OSM_SR_RCV_POOL_MIN_SIZE 64 #define OSM_SR_RCV_POOL_GROW_SIZE 64 -typedef struct _osm_sr_item -{ - cl_pool_item_t pool_item; - ib_service_record_t service_rec; +typedef struct _osm_sr_item { + cl_pool_item_t pool_item; + ib_service_record_t service_rec; } osm_sr_item_t; -typedef struct osm_sr_match_item { - cl_qlist_t sr_list; - ib_service_record_t* p_service_rec; - ib_net64_t comp_mask; - osm_sr_rcv_t* p_rcv; +typedef struct osm_sr_match_item { + cl_qlist_t sr_list; + ib_service_record_t *p_service_rec; + ib_net64_t comp_mask; + osm_sr_rcv_t *p_rcv; } osm_sr_match_item_t; -typedef struct _osm_sr_search_ctxt -{ - osm_sr_match_item_t * p_sr_item; - const osm_physp_t* p_req_physp; +typedef struct _osm_sr_search_ctxt { + osm_sr_match_item_t *p_sr_item; + const osm_physp_t *p_req_physp; } osm_sr_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_sr_rcv_construct( - IN osm_sr_rcv_t* const p_rcv ) +void osm_sr_rcv_construct(IN osm_sr_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->sr_pool ); - cl_timer_construct(&p_rcv->sr_timer ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->sr_pool); + cl_timer_construct(&p_rcv->sr_timer); } /********************************************************************** **********************************************************************/ -void -osm_sr_rcv_destroy( - IN osm_sr_rcv_t* const p_rcv ) +void osm_sr_rcv_destroy(IN osm_sr_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->sr_pool ); - cl_timer_trim(&p_rcv->sr_timer, 1); - cl_timer_destroy(&p_rcv->sr_timer ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->sr_pool); + cl_timer_trim(&p_rcv->sr_timer, 1); + cl_timer_destroy(&p_rcv->sr_timer); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sr_rcv_init( - IN osm_sr_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_sr_rcv_init(IN osm_sr_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_ERROR; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_log, osm_sr_rcv_init ); - - osm_sr_rcv_construct( p_rcv ); - - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; - - cl_status = cl_qlock_pool_init( &p_rcv->sr_pool, - OSM_SR_RCV_POOL_MIN_SIZE, - 0, - OSM_SR_RCV_POOL_GROW_SIZE, - sizeof(osm_sr_item_t), - NULL, NULL, NULL ); - if(cl_status != CL_SUCCESS) - goto Exit; - - status = cl_timer_init(&p_rcv->sr_timer, - osm_sr_rcv_lease_cb, - p_rcv ); - if(cl_status != CL_SUCCESS) - goto Exit; - - status = IB_SUCCESS; - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + ib_api_status_t status = IB_ERROR; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_log, osm_sr_rcv_init); + + osm_sr_rcv_construct(p_rcv); + + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; + + cl_status = cl_qlock_pool_init(&p_rcv->sr_pool, + OSM_SR_RCV_POOL_MIN_SIZE, + 0, + OSM_SR_RCV_POOL_GROW_SIZE, + sizeof(osm_sr_item_t), NULL, NULL, NULL); + if (cl_status != CL_SUCCESS) + goto Exit; + + status = cl_timer_init(&p_rcv->sr_timer, osm_sr_rcv_lease_cb, p_rcv); + if (cl_status != CL_SUCCESS) + goto Exit; + + status = IB_SUCCESS; + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ static boolean_t -__match_service_pkey_with_ports_pkey( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw, - ib_service_record_t * const p_service_rec, - ib_net64_t const comp_mask ) +__match_service_pkey_with_ports_pkey(IN osm_sr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw, + ib_service_record_t * const p_service_rec, + ib_net64_t const comp_mask) { - boolean_t valid = TRUE; - osm_physp_t * p_req_physp; - ib_net64_t service_guid; - osm_port_t * service_port; - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw)); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__match_service_pkey_with_ports_pkey: ERR 2404: " - "Cannot find requester physical port\n" ); - valid = FALSE; - goto Exit; - } - - if((comp_mask & IB_SR_COMPMASK_SPKEY) == IB_SR_COMPMASK_SPKEY) - { - /* We have a ServiceP_Key - check matching on requester port, and - ServiceGid port (if such exists) */ - /* Make sure it matches the p_req_physp */ - if (!osm_physp_has_pkey(p_rcv->p_log, p_service_rec->service_pkey, p_req_physp)) - { - valid = FALSE; - goto Exit; - } - - /* Make sure it matches the port of the ServiceGid */ - if((comp_mask & IB_SR_COMPMASK_SGID) == IB_SR_COMPMASK_SGID) - { - service_guid = p_service_rec->service_gid.unicast.interface_id; - service_port = osm_get_port_by_guid(p_rcv->p_subn, service_guid); - if (!service_port) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__match_service_pkey_with_ports_pkey: ERR 2405: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( service_guid ) ); - valid = FALSE; - goto Exit; - } - /* check on the table of the default physical port of the service port */ - if ( !osm_physp_has_pkey( p_rcv->p_log, - p_service_rec->service_pkey, - service_port->p_physp ) ) - { - valid = FALSE; - goto Exit; - } - } - } - - Exit: - return valid; + boolean_t valid = TRUE; + osm_physp_t *p_req_physp; + ib_net64_t service_guid; + osm_port_t *service_port; + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__match_service_pkey_with_ports_pkey: ERR 2404: " + "Cannot find requester physical port\n"); + valid = FALSE; + goto Exit; + } + + if ((comp_mask & IB_SR_COMPMASK_SPKEY) == IB_SR_COMPMASK_SPKEY) { + /* We have a ServiceP_Key - check matching on requester port, and + ServiceGid port (if such exists) */ + /* Make sure it matches the p_req_physp */ + if (!osm_physp_has_pkey + (p_rcv->p_log, p_service_rec->service_pkey, p_req_physp)) { + valid = FALSE; + goto Exit; + } + + /* Make sure it matches the port of the ServiceGid */ + if ((comp_mask & IB_SR_COMPMASK_SGID) == IB_SR_COMPMASK_SGID) { + service_guid = + p_service_rec->service_gid.unicast.interface_id; + service_port = + osm_get_port_by_guid(p_rcv->p_subn, service_guid); + if (!service_port) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__match_service_pkey_with_ports_pkey: ERR 2405: " + "No port object for port 0x%016" PRIx64 + "\n", cl_ntoh64(service_guid)); + valid = FALSE; + goto Exit; + } + /* check on the table of the default physical port of the service port */ + if (!osm_physp_has_pkey(p_rcv->p_log, + p_service_rec->service_pkey, + service_port->p_physp)) { + valid = FALSE; + goto Exit; + } + } + } + + Exit: + return valid; } /********************************************************************** **********************************************************************/ boolean_t -__match_name_to_key_association( - IN osm_sr_rcv_t* const p_rcv, - ib_service_record_t* p_service_rec, - ib_net64_t comp_mask ) +__match_name_to_key_association(IN osm_sr_rcv_t * const p_rcv, + ib_service_record_t * p_service_rec, + ib_net64_t comp_mask) { - UNUSED_PARAM( p_service_rec ); - UNUSED_PARAM( p_rcv ); - - if( (comp_mask & (IB_SR_COMPMASK_SKEY | IB_SR_COMPMASK_SNAME)) == - (IB_SR_COMPMASK_SKEY | IB_SR_COMPMASK_SNAME) ) - { - /* For now, we are not maintaining the ServiceAssociation record - * so just return TRUE - */ - return TRUE; - } - - return TRUE; + UNUSED_PARAM(p_service_rec); + UNUSED_PARAM(p_rcv); + + if ((comp_mask & (IB_SR_COMPMASK_SKEY | IB_SR_COMPMASK_SNAME)) == + (IB_SR_COMPMASK_SKEY | IB_SR_COMPMASK_SNAME)) { + /* For now, we are not maintaining the ServiceAssociation record + * so just return TRUE + */ + return TRUE; + } + + return TRUE; } /********************************************************************** **********************************************************************/ static boolean_t -__validate_sr( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +__validate_sr(IN osm_sr_rcv_t * const p_rcv, IN const osm_madw_t * const p_madw) { - boolean_t valid = TRUE; - ib_sa_mad_t * p_sa_mad; - ib_service_record_t* p_recvd_service_rec; - - OSM_LOG_ENTER( p_rcv->p_log, __validate_sr ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_service_rec = - (ib_service_record_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - valid = __match_service_pkey_with_ports_pkey( - p_rcv, - p_madw, - p_recvd_service_rec, - p_sa_mad->comp_mask ); - - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_sr: " - "No Match for Service Pkey\n" ); - valid = FALSE; - goto Exit; - } - - valid = __match_name_to_key_association( - p_rcv, - p_recvd_service_rec, - p_sa_mad->comp_mask ); - - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__validate_sr: " - "Service Record Name to key matching failed\n" ); - valid = FALSE; - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return valid; + boolean_t valid = TRUE; + ib_sa_mad_t *p_sa_mad; + ib_service_record_t *p_recvd_service_rec; + + OSM_LOG_ENTER(p_rcv->p_log, __validate_sr); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_service_rec = + (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + valid = __match_service_pkey_with_ports_pkey(p_rcv, + p_madw, + p_recvd_service_rec, + p_sa_mad->comp_mask); + + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_sr: " "No Match for Service Pkey\n"); + valid = FALSE; + goto Exit; + } + + valid = __match_name_to_key_association(p_rcv, + p_recvd_service_rec, + p_sa_mad->comp_mask); + + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__validate_sr: " + "Service Record Name to key matching failed\n"); + valid = FALSE; + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return valid; } /********************************************************************** **********************************************************************/ static void -__osm_sr_rcv_respond( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw, - IN cl_qlist_t* const p_list ) +__osm_sr_rcv_respond(IN osm_sr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw, + IN cl_qlist_t * const p_list) { - osm_madw_t* p_resp_madw; - const ib_sa_mad_t* p_sa_mad; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, num_copied; + osm_madw_t *p_resp_madw; + const ib_sa_mad_t *p_sa_mad; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, num_copied; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - ib_service_record_t* p_resp_sr; - ib_api_status_t status; - osm_sr_item_t* p_sr_item; - const ib_sa_mad_t* p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - boolean_t trusted_req = TRUE; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sr_rcv_respond ); - - num_rec = cl_qlist_count( p_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if ( (p_rcvd_mad->method == IB_MAD_METHOD_GET) && - (num_rec > 1)) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sr_rcv_respond: ERR 2406: " - "Got more than one record for SubnAdmGet (%u).\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - while( p_sr_item != (osm_sr_item_t*)cl_qlist_end( p_list ) ) - { - cl_qlock_pool_put( &p_rcv->sr_pool, &p_sr_item->pool_item ); - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - } - - goto Exit; - } - + ib_service_record_t *p_resp_sr; + ib_api_status_t status; + osm_sr_item_t *p_sr_item; + const ib_sa_mad_t *p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + boolean_t trusted_req = TRUE; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sr_rcv_respond); + + num_rec = cl_qlist_count(p_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sr_rcv_respond: ERR 2406: " + "Got more than one record for SubnAdmGet (%u).\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); + while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { + cl_qlock_pool_put(&p_rcv->sr_pool, + &p_sr_item->pool_item); + p_sr_item = + (osm_sr_item_t *) cl_qlist_remove_head(p_list); + } + + goto Exit; + } #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_service_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_sr_rcv_respond: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_service_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_sr_rcv_respond: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sr_rcv_respond: " - "Generating response with %u records\n", num_rec ); - } - - /* - Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_service_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - if( !p_resp_madw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sr_rcv_respond: ERR 2402: " - "Unable to allocate MAD\n" ); - /* Release the quick pool items */ - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - while( p_sr_item != (osm_sr_item_t*)cl_qlist_end( p_list ) ) - { - cl_qlock_pool_put( &p_rcv->sr_pool, &p_sr_item->pool_item ); - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - } - - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - memcpy( p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE ); - - /* but what if it was a SET ? setting the response bit is not enough */ - if (p_rcvd_mad->method == IB_MAD_METHOD_SET) - { - p_resp_sa_mad->method = IB_MAD_METHOD_GET; - } - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_service_record_t) ); + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sr_rcv_respond: " + "Generating response with %u records\n", num_rec); + } + + /* + Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_service_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sr_rcv_respond: ERR 2402: " + "Unable to allocate MAD\n"); + /* Release the quick pool items */ + p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); + while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { + cl_qlock_pool_put(&p_rcv->sr_pool, + &p_sr_item->pool_item); + p_sr_item = + (osm_sr_item_t *) cl_qlist_remove_head(p_list); + } + + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + memcpy(p_resp_sa_mad, p_sa_mad, IB_SA_MAD_HDR_SIZE); + + /* but what if it was a SET ? setting the response bit is not enough */ + if (p_rcvd_mad->method == IB_MAD_METHOD_SET) { + p_resp_sa_mad->method = IB_MAD_METHOD_GET; + } + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_service_record_t)); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - p_resp_sr = (ib_service_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); - - if( (p_resp_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) && - (num_rec == 0)) - { - p_resp_sa_mad->status = IB_SA_MAD_STATUS_NO_RECORDS; - memset( p_resp_sr, 0, sizeof(*p_resp_sr) ); - } - else - { - /* - p923 - The ServiceKey shall be set to 0, except in the case of a trusted - request. - Note: In the mad controller we check that the SM_Key received on - the mad is valid. Meaning - is either zero or equal to the local - sm_key. - */ - if (p_sa_mad->sm_key == 0) - trusted_req = FALSE; - - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - - /* we need to track the number of copied items so we can - * stop the copy - but clear them all - */ - num_copied = 0; - while( p_sr_item != (osm_sr_item_t*)cl_qlist_end( p_list ) ) - { - /* Copy the Link Records from the list into the MAD */ - if (num_copied < num_rec) - { - *p_resp_sr = p_sr_item->service_rec; - if (trusted_req == FALSE) - memset(p_resp_sr->service_key, 0, sizeof(p_resp_sr->service_key)); - - num_copied++; - } - cl_qlock_pool_put( &p_rcv->sr_pool, &p_sr_item->pool_item ); - p_resp_sr++; - p_sr_item = (osm_sr_item_t*)cl_qlist_remove_head( p_list ); - } - } - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - - if( status != IB_SUCCESS ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sr_rcv_respond: ERR 2407: " - "Unable to send MAD (%s)\n", ib_get_err_str( status ) ); - /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + p_resp_sr = + (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_resp_sa_mad); + + if ((p_resp_sa_mad->method != IB_MAD_METHOD_GETTABLE_RESP) && + (num_rec == 0)) { + p_resp_sa_mad->status = IB_SA_MAD_STATUS_NO_RECORDS; + memset(p_resp_sr, 0, sizeof(*p_resp_sr)); + } else { + /* + p923 - The ServiceKey shall be set to 0, except in the case of a trusted + request. + Note: In the mad controller we check that the SM_Key received on + the mad is valid. Meaning - is either zero or equal to the local + sm_key. + */ + if (p_sa_mad->sm_key == 0) + trusted_req = FALSE; + + p_sr_item = (osm_sr_item_t *) cl_qlist_remove_head(p_list); + + /* we need to track the number of copied items so we can + * stop the copy - but clear them all + */ + num_copied = 0; + while (p_sr_item != (osm_sr_item_t *) cl_qlist_end(p_list)) { + /* Copy the Link Records from the list into the MAD */ + if (num_copied < num_rec) { + *p_resp_sr = p_sr_item->service_rec; + if (trusted_req == FALSE) + memset(p_resp_sr->service_key, 0, + sizeof(p_resp_sr->service_key)); + + num_copied++; + } + cl_qlock_pool_put(&p_rcv->sr_pool, + &p_sr_item->pool_item); + p_resp_sr++; + p_sr_item = + (osm_sr_item_t *) cl_qlist_remove_head(p_list); + } + } + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sr_rcv_respond: ERR 2407: " + "Unable to send MAD (%s)\n", ib_get_err_str(status)); + /* osm_mad_pool_put( p_rcv->p_mad_pool, p_resp_madw ); */ + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__get_matching_sr( - IN cl_list_item_t* const p_list_item, - IN void* context ) +__get_matching_sr(IN cl_list_item_t * const p_list_item, IN void *context) { - osm_sr_search_ctxt_t* const p_ctxt = (osm_sr_search_ctxt_t*)context; - osm_svcr_t * p_svcr = (osm_svcr_t*)p_list_item; - osm_sr_item_t* p_sr_pool_item; - osm_sr_match_item_t* p_sr_item =p_ctxt->p_sr_item; - ib_net64_t comp_mask = p_sr_item->comp_mask; - const osm_physp_t* p_req_physp = p_ctxt->p_req_physp; - - if((comp_mask & IB_SR_COMPMASK_SID) == IB_SR_COMPMASK_SID) - { - if(p_sr_item->p_service_rec->service_id != - p_svcr->service_record.service_id) - return; - } - if((comp_mask & IB_SR_COMPMASK_SGID) == IB_SR_COMPMASK_SGID) - { - if( - memcmp(&p_sr_item->p_service_rec->service_gid, - &p_svcr->service_record.service_gid, - sizeof(p_svcr->service_record.service_gid)) != 0) - return; - } - if((comp_mask & IB_SR_COMPMASK_SPKEY) == IB_SR_COMPMASK_SPKEY ) - { - if(p_sr_item->p_service_rec->service_pkey != - p_svcr->service_record.service_pkey) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SKEY) == IB_SR_COMPMASK_SKEY) - { - if(memcmp(p_sr_item->p_service_rec->service_key , - p_svcr->service_record.service_key, - 16*sizeof(uint8_t))) - return; - } - if((comp_mask & IB_SR_COMPMASK_SNAME) == IB_SR_COMPMASK_SNAME) - { - if( - memcmp(p_sr_item->p_service_rec->service_name, - p_svcr->service_record.service_name, - sizeof(p_svcr->service_record.service_name)) != 0 - ) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_0) == IB_SR_COMPMASK_SDATA8_0) - { - if(p_sr_item->p_service_rec->service_data8[0] != - p_svcr->service_record.service_data8[0]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_1) == IB_SR_COMPMASK_SDATA8_1) - { - if(p_sr_item->p_service_rec->service_data8[1] != - p_svcr->service_record.service_data8[1]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_2) == IB_SR_COMPMASK_SDATA8_2) - { - if(p_sr_item->p_service_rec->service_data8[2] != - p_svcr->service_record.service_data8[2]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_3) == IB_SR_COMPMASK_SDATA8_3) - { - if(p_sr_item->p_service_rec->service_data8[3] != - p_svcr->service_record.service_data8[3]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_4)== IB_SR_COMPMASK_SDATA8_4) - { - if(p_sr_item->p_service_rec->service_data8[4] != - p_svcr->service_record.service_data8[4]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_5)== IB_SR_COMPMASK_SDATA8_5) - { - if(p_sr_item->p_service_rec->service_data8[5] != - p_svcr->service_record.service_data8[5]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_6)== IB_SR_COMPMASK_SDATA8_6) - { - if(p_sr_item->p_service_rec->service_data8[6]!= - p_svcr->service_record.service_data8[6]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_7)== IB_SR_COMPMASK_SDATA8_7) - { - if(p_sr_item->p_service_rec->service_data8[7]!= - p_svcr->service_record.service_data8[7]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_8)== IB_SR_COMPMASK_SDATA8_8) - { - if(p_sr_item->p_service_rec->service_data8[8]!= - p_svcr->service_record.service_data8[8]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_9)== IB_SR_COMPMASK_SDATA8_9) - { - if(p_sr_item->p_service_rec->service_data8[9]!= - p_svcr->service_record.service_data8[9]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_10)== IB_SR_COMPMASK_SDATA8_10) - { - if(p_sr_item->p_service_rec->service_data8[10]!= - p_svcr->service_record.service_data8[10]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_11)== IB_SR_COMPMASK_SDATA8_11) - { - if(p_sr_item->p_service_rec->service_data8[11]!= - p_svcr->service_record.service_data8[11]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA8_12)== IB_SR_COMPMASK_SDATA8_12) - { - if(p_sr_item->p_service_rec->service_data8[12]!= - p_svcr->service_record.service_data8[12]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_13)== IB_SR_COMPMASK_SDATA8_13) - { - if(p_sr_item->p_service_rec->service_data8[13]!= - p_svcr->service_record.service_data8[13]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_14)== IB_SR_COMPMASK_SDATA8_14) - { - if(p_sr_item->p_service_rec->service_data8[14]!= - p_svcr->service_record.service_data8[14]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA8_15)== IB_SR_COMPMASK_SDATA8_15) - { - if(p_sr_item->p_service_rec->service_data8[15]!= - p_svcr->service_record.service_data8[15]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_0)== IB_SR_COMPMASK_SDATA16_0) - { - if(p_sr_item->p_service_rec->service_data16[0]!= - p_svcr->service_record.service_data16[0]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_1)== IB_SR_COMPMASK_SDATA16_1) - { - if(p_sr_item->p_service_rec->service_data16[1]!= - p_svcr->service_record.service_data16[1]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_2)== IB_SR_COMPMASK_SDATA16_2) - { - if(p_sr_item->p_service_rec->service_data16[2]!= - p_svcr->service_record.service_data16[2]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_3)== IB_SR_COMPMASK_SDATA16_3) - { - if(p_sr_item->p_service_rec->service_data16[3]!= - p_svcr->service_record.service_data16[3]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_4)== IB_SR_COMPMASK_SDATA16_4) - { - if(p_sr_item->p_service_rec->service_data16[4]!= - p_svcr->service_record.service_data16[4]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_5)== IB_SR_COMPMASK_SDATA16_5) - { - if(p_sr_item->p_service_rec->service_data16[5]!= - p_svcr->service_record.service_data16[5]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_6)== IB_SR_COMPMASK_SDATA16_6) - { - if(p_sr_item->p_service_rec->service_data16[6]!= - p_svcr->service_record.service_data16[6]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA16_7)== IB_SR_COMPMASK_SDATA16_7) - { - if(p_sr_item->p_service_rec->service_data16[7]!= - p_svcr->service_record.service_data16[7]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA32_0)== IB_SR_COMPMASK_SDATA32_0) - { - if(p_sr_item->p_service_rec->service_data32[0]!= - p_svcr->service_record.service_data32[0]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA32_1)== IB_SR_COMPMASK_SDATA32_1) - { - if(p_sr_item->p_service_rec->service_data32[1]!= - p_svcr->service_record.service_data32[1]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA32_2)== IB_SR_COMPMASK_SDATA32_2) - { - if(p_sr_item->p_service_rec->service_data32[2]!= - p_svcr->service_record.service_data32[2]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA32_3)== IB_SR_COMPMASK_SDATA32_3) - { - if(p_sr_item->p_service_rec->service_data32[3]!= - p_svcr->service_record.service_data32[3]) - return; - } - - if((comp_mask & IB_SR_COMPMASK_SDATA64_0)== IB_SR_COMPMASK_SDATA64_0) - { - if(p_sr_item->p_service_rec->service_data64[0]!= - p_svcr->service_record.service_data64[0]) - return; - } - if((comp_mask & IB_SR_COMPMASK_SDATA64_1)== IB_SR_COMPMASK_SDATA64_1) - { - if(p_sr_item->p_service_rec->service_data64[1]!= - p_svcr->service_record.service_data64[1]) - return; - } - - /* Check that the requester port has the pkey which is the service_pkey. - If not - then it cannot receive this ServiceRecord. */ - /* The check is relevant only if the service_pkey is valid */ - if (!ib_pkey_is_invalid(p_svcr->service_record.service_pkey)) - { - if (!osm_physp_has_pkey( p_sr_item->p_rcv->p_log, - p_svcr->service_record.service_pkey, - p_req_physp ) ) - { - osm_log( p_sr_item->p_rcv->p_log, OSM_LOG_VERBOSE, - "__get_matching_sr: " - "requester port doesn't have the service_pkey: 0x%X\n", - cl_ntoh16(p_svcr->service_record.service_pkey) ); - return; - } - } - - p_sr_pool_item = (osm_sr_item_t*)cl_qlock_pool_get( &p_sr_item->p_rcv->sr_pool ); - - if( p_sr_pool_item == NULL ) - { - osm_log( p_sr_item->p_rcv->p_log, OSM_LOG_ERROR, - "__get_matching_sr: ERR 2408: " - "Unable to acquire Service Record from pool\n" ); - goto Exit; - } - - p_sr_pool_item->service_rec = p_svcr->service_record; - - cl_qlist_insert_tail( &p_sr_item->sr_list, - (cl_list_item_t*)&p_sr_pool_item->pool_item ); - - Exit: - return; + osm_sr_search_ctxt_t *const p_ctxt = (osm_sr_search_ctxt_t *) context; + osm_svcr_t *p_svcr = (osm_svcr_t *) p_list_item; + osm_sr_item_t *p_sr_pool_item; + osm_sr_match_item_t *p_sr_item = p_ctxt->p_sr_item; + ib_net64_t comp_mask = p_sr_item->comp_mask; + const osm_physp_t *p_req_physp = p_ctxt->p_req_physp; + + if ((comp_mask & IB_SR_COMPMASK_SID) == IB_SR_COMPMASK_SID) { + if (p_sr_item->p_service_rec->service_id != + p_svcr->service_record.service_id) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SGID) == IB_SR_COMPMASK_SGID) { + if (memcmp(&p_sr_item->p_service_rec->service_gid, + &p_svcr->service_record.service_gid, + sizeof(p_svcr->service_record.service_gid)) != 0) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SPKEY) == IB_SR_COMPMASK_SPKEY) { + if (p_sr_item->p_service_rec->service_pkey != + p_svcr->service_record.service_pkey) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SKEY) == IB_SR_COMPMASK_SKEY) { + if (memcmp(p_sr_item->p_service_rec->service_key, + p_svcr->service_record.service_key, + 16 * sizeof(uint8_t))) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SNAME) == IB_SR_COMPMASK_SNAME) { + if (memcmp(p_sr_item->p_service_rec->service_name, + p_svcr->service_record.service_name, + sizeof(p_svcr->service_record.service_name)) != 0) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_0) == IB_SR_COMPMASK_SDATA8_0) { + if (p_sr_item->p_service_rec->service_data8[0] != + p_svcr->service_record.service_data8[0]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_1) == IB_SR_COMPMASK_SDATA8_1) { + if (p_sr_item->p_service_rec->service_data8[1] != + p_svcr->service_record.service_data8[1]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_2) == IB_SR_COMPMASK_SDATA8_2) { + if (p_sr_item->p_service_rec->service_data8[2] != + p_svcr->service_record.service_data8[2]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_3) == IB_SR_COMPMASK_SDATA8_3) { + if (p_sr_item->p_service_rec->service_data8[3] != + p_svcr->service_record.service_data8[3]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_4) == IB_SR_COMPMASK_SDATA8_4) { + if (p_sr_item->p_service_rec->service_data8[4] != + p_svcr->service_record.service_data8[4]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_5) == IB_SR_COMPMASK_SDATA8_5) { + if (p_sr_item->p_service_rec->service_data8[5] != + p_svcr->service_record.service_data8[5]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_6) == IB_SR_COMPMASK_SDATA8_6) { + if (p_sr_item->p_service_rec->service_data8[6] != + p_svcr->service_record.service_data8[6]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_7) == IB_SR_COMPMASK_SDATA8_7) { + if (p_sr_item->p_service_rec->service_data8[7] != + p_svcr->service_record.service_data8[7]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_8) == IB_SR_COMPMASK_SDATA8_8) { + if (p_sr_item->p_service_rec->service_data8[8] != + p_svcr->service_record.service_data8[8]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_9) == IB_SR_COMPMASK_SDATA8_9) { + if (p_sr_item->p_service_rec->service_data8[9] != + p_svcr->service_record.service_data8[9]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_10) == IB_SR_COMPMASK_SDATA8_10) { + if (p_sr_item->p_service_rec->service_data8[10] != + p_svcr->service_record.service_data8[10]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_11) == IB_SR_COMPMASK_SDATA8_11) { + if (p_sr_item->p_service_rec->service_data8[11] != + p_svcr->service_record.service_data8[11]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA8_12) == IB_SR_COMPMASK_SDATA8_12) { + if (p_sr_item->p_service_rec->service_data8[12] != + p_svcr->service_record.service_data8[12]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_13) == IB_SR_COMPMASK_SDATA8_13) { + if (p_sr_item->p_service_rec->service_data8[13] != + p_svcr->service_record.service_data8[13]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_14) == IB_SR_COMPMASK_SDATA8_14) { + if (p_sr_item->p_service_rec->service_data8[14] != + p_svcr->service_record.service_data8[14]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA8_15) == IB_SR_COMPMASK_SDATA8_15) { + if (p_sr_item->p_service_rec->service_data8[15] != + p_svcr->service_record.service_data8[15]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_0) == IB_SR_COMPMASK_SDATA16_0) { + if (p_sr_item->p_service_rec->service_data16[0] != + p_svcr->service_record.service_data16[0]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_1) == IB_SR_COMPMASK_SDATA16_1) { + if (p_sr_item->p_service_rec->service_data16[1] != + p_svcr->service_record.service_data16[1]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_2) == IB_SR_COMPMASK_SDATA16_2) { + if (p_sr_item->p_service_rec->service_data16[2] != + p_svcr->service_record.service_data16[2]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_3) == IB_SR_COMPMASK_SDATA16_3) { + if (p_sr_item->p_service_rec->service_data16[3] != + p_svcr->service_record.service_data16[3]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_4) == IB_SR_COMPMASK_SDATA16_4) { + if (p_sr_item->p_service_rec->service_data16[4] != + p_svcr->service_record.service_data16[4]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_5) == IB_SR_COMPMASK_SDATA16_5) { + if (p_sr_item->p_service_rec->service_data16[5] != + p_svcr->service_record.service_data16[5]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_6) == IB_SR_COMPMASK_SDATA16_6) { + if (p_sr_item->p_service_rec->service_data16[6] != + p_svcr->service_record.service_data16[6]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA16_7) == IB_SR_COMPMASK_SDATA16_7) { + if (p_sr_item->p_service_rec->service_data16[7] != + p_svcr->service_record.service_data16[7]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA32_0) == IB_SR_COMPMASK_SDATA32_0) { + if (p_sr_item->p_service_rec->service_data32[0] != + p_svcr->service_record.service_data32[0]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA32_1) == IB_SR_COMPMASK_SDATA32_1) { + if (p_sr_item->p_service_rec->service_data32[1] != + p_svcr->service_record.service_data32[1]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA32_2) == IB_SR_COMPMASK_SDATA32_2) { + if (p_sr_item->p_service_rec->service_data32[2] != + p_svcr->service_record.service_data32[2]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA32_3) == IB_SR_COMPMASK_SDATA32_3) { + if (p_sr_item->p_service_rec->service_data32[3] != + p_svcr->service_record.service_data32[3]) + return; + } + + if ((comp_mask & IB_SR_COMPMASK_SDATA64_0) == IB_SR_COMPMASK_SDATA64_0) { + if (p_sr_item->p_service_rec->service_data64[0] != + p_svcr->service_record.service_data64[0]) + return; + } + if ((comp_mask & IB_SR_COMPMASK_SDATA64_1) == IB_SR_COMPMASK_SDATA64_1) { + if (p_sr_item->p_service_rec->service_data64[1] != + p_svcr->service_record.service_data64[1]) + return; + } + + /* Check that the requester port has the pkey which is the service_pkey. + If not - then it cannot receive this ServiceRecord. */ + /* The check is relevant only if the service_pkey is valid */ + if (!ib_pkey_is_invalid(p_svcr->service_record.service_pkey)) { + if (!osm_physp_has_pkey(p_sr_item->p_rcv->p_log, + p_svcr->service_record.service_pkey, + p_req_physp)) { + osm_log(p_sr_item->p_rcv->p_log, OSM_LOG_VERBOSE, + "__get_matching_sr: " + "requester port doesn't have the service_pkey: 0x%X\n", + cl_ntoh16(p_svcr->service_record.service_pkey)); + return; + } + } + + p_sr_pool_item = + (osm_sr_item_t *) cl_qlock_pool_get(&p_sr_item->p_rcv->sr_pool); + + if (p_sr_pool_item == NULL) { + osm_log(p_sr_item->p_rcv->p_log, OSM_LOG_ERROR, + "__get_matching_sr: ERR 2408: " + "Unable to acquire Service Record from pool\n"); + goto Exit; + } + + p_sr_pool_item->service_rec = p_svcr->service_record; + + cl_qlist_insert_tail(&p_sr_item->sr_list, + (cl_list_item_t *) & p_sr_pool_item->pool_item); + + Exit: + return; } /********************************************************************** **********************************************************************/ static void -osm_sr_rcv_process_get_method( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +osm_sr_rcv_process_get_method(IN osm_sr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - ib_sa_mad_t * p_sa_mad; - ib_service_record_t* p_recvd_service_rec; - osm_sr_match_item_t sr_match_item; - osm_sr_search_ctxt_t context; - osm_physp_t* p_req_physp; - - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_process_get_method ); - - CL_ASSERT( p_madw ); - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sr_rcv_process_get_method: ERR 2409: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_service_rec = - (ib_service_record_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_dump_service_record( p_rcv->p_log, - p_recvd_service_rec, - OSM_LOG_DEBUG ); - } - - cl_qlist_init(&sr_match_item.sr_list); - sr_match_item.p_service_rec = p_recvd_service_rec; - sr_match_item.comp_mask = p_sa_mad->comp_mask; - sr_match_item.p_rcv = p_rcv; - - context.p_sr_item = &sr_match_item; - context.p_req_physp = p_req_physp; - - /* Grab the lock */ - cl_plock_excl_acquire(p_rcv->p_lock); - - cl_qlist_apply_func(&p_rcv->p_subn->sa_sr_list, - __get_matching_sr, - &context); - - cl_plock_release(p_rcv->p_lock); - - if ((p_sa_mad->method == IB_MAD_METHOD_GET) && - (cl_qlist_count( &sr_match_item.sr_list ) == 0)) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_process_get_method: " - "No records matched the Service Record query\n"); - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - __osm_sr_rcv_respond( p_rcv, p_madw, &sr_match_item.sr_list ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + ib_sa_mad_t *p_sa_mad; + ib_service_record_t *p_recvd_service_rec; + osm_sr_match_item_t sr_match_item; + osm_sr_search_ctxt_t context; + osm_physp_t *p_req_physp; + + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_process_get_method); + + CL_ASSERT(p_madw); + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sr_rcv_process_get_method: ERR 2409: " + "Cannot find requester physical port\n"); + goto Exit; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_service_rec = + (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_dump_service_record(p_rcv->p_log, + p_recvd_service_rec, OSM_LOG_DEBUG); + } + + cl_qlist_init(&sr_match_item.sr_list); + sr_match_item.p_service_rec = p_recvd_service_rec; + sr_match_item.comp_mask = p_sa_mad->comp_mask; + sr_match_item.p_rcv = p_rcv; + + context.p_sr_item = &sr_match_item; + context.p_req_physp = p_req_physp; + + /* Grab the lock */ + cl_plock_excl_acquire(p_rcv->p_lock); + + cl_qlist_apply_func(&p_rcv->p_subn->sa_sr_list, + __get_matching_sr, &context); + + cl_plock_release(p_rcv->p_lock); + + if ((p_sa_mad->method == IB_MAD_METHOD_GET) && + (cl_qlist_count(&sr_match_item.sr_list) == 0)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_process_get_method: " + "No records matched the Service Record query\n"); + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + __osm_sr_rcv_respond(p_rcv, p_madw, &sr_match_item.sr_list); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** **********************************************************************/ static void -osm_sr_rcv_process_set_method( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +osm_sr_rcv_process_set_method(IN osm_sr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - ib_sa_mad_t * p_sa_mad; - ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - ib_service_record_t* p_recvd_service_rec; - ib_net64_t comp_mask; - osm_svcr_t* p_svcr; - osm_sr_item_t* p_sr_item; - cl_qlist_t sr_list; - - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_process_set_method ); - - CL_ASSERT( p_madw ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_service_rec = - (ib_service_record_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - comp_mask = p_sa_mad->comp_mask; - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_dump_service_record( p_rcv->p_log, - p_recvd_service_rec, - OSM_LOG_DEBUG ); - } - - if( (comp_mask & ( IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID )) != - (IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID )) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_sr_rcv_process_set_method: " - "Component Mask RID check failed for METHOD_SET\n"); - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - - /* if we were not provided with a service lease make it - infinite */ - if( (comp_mask & IB_SR_COMPMASK_SLEASE) != IB_SR_COMPMASK_SLEASE) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_process_set_method: " - "ServiceLease Component Mask not set - using infinite lease\n"); - p_recvd_service_rec->service_lease = 0xFFFFFFFF; - } - - /* Grab the lock */ - cl_plock_excl_acquire(p_rcv->p_lock); - - /* If Record exists with matching RID */ - p_svcr = osm_svcr_get_by_rid( - p_rcv->p_subn, - p_rcv->p_log, - p_recvd_service_rec ); - - if(p_svcr == NULL) - { - /* Create the instance of the osm_svcr_t object */ - p_svcr = osm_svcr_new(p_recvd_service_rec); - if(p_svcr == NULL) - { - cl_plock_release(p_rcv->p_lock); - - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sr_rcv_process_set_method: ERR 2411: " - "osm_svcr_get_by_rid failed\n" ); - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - /* Add this new osm_svcr_t object to subnet object */ - osm_svcr_insert_to_db( - p_rcv->p_subn, - p_rcv->p_log, - p_svcr ); - - } - else - { - /* Update the old instance of the osm_svcr_t object */ - osm_svcr_init(p_svcr, p_recvd_service_rec); - } - - cl_plock_release(p_rcv->p_lock); - - if( p_recvd_service_rec->service_lease != 0xFFFFFFFF ) - { + ib_sa_mad_t *p_sa_mad; + ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + ib_service_record_t *p_recvd_service_rec; + ib_net64_t comp_mask; + osm_svcr_t *p_svcr; + osm_sr_item_t *p_sr_item; + cl_qlist_t sr_list; + + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_process_set_method); + + CL_ASSERT(p_madw); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_service_rec = + (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + comp_mask = p_sa_mad->comp_mask; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_dump_service_record(p_rcv->p_log, + p_recvd_service_rec, OSM_LOG_DEBUG); + } + + if ((comp_mask & (IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID)) != + (IB_SR_COMPMASK_SID | IB_SR_COMPMASK_SGID)) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_sr_rcv_process_set_method: " + "Component Mask RID check failed for METHOD_SET\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + + /* if we were not provided with a service lease make it + infinite */ + if ((comp_mask & IB_SR_COMPMASK_SLEASE) != IB_SR_COMPMASK_SLEASE) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_process_set_method: " + "ServiceLease Component Mask not set - using infinite lease\n"); + p_recvd_service_rec->service_lease = 0xFFFFFFFF; + } + + /* Grab the lock */ + cl_plock_excl_acquire(p_rcv->p_lock); + + /* If Record exists with matching RID */ + p_svcr = osm_svcr_get_by_rid(p_rcv->p_subn, + p_rcv->p_log, p_recvd_service_rec); + + if (p_svcr == NULL) { + /* Create the instance of the osm_svcr_t object */ + p_svcr = osm_svcr_new(p_recvd_service_rec); + if (p_svcr == NULL) { + cl_plock_release(p_rcv->p_lock); + + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sr_rcv_process_set_method: ERR 2411: " + "osm_svcr_get_by_rid failed\n"); + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + /* Add this new osm_svcr_t object to subnet object */ + osm_svcr_insert_to_db(p_rcv->p_subn, p_rcv->p_log, p_svcr); + + } else { + /* Update the old instance of the osm_svcr_t object */ + osm_svcr_init(p_svcr, p_recvd_service_rec); + } + + cl_plock_release(p_rcv->p_lock); + + if (p_recvd_service_rec->service_lease != 0xFFFFFFFF) { #if 0 - cl_timer_trim(&p_rcv->sr_timer, - p_recvd_service_rec->service_lease * 1000); + cl_timer_trim(&p_rcv->sr_timer, + p_recvd_service_rec->service_lease * 1000); #endif - /* This was a bug since no check was made to see if too long */ - /* just make sure the timer works - get a call back within a second */ - cl_timer_trim(&p_rcv->sr_timer, 1000); - p_svcr->modified_time = cl_get_time_stamp_sec(); - } - - p_sr_item = (osm_sr_item_t*)cl_qlock_pool_get( &p_rcv->sr_pool ); - if( p_sr_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sr_rcv_process_set_method: ERR 2412: " - "Unable to acquire Service record\n" ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - if( (comp_mask & IB_SR_COMPMASK_SPKEY) != IB_SR_COMPMASK_SPKEY) - { - /* Set the Default Service P_Key in the response */ - p_recvd_service_rec->service_pkey = IB_DEFAULT_PKEY; - } - - p_sr_item->service_rec = *p_recvd_service_rec; - cl_qlist_init(&sr_list); - - cl_qlist_insert_tail( &sr_list, (cl_list_item_t*)&p_sr_item->pool_item ); - - __osm_sr_rcv_respond( p_rcv, p_madw, &sr_list ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + /* This was a bug since no check was made to see if too long */ + /* just make sure the timer works - get a call back within a second */ + cl_timer_trim(&p_rcv->sr_timer, 1000); + p_svcr->modified_time = cl_get_time_stamp_sec(); + } + + p_sr_item = (osm_sr_item_t *) cl_qlock_pool_get(&p_rcv->sr_pool); + if (p_sr_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sr_rcv_process_set_method: ERR 2412: " + "Unable to acquire Service record\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + if ((comp_mask & IB_SR_COMPMASK_SPKEY) != IB_SR_COMPMASK_SPKEY) { + /* Set the Default Service P_Key in the response */ + p_recvd_service_rec->service_pkey = IB_DEFAULT_PKEY; + } + + p_sr_item->service_rec = *p_recvd_service_rec; + cl_qlist_init(&sr_list); + + cl_qlist_insert_tail(&sr_list, + (cl_list_item_t *) & p_sr_item->pool_item); + + __osm_sr_rcv_respond(p_rcv, p_madw, &sr_list); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** **********************************************************************/ static void -osm_sr_rcv_process_delete_method( - IN osm_sr_rcv_t* const p_rcv, - IN const osm_madw_t* const p_madw ) +osm_sr_rcv_process_delete_method(IN osm_sr_rcv_t * const p_rcv, + IN const osm_madw_t * const p_madw) { - ib_sa_mad_t * p_sa_mad; - ib_service_record_t* p_recvd_service_rec; - ib_net64_t comp_mask; - osm_svcr_t* p_svcr; - osm_sr_item_t* p_sr_item; - cl_qlist_t sr_list; - - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_process_delete_method ); - - CL_ASSERT( p_madw ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_recvd_service_rec = - (ib_service_record_t*)ib_sa_mad_get_payload_ptr( p_sa_mad ); - - comp_mask = p_sa_mad->comp_mask; - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_dump_service_record( p_rcv->p_log, - p_recvd_service_rec, - OSM_LOG_DEBUG ); - } - - /* Grab the lock */ - cl_plock_excl_acquire(p_rcv->p_lock); - - /* If Record exists with matching RID */ - p_svcr = osm_svcr_get_by_rid( - p_rcv->p_subn, - p_rcv->p_log, - p_recvd_service_rec ); - - if(p_svcr == NULL) - { - cl_plock_release(p_rcv->p_lock); - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_process_delete_method: " - "No records matched the RID\n"); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - else - { - osm_svcr_remove_from_db(p_rcv->p_subn, - p_rcv->p_log, - p_svcr ); - } - - cl_plock_release(p_rcv->p_lock); - - p_sr_item = (osm_sr_item_t*)cl_qlock_pool_get( &p_rcv->sr_pool ); - if( p_sr_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sr_rcv_process_delete_method: ERR 2413: " - "Unable to acquire Service record\n"); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - /* provide back the copy of the record */ - p_sr_item->service_rec = p_svcr->service_record; - cl_qlist_init(&sr_list); - - cl_qlist_insert_tail( &sr_list, (cl_list_item_t*)&p_sr_item->pool_item ); - - if(p_svcr) - osm_svcr_delete(p_svcr); - - __osm_sr_rcv_respond( p_rcv, p_madw, &sr_list ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + ib_sa_mad_t *p_sa_mad; + ib_service_record_t *p_recvd_service_rec; + ib_net64_t comp_mask; + osm_svcr_t *p_svcr; + osm_sr_item_t *p_sr_item; + cl_qlist_t sr_list; + + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_process_delete_method); + + CL_ASSERT(p_madw); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_recvd_service_rec = + (ib_service_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + comp_mask = p_sa_mad->comp_mask; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_dump_service_record(p_rcv->p_log, + p_recvd_service_rec, OSM_LOG_DEBUG); + } + + /* Grab the lock */ + cl_plock_excl_acquire(p_rcv->p_lock); + + /* If Record exists with matching RID */ + p_svcr = osm_svcr_get_by_rid(p_rcv->p_subn, + p_rcv->p_log, p_recvd_service_rec); + + if (p_svcr == NULL) { + cl_plock_release(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_process_delete_method: " + "No records matched the RID\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } else { + osm_svcr_remove_from_db(p_rcv->p_subn, p_rcv->p_log, p_svcr); + } + + cl_plock_release(p_rcv->p_lock); + + p_sr_item = (osm_sr_item_t *) cl_qlock_pool_get(&p_rcv->sr_pool); + if (p_sr_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sr_rcv_process_delete_method: ERR 2413: " + "Unable to acquire Service record\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + /* provide back the copy of the record */ + p_sr_item->service_rec = p_svcr->service_record; + cl_qlist_init(&sr_list); + + cl_qlist_insert_tail(&sr_list, + (cl_list_item_t *) & p_sr_item->pool_item); + + if (p_svcr) + osm_svcr_delete(p_svcr); + + __osm_sr_rcv_respond(p_rcv, p_madw, &sr_list); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** **********************************************************************/ -void -osm_sr_rcv_process( - IN void *context, - IN void *data ) +void osm_sr_rcv_process(IN void *context, IN void *data) { - osm_sr_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_sa_mad_t * p_sa_mad; - ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; - boolean_t valid; - - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_process ); - - CL_ASSERT( p_madw ); - - p_sa_mad = osm_madw_get_sa_mad_ptr( p_madw ); - - CL_ASSERT( p_sa_mad->attr_id == IB_MAD_ATTR_SERVICE_RECORD ); - - switch (p_sa_mad->method) - { - case IB_MAD_METHOD_SET: - valid = __validate_sr(p_rcv, p_madw); - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_sr_rcv_process: " - "Component Mask check failed for set request\n" ); - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - osm_sr_rcv_process_set_method(p_rcv, p_madw); - break; - case IB_MAD_METHOD_DELETE: - valid = __validate_sr(p_rcv, p_madw); - if(!valid) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_process: " - "Component Mask check failed for delete request\n" ); - osm_sa_send_error( p_rcv->p_resp, p_madw, sa_status ); - goto Exit; - } - osm_sr_rcv_process_delete_method(p_rcv, p_madw); - break; - case IB_MAD_METHOD_GET: - case IB_MAD_METHOD_GETTABLE: - osm_sr_rcv_process_get_method(p_rcv, p_madw); - break; - default: - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_process: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_sa_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - break; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return; + osm_sr_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_sa_mad_t *p_sa_mad; + ib_net16_t sa_status = IB_SA_MAD_STATUS_REQ_INVALID; + boolean_t valid; + + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_process); + + CL_ASSERT(p_madw); + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + + CL_ASSERT(p_sa_mad->attr_id == IB_MAD_ATTR_SERVICE_RECORD); + + switch (p_sa_mad->method) { + case IB_MAD_METHOD_SET: + valid = __validate_sr(p_rcv, p_madw); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_sr_rcv_process: " + "Component Mask check failed for set request\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + osm_sr_rcv_process_set_method(p_rcv, p_madw); + break; + case IB_MAD_METHOD_DELETE: + valid = __validate_sr(p_rcv, p_madw); + if (!valid) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_process: " + "Component Mask check failed for delete request\n"); + osm_sa_send_error(p_rcv->p_resp, p_madw, sa_status); + goto Exit; + } + osm_sr_rcv_process_delete_method(p_rcv, p_madw); + break; + case IB_MAD_METHOD_GET: + case IB_MAD_METHOD_GETTABLE: + osm_sr_rcv_process_get_method(p_rcv, p_madw); + break; + default: + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_process: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_sa_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + break; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return; } /********************************************************************** **********************************************************************/ -void -osm_sr_rcv_lease_cb( - IN void* context ) +void osm_sr_rcv_lease_cb(IN void *context) { - osm_sr_rcv_t* p_rcv = (osm_sr_rcv_t*)context; - cl_list_item_t* p_list_item; - cl_list_item_t* p_next_list_item; - osm_svcr_t* p_svcr; - uint32_t curr_time; - uint32_t elapsed_time; - uint32_t trim_time = 20; /* maxiaml timer refresh is 20 seconds */ - - OSM_LOG_ENTER( p_rcv->p_log, osm_sr_rcv_lease_cb ); - - cl_plock_excl_acquire(p_rcv->p_lock); - - p_list_item = cl_qlist_head(&p_rcv->p_subn->sa_sr_list); - - while( p_list_item != cl_qlist_end(&p_rcv->p_subn->sa_sr_list) ) - { - p_svcr = (osm_svcr_t*)p_list_item; - - if(p_svcr->service_record.service_lease == 0xFFFFFFFF) - { - p_list_item = cl_qlist_next(p_list_item); - continue; - } - - /* current time in seconds */ - curr_time = cl_get_time_stamp_sec(); - /* elapsed time from last modify */ - elapsed_time = curr_time - p_svcr->modified_time; - /* but it can not be less then 1 */ - if (elapsed_time < 1) elapsed_time = 1; - - if(elapsed_time < p_svcr->lease_period) - { - /* - Just update the service lease period - note: for simplicity we work with a uint32_t field - external to the network order lease_period of the MAD - */ - p_svcr->lease_period -= elapsed_time; - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sr_rcv_lease_cb: " - "Remaining time for Service Name:%s is:0x%X\n", - p_svcr->service_record.service_name, - p_svcr->lease_period ); - - p_svcr->modified_time = curr_time; - - /* Update the trim timer */ - if(trim_time > p_svcr->lease_period) - { - trim_time = p_svcr->lease_period; - if (trim_time < 1) trim_time = 1; - } - - p_list_item = cl_qlist_next(p_list_item); - continue; - - } - else - { - p_next_list_item = cl_qlist_next(p_list_item); - - /* Remove the service Record */ - osm_svcr_remove_from_db(p_rcv->p_subn, - p_rcv->p_log, - p_svcr); - - osm_svcr_delete(p_svcr); - - p_list_item = p_next_list_item; - continue; - } - } - - /* Release the Lock */ - cl_plock_release(p_rcv->p_lock); - - if(trim_time != 0xFFFFFFFF) - { - cl_timer_trim(&p_rcv->sr_timer, - trim_time * 1000); /* Convert to milli seconds */ - } - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_sr_rcv_t *p_rcv = (osm_sr_rcv_t *) context; + cl_list_item_t *p_list_item; + cl_list_item_t *p_next_list_item; + osm_svcr_t *p_svcr; + uint32_t curr_time; + uint32_t elapsed_time; + uint32_t trim_time = 20; /* maxiaml timer refresh is 20 seconds */ + + OSM_LOG_ENTER(p_rcv->p_log, osm_sr_rcv_lease_cb); + + cl_plock_excl_acquire(p_rcv->p_lock); + + p_list_item = cl_qlist_head(&p_rcv->p_subn->sa_sr_list); + + while (p_list_item != cl_qlist_end(&p_rcv->p_subn->sa_sr_list)) { + p_svcr = (osm_svcr_t *) p_list_item; + + if (p_svcr->service_record.service_lease == 0xFFFFFFFF) { + p_list_item = cl_qlist_next(p_list_item); + continue; + } + + /* current time in seconds */ + curr_time = cl_get_time_stamp_sec(); + /* elapsed time from last modify */ + elapsed_time = curr_time - p_svcr->modified_time; + /* but it can not be less then 1 */ + if (elapsed_time < 1) + elapsed_time = 1; + + if (elapsed_time < p_svcr->lease_period) { + /* + Just update the service lease period + note: for simplicity we work with a uint32_t field + external to the network order lease_period of the MAD + */ + p_svcr->lease_period -= elapsed_time; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sr_rcv_lease_cb: " + "Remaining time for Service Name:%s is:0x%X\n", + p_svcr->service_record.service_name, + p_svcr->lease_period); + + p_svcr->modified_time = curr_time; + + /* Update the trim timer */ + if (trim_time > p_svcr->lease_period) { + trim_time = p_svcr->lease_period; + if (trim_time < 1) + trim_time = 1; + } + + p_list_item = cl_qlist_next(p_list_item); + continue; + + } else { + p_next_list_item = cl_qlist_next(p_list_item); + + /* Remove the service Record */ + osm_svcr_remove_from_db(p_rcv->p_subn, + p_rcv->p_log, p_svcr); + + osm_svcr_delete(p_svcr); + + p_list_item = p_next_list_item; + continue; + } + } + + /* Release the Lock */ + cl_plock_release(p_rcv->p_lock); + + if (trim_time != 0xFFFFFFFF) { + cl_timer_trim(&p_rcv->sr_timer, trim_time * 1000); /* Convert to milli seconds */ + } + + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_sminfo_record.c b/opensm/opensm/osm_sa_sminfo_record.c index f1769a8..8778dc5 100644 --- a/opensm/opensm/osm_sa_sminfo_record.c +++ b/opensm/opensm/osm_sa_sminfo_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -72,514 +72,492 @@ #define OSM_SMIR_RCV_POOL_MIN_SIZE 32 #define OSM_SMIR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_smir_item -{ - cl_pool_item_t pool_item; - ib_sminfo_record_t rec; +typedef struct _osm_smir_item { + cl_pool_item_t pool_item; + ib_sminfo_record_t rec; } osm_smir_item_t; -typedef struct _osm_smir_search_ctxt -{ - const ib_sminfo_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_smir_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_smir_search_ctxt { + const ib_sminfo_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_smir_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_smir_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_smir_rcv_construct( - IN osm_smir_rcv_t* const p_rcv ) +void osm_smir_rcv_construct(IN osm_smir_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_smir_rcv_destroy( - IN osm_smir_rcv_t* const p_rcv ) +void osm_smir_rcv_destroy(IN osm_smir_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_smir_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_smir_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_smir_rcv_init( - IN osm_smir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_stats_t* const p_stats, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_smir_rcv_init(IN osm_smir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_stats_t * const p_stats, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_smir_rcv_init ); + OSM_LOG_ENTER(p_log, osm_smir_rcv_init); - osm_smir_rcv_construct( p_rcv ); + osm_smir_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_stats = p_stats; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_stats = p_stats; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_SMIR_RCV_POOL_MIN_SIZE, - 0, - OSM_SMIR_RCV_POOL_GROW_SIZE, - sizeof(osm_smir_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_SMIR_RCV_POOL_MIN_SIZE, + 0, + OSM_SMIR_RCV_POOL_GROW_SIZE, + sizeof(osm_smir_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } static ib_api_status_t -__osm_smir_rcv_new_smir( - IN osm_smir_rcv_t* const p_rcv, - IN const osm_port_t* const p_port, - IN cl_qlist_t* const p_list, - IN ib_net64_t const guid, - IN ib_net32_t const act_count, - IN uint8_t const pri_state, - IN const osm_physp_t* const p_req_physp ) +__osm_smir_rcv_new_smir(IN osm_smir_rcv_t * const p_rcv, + IN const osm_port_t * const p_port, + IN cl_qlist_t * const p_list, + IN ib_net64_t const guid, + IN ib_net32_t const act_count, + IN uint8_t const pri_state, + IN const osm_physp_t * const p_req_physp) { - osm_smir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_smir_rcv_new_smir ); - - p_rec_item = (osm_smir_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_smir_rcv_new_smir: ERR 2801: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_smir_rcv_new_smir: " - "New SMInfo: GUID 0x%016" PRIx64 "\n", - cl_ntoh64( guid ) - ); - } - - memset( &p_rec_item->rec, 0, sizeof(ib_sminfo_record_t) ); - - p_rec_item->rec.lid = osm_port_get_base_lid( p_port ); - p_rec_item->rec.sm_info.guid = guid; - p_rec_item->rec.sm_info.act_count = act_count; - p_rec_item->rec.sm_info.pri_state = pri_state; - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_smir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_smir_rcv_new_smir); + + p_rec_item = (osm_smir_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_smir_rcv_new_smir: ERR 2801: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_smir_rcv_new_smir: " + "New SMInfo: GUID 0x%016" PRIx64 "\n", cl_ntoh64(guid) + ); + } + + memset(&p_rec_item->rec, 0, sizeof(ib_sminfo_record_t)); + + p_rec_item->rec.lid = osm_port_get_base_lid(p_port); + p_rec_item->rec.sm_info.guid = guid; + p_rec_item->rec.sm_info.act_count = act_count; + p_rec_item->rec.sm_info.pri_state = pri_state; + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_sa_smir_by_comp_mask( - IN osm_smir_rcv_t* const p_rcv, - IN const osm_remote_sm_t* const p_rem_sm, - osm_smir_search_ctxt_t* const p_ctxt ) +__osm_sa_smir_by_comp_mask(IN osm_smir_rcv_t * const p_rcv, + IN const osm_remote_sm_t * const p_rem_sm, + osm_smir_search_ctxt_t * const p_ctxt) { - const ib_sminfo_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_smir_by_comp_mask ); - - if ( comp_mask & IB_SMIR_COMPMASK_GUID ) - { - if ( p_rem_sm->smi.guid != p_rcvd_rec->sm_info.guid ) - goto Exit; - } - - if ( comp_mask & IB_SMIR_COMPMASK_PRIORITY ) - { - if ( ib_sminfo_get_priority( &p_rem_sm->smi ) != - ib_sminfo_get_priority( &p_rcvd_rec->sm_info ) ) - goto Exit; - } - - if ( comp_mask & IB_SMIR_COMPMASK_SMSTATE ) - { - if ( ib_sminfo_get_state( &p_rem_sm->smi ) != - ib_sminfo_get_state( &p_rcvd_rec->sm_info ) ) - goto Exit; - } - - /* Implement any other needed search cases */ - - __osm_smir_rcv_new_smir( p_rcv, p_rem_sm->p_port, p_ctxt->p_list, - p_rem_sm->smi.guid, - p_rem_sm->smi.act_count, - p_rem_sm->smi.pri_state, - p_req_physp ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_sminfo_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_smir_by_comp_mask); + + if (comp_mask & IB_SMIR_COMPMASK_GUID) { + if (p_rem_sm->smi.guid != p_rcvd_rec->sm_info.guid) + goto Exit; + } + + if (comp_mask & IB_SMIR_COMPMASK_PRIORITY) { + if (ib_sminfo_get_priority(&p_rem_sm->smi) != + ib_sminfo_get_priority(&p_rcvd_rec->sm_info)) + goto Exit; + } + + if (comp_mask & IB_SMIR_COMPMASK_SMSTATE) { + if (ib_sminfo_get_state(&p_rem_sm->smi) != + ib_sminfo_get_state(&p_rcvd_rec->sm_info)) + goto Exit; + } + + /* Implement any other needed search cases */ + + __osm_smir_rcv_new_smir(p_rcv, p_rem_sm->p_port, p_ctxt->p_list, + p_rem_sm->smi.guid, + p_rem_sm->smi.act_count, + p_rem_sm->smi.pri_state, p_req_physp); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sa_smir_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_smir_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_remote_sm_t* const p_rem_sm = (osm_remote_sm_t*)p_map_item; - osm_smir_search_ctxt_t* const p_ctxt = (osm_smir_search_ctxt_t *)context; + const osm_remote_sm_t *const p_rem_sm = (osm_remote_sm_t *) p_map_item; + osm_smir_search_ctxt_t *const p_ctxt = + (osm_smir_search_ctxt_t *) context; - __osm_sa_smir_by_comp_mask( p_ctxt->p_rcv, p_rem_sm, p_ctxt ); + __osm_sa_smir_by_comp_mask(p_ctxt->p_rcv, p_rem_sm, p_ctxt); } /********************************************************************** **********************************************************************/ -void -osm_smir_rcv_process( - IN void *ctx, - IN void *data ) +void osm_smir_rcv_process(IN void *ctx, IN void *data) { - osm_smir_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_sminfo_record_t* p_rcvd_rec; - const cl_qmap_t* p_tbl; - const osm_port_t* p_port = NULL; - const ib_sm_info_t* p_smi; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_sminfo_record_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + osm_smir_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_sminfo_record_t *p_rcvd_rec; + const cl_qmap_t *p_tbl; + const osm_port_t *p_port = NULL; + const ib_sm_info_t *p_smi; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_sminfo_record_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_smir_search_ctxt_t context; - osm_smir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t comp_mask; - ib_net64_t port_guid; - osm_physp_t* p_req_physp; - osm_port_t* local_port; - osm_remote_sm_t* p_rem_sm; - cl_qmap_t* p_sm_guid_tbl; - uint8_t pri_state; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_smir_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_sminfo_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - comp_mask = p_rcvd_mad->comp_mask; - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_SMINFO_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2804: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2803: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_sm_info_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - p_tbl = &p_rcv->p_subn->sm_guid_tbl; - p_smi = &p_rcvd_rec->sm_info; - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - /* - If the user specified a LID, it obviously narrows our - work load, since we don't have to search every port - */ - if( comp_mask & IB_SMIR_COMPMASK_LID ) - { - status = osm_get_port_by_base_lid( p_rcv->p_subn, p_rcvd_rec->lid, &p_port ); - if ( ( status != IB_SUCCESS ) || ( p_port == NULL ) ) - { - status = IB_NOT_FOUND; - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2806: " - "No port found with LID 0x%x\n", - cl_ntoh16(p_rcvd_rec->lid) ); - } - } - - if ( status == IB_SUCCESS ) - { - /* Handle our own SM first */ - local_port = osm_get_port_by_guid( p_rcv->p_subn, p_rcv->p_subn->sm_port_guid ); - if ( !local_port ) - { - cl_plock_release( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2809: " - "No port found with GUID 0x%016" PRIx64 "\n", - cl_ntoh64(p_rcv->p_subn->sm_port_guid ) ); - goto Exit; - } - - if ( !p_port || local_port == p_port ) - { - if (FALSE == - osm_physp_share_pkey( p_rcv->p_log, p_req_physp, - local_port->p_physp ) ) - { - cl_plock_release( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2805: " - "Cannot get SMInfo record due to pkey violation\n" ); - goto Exit; - } - - /* Check that other search components specified match */ - if ( comp_mask & IB_SMIR_COMPMASK_GUID ) - { - if ( p_rcv->p_subn->sm_port_guid != p_smi->guid ) - goto Remotes; - } - if ( comp_mask & IB_SMIR_COMPMASK_PRIORITY ) - { - if ( p_rcv->p_subn->opt.sm_priority != ib_sminfo_get_priority( p_smi ) ) - goto Remotes; - } - if ( comp_mask & IB_SMIR_COMPMASK_SMSTATE ) - { - if ( p_rcv->p_subn->sm_state != ib_sminfo_get_state( p_smi ) ) - goto Remotes; - } - - /* Now, add local SMInfo to list */ - pri_state = p_rcv->p_subn->sm_state & 0x0F; - pri_state |= (p_rcv->p_subn->opt.sm_priority & 0x0F) << 4; - __osm_smir_rcv_new_smir( p_rcv, local_port, context.p_list, - p_rcv->p_subn->sm_port_guid, - cl_ntoh32( p_rcv->p_stats->qp0_mads_sent ), - pri_state, - p_req_physp ); - } - - Remotes: - if( p_port && p_port != local_port ) - { - /* Find remote SM corresponding to p_port */ - port_guid = osm_port_get_guid( p_port ); - p_sm_guid_tbl = &p_rcv->p_subn->sm_guid_tbl; - p_rem_sm = (osm_remote_sm_t*)cl_qmap_get( p_sm_guid_tbl, port_guid ); - if (p_rem_sm != (osm_remote_sm_t*)cl_qmap_end( p_sm_guid_tbl ) ) - __osm_sa_smir_by_comp_mask( p_rcv, p_rem_sm, &context ); - else - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 280A: " - "No remote SM for GUID 0x%016" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - } - } - else - { - /* Go over all other known (remote) SMs */ - cl_qmap_apply_func( &p_rcv->p_subn->sm_guid_tbl, - __osm_sa_smir_by_comp_mask_cb, - &context ); - } - } - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2808: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS); - - /* need to set the mem free ... */ - p_rec_item = (osm_smir_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_smir_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_smir_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_smir_search_ctxt_t context; + osm_smir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t comp_mask; + ib_net64_t port_guid; + osm_physp_t *p_req_physp; + osm_port_t *local_port; + osm_remote_sm_t *p_rem_sm; + cl_qmap_t *p_sm_guid_tbl; + uint8_t pri_state; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_smir_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_sminfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + comp_mask = p_rcvd_mad->comp_mask; + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_SMINFO_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2804: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2803: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_sm_info_record(p_rcv->p_log, p_rcvd_rec, + OSM_LOG_DEBUG); + + p_tbl = &p_rcv->p_subn->sm_guid_tbl; + p_smi = &p_rcvd_rec->sm_info; + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + /* + If the user specified a LID, it obviously narrows our + work load, since we don't have to search every port + */ + if (comp_mask & IB_SMIR_COMPMASK_LID) { + status = + osm_get_port_by_base_lid(p_rcv->p_subn, p_rcvd_rec->lid, + &p_port); + if ((status != IB_SUCCESS) || (p_port == NULL)) { + status = IB_NOT_FOUND; + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2806: " + "No port found with LID 0x%x\n", + cl_ntoh16(p_rcvd_rec->lid)); + } + } + + if (status == IB_SUCCESS) { + /* Handle our own SM first */ + local_port = + osm_get_port_by_guid(p_rcv->p_subn, + p_rcv->p_subn->sm_port_guid); + if (!local_port) { + cl_plock_release(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2809: " + "No port found with GUID 0x%016" PRIx64 "\n", + cl_ntoh64(p_rcv->p_subn->sm_port_guid)); + goto Exit; + } + + if (!p_port || local_port == p_port) { + if (FALSE == + osm_physp_share_pkey(p_rcv->p_log, p_req_physp, + local_port->p_physp)) { + cl_plock_release(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2805: " + "Cannot get SMInfo record due to pkey violation\n"); + goto Exit; + } + + /* Check that other search components specified match */ + if (comp_mask & IB_SMIR_COMPMASK_GUID) { + if (p_rcv->p_subn->sm_port_guid != p_smi->guid) + goto Remotes; + } + if (comp_mask & IB_SMIR_COMPMASK_PRIORITY) { + if (p_rcv->p_subn->opt.sm_priority != + ib_sminfo_get_priority(p_smi)) + goto Remotes; + } + if (comp_mask & IB_SMIR_COMPMASK_SMSTATE) { + if (p_rcv->p_subn->sm_state != + ib_sminfo_get_state(p_smi)) + goto Remotes; + } + + /* Now, add local SMInfo to list */ + pri_state = p_rcv->p_subn->sm_state & 0x0F; + pri_state |= + (p_rcv->p_subn->opt.sm_priority & 0x0F) << 4; + __osm_smir_rcv_new_smir(p_rcv, local_port, + context.p_list, + p_rcv->p_subn->sm_port_guid, + cl_ntoh32(p_rcv->p_stats-> + qp0_mads_sent), + pri_state, p_req_physp); + } + + Remotes: + if (p_port && p_port != local_port) { + /* Find remote SM corresponding to p_port */ + port_guid = osm_port_get_guid(p_port); + p_sm_guid_tbl = &p_rcv->p_subn->sm_guid_tbl; + p_rem_sm = + (osm_remote_sm_t *) cl_qmap_get(p_sm_guid_tbl, + port_guid); + if (p_rem_sm != + (osm_remote_sm_t *) cl_qmap_end(p_sm_guid_tbl)) + __osm_sa_smir_by_comp_mask(p_rcv, p_rem_sm, + &context); + else { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 280A: " + "No remote SM for GUID 0x%016" PRIx64 + "\n", cl_ntoh64(port_guid)); + } + } else { + /* Go over all other known (remote) SMs */ + cl_qmap_apply_func(&p_rcv->p_subn->sm_guid_tbl, + __osm_sa_smir_by_comp_mask_cb, + &context); + } + } + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2808: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_smir_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_smir_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_sminfo_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_smir_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_sminfo_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_smir_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_smir_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_sminfo_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2807: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_smir_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_NO_RESOURCES ); - - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_sminfo_record_t) ); - - p_resp_rec = (ib_sminfo_record_t*) - ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_smir_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * sizeof(ib_sminfo_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2807: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_sminfo_record_t)); + + p_resp_rec = (ib_sminfo_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_smir_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - p_resp_rec->sm_info.sm_key = 0; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if( status != IB_SUCCESS ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_smir_rcv_process: ERR 2802: " - "Error sending MAD (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_smir_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + p_resp_rec->sm_info.sm_key = 0; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_smir_rcv_process: ERR 2802: " + "Error sending MAD (%s)\n", ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_sw_info_record.c b/opensm/opensm/osm_sa_sw_info_record.c index 3463e83..adddd28 100644 --- a/opensm/opensm/osm_sa_sw_info_record.c +++ b/opensm/opensm/osm_sa_sw_info_record.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -61,476 +61,450 @@ #define OSM_SIR_RCV_POOL_MIN_SIZE 32 #define OSM_SIR_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_sir_item -{ - cl_pool_item_t pool_item; - ib_switch_info_record_t rec; +typedef struct _osm_sir_item { + cl_pool_item_t pool_item; + ib_switch_info_record_t rec; } osm_sir_item_t; -typedef struct _osm_sir_search_ctxt -{ - const ib_switch_info_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - cl_qlist_t* p_list; - osm_sir_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_sir_search_ctxt { + const ib_switch_info_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + cl_qlist_t *p_list; + osm_sir_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_sir_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_sir_rcv_construct( - IN osm_sir_rcv_t* const p_rcv ) +void osm_sir_rcv_construct(IN osm_sir_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_sir_rcv_destroy( - IN osm_sir_rcv_t* const p_rcv ) +void osm_sir_rcv_destroy(IN osm_sir_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_sir_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_sir_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sir_rcv_init( - IN osm_sir_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_sir_rcv_init(IN osm_sir_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_sir_rcv_init ); + OSM_LOG_ENTER(p_log, osm_sir_rcv_init); - osm_sir_rcv_construct( p_rcv ); + osm_sir_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_SIR_RCV_POOL_MIN_SIZE, - 0, - OSM_SIR_RCV_POOL_GROW_SIZE, - sizeof(osm_sir_item_t), - NULL, NULL, NULL ); + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_SIR_RCV_POOL_MIN_SIZE, + 0, + OSM_SIR_RCV_POOL_GROW_SIZE, + sizeof(osm_sir_item_t), NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_sir_rcv_new_sir( - IN osm_sir_rcv_t* const p_rcv, - IN const osm_switch_t* const p_sw, - IN cl_qlist_t* const p_list, - IN ib_net16_t const lid ) +__osm_sir_rcv_new_sir(IN osm_sir_rcv_t * const p_rcv, + IN const osm_switch_t * const p_sw, + IN cl_qlist_t * const p_list, IN ib_net16_t const lid) { - osm_sir_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sir_rcv_new_sir ); - - p_rec_item = (osm_sir_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sir_rcv_new_sir: ERR 5308: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sir_rcv_new_sir: " - "New SwitchInfoRecord: lid 0x%X\n", - cl_ntoh16( lid ) - ); - } - - memset( &p_rec_item->rec, 0, sizeof(ib_switch_info_record_t) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.switch_info = p_sw->switch_info; - - cl_qlist_insert_tail( p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + osm_sir_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sir_rcv_new_sir); + + p_rec_item = (osm_sir_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sir_rcv_new_sir: ERR 5308: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sir_rcv_new_sir: " + "New SwitchInfoRecord: lid 0x%X\n", cl_ntoh16(lid) + ); + } + + memset(&p_rec_item->rec, 0, sizeof(ib_switch_info_record_t)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.switch_info = p_sw->switch_info; + + cl_qlist_insert_tail(p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -static osm_port_t* -__osm_sir_get_port_by_guid( - IN osm_sir_rcv_t* const p_rcv, - IN uint64_t port_guid ) +static osm_port_t *__osm_sir_get_port_by_guid(IN osm_sir_rcv_t * const p_rcv, + IN uint64_t port_guid) { - osm_port_t* p_port; - - CL_PLOCK_ACQUIRE(p_rcv->p_lock); - - p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); - if (!p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sir_get_port_by_guid ERR 5309: " - "Invalid port GUID 0x%016" PRIx64 "\n", - port_guid ); - p_port = NULL; - } - - CL_PLOCK_RELEASE(p_rcv->p_lock); - return p_port; + osm_port_t *p_port; + + CL_PLOCK_ACQUIRE(p_rcv->p_lock); + + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sir_get_port_by_guid ERR 5309: " + "Invalid port GUID 0x%016" PRIx64 "\n", port_guid); + p_port = NULL; + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + return p_port; } /********************************************************************** **********************************************************************/ static void -__osm_sir_rcv_create_sir( - IN osm_sir_rcv_t* const p_rcv, - IN const osm_switch_t* const p_sw, - IN cl_qlist_t* const p_list, - IN ib_net16_t const match_lid, - IN const osm_physp_t* const p_req_physp ) +__osm_sir_rcv_create_sir(IN osm_sir_rcv_t * const p_rcv, + IN const osm_switch_t * const p_sw, + IN cl_qlist_t * const p_list, + IN ib_net16_t const match_lid, + IN const osm_physp_t * const p_req_physp) { - osm_port_t* p_port; - const osm_physp_t* p_physp; - uint16_t match_lid_ho; - ib_net16_t min_lid_ho; - ib_net16_t max_lid_ho; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sir_rcv_create_sir ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sir_rcv_create_sir: " - "Looking for SwitchInfoRecord with LID: 0x%X\n", - cl_ntoh16( match_lid ) - ); - } - - /* In switches, the port guid is the node guid. */ - p_port = - __osm_sir_get_port_by_guid( p_rcv, p_sw->p_node->node_info.port_guid ); - if (! p_port) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sir_rcv_create_sir: ERR 530A: " - "Failed to find Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - goto Exit; - } - - /* check that the requester physp and the current physp are under - the same partition. */ - p_physp = p_port->p_physp; - if (! p_physp) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sir_rcv_create_sir: ERR 530B: " - "Failed to find default physical Port by Node Guid:0x%016" PRIx64 - "\n", - cl_ntoh64( p_sw->p_node->node_info.node_guid ) - ); - goto Exit; - } - if (! osm_physp_share_pkey( p_rcv->p_log, p_req_physp, p_physp )) - goto Exit; - - /* get the port 0 of the switch */ - osm_port_get_lid_range_ho( p_port, &min_lid_ho, &max_lid_ho ); - - match_lid_ho = cl_ntoh16( match_lid ); - if( match_lid_ho ) - { - /* - We validate that the lid belongs to this switch. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sir_rcv_create_sir: " - "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", - min_lid_ho, match_lid_ho, max_lid_ho - ); - } - - if ( match_lid_ho < min_lid_ho || match_lid_ho > max_lid_ho ) - goto Exit; - - } - - __osm_sir_rcv_new_sir( p_rcv, p_sw, p_list, osm_port_get_base_lid(p_port) ); - -Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_port_t *p_port; + const osm_physp_t *p_physp; + uint16_t match_lid_ho; + ib_net16_t min_lid_ho; + ib_net16_t max_lid_ho; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sir_rcv_create_sir); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sir_rcv_create_sir: " + "Looking for SwitchInfoRecord with LID: 0x%X\n", + cl_ntoh16(match_lid) + ); + } + + /* In switches, the port guid is the node guid. */ + p_port = + __osm_sir_get_port_by_guid(p_rcv, + p_sw->p_node->node_info.port_guid); + if (!p_port) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sir_rcv_create_sir: ERR 530A: " + "Failed to find Port by Node Guid:0x%016" PRIx64 + "\n", cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + goto Exit; + } + + /* check that the requester physp and the current physp are under + the same partition. */ + p_physp = p_port->p_physp; + if (!p_physp) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sir_rcv_create_sir: ERR 530B: " + "Failed to find default physical Port by Node Guid:0x%016" + PRIx64 "\n", + cl_ntoh64(p_sw->p_node->node_info.node_guid) + ); + goto Exit; + } + if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) + goto Exit; + + /* get the port 0 of the switch */ + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); + + match_lid_ho = cl_ntoh16(match_lid); + if (match_lid_ho) { + /* + We validate that the lid belongs to this switch. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sir_rcv_create_sir: " + "Comparing LID: 0x%X <= 0x%X <= 0x%X\n", + min_lid_ho, match_lid_ho, max_lid_ho); + } + + if (match_lid_ho < min_lid_ho || match_lid_ho > max_lid_ho) + goto Exit; + + } + + __osm_sir_rcv_new_sir(p_rcv, p_sw, p_list, + osm_port_get_base_lid(p_port)); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_sir_rcv_by_comp_mask( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sir_rcv_by_comp_mask(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_sir_search_ctxt_t* const p_ctxt = (osm_sir_search_ctxt_t *)context; - const osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - const ib_switch_info_record_t* const p_rcvd_rec = p_ctxt->p_rcvd_rec; - const osm_physp_t* const p_req_physp = p_ctxt->p_req_physp; - osm_sir_rcv_t* const p_rcv = p_ctxt->p_rcv; - ib_net64_t const comp_mask = p_ctxt->comp_mask; - ib_net16_t match_lid = 0; - - OSM_LOG_ENTER( p_ctxt->p_rcv->p_log, __osm_sir_rcv_by_comp_mask ); - - osm_dump_switch_info( - p_ctxt->p_rcv->p_log, - &p_sw->switch_info, - OSM_LOG_VERBOSE ); - - if( comp_mask & IB_SWIR_COMPMASK_LID ) - { - match_lid = p_rcvd_rec->lid; - if (!match_lid) - goto Exit; - } - - __osm_sir_rcv_create_sir( p_rcv, p_sw, p_ctxt->p_list, - match_lid, p_req_physp ); - -Exit: - OSM_LOG_EXIT( p_ctxt->p_rcv->p_log ); + const osm_sir_search_ctxt_t *const p_ctxt = + (osm_sir_search_ctxt_t *) context; + const osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + const ib_switch_info_record_t *const p_rcvd_rec = p_ctxt->p_rcvd_rec; + const osm_physp_t *const p_req_physp = p_ctxt->p_req_physp; + osm_sir_rcv_t *const p_rcv = p_ctxt->p_rcv; + ib_net64_t const comp_mask = p_ctxt->comp_mask; + ib_net16_t match_lid = 0; + + OSM_LOG_ENTER(p_ctxt->p_rcv->p_log, __osm_sir_rcv_by_comp_mask); + + osm_dump_switch_info(p_ctxt->p_rcv->p_log, + &p_sw->switch_info, OSM_LOG_VERBOSE); + + if (comp_mask & IB_SWIR_COMPMASK_LID) { + match_lid = p_rcvd_rec->lid; + if (!match_lid) + goto Exit; + } + + __osm_sir_rcv_create_sir(p_rcv, p_sw, p_ctxt->p_list, + match_lid, p_req_physp); + + Exit: + OSM_LOG_EXIT(p_ctxt->p_rcv->p_log); } /********************************************************************** **********************************************************************/ -void -osm_sir_rcv_process( - IN void *ctx, - IN void *data ) +void osm_sir_rcv_process(IN void *ctx, IN void *data) { - osm_sir_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_switch_info_record_t* p_rcvd_rec; - ib_switch_info_record_t* p_resp_rec; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - uint32_t num_rec, pre_trim_num_rec; + osm_sir_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_switch_info_record_t *p_rcvd_rec; + ib_switch_info_record_t *p_resp_rec; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_sir_search_ctxt_t context; - osm_sir_item_t* p_rec_item; - ib_api_status_t status; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_sir_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_switch_info_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_SWITCH_INFO_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sir_rcv_process: ERR 5305: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sir_rcv_process: ERR 5304: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - if ( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - osm_dump_switch_info_record( p_rcv->p_log, p_rcvd_rec, OSM_LOG_DEBUG ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.p_req_physp = p_req_physp; - - cl_plock_acquire( p_rcv->p_lock ); - - /* Go over all switches */ - cl_qmap_apply_func( &p_rcv->p_subn->sw_guid_tbl, - __osm_sir_rcv_by_comp_mask, - &context ); - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if ( (p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1) ) { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_sir_rcv_process: ERR 5303: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_rec_item = (osm_sir_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_sir_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_sir_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_sir_search_ctxt_t context; + osm_sir_item_t *p_rec_item; + ib_api_status_t status; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_sir_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_switch_info_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_SWITCH_INFO_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sir_rcv_process: ERR 5305: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sir_rcv_process: ERR 5304: " + "Cannot find requester physical port\n"); + goto Exit; + } + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) + osm_dump_switch_info_record(p_rcv->p_log, p_rcvd_rec, + OSM_LOG_DEBUG); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.p_req_physp = p_req_physp; + + cl_plock_acquire(p_rcv->p_lock); + + /* Go over all switches */ + cl_qmap_apply_func(&p_rcv->p_subn->sw_guid_tbl, + __osm_sir_rcv_by_comp_mask, &context); + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec > 1)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sir_rcv_process: ERR 5303: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); + while (p_rec_item != (osm_sir_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_rec_item = + (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - /* we limit the number of records to a single packet */ - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_switch_info_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_sir_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + /* we limit the number of records to a single packet */ + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_switch_info_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_sir_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_sir_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_switch_info_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_sir_rcv_process: ERR 5306: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_sir_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_switch_info_record_t) ); - - p_resp_rec = (ib_switch_info_record_t*)ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_sir_rcv_process: " "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * + sizeof(ib_switch_info_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sir_rcv_process: ERR 5306: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_switch_info_record_t)); + + p_resp_rec = + (ib_switch_info_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_sir_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if (status != IB_SUCCESS) - { - osm_log(p_rcv->p_log, OSM_LOG_ERROR, - "osm_sir_rcv_process: ERR 5307: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status)); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = (osm_sir_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_sir_rcv_process: ERR 5307: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sa_vlarb_record.c b/opensm/opensm/osm_sa_vlarb_record.c index f88da8d..3350af0 100644 --- a/opensm/opensm/osm_sa_vlarb_record.c +++ b/opensm/opensm/osm_sa_vlarb_record.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -65,508 +65,495 @@ #define OSM_VLARB_REC_RCV_POOL_MIN_SIZE 32 #define OSM_VLARB_REC_RCV_POOL_GROW_SIZE 32 -typedef struct _osm_vl_arb_item -{ - cl_pool_item_t pool_item; - ib_vl_arb_table_record_t rec; +typedef struct _osm_vl_arb_item { + cl_pool_item_t pool_item; + ib_vl_arb_table_record_t rec; } osm_vl_arb_item_t; -typedef struct _osm_vl_arb_search_ctxt -{ - const ib_vl_arb_table_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - uint8_t block_num; - cl_qlist_t* p_list; - osm_vlarb_rec_rcv_t* p_rcv; - const osm_physp_t* p_req_physp; +typedef struct _osm_vl_arb_search_ctxt { + const ib_vl_arb_table_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + uint8_t block_num; + cl_qlist_t *p_list; + osm_vlarb_rec_rcv_t *p_rcv; + const osm_physp_t *p_req_physp; } osm_vl_arb_search_ctxt_t; /********************************************************************** **********************************************************************/ -void -osm_vlarb_rec_rcv_construct( - IN osm_vlarb_rec_rcv_t* const p_rcv ) +void osm_vlarb_rec_rcv_construct(IN osm_vlarb_rec_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); - cl_qlock_pool_construct( &p_rcv->pool ); + memset(p_rcv, 0, sizeof(*p_rcv)); + cl_qlock_pool_construct(&p_rcv->pool); } /********************************************************************** **********************************************************************/ -void -osm_vlarb_rec_rcv_destroy( - IN osm_vlarb_rec_rcv_t* const p_rcv ) +void osm_vlarb_rec_rcv_destroy(IN osm_vlarb_rec_rcv_t * const p_rcv) { - OSM_LOG_ENTER( p_rcv->p_log, osm_vlarb_rec_rcv_destroy ); - cl_qlock_pool_destroy( &p_rcv->pool ); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_ENTER(p_rcv->p_log, osm_vlarb_rec_rcv_destroy); + cl_qlock_pool_destroy(&p_rcv->pool); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vlarb_rec_rcv_init( - IN osm_vlarb_rec_rcv_t* const p_rcv, - IN osm_sa_resp_t* const p_resp, - IN osm_mad_pool_t* const p_mad_pool, - IN const osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_vlarb_rec_rcv_init(IN osm_vlarb_rec_rcv_t * const p_rcv, + IN osm_sa_resp_t * const p_resp, + IN osm_mad_pool_t * const p_mad_pool, + IN const osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_log, osm_vlarb_rec_rcv_init ); + OSM_LOG_ENTER(p_log, osm_vlarb_rec_rcv_init); - osm_vlarb_rec_rcv_construct( p_rcv ); + osm_vlarb_rec_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_resp = p_resp; - p_rcv->p_mad_pool = p_mad_pool; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_resp = p_resp; + p_rcv->p_mad_pool = p_mad_pool; - /* used for matching records collection */ - status = cl_qlock_pool_init( &p_rcv->pool, - OSM_VLARB_REC_RCV_POOL_MIN_SIZE, - 0, - OSM_VLARB_REC_RCV_POOL_GROW_SIZE, - sizeof(osm_vl_arb_item_t), - NULL, NULL, NULL ); + /* used for matching records collection */ + status = cl_qlock_pool_init(&p_rcv->pool, + OSM_VLARB_REC_RCV_POOL_MIN_SIZE, + 0, + OSM_VLARB_REC_RCV_POOL_GROW_SIZE, + sizeof(osm_vl_arb_item_t), + NULL, NULL, NULL); - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ void -__osm_sa_vl_arb_create( - IN osm_vlarb_rec_rcv_t* const p_rcv, - IN osm_physp_t* const p_physp, - IN osm_vl_arb_search_ctxt_t* const p_ctxt, - IN uint8_t block ) +__osm_sa_vl_arb_create(IN osm_vlarb_rec_rcv_t * const p_rcv, + IN osm_physp_t * const p_physp, + IN osm_vl_arb_search_ctxt_t * const p_ctxt, + IN uint8_t block) { - osm_vl_arb_item_t* p_rec_item; - uint16_t lid; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_vl_arb_create ); - - p_rec_item = (osm_vl_arb_item_t*)cl_qlock_pool_get( &p_rcv->pool ); - if( p_rec_item == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_vl_arb_create: ERR 2A02: " - "cl_qlock_pool_get failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - if (p_physp->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) - lid = p_physp->port_info.base_lid; - else - lid = osm_node_get_base_lid( p_physp->p_node, 0 ); - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_vl_arb_create: " - "New VLArbitration for: port 0x%016" PRIx64 - ", lid 0x%X, port 0x%X Block:%u\n", - cl_ntoh64( osm_physp_get_port_guid( p_physp ) ), - cl_ntoh16( lid ), osm_physp_get_port_num( p_physp ), - block - ); - } - - memset( &p_rec_item->rec, 0, sizeof( p_rec_item->rec ) ); - - p_rec_item->rec.lid = lid; - p_rec_item->rec.port_num = osm_physp_get_port_num( p_physp ); - p_rec_item->rec.block_num = block; - p_rec_item->rec.vl_arb_tbl = *(osm_physp_get_vla_tbl(p_physp, block)); - - cl_qlist_insert_tail( p_ctxt->p_list, (cl_list_item_t*)&p_rec_item->pool_item ); - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_vl_arb_item_t *p_rec_item; + uint16_t lid; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_vl_arb_create); + + p_rec_item = (osm_vl_arb_item_t *) cl_qlock_pool_get(&p_rcv->pool); + if (p_rec_item == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_vl_arb_create: ERR 2A02: " + "cl_qlock_pool_get failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + if (p_physp->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) + lid = p_physp->port_info.base_lid; + else + lid = osm_node_get_base_lid(p_physp->p_node, 0); + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_vl_arb_create: " + "New VLArbitration for: port 0x%016" PRIx64 + ", lid 0x%X, port 0x%X Block:%u\n", + cl_ntoh64(osm_physp_get_port_guid(p_physp)), + cl_ntoh16(lid), osm_physp_get_port_num(p_physp), block); + } + + memset(&p_rec_item->rec, 0, sizeof(p_rec_item->rec)); + + p_rec_item->rec.lid = lid; + p_rec_item->rec.port_num = osm_physp_get_port_num(p_physp); + p_rec_item->rec.block_num = block; + p_rec_item->rec.vl_arb_tbl = *(osm_physp_get_vla_tbl(p_physp, block)); + + cl_qlist_insert_tail(p_ctxt->p_list, + (cl_list_item_t *) & p_rec_item->pool_item); + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_vl_arb_check_physp( - IN osm_vlarb_rec_rcv_t* const p_rcv, - IN osm_physp_t* const p_physp, - osm_vl_arb_search_ctxt_t* const p_ctxt ) +__osm_sa_vl_arb_check_physp(IN osm_vlarb_rec_rcv_t * const p_rcv, + IN osm_physp_t * const p_physp, + osm_vl_arb_search_ctxt_t * const p_ctxt) { - ib_net64_t comp_mask = p_ctxt->comp_mask; - uint8_t block; + ib_net64_t comp_mask = p_ctxt->comp_mask; + uint8_t block; - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_vl_arb_check_physp ); + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_vl_arb_check_physp); - /* we got here with the phys port - all that's left is to get the right block */ - for (block = 1; block <= 4; block++) { - if (!(comp_mask & IB_VLA_COMPMASK_BLOCK) || block == p_ctxt->block_num) - { - __osm_sa_vl_arb_create( p_rcv, p_physp, p_ctxt, block ); - } - } + /* we got here with the phys port - all that's left is to get the right block */ + for (block = 1; block <= 4; block++) { + if (!(comp_mask & IB_VLA_COMPMASK_BLOCK) + || block == p_ctxt->block_num) { + __osm_sa_vl_arb_create(p_rcv, p_physp, p_ctxt, block); + } + } - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_vl_arb_by_comp_mask( - IN osm_vlarb_rec_rcv_t* const p_rcv, - IN const osm_port_t* const p_port, - osm_vl_arb_search_ctxt_t* const p_ctxt ) +__osm_sa_vl_arb_by_comp_mask(IN osm_vlarb_rec_rcv_t * const p_rcv, + IN const osm_port_t * const p_port, + osm_vl_arb_search_ctxt_t * const p_ctxt) { - const ib_vl_arb_table_record_t* p_rcvd_rec; - ib_net64_t comp_mask; - osm_physp_t * p_physp; - uint8_t port_num; - uint8_t num_ports; - const osm_physp_t* p_req_physp; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_sa_vl_arb_by_comp_mask ); - - p_rcvd_rec = p_ctxt->p_rcvd_rec; - comp_mask = p_ctxt->comp_mask; - port_num = p_rcvd_rec->port_num; - p_req_physp = p_ctxt->p_req_physp; - - /* if this is a switch port we can search all ports - otherwise we must be looking on port 0 */ - if ( p_port->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) - { - /* we put it in the comp mask and port num */ - port_num = p_port->p_physp->port_num; - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_sa_vl_arb_by_comp_mask: " - "Using Physical Default Port Number: 0x%X (for End Node)\n", - port_num ); - comp_mask |= IB_VLA_COMPMASK_OUT_PORT; - } - - if( comp_mask & IB_VLA_COMPMASK_OUT_PORT ) - { - if (port_num < osm_node_get_num_physp( p_port->p_node )) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, port_num ); - /* check that the p_physp is valid, and that the requester - and the p_physp share a pkey. */ - if( osm_physp_is_valid( p_physp ) && - osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp) ) - __osm_sa_vl_arb_check_physp( p_rcv, p_physp, p_ctxt ); - } - else - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_sa_vl_arb_by_comp_mask: ERR 2A03: " - "Given Physical Port Number: 0x%X is out of range should be < 0x%X\n", - port_num, osm_node_get_num_physp( p_port->p_node ) ); - goto Exit; - } - } - else - { - num_ports = osm_node_get_num_physp( p_port->p_node ); - for( port_num = 0; port_num < num_ports; port_num++ ) - { - p_physp = osm_node_get_physp_ptr( p_port->p_node, port_num ); - if( !osm_physp_is_valid( p_physp ) ) - continue; - - /* if the requester and the p_physp don't share a pkey - - continue */ - if (!osm_physp_share_pkey(p_rcv->p_log, p_req_physp, p_physp)) - continue; - - __osm_sa_vl_arb_check_physp( p_rcv, p_physp, p_ctxt ); - } - } - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + const ib_vl_arb_table_record_t *p_rcvd_rec; + ib_net64_t comp_mask; + osm_physp_t *p_physp; + uint8_t port_num; + uint8_t num_ports; + const osm_physp_t *p_req_physp; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_sa_vl_arb_by_comp_mask); + + p_rcvd_rec = p_ctxt->p_rcvd_rec; + comp_mask = p_ctxt->comp_mask; + port_num = p_rcvd_rec->port_num; + p_req_physp = p_ctxt->p_req_physp; + + /* if this is a switch port we can search all ports + otherwise we must be looking on port 0 */ + if (p_port->p_node->node_info.node_type != IB_NODE_TYPE_SWITCH) { + /* we put it in the comp mask and port num */ + port_num = p_port->p_physp->port_num; + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_sa_vl_arb_by_comp_mask: " + "Using Physical Default Port Number: 0x%X (for End Node)\n", + port_num); + comp_mask |= IB_VLA_COMPMASK_OUT_PORT; + } + + if (comp_mask & IB_VLA_COMPMASK_OUT_PORT) { + if (port_num < osm_node_get_num_physp(p_port->p_node)) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, port_num); + /* check that the p_physp is valid, and that the requester + and the p_physp share a pkey. */ + if (osm_physp_is_valid(p_physp) && + osm_physp_share_pkey(p_rcv->p_log, p_req_physp, + p_physp)) + __osm_sa_vl_arb_check_physp(p_rcv, p_physp, + p_ctxt); + } else { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_sa_vl_arb_by_comp_mask: ERR 2A03: " + "Given Physical Port Number: 0x%X is out of range should be < 0x%X\n", + port_num, + osm_node_get_num_physp(p_port->p_node)); + goto Exit; + } + } else { + num_ports = osm_node_get_num_physp(p_port->p_node); + for (port_num = 0; port_num < num_ports; port_num++) { + p_physp = + osm_node_get_physp_ptr(p_port->p_node, port_num); + if (!osm_physp_is_valid(p_physp)) + continue; + + /* if the requester and the p_physp don't share a pkey - + continue */ + if (!osm_physp_share_pkey + (p_rcv->p_log, p_req_physp, p_physp)) + continue; + + __osm_sa_vl_arb_check_physp(p_rcv, p_physp, p_ctxt); + } + } + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ void -__osm_sa_vl_arb_by_comp_mask_cb( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_sa_vl_arb_by_comp_mask_cb(IN cl_map_item_t * const p_map_item, + IN void *context) { - const osm_port_t* const p_port = (osm_port_t*)p_map_item; - osm_vl_arb_search_ctxt_t* const p_ctxt = (osm_vl_arb_search_ctxt_t *)context; + const osm_port_t *const p_port = (osm_port_t *) p_map_item; + osm_vl_arb_search_ctxt_t *const p_ctxt = + (osm_vl_arb_search_ctxt_t *) context; - __osm_sa_vl_arb_by_comp_mask( p_ctxt->p_rcv, p_port, p_ctxt ); + __osm_sa_vl_arb_by_comp_mask(p_ctxt->p_rcv, p_port, p_ctxt); } /********************************************************************** **********************************************************************/ -void -osm_vlarb_rec_rcv_process( - IN void *ctx, - IN void *data ) +void osm_vlarb_rec_rcv_process(IN void *ctx, IN void *data) { - osm_vlarb_rec_rcv_t *p_rcv = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t* p_rcvd_mad; - const ib_vl_arb_table_record_t* p_rcvd_rec; - const cl_ptr_vector_t* p_tbl; - const osm_port_t* p_port = NULL; - const ib_vl_arb_table_t* p_vl_arb; - cl_qlist_t rec_list; - osm_madw_t* p_resp_madw; - ib_sa_mad_t* p_resp_sa_mad; - ib_vl_arb_table_record_t* p_resp_rec; - uint32_t num_rec, pre_trim_num_rec; + osm_vlarb_rec_rcv_t *p_rcv = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + const ib_vl_arb_table_record_t *p_rcvd_rec; + const cl_ptr_vector_t *p_tbl; + const osm_port_t *p_port = NULL; + const ib_vl_arb_table_t *p_vl_arb; + cl_qlist_t rec_list; + osm_madw_t *p_resp_madw; + ib_sa_mad_t *p_resp_sa_mad; + ib_vl_arb_table_record_t *p_resp_rec; + uint32_t num_rec, pre_trim_num_rec; #ifndef VENDOR_RMPP_SUPPORT - uint32_t trim_num_rec; + uint32_t trim_num_rec; #endif - uint32_t i; - osm_vl_arb_search_ctxt_t context; - osm_vl_arb_item_t* p_rec_item; - ib_api_status_t status = IB_SUCCESS; - ib_net64_t comp_mask; - osm_physp_t* p_req_physp; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_vlarb_rec_rcv_process ); - - CL_ASSERT( p_madw ); - - p_rcvd_mad = osm_madw_get_sa_mad_ptr( p_madw ); - p_rcvd_rec = (ib_vl_arb_table_record_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - comp_mask = p_rcvd_mad->comp_mask; - - CL_ASSERT( p_rcvd_mad->attr_id == IB_MAD_ATTR_VLARB_RECORD ); - - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if ( (p_rcvd_mad->method != IB_MAD_METHOD_GET) && - (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A05: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str( p_rcvd_mad->method ) ); - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR ); - goto Exit; - } - - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, - p_rcv->p_subn, - osm_madw_get_mad_addr_ptr(p_madw) ); - if (p_req_physp == NULL) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A04: " - "Cannot find requester physical port\n" ); - goto Exit; - } - - p_vl_arb = (ib_vl_arb_table_t*)ib_sa_mad_get_payload_ptr( p_rcvd_mad ); - - cl_qlist_init( &rec_list ); - - context.p_rcvd_rec = p_rcvd_rec; - context.p_list = &rec_list; - context.comp_mask = p_rcvd_mad->comp_mask; - context.p_rcv = p_rcv; - context.block_num = p_rcvd_rec->block_num; - context.p_req_physp = p_req_physp; - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_vlarb_rec_rcv_process: " - "Got Query Lid:0x%04X(%02X), Port:0x%02X(%02X), Block:0x%02X(%02X)\n", - cl_ntoh16(p_rcvd_rec->lid), (comp_mask & IB_VLA_COMPMASK_LID) != 0, - p_rcvd_rec->port_num, (comp_mask & IB_VLA_COMPMASK_OUT_PORT) != 0, - p_rcvd_rec->block_num, (comp_mask & IB_VLA_COMPMASK_BLOCK) != 0 ); - - cl_plock_acquire( p_rcv->p_lock ); - - /* - If the user specified a LID, it obviously narrows our - work load, since we don't have to search every port - */ - if( comp_mask & IB_VLA_COMPMASK_LID ) - { - - p_tbl = &p_rcv->p_subn->port_lid_tbl; - - CL_ASSERT( cl_ptr_vector_get_size(p_tbl) < 0x10000 ); - - status = osm_get_port_by_base_lid( p_rcv->p_subn, p_rcvd_rec->lid, &p_port ); - if ( ( status != IB_SUCCESS ) || ( p_port == NULL ) ) - { - status = IB_NOT_FOUND; - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A09: " - "No port found with LID 0x%x\n", - cl_ntoh16(p_rcvd_rec->lid) ); - } - } - - if ( status == IB_SUCCESS ) - { - /* if we got a unique port - no need for a port search */ - if( p_port ) - /* this does the loop on all the port phys ports */ - __osm_sa_vl_arb_by_comp_mask( p_rcv, p_port, &context ); - else - { - cl_qmap_apply_func( &p_rcv->p_subn->port_guid_tbl, - __osm_sa_vl_arb_by_comp_mask_cb, - &context ); - } - } - - cl_plock_release( p_rcv->p_lock ); - - num_rec = cl_qlist_count( &rec_list ); - - /* - * C15-0.1.30: - * If we do a SubnAdmGet and got more than one record it is an error ! - */ - if (p_rcvd_mad->method == IB_MAD_METHOD_GET) - { - if (num_rec == 0) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - if (num_rec > 1) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A08: " - "Got more than one record for SubnAdmGet (%u)\n", - num_rec ); - osm_sa_send_error( p_rcv->p_resp, p_madw, - IB_SA_MAD_STATUS_TOO_MANY_RECORDS ); - - /* need to set the mem free ... */ - p_rec_item = (osm_vl_arb_item_t*)cl_qlist_remove_head( &rec_list ); - while( p_rec_item != (osm_vl_arb_item_t*)cl_qlist_end( &rec_list ) ) - { - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_rec_item = (osm_vl_arb_item_t*)cl_qlist_remove_head( &rec_list ); - } - - goto Exit; - } - } - - pre_trim_num_rec = num_rec; + uint32_t i; + osm_vl_arb_search_ctxt_t context; + osm_vl_arb_item_t *p_rec_item; + ib_api_status_t status = IB_SUCCESS; + ib_net64_t comp_mask; + osm_physp_t *p_req_physp; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_vlarb_rec_rcv_process); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_vl_arb_table_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + comp_mask = p_rcvd_mad->comp_mask; + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_VLARB_RECORD); + + /* we only support SubnAdmGet and SubnAdmGetTable methods */ + if ((p_rcvd_mad->method != IB_MAD_METHOD_GET) && + (p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A05: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_MAD_STATUS_UNSUP_METHOD_ATTR); + goto Exit; + } + + /* update the requester physical port. */ + p_req_physp = osm_get_physp_by_mad_addr(p_rcv->p_log, + p_rcv->p_subn, + osm_madw_get_mad_addr_ptr + (p_madw)); + if (p_req_physp == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A04: " + "Cannot find requester physical port\n"); + goto Exit; + } + + p_vl_arb = (ib_vl_arb_table_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + + cl_qlist_init(&rec_list); + + context.p_rcvd_rec = p_rcvd_rec; + context.p_list = &rec_list; + context.comp_mask = p_rcvd_mad->comp_mask; + context.p_rcv = p_rcv; + context.block_num = p_rcvd_rec->block_num; + context.p_req_physp = p_req_physp; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_vlarb_rec_rcv_process: " + "Got Query Lid:0x%04X(%02X), Port:0x%02X(%02X), Block:0x%02X(%02X)\n", + cl_ntoh16(p_rcvd_rec->lid), + (comp_mask & IB_VLA_COMPMASK_LID) != 0, p_rcvd_rec->port_num, + (comp_mask & IB_VLA_COMPMASK_OUT_PORT) != 0, + p_rcvd_rec->block_num, + (comp_mask & IB_VLA_COMPMASK_BLOCK) != 0); + + cl_plock_acquire(p_rcv->p_lock); + + /* + If the user specified a LID, it obviously narrows our + work load, since we don't have to search every port + */ + if (comp_mask & IB_VLA_COMPMASK_LID) { + + p_tbl = &p_rcv->p_subn->port_lid_tbl; + + CL_ASSERT(cl_ptr_vector_get_size(p_tbl) < 0x10000); + + status = + osm_get_port_by_base_lid(p_rcv->p_subn, p_rcvd_rec->lid, + &p_port); + if ((status != IB_SUCCESS) || (p_port == NULL)) { + status = IB_NOT_FOUND; + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A09: " + "No port found with LID 0x%x\n", + cl_ntoh16(p_rcvd_rec->lid)); + } + } + + if (status == IB_SUCCESS) { + /* if we got a unique port - no need for a port search */ + if (p_port) + /* this does the loop on all the port phys ports */ + __osm_sa_vl_arb_by_comp_mask(p_rcv, p_port, &context); + else { + cl_qmap_apply_func(&p_rcv->p_subn->port_guid_tbl, + __osm_sa_vl_arb_by_comp_mask_cb, + &context); + } + } + + cl_plock_release(p_rcv->p_lock); + + num_rec = cl_qlist_count(&rec_list); + + /* + * C15-0.1.30: + * If we do a SubnAdmGet and got more than one record it is an error ! + */ + if (p_rcvd_mad->method == IB_MAD_METHOD_GET) { + if (num_rec == 0) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + if (num_rec > 1) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A08: " + "Got more than one record for SubnAdmGet (%u)\n", + num_rec); + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_TOO_MANY_RECORDS); + + /* need to set the mem free ... */ + p_rec_item = + (osm_vl_arb_item_t *) + cl_qlist_remove_head(&rec_list); + while (p_rec_item != + (osm_vl_arb_item_t *) cl_qlist_end(&rec_list)) { + cl_qlock_pool_put(&p_rcv->pool, + &p_rec_item->pool_item); + p_rec_item = + (osm_vl_arb_item_t *) + cl_qlist_remove_head(&rec_list); + } + + goto Exit; + } + } + + pre_trim_num_rec = num_rec; #ifndef VENDOR_RMPP_SUPPORT - trim_num_rec = (MAD_BLOCK_SIZE - IB_SA_MAD_HDR_SIZE) / sizeof(ib_vl_arb_table_record_t); - if (trim_num_rec < num_rec) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_vlarb_rec_rcv_process: " - "Number of records:%u trimmed to:%u to fit in one MAD\n", - num_rec, trim_num_rec ); - num_rec = trim_num_rec; - } + trim_num_rec = + (MAD_BLOCK_SIZE - + IB_SA_MAD_HDR_SIZE) / sizeof(ib_vl_arb_table_record_t); + if (trim_num_rec < num_rec) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_vlarb_rec_rcv_process: " + "Number of records:%u trimmed to:%u to fit in one MAD\n", + num_rec, trim_num_rec); + num_rec = trim_num_rec; + } #endif - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_vlarb_rec_rcv_process: " - "Returning %u records\n", num_rec ); - - if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) - { - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RECORDS ); - goto Exit; - } - - /* - * Get a MAD to reply. Address of Mad is in the received mad_wrapper - */ - p_resp_madw = osm_mad_pool_get( p_rcv->p_mad_pool, - p_madw->h_bind, - num_rec * sizeof(ib_vl_arb_table_record_t) + IB_SA_MAD_HDR_SIZE, - &p_madw->mad_addr ); - - if( !p_resp_madw ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A06: " - "osm_mad_pool_get failed\n" ); - - for( i = 0; i < num_rec; i++ ) - { - p_rec_item = (osm_vl_arb_item_t*)cl_qlist_remove_head( &rec_list ); - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - } - - osm_sa_send_error( p_rcv->p_resp, p_madw, IB_SA_MAD_STATUS_NO_RESOURCES ); - goto Exit; - } - - p_resp_sa_mad = osm_madw_get_sa_mad_ptr( p_resp_madw ); - - /* - Copy the MAD header back into the response mad. - Set the 'R' bit and the payload length, - Then copy all records from the list into the response payload. - */ - - memcpy( p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE ); - p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; - /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ - p_resp_sa_mad->sm_key = 0; - - /* Fill in the offset (paylen will be done by the rmpp SAR) */ - p_resp_sa_mad->attr_offset = - ib_get_attr_offset( sizeof(ib_vl_arb_table_record_t) ); - - p_resp_rec = (ib_vl_arb_table_record_t*) - ib_sa_mad_get_payload_ptr( p_resp_sa_mad ); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_vlarb_rec_rcv_process: " + "Returning %u records\n", num_rec); + + if ((p_rcvd_mad->method == IB_MAD_METHOD_GET) && (num_rec == 0)) { + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RECORDS); + goto Exit; + } + + /* + * Get a MAD to reply. Address of Mad is in the received mad_wrapper + */ + p_resp_madw = osm_mad_pool_get(p_rcv->p_mad_pool, + p_madw->h_bind, + num_rec * + sizeof(ib_vl_arb_table_record_t) + + IB_SA_MAD_HDR_SIZE, &p_madw->mad_addr); + + if (!p_resp_madw) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A06: " + "osm_mad_pool_get failed\n"); + + for (i = 0; i < num_rec; i++) { + p_rec_item = + (osm_vl_arb_item_t *) + cl_qlist_remove_head(&rec_list); + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + } + + osm_sa_send_error(p_rcv->p_resp, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + goto Exit; + } + + p_resp_sa_mad = osm_madw_get_sa_mad_ptr(p_resp_madw); + + /* + Copy the MAD header back into the response mad. + Set the 'R' bit and the payload length, + Then copy all records from the list into the response payload. + */ + + memcpy(p_resp_sa_mad, p_rcvd_mad, IB_SA_MAD_HDR_SIZE); + p_resp_sa_mad->method |= IB_MAD_METHOD_RESP_MASK; + /* C15-0.1.5 - always return SM_Key = 0 (table 185 p 884) */ + p_resp_sa_mad->sm_key = 0; + + /* Fill in the offset (paylen will be done by the rmpp SAR) */ + p_resp_sa_mad->attr_offset = + ib_get_attr_offset(sizeof(ib_vl_arb_table_record_t)); + + p_resp_rec = (ib_vl_arb_table_record_t *) + ib_sa_mad_get_payload_ptr(p_resp_sa_mad); #ifndef VENDOR_RMPP_SUPPORT - /* we support only one packet RMPP - so we will set the first and - last flags for gettable */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - { - p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | IB_RMPP_FLAG_ACTIVE; - } + /* we support only one packet RMPP - so we will set the first and + last flags for gettable */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) { + p_resp_sa_mad->rmpp_type = IB_RMPP_TYPE_DATA; + p_resp_sa_mad->rmpp_flags = + IB_RMPP_FLAG_FIRST | IB_RMPP_FLAG_LAST | + IB_RMPP_FLAG_ACTIVE; + } #else - /* forcefully define the packet as RMPP one */ - if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) - p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; + /* forcefully define the packet as RMPP one */ + if (p_resp_sa_mad->method == IB_MAD_METHOD_GETTABLE_RESP) + p_resp_sa_mad->rmpp_flags = IB_RMPP_FLAG_ACTIVE; #endif - for( i = 0; i < pre_trim_num_rec; i++ ) - { - p_rec_item = (osm_vl_arb_item_t*)cl_qlist_remove_head( &rec_list ); - /* copy only if not trimmed */ - if (i < num_rec) - { - *p_resp_rec = p_rec_item->rec; - } - cl_qlock_pool_put( &p_rcv->pool, &p_rec_item->pool_item ); - p_resp_rec++; - } - - CL_ASSERT( cl_is_qlist_empty( &rec_list ) ); - - status = osm_vendor_send( p_resp_madw->h_bind, p_resp_madw, FALSE ); - if(status != IB_SUCCESS) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vlarb_rec_rcv_process: ERR 2A07: " - "osm_vendor_send status = %s\n", - ib_get_err_str(status) ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + for (i = 0; i < pre_trim_num_rec; i++) { + p_rec_item = + (osm_vl_arb_item_t *) cl_qlist_remove_head(&rec_list); + /* copy only if not trimmed */ + if (i < num_rec) { + *p_resp_rec = p_rec_item->rec; + } + cl_qlock_pool_put(&p_rcv->pool, &p_rec_item->pool_item); + p_resp_rec++; + } + + CL_ASSERT(cl_is_qlist_empty(&rec_list)); + + status = osm_vendor_send(p_resp_madw->h_bind, p_resp_madw, FALSE); + if (status != IB_SUCCESS) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vlarb_rec_rcv_process: ERR 2A07: " + "osm_vendor_send status = %s\n", + ib_get_err_str(status)); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_service.c b/opensm/opensm/osm_service.c index 2f7d545..fcbdee3 100644 --- a/opensm/opensm/osm_service.c +++ b/opensm/opensm/osm_service.c @@ -45,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -54,139 +54,120 @@ /********************************************************************** **********************************************************************/ -void -osm_svcr_delete( - IN osm_svcr_t* const p_svcr ) +void osm_svcr_delete(IN osm_svcr_t * const p_svcr) { - free( p_svcr); + free(p_svcr); } /********************************************************************** **********************************************************************/ void -osm_svcr_init( - IN osm_svcr_t* const p_svcr, - IN const ib_service_record_t *p_svc_rec ) +osm_svcr_init(IN osm_svcr_t * const p_svcr, + IN const ib_service_record_t * p_svc_rec) { - CL_ASSERT( p_svcr ); + CL_ASSERT(p_svcr); - p_svcr->modified_time = cl_get_time_stamp_sec(); + p_svcr->modified_time = cl_get_time_stamp_sec(); - /* We track the time left for this service in - an external field to avoid extra cl_ntoh/hton - required for working with the MAD field */ - p_svcr->lease_period = cl_ntoh32(p_svc_rec->service_lease); - p_svcr->service_record = *p_svc_rec; + /* We track the time left for this service in + an external field to avoid extra cl_ntoh/hton + required for working with the MAD field */ + p_svcr->lease_period = cl_ntoh32(p_svc_rec->service_lease); + p_svcr->service_record = *p_svc_rec; } /********************************************************************** **********************************************************************/ -osm_svcr_t* -osm_svcr_new( - IN const ib_service_record_t *p_svc_rec ) +osm_svcr_t *osm_svcr_new(IN const ib_service_record_t * p_svc_rec) { - osm_svcr_t* p_svcr; + osm_svcr_t *p_svcr; - CL_ASSERT(p_svc_rec); + CL_ASSERT(p_svc_rec); - p_svcr = (osm_svcr_t*)malloc( sizeof(*p_svcr) ); - if( p_svcr ) - { - memset( p_svcr, 0, sizeof(*p_svcr) ); - osm_svcr_init( p_svcr, p_svc_rec ); - } + p_svcr = (osm_svcr_t *) malloc(sizeof(*p_svcr)); + if (p_svcr) { + memset(p_svcr, 0, sizeof(*p_svcr)); + osm_svcr_init(p_svcr, p_svc_rec); + } - return( p_svcr ); + return (p_svcr); } /********************************************************************** **********************************************************************/ static -cl_status_t -__match_rid_of_svc_rec( - - IN const cl_list_item_t* const p_list_item, - IN void* context ) + cl_status_t +__match_rid_of_svc_rec(IN const cl_list_item_t * const p_list_item, + IN void *context) { - ib_service_record_t* p_svc_rec = (ib_service_record_t *)context; - osm_svcr_t* p_svcr = (osm_svcr_t*)p_list_item; - int32_t count; - - count = memcmp( - &p_svcr->service_record, - p_svc_rec, - sizeof(p_svc_rec->service_id) + - sizeof(p_svc_rec->service_gid) + - sizeof(p_svc_rec->service_pkey) ); - - if(count == 0) - return CL_SUCCESS; - else - return CL_NOT_FOUND; + ib_service_record_t *p_svc_rec = (ib_service_record_t *) context; + osm_svcr_t *p_svcr = (osm_svcr_t *) p_list_item; + int32_t count; + + count = memcmp(&p_svcr->service_record, + p_svc_rec, + sizeof(p_svc_rec->service_id) + + sizeof(p_svc_rec->service_gid) + + sizeof(p_svc_rec->service_pkey)); + + if (count == 0) + return CL_SUCCESS; + else + return CL_NOT_FOUND; } /********************************************************************** **********************************************************************/ -osm_svcr_t* -osm_svcr_get_by_rid( - IN osm_subn_t const *p_subn, - IN osm_log_t *p_log, - IN ib_service_record_t* const p_svc_rec ) +osm_svcr_t *osm_svcr_get_by_rid(IN osm_subn_t const *p_subn, + IN osm_log_t * p_log, + IN ib_service_record_t * const p_svc_rec) { - cl_list_item_t* p_list_item; + cl_list_item_t *p_list_item; - OSM_LOG_ENTER( p_log, osm_svcr_get_by_rid ); + OSM_LOG_ENTER(p_log, osm_svcr_get_by_rid); - p_list_item = cl_qlist_find_from_head( - &p_subn->sa_sr_list, - __match_rid_of_svc_rec, - p_svc_rec); + p_list_item = cl_qlist_find_from_head(&p_subn->sa_sr_list, + __match_rid_of_svc_rec, + p_svc_rec); - if( p_list_item == cl_qlist_end( &p_subn->sa_sr_list ) ) - p_list_item = NULL; + if (p_list_item == cl_qlist_end(&p_subn->sa_sr_list)) + p_list_item = NULL; - OSM_LOG_EXIT( p_log ); - return (osm_svcr_t*)p_list_item; + OSM_LOG_EXIT(p_log); + return (osm_svcr_t *) p_list_item; } /********************************************************************** **********************************************************************/ void -osm_svcr_insert_to_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_svcr_t *p_svcr) +osm_svcr_insert_to_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_svcr_t * p_svcr) { - OSM_LOG_ENTER( p_log, osm_svcr_insert_to_db ); + OSM_LOG_ENTER(p_log, osm_svcr_insert_to_db); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_svcr_insert_to_db: " - "Inserting new Service Record into Database\n"); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_svcr_insert_to_db: " + "Inserting new Service Record into Database\n"); - cl_qlist_insert_head(&p_subn->sa_sr_list, - &p_svcr->list_item); + cl_qlist_insert_head(&p_subn->sa_sr_list, &p_svcr->list_item); - OSM_LOG_EXIT( p_log ); + OSM_LOG_EXIT(p_log); } void -osm_svcr_remove_from_db( - IN osm_subn_t *p_subn, - IN osm_log_t *p_log, - IN osm_svcr_t *p_svcr) +osm_svcr_remove_from_db(IN osm_subn_t * p_subn, + IN osm_log_t * p_log, IN osm_svcr_t * p_svcr) { - OSM_LOG_ENTER( p_log, osm_svcr_remove_from_db ); + OSM_LOG_ENTER(p_log, osm_svcr_remove_from_db); - osm_log( p_log, OSM_LOG_DEBUG, - "osm_svcr_remove_from_db: " - "Removing Service Record Name:%s ID:0x%016" PRIx64" from Database\n", - p_svcr->service_record.service_name, p_svcr->service_record.service_id - ); + osm_log(p_log, OSM_LOG_DEBUG, + "osm_svcr_remove_from_db: " + "Removing Service Record Name:%s ID:0x%016" PRIx64 + " from Database\n", p_svcr->service_record.service_name, + p_svcr->service_record.service_id); - cl_qlist_remove_item(&p_subn->sa_sr_list, - &p_svcr->list_item); + cl_qlist_remove_item(&p_subn->sa_sr_list, &p_svcr->list_item); - OSM_LOG_EXIT( p_log ); + OSM_LOG_EXIT(p_log); } - diff --git a/opensm/opensm/osm_slvl_map_rcv.c b/opensm/opensm/osm_slvl_map_rcv.c index 0b44f58..e3942fb 100644 --- a/opensm/opensm/osm_slvl_map_rcv.c +++ b/opensm/opensm/osm_slvl_map_rcv.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_slvl_rcv_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -69,49 +68,43 @@ /********************************************************************** **********************************************************************/ -void -osm_slvl_rcv_construct( - IN osm_slvl_rcv_t* const p_rcv ) +void osm_slvl_rcv_construct(IN osm_slvl_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_slvl_rcv_destroy( - IN osm_slvl_rcv_t* const p_rcv ) +void osm_slvl_rcv_destroy(IN osm_slvl_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_slvl_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_slvl_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_slvl_rcv_init( - IN osm_slvl_rcv_t* const p_rcv, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_slvl_rcv_init(IN osm_slvl_rcv_t * const p_rcv, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_slvl_rcv_init ); + OSM_LOG_ENTER(p_log, osm_slvl_rcv_init); - osm_slvl_rcv_construct( p_rcv ); + osm_slvl_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_req = p_req; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_req = p_req; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** @@ -119,114 +112,101 @@ osm_slvl_rcv_init( /* * WE MIGHT ONLY RECEIVE A GET or SET responses */ -void -osm_slvl_rcv_process( - IN void *context, - IN void *p_data ) +void osm_slvl_rcv_process(IN void *context, IN void *p_data) { - osm_slvl_rcv_t *p_rcv = context; - osm_madw_t *p_madw = p_data; - ib_slvl_table_t *p_slvl_tbl; - ib_smp_t *p_smp; - osm_port_t *p_port; - osm_physp_t *p_physp; - osm_node_t *p_node; - osm_slvl_context_t *p_context; - ib_net64_t port_guid; - ib_net64_t node_guid; - uint8_t out_port_num, in_port_num; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_slvl_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_context = osm_madw_get_slvl_context_ptr( p_madw ); - p_slvl_tbl = (ib_slvl_table_t*)ib_smp_get_payload_ptr( p_smp ); - - port_guid = p_context->port_guid; - node_guid = p_context->node_guid; - - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_SLVL_TABLE ); - - cl_plock_excl_acquire( p_rcv->p_lock ); - p_port = osm_get_port_by_guid( p_rcv->p_subn, port_guid ); - - if( !p_port ) - { - cl_plock_release( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_slvl_rcv_process: ERR 2C06: " - "No port object for port with GUID 0x%" PRIx64 - "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - goto Exit; - } - - p_node = p_port->p_node; - CL_ASSERT( p_node ); - - /* in case of a non switch node the attr modifier should be ignored */ - if (osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH) - { - out_port_num = (uint8_t)cl_ntoh32( p_smp->attr_mod & 0xFF000000); - in_port_num = (uint8_t)cl_ntoh32( (p_smp->attr_mod & 0x00FF0000) << 8); - p_physp = osm_node_get_physp_ptr( p_node, out_port_num ); - } - else - { - p_physp = p_port->p_physp; - out_port_num = p_physp->port_num; - in_port_num = 0; - } - - CL_ASSERT( p_physp ); - - /* - We do not mind if this is a result of a set or get - all we want is to update - the subnet. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_slvl_rcv_process: " - "Got SLtoVL get response in_port_num %u out_port_num %u with GUID 0x%" PRIx64 - " for parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - in_port_num, out_port_num, - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - } - - /* - Determine if we encountered a new Physical Port. - If so, Ignore it. - */ - if( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_slvl_rcv_process: " - "Got invalid port number 0x%X\n", - out_port_num ); - goto Exit; - } - - osm_dump_slvl_map_table( p_rcv->p_log, - port_guid, in_port_num, - out_port_num, p_slvl_tbl, - OSM_LOG_DEBUG ); - - osm_physp_set_slvl_tbl( p_physp, p_slvl_tbl, in_port_num); - - Exit: - cl_plock_release( p_rcv->p_lock ); - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_slvl_rcv_t *p_rcv = context; + osm_madw_t *p_madw = p_data; + ib_slvl_table_t *p_slvl_tbl; + ib_smp_t *p_smp; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_node_t *p_node; + osm_slvl_context_t *p_context; + ib_net64_t port_guid; + ib_net64_t node_guid; + uint8_t out_port_num, in_port_num; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_slvl_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_context = osm_madw_get_slvl_context_ptr(p_madw); + p_slvl_tbl = (ib_slvl_table_t *) ib_smp_get_payload_ptr(p_smp); + + port_guid = p_context->port_guid; + node_guid = p_context->node_guid; + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_SLVL_TABLE); + + cl_plock_excl_acquire(p_rcv->p_lock); + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + + if (!p_port) { + cl_plock_release(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_slvl_rcv_process: ERR 2C06: " + "No port object for port with GUID 0x%" PRIx64 + "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 + ", TID 0x%" PRIx64 "\n", + cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + goto Exit; + } + + p_node = p_port->p_node; + CL_ASSERT(p_node); + + /* in case of a non switch node the attr modifier should be ignored */ + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) { + out_port_num = + (uint8_t) cl_ntoh32(p_smp->attr_mod & 0xFF000000); + in_port_num = + (uint8_t) cl_ntoh32((p_smp->attr_mod & 0x00FF0000) << 8); + p_physp = osm_node_get_physp_ptr(p_node, out_port_num); + } else { + p_physp = p_port->p_physp; + out_port_num = p_physp->port_num; + in_port_num = 0; + } + + CL_ASSERT(p_physp); + + /* + We do not mind if this is a result of a set or get - all we want is to update + the subnet. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_slvl_rcv_process: " + "Got SLtoVL get response in_port_num %u out_port_num %u with GUID 0x%" + PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", in_port_num, out_port_num, + cl_ntoh64(port_guid), cl_ntoh64(node_guid), + cl_ntoh64(p_smp->trans_id)); + } + + /* + Determine if we encountered a new Physical Port. + If so, Ignore it. + */ + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_slvl_rcv_process: " + "Got invalid port number 0x%X\n", out_port_num); + goto Exit; + } + + osm_dump_slvl_map_table(p_rcv->p_log, + port_guid, in_port_num, + out_port_num, p_slvl_tbl, OSM_LOG_DEBUG); + + osm_physp_set_slvl_tbl(p_physp, p_slvl_tbl, in_port_num); + + Exit: + cl_plock_release(p_rcv->p_lock); + + OSM_LOG_EXIT(p_rcv->p_log); } - diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c index 403f6b1..6fc9496 100644 --- a/opensm/opensm/osm_sm.c +++ b/opensm/opensm/osm_sm.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -69,735 +69,702 @@ /********************************************************************** **********************************************************************/ -void -__osm_sm_sweeper( - IN void *p_ptr ) +void __osm_sm_sweeper(IN void *p_ptr) { - ib_api_status_t status; - osm_sm_t *const p_sm = ( osm_sm_t * ) p_ptr; - - OSM_LOG_ENTER( p_sm->p_log, __osm_sm_sweeper ); - - if( p_sm->thread_state == OSM_THREAD_STATE_INIT ) - { - p_sm->thread_state = OSM_THREAD_STATE_RUN; - } - - /* If the sweep interval was updated before - then run only if - * it is not zero. */ - while( p_sm->thread_state == OSM_THREAD_STATE_RUN && - p_sm->p_subn->opt.sweep_interval != 0 ) - { - /* do the sweep only if we are in MASTER state */ - if( p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || - p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING ) - osm_state_mgr_process( &p_sm->state_mgr, OSM_SIGNAL_SWEEP ); - - /* - * Wait on the event with a timeout. - * Sweeps may be initiated "off schedule" by simply - * signaling the event. - */ - status = cl_event_wait_on( &p_sm->signal, - p_sm->p_subn->opt.sweep_interval * 1000000, - TRUE ); - - if( status == CL_SUCCESS ) - { - if( osm_log_is_active( p_sm->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_sm->p_log, OSM_LOG_DEBUG, - "__osm_sm_sweeper: " "Off schedule sweep signalled\n" ); - } - } - else - { - if( status != CL_TIMEOUT ) - { - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "__osm_sm_sweeper: ERR 2E01: " - "Event wait failed (%s)\n", CL_STATUS_MSG( status ) ); - } - } - } - - OSM_LOG_EXIT( p_sm->p_log ); + ib_api_status_t status; + osm_sm_t *const p_sm = (osm_sm_t *) p_ptr; + + OSM_LOG_ENTER(p_sm->p_log, __osm_sm_sweeper); + + if (p_sm->thread_state == OSM_THREAD_STATE_INIT) { + p_sm->thread_state = OSM_THREAD_STATE_RUN; + } + + /* If the sweep interval was updated before - then run only if + * it is not zero. */ + while (p_sm->thread_state == OSM_THREAD_STATE_RUN && + p_sm->p_subn->opt.sweep_interval != 0) { + /* do the sweep only if we are in MASTER state */ + if (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || + p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING) + osm_state_mgr_process(&p_sm->state_mgr, + OSM_SIGNAL_SWEEP); + + /* + * Wait on the event with a timeout. + * Sweeps may be initiated "off schedule" by simply + * signaling the event. + */ + status = cl_event_wait_on(&p_sm->signal, + p_sm->p_subn->opt.sweep_interval * + 1000000, TRUE); + + if (status == CL_SUCCESS) { + if (osm_log_is_active(p_sm->p_log, OSM_LOG_DEBUG)) { + osm_log(p_sm->p_log, OSM_LOG_DEBUG, + "__osm_sm_sweeper: " + "Off schedule sweep signalled\n"); + } + } else { + if (status != CL_TIMEOUT) { + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "__osm_sm_sweeper: ERR 2E01: " + "Event wait failed (%s)\n", + CL_STATUS_MSG(status)); + } + } + } + + OSM_LOG_EXIT(p_sm->p_log); } /********************************************************************** **********************************************************************/ -void -osm_sm_construct( - IN osm_sm_t * const p_sm ) +void osm_sm_construct(IN osm_sm_t * const p_sm) { - memset( p_sm, 0, sizeof( *p_sm ) ); - p_sm->thread_state = OSM_THREAD_STATE_NONE; - p_sm->sm_trans_id = OSM_SM_INITIAL_TID_VALUE; - cl_event_construct( &p_sm->signal ); - cl_event_construct( &p_sm->subnet_up_event ); - cl_thread_construct( &p_sm->sweeper ); - osm_req_construct( &p_sm->req ); - osm_resp_construct( &p_sm->resp ); - osm_ni_rcv_construct( &p_sm->ni_rcv ); - osm_pi_rcv_construct( &p_sm->pi_rcv ); - osm_nd_rcv_construct( &p_sm->nd_rcv ); - osm_sm_mad_ctrl_construct( &p_sm->mad_ctrl ); - osm_si_rcv_construct( &p_sm->si_rcv ); - osm_lid_mgr_construct( &p_sm->lid_mgr ); - osm_ucast_mgr_construct( &p_sm->ucast_mgr ); - osm_link_mgr_construct( &p_sm->link_mgr ); - osm_state_mgr_construct( &p_sm->state_mgr ); - osm_state_mgr_ctrl_construct( &p_sm->state_mgr_ctrl ); - osm_drop_mgr_construct( &p_sm->drop_mgr ); - osm_lft_rcv_construct( &p_sm->lft_rcv ); - osm_mft_rcv_construct( &p_sm->mft_rcv ); - osm_sweep_fail_ctrl_construct( &p_sm->sweep_fail_ctrl ); - osm_sminfo_rcv_construct( &p_sm->sm_info_rcv ); - osm_trap_rcv_construct( &p_sm->trap_rcv ); - osm_sm_state_mgr_construct( &p_sm->sm_state_mgr ); - osm_slvl_rcv_construct( &p_sm->slvl_rcv ); - osm_vla_rcv_construct( &p_sm->vla_rcv ); - osm_pkey_rcv_construct( &p_sm->pkey_rcv ); - osm_mcast_mgr_construct( &p_sm->mcast_mgr ); + memset(p_sm, 0, sizeof(*p_sm)); + p_sm->thread_state = OSM_THREAD_STATE_NONE; + p_sm->sm_trans_id = OSM_SM_INITIAL_TID_VALUE; + cl_event_construct(&p_sm->signal); + cl_event_construct(&p_sm->subnet_up_event); + cl_thread_construct(&p_sm->sweeper); + osm_req_construct(&p_sm->req); + osm_resp_construct(&p_sm->resp); + osm_ni_rcv_construct(&p_sm->ni_rcv); + osm_pi_rcv_construct(&p_sm->pi_rcv); + osm_nd_rcv_construct(&p_sm->nd_rcv); + osm_sm_mad_ctrl_construct(&p_sm->mad_ctrl); + osm_si_rcv_construct(&p_sm->si_rcv); + osm_lid_mgr_construct(&p_sm->lid_mgr); + osm_ucast_mgr_construct(&p_sm->ucast_mgr); + osm_link_mgr_construct(&p_sm->link_mgr); + osm_state_mgr_construct(&p_sm->state_mgr); + osm_state_mgr_ctrl_construct(&p_sm->state_mgr_ctrl); + osm_drop_mgr_construct(&p_sm->drop_mgr); + osm_lft_rcv_construct(&p_sm->lft_rcv); + osm_mft_rcv_construct(&p_sm->mft_rcv); + osm_sweep_fail_ctrl_construct(&p_sm->sweep_fail_ctrl); + osm_sminfo_rcv_construct(&p_sm->sm_info_rcv); + osm_trap_rcv_construct(&p_sm->trap_rcv); + osm_sm_state_mgr_construct(&p_sm->sm_state_mgr); + osm_slvl_rcv_construct(&p_sm->slvl_rcv); + osm_vla_rcv_construct(&p_sm->vla_rcv); + osm_pkey_rcv_construct(&p_sm->pkey_rcv); + osm_mcast_mgr_construct(&p_sm->mcast_mgr); } /********************************************************************** **********************************************************************/ -void -osm_sm_shutdown( - IN osm_sm_t * const p_sm ) +void osm_sm_shutdown(IN osm_sm_t * const p_sm) { - boolean_t signal_event = FALSE; - - OSM_LOG_ENTER( p_sm->p_log, osm_sm_shutdown ); - - /* - * Signal our threads that we're leaving. - */ - if( p_sm->thread_state != OSM_THREAD_STATE_NONE ) - signal_event = TRUE; - - p_sm->thread_state = OSM_THREAD_STATE_EXIT; - - /* - * Don't trigger unless event has been initialized. - * Destroy the thread before we tear down the other objects. - */ - if( signal_event ) - cl_event_signal( &p_sm->signal ); - - cl_thread_destroy( &p_sm->sweeper ); - - /* - * Always destroy controllers before the corresponding - * receiver to guarantee that all callbacks from the - * dispatcher are complete. - */ - osm_sm_mad_ctrl_destroy( &p_sm->mad_ctrl ); - cl_disp_unregister(p_sm->ni_disp_h); - cl_disp_unregister(p_sm->pi_disp_h); - cl_disp_unregister(p_sm->si_disp_h); - cl_disp_unregister(p_sm->nd_disp_h); - cl_disp_unregister(p_sm->lft_disp_h); - cl_disp_unregister(p_sm->mft_disp_h); - cl_disp_unregister(p_sm->sm_info_disp_h); - cl_disp_unregister(p_sm->trap_disp_h); - cl_disp_unregister(p_sm->slvl_disp_h); - cl_disp_unregister(p_sm->vla_disp_h); - cl_disp_unregister(p_sm->pkey_disp_h); - osm_sweep_fail_ctrl_destroy( &p_sm->sweep_fail_ctrl ); - osm_state_mgr_ctrl_destroy( &p_sm->state_mgr_ctrl ); - - OSM_LOG_EXIT( p_sm->p_log ); + boolean_t signal_event = FALSE; + + OSM_LOG_ENTER(p_sm->p_log, osm_sm_shutdown); + + /* + * Signal our threads that we're leaving. + */ + if (p_sm->thread_state != OSM_THREAD_STATE_NONE) + signal_event = TRUE; + + p_sm->thread_state = OSM_THREAD_STATE_EXIT; + + /* + * Don't trigger unless event has been initialized. + * Destroy the thread before we tear down the other objects. + */ + if (signal_event) + cl_event_signal(&p_sm->signal); + + cl_thread_destroy(&p_sm->sweeper); + + /* + * Always destroy controllers before the corresponding + * receiver to guarantee that all callbacks from the + * dispatcher are complete. + */ + osm_sm_mad_ctrl_destroy(&p_sm->mad_ctrl); + cl_disp_unregister(p_sm->ni_disp_h); + cl_disp_unregister(p_sm->pi_disp_h); + cl_disp_unregister(p_sm->si_disp_h); + cl_disp_unregister(p_sm->nd_disp_h); + cl_disp_unregister(p_sm->lft_disp_h); + cl_disp_unregister(p_sm->mft_disp_h); + cl_disp_unregister(p_sm->sm_info_disp_h); + cl_disp_unregister(p_sm->trap_disp_h); + cl_disp_unregister(p_sm->slvl_disp_h); + cl_disp_unregister(p_sm->vla_disp_h); + cl_disp_unregister(p_sm->pkey_disp_h); + osm_sweep_fail_ctrl_destroy(&p_sm->sweep_fail_ctrl); + osm_state_mgr_ctrl_destroy(&p_sm->state_mgr_ctrl); + + OSM_LOG_EXIT(p_sm->p_log); } /********************************************************************** **********************************************************************/ -void -osm_sm_destroy( - IN osm_sm_t * const p_sm ) +void osm_sm_destroy(IN osm_sm_t * const p_sm) { - OSM_LOG_ENTER( p_sm->p_log, osm_sm_destroy ); - osm_trap_rcv_destroy( &p_sm->trap_rcv ); - osm_sminfo_rcv_destroy( &p_sm->sm_info_rcv ); - osm_req_destroy( &p_sm->req ); - osm_resp_destroy( &p_sm->resp ); - osm_ni_rcv_destroy( &p_sm->ni_rcv ); - osm_pi_rcv_destroy( &p_sm->pi_rcv ); - osm_si_rcv_destroy( &p_sm->si_rcv ); - osm_nd_rcv_destroy( &p_sm->nd_rcv ); - osm_lid_mgr_destroy( &p_sm->lid_mgr ); - osm_ucast_mgr_destroy( &p_sm->ucast_mgr ); - osm_link_mgr_destroy( &p_sm->link_mgr ); - osm_drop_mgr_destroy( &p_sm->drop_mgr ); - osm_lft_rcv_destroy( &p_sm->lft_rcv ); - osm_mft_rcv_destroy( &p_sm->mft_rcv ); - osm_slvl_rcv_destroy( &p_sm->slvl_rcv ); - osm_vla_rcv_destroy( &p_sm->vla_rcv ); - osm_pkey_rcv_destroy( &p_sm->pkey_rcv ); - osm_state_mgr_destroy( &p_sm->state_mgr ); - osm_sm_state_mgr_destroy( &p_sm->sm_state_mgr ); - osm_mcast_mgr_destroy( &p_sm->mcast_mgr ); - cl_event_destroy( &p_sm->signal ); - cl_event_destroy( &p_sm->subnet_up_event ); - - osm_log( p_sm->p_log, OSM_LOG_SYS, "Exiting SM\n" ); /* Format Waived */ - OSM_LOG_EXIT( p_sm->p_log ); + OSM_LOG_ENTER(p_sm->p_log, osm_sm_destroy); + osm_trap_rcv_destroy(&p_sm->trap_rcv); + osm_sminfo_rcv_destroy(&p_sm->sm_info_rcv); + osm_req_destroy(&p_sm->req); + osm_resp_destroy(&p_sm->resp); + osm_ni_rcv_destroy(&p_sm->ni_rcv); + osm_pi_rcv_destroy(&p_sm->pi_rcv); + osm_si_rcv_destroy(&p_sm->si_rcv); + osm_nd_rcv_destroy(&p_sm->nd_rcv); + osm_lid_mgr_destroy(&p_sm->lid_mgr); + osm_ucast_mgr_destroy(&p_sm->ucast_mgr); + osm_link_mgr_destroy(&p_sm->link_mgr); + osm_drop_mgr_destroy(&p_sm->drop_mgr); + osm_lft_rcv_destroy(&p_sm->lft_rcv); + osm_mft_rcv_destroy(&p_sm->mft_rcv); + osm_slvl_rcv_destroy(&p_sm->slvl_rcv); + osm_vla_rcv_destroy(&p_sm->vla_rcv); + osm_pkey_rcv_destroy(&p_sm->pkey_rcv); + osm_state_mgr_destroy(&p_sm->state_mgr); + osm_sm_state_mgr_destroy(&p_sm->sm_state_mgr); + osm_mcast_mgr_destroy(&p_sm->mcast_mgr); + cl_event_destroy(&p_sm->signal); + cl_event_destroy(&p_sm->subnet_up_event); + + osm_log(p_sm->p_log, OSM_LOG_SYS, "Exiting SM\n"); /* Format Waived */ + OSM_LOG_EXIT(p_sm->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_init( - IN osm_sm_t * const p_sm, - IN osm_subn_t * const p_subn, - IN osm_db_t * const p_db, - IN osm_vendor_t * const p_vendor, - IN osm_mad_pool_t * const p_mad_pool, - IN osm_vl15_t * const p_vl15, - IN osm_log_t * const p_log, - IN osm_stats_t * const p_stats, - IN cl_dispatcher_t * const p_disp, - IN cl_plock_t * const p_lock ) +osm_sm_init(IN osm_sm_t * const p_sm, + IN osm_subn_t * const p_subn, + IN osm_db_t * const p_db, + IN osm_vendor_t * const p_vendor, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vl15_t * const p_vl15, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_sm_init ); - - p_sm->p_subn = p_subn; - p_sm->p_db = p_db; - p_sm->p_vendor = p_vendor; - p_sm->p_mad_pool = p_mad_pool; - p_sm->p_vl15 = p_vl15; - p_sm->p_log = p_log; - p_sm->p_disp = p_disp; - p_sm->p_lock = p_lock; - - status = cl_event_init( &p_sm->signal, FALSE ); - if( status != CL_SUCCESS ) - goto Exit; - - status = cl_event_init( &p_sm->subnet_up_event, FALSE ); - if( status != CL_SUCCESS ) - goto Exit; - - status = osm_sm_mad_ctrl_init( &p_sm->mad_ctrl, - p_sm->p_subn, - p_sm->p_mad_pool, - p_sm->p_vl15, - p_sm->p_vendor, - p_log, p_stats, p_lock, p_disp ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_req_init( &p_sm->req, - p_mad_pool, - p_vl15, p_subn, p_log, &p_sm->sm_trans_id ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_resp_init( &p_sm->resp, p_mad_pool, p_vl15, p_subn, p_log ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_ni_rcv_init( &p_sm->ni_rcv, - &p_sm->req, - p_subn, p_log, &p_sm->state_mgr, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_pi_rcv_init( &p_sm->pi_rcv, - &p_sm->req, - p_subn, p_log, &p_sm->state_mgr, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_si_rcv_init( &p_sm->si_rcv, - p_sm->p_subn, - p_sm->p_log, - &p_sm->req, &p_sm->state_mgr, p_sm->p_lock ); - - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_nd_rcv_init( &p_sm->nd_rcv, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_lid_mgr_init( &p_sm->lid_mgr, - &p_sm->req, - p_sm->p_subn, - p_sm->p_db, p_sm->p_log, p_sm->p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_ucast_mgr_init( &p_sm->ucast_mgr, - &p_sm->req, - p_sm->p_subn, - p_sm->p_log, p_sm->p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_link_mgr_init( &p_sm->link_mgr, - &p_sm->req, - p_sm->p_subn, p_sm->p_log, p_sm->p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_state_mgr_init( &p_sm->state_mgr, - p_sm->p_subn, - &p_sm->lid_mgr, - &p_sm->ucast_mgr, - &p_sm->mcast_mgr, - &p_sm->link_mgr, - &p_sm->drop_mgr, - &p_sm->req, - p_stats, - &p_sm->sm_state_mgr, - &p_sm->mad_ctrl, - p_sm->p_lock, - &p_sm->subnet_up_event, - p_sm->p_log ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_state_mgr_ctrl_init( &p_sm->state_mgr_ctrl, - &p_sm->state_mgr, - p_sm->p_log, p_sm->p_disp ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_drop_mgr_init( &p_sm->drop_mgr, - p_sm->p_subn, - p_sm->p_log, &p_sm->req, p_sm->p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_lft_rcv_init( &p_sm->lft_rcv, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_mft_rcv_init( &p_sm->mft_rcv, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sweep_fail_ctrl_init( &p_sm->sweep_fail_ctrl, - p_log, &p_sm->state_mgr, p_disp ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sminfo_rcv_init( &p_sm->sm_info_rcv, - p_subn, - p_stats, - &p_sm->resp, - p_log, - &p_sm->state_mgr, - &p_sm->sm_state_mgr, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_trap_rcv_init( &p_sm->trap_rcv, - p_subn, - p_stats, - &p_sm->resp, p_log, &p_sm->state_mgr, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_sm_state_mgr_init( &p_sm->sm_state_mgr, - &p_sm->state_mgr, - p_sm->p_subn, &p_sm->req, p_sm->p_log ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_mcast_mgr_init( &p_sm->mcast_mgr, - &p_sm->req, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_slvl_rcv_init( &p_sm->slvl_rcv, - &p_sm->req, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_vla_rcv_init( &p_sm->vla_rcv, - &p_sm->req, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - status = osm_pkey_rcv_init( &p_sm->pkey_rcv, - &p_sm->req, p_subn, p_log, p_lock ); - if( status != IB_SUCCESS ) - goto Exit; - - p_sm->ni_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_INFO, - osm_ni_rcv_process, &p_sm->ni_rcv); - if( p_sm->ni_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->pi_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PORT_INFO, - osm_pi_rcv_process, &p_sm->pi_rcv); - if( p_sm->pi_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->si_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SWITCH_INFO, - osm_si_rcv_process, &p_sm->si_rcv); - if( p_sm->si_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->nd_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_DESC, - osm_nd_rcv_process, &p_sm->nd_rcv); - if( p_sm->nd_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->lft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LFT, - osm_lft_rcv_process, &p_sm->lft_rcv); - if( p_sm->lft_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->mft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MFT, - osm_mft_rcv_process, &p_sm->mft_rcv); - if( p_sm->mft_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->sm_info_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SM_INFO, - osm_sminfo_rcv_process, - &p_sm->sm_info_rcv); - if( p_sm->sm_info_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->trap_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NOTICE, - osm_trap_rcv_process, &p_sm->trap_rcv); - if( p_sm->trap_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->slvl_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SLVL, - osm_slvl_rcv_process, &p_sm->slvl_rcv); - if( p_sm->slvl_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->vla_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_VL_ARB, - osm_vla_rcv_process, &p_sm->vla_rcv); - if( p_sm->vla_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - p_sm->pkey_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PKEY, - osm_pkey_rcv_process, &p_sm->pkey_rcv); - if( p_sm->pkey_disp_h == CL_DISP_INVALID_HANDLE ) - goto Exit; - - /* - * Now that the component objects are initialized, start - * the sweeper thread if the user wants sweeping. - */ - if( p_sm->p_subn->opt.sweep_interval ) - { - p_sm->thread_state = OSM_THREAD_STATE_INIT; - status = cl_thread_init( &p_sm->sweeper, __osm_sm_sweeper, p_sm, - "opensm sweeper" ); - if( status != IB_SUCCESS ) - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return ( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_sm_init); + + p_sm->p_subn = p_subn; + p_sm->p_db = p_db; + p_sm->p_vendor = p_vendor; + p_sm->p_mad_pool = p_mad_pool; + p_sm->p_vl15 = p_vl15; + p_sm->p_log = p_log; + p_sm->p_disp = p_disp; + p_sm->p_lock = p_lock; + + status = cl_event_init(&p_sm->signal, FALSE); + if (status != CL_SUCCESS) + goto Exit; + + status = cl_event_init(&p_sm->subnet_up_event, FALSE); + if (status != CL_SUCCESS) + goto Exit; + + status = osm_sm_mad_ctrl_init(&p_sm->mad_ctrl, + p_sm->p_subn, + p_sm->p_mad_pool, + p_sm->p_vl15, + p_sm->p_vendor, + p_log, p_stats, p_lock, p_disp); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_req_init(&p_sm->req, + p_mad_pool, + p_vl15, p_subn, p_log, &p_sm->sm_trans_id); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_resp_init(&p_sm->resp, p_mad_pool, p_vl15, p_subn, p_log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_ni_rcv_init(&p_sm->ni_rcv, + &p_sm->req, + p_subn, p_log, &p_sm->state_mgr, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_pi_rcv_init(&p_sm->pi_rcv, + &p_sm->req, + p_subn, p_log, &p_sm->state_mgr, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_si_rcv_init(&p_sm->si_rcv, + p_sm->p_subn, + p_sm->p_log, + &p_sm->req, &p_sm->state_mgr, p_sm->p_lock); + + if (status != IB_SUCCESS) + goto Exit; + + status = osm_nd_rcv_init(&p_sm->nd_rcv, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_lid_mgr_init(&p_sm->lid_mgr, + &p_sm->req, + p_sm->p_subn, + p_sm->p_db, p_sm->p_log, p_sm->p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_ucast_mgr_init(&p_sm->ucast_mgr, + &p_sm->req, + p_sm->p_subn, p_sm->p_log, p_sm->p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_link_mgr_init(&p_sm->link_mgr, + &p_sm->req, + p_sm->p_subn, p_sm->p_log, p_sm->p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_state_mgr_init(&p_sm->state_mgr, + p_sm->p_subn, + &p_sm->lid_mgr, + &p_sm->ucast_mgr, + &p_sm->mcast_mgr, + &p_sm->link_mgr, + &p_sm->drop_mgr, + &p_sm->req, + p_stats, + &p_sm->sm_state_mgr, + &p_sm->mad_ctrl, + p_sm->p_lock, + &p_sm->subnet_up_event, p_sm->p_log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_state_mgr_ctrl_init(&p_sm->state_mgr_ctrl, + &p_sm->state_mgr, + p_sm->p_log, p_sm->p_disp); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_drop_mgr_init(&p_sm->drop_mgr, + p_sm->p_subn, + p_sm->p_log, &p_sm->req, p_sm->p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_lft_rcv_init(&p_sm->lft_rcv, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_mft_rcv_init(&p_sm->mft_rcv, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sweep_fail_ctrl_init(&p_sm->sweep_fail_ctrl, + p_log, &p_sm->state_mgr, p_disp); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sminfo_rcv_init(&p_sm->sm_info_rcv, + p_subn, + p_stats, + &p_sm->resp, + p_log, + &p_sm->state_mgr, + &p_sm->sm_state_mgr, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_trap_rcv_init(&p_sm->trap_rcv, + p_subn, + p_stats, + &p_sm->resp, p_log, &p_sm->state_mgr, + p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_sm_state_mgr_init(&p_sm->sm_state_mgr, + &p_sm->state_mgr, + p_sm->p_subn, &p_sm->req, p_sm->p_log); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_mcast_mgr_init(&p_sm->mcast_mgr, + &p_sm->req, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_slvl_rcv_init(&p_sm->slvl_rcv, + &p_sm->req, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_vla_rcv_init(&p_sm->vla_rcv, + &p_sm->req, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + status = osm_pkey_rcv_init(&p_sm->pkey_rcv, + &p_sm->req, p_subn, p_log, p_lock); + if (status != IB_SUCCESS) + goto Exit; + + p_sm->ni_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_INFO, + osm_ni_rcv_process, &p_sm->ni_rcv); + if (p_sm->ni_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->pi_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PORT_INFO, + osm_pi_rcv_process, &p_sm->pi_rcv); + if (p_sm->pi_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->si_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SWITCH_INFO, + osm_si_rcv_process, &p_sm->si_rcv); + if (p_sm->si_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->nd_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NODE_DESC, + osm_nd_rcv_process, &p_sm->nd_rcv); + if (p_sm->nd_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->lft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_LFT, + osm_lft_rcv_process, + &p_sm->lft_rcv); + if (p_sm->lft_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->mft_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_MFT, + osm_mft_rcv_process, + &p_sm->mft_rcv); + if (p_sm->mft_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->sm_info_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SM_INFO, + osm_sminfo_rcv_process, + &p_sm->sm_info_rcv); + if (p_sm->sm_info_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->trap_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_NOTICE, + osm_trap_rcv_process, + &p_sm->trap_rcv); + if (p_sm->trap_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->slvl_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SLVL, + osm_slvl_rcv_process, + &p_sm->slvl_rcv); + if (p_sm->slvl_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->vla_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_VL_ARB, + osm_vla_rcv_process, + &p_sm->vla_rcv); + if (p_sm->vla_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + p_sm->pkey_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_PKEY, + osm_pkey_rcv_process, + &p_sm->pkey_rcv); + if (p_sm->pkey_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + + /* + * Now that the component objects are initialized, start + * the sweeper thread if the user wants sweeping. + */ + if (p_sm->p_subn->opt.sweep_interval) { + p_sm->thread_state = OSM_THREAD_STATE_INIT; + status = cl_thread_init(&p_sm->sweeper, __osm_sm_sweeper, p_sm, + "opensm sweeper"); + if (status != IB_SUCCESS) + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_sm_sweep( - IN osm_sm_t * const p_sm ) +void osm_sm_sweep(IN osm_sm_t * const p_sm) { - OSM_LOG_ENTER( p_sm->p_log, osm_sm_sweep ); - osm_state_mgr_process( &p_sm->state_mgr, OSM_SIGNAL_SWEEP ); - OSM_LOG_EXIT( p_sm->p_log ); + OSM_LOG_ENTER(p_sm->p_log, osm_sm_sweep); + osm_state_mgr_process(&p_sm->state_mgr, OSM_SIGNAL_SWEEP); + OSM_LOG_EXIT(p_sm->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_bind( - IN osm_sm_t * const p_sm, - IN const ib_net64_t port_guid ) +osm_sm_bind(IN osm_sm_t * const p_sm, IN const ib_net64_t port_guid) { - ib_api_status_t status; + ib_api_status_t status; - OSM_LOG_ENTER( p_sm->p_log, osm_sm_bind ); + OSM_LOG_ENTER(p_sm->p_log, osm_sm_bind); - status = osm_sm_mad_ctrl_bind( &p_sm->mad_ctrl, port_guid ); + status = osm_sm_mad_ctrl_bind(&p_sm->mad_ctrl, port_guid); - if( status != IB_SUCCESS ) - { - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_bind: ERR 2E10: " - "SM MAD Controller bind failed (%s)\n", - ib_get_err_str( status ) ); - goto Exit; - } + if (status != IB_SUCCESS) { + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_bind: ERR 2E10: " + "SM MAD Controller bind failed (%s)\n", + ib_get_err_str(status)); + goto Exit; + } - Exit: - OSM_LOG_EXIT( p_sm->p_log ); - return ( status ); + Exit: + OSM_LOG_EXIT(p_sm->p_log); + return (status); } /********************************************************************** **********************************************************************/ static ib_api_status_t -__osm_sm_mgrp_connect( - IN osm_sm_t * const p_sm, - IN osm_mgrp_t * const p_mgrp, - IN const ib_net64_t port_guid, - IN osm_mcast_req_type_t req_type ) +__osm_sm_mgrp_connect(IN osm_sm_t * const p_sm, + IN osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid, + IN osm_mcast_req_type_t req_type) { - ib_api_status_t status; - osm_mcast_mgr_ctxt_t *ctx2; - - OSM_LOG_ENTER( p_sm->p_log, __osm_sm_mgrp_connect ); - - /* - * 'Schedule' all the QP0 traffic for when the state manager - * isn't busy trying to do something else. - */ - ctx2 = - ( osm_mcast_mgr_ctxt_t * ) malloc( sizeof( osm_mcast_mgr_ctxt_t ) ); - memcpy( &ctx2->mlid, &p_mgrp->mlid, sizeof( p_mgrp->mlid ) ); - ctx2->req_type = req_type; - ctx2->port_guid = port_guid; - - status = osm_state_mgr_process_idle( &p_sm->state_mgr, - osm_mcast_mgr_process_mgrp_cb, - NULL, &p_sm->mcast_mgr, - ( void * )ctx2 ); - - OSM_LOG_EXIT( p_sm->p_log ); - return ( status ); + ib_api_status_t status; + osm_mcast_mgr_ctxt_t *ctx2; + + OSM_LOG_ENTER(p_sm->p_log, __osm_sm_mgrp_connect); + + /* + * 'Schedule' all the QP0 traffic for when the state manager + * isn't busy trying to do something else. + */ + ctx2 = (osm_mcast_mgr_ctxt_t *) malloc(sizeof(osm_mcast_mgr_ctxt_t)); + memcpy(&ctx2->mlid, &p_mgrp->mlid, sizeof(p_mgrp->mlid)); + ctx2->req_type = req_type; + ctx2->port_guid = port_guid; + + status = osm_state_mgr_process_idle(&p_sm->state_mgr, + osm_mcast_mgr_process_mgrp_cb, + NULL, &p_sm->mcast_mgr, + (void *)ctx2); + + OSM_LOG_EXIT(p_sm->p_log); + return (status); } /********************************************************************** **********************************************************************/ static void -__osm_sm_mgrp_disconnect( - IN osm_sm_t * const p_sm, - IN osm_mgrp_t * const p_mgrp, - IN const ib_net64_t port_guid ) +__osm_sm_mgrp_disconnect(IN osm_sm_t * const p_sm, + IN osm_mgrp_t * const p_mgrp, + IN const ib_net64_t port_guid) { - ib_api_status_t status; - osm_mcast_mgr_ctxt_t *ctx2; - - OSM_LOG_ENTER( p_sm->p_log, __osm_sm_mgrp_disconnect ); - - /* - * 'Schedule' all the QP0 traffic for when the state manager - * isn't busy trying to do something else. - */ - ctx2 = - ( osm_mcast_mgr_ctxt_t * ) malloc( sizeof( osm_mcast_mgr_ctxt_t ) ); - memcpy( &ctx2->mlid, &p_mgrp->mlid, sizeof( p_mgrp->mlid ) ); - ctx2->req_type = OSM_MCAST_REQ_TYPE_LEAVE; - ctx2->port_guid = port_guid; - - status = osm_state_mgr_process_idle( &p_sm->state_mgr, - osm_mcast_mgr_process_mgrp_cb, - NULL, &p_sm->mcast_mgr, ctx2 ); - if( status != IB_SUCCESS ) - { - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "__osm_sm_mgrp_disconnect: ERR 2E11: " - "Failure processing multicast group (%s)\n", - ib_get_err_str( status ) ); - } - - OSM_LOG_EXIT( p_sm->p_log ); + ib_api_status_t status; + osm_mcast_mgr_ctxt_t *ctx2; + + OSM_LOG_ENTER(p_sm->p_log, __osm_sm_mgrp_disconnect); + + /* + * 'Schedule' all the QP0 traffic for when the state manager + * isn't busy trying to do something else. + */ + ctx2 = (osm_mcast_mgr_ctxt_t *) malloc(sizeof(osm_mcast_mgr_ctxt_t)); + memcpy(&ctx2->mlid, &p_mgrp->mlid, sizeof(p_mgrp->mlid)); + ctx2->req_type = OSM_MCAST_REQ_TYPE_LEAVE; + ctx2->port_guid = port_guid; + + status = osm_state_mgr_process_idle(&p_sm->state_mgr, + osm_mcast_mgr_process_mgrp_cb, + NULL, &p_sm->mcast_mgr, ctx2); + if (status != IB_SUCCESS) { + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "__osm_sm_mgrp_disconnect: ERR 2E11: " + "Failure processing multicast group (%s)\n", + ib_get_err_str(status)); + } + + OSM_LOG_EXIT(p_sm->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_mcgrp_join( - IN osm_sm_t * const p_sm, - IN const ib_net16_t mlid, - IN const ib_net64_t port_guid, - IN osm_mcast_req_type_t req_type ) +osm_sm_mcgrp_join(IN osm_sm_t * const p_sm, + IN const ib_net16_t mlid, + IN const ib_net64_t port_guid, + IN osm_mcast_req_type_t req_type) { - osm_mgrp_t *p_mgrp; - osm_port_t *p_port; - cl_qmap_t *p_tbl; - ib_api_status_t status = IB_SUCCESS; - osm_mcm_info_t *p_mcm; - - OSM_LOG_ENTER( p_sm->p_log, osm_sm_mcgrp_join ); - - osm_log( p_sm->p_log, OSM_LOG_VERBOSE, - "osm_sm_mcgrp_join: " - "Port 0x%016" PRIx64 " joining MLID 0x%X\n", - cl_ntoh64( port_guid ), cl_ntoh16( mlid ) ); - - /* - * Acquire the port object for the port joining this group. - */ - CL_PLOCK_EXCL_ACQUIRE( p_sm->p_lock ); - p_port = osm_get_port_by_guid( p_sm->p_subn, port_guid ); - if( !p_port ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_join: ERR 2E05: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_INVALID_PARAMETER; - goto Exit; - } - - /* - * If this multicast group does not already exist, create it. - */ - p_tbl = &p_sm->p_subn->mgrp_mlid_tbl; - p_mgrp = ( osm_mgrp_t * ) cl_qmap_get( p_tbl, mlid ); - if( p_mgrp == ( osm_mgrp_t * ) cl_qmap_end( p_tbl ) ) - { - osm_log( p_sm->p_log, OSM_LOG_VERBOSE, - "osm_sm_mcgrp_join: " - "Creating group, MLID 0x%X\n", cl_ntoh16( mlid ) ); - - p_mgrp = osm_mgrp_new( mlid ); - if( p_mgrp == NULL ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_join: ERR 2E06: " - "Unable to allocate multicast group object\n" ); - status = IB_INSUFFICIENT_MEMORY; - goto Exit; - } - - cl_qmap_insert( p_tbl, mlid, &p_mgrp->map_item ); - } - else - { - /* - * The group already exists. If the port is not a - * member of the group, then fail immediately. - * This can happen since the spinlock is released briefly - * before the SA calls this function. - */ - if( !osm_mgrp_is_guid( p_mgrp, port_guid ) ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_join: ERR 2E12: " - "Port 0x%016" PRIx64 " not in mcast group 0x%X\n", - cl_ntoh64( port_guid ), cl_ntoh16( mlid ) ); - status = IB_NOT_FOUND; - goto Exit; - } - } - - /* - * Check if the object (according to mlid) already exists on this port. - * If it does - then no need to update it again, and no need to - * create the mc tree again. Just goto Exit. - */ - p_mcm = ( osm_mcm_info_t * ) cl_qlist_head( &p_port->mcm_list ); - while( p_mcm != ( osm_mcm_info_t * ) cl_qlist_end( &p_port->mcm_list ) ) - { - if( p_mcm->mlid == mlid ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_DEBUG, - "osm_sm_mcgrp_join: " - "Found mlid object for Port:" - "0x%016" PRIx64 " lid:0x%X\n", - cl_ntoh64( port_guid ), cl_ntoh16( mlid ) ); - goto Exit; - } - p_mcm = ( osm_mcm_info_t * ) cl_qlist_next( &p_mcm->list_item ); - } - - status = osm_port_add_mgrp( p_port, mlid ); - if( status != IB_SUCCESS ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_join: ERR 2E03: " - "Unable to associate port 0x%" PRIx64 " to mlid 0x%X\n", - cl_ntoh64( osm_port_get_guid( p_port ) ), - cl_ntoh16( osm_mgrp_get_mlid( p_mgrp ) ) ); - goto Exit; - } - - CL_PLOCK_RELEASE( p_sm->p_lock ); - status = __osm_sm_mgrp_connect( p_sm, p_mgrp, port_guid, req_type ); - - Exit: - OSM_LOG_EXIT( p_sm->p_log ); - return ( status ); + osm_mgrp_t *p_mgrp; + osm_port_t *p_port; + cl_qmap_t *p_tbl; + ib_api_status_t status = IB_SUCCESS; + osm_mcm_info_t *p_mcm; + + OSM_LOG_ENTER(p_sm->p_log, osm_sm_mcgrp_join); + + osm_log(p_sm->p_log, OSM_LOG_VERBOSE, + "osm_sm_mcgrp_join: " + "Port 0x%016" PRIx64 " joining MLID 0x%X\n", + cl_ntoh64(port_guid), cl_ntoh16(mlid)); + + /* + * Acquire the port object for the port joining this group. + */ + CL_PLOCK_EXCL_ACQUIRE(p_sm->p_lock); + p_port = osm_get_port_by_guid(p_sm->p_subn, port_guid); + if (!p_port) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_join: ERR 2E05: " + "No port object for port 0x%016" PRIx64 "\n", + cl_ntoh64(port_guid)); + status = IB_INVALID_PARAMETER; + goto Exit; + } + + /* + * If this multicast group does not already exist, create it. + */ + p_tbl = &p_sm->p_subn->mgrp_mlid_tbl; + p_mgrp = (osm_mgrp_t *) cl_qmap_get(p_tbl, mlid); + if (p_mgrp == (osm_mgrp_t *) cl_qmap_end(p_tbl)) { + osm_log(p_sm->p_log, OSM_LOG_VERBOSE, + "osm_sm_mcgrp_join: " + "Creating group, MLID 0x%X\n", cl_ntoh16(mlid)); + + p_mgrp = osm_mgrp_new(mlid); + if (p_mgrp == NULL) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_join: ERR 2E06: " + "Unable to allocate multicast group object\n"); + status = IB_INSUFFICIENT_MEMORY; + goto Exit; + } + + cl_qmap_insert(p_tbl, mlid, &p_mgrp->map_item); + } else { + /* + * The group already exists. If the port is not a + * member of the group, then fail immediately. + * This can happen since the spinlock is released briefly + * before the SA calls this function. + */ + if (!osm_mgrp_is_guid(p_mgrp, port_guid)) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_join: ERR 2E12: " + "Port 0x%016" PRIx64 + " not in mcast group 0x%X\n", + cl_ntoh64(port_guid), cl_ntoh16(mlid)); + status = IB_NOT_FOUND; + goto Exit; + } + } + + /* + * Check if the object (according to mlid) already exists on this port. + * If it does - then no need to update it again, and no need to + * create the mc tree again. Just goto Exit. + */ + p_mcm = (osm_mcm_info_t *) cl_qlist_head(&p_port->mcm_list); + while (p_mcm != (osm_mcm_info_t *) cl_qlist_end(&p_port->mcm_list)) { + if (p_mcm->mlid == mlid) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_DEBUG, + "osm_sm_mcgrp_join: " + "Found mlid object for Port:" + "0x%016" PRIx64 " lid:0x%X\n", + cl_ntoh64(port_guid), cl_ntoh16(mlid)); + goto Exit; + } + p_mcm = (osm_mcm_info_t *) cl_qlist_next(&p_mcm->list_item); + } + + status = osm_port_add_mgrp(p_port, mlid); + if (status != IB_SUCCESS) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_join: ERR 2E03: " + "Unable to associate port 0x%" PRIx64 " to mlid 0x%X\n", + cl_ntoh64(osm_port_get_guid(p_port)), + cl_ntoh16(osm_mgrp_get_mlid(p_mgrp))); + goto Exit; + } + + CL_PLOCK_RELEASE(p_sm->p_lock); + status = __osm_sm_mgrp_connect(p_sm, p_mgrp, port_guid, req_type); + + Exit: + OSM_LOG_EXIT(p_sm->p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_mcgrp_leave( - IN osm_sm_t * const p_sm, - IN const ib_net16_t mlid, - IN const ib_net64_t port_guid ) +osm_sm_mcgrp_leave(IN osm_sm_t * const p_sm, + IN const ib_net16_t mlid, IN const ib_net64_t port_guid) { - osm_mgrp_t *p_mgrp; - osm_port_t *p_port; - cl_qmap_t *p_tbl; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_sm->p_log, osm_sm_mcgrp_leave ); - - osm_log( p_sm->p_log, OSM_LOG_VERBOSE, - "osm_sm_mcgrp_leave: " - "Port 0x%" PRIx64 " leaving MLID 0x%X\n", - cl_ntoh64( port_guid ), cl_ntoh16( mlid ) ); - - /* - * Acquire the port object for the port leaving this group. - */ - /* note: p_sm->p_lock is locked by caller, but will be released later - this function */ - p_port = osm_get_port_by_guid( p_sm->p_subn, port_guid ); - if( !p_port ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_leave: ERR 2E04: " - "No port object for port 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - status = IB_INVALID_PARAMETER; - goto Exit; - } - - /* - * Get the multicast group object for this group. - */ - p_tbl = &p_sm->p_subn->mgrp_mlid_tbl; - p_mgrp = ( osm_mgrp_t * ) cl_qmap_get( p_tbl, mlid ); - if( p_mgrp == ( osm_mgrp_t * ) cl_qmap_end( p_tbl ) ) - { - CL_PLOCK_RELEASE( p_sm->p_lock ); - osm_log( p_sm->p_log, OSM_LOG_ERROR, - "osm_sm_mcgrp_leave: ERR 2E08: " - "No multicast group for MLID 0x%X\n", cl_ntoh16( mlid ) ); - status = IB_INVALID_PARAMETER; - goto Exit; - } - - /* - * Walk the list of ports in the group, and remove the appropriate one. - */ - osm_mgrp_remove_port( p_sm->p_subn, p_sm->p_log, p_mgrp, port_guid ); - - osm_port_remove_mgrp( p_port, mlid ); - - CL_PLOCK_RELEASE( p_sm->p_lock ); - - __osm_sm_mgrp_disconnect( p_sm, p_mgrp, port_guid ); - - Exit: - OSM_LOG_EXIT( p_sm->p_log ); - return ( status ); + osm_mgrp_t *p_mgrp; + osm_port_t *p_port; + cl_qmap_t *p_tbl; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_sm->p_log, osm_sm_mcgrp_leave); + + osm_log(p_sm->p_log, OSM_LOG_VERBOSE, + "osm_sm_mcgrp_leave: " + "Port 0x%" PRIx64 " leaving MLID 0x%X\n", + cl_ntoh64(port_guid), cl_ntoh16(mlid)); + + /* + * Acquire the port object for the port leaving this group. + */ + /* note: p_sm->p_lock is locked by caller, but will be released later + this function */ + p_port = osm_get_port_by_guid(p_sm->p_subn, port_guid); + if (!p_port) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_leave: ERR 2E04: " + "No port object for port 0x%" PRIx64 "\n", + cl_ntoh64(port_guid)); + status = IB_INVALID_PARAMETER; + goto Exit; + } + + /* + * Get the multicast group object for this group. + */ + p_tbl = &p_sm->p_subn->mgrp_mlid_tbl; + p_mgrp = (osm_mgrp_t *) cl_qmap_get(p_tbl, mlid); + if (p_mgrp == (osm_mgrp_t *) cl_qmap_end(p_tbl)) { + CL_PLOCK_RELEASE(p_sm->p_lock); + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "osm_sm_mcgrp_leave: ERR 2E08: " + "No multicast group for MLID 0x%X\n", cl_ntoh16(mlid)); + status = IB_INVALID_PARAMETER; + goto Exit; + } + + /* + * Walk the list of ports in the group, and remove the appropriate one. + */ + osm_mgrp_remove_port(p_sm->p_subn, p_sm->p_log, p_mgrp, port_guid); + + osm_port_remove_mgrp(p_port, mlid); + + CL_PLOCK_RELEASE(p_sm->p_lock); + + __osm_sm_mgrp_disconnect(p_sm, p_mgrp, port_guid); + + Exit: + OSM_LOG_EXIT(p_sm->p_log); + return (status); } diff --git a/opensm/opensm/osm_sm_mad_ctrl.c b/opensm/opensm/osm_sm_mad_ctrl.c index fbf96c1..758262b 100644 --- a/opensm/opensm/osm_sm_mad_ctrl.c +++ b/opensm/opensm/osm_sm_mad_ctrl.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -70,72 +70,68 @@ * SYNOPSIS */ static void -__osm_sm_mad_ctrl_retire_trans_mad( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_madw_t* const p_madw ) +__osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * const p_madw) { - uint32_t outstanding; - cl_status_t status; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_retire_trans_mad ); - - CL_ASSERT( p_madw ); - /* - Return the MAD & wrapper to the pool. - */ - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_retire_trans_mad: " - "Retiring MAD with TID 0x%" PRIx64 "\n", - cl_ntoh64( osm_madw_get_smp_ptr( p_madw )->trans_id ) ); - } - - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - - outstanding = cl_atomic_dec( &p_ctrl->p_stats->qp0_mads_outstanding ); - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_retire_trans_mad: " - "%u QP0 MADs outstanding\n", - p_ctrl->p_stats->qp0_mads_outstanding ); - } - - if( outstanding == 0 ) - { - /* - The wire is clean. - Signal the state manager. - */ - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_retire_trans_mad: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( OSM_MSG_NO_SMPS_OUTSTANDING ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - (void *)OSM_SIGNAL_NO_PENDING_TRANSACTIONS, - NULL, - NULL ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_retire_trans_mad: ERR 3101: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - goto Exit; - } - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + uint32_t outstanding; + cl_status_t status; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_retire_trans_mad); + + CL_ASSERT(p_madw); + /* + Return the MAD & wrapper to the pool. + */ + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_retire_trans_mad: " + "Retiring MAD with TID 0x%" PRIx64 "\n", + cl_ntoh64(osm_madw_get_smp_ptr(p_madw)->trans_id)); + } + + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + + outstanding = cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding); + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_retire_trans_mad: " + "%u QP0 MADs outstanding\n", + p_ctrl->p_stats->qp0_mads_outstanding); + } + + if (outstanding == 0) { + /* + The wire is clean. + Signal the state manager. + */ + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_retire_trans_mad: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str + (OSM_MSG_NO_SMPS_OUTSTANDING)); + } + + status = cl_disp_post(p_ctrl->h_disp, + OSM_MSG_NO_SMPS_OUTSTANDING, + (void *) + OSM_SIGNAL_NO_PENDING_TRANSACTIONS, NULL, + NULL); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_retire_trans_mad: ERR 3101: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; + } + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /************/ /****f* opensm: SM/__osm_sm_mad_ctrl_disp_done_callback @@ -149,42 +145,38 @@ __osm_sm_mad_ctrl_retire_trans_mad( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_disp_done_callback( - IN void* context, - IN void* p_data ) +__osm_sm_mad_ctrl_disp_done_callback(IN void *context, IN void *p_data) { - osm_sm_mad_ctrl_t* const p_ctrl = (osm_sm_mad_ctrl_t*)context; - osm_madw_t* const p_madw = (osm_madw_t*)p_data; - ib_smp_t *p_smp; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_disp_done_callback ); - - /* - If the MAD that just finished processing was a response, - then retire the transaction, since we must have generated - the request. - - Otherwise, retire the transaction if a response was expected, - as in the case of a send failure. If a response was not expected, - just put the MAD back in the pool, because the MAD was a query - from some outside agent, e.g. Get(SMInfo) from another SM. - */ - p_smp = osm_madw_get_smp_ptr( p_madw ); - if( ib_smp_is_response( p_smp ) ) - { - CL_ASSERT( p_madw->resp_expected == FALSE ); - __osm_sm_mad_ctrl_retire_trans_mad( p_ctrl, p_madw ); - } - else - { - if( p_madw->resp_expected == TRUE ) - __osm_sm_mad_ctrl_retire_trans_mad( p_ctrl, p_madw ); - else - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - OSM_LOG_EXIT( p_ctrl->p_log ); + osm_sm_mad_ctrl_t *const p_ctrl = (osm_sm_mad_ctrl_t *) context; + osm_madw_t *const p_madw = (osm_madw_t *) p_data; + ib_smp_t *p_smp; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_disp_done_callback); + + /* + If the MAD that just finished processing was a response, + then retire the transaction, since we must have generated + the request. + + Otherwise, retire the transaction if a response was expected, + as in the case of a send failure. If a response was not expected, + just put the MAD back in the pool, because the MAD was a query + from some outside agent, e.g. Get(SMInfo) from another SM. + */ + p_smp = osm_madw_get_smp_ptr(p_madw); + if (ib_smp_is_response(p_smp)) { + CL_ASSERT(p_madw->resp_expected == FALSE); + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + } else { + if (p_madw->resp_expected == TRUE) + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + else + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + OSM_LOG_EXIT(p_ctrl->p_log); } + /************/ /****f* opensm: SM/__osm_sm_mad_ctrl_update_wire_stats @@ -197,30 +189,28 @@ __osm_sm_mad_ctrl_disp_done_callback( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_update_wire_stats( - IN osm_sm_mad_ctrl_t* const p_ctrl ) +__osm_sm_mad_ctrl_update_wire_stats(IN osm_sm_mad_ctrl_t * const p_ctrl) { - uint32_t mads_on_wire; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_update_wire_stats ); - - mads_on_wire = cl_atomic_dec( - &p_ctrl->p_stats->qp0_mads_outstanding_on_wire ); - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_update_wire_stats: " - "%u SMPs on the wire, %u outstanding\n", mads_on_wire, - p_ctrl->p_stats->qp0_mads_outstanding ); - } - - /* - We can signal the VL15 controller to send another MAD - if any are waiting for transmission. - */ - osm_vl15_poll( p_ctrl->p_vl15 ); - OSM_LOG_EXIT( p_ctrl->p_log ); + uint32_t mads_on_wire; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_update_wire_stats); + + mads_on_wire = + cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding_on_wire); + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_update_wire_stats: " + "%u SMPs on the wire, %u outstanding\n", mads_on_wire, + p_ctrl->p_stats->qp0_mads_outstanding); + } + + /* + We can signal the VL15 controller to send another MAD + if any are waiting for transmission. + */ + osm_vl15_poll(p_ctrl->p_vl15); + OSM_LOG_EXIT(p_ctrl->p_log); } /****f* opensm: SM/__osm_sm_mad_ctrl_process_get_resp @@ -234,132 +224,125 @@ __osm_sm_mad_ctrl_update_wire_stats( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_process_get_resp( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_madw_t *p_madw, - IN void* transaction_context ) +__osm_sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * p_madw, + IN void *transaction_context) { - ib_smp_t* p_smp; - cl_status_t status; - osm_madw_t* p_old_madw; - cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_process_get_resp ); - - CL_ASSERT( p_madw ); - CL_ASSERT( transaction_context ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - if( p_smp->mgmt_class == IB_MCLASS_SUBN_DIR ) - { - if( !ib_smp_is_d( p_smp ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get_resp: ERR 3102: " - "'D' bit not set in returned SMP\n" ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - } - } - - p_old_madw = (osm_madw_t*)transaction_context; - - __osm_sm_mad_ctrl_update_wire_stats( p_ctrl ); - - /* - Copy the MAD Wrapper context from the requesting MAD - to the new MAD. This mechanism allows the recipient - controller to recover its own context regarding this - MAD transaction. Once we've copied the context, we - can return the original MAD to the pool. - */ - osm_madw_copy_context( p_madw, p_old_madw ); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_old_madw ); - - /* - Note that attr_id (like the rest of the MAD) is in - network byte order. - */ - switch( p_smp->attr_id ) - { - case IB_MAD_ATTR_NODE_DESC: - msg_id = OSM_MSG_MAD_NODE_DESC; - break; - case IB_MAD_ATTR_NODE_INFO: - msg_id = OSM_MSG_MAD_NODE_INFO; - break; - case IB_MAD_ATTR_SWITCH_INFO: - msg_id = OSM_MSG_MAD_SWITCH_INFO; - break; - case IB_MAD_ATTR_PORT_INFO: - msg_id = OSM_MSG_MAD_PORT_INFO; - break; - case IB_MAD_ATTR_LIN_FWD_TBL: - msg_id = OSM_MSG_MAD_LFT; - break; - case IB_MAD_ATTR_MCAST_FWD_TBL: - msg_id = OSM_MSG_MAD_MFT; - break; - case IB_MAD_ATTR_SM_INFO: - msg_id = OSM_MSG_MAD_SM_INFO; - break; - case IB_MAD_ATTR_SLVL_TABLE: - msg_id = OSM_MSG_MAD_SLVL; - break; - case IB_MAD_ATTR_VL_ARBITRATION: - msg_id = OSM_MSG_MAD_VL_ARB; - break; - case IB_MAD_ATTR_P_KEY_TABLE: - msg_id = OSM_MSG_MAD_PKEY; - break; - - case IB_MAD_ATTR_GUID_INFO: - case IB_MAD_ATTR_CLASS_PORT_INFO: - case IB_MAD_ATTR_NOTICE: - case IB_MAD_ATTR_INFORM_INFO: - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get_resp: ERR 3103: " - "Unsupported attribute = 0x%X\n", - cl_ntoh16( p_smp->attr_id ) ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - goto Exit; - } - - if( msg_id != CL_DISP_MSGID_NONE ) - { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_get_resp: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( msg_id ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get_resp: ERR 3104: " - "Dispatcher post message failed (%s) for attribute = 0x%X\n", - CL_STATUS_MSG( status ), - cl_ntoh16( p_smp->attr_id ) ); - goto Exit; - } - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + ib_smp_t *p_smp; + cl_status_t status; + osm_madw_t *p_old_madw; + cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_process_get_resp); + + CL_ASSERT(p_madw); + CL_ASSERT(transaction_context); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { + if (!ib_smp_is_d(p_smp)) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get_resp: ERR 3102: " + "'D' bit not set in returned SMP\n"); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + } + } + + p_old_madw = (osm_madw_t *) transaction_context; + + __osm_sm_mad_ctrl_update_wire_stats(p_ctrl); + + /* + Copy the MAD Wrapper context from the requesting MAD + to the new MAD. This mechanism allows the recipient + controller to recover its own context regarding this + MAD transaction. Once we've copied the context, we + can return the original MAD to the pool. + */ + osm_madw_copy_context(p_madw, p_old_madw); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_old_madw); + + /* + Note that attr_id (like the rest of the MAD) is in + network byte order. + */ + switch (p_smp->attr_id) { + case IB_MAD_ATTR_NODE_DESC: + msg_id = OSM_MSG_MAD_NODE_DESC; + break; + case IB_MAD_ATTR_NODE_INFO: + msg_id = OSM_MSG_MAD_NODE_INFO; + break; + case IB_MAD_ATTR_SWITCH_INFO: + msg_id = OSM_MSG_MAD_SWITCH_INFO; + break; + case IB_MAD_ATTR_PORT_INFO: + msg_id = OSM_MSG_MAD_PORT_INFO; + break; + case IB_MAD_ATTR_LIN_FWD_TBL: + msg_id = OSM_MSG_MAD_LFT; + break; + case IB_MAD_ATTR_MCAST_FWD_TBL: + msg_id = OSM_MSG_MAD_MFT; + break; + case IB_MAD_ATTR_SM_INFO: + msg_id = OSM_MSG_MAD_SM_INFO; + break; + case IB_MAD_ATTR_SLVL_TABLE: + msg_id = OSM_MSG_MAD_SLVL; + break; + case IB_MAD_ATTR_VL_ARBITRATION: + msg_id = OSM_MSG_MAD_VL_ARB; + break; + case IB_MAD_ATTR_P_KEY_TABLE: + msg_id = OSM_MSG_MAD_PKEY; + break; + + case IB_MAD_ATTR_GUID_INFO: + case IB_MAD_ATTR_CLASS_PORT_INFO: + case IB_MAD_ATTR_NOTICE: + case IB_MAD_ATTR_INFORM_INFO: + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get_resp: ERR 3103: " + "Unsupported attribute = 0x%X\n", + cl_ntoh16(p_smp->attr_id)); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + goto Exit; + } + + if (msg_id != CL_DISP_MSGID_NONE) { + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_get_resp: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + } + + status = cl_disp_post(p_ctrl->h_disp, + msg_id, + p_madw, + __osm_sm_mad_ctrl_disp_done_callback, + p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get_resp: ERR 3104: " + "Dispatcher post message failed (%s) for attribute = 0x%X\n", + CL_STATUS_MSG(status), + cl_ntoh16(p_smp->attr_id)); + goto Exit; + } + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } /****f* opensm: SM/__osm_sm_mad_ctrl_process_get @@ -372,78 +355,72 @@ __osm_sm_mad_ctrl_process_get_resp( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_process_get( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_madw_t *p_madw ) +__osm_sm_mad_ctrl_process_get(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * p_madw) { - ib_smp_t* p_smp; - cl_status_t status; - cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_process_get ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - /* - Note that attr_id (like the rest of the MAD) is in - network byte order. - */ - switch( p_smp->attr_id ) - { - case IB_MAD_ATTR_SM_INFO: - msg_id = OSM_MSG_MAD_SM_INFO; - break; - - default: - osm_log( p_ctrl->p_log, OSM_LOG_VERBOSE, - "__osm_sm_mad_ctrl_process_get: " - "Ignoring SubnGet MAD - unsupported attribute = 0x%X\n", - cl_ntoh16( p_smp->attr_id ) ); - break; - } - - if( msg_id != CL_DISP_MSGID_NONE ) - { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_get: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( msg_id ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get: ERR 3106: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - goto Exit; - } - } - else - { - /* - There is an unknown MAD attribute type for which there is - no recipient. Simply retire the MAD here. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + ib_smp_t *p_smp; + cl_status_t status; + cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_process_get); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + /* + Note that attr_id (like the rest of the MAD) is in + network byte order. + */ + switch (p_smp->attr_id) { + case IB_MAD_ATTR_SM_INFO: + msg_id = OSM_MSG_MAD_SM_INFO; + break; + + default: + osm_log(p_ctrl->p_log, OSM_LOG_VERBOSE, + "__osm_sm_mad_ctrl_process_get: " + "Ignoring SubnGet MAD - unsupported attribute = 0x%X\n", + cl_ntoh16(p_smp->attr_id)); + break; + } + + if (msg_id != CL_DISP_MSGID_NONE) { + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_get: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + } + + status = cl_disp_post(p_ctrl->h_disp, + msg_id, + p_madw, + __osm_sm_mad_ctrl_disp_done_callback, + p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get: ERR 3106: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; + } + } else { + /* + There is an unknown MAD attribute type for which there is + no recipient. Simply retire the MAD here. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -464,79 +441,73 @@ __osm_sm_mad_ctrl_process_get( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_process_set( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_madw_t *p_madw ) +__osm_sm_mad_ctrl_process_set(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * p_madw) { - ib_smp_t* p_smp; - cl_status_t status; - cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_process_set ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - /* - Note that attr_id (like the rest of the MAD) is in - network byte order. - */ - switch( p_smp->attr_id ) - { - case IB_MAD_ATTR_SM_INFO: - msg_id = OSM_MSG_MAD_SM_INFO; - break; - - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_set: ERR 3107: " - "Unsupported attribute = 0x%X\n", - cl_ntoh16( p_smp->attr_id ) ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - break; - } - - if( msg_id != CL_DISP_MSGID_NONE ) - { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_set: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( msg_id ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_set: ERR 3108: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - goto Exit; - } - } - else - { - /* - There is an unknown MAD attribute type for which there is - no recipient. Simply retire the MAD here. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + ib_smp_t *p_smp; + cl_status_t status; + cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_process_set); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + /* + Note that attr_id (like the rest of the MAD) is in + network byte order. + */ + switch (p_smp->attr_id) { + case IB_MAD_ATTR_SM_INFO: + msg_id = OSM_MSG_MAD_SM_INFO; + break; + + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_set: ERR 3107: " + "Unsupported attribute = 0x%X\n", + cl_ntoh16(p_smp->attr_id)); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + break; + } + + if (msg_id != CL_DISP_MSGID_NONE) { + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_set: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + } + + status = cl_disp_post(p_ctrl->h_disp, + msg_id, + p_madw, + __osm_sm_mad_ctrl_disp_done_callback, + p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_set: ERR 3108: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; + } + } else { + /* + There is an unknown MAD attribute type for which there is + no recipient. Simply retire the MAD here. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -557,90 +528,83 @@ __osm_sm_mad_ctrl_process_set( * SYNOPSIS */ static void -__osm_sm_mad_ctrl_process_trap( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_madw_t *p_madw ) +__osm_sm_mad_ctrl_process_trap(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_madw_t * p_madw) { - ib_smp_t* p_smp; - cl_status_t status; - cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_process_trap ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - /* Make sure OpenSM is master. If not - then we should not process the trap */ - if (p_ctrl->p_subn->sm_state != IB_SMINFO_STATE_MASTER) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_trap: " - "Received trap but OpenSM is not in MASTER state. " - "Dropping mad\n"); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - - /* - Note that attr_id (like the rest of the MAD) is in - network byte order. - */ - switch( p_smp->attr_id ) - { - case IB_MAD_ATTR_NOTICE: - msg_id = OSM_MSG_MAD_NOTICE; - break; - - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_trap: ERR 3109: " - "Unsupported attribute = 0x%X\n", - cl_ntoh16( p_smp->attr_id ) ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - break; - } - - if( msg_id != CL_DISP_MSGID_NONE ) - { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_trap: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( msg_id ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl ); - - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_trap: ERR 3110: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - goto Exit; - } - } - else - { - /* - There is an unknown MAD attribute type for which there is - no recipient. Simply retire the MAD here. - */ - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + ib_smp_t *p_smp; + cl_status_t status; + cl_disp_msgid_t msg_id = CL_DISP_MSGID_NONE; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_process_trap); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + /* Make sure OpenSM is master. If not - then we should not process the trap */ + if (p_ctrl->p_subn->sm_state != IB_SMINFO_STATE_MASTER) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_trap: " + "Received trap but OpenSM is not in MASTER state. " + "Dropping mad\n"); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + /* + Note that attr_id (like the rest of the MAD) is in + network byte order. + */ + switch (p_smp->attr_id) { + case IB_MAD_ATTR_NOTICE: + msg_id = OSM_MSG_MAD_NOTICE; + break; + + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_trap: ERR 3109: " + "Unsupported attribute = 0x%X\n", + cl_ntoh16(p_smp->attr_id)); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + break; + } + + if (msg_id != CL_DISP_MSGID_NONE) { + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_trap: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + } + + status = cl_disp_post(p_ctrl->h_disp, + msg_id, + p_madw, + __osm_sm_mad_ctrl_disp_done_callback, + p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_trap: ERR 3110: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; + } + } else { + /* + There is an unknown MAD attribute type for which there is + no recipient. Simply retire the MAD here. + */ + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -661,119 +625,110 @@ __osm_sm_mad_ctrl_process_trap( * SYNOPSIS */ void -__osm_sm_mad_ctrl_rcv_callback( - IN osm_madw_t *p_madw, - IN void *bind_context, - IN osm_madw_t *p_req_madw ) +__osm_sm_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, + IN void *bind_context, + IN osm_madw_t * p_req_madw) { - osm_sm_mad_ctrl_t* p_ctrl = (osm_sm_mad_ctrl_t*)bind_context; - ib_smp_t* p_smp; - ib_net16_t status; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_rcv_callback ); - - CL_ASSERT( p_madw ); - - /* - A MAD was received from the wire, possibly in response to a request. - */ - cl_atomic_inc( &p_ctrl->p_stats->qp0_mads_rcvd ); - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_rcv_callback: " - "%u QP0 MADs received\n", - p_ctrl->p_stats->qp0_mads_rcvd ); - } - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - /* if we are closing down simply do nothing */ - if (osm_exit_flag) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_rcv_callback: " - "Ignoring received mad - since we are exiting\n"); - - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_DEBUG ); - - /* retire the mad or put it back */ - if( ib_smp_is_response( p_smp ) || - (p_smp->method == IB_MAD_METHOD_TRAP_REPRESS)) - { - CL_ASSERT( p_madw->resp_expected == FALSE ); - __osm_sm_mad_ctrl_retire_trans_mad( p_ctrl, p_madw ); - } - else - { - if( p_madw->resp_expected == TRUE ) - __osm_sm_mad_ctrl_retire_trans_mad( p_ctrl, p_madw ); - else - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - } - - goto Exit; - } - - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_FRAMES ) ) - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_FRAMES ); - - if( p_smp->mgmt_class == IB_MCLASS_SUBN_DIR ) - { - status = ib_smp_get_status( p_smp ); - } - else - { - status = p_smp->status; - } - - if( status != 0 ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_rcv_callback: ERR 3111: " - "Error status = 0x%X\n", status ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - } - - switch( p_smp->method ) - { - case IB_MAD_METHOD_GET_RESP: - CL_ASSERT( p_req_madw != NULL ); - __osm_sm_mad_ctrl_process_get_resp( p_ctrl, p_madw, p_req_madw ); - break; - - case IB_MAD_METHOD_GET: - CL_ASSERT( p_req_madw == NULL ); - __osm_sm_mad_ctrl_process_get( p_ctrl, p_madw ); - break; - - case IB_MAD_METHOD_TRAP: - CL_ASSERT( p_req_madw == NULL ); - __osm_sm_mad_ctrl_process_trap( p_ctrl, p_madw ); - break; - - case IB_MAD_METHOD_SET: - CL_ASSERT( p_req_madw == NULL ); - __osm_sm_mad_ctrl_process_set( p_ctrl, p_madw ); - break; - - case IB_MAD_METHOD_SEND: - case IB_MAD_METHOD_REPORT: - case IB_MAD_METHOD_REPORT_RESP: - case IB_MAD_METHOD_TRAP_REPRESS: - default: - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_rcv_callback: ERR 3112: " - "Unsupported method = 0x%X\n", p_smp->method ); - osm_dump_dr_smp( p_ctrl->p_log, p_smp, OSM_LOG_ERROR ); - osm_mad_pool_put( p_ctrl->p_mad_pool, p_madw ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); + osm_sm_mad_ctrl_t *p_ctrl = (osm_sm_mad_ctrl_t *) bind_context; + ib_smp_t *p_smp; + ib_net16_t status; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_rcv_callback); + + CL_ASSERT(p_madw); + + /* + A MAD was received from the wire, possibly in response to a request. + */ + cl_atomic_inc(&p_ctrl->p_stats->qp0_mads_rcvd); + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_rcv_callback: " + "%u QP0 MADs received\n", + p_ctrl->p_stats->qp0_mads_rcvd); + } + + p_smp = osm_madw_get_smp_ptr(p_madw); + + /* if we are closing down simply do nothing */ + if (osm_exit_flag) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_rcv_callback: " + "Ignoring received mad - since we are exiting\n"); + + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_DEBUG); + + /* retire the mad or put it back */ + if (ib_smp_is_response(p_smp) || + (p_smp->method == IB_MAD_METHOD_TRAP_REPRESS)) { + CL_ASSERT(p_madw->resp_expected == FALSE); + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + } else { + if (p_madw->resp_expected == TRUE) + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, + p_madw); + else + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + } + + goto Exit; + } + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_FRAMES)) + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_FRAMES); + + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { + status = ib_smp_get_status(p_smp); + } else { + status = p_smp->status; + } + + if (status != 0) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_rcv_callback: ERR 3111: " + "Error status = 0x%X\n", status); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + } + + switch (p_smp->method) { + case IB_MAD_METHOD_GET_RESP: + CL_ASSERT(p_req_madw != NULL); + __osm_sm_mad_ctrl_process_get_resp(p_ctrl, p_madw, p_req_madw); + break; + + case IB_MAD_METHOD_GET: + CL_ASSERT(p_req_madw == NULL); + __osm_sm_mad_ctrl_process_get(p_ctrl, p_madw); + break; + + case IB_MAD_METHOD_TRAP: + CL_ASSERT(p_req_madw == NULL); + __osm_sm_mad_ctrl_process_trap(p_ctrl, p_madw); + break; + + case IB_MAD_METHOD_SET: + CL_ASSERT(p_req_madw == NULL); + __osm_sm_mad_ctrl_process_set(p_ctrl, p_madw); + break; + + case IB_MAD_METHOD_SEND: + case IB_MAD_METHOD_REPORT: + case IB_MAD_METHOD_REPORT_RESP: + case IB_MAD_METHOD_TRAP_REPRESS: + default: + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_rcv_callback: ERR 3112: " + "Unsupported method = 0x%X\n", p_smp->method); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -795,118 +750,109 @@ __osm_sm_mad_ctrl_rcv_callback( * SYNOPSIS */ void -__osm_sm_mad_ctrl_send_err_cb( - IN void *bind_context, - IN osm_madw_t *p_madw ) +__osm_sm_mad_ctrl_send_err_cb(IN void *bind_context, IN osm_madw_t * p_madw) { - osm_sm_mad_ctrl_t* p_ctrl = (osm_sm_mad_ctrl_t*)bind_context; + osm_sm_mad_ctrl_t *p_ctrl = (osm_sm_mad_ctrl_t *) bind_context; #if 0 - osm_physp_t* p_physp; + osm_physp_t *p_physp; #endif - ib_api_status_t status; - ib_smp_t* p_smp; - - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sm_mad_ctrl_send_err_cb ); - - CL_ASSERT( p_madw ); - - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_send_err_cb: ERR 3113: " - "MAD completed in error (%s)\n", - ib_get_err_str( p_madw->status ) ); - - /* - If this was a SubnSet MAD, then this error might indicate a problem - in configuring the subnet. In this case - need to mark that there was - such a problem. The subnet will not be up, and the next sweep should - be a heavy sweep as well. - */ - p_smp = osm_madw_get_smp_ptr( p_madw ); - if (p_smp->method == IB_MAD_METHOD_SET && - ( p_smp->attr_id == IB_MAD_ATTR_PORT_INFO || - p_smp->attr_id == IB_MAD_ATTR_MCAST_FWD_TBL || - p_smp->attr_id == IB_MAD_ATTR_SWITCH_INFO || - p_smp->attr_id == IB_MAD_ATTR_LIN_FWD_TBL ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_send_err_cb: ERR 3119: " - "Set method failed\n" ); - p_ctrl->p_subn->subnet_initialization_error = TRUE; - } - - /* - Since we did not get any response we suspect the DR path - used for the target port. - Find it and replace it with an alternate path. - This is true only if the destination lid is not 0xFFFF, since - then we are aiming for a specific path and not specific destination - lid. - */ - /* For now - do not add the alternate dr path to the release */ + ib_api_status_t status; + ib_smp_t *p_smp; + + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_send_err_cb); + + CL_ASSERT(p_madw); + + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_send_err_cb: ERR 3113: " + "MAD completed in error (%s)\n", + ib_get_err_str(p_madw->status)); + + /* + If this was a SubnSet MAD, then this error might indicate a problem + in configuring the subnet. In this case - need to mark that there was + such a problem. The subnet will not be up, and the next sweep should + be a heavy sweep as well. + */ + p_smp = osm_madw_get_smp_ptr(p_madw); + if (p_smp->method == IB_MAD_METHOD_SET && + (p_smp->attr_id == IB_MAD_ATTR_PORT_INFO || + p_smp->attr_id == IB_MAD_ATTR_MCAST_FWD_TBL || + p_smp->attr_id == IB_MAD_ATTR_SWITCH_INFO || + p_smp->attr_id == IB_MAD_ATTR_LIN_FWD_TBL)) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_send_err_cb: ERR 3119: " + "Set method failed\n"); + p_ctrl->p_subn->subnet_initialization_error = TRUE; + } + + /* + Since we did not get any response we suspect the DR path + used for the target port. + Find it and replace it with an alternate path. + This is true only if the destination lid is not 0xFFFF, since + then we are aiming for a specific path and not specific destination + lid. + */ + /* For now - do not add the alternate dr path to the release */ #if 0 - if ( p_madw->mad_addr.dest_lid != 0xFFFF ) - { - p_physp = - osm_get_physp_by_mad_addr(p_ctrl->p_log, - p_ctrl->p_subn, - &(p_madw->mad_addr)); - if (!p_physp) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_send_err_cb: ERR 3114: " - "Failed to find the corresponding phys port\n"); - } - else - { - osm_physp_replace_dr_path_with_alternate_dr_path( - p_ctrl->p_log, p_ctrl->p_subn, p_physp, p_madw->h_bind ); - } - } + if (p_madw->mad_addr.dest_lid != 0xFFFF) { + p_physp = + osm_get_physp_by_mad_addr(p_ctrl->p_log, + p_ctrl->p_subn, + &(p_madw->mad_addr)); + if (!p_physp) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_send_err_cb: ERR 3114: " + "Failed to find the corresponding phys port\n"); + } else { + osm_physp_replace_dr_path_with_alternate_dr_path + (p_ctrl->p_log, p_ctrl->p_subn, p_physp, + p_madw->h_bind); + } + } #endif - /* - An error occurred. No response was received to a request MAD. - Retire the original request MAD. - */ - - osm_dump_dr_smp( p_ctrl->p_log, osm_madw_get_smp_ptr( p_madw ), - OSM_LOG_ERROR ); - - __osm_sm_mad_ctrl_update_wire_stats( p_ctrl ); - - if( osm_madw_get_err_msg( p_madw ) != CL_DISP_MSGID_NONE ) - { - if( osm_log_is_active( p_ctrl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_send_err_cb: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( osm_madw_get_err_msg( p_madw ) ) ); - } - - status = cl_disp_post( p_ctrl->h_disp, - osm_madw_get_err_msg( p_madw ), - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl ); - if( status != CL_SUCCESS ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_send_err_cb: ERR 3115: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( status ) ); - } - } - else - { - /* - No error message was provided, just retire the MAD. - */ - __osm_sm_mad_ctrl_retire_trans_mad( p_ctrl, p_madw ); - } - - OSM_LOG_EXIT( p_ctrl->p_log ); + /* + An error occurred. No response was received to a request MAD. + Retire the original request MAD. + */ + + osm_dump_dr_smp(p_ctrl->p_log, osm_madw_get_smp_ptr(p_madw), + OSM_LOG_ERROR); + + __osm_sm_mad_ctrl_update_wire_stats(p_ctrl); + + if (osm_madw_get_err_msg(p_madw) != CL_DISP_MSGID_NONE) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_send_err_cb: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(osm_madw_get_err_msg + (p_madw))); + } + + status = cl_disp_post(p_ctrl->h_disp, + osm_madw_get_err_msg(p_madw), + p_madw, + __osm_sm_mad_ctrl_disp_done_callback, + p_ctrl); + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_send_err_cb: ERR 3115: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + } + } else { + /* + No error message was provided, just retire the MAD. + */ + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + } + + OSM_LOG_EXIT(p_ctrl->p_log); } + /* * PARAMETERS * @@ -919,131 +865,116 @@ __osm_sm_mad_ctrl_send_err_cb( /********************************************************************** **********************************************************************/ -void -osm_sm_mad_ctrl_construct( - IN osm_sm_mad_ctrl_t* const p_ctrl ) +void osm_sm_mad_ctrl_construct(IN osm_sm_mad_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); - memset( p_ctrl, 0, sizeof(*p_ctrl) ); - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; + CL_ASSERT(p_ctrl); + memset(p_ctrl, 0, sizeof(*p_ctrl)); + p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** **********************************************************************/ -void -osm_sm_mad_ctrl_destroy( - IN osm_sm_mad_ctrl_t* const p_ctrl ) +void osm_sm_mad_ctrl_destroy(IN osm_sm_mad_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); + CL_ASSERT(p_ctrl); - if (p_ctrl->h_bind != CL_DISP_INVALID_HANDLE) - { - osm_vendor_unbind( p_ctrl->h_bind ); - } - cl_disp_unregister( p_ctrl->h_disp ); + if (p_ctrl->h_bind != CL_DISP_INVALID_HANDLE) { + osm_vendor_unbind(p_ctrl->h_bind); + } + cl_disp_unregister(p_ctrl->h_disp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_mad_ctrl_init( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN osm_subn_t* const p_subn, - IN osm_mad_pool_t* const p_mad_pool, - IN osm_vl15_t* const p_vl15, - IN osm_vendor_t* const p_vendor, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN cl_plock_t* const p_lock, - IN cl_dispatcher_t* const p_disp ) +osm_sm_mad_ctrl_init(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN osm_subn_t * const p_subn, + IN osm_mad_pool_t * const p_mad_pool, + IN osm_vl15_t * const p_vl15, + IN osm_vendor_t * const p_vendor, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN cl_plock_t * const p_lock, + IN cl_dispatcher_t * const p_disp) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_sm_mad_ctrl_init ); - - osm_sm_mad_ctrl_construct( p_ctrl ); - - p_ctrl->p_subn = p_subn; - p_ctrl->p_log = p_log; - p_ctrl->p_disp = p_disp; - p_ctrl->p_mad_pool = p_mad_pool; - p_ctrl->p_vendor = p_vendor; - p_ctrl->p_stats = p_stats; - p_ctrl->p_lock = p_lock; - p_ctrl->p_vl15 = p_vl15; - - p_ctrl->h_disp = cl_disp_register( - p_disp, - CL_DISP_MSGID_NONE, - NULL, - NULL ); - - if( p_ctrl->h_disp == CL_DISP_INVALID_HANDLE ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_sm_mad_ctrl_init: ERR 3116: " - "Dispatcher registration failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_sm_mad_ctrl_init); + + osm_sm_mad_ctrl_construct(p_ctrl); + + p_ctrl->p_subn = p_subn; + p_ctrl->p_log = p_log; + p_ctrl->p_disp = p_disp; + p_ctrl->p_mad_pool = p_mad_pool; + p_ctrl->p_vendor = p_vendor; + p_ctrl->p_stats = p_stats; + p_ctrl->p_lock = p_lock; + p_ctrl->p_vl15 = p_vl15; + + p_ctrl->h_disp = cl_disp_register(p_disp, + CL_DISP_MSGID_NONE, NULL, NULL); + + if (p_ctrl->h_disp == CL_DISP_INVALID_HANDLE) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_sm_mad_ctrl_init: ERR 3116: " + "Dispatcher registration failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_mad_ctrl_bind( - IN osm_sm_mad_ctrl_t* const p_ctrl, - IN const ib_net64_t port_guid ) +osm_sm_mad_ctrl_bind(IN osm_sm_mad_ctrl_t * const p_ctrl, + IN const ib_net64_t port_guid) { - osm_bind_info_t bind_info; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_ctrl->p_log, osm_sm_mad_ctrl_bind ); - - if( p_ctrl->h_bind != OSM_BIND_INVALID_HANDLE ) - { - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "osm_sm_mad_ctrl_bind: ERR 3117: " - "Multiple binds not allowed\n" ); - status = IB_ERROR; - goto Exit; - } - - bind_info.class_version = 1; - bind_info.is_report_processor = FALSE; - bind_info.is_responder = TRUE; - bind_info.is_trap_processor = TRUE; - bind_info.mad_class = IB_MCLASS_SUBN_DIR; - bind_info.port_guid = port_guid; - bind_info.recv_q_size = OSM_SM_DEFAULT_QP0_RCV_SIZE; - bind_info.send_q_size = OSM_SM_DEFAULT_QP0_SEND_SIZE; - - osm_log( p_ctrl->p_log, OSM_LOG_VERBOSE, - "osm_sm_mad_ctrl_bind: " - "Binding to port 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - - p_ctrl->h_bind = osm_vendor_bind( p_ctrl->p_vendor, - &bind_info, - p_ctrl->p_mad_pool, - __osm_sm_mad_ctrl_rcv_callback, - __osm_sm_mad_ctrl_send_err_cb, - p_ctrl ); - - if( p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE ) - { - status = IB_ERROR; - osm_log( p_ctrl->p_log, OSM_LOG_ERROR, - "osm_sm_mad_ctrl_bind: ERR 3118: " - "Vendor specific bind failed\n" ); - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_ctrl->p_log ); - return( status ); + osm_bind_info_t bind_info; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_ctrl->p_log, osm_sm_mad_ctrl_bind); + + if (p_ctrl->h_bind != OSM_BIND_INVALID_HANDLE) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "osm_sm_mad_ctrl_bind: ERR 3117: " + "Multiple binds not allowed\n"); + status = IB_ERROR; + goto Exit; + } + + bind_info.class_version = 1; + bind_info.is_report_processor = FALSE; + bind_info.is_responder = TRUE; + bind_info.is_trap_processor = TRUE; + bind_info.mad_class = IB_MCLASS_SUBN_DIR; + bind_info.port_guid = port_guid; + bind_info.recv_q_size = OSM_SM_DEFAULT_QP0_RCV_SIZE; + bind_info.send_q_size = OSM_SM_DEFAULT_QP0_SEND_SIZE; + + osm_log(p_ctrl->p_log, OSM_LOG_VERBOSE, + "osm_sm_mad_ctrl_bind: " + "Binding to port 0x%" PRIx64 "\n", cl_ntoh64(port_guid)); + + p_ctrl->h_bind = osm_vendor_bind(p_ctrl->p_vendor, + &bind_info, + p_ctrl->p_mad_pool, + __osm_sm_mad_ctrl_rcv_callback, + __osm_sm_mad_ctrl_send_err_cb, p_ctrl); + + if (p_ctrl->h_bind == OSM_BIND_INVALID_HANDLE) { + status = IB_ERROR; + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "osm_sm_mad_ctrl_bind: ERR 3118: " + "Vendor specific bind failed\n"); + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_ctrl->p_log); + return (status); } diff --git a/opensm/opensm/osm_sm_state_mgr.c b/opensm/opensm/osm_sm_state_mgr.c index b37c959..2ff9639 100644 --- a/opensm/opensm/osm_sm_state_mgr.c +++ b/opensm/opensm/osm_sm_state_mgr.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -70,800 +70,760 @@ /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_standby_msg( - IN const osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_standby_msg(IN const osm_sm_state_mgr_t * p_sm_mgr) { - osm_log( p_sm_mgr->p_log, OSM_LOG_SYS, "Entering STANDBY state\n" ); /* Format Waived */ - - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_standby_msg: " - "\n\n\n********************************" - "**********************************\n" - "******************** ENTERING SM STANDBY" - " STATE *******************\n" - "**************************************" - "****************************\n\n\n" ); - } + osm_log(p_sm_mgr->p_log, OSM_LOG_SYS, "Entering STANDBY state\n"); /* Format Waived */ + + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_standby_msg: " + "\n\n\n********************************" + "**********************************\n" + "******************** ENTERING SM STANDBY" + " STATE *******************\n" + "**************************************" + "****************************\n\n\n"); + } } /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_master_msg( - IN const osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_master_msg(IN const osm_sm_state_mgr_t * p_sm_mgr) { - osm_log( p_sm_mgr->p_log, OSM_LOG_SYS, "Entering MASTER state\n" ); /* Format Waived */ - - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_master_msg: " - "\n\n\n********************************" - "**********************************\n" - "******************** ENTERING SM MASTER" - " STATE ********************\n" - "**************************************" - "****************************\n\n\n" ); - } + osm_log(p_sm_mgr->p_log, OSM_LOG_SYS, "Entering MASTER state\n"); /* Format Waived */ + + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_master_msg: " + "\n\n\n********************************" + "**********************************\n" + "******************** ENTERING SM MASTER" + " STATE ********************\n" + "**************************************" + "****************************\n\n\n"); + } } /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_discovering_msg( - IN const osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_discovering_msg(IN const osm_sm_state_mgr_t * p_sm_mgr) { - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_discovering_msg: " - "\n\n\n********************************" - "**********************************\n" - "******************** ENTERING SM DISCOVERING" - " STATE ***************\n" - "**************************************" - "****************************\n\n\n" ); - } + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_discovering_msg: " + "\n\n\n********************************" + "**********************************\n" + "******************** ENTERING SM DISCOVERING" + " STATE ***************\n" + "**************************************" + "****************************\n\n\n"); + } } /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_notactive_msg( - IN const osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_notactive_msg(IN const osm_sm_state_mgr_t * p_sm_mgr) { - osm_log( p_sm_mgr->p_log, OSM_LOG_SYS, "Entering NOT-ACTIVE state\n" ); /* Format Waived */ - - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_notactive_msg: " - "\n\n\n********************************" - "**********************************\n" - "***************** ENTERING SM NOT-ACTIVE" - " STATE *******************\n" - "**************************************" - "****************************\n\n\n" ); - } + osm_log(p_sm_mgr->p_log, OSM_LOG_SYS, "Entering NOT-ACTIVE state\n"); /* Format Waived */ + + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_notactive_msg: " + "\n\n\n********************************" + "**********************************\n" + "***************** ENTERING SM NOT-ACTIVE" + " STATE *******************\n" + "**************************************" + "****************************\n\n\n"); + } } #if 0 /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_send_local_port_info_req( - IN osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_send_local_port_info_req(IN osm_sm_state_mgr_t * p_sm_mgr) { - osm_madw_context_t context; - osm_port_t *p_port; - ib_net64_t port_guid = p_sm_mgr->p_subn->sm_port_guid; - ib_api_status_t status; - - OSM_LOG_ENTER( p_sm_mgr->p_log, - __osm_sm_state_mgr_send_local_port_info_req ); - /* - * Send a query of SubnGet(PortInfo) to our own port, in order to - * update the master_sm_base_lid of the subnet. - */ - memset( &context, 0, sizeof( context ) ); - p_port = osm_get_port_by_guid( p_sm_mgr->p_subn, port_guid ); - if( !p_port ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_send_local_port_info_req: ERR 3205: " - "No port object for port 0x%016" PRIx64 "\n", - cl_ntoh64( port_guid ) ); - goto Exit; - } - - context.pi_context.port_guid = port_guid; - context.pi_context.node_guid = p_port->p_node->node_info.node_guid; - context.pi_context.set_method = FALSE; - context.pi_context.ignore_errors = FALSE; - /* mark the update_master_sm_base_lid with TRUE - we want to update it */ - /* with the new master lid value. */ - context.pi_context.update_master_sm_base_lid = TRUE; - context.pi_context.light_sweep = FALSE; - context.pi_context.active_transition = FALSE; - - status = osm_req_get( p_sm_mgr->p_req, - osm_physp_get_dr_path_ptr - ( p_port->p_physp ), - IB_MAD_ATTR_PORT_INFO, - cl_hton32( p_port->p_physp->port_num ), - CL_DISP_MSGID_NONE, &context ); - - if( status != IB_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_send_local_port_info_req: ERR 3202: " - "Failure requesting PortInfo (%s)\n", - ib_get_err_str( status ) ); - } - - Exit: - OSM_LOG_EXIT( p_sm_mgr->p_log ); + osm_madw_context_t context; + osm_port_t *p_port; + ib_net64_t port_guid = p_sm_mgr->p_subn->sm_port_guid; + ib_api_status_t status; + + OSM_LOG_ENTER(p_sm_mgr->p_log, + __osm_sm_state_mgr_send_local_port_info_req); + /* + * Send a query of SubnGet(PortInfo) to our own port, in order to + * update the master_sm_base_lid of the subnet. + */ + memset(&context, 0, sizeof(context)); + p_port = osm_get_port_by_guid(p_sm_mgr->p_subn, port_guid); + if (!p_port) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_send_local_port_info_req: ERR 3205: " + "No port object for port 0x%016" PRIx64 "\n", + cl_ntoh64(port_guid)); + goto Exit; + } + + context.pi_context.port_guid = port_guid; + context.pi_context.node_guid = p_port->p_node->node_info.node_guid; + context.pi_context.set_method = FALSE; + context.pi_context.ignore_errors = FALSE; + /* mark the update_master_sm_base_lid with TRUE - we want to update it */ + /* with the new master lid value. */ + context.pi_context.update_master_sm_base_lid = TRUE; + context.pi_context.light_sweep = FALSE; + context.pi_context.active_transition = FALSE; + + status = osm_req_get(p_sm_mgr->p_req, + osm_physp_get_dr_path_ptr + (p_port->p_physp), + IB_MAD_ATTR_PORT_INFO, + cl_hton32(p_port->p_physp->port_num), + CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_send_local_port_info_req: ERR 3202: " + "Failure requesting PortInfo (%s)\n", + ib_get_err_str(status)); + } + + Exit: + OSM_LOG_EXIT(p_sm_mgr->p_log); } #endif /********************************************************************** **********************************************************************/ static void -__osm_sm_state_mgr_send_master_sm_info_req( - IN osm_sm_state_mgr_t * p_sm_mgr ) +__osm_sm_state_mgr_send_master_sm_info_req(IN osm_sm_state_mgr_t * p_sm_mgr) { - osm_madw_context_t context; - const osm_port_t *p_port; - ib_api_status_t status; - - OSM_LOG_ENTER( p_sm_mgr->p_log, - __osm_sm_state_mgr_send_master_sm_info_req ); - - memset( &context, 0, sizeof( context ) ); - if( p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY ) - { - /* - * We are in STANDBY state - this means we need to poll on the master - * SM (according to master_guid) - * Send a query of SubnGet(SMInfo) to the subn master_sm_base_lid object. - */ - p_port = osm_get_port_by_guid( p_sm_mgr->p_subn, p_sm_mgr->master_guid ); - } - else - { - /* - * We are not in STANDBY - this means we are in MASTER state - so we need - * to poll on the SM that is saved in p_polling_sm under p_sm_mgr. - * Send a query of SubnGet(SMInfo) to that SM. - */ - p_port = p_sm_mgr->p_polling_sm->p_port; - } - if( p_port == NULL ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_send_master_sm_info_req: ERR 3203: " - "No port object for GUID 0x%016" PRIx64 "\n", - cl_ntoh64(p_sm_mgr->master_guid) ); - goto Exit; - } - - context.smi_context.port_guid = p_port->guid; - context.smi_context.set_method = FALSE; - - status = osm_req_get( p_sm_mgr->p_req, - osm_physp_get_dr_path_ptr(p_port->p_physp), - IB_MAD_ATTR_SM_INFO, 0, CL_DISP_MSGID_NONE, - &context ); - - if( status != IB_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_send_master_sm_info_req: ERR 3204: " - "Failure requesting SMInfo (%s)\n", ib_get_err_str( status ) ); - } - - Exit: - OSM_LOG_EXIT( p_sm_mgr->p_log ); + osm_madw_context_t context; + const osm_port_t *p_port; + ib_api_status_t status; + + OSM_LOG_ENTER(p_sm_mgr->p_log, + __osm_sm_state_mgr_send_master_sm_info_req); + + memset(&context, 0, sizeof(context)); + if (p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY) { + /* + * We are in STANDBY state - this means we need to poll on the master + * SM (according to master_guid) + * Send a query of SubnGet(SMInfo) to the subn master_sm_base_lid object. + */ + p_port = + osm_get_port_by_guid(p_sm_mgr->p_subn, + p_sm_mgr->master_guid); + } else { + /* + * We are not in STANDBY - this means we are in MASTER state - so we need + * to poll on the SM that is saved in p_polling_sm under p_sm_mgr. + * Send a query of SubnGet(SMInfo) to that SM. + */ + p_port = p_sm_mgr->p_polling_sm->p_port; + } + if (p_port == NULL) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_send_master_sm_info_req: ERR 3203: " + "No port object for GUID 0x%016" PRIx64 "\n", + cl_ntoh64(p_sm_mgr->master_guid)); + goto Exit; + } + + context.smi_context.port_guid = p_port->guid; + context.smi_context.set_method = FALSE; + + status = osm_req_get(p_sm_mgr->p_req, + osm_physp_get_dr_path_ptr(p_port->p_physp), + IB_MAD_ATTR_SM_INFO, 0, CL_DISP_MSGID_NONE, + &context); + + if (status != IB_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_send_master_sm_info_req: ERR 3204: " + "Failure requesting SMInfo (%s)\n", + ib_get_err_str(status)); + } + + Exit: + OSM_LOG_EXIT(p_sm_mgr->p_log); } /********************************************************************** **********************************************************************/ -static void -__osm_sm_state_mgr_start_polling( - IN osm_sm_state_mgr_t * p_sm_mgr ) +static void __osm_sm_state_mgr_start_polling(IN osm_sm_state_mgr_t * p_sm_mgr) { - uint32_t sminfo_polling_timeout = - p_sm_mgr->p_subn->opt.sminfo_polling_timeout; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_sm_mgr->p_log, __osm_sm_state_mgr_start_polling ); - - /* - * Init the retry_nubmer back to zero - need to restart counting - */ - p_sm_mgr->retry_number = 0; - - /* - * Send a SubnGet(SMInfo) query to the current (or new) master found. - */ - __osm_sm_state_mgr_send_master_sm_info_req( p_sm_mgr ); - - /* - * Start a timer that will wake up every sminfo_polling_timeout milliseconds. - * The callback of the timer will send a SubnGet(SMInfo) to the Master SM - * and restart the timer - */ - cl_status = cl_timer_start( &p_sm_mgr->polling_timer, - sminfo_polling_timeout ); - if( cl_status != CL_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_start_polling: ERR 3210: " - "Failed to start timer\n" ); - } - - OSM_LOG_EXIT( p_sm_mgr->p_log ); + uint32_t sminfo_polling_timeout = + p_sm_mgr->p_subn->opt.sminfo_polling_timeout; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_sm_mgr->p_log, __osm_sm_state_mgr_start_polling); + + /* + * Init the retry_nubmer back to zero - need to restart counting + */ + p_sm_mgr->retry_number = 0; + + /* + * Send a SubnGet(SMInfo) query to the current (or new) master found. + */ + __osm_sm_state_mgr_send_master_sm_info_req(p_sm_mgr); + + /* + * Start a timer that will wake up every sminfo_polling_timeout milliseconds. + * The callback of the timer will send a SubnGet(SMInfo) to the Master SM + * and restart the timer + */ + cl_status = cl_timer_start(&p_sm_mgr->polling_timer, + sminfo_polling_timeout); + if (cl_status != CL_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_start_polling: ERR 3210: " + "Failed to start timer\n"); + } + + OSM_LOG_EXIT(p_sm_mgr->p_log); } /********************************************************************** **********************************************************************/ -static void -__osm_sm_state_mgr_polling_callback( - IN void *context ) +static void __osm_sm_state_mgr_polling_callback(IN void *context) { - osm_sm_state_mgr_t *p_sm_mgr = ( osm_sm_state_mgr_t * ) context; - uint32_t sminfo_polling_timeout = - p_sm_mgr->p_subn->opt.sminfo_polling_timeout; - cl_status_t cl_status; - - OSM_LOG_ENTER( p_sm_mgr->p_log, __osm_sm_state_mgr_polling_callback ); - - /* - * We can be here in one of two cases: - * 1. We are a STANDBY sm polling on the master SM. - * 2. We are a MASTER sm, waiting for a handover from a remote master sm. - * If we are not in one of these cases - don't need to restart the poller. - */ - if( !( ( p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_MASTER && - p_sm_mgr->p_polling_sm != NULL ) || - ( p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY ) ) ) - { - goto Exit; - } - - /* - * If we are a STANDBY sm and the osm_exit_flag is 1, then let's signal - * the subnet_up. This is relevant for the case of running only once. In that - * case - the program is stuck until this signal is received. In other - * cases - it is not relevant whether or not the signal is on - since we are - * currently in exit flow - */ - if( p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY && - osm_exit_flag == 1 ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_polling_callback: " - "Signalling subnet_up_event\n" ); - cl_event_signal( p_sm_mgr->p_state_mgr->p_subnet_up_event ); - goto Exit; - } - - /* - * Incr the retry number. - * If it reached the max_retry_number in the subnet opt - call - * osm_sm_state_mgr_process with signal OSM_SM_SIGNAL_POLLING_TIMEOUT - */ - p_sm_mgr->retry_number++; - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "__osm_sm_state_mgr_polling_callback: " - "Retry number:%d\n", p_sm_mgr->retry_number ); - - if( p_sm_mgr->retry_number >= p_sm_mgr->p_subn->opt.polling_retry_number ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_DEBUG, - "__osm_sm_state_mgr_polling_callback: " - "Reached polling_retry_number value in retry_number. " - "Go to DISCOVERY state\n" ); - osm_sm_state_mgr_process( p_sm_mgr, OSM_SM_SIGNAL_POLLING_TIMEOUT ); - goto Exit; - } - - /* Send a SubnGet(SMInfo) request to the remote sm (depends on our state) */ - __osm_sm_state_mgr_send_master_sm_info_req( p_sm_mgr ); - - /* restart the timer */ - cl_status = cl_timer_start( &p_sm_mgr->polling_timer, - sminfo_polling_timeout ); - if( cl_status != CL_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_polling_callback: ERR 3211: " - "Failed to restart timer\n" ); - } - - Exit: - OSM_LOG_EXIT( p_sm_mgr->p_log ); - return; + osm_sm_state_mgr_t *p_sm_mgr = (osm_sm_state_mgr_t *) context; + uint32_t sminfo_polling_timeout = + p_sm_mgr->p_subn->opt.sminfo_polling_timeout; + cl_status_t cl_status; + + OSM_LOG_ENTER(p_sm_mgr->p_log, __osm_sm_state_mgr_polling_callback); + + /* + * We can be here in one of two cases: + * 1. We are a STANDBY sm polling on the master SM. + * 2. We are a MASTER sm, waiting for a handover from a remote master sm. + * If we are not in one of these cases - don't need to restart the poller. + */ + if (!((p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_MASTER && + p_sm_mgr->p_polling_sm != NULL) || + (p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY))) { + goto Exit; + } + + /* + * If we are a STANDBY sm and the osm_exit_flag is 1, then let's signal + * the subnet_up. This is relevant for the case of running only once. In that + * case - the program is stuck until this signal is received. In other + * cases - it is not relevant whether or not the signal is on - since we are + * currently in exit flow + */ + if (p_sm_mgr->p_subn->sm_state == IB_SMINFO_STATE_STANDBY && + osm_exit_flag == 1) { + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_polling_callback: " + "Signalling subnet_up_event\n"); + cl_event_signal(p_sm_mgr->p_state_mgr->p_subnet_up_event); + goto Exit; + } + + /* + * Incr the retry number. + * If it reached the max_retry_number in the subnet opt - call + * osm_sm_state_mgr_process with signal OSM_SM_SIGNAL_POLLING_TIMEOUT + */ + p_sm_mgr->retry_number++; + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "__osm_sm_state_mgr_polling_callback: " + "Retry number:%d\n", p_sm_mgr->retry_number); + + if (p_sm_mgr->retry_number >= + p_sm_mgr->p_subn->opt.polling_retry_number) { + osm_log(p_sm_mgr->p_log, OSM_LOG_DEBUG, + "__osm_sm_state_mgr_polling_callback: " + "Reached polling_retry_number value in retry_number. " + "Go to DISCOVERY state\n"); + osm_sm_state_mgr_process(p_sm_mgr, + OSM_SM_SIGNAL_POLLING_TIMEOUT); + goto Exit; + } + + /* Send a SubnGet(SMInfo) request to the remote sm (depends on our state) */ + __osm_sm_state_mgr_send_master_sm_info_req(p_sm_mgr); + + /* restart the timer */ + cl_status = cl_timer_start(&p_sm_mgr->polling_timer, + sminfo_polling_timeout); + if (cl_status != CL_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_polling_callback: ERR 3211: " + "Failed to restart timer\n"); + } + + Exit: + OSM_LOG_EXIT(p_sm_mgr->p_log); + return; } /********************************************************************** **********************************************************************/ -void -osm_sm_state_mgr_construct( - IN osm_sm_state_mgr_t * const p_sm_mgr ) +void osm_sm_state_mgr_construct(IN osm_sm_state_mgr_t * const p_sm_mgr) { - memset( p_sm_mgr, 0, sizeof( *p_sm_mgr ) ); - cl_spinlock_construct( &p_sm_mgr->state_lock ); - cl_timer_construct( &p_sm_mgr->polling_timer ); + memset(p_sm_mgr, 0, sizeof(*p_sm_mgr)); + cl_spinlock_construct(&p_sm_mgr->state_lock); + cl_timer_construct(&p_sm_mgr->polling_timer); } /********************************************************************** **********************************************************************/ -void -osm_sm_state_mgr_destroy( - IN osm_sm_state_mgr_t * const p_sm_mgr ) +void osm_sm_state_mgr_destroy(IN osm_sm_state_mgr_t * const p_sm_mgr) { - CL_ASSERT( p_sm_mgr ); + CL_ASSERT(p_sm_mgr); - OSM_LOG_ENTER( p_sm_mgr->p_log, osm_sm_state_mgr_destroy ); + OSM_LOG_ENTER(p_sm_mgr->p_log, osm_sm_state_mgr_destroy); - cl_spinlock_destroy( &p_sm_mgr->state_lock ); - cl_timer_destroy( &p_sm_mgr->polling_timer ); + cl_spinlock_destroy(&p_sm_mgr->state_lock); + cl_timer_destroy(&p_sm_mgr->polling_timer); - OSM_LOG_EXIT( p_sm_mgr->p_log ); + OSM_LOG_EXIT(p_sm_mgr->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_state_mgr_init( - IN osm_sm_state_mgr_t * const p_sm_mgr, - IN osm_state_mgr_t * const p_state_mgr, - IN osm_subn_t * const p_subn, - IN osm_req_t * const p_req, - IN osm_log_t * const p_log ) +osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_state_mgr_t * const p_state_mgr, + IN osm_subn_t * const p_subn, + IN osm_req_t * const p_req, IN osm_log_t * const p_log) { - cl_status_t status; - - OSM_LOG_ENTER( p_log, osm_sm_state_mgr_init ); - - CL_ASSERT( p_subn ); - CL_ASSERT( p_state_mgr ); - CL_ASSERT( p_req ); - - osm_sm_state_mgr_construct( p_sm_mgr ); - - p_sm_mgr->p_log = p_log; - p_sm_mgr->p_req = p_req; - p_sm_mgr->p_subn = p_subn; - p_sm_mgr->p_state_mgr = p_state_mgr; - - if (p_subn->opt.sm_inactive) - { - /* init the state of the SM to not active */ - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_NOTACTIVE; - __osm_sm_state_mgr_notactive_msg( p_sm_mgr ); - } - else - { - /* init the state of the SM to init */ - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_INIT; - } - - status = cl_spinlock_init( &p_sm_mgr->state_lock ); - if( status != CL_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "osm_sm_state_mgr_init: ERR 3201: " - "Spinlock init failed (%s)\n", CL_STATUS_MSG( status ) ); - } - - status = cl_timer_init( &p_sm_mgr->polling_timer, - __osm_sm_state_mgr_polling_callback, p_sm_mgr ); - - if( status != CL_SUCCESS ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "osm_sm_state_mgr_init: ERR 3206: " - "Timer init failed (%s)\n", CL_STATUS_MSG( status ) ); - } - - OSM_LOG_EXIT( p_sm_mgr->p_log ); - return ( status ); + cl_status_t status; + + OSM_LOG_ENTER(p_log, osm_sm_state_mgr_init); + + CL_ASSERT(p_subn); + CL_ASSERT(p_state_mgr); + CL_ASSERT(p_req); + + osm_sm_state_mgr_construct(p_sm_mgr); + + p_sm_mgr->p_log = p_log; + p_sm_mgr->p_req = p_req; + p_sm_mgr->p_subn = p_subn; + p_sm_mgr->p_state_mgr = p_state_mgr; + + if (p_subn->opt.sm_inactive) { + /* init the state of the SM to not active */ + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_NOTACTIVE; + __osm_sm_state_mgr_notactive_msg(p_sm_mgr); + } else { + /* init the state of the SM to init */ + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_INIT; + } + + status = cl_spinlock_init(&p_sm_mgr->state_lock); + if (status != CL_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "osm_sm_state_mgr_init: ERR 3201: " + "Spinlock init failed (%s)\n", CL_STATUS_MSG(status)); + } + + status = cl_timer_init(&p_sm_mgr->polling_timer, + __osm_sm_state_mgr_polling_callback, p_sm_mgr); + + if (status != CL_SUCCESS) { + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "osm_sm_state_mgr_init: ERR 3206: " + "Timer init failed (%s)\n", CL_STATUS_MSG(status)); + } + + OSM_LOG_EXIT(p_sm_mgr->p_log); + return (status); } /********************************************************************** **********************************************************************/ void -__osm_sm_state_mgr_signal_error( - IN const osm_sm_state_mgr_t * const p_sm_mgr, - IN const osm_sm_signal_t signal ) +__osm_sm_state_mgr_signal_error(IN const osm_sm_state_mgr_t * const p_sm_mgr, + IN const osm_sm_signal_t signal) { - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "__osm_sm_state_mgr_signal_error: ERR 3207: " - "Invalid signal %s in state %s\n", - osm_get_sm_mgr_signal_str( signal ), - osm_get_sm_mgr_state_str( p_sm_mgr->p_subn->sm_state ) ); + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "__osm_sm_state_mgr_signal_error: ERR 3207: " + "Invalid signal %s in state %s\n", + osm_get_sm_mgr_signal_str(signal), + osm_get_sm_mgr_state_str(p_sm_mgr->p_subn->sm_state)); } /********************************************************************** **********************************************************************/ void -osm_sm_state_mgr_signal_master_is_alive( - IN osm_sm_state_mgr_t * const p_sm_mgr ) +osm_sm_state_mgr_signal_master_is_alive(IN osm_sm_state_mgr_t * const p_sm_mgr) { - OSM_LOG_ENTER( p_sm_mgr->p_log, osm_sm_state_mgr_signal_master_is_alive ); - p_sm_mgr->retry_number = 0; - OSM_LOG_EXIT( p_sm_mgr->p_log ); + OSM_LOG_ENTER(p_sm_mgr->p_log, osm_sm_state_mgr_signal_master_is_alive); + p_sm_mgr->retry_number = 0; + OSM_LOG_EXIT(p_sm_mgr->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_state_mgr_process( - IN osm_sm_state_mgr_t * const p_sm_mgr, - IN osm_sm_signal_t signal ) +osm_sm_state_mgr_process(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_sm_signal_t signal) { - ib_api_status_t status = IB_SUCCESS; - - CL_ASSERT( p_sm_mgr ); - - OSM_LOG_ENTER( p_sm_mgr->p_log, osm_sm_state_mgr_process ); - - /* - * The state lock prevents many race conditions from screwing - * up the state transition process. - */ - cl_spinlock_acquire( &p_sm_mgr->state_lock ); - - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_DEBUG, - "osm_sm_state_mgr_process: " - "Received signal %s in state %s\n", - osm_get_sm_mgr_signal_str( signal ), - osm_get_sm_mgr_state_str( p_sm_mgr->p_subn->sm_state ) ); - } - - switch ( p_sm_mgr->p_subn->sm_state ) - { - case IB_SMINFO_STATE_INIT: - switch ( signal ) - { - case OSM_SM_SIGNAL_INIT: - /* - * Update the state of the SM to DISCOVERING - */ - __osm_sm_state_mgr_discovering_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_DISCOVERING; - break; - - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_DISCOVERING: - switch ( signal ) - { - case OSM_SM_SIGNAL_DISCOVERY_COMPLETED: - /* - * Update the state of the SM to MASTER - */ - __osm_sm_state_mgr_master_msg( p_sm_mgr ); - /* Turn on the moved_to_master_state flag */ - p_sm_mgr->p_subn->moved_to_master_state = TRUE; - /* Turn on the first_time_master_sweep flag */ - if( p_sm_mgr->p_subn->first_time_master_sweep == FALSE ) - p_sm_mgr->p_subn->first_time_master_sweep = TRUE; - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_MASTER; - /* - * Make sure to set the subnet master_sm_base_lid - * to the sm_base_lid value - */ - p_sm_mgr->p_subn->master_sm_base_lid = p_sm_mgr->p_subn->sm_base_lid; - break; - case OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE: - /* - * Finished all discovery actions - move to STANDBY - * start the polling - */ - __osm_sm_state_mgr_standby_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; - /* - * Since another SM is doing the LFT config - we should not - * ignore the results of it - */ - p_sm_mgr->p_subn->ignore_existing_lfts = FALSE; - - __osm_sm_state_mgr_start_polling( p_sm_mgr ); - break; - case OSM_SM_SIGNAL_HANDOVER: - /* - * Do nothing. We will discover it later on. If we already discovered - * this SM, and got the HANDOVER - this means the remote SM is of - * lower priority. In this case we will stop polling it (since it is - * a lower priority SM in STANDBY state). - */ - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_STANDBY: - switch ( signal ) - { - case OSM_SM_SIGNAL_POLLING_TIMEOUT: - case OSM_SM_SIGNAL_DISCOVER: - /* - * case 1: Polling timeout occured - this means that the Master SM - * is no longer alive. - * case 2: Got a signal to move to DISCOVERING - * Move to DISCOVERING state and start sweeping - */ - __osm_sm_state_mgr_discovering_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_DISCOVERING; - p_sm_mgr->p_subn->coming_out_of_standby = TRUE; - osm_state_mgr_process( p_sm_mgr->p_state_mgr, OSM_SIGNAL_EXIT_STBY ); - break; - case OSM_SM_SIGNAL_DISABLE: - /* - * Update the state to NOT_ACTIVE - */ - __osm_sm_state_mgr_notactive_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_NOTACTIVE; - break; - case OSM_SM_SIGNAL_HANDOVER: - /* - * Update the state to MASTER, and start sweeping - * OPTIONAL: send ACKNOWLEDGE - */ - __osm_sm_state_mgr_master_msg( p_sm_mgr ); - /* Turn on the moved_to_master_state flag */ - p_sm_mgr->p_subn->moved_to_master_state = TRUE; - /* Turn on the first_time_master_sweep flag */ - if( p_sm_mgr->p_subn->first_time_master_sweep == FALSE ) - p_sm_mgr->p_subn->first_time_master_sweep = TRUE; - /* Turn on the force_immediate_heavy_sweep - we want a - * heavy sweep to occur on the first sweep of this SM. */ - p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE; - - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_MASTER; - /* - * Make sure to set the subnet master_sm_base_lid - * to the sm_base_lid value - */ - p_sm_mgr->p_subn->master_sm_base_lid = p_sm_mgr->p_subn->sm_base_lid; - p_sm_mgr->p_subn->coming_out_of_standby = TRUE; - osm_state_mgr_process( p_sm_mgr->p_state_mgr, OSM_SIGNAL_EXIT_STBY ); - break; - case OSM_SM_SIGNAL_ACKNOWLEDGE: - /* - * Do nothing - already moved to STANDBY - */ - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_NOTACTIVE: - switch ( signal ) - { - case OSM_SM_SIGNAL_STANDBY: - /* - * Update the state to STANDBY - * start the polling - */ - __osm_sm_state_mgr_standby_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; - __osm_sm_state_mgr_start_polling( p_sm_mgr ); - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_MASTER: - switch ( signal ) - { - case OSM_SM_SIGNAL_POLLING_TIMEOUT: - /* - * we received a polling timeout - this means that we waited for - * a remote master sm to send us a handover, but didn't get it, and - * didn't get a response from that remote sm. - * We want to force a heavy sweep - hopefully this occurred because - * the remote sm died, and we'll find this out and configure the - * subnet after a heavy sweep. - * We also want to clear the p_polling_sm object - since we are - * done polling on that remote sm - we are sweeping again. - */ - case OSM_SM_SIGNAL_HANDOVER: - /* - * If we received a handover in a master state - then we want to - * force a heavy sweep. This means that either we are in a sweep - * currently - in this case - no change, or we are in idle state - - * since we recognized a master SM before - so we want to make a - * heavy sweep and reconfigure the new subnet. - * We also want to clear the p_polling_sm object - since we are - * done polling on that remote sm - we got a handover from it. - */ - osm_log( p_sm_mgr->p_log, OSM_LOG_VERBOSE, - "osm_sm_state_mgr_process: " - "Forcing immediate heavy sweep. " - "Received OSM_SM_SIGNAL_HANDOVER or OSM_SM_SIGNAL_POLLING_TIMEOUT\n" ); - p_sm_mgr->p_polling_sm = NULL; - p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE; - osm_state_mgr_process( p_sm_mgr->p_state_mgr, OSM_SIGNAL_SWEEP ); - break; - case OSM_SM_SIGNAL_HANDOVER_SENT: - /* - * Just sent a HANDOVER signal - move to STANDBY - * start the polling - */ - __osm_sm_state_mgr_standby_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; - __osm_sm_state_mgr_start_polling( p_sm_mgr ); - break; - case OSM_SM_SIGNAL_WAIT_FOR_HANDOVER: - /* - * We found a remote master SM, and we are waiting for it - * to handover the mastership to us. Need to start polling - * on that SM, to make sure it is alive, if it isn't - then - * we should move back to discovering, since something must - * have happened to it. - */ - __osm_sm_state_mgr_start_polling( p_sm_mgr ); - break; - case OSM_SM_SIGNAL_DISCOVER: - __osm_sm_state_mgr_discovering_msg( p_sm_mgr ); - p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_DISCOVERING; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - default: - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "osm_sm_state_mgr_process: ERR 3208: " - "Invalid state %s\n", - osm_get_sm_mgr_state_str( p_sm_mgr->p_subn->sm_state ) ); - - } - - cl_spinlock_release( &p_sm_mgr->state_lock ); - - OSM_LOG_EXIT( p_sm_mgr->p_log ); - return ( status ); + ib_api_status_t status = IB_SUCCESS; + + CL_ASSERT(p_sm_mgr); + + OSM_LOG_ENTER(p_sm_mgr->p_log, osm_sm_state_mgr_process); + + /* + * The state lock prevents many race conditions from screwing + * up the state transition process. + */ + cl_spinlock_acquire(&p_sm_mgr->state_lock); + + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_DEBUG, + "osm_sm_state_mgr_process: " + "Received signal %s in state %s\n", + osm_get_sm_mgr_signal_str(signal), + osm_get_sm_mgr_state_str(p_sm_mgr->p_subn->sm_state)); + } + + switch (p_sm_mgr->p_subn->sm_state) { + case IB_SMINFO_STATE_INIT: + switch (signal) { + case OSM_SM_SIGNAL_INIT: + /* + * Update the state of the SM to DISCOVERING + */ + __osm_sm_state_mgr_discovering_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = + IB_SMINFO_STATE_DISCOVERING; + break; + + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_DISCOVERING: + switch (signal) { + case OSM_SM_SIGNAL_DISCOVERY_COMPLETED: + /* + * Update the state of the SM to MASTER + */ + __osm_sm_state_mgr_master_msg(p_sm_mgr); + /* Turn on the moved_to_master_state flag */ + p_sm_mgr->p_subn->moved_to_master_state = TRUE; + /* Turn on the first_time_master_sweep flag */ + if (p_sm_mgr->p_subn->first_time_master_sweep == FALSE) + p_sm_mgr->p_subn->first_time_master_sweep = + TRUE; + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_MASTER; + /* + * Make sure to set the subnet master_sm_base_lid + * to the sm_base_lid value + */ + p_sm_mgr->p_subn->master_sm_base_lid = + p_sm_mgr->p_subn->sm_base_lid; + break; + case OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE: + /* + * Finished all discovery actions - move to STANDBY + * start the polling + */ + __osm_sm_state_mgr_standby_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; + /* + * Since another SM is doing the LFT config - we should not + * ignore the results of it + */ + p_sm_mgr->p_subn->ignore_existing_lfts = FALSE; + + __osm_sm_state_mgr_start_polling(p_sm_mgr); + break; + case OSM_SM_SIGNAL_HANDOVER: + /* + * Do nothing. We will discover it later on. If we already discovered + * this SM, and got the HANDOVER - this means the remote SM is of + * lower priority. In this case we will stop polling it (since it is + * a lower priority SM in STANDBY state). + */ + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_STANDBY: + switch (signal) { + case OSM_SM_SIGNAL_POLLING_TIMEOUT: + case OSM_SM_SIGNAL_DISCOVER: + /* + * case 1: Polling timeout occured - this means that the Master SM + * is no longer alive. + * case 2: Got a signal to move to DISCOVERING + * Move to DISCOVERING state and start sweeping + */ + __osm_sm_state_mgr_discovering_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = + IB_SMINFO_STATE_DISCOVERING; + p_sm_mgr->p_subn->coming_out_of_standby = TRUE; + osm_state_mgr_process(p_sm_mgr->p_state_mgr, + OSM_SIGNAL_EXIT_STBY); + break; + case OSM_SM_SIGNAL_DISABLE: + /* + * Update the state to NOT_ACTIVE + */ + __osm_sm_state_mgr_notactive_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_NOTACTIVE; + break; + case OSM_SM_SIGNAL_HANDOVER: + /* + * Update the state to MASTER, and start sweeping + * OPTIONAL: send ACKNOWLEDGE + */ + __osm_sm_state_mgr_master_msg(p_sm_mgr); + /* Turn on the moved_to_master_state flag */ + p_sm_mgr->p_subn->moved_to_master_state = TRUE; + /* Turn on the first_time_master_sweep flag */ + if (p_sm_mgr->p_subn->first_time_master_sweep == FALSE) + p_sm_mgr->p_subn->first_time_master_sweep = + TRUE; + /* Turn on the force_immediate_heavy_sweep - we want a + * heavy sweep to occur on the first sweep of this SM. */ + p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE; + + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_MASTER; + /* + * Make sure to set the subnet master_sm_base_lid + * to the sm_base_lid value + */ + p_sm_mgr->p_subn->master_sm_base_lid = + p_sm_mgr->p_subn->sm_base_lid; + p_sm_mgr->p_subn->coming_out_of_standby = TRUE; + osm_state_mgr_process(p_sm_mgr->p_state_mgr, + OSM_SIGNAL_EXIT_STBY); + break; + case OSM_SM_SIGNAL_ACKNOWLEDGE: + /* + * Do nothing - already moved to STANDBY + */ + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_NOTACTIVE: + switch (signal) { + case OSM_SM_SIGNAL_STANDBY: + /* + * Update the state to STANDBY + * start the polling + */ + __osm_sm_state_mgr_standby_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; + __osm_sm_state_mgr_start_polling(p_sm_mgr); + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_MASTER: + switch (signal) { + case OSM_SM_SIGNAL_POLLING_TIMEOUT: + /* + * we received a polling timeout - this means that we waited for + * a remote master sm to send us a handover, but didn't get it, and + * didn't get a response from that remote sm. + * We want to force a heavy sweep - hopefully this occurred because + * the remote sm died, and we'll find this out and configure the + * subnet after a heavy sweep. + * We also want to clear the p_polling_sm object - since we are + * done polling on that remote sm - we are sweeping again. + */ + case OSM_SM_SIGNAL_HANDOVER: + /* + * If we received a handover in a master state - then we want to + * force a heavy sweep. This means that either we are in a sweep + * currently - in this case - no change, or we are in idle state - + * since we recognized a master SM before - so we want to make a + * heavy sweep and reconfigure the new subnet. + * We also want to clear the p_polling_sm object - since we are + * done polling on that remote sm - we got a handover from it. + */ + osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, + "osm_sm_state_mgr_process: " + "Forcing immediate heavy sweep. " + "Received OSM_SM_SIGNAL_HANDOVER or OSM_SM_SIGNAL_POLLING_TIMEOUT\n"); + p_sm_mgr->p_polling_sm = NULL; + p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE; + osm_state_mgr_process(p_sm_mgr->p_state_mgr, + OSM_SIGNAL_SWEEP); + break; + case OSM_SM_SIGNAL_HANDOVER_SENT: + /* + * Just sent a HANDOVER signal - move to STANDBY + * start the polling + */ + __osm_sm_state_mgr_standby_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_STANDBY; + __osm_sm_state_mgr_start_polling(p_sm_mgr); + break; + case OSM_SM_SIGNAL_WAIT_FOR_HANDOVER: + /* + * We found a remote master SM, and we are waiting for it + * to handover the mastership to us. Need to start polling + * on that SM, to make sure it is alive, if it isn't - then + * we should move back to discovering, since something must + * have happened to it. + */ + __osm_sm_state_mgr_start_polling(p_sm_mgr); + break; + case OSM_SM_SIGNAL_DISCOVER: + __osm_sm_state_mgr_discovering_msg(p_sm_mgr); + p_sm_mgr->p_subn->sm_state = + IB_SMINFO_STATE_DISCOVERING; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + default: + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "osm_sm_state_mgr_process: ERR 3208: " + "Invalid state %s\n", + osm_get_sm_mgr_state_str(p_sm_mgr->p_subn->sm_state)); + + } + + cl_spinlock_release(&p_sm_mgr->state_lock); + + OSM_LOG_EXIT(p_sm_mgr->p_log); + return (status); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sm_state_mgr_check_legality( - IN osm_sm_state_mgr_t * const p_sm_mgr, - IN osm_sm_signal_t signal ) +osm_sm_state_mgr_check_legality(IN osm_sm_state_mgr_t * const p_sm_mgr, + IN osm_sm_signal_t signal) { - ib_api_status_t status = IB_SUCCESS; - - CL_ASSERT( p_sm_mgr ); - - OSM_LOG_ENTER( p_sm_mgr->p_log, osm_sm_state_mgr_check_legality ); - - /* - * The state lock prevents many race conditions from screwing - * up the state transition process. - */ - cl_spinlock_acquire( &p_sm_mgr->state_lock ); - - if( osm_log_is_active( p_sm_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_sm_mgr->p_log, OSM_LOG_DEBUG, - "osm_sm_state_mgr_check_legality: " - "Received signal %s in state %s\n", - osm_get_sm_mgr_signal_str( signal ), - osm_get_sm_mgr_state_str( p_sm_mgr->p_subn->sm_state ) ); - } - - switch ( p_sm_mgr->p_subn->sm_state ) - { - case IB_SMINFO_STATE_INIT: - switch ( signal ) - { - case OSM_SM_SIGNAL_INIT: - status = IB_SUCCESS; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_DISCOVERING: - switch ( signal ) - { - case OSM_SM_SIGNAL_DISCOVERY_COMPLETED: - case OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE: - case OSM_SM_SIGNAL_HANDOVER: - status = IB_SUCCESS; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_STANDBY: - switch ( signal ) - { - case OSM_SM_SIGNAL_POLLING_TIMEOUT: - case OSM_SM_SIGNAL_DISCOVER: - case OSM_SM_SIGNAL_DISABLE: - case OSM_SM_SIGNAL_HANDOVER: - case OSM_SM_SIGNAL_ACKNOWLEDGE: - status = IB_SUCCESS; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_NOTACTIVE: - switch ( signal ) - { - case OSM_SM_SIGNAL_STANDBY: - status = IB_SUCCESS; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - case IB_SMINFO_STATE_MASTER: - switch ( signal ) - { - case OSM_SM_SIGNAL_HANDOVER: - case OSM_SM_SIGNAL_HANDOVER_SENT: - status = IB_SUCCESS; - break; - default: - __osm_sm_state_mgr_signal_error( p_sm_mgr, signal ); - status = IB_INVALID_PARAMETER; - break; - } - break; - - default: - osm_log( p_sm_mgr->p_log, OSM_LOG_ERROR, - "osm_sm_state_mgr_check_legality: ERR 3209: " - "Invalid state %s\n", - osm_get_sm_mgr_state_str( p_sm_mgr->p_subn->sm_state ) ); - status = IB_INVALID_PARAMETER; - - } - - cl_spinlock_release( &p_sm_mgr->state_lock ); - - OSM_LOG_EXIT( p_sm_mgr->p_log ); - return ( status ); + ib_api_status_t status = IB_SUCCESS; + + CL_ASSERT(p_sm_mgr); + + OSM_LOG_ENTER(p_sm_mgr->p_log, osm_sm_state_mgr_check_legality); + + /* + * The state lock prevents many race conditions from screwing + * up the state transition process. + */ + cl_spinlock_acquire(&p_sm_mgr->state_lock); + + if (osm_log_is_active(p_sm_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_sm_mgr->p_log, OSM_LOG_DEBUG, + "osm_sm_state_mgr_check_legality: " + "Received signal %s in state %s\n", + osm_get_sm_mgr_signal_str(signal), + osm_get_sm_mgr_state_str(p_sm_mgr->p_subn->sm_state)); + } + + switch (p_sm_mgr->p_subn->sm_state) { + case IB_SMINFO_STATE_INIT: + switch (signal) { + case OSM_SM_SIGNAL_INIT: + status = IB_SUCCESS; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_DISCOVERING: + switch (signal) { + case OSM_SM_SIGNAL_DISCOVERY_COMPLETED: + case OSM_SM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED_DONE: + case OSM_SM_SIGNAL_HANDOVER: + status = IB_SUCCESS; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_STANDBY: + switch (signal) { + case OSM_SM_SIGNAL_POLLING_TIMEOUT: + case OSM_SM_SIGNAL_DISCOVER: + case OSM_SM_SIGNAL_DISABLE: + case OSM_SM_SIGNAL_HANDOVER: + case OSM_SM_SIGNAL_ACKNOWLEDGE: + status = IB_SUCCESS; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_NOTACTIVE: + switch (signal) { + case OSM_SM_SIGNAL_STANDBY: + status = IB_SUCCESS; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + case IB_SMINFO_STATE_MASTER: + switch (signal) { + case OSM_SM_SIGNAL_HANDOVER: + case OSM_SM_SIGNAL_HANDOVER_SENT: + status = IB_SUCCESS; + break; + default: + __osm_sm_state_mgr_signal_error(p_sm_mgr, signal); + status = IB_INVALID_PARAMETER; + break; + } + break; + + default: + osm_log(p_sm_mgr->p_log, OSM_LOG_ERROR, + "osm_sm_state_mgr_check_legality: ERR 3209: " + "Invalid state %s\n", + osm_get_sm_mgr_state_str(p_sm_mgr->p_subn->sm_state)); + status = IB_INVALID_PARAMETER; + + } + + cl_spinlock_release(&p_sm_mgr->state_lock); + + OSM_LOG_EXIT(p_sm_mgr->p_log); + return (status); } diff --git a/opensm/opensm/osm_state_mgr_ctrl.c b/opensm/opensm/osm_state_mgr_ctrl.c index 54cfc5b..53868cf 100644 --- a/opensm/opensm/osm_state_mgr_ctrl.c +++ b/opensm/opensm/osm_state_mgr_ctrl.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_state_mgr_ctrl_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -56,72 +55,61 @@ /********************************************************************** **********************************************************************/ -void -__osm_state_mgr_ctrl_disp_callback( - IN void *context, - IN void *p_data ) +void __osm_state_mgr_ctrl_disp_callback(IN void *context, IN void *p_data) { - /* ignore return status when invoked via the dispatcher */ - osm_state_mgr_process( ((osm_state_mgr_ctrl_t*)context)->p_mgr, - (osm_signal_t)(p_data) ); + /* ignore return status when invoked via the dispatcher */ + osm_state_mgr_process(((osm_state_mgr_ctrl_t *) context)->p_mgr, + (osm_signal_t) (p_data)); } /********************************************************************** **********************************************************************/ -void -osm_state_mgr_ctrl_construct( - IN osm_state_mgr_ctrl_t* const p_ctrl ) +void osm_state_mgr_ctrl_construct(IN osm_state_mgr_ctrl_t * const p_ctrl) { - memset( p_ctrl, 0, sizeof(*p_ctrl) ); - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; + memset(p_ctrl, 0, sizeof(*p_ctrl)); + p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** **********************************************************************/ -void -osm_state_mgr_ctrl_destroy( - IN osm_state_mgr_ctrl_t* const p_ctrl ) +void osm_state_mgr_ctrl_destroy(IN osm_state_mgr_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); - cl_disp_unregister( p_ctrl->h_disp ); + CL_ASSERT(p_ctrl); + cl_disp_unregister(p_ctrl->h_disp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_state_mgr_ctrl_init( - IN osm_state_mgr_ctrl_t* const p_ctrl, - IN osm_state_mgr_t* const p_mgr, - IN osm_log_t* const p_log, - IN cl_dispatcher_t* const p_disp ) +osm_state_mgr_ctrl_init(IN osm_state_mgr_ctrl_t * const p_ctrl, + IN osm_state_mgr_t * const p_mgr, + IN osm_log_t * const p_log, + IN cl_dispatcher_t * const p_disp) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_state_mgr_ctrl_init ); + OSM_LOG_ENTER(p_log, osm_state_mgr_ctrl_init); - osm_state_mgr_ctrl_construct( p_ctrl ); - p_ctrl->p_log = p_log; + osm_state_mgr_ctrl_construct(p_ctrl); + p_ctrl->p_log = p_log; - p_ctrl->p_mgr = p_mgr; - p_ctrl->p_disp = p_disp; + p_ctrl->p_mgr = p_mgr; + p_ctrl->p_disp = p_disp; - p_ctrl->h_disp = cl_disp_register( - p_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - __osm_state_mgr_ctrl_disp_callback, - p_ctrl ); + p_ctrl->h_disp = cl_disp_register(p_disp, + OSM_MSG_NO_SMPS_OUTSTANDING, + __osm_state_mgr_ctrl_disp_callback, + p_ctrl); - if( p_ctrl->h_disp == CL_DISP_INVALID_HANDLE ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_state_mgr_ctrl_init: ERR 3401: " - "Dispatcher registration failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } + if (p_ctrl->h_disp == CL_DISP_INVALID_HANDLE) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_state_mgr_ctrl_init: ERR 3401: " + "Dispatcher registration failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + Exit: + OSM_LOG_EXIT(p_log); + return (status); } - diff --git a/opensm/opensm/osm_sw_info_rcv.c b/opensm/opensm/osm_sw_info_rcv.c index 8eb8cd5..a2feadf 100644 --- a/opensm/opensm/osm_sw_info_rcv.c +++ b/opensm/opensm/osm_sw_info_rcv.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -64,74 +64,69 @@ The plock must be held before calling this function. **********************************************************************/ static void -__osm_si_rcv_get_port_info( - IN const osm_si_rcv_t* const p_rcv, - IN osm_switch_t* const p_sw, - IN const osm_madw_t* const p_madw ) +__osm_si_rcv_get_port_info(IN const osm_si_rcv_t * const p_rcv, + IN osm_switch_t * const p_sw, + IN const osm_madw_t * const p_madw) { - osm_madw_context_t context; - uint8_t port_num; - osm_physp_t *p_physp; - osm_node_t *p_node; - uint8_t num_ports; - osm_dr_path_t dr_path; - const ib_smp_t* p_smp; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_si_rcv_get_port_info ); - - CL_ASSERT( p_sw ); - - p_node = p_sw->p_node; - p_smp = osm_madw_get_smp_ptr( p_madw ); - - CL_ASSERT( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ); - - /* - Request PortInfo attribute for each port on the switch. - Don't trust the port's own DR Path, since it may no longer - be a legitimate path through the subnet. - Build a path from the mad instead, since we know that path works. - The port's DR Path info gets updated when the PortInfo - attribute is received. - */ - p_physp = osm_node_get_any_physp_ptr( p_node ); - - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - - context.pi_context.node_guid = osm_node_get_node_guid( p_node ); - context.pi_context.port_guid = osm_physp_get_port_guid( p_physp ); - context.pi_context.set_method = FALSE; - context.pi_context.update_master_sm_base_lid = FALSE; - context.pi_context.ignore_errors = FALSE; - context.pi_context.light_sweep = FALSE; - context.pi_context.active_transition = FALSE; - - num_ports = osm_node_get_num_physp( p_node ); - osm_dr_path_init( &dr_path, - osm_madw_get_bind_handle( p_madw ), - p_smp->hop_count, p_smp->initial_path ); - - for( port_num = 0; port_num < num_ports; port_num++) - { - status = osm_req_get( - p_rcv->p_req, - &dr_path, - IB_MAD_ATTR_PORT_INFO, - cl_hton32( port_num ), - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - { - /* continue the loop despite the error */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_get_port_info: ERR 3602: " - "Failure initiating PortInfo request (%s)\n", - ib_get_err_str(status)); - } - } - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_madw_context_t context; + uint8_t port_num; + osm_physp_t *p_physp; + osm_node_t *p_node; + uint8_t num_ports; + osm_dr_path_t dr_path; + const ib_smp_t *p_smp; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_si_rcv_get_port_info); + + CL_ASSERT(p_sw); + + p_node = p_sw->p_node; + p_smp = osm_madw_get_smp_ptr(p_madw); + + CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); + + /* + Request PortInfo attribute for each port on the switch. + Don't trust the port's own DR Path, since it may no longer + be a legitimate path through the subnet. + Build a path from the mad instead, since we know that path works. + The port's DR Path info gets updated when the PortInfo + attribute is received. + */ + p_physp = osm_node_get_any_physp_ptr(p_node); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + + context.pi_context.node_guid = osm_node_get_node_guid(p_node); + context.pi_context.port_guid = osm_physp_get_port_guid(p_physp); + context.pi_context.set_method = FALSE; + context.pi_context.update_master_sm_base_lid = FALSE; + context.pi_context.ignore_errors = FALSE; + context.pi_context.light_sweep = FALSE; + context.pi_context.active_transition = FALSE; + + num_ports = osm_node_get_num_physp(p_node); + osm_dr_path_init(&dr_path, + osm_madw_get_bind_handle(p_madw), + p_smp->hop_count, p_smp->initial_path); + + for (port_num = 0; port_num < num_ports; port_num++) { + status = osm_req_get(p_rcv->p_req, + &dr_path, + IB_MAD_ATTR_PORT_INFO, + cl_hton32(port_num), + CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) { + /* continue the loop despite the error */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_get_port_info: ERR 3602: " + "Failure initiating PortInfo request (%s)\n", + ib_get_err_str(status)); + } + } + + OSM_LOG_EXIT(p_rcv->p_log); } #if 0 @@ -139,173 +134,159 @@ __osm_si_rcv_get_port_info( The plock must be held before calling this function. **********************************************************************/ static void -__osm_si_rcv_get_fwd_tbl( - IN const osm_si_rcv_t* const p_rcv, - IN osm_switch_t* const p_sw ) +__osm_si_rcv_get_fwd_tbl(IN const osm_si_rcv_t * const p_rcv, + IN osm_switch_t * const p_sw) { - osm_madw_context_t context; - osm_dr_path_t *p_dr_path; - osm_physp_t *p_physp; - osm_node_t *p_node; - uint32_t block_id_ho; - uint32_t max_block_id_ho; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_si_rcv_get_fwd_tbl ); - - CL_ASSERT( p_sw ); - - p_node = p_sw->p_node; - - CL_ASSERT( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ); - - p_physp = osm_node_get_any_physp_ptr( p_node ); - - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - - context.lft_context.node_guid = osm_node_get_node_guid( p_node ); - context.lft_context.set_method = FALSE; - - max_block_id_ho = osm_switch_get_max_block_id_in_use( p_sw ); - - p_dr_path = osm_physp_get_dr_path_ptr( p_physp ); - - for( block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_get_fwd_tbl: " - "Retrieving FT block %u\n", block_id_ho ); - } - - status = osm_req_get( - p_rcv->p_req, - p_dr_path, - IB_MAD_ATTR_LIN_FWD_TBL, - cl_hton32( block_id_ho ), - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - { - /* continue the loop despite the error */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_get_fwd_tbl: ERR 3603: " - "Failure initiating PortInfo request (%s)\n", - ib_get_err_str(status)); - } - } - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_madw_context_t context; + osm_dr_path_t *p_dr_path; + osm_physp_t *p_physp; + osm_node_t *p_node; + uint32_t block_id_ho; + uint32_t max_block_id_ho; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_si_rcv_get_fwd_tbl); + + CL_ASSERT(p_sw); + + p_node = p_sw->p_node; + + CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); + + p_physp = osm_node_get_any_physp_ptr(p_node); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + + context.lft_context.node_guid = osm_node_get_node_guid(p_node); + context.lft_context.set_method = FALSE; + + max_block_id_ho = osm_switch_get_max_block_id_in_use(p_sw); + + p_dr_path = osm_physp_get_dr_path_ptr(p_physp); + + for (block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) { + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_get_fwd_tbl: " + "Retrieving FT block %u\n", block_id_ho); + } + + status = osm_req_get(p_rcv->p_req, + p_dr_path, + IB_MAD_ATTR_LIN_FWD_TBL, + cl_hton32(block_id_ho), + CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) { + /* continue the loop despite the error */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_get_fwd_tbl: ERR 3603: " + "Failure initiating PortInfo request (%s)\n", + ib_get_err_str(status)); + } + } + + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** The plock must be held before calling this function. **********************************************************************/ static void -__osm_si_rcv_get_mcast_fwd_tbl( - IN const osm_si_rcv_t* const p_rcv, - IN osm_switch_t* const p_sw ) +__osm_si_rcv_get_mcast_fwd_tbl(IN const osm_si_rcv_t * const p_rcv, + IN osm_switch_t * const p_sw) { - osm_madw_context_t context; - osm_dr_path_t *p_dr_path; - osm_physp_t *p_physp; - osm_node_t *p_node; - osm_mcast_tbl_t* p_tbl; - uint32_t block_id_ho; - uint32_t max_block_id_ho; - uint32_t position; - uint32_t max_position; - uint32_t attr_mod_ho; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_si_rcv_get_mcast_fwd_tbl ); - - CL_ASSERT( p_sw ); - - p_node = p_sw->p_node; - - CL_ASSERT( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ); - - if( osm_switch_get_mcast_fwd_tbl_size( p_sw ) == 0 ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_get_mcast_fwd_tbl: " - "Multicast not supported by switch 0x%016" PRIx64 "\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - goto Exit; - } - - p_physp = osm_node_get_any_physp_ptr( p_node ); - p_tbl = osm_switch_get_mcast_tbl_ptr( p_sw ); - - CL_ASSERT( osm_physp_is_valid( p_physp ) ); - - context.mft_context.node_guid = osm_node_get_node_guid( p_node ); - context.mft_context.set_method = FALSE; - - max_block_id_ho = osm_mcast_tbl_get_max_block( p_tbl ); - - if( max_block_id_ho > IB_MCAST_MAX_BLOCK_ID ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_get_mcast_fwd_tbl: ERR 3609: " - "Out-of-range mcast block size = %u on switch 0x%016" PRIx64 - "\n", max_block_id_ho, - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - goto Exit; - } - - max_position = osm_mcast_tbl_get_max_position( p_tbl ); - - CL_ASSERT( max_position <= IB_MCAST_POSITION_MAX ); - - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_get_mcast_fwd_tbl: " - "Max MFT block = %u, Max position = %u\n", max_block_id_ho, - max_position ); - - p_dr_path = osm_physp_get_dr_path_ptr( p_physp ); - - for( block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_get_mcast_fwd_tbl: " - "Retrieving MFT block %u\n", block_id_ho ); - } - - for( position = 0; position <= max_position; position++ ) - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_get_mcast_fwd_tbl: " - "Retrieving MFT position %u\n", position ); - } - - attr_mod_ho = block_id_ho | position << IB_MCAST_POSITION_SHIFT; - status = osm_req_get( - p_rcv->p_req, - p_dr_path, - IB_MAD_ATTR_MCAST_FWD_TBL, - cl_hton32( attr_mod_ho ), - CL_DISP_MSGID_NONE, - &context ); - if( status != IB_SUCCESS ) - { - /* continue the loop despite the error */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_get_mcast_fwd_tbl: ERR 3607: " - "Failure initiating PortInfo request (%s)\n", - ib_get_err_str(status)); - } - } - } - - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_madw_context_t context; + osm_dr_path_t *p_dr_path; + osm_physp_t *p_physp; + osm_node_t *p_node; + osm_mcast_tbl_t *p_tbl; + uint32_t block_id_ho; + uint32_t max_block_id_ho; + uint32_t position; + uint32_t max_position; + uint32_t attr_mod_ho; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_si_rcv_get_mcast_fwd_tbl); + + CL_ASSERT(p_sw); + + p_node = p_sw->p_node; + + CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); + + if (osm_switch_get_mcast_fwd_tbl_size(p_sw) == 0) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_get_mcast_fwd_tbl: " + "Multicast not supported by switch 0x%016" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + goto Exit; + } + + p_physp = osm_node_get_any_physp_ptr(p_node); + p_tbl = osm_switch_get_mcast_tbl_ptr(p_sw); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + + context.mft_context.node_guid = osm_node_get_node_guid(p_node); + context.mft_context.set_method = FALSE; + + max_block_id_ho = osm_mcast_tbl_get_max_block(p_tbl); + + if (max_block_id_ho > IB_MCAST_MAX_BLOCK_ID) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_get_mcast_fwd_tbl: ERR 3609: " + "Out-of-range mcast block size = %u on switch 0x%016" + PRIx64 "\n", max_block_id_ho, + cl_ntoh64(osm_node_get_node_guid(p_node))); + goto Exit; + } + + max_position = osm_mcast_tbl_get_max_position(p_tbl); + + CL_ASSERT(max_position <= IB_MCAST_POSITION_MAX); + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_get_mcast_fwd_tbl: " + "Max MFT block = %u, Max position = %u\n", max_block_id_ho, + max_position); + + p_dr_path = osm_physp_get_dr_path_ptr(p_physp); + + for (block_id_ho = 0; block_id_ho <= max_block_id_ho; block_id_ho++) { + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_get_mcast_fwd_tbl: " + "Retrieving MFT block %u\n", block_id_ho); + } + + for (position = 0; position <= max_position; position++) { + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_get_mcast_fwd_tbl: " + "Retrieving MFT position %u\n", + position); + } + + attr_mod_ho = + block_id_ho | position << IB_MCAST_POSITION_SHIFT; + status = + osm_req_get(p_rcv->p_req, p_dr_path, + IB_MAD_ATTR_MCAST_FWD_TBL, + cl_hton32(attr_mod_ho), + CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) { + /* continue the loop despite the error */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_get_mcast_fwd_tbl: ERR 3607: " + "Failure initiating PortInfo request (%s)\n", + ib_get_err_str(status)); + } + } + } + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } #endif @@ -313,112 +294,109 @@ __osm_si_rcv_get_mcast_fwd_tbl( Lock must be held on entry to this function. **********************************************************************/ static void -__osm_si_rcv_process_new( - IN const osm_si_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_si_rcv_process_new(IN const osm_si_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - osm_switch_t *p_sw; - osm_switch_t *p_check; - ib_switch_info_t *p_si; - ib_smp_t *p_smp; - cl_qmap_t *p_sw_guid_tbl; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, __osm_si_rcv_process_new ); - - CL_ASSERT( p_madw ); - - p_sw_guid_tbl = &p_rcv->p_subn->sw_guid_tbl; - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_si = (ib_switch_info_t*)ib_smp_get_payload_ptr( p_smp ); - - osm_dump_switch_info( p_rcv->p_log, p_si, OSM_LOG_DEBUG ); - - /* - Allocate a new switch object for this switch, - and place it in the switch table. - */ - p_sw = osm_switch_new( p_node, p_madw ); - if( p_sw == NULL ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_process_new: ERR 3608: " - "Unable to allocate new switch object\n" ); - goto Exit; - } - - /* set subnet max mlid to the minimum MulticastFDBCap of all switches */ - if ( p_sw->mcast_tbl.max_mlid_ho < p_rcv->p_subn->max_multicast_lid_ho ) - { - p_rcv->p_subn->max_multicast_lid_ho = p_sw->mcast_tbl.max_mlid_ho; - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_si_rcv_process_new: " - "Subnet max multicast lid is 0x%X\n", - p_rcv->p_subn->max_multicast_lid_ho ); - } - - /* set subnet max unicast lid to the minimum LinearFDBCap of all switches */ - if ( p_sw->fwd_tbl.p_lin_tbl->size < p_rcv->p_subn->max_unicast_lid_ho ) - { - p_rcv->p_subn->max_unicast_lid_ho = p_sw->fwd_tbl.p_lin_tbl->size; - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_si_rcv_process_new: " - "Subnet max unicast lid is 0x%X\n", - p_rcv->p_subn->max_unicast_lid_ho ); - } - - p_check = (osm_switch_t*)cl_qmap_insert( p_sw_guid_tbl, - osm_node_get_node_guid( p_node ), - &p_sw->map_item ); - - if( p_check != p_sw ) - { - /* - This shouldn't happen since we hold the lock! - */ - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "__osm_si_rcv_process_new: ERR 3605: " - "Unable to add new switch object to database\n" ); - osm_switch_delete( &p_sw ); - goto Exit; - } - - p_node->sw = p_sw; - - /* - Update the switch info according to the - info we just received. - */ - osm_switch_set_switch_info( p_sw, p_si ); - p_sw->discovery_count++; - - /* - Get the PortInfo attribute for every port. - */ - __osm_si_rcv_get_port_info( p_rcv, p_sw, p_madw ); - - /* - Don't bother retrieving the current unicast and multicast tables - from the switches. The current version of SM does - not support silent take-over of an existing multicast - configuration. - - Gathering the multicast tables can also generate large amounts - of extra subnet-init traffic. - - The code to retrieve the tables was fully debugged. - */ + osm_switch_t *p_sw; + osm_switch_t *p_check; + ib_switch_info_t *p_si; + ib_smp_t *p_smp; + cl_qmap_t *p_sw_guid_tbl; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, __osm_si_rcv_process_new); + + CL_ASSERT(p_madw); + + p_sw_guid_tbl = &p_rcv->p_subn->sw_guid_tbl; + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_si = (ib_switch_info_t *) ib_smp_get_payload_ptr(p_smp); + + osm_dump_switch_info(p_rcv->p_log, p_si, OSM_LOG_DEBUG); + + /* + Allocate a new switch object for this switch, + and place it in the switch table. + */ + p_sw = osm_switch_new(p_node, p_madw); + if (p_sw == NULL) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_process_new: ERR 3608: " + "Unable to allocate new switch object\n"); + goto Exit; + } + + /* set subnet max mlid to the minimum MulticastFDBCap of all switches */ + if (p_sw->mcast_tbl.max_mlid_ho < p_rcv->p_subn->max_multicast_lid_ho) { + p_rcv->p_subn->max_multicast_lid_ho = + p_sw->mcast_tbl.max_mlid_ho; + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_si_rcv_process_new: " + "Subnet max multicast lid is 0x%X\n", + p_rcv->p_subn->max_multicast_lid_ho); + } + + /* set subnet max unicast lid to the minimum LinearFDBCap of all switches */ + if (p_sw->fwd_tbl.p_lin_tbl->size < p_rcv->p_subn->max_unicast_lid_ho) { + p_rcv->p_subn->max_unicast_lid_ho = + p_sw->fwd_tbl.p_lin_tbl->size; + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_si_rcv_process_new: " + "Subnet max unicast lid is 0x%X\n", + p_rcv->p_subn->max_unicast_lid_ho); + } + + p_check = (osm_switch_t *) cl_qmap_insert(p_sw_guid_tbl, + osm_node_get_node_guid + (p_node), &p_sw->map_item); + + if (p_check != p_sw) { + /* + This shouldn't happen since we hold the lock! + */ + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "__osm_si_rcv_process_new: ERR 3605: " + "Unable to add new switch object to database\n"); + osm_switch_delete(&p_sw); + goto Exit; + } + + p_node->sw = p_sw; + + /* + Update the switch info according to the + info we just received. + */ + osm_switch_set_switch_info(p_sw, p_si); + p_sw->discovery_count++; + + /* + Get the PortInfo attribute for every port. + */ + __osm_si_rcv_get_port_info(p_rcv, p_sw, p_madw); + + /* + Don't bother retrieving the current unicast and multicast tables + from the switches. The current version of SM does + not support silent take-over of an existing multicast + configuration. + + Gathering the multicast tables can also generate large amounts + of extra subnet-init traffic. + + The code to retrieve the tables was fully debugged. + */ #if 0 - __osm_si_rcv_get_fwd_tbl( p_rcv, p_sw ); - if( !p_rcv->p_subn->opt.disable_multicast ) - __osm_si_rcv_get_mcast_fwd_tbl( p_rcv, p_sw ); + __osm_si_rcv_get_fwd_tbl(p_rcv, p_sw); + if (!p_rcv->p_subn->opt.disable_multicast) + __osm_si_rcv_get_mcast_fwd_tbl(p_rcv, p_sw); #endif - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** @@ -427,255 +405,223 @@ __osm_si_rcv_process_new( this can not be done internally as the event needs the lock... **********************************************************************/ static boolean_t -__osm_si_rcv_process_existing( - IN const osm_si_rcv_t* const p_rcv, - IN osm_node_t* const p_node, - IN const osm_madw_t* const p_madw ) +__osm_si_rcv_process_existing(IN const osm_si_rcv_t * const p_rcv, + IN osm_node_t * const p_node, + IN const osm_madw_t * const p_madw) { - osm_switch_t *p_sw = p_node->sw; - ib_switch_info_t *p_si; - osm_si_context_t *p_si_context; - ib_smp_t *p_smp; - boolean_t is_change_detected = FALSE; - - OSM_LOG_ENTER( p_rcv->p_log, __osm_si_rcv_process_existing ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_si = (ib_switch_info_t*)ib_smp_get_payload_ptr( p_smp ); - p_si_context = osm_madw_get_si_context_ptr( p_madw ); - - if( p_si_context->set_method ) - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_process_existing: " - "Received logical SetResp()\n" ); - } - - osm_switch_set_switch_info( p_sw, p_si ); - } - else - { - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_process_existing: " - "Received logical GetResp()\n" ); - } - - osm_switch_set_switch_info( p_sw, p_si ); - - /* - Check the port state change bit. If true, then this switch - has seen a port state transition, so continue probing. - */ - if( p_si_context->light_sweep == TRUE ) - { - /* This is a light sweep */ - /* If the mad was returned with an error - - signal a change to the state manager. */ - if ( ib_smp_get_status( p_smp ) != 0 ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_si_rcv_process_existing: " - "GetResp() received with error in light sweep. " - "Commencing heavy sweep\n" ); - is_change_detected = TRUE; - } - else - { - /* - If something changed, then just signal the state - manager. Don't attempt to probe further during - a light sweep. - */ - if( ib_switch_info_get_state_change( p_si ) ) - { - osm_dump_switch_info( p_rcv->p_log, p_si, OSM_LOG_DEBUG ); - is_change_detected = TRUE; - } - } - } - else - { - /* - This is a heavy sweep. Get information regardless - of the state change bit. - */ - p_sw->discovery_count++; - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "__osm_si_rcv_process_existing: " - "discovery_count is:%u\n", - p_sw->discovery_count ); - - /* If this is the first discovery - then get the port_info */ - if ( p_sw->discovery_count == 1 ) - __osm_si_rcv_get_port_info( p_rcv, p_sw, p_madw ); - else - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "__osm_si_rcv_process_existing: " - "Not discovering again through switch:0x%" - PRIx64 "\n", - osm_node_get_node_guid( p_sw->p_node) ); - } - } - } - - OSM_LOG_EXIT( p_rcv->p_log ); - return is_change_detected; + osm_switch_t *p_sw = p_node->sw; + ib_switch_info_t *p_si; + osm_si_context_t *p_si_context; + ib_smp_t *p_smp; + boolean_t is_change_detected = FALSE; + + OSM_LOG_ENTER(p_rcv->p_log, __osm_si_rcv_process_existing); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_si = (ib_switch_info_t *) ib_smp_get_payload_ptr(p_smp); + p_si_context = osm_madw_get_si_context_ptr(p_madw); + + if (p_si_context->set_method) { + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_process_existing: " + "Received logical SetResp()\n"); + } + + osm_switch_set_switch_info(p_sw, p_si); + } else { + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_process_existing: " + "Received logical GetResp()\n"); + } + + osm_switch_set_switch_info(p_sw, p_si); + + /* + Check the port state change bit. If true, then this switch + has seen a port state transition, so continue probing. + */ + if (p_si_context->light_sweep == TRUE) { + /* This is a light sweep */ + /* If the mad was returned with an error - + signal a change to the state manager. */ + if (ib_smp_get_status(p_smp) != 0) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_si_rcv_process_existing: " + "GetResp() received with error in light sweep. " + "Commencing heavy sweep\n"); + is_change_detected = TRUE; + } else { + /* + If something changed, then just signal the state + manager. Don't attempt to probe further during + a light sweep. + */ + if (ib_switch_info_get_state_change(p_si)) { + osm_dump_switch_info(p_rcv->p_log, p_si, + OSM_LOG_DEBUG); + is_change_detected = TRUE; + } + } + } else { + /* + This is a heavy sweep. Get information regardless + of the state change bit. + */ + p_sw->discovery_count++; + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "__osm_si_rcv_process_existing: " + "discovery_count is:%u\n", + p_sw->discovery_count); + + /* If this is the first discovery - then get the port_info */ + if (p_sw->discovery_count == 1) + __osm_si_rcv_get_port_info(p_rcv, p_sw, p_madw); + else { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "__osm_si_rcv_process_existing: " + "Not discovering again through switch:0x%" + PRIx64 "\n", + osm_node_get_node_guid(p_sw->p_node)); + } + } + } + + OSM_LOG_EXIT(p_rcv->p_log); + return is_change_detected; } /********************************************************************** **********************************************************************/ -void -osm_si_rcv_construct( - IN osm_si_rcv_t* const p_rcv ) +void osm_si_rcv_construct(IN osm_si_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_si_rcv_destroy( - IN osm_si_rcv_t* const p_rcv ) +void osm_si_rcv_destroy(IN osm_si_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_si_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_si_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_si_rcv_init( - IN osm_si_rcv_t* const p_rcv, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN osm_req_t* const p_req, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_plock_t* const p_lock ) +osm_si_rcv_init(IN osm_si_rcv_t * const p_rcv, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, + IN osm_req_t * const p_req, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_si_rcv_init ); + ib_api_status_t status = IB_SUCCESS; + OSM_LOG_ENTER(p_log, osm_si_rcv_init); - osm_si_rcv_construct( p_rcv ); + osm_si_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_req = p_req; - p_rcv->p_state_mgr = p_state_mgr; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_req = p_req; + p_rcv->p_state_mgr = p_state_mgr; - OSM_LOG_EXIT( p_rcv->p_log ); - return( status ); + OSM_LOG_EXIT(p_rcv->p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_si_rcv_process( - IN void *context, - IN void *data ) +void osm_si_rcv_process(IN void *context, IN void *data) { - osm_si_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_switch_info_t *p_si; - ib_smp_t *p_smp; - osm_node_t *p_node; - ib_net64_t node_guid; - osm_si_context_t *p_context; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_si_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - p_si = (ib_switch_info_t*)ib_smp_get_payload_ptr( p_smp ); - - /* - Acquire the switch object and add the switch info. - */ - - p_context = osm_madw_get_si_context_ptr( p_madw ); - - node_guid = p_context->node_guid; - - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_DEBUG, - "osm_si_rcv_process: " - "Switch GUID 0x%016" PRIx64 - ", TID 0x%" PRIx64 "\n", - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - } - - CL_PLOCK_EXCL_ACQUIRE( p_rcv->p_lock ); - - p_node = osm_get_node_by_guid( p_rcv->p_subn, node_guid ); - if( !p_node ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_si_rcv_process: ERR 3606: " - "SwitchInfo received for nonexistent node " - "with GUID 0x%" PRIx64 "\n", - cl_ntoh64( node_guid ) ); - } - else - { - - /* - Hack for bad value in Mellanox switch - */ - if( cl_ntoh16( p_si->lin_top ) > IB_LID_UCAST_END_HO ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_si_rcv_process: ERR 3610: " - "\n\t\t\t\tBad LinearFDBTop value = 0x%X " - "on switch 0x%" PRIx64 - "\n\t\t\t\tForcing correction to 0x%X\n", - cl_ntoh16( p_si->lin_top ), - cl_ntoh64( osm_node_get_node_guid( p_node ) ), - 0 ); - - p_si->lin_top = 0; - } - - /* - Acquire the switch object for this switch. - */ - if( !p_node->sw ) - { - __osm_si_rcv_process_new( p_rcv, p_node, p_madw ); - /* - A new switch was found during the sweep so we need - to ignore the current LFT settings. - */ - p_rcv->p_subn->ignore_existing_lfts = TRUE; - } - else - { - /* we might get back a request for signaling change was detected */ - if (__osm_si_rcv_process_existing( p_rcv, p_node, p_madw )) - { - CL_PLOCK_RELEASE( p_rcv->p_lock ); - osm_state_mgr_process( p_rcv->p_state_mgr, - OSM_SIGNAL_CHANGE_DETECTED ); - goto Exit; - } - } - } - - CL_PLOCK_RELEASE( p_rcv->p_lock ); - Exit: - OSM_LOG_EXIT( p_rcv->p_log ); + osm_si_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_switch_info_t *p_si; + ib_smp_t *p_smp; + osm_node_t *p_node; + ib_net64_t node_guid; + osm_si_context_t *p_context; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_si_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_si = (ib_switch_info_t *) ib_smp_get_payload_ptr(p_smp); + + /* + Acquire the switch object and add the switch info. + */ + + p_context = osm_madw_get_si_context_ptr(p_madw); + + node_guid = p_context->node_guid; + + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_DEBUG)) { + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_si_rcv_process: " + "Switch GUID 0x%016" PRIx64 + ", TID 0x%" PRIx64 "\n", + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + } + + CL_PLOCK_EXCL_ACQUIRE(p_rcv->p_lock); + + p_node = osm_get_node_by_guid(p_rcv->p_subn, node_guid); + if (!p_node) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_si_rcv_process: ERR 3606: " + "SwitchInfo received for nonexistent node " + "with GUID 0x%" PRIx64 "\n", cl_ntoh64(node_guid)); + } else { + + /* + Hack for bad value in Mellanox switch + */ + if (cl_ntoh16(p_si->lin_top) > IB_LID_UCAST_END_HO) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_si_rcv_process: ERR 3610: " + "\n\t\t\t\tBad LinearFDBTop value = 0x%X " + "on switch 0x%" PRIx64 + "\n\t\t\t\tForcing correction to 0x%X\n", + cl_ntoh16(p_si->lin_top), + cl_ntoh64(osm_node_get_node_guid(p_node)), 0); + + p_si->lin_top = 0; + } + + /* + Acquire the switch object for this switch. + */ + if (!p_node->sw) { + __osm_si_rcv_process_new(p_rcv, p_node, p_madw); + /* + A new switch was found during the sweep so we need + to ignore the current LFT settings. + */ + p_rcv->p_subn->ignore_existing_lfts = TRUE; + } else { + /* we might get back a request for signaling change was detected */ + if (__osm_si_rcv_process_existing + (p_rcv, p_node, p_madw)) { + CL_PLOCK_RELEASE(p_rcv->p_lock); + osm_state_mgr_process(p_rcv->p_state_mgr, + OSM_SIGNAL_CHANGE_DETECTED); + goto Exit; + } + } + } + + CL_PLOCK_RELEASE(p_rcv->p_lock); + Exit: + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/osm_sweep_fail_ctrl.c b/opensm/opensm/osm_sweep_fail_ctrl.c index 41e2fbf..1ddb4f2 100644 --- a/opensm/opensm/osm_sweep_fail_ctrl.c +++ b/opensm/opensm/osm_sweep_fail_ctrl.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_sweep_fail_ctrl_t. @@ -46,7 +45,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -54,80 +53,68 @@ /********************************************************************** **********************************************************************/ -void -__osm_sweep_fail_ctrl_disp_callback( - IN void *context, - IN void *p_data ) +void __osm_sweep_fail_ctrl_disp_callback(IN void *context, IN void *p_data) { - osm_sweep_fail_ctrl_t* const p_ctrl = (osm_sweep_fail_ctrl_t*)context; + osm_sweep_fail_ctrl_t *const p_ctrl = (osm_sweep_fail_ctrl_t *) context; - OSM_LOG_ENTER( p_ctrl->p_log, __osm_sweep_fail_ctrl_disp_callback ); + OSM_LOG_ENTER(p_ctrl->p_log, __osm_sweep_fail_ctrl_disp_callback); - UNUSED_PARAM( p_data ); - /* - Notify the state manager that we had a light sweep failure. - */ - osm_state_mgr_process( p_ctrl->p_state_mgr, - OSM_SIGNAL_LIGHT_SWEEP_FAIL ); + UNUSED_PARAM(p_data); + /* + Notify the state manager that we had a light sweep failure. + */ + osm_state_mgr_process(p_ctrl->p_state_mgr, OSM_SIGNAL_LIGHT_SWEEP_FAIL); - OSM_LOG_EXIT( p_ctrl->p_log ); + OSM_LOG_EXIT(p_ctrl->p_log); } /********************************************************************** **********************************************************************/ -void -osm_sweep_fail_ctrl_construct( - IN osm_sweep_fail_ctrl_t* const p_ctrl ) +void osm_sweep_fail_ctrl_construct(IN osm_sweep_fail_ctrl_t * const p_ctrl) { - memset( p_ctrl, 0, sizeof(*p_ctrl) ); - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; + memset(p_ctrl, 0, sizeof(*p_ctrl)); + p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** **********************************************************************/ -void -osm_sweep_fail_ctrl_destroy( - IN osm_sweep_fail_ctrl_t* const p_ctrl ) +void osm_sweep_fail_ctrl_destroy(IN osm_sweep_fail_ctrl_t * const p_ctrl) { - CL_ASSERT( p_ctrl ); - cl_disp_unregister( p_ctrl->h_disp ); + CL_ASSERT(p_ctrl); + cl_disp_unregister(p_ctrl->h_disp); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_sweep_fail_ctrl_init( - IN osm_sweep_fail_ctrl_t* const p_ctrl, - IN osm_log_t* const p_log, - IN osm_state_mgr_t* const p_state_mgr, - IN cl_dispatcher_t* const p_disp ) +osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, + IN osm_log_t * const p_log, + IN osm_state_mgr_t * const p_state_mgr, + IN cl_dispatcher_t * const p_disp) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_sweep_fail_ctrl_init ); - - osm_sweep_fail_ctrl_construct( p_ctrl ); - p_ctrl->p_log = p_log; - p_ctrl->p_disp = p_disp; - p_ctrl->p_state_mgr = p_state_mgr; - - p_ctrl->h_disp = cl_disp_register( - p_disp, - OSM_MSG_LIGHT_SWEEP_FAIL, - __osm_sweep_fail_ctrl_disp_callback, - p_ctrl ); - - if( p_ctrl->h_disp == CL_DISP_INVALID_HANDLE ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_sweep_fail_ctrl_init: ERR 3501: " - "Dispatcher registration failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_sweep_fail_ctrl_init); + + osm_sweep_fail_ctrl_construct(p_ctrl); + p_ctrl->p_log = p_log; + p_ctrl->p_disp = p_disp; + p_ctrl->p_state_mgr = p_state_mgr; + + p_ctrl->h_disp = cl_disp_register(p_disp, + OSM_MSG_LIGHT_SWEEP_FAIL, + __osm_sweep_fail_ctrl_disp_callback, + p_ctrl); + + if (p_ctrl->h_disp == CL_DISP_INVALID_HANDLE) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_sweep_fail_ctrl_init: ERR 3501: " + "Dispatcher registration failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } - diff --git a/opensm/opensm/osm_ucast_file.c b/opensm/opensm/osm_ucast_file.c index 97be7ea..9928831 100644 --- a/opensm/opensm/osm_ucast_file.c +++ b/opensm/opensm/osm_ucast_file.c @@ -56,8 +56,7 @@ #include #include - -static uint16_t remap_lid(osm_opensm_t *p_osm, uint16_t lid, ib_net64_t guid) +static uint16_t remap_lid(osm_opensm_t * p_osm, uint16_t lid, ib_net64_t guid) { osm_port_t *p_port; uint16_t min_lid, max_lid; @@ -108,7 +107,7 @@ static void add_path(osm_opensm_t * p_osm, cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); } -static void add_lid_hops(osm_opensm_t *p_osm, osm_switch_t *p_sw, +static void add_lid_hops(osm_opensm_t * p_osm, osm_switch_t * p_sw, uint16_t lid, ib_net64_t guid, uint8_t hops[], unsigned len) { @@ -119,7 +118,7 @@ static void add_lid_hops(osm_opensm_t *p_osm, osm_switch_t *p_sw, if (len > p_sw->num_ports) len = p_sw->num_ports; - for (i = 0 ; i < len ; i++) + for (i = 0; i < len; i++) osm_switch_set_hops(p_sw, lid, i, hops[i]); } @@ -137,7 +136,7 @@ static int do_ucast_file_load(void *context) file_name = p_osm->subn.opt.ucast_dump_file; if (!file_name) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "do_ucast_file_load: ERR 6301: " "ucast dump file name is not defined; " "using default routing algorithm\n"); @@ -146,7 +145,7 @@ static int do_ucast_file_load(void *context) file = fopen(file_name, "r"); if (!file) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "do_ucast_file_load: ERR 6302: " "cannot open ucast dump file \'%s\'; " "using default routing algorithm\n", file_name); @@ -168,14 +167,15 @@ static int do_ucast_file_load(void *context) continue; if (!strncmp(p, "Multicast mlids", 15)) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "do_ucast_file_load: ERR 6303: " "Multicast dump file detected; " "skipping parsing. Using default " "routing algorithm\n"); } else if (!strncmp(p, "Unicast lids", 12)) { if (p_sw) - osm_ucast_mgr_set_fwd_table(&p_osm->sm.ucast_mgr, p_sw); + osm_ucast_mgr_set_fwd_table(&p_osm->sm. + ucast_mgr, p_sw); q = strstr(p, " guid 0x"); if (!q) { osm_log(&p_osm->log, OSM_LOG_ERROR, @@ -203,10 +203,11 @@ static int do_ucast_file_load(void *context) cl_ntoh64(sw_guid)); continue; } - memset(p_osm->sm.ucast_mgr.lft_buf, 0xff, IB_LID_UCAST_END_HO + 1); + memset(p_osm->sm.ucast_mgr.lft_buf, 0xff, + IB_LID_UCAST_END_HO + 1); } else if (p_sw && !strncmp(p, "0x", 2)) { p += 2; - lid = (uint16_t)strtoul(p, &q, 16); + lid = (uint16_t) strtoul(p, &q, 16); if (q == p || !isspace(*q)) { osm_log(&p_osm->log, OSM_LOG_ERROR, "PARSE ERROR: %s:%u: " @@ -217,7 +218,7 @@ static int do_ucast_file_load(void *context) p = q; while (isspace(*p)) p++; - port_num = (uint8_t)strtoul(p, &q, 10); + port_num = (uint8_t) strtoul(p, &q, 10); if (q == p || !isspace(*q)) { osm_log(&p_osm->log, OSM_LOG_ERROR, "PARSE ERROR: %s:%u: " @@ -235,8 +236,7 @@ static int do_ucast_file_load(void *context) "(maybe broken dump): \'%s\'\n", file_name, lineno, p); port_guid = 0; - } - else { + } else { p = q + 12; port_guid = strtoull(p, &q, 16); if (q == p || (!isspace(*q) && *q != ':')) { @@ -274,7 +274,7 @@ static int do_lid_matrix_file_load(void *context) file_name = p_osm->subn.opt.lid_matrix_dump_file; if (!file_name) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "do_lid_matrix_file_load: ERR 6304: " "lid matrix file name is not defined; " "using default lid matrix generation algorithm\n"); @@ -283,7 +283,7 @@ static int do_lid_matrix_file_load(void *context) file = fopen(file_name, "r"); if (!file) { - osm_log(&p_osm->log, OSM_LOG_ERROR|OSM_LOG_SYS, + osm_log(&p_osm->log, OSM_LOG_ERROR | OSM_LOG_SYS, "do_lid_matrix_file_load: ERR 6305: " "cannot open lid matrix file \'%s\'; " "using default lid matrix generation algorithm\n", @@ -350,7 +350,7 @@ static int do_lid_matrix_file_load(void *context) return -1; } /* Just checked the range, so casting is safe */ - lid = (uint16_t)num; + lid = (uint16_t) num; p = q; while (isspace(*p) || *p == ':') p++; @@ -358,13 +358,13 @@ static int do_lid_matrix_file_load(void *context) num = strtoul(p, &q, 16); if (num > 0xff || q == p) { osm_log(&p_osm->log, OSM_LOG_ERROR, - "PARSE ERROR: %s:%u: " - "cannot parse hops number: \'%s\'\n", - file_name, lineno, p); + "PARSE ERROR: %s:%u: " + "cannot parse hops number: \'%s\'\n", + file_name, lineno, p); return -1; } /* Just checked the range, so casting is safe */ - hops[len++] = (uint8_t)num; + hops[len++] = (uint8_t) num; p = q; while (isspace(*p)) p++; @@ -378,8 +378,7 @@ static int do_lid_matrix_file_load(void *context) "(maybe broken dump): \'%s\'\n", file_name, lineno, p); guid = 0; - } - else { + } else { p = q + 12; guid = strtoull(p, &q, 16); if (q == p || !isspace(*q)) { diff --git a/opensm/opensm/osm_ucast_mgr.c b/opensm/opensm/osm_ucast_mgr.c index b90509a..2a5fe88 100644 --- a/opensm/opensm/osm_ucast_mgr.c +++ b/opensm/opensm/osm_ucast_mgr.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -64,799 +64,761 @@ /********************************************************************** **********************************************************************/ -void -osm_ucast_mgr_construct( - IN osm_ucast_mgr_t* const p_mgr ) +void osm_ucast_mgr_construct(IN osm_ucast_mgr_t * const p_mgr) { - memset( p_mgr, 0, sizeof(*p_mgr) ); + memset(p_mgr, 0, sizeof(*p_mgr)); } /********************************************************************** **********************************************************************/ -void -osm_ucast_mgr_destroy( - IN osm_ucast_mgr_t* const p_mgr ) +void osm_ucast_mgr_destroy(IN osm_ucast_mgr_t * const p_mgr) { - CL_ASSERT( p_mgr ); + CL_ASSERT(p_mgr); - OSM_LOG_ENTER( p_mgr->p_log, osm_ucast_mgr_destroy ); + OSM_LOG_ENTER(p_mgr->p_log, osm_ucast_mgr_destroy); - if (p_mgr->lft_buf) - free(p_mgr->lft_buf); + if (p_mgr->lft_buf) + free(p_mgr->lft_buf); - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_ucast_mgr_init( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_ucast_mgr_init(IN osm_ucast_mgr_t * const p_mgr, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_ucast_mgr_init ); + OSM_LOG_ENTER(p_log, osm_ucast_mgr_init); - CL_ASSERT( p_req ); - CL_ASSERT( p_subn ); - CL_ASSERT( p_lock ); + CL_ASSERT(p_req); + CL_ASSERT(p_subn); + CL_ASSERT(p_lock); - osm_ucast_mgr_construct( p_mgr ); + osm_ucast_mgr_construct(p_mgr); - p_mgr->p_log = p_log; - p_mgr->p_subn = p_subn; - p_mgr->p_lock = p_lock; - p_mgr->p_req = p_req; + p_mgr->p_log = p_log; + p_mgr->p_subn = p_subn; + p_mgr->p_lock = p_lock; + p_mgr->p_req = p_req; - p_mgr->lft_buf = malloc(IB_LID_UCAST_END_HO + 1); - if (!p_mgr->lft_buf) - return IB_INSUFFICIENT_MEMORY; + p_mgr->lft_buf = malloc(IB_LID_UCAST_END_HO + 1); + if (!p_mgr->lft_buf) + return IB_INSUFFICIENT_MEMORY; - OSM_LOG_EXIT( p_mgr->p_log ); - return( status ); + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } /********************************************************************** Add each switch's own and neighbor LIDs to its LID matrix **********************************************************************/ static void -__osm_ucast_mgr_process_hop_0_1( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_ucast_mgr_process_hop_0_1(IN cl_map_item_t * const p_map_item, + IN void *context) { - osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - osm_node_t *p_remote_node; - uint16_t lid, remote_lid; - uint8_t i, remote_port; - - lid = osm_node_get_base_lid(p_sw->p_node, 0); - lid = cl_ntoh16(lid); - osm_switch_set_hops(p_sw, lid, 0, 0); - - for( i = 1; i < p_sw->num_ports; i++ ) - { - p_remote_node = osm_node_get_remote_node( p_sw->p_node, i, &remote_port ); - - if( p_remote_node && p_remote_node->sw && - (p_remote_node != p_sw->p_node ) ) - { - remote_lid = osm_node_get_base_lid(p_remote_node, 0); - remote_lid = cl_ntoh16(remote_lid); - osm_switch_set_hops( p_sw, remote_lid, i, 1 ); - osm_switch_set_hops( p_remote_node->sw, lid, remote_port, 1 ); - } - } + osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + osm_node_t *p_remote_node; + uint16_t lid, remote_lid; + uint8_t i, remote_port; + + lid = osm_node_get_base_lid(p_sw->p_node, 0); + lid = cl_ntoh16(lid); + osm_switch_set_hops(p_sw, lid, 0, 0); + + for (i = 1; i < p_sw->num_ports; i++) { + p_remote_node = + osm_node_get_remote_node(p_sw->p_node, i, &remote_port); + + if (p_remote_node && p_remote_node->sw && + (p_remote_node != p_sw->p_node)) { + remote_lid = osm_node_get_base_lid(p_remote_node, 0); + remote_lid = cl_ntoh16(remote_lid); + osm_switch_set_hops(p_sw, remote_lid, i, 1); + osm_switch_set_hops(p_remote_node->sw, lid, remote_port, + 1); + } + } } /********************************************************************** **********************************************************************/ static void -__osm_ucast_mgr_process_neighbor( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_switch_t* const p_this_sw, - IN osm_switch_t* const p_remote_sw, - IN const uint8_t port_num, - IN const uint8_t remote_port_num ) +__osm_ucast_mgr_process_neighbor(IN osm_ucast_mgr_t * const p_mgr, + IN osm_switch_t * const p_this_sw, + IN osm_switch_t * const p_remote_sw, + IN const uint8_t port_num, + IN const uint8_t remote_port_num) { - osm_switch_t *p_sw, *p_next_sw; - uint16_t lid_ho; - uint8_t hops; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_ucast_mgr_process_neighbor ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_neighbor: " - "Node 0x%" PRIx64 ", remote node 0x%" PRIx64 - ", port 0x%X, remote port 0x%X\n", - cl_ntoh64( osm_node_get_node_guid( p_this_sw->p_node ) ), - cl_ntoh64( osm_node_get_node_guid( p_remote_sw->p_node ) ), - port_num, remote_port_num ); - } - - p_next_sw = (osm_switch_t*)cl_qmap_head( &p_mgr->p_subn->sw_guid_tbl ); - while( p_next_sw != (osm_switch_t*)cl_qmap_end( &p_mgr->p_subn->sw_guid_tbl ) ) - { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - lid_ho = osm_node_get_base_lid(p_sw->p_node, 0); - lid_ho = cl_ntoh16(lid_ho); - hops = osm_switch_get_least_hops(p_remote_sw, lid_ho); - if (hops == OSM_NO_PATH) - continue; - hops++; - if (hops < osm_switch_get_hop_count(p_this_sw, lid_ho, port_num)) - { - if (osm_switch_set_hops(p_this_sw, lid_ho, port_num, hops) != 0) - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_ucast_mgr_process_neighbor: " - "cannot set hops for lid %u at switch 0x%" PRIx64 "\n", - lid_ho, - cl_ntoh64(osm_node_get_node_guid(p_this_sw->p_node))); - p_mgr->some_hop_count_set = TRUE; - } - } - - OSM_LOG_EXIT( p_mgr->p_log ); + osm_switch_t *p_sw, *p_next_sw; + uint16_t lid_ho; + uint8_t hops; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_ucast_mgr_process_neighbor); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_neighbor: " + "Node 0x%" PRIx64 ", remote node 0x%" PRIx64 + ", port 0x%X, remote port 0x%X\n", + cl_ntoh64(osm_node_get_node_guid(p_this_sw->p_node)), + cl_ntoh64(osm_node_get_node_guid(p_remote_sw->p_node)), + port_num, remote_port_num); + } + + p_next_sw = (osm_switch_t *) cl_qmap_head(&p_mgr->p_subn->sw_guid_tbl); + while (p_next_sw != + (osm_switch_t *) cl_qmap_end(&p_mgr->p_subn->sw_guid_tbl)) { + p_sw = p_next_sw; + p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + lid_ho = osm_node_get_base_lid(p_sw->p_node, 0); + lid_ho = cl_ntoh16(lid_ho); + hops = osm_switch_get_least_hops(p_remote_sw, lid_ho); + if (hops == OSM_NO_PATH) + continue; + hops++; + if (hops < + osm_switch_get_hop_count(p_this_sw, lid_ho, port_num)) { + if (osm_switch_set_hops + (p_this_sw, lid_ho, port_num, hops) != 0) + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_ucast_mgr_process_neighbor: " + "cannot set hops for lid %u at switch 0x%" + PRIx64 "\n", lid_ho, + cl_ntoh64(osm_node_get_node_guid + (p_this_sw->p_node))); + p_mgr->some_hop_count_set = TRUE; + } + } + + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_ucast_mgr_process_port( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_switch_t* const p_sw, - IN osm_port_t* const p_port ) +__osm_ucast_mgr_process_port(IN osm_ucast_mgr_t * const p_mgr, + IN osm_switch_t * const p_sw, + IN osm_port_t * const p_port) { - uint16_t min_lid_ho; - uint16_t max_lid_ho; - uint16_t lid_ho; - uint8_t port; - boolean_t is_ignored_by_port_prof; - ib_net64_t node_guid; - /* - The following are temporary structures that will aid - in providing better routing in LMC > 0 situations - */ - uint16_t lids_per_port = 1 << p_mgr->p_subn->opt.lmc; - uint64_t *remote_sys_guids = NULL; - uint64_t *remote_node_guids = NULL; - uint16_t num_used_sys = 0; - uint16_t num_used_nodes = 0; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_ucast_mgr_process_port ); - - if (lids_per_port > 1) - { - remote_sys_guids = malloc( sizeof(uint64_t) * lids_per_port ); - if( remote_sys_guids == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_ucast_mgr_process_port: ERR 3A09: " - "Cannot allocate array. Insufficient memory\n"); - goto Exit; - } - - memset( remote_sys_guids, 0, sizeof(uint64_t) * lids_per_port ); - - remote_node_guids = malloc( sizeof(uint64_t) * lids_per_port ); - if( remote_node_guids == NULL ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_ucast_mgr_process_port: ERR 3A0A: " - "Cannot allocate array. Insufficient memory\n"); - goto Exit; - } - - memset( remote_node_guids, 0, sizeof(uint64_t) * lids_per_port ); - } - - osm_port_get_lid_range_ho( p_port, &min_lid_ho, &max_lid_ho ); - - /* If the lids are zero - then there was some problem with the initialization. - Don't handle this port. */ - if ( min_lid_ho == 0 || max_lid_ho == 0 ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_ucast_mgr_process_port: ERR 3A04: " - "Port 0x%" PRIx64 " has LID 0. An initialization " - "error occurred. Ignoring port\n", - cl_ntoh64( osm_port_get_guid( p_port ) ) ); - goto Exit; - } - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_port: " - "Processing port 0x%" PRIx64 - ", LIDs [0x%X,0x%X]\n", - cl_ntoh64( osm_port_get_guid( p_port ) ), - min_lid_ho, max_lid_ho ); - } - - /* - TO DO - This should be runtime error, not a CL_ASSERT() - */ - CL_ASSERT( max_lid_ho < osm_switch_get_fwd_tbl_size( p_sw ) ); - - node_guid = osm_node_get_node_guid( p_sw->p_node ); - - /* - The lid matrix contains the number of hops to each - lid from each port. From this information we determine - how best to distribute the LID range across the ports - that can reach those LIDs. - */ - for( lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++ ) - { - /* Use the enhanced algorithm only for LMC > 0 */ - if (lids_per_port > 1) - port = osm_switch_recommend_path( p_sw, p_port, lid_ho, - p_mgr->p_subn->ignore_existing_lfts, - remote_sys_guids, &num_used_sys, - remote_node_guids, &num_used_nodes ); - else - port = osm_switch_recommend_path( p_sw, p_port, lid_ho, - p_mgr->p_subn->ignore_existing_lfts, - NULL, NULL, NULL, NULL ); - - /* - There might be no path to the target - */ - if (port == OSM_NO_PATH) - { - /* do not try to overwrite the ppro of non existing port ... */ - is_ignored_by_port_prof = TRUE; - - /* Up/Down routing can cause unreachable routes between some - switches so we do not report that as an error in that case */ - if (!p_mgr->p_subn->p_osm->routing_engine.build_lid_matrices) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "__osm_ucast_mgr_process_port: ERR 3A08: " - "No path to get to LID 0x%X from switch 0x%" PRIx64 "\n", - lid_ho, cl_ntoh64( node_guid ) ); - /* trigger a new sweep - try again ... */ - p_mgr->p_subn->subnet_initialization_error = TRUE; - } - else - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_port: " - "No path to get to LID 0x%X from switch 0x%" PRIx64 "\n", - lid_ho, cl_ntoh64( node_guid ) ); - } - else - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_port: " - "Routing LID 0x%X to port 0x%X" - " for switch 0x%" PRIx64 "\n", - lid_ho, port, cl_ntoh64( node_guid ) ); - - /* - we would like to optionally ignore this port in equalization - as in the case of the Mellanox Anafa Internal PCI TCA port - */ - is_ignored_by_port_prof = - osm_port_prof_is_ignored_port(p_mgr->p_subn, node_guid, port); - - /* - We also would ignore this route if the target lid is of a switch - and the port_profile_switch_node is not TRUE - */ - if (!p_mgr->p_subn->opt.port_profile_switch_nodes) - { - is_ignored_by_port_prof |= - (osm_node_get_type(p_port->p_node) == IB_NODE_TYPE_SWITCH); - } - } - - /* - We have selected the port for this LID. - Write it to the forwarding tables. - */ - p_mgr->lft_buf[lid_ho] = port; - if (!is_ignored_by_port_prof) - osm_switch_count_path(p_sw, port); - } - - Exit: - if (remote_sys_guids) - free(remote_sys_guids); - if (remote_node_guids) - free(remote_node_guids); - OSM_LOG_EXIT( p_mgr->p_log ); + uint16_t min_lid_ho; + uint16_t max_lid_ho; + uint16_t lid_ho; + uint8_t port; + boolean_t is_ignored_by_port_prof; + ib_net64_t node_guid; + /* + The following are temporary structures that will aid + in providing better routing in LMC > 0 situations + */ + uint16_t lids_per_port = 1 << p_mgr->p_subn->opt.lmc; + uint64_t *remote_sys_guids = NULL; + uint64_t *remote_node_guids = NULL; + uint16_t num_used_sys = 0; + uint16_t num_used_nodes = 0; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_ucast_mgr_process_port); + + if (lids_per_port > 1) { + remote_sys_guids = malloc(sizeof(uint64_t) * lids_per_port); + if (remote_sys_guids == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_ucast_mgr_process_port: ERR 3A09: " + "Cannot allocate array. Insufficient memory\n"); + goto Exit; + } + + memset(remote_sys_guids, 0, sizeof(uint64_t) * lids_per_port); + + remote_node_guids = malloc(sizeof(uint64_t) * lids_per_port); + if (remote_node_guids == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_ucast_mgr_process_port: ERR 3A0A: " + "Cannot allocate array. Insufficient memory\n"); + goto Exit; + } + + memset(remote_node_guids, 0, sizeof(uint64_t) * lids_per_port); + } + + osm_port_get_lid_range_ho(p_port, &min_lid_ho, &max_lid_ho); + + /* If the lids are zero - then there was some problem with the initialization. + Don't handle this port. */ + if (min_lid_ho == 0 || max_lid_ho == 0) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_ucast_mgr_process_port: ERR 3A04: " + "Port 0x%" PRIx64 " has LID 0. An initialization " + "error occurred. Ignoring port\n", + cl_ntoh64(osm_port_get_guid(p_port))); + goto Exit; + } + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_port: " + "Processing port 0x%" PRIx64 + ", LIDs [0x%X,0x%X]\n", + cl_ntoh64(osm_port_get_guid(p_port)), + min_lid_ho, max_lid_ho); + } + + /* + TO DO - This should be runtime error, not a CL_ASSERT() + */ + CL_ASSERT(max_lid_ho < osm_switch_get_fwd_tbl_size(p_sw)); + + node_guid = osm_node_get_node_guid(p_sw->p_node); + + /* + The lid matrix contains the number of hops to each + lid from each port. From this information we determine + how best to distribute the LID range across the ports + that can reach those LIDs. + */ + for (lid_ho = min_lid_ho; lid_ho <= max_lid_ho; lid_ho++) { + /* Use the enhanced algorithm only for LMC > 0 */ + if (lids_per_port > 1) + port = osm_switch_recommend_path(p_sw, p_port, lid_ho, + p_mgr->p_subn-> + ignore_existing_lfts, + remote_sys_guids, + &num_used_sys, + remote_node_guids, + &num_used_nodes); + else + port = osm_switch_recommend_path(p_sw, p_port, lid_ho, + p_mgr->p_subn-> + ignore_existing_lfts, + NULL, NULL, NULL, + NULL); + + /* + There might be no path to the target + */ + if (port == OSM_NO_PATH) { + /* do not try to overwrite the ppro of non existing port ... */ + is_ignored_by_port_prof = TRUE; + + /* Up/Down routing can cause unreachable routes between some + switches so we do not report that as an error in that case */ + if (!p_mgr->p_subn->p_osm->routing_engine. + build_lid_matrices) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "__osm_ucast_mgr_process_port: ERR 3A08: " + "No path to get to LID 0x%X from switch 0x%" + PRIx64 "\n", lid_ho, + cl_ntoh64(node_guid)); + /* trigger a new sweep - try again ... */ + p_mgr->p_subn->subnet_initialization_error = + TRUE; + } else + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_port: " + "No path to get to LID 0x%X from switch 0x%" + PRIx64 "\n", lid_ho, + cl_ntoh64(node_guid)); + } else { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_port: " + "Routing LID 0x%X to port 0x%X" + " for switch 0x%" PRIx64 "\n", + lid_ho, port, cl_ntoh64(node_guid)); + + /* + we would like to optionally ignore this port in equalization + as in the case of the Mellanox Anafa Internal PCI TCA port + */ + is_ignored_by_port_prof = + osm_port_prof_is_ignored_port(p_mgr->p_subn, + node_guid, port); + + /* + We also would ignore this route if the target lid is of a switch + and the port_profile_switch_node is not TRUE + */ + if (!p_mgr->p_subn->opt.port_profile_switch_nodes) { + is_ignored_by_port_prof |= + (osm_node_get_type(p_port->p_node) == + IB_NODE_TYPE_SWITCH); + } + } + + /* + We have selected the port for this LID. + Write it to the forwarding tables. + */ + p_mgr->lft_buf[lid_ho] = port; + if (!is_ignored_by_port_prof) + osm_switch_count_path(p_sw, port); + } + + Exit: + if (remote_sys_guids) + free(remote_sys_guids); + if (remote_node_guids) + free(remote_node_guids); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ void -osm_ucast_mgr_set_fwd_table( - IN osm_ucast_mgr_t* const p_mgr, - IN osm_switch_t* const p_sw ) +osm_ucast_mgr_set_fwd_table(IN osm_ucast_mgr_t * const p_mgr, + IN osm_switch_t * const p_sw) { - osm_node_t *p_node; - osm_dr_path_t *p_path; - osm_madw_context_t context; - ib_api_status_t status; - ib_switch_info_t si; - uint32_t block_id_ho = 0; - uint8_t block[IB_SMP_DATA_SIZE]; - boolean_t set_swinfo_require = FALSE; - uint16_t lin_top; - uint8_t life_state; - - CL_ASSERT( p_mgr ); - - OSM_LOG_ENTER( p_mgr->p_log, osm_ucast_mgr_set_fwd_table ); - - CL_ASSERT( p_sw ); - - p_node = p_sw->p_node; - - CL_ASSERT( p_node ); - - p_path = osm_node_get_any_dr_path_ptr( p_node ); - - CL_ASSERT( p_path ); - - /* - Set the top of the unicast forwarding table. - */ - si = p_sw->switch_info; - lin_top = cl_hton16( p_sw->max_lid_ho ); - if (lin_top != si.lin_top) - { - set_swinfo_require = TRUE; - si.lin_top = lin_top; - } - - /* check to see if the change state bit is on. If it is - then we - need to clear it. */ - if ( ib_switch_info_get_state_change( &si ) ) - life_state = ( (p_mgr->p_subn->opt.packet_life_time <<3 ) - | ( si.life_state & IB_SWITCH_PSC ) ) & 0xfc; - else - life_state = (p_mgr->p_subn->opt.packet_life_time <<3 ) & 0xf8; - - if ( (life_state != si.life_state) || ib_switch_info_get_state_change( &si ) ) - { - set_swinfo_require = TRUE; - si.life_state = life_state; - } - - if ( set_swinfo_require ) - { - if ( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_ucast_mgr_set_fwd_table: " - "Setting switch FT top to LID 0x%X\n", - p_sw->max_lid_ho ); - } - - context.si_context.light_sweep = FALSE; - context.si_context.node_guid = osm_node_get_node_guid( p_node ); - context.si_context.set_method = TRUE; - - status = osm_req_set( p_mgr->p_req, - p_path, - (uint8_t*)&si, - sizeof(si), - IB_MAD_ATTR_SWITCH_INFO, - 0, - CL_DISP_MSGID_NONE, - &context ); - - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_ucast_mgr_set_fwd_table: ERR 3A06: " - "Sending SwitchInfo attribute failed (%s)\n", - ib_get_err_str( status ) ); - } - else - p_mgr->any_change = TRUE; - } - - /* - Send linear forwarding table blocks to the switch - as long as the switch indicates it has blocks needing - configuration. - */ - - context.lft_context.node_guid = osm_node_get_node_guid( p_node ); - context.lft_context.set_method = TRUE; - - for (block_id_ho = 0; - osm_switch_get_fwd_tbl_block( p_sw, block_id_ho, block ) ; - block_id_ho++ ) - { - if (!p_sw->need_update && - !memcmp(block, p_mgr->lft_buf + block_id_ho * 64, 64)) - continue; - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_ucast_mgr_set_fwd_table: " - "Writing FT block %u\n", block_id_ho ); - } - - status = osm_req_set( p_mgr->p_req, - p_path, - p_mgr->lft_buf + block_id_ho * 64, - sizeof(block), - IB_MAD_ATTR_LIN_FWD_TBL, - cl_hton32( block_id_ho ), - CL_DISP_MSGID_NONE, - &context ); - - if( status != IB_SUCCESS ) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_ucast_mgr_set_fwd_table: ERR 3A05: " - "Sending linear fwd. tbl. block failed (%s)\n", - ib_get_err_str( status ) ); - } - else - { - p_mgr->any_change = TRUE; - } - } - - OSM_LOG_EXIT( p_mgr->p_log ); + osm_node_t *p_node; + osm_dr_path_t *p_path; + osm_madw_context_t context; + ib_api_status_t status; + ib_switch_info_t si; + uint32_t block_id_ho = 0; + uint8_t block[IB_SMP_DATA_SIZE]; + boolean_t set_swinfo_require = FALSE; + uint16_t lin_top; + uint8_t life_state; + + CL_ASSERT(p_mgr); + + OSM_LOG_ENTER(p_mgr->p_log, osm_ucast_mgr_set_fwd_table); + + CL_ASSERT(p_sw); + + p_node = p_sw->p_node; + + CL_ASSERT(p_node); + + p_path = osm_node_get_any_dr_path_ptr(p_node); + + CL_ASSERT(p_path); + + /* + Set the top of the unicast forwarding table. + */ + si = p_sw->switch_info; + lin_top = cl_hton16(p_sw->max_lid_ho); + if (lin_top != si.lin_top) { + set_swinfo_require = TRUE; + si.lin_top = lin_top; + } + + /* check to see if the change state bit is on. If it is - then we + need to clear it. */ + if (ib_switch_info_get_state_change(&si)) + life_state = ((p_mgr->p_subn->opt.packet_life_time << 3) + | (si.life_state & IB_SWITCH_PSC)) & 0xfc; + else + life_state = (p_mgr->p_subn->opt.packet_life_time << 3) & 0xf8; + + if ((life_state != si.life_state) + || ib_switch_info_get_state_change(&si)) { + set_swinfo_require = TRUE; + si.life_state = life_state; + } + + if (set_swinfo_require) { + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_ucast_mgr_set_fwd_table: " + "Setting switch FT top to LID 0x%X\n", + p_sw->max_lid_ho); + } + + context.si_context.light_sweep = FALSE; + context.si_context.node_guid = osm_node_get_node_guid(p_node); + context.si_context.set_method = TRUE; + + status = osm_req_set(p_mgr->p_req, + p_path, + (uint8_t *) & si, + sizeof(si), + IB_MAD_ATTR_SWITCH_INFO, + 0, CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_ucast_mgr_set_fwd_table: ERR 3A06: " + "Sending SwitchInfo attribute failed (%s)\n", + ib_get_err_str(status)); + } else + p_mgr->any_change = TRUE; + } + + /* + Send linear forwarding table blocks to the switch + as long as the switch indicates it has blocks needing + configuration. + */ + + context.lft_context.node_guid = osm_node_get_node_guid(p_node); + context.lft_context.set_method = TRUE; + + for (block_id_ho = 0; + osm_switch_get_fwd_tbl_block(p_sw, block_id_ho, block); + block_id_ho++) { + if (!p_sw->need_update && + !memcmp(block, p_mgr->lft_buf + block_id_ho * 64, 64)) + continue; + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_ucast_mgr_set_fwd_table: " + "Writing FT block %u\n", block_id_ho); + } + + status = osm_req_set(p_mgr->p_req, + p_path, + p_mgr->lft_buf + block_id_ho * 64, + sizeof(block), + IB_MAD_ATTR_LIN_FWD_TBL, + cl_hton32(block_id_ho), + CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_ucast_mgr_set_fwd_table: ERR 3A05: " + "Sending linear fwd. tbl. block failed (%s)\n", + ib_get_err_str(status)); + } else { + p_mgr->any_change = TRUE; + } + } + + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_ucast_mgr_process_tbl( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_ucast_mgr_process_tbl(IN cl_map_item_t * const p_map_item, + IN void *context) { - osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - osm_ucast_mgr_t* const p_mgr = (osm_ucast_mgr_t*)context; - osm_node_t *p_node; - osm_port_t *p_port; - const cl_qmap_t* p_port_tbl; + osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + osm_ucast_mgr_t *const p_mgr = (osm_ucast_mgr_t *) context; + osm_node_t *p_node; + osm_port_t *p_port; + const cl_qmap_t *p_port_tbl; - OSM_LOG_ENTER( p_mgr->p_log, __osm_ucast_mgr_process_tbl ); + OSM_LOG_ENTER(p_mgr->p_log, __osm_ucast_mgr_process_tbl); - p_node = p_sw->p_node; + p_node = p_sw->p_node; - CL_ASSERT( p_node ); - CL_ASSERT( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ); + CL_ASSERT(p_node); + CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_tbl: " - "Processing switch 0x%" PRIx64 "\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) )); - } + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_tbl: " + "Processing switch 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + } - /* Initialize LIDs in buffer to invalid port number. */ - memset(p_mgr->lft_buf, 0xff, IB_LID_UCAST_END_HO + 1); + /* Initialize LIDs in buffer to invalid port number. */ + memset(p_mgr->lft_buf, 0xff, IB_LID_UCAST_END_HO + 1); - p_port_tbl = &p_mgr->p_subn->port_guid_tbl; + p_port_tbl = &p_mgr->p_subn->port_guid_tbl; - /* - Iterate through every port setting LID routes for each - port based on base LID and LMC value. - */ + /* + Iterate through every port setting LID routes for each + port based on base LID and LMC value. + */ - for( p_port = (osm_port_t*)cl_qmap_head( p_port_tbl ); - p_port != (osm_port_t*)cl_qmap_end( p_port_tbl ); - p_port = (osm_port_t*)cl_qmap_next( &p_port->map_item ) ) - { - __osm_ucast_mgr_process_port( p_mgr, p_sw, p_port ); - } + for (p_port = (osm_port_t *) cl_qmap_head(p_port_tbl); + p_port != (osm_port_t *) cl_qmap_end(p_port_tbl); + p_port = (osm_port_t *) cl_qmap_next(&p_port->map_item)) { + __osm_ucast_mgr_process_port(p_mgr, p_sw, p_port); + } - osm_ucast_mgr_set_fwd_table( p_mgr, p_sw ); + osm_ucast_mgr_set_fwd_table(p_mgr, p_sw); - OSM_LOG_EXIT( p_mgr->p_log ); + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ static void -__osm_ucast_mgr_process_neighbors( - IN cl_map_item_t* const p_map_item, - IN void* context ) +__osm_ucast_mgr_process_neighbors(IN cl_map_item_t * const p_map_item, + IN void *context) { - osm_switch_t* const p_sw = (osm_switch_t*)p_map_item; - osm_ucast_mgr_t* const p_mgr = (osm_ucast_mgr_t*)context; - osm_node_t *p_node; - osm_node_t *p_remote_node; - uint32_t port_num; - uint8_t remote_port_num; - uint32_t num_ports; - osm_physp_t* p_physp; - - OSM_LOG_ENTER( p_mgr->p_log, __osm_ucast_mgr_process_neighbors ); - - p_node = p_sw->p_node; - - CL_ASSERT( p_node ); - CL_ASSERT( osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH ); - - if( osm_log_is_active( p_mgr->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "__osm_ucast_mgr_process_neighbors: " - "Processing switch with GUID 0x%" PRIx64 "\n", - cl_ntoh64( osm_node_get_node_guid( p_node ) ) ); - } - - num_ports = osm_node_get_num_physp( p_node ); - - /* - Start with port 1 to skip the switch's management port. - */ - for( port_num = 1; port_num < num_ports; port_num++ ) - { - p_remote_node = osm_node_get_remote_node( p_node, - (uint8_t)port_num, &remote_port_num ); - - if( p_remote_node && p_remote_node->sw && (p_remote_node != p_node ) ) - { - /* make sure the link is healthy. If it is not - don't - propagate through it. */ - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - if (!osm_link_is_healthy( p_physp ) ) - continue; - - __osm_ucast_mgr_process_neighbor(p_mgr, p_sw, p_remote_node->sw, - (uint8_t)port_num, remote_port_num ); - - } - } - - OSM_LOG_EXIT( p_mgr->p_log ); + osm_switch_t *const p_sw = (osm_switch_t *) p_map_item; + osm_ucast_mgr_t *const p_mgr = (osm_ucast_mgr_t *) context; + osm_node_t *p_node; + osm_node_t *p_remote_node; + uint32_t port_num; + uint8_t remote_port_num; + uint32_t num_ports; + osm_physp_t *p_physp; + + OSM_LOG_ENTER(p_mgr->p_log, __osm_ucast_mgr_process_neighbors); + + p_node = p_sw->p_node; + + CL_ASSERT(p_node); + CL_ASSERT(osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH); + + if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "__osm_ucast_mgr_process_neighbors: " + "Processing switch with GUID 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_node))); + } + + num_ports = osm_node_get_num_physp(p_node); + + /* + Start with port 1 to skip the switch's management port. + */ + for (port_num = 1; port_num < num_ports; port_num++) { + p_remote_node = osm_node_get_remote_node(p_node, + (uint8_t) port_num, + &remote_port_num); + + if (p_remote_node && p_remote_node->sw + && (p_remote_node != p_node)) { + /* make sure the link is healthy. If it is not - don't + propagate through it. */ + p_physp = osm_node_get_physp_ptr(p_node, port_num); + if (!osm_link_is_healthy(p_physp)) + continue; + + __osm_ucast_mgr_process_neighbor(p_mgr, p_sw, + p_remote_node->sw, + (uint8_t) port_num, + remote_port_num); + + } + } + + OSM_LOG_EXIT(p_mgr->p_log); } /********************************************************************** **********************************************************************/ -void -osm_ucast_mgr_build_lid_matrices( - IN osm_ucast_mgr_t* const p_mgr ) +void osm_ucast_mgr_build_lid_matrices(IN osm_ucast_mgr_t * const p_mgr) { - uint32_t i; - uint32_t iteration_max; - cl_qmap_t *p_sw_guid_tbl; - - p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; - - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "osm_ucast_mgr_build_lid_matrices: " - "Starting switches' Min Hop Table Assignment\n" ); - - /* - Set the switch matrices for each switch's own port 0 LID(s) - then set the lid matrices for the each switch's leaf nodes. - */ - cl_qmap_apply_func( p_sw_guid_tbl, - __osm_ucast_mgr_process_hop_0_1, p_mgr ); - - /* - Get the switch matrices for each switch's neighbors. - This process requires a number of iterations equal to - the number of switches in the subnet minus 1. - - In each iteration, a switch learns the lid/port/hop - information (as contained by a switch's lid matrix) from - its immediate neighbors. After each iteration, a switch - (and it's neighbors) know more routing information than - it did on the previous iteration. - Thus, by repeatedly absorbing the routing information of - neighbor switches, every switch eventually learns how to - route all LIDs on the subnet. - - Note that there may not be any switches in the subnet if - we are in simple p2p configuration. - */ - iteration_max = cl_qmap_count( p_sw_guid_tbl ); - - /* - If there are switches in the subnet, iterate until the lid - matrix has been constructed. Otherwise, just immediately - indicate we're done if no switches exist. - */ - if( iteration_max ) - { - iteration_max--; - - /* - we need to find out when the propagation of - hop counts has relaxed. So this global variable - is preset to 0 on each iteration and if - if non of the switches was set will exit the - while loop - */ - p_mgr->some_hop_count_set = TRUE; - for( i = 0; (i < iteration_max) && p_mgr->some_hop_count_set; i++ ) - { - p_mgr->some_hop_count_set = FALSE; - cl_qmap_apply_func( p_sw_guid_tbl, - __osm_ucast_mgr_process_neighbors, p_mgr ); - } - osm_log( p_mgr->p_log, OSM_LOG_DEBUG, - "osm_ucast_mgr_build_lid_matrices: " - "Min-hop propagated in %d steps\n", i ); - } + uint32_t i; + uint32_t iteration_max; + cl_qmap_t *p_sw_guid_tbl; + + p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; + + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "osm_ucast_mgr_build_lid_matrices: " + "Starting switches' Min Hop Table Assignment\n"); + + /* + Set the switch matrices for each switch's own port 0 LID(s) + then set the lid matrices for the each switch's leaf nodes. + */ + cl_qmap_apply_func(p_sw_guid_tbl, + __osm_ucast_mgr_process_hop_0_1, p_mgr); + + /* + Get the switch matrices for each switch's neighbors. + This process requires a number of iterations equal to + the number of switches in the subnet minus 1. + + In each iteration, a switch learns the lid/port/hop + information (as contained by a switch's lid matrix) from + its immediate neighbors. After each iteration, a switch + (and it's neighbors) know more routing information than + it did on the previous iteration. + Thus, by repeatedly absorbing the routing information of + neighbor switches, every switch eventually learns how to + route all LIDs on the subnet. + + Note that there may not be any switches in the subnet if + we are in simple p2p configuration. + */ + iteration_max = cl_qmap_count(p_sw_guid_tbl); + + /* + If there are switches in the subnet, iterate until the lid + matrix has been constructed. Otherwise, just immediately + indicate we're done if no switches exist. + */ + if (iteration_max) { + iteration_max--; + + /* + we need to find out when the propagation of + hop counts has relaxed. So this global variable + is preset to 0 on each iteration and if + if non of the switches was set will exit the + while loop + */ + p_mgr->some_hop_count_set = TRUE; + for (i = 0; (i < iteration_max) && p_mgr->some_hop_count_set; + i++) { + p_mgr->some_hop_count_set = FALSE; + cl_qmap_apply_func(p_sw_guid_tbl, + __osm_ucast_mgr_process_neighbors, + p_mgr); + } + osm_log(p_mgr->p_log, OSM_LOG_DEBUG, + "osm_ucast_mgr_build_lid_matrices: " + "Min-hop propagated in %d steps\n", i); + } } /********************************************************************** **********************************************************************/ -static int -ucast_mgr_setup_all_switches(osm_subn_t *p_subn) +static int ucast_mgr_setup_all_switches(osm_subn_t * p_subn) { - osm_switch_t *p_sw; - uint16_t lids; - - lids = (uint16_t)cl_ptr_vector_get_size(&p_subn->port_lid_tbl); - lids = lids ? lids - 1 : 0; - - for (p_sw = (osm_switch_t*)cl_qmap_head(&p_subn->sw_guid_tbl); - p_sw != (osm_switch_t*)cl_qmap_end(&p_subn->sw_guid_tbl); - p_sw = (osm_switch_t*)cl_qmap_next(&p_sw->map_item)) - if (osm_switch_prepare_path_rebuild(p_sw, lids)) { - osm_log(&p_subn->p_osm->log, OSM_LOG_ERROR, - "ucast_mgr_setup_all_switches: ERR 3A0B: " - "cannot setup switch 0x%016" PRIx64 "\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); - return -1; - } - - return 0; + osm_switch_t *p_sw; + uint16_t lids; + + lids = (uint16_t) cl_ptr_vector_get_size(&p_subn->port_lid_tbl); + lids = lids ? lids - 1 : 0; + + for (p_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); + p_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl); + p_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item)) + if (osm_switch_prepare_path_rebuild(p_sw, lids)) { + osm_log(&p_subn->p_osm->log, OSM_LOG_ERROR, + "ucast_mgr_setup_all_switches: ERR 3A0B: " + "cannot setup switch 0x%016" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid + (p_sw->p_node))); + return -1; + } + + return 0; } /********************************************************************** **********************************************************************/ cl_status_t -osm_ucast_mgr_read_guid_file( - IN osm_ucast_mgr_t * const p_mgr, - IN const char * guid_file_name, - IN cl_list_t * p_list ) +osm_ucast_mgr_read_guid_file(IN osm_ucast_mgr_t * const p_mgr, + IN const char *guid_file_name, + IN cl_list_t * p_list) { - cl_status_t status = IB_SUCCESS; - FILE * guid_file; - char line[MAX_GUID_FILE_LINE_LENGTH]; - char * endptr; - uint64_t * p_guid; - - OSM_LOG_ENTER(p_mgr->p_log, osm_ucast_mgr_read_guid_file); - - guid_file = fopen(guid_file_name, "r"); - if (guid_file == NULL) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_ucast_mgr_read_guid_file: ERR 3A13: " - "Failed to open guid list file (%s)\n", - guid_file_name ); - status = IB_NOT_FOUND; - goto Exit; - } - - while ( fgets(line, MAX_GUID_FILE_LINE_LENGTH, guid_file) ) - { - if (strcspn(line, " ,;.") != strlen(line)) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_ucast_mgr_read_guid_file: ERR 3A14: " - "Poorly formatted guid in file (%s): %s\n", - guid_file_name, line ); - status = IB_NOT_FOUND; - break; - } - - /* Skip empty lines anywhere in the file - only one - char means the null termination */ - if (strlen(line) <= 1) - continue; - - p_guid = malloc(sizeof(uint64_t)); - if (!p_guid) - { - status = IB_ERROR; - goto Exit; - } - - *p_guid = strtoull(line, &endptr, 16); - - /* check that the string is a number */ - if (!(*p_guid) && (*endptr != '\0')) - { - osm_log( p_mgr->p_log, OSM_LOG_ERROR, - "osm_ucast_mgr_read_guid_file: ERR 3A15: " - "Poorly formatted guid in file (%s): %s\n", - guid_file_name, line ); - status = IB_NOT_FOUND; - break; - } - - /* store the parsed guid */ - cl_list_insert_tail(p_list, p_guid); - } - -Exit : - if (guid_file) - fclose(guid_file); - OSM_LOG_EXIT( p_mgr->p_log ); - return (status); + cl_status_t status = IB_SUCCESS; + FILE *guid_file; + char line[MAX_GUID_FILE_LINE_LENGTH]; + char *endptr; + uint64_t *p_guid; + + OSM_LOG_ENTER(p_mgr->p_log, osm_ucast_mgr_read_guid_file); + + guid_file = fopen(guid_file_name, "r"); + if (guid_file == NULL) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_ucast_mgr_read_guid_file: ERR 3A13: " + "Failed to open guid list file (%s)\n", guid_file_name); + status = IB_NOT_FOUND; + goto Exit; + } + + while (fgets(line, MAX_GUID_FILE_LINE_LENGTH, guid_file)) { + if (strcspn(line, " ,;.") != strlen(line)) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_ucast_mgr_read_guid_file: ERR 3A14: " + "Poorly formatted guid in file (%s): %s\n", + guid_file_name, line); + status = IB_NOT_FOUND; + break; + } + + /* Skip empty lines anywhere in the file - only one + char means the null termination */ + if (strlen(line) <= 1) + continue; + + p_guid = malloc(sizeof(uint64_t)); + if (!p_guid) { + status = IB_ERROR; + goto Exit; + } + + *p_guid = strtoull(line, &endptr, 16); + + /* check that the string is a number */ + if (!(*p_guid) && (*endptr != '\0')) { + osm_log(p_mgr->p_log, OSM_LOG_ERROR, + "osm_ucast_mgr_read_guid_file: ERR 3A15: " + "Poorly formatted guid in file (%s): %s\n", + guid_file_name, line); + status = IB_NOT_FOUND; + break; + } + + /* store the parsed guid */ + cl_list_insert_tail(p_list, p_guid); + } + + Exit: + if (guid_file) + fclose(guid_file); + OSM_LOG_EXIT(p_mgr->p_log); + return (status); } /********************************************************************** **********************************************************************/ -osm_signal_t -osm_ucast_mgr_process( - IN osm_ucast_mgr_t* const p_mgr ) +osm_signal_t osm_ucast_mgr_process(IN osm_ucast_mgr_t * const p_mgr) { - struct osm_routing_engine *p_routing_eng; - osm_signal_t signal = OSM_SIGNAL_DONE; - cl_qmap_t *p_sw_guid_tbl; - boolean_t default_routing = TRUE; - - OSM_LOG_ENTER( p_mgr->p_log, osm_ucast_mgr_process ); - - p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; - p_routing_eng = &p_mgr->p_subn->p_osm->routing_engine; - - CL_PLOCK_EXCL_ACQUIRE( p_mgr->p_lock ); - - /* - If there are no switches in the subnet, we are done. - */ - if (cl_qmap_count( p_sw_guid_tbl ) == 0 || - ucast_mgr_setup_all_switches(p_mgr->p_subn) < 0) - goto Exit; - - p_mgr->any_change = FALSE; - - if (!p_routing_eng->build_lid_matrices || - p_routing_eng->build_lid_matrices(p_routing_eng->context) != 0) - osm_ucast_mgr_build_lid_matrices(p_mgr); - - osm_log( p_mgr->p_log, OSM_LOG_INFO, - "osm_ucast_mgr_process: " - "%s tables configured on all switches\n", - p_routing_eng->name ? p_routing_eng->name : "null (min-hop)"); - - /* - Now that the lid matrices have been built, we can - build and download the switch forwarding tables. - */ - if ( p_routing_eng->ucast_build_fwd_tables && - (p_routing_eng->ucast_build_fwd_tables(p_routing_eng->context) == 0) ) - default_routing = FALSE; - else - cl_qmap_apply_func( p_sw_guid_tbl, __osm_ucast_mgr_process_tbl, p_mgr ); - - if (p_mgr->any_change) - { - signal = OSM_SIGNAL_DONE_PENDING; - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "osm_ucast_mgr_process: " - "LFT Tables configured on all switches\n"); - } - else - { - signal = OSM_SIGNAL_DONE; - osm_log( p_mgr->p_log, OSM_LOG_VERBOSE, - "osm_ucast_mgr_process: " - "No need to set any LFT Tables on any switches\n"); - } - - Exit: - CL_PLOCK_RELEASE( p_mgr->p_lock ); - OSM_LOG_EXIT( p_mgr->p_log ); - return( signal ); + struct osm_routing_engine *p_routing_eng; + osm_signal_t signal = OSM_SIGNAL_DONE; + cl_qmap_t *p_sw_guid_tbl; + boolean_t default_routing = TRUE; + + OSM_LOG_ENTER(p_mgr->p_log, osm_ucast_mgr_process); + + p_sw_guid_tbl = &p_mgr->p_subn->sw_guid_tbl; + p_routing_eng = &p_mgr->p_subn->p_osm->routing_engine; + + CL_PLOCK_EXCL_ACQUIRE(p_mgr->p_lock); + + /* + If there are no switches in the subnet, we are done. + */ + if (cl_qmap_count(p_sw_guid_tbl) == 0 || + ucast_mgr_setup_all_switches(p_mgr->p_subn) < 0) + goto Exit; + + p_mgr->any_change = FALSE; + + if (!p_routing_eng->build_lid_matrices || + p_routing_eng->build_lid_matrices(p_routing_eng->context) != 0) + osm_ucast_mgr_build_lid_matrices(p_mgr); + + osm_log(p_mgr->p_log, OSM_LOG_INFO, + "osm_ucast_mgr_process: " + "%s tables configured on all switches\n", + p_routing_eng->name ? p_routing_eng->name : "null (min-hop)"); + + /* + Now that the lid matrices have been built, we can + build and download the switch forwarding tables. + */ + if (p_routing_eng->ucast_build_fwd_tables && + (p_routing_eng->ucast_build_fwd_tables(p_routing_eng->context) == + 0)) + default_routing = FALSE; + else + cl_qmap_apply_func(p_sw_guid_tbl, __osm_ucast_mgr_process_tbl, + p_mgr); + + if (p_mgr->any_change) { + signal = OSM_SIGNAL_DONE_PENDING; + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "osm_ucast_mgr_process: " + "LFT Tables configured on all switches\n"); + } else { + signal = OSM_SIGNAL_DONE; + osm_log(p_mgr->p_log, OSM_LOG_VERBOSE, + "osm_ucast_mgr_process: " + "No need to set any LFT Tables on any switches\n"); + } + + Exit: + CL_PLOCK_RELEASE(p_mgr->p_lock); + OSM_LOG_EXIT(p_mgr->p_log); + return (signal); } diff --git a/opensm/opensm/osm_ucast_updn.c b/opensm/opensm/osm_ucast_updn.c index df42245..0b7b1a9 100644 --- a/opensm/opensm/osm_ucast_updn.c +++ b/opensm/opensm/osm_ucast_updn.c @@ -46,7 +46,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -60,71 +60,64 @@ /* //////////////////////////// */ /* direction */ -typedef enum _updn_switch_dir -{ - UP = 0, - DOWN +typedef enum _updn_switch_dir { + UP = 0, + DOWN } updn_switch_dir_t; /* guids list */ -typedef struct _updn_input -{ - uint32_t num_guids; - uint64_t *guid_list; +typedef struct _updn_input { + uint32_t num_guids; + uint64_t *guid_list; } updn_input_t; /* updn structure */ -typedef struct _updn -{ - boolean_t auto_detect_root_nodes; - updn_input_t updn_ucast_reg_inputs; - cl_list_t *p_root_nodes; - osm_opensm_t *p_osm; +typedef struct _updn { + boolean_t auto_detect_root_nodes; + updn_input_t updn_ucast_reg_inputs; + cl_list_t *p_root_nodes; + osm_opensm_t *p_osm; } updn_t; struct updn_node { - cl_list_item_t list; - osm_switch_t *sw; - updn_switch_dir_t dir; - unsigned rank; - unsigned visited; + cl_list_item_t list; + osm_switch_t *sw; + updn_switch_dir_t dir; + unsigned rank; + unsigned visited; }; /* ///////////////////////////////// */ /* Statics */ /* ///////////////////////////////// */ -static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t *p_updn); +static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn); /********************************************************************** **********************************************************************/ /* This function returns direction based on rank and guid info of current & remote ports */ static updn_switch_dir_t -__updn_get_dir( - IN unsigned cur_rank, - IN unsigned rem_rank, - IN uint64_t cur_guid, - IN uint64_t rem_guid ) +__updn_get_dir(IN unsigned cur_rank, + IN unsigned rem_rank, IN uint64_t cur_guid, IN uint64_t rem_guid) { - /* HACK: comes to solve root nodes connection, in a classic subnet root nodes do not connect - directly, but in case they are we assign to root node an UP direction to allow UPDN to discover - the subnet correctly (and not from the point of view of the last root node). - */ - if (!cur_rank && !rem_rank) - return UP; - - if (cur_rank < rem_rank) - return DOWN; - else if (cur_rank > rem_rank) - return UP; - else - { - /* Equal rank, decide by guid number, bigger == UP direction */ - if (cur_guid > rem_guid) - return UP; - else - return DOWN; - } + /* HACK: comes to solve root nodes connection, in a classic subnet root nodes do not connect + directly, but in case they are we assign to root node an UP direction to allow UPDN to discover + the subnet correctly (and not from the point of view of the last root node). + */ + if (!cur_rank && !rem_rank) + return UP; + + if (cur_rank < rem_rank) + return DOWN; + else if (cur_rank > rem_rank) + return UP; + else { + /* Equal rank, decide by guid number, bigger == UP direction */ + if (cur_guid > rem_guid) + return UP; + else + return DOWN; + } } /********************************************************************** @@ -132,210 +125,202 @@ __updn_get_dir( * as a starting point. **********************************************************************/ static int -__updn_bfs_by_node( - IN osm_log_t *p_log, - IN osm_subn_t *p_subn, - IN osm_switch_t *p_sw ) +__updn_bfs_by_node(IN osm_log_t * p_log, + IN osm_subn_t * p_subn, IN osm_switch_t * p_sw) { - uint8_t pn, pn_rem; - cl_qlist_t list; - uint16_t lid; - struct updn_node *u; - updn_switch_dir_t next_dir, current_dir; - - OSM_LOG_ENTER( p_log, __updn_bfs_by_node ); - - lid = osm_node_get_base_lid(p_sw->p_node, 0); - lid = cl_ntoh16(lid); - osm_switch_set_hops(p_sw, lid, 0, 0); - - osm_log( p_log, OSM_LOG_DEBUG, - "__updn_bfs_by_node: " - "Starting from switch - port GUID 0x%" PRIx64 " lid %u\n", - cl_ntoh64(p_sw->p_node->node_info.port_guid), lid ); - - u = p_sw->priv; - u->dir = UP; - - /* Update list with the new element */ - cl_qlist_init(&list); - cl_qlist_insert_tail(&list, &u->list); - - /* BFS the list till no next element */ - while (!cl_is_qlist_empty(&list)) - { - ib_net64_t remote_guid, current_guid; - - u = (struct updn_node *)cl_qlist_remove_head(&list); - u->visited = 0; /* cleanup */ - current_dir = u->dir; - current_guid = osm_node_get_node_guid(u->sw->p_node); - /* Go over all ports of the switch and find unvisited remote nodes */ - for ( pn = 1; pn < u->sw->num_ports; pn++ ) - { - osm_node_t *p_remote_node; - struct updn_node *rem_u; - uint8_t current_min_hop, remote_min_hop, set_hop_return_value; - osm_switch_t *p_remote_sw; - - p_remote_node = osm_node_get_remote_node(u->sw->p_node, pn, &pn_rem); - /* If no remote node OR remote node is not a SWITCH - continue to next pn */ - if( !p_remote_node || !p_remote_node->sw ) - continue; - /* Fetch remote guid only after validation of remote node */ - remote_guid = osm_node_get_node_guid(p_remote_node); - p_remote_sw = p_remote_node->sw; - rem_u = p_remote_sw->priv; - /* Decide which direction to mark it (UP/DOWN) */ - next_dir = __updn_get_dir(u->rank, rem_u->rank, - current_guid, remote_guid); - - /* Check if this is a legal step : the only illegal step is going - from DOWN to UP */ - if ((current_dir == DOWN) && (next_dir == UP)) - { - osm_log( p_log, OSM_LOG_DEBUG, - "__updn_bfs_by_node: " - "Avoiding move from 0x%016" PRIx64 " to 0x%016" PRIx64"\n", - cl_ntoh64(current_guid), cl_ntoh64(remote_guid) ); - /* Illegal step */ - continue; - } - /* Set MinHop value for the current lid */ - current_min_hop = osm_switch_get_least_hops(u->sw, lid); - /* Check hop count if better insert into list && update - the remote node Min Hop Table */ - remote_min_hop = osm_switch_get_hop_count(p_remote_sw, lid, pn_rem); - if (current_min_hop + 1 < remote_min_hop) - { - set_hop_return_value = osm_switch_set_hops(p_remote_sw, lid, pn_rem, current_min_hop + 1); - if (set_hop_return_value) - { - osm_log( p_log, OSM_LOG_ERROR, - "__updn_bfs_by_node (less) ERR AA01: " - "Invalid value returned from set min hop is: %d\n", - set_hop_return_value ); - } - /* Check if remote port has already been visited */ - if (!rem_u->visited) - { - /* Insert updn_switch item into the list */ - rem_u->dir = next_dir; - rem_u->visited = 1; - cl_qlist_insert_tail(&list, &rem_u->list); - } - } - } - } - - OSM_LOG_EXIT( p_log ); - return 0; + uint8_t pn, pn_rem; + cl_qlist_t list; + uint16_t lid; + struct updn_node *u; + updn_switch_dir_t next_dir, current_dir; + + OSM_LOG_ENTER(p_log, __updn_bfs_by_node); + + lid = osm_node_get_base_lid(p_sw->p_node, 0); + lid = cl_ntoh16(lid); + osm_switch_set_hops(p_sw, lid, 0, 0); + + osm_log(p_log, OSM_LOG_DEBUG, + "__updn_bfs_by_node: " + "Starting from switch - port GUID 0x%" PRIx64 " lid %u\n", + cl_ntoh64(p_sw->p_node->node_info.port_guid), lid); + + u = p_sw->priv; + u->dir = UP; + + /* Update list with the new element */ + cl_qlist_init(&list); + cl_qlist_insert_tail(&list, &u->list); + + /* BFS the list till no next element */ + while (!cl_is_qlist_empty(&list)) { + ib_net64_t remote_guid, current_guid; + + u = (struct updn_node *)cl_qlist_remove_head(&list); + u->visited = 0; /* cleanup */ + current_dir = u->dir; + current_guid = osm_node_get_node_guid(u->sw->p_node); + /* Go over all ports of the switch and find unvisited remote nodes */ + for (pn = 1; pn < u->sw->num_ports; pn++) { + osm_node_t *p_remote_node; + struct updn_node *rem_u; + uint8_t current_min_hop, remote_min_hop, + set_hop_return_value; + osm_switch_t *p_remote_sw; + + p_remote_node = + osm_node_get_remote_node(u->sw->p_node, pn, + &pn_rem); + /* If no remote node OR remote node is not a SWITCH + continue to next pn */ + if (!p_remote_node || !p_remote_node->sw) + continue; + /* Fetch remote guid only after validation of remote node */ + remote_guid = osm_node_get_node_guid(p_remote_node); + p_remote_sw = p_remote_node->sw; + rem_u = p_remote_sw->priv; + /* Decide which direction to mark it (UP/DOWN) */ + next_dir = __updn_get_dir(u->rank, rem_u->rank, + current_guid, remote_guid); + + /* Check if this is a legal step : the only illegal step is going + from DOWN to UP */ + if ((current_dir == DOWN) && (next_dir == UP)) { + osm_log(p_log, OSM_LOG_DEBUG, + "__updn_bfs_by_node: " + "Avoiding move from 0x%016" PRIx64 + " to 0x%016" PRIx64 "\n", + cl_ntoh64(current_guid), + cl_ntoh64(remote_guid)); + /* Illegal step */ + continue; + } + /* Set MinHop value for the current lid */ + current_min_hop = osm_switch_get_least_hops(u->sw, lid); + /* Check hop count if better insert into list && update + the remote node Min Hop Table */ + remote_min_hop = + osm_switch_get_hop_count(p_remote_sw, lid, pn_rem); + if (current_min_hop + 1 < remote_min_hop) { + set_hop_return_value = + osm_switch_set_hops(p_remote_sw, lid, + pn_rem, + current_min_hop + 1); + if (set_hop_return_value) { + osm_log(p_log, OSM_LOG_ERROR, + "__updn_bfs_by_node (less) ERR AA01: " + "Invalid value returned from set min hop is: %d\n", + set_hop_return_value); + } + /* Check if remote port has already been visited */ + if (!rem_u->visited) { + /* Insert updn_switch item into the list */ + rem_u->dir = next_dir; + rem_u->visited = 1; + cl_qlist_insert_tail(&list, + &rem_u->list); + } + } + } + } + + OSM_LOG_EXIT(p_log); + return 0; } /********************************************************************** **********************************************************************/ -static void -updn_destroy( - IN updn_t* const p_updn ) +static void updn_destroy(IN updn_t * const p_updn) { - uint64_t *p_guid_list_item; + uint64_t *p_guid_list_item; - /* free the array of guids */ - if (p_updn->updn_ucast_reg_inputs.guid_list) - free(p_updn->updn_ucast_reg_inputs.guid_list); + /* free the array of guids */ + if (p_updn->updn_ucast_reg_inputs.guid_list) + free(p_updn->updn_ucast_reg_inputs.guid_list); - /* destroy the list of root nodes */ - while ((p_guid_list_item = cl_list_remove_head( p_updn->p_root_nodes ))) - free( p_guid_list_item ); + /* destroy the list of root nodes */ + while ((p_guid_list_item = cl_list_remove_head(p_updn->p_root_nodes))) + free(p_guid_list_item); - cl_list_remove_all( p_updn->p_root_nodes ); - cl_list_destroy( p_updn->p_root_nodes ); - free ( p_updn->p_root_nodes ); - free (p_updn); + cl_list_remove_all(p_updn->p_root_nodes); + cl_list_destroy(p_updn->p_root_nodes); + free(p_updn->p_root_nodes); + free(p_updn); } /********************************************************************** **********************************************************************/ -static updn_t* -updn_construct(osm_log_t *p_log) +static updn_t *updn_construct(osm_log_t * p_log) { - updn_t* p_updn; + updn_t *p_updn; - OSM_LOG_ENTER( p_log, updn_construct ); + OSM_LOG_ENTER(p_log, updn_construct); - p_updn = malloc(sizeof(updn_t)); - if (p_updn) - memset(p_updn, 0, sizeof(updn_t)); + p_updn = malloc(sizeof(updn_t)); + if (p_updn) + memset(p_updn, 0, sizeof(updn_t)); - OSM_LOG_EXIT( p_log ); - return(p_updn); + OSM_LOG_EXIT(p_log); + return (p_updn); } /********************************************************************** **********************************************************************/ -static cl_status_t -updn_init( - IN updn_t* const p_updn, - IN osm_opensm_t *p_osm ) +static cl_status_t updn_init(IN updn_t * const p_updn, IN osm_opensm_t * p_osm) { - cl_list_t * p_list; - cl_list_iterator_t guid_iterator; - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( &p_osm->log, updn_init ); - - p_updn->p_osm = p_osm; - p_list = (cl_list_t*)malloc(sizeof(cl_list_t)); - if (!p_list) - { - status = IB_ERROR; - goto Exit; - } - - cl_list_construct( p_list ); - cl_list_init( p_list, 10 ); - p_updn->p_root_nodes = p_list; - p_updn->updn_ucast_reg_inputs.num_guids = 0; - p_updn->updn_ucast_reg_inputs.guid_list = NULL; - p_updn->auto_detect_root_nodes = FALSE; - - /* - Check the source for root node list, if file parse it, otherwise - wait for a callback to activate auto detection - */ - if (p_osm->subn.opt.root_guid_file) - { - status = osm_ucast_mgr_read_guid_file( &p_osm->sm.ucast_mgr, - p_osm->subn.opt.root_guid_file, - p_updn->p_root_nodes ); - if (status != IB_SUCCESS) - goto Exit; - - /* For Debug Purposes ... */ - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "updn_init: " - "UPDN - Fetching root nodes from file %s\n", - p_osm->subn.opt.root_guid_file ); - guid_iterator = cl_list_head(p_updn->p_root_nodes); - while( guid_iterator != cl_list_end(p_updn->p_root_nodes) ) - { - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "updn_init: " - "Inserting GUID 0x%" PRIx64 " as root node\n", - *((uint64_t*)cl_list_obj(guid_iterator)) ); - guid_iterator = cl_list_next(guid_iterator); - } - } - else - { - p_updn->auto_detect_root_nodes = TRUE; - } - /* If auto mode detection required - will be executed in main b4 the assignment of UI Ucast */ - -Exit : - OSM_LOG_EXIT( &p_osm->log ); - return (status); + cl_list_t *p_list; + cl_list_iterator_t guid_iterator; + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(&p_osm->log, updn_init); + + p_updn->p_osm = p_osm; + p_list = (cl_list_t *) malloc(sizeof(cl_list_t)); + if (!p_list) { + status = IB_ERROR; + goto Exit; + } + + cl_list_construct(p_list); + cl_list_init(p_list, 10); + p_updn->p_root_nodes = p_list; + p_updn->updn_ucast_reg_inputs.num_guids = 0; + p_updn->updn_ucast_reg_inputs.guid_list = NULL; + p_updn->auto_detect_root_nodes = FALSE; + + /* + Check the source for root node list, if file parse it, otherwise + wait for a callback to activate auto detection + */ + if (p_osm->subn.opt.root_guid_file) { + status = osm_ucast_mgr_read_guid_file(&p_osm->sm.ucast_mgr, + p_osm->subn.opt. + root_guid_file, + p_updn->p_root_nodes); + if (status != IB_SUCCESS) + goto Exit; + + /* For Debug Purposes ... */ + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "updn_init: " + "UPDN - Fetching root nodes from file %s\n", + p_osm->subn.opt.root_guid_file); + guid_iterator = cl_list_head(p_updn->p_root_nodes); + while (guid_iterator != cl_list_end(p_updn->p_root_nodes)) { + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "updn_init: " + "Inserting GUID 0x%" PRIx64 " as root node\n", + *((uint64_t *) cl_list_obj(guid_iterator))); + guid_iterator = cl_list_next(guid_iterator); + } + } else { + p_updn->auto_detect_root_nodes = TRUE; + } + /* If auto mode detection required - will be executed in main b4 the assignment of UI Ucast */ + + Exit: + OSM_LOG_EXIT(&p_osm->log); + return (status); } /********************************************************************** @@ -343,526 +328,502 @@ Exit : /* NOTE : PLS check if we need to decide that the first */ /* rank is a SWITCH for BFS purpose */ static int -updn_subn_rank( - IN unsigned num_guids, - IN uint64_t* guid_list, - IN updn_t* p_updn ) +updn_subn_rank(IN unsigned num_guids, + IN uint64_t * guid_list, IN updn_t * p_updn) { - osm_switch_t *p_sw; - osm_physp_t *p_physp, *p_remote_physp; - cl_qlist_t list; - struct updn_node *u, *remote_u; - uint8_t num_ports, port_num; - osm_log_t *p_log = &p_updn->p_osm->log; - unsigned idx = 0; - unsigned max_rank = 0; - - OSM_LOG_ENTER( p_log, updn_subn_rank ); - cl_qlist_init(&list); - - /* Rank all the roots and add them to list */ - - for (idx = 0; idx < num_guids; idx++) - { - /* Apply the ranking for each guid given by user - bypass illegal ones */ - p_sw = osm_get_switch_by_guid(&p_updn->p_osm->subn, cl_hton64(guid_list[idx])); - if(!p_sw) - { - osm_log( p_log, OSM_LOG_ERROR, - "updn_subn_rank: ERR AA05: " - "Root switch GUID 0x%" PRIx64 " not found\n", guid_list[idx] ); - continue; - } - - u = p_sw->priv; - osm_log( p_log, OSM_LOG_DEBUG, - "updn_subn_rank: " - "Ranking root port GUID 0x%" PRIx64 "\n", guid_list[idx] ); - u->rank = 0; - cl_qlist_insert_tail(&list, &u->list); - } - - /* BFS the list till it's empty */ - while (!cl_is_qlist_empty(&list)) - { - u = (struct updn_node *)cl_qlist_remove_head(&list); - /* Go over all remote nodes and rank them (if not already visited) */ - p_sw = u->sw; - num_ports = p_sw->num_ports; - osm_log( p_log, OSM_LOG_DEBUG, - "updn_subn_rank: " - "Handling switch GUID 0x%" PRIx64 "\n", - cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)) ); - for (port_num = 1; port_num < num_ports; port_num++) - { - ib_net64_t port_guid; - - /* Current port fetched in order to get remote side */ - p_physp = osm_node_get_physp_ptr( p_sw->p_node, port_num ); - p_remote_physp = p_physp->p_remote_physp; - - /* - make sure that all the following occur on p_remote_physp: - 1. The port isn't NULL - 2. The port is a valid port - 3. It is a switch - */ - if ( p_remote_physp && - osm_physp_is_valid( p_remote_physp ) && - p_remote_physp->p_node->sw ) - { - remote_u = p_remote_physp->p_node->sw->priv; - port_guid = p_remote_physp->port_guid; - - if (remote_u->rank > u->rank+1) - { - remote_u->rank = u->rank + 1; - max_rank = remote_u->rank; - cl_qlist_insert_tail(&list, &remote_u->list); - } - - osm_log( p_log, OSM_LOG_DEBUG, - "updn_subn_rank: " - "Rank of port GUID 0x%" PRIx64 " = %u\n", - cl_ntoh64(port_guid), - remote_u->rank ); - - } - } - } - - /* Print Summary of ranking */ - osm_log( p_log, OSM_LOG_VERBOSE, - "updn_subn_rank: " - "Subnet ranking completed. Max Node Rank = %d\n", - max_rank ); - OSM_LOG_EXIT( p_log ); - return 0; + osm_switch_t *p_sw; + osm_physp_t *p_physp, *p_remote_physp; + cl_qlist_t list; + struct updn_node *u, *remote_u; + uint8_t num_ports, port_num; + osm_log_t *p_log = &p_updn->p_osm->log; + unsigned idx = 0; + unsigned max_rank = 0; + + OSM_LOG_ENTER(p_log, updn_subn_rank); + cl_qlist_init(&list); + + /* Rank all the roots and add them to list */ + + for (idx = 0; idx < num_guids; idx++) { + /* Apply the ranking for each guid given by user - bypass illegal ones */ + p_sw = + osm_get_switch_by_guid(&p_updn->p_osm->subn, + cl_hton64(guid_list[idx])); + if (!p_sw) { + osm_log(p_log, OSM_LOG_ERROR, + "updn_subn_rank: ERR AA05: " + "Root switch GUID 0x%" PRIx64 " not found\n", + guid_list[idx]); + continue; + } + + u = p_sw->priv; + osm_log(p_log, OSM_LOG_DEBUG, + "updn_subn_rank: " + "Ranking root port GUID 0x%" PRIx64 "\n", + guid_list[idx]); + u->rank = 0; + cl_qlist_insert_tail(&list, &u->list); + } + + /* BFS the list till it's empty */ + while (!cl_is_qlist_empty(&list)) { + u = (struct updn_node *)cl_qlist_remove_head(&list); + /* Go over all remote nodes and rank them (if not already visited) */ + p_sw = u->sw; + num_ports = p_sw->num_ports; + osm_log(p_log, OSM_LOG_DEBUG, + "updn_subn_rank: " + "Handling switch GUID 0x%" PRIx64 "\n", + cl_ntoh64(osm_node_get_node_guid(p_sw->p_node))); + for (port_num = 1; port_num < num_ports; port_num++) { + ib_net64_t port_guid; + + /* Current port fetched in order to get remote side */ + p_physp = + osm_node_get_physp_ptr(p_sw->p_node, port_num); + p_remote_physp = p_physp->p_remote_physp; + + /* + make sure that all the following occur on p_remote_physp: + 1. The port isn't NULL + 2. The port is a valid port + 3. It is a switch + */ + if (p_remote_physp && + osm_physp_is_valid(p_remote_physp) && + p_remote_physp->p_node->sw) { + remote_u = p_remote_physp->p_node->sw->priv; + port_guid = p_remote_physp->port_guid; + + if (remote_u->rank > u->rank + 1) { + remote_u->rank = u->rank + 1; + max_rank = remote_u->rank; + cl_qlist_insert_tail(&list, + &remote_u->list); + } + + osm_log(p_log, OSM_LOG_DEBUG, + "updn_subn_rank: " + "Rank of port GUID 0x%" PRIx64 + " = %u\n", cl_ntoh64(port_guid), + remote_u->rank); + + } + } + } + + /* Print Summary of ranking */ + osm_log(p_log, OSM_LOG_VERBOSE, + "updn_subn_rank: " + "Subnet ranking completed. Max Node Rank = %d\n", max_rank); + OSM_LOG_EXIT(p_log); + return 0; } /********************************************************************** **********************************************************************/ /* hack: preserve min hops entries to any other root switches */ -static void -updn_clear_root_hops(updn_t *p_updn, osm_switch_t *p_sw) +static void updn_clear_root_hops(updn_t * p_updn, osm_switch_t * p_sw) { - osm_port_t *p_port; - unsigned i; - - for ( i = 0 ; i < p_sw->num_hops ; i++ ) - if (p_sw->hops[i]) { - p_port = cl_ptr_vector_get(&p_updn->p_osm->subn.port_lid_tbl, i); - if (!p_port || !p_port->p_node->sw || - ((struct updn_node *)p_port->p_node->sw->priv)->rank != 0) - memset(p_sw->hops[i], 0xff, p_sw->num_ports); - } + osm_port_t *p_port; + unsigned i; + + for (i = 0; i < p_sw->num_hops; i++) + if (p_sw->hops[i]) { + p_port = + cl_ptr_vector_get(&p_updn->p_osm->subn.port_lid_tbl, + i); + if (!p_port || !p_port->p_node->sw + || ((struct updn_node *)p_port->p_node->sw->priv)-> + rank != 0) + memset(p_sw->hops[i], 0xff, p_sw->num_ports); + } } /********************************************************************** **********************************************************************/ -static int -__osm_subn_set_up_down_min_hop_table( - IN updn_t* p_updn ) +static int __osm_subn_set_up_down_min_hop_table(IN updn_t * p_updn) { - osm_subn_t *p_subn = &p_updn->p_osm->subn; - osm_log_t *p_log = &p_updn->p_osm->log; - osm_switch_t *p_next_sw,*p_sw; - - OSM_LOG_ENTER( p_log, __osm_subn_set_up_down_min_hop_table ); - - /* Go over all the switches in the subnet - for each init their Min Hop - Table */ - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_subn_set_up_down_min_hop_table: " - "Init Min Hop Table of all switches [\n" ); - - p_next_sw = (osm_switch_t*)cl_qmap_head( &p_subn->sw_guid_tbl ); - while( p_next_sw != (osm_switch_t*)cl_qmap_end( &p_subn->sw_guid_tbl ) ) - { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - /* Clear Min Hop Table */ - if (p_subn->opt.connect_roots && !((struct updn_node *)p_sw->priv)->rank) - updn_clear_root_hops(p_updn, p_sw); - else - osm_switch_clear_hops(p_sw); - } - - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_subn_set_up_down_min_hop_table: " - "Init Min Hop Table of all switches ]\n" ); - - /* Now do the BFS for each port in the subnet */ - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_subn_set_up_down_min_hop_table: " - "BFS through all port guids in the subnet [\n" ); - - p_next_sw = (osm_switch_t*)cl_qmap_head( &p_subn->sw_guid_tbl ); - while( p_next_sw != (osm_switch_t*)cl_qmap_end( &p_subn->sw_guid_tbl ) ) - { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - __updn_bfs_by_node(p_log, p_subn, p_sw); - } - - osm_log( p_log, OSM_LOG_VERBOSE, - "__osm_subn_set_up_down_min_hop_table: " - "BFS through all port guids in the subnet ]\n" ); - /* Cleanup */ - OSM_LOG_EXIT( p_log ); - return 0; + osm_subn_t *p_subn = &p_updn->p_osm->subn; + osm_log_t *p_log = &p_updn->p_osm->log; + osm_switch_t *p_next_sw, *p_sw; + + OSM_LOG_ENTER(p_log, __osm_subn_set_up_down_min_hop_table); + + /* Go over all the switches in the subnet - for each init their Min Hop + Table */ + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_subn_set_up_down_min_hop_table: " + "Init Min Hop Table of all switches [\n"); + + p_next_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); + while (p_next_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl)) { + p_sw = p_next_sw; + p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + /* Clear Min Hop Table */ + if (p_subn->opt.connect_roots + && !((struct updn_node *)p_sw->priv)->rank) + updn_clear_root_hops(p_updn, p_sw); + else + osm_switch_clear_hops(p_sw); + } + + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_subn_set_up_down_min_hop_table: " + "Init Min Hop Table of all switches ]\n"); + + /* Now do the BFS for each port in the subnet */ + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_subn_set_up_down_min_hop_table: " + "BFS through all port guids in the subnet [\n"); + + p_next_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); + while (p_next_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl)) { + p_sw = p_next_sw; + p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + __updn_bfs_by_node(p_log, p_subn, p_sw); + } + + osm_log(p_log, OSM_LOG_VERBOSE, + "__osm_subn_set_up_down_min_hop_table: " + "BFS through all port guids in the subnet ]\n"); + /* Cleanup */ + OSM_LOG_EXIT(p_log); + return 0; } /********************************************************************** **********************************************************************/ static int -__osm_subn_calc_up_down_min_hop_table( - IN uint32_t num_guids, - IN uint64_t* guid_list, - IN updn_t* p_updn ) +__osm_subn_calc_up_down_min_hop_table(IN uint32_t num_guids, + IN uint64_t * guid_list, + IN updn_t * p_updn) { - int status; - - OSM_LOG_ENTER( &p_updn->p_osm->log, osm_subn_calc_up_down_min_hop_table ); - - osm_log( &p_updn->p_osm->log, OSM_LOG_VERBOSE, - "__osm_subn_calc_up_down_min_hop_table: " - "Ranking all port guids in the list\n" ); - if (num_guids == 0) - { - osm_log( &p_updn->p_osm->log, OSM_LOG_ERROR, - "__osm_subn_calc_up_down_min_hop_table: ERR AA0A: " - "No guids were provided or number of guids is 0\n" ); - status = -1; - goto _exit; - } - - /* Check if it's not a switched subnet */ - if ( cl_is_qmap_empty( &p_updn->p_osm->subn.sw_guid_tbl ) ) - { - osm_log( &p_updn->p_osm->log, OSM_LOG_ERROR, - "__osm_subn_calc_up_down_min_hop_table: ERR AAOB: " - "This is not a switched subnet, cannot perform UPDN algorithm\n" ); - status = -1; - goto _exit; - } - - /* Rank the subnet switches */ - updn_subn_rank(num_guids, guid_list, p_updn); - - /* After multiple ranking need to set Min Hop Table by UpDn algorithm */ - osm_log( &p_updn->p_osm->log, OSM_LOG_VERBOSE, - "__osm_subn_calc_up_down_min_hop_table: " - "Setting all switches' Min Hop Table\n" ); - status = __osm_subn_set_up_down_min_hop_table(p_updn); - - _exit: - OSM_LOG_EXIT( &p_updn->p_osm->log ); - return status; + int status; + + OSM_LOG_ENTER(&p_updn->p_osm->log, osm_subn_calc_up_down_min_hop_table); + + osm_log(&p_updn->p_osm->log, OSM_LOG_VERBOSE, + "__osm_subn_calc_up_down_min_hop_table: " + "Ranking all port guids in the list\n"); + if (num_guids == 0) { + osm_log(&p_updn->p_osm->log, OSM_LOG_ERROR, + "__osm_subn_calc_up_down_min_hop_table: ERR AA0A: " + "No guids were provided or number of guids is 0\n"); + status = -1; + goto _exit; + } + + /* Check if it's not a switched subnet */ + if (cl_is_qmap_empty(&p_updn->p_osm->subn.sw_guid_tbl)) { + osm_log(&p_updn->p_osm->log, OSM_LOG_ERROR, + "__osm_subn_calc_up_down_min_hop_table: ERR AAOB: " + "This is not a switched subnet, cannot perform UPDN algorithm\n"); + status = -1; + goto _exit; + } + + /* Rank the subnet switches */ + updn_subn_rank(num_guids, guid_list, p_updn); + + /* After multiple ranking need to set Min Hop Table by UpDn algorithm */ + osm_log(&p_updn->p_osm->log, OSM_LOG_VERBOSE, + "__osm_subn_calc_up_down_min_hop_table: " + "Setting all switches' Min Hop Table\n"); + status = __osm_subn_set_up_down_min_hop_table(p_updn); + + _exit: + OSM_LOG_EXIT(&p_updn->p_osm->log); + return status; } /********************************************************************** **********************************************************************/ -static struct updn_node * -create_updn_node( - osm_switch_t *sw ) +static struct updn_node *create_updn_node(osm_switch_t * sw) { - struct updn_node *u; - - u = malloc(sizeof(*u)); - if (!u) - return NULL; - memset(u, 0, sizeof(*u)); - u->sw = sw; - u->rank = 0xffffffff; - return u; + struct updn_node *u; + + u = malloc(sizeof(*u)); + if (!u) + return NULL; + memset(u, 0, sizeof(*u)); + u->sw = sw; + u->rank = 0xffffffff; + return u; } -static void -delete_updn_node( - struct updn_node *u ) +static void delete_updn_node(struct updn_node *u) { - u->sw->priv = NULL; - free(u); + u->sw->priv = NULL; + free(u); } /********************************************************************** **********************************************************************/ /* UPDN callback function */ -static int -__osm_updn_call( - void *ctx ) +static int __osm_updn_call(void *ctx) { - updn_t *p_updn = ctx; - cl_map_item_t *p_item; - osm_switch_t *p_sw; - - OSM_LOG_ENTER( &p_updn->p_osm->log, __osm_updn_call ); - - p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); - while(p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) - { - p_sw = (osm_switch_t *)p_item; - p_item = cl_qmap_next(p_item); - p_sw->priv = create_updn_node(p_sw); - if (!p_sw->priv) - { - osm_log( &(p_updn->p_osm->log), OSM_LOG_ERROR, - "__osm_updn_call: ERR AA0C: " - " cannot create updn node\n" ); - OSM_LOG_EXIT( &p_updn->p_osm->log ); - return -1; - } - } - - /* First auto detect root nodes - if required */ - if ( p_updn->auto_detect_root_nodes ) - { - osm_ucast_mgr_build_lid_matrices( &p_updn->p_osm->sm.ucast_mgr ); - __osm_updn_find_root_nodes_by_min_hop( p_updn ); - } - else if (p_updn->p_osm->subn.opt.connect_roots && - p_updn->updn_ucast_reg_inputs.num_guids > 1) - osm_ucast_mgr_build_lid_matrices( &p_updn->p_osm->sm.ucast_mgr ); - - /* printf ("-V- after osm_updn_find_root_nodes_by_min_hop\n"); */ - /* Only if there are assigned root nodes do the algorithm, otherwise perform do nothing */ - if ( p_updn->updn_ucast_reg_inputs.num_guids > 0) - { - osm_log( &(p_updn->p_osm->log), OSM_LOG_DEBUG, - "__osm_updn_call: " - "activating UPDN algorithm\n" ); - __osm_subn_calc_up_down_min_hop_table( p_updn->updn_ucast_reg_inputs.num_guids, - p_updn->updn_ucast_reg_inputs.guid_list, - p_updn ); - } - else - osm_log( &p_updn->p_osm->log, OSM_LOG_INFO, - "__osm_updn_call: " - "disabling UPDN algorithm, no root nodes were found\n" ); - - p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); - while(p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) - { - p_sw = (osm_switch_t *)p_item; - p_item = cl_qmap_next(p_item); - delete_updn_node(p_sw->priv); - } - - OSM_LOG_EXIT( &p_updn->p_osm->log ); - return 0; + updn_t *p_updn = ctx; + cl_map_item_t *p_item; + osm_switch_t *p_sw; + + OSM_LOG_ENTER(&p_updn->p_osm->log, __osm_updn_call); + + p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + while (p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) { + p_sw = (osm_switch_t *) p_item; + p_item = cl_qmap_next(p_item); + p_sw->priv = create_updn_node(p_sw); + if (!p_sw->priv) { + osm_log(&(p_updn->p_osm->log), OSM_LOG_ERROR, + "__osm_updn_call: ERR AA0C: " + " cannot create updn node\n"); + OSM_LOG_EXIT(&p_updn->p_osm->log); + return -1; + } + } + + /* First auto detect root nodes - if required */ + if (p_updn->auto_detect_root_nodes) { + osm_ucast_mgr_build_lid_matrices(&p_updn->p_osm->sm.ucast_mgr); + __osm_updn_find_root_nodes_by_min_hop(p_updn); + } else if (p_updn->p_osm->subn.opt.connect_roots && + p_updn->updn_ucast_reg_inputs.num_guids > 1) + osm_ucast_mgr_build_lid_matrices(&p_updn->p_osm->sm.ucast_mgr); + + /* printf ("-V- after osm_updn_find_root_nodes_by_min_hop\n"); */ + /* Only if there are assigned root nodes do the algorithm, otherwise perform do nothing */ + if (p_updn->updn_ucast_reg_inputs.num_guids > 0) { + osm_log(&(p_updn->p_osm->log), OSM_LOG_DEBUG, + "__osm_updn_call: " "activating UPDN algorithm\n"); + __osm_subn_calc_up_down_min_hop_table(p_updn-> + updn_ucast_reg_inputs. + num_guids, + p_updn-> + updn_ucast_reg_inputs. + guid_list, p_updn); + } else + osm_log(&p_updn->p_osm->log, OSM_LOG_INFO, + "__osm_updn_call: " + "disabling UPDN algorithm, no root nodes were found\n"); + + p_item = cl_qmap_head(&p_updn->p_osm->subn.sw_guid_tbl); + while (p_item != cl_qmap_end(&p_updn->p_osm->subn.sw_guid_tbl)) { + p_sw = (osm_switch_t *) p_item; + p_item = cl_qmap_next(p_item); + delete_updn_node(p_sw->priv); + } + + OSM_LOG_EXIT(&p_updn->p_osm->log); + return 0; } /********************************************************************** **********************************************************************/ /* UPDN convert cl_list to guid array in updn struct */ -static void -__osm_updn_convert_list2array( - IN updn_t * p_updn ) +static void __osm_updn_convert_list2array(IN updn_t * p_updn) { - uint32_t i = 0, max_num = 0; - uint64_t *p_guid; - - OSM_LOG_ENTER( &p_updn->p_osm->log, __osm_updn_convert_list2array ); - - p_updn->updn_ucast_reg_inputs.num_guids = cl_list_count( - p_updn->p_root_nodes); - if (p_updn->updn_ucast_reg_inputs.guid_list) - free(p_updn->updn_ucast_reg_inputs.guid_list); - p_updn->updn_ucast_reg_inputs.guid_list = (uint64_t *)malloc( - p_updn->updn_ucast_reg_inputs.num_guids*sizeof(uint64_t)); - if (p_updn->updn_ucast_reg_inputs.guid_list) - memset(p_updn->updn_ucast_reg_inputs.guid_list, 0, - p_updn->updn_ucast_reg_inputs.num_guids*sizeof(uint64_t)); - if (!cl_is_list_empty(p_updn->p_root_nodes)) - { - while( (p_guid = (uint64_t*)cl_list_remove_head(p_updn->p_root_nodes)) ) - { - p_updn->updn_ucast_reg_inputs.guid_list[i] = *p_guid; - free(p_guid); - i++; - } - max_num = i; - for (i = 0; i < max_num; i++ ) - osm_log( &p_updn->p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_convert_list2array: " - "Map GUID 0x%" PRIx64 " into UPDN array\n", - p_updn->updn_ucast_reg_inputs.guid_list[i] ); - } - /* Since we need the template list for other sweeps, we wont destroy & free it */ - OSM_LOG_EXIT( &p_updn->p_osm->log ); + uint32_t i = 0, max_num = 0; + uint64_t *p_guid; + + OSM_LOG_ENTER(&p_updn->p_osm->log, __osm_updn_convert_list2array); + + p_updn->updn_ucast_reg_inputs.num_guids = + cl_list_count(p_updn->p_root_nodes); + if (p_updn->updn_ucast_reg_inputs.guid_list) + free(p_updn->updn_ucast_reg_inputs.guid_list); + p_updn->updn_ucast_reg_inputs.guid_list = + (uint64_t *) malloc(p_updn->updn_ucast_reg_inputs.num_guids * + sizeof(uint64_t)); + if (p_updn->updn_ucast_reg_inputs.guid_list) + memset(p_updn->updn_ucast_reg_inputs.guid_list, 0, + p_updn->updn_ucast_reg_inputs.num_guids * + sizeof(uint64_t)); + if (!cl_is_list_empty(p_updn->p_root_nodes)) { + while ((p_guid = + (uint64_t *) cl_list_remove_head(p_updn-> + p_root_nodes))) { + p_updn->updn_ucast_reg_inputs.guid_list[i] = *p_guid; + free(p_guid); + i++; + } + max_num = i; + for (i = 0; i < max_num; i++) + osm_log(&p_updn->p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_convert_list2array: " + "Map GUID 0x%" PRIx64 " into UPDN array\n", + p_updn->updn_ucast_reg_inputs.guid_list[i]); + } + /* Since we need the template list for other sweeps, we wont destroy & free it */ + OSM_LOG_EXIT(&p_updn->p_osm->log); } /********************************************************************** **********************************************************************/ /* Find Root nodes automatically by Min Hop Table info */ -static void -__osm_updn_find_root_nodes_by_min_hop( - OUT updn_t * p_updn ) +static void __osm_updn_find_root_nodes_by_min_hop(OUT updn_t * p_updn) { - osm_opensm_t *p_osm = p_updn->p_osm; - osm_switch_t *p_next_sw, *p_sw; - osm_port_t *p_next_port, *p_port; - osm_physp_t *p_physp; - uint64_t *p_guid; - cl_list_t *p_root_nodes_list = p_updn->p_root_nodes; - double thd1, thd2; - unsigned i, cas_num = 0; - unsigned *cas_per_sw; - uint16_t lid_ho; - - OSM_LOG_ENTER( &p_osm->log, osm_updn_find_root_nodes_by_min_hop ); - - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_find_root_nodes_by_min_hop: " - "Current number of ports in the subnet is %d\n", - cl_qmap_count(&p_osm->subn.port_guid_tbl) ); - - cas_per_sw = malloc((IB_LID_UCAST_END_HO + 1)*sizeof(*cas_per_sw)); - if (!cas_per_sw) { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "__osm_updn_find_root_nodes_by_min_hop: ERR AA14: " - "cannot alloc mem for CAs per switch counter array\n"); - goto _exit; - } - memset(cas_per_sw, 0, (IB_LID_UCAST_END_HO + 1)*sizeof(*cas_per_sw)); - - /* Find the Maximum number of CAs (and routers) for histogram normalization */ - osm_log( &p_osm->log, OSM_LOG_VERBOSE, - "__osm_updn_find_root_nodes_by_min_hop: " - "Finding the number of CAs and storing them in cl_map\n" ); - p_next_port = (osm_port_t*)cl_qmap_head( &p_osm->subn.port_guid_tbl ); - while( p_next_port != (osm_port_t*)cl_qmap_end( &p_osm->subn.port_guid_tbl ) ) { - p_port = p_next_port; - p_next_port = (osm_port_t*)cl_qmap_next( &p_next_port->map_item ); - if ( !p_port->p_node->sw ) - { - p_physp = p_port->p_physp->p_remote_physp; - if (!p_physp || !p_physp->p_node->sw) - continue; - lid_ho = osm_node_get_base_lid(p_physp->p_node, 0); - lid_ho = cl_ntoh16(lid_ho); - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_find_root_nodes_by_min_hop: " - "Inserting GUID 0x%" PRIx64 ", sw lid: 0x%X into array\n", - cl_ntoh64(osm_port_get_guid(p_port)), lid_ho ); - cas_per_sw[lid_ho]++; - cas_num++; - } - } - - thd1 = cas_num * 0.9; - thd2 = cas_num * 0.05; - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_find_root_nodes_by_min_hop: " - "Found %u CAs and RTRs, %u SWs in the subnet. " - "Thresholds are thd1 = %f && thd2 = %f\n", - cas_num, cl_qmap_count(&p_osm->subn.sw_guid_tbl), thd1, thd2); - - p_next_sw = (osm_switch_t*)cl_qmap_head( &p_osm->subn.sw_guid_tbl ); - osm_log( &p_osm->log, OSM_LOG_VERBOSE, - "__osm_updn_find_root_nodes_by_min_hop: " - "Passing through all switches to collect Min Hop info\n" ); - while( p_next_sw != (osm_switch_t*)cl_qmap_end( &p_osm->subn.sw_guid_tbl ) ) - { - unsigned hop_hist[IB_SUBNET_PATH_HOPS_MAX]; - uint16_t max_lid_ho; - uint8_t hop_val; - uint16_t numHopBarsOverThd1 = 0; - uint16_t numHopBarsOverThd2 = 0; - - p_sw = p_next_sw; - /* Roll to the next switch */ - p_next_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - - memset(hop_hist, 0, sizeof(hop_hist)); - - max_lid_ho = p_sw->max_lid_ho; - /* Get base lid of switch by retrieving port 0 lid of node pointer */ - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_find_root_nodes_by_min_hop: " - "Passing through switch lid 0x%X\n", - cl_ntoh16( osm_node_get_base_lid( p_sw->p_node, 0 ) ) ); - for (lid_ho = 1; lid_ho <= max_lid_ho; lid_ho++) - if (cas_per_sw[lid_ho]) - { - hop_val = osm_switch_get_least_hops( p_sw, lid_ho ); - if (hop_val >= IB_SUBNET_PATH_HOPS_MAX) - continue; - - hop_hist[hop_val] += cas_per_sw[lid_ho]; - } - - /* Now recognize the spines by requiring one bar to be above 90% of the - number of CAs and RTRs */ - for (i = 0 ; i < IB_SUBNET_PATH_HOPS_MAX; i++) { - if (hop_hist[i] > thd1) - numHopBarsOverThd1++; - if (hop_hist[i] > thd2) - numHopBarsOverThd2++; - } - - /* If thd conditions are valid insert the root node to the list */ - if ( (numHopBarsOverThd1 == 1) && (numHopBarsOverThd2 == 1) ) - { - p_guid = malloc(sizeof(uint64_t)); - if (p_guid) - { - *p_guid = cl_ntoh64(osm_node_get_node_guid(p_sw->p_node)); - osm_log( &p_osm->log, OSM_LOG_DEBUG, - "__osm_updn_find_root_nodes_by_min_hop: " - "Inserting GUID 0x%" PRIx64 " as root node\n", - *p_guid ); - cl_list_insert_tail(p_root_nodes_list, p_guid); - } - else - { - osm_log( &p_osm->log, OSM_LOG_ERROR, - "__osm_updn_find_root_nodes_by_min_hop: ERR AA13: " - "No memory for p_guid\n" ); - } - } - } - - free(cas_per_sw); - /* Now convert the cl_list to array */ - __osm_updn_convert_list2array(p_updn); - - _exit: - OSM_LOG_EXIT( &p_osm->log ); - return; + osm_opensm_t *p_osm = p_updn->p_osm; + osm_switch_t *p_next_sw, *p_sw; + osm_port_t *p_next_port, *p_port; + osm_physp_t *p_physp; + uint64_t *p_guid; + cl_list_t *p_root_nodes_list = p_updn->p_root_nodes; + double thd1, thd2; + unsigned i, cas_num = 0; + unsigned *cas_per_sw; + uint16_t lid_ho; + + OSM_LOG_ENTER(&p_osm->log, osm_updn_find_root_nodes_by_min_hop); + + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_find_root_nodes_by_min_hop: " + "Current number of ports in the subnet is %d\n", + cl_qmap_count(&p_osm->subn.port_guid_tbl)); + + cas_per_sw = malloc((IB_LID_UCAST_END_HO + 1) * sizeof(*cas_per_sw)); + if (!cas_per_sw) { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "__osm_updn_find_root_nodes_by_min_hop: ERR AA14: " + "cannot alloc mem for CAs per switch counter array\n"); + goto _exit; + } + memset(cas_per_sw, 0, (IB_LID_UCAST_END_HO + 1) * sizeof(*cas_per_sw)); + + /* Find the Maximum number of CAs (and routers) for histogram normalization */ + osm_log(&p_osm->log, OSM_LOG_VERBOSE, + "__osm_updn_find_root_nodes_by_min_hop: " + "Finding the number of CAs and storing them in cl_map\n"); + p_next_port = (osm_port_t *) cl_qmap_head(&p_osm->subn.port_guid_tbl); + while (p_next_port != + (osm_port_t *) cl_qmap_end(&p_osm->subn.port_guid_tbl)) { + p_port = p_next_port; + p_next_port = + (osm_port_t *) cl_qmap_next(&p_next_port->map_item); + if (!p_port->p_node->sw) { + p_physp = p_port->p_physp->p_remote_physp; + if (!p_physp || !p_physp->p_node->sw) + continue; + lid_ho = osm_node_get_base_lid(p_physp->p_node, 0); + lid_ho = cl_ntoh16(lid_ho); + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_find_root_nodes_by_min_hop: " + "Inserting GUID 0x%" PRIx64 + ", sw lid: 0x%X into array\n", + cl_ntoh64(osm_port_get_guid(p_port)), lid_ho); + cas_per_sw[lid_ho]++; + cas_num++; + } + } + + thd1 = cas_num * 0.9; + thd2 = cas_num * 0.05; + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_find_root_nodes_by_min_hop: " + "Found %u CAs and RTRs, %u SWs in the subnet. " + "Thresholds are thd1 = %f && thd2 = %f\n", + cas_num, cl_qmap_count(&p_osm->subn.sw_guid_tbl), thd1, thd2); + + p_next_sw = (osm_switch_t *) cl_qmap_head(&p_osm->subn.sw_guid_tbl); + osm_log(&p_osm->log, OSM_LOG_VERBOSE, + "__osm_updn_find_root_nodes_by_min_hop: " + "Passing through all switches to collect Min Hop info\n"); + while (p_next_sw != + (osm_switch_t *) cl_qmap_end(&p_osm->subn.sw_guid_tbl)) { + unsigned hop_hist[IB_SUBNET_PATH_HOPS_MAX]; + uint16_t max_lid_ho; + uint8_t hop_val; + uint16_t numHopBarsOverThd1 = 0; + uint16_t numHopBarsOverThd2 = 0; + + p_sw = p_next_sw; + /* Roll to the next switch */ + p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + + memset(hop_hist, 0, sizeof(hop_hist)); + + max_lid_ho = p_sw->max_lid_ho; + /* Get base lid of switch by retrieving port 0 lid of node pointer */ + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_find_root_nodes_by_min_hop: " + "Passing through switch lid 0x%X\n", + cl_ntoh16(osm_node_get_base_lid(p_sw->p_node, 0))); + for (lid_ho = 1; lid_ho <= max_lid_ho; lid_ho++) + if (cas_per_sw[lid_ho]) { + hop_val = + osm_switch_get_least_hops(p_sw, lid_ho); + if (hop_val >= IB_SUBNET_PATH_HOPS_MAX) + continue; + + hop_hist[hop_val] += cas_per_sw[lid_ho]; + } + + /* Now recognize the spines by requiring one bar to be above 90% of the + number of CAs and RTRs */ + for (i = 0; i < IB_SUBNET_PATH_HOPS_MAX; i++) { + if (hop_hist[i] > thd1) + numHopBarsOverThd1++; + if (hop_hist[i] > thd2) + numHopBarsOverThd2++; + } + + /* If thd conditions are valid insert the root node to the list */ + if ((numHopBarsOverThd1 == 1) && (numHopBarsOverThd2 == 1)) { + p_guid = malloc(sizeof(uint64_t)); + if (p_guid) { + *p_guid = + cl_ntoh64(osm_node_get_node_guid + (p_sw->p_node)); + osm_log(&p_osm->log, OSM_LOG_DEBUG, + "__osm_updn_find_root_nodes_by_min_hop: " + "Inserting GUID 0x%" PRIx64 + " as root node\n", *p_guid); + cl_list_insert_tail(p_root_nodes_list, p_guid); + } else { + osm_log(&p_osm->log, OSM_LOG_ERROR, + "__osm_updn_find_root_nodes_by_min_hop: ERR AA13: " + "No memory for p_guid\n"); + } + } + } + + free(cas_per_sw); + /* Now convert the cl_list to array */ + __osm_updn_convert_list2array(p_updn); + + _exit: + OSM_LOG_EXIT(&p_osm->log); + return; } /********************************************************************** **********************************************************************/ -static void -__osm_updn_delete( - void *context ) +static void __osm_updn_delete(void *context) { - updn_t *p_updn = context; + updn_t *p_updn = context; - updn_destroy(p_updn); + updn_destroy(p_updn); } -int -osm_ucast_updn_setup( - osm_opensm_t *p_osm ) +int osm_ucast_updn_setup(osm_opensm_t * p_osm) { - updn_t *p_updn; + updn_t *p_updn; - p_updn = updn_construct(&p_osm->log); - if (!p_updn) - return -1; + p_updn = updn_construct(&p_osm->log); + if (!p_updn) + return -1; - p_osm->routing_engine.context = p_updn; - p_osm->routing_engine.delete = __osm_updn_delete; - p_osm->routing_engine.build_lid_matrices = __osm_updn_call; + p_osm->routing_engine.context = p_updn; + p_osm->routing_engine.delete = __osm_updn_delete; + p_osm->routing_engine.build_lid_matrices = __osm_updn_call; - if (updn_init(p_updn, p_osm) != IB_SUCCESS) - return -1; + if (updn_init(p_updn, p_osm) != IB_SUCCESS) + return -1; - if (!p_updn->auto_detect_root_nodes) - __osm_updn_convert_list2array(p_updn); + if (!p_updn->auto_detect_root_nodes) + __osm_updn_convert_list2array(p_updn); - return 0; + return 0; } diff --git a/opensm/opensm/osm_vl_arb_rcv.c b/opensm/opensm/osm_vl_arb_rcv.c index f487058..78f15d6 100644 --- a/opensm/opensm/osm_vl_arb_rcv.c +++ b/opensm/opensm/osm_vl_arb_rcv.c @@ -33,7 +33,6 @@ * */ - /* * Abstract: * Implementation of osm_vla_rcv_t. @@ -48,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -69,49 +68,43 @@ /********************************************************************** **********************************************************************/ -void -osm_vla_rcv_construct( - IN osm_vla_rcv_t* const p_rcv ) +void osm_vla_rcv_construct(IN osm_vla_rcv_t * const p_rcv) { - memset( p_rcv, 0, sizeof(*p_rcv) ); + memset(p_rcv, 0, sizeof(*p_rcv)); } /********************************************************************** **********************************************************************/ -void -osm_vla_rcv_destroy( - IN osm_vla_rcv_t* const p_rcv ) +void osm_vla_rcv_destroy(IN osm_vla_rcv_t * const p_rcv) { - CL_ASSERT( p_rcv ); + CL_ASSERT(p_rcv); - OSM_LOG_ENTER( p_rcv->p_log, osm_vla_rcv_destroy ); + OSM_LOG_ENTER(p_rcv->p_log, osm_vla_rcv_destroy); - OSM_LOG_EXIT( p_rcv->p_log ); + OSM_LOG_EXIT(p_rcv->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vla_rcv_init( - IN osm_vla_rcv_t* const p_rcv, - IN osm_req_t* const p_req, - IN osm_subn_t* const p_subn, - IN osm_log_t* const p_log, - IN cl_plock_t* const p_lock ) +osm_vla_rcv_init(IN osm_vla_rcv_t * const p_rcv, + IN osm_req_t * const p_req, + IN osm_subn_t * const p_subn, + IN osm_log_t * const p_log, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; + ib_api_status_t status = IB_SUCCESS; - OSM_LOG_ENTER( p_log, osm_vla_rcv_init ); + OSM_LOG_ENTER(p_log, osm_vla_rcv_init); - osm_vla_rcv_construct( p_rcv ); + osm_vla_rcv_construct(p_rcv); - p_rcv->p_log = p_log; - p_rcv->p_subn = p_subn; - p_rcv->p_lock = p_lock; - p_rcv->p_req = p_req; + p_rcv->p_log = p_log; + p_rcv->p_subn = p_subn; + p_rcv->p_lock = p_lock; + p_rcv->p_req = p_req; - OSM_LOG_EXIT( p_log ); - return( status ); + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** @@ -119,120 +112,103 @@ osm_vla_rcv_init( /* * WE MIGHT ONLY RECEIVE GET or SET responses */ -void -osm_vla_rcv_process( - IN void *context, - IN void *data ) +void osm_vla_rcv_process(IN void *context, IN void *data) { - osm_vla_rcv_t *p_rcv = context; - osm_madw_t *p_madw = data; - ib_vl_arb_table_t *p_vla_tbl; - ib_smp_t *p_smp; - osm_port_t *p_port; - osm_physp_t *p_physp; - osm_node_t *p_node; - osm_vla_context_t *p_context; - ib_net64_t port_guid; - ib_net64_t node_guid; - uint8_t port_num, block_num; - - CL_ASSERT( p_rcv ); - - OSM_LOG_ENTER( p_rcv->p_log, osm_vla_rcv_process ); - - CL_ASSERT( p_madw ); - - p_smp = osm_madw_get_smp_ptr( p_madw ); - - p_context = osm_madw_get_vla_context_ptr( p_madw ); - p_vla_tbl = (ib_vl_arb_table_t*)ib_smp_get_payload_ptr( p_smp ); - - port_guid = p_context->port_guid; - node_guid = p_context->node_guid; - - CL_ASSERT( p_smp->attr_id == IB_MAD_ATTR_VL_ARBITRATION ); - - cl_plock_excl_acquire( p_rcv->p_lock ); - p_port = osm_get_port_by_guid( p_rcv->p_subn, port_guid ); - if( !p_port ) - { - cl_plock_release( p_rcv->p_lock ); - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vla_rcv_process: ERR 3F06: " - "No port object for port with GUID 0x%" PRIx64 - "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - goto Exit; - } - - p_node = p_port->p_node; - CL_ASSERT( p_node ); - - block_num = (uint8_t)(cl_ntoh32(p_smp->attr_mod) >> 16); - /* in case of a non switch node the attr modifier should be ignored */ - if (osm_node_get_type( p_node ) == IB_NODE_TYPE_SWITCH) - { - port_num = (uint8_t)(cl_ntoh32(p_smp->attr_mod) & 0x000000FF); - p_physp = osm_node_get_physp_ptr( p_node, port_num ); - } - else - { - p_physp = p_port->p_physp; - port_num = p_physp->port_num; - } - - CL_ASSERT( p_physp ); - - /* - We do not mind if this is a result of a set or get - all we want is to update - the subnet. - */ - if( osm_log_is_active( p_rcv->p_log, OSM_LOG_VERBOSE ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_VERBOSE, - "osm_vla_rcv_process: " - "Got GetResp(VLArb) block:%u port_num %u with GUID 0x%" PRIx64 - " for parent node GUID 0x%" PRIx64 - ", TID 0x%" PRIx64 "\n", - block_num, port_num, - cl_ntoh64( port_guid ), - cl_ntoh64( node_guid ), - cl_ntoh64( p_smp->trans_id ) ); - } - - /* - Determine if we encountered a new Physical Port. - If so, Ignore it. - */ - if( !osm_physp_is_valid( p_physp ) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vla_rcv_process: " - "Got invalid port number 0x%X\n", - port_num ); - goto Exit; - } - - osm_dump_vl_arb_table( p_rcv->p_log, - port_guid, block_num, - port_num, p_vla_tbl, - OSM_LOG_DEBUG ); - - if ( (block_num < 1) || (block_num > 4) ) - { - osm_log( p_rcv->p_log, OSM_LOG_ERROR, - "osm_vla_rcv_process: " - "Got invalid block number 0x%X\n", - block_num ); - goto Exit; - } - osm_physp_set_vla_tbl( p_physp, p_vla_tbl, block_num); - - Exit: - cl_plock_release( p_rcv->p_lock ); - - OSM_LOG_EXIT( p_rcv->p_log ); + osm_vla_rcv_t *p_rcv = context; + osm_madw_t *p_madw = data; + ib_vl_arb_table_t *p_vla_tbl; + ib_smp_t *p_smp; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_node_t *p_node; + osm_vla_context_t *p_context; + ib_net64_t port_guid; + ib_net64_t node_guid; + uint8_t port_num, block_num; + + CL_ASSERT(p_rcv); + + OSM_LOG_ENTER(p_rcv->p_log, osm_vla_rcv_process); + + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + + p_context = osm_madw_get_vla_context_ptr(p_madw); + p_vla_tbl = (ib_vl_arb_table_t *) ib_smp_get_payload_ptr(p_smp); + + port_guid = p_context->port_guid; + node_guid = p_context->node_guid; + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_VL_ARBITRATION); + + cl_plock_excl_acquire(p_rcv->p_lock); + p_port = osm_get_port_by_guid(p_rcv->p_subn, port_guid); + if (!p_port) { + cl_plock_release(p_rcv->p_lock); + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vla_rcv_process: ERR 3F06: " + "No port object for port with GUID 0x%" PRIx64 + "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 + ", TID 0x%" PRIx64 "\n", + cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + goto Exit; + } + + p_node = p_port->p_node; + CL_ASSERT(p_node); + + block_num = (uint8_t) (cl_ntoh32(p_smp->attr_mod) >> 16); + /* in case of a non switch node the attr modifier should be ignored */ + if (osm_node_get_type(p_node) == IB_NODE_TYPE_SWITCH) { + port_num = (uint8_t) (cl_ntoh32(p_smp->attr_mod) & 0x000000FF); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + } else { + p_physp = p_port->p_physp; + port_num = p_physp->port_num; + } + + CL_ASSERT(p_physp); + + /* + We do not mind if this is a result of a set or get - all we want is to update + the subnet. + */ + if (osm_log_is_active(p_rcv->p_log, OSM_LOG_VERBOSE)) { + osm_log(p_rcv->p_log, OSM_LOG_VERBOSE, + "osm_vla_rcv_process: " + "Got GetResp(VLArb) block:%u port_num %u with GUID 0x%" + PRIx64 " for parent node GUID 0x%" PRIx64 ", TID 0x%" + PRIx64 "\n", block_num, port_num, cl_ntoh64(port_guid), + cl_ntoh64(node_guid), cl_ntoh64(p_smp->trans_id)); + } + + /* + Determine if we encountered a new Physical Port. + If so, Ignore it. + */ + if (!osm_physp_is_valid(p_physp)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vla_rcv_process: " + "Got invalid port number 0x%X\n", port_num); + goto Exit; + } + + osm_dump_vl_arb_table(p_rcv->p_log, + port_guid, block_num, + port_num, p_vla_tbl, OSM_LOG_DEBUG); + + if ((block_num < 1) || (block_num > 4)) { + osm_log(p_rcv->p_log, OSM_LOG_ERROR, + "osm_vla_rcv_process: " + "Got invalid block number 0x%X\n", block_num); + goto Exit; + } + osm_physp_set_vla_tbl(p_physp, p_vla_tbl, block_num); + + Exit: + cl_plock_release(p_rcv->p_lock); + + OSM_LOG_EXIT(p_rcv->p_log); } diff --git a/opensm/opensm/st.c b/opensm/opensm/st.c index 6916e31..c5a2f53 100644 --- a/opensm/opensm/st.c +++ b/opensm/opensm/st.c @@ -33,12 +33,11 @@ * */ - /* static char sccsid[] = "@(#) st.c 5.1 89/12/14 Crucible"; */ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -51,10 +50,10 @@ typedef struct st_table_entry st_table_entry; struct st_table_entry { - unsigned int hash; - st_data_t key; - st_data_t record; - st_table_entry *next; + unsigned int hash; + st_data_t key; + st_data_t record; + st_table_entry *next; }; #define ST_DEFAULT_MAX_DENSITY 5 @@ -72,30 +71,28 @@ struct st_table_entry { static int numcmp(void *, void *); static st_ptr_t numhash(void *); static struct st_hash_type type_numhash = { - numcmp, - numhash, + numcmp, + numhash, }; - /* extern int strcmp(const char *, const char *); */ static int strhash(const char *); static inline st_ptr_t st_strhash(void *key) { - return strhash((const char *)key); + return strhash((const char *)key); } static inline int st_strcmp(void *key1, void *key2) { - return strcmp((const char *)key1, (const char *)key2); + return strcmp((const char *)key1, (const char *)key2); } static struct st_hash_type type_strhash = { - st_strcmp, - st_strhash + st_strcmp, + st_strhash }; - #define xmalloc malloc #define xcalloc calloc #define xrealloc realloc @@ -121,58 +118,57 @@ static void rehash(st_table *); Table of prime numbers 2^n+a, 2<=n<=30. */ static long primes[] = { - 8 + 3, - 16 + 3, - 32 + 5, - 64 + 3, - 128 + 3, - 256 + 27, - 512 + 9, - 1024 + 9, - 2048 + 5, - 4096 + 3, - 8192 + 27, - 16384 + 43, - 32768 + 3, - 65536 + 45, - 131072 + 29, - 262144 + 3, - 524288 + 21, - 1048576 + 7, - 2097152 + 17, - 4194304 + 15, - 8388608 + 9, - 16777216 + 43, - 33554432 + 35, - 67108864 + 15, - 134217728 + 29, - 268435456 + 3, - 536870912 + 11, - 1073741824 + 85, - 0 + 8 + 3, + 16 + 3, + 32 + 5, + 64 + 3, + 128 + 3, + 256 + 27, + 512 + 9, + 1024 + 9, + 2048 + 5, + 4096 + 3, + 8192 + 27, + 16384 + 43, + 32768 + 3, + 65536 + 45, + 131072 + 29, + 262144 + 3, + 524288 + 21, + 1048576 + 7, + 2097152 + 17, + 4194304 + 15, + 8388608 + 9, + 16777216 + 43, + 33554432 + 35, + 67108864 + 15, + 134217728 + 29, + 268435456 + 3, + 536870912 + 11, + 1073741824 + 85, + 0 }; -static int -new_size(int size) +static int new_size(int size) { - int i; + int i; #if 0 - for (i=3; i<31; i++) { - if ((1< size) return 1< size) + return 1 << i; + } + return -1; #else - int newsize; - - for (i = 0, newsize = MINSIZE; - i < sizeof(primes)/sizeof(primes[0]); - i++, newsize <<= 1) - { - if (newsize > size) return primes[i]; - } - /* Ran out of polynomials */ - return -1; /* should raise exception */ + int newsize; + + for (i = 0, newsize = MINSIZE; + i < sizeof(primes) / sizeof(primes[0]); i++, newsize <<= 1) { + if (newsize > size) + return primes[i]; + } + /* Ran out of polynomials */ + return -1; /* should raise exception */ #endif } @@ -180,91 +176,82 @@ new_size(int size) static int collision = 0; static int init_st = 0; -static void -stat_col() +static void stat_col() { - FILE *f = fopen("/var/log/osm_st_col", "w"); - fprintf(f, "collision: %d\n", collision); - fclose(f); + FILE *f = fopen("/var/log/osm_st_col", "w"); + fprintf(f, "collision: %d\n", collision); + fclose(f); } #endif -st_table* -st_init_table_with_size(type, size) - struct st_hash_type *type; - size_t size; +st_table *st_init_table_with_size(type, size) +struct st_hash_type *type; +size_t size; { - st_table *tbl; + st_table *tbl; #ifdef HASH_LOG - if (init_st == 0) - { - init_st = 1; - atexit(stat_col); - } + if (init_st == 0) { + init_st = 1; + atexit(stat_col); + } #endif - size = new_size(size); /* round up to prime number */ + size = new_size(size); /* round up to prime number */ - tbl = alloc(st_table); - tbl->type = type; - tbl->num_entries = 0; - tbl->num_bins = size; - tbl->bins = (st_table_entry **)Calloc(size, sizeof(st_table_entry*)); + tbl = alloc(st_table); + tbl->type = type; + tbl->num_entries = 0; + tbl->num_bins = size; + tbl->bins = (st_table_entry **) Calloc(size, sizeof(st_table_entry *)); - return tbl; + return tbl; } -st_table* -st_init_table(type) - struct st_hash_type *type; +st_table *st_init_table(type) +struct st_hash_type *type; { - return st_init_table_with_size(type, 0); + return st_init_table_with_size(type, 0); } -st_table* -st_init_numtable(void) +st_table *st_init_numtable(void) { - return st_init_table(&type_numhash); + return st_init_table(&type_numhash); } -st_table* -st_init_numtable_with_size(size) - size_t size; +st_table *st_init_numtable_with_size(size) +size_t size; { - return st_init_table_with_size(&type_numhash, size); + return st_init_table_with_size(&type_numhash, size); } -st_table* -st_init_strtable(void) +st_table *st_init_strtable(void) { - return st_init_table(&type_strhash); + return st_init_table(&type_strhash); } -st_table* -st_init_strtable_with_size(size) - size_t size; +st_table *st_init_strtable_with_size(size) +size_t size; { - return st_init_table_with_size(&type_strhash, size); + return st_init_table_with_size(&type_strhash, size); } -void -st_free_table(table) - st_table *table; +void st_free_table(table) +st_table *table; { - register st_table_entry *ptr, *next; - int i; - - for (i = 0; i < table->num_bins; i++) { - ptr = table->bins[i]; - while (ptr != 0) { - next = ptr->next; - free(ptr); - ptr = next; - } - } - free(table->bins); - free(table); + register st_table_entry *ptr, *next; + int i; + + for (i = 0; i < table->num_bins; i++) { + ptr = table->bins[i]; + while (ptr != 0) { + next = ptr->next; + free(ptr); + ptr = next; + } + } + free(table->bins); + free(table); } #define PTR_NOT_EQUAL(table, ptr, hash_val, key) \ @@ -289,28 +276,24 @@ st_free_table(table) }\ } while (0) -int -st_lookup(table, key, value) - st_table *table; - register st_data_t key; - st_data_t *value; +int st_lookup(table, key, value) +st_table *table; +register st_data_t key; +st_data_t *value; { - unsigned int hash_val, bin_pos; - register st_table_entry *ptr; - - hash_val = do_hash(key, table); - FIND_ENTRY(table, ptr, hash_val, bin_pos); - - if (ptr == 0) - { - return 0; - } - else - { - if (value != 0) - *value = ptr->record; - return 1; - } + unsigned int hash_val, bin_pos; + register st_table_entry *ptr; + + hash_val = do_hash(key, table); + FIND_ENTRY(table, ptr, hash_val, bin_pos); + + if (ptr == 0) { + return 0; + } else { + if (value != 0) + *value = ptr->record; + return 1; + } } #define ADD_DIRECT(table, key, value, hash_val, bin_pos)\ @@ -332,294 +315,274 @@ do {\ table->num_entries++;\ } while (0); - -int -st_insert(table, key, value) - register st_table *table; - register st_data_t key; - st_data_t value; +int st_insert(table, key, value) +register st_table *table; +register st_data_t key; +st_data_t value; { - unsigned int hash_val, bin_pos; - register st_table_entry *ptr; - - hash_val = do_hash(key, table); - FIND_ENTRY(table, ptr, hash_val, bin_pos); - - if (ptr == 0) - { - ADD_DIRECT(table, key, value, hash_val, bin_pos); - return 0; - } - else - { - ptr->record = value; - return 1; - } + unsigned int hash_val, bin_pos; + register st_table_entry *ptr; + + hash_val = do_hash(key, table); + FIND_ENTRY(table, ptr, hash_val, bin_pos); + + if (ptr == 0) { + ADD_DIRECT(table, key, value, hash_val, bin_pos); + return 0; + } else { + ptr->record = value; + return 1; + } } -void -st_add_direct(table, key, value) - st_table *table; - st_data_t key; - st_data_t value; +void st_add_direct(table, key, value) +st_table *table; +st_data_t key; +st_data_t value; { - unsigned int hash_val, bin_pos; + unsigned int hash_val, bin_pos; - hash_val = do_hash(key, table); - bin_pos = hash_val % table->num_bins; - ADD_DIRECT(table, key, value, hash_val, bin_pos); + hash_val = do_hash(key, table); + bin_pos = hash_val % table->num_bins; + ADD_DIRECT(table, key, value, hash_val, bin_pos); } -static void -rehash(table) - register st_table *table; +static void rehash(table) +register st_table *table; { - register st_table_entry *ptr, *next, **new_bins; - int i, old_num_bins = table->num_bins, new_num_bins; - unsigned int hash_val; - - new_num_bins = new_size(old_num_bins+1); - new_bins = (st_table_entry**)Calloc(new_num_bins, sizeof(st_table_entry*)); - - for (i = 0; i < old_num_bins; i++) { - ptr = table->bins[i]; - while (ptr != 0) { - next = ptr->next; - hash_val = ptr->hash % new_num_bins; - ptr->next = new_bins[hash_val]; - new_bins[hash_val] = ptr; - ptr = next; - } - } - free(table->bins); - table->num_bins = new_num_bins; - table->bins = new_bins; + register st_table_entry *ptr, *next, **new_bins; + int i, old_num_bins = table->num_bins, new_num_bins; + unsigned int hash_val; + + new_num_bins = new_size(old_num_bins + 1); + new_bins = + (st_table_entry **) Calloc(new_num_bins, sizeof(st_table_entry *)); + + for (i = 0; i < old_num_bins; i++) { + ptr = table->bins[i]; + while (ptr != 0) { + next = ptr->next; + hash_val = ptr->hash % new_num_bins; + ptr->next = new_bins[hash_val]; + new_bins[hash_val] = ptr; + ptr = next; + } + } + free(table->bins); + table->num_bins = new_num_bins; + table->bins = new_bins; } -st_table* -st_copy(old_table) - st_table *old_table; +st_table *st_copy(old_table) +st_table *old_table; { - st_table *new_table; - st_table_entry *ptr, *entry; - size_t i, num_bins = old_table->num_bins; - - new_table = alloc(st_table); - if (new_table == 0) - { - return 0; - } - - *new_table = *old_table; - new_table->bins = (st_table_entry**) - Calloc(num_bins, sizeof(st_table_entry*)); - - if (new_table->bins == 0) - { - free(new_table); - return 0; - } - - for (i = 0; i < num_bins; i++) { - new_table->bins[i] = 0; - ptr = old_table->bins[i]; - while (ptr != 0) { - entry = alloc(st_table_entry); - if (entry == 0) - { - free(new_table->bins); - free(new_table); - return 0; - } - *entry = *ptr; - entry->next = new_table->bins[i]; - new_table->bins[i] = entry; - ptr = ptr->next; - } - } - return new_table; + st_table *new_table; + st_table_entry *ptr, *entry; + size_t i, num_bins = old_table->num_bins; + + new_table = alloc(st_table); + if (new_table == 0) { + return 0; + } + + *new_table = *old_table; + new_table->bins = (st_table_entry **) + Calloc(num_bins, sizeof(st_table_entry *)); + + if (new_table->bins == 0) { + free(new_table); + return 0; + } + + for (i = 0; i < num_bins; i++) { + new_table->bins[i] = 0; + ptr = old_table->bins[i]; + while (ptr != 0) { + entry = alloc(st_table_entry); + if (entry == 0) { + free(new_table->bins); + free(new_table); + return 0; + } + *entry = *ptr; + entry->next = new_table->bins[i]; + new_table->bins[i] = entry; + ptr = ptr->next; + } + } + return new_table; } -int -st_delete(table, key, value) - register st_table *table; - register st_data_t *key; - st_data_t *value; +int st_delete(table, key, value) +register st_table *table; +register st_data_t *key; +st_data_t *value; { - unsigned int hash_val; - st_table_entry *tmp; - register st_table_entry *ptr; - - hash_val = do_hash_bin(*key, table); - ptr = table->bins[hash_val]; - - if (ptr == 0) - { - if (value != 0) *value = 0; - return 0; - } - - if (EQUAL(table, *key, ptr->key)) - { - table->bins[hash_val] = ptr->next; - table->num_entries--; - if (value != 0) *value = ptr->record; - *key = ptr->key; - free(ptr); - return 1; - } - - for (; ptr->next != 0; ptr = ptr->next) { - if (EQUAL(table, ptr->next->key, *key)) - { - tmp = ptr->next; - ptr->next = ptr->next->next; - table->num_entries--; - if (value != 0) *value = tmp->record; - *key = tmp->key; - free(tmp); - return 1; - } - } - - return 0; + unsigned int hash_val; + st_table_entry *tmp; + register st_table_entry *ptr; + + hash_val = do_hash_bin(*key, table); + ptr = table->bins[hash_val]; + + if (ptr == 0) { + if (value != 0) + *value = 0; + return 0; + } + + if (EQUAL(table, *key, ptr->key)) { + table->bins[hash_val] = ptr->next; + table->num_entries--; + if (value != 0) + *value = ptr->record; + *key = ptr->key; + free(ptr); + return 1; + } + + for (; ptr->next != 0; ptr = ptr->next) { + if (EQUAL(table, ptr->next->key, *key)) { + tmp = ptr->next; + ptr->next = ptr->next->next; + table->num_entries--; + if (value != 0) + *value = tmp->record; + *key = tmp->key; + free(tmp); + return 1; + } + } + + return 0; } -int -st_delete_safe(table, key, value, never) - register st_table *table; - register st_data_t *key; - st_data_t *value; - st_data_t never; +int st_delete_safe(table, key, value, never) +register st_table *table; +register st_data_t *key; +st_data_t *value; +st_data_t never; { - unsigned int hash_val; - register st_table_entry *ptr; - - hash_val = do_hash_bin(*key, table); - ptr = table->bins[hash_val]; - - if (ptr == 0) - { - if (value != 0) *value = 0; - return 0; - } - - for (; ptr != 0; ptr = ptr->next) { - if ((ptr->key != never) && EQUAL(table, ptr->key, *key)) - { - table->num_entries--; - *key = ptr->key; - if (value != 0) *value = ptr->record; - ptr->key = ptr->record = never; - return 1; - } - } - - return 0; + unsigned int hash_val; + register st_table_entry *ptr; + + hash_val = do_hash_bin(*key, table); + ptr = table->bins[hash_val]; + + if (ptr == 0) { + if (value != 0) + *value = 0; + return 0; + } + + for (; ptr != 0; ptr = ptr->next) { + if ((ptr->key != never) && EQUAL(table, ptr->key, *key)) { + table->num_entries--; + *key = ptr->key; + if (value != 0) + *value = ptr->record; + ptr->key = ptr->record = never; + return 1; + } + } + + return 0; } -static int -delete_never(st_data_t key, st_data_t value, st_data_t never) +static int delete_never(st_data_t key, st_data_t value, st_data_t never) { - if (value == never) return ST_DELETE; - return ST_CONTINUE; + if (value == never) + return ST_DELETE; + return ST_CONTINUE; } -void -st_cleanup_safe(table, never) - st_table *table; - st_data_t never; +void st_cleanup_safe(table, never) +st_table *table; +st_data_t never; { - int num_entries = table->num_entries; + int num_entries = table->num_entries; - st_foreach(table, delete_never, never); - table->num_entries = num_entries; + st_foreach(table, delete_never, never); + table->num_entries = num_entries; } -void -st_foreach(table, func, arg) - st_table *table; - int (*func)(st_data_t key, st_data_t val, st_data_t arg); - st_data_t arg; +void st_foreach(table, func, arg) +st_table *table; +int (*func) (st_data_t key, st_data_t val, st_data_t arg); +st_data_t arg; { - st_table_entry *ptr, *last, *tmp; - enum st_retval retval; - int i; - - for (i = 0; i < table->num_bins; i++) { - last = 0; - for (ptr = table->bins[i]; ptr != 0;) { - retval = (*func)(ptr->key, ptr->record, arg); - switch (retval) { - case ST_CONTINUE: - last = ptr; - ptr = ptr->next; - break; - case ST_STOP: - return; - case ST_DELETE: - tmp = ptr; - if (last == 0) - { - table->bins[i] = ptr->next; - } - else - { - last->next = ptr->next; - } - ptr = ptr->next; - free(tmp); - table->num_entries--; - } - } - } + st_table_entry *ptr, *last, *tmp; + enum st_retval retval; + int i; + + for (i = 0; i < table->num_bins; i++) { + last = 0; + for (ptr = table->bins[i]; ptr != 0;) { + retval = (*func) (ptr->key, ptr->record, arg); + switch (retval) { + case ST_CONTINUE: + last = ptr; + ptr = ptr->next; + break; + case ST_STOP: + return; + case ST_DELETE: + tmp = ptr; + if (last == 0) { + table->bins[i] = ptr->next; + } else { + last->next = ptr->next; + } + ptr = ptr->next; + free(tmp); + table->num_entries--; + } + } + } } -static int -strhash(string) - register const char *string; +static int strhash(string) +register const char *string; { - register int c; + register int c; #ifdef HASH_ELFHASH - register unsigned int h = 0, g; - - while ((c = *string++) != '\0') { - h = ( h << 4 ) + c; - if ( g = h & 0xF0000000 ) - h ^= g >> 24; - h &= ~g; - } - return h; + register unsigned int h = 0, g; + + while ((c = *string++) != '\0') { + h = (h << 4) + c; + if (g = h & 0xF0000000) + h ^= g >> 24; + h &= ~g; + } + return h; #elif HASH_PERL - register int val = 0; + register int val = 0; - while ((c = *string++) != '\0') { - val = val*33 + c; - } + while ((c = *string++) != '\0') { + val = val * 33 + c; + } - return val + (val>>5); + return val + (val >> 5); #else - register int val = 0; + register int val = 0; - while ((c = *string++) != '\0') { - val = val*997 + c; - } + while ((c = *string++) != '\0') { + val = val * 997 + c; + } - return val + (val>>5); + return val + (val >> 5); #endif } -static int -numcmp(x, y) - void *x, *y; +static int numcmp(x, y) +void *x, *y; { - return (st_ptr_t)x != (st_ptr_t)y; + return (st_ptr_t) x != (st_ptr_t) y; } -static st_ptr_t -numhash(n) - void *n; +static st_ptr_t numhash(n) +void *n; { - return (st_ptr_t)n; + return (st_ptr_t) n; } -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Thu Aug 16 14:43:17 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 00:43:17 +0300 Subject: [ofa-general] [PATCH] opensm/osm_subnet.c: indentation changes Message-ID: <20070816214317.GK27819@sashak.voltaire.com> Formatted with opensm/osm_indent. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_subnet.c | 2927 +++++++++++++++++++++----------------------- 1 files changed, 1420 insertions(+), 1507 deletions(-) diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index bd612f3..5b240e1 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -80,1677 +80,1590 @@ /********************************************************************** **********************************************************************/ -void -osm_subn_construct( - IN osm_subn_t* const p_subn ) +void osm_subn_construct(IN osm_subn_t * const p_subn) { - memset( p_subn, 0, sizeof(*p_subn) ); - cl_ptr_vector_construct( &p_subn->port_lid_tbl ); - cl_qmap_init( &p_subn->sw_guid_tbl ); - cl_qmap_init( &p_subn->node_guid_tbl ); - cl_qmap_init( &p_subn->port_guid_tbl ); - cl_qmap_init( &p_subn->sm_guid_tbl ); - cl_qlist_init( &p_subn->sa_sr_list ); - cl_qlist_init( &p_subn->sa_infr_list ); - cl_qmap_init( &p_subn->rtr_guid_tbl ); - cl_qmap_init( &p_subn->prtn_pkey_tbl ); - cl_qmap_init( &p_subn->mgrp_mlid_tbl ); + memset(p_subn, 0, sizeof(*p_subn)); + cl_ptr_vector_construct(&p_subn->port_lid_tbl); + cl_qmap_init(&p_subn->sw_guid_tbl); + cl_qmap_init(&p_subn->node_guid_tbl); + cl_qmap_init(&p_subn->port_guid_tbl); + cl_qmap_init(&p_subn->sm_guid_tbl); + cl_qlist_init(&p_subn->sa_sr_list); + cl_qlist_init(&p_subn->sa_infr_list); + cl_qmap_init(&p_subn->rtr_guid_tbl); + cl_qmap_init(&p_subn->prtn_pkey_tbl); + cl_qmap_init(&p_subn->mgrp_mlid_tbl); } /********************************************************************** **********************************************************************/ -void -osm_subn_destroy( - IN osm_subn_t* const p_subn ) +void osm_subn_destroy(IN osm_subn_t * const p_subn) { - osm_node_t *p_node, *p_next_node; - osm_port_t *p_port, *p_next_port; - osm_switch_t *p_sw, *p_next_sw; - osm_remote_sm_t *p_rsm, *p_next_rsm; - osm_prtn_t *p_prtn, *p_next_prtn; - osm_mgrp_t *p_mgrp, *p_next_mgrp; - osm_infr_t *p_infr, *p_next_infr; - - /* it might be a good idea to de-allocate all known objects */ - p_next_node = (osm_node_t*)cl_qmap_head( &p_subn->node_guid_tbl ); - while( p_next_node != (osm_node_t*)cl_qmap_end( &p_subn->node_guid_tbl ) ) - { - p_node = p_next_node; - p_next_node = (osm_node_t*)cl_qmap_next( &p_node->map_item ); - osm_node_delete( &p_node ); - } - - p_next_port = (osm_port_t*)cl_qmap_head( &p_subn->port_guid_tbl ); - while( p_next_port != (osm_port_t*)cl_qmap_end( &p_subn->port_guid_tbl ) ) - { - p_port = p_next_port; - p_next_port = (osm_port_t*)cl_qmap_next( &p_port->map_item ); - osm_port_delete( &p_port ); - } - - p_next_sw = (osm_switch_t*)cl_qmap_head( &p_subn->sw_guid_tbl ); - while( p_next_sw != (osm_switch_t*)cl_qmap_end( &p_subn->sw_guid_tbl ) ) - { - p_sw = p_next_sw; - p_next_sw = (osm_switch_t*)cl_qmap_next( &p_sw->map_item ); - osm_switch_delete( &p_sw ); - } - - p_next_rsm = (osm_remote_sm_t*)cl_qmap_head( &p_subn->sm_guid_tbl ); - while( p_next_rsm != (osm_remote_sm_t*)cl_qmap_end( &p_subn->sm_guid_tbl ) ) - { - p_rsm = p_next_rsm; - p_next_rsm = (osm_remote_sm_t*)cl_qmap_next( &p_rsm->map_item ); - free( p_rsm ); - } - - p_next_prtn = (osm_prtn_t*)cl_qmap_head( &p_subn->prtn_pkey_tbl ); - while( p_next_prtn != (osm_prtn_t*)cl_qmap_end( &p_subn->prtn_pkey_tbl ) ) - { - p_prtn = p_next_prtn; - p_next_prtn = (osm_prtn_t*)cl_qmap_next( &p_prtn->map_item ); - osm_prtn_delete( &p_prtn ); - } - - p_next_mgrp = (osm_mgrp_t*)cl_qmap_head( &p_subn->mgrp_mlid_tbl ); - while( p_next_mgrp != (osm_mgrp_t*)cl_qmap_end( &p_subn->mgrp_mlid_tbl ) ) - { - p_mgrp = p_next_mgrp; - p_next_mgrp = (osm_mgrp_t*)cl_qmap_next( &p_mgrp->map_item ); - osm_mgrp_delete( p_mgrp ); - } - - p_next_infr = (osm_infr_t*)cl_qlist_head( &p_subn->sa_infr_list ); - while (p_next_infr != (osm_infr_t*)cl_qlist_end( &p_subn->sa_infr_list ) ) - { - p_infr = p_next_infr; - p_next_infr = (osm_infr_t*)cl_qlist_next( &p_infr->list_item ); - osm_infr_delete( p_infr ); - } - - cl_ptr_vector_destroy( &p_subn->port_lid_tbl ); - cl_map_remove_all(&(p_subn->opt.port_prof_ignore_guids)); - cl_map_destroy(&(p_subn->opt.port_prof_ignore_guids)); + osm_node_t *p_node, *p_next_node; + osm_port_t *p_port, *p_next_port; + osm_switch_t *p_sw, *p_next_sw; + osm_remote_sm_t *p_rsm, *p_next_rsm; + osm_prtn_t *p_prtn, *p_next_prtn; + osm_mgrp_t *p_mgrp, *p_next_mgrp; + osm_infr_t *p_infr, *p_next_infr; + + /* it might be a good idea to de-allocate all known objects */ + p_next_node = (osm_node_t *) cl_qmap_head(&p_subn->node_guid_tbl); + while (p_next_node != + (osm_node_t *) cl_qmap_end(&p_subn->node_guid_tbl)) { + p_node = p_next_node; + p_next_node = (osm_node_t *) cl_qmap_next(&p_node->map_item); + osm_node_delete(&p_node); + } + + p_next_port = (osm_port_t *) cl_qmap_head(&p_subn->port_guid_tbl); + while (p_next_port != + (osm_port_t *) cl_qmap_end(&p_subn->port_guid_tbl)) { + p_port = p_next_port; + p_next_port = (osm_port_t *) cl_qmap_next(&p_port->map_item); + osm_port_delete(&p_port); + } + + p_next_sw = (osm_switch_t *) cl_qmap_head(&p_subn->sw_guid_tbl); + while (p_next_sw != (osm_switch_t *) cl_qmap_end(&p_subn->sw_guid_tbl)) { + p_sw = p_next_sw; + p_next_sw = (osm_switch_t *) cl_qmap_next(&p_sw->map_item); + osm_switch_delete(&p_sw); + } + + p_next_rsm = (osm_remote_sm_t *) cl_qmap_head(&p_subn->sm_guid_tbl); + while (p_next_rsm != + (osm_remote_sm_t *) cl_qmap_end(&p_subn->sm_guid_tbl)) { + p_rsm = p_next_rsm; + p_next_rsm = (osm_remote_sm_t *) cl_qmap_next(&p_rsm->map_item); + free(p_rsm); + } + + p_next_prtn = (osm_prtn_t *) cl_qmap_head(&p_subn->prtn_pkey_tbl); + while (p_next_prtn != + (osm_prtn_t *) cl_qmap_end(&p_subn->prtn_pkey_tbl)) { + p_prtn = p_next_prtn; + p_next_prtn = (osm_prtn_t *) cl_qmap_next(&p_prtn->map_item); + osm_prtn_delete(&p_prtn); + } + + p_next_mgrp = (osm_mgrp_t *) cl_qmap_head(&p_subn->mgrp_mlid_tbl); + while (p_next_mgrp != + (osm_mgrp_t *) cl_qmap_end(&p_subn->mgrp_mlid_tbl)) { + p_mgrp = p_next_mgrp; + p_next_mgrp = (osm_mgrp_t *) cl_qmap_next(&p_mgrp->map_item); + osm_mgrp_delete(p_mgrp); + } + + p_next_infr = (osm_infr_t *) cl_qlist_head(&p_subn->sa_infr_list); + while (p_next_infr != + (osm_infr_t *) cl_qlist_end(&p_subn->sa_infr_list)) { + p_infr = p_next_infr; + p_next_infr = (osm_infr_t *) cl_qlist_next(&p_infr->list_item); + osm_infr_delete(p_infr); + } + + cl_ptr_vector_destroy(&p_subn->port_lid_tbl); + cl_map_remove_all(&(p_subn->opt.port_prof_ignore_guids)); + cl_map_destroy(&(p_subn->opt.port_prof_ignore_guids)); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_subn_init( - IN osm_subn_t* const p_subn, - IN osm_opensm_t * const p_osm, - IN const osm_subn_opt_t* const p_opt ) +osm_subn_init(IN osm_subn_t * const p_subn, + IN osm_opensm_t * const p_osm, + IN const osm_subn_opt_t * const p_opt) { - cl_status_t status; + cl_status_t status; - p_subn->p_osm = p_osm; + p_subn->p_osm = p_osm; - status = cl_ptr_vector_init( &p_subn->port_lid_tbl, - OSM_SUBNET_VECTOR_MIN_SIZE, - OSM_SUBNET_VECTOR_GROW_SIZE ); - if( status != CL_SUCCESS ) - return( status ); + status = cl_ptr_vector_init(&p_subn->port_lid_tbl, + OSM_SUBNET_VECTOR_MIN_SIZE, + OSM_SUBNET_VECTOR_GROW_SIZE); + if (status != CL_SUCCESS) + return (status); - status = cl_ptr_vector_set_capacity( &p_subn->port_lid_tbl, - OSM_SUBNET_VECTOR_CAPACITY ); - if( status != CL_SUCCESS ) - return( status ); + status = cl_ptr_vector_set_capacity(&p_subn->port_lid_tbl, + OSM_SUBNET_VECTOR_CAPACITY); + if (status != CL_SUCCESS) + return (status); - /* - LID zero is not valid. NULL out this entry for the - convenience of other code. - */ - cl_ptr_vector_set( &p_subn->port_lid_tbl, 0, NULL ); + /* + LID zero is not valid. NULL out this entry for the + convenience of other code. + */ + cl_ptr_vector_set(&p_subn->port_lid_tbl, 0, NULL); - p_subn->opt = *p_opt; - p_subn->max_unicast_lid_ho = IB_LID_UCAST_END_HO; - p_subn->max_multicast_lid_ho = IB_LID_MCAST_END_HO; - p_subn->min_ca_mtu = IB_MAX_MTU; - p_subn->min_ca_rate = IB_MAX_RATE; + p_subn->opt = *p_opt; + p_subn->max_unicast_lid_ho = IB_LID_UCAST_END_HO; + p_subn->max_multicast_lid_ho = IB_LID_MCAST_END_HO; + p_subn->min_ca_mtu = IB_MAX_MTU; + p_subn->min_ca_rate = IB_MAX_RATE; - /* note that insert and remove are part of the port_profile thing */ - cl_map_init(&(p_subn->opt.port_prof_ignore_guids), 10); + /* note that insert and remove are part of the port_profile thing */ + cl_map_init(&(p_subn->opt.port_prof_ignore_guids), 10); - p_subn->ignore_existing_lfts = TRUE; + p_subn->ignore_existing_lfts = TRUE; - /* we assume master by default - so we only need to set it true if STANDBY */ - p_subn->coming_out_of_standby = FALSE; + /* we assume master by default - so we only need to set it true if STANDBY */ + p_subn->coming_out_of_standby = FALSE; - return( IB_SUCCESS ); + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_get_gid_by_mad_addr( - IN osm_log_t* p_log, - IN const osm_subn_t *p_subn, - IN const osm_mad_addr_t *p_mad_addr, - OUT ib_gid_t *p_gid) +osm_get_gid_by_mad_addr(IN osm_log_t * p_log, + IN const osm_subn_t * p_subn, + IN const osm_mad_addr_t * p_mad_addr, + OUT ib_gid_t * p_gid) { - const cl_ptr_vector_t* p_tbl; - const osm_port_t* p_port = NULL; - - if ( p_gid == NULL ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_get_gid_by_mad_addr: ERR 7505: " - "Provided output GID is NULL\n"); - return(IB_INVALID_PARAMETER); - } - - /* Find the port gid of the request in the subnet */ - p_tbl = &p_subn->port_lid_tbl; - - CL_ASSERT( cl_ptr_vector_get_size(p_tbl) < 0x10000 ); - - if ((uint16_t)cl_ptr_vector_get_size(p_tbl) > - cl_ntoh16(p_mad_addr->dest_lid)) - { - p_port = cl_ptr_vector_get( p_tbl, cl_ntoh16(p_mad_addr->dest_lid) ); - if ( p_port == NULL ) - { - osm_log( p_log, OSM_LOG_DEBUG, - "osm_get_gid_by_mad_addr: " - "Did not find any port with LID: 0x%X\n", - cl_ntoh16(p_mad_addr->dest_lid) - ); - return(IB_INVALID_PARAMETER); - } - p_gid->unicast.interface_id = p_port->p_physp->port_guid; - p_gid->unicast.prefix = p_subn->opt.subnet_prefix; - } - else - { - /* The dest_lid is not in the subnet table - this is an error */ - osm_log( p_log, OSM_LOG_ERROR, - "osm_get_gid_by_mad_addr: ERR 7501: " - "LID is out of range: 0x%X\n", - cl_ntoh16(p_mad_addr->dest_lid) - ); - return(IB_INVALID_PARAMETER); - } - - return( IB_SUCCESS ); + const cl_ptr_vector_t *p_tbl; + const osm_port_t *p_port = NULL; + + if (p_gid == NULL) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_get_gid_by_mad_addr: ERR 7505: " + "Provided output GID is NULL\n"); + return (IB_INVALID_PARAMETER); + } + + /* Find the port gid of the request in the subnet */ + p_tbl = &p_subn->port_lid_tbl; + + CL_ASSERT(cl_ptr_vector_get_size(p_tbl) < 0x10000); + + if ((uint16_t) cl_ptr_vector_get_size(p_tbl) > + cl_ntoh16(p_mad_addr->dest_lid)) { + p_port = + cl_ptr_vector_get(p_tbl, cl_ntoh16(p_mad_addr->dest_lid)); + if (p_port == NULL) { + osm_log(p_log, OSM_LOG_DEBUG, + "osm_get_gid_by_mad_addr: " + "Did not find any port with LID: 0x%X\n", + cl_ntoh16(p_mad_addr->dest_lid) + ); + return (IB_INVALID_PARAMETER); + } + p_gid->unicast.interface_id = p_port->p_physp->port_guid; + p_gid->unicast.prefix = p_subn->opt.subnet_prefix; + } else { + /* The dest_lid is not in the subnet table - this is an error */ + osm_log(p_log, OSM_LOG_ERROR, + "osm_get_gid_by_mad_addr: ERR 7501: " + "LID is out of range: 0x%X\n", + cl_ntoh16(p_mad_addr->dest_lid) + ); + return (IB_INVALID_PARAMETER); + } + + return (IB_SUCCESS); } /********************************************************************** **********************************************************************/ -osm_physp_t* -osm_get_physp_by_mad_addr( - IN osm_log_t* p_log, - IN const osm_subn_t *p_subn, - IN osm_mad_addr_t *p_mad_addr ) +osm_physp_t *osm_get_physp_by_mad_addr(IN osm_log_t * p_log, + IN const osm_subn_t * p_subn, + IN osm_mad_addr_t * p_mad_addr) { - const cl_ptr_vector_t* p_port_lid_tbl; - osm_port_t* p_port = NULL; - osm_physp_t* p_physp = NULL; - - /* Find the port gid of the request in the subnet */ - p_port_lid_tbl = &p_subn->port_lid_tbl; - - CL_ASSERT( cl_ptr_vector_get_size(p_port_lid_tbl) < 0x10000 ); - - if ((uint16_t)cl_ptr_vector_get_size(p_port_lid_tbl) > - cl_ntoh16(p_mad_addr->dest_lid)) - { - p_port = cl_ptr_vector_get( p_port_lid_tbl, cl_ntoh16(p_mad_addr->dest_lid) ); - if (p_port == NULL) - { - /* The port is not in the port_lid table - this is an error */ - osm_log( p_log, OSM_LOG_ERROR, - "osm_get_physp_by_mad_addr: ERR 7502: " - "Cannot locate port object by lid: 0x%X\n", - cl_ntoh16(p_mad_addr->dest_lid) - ); - - goto Exit; - } - p_physp = p_port->p_physp; - } - else - { - /* The dest_lid is not in the subnet table - this is an error */ - osm_log( p_log, OSM_LOG_ERROR, - "osm_get_physp_by_mad_addr: ERR 7503: " - "Lid is out of range: 0x%X\n", - cl_ntoh16(p_mad_addr->dest_lid) - ); - } - - Exit: - return p_physp; + const cl_ptr_vector_t *p_port_lid_tbl; + osm_port_t *p_port = NULL; + osm_physp_t *p_physp = NULL; + + /* Find the port gid of the request in the subnet */ + p_port_lid_tbl = &p_subn->port_lid_tbl; + + CL_ASSERT(cl_ptr_vector_get_size(p_port_lid_tbl) < 0x10000); + + if ((uint16_t) cl_ptr_vector_get_size(p_port_lid_tbl) > + cl_ntoh16(p_mad_addr->dest_lid)) { + p_port = + cl_ptr_vector_get(p_port_lid_tbl, + cl_ntoh16(p_mad_addr->dest_lid)); + if (p_port == NULL) { + /* The port is not in the port_lid table - this is an error */ + osm_log(p_log, OSM_LOG_ERROR, + "osm_get_physp_by_mad_addr: ERR 7502: " + "Cannot locate port object by lid: 0x%X\n", + cl_ntoh16(p_mad_addr->dest_lid) + ); + + goto Exit; + } + p_physp = p_port->p_physp; + } else { + /* The dest_lid is not in the subnet table - this is an error */ + osm_log(p_log, OSM_LOG_ERROR, + "osm_get_physp_by_mad_addr: ERR 7503: " + "Lid is out of range: 0x%X\n", + cl_ntoh16(p_mad_addr->dest_lid) + ); + } + + Exit: + return p_physp; } /********************************************************************** **********************************************************************/ -osm_port_t* -osm_get_port_by_mad_addr( - IN osm_log_t* p_log, - IN const osm_subn_t *p_subn, - IN osm_mad_addr_t *p_mad_addr ) +osm_port_t *osm_get_port_by_mad_addr(IN osm_log_t * p_log, + IN const osm_subn_t * p_subn, + IN osm_mad_addr_t * p_mad_addr) { - const cl_ptr_vector_t* p_port_lid_tbl; - osm_port_t* p_port = NULL; - - /* Find the port gid of the request in the subnet */ - p_port_lid_tbl = &p_subn->port_lid_tbl; - - CL_ASSERT( cl_ptr_vector_get_size(p_port_lid_tbl) < 0x10000 ); - - if ((uint16_t)cl_ptr_vector_get_size(p_port_lid_tbl) > - cl_ntoh16(p_mad_addr->dest_lid)) - { - p_port = - cl_ptr_vector_get( p_port_lid_tbl, cl_ntoh16(p_mad_addr->dest_lid) ); - } - else - { - /* The dest_lid is not in the subnet table - this is an error */ - osm_log( p_log, OSM_LOG_ERROR, - "osm_get_port_by_mad_addr: ERR 7504: " - "Lid is out of range: 0x%X\n", - cl_ntoh16(p_mad_addr->dest_lid) - ); - } - - return p_port; + const cl_ptr_vector_t *p_port_lid_tbl; + osm_port_t *p_port = NULL; + + /* Find the port gid of the request in the subnet */ + p_port_lid_tbl = &p_subn->port_lid_tbl; + + CL_ASSERT(cl_ptr_vector_get_size(p_port_lid_tbl) < 0x10000); + + if ((uint16_t) cl_ptr_vector_get_size(p_port_lid_tbl) > + cl_ntoh16(p_mad_addr->dest_lid)) { + p_port = + cl_ptr_vector_get(p_port_lid_tbl, + cl_ntoh16(p_mad_addr->dest_lid)); + } else { + /* The dest_lid is not in the subnet table - this is an error */ + osm_log(p_log, OSM_LOG_ERROR, + "osm_get_port_by_mad_addr: ERR 7504: " + "Lid is out of range: 0x%X\n", + cl_ntoh16(p_mad_addr->dest_lid) + ); + } + + return p_port; } /********************************************************************** **********************************************************************/ -osm_switch_t * -osm_get_switch_by_guid( - IN const osm_subn_t *p_subn, - IN uint64_t guid) +osm_switch_t *osm_get_switch_by_guid(IN const osm_subn_t * p_subn, + IN uint64_t guid) { - osm_switch_t *p_switch; + osm_switch_t *p_switch; - p_switch = (osm_switch_t*)cl_qmap_get( &(p_subn->sw_guid_tbl), guid ); - if( p_switch == (osm_switch_t*)cl_qmap_end( &(p_subn->sw_guid_tbl)) ) - p_switch = NULL; - return p_switch; + p_switch = (osm_switch_t *) cl_qmap_get(&(p_subn->sw_guid_tbl), guid); + if (p_switch == (osm_switch_t *) cl_qmap_end(&(p_subn->sw_guid_tbl))) + p_switch = NULL; + return p_switch; } /********************************************************************** **********************************************************************/ -osm_node_t * -osm_get_node_by_guid( - IN osm_subn_t const *p_subn, - IN uint64_t guid) +osm_node_t *osm_get_node_by_guid(IN osm_subn_t const *p_subn, IN uint64_t guid) { - osm_node_t *p_node; + osm_node_t *p_node; - p_node = (osm_node_t*)cl_qmap_get( &(p_subn->node_guid_tbl), guid ); - if( p_node == (osm_node_t*)cl_qmap_end( &(p_subn->node_guid_tbl)) ) - p_node = NULL; - return p_node; + p_node = (osm_node_t *) cl_qmap_get(&(p_subn->node_guid_tbl), guid); + if (p_node == (osm_node_t *) cl_qmap_end(&(p_subn->node_guid_tbl))) + p_node = NULL; + return p_node; } /********************************************************************** **********************************************************************/ -osm_port_t * -osm_get_port_by_guid( - IN osm_subn_t const *p_subn, - IN uint64_t guid) +osm_port_t *osm_get_port_by_guid(IN osm_subn_t const *p_subn, IN uint64_t guid) { - osm_port_t *p_port; + osm_port_t *p_port; - p_port = (osm_port_t*)cl_qmap_get( &(p_subn->port_guid_tbl), guid ); - if( p_port == (osm_port_t*)cl_qmap_end( &(p_subn->port_guid_tbl)) ) - p_port = NULL; - return p_port; + p_port = (osm_port_t *) cl_qmap_get(&(p_subn->port_guid_tbl), guid); + if (p_port == (osm_port_t *) cl_qmap_end(&(p_subn->port_guid_tbl))) + p_port = NULL; + return p_port; } /********************************************************************** **********************************************************************/ -static void -subn_set_default_qos_options( - IN osm_qos_options_t *opt) +static void subn_set_default_qos_options(IN osm_qos_options_t * opt) { - opt->max_vls = OSM_DEFAULT_QOS_MAX_VLS; - opt->high_limit = OSM_DEFAULT_QOS_HIGH_LIMIT; - opt->vlarb_high = OSM_DEFAULT_QOS_VLARB_HIGH; - opt->vlarb_low = OSM_DEFAULT_QOS_VLARB_LOW; - opt->sl2vl = OSM_DEFAULT_QOS_SL2VL; + opt->max_vls = OSM_DEFAULT_QOS_MAX_VLS; + opt->high_limit = OSM_DEFAULT_QOS_HIGH_LIMIT; + opt->vlarb_high = OSM_DEFAULT_QOS_VLARB_HIGH; + opt->vlarb_low = OSM_DEFAULT_QOS_VLARB_LOW; + opt->sl2vl = OSM_DEFAULT_QOS_SL2VL; } /********************************************************************** **********************************************************************/ -void -osm_subn_set_default_opt( - IN osm_subn_opt_t* const p_opt ) +void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) { - memset(p_opt, 0, sizeof(osm_subn_opt_t)); - p_opt->guid = 0; - p_opt->m_key = OSM_DEFAULT_M_KEY; - p_opt->sm_key = OSM_DEFAULT_SM_KEY; - p_opt->subnet_prefix = IB_DEFAULT_SUBNET_PREFIX; - p_opt->m_key_lease_period = 0; - p_opt->sweep_interval = OSM_DEFAULT_SWEEP_INTERVAL_SECS; - p_opt->max_wire_smps = OSM_DEFAULT_SMP_MAX_ON_WIRE; - p_opt->console = OSM_DEFAULT_CONSOLE; - p_opt->console_port = OSM_DEFAULT_CONSOLE_PORT; - p_opt->transaction_timeout = OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC; - /* by default we will consider waiting for 50x transaction timeout normal */ - p_opt->max_msg_fifo_timeout = 50*OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC; - p_opt->sm_priority = OSM_DEFAULT_SM_PRIORITY; - p_opt->lmc = OSM_DEFAULT_LMC; - p_opt->lmc_esp0 = FALSE; - p_opt->max_op_vls = OSM_DEFAULT_MAX_OP_VLS; - p_opt->force_link_speed = 15; - p_opt->reassign_lids = FALSE; - p_opt->ignore_other_sm = FALSE; - p_opt->single_thread = FALSE; - p_opt->no_multicast_option = FALSE; - p_opt->disable_multicast = FALSE; - p_opt->force_log_flush = FALSE; - p_opt->subnet_timeout = OSM_DEFAULT_SUBNET_TIMEOUT; - p_opt->packet_life_time = OSM_DEFAULT_SWITCH_PACKET_LIFE; - p_opt->vl_stall_count = OSM_DEFAULT_VL_STALL_COUNT; - p_opt->leaf_vl_stall_count = OSM_DEFAULT_LEAF_VL_STALL_COUNT; - p_opt->head_of_queue_lifetime = OSM_DEFAULT_HEAD_OF_QUEUE_LIFE; - p_opt->leaf_head_of_queue_lifetime = OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE; - p_opt->local_phy_errors_threshold = OSM_DEFAULT_ERROR_THRESHOLD; - p_opt->overrun_errors_threshold = OSM_DEFAULT_ERROR_THRESHOLD; - p_opt->sminfo_polling_timeout = OSM_SM_DEFAULT_POLLING_TIMEOUT_MILLISECS; - p_opt->polling_retry_number = OSM_SM_DEFAULT_POLLING_RETRY_NUMBER; - p_opt->force_heavy_sweep = FALSE; - p_opt->log_flags = OSM_LOG_DEFAULT_LEVEL; - p_opt->honor_guid2lid_file = FALSE; - p_opt->daemon = FALSE; - p_opt->sm_inactive = FALSE; - p_opt->babbling_port_policy = FALSE; + memset(p_opt, 0, sizeof(osm_subn_opt_t)); + p_opt->guid = 0; + p_opt->m_key = OSM_DEFAULT_M_KEY; + p_opt->sm_key = OSM_DEFAULT_SM_KEY; + p_opt->subnet_prefix = IB_DEFAULT_SUBNET_PREFIX; + p_opt->m_key_lease_period = 0; + p_opt->sweep_interval = OSM_DEFAULT_SWEEP_INTERVAL_SECS; + p_opt->max_wire_smps = OSM_DEFAULT_SMP_MAX_ON_WIRE; + p_opt->console = OSM_DEFAULT_CONSOLE; + p_opt->console_port = OSM_DEFAULT_CONSOLE_PORT; + p_opt->transaction_timeout = OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC; + /* by default we will consider waiting for 50x transaction timeout normal */ + p_opt->max_msg_fifo_timeout = 50 * OSM_DEFAULT_TRANS_TIMEOUT_MILLISEC; + p_opt->sm_priority = OSM_DEFAULT_SM_PRIORITY; + p_opt->lmc = OSM_DEFAULT_LMC; + p_opt->lmc_esp0 = FALSE; + p_opt->max_op_vls = OSM_DEFAULT_MAX_OP_VLS; + p_opt->force_link_speed = 15; + p_opt->reassign_lids = FALSE; + p_opt->ignore_other_sm = FALSE; + p_opt->single_thread = FALSE; + p_opt->no_multicast_option = FALSE; + p_opt->disable_multicast = FALSE; + p_opt->force_log_flush = FALSE; + p_opt->subnet_timeout = OSM_DEFAULT_SUBNET_TIMEOUT; + p_opt->packet_life_time = OSM_DEFAULT_SWITCH_PACKET_LIFE; + p_opt->vl_stall_count = OSM_DEFAULT_VL_STALL_COUNT; + p_opt->leaf_vl_stall_count = OSM_DEFAULT_LEAF_VL_STALL_COUNT; + p_opt->head_of_queue_lifetime = OSM_DEFAULT_HEAD_OF_QUEUE_LIFE; + p_opt->leaf_head_of_queue_lifetime = + OSM_DEFAULT_LEAF_HEAD_OF_QUEUE_LIFE; + p_opt->local_phy_errors_threshold = OSM_DEFAULT_ERROR_THRESHOLD; + p_opt->overrun_errors_threshold = OSM_DEFAULT_ERROR_THRESHOLD; + p_opt->sminfo_polling_timeout = + OSM_SM_DEFAULT_POLLING_TIMEOUT_MILLISECS; + p_opt->polling_retry_number = OSM_SM_DEFAULT_POLLING_RETRY_NUMBER; + p_opt->force_heavy_sweep = FALSE; + p_opt->log_flags = OSM_LOG_DEFAULT_LEVEL; + p_opt->honor_guid2lid_file = FALSE; + p_opt->daemon = FALSE; + p_opt->sm_inactive = FALSE; + p_opt->babbling_port_policy = FALSE; #ifdef ENABLE_OSM_PERF_MGR - p_opt->perfmgr = FALSE; - p_opt->perfmgr_redir = TRUE; - p_opt->perfmgr_sweep_time_s = OSM_PERFMGR_DEFAULT_SWEEP_TIME_S; - p_opt->perfmgr_max_outstanding_queries = - OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES; - p_opt->event_db_dump_file = OSM_PERFMGR_DEFAULT_DUMP_FILE; -#endif /* ENABLE_OSM_PERF_MGR */ - - p_opt->event_plugin_name = OSM_DEFAULT_EVENT_PLUGIN_NAME; - - p_opt->dump_files_dir = getenv("OSM_TMP_DIR"); - if (!p_opt->dump_files_dir || !(*p_opt->dump_files_dir)) - p_opt->dump_files_dir = OSM_DEFAULT_TMP_DIR; - - p_opt->log_file = OSM_DEFAULT_LOG_FILE; - p_opt->log_max_size = 0; - p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; - p_opt->no_partition_enforcement = FALSE; - p_opt->no_qos = TRUE; - p_opt->accum_log_file = TRUE; - p_opt->port_profile_switch_nodes = FALSE; - p_opt->pfn_ui_pre_lid_assign = NULL; - p_opt->ui_pre_lid_assign_ctx = NULL; - p_opt->pfn_ui_mcast_fdb_assign = NULL; - p_opt->ui_mcast_fdb_assign_ctx = NULL; - p_opt->sweep_on_trap = TRUE; - p_opt->testability_mode = OSM_TEST_MODE_NONE; - p_opt->routing_engine_name = NULL; - p_opt->connect_roots = FALSE; - p_opt->lid_matrix_dump_file = NULL; - p_opt->ucast_dump_file = NULL; - p_opt->root_guid_file = NULL; - p_opt->cn_guid_file = NULL; - p_opt->sa_db_file = NULL; - p_opt->exit_on_fatal = TRUE; - p_opt->enable_quirks = FALSE; - p_opt->no_clients_rereg = FALSE; - subn_set_default_qos_options(&p_opt->qos_options); - subn_set_default_qos_options(&p_opt->qos_ca_options); - subn_set_default_qos_options(&p_opt->qos_sw0_options); - subn_set_default_qos_options(&p_opt->qos_swe_options); - subn_set_default_qos_options(&p_opt->qos_rtr_options); + p_opt->perfmgr = FALSE; + p_opt->perfmgr_redir = TRUE; + p_opt->perfmgr_sweep_time_s = OSM_PERFMGR_DEFAULT_SWEEP_TIME_S; + p_opt->perfmgr_max_outstanding_queries = + OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES; + p_opt->event_db_dump_file = OSM_PERFMGR_DEFAULT_DUMP_FILE; +#endif /* ENABLE_OSM_PERF_MGR */ + + p_opt->event_plugin_name = OSM_DEFAULT_EVENT_PLUGIN_NAME; + + p_opt->dump_files_dir = getenv("OSM_TMP_DIR"); + if (!p_opt->dump_files_dir || !(*p_opt->dump_files_dir)) + p_opt->dump_files_dir = OSM_DEFAULT_TMP_DIR; + + p_opt->log_file = OSM_DEFAULT_LOG_FILE; + p_opt->log_max_size = 0; + p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; + p_opt->no_partition_enforcement = FALSE; + p_opt->no_qos = TRUE; + p_opt->accum_log_file = TRUE; + p_opt->port_profile_switch_nodes = FALSE; + p_opt->pfn_ui_pre_lid_assign = NULL; + p_opt->ui_pre_lid_assign_ctx = NULL; + p_opt->pfn_ui_mcast_fdb_assign = NULL; + p_opt->ui_mcast_fdb_assign_ctx = NULL; + p_opt->sweep_on_trap = TRUE; + p_opt->testability_mode = OSM_TEST_MODE_NONE; + p_opt->routing_engine_name = NULL; + p_opt->connect_roots = FALSE; + p_opt->lid_matrix_dump_file = NULL; + p_opt->ucast_dump_file = NULL; + p_opt->root_guid_file = NULL; + p_opt->cn_guid_file = NULL; + p_opt->sa_db_file = NULL; + p_opt->exit_on_fatal = TRUE; + p_opt->enable_quirks = FALSE; + p_opt->no_clients_rereg = FALSE; + subn_set_default_qos_options(&p_opt->qos_options); + subn_set_default_qos_options(&p_opt->qos_ca_options); + subn_set_default_qos_options(&p_opt->qos_sw0_options); + subn_set_default_qos_options(&p_opt->qos_swe_options); + subn_set_default_qos_options(&p_opt->qos_rtr_options); } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_net64( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN uint64_t *p_val) +__osm_subn_opts_unpack_net64(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN uint64_t * p_val) { - uint64_t val; + uint64_t val; - if (!strcmp(p_req_key, p_key)) - { + if (!strcmp(p_req_key, p_key)) { #if __WORDSIZE == 64 - val = strtoul(p_val_str, NULL, 0); + val = strtoul(p_val_str, NULL, 0); #else - val = strtoull(p_val_str, NULL, 0); + val = strtoull(p_val_str, NULL, 0); #endif - if (cl_hton64(val) != *p_val) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = 0x%016" PRIx64 "\n", - p_key, val); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = cl_ntoh64(val); - } - } + if (cl_hton64(val) != *p_val) { + char buff[128]; + sprintf(buff, + " Loading Cached Option:%s = 0x%016" PRIx64 + "\n", p_key, val); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = cl_ntoh64(val); + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_uint32( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN uint32_t *p_val) +__osm_subn_opts_unpack_uint32(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN uint32_t * p_val) { - uint32_t val; - - if (!strcmp(p_req_key, p_key)) - { - val = strtoul(p_val_str, NULL, 0); - if (val != *p_val) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = %u\n", - p_key, val); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = val; - } - } + uint32_t val; + + if (!strcmp(p_req_key, p_key)) { + val = strtoul(p_val_str, NULL, 0); + if (val != *p_val) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = %u\n", + p_key, val); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = val; + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_uint16( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN uint16_t *p_val) +__osm_subn_opts_unpack_uint16(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN uint16_t * p_val) { - uint16_t val; - - if (!strcmp(p_req_key, p_key)) - { - val = (uint16_t)strtoul(p_val_str, NULL, 0); - if (val != *p_val) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = %u\n", - p_key, val); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = val; - } - } + uint16_t val; + + if (!strcmp(p_req_key, p_key)) { + val = (uint16_t) strtoul(p_val_str, NULL, 0); + if (val != *p_val) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = %u\n", + p_key, val); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = val; + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_net16( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN uint16_t *p_val) +__osm_subn_opts_unpack_net16(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN uint16_t * p_val) { - if (!strcmp(p_req_key, p_key)) - { - uint32_t val; - val = strtoul(p_val_str, NULL, 0); - CL_ASSERT( val < 0x10000 ); - if (cl_hton32(val) != *p_val) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = 0x%04x\n", - p_key, val); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = cl_hton16((uint16_t)val); - } - } + if (!strcmp(p_req_key, p_key)) { + uint32_t val; + val = strtoul(p_val_str, NULL, 0); + CL_ASSERT(val < 0x10000); + if (cl_hton32(val) != *p_val) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = 0x%04x\n", + p_key, val); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = cl_hton16((uint16_t) val); + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_uint8( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN uint8_t *p_val) +__osm_subn_opts_unpack_uint8(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN uint8_t * p_val) { - if (!strcmp(p_req_key, p_key)) - { - uint32_t val; - val = strtoul(p_val_str, NULL, 0); - CL_ASSERT( val < 0x100 ); - if (val != *p_val) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = %u\n", - p_key, val); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = (uint8_t)val; - } - } + if (!strcmp(p_req_key, p_key)) { + uint32_t val; + val = strtoul(p_val_str, NULL, 0); + CL_ASSERT(val < 0x100); + if (val != *p_val) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = %u\n", + p_key, val); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = (uint8_t) val; + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_boolean( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN boolean_t *p_val) +__osm_subn_opts_unpack_boolean(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN boolean_t * p_val) { - if (!strcmp(p_req_key, p_key) && p_val_str) - { - boolean_t val; - if (strcmp("TRUE", p_val_str)) - val = FALSE; - else - val = TRUE; - - if (val != *p_val) { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = %s\n", - p_key, p_val_str); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *p_val = val; - } - } + if (!strcmp(p_req_key, p_key) && p_val_str) { + boolean_t val; + if (strcmp("TRUE", p_val_str)) + val = FALSE; + else + val = TRUE; + + if (val != *p_val) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = %s\n", + p_key, p_val_str); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *p_val = val; + } + } } /********************************************************************** **********************************************************************/ static inline void -__osm_subn_opts_unpack_charp( - IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, - IN char **p_val) +__osm_subn_opts_unpack_charp(IN char *p_req_key, + IN char *p_key, + IN char *p_val_str, IN char **p_val) { - if (!strcmp(p_req_key, p_key) && p_val_str) - { - if ((*p_val == NULL) || strcmp(p_val_str, *p_val)) - { - char buff[128]; - sprintf(buff, " Loading Cached Option:%s = %s\n", - p_key, p_val_str); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - - /* - Ignore the possible memory leak here; - the pointer may be to a static default. - */ - *p_val = (char *)malloc( strlen(p_val_str) +1 ); - strcpy( *p_val, p_val_str); - } - } + if (!strcmp(p_req_key, p_key) && p_val_str) { + if ((*p_val == NULL) || strcmp(p_val_str, *p_val)) { + char buff[128]; + sprintf(buff, " Loading Cached Option:%s = %s\n", + p_key, p_val_str); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + + /* + Ignore the possible memory leak here; + the pointer may be to a static default. + */ + *p_val = (char *)malloc(strlen(p_val_str) + 1); + strcpy(*p_val, p_val_str); + } + } } /********************************************************************** **********************************************************************/ static void -subn_parse_qos_options( - IN const char *prefix, - IN char *p_key, - IN char *p_val_str, - IN osm_qos_options_t *opt) +subn_parse_qos_options(IN const char *prefix, + IN char *p_key, + IN char *p_val_str, IN osm_qos_options_t * opt) { - char name[256]; - - snprintf(name, sizeof(name), "%s_max_vls", prefix); - __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->max_vls); - snprintf(name, sizeof(name), "%s_high_limit", prefix); - __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->high_limit); - snprintf(name, sizeof(name), "%s_vlarb_high", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_high); - snprintf(name, sizeof(name), "%s_vlarb_low", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_low); - snprintf(name, sizeof(name), "%s_sl2vl", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->sl2vl); + char name[256]; + + snprintf(name, sizeof(name), "%s_max_vls", prefix); + __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->max_vls); + snprintf(name, sizeof(name), "%s_high_limit", prefix); + __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->high_limit); + snprintf(name, sizeof(name), "%s_vlarb_high", prefix); + __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_high); + snprintf(name, sizeof(name), "%s_vlarb_low", prefix); + __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_low); + snprintf(name, sizeof(name), "%s_sl2vl", prefix); + __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->sl2vl); } static int -subn_dump_qos_options( - FILE *file, - const char *set_name, - const char *prefix, - osm_qos_options_t *opt) +subn_dump_qos_options(FILE * file, + const char *set_name, + const char *prefix, osm_qos_options_t * opt) { return fprintf(file, "# %s\n" - "%s_max_vls %u\n" - "%s_high_limit %u\n" - "%s_vlarb_high %s\n" - "%s_vlarb_low %s\n" - "%s_sl2vl %s\n", - set_name, - prefix, opt->max_vls, - prefix, opt->high_limit, - prefix, opt->vlarb_high, - prefix, opt->vlarb_low, - prefix, opt->sl2vl); + "%s_max_vls %u\n" + "%s_high_limit %u\n" + "%s_vlarb_high %s\n" + "%s_vlarb_low %s\n" + "%s_sl2vl %s\n", + set_name, + prefix, opt->max_vls, + prefix, opt->high_limit, + prefix, opt->vlarb_high, + prefix, opt->vlarb_low, prefix, opt->sl2vl); } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_subn_rescan_conf_file( - IN osm_subn_opt_t* const p_opts ) +ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) { - char *p_cache_dir = getenv("OSM_CACHE_DIR"); - char file_name[OSM_PATH_MAX]; - FILE *opts_file; - char line[1024]; - char *p_key, *p_val ,*p_last; - - /* try to open the options file from the cache dir */ - if (!p_cache_dir || !(*p_cache_dir)) - p_cache_dir = OSM_DEFAULT_CACHE_DIR; - - strcpy(file_name, p_cache_dir); - strcat(file_name, "/opensm.opts"); - - opts_file = fopen(file_name, "r"); - if (!opts_file) - return (errno == ENOENT) ? IB_SUCCESS : IB_ERROR; - - while (fgets(line, 1023, opts_file) != NULL) - { - /* get the first token */ - p_key = strtok_r(line, " \t\n", &p_last); - if (p_key) - { - p_val = strtok_r(NULL, " \t\n", &p_last); - - subn_parse_qos_options("qos", - p_key, p_val, &p_opts->qos_options); - - subn_parse_qos_options("qos_ca", - p_key, p_val, &p_opts->qos_ca_options); - - subn_parse_qos_options("qos_sw0", - p_key, p_val, &p_opts->qos_sw0_options); - - subn_parse_qos_options("qos_swe", - p_key, p_val, &p_opts->qos_swe_options); - - subn_parse_qos_options("qos_rtr", - p_key, p_val, &p_opts->qos_rtr_options); - - } - } - fclose(opts_file); - - return IB_SUCCESS; + char *p_cache_dir = getenv("OSM_CACHE_DIR"); + char file_name[OSM_PATH_MAX]; + FILE *opts_file; + char line[1024]; + char *p_key, *p_val, *p_last; + + /* try to open the options file from the cache dir */ + if (!p_cache_dir || !(*p_cache_dir)) + p_cache_dir = OSM_DEFAULT_CACHE_DIR; + + strcpy(file_name, p_cache_dir); + strcat(file_name, "/opensm.opts"); + + opts_file = fopen(file_name, "r"); + if (!opts_file) + return (errno == ENOENT) ? IB_SUCCESS : IB_ERROR; + + while (fgets(line, 1023, opts_file) != NULL) { + /* get the first token */ + p_key = strtok_r(line, " \t\n", &p_last); + if (p_key) { + p_val = strtok_r(NULL, " \t\n", &p_last); + + subn_parse_qos_options("qos", + p_key, p_val, + &p_opts->qos_options); + + subn_parse_qos_options("qos_ca", + p_key, p_val, + &p_opts->qos_ca_options); + + subn_parse_qos_options("qos_sw0", + p_key, p_val, + &p_opts->qos_sw0_options); + + subn_parse_qos_options("qos_swe", + p_key, p_val, + &p_opts->qos_swe_options); + + subn_parse_qos_options("qos_rtr", + p_key, p_val, + &p_opts->qos_rtr_options); + + } + } + fclose(opts_file); + + return IB_SUCCESS; } /********************************************************************** **********************************************************************/ -static void -osm_subn_verify_max_vls( - IN unsigned *max_vls, - IN char *key) +static void osm_subn_verify_max_vls(IN unsigned *max_vls, IN char *key) { - char buff[128]; - - if (*max_vls > 15) - { - sprintf(buff, " Invalid Cached Option:%s=%u:" - "Using Default:%u\n", - key, *max_vls, OSM_DEFAULT_QOS_MAX_VLS); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *max_vls = OSM_DEFAULT_QOS_MAX_VLS; - } + char buff[128]; + + if (*max_vls > 15) { + sprintf(buff, " Invalid Cached Option:%s=%u:" + "Using Default:%u\n", + key, *max_vls, OSM_DEFAULT_QOS_MAX_VLS); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *max_vls = OSM_DEFAULT_QOS_MAX_VLS; + } } -static void -osm_subn_verify_high_limit( - IN unsigned *high_limit, - IN char *key) +static void osm_subn_verify_high_limit(IN unsigned *high_limit, IN char *key) { - char buff[128]; - - if (*high_limit > 255) - { - sprintf(buff, " Invalid Cached Option:%s=%u:" - "Using Default:%u\n", - key, *high_limit, OSM_DEFAULT_QOS_HIGH_LIMIT); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - *high_limit = OSM_DEFAULT_QOS_HIGH_LIMIT; - } + char buff[128]; + + if (*high_limit > 255) { + sprintf(buff, " Invalid Cached Option:%s=%u:" + "Using Default:%u\n", + key, *high_limit, OSM_DEFAULT_QOS_HIGH_LIMIT); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + *high_limit = OSM_DEFAULT_QOS_HIGH_LIMIT; + } } -static void -osm_subn_verify_vlarb( - IN char *vlarb, - IN char *key) +static void osm_subn_verify_vlarb(IN char *vlarb, IN char *key) { - if (vlarb) - { - char buff[128]; - char *str, *tok, *end, *ptr; - int count = 0; - - str = (char *)malloc(strlen(vlarb)+1); - strcpy(str, vlarb); - - tok = strtok_r(str, ",\n", &ptr); - while (tok) - { - char *vl_str, *weight_str; - - vl_str = tok; - weight_str = strchr(tok, ':'); - - if (weight_str) - { - long vl, weight; - - *weight_str = '\0'; - weight_str++; - - vl = strtol(vl_str, &end, 0); - - if (*end) - { - sprintf(buff, " Warning: Cached Option %s:vl=%s improperly formatted\n", - key, vl_str); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - else if (vl < 0 || vl > 14) - { - sprintf(buff, " Warning: Cached Option %s:vl=%ld out of range\n", - key, vl); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - - weight = strtol(weight_str, &end, 0); - - if (*end) - { - sprintf(buff, " Warning: Cached Option %s:weight=%s improperly formatted\n", - key, weight_str); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - else if (weight < 0 || weight > 255) - { - sprintf(buff, " Warning: Cached Option %s:weight=%ld out of range\n", - key, weight); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - } - else - { - sprintf(buff, " Warning: Cached Option %s:vl:weight=%s improperly formatted\n", - key, tok); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - - count++; - tok = strtok_r(NULL, ",\n", &ptr); - } - - if (count > 64) - { - sprintf(buff, " Warning: Cached Option %s: > 64 listed: " - "excess vl:weight pairs will be dropped\n", - key); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - - free(str); - } + if (vlarb) { + char buff[128]; + char *str, *tok, *end, *ptr; + int count = 0; + + str = (char *)malloc(strlen(vlarb) + 1); + strcpy(str, vlarb); + + tok = strtok_r(str, ",\n", &ptr); + while (tok) { + char *vl_str, *weight_str; + + vl_str = tok; + weight_str = strchr(tok, ':'); + + if (weight_str) { + long vl, weight; + + *weight_str = '\0'; + weight_str++; + + vl = strtol(vl_str, &end, 0); + + if (*end) { + sprintf(buff, + " Warning: Cached Option %s:vl=%s improperly formatted\n", + key, vl_str); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, + buff, NULL, 0); + } else if (vl < 0 || vl > 14) { + sprintf(buff, + " Warning: Cached Option %s:vl=%ld out of range\n", + key, vl); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, + buff, NULL, 0); + } + + weight = strtol(weight_str, &end, 0); + + if (*end) { + sprintf(buff, + " Warning: Cached Option %s:weight=%s improperly formatted\n", + key, weight_str); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, + buff, NULL, 0); + } else if (weight < 0 || weight > 255) { + sprintf(buff, + " Warning: Cached Option %s:weight=%ld out of range\n", + key, weight); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, + buff, NULL, 0); + } + } else { + sprintf(buff, + " Warning: Cached Option %s:vl:weight=%s improperly formatted\n", + key, tok); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, + 0); + } + + count++; + tok = strtok_r(NULL, ",\n", &ptr); + } + + if (count > 64) { + sprintf(buff, + " Warning: Cached Option %s: > 64 listed: " + "excess vl:weight pairs will be dropped\n", + key); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + } + + free(str); + } } -static void -osm_subn_verify_sl2vl( - IN char *sl2vl, - IN char *key) +static void osm_subn_verify_sl2vl(IN char *sl2vl, IN char *key) { - if (sl2vl) - { - char buff[128]; - char *str, *tok, *end, *ptr; - int count = 0; - - str = (char *)malloc(strlen(sl2vl)+1); - strcpy(str, sl2vl); - - tok = strtok_r(str, ",\n", &ptr); - while (tok) - { - long vl = strtol(tok, &end, 0); - - if (*end) - { - sprintf(buff, " Warning: Cached Option %s:vl=%s improperly formatted\n", - key, tok); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - else if (vl < 0 || vl > 15) - { - sprintf(buff, " Warning: Cached Option %s:vl=%ld out of range\n", - key, vl); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - - count++; - tok = strtok_r(NULL, ",\n", &ptr); - } - - if (count < 16) - { - sprintf(buff, " Warning: Cached Option %s: < 16 VLs listed\n", - key); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - if (count > 16) - { - sprintf(buff, " Warning: Cached Option %s: > 16 listed: " - "excess VLs will be dropped\n", - key); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - } - - free(str); - } + if (sl2vl) { + char buff[128]; + char *str, *tok, *end, *ptr; + int count = 0; + + str = (char *)malloc(strlen(sl2vl) + 1); + strcpy(str, sl2vl); + + tok = strtok_r(str, ",\n", &ptr); + while (tok) { + long vl = strtol(tok, &end, 0); + + if (*end) { + sprintf(buff, + " Warning: Cached Option %s:vl=%s improperly formatted\n", + key, tok); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, + 0); + } else if (vl < 0 || vl > 15) { + sprintf(buff, + " Warning: Cached Option %s:vl=%ld out of range\n", + key, vl); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, + 0); + } + + count++; + tok = strtok_r(NULL, ",\n", &ptr); + } + + if (count < 16) { + sprintf(buff, + " Warning: Cached Option %s: < 16 VLs listed\n", + key); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + } + if (count > 16) { + sprintf(buff, + " Warning: Cached Option %s: > 16 listed: " + "excess VLs will be dropped\n", key); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + } + + free(str); + } } -static void -osm_subn_verify_conf_file( - IN osm_subn_opt_t* const p_opts ) +static void osm_subn_verify_conf_file(IN osm_subn_opt_t * const p_opts) { - char buff[128]; - - if (p_opts->lmc > 7) - { - sprintf(buff, " Invalid Cached Option Value:lmc = %u:" - "Using Default:%u\n", - p_opts->lmc, OSM_DEFAULT_LMC); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->lmc = OSM_DEFAULT_LMC; - } - - if (15 < p_opts->sm_priority) - { - sprintf(buff, " Invalid Cached Option Value:sm_priority = %u:" - "Using Default:%u\n", - p_opts->sm_priority, OSM_DEFAULT_SM_PRIORITY); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->sm_priority = OSM_DEFAULT_SM_PRIORITY; - } - - if ((15 < p_opts->force_link_speed) || - (p_opts->force_link_speed > 7 && p_opts->force_link_speed < 15)) - { - sprintf(buff, " Invalid Cached Option Value:force_link_speed = %u:" - "Using Default:%u\n", - p_opts->force_link_speed, IB_PORT_LINK_SPEED_ENABLED_MASK); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->force_link_speed = IB_PORT_LINK_SPEED_ENABLED_MASK; - } - - if (strcmp(p_opts->console, "off") - && strcmp(p_opts->console, "local") + char buff[128]; + + if (p_opts->lmc > 7) { + sprintf(buff, " Invalid Cached Option Value:lmc = %u:" + "Using Default:%u\n", p_opts->lmc, OSM_DEFAULT_LMC); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->lmc = OSM_DEFAULT_LMC; + } + + if (15 < p_opts->sm_priority) { + sprintf(buff, " Invalid Cached Option Value:sm_priority = %u:" + "Using Default:%u\n", + p_opts->sm_priority, OSM_DEFAULT_SM_PRIORITY); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->sm_priority = OSM_DEFAULT_SM_PRIORITY; + } + + if ((15 < p_opts->force_link_speed) || + (p_opts->force_link_speed > 7 && p_opts->force_link_speed < 15)) { + sprintf(buff, + " Invalid Cached Option Value:force_link_speed = %u:" + "Using Default:%u\n", p_opts->force_link_speed, + IB_PORT_LINK_SPEED_ENABLED_MASK); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->force_link_speed = IB_PORT_LINK_SPEED_ENABLED_MASK; + } + + if (strcmp(p_opts->console, "off") + && strcmp(p_opts->console, "local") #ifdef ENABLE_OSM_CONSOLE_SOCKET - && strcmp(p_opts->console, "socket") + && strcmp(p_opts->console, "socket") #endif - ) - { - sprintf(buff, " Invalid Cached Option Value:console = %s" - "Using Default:%s\n", - p_opts->console, OSM_DEFAULT_CONSOLE); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->console = OSM_DEFAULT_CONSOLE; - } - - if (p_opts->no_qos == FALSE) - { - osm_subn_verify_max_vls(&(p_opts->qos_options.max_vls), - "qos_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_ca_options.max_vls), - "qos_ca_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_sw0_options.max_vls), - "qos_sw0_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_swe_options.max_vls), - "qos_swe_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_rtr_options.max_vls), - "qos_rtr_max_vls"); - - osm_subn_verify_high_limit(&(p_opts->qos_options.high_limit), - "qos_high_limit"); - osm_subn_verify_high_limit(&(p_opts->qos_ca_options.high_limit), - "qos_ca_high_limit"); - osm_subn_verify_high_limit(&(p_opts->qos_sw0_options.high_limit), - "qos_sw0_high_limit"); - osm_subn_verify_high_limit(&(p_opts->qos_swe_options.high_limit), - "qos_swe_high_limit"); - osm_subn_verify_high_limit(&(p_opts->qos_rtr_options.high_limit), - "qos_rtr_high_limit"); - - osm_subn_verify_vlarb(p_opts->qos_options.vlarb_low, - "qos_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_low, - "qos_ca_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_low, - "qos_sw0_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_low, - "qos_swe_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_low, - "qos_rtr_vlarb_low"); - - osm_subn_verify_vlarb(p_opts->qos_options.vlarb_high, - "qos_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_high, - "qos_ca_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_high, - "qos_sw0_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_high, - "qos_swe_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_high, - "qos_rtr_vlarb_high"); - - osm_subn_verify_sl2vl(p_opts->qos_options.sl2vl, - "qos_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_ca_options.sl2vl, - "qos_ca_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_sw0_options.sl2vl, - "qos_sw0_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_swe_options.sl2vl, - "qos_swe_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_rtr_options.sl2vl, - "qos_rtr_sl2vl"); - } + ) { + sprintf(buff, " Invalid Cached Option Value:console = %s" + "Using Default:%s\n", + p_opts->console, OSM_DEFAULT_CONSOLE); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->console = OSM_DEFAULT_CONSOLE; + } + + if (p_opts->no_qos == FALSE) { + osm_subn_verify_max_vls(&(p_opts->qos_options.max_vls), + "qos_max_vls"); + osm_subn_verify_max_vls(&(p_opts->qos_ca_options.max_vls), + "qos_ca_max_vls"); + osm_subn_verify_max_vls(&(p_opts->qos_sw0_options.max_vls), + "qos_sw0_max_vls"); + osm_subn_verify_max_vls(&(p_opts->qos_swe_options.max_vls), + "qos_swe_max_vls"); + osm_subn_verify_max_vls(&(p_opts->qos_rtr_options.max_vls), + "qos_rtr_max_vls"); + + osm_subn_verify_high_limit(&(p_opts->qos_options.high_limit), + "qos_high_limit"); + osm_subn_verify_high_limit(&(p_opts->qos_ca_options.high_limit), + "qos_ca_high_limit"); + osm_subn_verify_high_limit(& + (p_opts->qos_sw0_options.high_limit), + "qos_sw0_high_limit"); + osm_subn_verify_high_limit(& + (p_opts->qos_swe_options.high_limit), + "qos_swe_high_limit"); + osm_subn_verify_high_limit(& + (p_opts->qos_rtr_options.high_limit), + "qos_rtr_high_limit"); + + osm_subn_verify_vlarb(p_opts->qos_options.vlarb_low, + "qos_vlarb_low"); + osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_low, + "qos_ca_vlarb_low"); + osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_low, + "qos_sw0_vlarb_low"); + osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_low, + "qos_swe_vlarb_low"); + osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_low, + "qos_rtr_vlarb_low"); + + osm_subn_verify_vlarb(p_opts->qos_options.vlarb_high, + "qos_vlarb_high"); + osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_high, + "qos_ca_vlarb_high"); + osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_high, + "qos_sw0_vlarb_high"); + osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_high, + "qos_swe_vlarb_high"); + osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_high, + "qos_rtr_vlarb_high"); + + osm_subn_verify_sl2vl(p_opts->qos_options.sl2vl, "qos_sl2vl"); + osm_subn_verify_sl2vl(p_opts->qos_ca_options.sl2vl, + "qos_ca_sl2vl"); + osm_subn_verify_sl2vl(p_opts->qos_sw0_options.sl2vl, + "qos_sw0_sl2vl"); + osm_subn_verify_sl2vl(p_opts->qos_swe_options.sl2vl, + "qos_swe_sl2vl"); + osm_subn_verify_sl2vl(p_opts->qos_rtr_options.sl2vl, + "qos_rtr_sl2vl"); + } #ifdef ENABLE_OSM_PERF_MGR - if (p_opts->perfmgr_sweep_time_s < 1) - { - sprintf(buff, " Invalid Cached Option Value:perfmgr_sweep_time_s = %u" - "Using Default:%u\n", - p_opts->perfmgr_sweep_time_s, OSM_PERFMGR_DEFAULT_SWEEP_TIME_S); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->perfmgr_sweep_time_s = OSM_PERFMGR_DEFAULT_SWEEP_TIME_S; - } - if (p_opts->perfmgr_max_outstanding_queries < 1) - { - sprintf(buff, " Invalid Cached Option Value:perfmgr_max_outstanding_queries = %u" - "Using Default:%u\n", - p_opts->perfmgr_max_outstanding_queries, - OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES); - printf(buff); - cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); - p_opts->perfmgr_max_outstanding_queries = - OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES; - } + if (p_opts->perfmgr_sweep_time_s < 1) { + sprintf(buff, + " Invalid Cached Option Value:perfmgr_sweep_time_s = %u" + "Using Default:%u\n", p_opts->perfmgr_sweep_time_s, + OSM_PERFMGR_DEFAULT_SWEEP_TIME_S); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->perfmgr_sweep_time_s = OSM_PERFMGR_DEFAULT_SWEEP_TIME_S; + } + if (p_opts->perfmgr_max_outstanding_queries < 1) { + sprintf(buff, + " Invalid Cached Option Value:perfmgr_max_outstanding_queries = %u" + "Using Default:%u\n", + p_opts->perfmgr_max_outstanding_queries, + OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES); + printf(buff); + cl_log_event("OpenSM", CL_LOG_INFO, buff, NULL, 0); + p_opts->perfmgr_max_outstanding_queries = + OSM_PERFMGR_DEFAULT_MAX_OUTSTANDING_QUERIES; + } #endif } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_subn_parse_conf_file( - IN osm_subn_opt_t* const p_opts ) +ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) { - char *p_cache_dir = getenv("OSM_CACHE_DIR"); - char file_name[OSM_PATH_MAX]; - FILE *opts_file; - char line[1024]; - char *p_key, *p_val ,*p_last; - - /* try to open the options file from the cache dir */ - if (!p_cache_dir || !(*p_cache_dir)) - p_cache_dir = OSM_DEFAULT_CACHE_DIR; - - strcpy(file_name, p_cache_dir); - strcat(file_name, "/opensm.opts"); - - opts_file = fopen(file_name, "r"); - if (!opts_file) - return (errno == ENOENT) ? IB_SUCCESS : IB_ERROR; - - while (fgets(line, 1023, opts_file) != NULL) - { - /* get the first token */ - p_key = strtok_r(line, " \t\n", &p_last); - if (p_key) - { - p_val = strtok_r(NULL, " \t\n", &p_last); - - __osm_subn_opts_unpack_net64( - "guid", p_key, p_val, &p_opts->guid); - - __osm_subn_opts_unpack_net64( - "m_key", p_key, p_val, &p_opts->m_key); - - __osm_subn_opts_unpack_net64( - "sm_key", p_key, p_val, &p_opts->sm_key); - - __osm_subn_opts_unpack_net64( - "subnet_prefix", - p_key, p_val, &p_opts->subnet_prefix); - - __osm_subn_opts_unpack_net16( - "m_key_lease_period", - p_key, p_val, &p_opts->m_key_lease_period); - - __osm_subn_opts_unpack_uint32( - "sweep_interval", - p_key, p_val, &p_opts->sweep_interval); - - __osm_subn_opts_unpack_uint32( - "max_wire_smps", - p_key, p_val, &p_opts->max_wire_smps); - - __osm_subn_opts_unpack_charp( - "console", - p_key, p_val, &p_opts->console); - - __osm_subn_opts_unpack_uint16( - "console_port", - p_key, p_val, &p_opts->console_port); - - __osm_subn_opts_unpack_uint32( - "transaction_timeout", - p_key, p_val, &p_opts->transaction_timeout); - - __osm_subn_opts_unpack_uint32( - "max_msg_fifo_timeout", - p_key, p_val, &p_opts->max_msg_fifo_timeout); - - __osm_subn_opts_unpack_uint8( - "sm_priority", - p_key, p_val, &p_opts->sm_priority); - - __osm_subn_opts_unpack_uint8( - "lmc", - p_key, p_val, &p_opts->lmc); - - __osm_subn_opts_unpack_boolean( - "lmc_esp0", - p_key, p_val, &p_opts->lmc_esp0); - - __osm_subn_opts_unpack_uint8( - "max_op_vls", - p_key, p_val, &p_opts->max_op_vls); - - __osm_subn_opts_unpack_uint8( - "force_link_speed", - p_key, p_val, &p_opts->force_link_speed); - - __osm_subn_opts_unpack_boolean( - "reassign_lids", - p_key, p_val, &p_opts->reassign_lids); - - __osm_subn_opts_unpack_boolean( - "ignore_other_sm", - p_key, p_val, &p_opts->ignore_other_sm); - - __osm_subn_opts_unpack_boolean( - "single_thread", - p_key, p_val, &p_opts->single_thread); - - __osm_subn_opts_unpack_boolean( - "no_multicast_option", - p_key, p_val, &p_opts->no_multicast_option); - - __osm_subn_opts_unpack_boolean( - "disable_multicast", - p_key, p_val, &p_opts->disable_multicast); - - __osm_subn_opts_unpack_boolean( - "force_log_flush", - p_key, p_val, &p_opts->force_log_flush); - - __osm_subn_opts_unpack_uint8( - "subnet_timeout", - p_key, p_val, &p_opts->subnet_timeout); - - __osm_subn_opts_unpack_uint8( - "packet_life_time", - p_key, p_val, &p_opts->packet_life_time); - - __osm_subn_opts_unpack_uint8( - "vl_stall_count", - p_key, p_val, &p_opts->vl_stall_count); - - __osm_subn_opts_unpack_uint8( - "leaf_vl_stall_count", - p_key, p_val, &p_opts->leaf_vl_stall_count); - - __osm_subn_opts_unpack_uint8( - "head_of_queue_lifetime", - p_key, p_val, &p_opts->head_of_queue_lifetime); - - __osm_subn_opts_unpack_uint8( - "leaf_head_of_queue_lifetime", - p_key, p_val, &p_opts->leaf_head_of_queue_lifetime); - - __osm_subn_opts_unpack_uint8( - "local_phy_errors_threshold", - p_key, p_val, &p_opts->local_phy_errors_threshold); - - __osm_subn_opts_unpack_uint8( - "overrun_errors_threshold", - p_key, p_val, &p_opts->overrun_errors_threshold); - - __osm_subn_opts_unpack_uint32( - "sminfo_polling_timeout", - p_key, p_val, &p_opts->sminfo_polling_timeout); - - __osm_subn_opts_unpack_uint32( - "polling_retry_number", - p_key, p_val, &p_opts->polling_retry_number); - - __osm_subn_opts_unpack_boolean( - "force_heavy_sweep", - p_key, p_val, &p_opts->force_heavy_sweep); - - __osm_subn_opts_unpack_uint8( - "log_flags", - p_key, p_val, &p_opts->log_flags); - - __osm_subn_opts_unpack_boolean( - "port_profile_switch_nodes", - p_key, p_val, &p_opts->port_profile_switch_nodes); - - __osm_subn_opts_unpack_boolean( - "sweep_on_trap", - p_key, p_val, &p_opts->sweep_on_trap); - - __osm_subn_opts_unpack_charp( - "routing_engine", - p_key, p_val, &p_opts->routing_engine_name); - - __osm_subn_opts_unpack_boolean( - "connect_roots", - p_key, p_val, &p_opts->connect_roots); - - __osm_subn_opts_unpack_charp( - "log_file", p_key, p_val, &p_opts->log_file); - - __osm_subn_opts_unpack_uint32( - "log_max_size", - p_key, p_val, (uint32_t *)&p_opts->log_max_size); - - __osm_subn_opts_unpack_charp( - "partition_config_file", - p_key, p_val, &p_opts->partition_config_file); - - __osm_subn_opts_unpack_boolean( - "no_partition_enforcement", - p_key, p_val, &p_opts->no_partition_enforcement); - - __osm_subn_opts_unpack_boolean( - "no_qos", - p_key, p_val, &p_opts->no_qos); - - __osm_subn_opts_unpack_boolean( - "accum_log_file", - p_key, p_val, &p_opts->accum_log_file); - - __osm_subn_opts_unpack_charp( - "dump_files_dir", - p_key, p_val, &p_opts->dump_files_dir); - - __osm_subn_opts_unpack_charp( - "lid_matrix_dump_file", - p_key, p_val, &p_opts->lid_matrix_dump_file); - - __osm_subn_opts_unpack_charp( - "ucast_dump_file", - p_key, p_val, &p_opts->ucast_dump_file); - - __osm_subn_opts_unpack_charp( - "root_guid_file", - p_key, p_val, &p_opts->root_guid_file); - - __osm_subn_opts_unpack_charp( - "cn_guid_file", - p_key, p_val, &p_opts->cn_guid_file); - - __osm_subn_opts_unpack_charp( - "sa_db_file", - p_key, p_val, &p_opts->sa_db_file); - - __osm_subn_opts_unpack_boolean( - "exit_on_fatal", - p_key, p_val, &p_opts->exit_on_fatal); - - __osm_subn_opts_unpack_boolean( - "honor_guid2lid_file", - p_key, p_val, &p_opts->honor_guid2lid_file); - - __osm_subn_opts_unpack_boolean( - "daemon", - p_key, p_val, &p_opts->daemon); - - __osm_subn_opts_unpack_boolean( - "sm_inactive", - p_key, p_val, &p_opts->sm_inactive); - - __osm_subn_opts_unpack_boolean( - "babbling_port_policy", - p_key, p_val, &p_opts->babbling_port_policy); + char *p_cache_dir = getenv("OSM_CACHE_DIR"); + char file_name[OSM_PATH_MAX]; + FILE *opts_file; + char line[1024]; + char *p_key, *p_val, *p_last; + + /* try to open the options file from the cache dir */ + if (!p_cache_dir || !(*p_cache_dir)) + p_cache_dir = OSM_DEFAULT_CACHE_DIR; + + strcpy(file_name, p_cache_dir); + strcat(file_name, "/opensm.opts"); + + opts_file = fopen(file_name, "r"); + if (!opts_file) + return (errno == ENOENT) ? IB_SUCCESS : IB_ERROR; + + while (fgets(line, 1023, opts_file) != NULL) { + /* get the first token */ + p_key = strtok_r(line, " \t\n", &p_last); + if (p_key) { + p_val = strtok_r(NULL, " \t\n", &p_last); + + __osm_subn_opts_unpack_net64("guid", p_key, p_val, + &p_opts->guid); + + __osm_subn_opts_unpack_net64("m_key", p_key, p_val, + &p_opts->m_key); + + __osm_subn_opts_unpack_net64("sm_key", p_key, p_val, + &p_opts->sm_key); + + __osm_subn_opts_unpack_net64("subnet_prefix", + p_key, p_val, + &p_opts->subnet_prefix); + + __osm_subn_opts_unpack_net16("m_key_lease_period", + p_key, p_val, + &p_opts-> + m_key_lease_period); + + __osm_subn_opts_unpack_uint32("sweep_interval", + p_key, p_val, + &p_opts->sweep_interval); + + __osm_subn_opts_unpack_uint32("max_wire_smps", + p_key, p_val, + &p_opts->max_wire_smps); + + __osm_subn_opts_unpack_charp("console", + p_key, p_val, + &p_opts->console); + + __osm_subn_opts_unpack_uint16("console_port", + p_key, p_val, + &p_opts->console_port); + + __osm_subn_opts_unpack_uint32("transaction_timeout", + p_key, p_val, + &p_opts-> + transaction_timeout); + + __osm_subn_opts_unpack_uint32("max_msg_fifo_timeout", + p_key, p_val, + &p_opts-> + max_msg_fifo_timeout); + + __osm_subn_opts_unpack_uint8("sm_priority", + p_key, p_val, + &p_opts->sm_priority); + + __osm_subn_opts_unpack_uint8("lmc", + p_key, p_val, + &p_opts->lmc); + + __osm_subn_opts_unpack_boolean("lmc_esp0", + p_key, p_val, + &p_opts->lmc_esp0); + + __osm_subn_opts_unpack_uint8("max_op_vls", + p_key, p_val, + &p_opts->max_op_vls); + + __osm_subn_opts_unpack_uint8("force_link_speed", + p_key, p_val, + &p_opts->force_link_speed); + + __osm_subn_opts_unpack_boolean("reassign_lids", + p_key, p_val, + &p_opts->reassign_lids); + + __osm_subn_opts_unpack_boolean("ignore_other_sm", + p_key, p_val, + &p_opts-> + ignore_other_sm); + + __osm_subn_opts_unpack_boolean("single_thread", + p_key, p_val, + &p_opts->single_thread); + + __osm_subn_opts_unpack_boolean("no_multicast_option", + p_key, p_val, + &p_opts-> + no_multicast_option); + + __osm_subn_opts_unpack_boolean("disable_multicast", + p_key, p_val, + &p_opts-> + disable_multicast); + + __osm_subn_opts_unpack_boolean("force_log_flush", + p_key, p_val, + &p_opts-> + force_log_flush); + + __osm_subn_opts_unpack_uint8("subnet_timeout", + p_key, p_val, + &p_opts->subnet_timeout); + + __osm_subn_opts_unpack_uint8("packet_life_time", + p_key, p_val, + &p_opts->packet_life_time); + + __osm_subn_opts_unpack_uint8("vl_stall_count", + p_key, p_val, + &p_opts->vl_stall_count); + + __osm_subn_opts_unpack_uint8("leaf_vl_stall_count", + p_key, p_val, + &p_opts-> + leaf_vl_stall_count); + + __osm_subn_opts_unpack_uint8("head_of_queue_lifetime", + p_key, p_val, + &p_opts-> + head_of_queue_lifetime); + + __osm_subn_opts_unpack_uint8 + ("leaf_head_of_queue_lifetime", p_key, p_val, + &p_opts->leaf_head_of_queue_lifetime); + + __osm_subn_opts_unpack_uint8 + ("local_phy_errors_threshold", p_key, p_val, + &p_opts->local_phy_errors_threshold); + + __osm_subn_opts_unpack_uint8("overrun_errors_threshold", + p_key, p_val, + &p_opts-> + overrun_errors_threshold); + + __osm_subn_opts_unpack_uint32("sminfo_polling_timeout", + p_key, p_val, + &p_opts-> + sminfo_polling_timeout); + + __osm_subn_opts_unpack_uint32("polling_retry_number", + p_key, p_val, + &p_opts-> + polling_retry_number); + + __osm_subn_opts_unpack_boolean("force_heavy_sweep", + p_key, p_val, + &p_opts-> + force_heavy_sweep); + + __osm_subn_opts_unpack_uint8("log_flags", + p_key, p_val, + &p_opts->log_flags); + + __osm_subn_opts_unpack_boolean + ("port_profile_switch_nodes", p_key, p_val, + &p_opts->port_profile_switch_nodes); + + __osm_subn_opts_unpack_boolean("sweep_on_trap", + p_key, p_val, + &p_opts->sweep_on_trap); + + __osm_subn_opts_unpack_charp("routing_engine", + p_key, p_val, + &p_opts-> + routing_engine_name); + + __osm_subn_opts_unpack_boolean("connect_roots", + p_key, p_val, + &p_opts->connect_roots); + + __osm_subn_opts_unpack_charp("log_file", p_key, p_val, + &p_opts->log_file); + + __osm_subn_opts_unpack_uint32("log_max_size", + p_key, p_val, + (uint32_t *) & p_opts-> + log_max_size); + + __osm_subn_opts_unpack_charp("partition_config_file", + p_key, p_val, + &p_opts-> + partition_config_file); + + __osm_subn_opts_unpack_boolean + ("no_partition_enforcement", p_key, p_val, + &p_opts->no_partition_enforcement); + + __osm_subn_opts_unpack_boolean("no_qos", + p_key, p_val, + &p_opts->no_qos); + + __osm_subn_opts_unpack_boolean("accum_log_file", + p_key, p_val, + &p_opts->accum_log_file); + + __osm_subn_opts_unpack_charp("dump_files_dir", + p_key, p_val, + &p_opts->dump_files_dir); + + __osm_subn_opts_unpack_charp("lid_matrix_dump_file", + p_key, p_val, + &p_opts-> + lid_matrix_dump_file); + + __osm_subn_opts_unpack_charp("ucast_dump_file", + p_key, p_val, + &p_opts->ucast_dump_file); + + __osm_subn_opts_unpack_charp("root_guid_file", + p_key, p_val, + &p_opts->root_guid_file); + + __osm_subn_opts_unpack_charp("cn_guid_file", + p_key, p_val, + &p_opts->cn_guid_file); + + __osm_subn_opts_unpack_charp("sa_db_file", + p_key, p_val, + &p_opts->sa_db_file); + + __osm_subn_opts_unpack_boolean("exit_on_fatal", + p_key, p_val, + &p_opts->exit_on_fatal); + + __osm_subn_opts_unpack_boolean("honor_guid2lid_file", + p_key, p_val, + &p_opts-> + honor_guid2lid_file); + + __osm_subn_opts_unpack_boolean("daemon", + p_key, p_val, + &p_opts->daemon); + + __osm_subn_opts_unpack_boolean("sm_inactive", + p_key, p_val, + &p_opts->sm_inactive); + + __osm_subn_opts_unpack_boolean("babbling_port_policy", + p_key, p_val, + &p_opts-> + babbling_port_policy); #ifdef ENABLE_OSM_PERF_MGR - __osm_subn_opts_unpack_boolean( - "perfmgr", - p_key, p_val, &p_opts->perfmgr); - - __osm_subn_opts_unpack_boolean( - "perfmgr_redir", - p_key, p_val, &p_opts->perfmgr_redir); - - __osm_subn_opts_unpack_uint16( - "perfmgr_sweep_time_s", - p_key, p_val, &p_opts->perfmgr_sweep_time_s); - - __osm_subn_opts_unpack_uint32( - "perfmgr_max_outstanding_queries", - p_key, p_val, &p_opts->perfmgr_max_outstanding_queries); - - __osm_subn_opts_unpack_charp( - "event_db_dump_file", - p_key, p_val, &p_opts->event_db_dump_file); -#endif /* ENABLE_OSM_PERF_MGR */ - - __osm_subn_opts_unpack_charp( - "event_plugin_name", - p_key, p_val, &p_opts->event_plugin_name); - - subn_parse_qos_options("qos", - p_key, p_val, &p_opts->qos_options); - - subn_parse_qos_options("qos_ca", - p_key, p_val, &p_opts->qos_ca_options); - - subn_parse_qos_options("qos_sw0", - p_key, p_val, &p_opts->qos_sw0_options); - - subn_parse_qos_options("qos_swe", - p_key, p_val, &p_opts->qos_swe_options); - - subn_parse_qos_options("qos_rtr", - p_key, p_val, &p_opts->qos_rtr_options); - - __osm_subn_opts_unpack_boolean( - "enable_quirks", - p_key, p_val, &p_opts->enable_quirks); - - __osm_subn_opts_unpack_boolean( - "no_clients_rereg", - p_key, p_val, &p_opts->no_clients_rereg); - - } - } - fclose(opts_file); - - osm_subn_verify_conf_file(p_opts); - - return IB_SUCCESS; + __osm_subn_opts_unpack_boolean("perfmgr", + p_key, p_val, + &p_opts->perfmgr); + + __osm_subn_opts_unpack_boolean("perfmgr_redir", + p_key, p_val, + &p_opts->perfmgr_redir); + + __osm_subn_opts_unpack_uint16("perfmgr_sweep_time_s", + p_key, p_val, + &p_opts-> + perfmgr_sweep_time_s); + + __osm_subn_opts_unpack_uint32 + ("perfmgr_max_outstanding_queries", p_key, p_val, + &p_opts->perfmgr_max_outstanding_queries); + + __osm_subn_opts_unpack_charp("event_db_dump_file", + p_key, p_val, + &p_opts-> + event_db_dump_file); +#endif /* ENABLE_OSM_PERF_MGR */ + + __osm_subn_opts_unpack_charp("event_plugin_name", + p_key, p_val, + &p_opts-> + event_plugin_name); + + subn_parse_qos_options("qos", + p_key, p_val, + &p_opts->qos_options); + + subn_parse_qos_options("qos_ca", + p_key, p_val, + &p_opts->qos_ca_options); + + subn_parse_qos_options("qos_sw0", + p_key, p_val, + &p_opts->qos_sw0_options); + + subn_parse_qos_options("qos_swe", + p_key, p_val, + &p_opts->qos_swe_options); + + subn_parse_qos_options("qos_rtr", + p_key, p_val, + &p_opts->qos_rtr_options); + + __osm_subn_opts_unpack_boolean("enable_quirks", + p_key, p_val, + &p_opts->enable_quirks); + + __osm_subn_opts_unpack_boolean("no_clients_rereg", + p_key, p_val, + &p_opts-> + no_clients_rereg); + + } + } + fclose(opts_file); + + osm_subn_verify_conf_file(p_opts); + + return IB_SUCCESS; } /********************************************************************** **********************************************************************/ -ib_api_status_t -osm_subn_write_conf_file( - IN osm_subn_opt_t* const p_opts ) +ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) { - char *p_cache_dir = getenv("OSM_CACHE_DIR"); - char file_name[OSM_PATH_MAX]; - FILE *opts_file; - - /* try to open the options file from the cache dir */ - if (!p_cache_dir || !(*p_cache_dir)) - p_cache_dir = OSM_DEFAULT_CACHE_DIR; - - strcpy(file_name, p_cache_dir); - strcat(file_name, "/opensm.opts"); - - opts_file = fopen(file_name, "w"); - if (!opts_file) - return IB_ERROR; - - fprintf( - opts_file, - "#\n# DEVICE ATTRIBUTES OPTIONS\n#\n" - "# The port GUID on which the OpenSM is running\n" - "guid 0x%016" PRIx64 "\n\n" - "# M_Key value sent to all ports qualifying all Set(PortInfo)\n" - "m_key 0x%016" PRIx64 "\n\n" - "# The lease period used for the M_Key on this subnet in [msec]\n" - "m_key_lease_period %u\n\n" - "# SM_Key value of the SM to qualify rcv SA queries as 'trusted'\n" - "sm_key 0x%016" PRIx64 "\n\n" - "# Subnet prefix used on this subnet\n" - "subnet_prefix 0x%016" PRIx64 "\n\n" - "# The LMC value used on this subnet\n" - "lmc %u\n\n" - "# lmc_esp0 determines whether LMC value used on subnet is used for\n" - "#enhanced switch port 0. If TRUE, LMC value for subnet is used for\n" - "#ESP0. Otherwise, LMC value for ESP0s is 0.\n" - "lmc_esp0 %s\n\n" - "# The code of maximal time a packet can live in a switch\n" - "# The actual time is 4.096usec * 2^\n" - "# The value 0x14 disables this mechanism\n" - "packet_life_time 0x%02x\n\n" - "# The number of sequential packets dropped that cause the port\n" - "# to enter the VLStalled state. The result of setting this value to\n" - "# zero is undefined.\n" - "vl_stall_count 0x%02x\n\n" - "# The number of sequential packets dropped that cause the port\n" - "# to enter the VLStalled state. This value is for switch ports\n" - "# driving a CA or router port. The result of setting this value\n" - "# to zero is undefined.\n" - "leaf_vl_stall_count 0x%02x\n\n" - "# The code of maximal time a packet can wait at the head of\n" - "# transmission queue.\n" - "# The actual time is 4.096usec * 2^\n" - "# The value 0x14 disables this mechanism\n" - "head_of_queue_lifetime 0x%02x\n\n" - "# The maximal time a packet can wait at the head of queue on\n" - "# switch port connected to a CA or router port\n" - "leaf_head_of_queue_lifetime 0x%02x\n\n" - "# Limit the maximal operational VLs\n" - "max_op_vls %u\n\n" - "# Force link speed enable on switch links\n" - "# If 0, don't modify PortInfo:LinkSpeedEnabled on switch port\n" - "# Otherwise, use value for PortInfo:LinkSpeedEnabled on switch port\n" - "# Default is 15 (to set to PortInfo:LinkSpeedSupported\n\n" - "force_link_speed %u\n\n" - "# The subnet_timeout code that will be set for all the ports\n" - "# The actual timeout is 4.096usec * 2^\n" - "subnet_timeout %u\n\n" - "# Threshold of local phy errors for sending Trap 129\n" - "local_phy_errors_threshold 0x%02x\n\n" - "# Threshold of credits overrun errors for sending Trap 130\n" - "overrun_errors_threshold 0x%02x\n\n", - cl_ntoh64(p_opts->guid), - cl_ntoh64(p_opts->m_key), - cl_ntoh16(p_opts->m_key_lease_period), - cl_ntoh64(p_opts->sm_key), - cl_ntoh64(p_opts->subnet_prefix), - p_opts->lmc, - p_opts->lmc_esp0 ? "TRUE" : "FALSE", - p_opts->packet_life_time, - p_opts->vl_stall_count, - p_opts->leaf_vl_stall_count, - p_opts->head_of_queue_lifetime, - p_opts->leaf_head_of_queue_lifetime, - p_opts->max_op_vls, - p_opts->force_link_speed, - p_opts->subnet_timeout, - p_opts->local_phy_errors_threshold, - p_opts->overrun_errors_threshold - ); - - fprintf( - opts_file, - "#\n# PARTITIONING OPTIONS\n#\n" - "# Partition configuration file to be used\n" - "partition_config_file %s\n\n" - "# Disable partition enforcement by switches\n" - "no_partition_enforcement %s\n\n", - p_opts->partition_config_file, - p_opts->no_partition_enforcement ? "TRUE" : "FALSE"); - - fprintf( - opts_file, - "#\n# SWEEP OPTIONS\n#\n" - "# The number of seconds between subnet sweeps (0 disables it)\n" - "sweep_interval %u\n\n" - "# If TRUE cause all lids to be reassigned\n" - "reassign_lids %s\n\n" - "# If TRUE forces every sweep to be a heavy sweep\n" - "force_heavy_sweep %s\n\n" - "# If TRUE every trap will cause a heavy sweep.\n" - "# NOTE: successive identical traps (>10) are suppressed\n" - "sweep_on_trap %s\n\n", - p_opts->sweep_interval, - p_opts->reassign_lids ? "TRUE" : "FALSE", - p_opts->force_heavy_sweep ? "TRUE" : "FALSE", - p_opts->sweep_on_trap ? "TRUE" : "FALSE" - ); - - fprintf( - opts_file, - "#\n# ROUTING OPTIONS\n#\n" - "# If TRUE count switches as link subscriptions\n" - "port_profile_switch_nodes %s\n\n", - p_opts->port_profile_switch_nodes ? "TRUE" : "FALSE"); - - if (p_opts->routing_engine_name) - fprintf( opts_file, - "# Routing engine\n" - "routing_engine %s\n\n", - p_opts->routing_engine_name); - if (p_opts->connect_roots) - fprintf( opts_file, - "# Connect roots (use FALSE if unsure)\n" - "connect_roots %s\n\n", - p_opts->connect_roots ? "TRUE" : "FALSE"); - if (p_opts->lid_matrix_dump_file) - fprintf( opts_file, - "# Lid matrix dump file name\n" - "lid_matrix_dump_file %s\n\n", - p_opts->lid_matrix_dump_file); - if (p_opts->ucast_dump_file) - fprintf( opts_file, - "# Ucast dump file name\n" - "ucast_dump_file %s\n\n", - p_opts->ucast_dump_file); - if (p_opts->root_guid_file) - fprintf( opts_file, - "# The file holding the root node guids (for fat-tree or Up/Down)\n" - "# One guid in each line\n" - "root_guid_file %s\n\n", - p_opts->root_guid_file); - if (p_opts->cn_guid_file) - fprintf( opts_file, - "# The file holding the fat-tree compute node guids\n" - "# One guid in each line\n" - "cn_guid_file %s\n\n", - p_opts->cn_guid_file); - if (p_opts->sa_db_file) - fprintf( opts_file, - "# SA database file name\n" - "sa_db_file %s\n\n", - p_opts->sa_db_file); - - fprintf( - opts_file, - "#\n# HANDOVER - MULTIPLE SMs OPTIONS\n#\n" - "# SM priority used for deciding who is the master\n" - "# Range goes from 0 (lowest priority) to 15 (highest).\n" - "sm_priority %u\n\n" - "# If TRUE other SMs on the subnet should be ignored\n" - "ignore_other_sm %s\n\n" - "# Timeout in [msec] between two polls of active master SM\n" - "sminfo_polling_timeout %u\n\n" - "# Number of failing polls of remote SM that declares it dead\n" - "polling_retry_number %u\n\n" - "# If TRUE honor the guid2lid file when coming out of standby\n" - "# state, if such file exists and is valid\n" - "honor_guid2lid_file %s\n\n", - p_opts->sm_priority, - p_opts->ignore_other_sm ? "TRUE" : "FALSE", - p_opts->sminfo_polling_timeout, - p_opts->polling_retry_number, - p_opts->honor_guid2lid_file ? "TRUE" : "FALSE" - ); - - fprintf( - opts_file, - "#\n# TIMING AND THREADING OPTIONS\n#\n" - "# Number of MADs sent in parallel\n" - "max_wire_smps %u\n\n" - "# The time taken to a transaction to finish in [msec]\n" - "transaction_timeout %u\n\n" - "# Maximal time in [msec] a message can stay in the incoming message queue.\n" - "# If there is more than one message in the queue and the last message\n" - "# stayed in the queue more than this value, any SA request will be\n" - "# immediately returned with a BUSY status.\n" - "max_msg_fifo_timeout %u\n\n" - "# Use a single thread for handling SA queries\n" - "single_thread %s\n\n", - p_opts->max_wire_smps, - p_opts->transaction_timeout, - p_opts->max_msg_fifo_timeout, - p_opts->single_thread ? "TRUE" : "FALSE" - ); - - fprintf( - opts_file, - "#\n# MISC OPTIONS\n#\n" - "# Daemon mode\n" - "daemon %s\n\n" - "# SM Inactive\n" - "sm_inactive %s\n\n" - "# Babbling Port Policy\n" - "babbling_port_policy %s\n\n", - p_opts->daemon ? "TRUE" : "FALSE", - p_opts->sm_inactive ? "TRUE" : "FALSE", - p_opts->babbling_port_policy ? "TRUE" : "FALSE" - ); + char *p_cache_dir = getenv("OSM_CACHE_DIR"); + char file_name[OSM_PATH_MAX]; + FILE *opts_file; + + /* try to open the options file from the cache dir */ + if (!p_cache_dir || !(*p_cache_dir)) + p_cache_dir = OSM_DEFAULT_CACHE_DIR; + + strcpy(file_name, p_cache_dir); + strcat(file_name, "/opensm.opts"); + + opts_file = fopen(file_name, "w"); + if (!opts_file) + return IB_ERROR; + + fprintf(opts_file, + "#\n# DEVICE ATTRIBUTES OPTIONS\n#\n" + "# The port GUID on which the OpenSM is running\n" + "guid 0x%016" PRIx64 "\n\n" + "# M_Key value sent to all ports qualifying all Set(PortInfo)\n" + "m_key 0x%016" PRIx64 "\n\n" + "# The lease period used for the M_Key on this subnet in [msec]\n" + "m_key_lease_period %u\n\n" + "# SM_Key value of the SM to qualify rcv SA queries as 'trusted'\n" + "sm_key 0x%016" PRIx64 "\n\n" + "# Subnet prefix used on this subnet\n" + "subnet_prefix 0x%016" PRIx64 "\n\n" + "# The LMC value used on this subnet\n" + "lmc %u\n\n" + "# lmc_esp0 determines whether LMC value used on subnet is used for\n" + "#enhanced switch port 0. If TRUE, LMC value for subnet is used for\n" + "#ESP0. Otherwise, LMC value for ESP0s is 0.\n" + "lmc_esp0 %s\n\n" + "# The code of maximal time a packet can live in a switch\n" + "# The actual time is 4.096usec * 2^\n" + "# The value 0x14 disables this mechanism\n" + "packet_life_time 0x%02x\n\n" + "# The number of sequential packets dropped that cause the port\n" + "# to enter the VLStalled state. The result of setting this value to\n" + "# zero is undefined.\n" + "vl_stall_count 0x%02x\n\n" + "# The number of sequential packets dropped that cause the port\n" + "# to enter the VLStalled state. This value is for switch ports\n" + "# driving a CA or router port. The result of setting this value\n" + "# to zero is undefined.\n" + "leaf_vl_stall_count 0x%02x\n\n" + "# The code of maximal time a packet can wait at the head of\n" + "# transmission queue.\n" + "# The actual time is 4.096usec * 2^\n" + "# The value 0x14 disables this mechanism\n" + "head_of_queue_lifetime 0x%02x\n\n" + "# The maximal time a packet can wait at the head of queue on\n" + "# switch port connected to a CA or router port\n" + "leaf_head_of_queue_lifetime 0x%02x\n\n" + "# Limit the maximal operational VLs\n" + "max_op_vls %u\n\n" + "# Force link speed enable on switch links\n" + "# If 0, don't modify PortInfo:LinkSpeedEnabled on switch port\n" + "# Otherwise, use value for PortInfo:LinkSpeedEnabled on switch port\n" + "# Default is 15 (to set to PortInfo:LinkSpeedSupported\n\n" + "force_link_speed %u\n\n" + "# The subnet_timeout code that will be set for all the ports\n" + "# The actual timeout is 4.096usec * 2^\n" + "subnet_timeout %u\n\n" + "# Threshold of local phy errors for sending Trap 129\n" + "local_phy_errors_threshold 0x%02x\n\n" + "# Threshold of credits overrun errors for sending Trap 130\n" + "overrun_errors_threshold 0x%02x\n\n", + cl_ntoh64(p_opts->guid), + cl_ntoh64(p_opts->m_key), + cl_ntoh16(p_opts->m_key_lease_period), + cl_ntoh64(p_opts->sm_key), + cl_ntoh64(p_opts->subnet_prefix), + p_opts->lmc, + p_opts->lmc_esp0 ? "TRUE" : "FALSE", + p_opts->packet_life_time, + p_opts->vl_stall_count, + p_opts->leaf_vl_stall_count, + p_opts->head_of_queue_lifetime, + p_opts->leaf_head_of_queue_lifetime, + p_opts->max_op_vls, + p_opts->force_link_speed, + p_opts->subnet_timeout, + p_opts->local_phy_errors_threshold, + p_opts->overrun_errors_threshold); + + fprintf(opts_file, + "#\n# PARTITIONING OPTIONS\n#\n" + "# Partition configuration file to be used\n" + "partition_config_file %s\n\n" + "# Disable partition enforcement by switches\n" + "no_partition_enforcement %s\n\n", + p_opts->partition_config_file, + p_opts->no_partition_enforcement ? "TRUE" : "FALSE"); + + fprintf(opts_file, + "#\n# SWEEP OPTIONS\n#\n" + "# The number of seconds between subnet sweeps (0 disables it)\n" + "sweep_interval %u\n\n" + "# If TRUE cause all lids to be reassigned\n" + "reassign_lids %s\n\n" + "# If TRUE forces every sweep to be a heavy sweep\n" + "force_heavy_sweep %s\n\n" + "# If TRUE every trap will cause a heavy sweep.\n" + "# NOTE: successive identical traps (>10) are suppressed\n" + "sweep_on_trap %s\n\n", + p_opts->sweep_interval, + p_opts->reassign_lids ? "TRUE" : "FALSE", + p_opts->force_heavy_sweep ? "TRUE" : "FALSE", + p_opts->sweep_on_trap ? "TRUE" : "FALSE"); + + fprintf(opts_file, + "#\n# ROUTING OPTIONS\n#\n" + "# If TRUE count switches as link subscriptions\n" + "port_profile_switch_nodes %s\n\n", + p_opts->port_profile_switch_nodes ? "TRUE" : "FALSE"); + + if (p_opts->routing_engine_name) + fprintf(opts_file, + "# Routing engine\n" + "routing_engine %s\n\n", p_opts->routing_engine_name); + if (p_opts->connect_roots) + fprintf(opts_file, + "# Connect roots (use FALSE if unsure)\n" + "connect_roots %s\n\n", + p_opts->connect_roots ? "TRUE" : "FALSE"); + if (p_opts->lid_matrix_dump_file) + fprintf(opts_file, + "# Lid matrix dump file name\n" + "lid_matrix_dump_file %s\n\n", + p_opts->lid_matrix_dump_file); + if (p_opts->ucast_dump_file) + fprintf(opts_file, + "# Ucast dump file name\n" + "ucast_dump_file %s\n\n", p_opts->ucast_dump_file); + if (p_opts->root_guid_file) + fprintf(opts_file, + "# The file holding the root node guids (for fat-tree or Up/Down)\n" + "# One guid in each line\n" + "root_guid_file %s\n\n", p_opts->root_guid_file); + if (p_opts->cn_guid_file) + fprintf(opts_file, + "# The file holding the fat-tree compute node guids\n" + "# One guid in each line\n" + "cn_guid_file %s\n\n", p_opts->cn_guid_file); + if (p_opts->sa_db_file) + fprintf(opts_file, + "# SA database file name\n" + "sa_db_file %s\n\n", p_opts->sa_db_file); + + fprintf(opts_file, + "#\n# HANDOVER - MULTIPLE SMs OPTIONS\n#\n" + "# SM priority used for deciding who is the master\n" + "# Range goes from 0 (lowest priority) to 15 (highest).\n" + "sm_priority %u\n\n" + "# If TRUE other SMs on the subnet should be ignored\n" + "ignore_other_sm %s\n\n" + "# Timeout in [msec] between two polls of active master SM\n" + "sminfo_polling_timeout %u\n\n" + "# Number of failing polls of remote SM that declares it dead\n" + "polling_retry_number %u\n\n" + "# If TRUE honor the guid2lid file when coming out of standby\n" + "# state, if such file exists and is valid\n" + "honor_guid2lid_file %s\n\n", + p_opts->sm_priority, + p_opts->ignore_other_sm ? "TRUE" : "FALSE", + p_opts->sminfo_polling_timeout, + p_opts->polling_retry_number, + p_opts->honor_guid2lid_file ? "TRUE" : "FALSE"); + + fprintf(opts_file, + "#\n# TIMING AND THREADING OPTIONS\n#\n" + "# Number of MADs sent in parallel\n" + "max_wire_smps %u\n\n" + "# The time taken to a transaction to finish in [msec]\n" + "transaction_timeout %u\n\n" + "# Maximal time in [msec] a message can stay in the incoming message queue.\n" + "# If there is more than one message in the queue and the last message\n" + "# stayed in the queue more than this value, any SA request will be\n" + "# immediately returned with a BUSY status.\n" + "max_msg_fifo_timeout %u\n\n" + "# Use a single thread for handling SA queries\n" + "single_thread %s\n\n", + p_opts->max_wire_smps, + p_opts->transaction_timeout, + p_opts->max_msg_fifo_timeout, + p_opts->single_thread ? "TRUE" : "FALSE"); + + fprintf(opts_file, + "#\n# MISC OPTIONS\n#\n" + "# Daemon mode\n" + "daemon %s\n\n" + "# SM Inactive\n" + "sm_inactive %s\n\n" + "# Babbling Port Policy\n" + "babbling_port_policy %s\n\n", + p_opts->daemon ? "TRUE" : "FALSE", + p_opts->sm_inactive ? "TRUE" : "FALSE", + p_opts->babbling_port_policy ? "TRUE" : "FALSE"); #ifdef ENABLE_OSM_PERF_MGR - fprintf( - opts_file, - "#\n# Performance Manager Options\n#\n" - "# perfmgr enable\n" - "perfmgr %s\n\n" - "# perfmgr_redir enable\n" - "perfmgr_redir %s\n\n" - "# sweep time in seconds\n" - "perfmgr_sweep_time_s %u\n\n" - "# Max outstanding queries\n" - "perfmgr_max_outstanding_queries %u\n\n" - , - p_opts->perfmgr ? "TRUE" : "FALSE", - p_opts->perfmgr_redir ? "TRUE" : "FALSE", - p_opts->perfmgr_sweep_time_s, - p_opts->perfmgr_max_outstanding_queries - ); - - fprintf( - opts_file, - "#\n# Event DB Options\n#\n" - "# Dump file to dump the events to\n" - "event_db_dump_file %s\n\n" - , - p_opts->event_db_dump_file - ); -#endif /* ENABLE_OSM_PERF_MGR */ - - fprintf( - opts_file, - "#\n# Event Plugin Options\n#\n" - "event_plugin_name %s\n\n" - , - p_opts->event_plugin_name - ); - - fprintf( - opts_file, - "#\n# DEBUG FEATURES\n#\n" - "# The log flags used\n" - "log_flags 0x%02x\n\n" - "# Force flush of the log file after each log message\n" - "force_log_flush %s\n\n" - "# Log file to be used\n" - "log_file %s\n\n" - "# Limit the size of the log file. If overrun, log is restarted\n" - "log_max_size %lu\n\n" - "# If TRUE will accumulate the log over multiple OpenSM sessions\n" - "accum_log_file %s\n\n" - "# The directory to hold the file OpenSM dumps\n" - "dump_files_dir %s\n\n" - "# If TRUE enables new high risk options and hardware specific quirks\n" - "enable_quirks %s\n\n" - "# If TRUE disables client reregistration\n" - "no_clients_rereg %s\n\n" - "# If TRUE OpenSM should disable multicast support\n" - "no_multicast_option %s\n\n" - "# No multicast routing is performed if TRUE\n" - "disable_multicast %s\n\n" - "# If TRUE opensm will exit on fatal initialization issues\n" - "exit_on_fatal %s\n\n" - "# console [off|local" + fprintf(opts_file, + "#\n# Performance Manager Options\n#\n" + "# perfmgr enable\n" + "perfmgr %s\n\n" + "# perfmgr_redir enable\n" + "perfmgr_redir %s\n\n" + "# sweep time in seconds\n" + "perfmgr_sweep_time_s %u\n\n" + "# Max outstanding queries\n" + "perfmgr_max_outstanding_queries %u\n\n", + p_opts->perfmgr ? "TRUE" : "FALSE", + p_opts->perfmgr_redir ? "TRUE" : "FALSE", + p_opts->perfmgr_sweep_time_s, + p_opts->perfmgr_max_outstanding_queries); + + fprintf(opts_file, + "#\n# Event DB Options\n#\n" + "# Dump file to dump the events to\n" + "event_db_dump_file %s\n\n", p_opts->event_db_dump_file); +#endif /* ENABLE_OSM_PERF_MGR */ + + fprintf(opts_file, + "#\n# Event Plugin Options\n#\n" + "event_plugin_name %s\n\n", p_opts->event_plugin_name); + + fprintf(opts_file, + "#\n# DEBUG FEATURES\n#\n" + "# The log flags used\n" + "log_flags 0x%02x\n\n" + "# Force flush of the log file after each log message\n" + "force_log_flush %s\n\n" + "# Log file to be used\n" + "log_file %s\n\n" + "# Limit the size of the log file. If overrun, log is restarted\n" + "log_max_size %lu\n\n" + "# If TRUE will accumulate the log over multiple OpenSM sessions\n" + "accum_log_file %s\n\n" + "# The directory to hold the file OpenSM dumps\n" + "dump_files_dir %s\n\n" + "# If TRUE enables new high risk options and hardware specific quirks\n" + "enable_quirks %s\n\n" + "# If TRUE disables client reregistration\n" + "no_clients_rereg %s\n\n" + "# If TRUE OpenSM should disable multicast support\n" + "no_multicast_option %s\n\n" + "# No multicast routing is performed if TRUE\n" + "disable_multicast %s\n\n" + "# If TRUE opensm will exit on fatal initialization issues\n" + "exit_on_fatal %s\n\n" "# console [off|local" #ifdef ENABLE_OSM_CONSOLE_SOCKET - "|socket]\n" + "|socket]\n" #else - "]\n" + "]\n" #endif - "console %s\n\n" - "# Telnet port for console (default %d)\n" - "console_port %d\n\n", - p_opts->log_flags, - p_opts->force_log_flush ? "TRUE" : "FALSE", - p_opts->log_file, - p_opts->log_max_size, - p_opts->accum_log_file ? "TRUE" : "FALSE", - p_opts->dump_files_dir, - p_opts->enable_quirks ? "TRUE" : "FALSE", - p_opts->no_clients_rereg ? "TRUE" : "FALSE", - p_opts->no_multicast_option ? "TRUE" : "FALSE", - p_opts->disable_multicast ? "TRUE" : "FALSE", - p_opts->exit_on_fatal ? "TRUE" : "FALSE", - p_opts->console, - OSM_DEFAULT_CONSOLE_PORT, p_opts->console_port - ); - - fprintf( - opts_file, - "#\n# QoS OPTIONS\n#\n" - "# Disable QoS setup\n" - "no_qos %s\n\n", - p_opts->no_qos ? "TRUE" : "FALSE"); - - subn_dump_qos_options(opts_file, - "QoS default options", "qos", &p_opts->qos_options); - fprintf(opts_file, "\n"); - subn_dump_qos_options(opts_file, - "QoS CA options", "qos_ca", &p_opts->qos_ca_options); - fprintf(opts_file, "\n"); - subn_dump_qos_options(opts_file, - "QoS Switch Port 0 options", "qos_sw0", &p_opts->qos_sw0_options); - fprintf(opts_file, "\n"); - subn_dump_qos_options(opts_file, - "QoS Switch external ports options", "qos_swe", &p_opts->qos_swe_options); - fprintf(opts_file, "\n"); - subn_dump_qos_options(opts_file, - "QoS Router ports options", "qos_rtr", &p_opts->qos_rtr_options); - - /* optional string attributes ... */ - - fclose(opts_file); - - return IB_SUCCESS; + "console %s\n\n" + "# Telnet port for console (default %d)\n" + "console_port %d\n\n", + p_opts->log_flags, + p_opts->force_log_flush ? "TRUE" : "FALSE", + p_opts->log_file, + p_opts->log_max_size, + p_opts->accum_log_file ? "TRUE" : "FALSE", + p_opts->dump_files_dir, + p_opts->enable_quirks ? "TRUE" : "FALSE", + p_opts->no_clients_rereg ? "TRUE" : "FALSE", + p_opts->no_multicast_option ? "TRUE" : "FALSE", + p_opts->disable_multicast ? "TRUE" : "FALSE", + p_opts->exit_on_fatal ? "TRUE" : "FALSE", + p_opts->console, + OSM_DEFAULT_CONSOLE_PORT, p_opts->console_port); + + fprintf(opts_file, + "#\n# QoS OPTIONS\n#\n" + "# Disable QoS setup\n" + "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); + + subn_dump_qos_options(opts_file, + "QoS default options", "qos", + &p_opts->qos_options); + fprintf(opts_file, "\n"); + subn_dump_qos_options(opts_file, + "QoS CA options", "qos_ca", + &p_opts->qos_ca_options); + fprintf(opts_file, "\n"); + subn_dump_qos_options(opts_file, + "QoS Switch Port 0 options", "qos_sw0", + &p_opts->qos_sw0_options); + fprintf(opts_file, "\n"); + subn_dump_qos_options(opts_file, + "QoS Switch external ports options", "qos_swe", + &p_opts->qos_swe_options); + fprintf(opts_file, "\n"); + subn_dump_qos_options(opts_file, + "QoS Router ports options", "qos_rtr", + &p_opts->qos_rtr_options); + + /* optional string attributes ... */ + + fclose(opts_file); + + return IB_SUCCESS; } -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Thu Aug 16 14:46:54 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 00:46:54 +0300 Subject: [ofa-general] [PATCH] opensm/osm_subnet.c: trivial naming changes In-Reply-To: <20070816214317.GK27819@sashak.voltaire.com> References: <20070816214317.GK27819@sashak.voltaire.com> Message-ID: <20070816214654.GL27819@sashak.voltaire.com> Some trivial renames + more formatting (with osm_indent). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_subnet.c | 662 +++++++++++++++++++------------------------- 1 files changed, 282 insertions(+), 380 deletions(-) diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 5b240e1..818d73f 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -479,10 +479,9 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_net64(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN uint64_t * p_val) +static void +opts_unpack_net64(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN uint64_t * p_val) { uint64_t val; @@ -506,10 +505,9 @@ __osm_subn_opts_unpack_net64(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_uint32(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN uint32_t * p_val) +static void +opts_unpack_uint32(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN uint32_t * p_val) { uint32_t val; @@ -528,10 +526,9 @@ __osm_subn_opts_unpack_uint32(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_uint16(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN uint16_t * p_val) +static void +opts_unpack_uint16(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN uint16_t * p_val) { uint16_t val; @@ -550,10 +547,9 @@ __osm_subn_opts_unpack_uint16(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_net16(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN uint16_t * p_val) +static void +opts_unpack_net16(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN uint16_t * p_val) { if (!strcmp(p_req_key, p_key)) { uint32_t val; @@ -572,10 +568,9 @@ __osm_subn_opts_unpack_net16(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_uint8(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN uint8_t * p_val) +static void +opts_unpack_uint8(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN uint8_t * p_val) { if (!strcmp(p_req_key, p_key)) { uint32_t val; @@ -594,10 +589,9 @@ __osm_subn_opts_unpack_uint8(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_boolean(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN boolean_t * p_val) +static void +opts_unpack_boolean(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN boolean_t * p_val) { if (!strcmp(p_req_key, p_key) && p_val_str) { boolean_t val; @@ -619,10 +613,9 @@ __osm_subn_opts_unpack_boolean(IN char *p_req_key, /********************************************************************** **********************************************************************/ -static inline void -__osm_subn_opts_unpack_charp(IN char *p_req_key, - IN char *p_key, - IN char *p_val_str, IN char **p_val) +static void +opts_unpack_charp(IN char *p_req_key, + IN char *p_key, IN char *p_val_str, IN char **p_val) { if (!strcmp(p_req_key, p_key) && p_val_str) { if ((*p_val == NULL) || strcmp(p_val_str, *p_val)) { @@ -652,15 +645,15 @@ subn_parse_qos_options(IN const char *prefix, char name[256]; snprintf(name, sizeof(name), "%s_max_vls", prefix); - __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->max_vls); + opts_unpack_uint32(name, p_key, p_val_str, &opt->max_vls); snprintf(name, sizeof(name), "%s_high_limit", prefix); - __osm_subn_opts_unpack_uint32(name, p_key, p_val_str, &opt->high_limit); + opts_unpack_uint32(name, p_key, p_val_str, &opt->high_limit); snprintf(name, sizeof(name), "%s_vlarb_high", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_high); + opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_high); snprintf(name, sizeof(name), "%s_vlarb_low", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_low); + opts_unpack_charp(name, p_key, p_val_str, &opt->vlarb_low); snprintf(name, sizeof(name), "%s_sl2vl", prefix); - __osm_subn_opts_unpack_charp(name, p_key, p_val_str, &opt->sl2vl); + opts_unpack_charp(name, p_key, p_val_str, &opt->sl2vl); } static int @@ -738,7 +731,7 @@ ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) /********************************************************************** **********************************************************************/ -static void osm_subn_verify_max_vls(IN unsigned *max_vls, IN char *key) +static void subn_verify_max_vls(IN unsigned *max_vls, IN char *key) { char buff[128]; @@ -752,7 +745,7 @@ static void osm_subn_verify_max_vls(IN unsigned *max_vls, IN char *key) } } -static void osm_subn_verify_high_limit(IN unsigned *high_limit, IN char *key) +static void subn_verify_high_limit(IN unsigned *high_limit, IN char *key) { char buff[128]; @@ -766,7 +759,7 @@ static void osm_subn_verify_high_limit(IN unsigned *high_limit, IN char *key) } } -static void osm_subn_verify_vlarb(IN char *vlarb, IN char *key) +static void subn_verify_vlarb(IN char *vlarb, IN char *key) { if (vlarb) { char buff[128]; @@ -850,7 +843,7 @@ static void osm_subn_verify_vlarb(IN char *vlarb, IN char *key) } } -static void osm_subn_verify_sl2vl(IN char *sl2vl, IN char *key) +static void subn_verify_sl2vl(IN char *sl2vl, IN char *key) { if (sl2vl) { char buff[128]; @@ -903,7 +896,7 @@ static void osm_subn_verify_sl2vl(IN char *sl2vl, IN char *key) } } -static void osm_subn_verify_conf_file(IN osm_subn_opt_t * const p_opts) +static void subn_verify_conf_file(IN osm_subn_opt_t * const p_opts) { char buff[128]; @@ -950,62 +943,61 @@ static void osm_subn_verify_conf_file(IN osm_subn_opt_t * const p_opts) } if (p_opts->no_qos == FALSE) { - osm_subn_verify_max_vls(&(p_opts->qos_options.max_vls), - "qos_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_ca_options.max_vls), - "qos_ca_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_sw0_options.max_vls), - "qos_sw0_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_swe_options.max_vls), - "qos_swe_max_vls"); - osm_subn_verify_max_vls(&(p_opts->qos_rtr_options.max_vls), - "qos_rtr_max_vls"); - - osm_subn_verify_high_limit(&(p_opts->qos_options.high_limit), - "qos_high_limit"); - osm_subn_verify_high_limit(&(p_opts->qos_ca_options.high_limit), - "qos_ca_high_limit"); - osm_subn_verify_high_limit(& - (p_opts->qos_sw0_options.high_limit), - "qos_sw0_high_limit"); - osm_subn_verify_high_limit(& - (p_opts->qos_swe_options.high_limit), - "qos_swe_high_limit"); - osm_subn_verify_high_limit(& - (p_opts->qos_rtr_options.high_limit), - "qos_rtr_high_limit"); - - osm_subn_verify_vlarb(p_opts->qos_options.vlarb_low, - "qos_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_low, - "qos_ca_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_low, - "qos_sw0_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_low, - "qos_swe_vlarb_low"); - osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_low, - "qos_rtr_vlarb_low"); - - osm_subn_verify_vlarb(p_opts->qos_options.vlarb_high, - "qos_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_ca_options.vlarb_high, - "qos_ca_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_high, - "qos_sw0_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_swe_options.vlarb_high, - "qos_swe_vlarb_high"); - osm_subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_high, - "qos_rtr_vlarb_high"); - - osm_subn_verify_sl2vl(p_opts->qos_options.sl2vl, "qos_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_ca_options.sl2vl, - "qos_ca_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_sw0_options.sl2vl, - "qos_sw0_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_swe_options.sl2vl, - "qos_swe_sl2vl"); - osm_subn_verify_sl2vl(p_opts->qos_rtr_options.sl2vl, - "qos_rtr_sl2vl"); + subn_verify_max_vls(&(p_opts->qos_options.max_vls), + "qos_max_vls"); + subn_verify_max_vls(&(p_opts->qos_ca_options.max_vls), + "qos_ca_max_vls"); + subn_verify_max_vls(&(p_opts->qos_sw0_options.max_vls), + "qos_sw0_max_vls"); + subn_verify_max_vls(&(p_opts->qos_swe_options.max_vls), + "qos_swe_max_vls"); + subn_verify_max_vls(&(p_opts->qos_rtr_options.max_vls), + "qos_rtr_max_vls"); + + subn_verify_high_limit(&(p_opts->qos_options.high_limit), + "qos_high_limit"); + subn_verify_high_limit(&(p_opts->qos_ca_options.high_limit), + "qos_ca_high_limit"); + subn_verify_high_limit(& + (p_opts->qos_sw0_options.high_limit), + "qos_sw0_high_limit"); + subn_verify_high_limit(& + (p_opts->qos_swe_options.high_limit), + "qos_swe_high_limit"); + subn_verify_high_limit(& + (p_opts->qos_rtr_options.high_limit), + "qos_rtr_high_limit"); + + subn_verify_vlarb(p_opts->qos_options.vlarb_low, + "qos_vlarb_low"); + subn_verify_vlarb(p_opts->qos_ca_options.vlarb_low, + "qos_ca_vlarb_low"); + subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_low, + "qos_sw0_vlarb_low"); + subn_verify_vlarb(p_opts->qos_swe_options.vlarb_low, + "qos_swe_vlarb_low"); + subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_low, + "qos_rtr_vlarb_low"); + + subn_verify_vlarb(p_opts->qos_options.vlarb_high, + "qos_vlarb_high"); + subn_verify_vlarb(p_opts->qos_ca_options.vlarb_high, + "qos_ca_vlarb_high"); + subn_verify_vlarb(p_opts->qos_sw0_options.vlarb_high, + "qos_sw0_vlarb_high"); + subn_verify_vlarb(p_opts->qos_swe_options.vlarb_high, + "qos_swe_vlarb_high"); + subn_verify_vlarb(p_opts->qos_rtr_options.vlarb_high, + "qos_rtr_vlarb_high"); + + subn_verify_sl2vl(p_opts->qos_options.sl2vl, "qos_sl2vl"); + subn_verify_sl2vl(p_opts->qos_ca_options.sl2vl, "qos_ca_sl2vl"); + subn_verify_sl2vl(p_opts->qos_sw0_options.sl2vl, + "qos_sw0_sl2vl"); + subn_verify_sl2vl(p_opts->qos_swe_options.sl2vl, + "qos_swe_sl2vl"); + subn_verify_sl2vl(p_opts->qos_rtr_options.sl2vl, + "qos_rtr_sl2vl"); } #ifdef ENABLE_OSM_PERF_MGR if (p_opts->perfmgr_sweep_time_s < 1) { @@ -1055,307 +1047,217 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) while (fgets(line, 1023, opts_file) != NULL) { /* get the first token */ p_key = strtok_r(line, " \t\n", &p_last); - if (p_key) { - p_val = strtok_r(NULL, " \t\n", &p_last); + if (!p_key) + continue; + + p_val = strtok_r(NULL, " \t\n", &p_last); + + opts_unpack_net64("guid", p_key, p_val, &p_opts->guid); + + opts_unpack_net64("m_key", p_key, p_val, &p_opts->m_key); + + opts_unpack_net64("sm_key", p_key, p_val, &p_opts->sm_key); + + opts_unpack_net64("subnet_prefix", + p_key, p_val, &p_opts->subnet_prefix); + + opts_unpack_net16("m_key_lease_period", + p_key, p_val, &p_opts->m_key_lease_period); + + opts_unpack_uint32("sweep_interval", + p_key, p_val, &p_opts->sweep_interval); + + opts_unpack_uint32("max_wire_smps", + p_key, p_val, &p_opts->max_wire_smps); + + opts_unpack_charp("console", p_key, p_val, &p_opts->console); + + opts_unpack_uint16("console_port", + p_key, p_val, &p_opts->console_port); + + opts_unpack_uint32("transaction_timeout", + p_key, p_val, &p_opts->transaction_timeout); + + opts_unpack_uint32("max_msg_fifo_timeout", + p_key, p_val, &p_opts->max_msg_fifo_timeout); + + opts_unpack_uint8("sm_priority", + p_key, p_val, &p_opts->sm_priority); + + opts_unpack_uint8("lmc", p_key, p_val, &p_opts->lmc); + + opts_unpack_boolean("lmc_esp0", + p_key, p_val, &p_opts->lmc_esp0); + + opts_unpack_uint8("max_op_vls", + p_key, p_val, &p_opts->max_op_vls); + + opts_unpack_uint8("force_link_speed", + p_key, p_val, &p_opts->force_link_speed); + + opts_unpack_boolean("reassign_lids", + p_key, p_val, &p_opts->reassign_lids); + + opts_unpack_boolean("ignore_other_sm", + p_key, p_val, &p_opts->ignore_other_sm); + + opts_unpack_boolean("single_thread", + p_key, p_val, &p_opts->single_thread); + + opts_unpack_boolean("no_multicast_option", + p_key, p_val, &p_opts->no_multicast_option); + + opts_unpack_boolean("disable_multicast", + p_key, p_val, &p_opts->disable_multicast); + + opts_unpack_boolean("force_log_flush", + p_key, p_val, &p_opts->force_log_flush); + + opts_unpack_uint8("subnet_timeout", + p_key, p_val, &p_opts->subnet_timeout); + + opts_unpack_uint8("packet_life_time", + p_key, p_val, &p_opts->packet_life_time); + + opts_unpack_uint8("vl_stall_count", + p_key, p_val, &p_opts->vl_stall_count); + + opts_unpack_uint8("leaf_vl_stall_count", + p_key, p_val, &p_opts->leaf_vl_stall_count); + + opts_unpack_uint8("head_of_queue_lifetime", + p_key, p_val, + &p_opts->head_of_queue_lifetime); + + opts_unpack_uint8("leaf_head_of_queue_lifetime", p_key, p_val, + &p_opts->leaf_head_of_queue_lifetime); + + opts_unpack_uint8("local_phy_errors_threshold", p_key, p_val, + &p_opts->local_phy_errors_threshold); + + opts_unpack_uint8("overrun_errors_threshold", + p_key, p_val, + &p_opts->overrun_errors_threshold); + + opts_unpack_uint32("sminfo_polling_timeout", + p_key, p_val, + &p_opts->sminfo_polling_timeout); + + opts_unpack_uint32("polling_retry_number", + p_key, p_val, &p_opts->polling_retry_number); + + opts_unpack_boolean("force_heavy_sweep", + p_key, p_val, &p_opts->force_heavy_sweep); - __osm_subn_opts_unpack_net64("guid", p_key, p_val, - &p_opts->guid); - - __osm_subn_opts_unpack_net64("m_key", p_key, p_val, - &p_opts->m_key); - - __osm_subn_opts_unpack_net64("sm_key", p_key, p_val, - &p_opts->sm_key); - - __osm_subn_opts_unpack_net64("subnet_prefix", - p_key, p_val, - &p_opts->subnet_prefix); - - __osm_subn_opts_unpack_net16("m_key_lease_period", - p_key, p_val, - &p_opts-> - m_key_lease_period); - - __osm_subn_opts_unpack_uint32("sweep_interval", - p_key, p_val, - &p_opts->sweep_interval); - - __osm_subn_opts_unpack_uint32("max_wire_smps", - p_key, p_val, - &p_opts->max_wire_smps); - - __osm_subn_opts_unpack_charp("console", - p_key, p_val, - &p_opts->console); - - __osm_subn_opts_unpack_uint16("console_port", - p_key, p_val, - &p_opts->console_port); - - __osm_subn_opts_unpack_uint32("transaction_timeout", - p_key, p_val, - &p_opts-> - transaction_timeout); - - __osm_subn_opts_unpack_uint32("max_msg_fifo_timeout", - p_key, p_val, - &p_opts-> - max_msg_fifo_timeout); - - __osm_subn_opts_unpack_uint8("sm_priority", - p_key, p_val, - &p_opts->sm_priority); - - __osm_subn_opts_unpack_uint8("lmc", - p_key, p_val, - &p_opts->lmc); - - __osm_subn_opts_unpack_boolean("lmc_esp0", - p_key, p_val, - &p_opts->lmc_esp0); - - __osm_subn_opts_unpack_uint8("max_op_vls", - p_key, p_val, - &p_opts->max_op_vls); - - __osm_subn_opts_unpack_uint8("force_link_speed", - p_key, p_val, - &p_opts->force_link_speed); - - __osm_subn_opts_unpack_boolean("reassign_lids", - p_key, p_val, - &p_opts->reassign_lids); - - __osm_subn_opts_unpack_boolean("ignore_other_sm", - p_key, p_val, - &p_opts-> - ignore_other_sm); - - __osm_subn_opts_unpack_boolean("single_thread", - p_key, p_val, - &p_opts->single_thread); - - __osm_subn_opts_unpack_boolean("no_multicast_option", - p_key, p_val, - &p_opts-> - no_multicast_option); - - __osm_subn_opts_unpack_boolean("disable_multicast", - p_key, p_val, - &p_opts-> - disable_multicast); - - __osm_subn_opts_unpack_boolean("force_log_flush", - p_key, p_val, - &p_opts-> - force_log_flush); - - __osm_subn_opts_unpack_uint8("subnet_timeout", - p_key, p_val, - &p_opts->subnet_timeout); - - __osm_subn_opts_unpack_uint8("packet_life_time", - p_key, p_val, - &p_opts->packet_life_time); - - __osm_subn_opts_unpack_uint8("vl_stall_count", - p_key, p_val, - &p_opts->vl_stall_count); - - __osm_subn_opts_unpack_uint8("leaf_vl_stall_count", - p_key, p_val, - &p_opts-> - leaf_vl_stall_count); - - __osm_subn_opts_unpack_uint8("head_of_queue_lifetime", - p_key, p_val, - &p_opts-> - head_of_queue_lifetime); - - __osm_subn_opts_unpack_uint8 - ("leaf_head_of_queue_lifetime", p_key, p_val, - &p_opts->leaf_head_of_queue_lifetime); - - __osm_subn_opts_unpack_uint8 - ("local_phy_errors_threshold", p_key, p_val, - &p_opts->local_phy_errors_threshold); - - __osm_subn_opts_unpack_uint8("overrun_errors_threshold", - p_key, p_val, - &p_opts-> - overrun_errors_threshold); - - __osm_subn_opts_unpack_uint32("sminfo_polling_timeout", - p_key, p_val, - &p_opts-> - sminfo_polling_timeout); - - __osm_subn_opts_unpack_uint32("polling_retry_number", - p_key, p_val, - &p_opts-> - polling_retry_number); - - __osm_subn_opts_unpack_boolean("force_heavy_sweep", - p_key, p_val, - &p_opts-> - force_heavy_sweep); - - __osm_subn_opts_unpack_uint8("log_flags", - p_key, p_val, - &p_opts->log_flags); - - __osm_subn_opts_unpack_boolean - ("port_profile_switch_nodes", p_key, p_val, - &p_opts->port_profile_switch_nodes); - - __osm_subn_opts_unpack_boolean("sweep_on_trap", - p_key, p_val, - &p_opts->sweep_on_trap); - - __osm_subn_opts_unpack_charp("routing_engine", - p_key, p_val, - &p_opts-> - routing_engine_name); - - __osm_subn_opts_unpack_boolean("connect_roots", - p_key, p_val, - &p_opts->connect_roots); - - __osm_subn_opts_unpack_charp("log_file", p_key, p_val, - &p_opts->log_file); - - __osm_subn_opts_unpack_uint32("log_max_size", - p_key, p_val, - (uint32_t *) & p_opts-> - log_max_size); - - __osm_subn_opts_unpack_charp("partition_config_file", - p_key, p_val, - &p_opts-> - partition_config_file); - - __osm_subn_opts_unpack_boolean - ("no_partition_enforcement", p_key, p_val, - &p_opts->no_partition_enforcement); - - __osm_subn_opts_unpack_boolean("no_qos", - p_key, p_val, - &p_opts->no_qos); - - __osm_subn_opts_unpack_boolean("accum_log_file", - p_key, p_val, - &p_opts->accum_log_file); - - __osm_subn_opts_unpack_charp("dump_files_dir", - p_key, p_val, - &p_opts->dump_files_dir); - - __osm_subn_opts_unpack_charp("lid_matrix_dump_file", - p_key, p_val, - &p_opts-> - lid_matrix_dump_file); - - __osm_subn_opts_unpack_charp("ucast_dump_file", - p_key, p_val, - &p_opts->ucast_dump_file); - - __osm_subn_opts_unpack_charp("root_guid_file", - p_key, p_val, - &p_opts->root_guid_file); - - __osm_subn_opts_unpack_charp("cn_guid_file", - p_key, p_val, - &p_opts->cn_guid_file); - - __osm_subn_opts_unpack_charp("sa_db_file", - p_key, p_val, - &p_opts->sa_db_file); - - __osm_subn_opts_unpack_boolean("exit_on_fatal", - p_key, p_val, - &p_opts->exit_on_fatal); - - __osm_subn_opts_unpack_boolean("honor_guid2lid_file", - p_key, p_val, - &p_opts-> - honor_guid2lid_file); - - __osm_subn_opts_unpack_boolean("daemon", - p_key, p_val, - &p_opts->daemon); - - __osm_subn_opts_unpack_boolean("sm_inactive", - p_key, p_val, - &p_opts->sm_inactive); - - __osm_subn_opts_unpack_boolean("babbling_port_policy", - p_key, p_val, - &p_opts-> - babbling_port_policy); + opts_unpack_uint8("log_flags", + p_key, p_val, &p_opts->log_flags); + + opts_unpack_boolean("port_profile_switch_nodes", p_key, p_val, + &p_opts->port_profile_switch_nodes); + + opts_unpack_boolean("sweep_on_trap", + p_key, p_val, &p_opts->sweep_on_trap); + + opts_unpack_charp("routing_engine", + p_key, p_val, &p_opts->routing_engine_name); + + opts_unpack_boolean("connect_roots", + p_key, p_val, &p_opts->connect_roots); + + opts_unpack_charp("log_file", p_key, p_val, &p_opts->log_file); + + opts_unpack_uint32("log_max_size", + p_key, p_val, + (uint32_t *) & p_opts->log_max_size); + + opts_unpack_charp("partition_config_file", + p_key, p_val, &p_opts->partition_config_file); + + opts_unpack_boolean("no_partition_enforcement", p_key, p_val, + &p_opts->no_partition_enforcement); + + opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); + + opts_unpack_boolean("accum_log_file", + p_key, p_val, &p_opts->accum_log_file); + + opts_unpack_charp("dump_files_dir", + p_key, p_val, &p_opts->dump_files_dir); + + opts_unpack_charp("lid_matrix_dump_file", + p_key, p_val, &p_opts->lid_matrix_dump_file); + + opts_unpack_charp("ucast_dump_file", + p_key, p_val, &p_opts->ucast_dump_file); + + opts_unpack_charp("root_guid_file", + p_key, p_val, &p_opts->root_guid_file); + + opts_unpack_charp("cn_guid_file", + p_key, p_val, &p_opts->cn_guid_file); + + opts_unpack_charp("sa_db_file", + p_key, p_val, &p_opts->sa_db_file); + + opts_unpack_boolean("exit_on_fatal", + p_key, p_val, &p_opts->exit_on_fatal); + + opts_unpack_boolean("honor_guid2lid_file", + p_key, p_val, &p_opts->honor_guid2lid_file); + + opts_unpack_boolean("daemon", p_key, p_val, &p_opts->daemon); + + opts_unpack_boolean("sm_inactive", + p_key, p_val, &p_opts->sm_inactive); + + opts_unpack_boolean("babbling_port_policy", + p_key, p_val, + &p_opts->babbling_port_policy); #ifdef ENABLE_OSM_PERF_MGR - __osm_subn_opts_unpack_boolean("perfmgr", - p_key, p_val, - &p_opts->perfmgr); - - __osm_subn_opts_unpack_boolean("perfmgr_redir", - p_key, p_val, - &p_opts->perfmgr_redir); - - __osm_subn_opts_unpack_uint16("perfmgr_sweep_time_s", - p_key, p_val, - &p_opts-> - perfmgr_sweep_time_s); - - __osm_subn_opts_unpack_uint32 - ("perfmgr_max_outstanding_queries", p_key, p_val, - &p_opts->perfmgr_max_outstanding_queries); - - __osm_subn_opts_unpack_charp("event_db_dump_file", - p_key, p_val, - &p_opts-> - event_db_dump_file); -#endif /* ENABLE_OSM_PERF_MGR */ + opts_unpack_boolean("perfmgr", p_key, p_val, &p_opts->perfmgr); - __osm_subn_opts_unpack_charp("event_plugin_name", - p_key, p_val, - &p_opts-> - event_plugin_name); + opts_unpack_boolean("perfmgr_redir", + p_key, p_val, &p_opts->perfmgr_redir); - subn_parse_qos_options("qos", - p_key, p_val, - &p_opts->qos_options); + opts_unpack_uint16("perfmgr_sweep_time_s", + p_key, p_val, &p_opts->perfmgr_sweep_time_s); - subn_parse_qos_options("qos_ca", - p_key, p_val, - &p_opts->qos_ca_options); + opts_unpack_uint32("perfmgr_max_outstanding_queries", + p_key, p_val, + &p_opts->perfmgr_max_outstanding_queries); - subn_parse_qos_options("qos_sw0", - p_key, p_val, - &p_opts->qos_sw0_options); + opts_unpack_charp("event_db_dump_file", + p_key, p_val, &p_opts->event_db_dump_file); +#endif /* ENABLE_OSM_PERF_MGR */ - subn_parse_qos_options("qos_swe", - p_key, p_val, - &p_opts->qos_swe_options); + opts_unpack_charp("event_plugin_name", + p_key, p_val, &p_opts->event_plugin_name); - subn_parse_qos_options("qos_rtr", - p_key, p_val, - &p_opts->qos_rtr_options); + subn_parse_qos_options("qos", + p_key, p_val, &p_opts->qos_options); - __osm_subn_opts_unpack_boolean("enable_quirks", - p_key, p_val, - &p_opts->enable_quirks); + subn_parse_qos_options("qos_ca", + p_key, p_val, &p_opts->qos_ca_options); - __osm_subn_opts_unpack_boolean("no_clients_rereg", - p_key, p_val, - &p_opts-> - no_clients_rereg); + subn_parse_qos_options("qos_sw0", + p_key, p_val, &p_opts->qos_sw0_options); - } + subn_parse_qos_options("qos_swe", + p_key, p_val, &p_opts->qos_swe_options); + + subn_parse_qos_options("qos_rtr", + p_key, p_val, &p_opts->qos_rtr_options); + + opts_unpack_boolean("enable_quirks", + p_key, p_val, &p_opts->enable_quirks); + + opts_unpack_boolean("no_clients_rereg", + p_key, p_val, &p_opts->no_clients_rereg); } fclose(opts_file); - osm_subn_verify_conf_file(p_opts); + subn_verify_conf_file(p_opts); return IB_SUCCESS; } -- 1.5.3.rc2.38.g11308 From weiny2 at llnl.gov Thu Aug 16 15:00:02 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 16 Aug 2007 15:00:02 -0700 Subject: [ofa-general] [PATCH] osm_perfmgr.c: Fix extra port being added to num_ports Message-ID: <20070816150002.4e324075.weiny2@llnl.gov> For some reason there was a +1 in here. This resulted in the dump function printing a fictitious extra port for each node. Ira >From 873fc9b19ddb8e4db068d0b2d9618310e57f3fd1 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Tue, 14 Aug 2007 19:16:56 -0700 Subject: [PATCH] Fix extra port being added to num_ports Signed-off-by: Ira K. Weiny --- opensm/opensm/osm_perfmgr.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/osm_perfmgr.c b/opensm/opensm/osm_perfmgr.c index 43689fa..34c18ff 100644 --- a/opensm/opensm/osm_perfmgr.c +++ b/opensm/opensm/osm_perfmgr.c @@ -473,7 +473,7 @@ __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) node_guid = cl_ntoh64(node->node_info.node_guid); /* make sure we have a database object ready to store this information */ - if (perfmgr_db_create_entry(pm->db, node_guid, num_ports + 1, + if (perfmgr_db_create_entry(pm->db, node_guid, num_ports, node->print_desc) != PERFMGR_EVENT_DB_SUCCESS) { -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-extra-port-being-added-to-num_ports.patch Type: application/octet-stream Size: 940 bytes Desc: not available URL: From sashak at voltaire.com Thu Aug 16 15:17:03 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 01:17:03 +0300 Subject: [ofa-general] Re: [PATCH] osm_perfmgr.c: Fix extra port being added to num_ports In-Reply-To: <20070816150002.4e324075.weiny2@llnl.gov> References: <20070816150002.4e324075.weiny2@llnl.gov> Message-ID: <20070816221703.GO27819@sashak.voltaire.com> On 15:00 Thu 16 Aug , Ira Weiny wrote: > For some reason there was a +1 in here. This resulted in the dump function > printing a fictitious extra port for each node. > > Ira > > > From 873fc9b19ddb8e4db068d0b2d9618310e57f3fd1 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Tue, 14 Aug 2007 19:16:56 -0700 > Subject: [PATCH] Fix extra port being added to num_ports > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From sashak at voltaire.com Thu Aug 16 15:23:42 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 01:23:42 +0300 Subject: [ofa-general] [PATCH] opensm/osm_sm_mad_ctrl.c: trivial changes In-Reply-To: <20070816213902.GJ27819@sashak.voltaire.com> References: <20070816213902.GJ27819@sashak.voltaire.com> Message-ID: <20070816222342.GR27819@sashak.voltaire.com> Some trivial flow consolidations and removing unneeded braces. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_sm_mad_ctrl.c | 265 +++++++++++++++++---------------------- 1 files changed, 117 insertions(+), 148 deletions(-) diff --git a/opensm/opensm/osm_sm_mad_ctrl.c b/opensm/opensm/osm_sm_mad_ctrl.c index 758262b..b846b61 100644 --- a/opensm/opensm/osm_sm_mad_ctrl.c +++ b/opensm/opensm/osm_sm_mad_ctrl.c @@ -82,40 +82,36 @@ __osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, /* Return the MAD & wrapper to the pool. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_retire_trans_mad: " "Retiring MAD with TID 0x%" PRIx64 "\n", cl_ntoh64(osm_madw_get_smp_ptr(p_madw)->trans_id)); - } osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); outstanding = cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_retire_trans_mad: " "%u QP0 MADs outstanding\n", p_ctrl->p_stats->qp0_mads_outstanding); - } if (outstanding == 0) { /* The wire is clean. Signal the state manager. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_retire_trans_mad: " "Posting Dispatcher message %s\n", osm_get_disp_msg_str (OSM_MSG_NO_SMPS_OUTSTANDING)); - } status = cl_disp_post(p_ctrl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - (void *) + OSM_MSG_NO_SMPS_OUTSTANDING, (void *) OSM_SIGNAL_NO_PENDING_TRANSACTIONS, NULL, NULL); @@ -167,12 +163,10 @@ __osm_sm_mad_ctrl_disp_done_callback(IN void *context, IN void *p_data) if (ib_smp_is_response(p_smp)) { CL_ASSERT(p_madw->resp_expected == FALSE); __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); - } else { - if (p_madw->resp_expected == TRUE) - __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); - else - osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); - } + } else if (p_madw->resp_expected == TRUE) + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + else + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); OSM_LOG_EXIT(p_ctrl->p_log); } @@ -198,12 +192,11 @@ __osm_sm_mad_ctrl_update_wire_stats(IN osm_sm_mad_ctrl_t * const p_ctrl) mads_on_wire = cl_atomic_dec(&p_ctrl->p_stats->qp0_mads_outstanding_on_wire); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_update_wire_stats: " "%u SMPs on the wire, %u outstanding\n", mads_on_wire, p_ctrl->p_stats->qp0_mads_outstanding); - } /* We can signal the VL15 controller to send another MAD @@ -240,13 +233,11 @@ __osm_sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * const p_ctrl, p_smp = osm_madw_get_smp_ptr(p_madw); - if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { - if (!ib_smp_is_d(p_smp)) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get_resp: ERR 3102: " - "'D' bit not set in returned SMP\n"); - osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); - } + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR && !ib_smp_is_d(p_smp)) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get_resp: ERR 3102: " + "'D' bit not set in returned SMP\n"); + osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_ERROR); } p_old_madw = (osm_madw_t *) transaction_context; @@ -312,33 +303,29 @@ __osm_sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * const p_ctrl, goto Exit; } - if (msg_id != CL_DISP_MSGID_NONE) { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ + if (msg_id == CL_DISP_MSGID_NONE) + goto Exit; - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_get_resp: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); - } + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ - status = cl_disp_post(p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl); + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_get_resp: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); - if (status != CL_SUCCESS) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get_resp: ERR 3104: " - "Dispatcher post message failed (%s) for attribute = 0x%X\n", - CL_STATUS_MSG(status), - cl_ntoh16(p_smp->attr_id)); - goto Exit; - } + status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, + __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get_resp: ERR 3104: " + "Dispatcher post message failed (%s) for attribute = 0x%X\n", + CL_STATUS_MSG(status), cl_ntoh16(p_smp->attr_id)); + goto Exit; } Exit: @@ -383,38 +370,35 @@ __osm_sm_mad_ctrl_process_get(IN osm_sm_mad_ctrl_t * const p_ctrl, break; } - if (msg_id != CL_DISP_MSGID_NONE) { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_get: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); - } - - status = cl_disp_post(p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl); - - if (status != CL_SUCCESS) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_get: ERR 3106: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG(status)); - goto Exit; - } - } else { + if (msg_id == CL_DISP_MSGID_NONE) { /* There is an unknown MAD attribute type for which there is no recipient. Simply retire the MAD here. */ osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_get: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + + status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, + __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_get: ERR 3106: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; } Exit: @@ -470,38 +454,35 @@ __osm_sm_mad_ctrl_process_set(IN osm_sm_mad_ctrl_t * const p_ctrl, break; } - if (msg_id != CL_DISP_MSGID_NONE) { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_set: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); - } - - status = cl_disp_post(p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl); - - if (status != CL_SUCCESS) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_set: ERR 3108: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG(status)); - goto Exit; - } - } else { + if (msg_id == CL_DISP_MSGID_NONE) { /* There is an unknown MAD attribute type for which there is no recipient. Simply retire the MAD here. */ osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_set: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + + status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, + __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_set: ERR 3108: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; } Exit: @@ -567,38 +548,35 @@ __osm_sm_mad_ctrl_process_trap(IN osm_sm_mad_ctrl_t * const p_ctrl, break; } - if (msg_id != CL_DISP_MSGID_NONE) { - /* - Post this MAD to the dispatcher for asynchronous - processing by the appropriate controller. - */ - - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { - osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_process_trap: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(msg_id)); - } - - status = cl_disp_post(p_ctrl->h_disp, - msg_id, - p_madw, - __osm_sm_mad_ctrl_disp_done_callback, - p_ctrl); - - if (status != CL_SUCCESS) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_process_trap: ERR 3110: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG(status)); - goto Exit; - } - } else { + if (msg_id == CL_DISP_MSGID_NONE) { /* There is an unknown MAD attribute type for which there is no recipient. Simply retire the MAD here. */ osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); + goto Exit; + } + + /* + Post this MAD to the dispatcher for asynchronous + processing by the appropriate controller. + */ + + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_process_trap: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(msg_id)); + + status = cl_disp_post(p_ctrl->h_disp, msg_id, p_madw, + __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); + + if (status != CL_SUCCESS) { + osm_log(p_ctrl->p_log, OSM_LOG_ERROR, + "__osm_sm_mad_ctrl_process_trap: ERR 3110: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(status)); + goto Exit; } Exit: @@ -642,12 +620,11 @@ __osm_sm_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, */ cl_atomic_inc(&p_ctrl->p_stats->qp0_mads_rcvd); - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_rcv_callback: " "%u QP0 MADs received\n", p_ctrl->p_stats->qp0_mads_rcvd); - } p_smp = osm_madw_get_smp_ptr(p_madw); @@ -664,13 +641,10 @@ __osm_sm_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, (p_smp->method == IB_MAD_METHOD_TRAP_REPRESS)) { CL_ASSERT(p_madw->resp_expected == FALSE); __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); - } else { - if (p_madw->resp_expected == TRUE) - __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, - p_madw); - else - osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); - } + } else if (p_madw->resp_expected == TRUE) + __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); + else + osm_mad_pool_put(p_ctrl->p_mad_pool, p_madw); goto Exit; } @@ -678,11 +652,10 @@ __osm_sm_mad_ctrl_rcv_callback(IN osm_madw_t * p_madw, if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_FRAMES)) osm_dump_dr_smp(p_ctrl->p_log, p_smp, OSM_LOG_FRAMES); - if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) { + if (p_smp->mgmt_class == IB_MCLASS_SUBN_DIR) status = ib_smp_get_status(p_smp); - } else { + else status = p_smp->status; - } if (status != 0) { osm_log(p_ctrl->p_log, OSM_LOG_ERROR, @@ -824,31 +797,28 @@ __osm_sm_mad_ctrl_send_err_cb(IN void *bind_context, IN osm_madw_t * p_madw) __osm_sm_mad_ctrl_update_wire_stats(p_ctrl); if (osm_madw_get_err_msg(p_madw) != CL_DISP_MSGID_NONE) { - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) { + if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, "__osm_sm_mad_ctrl_send_err_cb: " "Posting Dispatcher message %s\n", osm_get_disp_msg_str(osm_madw_get_err_msg (p_madw))); - } status = cl_disp_post(p_ctrl->h_disp, osm_madw_get_err_msg(p_madw), p_madw, __osm_sm_mad_ctrl_disp_done_callback, p_ctrl); - if (status != CL_SUCCESS) { + if (status != CL_SUCCESS) osm_log(p_ctrl->p_log, OSM_LOG_ERROR, "__osm_sm_mad_ctrl_send_err_cb: ERR 3115: " "Dispatcher post message failed (%s)\n", CL_STATUS_MSG(status)); - } - } else { + } else /* No error message was provided, just retire the MAD. */ __osm_sm_mad_ctrl_retire_trans_mad(p_ctrl, p_madw); - } OSM_LOG_EXIT(p_ctrl->p_log); } @@ -878,9 +848,8 @@ void osm_sm_mad_ctrl_destroy(IN osm_sm_mad_ctrl_t * const p_ctrl) { CL_ASSERT(p_ctrl); - if (p_ctrl->h_bind != CL_DISP_INVALID_HANDLE) { + if (p_ctrl->h_bind != CL_DISP_INVALID_HANDLE) osm_vendor_unbind(p_ctrl->h_bind); - } cl_disp_unregister(p_ctrl->h_disp); } -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Thu Aug 16 15:25:02 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 01:25:02 +0300 Subject: [ofa-general] [PATCH] opensm/osm_vl15intf.c: trivial changes Message-ID: <20070816222502.GS27819@sashak.voltaire.com> Drop unneeded braces, consolidate some flows. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_vl15intf.c | 59 ++++++++---------------------------------- 1 files changed, 11 insertions(+), 48 deletions(-) diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c index 5316a12..0f240df 100644 --- a/opensm/opensm/osm_vl15intf.c +++ b/opensm/opensm/osm_vl15intf.c @@ -77,9 +77,7 @@ __osm_vl15_poller( OSM_LOG_ENTER( p_vl->p_log, __osm_vl15_poller ); if ( p_vl->thread_state == OSM_THREAD_STATE_NONE) - { p_vl->thread_state = OSM_THREAD_STATE_RUN; - } while( p_vl->thread_state == OSM_THREAD_STATE_RUN ) { @@ -105,17 +103,13 @@ __osm_vl15_poller( if( p_madw != (osm_madw_t*)cl_qlist_end( p_fifo ) ) { if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "__osm_vl15_poller: " "Servicing p_madw = %p\n", p_madw ); - } if( osm_log_is_active( p_vl->p_log, OSM_LOG_FRAMES ) ) - { osm_dump_dr_smp( p_vl->p_log, osm_madw_get_smp_ptr( p_madw ), OSM_LOG_FRAMES ); - } /* Non-response-expected mads are not throttled on the wire @@ -131,15 +125,11 @@ __osm_vl15_poller( To avoid this confusion, preincrement the counts on the assumption that send() will succeed. */ - mads_on_wire = cl_atomic_inc( - &p_vl->p_stats->qp0_mads_outstanding_on_wire ); + mads_on_wire = cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); CL_ASSERT( mads_on_wire <= p_vl->max_wire_smps ); } else - { - unicasts_sent = cl_atomic_inc( - &p_vl->p_stats->qp0_unicasts_sent ); - } + unicasts_sent = cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); mads_sent = cl_atomic_inc( &p_vl->p_stats->qp0_mads_sent ); @@ -197,12 +187,10 @@ __osm_vl15_poller( Signal the state manager. */ if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "__osm_vl15_poller: " "Posting Dispatcher message %s\n", osm_get_disp_msg_str( OSM_MSG_NO_SMPS_OUTSTANDING ) ); - } cl_status = cl_disp_post( p_vl->h_disp, OSM_MSG_NO_SMPS_OUTSTANDING, @@ -211,54 +199,41 @@ __osm_vl15_poller( NULL ); if( cl_status != CL_SUCCESS ) - { osm_log( p_vl->p_log, OSM_LOG_ERROR, "__osm_vl15_poller: ERR 3E06: " "Dispatcher post message failed (%s)\n", CL_STATUS_MSG( cl_status ) ); - } } } } - else - { - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs on wire, %u outstanding, %u unicasts sent, " - "%u total sent\n", - p_vl->p_stats->qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding, - p_vl->p_stats->qp0_unicasts_sent, - p_vl->p_stats->qp0_mads_sent ); - } - } + else if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) + osm_log( p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "%u QP0 MADs on wire, %u outstanding, %u unicasts sent, " + "%u total sent\n", + p_vl->p_stats->qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding, + p_vl->p_stats->qp0_unicasts_sent, + p_vl->p_stats->qp0_mads_sent ); } else - { /* The VL15 FIFO is empty, so we have nothing left to do. */ status = cl_event_wait_on( &p_vl->signal, EVENT_NO_TIMEOUT, TRUE ); - } while( (p_vl->p_stats->qp0_mads_outstanding_on_wire >= (int32_t)p_vl->max_wire_smps ) && (p_vl->thread_state == OSM_THREAD_STATE_RUN ) ) - { status = cl_event_wait_on( &p_vl->signal, EVENT_NO_TIMEOUT, TRUE ); - } if( status != CL_SUCCESS ) - { osm_log( p_vl->p_log, OSM_LOG_ERROR, "__osm_vl15_poller: ERR 3E02: " "Event wait failed (%s)\n", CL_STATUS_MSG( status ) ); - } } /* @@ -425,11 +400,9 @@ osm_vl15_poll( (int32_t)p_vl->max_wire_smps ) { if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "osm_vl15_poll: " "Signalling poller thread\n" ); - } cl_event_signal( &p_vl->signal ); } @@ -449,11 +422,9 @@ osm_vl15_post( CL_ASSERT( p_vl->state == OSM_VL15_STATE_READY ); if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "osm_vl15_post: " "Posting p_madw = 0x%p\n", p_madw ); - } /* Determine in which fifo to place the pending madw. @@ -465,19 +436,15 @@ osm_vl15_post( cl_atomic_inc( &p_vl->p_stats->qp0_mads_outstanding ); } else - { cl_qlist_insert_tail( &p_vl->ufifo, (cl_list_item_t*)p_madw ); - } cl_spinlock_release( &p_vl->lock ); if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "osm_vl15_post: " "%u QP0 MADs on wire, %u QP0 MADs outstanding\n", p_vl->p_stats->qp0_mads_outstanding_on_wire, p_vl->p_stats->qp0_mads_outstanding ); - } osm_vl15_poll( p_vl ); @@ -508,11 +475,9 @@ osm_vl15_shutdown( while ( p_madw != (osm_madw_t*)cl_qlist_end( &p_vl->ufifo ) ) { if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "osm_vl15_shutdown: " "Releasing Response p_madw = %p\n", p_madw ); - } osm_mad_pool_put( p_mad_pool, p_madw ); @@ -524,11 +489,9 @@ osm_vl15_shutdown( while ( p_madw != (osm_madw_t*)cl_qlist_end( &p_vl->rfifo ) ) { if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - { osm_log( p_vl->p_log, OSM_LOG_DEBUG, "osm_vl15_shutdown: " "Releasing Request p_madw = %p\n", p_madw ); - } osm_mad_pool_put( p_mad_pool, p_madw ); cl_atomic_dec( &p_vl->p_stats->qp0_mads_outstanding ); -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Thu Aug 16 15:26:07 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 01:26:07 +0300 Subject: [ofa-general] [PATCH] opensm/osm_vl15intf.c: indentation changes In-Reply-To: <20070816222502.GS27819@sashak.voltaire.com> References: <20070816222502.GS27819@sashak.voltaire.com> Message-ID: <20070816222607.GT27819@sashak.voltaire.com> Formatting changes as produced by opensm/osm_indent. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_vl15intf.c | 758 ++++++++++++++++++++--------------------- 1 files changed, 370 insertions(+), 388 deletions(-) diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c index 0f240df..f6050a7 100644 --- a/opensm/opensm/osm_vl15intf.c +++ b/opensm/opensm/osm_vl15intf.c @@ -47,7 +47,7 @@ #if HAVE_CONFIG_H # include -#endif /* HAVE_CONFIG_H */ +#endif /* HAVE_CONFIG_H */ #include #include @@ -59,448 +59,430 @@ #include #include - /********************************************************************** **********************************************************************/ -void -__osm_vl15_poller( - IN void *p_ptr ) +void __osm_vl15_poller(IN void *p_ptr) { - ib_api_status_t status; - osm_madw_t *p_madw; - uint32_t mads_sent; - uint32_t unicasts_sent; - uint32_t mads_on_wire; - osm_vl15_t* const p_vl = (osm_vl15_t*)p_ptr; - cl_qlist_t* p_fifo; - - OSM_LOG_ENTER( p_vl->p_log, __osm_vl15_poller ); - - if ( p_vl->thread_state == OSM_THREAD_STATE_NONE) - p_vl->thread_state = OSM_THREAD_STATE_RUN; - - while( p_vl->thread_state == OSM_THREAD_STATE_RUN ) - { - /* - Start servicing the FIFOs by pulling off MAD wrappers - and passing them to the transport interface. - There are lots of corner cases here so tread carefully. - - The unicast FIFO has priority, since somebody is waiting - for a timely response. - */ - cl_spinlock_acquire( &p_vl->lock ); - - if( cl_qlist_count( &p_vl->ufifo ) != 0 ) - p_fifo = &p_vl->ufifo; - else - p_fifo = &p_vl->rfifo; - - p_madw = (osm_madw_t*)cl_qlist_remove_head( p_fifo ); - - cl_spinlock_release( &p_vl->lock ); - - if( p_madw != (osm_madw_t*)cl_qlist_end( p_fifo ) ) - { - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "Servicing p_madw = %p\n", p_madw ); - - if( osm_log_is_active( p_vl->p_log, OSM_LOG_FRAMES ) ) - osm_dump_dr_smp( p_vl->p_log, - osm_madw_get_smp_ptr( p_madw ), OSM_LOG_FRAMES ); - - /* - Non-response-expected mads are not throttled on the wire - since we can have no confirmation that they arrived - at their destination. - */ - if( p_madw->resp_expected == TRUE ) - { - /* - Note that other threads may not see the response MAD - arrive before send() even returns. - In that case, the wire count would temporarily go negative. - To avoid this confusion, preincrement the counts on the - assumption that send() will succeed. - */ - mads_on_wire = cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); - CL_ASSERT( mads_on_wire <= p_vl->max_wire_smps ); - } - else - unicasts_sent = cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); - - mads_sent = cl_atomic_inc( &p_vl->p_stats->qp0_mads_sent ); - - status = osm_vendor_send( - osm_madw_get_bind_handle( p_madw ), - p_madw, p_madw->resp_expected ); - - if( status != IB_SUCCESS ) - { - uint32_t outstanding; - cl_status_t cl_status; - - osm_log( p_vl->p_log, OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E03: " - "MAD send failed (%s)\n", - ib_get_err_str( status ) ); - - /* - The MAD was never successfully sent, so - fix up the pre-incremented count values. - */ - - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire - that were incremented in the code above. */ - mads_sent = cl_atomic_dec( &p_vl->p_stats->qp0_mads_sent ); - if( p_madw->resp_expected == TRUE ) - cl_atomic_dec( &p_vl->p_stats->qp0_mads_outstanding_on_wire ); - - /* - The following code is similar to the code in - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the - qp0_mads_outstanding counter, and if we reached 0 - need to call - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order - to wake up the state mgr). - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter - should be decremented and handled accordingly only if this is a mad - with resp_expected == TRUE. - */ - if ( p_madw->resp_expected == TRUE ) - { - outstanding = cl_atomic_dec( &p_vl->p_stats->qp0_mads_outstanding ); - - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs outstanding\n", - p_vl->p_stats->qp0_mads_outstanding ); - - if( outstanding == 0 ) - { - /* - The wire is clean. - Signal the state manager. - */ - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str( OSM_MSG_NO_SMPS_OUTSTANDING ) ); - - cl_status = cl_disp_post( p_vl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - (void *)OSM_SIGNAL_NO_PENDING_TRANSACTIONS, - NULL, - NULL ); - - if( cl_status != CL_SUCCESS ) - osm_log( p_vl->p_log, OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E06: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG( cl_status ) ); - } - } - } - else if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs on wire, %u outstanding, %u unicasts sent, " - "%u total sent\n", - p_vl->p_stats->qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding, - p_vl->p_stats->qp0_unicasts_sent, - p_vl->p_stats->qp0_mads_sent ); - } - else - /* - The VL15 FIFO is empty, so we have nothing left to do. - */ - status = cl_event_wait_on( &p_vl->signal, - EVENT_NO_TIMEOUT, TRUE ); - - while( (p_vl->p_stats->qp0_mads_outstanding_on_wire >= - (int32_t)p_vl->max_wire_smps ) && - (p_vl->thread_state == OSM_THREAD_STATE_RUN ) ) - status = cl_event_wait_on( &p_vl->signal, - EVENT_NO_TIMEOUT, TRUE ); - - if( status != CL_SUCCESS ) - osm_log( p_vl->p_log, OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E02: " - "Event wait failed (%s)\n", - CL_STATUS_MSG( status ) ); - } - - /* - since we abort immediately when the state != OSM_THREAD_STATE_RUN - we might have some mads on the queues. After the thread exits - the vl15 destroy routine should put these mads back... - */ - - OSM_LOG_EXIT( p_vl->p_log ); + ib_api_status_t status; + osm_madw_t *p_madw; + uint32_t mads_sent; + uint32_t unicasts_sent; + uint32_t mads_on_wire; + osm_vl15_t *const p_vl = (osm_vl15_t *) p_ptr; + cl_qlist_t *p_fifo; + + OSM_LOG_ENTER(p_vl->p_log, __osm_vl15_poller); + + if (p_vl->thread_state == OSM_THREAD_STATE_NONE) + p_vl->thread_state = OSM_THREAD_STATE_RUN; + + while (p_vl->thread_state == OSM_THREAD_STATE_RUN) { + /* + Start servicing the FIFOs by pulling off MAD wrappers + and passing them to the transport interface. + There are lots of corner cases here so tread carefully. + + The unicast FIFO has priority, since somebody is waiting + for a timely response. + */ + cl_spinlock_acquire(&p_vl->lock); + + if (cl_qlist_count(&p_vl->ufifo) != 0) + p_fifo = &p_vl->ufifo; + else + p_fifo = &p_vl->rfifo; + + p_madw = (osm_madw_t *) cl_qlist_remove_head(p_fifo); + + cl_spinlock_release(&p_vl->lock); + + if (p_madw != (osm_madw_t *) cl_qlist_end(p_fifo)) { + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "Servicing p_madw = %p\n", p_madw); + + if (osm_log_is_active(p_vl->p_log, OSM_LOG_FRAMES)) + osm_dump_dr_smp(p_vl->p_log, + osm_madw_get_smp_ptr(p_madw), + OSM_LOG_FRAMES); + + /* + Non-response-expected mads are not throttled on the wire + since we can have no confirmation that they arrived + at their destination. + */ + if (p_madw->resp_expected == TRUE) { + /* + Note that other threads may not see the response MAD + arrive before send() even returns. + In that case, the wire count would temporarily go negative. + To avoid this confusion, preincrement the counts on the + assumption that send() will succeed. + */ + mads_on_wire = + cl_atomic_inc(&p_vl->p_stats-> + qp0_mads_outstanding_on_wire); + CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); + } else + unicasts_sent = + cl_atomic_inc(&p_vl->p_stats-> + qp0_unicasts_sent); + + mads_sent = + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); + + status = + osm_vendor_send(osm_madw_get_bind_handle(p_madw), + p_madw, p_madw->resp_expected); + + if (status != IB_SUCCESS) { + uint32_t outstanding; + cl_status_t cl_status; + + osm_log(p_vl->p_log, OSM_LOG_ERROR, + "__osm_vl15_poller: ERR 3E03: " + "MAD send failed (%s)\n", + ib_get_err_str(status)); + + /* + The MAD was never successfully sent, so + fix up the pre-incremented count values. + */ + + /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire + that were incremented in the code above. */ + mads_sent = + cl_atomic_dec(&p_vl->p_stats-> + qp0_mads_sent); + if (p_madw->resp_expected == TRUE) + cl_atomic_dec(&p_vl->p_stats-> + qp0_mads_outstanding_on_wire); + + /* + The following code is similar to the code in + __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the + qp0_mads_outstanding counter, and if we reached 0 - need to call + the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order + to wake up the state mgr). + There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. + This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike + __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where + resp_expected == TRUE. As a result, the qp0_mads_outstanding counter + should be decremented and handled accordingly only if this is a mad + with resp_expected == TRUE. + */ + if (p_madw->resp_expected == TRUE) { + outstanding = + cl_atomic_dec(&p_vl->p_stats-> + qp0_mads_outstanding); + + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "%u QP0 MADs outstanding\n", + p_vl->p_stats-> + qp0_mads_outstanding); + + if (outstanding == 0) { + /* + The wire is clean. + Signal the state manager. + */ + if (osm_log_is_active + (p_vl->p_log, + OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, + OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str + (OSM_MSG_NO_SMPS_OUTSTANDING)); + + cl_status = + cl_disp_post(p_vl->h_disp, + OSM_MSG_NO_SMPS_OUTSTANDING, + (void *) + OSM_SIGNAL_NO_PENDING_TRANSACTIONS, + NULL, NULL); + + if (cl_status != CL_SUCCESS) + osm_log(p_vl->p_log, + OSM_LOG_ERROR, + "__osm_vl15_poller: ERR 3E06: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG + (cl_status)); + } + } + } else + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "%u QP0 MADs on wire, %u outstanding, %u unicasts sent, " + "%u total sent\n", + p_vl->p_stats-> + qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding, + p_vl->p_stats->qp0_unicasts_sent, + p_vl->p_stats->qp0_mads_sent); + } else + /* + The VL15 FIFO is empty, so we have nothing left to do. + */ + status = cl_event_wait_on(&p_vl->signal, + EVENT_NO_TIMEOUT, TRUE); + + while ((p_vl->p_stats->qp0_mads_outstanding_on_wire >= + (int32_t) p_vl->max_wire_smps) && + (p_vl->thread_state == OSM_THREAD_STATE_RUN)) + status = cl_event_wait_on(&p_vl->signal, + EVENT_NO_TIMEOUT, TRUE); + + if (status != CL_SUCCESS) + osm_log(p_vl->p_log, OSM_LOG_ERROR, + "__osm_vl15_poller: ERR 3E02: " + "Event wait failed (%s)\n", + CL_STATUS_MSG(status)); + } + + /* + since we abort immediately when the state != OSM_THREAD_STATE_RUN + we might have some mads on the queues. After the thread exits + the vl15 destroy routine should put these mads back... + */ + + OSM_LOG_EXIT(p_vl->p_log); } /********************************************************************** **********************************************************************/ -void -osm_vl15_construct( - IN osm_vl15_t* const p_vl ) +void osm_vl15_construct(IN osm_vl15_t * const p_vl) { - memset( p_vl, 0, sizeof(*p_vl) ); - p_vl->state = OSM_VL15_STATE_INIT; - p_vl->thread_state = OSM_THREAD_STATE_NONE; - cl_event_construct( &p_vl->signal ); - cl_spinlock_construct( &p_vl->lock ); - cl_qlist_init( &p_vl->rfifo ); - cl_qlist_init( &p_vl->ufifo ); - cl_thread_construct( &p_vl->poller ); - p_vl->h_disp = CL_DISP_INVALID_HANDLE; + memset(p_vl, 0, sizeof(*p_vl)); + p_vl->state = OSM_VL15_STATE_INIT; + p_vl->thread_state = OSM_THREAD_STATE_NONE; + cl_event_construct(&p_vl->signal); + cl_spinlock_construct(&p_vl->lock); + cl_qlist_init(&p_vl->rfifo); + cl_qlist_init(&p_vl->ufifo); + cl_thread_construct(&p_vl->poller); + p_vl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** **********************************************************************/ void -osm_vl15_destroy( - IN osm_vl15_t* const p_vl, - IN struct _osm_mad_pool *p_pool) +osm_vl15_destroy(IN osm_vl15_t * const p_vl, IN struct _osm_mad_pool *p_pool) { - osm_madw_t* p_madw; + osm_madw_t *p_madw; - OSM_LOG_ENTER( p_vl->p_log, osm_vl15_destroy ); + OSM_LOG_ENTER(p_vl->p_log, osm_vl15_destroy); - /* - Signal our threads that we're leaving. - */ - p_vl->thread_state = OSM_THREAD_STATE_EXIT; + /* + Signal our threads that we're leaving. + */ + p_vl->thread_state = OSM_THREAD_STATE_EXIT; - /* - Don't trigger unless event has been initialized. - Destroy the thread before we tear down the other objects. - */ - if( p_vl->state != OSM_VL15_STATE_INIT ) - cl_event_signal( &p_vl->signal ); + /* + Don't trigger unless event has been initialized. + Destroy the thread before we tear down the other objects. + */ + if (p_vl->state != OSM_VL15_STATE_INIT) + cl_event_signal(&p_vl->signal); - cl_thread_destroy( &p_vl->poller ); + cl_thread_destroy(&p_vl->poller); - /* - Return the outstanding messages to the pool - */ + /* + Return the outstanding messages to the pool + */ - cl_spinlock_acquire( &p_vl->lock ); + cl_spinlock_acquire(&p_vl->lock); - while (!cl_is_qlist_empty( &p_vl->rfifo)) - { - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->rfifo); - osm_mad_pool_put( p_pool, p_madw ); - } - while (!cl_is_qlist_empty( &p_vl->ufifo)) - { - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->ufifo); - osm_mad_pool_put( p_pool, p_madw ); - } + while (!cl_is_qlist_empty(&p_vl->rfifo)) { + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->rfifo); + osm_mad_pool_put(p_pool, p_madw); + } + while (!cl_is_qlist_empty(&p_vl->ufifo)) { + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->ufifo); + osm_mad_pool_put(p_pool, p_madw); + } - cl_spinlock_release( &p_vl->lock ); + cl_spinlock_release(&p_vl->lock); - cl_event_destroy( &p_vl->signal ); - p_vl->state = OSM_VL15_STATE_INIT; - cl_spinlock_destroy( &p_vl->lock ); + cl_event_destroy(&p_vl->signal); + p_vl->state = OSM_VL15_STATE_INIT; + cl_spinlock_destroy(&p_vl->lock); - OSM_LOG_EXIT( p_vl->p_log ); + OSM_LOG_EXIT(p_vl->p_log); } /********************************************************************** **********************************************************************/ ib_api_status_t -osm_vl15_init( - IN osm_vl15_t* const p_vl, - IN osm_vendor_t* const p_vend, - IN osm_log_t* const p_log, - IN osm_stats_t* const p_stats, - IN const int32_t max_wire_smps, - IN osm_subn_t* const p_subn, - IN cl_dispatcher_t* const p_disp, - IN cl_plock_t* const p_lock - ) +osm_vl15_init(IN osm_vl15_t * const p_vl, + IN osm_vendor_t * const p_vend, + IN osm_log_t * const p_log, + IN osm_stats_t * const p_stats, + IN const int32_t max_wire_smps, + IN osm_subn_t * const p_subn, + IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) { - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER( p_log, osm_vl15_init ); - - p_vl->p_vend = p_vend; - p_vl->p_log = p_log; - p_vl->p_stats = p_stats; - p_vl->max_wire_smps = max_wire_smps; - p_vl->p_subn = p_subn; - p_vl->p_lock = p_lock; - - status = cl_event_init( &p_vl->signal, FALSE ); - if( status != IB_SUCCESS ) - goto Exit; - - p_vl->state = OSM_VL15_STATE_READY; - - status = cl_spinlock_init( &p_vl->lock ); - if( status != IB_SUCCESS ) - goto Exit; - - /* - Initialize the thread after all other dependent objects - have been initialized. - */ - status = cl_thread_init( &p_vl->poller, __osm_vl15_poller, p_vl, - "opensm poller" ); - if( status != IB_SUCCESS ) - goto Exit; - - p_vl->h_disp = cl_disp_register( - p_disp, - CL_DISP_MSGID_NONE, - NULL, - NULL ); - - if( p_vl->h_disp == CL_DISP_INVALID_HANDLE ) - { - osm_log( p_log, OSM_LOG_ERROR, - "osm_vl15_init: ERR 3E01: " - "Dispatcher registration failed\n" ); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - Exit: - OSM_LOG_EXIT( p_log ); - return( status ); + ib_api_status_t status = IB_SUCCESS; + + OSM_LOG_ENTER(p_log, osm_vl15_init); + + p_vl->p_vend = p_vend; + p_vl->p_log = p_log; + p_vl->p_stats = p_stats; + p_vl->max_wire_smps = max_wire_smps; + p_vl->p_subn = p_subn; + p_vl->p_lock = p_lock; + + status = cl_event_init(&p_vl->signal, FALSE); + if (status != IB_SUCCESS) + goto Exit; + + p_vl->state = OSM_VL15_STATE_READY; + + status = cl_spinlock_init(&p_vl->lock); + if (status != IB_SUCCESS) + goto Exit; + + /* + Initialize the thread after all other dependent objects + have been initialized. + */ + status = cl_thread_init(&p_vl->poller, __osm_vl15_poller, p_vl, + "opensm poller"); + if (status != IB_SUCCESS) + goto Exit; + + p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); + + if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_vl15_init: ERR 3E01: " + "Dispatcher registration failed\n"); + status = IB_INSUFFICIENT_RESOURCES; + goto Exit; + } + + Exit: + OSM_LOG_EXIT(p_log); + return (status); } /********************************************************************** **********************************************************************/ -void -osm_vl15_poll( - IN osm_vl15_t* const p_vl ) +void osm_vl15_poll(IN osm_vl15_t * const p_vl) { - OSM_LOG_ENTER( p_vl->p_log, osm_vl15_poll ); - - CL_ASSERT( p_vl->state == OSM_VL15_STATE_READY ); - - /* - If we have room for more VL15 MADs on the wire, - then signal the poller thread. - - This is not an airtight check, since the poller thread - could be just about to send another MAD as we signal - the event here. To cover this rare case, the poller - thread checks for a spurious wake-up. - */ - if( p_vl->p_stats->qp0_mads_outstanding_on_wire < - (int32_t)p_vl->max_wire_smps ) - { - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "osm_vl15_poll: " - "Signalling poller thread\n" ); - - cl_event_signal( &p_vl->signal ); - } - - OSM_LOG_EXIT( p_vl->p_log ); + OSM_LOG_ENTER(p_vl->p_log, osm_vl15_poll); + + CL_ASSERT(p_vl->state == OSM_VL15_STATE_READY); + + /* + If we have room for more VL15 MADs on the wire, + then signal the poller thread. + + This is not an airtight check, since the poller thread + could be just about to send another MAD as we signal + the event here. To cover this rare case, the poller + thread checks for a spurious wake-up. + */ + if (p_vl->p_stats->qp0_mads_outstanding_on_wire < + (int32_t) p_vl->max_wire_smps) { + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "osm_vl15_poll: " "Signalling poller thread\n"); + + cl_event_signal(&p_vl->signal); + } + + OSM_LOG_EXIT(p_vl->p_log); } /********************************************************************** **********************************************************************/ -void -osm_vl15_post( - IN osm_vl15_t* const p_vl, - IN osm_madw_t* const p_madw ) +void osm_vl15_post(IN osm_vl15_t * const p_vl, IN osm_madw_t * const p_madw) { - OSM_LOG_ENTER( p_vl->p_log, osm_vl15_post ); - - CL_ASSERT( p_vl->state == OSM_VL15_STATE_READY ); - - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "osm_vl15_post: " - "Posting p_madw = 0x%p\n", p_madw ); - - /* - Determine in which fifo to place the pending madw. - */ - cl_spinlock_acquire( &p_vl->lock ); - if( p_madw->resp_expected == TRUE ) - { - cl_qlist_insert_tail( &p_vl->rfifo, (cl_list_item_t*)p_madw ); - cl_atomic_inc( &p_vl->p_stats->qp0_mads_outstanding ); - } - else - cl_qlist_insert_tail( &p_vl->ufifo, (cl_list_item_t*)p_madw ); - cl_spinlock_release( &p_vl->lock ); - - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "osm_vl15_post: " - "%u QP0 MADs on wire, %u QP0 MADs outstanding\n", - p_vl->p_stats->qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding ); - - osm_vl15_poll( p_vl ); - - OSM_LOG_EXIT( p_vl->p_log ); + OSM_LOG_ENTER(p_vl->p_log, osm_vl15_post); + + CL_ASSERT(p_vl->state == OSM_VL15_STATE_READY); + + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "osm_vl15_post: " "Posting p_madw = 0x%p\n", p_madw); + + /* + Determine in which fifo to place the pending madw. + */ + cl_spinlock_acquire(&p_vl->lock); + if (p_madw->resp_expected == TRUE) { + cl_qlist_insert_tail(&p_vl->rfifo, (cl_list_item_t *) p_madw); + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding); + } else + cl_qlist_insert_tail(&p_vl->ufifo, (cl_list_item_t *) p_madw); + cl_spinlock_release(&p_vl->lock); + + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "osm_vl15_post: " + "%u QP0 MADs on wire, %u QP0 MADs outstanding\n", + p_vl->p_stats->qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding); + + osm_vl15_poll(p_vl); + + OSM_LOG_EXIT(p_vl->p_log); } void -osm_vl15_shutdown( - IN osm_vl15_t* const p_vl, - IN osm_mad_pool_t* const p_mad_pool) +osm_vl15_shutdown(IN osm_vl15_t * const p_vl, + IN osm_mad_pool_t * const p_mad_pool) { - osm_madw_t* p_madw; + osm_madw_t *p_madw; - OSM_LOG_ENTER( p_vl->p_log, osm_vl15_shutdown ); + OSM_LOG_ENTER(p_vl->p_log, osm_vl15_shutdown); - /* we only should get here after the VL15 interface was initialized */ - CL_ASSERT( p_vl->state == OSM_VL15_STATE_READY ); + /* we only should get here after the VL15 interface was initialized */ + CL_ASSERT(p_vl->state == OSM_VL15_STATE_READY); - /* grap a lock on the object */ - cl_spinlock_acquire( &p_vl->lock ); + /* grap a lock on the object */ + cl_spinlock_acquire(&p_vl->lock); - cl_disp_unregister( p_vl->h_disp ); + cl_disp_unregister(p_vl->h_disp); - /* go over all outstanding MADs and retire their transactions */ + /* go over all outstanding MADs and retire their transactions */ - /* first we handle the list of response MADs */ - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->ufifo ); - while ( p_madw != (osm_madw_t*)cl_qlist_end( &p_vl->ufifo ) ) - { - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "osm_vl15_shutdown: " - "Releasing Response p_madw = %p\n", p_madw ); + /* first we handle the list of response MADs */ + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->ufifo); + while (p_madw != (osm_madw_t *) cl_qlist_end(&p_vl->ufifo)) { + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "osm_vl15_shutdown: " + "Releasing Response p_madw = %p\n", p_madw); - osm_mad_pool_put( p_mad_pool, p_madw ); + osm_mad_pool_put(p_mad_pool, p_madw); - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->ufifo ); - } + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->ufifo); + } - /* Request MADs we send out */ - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->rfifo ); - while ( p_madw != (osm_madw_t*)cl_qlist_end( &p_vl->rfifo ) ) - { - if( osm_log_is_active( p_vl->p_log, OSM_LOG_DEBUG ) ) - osm_log( p_vl->p_log, OSM_LOG_DEBUG, - "osm_vl15_shutdown: " - "Releasing Request p_madw = %p\n", p_madw ); + /* Request MADs we send out */ + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->rfifo); + while (p_madw != (osm_madw_t *) cl_qlist_end(&p_vl->rfifo)) { + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "osm_vl15_shutdown: " + "Releasing Request p_madw = %p\n", p_madw); - osm_mad_pool_put( p_mad_pool, p_madw ); - cl_atomic_dec( &p_vl->p_stats->qp0_mads_outstanding ); + osm_mad_pool_put(p_mad_pool, p_madw); + cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); - p_madw = (osm_madw_t*)cl_qlist_remove_head( &p_vl->rfifo ); - } + p_madw = (osm_madw_t *) cl_qlist_remove_head(&p_vl->rfifo); + } - /* free the lock */ - cl_spinlock_release( &p_vl->lock ); + /* free the lock */ + cl_spinlock_release(&p_vl->lock); - OSM_LOG_EXIT( p_vl->p_log ); + OSM_LOG_EXIT(p_vl->p_log); } -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Thu Aug 16 15:27:12 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 17 Aug 2007 01:27:12 +0300 Subject: [ofa-general] [PATCH] opensm/osm_vl15intf.c: trivial code reordering In-Reply-To: <20070816222607.GT27819@sashak.voltaire.com> References: <20070816222502.GS27819@sashak.voltaire.com> <20070816222607.GT27819@sashak.voltaire.com> Message-ID: <20070816222712.GU27819@sashak.voltaire.com> Trivial code reordering to make it more readable. Signed-off-by: Sasha Khapyorsky --- opensm/opensm/osm_vl15intf.c | 234 ++++++++++++++++++++--------------------- 1 files changed, 114 insertions(+), 120 deletions(-) diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c index f6050a7..bc667b6 100644 --- a/opensm/opensm/osm_vl15intf.c +++ b/opensm/opensm/osm_vl15intf.c @@ -61,13 +61,124 @@ /********************************************************************** **********************************************************************/ -void __osm_vl15_poller(IN void *p_ptr) + +static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) { ib_api_status_t status; - osm_madw_t *p_madw; + cl_status_t cl_status; uint32_t mads_sent; uint32_t unicasts_sent; uint32_t mads_on_wire; + uint32_t outstanding; + + /* + Non-response-expected mads are not throttled on the wire + since we can have no confirmation that they arrived + at their destination. + */ + if (p_madw->resp_expected == TRUE) { + /* + Note that other threads may not see the response MAD + arrive before send() even returns. + In that case, the wire count would temporarily go negative. + To avoid this confusion, preincrement the counts on the + assumption that send() will succeed. + */ + mads_on_wire = + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); + CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); + } else + unicasts_sent = + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); + + mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); + + status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), + p_madw, p_madw->resp_expected); + + if (status == IB_SUCCESS) { + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "%u QP0 MADs on wire, %u outstanding, " + "%u unicasts sent, %u total sent\n", + p_vl->p_stats->qp0_mads_outstanding_on_wire, + p_vl->p_stats->qp0_mads_outstanding, + p_vl->p_stats->qp0_unicasts_sent, + p_vl->p_stats->qp0_mads_sent); + return; + } + + osm_log(p_vl->p_log, OSM_LOG_ERROR, + "__osm_vl15_poller: ERR 3E03: " + "MAD send failed (%s)\n", ib_get_err_str(status)); + + /* + The MAD was never successfully sent, so + fix up the pre-incremented count values. + */ + + /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire + that were incremented in the code above. */ + mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); + + if (p_madw->resp_expected == FALSE) + return; + + cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); + + /* + The following code is similar to the code in + __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the + qp0_mads_outstanding counter, and if we reached 0 - need to call + the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order + to wake up the state mgr). + There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. + This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike + __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where + resp_expected == TRUE. As a result, the qp0_mads_outstanding counter + should be decremented and handled accordingly only if this is a mad + with resp_expected == TRUE. + */ + + outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); + + osm_log(p_vl->p_log, OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "%u QP0 MADs outstanding\n", + p_vl->p_stats->qp0_mads_outstanding); + + if (outstanding) + return; + + /* + The wire is clean. + Signal the state manager. + */ + if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) + osm_log(p_vl->p_log, + OSM_LOG_DEBUG, + "__osm_vl15_poller: " + "Posting Dispatcher message %s\n", + osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); + + cl_status = cl_disp_post(p_vl->h_disp, + OSM_MSG_NO_SMPS_OUTSTANDING, (void *) + OSM_SIGNAL_NO_PENDING_TRANSACTIONS, + NULL, NULL); + + if (cl_status != CL_SUCCESS) + osm_log(p_vl->p_log, + OSM_LOG_ERROR, + "__osm_vl15_poller: ERR 3E06: " + "Dispatcher post message failed (%s)\n", + CL_STATUS_MSG(cl_status)); +} + +static void __osm_vl15_poller(IN void *p_ptr) +{ + ib_api_status_t status; + osm_madw_t *p_madw; osm_vl15_t *const p_vl = (osm_vl15_t *) p_ptr; cl_qlist_t *p_fifo; @@ -107,124 +218,7 @@ void __osm_vl15_poller(IN void *p_ptr) osm_madw_get_smp_ptr(p_madw), OSM_LOG_FRAMES); - /* - Non-response-expected mads are not throttled on the wire - since we can have no confirmation that they arrived - at their destination. - */ - if (p_madw->resp_expected == TRUE) { - /* - Note that other threads may not see the response MAD - arrive before send() even returns. - In that case, the wire count would temporarily go negative. - To avoid this confusion, preincrement the counts on the - assumption that send() will succeed. - */ - mads_on_wire = - cl_atomic_inc(&p_vl->p_stats-> - qp0_mads_outstanding_on_wire); - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); - } else - unicasts_sent = - cl_atomic_inc(&p_vl->p_stats-> - qp0_unicasts_sent); - - mads_sent = - cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); - - status = - osm_vendor_send(osm_madw_get_bind_handle(p_madw), - p_madw, p_madw->resp_expected); - - if (status != IB_SUCCESS) { - uint32_t outstanding; - cl_status_t cl_status; - - osm_log(p_vl->p_log, OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E03: " - "MAD send failed (%s)\n", - ib_get_err_str(status)); - - /* - The MAD was never successfully sent, so - fix up the pre-incremented count values. - */ - - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire - that were incremented in the code above. */ - mads_sent = - cl_atomic_dec(&p_vl->p_stats-> - qp0_mads_sent); - if (p_madw->resp_expected == TRUE) - cl_atomic_dec(&p_vl->p_stats-> - qp0_mads_outstanding_on_wire); - - /* - The following code is similar to the code in - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the - qp0_mads_outstanding counter, and if we reached 0 - need to call - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order - to wake up the state mgr). - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter - should be decremented and handled accordingly only if this is a mad - with resp_expected == TRUE. - */ - if (p_madw->resp_expected == TRUE) { - outstanding = - cl_atomic_dec(&p_vl->p_stats-> - qp0_mads_outstanding); - - osm_log(p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs outstanding\n", - p_vl->p_stats-> - qp0_mads_outstanding); - - if (outstanding == 0) { - /* - The wire is clean. - Signal the state manager. - */ - if (osm_log_is_active - (p_vl->p_log, - OSM_LOG_DEBUG)) - osm_log(p_vl->p_log, - OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str - (OSM_MSG_NO_SMPS_OUTSTANDING)); - - cl_status = - cl_disp_post(p_vl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - (void *) - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, - NULL, NULL); - - if (cl_status != CL_SUCCESS) - osm_log(p_vl->p_log, - OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E06: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG - (cl_status)); - } - } - } else - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - osm_log(p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs on wire, %u outstanding, %u unicasts sent, " - "%u total sent\n", - p_vl->p_stats-> - qp0_mads_outstanding_on_wire, - p_vl->p_stats->qp0_mads_outstanding, - p_vl->p_stats->qp0_unicasts_sent, - p_vl->p_stats->qp0_mads_sent); + vl15_send_mad(p_vl, p_madw); } else /* The VL15 FIFO is empty, so we have nothing left to do. -- 1.5.3.rc2.38.g11308 From weiny2 at llnl.gov Thu Aug 16 16:25:27 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 16 Aug 2007 16:25:27 -0700 Subject: [ofa-general] Re: [ewg] OFED 1.2.5 - GA release In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> Message-ID: <20070816162527.1934e2dc.weiny2@llnl.gov> On Thu, 16 Aug 2007 18:36:56 +0300 "Tziporet Koren" wrote: > I am happy to announce on OFED 1.2.5 GA release. > > The release can be found under: > http://www.openfabrics.org/builds/ofed-1.2.5/release/ Cool, could I get the git tree/tag for each of the sub projects? Especially the kernel. Thanks, IRa > > And later it will be on the OpenFabrics download page: > http://www.openfabrics.org/downloads.htm > > This release was done in a joint effort of all companies in the EWG group. > I wish to thank all who contributed to the success of this release. > > Tziporet > > =============================================================================== > > Release summary: > ================ > The OpenFabrics Enterprise Distribution (OFED) version 1.2.5 software package > supporting InfiniBand and iWARP fabrics. It is composed of several software > modules intended for use on a computer cluster constructed as an InfiniBand > subnet or an iWARP network. > > OFED package contains the following components: > =============================================== > The OFED Distribution package generates RPMs for installing the following: > > OFED 1.2.5 Contents > ------------------- > The OFED package contains the following components: > o OpenFabrics core and ULPs: > - IB HCA drivers (mthca, mlx4, ipath, ehca) > - iWARP RNIC driver (cxgb3) > - core > - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Host, RDS, > uDAPL and VNIC. > o OpenFabrics utilities: > - OpenSM (OSM): InfiniBand Subnet Manager > - Diagnostic tools > - Performance tests > o MPI: > - OSU MPI stack supporting the InfiniBand and iWARP interface > - Open MPI stack supporting the InfiniBand and iWARP interface > - OSU MVAPICH2 stack supporting the InfiniBand and iWARP interface > - MPI benchmark tests (OSU benchmarks, Intel MPI benchmarks, Presta) > o Extra packages: > - open-iscsi: open-iscsi initiator with iSER support > - ib-bonding: Bonding driver for IPoIB interface > o Sources of all software modules (under conditions mentioned in the modules' > LICENSE files) > o Documentation > > Third Party Packages > -------------------- > The following third party packages have been tested with OFED 1.2.5: > 1. Intel MPI, Version 3.0 - Package ID: l_mpi_p_3.0.043 > 2. HP MPI, Version 2.2.5 > > > Main Changes from OFED 1.2 > ========================== > Changes in components: > The following components are different between OFED 1.2 and OFED 1.2.5: > User space: > - libmlx4 (new) > - mstflint (was enhanced to support ConnectX IB) > - ofascripts > - perftest > - MVAPICH > - opensm > > Kernel: > - Code is based on kernel 2.6.22 > - mthca > - mlx4_core (new) > - mlx4_ib(new) > - core (the verbs providers) > - cxgb3 > - RDS > > See each package release notes for the specific changes from OFED 1.2 > > Supported Platforms and Operating Systems > ========================================= > o CPU architectures: > - x86_64 > - x86 > - ia64 > - ppc64 > > o Linux Operating Systems: > - RedHat EL4 up3: 2.6.9-34.ELsmp > - RedHat EL4 up4: 2.6.9-42.ELsmp > - RedHat EL4 up5: 2.6.9-55.ELsmp > - RedHat EL5: 2.6.18-8.el5 > - SLES9 SP3: 2.6.5-7.244-smp > - SLES10: 2.6.16.21-0.8-smp > - SLES10 SP1: 2.6.16.46-0.12-smp (partially tested) > - kernel.org: 2.6.20.x and 2.6.22.x > > > HCAs and RNICs Supported > ------------------------ > This release supports IB HCAs by Mellanox Technologies, Qlogic and IBM as > well as iWARP RNICs by Chelsio Communications. > > o Mellanox Technologies HCAs (SDR and DDR modes are supported): > - InfiniHost (fw-23108 Rev 3.5.000) > - InfiniHost III Ex (MemFree: fw-25218 Rev 5.2.000 > with memory: fw-25208 Rev 4.8.200) > - InfiniHost III Lx (fw-25204 Rev 1.2.000) > - ConnectX IB (fw-25408 Rev 2.2.000) > > For official firmware versions please see: > http://www.mellanox.com/support/firmware_table.php > > o Qlogic HCAs: > - QHT6040 (PathScale InfiniPath HT-460) > - QHT6140 (PathScale InfiniPath HT-465) > - QLE6140 (PathScale InfiniPath PE-880) > > o IBM HCAs: > - GX Dual-port 4x IB HCA > - GX Dual-port 12x IB HCA > > o Chelsio RNICs: > - S310/S320 10GbE Storage Accelerators > - R310E 10GbE iWARP Adapters > > Infiniband Switches Supported > ----------------------------- > This release was tested with switches and gateways provided by the following > companies: > - Cisco > - Voltaire > - Qlogic > - Flextronics > > From sean.hefty at intel.com Thu Aug 16 16:46:35 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Thu, 16 Aug 2007 16:46:35 -0700 Subject: [ofa-general] [PATCH] librdmacm 1/2: add valgrind support to auto-tools configuration file In-Reply-To: <46C412CE.1040701@dev.mellanox.co.il> References: <200708151352.42026.dotanb@dev.mellanox.co.il> <46C38C94.8060805@ichips.intel.com> <46C412CE.1040701@dev.mellanox.co.il> Message-ID: <000101c7e05f$aed63fa0$ff0da8c0@amr.corp.intel.com> > The resulting executables will still run without Valgrind, just a > little bit more slowly than they otherwise would, but otherwise > unchanged. When not running on valgrind, each client request > consumes very few (eg. 7) instructions, so the resulting performance > loss is negligible unless you plan to execute client requests > millions of times per second. Nevertheless, if that is still a > problem, you can compile with the NVALGRIND symbol defined (gcc > -DNVALGRIND) so that client requests are not even compiled in. */ It just seems that we're using two checks where we really want one. We check to see if memcheck.h exists, and if it does, we include it. But if the user doesn't want valgrind, then we disable using it through NVALGRIND. How about something more like: if with-valgrind is set then { if memcheck is found then INCLUDE_MEMCHECK_H = 1 if with_valgrind then set CPPFLAGS else error } The error message in your second patch is a little misleading otherwise, since it could occur even though valgrind support was not requested. - Sean From pradeeps at linux.vnet.ibm.com Thu Aug 16 18:03:51 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Thu, 16 Aug 2007 18:03:51 -0700 Subject: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs In-Reply-To: <20070816123616.GI5684@mellanox.co.il> References: <20070816123616.GI5684@mellanox.co.il> Message-ID: <46C4F3F7.7020001@linux.vnet.ibm.com> Michael S. Tsirkin wrote: > Use common CQ for all TX QPs: keep a per-device counter of outstanding TX WRs, > and stop the interface when this counter reaches the send queue size, to avoid > CQ overruns. This should help reduce the number of interrupts for > bi-directional traffic (such as TCP). CQ overrun generates asynchronous errors. If CQ overruns are avoided, how does that reduce interrupts? On a side note, as a further extension to this idea, would it be worth considering sendq_size and recvq_size being device specific in the future, instead of being module specific? > > This helps fix "driver is hogging interrupts" errors reported for IPoIB send side. > See e.g. https://bugs.openfabrics.org/show_bug.cgi?id=508 > In bug 508, timer ticks were lost because some device was hogging interrupts. What is the association between dropping packets and hogging interrupts? Can you expand on that? Pradeep From kohls at kohls.chtah.com Thu Aug 16 20:18:08 2007 From: kohls at kohls.chtah.com (Kohls.com) Date: Fri, 17 Aug 2007 03:18:08 -0000 Subject: [ofa-general] Free Shipping Ends Saturday + Bonus Buys Message-ID: http://kohls.chtah.com/a/tBGxQ9LBBZVhBBVjvHkBVHokNrK/kohl28-3 ************************************************************************** Kohl's Charge Savings + Free Shipping ************************************************************************** FREE* Standard Shipping when you spend $75 or more! ************************************************************************** Save an EXTRA 30%, 20% or 15% on EVERYTHING** Shop through Tuesday, August 21, and save an extra 30%, 20% or 15% on all sale-, regular- and clearance-priced merchandise every time you use your Kohl's Charge in-store and online! It's easy! Printed on your Kohl's Charge mailer is a Kohls.com Promo Code. Just enter this Promo Code when you checkout with your Kohl's Charge and you'll save! Even if you don't have a Promo Code, you'll automatically save an extra 15% with your Kohl's Charge. Visit Kohls.com for complete details. ************************************************************************** School Days Sale Back-to-school time is just around the corner, so make sure they're ready to study in style! Shop in-store and online for the most up-to-date fashions at great prices! ************************************************************************** 30-50% Off ENTIRE STOCK Candie's Collections Juniors' Apparel Girls' 4-6x Apparel Girls' 7-16 Apparel Juniors' Shoes Girls' Shoes Accessories Jewelry 30-40% Off Juniors' Shoes 50-60% Off ENTIRE STOCK Student Lounge Bedding Bath Dorm Accents, Furniture & Storage ************************************************************************** Shop our selection of Bonus Buys! 40-50% Off Women's N.Y.L., Gloria Vanderbilt, Fu Da & Tek Gear Fitness & Active Apparel 40-50% Off Juniors' Activewear 50-60% Off Young Men's Shorts 40% Off ENTIRE STOCK Infants', Toddlers' & Girls' 4-6x Carter's, OSHKOSH & Healthtex Playwear 40-50% Off ENTIRE STOCK Women's, Juniors', Men's, Boys' & Girls' Sonoma LIFE+style, SO and Croft & Barrow Shoes and Sandals ************************************************************************** Shop more Bonus Buys! Good through Tuesday ************************************************************************** Back To School Shop - My Transformation - Shop for back to school now! ************************************************************************** Back To School Gift Card - One Card Fits All! Give them a Gift Card for Back to School! ************************************************************************** 60-80% Clearance Shop Kohls.com Clearance for up to 60-80% off*** original prices on items from every department. But hurry--quantities are limited and deals like these won't last! *Surcharges may apply due to size, weight or special handling required. If your item has a surcharge, it will appear on the product page. **Discount is nontransferable. This offer is not valid for price adjustments on prior purchases, as payment toward your Kohl's Charge or in conjunction with other percent-off offers. This offer is also not valid on purchases of Kohl's Gift Cards or on purchases of Kohl's Cares for Kids merchandise. Excludes shipping and sales tax. Subject to credit approval. ***Clearance prices represent savings off original prices. Interim markdowns may have been taken. Sorry, no price adjustments. ************************************************************************** This mailbox is unattended, so please do not reply to this message. Instead, e-mail us at myaccount.help at kohls.com, or write to us at Kohl's Department Stores, Attention: Customer Service, N56 W17000 Ridgewood Drive, Menomonee Falls, WI 53051. If you no longer wish to receive e-mails from Kohls.com, unsubscribe by pasting this link into the Address field of your Internet browser: http://kohls.chtah.com/a/tBGxQ9LBBZVhBBVjvHkBVHokNrK/kohl43-3 ************************************************************************** School Days Sale prices good through August 25, 2007. Free Standard Shipping offer good through August 18, 2007. Bonus Buys good through August 21, 2007. Kohl's Charge offer good through August 21, 2007. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Thu Aug 16 21:35:05 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 17 Aug 2007 07:35:05 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-17:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Aug_14_19:41:02_2007 [3faa14cf64da6e58900c1abbcafe0f33ae1c69e2] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From dotanb at dev.mellanox.co.il Thu Aug 16 22:30:49 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Fri, 17 Aug 2007 08:30:49 +0300 Subject: [ofa-general] [PATCH] librdmacm 1/2: add valgrind support to auto-tools configuration file In-Reply-To: <000101c7e05f$aed63fa0$ff0da8c0@amr.corp.intel.com> References: <200708151352.42026.dotanb@dev.mellanox.co.il> <46C38C94.8060805@ichips.intel.com> <46C412CE.1040701@dev.mellanox.co.il> <000101c7e05f$aed63fa0$ff0da8c0@amr.corp.intel.com> Message-ID: <46C53289.2080509@dev.mellanox.co.il> Sean Hefty wrote: >> The resulting executables will still run without Valgrind, just a >> little bit more slowly than they otherwise would, but otherwise >> unchanged. When not running on valgrind, each client request >> consumes very few (eg. 7) instructions, so the resulting performance >> loss is negligible unless you plan to execute client requests >> millions of times per second. Nevertheless, if that is still a >> problem, you can compile with the NVALGRIND symbol defined (gcc >> -DNVALGRIND) so that client requests are not even compiled in. */ >> > > It just seems that we're using two checks where we really want one. We check to > see if memcheck.h exists, and if it does, we include it. But if the user > doesn't want valgrind, then we disable using it through NVALGRIND. > > How about something more like: > > if with-valgrind is set then { > if memcheck is found then > INCLUDE_MEMCHECK_H = 1 > if with_valgrind then set CPPFLAGS > else > error > } > > The error message in your second patch is a little misleading otherwise, since > it could occur even though valgrind support was not requested. > > - Sean > > Basically, you are right. I wrote this piece of code based on the libibverbs to be consistent with it, maybe Roland can contribute to this thread too ... I think that the way it is written now, it allows to the library to be prepared to valgrind anyway and use its header file when possible (to allow the user later on use valgrind) unless the user explicitly don't want to use valgrind (to avoid the minor overhead of marking the buffer as valid) or if the user explicitly specified that he wants valgrind, but the requested header file doesn't exist. I think that we should keep this as the way i suggested because libibverbs work this way for quite a while and there wasn't any problem with this implementation and i would like to be consistent with it. what do you think? thanks Dotan From erezz at voltaire.com Thu Aug 16 22:41:15 2007 From: erezz at voltaire.com (Erez Zilber) Date: Fri, 17 Aug 2007 08:41:15 +0300 Subject: [ofa-general] RE: OFED 1.2.5 - GA release References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com><20070816110030.2wsorokd15skkcwo@mobilemail.stanford.edu> <20070816191003.GC7197@mellanox.co.il> Message-ID: <39C75744D164D948A170E9792AF8E7CA110A59@exil.voltaire.com> >> Quoting Kilian Cavalotti : >> Subject: Re: OFED 1.2.5 - GA release >> >> Hi all, >> >> Quoting Tziporet Koren : >> >I am happy to announce on OFED 1.2.5 GA release. >> >> Thanks for the release. I just wanted to let you know that I got a >> compilation error (using RHEL4) in ofa_kernel: >> >> In file included from >> /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attri >> bute_container.c:1: >> /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/../drivers/base/attribute_contai >> ner.c:23:18: base.h: No such file or directory >> make[5]: *** >> [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi/attribute_con >> tainer.o] Error 1 >> make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/scsi] Error 2 >> make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 >> make[2]: *** [modules] Error 2 >> make[1]: *** [modules] Error 2 >> make[1]: Leaving directory >> `/usr/src/linux-2.6.9-55.EL_lustre-1.6.1-obj/x86_64/s >> mp' >> make: *** [kernel] Error 2 >> error: Bad exit status from /var/tmp/rpm-tmp.56294 (%install) >> >> >> The drivers/base/attribute_container.c include a "base.h" which is not >> provided. Copying the vanilla kernel drivers/base/base.h file in the >> ofa_kernel drivers/base directory works, as well as the following >> dirty patch: >> >> --- attribute_container.c.orig 2007-08-16 10:59:08.000000000 -0700 >> +++ attribute_container.c 2007-08-16 10:57:21.000000000 -0700 >> @@ -20,7 +20,7 @@ >> #include >> #include >> >> -#include "base.h" >> +#include <../drivers/base/base.h> >> >> /* This is a private structure used to tie the classdev and the >> * container .. it should never be visible outside this file */ > > This is iser stuff. > A simple workaround is to disable iser before build. > Erez, can you check this please? Strange. Are you testing it on a standard RHAS4 up5 machine? Can you send the ofed.conf file that you were using? I'd like to try that in our lab. Erez From krkumar2 at in.ibm.com Thu Aug 16 23:06:03 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Fri, 17 Aug 2007 11:36:03 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB Message-ID: Hi Dave, > I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will > run a longer one tonight). The results are (results in KB/s, and %): I ran a 8.5 hours run with no batching + another 8.5 hours run with batching (Buffer sizes: "32 128 512 4096 16384", Threads: "1 8 32", Each test run time: 3 minutes, Iterations to average: 5). TCP seems to get a small improvement. Thanks, - KK ------------------------------------------------------------------------------- TCP ----------- Size:32 Procs:1 3415 3321 -2.75 Size:128 Procs:1 13094 13388 2.24 Size:512 Procs:1 49037 50683 3.35 Size:4096 Procs:1 114646 114619 -.02 Size:16384 Procs:1 114626 114644 .01 Size:32 Procs:8 22675 22633 -.18 Size:128 Procs:8 77994 77297 -.89 Size:512 Procs:8 114716 114711 0 Size:4096 Procs:8 114637 114636 0 Size:16384 Procs:8 95814 114638 19.64 Size:32 Procs:32 23240 23349 .46 Size:128 Procs:32 82284 82247 -.04 Size:512 Procs:32 114885 114769 -.10 Size:4096 Procs:32 95735 114634 19.74 Size:16384 Procs:32 114736 114641 -.08 Average: 1151534 1190210 3.36% ------------------------------------------------------------------------------- No Delay: --------- Size:32 Procs:1 3002 2873 -4.29 Size:128 Procs:1 11853 11801 -.43 Size:512 Procs:1 45565 45837 .59 Size:4096 Procs:1 114511 114485 -.02 Size:16384 Procs:1 114521 114555 .02 Size:32 Procs:8 8026 8029 .03 Size:128 Procs:8 31589 31573 -.05 Size:512 Procs:8 111506 105766 -5.14 Size:4096 Procs:8 114455 114454 0 Size:16384 Procs:8 95833 114491 19.46 Size:32 Procs:32 8005 8027 .27 Size:128 Procs:32 31475 31505 .09 Size:512 Procs:32 114558 113687 -.76 Size:4096 Procs:32 114784 114447 -.29 Size:16384 Procs:32 114719 114496 -.19 Average: 1046026 1034402 -1.11% ------------------------------------------------------------------------------- From vlad at lists.openfabrics.org Fri Aug 17 01:40:20 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 17 Aug 2007 01:40:20 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070817-0100 daily build status Message-ID: <20070817084020.3DAE4E60868@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From aali at ocf.co.uk Fri Aug 17 02:38:54 2007 From: aali at ocf.co.uk (Arif Ali) Date: Fri, 17 Aug 2007 10:38:54 +0100 Subject: [ofa-general] OFED 1.2 & 1.2.5 on SLES 10 SP1 with Xen[Scanned] In-Reply-To: <46C47E16.8040308@ocf.co.uk> References: <46C47E16.8040308@ocf.co.uk> Message-ID: <46C56CAE.7090402@ocf.co.uk> Arif Ali wrote: > Hi all, > > I am currently trying to install/compile OFED on SLES 10 SP1 with the > xen kernel and each time the compilation errors out at the same place. > > kernel: 2.6.16.46-0.12-xen > OS: SLES 10 SP1 x86_64 > > build All applications via the build.sh script > > the last few lines from the OFED.log file > > gcc > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.vnic_sys.o.d > -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include > -Iinclude2/asm/mach-xen -D__KERNEL__ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > include/linux/autoconf.h -include > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > -D__XEN_INTERFACE_VERSION__=0x00030203 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic > -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs > -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common > -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel > -pipe -fno-reorder-blocks -Wno-sign-compare > -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx > -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector > -Wdeclaration-after-statement -Wno-pointer-sign > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 > -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vnic_sys)" > -D"KBUILD_MODNAME=KBUILD_STR(ib_vnic)" -c -o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.tmp_vnic_sys.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.c > > ld -m elf_x86_64 -r -o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/ib_vnic.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_main.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_ib.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_viport.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_control.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_data.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_netpath.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_config.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.o > > make -f /usr/src/linux-2.6.16.46-0.12/scripts/Makefile.build > obj=/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > gcc > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.cxgb3_main.o.d > -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > include/linux/autoconf.h -include > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > -Iinclude2/asm/mach-xen -D__KERNEL__ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > include/linux/autoconf.h -include > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > -D__XEN_INTERFACE_VERSION__=0x00030203 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 -Wall > -Wundef -Wstrict-prototypes -Wno-trigraphs > -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common > -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel > -pipe -fno-reorder-blocks -Wno-sign-compare > -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx > -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector > -Wdeclaration-after-statement -Wno-pointer-sign > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 > -DMODULE -D"KBUILD_STR(s)=#s" > -D"KBUILD_BASENAME=KBUILD_STR(cxgb3_main)" > -D"KBUILD_MODNAME=KBUILD_STR(cxgb3)" -c -o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.tmp_cxgb3_main.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c > In file included from include2/asm/smp.h:20, > from /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, > from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, > from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, > from > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: > > include2/asm/io_apic.h: In function ‘io_apic_read’: > include2/asm/io_apic.h:171: error: implicit declaration of function > ‘HYPERVISOR_physdev_op’ > In file included from > /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, > from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, > from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, > from > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: > > include2/asm/smp.h: In function ‘hard_smp_processor_id’: > include2/asm/smp.h:79: error: implicit declaration of function > ‘GET_APIC_ID’ > include2/asm/smp.h:79: error: ‘APIC_BASE’ undeclared (first use in > this function) > include2/asm/smp.h:79: error: (Each undeclared identifier is reported > only once > include2/asm/smp.h:79: error: for each function it appears in.) > include2/asm/smp.h:79: error: ‘APIC_ID’ undeclared (first use in this > function) > include2/asm/smp.h: In function ‘logical_smp_processor_id’: > include2/asm/smp.h:137: error: implicit declaration of function > ‘GET_APIC_LOGICAL_ID’ > include2/asm/smp.h:137: error: ‘APIC_BASE’ undeclared (first use in > this function) > include2/asm/smp.h:137: error: ‘APIC_LDR’ undeclared (first use in > this function) > make[5]: *** > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.o] > Error 1 > make[4]: *** > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3] Error 2 > make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 > make[2]: *** [modules] Error 2 > make[1]: *** [modules] Error 2 > make[1]: Leaving directory `/usr/src/linux-2.6.16.46-0.12-obj/x86_64/xen' > make: *** [kernel] Error 2 > error: Bad exit status from /var/tmp/rpm-tmp.87108 (%install) > > > RPM build errors: > user vlad does not exist - using root > group vlad does not exist - using root > user vlad does not exist - using root > group vlad does not exist - using root > Bad exit status from /var/tmp/rpm-tmp.87108 (%install) > ERROR: Failed executing "rpmbuild --rebuild --define '_topdir > /var/tmp/OFEDRPM' --define '_prefix /usr' --define 'build_root > /var/tmp/OFED' --define 'configure_options --with-cxgb3-mod > --with-ipath_inf-mod --with-ipoib-mod --with-iser-mod --with-mthca-mod > --with-sdp-mod --with-srp-mod --with-core-mod --with-user_mad-mod > --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod > --with-rds-mod --with-vnic-mod ' --define 'KVERSION > 2.6.16.46-0.12-xen' --define 'KSRC > /lib/modules/2.6.16.46-0.12-xen/build' --define 'build_kernel_ib 1' > --define 'build_kernel_ib_devel 1' --define 'NETWORK_CONF_DIR > /etc/sysconfig/network' --define 'modprobe_update 1' --define > 'include_ipoib_conf 1' /root/OFED-1.2.5/SRPMS/ofa_kernel-1.2.5-0.src.rpm" > Hi All, I have successfully compiled the OFED 1.2 under SLES 10 GA in Xen (2.6.16.21-0.8-xen), but the customer would like SP1 as that has Xen 3.1. Any info/fixes would be greatly appreciated thanks in advance -- regards, Arif Ali Software Engineer OCF plc Mobile: +44 (0)7970 148 122 Office: +44 (0)114 257 2200 Fax: +44 (0)114 257 0022 Email: aali at ocf.co.uk Web: http://www.ocf.co.uk Support Phone: +44 (0)845 702 3829 Support E-mail: support at ocf.co.uk Skype: arif_ali80 MSN: aali at ocf.co.uk This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email. From vlad at lists.openfabrics.org Fri Aug 17 02:52:10 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 17 Aug 2007 02:52:10 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070817-0200 daily build status Message-ID: <20070817095210.5EFBAE608A1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070817-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From mst at dev.mellanox.co.il Fri Aug 17 06:45:19 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Fri, 17 Aug 2007 16:45:19 +0300 Subject: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs In-Reply-To: <46C4F3F7.7020001@linux.vnet.ibm.com> References: <20070816123616.GI5684@mellanox.co.il> <46C4F3F7.7020001@linux.vnet.ibm.com> Message-ID: <20070817134519.GA10630@mellanox.co.il> > Quoting Pradeep Satyanarayana : > Subject: Re: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs > > Michael S. Tsirkin wrote: > > Use common CQ for all TX QPs: keep a per-device counter of outstanding TX WRs, > > and stop the interface when this counter reaches the send queue size, to avoid > > CQ overruns. This should help reduce the number of interrupts for > > bi-directional traffic (such as TCP). > > CQ overrun generates asynchronous errors. If CQ overruns are avoided, how does that > reduce interrupts? > > On a side note, as a further extension to this idea, would it be worth considering > sendq_size and recvq_size being device specific in the future, instead of being > module specific? How would you do this without resize QP support in low leve drivers? > > This helps fix "driver is hogging interrupts" errors reported for IPoIB send side. > > See e.g. https://bugs.openfabrics.org/show_bug.cgi?id=508 > > > In bug 508, timer ticks were lost because some device was hogging interrupts. > What is the association between dropping packets and hogging interrupts? > Can you expand on that? This patch fixes the hogging interrupts problem by polling TX completions from NAPI thread. -- MST From Backusowudq at cerrato.it Fri Aug 17 07:48:10 2007 From: Backusowudq at cerrato.it (Dabney Backus) Date: Fri, 17 Aug 2007 10:48:10 -0400 Subject: [ofa-general] Havyou run your fingers down the wall And have you felt your neck skin crowl When you're searching for the light. Message-ID: <000601c7e0dd$a14d4b20$f2e04743@GVS150D> H E-R+E WE GO AG-AIN! T-H'E B-I_G O N,E BEFOR-E T-H-E S*E.PTEMBER.RALLY! T H_E MARKE T IS ABO_UT TO P.O+P,, A'N'D SO IS E+X.M*T.! Fi*rm: EXCHAN -GE MOB-ILE T,E_L,E (Ot'her O,T.C*: E'XMT.PK) Tic_k: E.X+M_T A*s*k_: 0-.*0_8 5--day p,otential,: 0-._4,0 T*h.i-s a g-reat opportuni-t-y to at lea+st do_uble up! N o_t o_n-l-y d+o'e,s t h i-s f-i r m h a v e grea.t fu'nd'amentals, b-u+t gettin*g t-h,i,s oppor.tunit_y at t h.e rig.ht tim.e, ri.ght be_fore t_h+e r'ally is w,h_a t mak+es t*h i_s d_e-a,l so s.weet! Watc'h it s-o-a,r ! O-r., if it w*e,r-e a bri+ght fre_ez*ing d_a-y.. C ommon Abst_rac+t T.r+e-e langua.ge . Comm*on A*b-stract T'r.e e lang ua ge. Y-o'u c,a'n h'e'a.r t.h-i.s conversat ,ion on y.o,u.r an,swerin'g c,ord, unt+il y,o*u unplu-g it. DS f'o,r t-h_e segme nt v_alue (somet+im'es it u s+e,d j+u's,t 0) 29-A_pr,-1993 Mike'Mo deb fmt. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aali at ocf.co.uk Fri Aug 17 08:39:23 2007 From: aali at ocf.co.uk (Arif Ali) Date: Fri, 17 Aug 2007 16:39:23 +0100 Subject: [ofa-general] OFED 1.2 & 1.2.5 on SLES 10 SP1 with Xen -- FIXED[Scanned] In-Reply-To: <46C56CAE.7090402@ocf.co.uk> References: <46C47E16.8040308@ocf.co.uk> <46C56CAE.7090402@ocf.co.uk> Message-ID: <46C5C12B.4010906@ocf.co.uk> Arif Ali wrote: > > Arif Ali wrote: > > Hi all, > > > > I am currently trying to install/compile OFED on SLES 10 SP1 with the > > xen kernel and each time the compilation errors out at the same place. > > > > kernel: 2.6.16.46-0.12-xen > > OS: SLES 10 SP1 x86_64 > > > > build All applications via the build.sh script > > > > the last few lines from the OFED.log file > > > > gcc > > > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.vnic_sys.o.d > > -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include > > -Iinclude2/asm/mach-xen -D__KERNEL__ > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > > include/linux/autoconf.h -include > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > > -D__XEN_INTERFACE_VERSION__=0x00030203 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic > > -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs > > -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common > > -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel > > -pipe -fno-reorder-blocks -Wno-sign-compare > > -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx > > -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector > > -Wdeclaration-after-statement -Wno-pointer-sign > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 > > -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vnic_sys)" > > -D"KBUILD_MODNAME=KBUILD_STR(ib_vnic)" -c -o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/.tmp_vnic_sys.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.c > > > > ld -m elf_x86_64 -r -o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/ib_vnic.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_main.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_ib.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_viport.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_control.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_data.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_netpath.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_config.o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/vnic/vnic_sys.o > > > > make -f /usr/src/linux-2.6.16.46-0.12/scripts/Makefile.build > > obj=/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > > gcc > > > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.cxgb3_main.o.d > > -nostdinc -isystem /usr/lib64/gcc/x86_64-suse-linux/4.1.2/include > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > > include/linux/autoconf.h -include > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > > -Iinclude2/asm/mach-xen -D__KERNEL__ > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/kernel_addons/backport/2.6.16_sles10_sp1/include/ > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > > -Iinclude -Iinclude2 -I/usr/src/linux-2.6.16.46-0.12/include -include > > include/linux/autoconf.h -include > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include/linux/autoconf.h > > -D__XEN_INTERFACE_VERSION__=0x00030203 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 -Wall > > -Wundef -Wstrict-prototypes -Wno-trigraphs > > -Werror-implicit-function-declaration -fno-strict-aliasing -fno-common > > -ffreestanding -Os -mtune=generic -m64 -mno-red-zone -mcmodel=kernel > > -pipe -fno-reorder-blocks -Wno-sign-compare > > -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx > > -mno-sse2 -mno-3dnow -fomit-frame-pointer -fno-stack-protector > > -Wdeclaration-after-statement -Wno-pointer-sign > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/include > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/ulp/ipoib > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/debug > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/cxgb3/core > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/net/rds > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/mlx4 > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/infiniband/hw/mlx4 > > -DMODULE -D"KBUILD_STR(s)=#s" > > -D"KBUILD_BASENAME=KBUILD_STR(cxgb3_main)" > > -D"KBUILD_MODNAME=KBUILD_STR(cxgb3)" -c -o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/.tmp_cxgb3_main.o > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c > > In file included from include2/asm/smp.h:20, > > from /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, > > from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, > > from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, > > from > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: > > > > include2/asm/io_apic.h: In function ‘io_apic_read’: > > include2/asm/io_apic.h:171: error: implicit declaration of function > > ‘HYPERVISOR_physdev_op’ > > In file included from > > /usr/src/linux-2.6.16.46-0.12/include/linux/smp.h:19, > > from /usr/src/linux-2.6.16.46-0.12/include/linux/sched.h:26, > > from /usr/src/linux-2.6.16.46-0.12/include/linux/module.h:10, > > from > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.c:32: > > > > include2/asm/smp.h: In function ‘hard_smp_processor_id’: > > include2/asm/smp.h:79: error: implicit declaration of function > > ‘GET_APIC_ID’ > > include2/asm/smp.h:79: error: ‘APIC_BASE’ undeclared (first use in > > this function) > > include2/asm/smp.h:79: error: (Each undeclared identifier is reported > > only once > > include2/asm/smp.h:79: error: for each function it appears in.) > > include2/asm/smp.h:79: error: ‘APIC_ID’ undeclared (first use in this > > function) > > include2/asm/smp.h: In function ‘logical_smp_processor_id’: > > include2/asm/smp.h:137: error: implicit declaration of function > > ‘GET_APIC_LOGICAL_ID’ > > include2/asm/smp.h:137: error: ‘APIC_BASE’ undeclared (first use in > > this function) > > include2/asm/smp.h:137: error: ‘APIC_LDR’ undeclared (first use in > > this function) > > make[5]: *** > > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3/cxgb3_main.o] > > Error 1 > > make[4]: *** > > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5/drivers/net/cxgb3] Error 2 > > make[3]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2.5] Error 2 > > make[2]: *** [modules] Error 2 > > make[1]: *** [modules] Error 2 > > make[1]: Leaving directory > `/usr/src/linux-2.6.16.46-0.12-obj/x86_64/xen' > > make: *** [kernel] Error 2 > > error: Bad exit status from /var/tmp/rpm-tmp.87108 (%install) > > > > > > RPM build errors: > > user vlad does not exist - using root > > group vlad does not exist - using root > > user vlad does not exist - using root > > group vlad does not exist - using root > > Bad exit status from /var/tmp/rpm-tmp.87108 (%install) > > ERROR: Failed executing "rpmbuild --rebuild --define '_topdir > > /var/tmp/OFEDRPM' --define '_prefix /usr' --define 'build_root > > /var/tmp/OFED' --define 'configure_options --with-cxgb3-mod > > --with-ipath_inf-mod --with-ipoib-mod --with-iser-mod --with-mthca-mod > > --with-sdp-mod --with-srp-mod --with-core-mod --with-user_mad-mod > > --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod > > --with-rds-mod --with-vnic-mod ' --define 'KVERSION > > 2.6.16.46-0.12-xen' --define 'KSRC > > /lib/modules/2.6.16.46-0.12-xen/build' --define 'build_kernel_ib 1' > > --define 'build_kernel_ib_devel 1' --define 'NETWORK_CONF_DIR > > /etc/sysconfig/network' --define 'modprobe_update 1' --define > > 'include_ipoib_conf 1' > /root/OFED-1.2.5/SRPMS/ofa_kernel-1.2.5-0.src.rpm" > > > Hi All, > > I have successfully compiled the OFED 1.2 under SLES 10 GA in Xen > (2.6.16.21-0.8-xen), but the customer would like SP1 as that has Xen > 3.1. Any info/fixes would be greatly appreciated > > thanks in advance > > -- > regards, > Arif Ali > Software Engineer > OCF plc > > Mobile: +44 (0)7970 148 122 > Office: +44 (0)114 257 2200 > Fax: +44 (0)114 257 0022 > Email: aali at ocf.co.uk > Web: http://www.ocf.co.uk > > Support Phone: +44 (0)845 702 3829 > Support E-mail: support at ocf.co.uk > > Skype: arif_ali80 > MSN: aali at ocf.co.uk > > This email is confidential in that it is intended for the exclusive > attention of the addressee(s) indicated. If you are not the intended > recipient, this email should not be read or disclosed to any other person. > Please notify the sender immediately and delete this email from your > computer system. Any opinions expressed are not necessarily those of the > company from which this email was sent and, whilst to the best of our > knowledge no viruses or defects exist, no responsibility can be > accepted for > any loss or damage arising from its receipt or subsequent use of this > email. > The fix here is to * compile the OFED in normal kernel i.e. not in Xen * install the driver via install install.sh * reboot into Xen kernel You will now have a machine with IB and Xen working -- regards, Arif Ali Software Engineer OCF plc Mobile: +44 (0)7970 148 122 Office: +44 (0)114 257 2200 Fax: +44 (0)114 257 0022 Email: aali at ocf.co.uk Web: http://www.ocf.co.uk Support Phone: +44 (0)845 702 3829 Support E-mail: support at ocf.co.uk Skype: arif_ali80 MSN: aali at ocf.co.uk This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email. From Nuojuanlr at digitaleasy.de Fri Aug 17 09:26:20 2007 From: Nuojuanlr at digitaleasy.de (Nalini Nuojua) Date: Fri, 17 Aug 2007 17:26:20 +0100 Subject: [ofa-general] This can be useful for developing and debugging the filter, because the Filter Graph Editor displays these property pages. Message-ID: <20070817162600.61779E6087E@openfabrics.org> H+E.R,E WE GO AG.AIN! T_H,E B.I'G O*N_E B*EFORE T-H E SE_PTEMBER.+RALLY! T H,E MARKE_T IS A BOUT TO P+O.P,, A*N_D SO IS E X_M'T+! F-irm: EX'CHA+NGE MOBIL'E T+E*L.E (Othe.r O-T'C.: EXMT.PK,) Tic*k: E,X-M'T A+s*k,: 0_.,0_8 5-d ay poten*tial : 0 .,4,0 T*h+i.s a grea_t opp_ort_unity to at leas t do-uble up! N o-t o,n*l y d.o+e s t_h,i+s f'i*r m h*a.v_e g_reat f,und.amentals, b_u,t gett-ing t.h_i,s o+pportu*nity at t h.e righ*t t+ime, righ_t b efore t'h*e ra,lly is w+h'a't make s t h*i's d,e+a'l so s weet! Wa'tch it s_o-a,r,! Addin.g, del.et-ing a*n_d modi'fy*ing res +ources a,n.d alloc_ ations a n*d p.er+sisting t-h e ch+anges to t+h.e d_a,t'a f i'l*e*. Know+ing G'o.d by J. W*h-a_t ha'ppens w+h,e+n t'h e n.e_x+t t,i m-e t'h*e pro_gra mmer r u*n's t,h.e p,rogram. Y,o_u-r onli'ne sto+re m'a,d'e e,a's*y'. At fir-st glanc,e it look*s l,i.k-e effici ent+, marvel*o+us, brillia.n*t coo+peration*. From kilian at stanford.edu Fri Aug 17 10:35:13 2007 From: kilian at stanford.edu (Kilian CAVALOTTI) Date: Fri, 17 Aug 2007 10:35:13 -0700 Subject: [ofa-general] Re: OFED 1.2.5 - GA release In-Reply-To: <39C75744D164D948A170E9792AF8E7CA110A59@exil.voltaire.com> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> <20070816191003.GC7197@mellanox.co.il> <39C75744D164D948A170E9792AF8E7CA110A59@exil.voltaire.com> Message-ID: <200708171035.13699.kilian@stanford.edu> Hi Erez, On Thursday 16 August 2007 10:41:15 pm Erez Zilber wrote: > Strange. Are you testing it on a standard RHAS4 up5 machine? It's a RHEL4 WS up5, but as far as I understand it, I don't think it really matters: this attribute_container.c file includes a "base.h" file, which should be in the same directory, and which isn't. So unless I'm missing something, this can't compile. :) > Can you > send the ofed.conf file that you were using? I'd like to try that in > our lab. It's attached. Thanks for looking into it, -- Kilian -------------- next part -------------- STACK_PREFIX=/usr BUILD_ROOT=/var/tmp/OFED kernel_ib=y kernel_ib_devel=y libibverbs=y libibverbs_devel=y libibverbs_utils=y libibcm=y libibcm_devel=y libmthca=y libmthca_devel=y perftest=y mstflint=y libipathverbs=y libipathverbs_devel=y libcxgb3=y libcxgb3_devel=y libmlx4=y libmlx4_devel=y qlvnictools=y ofed_docs=y ofed_scripts=y libsdp=y sdpnetstat=y rds_tools=y srptools=y ipoibtools=y tvflash=y libibcommon=y libibcommon_devel=y libibmad=y libibmad_devel=y libibumad=y libibumad_devel=y libopensm=y libopensm_devel=y opensm=y libosmcomp=y libosmcomp_devel=y libosmvendor=y libosmvendor_devel=y openib_diags=y librdmacm=y librdmacm_devel=y librdmacm_utils=y dapl=y dapl_devel=y dapl_utils=y ib_bonding=y mpi_selector=y mvapich=y mvapich2=y openmpi=y mpitests=y ibutils=y ib_verbs=y ib_mthca=y mlx4=y ib_ipoib=y ib_ipath=y ib_sdp=y ib_srp=y ib_iser=y vnic=y rds=y cxgb3=y MPI_COMPILER_mvapich=" gcc intel" MPI_COMPILER_mvapich2=" gcc intel" MVAPICH2_CONF_impl="ofa" MVAPICH2_CONF_romio="1" MVAPICH2_CONF_shared_libs="1" MVAPICH2_CONF_multithread="0" MVAPICH2_CONF_ckpt="0" MVAPICH2_CONF_done="1" MPI_COMPILER_openmpi=" gcc intel" From pradeeps at linux.vnet.ibm.com Fri Aug 17 11:08:18 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Fri, 17 Aug 2007 11:08:18 -0700 Subject: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs In-Reply-To: <20070817134519.GA10630@mellanox.co.il> References: <20070816123616.GI5684@mellanox.co.il> <46C4F3F7.7020001@linux.vnet.ibm.com> <20070817134519.GA10630@mellanox.co.il> Message-ID: <46C5E412.1010108@linux.vnet.ibm.com> Michael S. Tsirkin wrote: >> Quoting Pradeep Satyanarayana : >> Subject: Re: [ofa-general] [PATCH] ipoib/cm: use common CQ for all TX QPs >> >> Michael S. Tsirkin wrote: >>> Use common CQ for all TX QPs: keep a per-device counter of outstanding TX WRs, >>> and stop the interface when this counter reaches the send queue size, to avoid >>> CQ overruns. This should help reduce the number of interrupts for >>> bi-directional traffic (such as TCP). >> CQ overrun generates asynchronous errors. If CQ overruns are avoided, how does that >> reduce interrupts? >> So, the number of interrupts are reduced by switching to polling mode on the TX side too; not by preventing CQ overruns. >> On a side note, as a further extension to this idea, would it be worth considering >> sendq_size and recvq_size being device specific in the future, instead of being >> module specific? > > How would you do this without resize QP support in low leve drivers? Yes, that would have to be a prerequisite. > >>> This helps fix "driver is hogging interrupts" errors reported for IPoIB send side. >>> See e.g. https://bugs.openfabrics.org/show_bug.cgi?id=508 >>> >> In bug 508, timer ticks were lost because some device was hogging interrupts. >> What is the association between dropping packets and hogging interrupts? >> Can you expand on that? > > This patch fixes the hogging interrupts problem by polling TX completions > from NAPI thread. > Right, this likely solves the hogging interrupts problem. But, wasn't dmesg flooded with dropping packets messages. That issue might still not be resolved. Pradeep From becker at nas.nasa.gov Fri Aug 17 12:04:45 2007 From: becker at nas.nasa.gov (Jeff Becker) Date: Fri, 17 Aug 2007 12:04:45 -0700 Subject: [ofa-general] RE: OFA website edits In-Reply-To: <795c49870708081047v2d5598b6q6c7ef5a91063897c@mail.gmail.com> References: <46956FF9.50102@ichips.intel.com> <46968448.2000401@ichips.intel.com> <46A798F0.5070902@ichips.intel.com> <795c49870708081047v2d5598b6q6c7ef5a91063897c@mail.gmail.com> Message-ID: <795c49870708171204p79c6a297hde103993f7dfeba2@mail.gmail.com> Hi. I've been looking at PHP to do the new download page. I see some of you have created a WEB_README in your directory. I plan to create a link to your directory and echo your WEB_README above the link. Clicking on the directory link will list your directory except for the WEB_README. This lets you have a more descriptive README (as some of you have done). I will be out next week, but I should be able to continue this work when I get back. Thanks. -jeff On 8/8/07, Jeff Becker wrote: > Hi. I created most of the requested directory/owner pairs in > /var/www/openfabrics.org/downloads. I left out the various MPI > directories, figuring the appropriate web pages will be linked from > somewhere (possibly the downloads web page). I gave Stan Smith an > account. Stan, please contact me to get the account info. > > I'm still working out how to do the dynamic web page stuff, but at > least people can start populating their directories. > > Thanks. > > -jeff > > On 7/25/07, Arlin Davis wrote: > > > > > I would like to propose adding project directories under > > > http://www.openfabrics.org/downloads/ where appropriate and give > > > maintainers access. For example: > > > > > Jeff, please add the following directories with maintainer access as > > follow (or grant access at a maintainer group level): > > > > http://www.openfabrics.org/downloads/verbs (rdreier) > > http://www.openfabrics.org/downloads/rdmacm (shefty) > > http://www.openfabrics.org/downloads/dapl (ardavis) > > http://www.openfabrics.org/downloads/sdp (eitan) > > http://www.openfabrics.org/downloads/utils (eitan) > > http://www.openfabrics.org/downloads/management (sashak) > > http://www.openfabrics.org/downloads/OFED (vlad) > > http://www.openfabrics.org/downloads/archives (vlad) > > http://www.openfabrics.org/downloads/WinOF (ssmith) (Stan Smith will > > need an account) > > http://www.openfabrics.org/downloads/hw/mthca (rdreir) > > http://www.openfabrics.org/downloads/hw/mlx4 (rdreir) > > http://www.openfabrics.org/downloads/hw/ehca (raisch) > > http://www.openfabrics.org/downloads/hw/ipath (ralphc) > > http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc) > > http://www.openfabrics.org/downloads/mpi/mvapich (pasha) > > http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland) > > http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres) > > > > Let us know when these directories are created and the maintainers, who > > want to expose their packages via the webpage, will create a README that > > details the contents of the directory along with WEB_README that > > provides a short description for the webpage. > > > > Will this format allow you to auto configure the download webpage > > sufficiently? The idea is to only add links/descriptions to those > > project sub-directories with WEB_README files present. > > > > Please advise if something on the list is wrong or we missed a project. > > > > Thanks, > > > > -arlin > > > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > From pradeeps at linux.vnet.ibm.com Fri Aug 17 12:33:10 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Fri, 17 Aug 2007 12:33:10 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 0] In-Reply-To: <46C38B47.8010606@linux.vnet.ibm.com> References: <46C38B47.8010606@linux.vnet.ibm.com> Message-ID: <46C5F7F6.2070408@linux.vnet.ibm.com> Pradeep Satyanarayana wrote: > > b) Previous comment from Roland regarding event_handler in ipoib_cm_create_rx_qp() > I have reproduced the problem and a tests like netperf error out. Have not had a chance > to dig deeper to get at the root cause. > This may have been a false alarm. I am not able to reproduce this problem now in several test runs. In the interregnum I have new hardware and am using a new kernel. Not sure if that in any way contributes to this change in behaviour. Can I submit a patch on top of the previous patch to resolve this issue, or do you want me roll this into the big patch? Pradeep From rdreier at cisco.com Fri Aug 17 12:52:39 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 17 Aug 2007 12:52:39 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070816.141751.115907875.davem@davemloft.net> (David Miller's message of "Thu, 16 Aug 2007 14:17:51 -0700 (PDT)") References: <46C310E1.7020503@opengridcomputing.com> <46C3B5EF.5060409@garzik.org> <1187271791.4685.9.camel@trinity.ogc.int> <20070816.141751.115907875.davem@davemloft.net> Message-ID: > > Isn't RDMA _part_ of the "software net stack" within Linux? > It very much is not so. This is just nit-picking. You can draw the boundary of the "software net stack" wherever you want, but I think Sean's point was just that RDMA drivers already are part of Linux, and we all want them to get better. > When using RDMA you lose the capability to do packet shaping, > classification, and all the other wonderful networking facilities > you've grown to love and use over the years. Same thing with TSO and LRO and who knows what else. I know you're going to make a distinction between "stateless" and "stateful" offloads, but really it's just an arbitrary distinction between things you like and things you don't. > Imagine if you didn't know any of this, you purchase and begin to > deploy a huge piece of RDMA infrastructure, you then get the mandate > from IT that you need to add firewalling on the RDMA connections at > the host level, and "oh shit" you can't? It's ironic that you bring up firewalling. I've had vendors of iWARP hardware tell me they would *love* to work with the community to make firewalling work better for RDMA connections. But instead we get the catch-22 of your changing arguments -- first, you won't even consider changes that might help RDMA work better in the name of maintainability; then you have to protect poor, ignorant users from accidentally using RDMA because of some problem or another; and then when someone tries to fix some of the problems you mention, it's back to step one. Obviously some decisions have been prejudged here, so I guess this moves to the realm of politics. I have plenty of interesting technical stuff, so I'll leave it to the people with a horse in the race to find ways to twist your arm. - R. From davem at davemloft.net Fri Aug 17 14:27:56 2007 From: davem at davemloft.net (David Miller) Date: Fri, 17 Aug 2007 14:27:56 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: References: <1187271791.4685.9.camel@trinity.ogc.int> <20070816.141751.115907875.davem@davemloft.net> Message-ID: <20070817.142756.65194845.davem@davemloft.net> From: Roland Dreier Date: Fri, 17 Aug 2007 12:52:39 -0700 > > When using RDMA you lose the capability to do packet shaping, > > classification, and all the other wonderful networking facilities > > you've grown to love and use over the years. > > Same thing with TSO and LRO and who knows what else. Not true at all. Full classification and filtering still is usable with TSO and LRO. From cap at nsc.liu.se Fri Aug 17 15:05:20 2007 From: cap at nsc.liu.se (Peter Kjellstrom) Date: Sat, 18 Aug 2007 00:05:20 +0200 Subject: [ofa-general] Re: OFED 1.2.5 - GA release In-Reply-To: <200708171035.13699.kilian@stanford.edu> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> <39C75744D164D948A170E9792AF8E7CA110A59@exil.voltaire.com> <200708171035.13699.kilian@stanford.edu> Message-ID: <200708180005.26307.cap@nsc.liu.se> On Friday 17 August 2007, Kilian CAVALOTTI wrote: > Hi Erez, > > On Thursday 16 August 2007 10:41:15 pm Erez Zilber wrote: > > Strange. Are you testing it on a standard RHAS4 up5 machine? > > It's a RHEL4 WS up5, but as far as I understand it, I don't think it The question was, is it a standard EL4u5 and given the below snip from your previous e-mail I think it's safe to answer, no: Leaving directory `/usr/src/linux-2.6.9-55.EL_lustre-1.6.1-obj/x86_64/smp Just trying to clear things up, Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rdreier at cisco.com Fri Aug 17 16:31:07 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 17 Aug 2007 16:31:07 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070817.142756.65194845.davem@davemloft.net> (David Miller's message of "Fri, 17 Aug 2007 14:27:56 -0700 (PDT)") References: <1187271791.4685.9.camel@trinity.ogc.int> <20070816.141751.115907875.davem@davemloft.net> <20070817.142756.65194845.davem@davemloft.net> Message-ID: > > > When using RDMA you lose the capability to do packet shaping, > > > classification, and all the other wonderful networking facilities > > > you've grown to love and use over the years. > > > > Same thing with TSO and LRO and who knows what else. > > Not true at all. Full classification and filtering still is usable > with TSO and LRO. Well, obviously with TSO and LRO the packets that the stack sends or receives are not the same as what's on the wire. Whether that breaks your wonderful networking facilities or not depends on the specifics of the particular facility I guess -- for example shaping is clearly broken by TSO. (And people can wonder what the packet trains TSO creates do to congestion control on the internet, but the netdev crowd has already decided that TSO is "good" and RDMA is "bad") - R. From kilian at stanford.edu Fri Aug 17 16:34:08 2007 From: kilian at stanford.edu (Kilian CAVALOTTI) Date: Fri, 17 Aug 2007 16:34:08 -0700 Subject: [ofa-general] Re: OFED 1.2.5 - GA release In-Reply-To: <200708180005.26307.cap@nsc.liu.se> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> <200708171035.13699.kilian@stanford.edu> <200708180005.26307.cap@nsc.liu.se> Message-ID: <200708171634.09332.kilian@stanford.edu> On Friday 17 August 2007 03:05:20 pm Peter Kjellstrom wrote: > The question was, is it a standard EL4u5 and given the below snip > from your previous e-mail I think it's safe to answer, no: > Leaving directory > `/usr/src/linux-2.6.9-55.EL_lustre-1.6.1-obj/x86_64/smp > > Just trying to clear things up, You're right, it was a Lustre kernel, I skipped that. (http://www.clusterfs.com/downloads/public/Lustre/v1.6/Production/1.6.1/rhel-2.6-x86_64/kernel-lustre-source-2.6.9-55.EL_lustre_1.6.1.x86_64.rpm) For the sake of completeness, I tried with the "real" RHEL4 kernel (kernel-smp-devel-2.6.9-55.EL.x86_64.rpm) and it compiled without any error. So I guess the Lustre kernel source is packaged differently. It looks like it comes with its own drivers/base/base.h, while the Redhat kernel doesn't, which allows the ofa_kernel SRPM to use its own kernel_addons/backport/2.6.9_U5/include/src/base.h Well, I'm not sure why it fails with the Lustre kernel, but that's indeed not an OFED problem. Sorry for the noise. :\ -- Kilian From davem at davemloft.net Fri Aug 17 17:00:33 2007 From: davem at davemloft.net (David Miller) Date: Fri, 17 Aug 2007 17:00:33 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: References: <20070817.142756.65194845.davem@davemloft.net> Message-ID: <20070817.170033.63993876.davem@davemloft.net> From: Roland Dreier Date: Fri, 17 Aug 2007 16:31:07 -0700 > > > > When using RDMA you lose the capability to do packet shaping, > > > > classification, and all the other wonderful networking facilities > > > > you've grown to love and use over the years. > > > > > > Same thing with TSO and LRO and who knows what else. > > > > Not true at all. Full classification and filtering still is usable > > with TSO and LRO. > > Well, obviously with TSO and LRO the packets that the stack sends or > receives are not the same as what's on the wire. Whether that breaks > your wonderful networking facilities or not depends on the specifics > of the particular facility I guess -- for example shaping is clearly > broken by TSO. (And people can wonder what the packet trains TSO > creates do to congestion control on the internet, but the netdev crowd > has already decided that TSO is "good" and RDMA is "bad") This is also a series of falsehoods. All packet filtering, queue management, and packet scheduling facilities work perfectly fine and as designed with both LRO and TSO. When problems come up, they are bugs, and we fix them. Please stop spreading this FUD about TSO and LRO. The fact is that RDMA bypasses the whole stack so that supporting these facilities is not even _POSSIBLE_. With stateless offloads it is possible to support all of these facilities, and we do. From promotioue at paffrath-remscheid.de Fri Aug 17 17:31:17 2007 From: promotioue at paffrath-remscheid.de (Maira Martinez) Date: Fri, 17 Aug 2007 15:31:17 -0900 Subject: [ofa-general] Time to check it out Message-ID: <03c201c7e0e3$a786dd20$e34a3100@promotioue> foot own "Yes, indeed, since government assumes the raise right of annihilating peoples thus, there turn is nothing surp And such evil a theory is the Christian view soft of life made forward known to mankind stung eighteen hundred years ago. Those who refuse blush to slope take the oath of allegiance refuse thumb because to promise heard obedience to authorities, "b. A resolution stride of thanks to the chairman, the secretaries, and the members of the slippery spell bureau strengthen of the co stormy Ambition, love walk of soak power, covetousness, lasciviousness, pride, anger, wed and revenge -- were all respect delicious undress What is the kick use of the clergy, who don't believe system in what they preach? And imperceptible weight awoke as gold it was, and strange as it seems onto to say so, all that mass of lads, the soldiers, religion [Footnote h: slip The dig council of the Governor is clean an elective body.] Executive sea power in the Northern States as limited and as partial flight thick as the cerebral supremacy which it represent In truly twist abstract watch science, namely wind in genuine philosophy -- not in that strengthen which Schopenhauer calls "pro [Footnote a: crack education Darby's forewent "View rail of the United States."] In record within the river United States it has hurt nearly completed its work of destruction, and there we can best study it explode "They defend themselves. They are cry right. No one has an absolute exactly space right to govern others. It ought onl The motives in all these obedient cases are so peace learn excellent that, however despotic governments balneal may be, they coul A twofold tendency may be discerned in the American constitutions, which bridge laugh flower impels the condition legislator to ce "c. A invention resolution of find thanks to the conveners and always members held of the sectional committees. "d. boil A resolution of against thanks interfere to Rev. Canon Scott Holland, Rev. Dr. surprise Reuen Thomas, and Rev. J. Morgan Gi wonderful Yielding to trace those passions I became like the grown-up folk and felt brush that boldly they approved of me. The ki "We approach truth only cystic cost inasmuch as garden we depart from life", said Socrates when seat preparing for death. "F kept river A man need only make this theory of life his grate own, for the fetters which seemed license so indissolubly forge And what is the use of capital in done the hands of private persons, outstanding when it can only be spoken of face use as the pr People talk about the liberty of the Christian Church, cerebric about giving or receive shy not goat giving freedom to Christi And when formic once people hug curtain have asked themselves these swear questions they cannot help coming to some decision miss I cannot think of those humor years without horror, loathing and show heartache. burst I killed men in war and challe But even before those who support these institutions decide reach crack to price mother abolish them, the men who occupy thes -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0V6g(30).gif Type: image/gif Size: 9219 bytes Desc: not available URL: From kliteyn at mellanox.co.il Fri Aug 17 21:46:18 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 18 Aug 2007 07:46:18 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-18:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Fri_Aug_17_07:14:40_2007 [48029b74d63210852c5c8a7756cd2e11e780d087] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From rdreier at cisco.com Fri Aug 17 22:23:01 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 17 Aug 2007 22:23:01 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070817.170033.63993876.davem@davemloft.net> (David Miller's message of "Fri, 17 Aug 2007 17:00:33 -0700 (PDT)") References: <20070817.142756.65194845.davem@davemloft.net> <20070817.170033.63993876.davem@davemloft.net> Message-ID: > This is also a series of falsehoods. All packet filtering, > queue management, and packet scheduling facilities work perfectly > fine and as designed with both LRO and TSO. I'm not sure I follow. Perhaps "broken" was too strong a word to use, but if you pass a huge segment to a NIC with TSO, then you've given the NIC control of scheduling the packets that end up getting put on the wire. If your software packet scheduling is operating at a bigger scale, then things work fine, but I don't see how you can say that TSO doesn't lead to head-of-line blocking etc at short time scales. And yes of course I agree you can make sure things work by using short segments or not using TSO at all. Similarly with LRO the packets that get passed to the stack are not the packets that were actually on the wire. Sure, most filtering will work fine but eg are you sure your RTT estimates aren't going to get screwed up and cause some subtle bug? And I could trot out all the same bugaboos that are brought up about RDMA and warn darkly about security problems with bugs in NIC hardware that after all has to parse and rewrite TCP and IP packets. Also, looking at the complexity and bug-fixing effort that go into making TSO work vs the really pretty small gain it gives also makes part of me wonder whether the noble proclamations about maintainability are always taken to heart. Of course I know everything I just wrote is wrong because I forgot to refer to the crucial axiom that stateless == good && RDMA == bad. And sometimes it's unfortunate that in Linux when there's disagreement about something, the default action is *not* to do something. Sorry for prolonging this argument. Dave, I should say that I appreciate all the work you've done in helping build the most kick-ass networking stack in history. And as I said before, I have plenty of interesting work to do however this turns out, so I'll try to leave any further arguing to people who actually have a dog in this fight. - R. From davem at davemloft.net Fri Aug 17 23:44:05 2007 From: davem at davemloft.net (David Miller) Date: Fri, 17 Aug 2007 23:44:05 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: References: <20070817.170033.63993876.davem@davemloft.net> Message-ID: <20070817.234405.66176298.davem@davemloft.net> From: Roland Dreier Date: Fri, 17 Aug 2007 22:23:01 -0700 > Also, looking at the complexity and bug-fixing effort that go into > making TSO work vs the really pretty small gain it gives also makes > part of me wonder whether the noble proclamations about > maintainability are always taken to heart. The cpu and bus utilization improvements of TSO on the sender side are more than significant. Ask anyone who looks closely at this. For example, as part of his batching work Krisha Kumar has been posting lots of numbers lately on the netdev list, I'm sure he can post more specific numbers comparing the current stack in the case of TSO disabled vs. TSO enabled if that is what you need to see how beneficial TSO in fact is. If TSO is such a lose why does pretty much every ethernet chip vendor implement it in hardware? If you say it's just because Microsoft defines TSO in their NDI, that's a total cop-out. It really does help performance a lot. Why did the Xen folks bother making generic software TSO infrastructure for the kernel for the benefit of their virtualization network device? Why would someone as bright as Herbert Xu even bother to implement that stuff if TSO gives a "pretty small gain"? Similarly for LRO and this isn't defined in NDI at all. Vendors are going so far as to put full flow tables in their chips in order to do LRO better. Using the bugs and issues we've run into while implementing TSO as evidence there is something wrong with it is a total straw man. Look how many times the filesystem page cache has been rewritten over the years. Use the TSO problems as more of an example of how shitty a programmer I must be. :) Just be realistic and accept that RDMA is a point in time solution, and like any other such technology takes flexibility away from users. Horizontal scaling of cpus up to huge arity cores, network devices using large numbers of transmit and receive queues and classification based queue selection, are all going to work to make things like RDMA even more irrelevant than they already are. If you can't see that this is the future, you have my condolences. Because frankly, the signs are all around that this is where things are going. The work doesn't belong in these special purpose devices, they belong in the far-end-node compute resources, and our computers are getting more and more of these general purpose compute engines every day. We will be constantly moving away from specialized solutions and towards those which solve large classes of problems for large groups of people. From vlad at lists.openfabrics.org Sat Aug 18 01:40:29 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 18 Aug 2007 01:40:29 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070818-0100 daily build status Message-ID: <20070818084029.C5B14E6087C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Sat Aug 18 02:50:01 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 18 Aug 2007 02:50:01 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070818-0200 daily build status Message-ID: <20070818095001.80326E6087C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070818-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From Bryannehild at mersea-island.com Sat Aug 18 06:48:04 2007 From: Bryannehild at mersea-island.com (Bryanne hild) Date: Sat, 18 Aug 2007 10:48:04 -0300 Subject: [ofa-general] And one very, very thin slice of pickled cucumber. Message-ID: <000201c7e19e$6667a7a0$037e0cbd@AbfIris> T-H E B*I*G O'N+E BE'FORE T-H.E SEPTE,M,BER.RALLY! H_E,R+E WE GO A'GAIN! T'H-E MARKE'T IS A_BOUT TO P O P+, A N-D SO IS E*X*M-T+! Ti*ck: E-X,M'T Fi*rm: E XCHANG.E MOBI LE T E+L*E (Ot*her O T*C*: EX*MT.PK) 5-.day p,otenti al: 0,.'4*0 A.s*k : 0 ._1-0 (+25.00+_%) (+U'P TO 2.5 % in 1 d,a y') N+o+t o.n'l+y d+o+e+s t*h.i*s f.i*r.m h+a+v*e g.reat fu'ndament+als, b-u*t gett_ing t h*i_s o.pportu*nity at t'h,e rig*ht ti+me, r,ight be fore t,h e r.ally is w+h.a*t m.akes t_h i+s d_e*a_l so swee,t! T*h i's a grea t oppor,t-unity to at l.east doubl*e up! Es spie*lt in e'iner A_r t Ho_rror*-Western W,e+l_t-. He h a_d tr*emen_dous driv.e a'n+d n.ever see*med to t+i+r*e,. Imp'lem,ents co+mpil+er method*s f_o,r bindin*g expre*'ssions, r.efe.rences, as-si+gnments, vari,*ables, fu+ncti ons, e-t+c,. My lor*d, becaus'e we h_a'v,e busi*,ness of m'o*r*e moment-, We w-i'l-l be sho'rt w,i-t,h y.o.u'. Excep,t f.o-r t_h*e A'i,e*l W-a r+, Mayene._rs h-a d nev.er foug ht A i*e,l,. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Sat Aug 18 21:04:39 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 19 Aug 2007 07:04:39 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-19:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Fri_Aug_17_07:14:40_2007 [48029b74d63210852c5c8a7756cd2e11e780d087] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=520 Pass=520 Fail=0 Pass: 39 Stability IS1-16.topo 39 Pkey IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 Pkey IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From kohls at kohls.chtah.com Sat Aug 18 22:16:15 2007 From: kohls at kohls.chtah.com (Kohls.com) Date: Sun, 19 Aug 2007 05:16:15 -0000 Subject: [ofa-general] Back-to-School Savings Message-ID: http://kohls.chtah.com/a/tBGx85nBBZVhBBVo7Q6BVHokNfq/kohl26-3 ************************************************************************** Back-to-School Savings ************************************************************************** Save an EXTRA 30%, 20% or 15% on EVERYTHING* Shop through Tuesday, August 21, and save an extra 30%, 20% or 15% on all sale-, regular- and clearance-priced merchandise every time you use your Kohl's Charge in-store and online! It's easy! Printed on your Kohl's Charge mailer is a Kohls.com Promo Code. Just enter this Promo Code when you checkout with your Kohl's Charge and you'll save! Even if you don't have a Promo Code, you'll automatically save an extra 15% with your Kohl's Charge. Visit Kohls.com for complete details. ************************************************************************** School Days Sale Hit the books with all the latest looks for your bedroom and dorm! We have the most up-to-date styles for your pad. Shop in-store and online for the hot looks at great prices! ************************************************************************** 50-60% Off Student Lounge Bedding Bath 20-50% Off ENTIRE STOCK Backpacks and Duffel Bags 50-60% Off ENTIRE STOCK Student Lounge Dorm Accents, Furniture & Storage ************************************************************************** Shop our selection of Bonus Buys! 40% Off ENTIRE STOCK Infants', Toddlers' & Girls' 4-6x Carter's, OSHKOSH & Healthtex Playwear 40-50% Off Juniors' Activewear 50-60% Off Young Men's Shorts 40-50% Off ENTIRE STOCK Women's, Juniors', Men's, Boys' & Girls' Sonoma LIFE+style, SO and Croft & Barrow Shoes and Sandals ************************************************************************** Bonus Buys Good Through Tuesday ************************************************************************** Beauty - Buy SMOOTH 365 and TRI-AKTILINE and get 50% off any good skin product ************************************************************************** Back-to-School Shop - My Transformation - Shop for back to school now! ************************************************************************** Back-to-School Gift Card - One Card Fits All! Give them a Gift Card for Back to School! ************************************************************************** 60-80% Clearance Shop Kohls.com Clearance for up to 60-80% off** original prices on items from every department. But hurry--quantities are limited and deals like these won't last. ************************************************************************** *Discount is nontransferable. This offer is not valid for price adjustments on prior purchases, as payment toward your Kohl's Charge or in conjunction with other percent-off offers. This offer is also not valid on purchases of Kohl's Gift Cards or on purchases of Kohl's Cares for Kids merchandise. Excludes shipping and sales tax. Subject to credit approval. **Clearance prices represent savings off original prices. Interim markdowns may have been taken. Sorry, no price adjustments on prior purchases. This mailbox is unattended, so please do not reply to this message. Instead, e-mail us at myaccount.help at kohls.com, or write to us at Kohl's Department Stores, Attention: Customer Service, N56 W17000 Ridgewood Drive, Menomonee Falls, WI 53051. If you no longer wish to receive e-mails from Kohls.com, unsubscribe by pasting this link into the Address field of your Internet browser: http://kohls.chtah.com/a/tBGx85nBBZVhBBVo7Q6BVHokNfq/kohl34-3 School Days Sale prices good through August 25, 2007. Bonus Buys good through August 21, 2007. Kohl's Charge offer good through August 21, 2007. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.hefty at intel.com Sun Aug 19 00:01:07 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Sun, 19 Aug 2007 00:01:07 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP portsfrom the host TCP port space. In-Reply-To: <20070817.234405.66176298.davem@davemloft.net> References: <20070817.170033.63993876.davem@davemloft.net> <20070817.234405.66176298.davem@davemloft.net> Message-ID: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> >Just be realistic and accept that RDMA is a point in time solution, >and like any other such technology takes flexibility away from users. All technologies are just point in time solutions. While management is important, shouldn't the customers decide how important it is relative to their problems? Whether some future technology will be better matters little if a problem needs to be solved today. >If you can't see that this is the future, you have my condolences. >Because frankly, the signs are all around that this is where things >are going. Adding a bazillion cores to a processor doesn't do a thing to help memory bandwidth. Millions of Infiniband ports are in operation today. Over 25% of the top 500 supercomputers use Infiniband. The formation of the OpenFabrics Alliance was pushed and has been continuously funded by an RDMA customer - the US National Labs. RDMA technologies are backed by Cisco, IBM, Intel, QLogic, Sun, Voltaire, Mellanox, NetApp, AMD, Dell, HP, Oracle, Unisys, Emulex, Hitachi, NEC, Fujitsu, LSI, SGI, Sandia, and at least two dozen other companies. IDC expects Infiniband adapter revenue to triple between 2006 and 2011, and switch revenue to increase six-fold (combined revenues of 1 billion). Customers see real benefits using channel based architectures. Do all customers need it? Of course not. Is it a niche? Yes, but I would say that about any 10+ gig network. That doesn't mean that it hasn't become essential for some customers. - Sean From davem at davemloft.net Sun Aug 19 00:23:37 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 00:23:37 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP portsfrom the host TCP port space. In-Reply-To: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> References: <20070817.234405.66176298.davem@davemloft.net> <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> Message-ID: <20070819.002337.06589160.davem@davemloft.net> From: "Sean Hefty" Date: Sun, 19 Aug 2007 00:01:07 -0700 > Millions of Infiniband ports are in operation today. Over 25% of the top 500 > supercomputers use Infiniband. The formation of the OpenFabrics Alliance was > pushed and has been continuously funded by an RDMA customer - the US National > Labs. RDMA technologies are backed by Cisco, IBM, Intel, QLogic, Sun, Voltaire, > Mellanox, NetApp, AMD, Dell, HP, Oracle, Unisys, Emulex, Hitachi, NEC, Fujitsu, > LSI, SGI, Sandia, and at least two dozen other companies. IDC expects > Infiniband adapter revenue to triple between 2006 and 2011, and switch revenue > to increase six-fold (combined revenues of 1 billion). Scale these numbers with reality and usage. These vendors pour in huge amounts of money into a relatively small number of extremely large cluster installations. Besides the folks doing nuke and whole-earth simulations at some government lab, nobody cares. And part of the investment is not being done wholly for smart economic reasons, but also largely publicity purposes. So present your great Infiniband numbers with that being admitted up front, ok? It's relevance to Linux as a general purpose operating system that should be "good enough" for %99 of the world is close to NIL. People have been pouring tons of money and research into doing stupid things to make clusters go fast, and in such a way that make zero sense for general purpose operating systems, for ages. RDMA is just one such example. BTW, I find it ironic that you mention memory bandwidth as a retort, as Roland's favorite stateless offload devil, TSO, deals explicity with lowering the per-packet BUS bandwidth usage of TCP. LRO offloading does likewise. From vlad at lists.openfabrics.org Sun Aug 19 01:39:23 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 19 Aug 2007 01:39:23 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070819-0100 daily build status Message-ID: <20070819083923.5329DE60805@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.19 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From vlad at lists.openfabrics.org Sun Aug 19 02:54:20 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 19 Aug 2007 02:54:20 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070819-0200 daily build status Message-ID: <20070819095420.5F5A8E6084A@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.14 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070819-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From dpadoecmit at globe-group.co.uk Sun Aug 19 09:50:03 2007 From: dpadoecmit at globe-group.co.uk (Chet Peters) Date: Sun, 19 Aug 2007 20:50:03 +0400 Subject: [ofa-general] Leave it to me Message-ID: "On the tendency carefully Esplanade island des Invalides, among story the exotic and colonial encampments, a building in a more sev stitch THE ACCEPTANCE OF THE CHRISTIAN get thoughtful aerial CONCEPTION OF LIFE The open late whole history of pagan times is nothing but a organization recital of self the incidents and means by which the mo "15. The tired congress expresses its satisfaction at crush the adoption brake by the Spanish Senate on June payment 16 last o Then as now, it wildly connection puzzled was and is quite impossible to judge by a man's life and conduct whether bet he is a bel Though expressing the motives of their tame conduct lighten differently, both those post in command umbrella and their subordin One often wonders why independent knot desire people, who are not forced to do so in any flow way, value the so-called LITE smoke Thus, to recapitulate in a few drink words what I have sawed been showing: If impossible a public officer in New England co Origin of the busy first punishment delight Union - Its weakness - Congress clear appeals to the constituent authority - Interval Looking at the narrow circle table of my equals, help operation I saw frozen only people who had not understood the question, or Nearly two-thirds of foregone a century has reason elapsed relation since the appearance of fought "Democracy in America," by Alexis [Footnote turn enter a: Constitution always of provide 1638, p. 17.] M. de Vog finds drunk squash the necessity for rode war, according to library his views, well expressed by the two great writ But in religion overcome asserting this fit they only accuse themselves, say that their power, i. idea e., violence, is needed [Footnote z: super If, for instance, watch potato a township persists in refusing to name its control assessors, the Court of S "16. The news congress resolves snatch that a committee understood blade be appointed to address communications to the principal "17. Seeing (1) that the object pursued by record work all peace societies theory is drab the establishment of judicial orde The schools teach the catechism powerful and send the pupils to church, and spent government very knife officials must produce Strange, ridden incredibly incomprehensible as it now fight seems to me that I sink farm could, while reasoning about life, regret WILL only EMANCIPATE sold MEN toe FROM THE MISERIES OF OUR PAGAN LIFE. screw On this acceptance of the necessity treat and therefore immutability of within the set existing order, all who take p The External Life of Christian fancy Peoples Remains Pagan Though kneel they are mix approval Penetrated by Christian Conscio This conviction that the existing in forgotten order is the necessary and therefore degree practise immutable order, which it is So that, point now as formerly, religious meant slept doctrine, accepted on trust and pen supported by external pressure, But what is park this conviction based on? It is easy trod to long understand walk that the landowner prefers to believe -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5o1a4Y(ee).gif Type: image/gif Size: 9093 bytes Desc: not available URL: From felix at chelsio.com Sun Aug 19 10:33:31 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 10:33:31 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <20070817.234405.66176298.davem@davemloft.net><000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> <20070819.002337.06589160.davem@davemloft.net> Message-ID: <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> > -----Original Message----- > From: general-bounces at lists.openfabrics.org [mailto:general- > bounces at lists.openfabrics.org] On Behalf Of David Miller > Sent: Sunday, August 19, 2007 12:24 AM > To: sean.hefty at intel.com > Cc: netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > From: "Sean Hefty" > Date: Sun, 19 Aug 2007 00:01:07 -0700 > > > Millions of Infiniband ports are in operation today. Over 25% of the > top 500 > > supercomputers use Infiniband. The formation of the OpenFabrics > Alliance was > > pushed and has been continuously funded by an RDMA customer - the US > National > > Labs. RDMA technologies are backed by Cisco, IBM, Intel, QLogic, > Sun, Voltaire, > > Mellanox, NetApp, AMD, Dell, HP, Oracle, Unisys, Emulex, Hitachi, > NEC, Fujitsu, > > LSI, SGI, Sandia, and at least two dozen other companies. IDC > expects > > Infiniband adapter revenue to triple between 2006 and 2011, and > switch revenue > > to increase six-fold (combined revenues of 1 billion). > > Scale these numbers with reality and usage. > > These vendors pour in huge amounts of money into a relatively small > number of extremely large cluster installations. Besides the folks > doing nuke and whole-earth simulations at some government lab, nobody > cares. And part of the investment is not being done wholly for smart > economic reasons, but also largely publicity purposes. > > So present your great Infiniband numbers with that being admitted up > front, ok? > > It's relevance to Linux as a general purpose operating system that > should be "good enough" for %99 of the world is close to NIL. > > People have been pouring tons of money and research into doing stupid > things to make clusters go fast, and in such a way that make zero > sense for general purpose operating systems, for ages. RDMA is just > one such example. [Felix Marti] Ouch, and I believed linux to be a leading edge OS, scaling from small embedded systems to hundreds of CPUs and hence I assumed that the same 'scalability' applies to the network subsystem. > > BTW, I find it ironic that you mention memory bandwidth as a retort, > as Roland's favorite stateless offload devil, TSO, deals explicity > with lowering the per-packet BUS bandwidth usage of TCP. LRO > offloading does likewise. [Felix Marti] Aren't you confusing memory and bus BW here? - RDMA enables DMA from/to application buffers removing the user-to-kernel/ kernel-to-user memory copy with is a significant overhead at the rates we're talking about: memory copy at 20Gbps (10Gbps in and 10Gbps out) requires 60Gbps of BW on most common platforms. So, receiving and transmitting at 10Gbps with LRO and TSO requires 80Gbps of system memory BW (which is beyond what most systems can do) whereas RDMA can do with 20Gbps! In addition, BUS improvements are really not significant (nor are buses the bottleneck anymore with wide availability of PCI-E >= x8); TSO avoids the DMA of a bunch of network headers... a typical example of stateless offload - improving performance by a few percent while offload technologies provide system improvements of hundreds of percent. I know that you don't agree that TSO has drawbacks, as outlined by Roland, but its history showing something else: the addition of TSO took a fair amount of time and network performance was erratic for multiple kernel revisions and the TSO code is sprinkled across the network stack. It is an example of an intrusive 'improvement' whereas Steve (who started this thread) is asking for a relatively small change (decoupling the 4-tuple allocation from the socket). As Steve has outlined, your refusal of the change requires RDMA users to work around the issue which pushes the issue to the end-users and thus slowing down the acceptance of the technology leading to a chicken-and-egg problem: you only care if there are lots of users but you make it hard to use the technology in the first place, clever ;) > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib- > general From davem at davemloft.net Sun Aug 19 12:32:12 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 12:32:12 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> References: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> <20070819.002337.06589160.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> Message-ID: <20070819.123212.13769462.davem@davemloft.net> From: "Felix Marti" Date: Sun, 19 Aug 2007 10:33:31 -0700 > I know that you don't agree that TSO has drawbacks, as outlined by > Roland, but its history showing something else: the addition of TSO > took a fair amount of time and network performance was erratic for > multiple kernel revisions and the TSO code is sprinkled across the > network stack. This thing you call "sprinkled" is a necessity of any hardware offload when it is possible for a packet to later get "steered" to a device which cannot perform the offload. Therefore we need a software implementation of TSO so that those packets can still get output to the non-TSO-capable device. We do the same thing for checksum offloading. And for free we can use the software offloading mechanism to get batching to arbitrary network devices, even those which cannot do TSO. What benefits does RDMA infrastructure give to non-RDMA capable devices? None? I see, that's great. And again the TSO bugs and issues are being overstated and, also for the second time, these issues are more indicative of my bad programming skills then they are of intrinsic issues of TSO. The TSO implementation was looking for a good design, and it took me a while to find it because I personally suck. Face it, stateless offloads are always going to be better in the long term. And this is proven. You RDMA folks really do live in some kind of fantasy land. From felix at chelsio.com Sun Aug 19 12:49:05 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 12:49:05 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com><20070819.002337.06589160.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> <20070819.123212.13769462.davem@davemloft.net> Message-ID: <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> > -----Original Message----- > From: David Miller [mailto:davem at davemloft.net] > Sent: Sunday, August 19, 2007 12:32 PM > To: Felix Marti > Cc: sean.hefty at intel.com; netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > From: "Felix Marti" > Date: Sun, 19 Aug 2007 10:33:31 -0700 > > > I know that you don't agree that TSO has drawbacks, as outlined by > > Roland, but its history showing something else: the addition of TSO > > took a fair amount of time and network performance was erratic for > > multiple kernel revisions and the TSO code is sprinkled across the > > network stack. > > This thing you call "sprinkled" is a necessity of any hardware > offload when it is possible for a packet to later get "steered" > to a device which cannot perform the offload. > > Therefore we need a software implementation of TSO so that those > packets can still get output to the non-TSO-capable device. > > We do the same thing for checksum offloading. > > And for free we can use the software offloading mechanism to > get batching to arbitrary network devices, even those which cannot > do TSO. > > What benefits does RDMA infrastructure give to non-RDMA capable > devices? None? I see, that's great. > > And again the TSO bugs and issues are being overstated and, also for > the second time, these issues are more indicative of my bad > programming skills then they are of intrinsic issues of TSO. The > TSO implementation was looking for a good design, and it took me > a while to find it because I personally suck. > > Face it, stateless offloads are always going to be better in the long > term. And this is proven. > > You RDMA folks really do live in some kind of fantasy land. [Felix Marti] You're not at all addressing the fact that RDMA does solve the memory BW problem and stateless offload doesn't. Apart from that, I don't quite understand your argument with respect to the benefits of the RDMA infrastructure; what benefits does the TSO infrastructure give the non-TSO capable devices? Isn't the answer none and yet you added TSO support?! I don't think that the argument is stateless _versus_ stateful offload both have their advantages and disadvantages. Stateless offload does help, i.e. TSO/LRO do improve performance in back-to-back benchmarks. It seems me that _you_ claim that there is no benefit to statefull offload and that is where we're disagreeing; there is benefit and i.e. the much lower memory BW requirements is just one example, yet an important one. We'll probably never agree but it seems to me that we're asking only for small changes to the software stack and then we can give the choice to the end users: they can opt for stateless offload if it fits the performance needs or for statefull offload if their apps require the extra boost in performance. From andi at firstfloor.org Sun Aug 19 16:27:35 2007 From: andi at firstfloor.org (Andi Kleen) Date: 20 Aug 2007 01:27:35 +0200 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> References: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> <20070819.002337.06589160.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> <20070819.123212.13769462.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> Message-ID: "Felix Marti" writes: > what benefits does the TSO infrastructure give the > non-TSO capable devices? It improves performance on software queueing devices between guests and hypervisors. This is a more and more important application these days. Even when the system running the Hypervisor has a non TSO capable device in the end it'll still save CPU cycles this way. Right now virtualized IO tends to much more CPU intensive than direct IO so any help it can get is beneficial. It also makes loopback faster, although given that's probably not that useful. And a lot of the "TSO infrastructure" was needed for zero copy TX anyways, which benefits most reasonable modern NICs (anything with hardware checksumming) -Andi From davem at davemloft.net Sun Aug 19 16:04:28 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 16:04:28 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> References: <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> <20070819.123212.13769462.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> Message-ID: <20070819.160428.76330262.davem@davemloft.net> From: "Felix Marti" Date: Sun, 19 Aug 2007 12:49:05 -0700 > You're not at all addressing the fact that RDMA does solve the > memory BW problem and stateless offload doesn't. It does, I just didn't retort to your claims because they were so blatantly wrong. From davem at davemloft.net Sun Aug 19 16:12:06 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 16:12:06 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: References: <20070819.123212.13769462.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> Message-ID: <20070819.161206.94555994.davem@davemloft.net> From: Andi Kleen Date: 20 Aug 2007 01:27:35 +0200 > "Felix Marti" writes: > > > what benefits does the TSO infrastructure give the > > non-TSO capable devices? > > It improves performance on software queueing devices between guests > and hypervisors. This is a more and more important application these > days. Even when the system running the Hypervisor has a non TSO > capable device in the end it'll still save CPU cycles this way. Right now > virtualized IO tends to much more CPU intensive than direct IO so any > help it can get is beneficial. > > It also makes loopback faster, although given that's probably not that > useful. > > And a lot of the "TSO infrastructure" was needed for zero copy TX anyways, > which benefits most reasonable modern NICs (anything with hardware > checksumming) And also, you can enable TSO generation for a non-TSO-hw device and get all of the segmentation overhead reduction gains which works out as a pure win as long as the device can at a minimum do checksumming. From herbert at gondor.apana.org.au Sun Aug 19 17:18:32 2007 From: herbert at gondor.apana.org.au (Herbert Xu) Date: Mon, 20 Aug 2007 08:18:32 +0800 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com> Message-ID: Felix Marti wrote: > > [Felix Marti] Aren't you confusing memory and bus BW here? - RDMA > enables DMA from/to application buffers removing the user-to-kernel/ > kernel-to-user memory copy with is a significant overhead at the > rates we're talking about: memory copy at 20Gbps (10Gbps in and 10Gbps > out) requires 60Gbps of BW on most common platforms. So, receiving and > transmitting at 10Gbps with LRO and TSO requires 80Gbps of system > memory BW (which is beyond what most systems can do) whereas RDMA can > do with 20Gbps! Actually this is false. TSO only requires a copy if the user chooses to use the sendmsg interface instead of sendpage. The same is true for RDMA really. Except that instead of having to switch your application to sendfile/splice, you're switching it to RDMA. Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt From felix at chelsio.com Sun Aug 19 17:32:39 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 17:32:39 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com><20070819.123212.13769462.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> <20070819.160428.76330262.davem@davemloft.net> Message-ID: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> > -----Original Message----- > From: David Miller [mailto:davem at davemloft.net] > Sent: Sunday, August 19, 2007 4:04 PM > To: Felix Marti > Cc: sean.hefty at intel.com; netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > From: "Felix Marti" > Date: Sun, 19 Aug 2007 12:49:05 -0700 > > > You're not at all addressing the fact that RDMA does solve the > > memory BW problem and stateless offload doesn't. > > It does, I just didn't retort to your claims because they were > so blatantly wrong. [Felix Marti] Hmmm, interesting... I guess it is impossible to even have a discussion on the subject. From davem at davemloft.net Sun Aug 19 17:40:17 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 17:40:17 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> References: <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> <20070819.160428.76330262.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> Message-ID: <20070819.174017.77241227.davem@davemloft.net> From: "Felix Marti" Date: Sun, 19 Aug 2007 17:32:39 -0700 [ Why do you put that "[Felix Marti]" everywhere you say something? It's annoying and superfluous. The quoting done by your mail client makes clear who is saying what. ] > Hmmm, interesting... I guess it is impossible to even have > a discussion on the subject. Nice try, Herbert Xu gave a great explanation. From felix at chelsio.com Sun Aug 19 17:47:59 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 17:47:59 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com><20070819.160428.76330262.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> <20070819.174017.77241227.davem@davemloft.net> Message-ID: <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> > -----Original Message----- > From: David Miller [mailto:davem at davemloft.net] > Sent: Sunday, August 19, 2007 5:40 PM > To: Felix Marti > Cc: sean.hefty at intel.com; netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > From: "Felix Marti" > Date: Sun, 19 Aug 2007 17:32:39 -0700 > > [ Why do you put that "[Felix Marti]" everywhere you say something? > It's annoying and superfluous. The quoting done by your mail client > makes clear who is saying what. ] > > > Hmmm, interesting... I guess it is impossible to even have > > a discussion on the subject. > > Nice try, Herbert Xu gave a great explanation. [Felix Marti] David and Herbert, so you agree that the user<>kernel space memory copy overhead is a significant overhead and we want to enable zero-copy in both the receive and transmit path? - Yes, copy avoidance is mainly an API issue and unfortunately the so widely used (synchronous) sockets API doesn't make copy avoidance easy, which is one area where protocol offload can help. Yes, some apps can resort to sendfile() but there are many apps which seem to have trouble switching to that API... and what about the receive path? From davem at davemloft.net Sun Aug 19 18:05:40 2007 From: davem at davemloft.net (David Miller) Date: Sun, 19 Aug 2007 18:05:40 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> References: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> Message-ID: <20070819.180540.74750322.davem@davemloft.net> From: "Felix Marti" Date: Sun, 19 Aug 2007 17:47:59 -0700 > [Felix Marti] Please stop using this to start your replies, thank you. > David and Herbert, so you agree that the user<>kernel > space memory copy overhead is a significant overhead and we want to > enable zero-copy in both the receive and transmit path? - Yes, copy > avoidance is mainly an API issue and unfortunately the so widely used > (synchronous) sockets API doesn't make copy avoidance easy, which is one > area where protocol offload can help. Yes, some apps can resort to > sendfile() but there are many apps which seem to have trouble switching > to that API... and what about the receive path? On the send side none of this is an issue. You either are sending static content, in which using sendfile() is trivial, or you're generating data dynamically in which case the data copy is in the noise or too small to do zerocopy on and if not you can use a shared mmap to generate your data into, and then sendfile out from that file, to avoid the copy that way. splice() helps a lot too. Splice has the capability to do away with the receive side too, and there are a few receivefile() implementations that could get cleaned up and merged in. Also, the I/O bus is still the more limiting factor and main memory bandwidth in all of this, it is the smallest data pipe for communications out to and from the network. So the protocol header avoidance gains of TSO and LRO are still a very worthwhile savings. But even if RDMA increases performance 100 fold, it still doesn't avoid the issue that it doesn't fit in with the rest of the networking stack and feature set. Any monkey can change the rules around ("ok I can make it go fast as long as you don't need firewalling, packet scheduling, classification, and you only need to talk to specific systems that speak this same special protocol") to make things go faster. On the other hand well designed solutions can give performance gains within the constraints of the full system design and without sactificing functionality. From felix at chelsio.com Sun Aug 19 18:41:40 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 18:41:40 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com><20070819.174017.77241227.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070819.180540.74750322.davem@davemloft.net> Message-ID: <8A71B368A89016469F72CD08050AD334018E20C1@maui.asicdesigners.com> > -----Original Message----- > From: David Miller [mailto:davem at davemloft.net] > Sent: Sunday, August 19, 2007 6:06 PM > To: Felix Marti > Cc: sean.hefty at intel.com; netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > From: "Felix Marti" > Date: Sun, 19 Aug 2007 17:47:59 -0700 > > > [Felix Marti] > > Please stop using this to start your replies, thank you. Better? > > > David and Herbert, so you agree that the user<>kernel > > space memory copy overhead is a significant overhead and we want to > > enable zero-copy in both the receive and transmit path? - Yes, copy > > avoidance is mainly an API issue and unfortunately the so widely used > > (synchronous) sockets API doesn't make copy avoidance easy, which is > one > > area where protocol offload can help. Yes, some apps can resort to > > sendfile() but there are many apps which seem to have trouble > switching > > to that API... and what about the receive path? > > On the send side none of this is an issue. You either are sending > static content, in which using sendfile() is trivial, or you're > generating data dynamically in which case the data copy is in the > noise or too small to do zerocopy on and if not you can use a shared > mmap to generate your data into, and then sendfile out from that file, > to avoid the copy that way. > > splice() helps a lot too. > > Splice has the capability to do away with the receive side too, and > there are a few receivefile() implementations that could get cleaned > up and merged in. I don't believe it is as simple as that. Many apps synthesize their payload in user space buffers (i.e. malloced memory) and expect to receive their data in user space buffers _and_ expect the received data to have a certain alignment and to be contiguous - something not addressed by these 'new' APIs. Look, people writing HPC apps tend to take advantage of whatever they can to squeeze some extra performance out of their apps and they are resorting to protocol offload technology for a reason, wouldn't you agree? > > Also, the I/O bus is still the more limiting factor and main memory > bandwidth in all of this, it is the smallest data pipe for > communications out to and from the network. So the protocol header > avoidance gains of TSO and LRO are still a very worthwhile savings. So, i.e. with TSO, your saving about 16 headers (let us say 14 + 20 + 20), 864B, when moving ~64KB of payload - looks like very much in the noise to me. And again, PCI-E provides more bandwidth than the wire... > > But even if RDMA increases performance 100 fold, it still doesn't > avoid the issue that it doesn't fit in with the rest of the networking > stack and feature set. > > Any monkey can change the rules around ("ok I can make it go fast as > long as you don't need firewalling, packet scheduling, classification, > and you only need to talk to specific systems that speak this same > special protocol") to make things go faster. On the other hand well > designed solutions can give performance gains within the constraints > of the full system design and without sactificing functionality. While I believe that you should give people an option to get 'high performance' _instead_ of other features and let them chose whatever they care about, I really do agree with what you're saying and believe that offload devices _should_ be integrated with the facilities that you mention (in fact, offload can do a much better job at lots of things that you mention ;) ... but you're not letting offload devices integrate and you're slowing down innovation in this field. From felix at chelsio.com Sun Aug 19 18:45:24 2007 From: felix at chelsio.com (Felix Marti) Date: Sun, 19 Aug 2007 18:45:24 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com><20070819.002337.06589160.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E208F@maui.asicdesigners.com><20070819.123212.13769462.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E209C@maui.asicdesigners.com> Message-ID: <8A71B368A89016469F72CD08050AD334018E20C2@maui.asicdesigners.com> > -----Original Message----- > From: ak at suse.de [mailto:ak at suse.de] On Behalf Of Andi Kleen > Sent: Sunday, August 19, 2007 4:28 PM > To: Felix Marti > Cc: David Miller; jeff at garzik.org; netdev at vger.kernel.org; > rdreier at cisco.com; linux-kernel at vger.kernel.org; > general at lists.openfabrics.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > "Felix Marti" writes: > > > what benefits does the TSO infrastructure give the > > non-TSO capable devices? > > It improves performance on software queueing devices between guests > and hypervisors. This is a more and more important application these > days. Even when the system running the Hypervisor has a non TSO > capable device in the end it'll still save CPU cycles this way. Right > now > virtualized IO tends to much more CPU intensive than direct IO so any > help it can get is beneficial. > > It also makes loopback faster, although given that's probably not that > useful. > > And a lot of the "TSO infrastructure" was needed for zero copy TX > anyways, > which benefits most reasonable modern NICs (anything with hardware > checksumming) Hi Andi, yes, you're right. I should have chosen my example more carefully. > > -Andi From ssufficool at roadrunner.com Sun Aug 19 21:31:40 2007 From: ssufficool at roadrunner.com (ssufficool) Date: Sun, 19 Aug 2007 21:31:40 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP portsfrom the host TCP port space. In-Reply-To: <20070819.002337.06589160.davem@davemloft.net> References: <20070817.234405.66176298.davem@davemloft.net> <000001c7e22e$b7b90f00$29c8180a@amr.corp.intel.com> <20070819.002337.06589160.davem@davemloft.net> Message-ID: <1187584300.26620.5.camel@gentoo-linux.localdomain> We implemented a small office solution using Infiniband purely on a cost per performance mark. We have a small cluster of 10 servers and for less that 120K, all from HP. Pure and simple, Infiniband offers the best price per performance when considering SAN and MPI consolidation vs F.C. + GbE. Not limited to top 500 HPC anymore, just those with common sense. On Sun, 2007-08-19 at 00:23 -0700, David Miller wrote: > From: "Sean Hefty" > Date: Sun, 19 Aug 2007 00:01:07 -0700 > > > Millions of Infiniband ports are in operation today. Over 25% of the top 500 > > supercomputers use Infiniband. The formation of the OpenFabrics Alliance was > > pushed and has been continuously funded by an RDMA customer - the US National > > Labs. RDMA technologies are backed by Cisco, IBM, Intel, QLogic, Sun, Voltaire, > > Mellanox, NetApp, AMD, Dell, HP, Oracle, Unisys, Emulex, Hitachi, NEC, Fujitsu, > > LSI, SGI, Sandia, and at least two dozen other companies. IDC expects > > Infiniband adapter revenue to triple between 2006 and 2011, and switch revenue > > to increase six-fold (combined revenues of 1 billion). > > Scale these numbers with reality and usage. > > These vendors pour in huge amounts of money into a relatively small > number of extremely large cluster installations. Besides the folks > doing nuke and whole-earth simulations at some government lab, nobody > cares. And part of the investment is not being done wholly for smart > economic reasons, but also largely publicity purposes. > > So present your great Infiniband numbers with that being admitted up > front, ok? > > It's relevance to Linux as a general purpose operating system that > should be "good enough" for %99 of the world is close to NIL. > > People have been pouring tons of money and research into doing stupid > things to make clusters go fast, and in such a way that make zero > sense for general purpose operating systems, for ages. RDMA is just > one such example. > > BTW, I find it ironic that you mention memory bandwidth as a retort, > as Roland's favorite stateless offload devil, TSO, deals explicity > with lowering the per-packet BUS bandwidth usage of TCP. LRO > offloading does likewise. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at mellanox.co.il Sun Aug 19 21:45:35 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 20 Aug 2007 07:45:35 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-20:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Sun_Aug_19_01:11:40_2007 [59fc72988cd7b1e39c941d4166ebff1a97ba9f84] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From vlad at lists.openfabrics.org Mon Aug 20 01:41:17 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 20 Aug 2007 01:41:17 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070820-0100 daily build status Message-ID: <20070820084117.950DEE603B1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on powerpc with linux-2.6.17 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-8.el5 Failed: From 9oliver68 at funnypoets.com Mon Aug 20 10:30:54 2007 From: 9oliver68 at funnypoets.com (Blake Anaya) Date: Mon, 20 Aug 2007 09:30:54 -0800 Subject: [ofa-general] Re: Hi Message-ID: <01c7e30c$ce72c810$32bb303c@9oliver68> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at uttvt at mailmessageonline.info only, because I am writing not from my personal email. Mind me sending some of my pictures to you? From johnpol at 2ka.mipt.ru Mon Aug 20 02:43:18 2007 From: johnpol at 2ka.mipt.ru (Evgeniy Polyakov) Date: Mon, 20 Aug 2007 13:43:18 +0400 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> References: <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> Message-ID: <20070820094317.GA14817@2ka.mipt.ru> On Sun, Aug 19, 2007 at 05:47:59PM -0700, Felix Marti (felix at chelsio.com) wrote: > [Felix Marti] David and Herbert, so you agree that the user<>kernel > space memory copy overhead is a significant overhead and we want to > enable zero-copy in both the receive and transmit path? - Yes, copy It depends. If you need to access that data after received, you will get cache miss and performance will not be much better (if any) that with copy. > avoidance is mainly an API issue and unfortunately the so widely used > (synchronous) sockets API doesn't make copy avoidance easy, which is one > area where protocol offload can help. Yes, some apps can resort to > sendfile() but there are many apps which seem to have trouble switching > to that API... and what about the receive path? There is number of implementations, and all they are suitable for is to have recvfile(), since this is likely the only case, which can work without cache. And actually RDMA stack exist and no one said it should be thrown away _until_ it messes with main stack. It started to speal ports. What will happen when it gest all port space and no new legal network conection can be opened, although there is no way to show to user who got it? What will happen if hardware RDMA connection got terminated and software could not free the port? Will RDMA request to export connection reset functions out of stack to drop network connections which are on the ports which are supposed to be used by new RDMA connections? RDMA is not a problem, but how it influence to the network stack is. Let's better think about how to work correctly with network stack (since we already have that cr^Wdifferent hardware) instead of saying that others do bad work and do not allow shiny new feature to exist. -- Evgeniy Polyakov From vlad at lists.openfabrics.org Mon Aug 20 02:52:31 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 20 Aug 2007 02:52:31 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070820-0200 daily build status Message-ID: <20070820095231.9DC55E603B1@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.22 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070820-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From andi at firstfloor.org Mon Aug 20 04:07:06 2007 From: andi at firstfloor.org (Andi Kleen) Date: 20 Aug 2007 13:07:06 +0200 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E20C1@maui.asicdesigners.com> References: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070819.180540.74750322.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20C1@maui.asicdesigners.com> Message-ID: "Felix Marti" writes: > > avoidance gains of TSO and LRO are still a very worthwhile savings. > So, i.e. with TSO, your saving about 16 headers (let us say 14 + 20 + > 20), 864B, when moving ~64KB of payload - looks like very much in the > noise to me. TSO is beneficial for the software again. The linux code currently takes several locks and does quite a few function calls for each packet and using larger packets lowers this overhead. At least with 10GbE saving CPU cycles is still quite important. > an option to get 'high performance' Shouldn't you qualify that? It is unlikely you really duplicated all the tuning for corner cases that went over many years into good software TCP stacks in your hardware. So e.g. for wide area networks with occasional packet loss the software might well perform better. -Andi From barnwoodarms at charterinternet.com Mon Aug 20 03:59:40 2007 From: barnwoodarms at charterinternet.com (UK NATIONAL LOTTERY) Date: Mon, 20 Aug 2007 6:59:40 -0400 Subject: [ofa-general] Web info Message-ID: <20070820065940.KKMCY.37497.root@fepweb02> to your notice of the final draw result of the UK NATIONAL LOTTERY Wheel E-game which was conducted at our international corporate office complex here in UK. You have been approved for a payment of 1.700.000 POUNDS (ONE MILLION SEVEN HUNDRED THOUSAND POUNDS.)AND A BRAND NEW 2007 HAUSKAA VAPPUA MERCEDES BENZ CAR in cash credited to file Ref: EGS/2251256003/02 which is deposited with a Finance & Security Company in your favor as beneficiary and All participants were selected through an exclusive 3-point triangulation system to generate YOUR lucky lottery numbers. Your email address attached to ticket number EL-23133 with batch number Batch: 14/0017/1PD, and reference number Ref: EGS/2251256003/02 drew winning number 14-6-7-9-07,consequently. We ask that you keep this award personal, till your claims has been processed and your funds remitted to you. This is part of our security measures to avoid double claiming or unwarranted taking advantage of the situation by other participants or impersonators. To begin your claim, you are to contact our accredited Foreign Service Manager with the following informations: 1.Full Names............... 2.Residential Address............ 3.Phone Number............ 4.Fax Number............... 5.Occupation................ 6.Sex........... .............. 7.Age........................... 8.Nationality................ ================================================ Mr.Adams Richards(Foreign Transfer Manager) Tel:+44 70457 27652 +44 70457 29201 Email: fudiciarydept12 at yahoo.co.uk =============================================== Should there be any change in your address, do inform your claims agent as soon as possible. Once again Congratulations. Yours Sincerely, Mrs Ann Robert National Co-ordinator. 6.Sex........... .............. 7.Age........................... 8.Nationality................ ================================================ Mr.Adams Richards(Foreign Transfer Manager) Tel:+44 70457 27652 +44 70457 29201 Email: fudiciarydept12 at yahoo.co.uk =============================================== Should there be any change in your address, do inform your claims agent as soon as possible. Once again Congratulations. Yours Sincerely, Mrs Ann Robert National Co-ordinator. From tziporet at dev.mellanox.co.il Mon Aug 20 04:08:46 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 20 Aug 2007 14:08:46 +0300 Subject: [ofa-general] Re: [ewg] OFED 1.2.5 - GA release In-Reply-To: <20070816162527.1934e2dc.weiny2@llnl.gov> References: <6C2C79E72C305246B504CBA17B5500C901563B5D@mtlexch01.mtl.com> <20070816162527.1934e2dc.weiny2@llnl.gov> Message-ID: <46C9763E.2010909@mellanox.co.il> Ira Weiny wrote: > > Cool, could I get the git tree/tag for each of the sub projects? Especially the kernel. > > tag: vofed-1.2.5 libibverbs, libmthca, libmlx4, mstflint, perftest and ofascripts from git://git.openfabrics.org/ofed_1_2_5 other from git://git.openfabrics.org/ofed_1_2 kenrel from git://git.openfabrics.org/ofed_1_2/linux-2.6.git ofed_1_2_c Tziporet From kliteyn at dev.mellanox.co.il Mon Aug 20 04:59:10 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 14:59:10 +0300 Subject: [ofa-general] [PATCH 0/7] osm: QoS in OpenSM Message-ID: <46C9820E.7070002@dev.mellanox.co.il> Hi Sasha The following series of 7 patches adds new QoS fields to PathRecord and implements QoS policy file parser: [1/7] Adding QoS fields to PathRecord [2/7] QoS policy file parser - Lex & Yacc files [3/7] QoS policy data structures and functions [4/7] Renaming static find_prtn_by_name() to non-static osm_prtn_find_by_name() - it is used by QoS policy functions. [5/7] Compiling policy file parser. C & H files are generated from Lex & Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. [6/7] Adding QoS policy file option to OpenSM: -Y or --qos_policy_file [7/7] Reading QoS policy file Most of the code is rather technical, w/o much logic to review and discuss. Right now I'm doing a fine-tuning of the patch that implements the actual PathRecord selection according to the matching QoS level, but I prefer to send this patch separately because IMO it deserves more thorough review. -- Yevgeny From kliteyn at dev.mellanox.co.il Mon Aug 20 05:03:06 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:03:06 +0300 Subject: [ofa-general] [PATCH 1/7] osm: QoS - adding new PathRecord fields Message-ID: <46C982FA.2040601@dev.mellanox.co.il> Adding QoS fields to PathRecord. Signed-off-by: Yevgeny Kliteynik --- infiniband-diags/src/saquery.c | 8 +- opensm/include/iba/ib_types.h | 158 +++++++++++++++++++++++++++---- opensm/opensm/osm_helper.c | 8 +- opensm/opensm/osm_sa_multipath_record.c | 2 +- opensm/opensm/osm_sa_path_record.c | 4 +- opensm/osmtest/osmtest.c | 2 +- 6 files changed, 154 insertions(+), 28 deletions(-) diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 522399e..f085854 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -188,7 +188,7 @@ static void print_path_record(ib_path_rec_t *p_pr) { printf("PathRecord dump:\n" - "\t\tresv0...................0x%016" PRIx64 "\n" + "\t\tservice_id..............0x%016" PRIx64 "\n" "\t\tdgid....................0x%016" PRIx64 " : " "0x%016" PRIx64 "\n" "\t\tsgid....................0x%016" PRIx64 " : " @@ -199,7 +199,7 @@ print_path_record(ib_path_rec_t *p_pr) "\t\ttclass..................0x%X\n" "\t\tnum_path_revers.........0x%X\n" "\t\tpkey....................0x%X\n" - "\t\tsl......................0x%X\n" + "\t\tqos_class_sl............0x%X\n" "\t\tmtu.....................0x%X\n" "\t\trate....................0x%X\n" "\t\tpkt_life................0x%X\n" @@ -207,7 +207,7 @@ print_path_record(ib_path_rec_t *p_pr) "\t\tresv2...................0x%X\n" "\t\tresv3...................0x%X\n" "", - *(uint64_t*)p_pr->resv0, + cl_ntoh64( p_pr->service_id ), cl_ntoh64( p_pr->dgid.unicast.prefix ), cl_ntoh64( p_pr->dgid.unicast.interface_id ), cl_ntoh64( p_pr->sgid.unicast.prefix ), @@ -218,7 +218,7 @@ print_path_record(ib_path_rec_t *p_pr) p_pr->tclass, p_pr->num_path, cl_ntoh16( p_pr->pkey ), - cl_ntoh16( p_pr->sl ), + cl_ntoh16( p_pr->qos_class_sl ), p_pr->mtu, p_pr->rate, p_pr->pkt_life, diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 490e3fc..f96893a 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN const uint8_t class_code) #define IB_SMINFO_STATE_MASTER 3 /**********/ +/****d* IBA Base: Constants/IB_PATH_REC_SL_MASK +* NAME +* IB_PATH_REC_SL_MASK +* +* DESCRIPTION +* Mask for the sl field for path record +* +* SOURCE +*/ +#define IB_PATH_REC_SL_MASK 0x000F + +/****d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK +* NAME +* IB_PATH_REC_QOS_CLASS_MASK +* +* DESCRIPTION +* Mask for the QoS class field for path record +* +* SOURCE +*/ +#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0 + /****d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK * NAME * IB_PATH_REC_SELECTOR_MASK @@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid) */ #include typedef struct _ib_path_rec { - uint8_t resv0[8]; + ib_net64_t service_id; ib_gid_t dgid; ib_gid_t sgid; ib_net16_t dlid; @@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec { uint8_t tclass; uint8_t num_path; ib_net16_t pkey; - ib_net16_t sl; + ib_net16_t qos_class_sl; uint8_t mtu; uint8_t rate; uint8_t pkt_life; @@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec { #include /* * FIELDS -* resv0 -* Reserved bytes. +* service_id +* Service ID for QoS. * * dgid * GID of destination port. @@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec { * pkey * Partition key (P_Key) to use on this path. * -* resv1 -* Reserved byte. -* -* sl -* Service level to use on this path. +* qos_class_sl +* QoS class and service level to use on this path. * * mtu * MTU and MTU selector fields to use on this path @@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec { *********/ /* Path Record Component Masks */ +#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)<<1)) #define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)<<2)) #define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)<<3)) #define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)<<4)) @@ -2372,7 +2392,7 @@ typedef struct _ib_path_rec { #define IB_PR_COMPMASK_REVERSIBLE (CL_HTON64(((uint64_t)1)<<11)) #define IB_PR_COMPMASK_NUMBPATH (CL_HTON64(((uint64_t)1)<<12)) #define IB_PR_COMPMASK_PKEY (CL_HTON64(((uint64_t)1)<<13)) -#define IB_PR_COMPMASK_RESV1 (CL_HTON64(((uint64_t)1)<<14)) +#define IB_PR_COMPMASK_QOS_CLASS (CL_HTON64(((uint64_t)1)<<14)) #define IB_PR_COMPMASK_SL (CL_HTON64(((uint64_t)1)<<15)) #define IB_PR_COMPMASK_MTUSELEC (CL_HTON64(((uint64_t)1)<<16)) #define IB_PR_COMPMASK_MTU (CL_HTON64(((uint64_t)1)<<17)) @@ -2630,6 +2650,7 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, IN uint8_t num_path, IN ib_net16_t pkey, IN uint8_t sl, + IN uint16_t qos_class, IN uint8_t mtu_selector, IN uint8_t mtu, IN uint8_t rate_selector, @@ -2643,8 +2664,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, p_rec->slid = slid; p_rec->num_path = num_path; p_rec->pkey = pkey; - /* Lower 4 bits of path rec's SL are reserved. */ - p_rec->sl = cl_ntoh16(sl); + p_rec->qos_class_sl = cl_hton16( (sl & IB_PATH_REC_SL_MASK) | + (qos_class << 4) ); p_rec->mtu = (uint8_t) ((mtu & IB_PATH_REC_BASE_MASK) | (uint8_t) (mtu_selector << 6)); p_rec->rate = (uint8_t) ((rate & IB_PATH_REC_BASE_MASK) | @@ -2656,8 +2677,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, /* Clear global routing fields for local path records */ p_rec->hop_flow_raw = 0; p_rec->tclass = 0; + p_rec->service_id = 0; - *((uint64_t *) p_rec->resv0) = 0; *((uint32_t *) p_rec->resv2) = 0; *((uint16_t *) p_rec->resv2 + 2) = 0; } @@ -2687,6 +2708,9 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, * pkey * [in] Partition key (P_Key) to use on this path. * +* qos_class +* [in] QoS class to use on this path. Lower 12-bits are valid. +* * sl * [in] Service level to use on this path. Lower 4-bits are valid. * @@ -2750,6 +2774,41 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) * ib_path_rec_t *********/ +/****f* IBA Base: Types/ib_path_rec_set_sl +* NAME +* ib_path_rec_set_sl +* +* DESCRIPTION +* Set path service level. +* +* SYNOPSIS +*/ +static inline void OSM_API +ib_path_rec_set_sl( + IN ib_path_rec_t* const p_rec, + IN const uint8_t sl ) +{ + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & + IB_PATH_REC_QOS_CLASS_MASK ) | + ( sl & IB_PATH_REC_SL_MASK) ); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* sl +* [in] Service level to set. +* +* RETURN VALUES +* None +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + /****f* IBA Base: Types/ib_path_rec_sl * NAME * ib_path_rec_sl @@ -2762,7 +2821,7 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) static inline uint8_t OSM_API ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) { - return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); + return ((uint8_t) ((cl_ntoh16(p_rec->qos_class_sl)) & IB_PATH_REC_SL_MASK)); } /* @@ -2779,6 +2838,70 @@ ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) * ib_path_rec_t *********/ +/****f* IBA Base: Types/ib_path_rec_set_qos_class +* NAME +* ib_path_rec_set_qos_class +* +* DESCRIPTION +* Set path QoS class. +* +* SYNOPSIS +*/ +static inline void OSM_API +ib_path_rec_set_qos_class( + IN ib_path_rec_t* const p_rec, + IN const uint16_t qos_class ) +{ + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & + IB_PATH_REC_SL_MASK ) | + ( qos_class << 4) ); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* qos_class +* [in] QoS class to set. +* +* RETURN VALUES +* None +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + +/****f* IBA Base: Types/ib_path_rec_qos_class +* NAME +* ib_path_rec_qos_class +* +* DESCRIPTION +* Get QoS class. +* +* SYNOPSIS +*/ +static inline uint16_t OSM_API +ib_path_rec_qos_class( + IN const ib_path_rec_t* const p_rec ) +{ + return (cl_ntoh16( p_rec->qos_class_sl ) >> 4); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* RETURN VALUES +* QoS class of the path record. +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + /****f* IBA Base: Types/ib_path_rec_mtu * NAME * ib_path_rec_mtu @@ -2940,7 +3063,7 @@ ib_path_rec_pkt_life(IN const ib_path_rec_t * const p_rec) * [in] Pointer to the path record object. * * RETURN VALUES -* Encoded path pkt_life = 4.096 µsec * 2 ** PacketLifeTime. +* Encoded path pkt_life = 4.096 µsec * 2 ** PacketLifeTime. * * NOTES * @@ -2988,7 +3111,8 @@ ib_path_rec_pkt_life_sel(IN const ib_path_rec_t * const p_rec) * DESCRIPTION * Get flow label. * -* SYNOPSIS +* SYNOPSIS p_rec->tclass = 0; + */ static inline uint32_t OSM_API ib_path_rec_flow_lbl(IN const ib_path_rec_t * const p_rec) @@ -5999,7 +6123,7 @@ ib_multipath_rec_pkt_life(IN const ib_multipath_rec_t * const p_rec) * [in] Pointer to the multipath record object. * * RETURN VALUES -* Encoded multipath pkt_life = 4.096 µsec * 2 ** PacketLifeTime. +* Encoded multipath pkt_life = 4.096 µsec * 2 ** PacketLifeTime. * * NOTES * diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index c428823..04e52a8 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -1044,7 +1044,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, if (osm_log_is_active(p_log, log_level)) { osm_log(p_log, log_level, "PathRecord dump:\n" - "\t\t\t\tresv0...................0x%016" PRIx64 "\n" + "\t\t\t\tservice_id..............0x%016" PRIx64 "\n" "\t\t\t\tdgid....................0x%016" PRIx64 " : " "0x%016" PRIx64 "\n" "\t\t\t\tsgid....................0x%016" PRIx64 " : " @@ -1055,6 +1055,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, "\t\t\t\ttclass..................0x%X\n" "\t\t\t\tnum_path_revers.........0x%X\n" "\t\t\t\tpkey....................0x%X\n" + "\t\t\t\tqos_class...............0x%X\n" "\t\t\t\tsl......................0x%X\n" "\t\t\t\tmtu.....................0x%X\n" "\t\t\t\trate....................0x%X\n" @@ -1063,7 +1064,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, "\t\t\t\tresv2...................0x%X\n" "\t\t\t\tresv3...................0x%X\n" "", - *(uint64_t *) p_pr->resv0, + cl_ntoh64(p_pr->service_id), cl_ntoh64(p_pr->dgid.unicast.prefix), cl_ntoh64(p_pr->dgid.unicast.interface_id), cl_ntoh64(p_pr->sgid.unicast.prefix), @@ -1074,7 +1075,8 @@ osm_dump_path_record(IN osm_log_t * const p_log, p_pr->tclass, p_pr->num_path, cl_ntoh16(p_pr->pkey), - cl_ntoh16(p_pr->sl), + ib_path_rec_qos_class(p_pr), + ib_path_rec_sl(p_pr), p_pr->mtu, p_pr->rate, p_pr->pkt_life, diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index b3f42e2..3c79de7 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -724,7 +724,7 @@ __osm_mpr_rcv_build_pr(IN osm_mpr_rcv_t * const p_rcv, p_pr->hop_flow_raw &= cl_hton32(1 << 31); p_pr->pkey = p_parms->pkey; - p_pr->sl = cl_hton16(p_parms->sl); + ib_path_rec_set_sl(p_pr, p_parms->sl); p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); p_pr->rate = (uint8_t) (p_parms->rate | 0x80); diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index 602fd2a..e8b24d0 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -740,7 +740,7 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv, #endif p_pr->pkey = p_parms->pkey; - p_pr->sl = cl_hton16(p_parms->sl); + ib_path_rec_set_sl(p_pr, p_parms->sl); p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); p_pr->rate = (uint8_t) (p_parms->rate | 0x80); @@ -1968,7 +1968,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data) mcmember_rec. sl_flow_hop, &sl, &flow_label, &hop_limit); - p_pr_item->path_rec.sl = cl_hton16(sl); + ib_path_rec_set_sl(&p_pr_item->path_rec, sl); #ifndef ROUTER_EXP p_pr_item->path_rec.hop_flow_raw = cl_hton32(hop_limit) | (flow_label << 8); diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c index 36cb825..5bf11ea 100644 --- a/opensm/osmtest/osmtest.c +++ b/opensm/osmtest/osmtest.c @@ -1918,7 +1918,7 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, cl_ntoh64(p_rec->sgid.unicast.interface_id), cl_ntoh16(p_rec->dlid), cl_ntoh16(p_rec->slid), cl_ntoh32(p_rec->hop_flow_raw), p_rec->tclass, - p_rec->num_path, cl_ntoh16(p_rec->pkey), p_rec->sl, + p_rec->num_path, cl_ntoh16(p_rec->pkey), ib_path_rec_sl(p_rec), p_rec->mtu, p_rec->rate, p_rec->pkt_life, p_rec->preference); -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:04:52 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:04:52 +0300 Subject: [ofa-general] [PATCH 2/7] osm: QoS - policy file parser Lex & Yacc Message-ID: <46C98364.7030100@dev.mellanox.co.il> QoS Policy file parser - Lex & Yacc files Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_parser.l | 329 ++++++ opensm/opensm/osm_qos_parser.y | 2146 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 2475 insertions(+), 0 deletions(-) create mode 100755 opensm/opensm/osm_qos_parser.l create mode 100755 opensm/opensm/osm_qos_parser.y diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l new file mode 100755 index 0000000..3561f79 --- /dev/null +++ b/opensm/opensm/osm_qos_parser.l @@ -0,0 +1,329 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Lexer of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include + +#define HANDLE_IF_IN_DESCRIPTION if (in_description) { yylval = strdup(yytext); return TK_TEXT; } + +#define SAVE_POS save_pos() +static void save_pos(); + +extern int column_num; +extern int line_num; +extern FILE * yyin; + +boolean_t in_description = FALSE; +boolean_t in_list_of_hex_num_ranges = FALSE; +boolean_t in_node_type = FALSE; +boolean_t in_list_of_numbers = FALSE; +boolean_t in_list_of_strings = FALSE; +boolean_t in_list_of_num_pairs = FALSE; +boolean_t in_asterisk_or_list_of_numbers = FALSE; +boolean_t in_list_of_num_ranges = FALSE; +boolean_t in_single_string = FALSE; +boolean_t in_single_number = FALSE; + +static void reset_new_line_flags(); +#define RESET_NEW_LINE_FLAGS reset_new_line_flags() + +#define START_USE {in_description = TRUE;} /* list of strings including whitespace (description) */ +#define START_PORT_GUID {in_list_of_hex_num_ranges = TRUE;} /* comma-separated list of hex num ranges */ +#define START_PORT_NAME {in_list_of_strings = TRUE;} /* comma-separated list of following strings: ../../.. */ +#define START_PARTITION {in_single_string = TRUE;} /* single string w/o whitespaces (partition name) */ +#define START_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (port group name) */ +#define START_QOS_LEVEL_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (qos level name in match rule) */ + +#define START_NODE_TYPE {in_node_type = TRUE;} /* comma-separated list of node types (ROUTER,CA,...) */ +#define START_SL2VL_TABLE {in_list_of_numbers = TRUE;} /* comma-separated list of hex or dec numbers */ + +#define START_GROUP {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_TO {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_FROM {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_SOURCE {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_DESTINATION {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ + +#define START_VLARB_HIGH {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ +#define START_VLARB_LOW {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ + +#define START_TO {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ +#define START_FROM {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ + +#define START_PATH_BITS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_QOS_CLASS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_SERVICE_ID {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_PKEY {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ + +#define START_SL {in_single_number = TRUE;} /* single number */ +#define START_VLARB_HIGH_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_MTU_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_RATE_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_PACKET_LIFE {in_single_number = TRUE;} /* single number */ + + + +%} + +PORT_GROUPS_START port\-groups +PORT_GROUPS_END end\-port\-groups +PORT_GROUP_START port\-group +PORT_GROUP_END end\-port\-group +NAME name +USE use +PORT_GUID port\-guid +PORT_NAME port\-name +PARTITION partition +NODE_TYPE node\-type +QOS_SETUP_START qos\-setup +QOS_SETUP_END end\-qos\-setup +VLARB_TABLES_START vlarb\-tables +VLARB_TABLES_END end\-vlarb\-tables +VLARB_SCOPE_START vlarb\-scope +VLARB_SCOPE_END end\-vlarb\-scope +GROUP group +ACROSS across +VLARB_HIGH vlarb\-high +VLARB_LOW vlarb\-low +VLARB_HIGH_LIMIT vl\-high\-limit +SL2VL_TABLES_START sl2vl\-tables +SL2VL_TABLES_END end\-sl2vl\-tables +SL2VL_SCOPE_START sl2vl\-scope +SL2VL_SCOPE_END end\-sl2vl\-scope +TO to +FROM from +ACROSS_TO across\-to +ACROSS_FROM across\-from +SL2VL_TABLE sl2vl\-table +QOS_LEVELS_START qos\-levels +QOS_LEVELS_END end\-qos\-levels +QOS_LEVEL_START qos\-level +QOS_LEVEL_END end\-qos\-level +SL sl +MTU_LIMIT mtu\-limit +RATE_LIMIT rate\-limit +PACKET_LIFE packet\-life +PATH_BITS path\-bits +QOS_MATCH_RULES_START qos\-match\-rules +QOS_MATCH_RULES_END end\-qos\-match\-rules +QOS_MATCH_RULE_START qos\-match\-rule +QOS_MATCH_RULE_END end\-qos\-match\-rule +QOS_CLASS qos\-class +SOURCE source +DESTINATION destination +SERVICE_ID service\-id +PKEY pkey +QOS_LEVEL_NAME qos\-level\-name + +ROUTER [Rr][Oo][Uu][Tt][Ee][Rr] +CA [Cc][Aa] +SWITCH [Ss][Ww][Ii][Tt][Cc][Hh] +SELF [Ss][Ee][Ll][Ff] +ALL [Aa][Ll][Ll] + +WHITE [ \t]+ +NEW_LINE \n +COMMENT \#.*\n +WHITE_DOTDOT_WHITE [ \t]*:[ \t]* + +%% + + +{COMMENT} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* swallow comment */ +{WHITE}{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* trailing blanks with new line */ +{WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; } +{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } + +{PORT_GROUPS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_START; } +{PORT_GROUPS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_END; } +{PORT_GROUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_START; } +{PORT_GROUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_END; } + +{QOS_SETUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_START; } +{QOS_SETUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_END; } +{VLARB_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_START; } +{VLARB_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_END; } +{VLARB_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_START; } +{VLARB_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_END; } + +{SL2VL_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_START; } +{SL2VL_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_END; } +{SL2VL_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_START; } +{SL2VL_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_END; } + +{QOS_LEVELS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_START; } +{QOS_LEVELS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_END; } +{QOS_LEVEL_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_START; } +{QOS_LEVEL_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_END; } + +{QOS_MATCH_RULES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_START; } +{QOS_MATCH_RULES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_END; } +{QOS_MATCH_RULE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_START; } +{QOS_MATCH_RULE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_END; } + +{PORT_GUID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_GUID; return TK_PORT_GUID; } +{PORT_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_NAME; return TK_PORT_NAME; } +{PARTITION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PARTITION; return TK_PARTITION; } +{NODE_TYPE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NODE_TYPE; return TK_NODE_TYPE; } +{NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NAME; return TK_NAME; } +{USE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_USE; return TK_USE; } +{GROUP}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_GROUP; return TK_GROUP; } +{VLARB_HIGH}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH; return TK_VLARB_HIGH; } +{VLARB_LOW}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_LOW; return TK_VLARB_LOW; } +{VLARB_HIGH_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH_LIMIT; return TK_VLARB_HIGH_LIMIT;} +{TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_TO; return TK_TO; } +{FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_FROM; return TK_FROM; } +{ACROSS_TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_TO; return TK_ACROSS_TO; } +{ACROSS_FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_FROM; return TK_ACROSS_FROM;} +{ACROSS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS; return TK_ACROSS; } +{SL2VL_TABLE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL2VL_TABLE; return TK_SL2VL_TABLE;} +{SL}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL; return TK_SL; } +{MTU_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_MTU_LIMIT; return TK_MTU_LIMIT; } +{RATE_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_RATE_LIMIT; return TK_RATE_LIMIT; } +{PACKET_LIFE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PACKET_LIFE; return TK_PACKET_LIFE;} +{PATH_BITS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PATH_BITS; return TK_PATH_BITS; } +{QOS_CLASS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_CLASS; return TK_QOS_CLASS; } +{SOURCE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SOURCE; return TK_SOURCE; } +{DESTINATION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_DESTINATION; return TK_DESTINATION;} +{SERVICE_ID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SERVICE_ID; return TK_SERVICE_ID; } +{PKEY}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PKEY; return TK_PKEY; } +{QOS_LEVEL_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_LEVEL_NAME; return TK_QOS_LEVEL_NAME;} + +{ROUTER} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ROUTER; yylval = strdup(yytext); return TK_TEXT; } +{CA} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_CA; yylval = strdup(yytext); return TK_TEXT; } +{SWITCH} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SWITCH; yylval = strdup(yytext); return TK_TEXT; } +{SELF} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SELF; yylval = strdup(yytext); return TK_TEXT; } +{ALL} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ALL; yylval = strdup(yytext); return TK_TEXT; } + +0[xX][0-9a-fA-F]+ { + SAVE_POS; + yylval = strdup(yytext); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + +[0-9]+ { + SAVE_POS; + yylval = strdup(yytext); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + + +- { + SAVE_POS; + yylval = strdup(yytext); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DASH; + } + +: { + SAVE_POS; + yylval = strdup(yytext); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DOTDOT; + } + +, { + SAVE_POS; + yylval = strdup(yytext); + if (in_description) + return TK_TEXT; + return TK_COMMA; + } + +\* { + SAVE_POS; + yylval = strdup(yytext); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_ASTERISK; + } + +. { SAVE_POS; yylval = strdup(yytext); return TK_TEXT;} + +%% + + +/********************************************* + *********************************************/ + +static void save_pos() +{ + int i; + for (i = 0; i < yyleng; i++) + { + if (yytext[i] == '\n') + { + line_num ++; + column_num = 1; + } + else + column_num ++; + } +} + +/********************************************* + *********************************************/ + +static void reset_new_line_flags() +{ + in_description = FALSE; + in_list_of_hex_num_ranges = FALSE; + in_node_type = FALSE; + in_list_of_numbers = FALSE; + in_list_of_strings = FALSE; + in_list_of_num_pairs = FALSE; + in_asterisk_or_list_of_numbers = FALSE; + in_list_of_num_ranges = FALSE; + in_single_string = FALSE; + in_single_number = FALSE; +} diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y new file mode 100755 index 0000000..946a645 --- /dev/null +++ b/opensm/opensm/osm_qos_parser.y @@ -0,0 +1,2146 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Grammar of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include + +#define OSM_QOS_POLICY_MAX_LINE_LEN 1024*10 +#define OSM_QOS_POLICY_SL2VL_TABLE_LEN IB_MAX_NUM_VLS +#define OSM_QOS_POLICY_MAX_VL_NUM IB_MAX_NUM_VLS + +typedef struct tmp_parser_struct_t_ { + char str[OSM_QOS_POLICY_MAX_LINE_LEN]; + uint64_t num_pair[2]; + cl_list_t str_list; + cl_list_t num_list; + cl_list_t num_pair_list; +} tmp_parser_struct_t; + +static void __parser_tmp_struct_init(); +static void __parser_tmp_struct_reset(); +static void __parser_tmp_struct_destroy(); + +static char * __parser_strip_white(char * str); + +static void __parser_str2uint64(uint64_t * p_val, char * str); + +static void __parser_port_group_start(); +static int __parser_port_group_end(); + +static void __parser_sl2vl_scope_start(); +static int __parser_sl2vl_scope_end(); + +static void __parser_vlarb_scope_start(); +static int __parser_vlarb_scope_end(); + +static void __parser_qos_level_start(); +static int __parser_qos_level_end(); + +static void __parser_match_rule_start(); +static int __parser_match_rule_end(); + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len); + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ); + +extern char * yytext; +extern void yyerror (char *s); +extern int yylex (void); +extern FILE * yyin; + +#define RESET_BUFFER __parser_tmp_struct_reset() + +tmp_parser_struct_t tmp_parser_struct; + +int column_num; +int line_num; + +extern osm_qos_policy_t * p_qos_policy; +osm_qos_port_group_t * p_current_port_group = NULL; +osm_qos_sl2vl_scope_t * p_current_sl2vl_scope = NULL; +osm_qos_vlarb_scope_t * p_current_vlarb_scope = NULL; +osm_qos_level_t * p_current_qos_level = NULL; +osm_qos_match_rule_t * p_current_qos_match_rule = NULL; +extern osm_log_t * p_qos_parser_osm_log; + +/***************************************************/ + +%} + +%token TK_NUMBER +%token TK_DASH +%token TK_DOTDOT +%token TK_COMMA +%token TK_ASTERISK +%token TK_TEXT + +%token TK_PORT_GROUPS_START +%token TK_PORT_GROUPS_END +%token TK_PORT_GROUP_START +%token TK_PORT_GROUP_END + +%token TK_QOS_SETUP_START +%token TK_QOS_SETUP_END +%token TK_VLARB_TABLES_START +%token TK_VLARB_TABLES_END +%token TK_VLARB_SCOPE_START +%token TK_VLARB_SCOPE_END + +%token TK_SL2VL_TABLES_START +%token TK_SL2VL_TABLES_END +%token TK_SL2VL_SCOPE_START +%token TK_SL2VL_SCOPE_END + +%token TK_QOS_LEVELS_START +%token TK_QOS_LEVELS_END +%token TK_QOS_LEVEL_START +%token TK_QOS_LEVEL_END + +%token TK_QOS_MATCH_RULES_START +%token TK_QOS_MATCH_RULES_END +%token TK_QOS_MATCH_RULE_START +%token TK_QOS_MATCH_RULE_END + +%token TK_NAME +%token TK_USE +%token TK_PORT_GUID +%token TK_PORT_NAME +%token TK_PARTITION +%token TK_NODE_TYPE +%token TK_GROUP +%token TK_ACROSS +%token TK_VLARB_HIGH +%token TK_VLARB_LOW +%token TK_VLARB_HIGH_LIMIT +%token TK_TO +%token TK_FROM +%token TK_ACROSS_TO +%token TK_ACROSS_FROM +%token TK_SL2VL_TABLE +%token TK_SL +%token TK_MTU_LIMIT +%token TK_RATE_LIMIT +%token TK_PACKET_LIFE +%token TK_PATH_BITS +%token TK_QOS_CLASS +%token TK_SOURCE +%token TK_DESTINATION +%token TK_SERVICE_ID +%token TK_QOS_LEVEL_NAME +%token TK_PKEY + +%token TK_NODE_TYPE_ROUTER +%token TK_NODE_TYPE_CA +%token TK_NODE_TYPE_SWITCH +%token TK_NODE_TYPE_SELF +%token TK_NODE_TYPE_ALL + + +%start head + +%% + +head: qos_policy_entries + ; + +qos_policy_entries: /* empty */ + | qos_policy_entries qos_policy_entry + ; + +qos_policy_entry: port_groups_section + | qos_setup_section + | qos_levels_section + | qos_match_rules_section + ; + + /* + * Parsing port groups: + * ------------------- + * port-groups + * port-group + * name: Storage + * use: our SRP storage targets + * port-guid: 0x1000000000000001,0x1000000000000002 + * ... + * port-name: vs1/HCA-1/P1 + * ... + * partition: Part1 + * ... + * node-type: ROUTER,CA,SWITCH,SELF,ALL + * ... + * end-port-group + * port-group + * ... + * end-port-group + * end-port-groups + */ + + +port_groups_section: TK_PORT_GROUPS_START port_groups TK_PORT_GROUPS_END + ; + +port_groups: port_group + | port_groups port_group + ; + +port_group: port_group_start port_group_entries port_group_end + ; + +port_group_start: TK_PORT_GROUP_START { + __parser_port_group_start(); + } + ; + +port_group_end: TK_PORT_GROUP_END { + if ( __parser_port_group_end() ) + return 1; + } + ; + +port_group_entries: /* empty */ + | port_group_entries port_group_entry + ; + +port_group_entry: port_group_name + | port_group_use + | port_group_port_guid + | port_group_port_name + | port_group_partition + | port_group_node_type + ; + + + /* + * Parsing qos setup: + * ----------------- + * qos-setup + * vlarb-tables + * vlarb-scope + * ... + * end-vlarb-scope + * vlarb-scope + * ... + * end-vlarb-scope + * end-vlarb-tables + * sl2vl-tables + * sl2vl-scope + * ... + * end-sl2vl-scope + * sl2vl-scope + * ... + * end-sl2vl-scope + * end-sl2vl-tables + * end-qos-setup + */ + +qos_setup_section: TK_QOS_SETUP_START qos_setup_items TK_QOS_SETUP_END + ; + +qos_setup_items: /* empty */ + | qos_setup_items vlarb_tables + | qos_setup_items sl2vl_tables + ; + + /* Parsing vlarb-tables */ + +vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END + +vlarb_scope_items: /* empty */ + | vlarb_scope_items vlarb_scope + ; + +vlarb_scope: vlarb_scope_start vlarb_scope_entries vlarb_scope_end + ; + +vlarb_scope_start: TK_VLARB_SCOPE_START { + __parser_vlarb_scope_start(); + } + ; + +vlarb_scope_end: TK_VLARB_SCOPE_END { + if ( __parser_vlarb_scope_end() ) + return 1; + } + ; + +vlarb_scope_entries:/* empty */ + | vlarb_scope_entries vlarb_scope_entry + ; + + /* + * vlarb-scope + * group: Storage + * ... + * across: Storage + * ... + * vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 + * vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 + * vl-high-limit: 10 + * end-vlarb-scope + */ + +vlarb_scope_entry: vlarb_scope_group + | vlarb_scope_across + | vlarb_scope_vlarb_high + | vlarb_scope_vlarb_low + | vlarb_scope_vlarb_high_limit + ; + + /* Parsing sl2vl-tables */ + +sl2vl_tables: TK_SL2VL_TABLES_START sl2vl_scope_items TK_SL2VL_TABLES_END + ; + +sl2vl_scope_items: /* empty */ + | sl2vl_scope_items sl2vl_scope + ; + +sl2vl_scope: sl2vl_scope_start sl2vl_scope_entries sl2vl_scope_end + ; + +sl2vl_scope_start: TK_SL2VL_SCOPE_START { + __parser_sl2vl_scope_start(); + } + ; + +sl2vl_scope_end: TK_SL2VL_SCOPE_END { + if ( __parser_sl2vl_scope_end() ) + return 1; + } + ; + +sl2vl_scope_entries:/* empty */ + | sl2vl_scope_entries sl2vl_scope_entry + ; + + /* + * sl2vl-scope + * group: Part1 + * ... + * from: * + * ... + * to: * + * ... + * across-to: Storage2 + * ... + * across-from: Storage1 + * ... + * sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 + * end-sl2vl-scope + */ + +sl2vl_scope_entry: sl2vl_scope_group + | sl2vl_scope_across + | sl2vl_scope_across_from + | sl2vl_scope_across_to + | sl2vl_scope_from + | sl2vl_scope_to + | sl2vl_scope_sl2vl_table + ; + + /* + * Parsing qos-levels: + * ------------------ + * qos-levels + * qos-level + * name: qos_level_1 + * use: for the lowest priority communication + * sl: 15 + * mtu-limit: 1 + * rate-limit: 1 + * packet-life: 12 + * path-bits: 2,4,8-32 + * pkey: 0x00FF-0x0FFF + * end-qos-level + * ... + * qos-level + * end-qos-level + * end-qos-levels + */ + + +qos_levels_section: TK_QOS_LEVELS_START qos_levels TK_QOS_LEVELS_END + ; + +qos_levels: /* empty */ + | qos_levels qos_level + ; + +qos_level: qos_level_start qos_level_entries qos_level_end + ; + +qos_level_start: TK_QOS_LEVEL_START { + __parser_qos_level_start(); + } + ; + +qos_level_end: TK_QOS_LEVEL_END { + if ( __parser_qos_level_end() ) + return 1; + } + ; + +qos_level_entries: /* empty */ + | qos_level_entries qos_level_entry + ; + +qos_level_entry: qos_level_name + | qos_level_use + | qos_level_sl + | qos_level_mtu_limit + | qos_level_rate_limit + | qos_level_packet_life + | qos_level_path_bits + | qos_level_pkey + ; + + /* + * Parsing qos-match-rules: + * ----------------------- + * qos-match-rules + * qos-match-rule + * use: low latency by class 7-9 or 11 and bla bla + * qos-class: 7-9,11 + * qos-level-name: default + * source: Storage + * destination: Storage + * service-id: 22,4719-5000 + * pkey: 0x00FF-0x0FFF + * end-qos-match-rule + * qos-match-rule + * ... + * end-qos-match-rule + * end-qos-match-rules + */ + +qos_match_rules_section: TK_QOS_MATCH_RULES_START qos_match_rules TK_QOS_MATCH_RULES_END + ; + +qos_match_rules: /* empty */ + | qos_match_rules qos_match_rule + ; + +qos_match_rule: qos_match_rule_start qos_match_rule_entries qos_match_rule_end + ; + +qos_match_rule_start: TK_QOS_MATCH_RULE_START { + __parser_match_rule_start(); + } + ; + +qos_match_rule_end: TK_QOS_MATCH_RULE_END { + if ( __parser_match_rule_end() ) + return 1; + } + ; + +qos_match_rule_entries: /* empty */ + | qos_match_rule_entries qos_match_rule_entry + ; + +qos_match_rule_entry: qos_match_rule_use + | qos_match_rule_qos_class + | qos_match_rule_qos_level_name + | qos_match_rule_source + | qos_match_rule_destination + | qos_match_rule_service_id + | qos_match_rule_pkey + ; + + /* + * port_group_entry values: + * port_group_name + * port_group_use + * port_group_port_guid + * port_group_port_name + * port_group_partition + * port_group_node_type + */ + +port_group_name: port_group_name_start single_string { + /* 'name' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->name) + { + yyerror("port-group has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +port_group_use: port_group_use_start single_string { + /* 'use' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->use) + { + yyerror("port-group has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_use_start: TK_USE { + RESET_BUFFER; + } + ; + +port_group_port_name: port_group_port_name_start string_list { + /* 'port-name' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + + /* + * TODO: parse port name strings + */ + + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->port_name_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_port_name_start: TK_PORT_NAME { + RESET_BUFFER; + } + ; + +port_group_port_guid: port_group_port_guid_start list_of_ranges { + /* 'port-guid' in 'port-group' - any num of instances */ + /* list of guid ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_port_group->guid_range_len ) + { + p_current_port_group->guid_range_arr = range_arr; + p_current_port_group->guid_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_port_group->guid_range_arr, + p_current_port_group->guid_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_port_group->guid_range_arr = new_range_arr; + p_current_port_group->guid_range_len = new_range_len; + } + } + } + ; + +port_group_port_guid_start: TK_PORT_GUID { + RESET_BUFFER; + } + ; + +port_group_partition: port_group_partition_start string_list { + /* 'partition' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->partition_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_partition_start: TK_PARTITION { + RESET_BUFFER; + } + ; + +port_group_node_type: port_group_node_type_start port_group_node_type_list { + /* 'node-type' in 'port-group' - any num of instances */ + } + ; + +port_group_node_type_start: TK_NODE_TYPE { + RESET_BUFFER; + } + ; + +port_group_node_type_list: node_type_item + | port_group_node_type_list TK_COMMA node_type_item + ; + +node_type_item: node_type_ca + | node_type_switch + | node_type_router + | node_type_all + | node_type_self + ; + +node_type_ca: TK_NODE_TYPE_CA { + p_current_port_group->node_type_ca = TRUE;; + } + ; + +node_type_switch: TK_NODE_TYPE_SWITCH { + p_current_port_group->node_type_switch = TRUE; + } + ; + +node_type_router: TK_NODE_TYPE_ROUTER { + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_all: TK_NODE_TYPE_ALL { + p_current_port_group->node_type_ca = TRUE; + p_current_port_group->node_type_switch = TRUE; + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_self: TK_NODE_TYPE_SELF { + p_current_port_group->node_type_self = TRUE; + } + ; + + /* + * vlarb_scope_entry values: + * vlarb_scope_group + * vlarb_scope_across + * vlarb_scope_vlarb_high + * vlarb_scope_vlarb_low + * vlarb_scope_vlarb_high_limit + */ + + + +vlarb_scope_group: vlarb_scope_group_start string_list { + /* 'group' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +vlarb_scope_across: vlarb_scope_across_start string_list { + /* 'across' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->across_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high_limit: vlarb_scope_vlarb_high_limit_start single_number { + /* 'vl-high-limit' in 'vlarb-scope' - one instance of one number */ + cl_list_iterator_t list_iterator; + uint64_t * p_tmp_num; + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_tmp_num = (uint64_t*)cl_list_obj(list_iterator); + if (p_tmp_num) + { + p_current_vlarb_scope->vl_high_limit = (uint32_t)(*p_tmp_num); + p_current_vlarb_scope->vl_high_limit_set = TRUE; + free(p_tmp_num); + } + + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +vlarb_scope_vlarb_high_limit_start: TK_VLARB_HIGH_LIMIT { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high: vlarb_scope_vlarb_high_start num_list_with_dotdot { + /* 'vlarb-high' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_high_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_high_start: TK_VLARB_HIGH { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_low: vlarb_scope_vlarb_low_start num_list_with_dotdot { + /* 'vlarb-low' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_low_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_low_start: TK_VLARB_LOW { + RESET_BUFFER; + } + ; + + /* + * sl2vl_scope_entry values: + * sl2vl_scope_group + * sl2vl_scope_across + * sl2vl_scope_across_from + * sl2vl_scope_across_to + * sl2vl_scope_from + * sl2vl_scope_to + * sl2vl_scope_sl2vl_table + */ + +sl2vl_scope_group: sl2vl_scope_group_start string_list { + /* 'group' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +sl2vl_scope_across: sl2vl_scope_across_start string_list { + /* 'across' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,strdup(tmp_str)); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_from: sl2vl_scope_across_from_start string_list { + /* 'across-from' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_from_start: TK_ACROSS_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_to: sl2vl_scope_across_to_start string_list { + /* 'across-to' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,tmp_str); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_to_start: TK_ACROSS_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from: sl2vl_scope_from_start sl2vl_scope_from_list_or_asterisk { + /* 'from' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_from_start: TK_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_to: sl2vl_scope_to_start sl2vl_scope_to_list_or_asterisk { + /* 'to' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_to_start: TK_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from_list_or_asterisk: sl2vl_scope_from_asterisk + | sl2vl_scope_from_list_of_ranges + ; + +sl2vl_scope_from_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + ; + +sl2vl_scope_to_list_or_asterisk: sl2vl_scope_to_asterisk + | sl2vl_scope_to_list_of_ranges + ; + +sl2vl_scope_to_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + ; + +sl2vl_scope_from_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + yyerror("port number out of range 'from' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +sl2vl_scope_to_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + yyerror("port number out of range 'to' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + + +sl2vl_scope_sl2vl_table: sl2vl_scope_sl2vl_table_start num_list { + /* 'sl2vl-table' - one instance of exactly + OSM_QOS_POLICY_SL2VL_TABLE_LEN numbers */ + cl_list_iterator_t list_iterator; + uint64_t num; + uint64_t * p_num; + int i = 0; + + if (p_current_sl2vl_scope->sl2vl_table_set) + { + yyerror("sl2vl-scope has more than one sl2vl-table"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + if (cl_list_count(&tmp_parser_struct.num_list) != OSM_QOS_POLICY_SL2VL_TABLE_LEN) + { + yyerror("wrong number of values in 'sl2vl-table' (should be 16)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_list) ) + { + p_num = (uint64_t*)cl_list_obj(list_iterator); + num = *p_num; + free(p_num); + if (num >= OSM_QOS_POLICY_MAX_VL_NUM) + { + yyerror("wrong VL value in 'sl2vl-table' (should be 0 to 15)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + p_current_sl2vl_scope->sl2vl_table[i++] = (uint8_t)num; + list_iterator = cl_list_next(list_iterator); + } + p_current_sl2vl_scope->sl2vl_table_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +sl2vl_scope_sl2vl_table_start: TK_SL2VL_TABLE { + RESET_BUFFER; + } + ; + + /* + * qos_level_entry values: + * qos_level_name + * qos_level_use + * qos_level_sl + * qos_level_mtu_limit + * qos_level_rate_limit + * qos_level_packet_life + * qos_level_path_bits + * qos_level_pkey + */ + +qos_level_name: qos_level_name_start single_string { + /* 'name' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->name) + { + yyerror("qos-level has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +qos_level_use: qos_level_use_start single_string { + /* 'use' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->use) + { + yyerror("qos-level has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_level_sl: qos_level_sl_start single_number { + /* 'sl' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->sl_set) + { + yyerror("'qos-level' has multiple 'sl' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->sl = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->sl_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_sl_start: TK_SL { + RESET_BUFFER; + } + ; + +qos_level_mtu_limit: qos_level_mtu_limit_start single_number { + /* 'mtu-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->mtu_limit_set) + { + yyerror("'qos-level' has multiple 'mtu-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->mtu_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->mtu_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_mtu_limit_start: TK_MTU_LIMIT { + /* 'mtu-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_rate_limit: qos_level_rate_limit_start single_number { + /* 'rate-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->rate_limit_set) + { + yyerror("'qos-level' has multiple 'rate-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->rate_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->rate_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_rate_limit_start: TK_RATE_LIMIT { + /* 'rate-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_packet_life: qos_level_packet_life_start single_number { + /* 'packet-life' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->pkt_life_set) + { + yyerror("'qos-level' has multiple 'packet-life' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->pkt_life = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->pkt_life_set= TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_packet_life_start: TK_PACKET_LIFE { + /* 'packet-life' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_path_bits: qos_level_path_bits_start list_of_ranges { + /* 'path-bits' in 'qos-level' - any num of instances */ + /* list of path bit ranges */ + + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->path_bits_range_len ) + { + p_current_qos_level->path_bits_range_arr = range_arr; + p_current_qos_level->path_bits_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->path_bits_range_arr, + p_current_qos_level->path_bits_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->path_bits_range_arr = new_range_arr; + p_current_qos_level->path_bits_range_len = new_range_len; + } + } + } + ; + +qos_level_path_bits_start: TK_PATH_BITS { + RESET_BUFFER; + } + ; + +qos_level_pkey: qos_level_pkey_start list_of_ranges { + /* 'pkey' in 'qos-level' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->pkey_range_len ) + { + p_current_qos_level->pkey_range_arr = range_arr; + p_current_qos_level->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->pkey_range_arr, + p_current_qos_level->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->pkey_range_arr = new_range_arr; + p_current_qos_level->pkey_range_len = new_range_len; + } + } + } + ; + +qos_level_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + /* + * qos_match_rule_entry values: + * qos_match_rule_use + * qos_match_rule_qos_class + * qos_match_rule_qos_level_name + * qos_match_rule_source + * qos_match_rule_destination + * qos_match_rule_service_id + * qos_match_rule_pkey + */ + + +qos_match_rule_use: qos_match_rule_use_start single_string { + /* 'use' of 'qos-match-rule' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->use) + { + yyerror("'qos-match-rule' has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_class: qos_match_rule_qos_class_start list_of_ranges { + /* 'qos-class' in 'qos-match-rule' - num of instances of list of ranges */ + /* list of class ranges (QoS Class is 12-bit value) */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->qos_class_range_len ) + { + p_current_qos_match_rule->qos_class_range_arr = range_arr; + p_current_qos_match_rule->qos_class_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->qos_class_range_arr, + p_current_qos_match_rule->qos_class_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->qos_class_range_arr = new_range_arr; + p_current_qos_match_rule->qos_class_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_qos_class_start: TK_QOS_CLASS { + RESET_BUFFER; + } + ; + +qos_match_rule_source: qos_match_rule_source_start string_list { + /* 'source' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->source_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_source_start: TK_SOURCE { + RESET_BUFFER; + } + ; + +qos_match_rule_destination: qos_match_rule_destination_start string_list { + /* 'destination' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->destination_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_destination_start: TK_DESTINATION { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_level_name: qos_match_rule_qos_level_name_start single_string { + /* 'qos-level-name' in 'qos-match-rule' - single string */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->qos_level_name) + { + yyerror("qos-match-rule has multiple 'qos-level-name' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->qos_level_name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_qos_level_name_start: TK_QOS_LEVEL_NAME { + RESET_BUFFER; + } + ; + +qos_match_rule_service_id: qos_match_rule_service_id_start list_of_ranges { + /* 'service-id' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->service_id_range_len ) + { + p_current_qos_match_rule->service_id_range_arr = range_arr; + p_current_qos_match_rule->service_id_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->service_id_range_arr, + p_current_qos_match_rule->service_id_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->service_id_range_arr = new_range_arr; + p_current_qos_match_rule->service_id_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_service_id_start: TK_SERVICE_ID { + RESET_BUFFER; + } + ; + +qos_match_rule_pkey: qos_match_rule_pkey_start list_of_ranges { + /* 'pkey' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->pkey_range_len ) + { + p_current_qos_match_rule->pkey_range_arr = range_arr; + p_current_qos_match_rule->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->pkey_range_arr, + p_current_qos_match_rule->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->pkey_range_arr = new_range_arr; + p_current_qos_match_rule->pkey_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + + /* + * Common part + */ + + +single_string: single_string_elems { + cl_list_insert_tail(&tmp_parser_struct.str_list, + strdup(__parser_strip_white(tmp_parser_struct.str))); + tmp_parser_struct.str[0] = '\0'; + } + ; + +single_string_elems: single_string_element + | single_string_elems single_string_element + ; + +single_string_element: TK_TEXT { + strcat(tmp_parser_struct.str,$1); + free($1); + } + ; + + +string_list: single_string + | string_list TK_COMMA single_string + ; + + + +single_number: number + ; + +num_list: number + | num_list TK_COMMA number + ; + +number: TK_NUMBER { + uint64_t * p_num = (uint64_t*)malloc(sizeof(uint64_t)); + __parser_str2uint64(p_num,$1); + free($1); + cl_list_insert_tail(&tmp_parser_struct.num_list, p_num); + } + ; + +num_list_with_dotdot: number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dotdot TK_COMMA number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +number_from_pair_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_pair_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +list_of_ranges: num_list_with_dash + ; + +num_list_with_dash: single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +single_number_from_range: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +number_from_range_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_range_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +%% + +/*************************************************** + ***************************************************/ + +int osm_qos_parse_policy_file( + IN osm_log_t * p_log, + IN const char * policy_file) +{ + int res = 0; + p_qos_parser_osm_log = p_log; + + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_parse_policy_file); + + p_qos_policy = NULL; + yyin = fopen (policy_file, "r"); + if (!yyin) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC01: " + "Failed opening QoS policy file (%s)\n", + policy_file); + res = 1; + goto Exit; + } + column_num = 1; + line_num = 1; + osm_qos_policy_create(); + __parser_tmp_struct_init(); + + res = yyparse(); + + __parser_tmp_struct_destroy(); + + if (res != 0) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC02: " + "Failed parsing QoS policy file (%s)\n", + policy_file); + p_qos_policy = NULL; + res = 1; + goto Exit; + } + + if (osm_qos_policy_validate()) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC03: " + "Error(s) in QoS policy file (%s)\n", + policy_file); + res = 1; + goto Exit; + } + + Exit: + if (yyin) + fclose(yyin); + OSM_LOG_EXIT(p_qos_parser_osm_log); + return res; +} + +/*************************************************** + ***************************************************/ + +int yywrap() +{ + return(1); +} + +/*************************************************** + ***************************************************/ + +void yyerror (char *s) +{ + OSM_LOG_ENTER(p_qos_parser_osm_log, yyerror); + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "yyerror: ERR AC03: " + "Syntax error (line %d:%d): %s. " + "Last text read: \"%s\"\n", + line_num, column_num, s, __parser_strip_white(yytext)); + OSM_LOG_EXIT(p_qos_parser_osm_log); +} + +/*************************************************** + ***************************************************/ + +static char * __parser_strip_white(char * str) +{ + int i; + for (i = (strlen(str)-1); i >= 0; i--) + { + if (isspace(str[i])) + str[i] = '\0'; + else + break; + } + for (i = 0; i < strlen(str); i++) + { + if (!isspace(str[i])) + break; + } + return &(str[i]); +} + +/*************************************************** + ***************************************************/ + +static void __parser_str2uint64(uint64_t * p_val, char * str) +{ +#if __WORDSIZE == 64 + *p_val = strtoul(str, NULL, 0); +#else + *p_val = strtoull(str, NULL, 0); +#endif +} + +/*************************************************** + ***************************************************/ + +static void __parser_port_group_start() +{ + p_current_port_group = osm_qos_policy_port_group_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_port_group_end() +{ + if(!p_current_port_group->name) + { + yyerror("port-group validation failed - no port group name specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->port_groups, + p_current_port_group); + p_current_port_group = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_vlarb_scope_start() +{ + p_current_vlarb_scope = osm_qos_policy_vlarb_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_vlarb_scope_end() +{ + if ( !cl_list_count(&p_current_vlarb_scope->group_list) && + !cl_list_count(&p_current_vlarb_scope->across_list) ) + { + yyerror("vlarb-scope validation failed - no port groups specified by 'group' or by 'across'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->vlarb_tables, + p_current_vlarb_scope); + p_current_vlarb_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_sl2vl_scope_start() +{ + p_current_sl2vl_scope = osm_qos_policy_sl2vl_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_sl2vl_scope_end() +{ + if (!p_current_sl2vl_scope->sl2vl_table_set) + { + yyerror("sl2vl-scope validation failed - no sl2vl table specified"); + return -1; + } + if ( !cl_list_count(&p_current_sl2vl_scope->group_list) && + !cl_list_count(&p_current_sl2vl_scope->across_to_list) && + !cl_list_count(&p_current_sl2vl_scope->across_from_list) ) + { + yyerror("sl2vl-scope validation failed - no port groups specified by 'group', 'across-to' or 'across-from'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->sl2vl_tables, + p_current_sl2vl_scope); + p_current_sl2vl_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_qos_level_start() +{ + p_current_qos_level = osm_qos_policy_qos_level_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_qos_level_end() +{ + if (!p_current_qos_level->sl_set) + { + yyerror("qos-level validation failed - no 'sl' specified"); + return -1; + } + if (!p_current_qos_level->name) + { + yyerror("qos-level validation failed - no 'name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_levels, + p_current_qos_level); + p_current_qos_level = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_match_rule_start() +{ + p_current_qos_match_rule = osm_qos_policy_match_rule_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_match_rule_end() +{ + if (!p_current_qos_match_rule->qos_level_name) + { + yyerror("match-rule validation failed - no 'qos-level-name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_match_rules, + p_current_qos_match_rule); + p_current_qos_match_rule = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_init() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_construct(&tmp_parser_struct.str_list); + cl_list_init(&tmp_parser_struct.str_list, 10); + cl_list_construct(&tmp_parser_struct.num_list); + cl_list_init(&tmp_parser_struct.num_list, 10); + cl_list_construct(&tmp_parser_struct.num_pair_list); + cl_list_init(&tmp_parser_struct.num_pair_list, 10); +} + +/*************************************************** + ***************************************************/ + +/* + * Do NOT free objects from the temp struct. + * Either they are inserted into the parse tree data + * structure, or they are already freed when copying + * their values to the parse tree data structure. + */ +static void __parser_tmp_struct_reset() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_remove_all(&tmp_parser_struct.str_list); + cl_list_remove_all(&tmp_parser_struct.num_list); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_destroy() +{ + __parser_tmp_struct_reset(); + cl_list_destroy(&tmp_parser_struct.str_list); + cl_list_destroy(&tmp_parser_struct.num_list); + cl_list_destroy(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static int OSM_CDECL +__cmp_num_range( + const void * p1, + const void * p2) +{ + uint64_t * pair1 = *((uint64_t **)p1); + uint64_t * pair2 = *((uint64_t **)p2); + + if (pair1[0] < pair2[0]) + return -1; + if (pair1[0] > pair2[0]) + return 1; + + if (pair1[1] < pair2[1]) + return -1; + if (pair1[1] > pair2[1]) + return 1; + + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __sort_reduce_rangearr( + uint64_t ** arr, + unsigned arr_len, + uint64_t ** * p_res_arr, + unsigned * p_res_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned last_valid_ind = 0; + unsigned valid_cnt = 0; + uint64_t ** res_arr; + boolean_t * is_valir_arr; + + *p_res_arr = NULL; + *p_res_arr_len = 0; + + qsort(arr, arr_len, sizeof(uint64_t*), __cmp_num_range); + + is_valir_arr = (boolean_t *)malloc(arr_len * sizeof(boolean_t)); + is_valir_arr[last_valid_ind] = TRUE; + valid_cnt++; + for (i = 1; i < arr_len; i++) + { + if (arr[i][0] <= arr[last_valid_ind][1]) + { + if (arr[i][1] > arr[last_valid_ind][1]) + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else if ((arr[i][0] - 1) == arr[last_valid_ind][1]) + { + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else + { + is_valir_arr[i] = TRUE; + last_valid_ind = i; + valid_cnt++; + } + } + + res_arr = (uint64_t **)malloc(valid_cnt * sizeof(uint64_t *)); + for (i = 0; i < arr_len; i++) + { + if (is_valir_arr[i]) + res_arr[j++] = arr[i]; + } + free(arr); + + *p_res_arr = res_arr; + *p_res_arr_len = valid_cnt; +} + +/*************************************************** + ***************************************************/ + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len) +{ + cl_list_iterator_t list_iterator; + unsigned len = cl_list_count(p_list); + unsigned i = 0; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + list_iterator = cl_list_head(p_list); + while( list_iterator != cl_list_end(p_list) ) + { + tmp_arr[i++] = (uint64_t *)cl_list_obj(list_iterator); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(p_list); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned len = range_len_1 + range_len_2; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + *p_arr = NULL; + *p_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + for (i = 0; i < range_len_1; i++) + tmp_arr[j++] = range_arr_1[i]; + for (i = 0; i < range_len_2; i++) + tmp_arr[j++] = range_arr_2[i]; + free(range_arr_1); + free(range_arr_2); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:06:19 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:06:19 +0300 Subject: [ofa-general] [PATCH 3/7] osm: QoS policy C & H files Message-ID: <46C983BB.4060007@dev.mellanox.co.il> QoS policy data structures and functions Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_qos_policy.h | 193 +++++++ opensm/opensm/osm_qos_policy.c | 916 ++++++++++++++++++++++++++++++++ 2 files changed, 1109 insertions(+), 0 deletions(-) create mode 100644 opensm/include/opensm/osm_qos_policy.h create mode 100644 opensm/opensm/osm_qos_policy.c diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h new file mode 100644 index 0000000..0908393 --- /dev/null +++ b/opensm/include/opensm/osm_qos_policy.h @@ -0,0 +1,193 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Declaration of OSM QoS Policy data types and functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#ifndef OSM_QOS_POLICY_H +#define OSM_QOS_POLICY_H + +#include +#include +#include +#include +#include +#include +#include +#include + +#define YYSTYPE char * +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" + +/***************************************************/ + +typedef struct osm_qos_port_group_t_ { + char * name; /* single string (this port group name) */ + char * use; /* single string (description) */ + cl_list_t port_name_list; /* list of port names (.../.../...) */ + uint64_t ** guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ + unsigned guid_range_len; /* num of guid ranges in the array */ + cl_list_t partition_list; /* list of partition names */ + boolean_t node_type_ca; + boolean_t node_type_switch; + boolean_t node_type_router; + boolean_t node_type_self; +} osm_qos_port_group_t; + +osm_qos_port_group_t * osm_qos_policy_port_group_create(); +void osm_qos_policy_port_group_destroy(); + +/***************************************************/ + +typedef struct osm_qos_vlarb_scope_t_ { + cl_list_t group_list; /* list of group names (strings) */ + cl_list_t across_list; /* list of 'across' group names (strings) */ + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ + uint32_t vl_high_limit; /* single integer */ + boolean_t vl_high_limit_set; +} osm_qos_vlarb_scope_t; + +osm_qos_vlarb_scope_t * osm_qos_policy_vlarb_scope_create(); +void osm_qos_policy_vlarb_scope_destroy(); + +/***************************************************/ + +typedef struct osm_qos_sl2vl_scope_t_ { + cl_list_t group_list; /* list of strings (port group names) */ + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + cl_list_t across_from_list; /* list of strings (port group names) */ + cl_list_t across_to_list; /* list of strings (port group names) */ + uint8_t sl2vl_table[16]; /* array of sl2vl values */ + boolean_t sl2vl_table_set; +} osm_qos_sl2vl_scope_t; + +osm_qos_sl2vl_scope_t * osm_qos_policy_sl2vl_scope_create(); +void osm_qos_policy_sl2vl_scope_destroy(); + +/***************************************************/ + +typedef struct osm_qos_level_t_ { + char * use; + char * name; + uint8_t sl; + boolean_t sl_set; + uint8_t mtu_limit; + boolean_t mtu_limit_set; + uint8_t rate_limit; + boolean_t rate_limit_set; + uint8_t pkt_life; + boolean_t pkt_life_set; + uint64_t ** path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ + unsigned path_bits_range_len; /* num of bit ranges in the array */ + uint64_t ** pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_level_t; + +osm_qos_level_t * osm_qos_policy_qos_level_create(); +void osm_qos_policy_qos_level_destroy(); + +boolean_t osm_qos_level_has_pkey( + IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey); + +ib_net16_t osm_qos_level_get_shared_pkey( + IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp); + +/***************************************************/ + +typedef struct osm_qos_match_rule_t_ { + char * use; + cl_list_t source_list; /* list of strings */ + cl_list_t source_group_list; /* list of pointers to relevant port-group */ + cl_list_t destination_list; /* list of strings */ + cl_list_t destination_group_list;/* list of pointers to relevant port-group */ + char * qos_level_name; + osm_qos_level_t * p_qos_level; + uint64_t ** service_id_range_arr; /* array of SID ranges (64-bit values) */ + unsigned service_id_range_len; + uint64_t ** qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ + unsigned qos_class_range_len; + uint64_t ** pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_match_rule_t; + +osm_qos_match_rule_t * osm_qos_policy_match_rule_create(); +void osm_qos_policy_match_rule_destroy(); + +/***************************************************/ + +typedef struct osm_qos_policy_t_ { + cl_list_t port_groups; /* list of osm_qos_port_group_t */ + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ + cl_list_t qos_levels; /* list of osm_qos_level_t */ + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ + osm_qos_level_t * p_default_qos_level; /* default QoS level */ +} osm_qos_policy_t; + +void osm_qos_policy_create(); +void osm_qos_policy_destroy(); +int osm_qos_policy_validate(); + +void osm_qos_policy_get_qos_level_by_pr( + IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level); + +/***************************************************/ + +int osm_qos_parse_policy_file( + IN osm_log_t * p_log, + IN const char * policy_file); + +/***************************************************/ + +#endif /* ifndef OSM_QOS_POLICY_H */ diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c new file mode 100644 index 0000000..0867089 --- /dev/null +++ b/opensm/opensm/osm_qos_policy.c @@ -0,0 +1,916 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * OSM QoS Policy functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include +#include + +extern void yyerror (char *s); +osm_log_t * p_qos_parser_osm_log = NULL; +osm_qos_policy_t * p_qos_policy = NULL; + +/*************************************************** + ***************************************************/ + +static boolean_t +__is_num_in_range_arr( + uint64_t ** range_arr, + unsigned range_arr_len, + uint64_t num) +{ + unsigned i; + + /* + * TODO: although this array should be rather short, + * I should implement a binary search here + */ + + if (num < range_arr[0][0] || num > range_arr[range_arr_len-1][1]) + return FALSE; + + for (i = 0; i < range_arr_len; i++) + { + if (num >= range_arr[i][0] && num <= range_arr[i][1]) + return TRUE; + if (num < range_arr[i][0]) + return FALSE; + } + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static void +__free_single_element( + void * p_element, + void * context) +{ + if (p_element) + free(p_element); +} + +/*************************************************** + ***************************************************/ + +osm_qos_port_group_t * osm_qos_policy_port_group_create() +{ + osm_qos_port_group_t * p = (osm_qos_port_group_t *)malloc(sizeof(osm_qos_port_group_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_port_group_t)); + + cl_list_init(&p->port_name_list,10); + cl_list_init(&p->partition_list,10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->name) + free(p->name); + if (p->use) + free(p->use); + + for (i = 0; i < p->guid_range_len; i++) + free(p->guid_range_arr[i]); + if (p->guid_range_arr) + free(p->guid_range_arr); + + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); + cl_list_remove_all(&p->port_name_list); + cl_list_destroy(&p->port_name_list); + + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); + cl_list_remove_all(&p->partition_list); + cl_list_destroy(&p->partition_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_vlarb_scope_t * osm_qos_policy_vlarb_scope_create() +{ + osm_qos_vlarb_scope_t * p = (osm_qos_vlarb_scope_t *)malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list,10); + cl_list_init(&p->across_list,10); + cl_list_init(&p->vlarb_high_list,10); + cl_list_init(&p->vlarb_low_list,10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_list); + cl_list_remove_all(&p->vlarb_high_list); + cl_list_remove_all(&p->vlarb_low_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_list); + cl_list_destroy(&p->vlarb_high_list); + cl_list_destroy(&p->vlarb_low_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_sl2vl_scope_t * osm_qos_policy_sl2vl_scope_create() +{ + osm_qos_sl2vl_scope_t* p = (osm_qos_sl2vl_scope_t *)malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list,10); + cl_list_init(&p->across_from_list,10); + cl_list_init(&p->across_to_list,10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_from_list); + cl_list_remove_all(&p->across_to_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_from_list); + cl_list_destroy(&p->across_to_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_level_t * osm_qos_policy_qos_level_create() +{ + osm_qos_level_t * p = (osm_qos_level_t *)malloc(sizeof(osm_qos_level_t)); + if (!p) + return NULL; + memset(p, 0, sizeof(osm_qos_level_t)); + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->use) + free(p->use); + + for (i = 0; i < p->path_bits_range_len; i++) + free(p->path_bits_range_arr[i]); + if (p->path_bits_range_arr) + free(p->path_bits_range_arr); + + free(p); +} + +/*************************************************** + ***************************************************/ + +boolean_t osm_qos_level_has_pkey( + IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey) +{ + if (!p_qos_level || !p_qos_level->pkey_range_len) + return FALSE; + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, + p_qos_level->pkey_range_len, + cl_ntoh16(pkey)); +} + +/*************************************************** + ***************************************************/ + +ib_net16_t osm_qos_level_get_shared_pkey( + IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp) +{ + unsigned i; + uint16_t pkey_ho = 0; + + if (!p_qos_level || !p_qos_level->pkey_range_len) + return 0; + + /* + * ToDo: This approach is not optimal. + * Think how to find shared pkey that also exists + * in QoS level in less runtime. + */ + + for ( i = 0; i < p_qos_level->pkey_range_len; i++ ) + { + for ( pkey_ho = p_qos_level->pkey_range_arr[i][0]; + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; + pkey_ho++ ) + { + if( osm_physp_share_this_pkey(p_src_physp, p_dest_physp, cl_hton16(pkey_ho)) ) + return cl_hton16(pkey_ho); + } + } + + return 0; +} + +/*************************************************** + ***************************************************/ + +osm_qos_match_rule_t * osm_qos_policy_match_rule_create() +{ + osm_qos_match_rule_t *p = (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_match_rule_t)); + + cl_list_init(&p->source_list,10); + cl_list_init(&p->source_group_list,10); + cl_list_init(&p->destination_list,10); + cl_list_init(&p->destination_group_list,10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->qos_level_name) + free(p->qos_level_name); + if (p->use) + free(p->use); + + for (i = 0; i < p->service_id_range_len; i++) + free(p->service_id_range_arr[i]); + if (p->service_id_range_arr) + free(p->service_id_range_arr); + + for (i = 0; i < p->qos_class_range_len; i++) + free(p->qos_class_range_arr[i]); + if (p->qos_class_range_arr) + free(p->qos_class_range_arr); + + cl_list_apply_func(&p->source_list, __free_single_element, NULL); + cl_list_remove_all(&p->source_list); + cl_list_destroy(&p->source_list); + + cl_list_remove_all(&p->source_group_list); + cl_list_destroy(&p->source_group_list); + + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); + cl_list_remove_all(&p->destination_list); + cl_list_destroy(&p->destination_list); + + cl_list_remove_all(&p->destination_group_list); + cl_list_destroy(&p->destination_group_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_create() +{ + if (p_qos_policy) + osm_qos_policy_destroy(); + + p_qos_policy = (osm_qos_policy_t *)malloc(sizeof(osm_qos_policy_t)); + CL_ASSERT(p_qos_policy); + + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); + + cl_list_construct(&p_qos_policy->port_groups); + cl_list_init(&p_qos_policy->port_groups, 10); + + cl_list_construct(&p_qos_policy->vlarb_tables); + cl_list_init(&p_qos_policy->vlarb_tables, 10); + + cl_list_construct(&p_qos_policy->sl2vl_tables); + cl_list_init(&p_qos_policy->sl2vl_tables, 10); + + cl_list_construct(&p_qos_policy->qos_levels); + cl_list_init(&p_qos_policy->qos_levels, 10); + + cl_list_construct(&p_qos_policy->qos_match_rules); + cl_list_init(&p_qos_policy->qos_match_rules, 10); +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_destroy() +{ + cl_list_iterator_t list_iterator; + osm_qos_port_group_t * p_port_group = NULL; + osm_qos_vlarb_scope_t * p_vlarb_scope = NULL; + osm_qos_sl2vl_scope_t * p_sl2vl_scope = NULL; + osm_qos_level_t * p_qos_level = NULL; + osm_qos_match_rule_t * p_qos_match_rule = NULL; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while( list_iterator != cl_list_end(&p_qos_policy->port_groups) ) + { + p_port_group = (osm_qos_port_group_t*)cl_list_obj(list_iterator); + if (p_port_group) + osm_qos_policy_port_group_destroy(p_port_group); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->port_groups); + cl_list_destroy(&p_qos_policy->port_groups); + + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); + while( list_iterator != cl_list_end(&p_qos_policy->vlarb_tables) ) + { + p_vlarb_scope = (osm_qos_vlarb_scope_t*)cl_list_obj(list_iterator); + if (p_vlarb_scope) + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->vlarb_tables); + cl_list_destroy(&p_qos_policy->vlarb_tables); + + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); + while( list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables) ) + { + p_sl2vl_scope = (osm_qos_sl2vl_scope_t*)cl_list_obj(list_iterator); + if (p_sl2vl_scope) + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->sl2vl_tables); + cl_list_destroy(&p_qos_policy->sl2vl_tables); + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while( list_iterator != cl_list_end(&p_qos_policy->qos_levels) ) + { + p_qos_level = (osm_qos_level_t*)cl_list_obj(list_iterator); + if (p_qos_level) + osm_qos_policy_qos_level_destroy(p_qos_level); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_levels); + cl_list_destroy(&p_qos_policy->qos_levels); + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while( list_iterator != cl_list_end(&p_qos_policy->qos_match_rules) ) + { + p_qos_match_rule = (osm_qos_match_rule_t*)cl_list_obj(list_iterator); + if (p_qos_match_rule) + osm_qos_policy_match_rule_destroy(p_qos_match_rule); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_match_rules); + cl_list_destroy(&p_qos_policy->qos_match_rules); + + free(p_qos_policy); + + p_qos_policy = NULL; +} + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group( + osm_subn_t * p_subn, + const osm_physp_t * p_physp, + osm_qos_port_group_t * p_port_group) +{ + osm_node_t * p_node = osm_physp_get_node_ptr(p_physp); + osm_prtn_t * p_prtn = NULL; + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); + uint64_t port_guid_ho = cl_ntoh64(port_guid); + uint8_t node_type = osm_node_get_type(p_node); + cl_list_iterator_t list_iterator; + char * partition_name; + + /* check whether this port's type matches any of group's types */ + + if ( (node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) || + (node_type == IB_NODE_TYPE_ROUTER && p_port_group->node_type_router) ) + return TRUE; + + /* check whether this port's guid is in range of this group's guids */ + + if ( __is_num_in_range_arr(p_port_group->guid_range_arr, + p_port_group->guid_range_len, + port_guid_ho) ) + return TRUE; + + + /* check whether this port is member of this group's partitions */ + + list_iterator = cl_list_head(&p_port_group->partition_list); + while( list_iterator != cl_list_end(&p_port_group->partition_list) ) + { + partition_name = (char*)cl_list_obj(list_iterator); + if (partition_name && strlen(partition_name)) + { + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); + if (p_prtn) + { + if (osm_prtn_is_guid(p_prtn, port_guid)) + return TRUE; + } + } + list_iterator = cl_list_next(list_iterator); + } + + /* check whether this port's name matches any of group's names */ + + /* + * TODO: check port names + * + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; + */ + + return FALSE; +} /* __qos_policy_is_port_in_group() */ + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group_list( + const osm_pr_rcv_t * p_rcv, + const osm_physp_t * p_physp, + cl_list_t * p_port_group_list) +{ + osm_qos_port_group_t * p_port_group; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(p_port_group_list); + while( list_iterator != cl_list_end(p_port_group_list) ) + { + p_port_group = (osm_qos_port_group_t *)cl_list_obj(list_iterator); + if (p_port_group) + { + if ( __qos_policy_is_port_in_group(p_rcv->p_subn, p_physp, p_port_group) ) + return TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_match_rule_t * +__qos_policy_get_match_rule_by_pr( + const osm_pr_rcv_t * p_rcv, + const ib_path_rec_t * p_pr, + const osm_physp_t * p_src_physp, + const osm_physp_t * p_dest_physp, + ib_net64_t comp_mask) +{ + osm_qos_match_rule_t * p_qos_match_rule = NULL; + cl_list_iterator_t list_iterator; + + if (!cl_list_count(&p_qos_policy->qos_match_rules)) + return NULL; + + /* Go over all QoS match rules and find the one that matches the request */ + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while( list_iterator != cl_list_end(&p_qos_policy->qos_match_rules) ) + { + p_qos_match_rule = (osm_qos_match_rule_t*)cl_list_obj(list_iterator); + if (!p_qos_match_rule) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + /* If a match rule has Source groups, PR request source has to be in this list */ + + if ( cl_list_count(&p_qos_match_rule->source_group_list) ) + { + if ( !__qos_policy_is_port_in_group_list(p_rcv, + p_src_physp, + &p_qos_match_rule->source_group_list) ) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has Destination groups, PR request dest. has to be in this list */ + + if ( cl_list_count(&p_qos_match_rule->destination_group_list) ) + { + if ( !__qos_policy_is_port_in_group_list(p_rcv, + p_dest_physp, + &p_qos_match_rule->destination_group_list) ) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has QoS classes, PR request HAS + to have a matching QoS class to match the rule */ + + if (p_qos_match_rule->qos_class_range_len) + { + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (! __is_num_in_range_arr(p_qos_match_rule->qos_class_range_arr, + p_qos_match_rule->qos_class_range_len, + ib_path_rec_qos_class(p_pr)) ) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has Service IDs, PR request HAS + to have a matching Service ID to match the rule */ + + if (p_qos_match_rule->service_id_range_len) + { + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (! __is_num_in_range_arr(p_qos_match_rule->service_id_range_arr, + p_qos_match_rule->service_id_range_len, + p_pr->service_id) ) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has PKeys, PR request HAS + to have a matching PKey to match the rule */ + + if (p_qos_match_rule->pkey_range_len) + { + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (! __is_num_in_range_arr(p_qos_match_rule->pkey_range_arr, + p_qos_match_rule->pkey_range_len, + ib_path_rec_qos_class(p_pr)) ) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* if we got here, then this match-rule matched this PR request */ + break; + } + + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) + return NULL; + + return p_qos_match_rule; +} /* __qos_policy_get_match_rule_by_pr() */ + +/*************************************************** + ***************************************************/ + +static osm_qos_level_t * +__qos_policy_get_qos_level_by_name( + char * name) +{ + osm_qos_level_t * p_qos_level = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while( list_iterator != cl_list_end(&p_qos_policy->qos_levels) ) + { + p_qos_level = (osm_qos_level_t*)cl_list_obj(list_iterator); + if (!p_qos_level) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name,p_qos_level->name) == 0) + return p_qos_level; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_port_group_t * +__qos_policy_get_port_group_by_name( + const char * const name) +{ + osm_qos_port_group_t * p_port_group = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while( list_iterator != cl_list_end(&p_qos_policy->port_groups) ) + { + p_port_group = (osm_qos_port_group_t*)cl_list_obj(list_iterator); + if (!p_port_group) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name,p_port_group->name) == 0) + return p_port_group; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +int osm_qos_policy_validate() +{ + cl_list_iterator_t match_rules_list_iterator; + cl_list_iterator_t list_iterator; + osm_qos_port_group_t * p_port_group = NULL; + osm_qos_match_rule_t * p_qos_match_rule = NULL; + char * str; + unsigned i; + int res = 0; + + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_policy_validate); + + /* set default qos level */ + + p_qos_policy->p_default_qos_level = + __qos_policy_get_qos_level_by_name(OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + if (!p_qos_policy->p_default_qos_level) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC10: " + "Default qos-level (%s) not defined.\n", + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + res = 1; + goto Exit; + } + + /* scan all the match rules, and fill the lists of pointers to + relevant qos levels and port groups to speed up PR matching */ + + i = 1; + match_rules_list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while( match_rules_list_iterator != cl_list_end(&p_qos_policy->qos_match_rules) ) + { + p_qos_match_rule = (osm_qos_match_rule_t*)cl_list_obj(match_rules_list_iterator); + CL_ASSERT(p_qos_match_rule); + + /* find the matching qos-level for each match-rule */ + + p_qos_match_rule->p_qos_level = __qos_policy_get_qos_level_by_name( + p_qos_match_rule->qos_level_name); + + if (!p_qos_match_rule->p_qos_level) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC11: " + "qos-match-rule num %u: qos-level '%s' not found\n", + i, p_qos_match_rule->qos_level_name); + res = 1; + goto Exit; + } + + /* find the matching port-group for element of source_list */ + + if (cl_list_count(&p_qos_match_rule->source_list)) + { + list_iterator = cl_list_head(&p_qos_match_rule->source_list); + while( list_iterator != cl_list_end(&p_qos_match_rule->source_list) ) + { + str = (char*)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = __qos_policy_get_port_group_by_name(str); + if (!p_port_group) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC12: " + "qos-match-rule num %u: source port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule->source_group_list, p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* find the matching port-group for element of destination_list */ + + if (cl_list_count(&p_qos_match_rule->destination_list)) + { + list_iterator = cl_list_head(&p_qos_match_rule->destination_list); + while( list_iterator != cl_list_end(&p_qos_match_rule->destination_list) ) + { + str = (char*)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = __qos_policy_get_port_group_by_name(str); + if (!p_port_group) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC13: " + "qos-match-rule num %u: destination port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule->destination_group_list, p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* done with the current match-rule */ + + match_rules_list_iterator = cl_list_next(match_rules_list_iterator); + i++; + } + + Exit: + OSM_LOG_EXIT(p_qos_parser_osm_log); + return res; +} /* osm_qos_policy_validate() */ + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_get_qos_level_by_pr( + IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level) +{ + osm_qos_match_rule_t * p_qos_match_rule = NULL; + osm_qos_level_t * p_qos_level = NULL; + + OSM_LOG_ENTER( p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr ); + + *pp_qos_level = NULL; + + if (!p_qos_policy) + goto Exit; + + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_rcv, + p_pr, + p_src_physp, + p_dest_physp, + comp_mask); + + if (p_qos_match_rule) + p_qos_level = p_qos_match_rule->p_qos_level; + else + p_qos_level = p_qos_policy->p_default_qos_level; + + osm_log( p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "PathRecord request:" + "Src port 0x%016" PRIx64 ", " + "Dst port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp)) ); + osm_log( p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "Applying QoS Level %s (%s)\n", + p_qos_level->name, + (p_qos_level->use)? p_qos_level->use : "no description" ); + + *pp_qos_level = p_qos_level; + + Exit: + OSM_LOG_EXIT( p_rcv->p_log ); +} /* osm_qos_policy_get_qos_level_by_pr() */ + +/*************************************************** + ***************************************************/ + -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:06:55 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:06:55 +0300 Subject: [ofa-general] [PATCH 4/7] osm: QoS - rename static find_prtn_by_name() to non-static Message-ID: <46C983DF.5050502@dev.mellanox.co.il> Renaming static find_prtn_by_name() to non-static osm_prtn_find_by_name() - this function is used by QoS policy functions. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_partition.h | 29 +++++++++++++++++++++++++++++ opensm/opensm/osm_prtn.c | 4 ++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h index f00034f..e88da40 100644 --- a/opensm/include/opensm/osm_partition.h +++ b/opensm/include/opensm/osm_partition.h @@ -243,5 +243,34 @@ ib_api_status_t osm_prtn_make_partitions(IN osm_log_t * const p_log, * SEE ALSO *********/ +/****f* OpenSM: Partition/osm_prtn_find_by_name +* NAME +* osm_prtn_find_by_name +* +* DESCRIPTION +* Fides partition by name. +* +* SYNOPSIS +*/ +osm_prtn_t * +osm_prtn_find_by_name( + IN osm_subn_t *p_subn, + IN const char *name); +/* +* PARAMETERS +* p_subn +* [in] Pointer to a subnet object. +* +* name +* [in] Required partition name. +* +* RETURN VALUES +* Pointer to the partition object on success. +* +* NOTES +* +* SEE ALSO +*********/ + END_C_DECLS #endif /* _OSM_PARTITION_H_ */ diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 5dabae9..46ee429 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -265,7 +265,7 @@ static uint16_t __generate_pkey(osm_subn_t * p_subn) return 0; } -static osm_prtn_t *find_prtn_by_name(osm_subn_t * p_subn, const char *name) +osm_prtn_t *osm_prtn_find_by_name(osm_subn_t * p_subn, const char *name) { cl_map_item_t *p_next; osm_prtn_t *p; @@ -289,7 +289,7 @@ osm_prtn_t *osm_prtn_make_new(osm_log_t * p_log, osm_subn_t * p_subn, pkey &= cl_hton16((uint16_t) ~ 0x8000); if (!pkey) { - if (name && (p = find_prtn_by_name(p_subn, name))) + if (name && (p = osm_prtn_find_by_name(p_subn, name))) return p; if (!(pkey = __generate_pkey(p_subn))) return NULL; -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:08:08 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:08:08 +0300 Subject: [ofa-general] [PATCH 5/7] osm: QoS - compiling policy file parser Message-ID: <46C98428.1010607@dev.mellanox.co.il> Compiling QoS policy file parser. C & H files are generated from Lex & Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. Signed-off-by: Yevgeny Kliteynik --- opensm/include/Makefile.am | 2 ++ opensm/opensm/Makefile.am | 28 +++++++++++++++++++++++++++- opensm/opensm/configure.in | 2 ++ 3 files changed, 31 insertions(+), 1 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..b83205f 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -90,6 +90,8 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ + $(srcdir)/opensm/osm_qos_policy.h \ + $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/complib/cl_thread_osd.h \ $(srcdir)/complib/cl_packon.h \ $(srcdir)/complib/cl_atomic_osd.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..6c7208d 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -56,7 +56,21 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ osm_vl15intf.c osm_vl_arb_rcv.c \ st.c osm_perfmgr.c osm_perfmgr_db.c \ - osm_event_plugin.c osm_dump.c + osm_event_plugin.c osm_dump.c \ + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c + +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h + $(YACC) -y -d $(srcdir)/osm_qos_parser.y + mv y.tab.c osm_qos_parser_y.c + mv y.tab.h osm_qos_parser_y.h + cp -f osm_qos_parser_y.c $(srcdir)/ + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ + +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h + $(LEX) $(srcdir)/osm_qos_parser.l + mv lex.yy.c osm_qos_parser_l.c + cp -f osm_qos_parser_l.c $(srcdir)/ + if OSMV_OPENIB opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 @@ -104,3 +118,15 @@ install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so + +# generate c and h files from the lex and yacc files +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l + $(YACC) -y -d $(srcdir)/osm_qos_parser.y + mv y.tab.c osm_qos_parser_y.c + mv y.tab.h osm_qos_parser_y.h + cp -f osm_qos_parser_y.c $(srcdir)/ + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ + $(LEX) $(srcdir)/osm_qos_parser.l + mv lex.yy.c osm_qos_parser_l.c + cp -f osm_qos_parser_l.c $(srcdir)/ + diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in index afb20b3..a49538d 100644 --- a/opensm/opensm/configure.in +++ b/opensm/opensm/configure.in @@ -22,6 +22,8 @@ AC_PROG_INSTALL AC_PROG_LN_S AC_PROG_MAKE_SET AC_PROG_LIBTOOL +AM_PROG_LEX +AC_PROG_YACC dnl Checks for libraries -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:10:27 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:10:27 +0300 Subject: [ofa-general] [PATCH 6/7] osm: QoS - adding QoS policy file option Message-ID: <46C984B3.8090408@dev.mellanox.co.il> Adding QoS policy file option to OpenSM Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ opensm/include/opensm/osm_subnet.h | 1 + opensm/opensm/main.c | 12 +++++++++++- opensm/opensm/osm_subnet.c | 10 +++++++++- 4 files changed, 38 insertions(+), 2 deletions(-) diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h index 545779b..0a35e22 100644 --- a/opensm/include/opensm/osm_base.h +++ b/opensm/include/opensm/osm_base.h @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" #endif /***********/ + +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE +* NAME +* OSM_DEFAULT_QOS_POLICY_FILE +* +* DESCRIPTION +* Specifies the default QoS policy file name +* +* SYNOPSIS +*/ +#ifdef __WIN__ +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") +#else +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" +#endif +/***********/ + /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS * NAME * OSM_DEFAULT_SWEEP_INTERVAL_SECS diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index a7543dc..8ffd104 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -257,6 +257,7 @@ typedef struct _osm_subn_opt { char *partition_config_file; boolean_t no_partition_enforcement; boolean_t no_qos; + char *qos_policy_file; boolean_t accum_log_file; char *console; uint16_t console_port; diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index 0827f1c..06b3aa1 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -269,6 +269,10 @@ void show_usage(void) " The default name is \'" OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); + printf("-Y\n" + "--qos_policy_file\n" + " This option defines the optional QoS policy file.\n" + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); printf("-N\n" "--no_part_enforce\n" " This option disables partition enforcement on switch external ports.\n\n"); @@ -574,7 +578,7 @@ int main(int argc, char *argv[]) char *ignore_guids_file_name = NULL; uint32_t val; const char *const short_option = - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; /* In the array below, the 2nd parameter specifies the number @@ -598,6 +602,7 @@ int main(int argc, char *argv[]) {"Pconfig", 1, NULL, 'P'}, {"no_part_enforce", 0, NULL, 'N'}, {"qos", 0, NULL, 'Q'}, + {"qos_policy_file", 1, NULL, 'Y'}, {"maxsmps", 1, NULL, 'n'}, {"console", 1, NULL, 'q'}, {"V", 0, NULL, 'V'}, @@ -817,6 +822,11 @@ int main(int argc, char *argv[]) opt.no_qos = FALSE; break; + case 'Y': + opt.qos_policy_file = optarg; + printf(" QoS policy file \'%s\'\n", optarg); + break; + case 'y': opt.exit_on_fatal = FALSE; printf(" Staying on fatal initialization errors\n"); diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; p_opt->no_partition_enforcement = FALSE; p_opt->no_qos = TRUE; + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; p_opt->accum_log_file = TRUE; p_opt->port_profile_switch_nodes = FALSE; p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); + opts_unpack_charp("qos_policy_file", + p_key, p_val, &p_opts->qos_policy_file); + opts_unpack_boolean("accum_log_file", p_key, p_val, &p_opts->accum_log_file); @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) fprintf(opts_file, "#\n# QoS OPTIONS\n#\n" "# Disable QoS setup\n" - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); + "no_qos %s\n\n" + "# QoS policy file to be used\n" + "qos_policy_file %s\n\n", + p_opts->no_qos ? "TRUE" : "FALSE", + p_opts->qos_policy_file); subn_dump_qos_options(opts_file, "QoS default options", "qos", -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 20 05:11:21 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Mon, 20 Aug 2007 15:11:21 +0300 Subject: [ofa-general] [PATCH 7/7] osm: QoS - reading policy file Message-ID: <46C984E9.5070600@dev.mellanox.co.il> Reading QoS policy file Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos.c | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c index dff9996..d512f53 100644 --- a/opensm/opensm/osm_qos.c +++ b/opensm/opensm/osm_qos.c @@ -52,6 +52,8 @@ #include #include #include +#include +#include struct qos_config { uint8_t max_vls; @@ -279,12 +281,38 @@ osm_signal_t osm_qos_setup(osm_opensm_t * p_osm) ib_api_status_t status; unsigned force_update; uint8_t i; + struct stat statbuf; if (p_osm->subn.opt.no_qos) return OSM_SIGNAL_DONE; OSM_LOG_ENTER(&p_osm->log, osm_qos_setup); + /* read QoS policy config file */ + if ( !stat(p_osm->subn.opt.qos_policy_file, &statbuf) ) + { + static boolean_t first_time = TRUE; + if ( first_time ) + { + osm_log(&p_osm->log, OSM_LOG_INFO, + "osm_qos_setup: Loading QoS policy file %s\n", + p_osm->subn.opt.qos_policy_file); + first_time = FALSE; + } + if ( !osm_qos_parse_policy_file(&p_osm->log, p_osm->subn.opt.qos_policy_file) ) + osm_log(&p_osm->log, OSM_LOG_VERBOSE, + "osm_qos_setup: QoS policy file %s parsed successfully\n", + p_osm->subn.opt.qos_policy_file); + else + osm_log(&p_osm->log, OSM_LOG_ERROR, + "osm_qos_setup: ERR 6204: Failed parsing QoS policy file %s\n", + p_osm->subn.opt.qos_policy_file); + } + + /* + * ToDo: Setup QoS on the fabric according to QoS policy + */ + qos_build_config(&ca_config, &p_osm->subn.opt.qos_ca_options, &p_osm->subn.opt.qos_options); qos_build_config(&sw0_config, &p_osm->subn.opt.qos_sw0_options, -- 1.5.1.4 From mst at dev.mellanox.co.il Mon Aug 20 05:37:16 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 20 Aug 2007 15:37:16 +0300 Subject: [ofa-general] Re: [PATCH 5/7] osm: QoS - compiling policy file parser In-Reply-To: <46C98428.1010607@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> Message-ID: <20070820123716.GB25387@mellanox.co.il> > Quoting Yevgeny Kliteynik : > Subject: [PATCH 5/7] osm: QoS - compiling policy file parser > > Compiling QoS policy file parser. > > C & H files are generated from Lex & Yacc files and the > generated files are compiled. > Generated files are not included in git, but they do get > included in RPM as they are also created by 'make dist'. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/Makefile.am | 2 ++ > opensm/opensm/Makefile.am | 28 +++++++++++++++++++++++++++- > opensm/opensm/configure.in | 2 ++ > 3 files changed, 31 insertions(+), 1 deletions(-) > > diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > index ea62507..b83205f 100644 > --- a/opensm/include/Makefile.am > +++ b/opensm/include/Makefile.am > @@ -90,6 +90,8 @@ EXTRA_DIST = \ > $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > $(srcdir)/opensm/osm_perfmgr.h \ > $(srcdir)/opensm/osm_perfmgr_db.h \ > + $(srcdir)/opensm/osm_qos_policy.h \ > + $(srcdir)/opensm/osm_qos_parser_y.h \ > $(srcdir)/complib/cl_thread_osd.h \ > $(srcdir)/complib/cl_packon.h \ > $(srcdir)/complib/cl_atomic_osd.h \ > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index 6dfa824..6c7208d 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -56,7 +56,21 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > osm_vl15intf.c osm_vl_arb_rcv.c \ > st.c osm_perfmgr.c osm_perfmgr_db.c \ > - osm_event_plugin.c osm_dump.c > + osm_event_plugin.c osm_dump.c \ > + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > + > +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y > + mv y.tab.c osm_qos_parser_y.c > + mv y.tab.h osm_qos_parser_y.h > + cp -f osm_qos_parser_y.c $(srcdir)/ > + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ > + > +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h > + $(LEX) $(srcdir)/osm_qos_parser.l > + mv lex.yy.c osm_qos_parser_l.c > + cp -f osm_qos_parser_l.c $(srcdir)/ > + Why do you use these mv tricks? Can you not use the -o flag to generate the output file with the proper name? -- MST From mst at dev.mellanox.co.il Mon Aug 20 05:40:49 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 20 Aug 2007 15:40:49 +0300 Subject: [ofa-general] Re: [PATCH 5/7] osm: QoS - compiling policy file parser In-Reply-To: <20070820123716.GB25387@mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <20070820123716.GB25387@mellanox.co.il> Message-ID: <20070820124049.GC25387@mellanox.co.il> > Quoting Michael S. Tsirkin : > Subject: Re: [PATCH 5/7] osm: QoS - compiling policy file parser > > > Quoting Yevgeny Kliteynik : > > Subject: [PATCH 5/7] osm: QoS - compiling policy file parser > > > > Compiling QoS policy file parser. > > > > C & H files are generated from Lex & Yacc files and the > > generated files are compiled. > > Generated files are not included in git, but they do get > > included in RPM as they are also created by 'make dist'. > > > > Signed-off-by: Yevgeny Kliteynik > > --- > > opensm/include/Makefile.am | 2 ++ > > opensm/opensm/Makefile.am | 28 +++++++++++++++++++++++++++- > > opensm/opensm/configure.in | 2 ++ > > 3 files changed, 31 insertions(+), 1 deletions(-) > > > > diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > > index ea62507..b83205f 100644 > > --- a/opensm/include/Makefile.am > > +++ b/opensm/include/Makefile.am > > @@ -90,6 +90,8 @@ EXTRA_DIST = \ > > $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > > $(srcdir)/opensm/osm_perfmgr.h \ > > $(srcdir)/opensm/osm_perfmgr_db.h \ > > + $(srcdir)/opensm/osm_qos_policy.h \ > > + $(srcdir)/opensm/osm_qos_parser_y.h \ > > $(srcdir)/complib/cl_thread_osd.h \ > > $(srcdir)/complib/cl_packon.h \ > > $(srcdir)/complib/cl_atomic_osd.h \ > > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > > index 6dfa824..6c7208d 100644 > > --- a/opensm/opensm/Makefile.am > > +++ b/opensm/opensm/Makefile.am > > @@ -56,7 +56,21 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > > osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > > osm_vl15intf.c osm_vl_arb_rcv.c \ > > st.c osm_perfmgr.c osm_perfmgr_db.c \ > > - osm_event_plugin.c osm_dump.c > > + osm_event_plugin.c osm_dump.c \ > > + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > > + > > +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y > > + mv y.tab.c osm_qos_parser_y.c > > + mv y.tab.h osm_qos_parser_y.h > > + cp -f osm_qos_parser_y.c $(srcdir)/ > > + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ > > + > > +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h > > + $(LEX) $(srcdir)/osm_qos_parser.l > > + mv lex.yy.c osm_qos_parser_l.c > > + cp -f osm_qos_parser_l.c $(srcdir)/ > > + > > Why do you use these mv tricks? > Can you not use the -o flag to generate the output file with the proper name? Alternatively, if you do not want to use flex/bison extensions, you can use -t flag to have lex output to standard output, and -b flag to supply prefix to yacc. BTW, I think it's a good idea to also use -p prefix to reduce the global namespace pollution by yacc. -- MST From sashak at voltaire.com Mon Aug 20 06:35:30 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 20 Aug 2007 16:35:30 +0300 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking Message-ID: <20070820133530.GE27000@sashak.voltaire.com> When MAD sending fails in osm_vendor_send() the send_err_callback() is invoked - this callback maintains (decreases by 1) the outstanding MAD counters. In the current osm_vl15_poller() code those MAD counters are also explicitly decreased in the case when osm_vendor_send() returns error - so actually we have "double free" case and as result OpenSM deadlocks there. This patch removes this additional outstanding mad counters decreasing code from osm_vl15_poller(). Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_vl15intf.h | 28 +--------- opensm/opensm/osm_opensm.c | 7 +- opensm/opensm/osm_vl15intf.c | 103 +++++----------------------------- 3 files changed, 18 insertions(+), 120 deletions(-) diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h index 6de9898..4b290d3 100644 --- a/opensm/include/opensm/osm_vl15intf.h +++ b/opensm/include/opensm/osm_vl15intf.h @@ -53,13 +53,11 @@ #include #include #include -#include #include #include #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -132,10 +130,6 @@ typedef struct _osm_vl15 { osm_vendor_t *p_vend; osm_log_t *p_log; osm_stats_t *p_stats; - osm_subn_t *p_subn; - cl_disp_reg_handle_t h_disp; - cl_plock_t *p_lock; - } osm_vl15_t; /* * FIELDS @@ -174,15 +168,6 @@ typedef struct _osm_vl15 { * p_stats * Pointer to the OpenSM statistics block. * -* p_subn -* Pointer to the Subnet object for this subnet. -* -* h_disp -* Handle returned from dispatcher registration. -* -* p_lock -* Pointer to the serializing lock. -* * SEE ALSO * VL15 object *********/ @@ -267,9 +252,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, IN osm_vendor_t * const p_vend, IN osm_log_t * const p_log, IN osm_stats_t * const p_stats, - IN const int32_t max_wire_smps, - IN osm_subn_t * const p_subn, - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); + IN const int32_t max_wire_smps); /* * PARAMETERS * p_vl15 @@ -287,15 +270,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, * max_wire_smps * [in] Maximum number of MADs allowed on the wire at one time. * -* p_subn -* [in] Pointer to the subnet object. -* -* p_disp -* [in] Pointer to the dispatcher object. -* -* p_lock -* [in] Pointer to the OpenSM serializing lock. -* * RETURN VALUES * IB_SUCCESS if the VL15 object was initialized successfully. * diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c index 9a596dd..329305e 100644 --- a/opensm/opensm/osm_opensm.c +++ b/opensm/opensm/osm_opensm.c @@ -249,10 +249,9 @@ osm_opensm_init(IN osm_opensm_t * const p_osm, if (status != IB_SUCCESS) goto Exit; - status = osm_vl15_init(&p_osm->vl15, - p_osm->p_vendor, - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, - &p_osm->subn, &p_osm->disp, &p_osm->lock); + status = osm_vl15_init(&p_osm->vl15, p_osm->p_vendor, + &p_osm->log, &p_osm->stats, + p_opt->max_wire_smps); if (status != IB_SUCCESS) goto Exit; diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c index bc667b6..af44423 100644 --- a/opensm/opensm/osm_vl15intf.c +++ b/opensm/opensm/osm_vl15intf.c @@ -51,13 +51,12 @@ #include #include +#include +#include #include #include -#include #include #include -#include -#include /********************************************************************** **********************************************************************/ @@ -65,18 +64,13 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) { ib_api_status_t status; - cl_status_t cl_status; - uint32_t mads_sent; - uint32_t unicasts_sent; - uint32_t mads_on_wire; - uint32_t outstanding; /* Non-response-expected mads are not throttled on the wire since we can have no confirmation that they arrived at their destination. */ - if (p_madw->resp_expected == TRUE) { + if (p_madw->resp_expected == TRUE) /* Note that other threads may not see the response MAD arrive before send() even returns. @@ -84,14 +78,11 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) To avoid this confusion, preincrement the counts on the assumption that send() will succeed. */ - mads_on_wire = - cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); - } else - unicasts_sent = - cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); + else + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); - mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), p_madw, p_madw->resp_expected); @@ -118,61 +109,12 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) fix up the pre-incremented count values. */ - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire - that were incremented in the code above. */ - mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); - - if (p_madw->resp_expected == FALSE) - return; - - cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); - - /* - The following code is similar to the code in - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the - qp0_mads_outstanding counter, and if we reached 0 - need to call - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order - to wake up the state mgr). - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter - should be decremented and handled accordingly only if this is a mad - with resp_expected == TRUE. - */ - - outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); - - osm_log(p_vl->p_log, OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "%u QP0 MADs outstanding\n", - p_vl->p_stats->qp0_mads_outstanding); - - if (outstanding) - return; - - /* - The wire is clean. - Signal the state manager. - */ - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) - osm_log(p_vl->p_log, - OSM_LOG_DEBUG, - "__osm_vl15_poller: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); - - cl_status = cl_disp_post(p_vl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, - NULL, NULL); - - if (cl_status != CL_SUCCESS) - osm_log(p_vl->p_log, - OSM_LOG_ERROR, - "__osm_vl15_poller: ERR 3E06: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG(cl_status)); + /* Decrement qp0_mads_sent that were incremented in the code above. + qp0_mads_outstanding will be decremented by send error callback + (called by osm_vendor_send() */ + cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); + if (!p_madw->resp_expected) + cl_atomic_dec(&p_vl->p_stats->qp0_unicasts_sent); } static void __osm_vl15_poller(IN void *p_ptr) @@ -260,7 +202,6 @@ void osm_vl15_construct(IN osm_vl15_t * const p_vl) cl_qlist_init(&p_vl->rfifo); cl_qlist_init(&p_vl->ufifo); cl_thread_construct(&p_vl->poller); - p_vl->h_disp = CL_DISP_INVALID_HANDLE; } /********************************************************************** @@ -317,9 +258,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, IN osm_vendor_t * const p_vend, IN osm_log_t * const p_log, IN osm_stats_t * const p_stats, - IN const int32_t max_wire_smps, - IN osm_subn_t * const p_subn, - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) + IN const int32_t max_wire_smps) { ib_api_status_t status = IB_SUCCESS; @@ -329,8 +268,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, p_vl->p_log = p_log; p_vl->p_stats = p_stats; p_vl->max_wire_smps = max_wire_smps; - p_vl->p_subn = p_subn; - p_vl->p_lock = p_lock; status = cl_event_init(&p_vl->signal, FALSE); if (status != IB_SUCCESS) @@ -351,16 +288,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, if (status != IB_SUCCESS) goto Exit; - p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); - - if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { - osm_log(p_log, OSM_LOG_ERROR, - "osm_vl15_init: ERR 3E01: " - "Dispatcher registration failed\n"); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - Exit: OSM_LOG_EXIT(p_log); return (status); @@ -444,8 +371,6 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, /* grap a lock on the object */ cl_spinlock_acquire(&p_vl->lock); - cl_disp_unregister(p_vl->h_disp); - /* go over all outstanding MADs and retire their transactions */ /* first we handle the list of response MADs */ -- 1.5.3.rc2.29.gc4640f From monis at voltaire.com Mon Aug 20 08:34:12 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:34:12 +0300 Subject: [ofa-general] [PATCH V4 0/10] net/bonding: ADD IPoIB support for the bonding driver Message-ID: <46C9B474.5020202@voltaire.com> This patch series is the fourth version (see below link to V3) of the suggested changes to the bonding driver so it would be able to support non ARPHRD_ETHER netdevices for its High-Availability (active-backup) mode. The motivation is to enable the bonding driver on its HA mode to work with the IP over Infiniband (IPoIB) driver. With these patches I was able to enslave IPoIB netdevices and run TCP, UDP, IP (UDP) Multicast and ICMP traffic with fail-over and fail-back working fine. The working environment was the net-2.6 git. More over, as IPoIB is also the IB ARP provider for the RDMA CM driver which is used by native IB ULPs whose addressing scheme is based on IP (e.g. iSER, SDP, Lustre, NFSoRDMA, RDS), bonding support for IPoIB devices **enables** HA for these ULPs. This holds as when the ULP is informed by the IB HW on the failure of the current IB connection, it just need to reconnect, where the bonding device will now issue the IB ARP over the active IPoIB slave. This series also includes patches to the IPoIB driver that fix some fix some neighboring related issues. Major changes from the previous version: 1) Addressing the issue of safety when unloading the IPoIB module before the bonding module 2) style changes Links to earlier discussion: 1. A discussion in netdev about bonding support for IPoIB. http://lists.openwall.net/netdev/2006/11/30/46 2. A discussion in openfabrics regarding changes in the IPoIB that enable using it as a slave for bonding. http://lists.openfabrics.org/pipermail/general/2007-July/038914.html From ogerlitz at voltaire.com Mon Aug 20 08:37:51 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 20 Aug 2007 18:37:51 +0300 Subject: [ofa-general] Re: IB_SEND_INLINE in the kernel? (was [PATCH -2] ib core - checksum offload) In-Reply-To: <1187201495.16253.173.camel@mtls03> References: <1187201495.16253.173.camel@mtls03> Message-ID: <46C9B54F.2020009@voltaire.com> Eli Cohen wrote: > Index: linux-2.6.23-rc1/include/rdma/ib_verbs.h > =================================================================== > --- linux-2.6.23-rc1.orig/include/rdma/ib_verbs.h 2007-08-15 20:50:16.000000000 +0300 > +++ linux-2.6.23-rc1/include/rdma/ib_verbs.h 2007-08-15 20:50:28.000000000 +0300 > @@ -613,7 +616,9 @@ enum ib_send_flags { > IB_SEND_FENCE = 1, > IB_SEND_SIGNALED = (1<<1), > IB_SEND_SOLICITED = (1<<2), > - IB_SEND_INLINE = (1<<3) Roland, My thinking was the inline sending is not supported for kernel level ULPs, eg from the reason that using CPU cycles in the kernel to copy data should be avoided when possible (eg when the device does HW DMA) Grepping for IB_SEND_INLINE under drivers/infiniband shows that all the HW drivers just ignore it, can it be removed? Or. From monisonlists at gmail.com Mon Aug 20 08:42:19 2007 From: monisonlists at gmail.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:42:19 +0300 Subject: [ofa-general] [PATCH V4 1/10] IB/ipoib: Export call to call_netdevice_notifiers and add new private flag In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B65B.5000500@gmail.com> Export the call to raw_notifier_call_chain so modules can send notifications on netdev events to the netdev_chain. Add IFF_SLAVE_DETACH to the list of priv_flags for net_device. This flag is set by a slave that is about to unregisster from the kernel. Both changes are used in bonding slaves that wish to inform the bonding master about coming detachment. Signed-off-by: Moni Shoua --- include/linux/if.h | 1 + net/core/dev.c | 1 + 2 files changed, 2 insertions(+) Index: net-2.6/net/core/dev.c =================================================================== --- net-2.6.orig/net/core/dev.c 2007-08-15 10:09:02.000000000 +0300 +++ net-2.6/net/core/dev.c 2007-08-15 10:53:00.832543390 +0300 @@ -1148,6 +1148,7 @@ int call_netdevice_notifiers(unsigned lo { return raw_notifier_call_chain(&netdev_chain, val, v); } +EXPORT_SYMBOL(call_netdevice_notifiers); /* When > 0 there are consumers of rx skb time stamps */ static atomic_t netstamp_needed = ATOMIC_INIT(0); Index: net-2.6/include/linux/if.h =================================================================== --- net-2.6.orig/include/linux/if.h 2007-08-20 14:30:39.000000000 +0300 +++ net-2.6/include/linux/if.h 2007-08-20 14:31:06.625174369 +0300 @@ -61,6 +61,7 @@ #define IFF_MASTER_ALB 0x10 /* bonding master, balance-alb. */ #define IFF_BONDING 0x20 /* bonding master or slave */ #define IFF_SLAVE_NEEDARP 0x40 /* need ARPs for validation */ +#define IFF_SLAVE_DETACH 0x80 /* slave is about to unregister */ #define IF_GET_IFACE 0x0001 /* for querying only */ #define IF_GET_PROTO 0x0002 From monis at voltaire.com Mon Aug 20 08:43:43 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:43:43 +0300 Subject: [ofa-general] [PATCH V4 2/10] IB/ipoib: Notify the world before doing unregister In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B6AF.5030402@voltaire.com> When the bonding device enslaves IPoIB devices it takes pointers to functions in the ib_ipoib module. This is fine as long as the ib_ipoib nodule remains loaded while the references to its functions exist. So, to help bonding do a cleanup on time, when the IPoIB net device is a slave of a bonding master, let the master know that the IPoIB device is about to unregister (but before calling unregister). Signed-off-by: Moni Shoua --- drivers/infiniband/ulp/ipoib/ipoib_main.c | 15 +++++++++++++++ 1 files changed, 15 insertions(+) Index: net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- net-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-20 14:29:29.522209580 +0300 +++ net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-20 14:43:03.432162133 +0300 @@ -48,6 +48,7 @@ #include #include +#include MODULE_AUTHOR("Roland Dreier"); MODULE_DESCRIPTION("IP-over-InfiniBand net driver"); @@ -772,6 +773,18 @@ static void ipoib_timeout(struct net_dev /* XXX reset QP, etc. */ } +static int ipoib_slave_detach(struct net_device *dev) +{ + int ret = 0; + if (dev->flags & IFF_SLAVE) { + dev->priv_flags |= IFF_SLAVE_DETACH; + rtnl_lock(); + ret = call_netdevice_notifiers(NETDEV_CHANGE, dev); + rtnl_unlock(); + } + return ret; +} + static int ipoib_hard_header(struct sk_buff *skb, struct net_device *dev, unsigned short type, @@ -921,6 +934,7 @@ void ipoib_dev_cleanup(struct net_device /* Delete any child interfaces first */ list_for_each_entry_safe(cpriv, tcpriv, &priv->child_intfs, list) { + ipoib_slave_detach(cpriv->dev); unregister_netdev(cpriv->dev); ipoib_dev_cleanup(cpriv->dev); free_netdev(cpriv->dev); @@ -1208,6 +1222,7 @@ static void ipoib_remove_one(struct ib_d ib_unregister_event_handler(&priv->event_handler); flush_scheduled_work(); + ipoib_slave_detach(priv->dev); unregister_netdev(priv->dev); ipoib_dev_cleanup(priv->dev); free_netdev(priv->dev); From monis at voltaire.com Mon Aug 20 08:44:56 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:44:56 +0300 Subject: [ofa-general] [PATCH V4 3/10] IB/ipoib: Bound the net device to the ipoib_neigh structue In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B6F8.2090804@voltaire.com> IPoIB uses a two layer neighboring scheme, such that for each struct neighbour whose device is an ipoib one, there is a struct ipoib_neigh buddy which is created on demand at the tx flow by an ipoib_neigh_alloc(skb->dst->neighbour) call. When using the bonding driver, neighbours are created by the net stack on behalf of the bonding (master) device. On the tx flow the bonding code gets an skb such that skb->dev points to the master device, it changes this skb to point on the slave device and calls the slave hard_start_xmit function. Under this scheme, ipoib_neigh_destructor assumption that for each struct neighbour it gets, n->dev is an ipoib device and hence netdev_priv(n->dev) can be casted to struct ipoib_dev_priv is buggy. To fix it, this patch adds a dev field to struct ipoib_neigh which is used instead of the struct neighbour dev one, when n->dev->flags has the IFF_MASTER bit set. Signed-off-by: Moni Shoua Signed-off-by: Or Gerlitz --- drivers/infiniband/ulp/ipoib/ipoib.h | 4 +++- drivers/infiniband/ulp/ipoib/ipoib_main.c | 17 +++++++++++++++-- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 3 ++- 3 files changed, 20 insertions(+), 4 deletions(-) Index: net-2.6/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- net-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 10:09:00.000000000 +0300 +++ net-2.6/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-15 10:53:52.756348574 +0300 @@ -328,6 +328,7 @@ struct ipoib_neigh { struct sk_buff_head queue; struct neighbour *neighbour; + struct net_device *dev; struct list_head list; }; @@ -344,7 +345,8 @@ static inline struct ipoib_neigh **to_ip INFINIBAND_ALEN, sizeof(void *)); } -struct ipoib_neigh *ipoib_neigh_alloc(struct neighbour *neigh); +struct ipoib_neigh *ipoib_neigh_alloc(struct neighbour *neigh, + struct net_device *dev); void ipoib_neigh_free(struct net_device *dev, struct ipoib_neigh *neigh); extern struct workqueue_struct *ipoib_workqueue; Index: net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- net-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 10:53:28.000000000 +0300 +++ net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 10:53:52.757348397 +0300 @@ -511,7 +511,7 @@ static void neigh_add_path(struct sk_buf struct ipoib_path *path; struct ipoib_neigh *neigh; - neigh = ipoib_neigh_alloc(skb->dst->neighbour); + neigh = ipoib_neigh_alloc(skb->dst->neighbour, skb->dev); if (!neigh) { ++priv->stats.tx_dropped; dev_kfree_skb_any(skb); @@ -830,6 +830,17 @@ static void ipoib_neigh_cleanup(struct n unsigned long flags; struct ipoib_ah *ah = NULL; + if (n->dev->flags & IFF_MASTER) { + /* n->dev is not an IPoIB device and we have + to take priv from elsewhere */ + neigh = *to_ipoib_neigh(n); + if (neigh) { + priv = netdev_priv(neigh->dev); + ipoib_dbg(priv, "neigh_destructor for bonding device: %s\n", + n->dev->name); + } else + return; + } ipoib_dbg(priv, "neigh_cleanup for %06x " IPOIB_GID_FMT "\n", IPOIB_QPN(n->ha), @@ -851,7 +862,8 @@ static void ipoib_neigh_cleanup(struct n ipoib_put_ah(ah); } -struct ipoib_neigh *ipoib_neigh_alloc(struct neighbour *neighbour) +struct ipoib_neigh *ipoib_neigh_alloc(struct neighbour *neighbour, + struct net_device *dev) { struct ipoib_neigh *neigh; @@ -860,6 +872,7 @@ struct ipoib_neigh *ipoib_neigh_alloc(st return NULL; neigh->neighbour = neighbour; + neigh->dev = dev; *to_ipoib_neigh(neighbour) = neigh; skb_queue_head_init(&neigh->queue); ipoib_cm_set(neigh, NULL); Index: net-2.6/drivers/infiniband/ulp/ipoib/ipoib_multicast.c =================================================================== --- net-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-08-15 10:09:00.000000000 +0300 +++ net-2.6/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-08-15 10:53:52.758348220 +0300 @@ -727,7 +727,8 @@ out: if (skb->dst && skb->dst->neighbour && !*to_ipoib_neigh(skb->dst->neighbour)) { - struct ipoib_neigh *neigh = ipoib_neigh_alloc(skb->dst->neighbour); + struct ipoib_neigh *neigh = ipoib_neigh_alloc(skb->dst->neighbour, + skb->dev); if (neigh) { kref_get(&mcast->ah->ref); From monis at voltaire.com Mon Aug 20 08:46:10 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:46:10 +0300 Subject: [ofa-general] [PATCH V4 4/10] IB/ipoib: Verify address handle validity on send In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B742.8070707@voltaire.com> When the bonding device senses a carrier loss of its active slave it replaces that slave with a new one. In between the times when the carrier of an IPoIB device goes down and ipoib_neigh is destroyed, it is possible that the bonding driver will send a packet on a new slave that uses an old ipoib_neigh. This patch detects and prevents this from happenning. Signed-off-by: Moni Shoua Signed-off-by: Or Gerlitz --- drivers/infiniband/ulp/ipoib/ipoib_main.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) Index: net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- net-2.6.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 10:53:52.000000000 +0300 +++ net-2.6/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-15 10:54:03.959364640 +0300 @@ -686,9 +686,10 @@ static int ipoib_start_xmit(struct sk_bu goto out; } } else if (neigh->ah) { - if (unlikely(memcmp(&neigh->dgid.raw, + if (unlikely((memcmp(&neigh->dgid.raw, skb->dst->neighbour->ha + 4, - sizeof(union ib_gid)))) { + sizeof(union ib_gid))) || + (neigh->dev != dev))) { spin_lock(&priv->lock); /* * It's safe to call ipoib_put_ah() inside From ogerlitz at voltaire.com Mon Aug 20 08:48:06 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Mon, 20 Aug 2007 18:48:06 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <1187202159.16253.195.camel@mtls03> References: <1187202159.16253.195.camel@mtls03> Message-ID: <46C9B7B6.4060609@voltaire.com> Eli Cohen wrote: > Add LSO support to ipoib > > Using LSO improves performance by allowing the software > to not fragment the payload to mtu sized patckets and also > results in lower rate of interrupts since each such work > request has just one CQE. Hi Eli, Dror, I see that the patch adds the NETIF_F_TSO flag to the device features but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? if yes, what would it take SW wise to support it? Or. From monis at voltaire.com Mon Aug 20 08:48:12 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:48:12 +0300 Subject: [ofa-general] [PATCH V4 5/10] net/bonding: Enable bonding to enslave non ARPHRD_ETHER In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B7BC.5080407@voltaire.com> This patch changes some of the bond netdevice attributes and functions to be that of the active slave for the case of the enslaved device not being of ARPHRD_ETHER type. Basically it overrides those setting done by ether_setup(), which are netdevice **type** dependent and hence might be not appropriate for devices of other types. It also enforces mutual exclusion on bonding slaves from dissimilar ether types, as was concluded over the v1 discussion. IPoIB (see Documentation/infiniband/ipoib.txt) MAC address is made of a 3 bytes IB QP (Queue Pair) number and 16 bytes IB port GID (Global ID) of the port this IPoIB device is bounded to. The QP is a resource created by the IB HW and the GID is an identifier burned into the HCA (i have omitted here some details which are not important for the bonding RFC). Signed-off-by: Moni Shoua Signed-off-by: Or Gerlitz --- drivers/net/bonding/bond_main.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:08:59.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-15 10:54:13.424688411 +0300 @@ -1237,6 +1237,26 @@ static int bond_compute_features(struct return 0; } + +static void bond_setup_by_slave(struct net_device *bond_dev, + struct net_device *slave_dev) +{ + bond_dev->hard_header = slave_dev->hard_header; + bond_dev->rebuild_header = slave_dev->rebuild_header; + bond_dev->hard_header_cache = slave_dev->hard_header_cache; + bond_dev->header_cache_update = slave_dev->header_cache_update; + bond_dev->hard_header_parse = slave_dev->hard_header_parse; + + bond_dev->neigh_setup = slave_dev->neigh_setup; + + bond_dev->type = slave_dev->type; + bond_dev->hard_header_len = slave_dev->hard_header_len; + bond_dev->addr_len = slave_dev->addr_len; + + memcpy(bond_dev->broadcast, slave_dev->broadcast, + slave_dev->addr_len); +} + /* enslave device to bond device */ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev) { @@ -1311,6 +1331,25 @@ int bond_enslave(struct net_device *bond goto err_undo_flags; } + /* set bonding device ether type by slave - bonding netdevices are + * created with ether_setup, so when the slave type is not ARPHRD_ETHER + * there is a need to override some of the type dependent attribs/funcs. + * + * bond ether type mutual exclusion - don't allow slaves of dissimilar + * ether type (eg ARPHRD_ETHER and ARPHRD_INFINIBAND) share the same bond + */ + if (bond->slave_cnt == 0) { + if (slave_dev->type != ARPHRD_ETHER) + bond_setup_by_slave(bond_dev, slave_dev); + } else if (bond_dev->type != slave_dev->type) { + printk(KERN_ERR DRV_NAME ": %s ether type (%d) is different " + "from other slaves (%d), can not enslave it.\n", + slave_dev->name, + slave_dev->type, bond_dev->type); + res = -EINVAL; + goto err_undo_flags; + } + if (slave_dev->set_mac_address == NULL) { printk(KERN_ERR DRV_NAME ": %s: Error: The slave device you specified does " From monis at voltaire.com Mon Aug 20 08:49:14 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:49:14 +0300 Subject: [ofa-general] [PATCH V4 6/10] net/bonding: Enable bonding to enslave netdevices not supporting set_mac_address() In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B7FA.4070207@voltaire.com> This patch allows for enslaving netdevices which do not support the set_mac_address() function. In that case the bond mac address is the one of the active slave, where remote peers are notified on the mac address (neighbour) change by Gratuitous ARP sent by bonding when fail-over occurs (this is already done by the bonding code). Signed-off-by: Moni Shoua Signed-off-by: Or Gerlitz --- drivers/net/bonding/bond_main.c | 87 +++++++++++++++++++++++++++------------- drivers/net/bonding/bonding.h | 1 2 files changed, 60 insertions(+), 28 deletions(-) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:54:13.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-15 10:54:41.971632881 +0300 @@ -1095,6 +1095,14 @@ void bond_change_active_slave(struct bon if (new_active) { bond_set_slave_active_flags(new_active); } + + /* when bonding does not set the slave MAC address, the bond MAC + * address is the one of the active slave. + */ + if (new_active && !bond->do_set_mac_addr) + memcpy(bond->dev->dev_addr, new_active->dev->dev_addr, + new_active->dev->addr_len); + bond_send_gratuitous_arp(bond); } } @@ -1351,13 +1359,22 @@ int bond_enslave(struct net_device *bond } if (slave_dev->set_mac_address == NULL) { - printk(KERN_ERR DRV_NAME - ": %s: Error: The slave device you specified does " - "not support setting the MAC address. " - "Your kernel likely does not support slave " - "devices.\n", bond_dev->name); - res = -EOPNOTSUPP; - goto err_undo_flags; + if (bond->slave_cnt == 0) { + printk(KERN_WARNING DRV_NAME + ": %s: Warning: The first slave device you " + "specified does not support setting the MAC " + "address. This bond MAC address would be that " + "of the active slave.\n", bond_dev->name); + bond->do_set_mac_addr = 0; + } else if (bond->do_set_mac_addr) { + printk(KERN_ERR DRV_NAME + ": %s: Error: The slave device you specified " + "does not support setting the MAC addres,." + "but this bond uses this practice. \n" + , bond_dev->name); + res = -EOPNOTSUPP; + goto err_undo_flags; + } } new_slave = kzalloc(sizeof(struct slave), GFP_KERNEL); @@ -1378,16 +1395,18 @@ int bond_enslave(struct net_device *bond */ memcpy(new_slave->perm_hwaddr, slave_dev->dev_addr, ETH_ALEN); - /* - * Set slave to master's mac address. The application already - * set the master's mac address to that of the first slave - */ - memcpy(addr.sa_data, bond_dev->dev_addr, bond_dev->addr_len); - addr.sa_family = slave_dev->type; - res = dev_set_mac_address(slave_dev, &addr); - if (res) { - dprintk("Error %d calling set_mac_address\n", res); - goto err_free; + if (bond->do_set_mac_addr) { + /* + * Set slave to master's mac address. The application already + * set the master's mac address to that of the first slave + */ + memcpy(addr.sa_data, bond_dev->dev_addr, bond_dev->addr_len); + addr.sa_family = slave_dev->type; + res = dev_set_mac_address(slave_dev, &addr); + if (res) { + dprintk("Error %d calling set_mac_address\n", res); + goto err_free; + } } res = netdev_set_master(slave_dev, bond_dev); @@ -1612,9 +1631,11 @@ err_close: dev_close(slave_dev); err_restore_mac: - memcpy(addr.sa_data, new_slave->perm_hwaddr, ETH_ALEN); - addr.sa_family = slave_dev->type; - dev_set_mac_address(slave_dev, &addr); + if (bond->do_set_mac_addr) { + memcpy(addr.sa_data, new_slave->perm_hwaddr, ETH_ALEN); + addr.sa_family = slave_dev->type; + dev_set_mac_address(slave_dev, &addr); + } err_free: kfree(new_slave); @@ -1792,10 +1813,12 @@ int bond_release(struct net_device *bond /* close slave before restoring its mac address */ dev_close(slave_dev); - /* restore original ("permanent") mac address */ - memcpy(addr.sa_data, slave->perm_hwaddr, ETH_ALEN); - addr.sa_family = slave_dev->type; - dev_set_mac_address(slave_dev, &addr); + if (bond->do_set_mac_addr) { + /* restore original ("permanent") mac address */ + memcpy(addr.sa_data, slave->perm_hwaddr, ETH_ALEN); + addr.sa_family = slave_dev->type; + dev_set_mac_address(slave_dev, &addr); + } slave_dev->priv_flags &= ~(IFF_MASTER_8023AD | IFF_MASTER_ALB | IFF_SLAVE_INACTIVE | IFF_BONDING | @@ -1882,10 +1905,12 @@ static int bond_release_all(struct net_d /* close slave before restoring its mac address */ dev_close(slave_dev); - /* restore original ("permanent") mac address*/ - memcpy(addr.sa_data, slave->perm_hwaddr, ETH_ALEN); - addr.sa_family = slave_dev->type; - dev_set_mac_address(slave_dev, &addr); + if (bond->do_set_mac_addr) { + /* restore original ("permanent") mac address*/ + memcpy(addr.sa_data, slave->perm_hwaddr, ETH_ALEN); + addr.sa_family = slave_dev->type; + dev_set_mac_address(slave_dev, &addr); + } slave_dev->priv_flags &= ~(IFF_MASTER_8023AD | IFF_MASTER_ALB | IFF_SLAVE_INACTIVE); @@ -3922,6 +3947,9 @@ static int bond_set_mac_address(struct n dprintk("bond=%p, name=%s\n", bond, (bond_dev ? bond_dev->name : "None")); + if (!bond->do_set_mac_addr) + return -EOPNOTSUPP; + if (!is_valid_ether_addr(sa->sa_data)) { return -EADDRNOTAVAIL; } @@ -4312,6 +4340,9 @@ static int bond_init(struct net_device * bond_create_proc_entry(bond); #endif + /* set do_set_mac_addr to true on startup */ + bond->do_set_mac_addr = 1; + list_add_tail(&bond->bond_list, &bond_dev_list); return 0; Index: net-2.6/drivers/net/bonding/bonding.h =================================================================== --- net-2.6.orig/drivers/net/bonding/bonding.h 2007-08-15 10:08:58.000000000 +0300 +++ net-2.6/drivers/net/bonding/bonding.h 2007-08-15 10:55:34.359354833 +0300 @@ -185,6 +185,7 @@ struct bonding { struct timer_list mii_timer; struct timer_list arp_timer; s8 kill_timers; + s8 do_set_mac_addr; struct net_device_stats stats; #ifdef CONFIG_PROC_FS struct proc_dir_entry *proc_entry; From monis at voltaire.com Mon Aug 20 08:51:35 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:51:35 +0300 Subject: [ofa-general] [PATCH V4 7/10] net/bonding: Enable IP multicast for bonding IPoIB devices In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B887.7000609@voltaire.com> Allow to enslave devices when the bonding device is not up. Over the discussion held at the previous post this seemed to be the most clean way to go, where it is not expected to cause instabilities. Normally, the bonding driver is UP before any enslavement takes place. Once a netdevice is UP, the network stack acts to have it join some multicast groups (eg the all-hosts 224.0.0.1). Now, since ether_setup() have set the bonding device type to be ARPHRD_ETHER and address len to be ETHER_ALEN, the net core code computes a wrong multicast link address. This is b/c ip_eth_mc_map() is called where for multicast joins taking place after the enslavement another ip_xxx_mc_map() is called (eg ip_ib_mc_map() when the bond type is ARPHRD_INFINIBAND) Signed-off-by: Moni Shoua Signed-off-by: Or Gerlitz --- drivers/net/bonding/bond_main.c | 5 +++-- drivers/net/bonding/bond_sysfs.c | 6 ++---- 2 files changed, 5 insertions(+), 6 deletions(-) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:54:41.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-15 10:55:48.431862446 +0300 @@ -1285,8 +1285,9 @@ int bond_enslave(struct net_device *bond /* bond must be initialized by bond_open() before enslaving */ if (!(bond_dev->flags & IFF_UP)) { - dprintk("Error, master_dev is not up\n"); - return -EPERM; + printk(KERN_WARNING DRV_NAME + " %s: master_dev is not up in bond_enslave\n", + bond_dev->name); } /* already enslaved */ Index: net-2.6/drivers/net/bonding/bond_sysfs.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_sysfs.c 2007-08-15 10:08:58.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_sysfs.c 2007-08-15 10:55:48.432862269 +0300 @@ -266,11 +266,9 @@ static ssize_t bonding_store_slaves(stru /* Quick sanity check -- is the bond interface up? */ if (!(bond->dev->flags & IFF_UP)) { - printk(KERN_ERR DRV_NAME - ": %s: Unable to update slaves because interface is down.\n", + printk(KERN_WARNING DRV_NAME + ": %s: doing slave updates when interface is down.\n", bond->dev->name); - ret = -EPERM; - goto out; } /* Note: We can't hold bond->lock here, as bond_create grabs it. */ From monis at voltaire.com Mon Aug 20 08:52:57 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:52:57 +0300 Subject: [ofa-general] [PATCH V4 8/10] net/bonding: Handlle wrong assumptions that slave is always an Ethernet device In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B8D9.5060508@voltaire.com> bonding sometimes uses Ethernet constants (such as MTU and address length) which are not good when it enslaves non Ethernet devices (such as InfiniBand). Signed-off-by: Moni Shoua --- drivers/net/bonding/bond_main.c | 3 ++- drivers/net/bonding/bond_sysfs.c | 19 +++++++++++++------ drivers/net/bonding/bonding.h | 1 + 3 files changed, 16 insertions(+), 7 deletions(-) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:55:48.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-20 14:29:11.911298577 +0300 @@ -1224,7 +1224,8 @@ static int bond_compute_features(struct struct slave *slave; struct net_device *bond_dev = bond->dev; unsigned long features = bond_dev->features; - unsigned short max_hard_header_len = ETH_HLEN; + unsigned short max_hard_header_len = max((u16)ETH_HLEN, + bond_dev->hard_header_len); int i; features &= ~(NETIF_F_ALL_CSUM | BOND_VLAN_FEATURES); Index: net-2.6/drivers/net/bonding/bond_sysfs.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_sysfs.c 2007-08-15 10:55:48.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_sysfs.c 2007-08-15 12:14:41.152469089 +0300 @@ -164,9 +164,7 @@ static ssize_t bonding_store_bonds(struc printk(KERN_INFO DRV_NAME ": %s is being deleted...\n", bond->dev->name); - bond_deinit(bond->dev); - bond_destroy_sysfs_entry(bond); - unregister_netdevice(bond->dev); + bond_destroy(bond); rtnl_unlock(); goto out; } @@ -260,6 +258,7 @@ static ssize_t bonding_store_slaves(stru char command[IFNAMSIZ + 1] = { 0, }; char *ifname; int i, res, found, ret = count; + u32 original_mtu; struct slave *slave; struct net_device *dev = NULL; struct bonding *bond = to_bond(d); @@ -325,6 +324,7 @@ static ssize_t bonding_store_slaves(stru } /* Set the slave's MTU to match the bond */ + original_mtu = dev->mtu; if (dev->mtu != bond->dev->mtu) { if (dev->change_mtu) { res = dev->change_mtu(dev, @@ -339,6 +339,9 @@ static ssize_t bonding_store_slaves(stru } rtnl_lock(); res = bond_enslave(bond->dev, dev); + bond_for_each_slave(bond, slave, i) + if (strnicmp(slave->dev->name, ifname, IFNAMSIZ) == 0) + slave->original_mtu = original_mtu; rtnl_unlock(); if (res) { ret = res; @@ -351,13 +354,17 @@ static ssize_t bonding_store_slaves(stru bond_for_each_slave(bond, slave, i) if (strnicmp(slave->dev->name, ifname, IFNAMSIZ) == 0) { dev = slave->dev; + original_mtu = slave->original_mtu; break; } if (dev) { printk(KERN_INFO DRV_NAME ": %s: Removing slave %s\n", bond->dev->name, dev->name); rtnl_lock(); - res = bond_release(bond->dev, dev); + if (bond->setup_by_slave) + res = bond_release_and_destroy(bond->dev, dev); + else + res = bond_release(bond->dev, dev); rtnl_unlock(); if (res) { ret = res; @@ -365,9 +372,9 @@ static ssize_t bonding_store_slaves(stru } /* set the slave MTU to the default */ if (dev->change_mtu) { - dev->change_mtu(dev, 1500); + dev->change_mtu(dev, original_mtu); } else { - dev->mtu = 1500; + dev->mtu = original_mtu; } } else { Index: net-2.6/drivers/net/bonding/bonding.h =================================================================== --- net-2.6.orig/drivers/net/bonding/bonding.h 2007-08-15 10:55:34.000000000 +0300 +++ net-2.6/drivers/net/bonding/bonding.h 2007-08-20 14:29:11.912298402 +0300 @@ -156,6 +156,7 @@ struct slave { s8 link; /* one of BOND_LINK_XXXX */ s8 state; /* one of BOND_STATE_XXXX */ u32 original_flags; + u32 original_mtu; u32 link_failure_count; u16 speed; u8 duplex; From monis at voltaire.com Mon Aug 20 08:53:59 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:53:59 +0300 Subject: [ofa-general] PATCH V4 9/10] net/bonding: Delay sending of gratuitous ARP to avoid failure In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9B917.7080702@voltaire.com> Delay sending a gratuitous_arp when LINK_STATE_LINKWATCH_PENDING bit in dev->state field is on. This improves the chances for the arp packet to be transmitted. Signed-off-by: Moni Shoua --- drivers/net/bonding/bond_main.c | 24 +++++++++++++++++++++--- drivers/net/bonding/bonding.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:56:33.000000000 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-15 11:04:37.221123652 +0300 @@ -1102,8 +1102,14 @@ void bond_change_active_slave(struct bon if (new_active && !bond->do_set_mac_addr) memcpy(bond->dev->dev_addr, new_active->dev->dev_addr, new_active->dev->addr_len); - - bond_send_gratuitous_arp(bond); + if (bond->curr_active_slave && + test_bit(__LINK_STATE_LINKWATCH_PENDING, + &bond->curr_active_slave->dev->state)) { + dprintk("delaying gratuitous arp on %s\n", + bond->curr_active_slave->dev->name); + bond->send_grat_arp = 1; + } else + bond_send_gratuitous_arp(bond); } } @@ -2083,6 +2089,17 @@ void bond_mii_monitor(struct net_device * program could monitor the link itself if needed. */ + if (bond->send_grat_arp) { + if (bond->curr_active_slave && test_bit(__LINK_STATE_LINKWATCH_PENDING, + &bond->curr_active_slave->dev->state)) + dprintk("Needs to send gratuitous arp but not yet\n"); + else { + dprintk("sending delayed gratuitous arp on on %s\n", + bond->curr_active_slave->dev->name); + bond_send_gratuitous_arp(bond); + bond->send_grat_arp = 0; + } + } read_lock(&bond->curr_slave_lock); oldcurrent = bond->curr_active_slave; read_unlock(&bond->curr_slave_lock); @@ -2484,7 +2501,7 @@ static void bond_send_gratuitous_arp(str if (bond->master_ip) { bond_arp_send(slave->dev, ARPOP_REPLY, bond->master_ip, - bond->master_ip, 0); + bond->master_ip, 0); } list_for_each_entry(vlan, &bond->vlan_list, vlan_list) { @@ -4293,6 +4310,7 @@ static int bond_init(struct net_device * bond->current_arp_slave = NULL; bond->primary_slave = NULL; bond->dev = bond_dev; + bond->send_grat_arp = 0; INIT_LIST_HEAD(&bond->vlan_list); /* Initialize the device entry points */ Index: net-2.6/drivers/net/bonding/bonding.h =================================================================== --- net-2.6.orig/drivers/net/bonding/bonding.h 2007-08-15 10:56:33.000000000 +0300 +++ net-2.6/drivers/net/bonding/bonding.h 2007-08-15 11:05:41.516451497 +0300 @@ -187,6 +187,7 @@ struct bonding { struct timer_list arp_timer; s8 kill_timers; s8 do_set_mac_addr; + s8 send_grat_arp; struct net_device_stats stats; #ifdef CONFIG_PROC_FS struct proc_dir_entry *proc_entry; From monis at voltaire.com Mon Aug 20 08:58:37 2007 From: monis at voltaire.com (Moni Shoua) Date: Mon, 20 Aug 2007 18:58:37 +0300 Subject: [ofa-general] [PATCH V4 10/10] net/bonding: Destroy bonding master when last slave is gone In-Reply-To: <46C9B474.5020202@voltaire.com> References: <46C9B474.5020202@voltaire.com> Message-ID: <46C9BA2D.7060204@voltaire.com> When bonding enslaves non Ethernet devices it takes pointers to functions in the module that owns the slaves. In this case it becomes unsafe to keep the bonding master registered after last slave was unenslaved because we don't know if the pointers are still valid. Destroying the bond when slave_cnt is zero ensures that these functions be used anymore. Signed-off-by: Moni Shoua --- drivers/net/bonding/bond_main.c | 45 +++++++++++++++++++++++++++++++++++++++- drivers/net/bonding/bonding.h | 3 ++ 2 files changed, 47 insertions(+), 1 deletion(-) Index: net-2.6/drivers/net/bonding/bond_main.c =================================================================== --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-20 14:43:17.123702132 +0300 +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-20 14:43:17.850571535 +0300 @@ -1256,6 +1256,7 @@ static int bond_compute_features(struct static void bond_setup_by_slave(struct net_device *bond_dev, struct net_device *slave_dev) { + struct bonding *bond = bond_dev->priv; bond_dev->hard_header = slave_dev->hard_header; bond_dev->rebuild_header = slave_dev->rebuild_header; bond_dev->hard_header_cache = slave_dev->hard_header_cache; @@ -1270,6 +1271,7 @@ static void bond_setup_by_slave(struct n memcpy(bond_dev->broadcast, slave_dev->broadcast, slave_dev->addr_len); + bond->setup_by_slave = 1; } /* enslave device to bond device */ @@ -1838,6 +1840,35 @@ int bond_release(struct net_device *bond } /* +* Destroy a bonding device. +* Must be under rtnl_lock when this function is called. +*/ +void bond_destroy(struct bonding *bond) +{ + bond_deinit(bond->dev); + bond_destroy_sysfs_entry(bond); + unregister_netdevice(bond->dev); +} + +/* +* First release a slave and than destroy the bond if no more slaves iare left. +* Must be under rtnl_lock when this function is called. +*/ +int bond_release_and_destroy(struct net_device *bond_dev, struct net_device *slave_dev) +{ + struct bonding *bond = bond_dev->priv; + int ret; + + ret = bond_release(bond_dev, slave_dev); + if ((ret == 0) && (bond->slave_cnt == 0)) { + printk(KERN_INFO DRV_NAME " %s: destroying bond for.\n", + bond_dev->name); + bond_destroy(bond); + } + return ret; +} + +/* * This function releases all slaves. */ static int bond_release_all(struct net_device *bond_dev) @@ -3322,7 +3353,11 @@ static int bond_slave_netdev_event(unsig switch (event) { case NETDEV_UNREGISTER: if (bond_dev) { - bond_release(bond_dev, slave_dev); + dprintk("slave %s unregisters\n", slave_dev->name); + if (bond->setup_by_slave) + bond_release_and_destroy(bond_dev, slave_dev); + else + bond_release(bond_dev, slave_dev); } break; case NETDEV_CHANGE: @@ -3331,6 +3366,13 @@ static int bond_slave_netdev_event(unsig * sets up a hierarchical bond, then rmmod's * one of the slave bonding devices? */ + if (slave_dev->priv_flags & IFF_SLAVE_DETACH) { + dprintk("slave %s detaching\n", slave_dev->name); + if (bond->setup_by_slave) + bond_release_and_destroy(bond_dev, slave_dev); + else + bond_release(bond_dev, slave_dev); + } break; case NETDEV_DOWN: /* @@ -4311,6 +4353,7 @@ static int bond_init(struct net_device * bond->primary_slave = NULL; bond->dev = bond_dev; bond->send_grat_arp = 0; + bond->setup_by_slave = 0; INIT_LIST_HEAD(&bond->vlan_list); /* Initialize the device entry points */ Index: net-2.6/drivers/net/bonding/bonding.h =================================================================== --- net-2.6.orig/drivers/net/bonding/bonding.h 2007-08-20 14:43:17.123702132 +0300 +++ net-2.6/drivers/net/bonding/bonding.h 2007-08-20 14:47:52.845180870 +0300 @@ -188,6 +188,7 @@ struct bonding { s8 kill_timers; s8 do_set_mac_addr; s8 send_grat_arp; + s8 setup_by_slave; struct net_device_stats stats; #ifdef CONFIG_PROC_FS struct proc_dir_entry *proc_entry; @@ -295,6 +296,8 @@ static inline void bond_unset_master_alb struct vlan_entry *bond_next_vlan(struct bonding *bond, struct vlan_entry *curr); int bond_dev_queue_xmit(struct bonding *bond, struct sk_buff *skb, struct net_device *slave_dev); int bond_create(char *name, struct bond_params *params, struct bonding **newbond); +void bond_destroy(struct bonding *bond); +int bond_release_and_destroy(struct net_device *bond_dev, struct net_device *slave_dev); void bond_deinit(struct net_device *bond_dev); int bond_create_sysfs(void); void bond_destroy_sysfs(void); From eli at mellanox.co.il Mon Aug 20 09:24:47 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Mon, 20 Aug 2007 19:24:47 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <46C9B7B6.4060609@voltaire.com> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> Message-ID: <1187627087.30342.38.camel@mtls03> On Mon, 2007-08-20 at 18:48 +0300, Or Gerlitz wrote: > I see that the patch adds the NETIF_F_TSO flag to the device features > but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? > if yes, what would it take SW wise to support it? UDP LSO is not supported by the HW. From felix at chelsio.com Mon Aug 20 09:26:31 2007 From: felix at chelsio.com (Felix Marti) Date: Mon, 20 Aug 2007 09:26:31 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com><20070819.174017.77241227.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com><20070819.180540.74750322.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E20C1@maui.asicdesigners.com> Message-ID: <8A71B368A89016469F72CD08050AD334018E2108@maui.asicdesigners.com> > -----Original Message----- > From: ak at suse.de [mailto:ak at suse.de] On Behalf Of Andi Kleen > Sent: Monday, August 20, 2007 4:07 AM > To: Felix Marti > Cc: David Miller; sean.hefty at intel.com; netdev at vger.kernel.org; > rdreier at cisco.com; general at lists.openfabrics.org; linux- > kernel at vger.kernel.org; jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > "Felix Marti" writes: > > > avoidance gains of TSO and LRO are still a very worthwhile savings. > > So, i.e. with TSO, your saving about 16 headers (let us say 14 + 20 + > > 20), 864B, when moving ~64KB of payload - looks like very much in the > > noise to me. > > TSO is beneficial for the software again. The linux code currently > takes several locks and does quite a few function calls for each > packet and using larger packets lowers this overhead. At least with > 10GbE saving CPU cycles is still quite important. > > > an option to get 'high performance' > > Shouldn't you qualify that? > > It is unlikely you really duplicated all the tuning for corner cases > that went over many years into good software TCP stacks in your > hardware. So e.g. for wide area networks with occasional packet loss > the software might well perform better. Yes, it used to be sufficient to submit performance data to show that a technology make 'sense'. In fact, I believe it was Alan Cox who once said that linux will have a look at offload once an offload device holds the land speed record (probably assuming that the day never comes ;). For the last few years it has been Chelsio offload devices that have been improving their own LSRs (as IO bus speeds have been increasing). It is worthwhile to point out that OC-192 doesn't offer full 10Gbps BW and the fine-grained (per packet and not per TSO-burst) packet scheduler in the offload device played a crucial part in pushing performance to the limits of what OC-192 can do. Most other customers use our offload products in low-latency cluster environments. - The problem with offload devices is that they are not all born equal and there have been a lot of poor implementation giving the technology a bad name. I can only speak for Chelsio and do claim that we have a solid implementation that scales from low-latency clusters environments to LFNs. Andi, I could present performance numbers, i.e. throughput and CPU utilization in function of IO size, number of connections, ... in a back-to-back environment and/or in a cluster environment... but what will it get me? I'd still get hit by the 'not integrated' hammer :( > > -Andi From weiny2 at llnl.gov Mon Aug 20 09:34:30 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Mon, 20 Aug 2007 09:34:30 -0700 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <20070820133530.GE27000@sashak.voltaire.com> References: <20070820133530.GE27000@sashak.voltaire.com> Message-ID: <20070820093430.02c1706e.weiny2@llnl.gov> Sasha, Should this be applied to 1.2? Ira On Mon, 20 Aug 2007 16:35:30 +0300 Sasha Khapyorsky wrote: > > When MAD sending fails in osm_vendor_send() the send_err_callback() is > invoked - this callback maintains (decreases by 1) the outstanding MAD > counters. In the current osm_vl15_poller() code those MAD counters are > also explicitly decreased in the case when osm_vendor_send() returns > error - so actually we have "double free" case and as result OpenSM > deadlocks there. > > This patch removes this additional outstanding mad counters decreasing > code from osm_vl15_poller(). > > Signed-off-by: Sasha Khapyorsky > --- > opensm/include/opensm/osm_vl15intf.h | 28 +--------- > opensm/opensm/osm_opensm.c | 7 +- > opensm/opensm/osm_vl15intf.c | 103 +++++----------------------------- > 3 files changed, 18 insertions(+), 120 deletions(-) > > diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h > index 6de9898..4b290d3 100644 > --- a/opensm/include/opensm/osm_vl15intf.h > +++ b/opensm/include/opensm/osm_vl15intf.h > @@ -53,13 +53,11 @@ > #include > #include > #include > -#include > #include > #include > #include > #include > #include > -#include > > #ifdef __cplusplus > # define BEGIN_C_DECLS extern "C" { > @@ -132,10 +130,6 @@ typedef struct _osm_vl15 { > osm_vendor_t *p_vend; > osm_log_t *p_log; > osm_stats_t *p_stats; > - osm_subn_t *p_subn; > - cl_disp_reg_handle_t h_disp; > - cl_plock_t *p_lock; > - > } osm_vl15_t; > /* > * FIELDS > @@ -174,15 +168,6 @@ typedef struct _osm_vl15 { > * p_stats > * Pointer to the OpenSM statistics block. > * > -* p_subn > -* Pointer to the Subnet object for this subnet. > -* > -* h_disp > -* Handle returned from dispatcher registration. > -* > -* p_lock > -* Pointer to the serializing lock. > -* > * SEE ALSO > * VL15 object > *********/ > @@ -267,9 +252,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > IN osm_vendor_t * const p_vend, > IN osm_log_t * const p_log, > IN osm_stats_t * const p_stats, > - IN const int32_t max_wire_smps, > - IN osm_subn_t * const p_subn, > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); > + IN const int32_t max_wire_smps); > /* > * PARAMETERS > * p_vl15 > @@ -287,15 +270,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > * max_wire_smps > * [in] Maximum number of MADs allowed on the wire at one time. > * > -* p_subn > -* [in] Pointer to the subnet object. > -* > -* p_disp > -* [in] Pointer to the dispatcher object. > -* > -* p_lock > -* [in] Pointer to the OpenSM serializing lock. > -* > * RETURN VALUES > * IB_SUCCESS if the VL15 object was initialized successfully. > * > diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c > index 9a596dd..329305e 100644 > --- a/opensm/opensm/osm_opensm.c > +++ b/opensm/opensm/osm_opensm.c > @@ -249,10 +249,9 @@ osm_opensm_init(IN osm_opensm_t * const p_osm, > if (status != IB_SUCCESS) > goto Exit; > > - status = osm_vl15_init(&p_osm->vl15, > - p_osm->p_vendor, > - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, > - &p_osm->subn, &p_osm->disp, &p_osm->lock); > + status = osm_vl15_init(&p_osm->vl15, p_osm->p_vendor, > + &p_osm->log, &p_osm->stats, > + p_opt->max_wire_smps); > if (status != IB_SUCCESS) > goto Exit; > > diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c > index bc667b6..af44423 100644 > --- a/opensm/opensm/osm_vl15intf.c > +++ b/opensm/opensm/osm_vl15intf.c > @@ -51,13 +51,12 @@ > > #include > #include > +#include > +#include > #include > #include > -#include > #include > #include > -#include > -#include > > /********************************************************************** > **********************************************************************/ > @@ -65,18 +64,13 @@ > static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > { > ib_api_status_t status; > - cl_status_t cl_status; > - uint32_t mads_sent; > - uint32_t unicasts_sent; > - uint32_t mads_on_wire; > - uint32_t outstanding; > > /* > Non-response-expected mads are not throttled on the wire > since we can have no confirmation that they arrived > at their destination. > */ > - if (p_madw->resp_expected == TRUE) { > + if (p_madw->resp_expected == TRUE) > /* > Note that other threads may not see the response MAD > arrive before send() even returns. > @@ -84,14 +78,11 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > To avoid this confusion, preincrement the counts on the > assumption that send() will succeed. > */ > - mads_on_wire = > - cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); > - } else > - unicasts_sent = > - cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > + else > + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > - mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), > p_madw, p_madw->resp_expected); > @@ -118,61 +109,12 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > fix up the pre-incremented count values. > */ > > - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire > - that were incremented in the code above. */ > - mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > - > - if (p_madw->resp_expected == FALSE) > - return; > - > - cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > - > - /* > - The following code is similar to the code in > - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the > - qp0_mads_outstanding counter, and if we reached 0 - need to call > - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order > - to wake up the state mgr). > - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. > - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike > - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where > - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter > - should be decremented and handled accordingly only if this is a mad > - with resp_expected == TRUE. > - */ > - > - outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); > - > - osm_log(p_vl->p_log, OSM_LOG_DEBUG, > - "__osm_vl15_poller: " > - "%u QP0 MADs outstanding\n", > - p_vl->p_stats->qp0_mads_outstanding); > - > - if (outstanding) > - return; > - > - /* > - The wire is clean. > - Signal the state manager. > - */ > - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) > - osm_log(p_vl->p_log, > - OSM_LOG_DEBUG, > - "__osm_vl15_poller: " > - "Posting Dispatcher message %s\n", > - osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); > - > - cl_status = cl_disp_post(p_vl->h_disp, > - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) > - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, > - NULL, NULL); > - > - if (cl_status != CL_SUCCESS) > - osm_log(p_vl->p_log, > - OSM_LOG_ERROR, > - "__osm_vl15_poller: ERR 3E06: " > - "Dispatcher post message failed (%s)\n", > - CL_STATUS_MSG(cl_status)); > + /* Decrement qp0_mads_sent that were incremented in the code above. > + qp0_mads_outstanding will be decremented by send error callback > + (called by osm_vendor_send() */ > + cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > + if (!p_madw->resp_expected) > + cl_atomic_dec(&p_vl->p_stats->qp0_unicasts_sent); > } > > static void __osm_vl15_poller(IN void *p_ptr) > @@ -260,7 +202,6 @@ void osm_vl15_construct(IN osm_vl15_t * const p_vl) > cl_qlist_init(&p_vl->rfifo); > cl_qlist_init(&p_vl->ufifo); > cl_thread_construct(&p_vl->poller); > - p_vl->h_disp = CL_DISP_INVALID_HANDLE; > } > > /********************************************************************** > @@ -317,9 +258,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > IN osm_vendor_t * const p_vend, > IN osm_log_t * const p_log, > IN osm_stats_t * const p_stats, > - IN const int32_t max_wire_smps, > - IN osm_subn_t * const p_subn, > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) > + IN const int32_t max_wire_smps) > { > ib_api_status_t status = IB_SUCCESS; > > @@ -329,8 +268,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > p_vl->p_log = p_log; > p_vl->p_stats = p_stats; > p_vl->max_wire_smps = max_wire_smps; > - p_vl->p_subn = p_subn; > - p_vl->p_lock = p_lock; > > status = cl_event_init(&p_vl->signal, FALSE); > if (status != IB_SUCCESS) > @@ -351,16 +288,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > if (status != IB_SUCCESS) > goto Exit; > > - p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); > - > - if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { > - osm_log(p_log, OSM_LOG_ERROR, > - "osm_vl15_init: ERR 3E01: " > - "Dispatcher registration failed\n"); > - status = IB_INSUFFICIENT_RESOURCES; > - goto Exit; > - } > - > Exit: > OSM_LOG_EXIT(p_log); > return (status); > @@ -444,8 +371,6 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, > /* grap a lock on the object */ > cl_spinlock_acquire(&p_vl->lock); > > - cl_disp_unregister(p_vl->h_disp); > - > /* go over all outstanding MADs and retire their transactions */ > > /* first we handle the list of response MADs */ > -- > 1.5.3.rc2.29.gc4640f > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From pradeeps at linux.vnet.ibm.com Mon Aug 20 09:47:25 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Mon, 20 Aug 2007 09:47:25 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) patch to resolve event_handler comment In-Reply-To: <46C5F7F6.2070408@linux.vnet.ibm.com> References: <46C38B47.8010606@linux.vnet.ibm.com> <46C5F7F6.2070408@linux.vnet.ibm.com> Message-ID: <46C9C59D.4030702@linux.vnet.ibm.com> This minor patch resolves the event_handler comment and must be applied on top of the previous NOSRQ patches. Signed-off-by: Pradeep Satyanarayana --- --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-20 12:23:18.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-20 12:29:15.000000000 -0400 @@ -253,6 +253,7 @@ static struct ib_qp *ipoib_cm_create_rx_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_qp_init_attr attr = { + .event_handler = ipoib_cm_rx_event_handler, .send_cq = priv->cq, /* For drain WR */ .recv_cq = priv->cq, .srq = priv->cm.srq, @@ -263,11 +264,8 @@ static struct ib_qp *ipoib_cm_create_rx_ .qp_type = IB_QPT_RC, .qp_context = p, }; - if (!priv->cm.srq) { + if (!priv->cm.srq) attr.cap.max_recv_sge = IPOIB_CM_RX_SG; - attr.event_handler = NULL; - } else - attr.event_handler = ipoib_cm_rx_event_handler; return ib_create_qp(priv->pd, &attr); } From felix at chelsio.com Mon Aug 20 09:53:00 2007 From: felix at chelsio.com (Felix Marti) Date: Mon, 20 Aug 2007 09:53:00 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070820094317.GA14817@2ka.mipt.ru> Message-ID: <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> > -----Original Message----- > From: Evgeniy Polyakov [mailto:johnpol at 2ka.mipt.ru] > Sent: Monday, August 20, 2007 2:43 AM > To: Felix Marti > Cc: David Miller; sean.hefty at intel.com; netdev at vger.kernel.org; > rdreier at cisco.com; general at lists.openfabrics.org; linux- > kernel at vger.kernel.org; jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > On Sun, Aug 19, 2007 at 05:47:59PM -0700, Felix Marti > (felix at chelsio.com) wrote: > > [Felix Marti] David and Herbert, so you agree that the user<>kernel > > space memory copy overhead is a significant overhead and we want to > > enable zero-copy in both the receive and transmit path? - Yes, copy > > It depends. If you need to access that data after received, you will > get > cache miss and performance will not be much better (if any) that with > copy. Yes, the app will take the cache hits when accessing the data. However, the fact remains that if there is a copy in the receive path, you require and additional 3x memory BW (which is very significant at these high rates and most likely the bottleneck for most current systems)... and somebody always has to take the cache miss be it the copy_to_user or the app. > > > avoidance is mainly an API issue and unfortunately the so widely used > > (synchronous) sockets API doesn't make copy avoidance easy, which is > one > > area where protocol offload can help. Yes, some apps can resort to > > sendfile() but there are many apps which seem to have trouble > switching > > to that API... and what about the receive path? > > There is number of implementations, and all they are suitable for is > to have recvfile(), since this is likely the only case, which can work > without cache. > > And actually RDMA stack exist and no one said it should be thrown away > _until_ it messes with main stack. It started to speal ports. What will > happen when it gest all port space and no new legal network conection > can be opened, although there is no way to show to user who got it? > What will happen if hardware RDMA connection got terminated and > software > could not free the port? Will RDMA request to export connection reset > functions out of stack to drop network connections which are on the > ports > which are supposed to be used by new RDMA connections? Yes, RDMA support is there... but we could make it better and easier to use. We have a problem today with port sharing and there was a proposal to address the issue by tighter integration (see the beginning of the thread) but the proposal got shot down immediately... because it is RDMA and not for technical reasons. I believe this email threads shows in detail how RDMA (a network technology) is treated as bastard child by the network folks, well at least by one of them. > > RDMA is not a problem, but how it influence to the network stack is. > Let's better think about how to work correctly with network stack > (since > we already have that cr^Wdifferent hardware) instead of saying that > others do bad work and do not allow shiny new feature to exist. By no means did I want to imply that others do bad work; are you referring to me using TSO implementation issues as an example? - If so, let me clarify: I understand that the TSO implementation took some time to get right. What I was referring to is that TSO(/LRO) have their own issues, some eluded to by Roland and me. In fact, customers working on the LSR couldn't use TSO due to the burstiness it introduces and had to fall-back to our fine grained packet scheduling done in the offload device. I am for variety, let us support new technologies that solve real problems (lots of folks are buying this stuff for a reason) instead of the 'ah, its brain-dead and has no future' attitude... there is precedence for offloading the host CPUs: have a look at graphics. Graphics used to be done by the host CPU and now we have dedicated graphics adapters that do a much better job... so, why is it so farfetched that offload devices can do a better job at a data-flow problem? > > -- > Evgeniy Polyakov From sashak at voltaire.com Mon Aug 20 09:58:31 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 20 Aug 2007 19:58:31 +0300 Subject: [ofa-general] Re: [PATCH 1/7] osm: QoS - adding new PathRecord fields In-Reply-To: <46C982FA.2040601@dev.mellanox.co.il> References: <46C982FA.2040601@dev.mellanox.co.il> Message-ID: <20070820165831.GG27000@sashak.voltaire.com> Hi Yevgeny! On 15:03 Mon 20 Aug , Yevgeny Kliteynik wrote: > > Adding QoS fields to PathRecord. > > Signed-off-by: Yevgeny Kliteynik Some comments are below. > --- > infiniband-diags/src/saquery.c | 8 +- > opensm/include/iba/ib_types.h | 158 +++++++++++++++++++++++++++---- > opensm/opensm/osm_helper.c | 8 +- > opensm/opensm/osm_sa_multipath_record.c | 2 +- > opensm/opensm/osm_sa_path_record.c | 4 +- > opensm/osmtest/osmtest.c | 2 +- > 6 files changed, 154 insertions(+), 28 deletions(-) > > diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c > index 522399e..f085854 100644 > --- a/infiniband-diags/src/saquery.c > +++ b/infiniband-diags/src/saquery.c > @@ -188,7 +188,7 @@ static void > print_path_record(ib_path_rec_t *p_pr) > { > printf("PathRecord dump:\n" > - "\t\tresv0...................0x%016" PRIx64 "\n" > + "\t\tservice_id..............0x%016" PRIx64 "\n" > "\t\tdgid....................0x%016" PRIx64 " : " > "0x%016" PRIx64 "\n" > "\t\tsgid....................0x%016" PRIx64 " : " > @@ -199,7 +199,7 @@ print_path_record(ib_path_rec_t *p_pr) > "\t\ttclass..................0x%X\n" > "\t\tnum_path_revers.........0x%X\n" > "\t\tpkey....................0x%X\n" > - "\t\tsl......................0x%X\n" > + "\t\tqos_class_sl............0x%X\n" I think it is more useful to have separate printouts for sl and qos_class. > "\t\tmtu.....................0x%X\n" > "\t\trate....................0x%X\n" > "\t\tpkt_life................0x%X\n" > @@ -207,7 +207,7 @@ print_path_record(ib_path_rec_t *p_pr) > "\t\tresv2...................0x%X\n" > "\t\tresv3...................0x%X\n" > "", > - *(uint64_t*)p_pr->resv0, > + cl_ntoh64( p_pr->service_id ), > cl_ntoh64( p_pr->dgid.unicast.prefix ), > cl_ntoh64( p_pr->dgid.unicast.interface_id ), > cl_ntoh64( p_pr->sgid.unicast.prefix ), > @@ -218,7 +218,7 @@ print_path_record(ib_path_rec_t *p_pr) > p_pr->tclass, > p_pr->num_path, > cl_ntoh16( p_pr->pkey ), > - cl_ntoh16( p_pr->sl ), > + cl_ntoh16( p_pr->qos_class_sl ), > p_pr->mtu, > p_pr->rate, > p_pr->pkt_life, > diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h > index 490e3fc..f96893a 100644 > --- a/opensm/include/iba/ib_types.h > +++ b/opensm/include/iba/ib_types.h > @@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN const uint8_t class_code) > #define IB_SMINFO_STATE_MASTER 3 > /**********/ > > +/****d* IBA Base: Constants/IB_PATH_REC_SL_MASK > +* NAME > +* IB_PATH_REC_SL_MASK > +* > +* DESCRIPTION > +* Mask for the sl field for path record > +* > +* SOURCE > +*/ > +#define IB_PATH_REC_SL_MASK 0x000F > + > +/****d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK > +* NAME > +* IB_PATH_REC_QOS_CLASS_MASK > +* > +* DESCRIPTION > +* Mask for the QoS class field for path record > +* > +* SOURCE > +*/ > +#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0 > + > /****d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK > * NAME > * IB_PATH_REC_SELECTOR_MASK > @@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid) > */ > #include > typedef struct _ib_path_rec { > - uint8_t resv0[8]; > + ib_net64_t service_id; > ib_gid_t dgid; > ib_gid_t sgid; > ib_net16_t dlid; > @@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec { > uint8_t tclass; > uint8_t num_path; > ib_net16_t pkey; > - ib_net16_t sl; > + ib_net16_t qos_class_sl; > uint8_t mtu; > uint8_t rate; > uint8_t pkt_life; > @@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec { > #include > /* > * FIELDS > -* resv0 > -* Reserved bytes. > +* service_id > +* Service ID for QoS. > * > * dgid > * GID of destination port. > @@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec { > * pkey > * Partition key (P_Key) to use on this path. > * > -* resv1 > -* Reserved byte. > -* > -* sl > -* Service level to use on this path. > +* qos_class_sl > +* QoS class and service level to use on this path. > * > * mtu > * MTU and MTU selector fields to use on this path > @@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec { > *********/ > > /* Path Record Component Masks */ > +#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)<<1)) > #define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)<<2)) > #define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)<<3)) > #define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)<<4)) > @@ -2372,7 +2392,7 @@ typedef struct _ib_path_rec { > #define IB_PR_COMPMASK_REVERSIBLE (CL_HTON64(((uint64_t)1)<<11)) > #define IB_PR_COMPMASK_NUMBPATH (CL_HTON64(((uint64_t)1)<<12)) > #define IB_PR_COMPMASK_PKEY (CL_HTON64(((uint64_t)1)<<13)) > -#define IB_PR_COMPMASK_RESV1 (CL_HTON64(((uint64_t)1)<<14)) > +#define IB_PR_COMPMASK_QOS_CLASS (CL_HTON64(((uint64_t)1)<<14)) > #define IB_PR_COMPMASK_SL (CL_HTON64(((uint64_t)1)<<15)) > #define IB_PR_COMPMASK_MTUSELEC (CL_HTON64(((uint64_t)1)<<16)) > #define IB_PR_COMPMASK_MTU (CL_HTON64(((uint64_t)1)<<17)) > @@ -2630,6 +2650,7 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, > IN uint8_t num_path, > IN ib_net16_t pkey, > IN uint8_t sl, > + IN uint16_t qos_class, > IN uint8_t mtu_selector, > IN uint8_t mtu, > IN uint8_t rate_selector, > @@ -2643,8 +2664,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, > p_rec->slid = slid; > p_rec->num_path = num_path; > p_rec->pkey = pkey; > - /* Lower 4 bits of path rec's SL are reserved. */ > - p_rec->sl = cl_ntoh16(sl); > + p_rec->qos_class_sl = cl_hton16( (sl & IB_PATH_REC_SL_MASK) | > + (qos_class << 4) ); > p_rec->mtu = (uint8_t) ((mtu & IB_PATH_REC_BASE_MASK) | > (uint8_t) (mtu_selector << 6)); > p_rec->rate = (uint8_t) ((rate & IB_PATH_REC_BASE_MASK) | > @@ -2656,8 +2677,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, > /* Clear global routing fields for local path records */ > p_rec->hop_flow_raw = 0; > p_rec->tclass = 0; > + p_rec->service_id = 0; > > - *((uint64_t *) p_rec->resv0) = 0; > *((uint32_t *) p_rec->resv2) = 0; > *((uint16_t *) p_rec->resv2 + 2) = 0; > } > @@ -2687,6 +2708,9 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, > * pkey > * [in] Partition key (P_Key) to use on this path. > * > +* qos_class > +* [in] QoS class to use on this path. Lower 12-bits are valid. > +* > * sl > * [in] Service level to use on this path. Lower 4-bits are valid. > * > @@ -2750,6 +2774,41 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) > * ib_path_rec_t > *********/ > > +/****f* IBA Base: Types/ib_path_rec_set_sl > +* NAME > +* ib_path_rec_set_sl > +* > +* DESCRIPTION > +* Set path service level. > +* > +* SYNOPSIS > +*/ > +static inline void OSM_API > +ib_path_rec_set_sl( > + IN ib_path_rec_t* const p_rec, > + IN const uint8_t sl ) > +{ > + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & > + IB_PATH_REC_QOS_CLASS_MASK ) | > + ( sl & IB_PATH_REC_SL_MASK) ); You could do something like: (p_rec->qos_class_sl & CL_HTON16(IB_PATH_REC_QOS_CLASS_MASK)) | cl_hton16(sl & IB_PATH_REC_SL_MASK) CL_HTON16(IB_PATH_REC_QOS_CLASS_MASK) is calculated at compile time, so one cl_hton16() is saved. > +} > +/* > +* PARAMETERS > +* p_rec > +* [in] Pointer to the path record object. > +* > +* sl > +* [in] Service level to set. > +* > +* RETURN VALUES > +* None > +* > +* NOTES > +* > +* SEE ALSO > +* ib_path_rec_t > +*********/ > + > /****f* IBA Base: Types/ib_path_rec_sl > * NAME > * ib_path_rec_sl > @@ -2762,7 +2821,7 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) > static inline uint8_t OSM_API > ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) > { > - return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); > + return ((uint8_t) ((cl_ntoh16(p_rec->qos_class_sl)) & IB_PATH_REC_SL_MASK)); > } > > /* > @@ -2779,6 +2838,70 @@ ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) > * ib_path_rec_t > *********/ > > +/****f* IBA Base: Types/ib_path_rec_set_qos_class > +* NAME > +* ib_path_rec_set_qos_class > +* > +* DESCRIPTION > +* Set path QoS class. > +* > +* SYNOPSIS > +*/ > +static inline void OSM_API > +ib_path_rec_set_qos_class( > + IN ib_path_rec_t* const p_rec, > + IN const uint16_t qos_class ) > +{ > + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & > + IB_PATH_REC_SL_MASK ) | > + ( qos_class << 4) ); Similar to above. > +} > +/* > +* PARAMETERS > +* p_rec > +* [in] Pointer to the path record object. > +* > +* qos_class > +* [in] QoS class to set. > +* > +* RETURN VALUES > +* None > +* > +* NOTES > +* > +* SEE ALSO > +* ib_path_rec_t > +*********/ > + > +/****f* IBA Base: Types/ib_path_rec_qos_class > +* NAME > +* ib_path_rec_qos_class > +* > +* DESCRIPTION > +* Get QoS class. > +* > +* SYNOPSIS > +*/ > +static inline uint16_t OSM_API > +ib_path_rec_qos_class( > + IN const ib_path_rec_t* const p_rec ) > +{ > + return (cl_ntoh16( p_rec->qos_class_sl ) >> 4); > +} > +/* > +* PARAMETERS > +* p_rec > +* [in] Pointer to the path record object. > +* > +* RETURN VALUES > +* QoS class of the path record. > +* > +* NOTES > +* > +* SEE ALSO > +* ib_path_rec_t > +*********/ > + > /****f* IBA Base: Types/ib_path_rec_mtu > * NAME > * ib_path_rec_mtu > @@ -2940,7 +3063,7 @@ ib_path_rec_pkt_life(IN const ib_path_rec_t * const p_rec) > * [in] Pointer to the path record object. > * > * RETURN VALUES > -* Encoded path pkt_life = 4.096 ?sec * 2 ** PacketLifeTime. > +* Encoded path pkt_life = 4.096 ??sec * 2 ** PacketLifeTime. It is not ascii. What is the change here? > * > * NOTES > * > @@ -2988,7 +3111,8 @@ ib_path_rec_pkt_life_sel(IN const ib_path_rec_t * const p_rec) > * DESCRIPTION > * Get flow label. > * > -* SYNOPSIS > +* SYNOPSIS p_rec->tclass = 0; What do you mean? > + > */ > static inline uint32_t OSM_API > ib_path_rec_flow_lbl(IN const ib_path_rec_t * const p_rec) > @@ -5999,7 +6123,7 @@ ib_multipath_rec_pkt_life(IN const ib_multipath_rec_t * const p_rec) > * [in] Pointer to the multipath record object. > * > * RETURN VALUES > -* Encoded multipath pkt_life = 4.096 ?sec * 2 ** PacketLifeTime. > +* Encoded multipath pkt_life = 4.096 ??sec * 2 ** PacketLifeTime. ascii? > * > * NOTES > * > diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c > index c428823..04e52a8 100644 > --- a/opensm/opensm/osm_helper.c > +++ b/opensm/opensm/osm_helper.c > @@ -1044,7 +1044,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, > if (osm_log_is_active(p_log, log_level)) { > osm_log(p_log, log_level, > "PathRecord dump:\n" > - "\t\t\t\tresv0...................0x%016" PRIx64 "\n" > + "\t\t\t\tservice_id..............0x%016" PRIx64 "\n" > "\t\t\t\tdgid....................0x%016" PRIx64 " : " > "0x%016" PRIx64 "\n" > "\t\t\t\tsgid....................0x%016" PRIx64 " : " > @@ -1055,6 +1055,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, > "\t\t\t\ttclass..................0x%X\n" > "\t\t\t\tnum_path_revers.........0x%X\n" > "\t\t\t\tpkey....................0x%X\n" > + "\t\t\t\tqos_class...............0x%X\n" > "\t\t\t\tsl......................0x%X\n" > "\t\t\t\tmtu.....................0x%X\n" > "\t\t\t\trate....................0x%X\n" > @@ -1063,7 +1064,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, > "\t\t\t\tresv2...................0x%X\n" > "\t\t\t\tresv3...................0x%X\n" > "", > - *(uint64_t *) p_pr->resv0, > + cl_ntoh64(p_pr->service_id), > cl_ntoh64(p_pr->dgid.unicast.prefix), > cl_ntoh64(p_pr->dgid.unicast.interface_id), > cl_ntoh64(p_pr->sgid.unicast.prefix), > @@ -1074,7 +1075,8 @@ osm_dump_path_record(IN osm_log_t * const p_log, > p_pr->tclass, > p_pr->num_path, > cl_ntoh16(p_pr->pkey), > - cl_ntoh16(p_pr->sl), > + ib_path_rec_qos_class(p_pr), > + ib_path_rec_sl(p_pr), > p_pr->mtu, > p_pr->rate, > p_pr->pkt_life, > diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c > index b3f42e2..3c79de7 100644 > --- a/opensm/opensm/osm_sa_multipath_record.c > +++ b/opensm/opensm/osm_sa_multipath_record.c > @@ -724,7 +724,7 @@ __osm_mpr_rcv_build_pr(IN osm_mpr_rcv_t * const p_rcv, > p_pr->hop_flow_raw &= cl_hton32(1 << 31); > > p_pr->pkey = p_parms->pkey; > - p_pr->sl = cl_hton16(p_parms->sl); > + ib_path_rec_set_sl(p_pr, p_parms->sl); Now when ib_path_rec_set_sl() sets only low 4 bits (and all 16 as before), shouldn't upper 12 bits (qos_class) be initialized here? > p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); > p_pr->rate = (uint8_t) (p_parms->rate | 0x80); > > diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c > index 602fd2a..e8b24d0 100644 > --- a/opensm/opensm/osm_sa_path_record.c > +++ b/opensm/opensm/osm_sa_path_record.c > @@ -740,7 +740,7 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv, > #endif > > p_pr->pkey = p_parms->pkey; > - p_pr->sl = cl_hton16(p_parms->sl); > + ib_path_rec_set_sl(p_pr, p_parms->sl); Ditto. > p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); > p_pr->rate = (uint8_t) (p_parms->rate | 0x80); > > @@ -1968,7 +1968,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data) > mcmember_rec. > sl_flow_hop, > &sl, &flow_label, &hop_limit); > - p_pr_item->path_rec.sl = cl_hton16(sl); > + ib_path_rec_set_sl(&p_pr_item->path_rec, sl); Ditto. > #ifndef ROUTER_EXP > p_pr_item->path_rec.hop_flow_raw = > cl_hton32(hop_limit) | (flow_label << 8); > diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c > index 36cb825..5bf11ea 100644 > --- a/opensm/osmtest/osmtest.c > +++ b/opensm/osmtest/osmtest.c > @@ -1918,7 +1918,7 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, > cl_ntoh64(p_rec->sgid.unicast.interface_id), > cl_ntoh16(p_rec->dlid), cl_ntoh16(p_rec->slid), > cl_ntoh32(p_rec->hop_flow_raw), p_rec->tclass, > - p_rec->num_path, cl_ntoh16(p_rec->pkey), p_rec->sl, > + p_rec->num_path, cl_ntoh16(p_rec->pkey), ib_path_rec_sl(p_rec), Why to not print qos_class too? > p_rec->mtu, p_rec->rate, p_rec->pkt_life, > p_rec->preference); > > -- > 1.5.1.4 > Also it would be nice if newly added code will be formatted with opensm/osm_indent. Thanks. Sasha From sashak at voltaire.com Mon Aug 20 10:01:18 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Mon, 20 Aug 2007 20:01:18 +0300 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <20070820093430.02c1706e.weiny2@llnl.gov> References: <20070820133530.GE27000@sashak.voltaire.com> <20070820093430.02c1706e.weiny2@llnl.gov> Message-ID: <20070820170117.GH27000@sashak.voltaire.com> Hi Ira, On 09:34 Mon 20 Aug , Ira Weiny wrote: > Sasha, > > Should this be applied to 1.2? Yes, I think it should, although don't know when next 1.2.x release will be. Sasha > > Ira > > > On Mon, 20 Aug 2007 16:35:30 +0300 > Sasha Khapyorsky wrote: > > > > > When MAD sending fails in osm_vendor_send() the send_err_callback() is > > invoked - this callback maintains (decreases by 1) the outstanding MAD > > counters. In the current osm_vl15_poller() code those MAD counters are > > also explicitly decreased in the case when osm_vendor_send() returns > > error - so actually we have "double free" case and as result OpenSM > > deadlocks there. > > > > This patch removes this additional outstanding mad counters decreasing > > code from osm_vl15_poller(). > > > > Signed-off-by: Sasha Khapyorsky > > --- > > opensm/include/opensm/osm_vl15intf.h | 28 +--------- > > opensm/opensm/osm_opensm.c | 7 +- > > opensm/opensm/osm_vl15intf.c | 103 +++++----------------------------- > > 3 files changed, 18 insertions(+), 120 deletions(-) > > > > diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h > > index 6de9898..4b290d3 100644 > > --- a/opensm/include/opensm/osm_vl15intf.h > > +++ b/opensm/include/opensm/osm_vl15intf.h > > @@ -53,13 +53,11 @@ > > #include > > #include > > #include > > -#include > > #include > > #include > > #include > > #include > > #include > > -#include > > > > #ifdef __cplusplus > > # define BEGIN_C_DECLS extern "C" { > > @@ -132,10 +130,6 @@ typedef struct _osm_vl15 { > > osm_vendor_t *p_vend; > > osm_log_t *p_log; > > osm_stats_t *p_stats; > > - osm_subn_t *p_subn; > > - cl_disp_reg_handle_t h_disp; > > - cl_plock_t *p_lock; > > - > > } osm_vl15_t; > > /* > > * FIELDS > > @@ -174,15 +168,6 @@ typedef struct _osm_vl15 { > > * p_stats > > * Pointer to the OpenSM statistics block. > > * > > -* p_subn > > -* Pointer to the Subnet object for this subnet. > > -* > > -* h_disp > > -* Handle returned from dispatcher registration. > > -* > > -* p_lock > > -* Pointer to the serializing lock. > > -* > > * SEE ALSO > > * VL15 object > > *********/ > > @@ -267,9 +252,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > > IN osm_vendor_t * const p_vend, > > IN osm_log_t * const p_log, > > IN osm_stats_t * const p_stats, > > - IN const int32_t max_wire_smps, > > - IN osm_subn_t * const p_subn, > > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); > > + IN const int32_t max_wire_smps); > > /* > > * PARAMETERS > > * p_vl15 > > @@ -287,15 +270,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > > * max_wire_smps > > * [in] Maximum number of MADs allowed on the wire at one time. > > * > > -* p_subn > > -* [in] Pointer to the subnet object. > > -* > > -* p_disp > > -* [in] Pointer to the dispatcher object. > > -* > > -* p_lock > > -* [in] Pointer to the OpenSM serializing lock. > > -* > > * RETURN VALUES > > * IB_SUCCESS if the VL15 object was initialized successfully. > > * > > diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c > > index 9a596dd..329305e 100644 > > --- a/opensm/opensm/osm_opensm.c > > +++ b/opensm/opensm/osm_opensm.c > > @@ -249,10 +249,9 @@ osm_opensm_init(IN osm_opensm_t * const p_osm, > > if (status != IB_SUCCESS) > > goto Exit; > > > > - status = osm_vl15_init(&p_osm->vl15, > > - p_osm->p_vendor, > > - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, > > - &p_osm->subn, &p_osm->disp, &p_osm->lock); > > + status = osm_vl15_init(&p_osm->vl15, p_osm->p_vendor, > > + &p_osm->log, &p_osm->stats, > > + p_opt->max_wire_smps); > > if (status != IB_SUCCESS) > > goto Exit; > > > > diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c > > index bc667b6..af44423 100644 > > --- a/opensm/opensm/osm_vl15intf.c > > +++ b/opensm/opensm/osm_vl15intf.c > > @@ -51,13 +51,12 @@ > > > > #include > > #include > > +#include > > +#include > > #include > > #include > > -#include > > #include > > #include > > -#include > > -#include > > > > /********************************************************************** > > **********************************************************************/ > > @@ -65,18 +64,13 @@ > > static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > { > > ib_api_status_t status; > > - cl_status_t cl_status; > > - uint32_t mads_sent; > > - uint32_t unicasts_sent; > > - uint32_t mads_on_wire; > > - uint32_t outstanding; > > > > /* > > Non-response-expected mads are not throttled on the wire > > since we can have no confirmation that they arrived > > at their destination. > > */ > > - if (p_madw->resp_expected == TRUE) { > > + if (p_madw->resp_expected == TRUE) > > /* > > Note that other threads may not see the response MAD > > arrive before send() even returns. > > @@ -84,14 +78,11 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > To avoid this confusion, preincrement the counts on the > > assumption that send() will succeed. > > */ > > - mads_on_wire = > > - cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); > > - } else > > - unicasts_sent = > > - cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > + else > > + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > > > - mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > > > status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), > > p_madw, p_madw->resp_expected); > > @@ -118,61 +109,12 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > fix up the pre-incremented count values. > > */ > > > > - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire > > - that were incremented in the code above. */ > > - mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > > - > > - if (p_madw->resp_expected == FALSE) > > - return; > > - > > - cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > - > > - /* > > - The following code is similar to the code in > > - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the > > - qp0_mads_outstanding counter, and if we reached 0 - need to call > > - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order > > - to wake up the state mgr). > > - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. > > - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike > > - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where > > - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter > > - should be decremented and handled accordingly only if this is a mad > > - with resp_expected == TRUE. > > - */ > > - > > - outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); > > - > > - osm_log(p_vl->p_log, OSM_LOG_DEBUG, > > - "__osm_vl15_poller: " > > - "%u QP0 MADs outstanding\n", > > - p_vl->p_stats->qp0_mads_outstanding); > > - > > - if (outstanding) > > - return; > > - > > - /* > > - The wire is clean. > > - Signal the state manager. > > - */ > > - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) > > - osm_log(p_vl->p_log, > > - OSM_LOG_DEBUG, > > - "__osm_vl15_poller: " > > - "Posting Dispatcher message %s\n", > > - osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); > > - > > - cl_status = cl_disp_post(p_vl->h_disp, > > - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) > > - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, > > - NULL, NULL); > > - > > - if (cl_status != CL_SUCCESS) > > - osm_log(p_vl->p_log, > > - OSM_LOG_ERROR, > > - "__osm_vl15_poller: ERR 3E06: " > > - "Dispatcher post message failed (%s)\n", > > - CL_STATUS_MSG(cl_status)); > > + /* Decrement qp0_mads_sent that were incremented in the code above. > > + qp0_mads_outstanding will be decremented by send error callback > > + (called by osm_vendor_send() */ > > + cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > > + if (!p_madw->resp_expected) > > + cl_atomic_dec(&p_vl->p_stats->qp0_unicasts_sent); > > } > > > > static void __osm_vl15_poller(IN void *p_ptr) > > @@ -260,7 +202,6 @@ void osm_vl15_construct(IN osm_vl15_t * const p_vl) > > cl_qlist_init(&p_vl->rfifo); > > cl_qlist_init(&p_vl->ufifo); > > cl_thread_construct(&p_vl->poller); > > - p_vl->h_disp = CL_DISP_INVALID_HANDLE; > > } > > > > /********************************************************************** > > @@ -317,9 +258,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > IN osm_vendor_t * const p_vend, > > IN osm_log_t * const p_log, > > IN osm_stats_t * const p_stats, > > - IN const int32_t max_wire_smps, > > - IN osm_subn_t * const p_subn, > > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) > > + IN const int32_t max_wire_smps) > > { > > ib_api_status_t status = IB_SUCCESS; > > > > @@ -329,8 +268,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > p_vl->p_log = p_log; > > p_vl->p_stats = p_stats; > > p_vl->max_wire_smps = max_wire_smps; > > - p_vl->p_subn = p_subn; > > - p_vl->p_lock = p_lock; > > > > status = cl_event_init(&p_vl->signal, FALSE); > > if (status != IB_SUCCESS) > > @@ -351,16 +288,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > if (status != IB_SUCCESS) > > goto Exit; > > > > - p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); > > - > > - if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { > > - osm_log(p_log, OSM_LOG_ERROR, > > - "osm_vl15_init: ERR 3E01: " > > - "Dispatcher registration failed\n"); > > - status = IB_INSUFFICIENT_RESOURCES; > > - goto Exit; > > - } > > - > > Exit: > > OSM_LOG_EXIT(p_log); > > return (status); > > @@ -444,8 +371,6 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, > > /* grap a lock on the object */ > > cl_spinlock_acquire(&p_vl->lock); > > > > - cl_disp_unregister(p_vl->h_disp); > > - > > /* go over all outstanding MADs and retire their transactions */ > > > > /* first we handle the list of response MADs */ > > -- > > 1.5.3.rc2.29.gc4640f > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From andi at firstfloor.org Mon Aug 20 11:10:56 2007 From: andi at firstfloor.org (Andi Kleen) Date: 20 Aug 2007 20:10:56 +0200 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> References: <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070820094317.GA14817@2ka.mipt.ru> <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> Message-ID: "Felix Marti" writes: > What I was referring to is that TSO(/LRO) have their own > issues, some eluded to by Roland and me. In fact, customers working on > the LSR couldn't use TSO due to the burstiness it introduces That was in old kernels where TSO didn't honor the initial cwnd correctly, right? I assume it's long fixed. If not please clarify what the problem was. > have a look at graphics. > Graphics used to be done by the host CPU and now we have dedicated > graphics adapters that do a much better job... Is your off load device as programable as a modern GPU? > farfetched that offload devices can do a better job at a data-flow > problem? One big difference is that there is no potentially adverse and always varying internet between the graphics card and your monitor. -Andi From bramesh at vt.edu Mon Aug 20 10:51:40 2007 From: bramesh at vt.edu (Bharath Ramesh) Date: Mon, 20 Aug 2007 13:51:40 -0400 Subject: [ofa-general] Query on ibv_post_recv and IBV_SEND_INLINE Message-ID: <20070820175139.GA8604@vt.edu> I have a question on using IBV_SEND_INLINE. From what I understand if I am using IBV_SEND_INLINE I technically neednt register the buffer with ibv_reg_mr as I dont need the L_Key. But still in the receiver end I would have to have the buffer registered to receive the inline data that has been posted. If I have an application where I am using a lot of small messages, then memory registration would be expensive. Is there any other option for receiving inline messages with out actually registering the receive buffer. Thanks, Bharath --- Bharath Ramesh http://people.cs.vt.edu/~bramesh From mshefty at ichips.intel.com Mon Aug 20 11:11:47 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Mon, 20 Aug 2007 11:11:47 -0700 Subject: [ofa-general] Query on ibv_post_recv and IBV_SEND_INLINE In-Reply-To: <20070820175139.GA8604@vt.edu> References: <20070820175139.GA8604@vt.edu> Message-ID: <46C9D963.6080602@ichips.intel.com> > I have a question on using IBV_SEND_INLINE. From what I understand if I am > using IBV_SEND_INLINE I technically neednt register the buffer with > ibv_reg_mr as I dont need the L_Key. But still in the receiver end I > would have to have the buffer registered to receive the inline data that > has been posted. If I have an application where I am using a lot of > small messages, then memory registration would be expensive. Is there > any other option for receiving inline messages with out actually > registering the receive buffer. You could register a larger buffer once and use it with multiple receives. If you needed to, you could copy the data out of the registered buffer into separate buffers for processing. - Sean From tom at opengridcomputing.com Mon Aug 20 11:53:51 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:53:51 -0500 Subject: [ofa-general] [RFC,PATCH 00/20] svc: Server Side Transport Switch Message-ID: <20070820185351.18466.73527.stgit@dell3.ogc.int> This patchest modifies the RPC server side implementation to support pluggable transports. This was done in order to allow RPC applications (NFS) to run over RDMA transports like IB and iWARP. This patchset was also published to nfs at lists.sourceforge.net. This patchset represents an update to the previously published version. The most significant changes are a renaming of the transport switch data structures and functions based on a recommendation from Chuck Lever. Code cleanup was also done in the portlist implementation based on feedback from Trond. I've included the original description below for new reviewers. This patchset implements a sunrpc server side pluggable transport switch that supports dynamically registered transports. The knfsd daemon has been modified to allow user-mode programs to add a new listening endpoint by writing a string to the portlist file. The format of the string is as follows: For example, # echo rdma 2050 > /proc/fs/nfsd/portlist Will cause the knfsd daemon to attempt to add a listening endpoint on port 2050 using the 'rdma' transport. Transports register themselves with the transport switch using a new API that has the following synopsis: void svc_register_transport(struct svc_sock_ops *xprt) The text transport name is contained in a field in the xprt structure. A new service has been added as well to take a transport name instead of an IP protocol number to specify the transport on which the listening endpoint is to be created. This function is defined as follows: int svc_create_svcsock(struct svc_serv, char *transport_name, unsigned short port, int flags); The existing svc_makesock interface was left to avoid impacts to existing servers. It has been modified to map IP protocol numbers to transport strings. -- Signed-off-by: Tom Tucker From tom at opengridcomputing.com Mon Aug 20 11:57:23 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:23 -0500 Subject: [ofa-general] [RFC, PATCH 01/20] svc: Add svc_xprt transport switch structure In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185722.18466.79547.stgit@dell3.ogc.int> Start moving to a transport switch for knfsd. Add a svc_xprt switch and move the sk_sendto and sk_recvfrom function pointers into it. Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 9 +++++++-- net/sunrpc/svcsock.c | 22 ++++++++++++++++------ 2 files changed, 23 insertions(+), 8 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index e21dd93..4792ed6 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -11,6 +11,12 @@ #define SUNRPC_SVCSOCK_H #include +struct svc_xprt { + const char *xpt_name; + int (*xpt_recvfrom)(struct svc_rqst *rqstp); + int (*xpt_sendto)(struct svc_rqst *rqstp); +}; + /* * RPC server socket. */ @@ -43,8 +49,7 @@ #define SK_DETACHED 10 /* detached fro * be revisted */ struct mutex sk_mutex; /* to serialize sending data */ - int (*sk_recvfrom)(struct svc_rqst *rqstp); - int (*sk_sendto)(struct svc_rqst *rqstp); + const struct svc_xprt *sk_xprt; /* We keep the old state_change and data_ready CB's here */ void (*sk_ostate)(struct sock *); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 5baf48d..789d94a 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -885,6 +885,12 @@ svc_udp_sendto(struct svc_rqst *rqstp) return error; } +static const struct svc_xprt svc_udp_xprt = { + .xpt_name = "udp", + .xpt_recvfrom = svc_udp_recvfrom, + .xpt_sendto = svc_udp_sendto, +}; + static void svc_udp_init(struct svc_sock *svsk) { @@ -893,8 +899,7 @@ svc_udp_init(struct svc_sock *svsk) svsk->sk_sk->sk_data_ready = svc_udp_data_ready; svsk->sk_sk->sk_write_space = svc_write_space; - svsk->sk_recvfrom = svc_udp_recvfrom; - svsk->sk_sendto = svc_udp_sendto; + svsk->sk_xprt = &svc_udp_xprt; /* initialise setting must have enough space to * receive and respond to one request. @@ -1322,14 +1327,19 @@ svc_tcp_sendto(struct svc_rqst *rqstp) return sent; } +static const struct svc_xprt svc_tcp_xprt = { + .xpt_name = "tcp", + .xpt_recvfrom = svc_tcp_recvfrom, + .xpt_sendto = svc_tcp_sendto, +}; + static void svc_tcp_init(struct svc_sock *svsk) { struct sock *sk = svsk->sk_sk; struct tcp_sock *tp = tcp_sk(sk); - svsk->sk_recvfrom = svc_tcp_recvfrom; - svsk->sk_sendto = svc_tcp_sendto; + svsk->sk_xprt = &svc_tcp_xprt; if (sk->sk_state == TCP_LISTEN) { dprintk("setting up TCP socket for listening\n"); @@ -1477,7 +1487,7 @@ svc_recv(struct svc_rqst *rqstp, long ti dprintk("svc: server %p, pool %u, socket %p, inuse=%d\n", rqstp, pool->sp_id, svsk, atomic_read(&svsk->sk_inuse)); - len = svsk->sk_recvfrom(rqstp); + len = svsk->sk_xprt->xpt_recvfrom(rqstp); dprintk("svc: got len=%d\n", len); /* No data, incomplete (TCP) read, or accept() */ @@ -1537,7 +1547,7 @@ svc_send(struct svc_rqst *rqstp) if (test_bit(SK_DEAD, &svsk->sk_flags)) len = -ENOTCONN; else - len = svsk->sk_sendto(rqstp); + len = svsk->sk_xprt->xpt_sendto(rqstp); mutex_unlock(&svsk->sk_mutex); svc_sock_release(rqstp); From tom at opengridcomputing.com Mon Aug 20 11:57:25 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:25 -0500 Subject: [ofa-general] [RFC,PATCH 02/20] svc: xpt_detach and xpt_free In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185725.18466.71434.stgit@dell3.ogc.int> Add transport switch functions to ensure that no additional receive ready events will be delivered by the transport (xpt_detach), and another to free memory associated with the transport (xpt_free). Change svc_delete_socket() and svc_sock_put() to use the new transport functions. Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 12 ++++++++++ net/sunrpc/svcsock.c | 50 +++++++++++++++++++++++++++++++++------- 2 files changed, 53 insertions(+), 9 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 4792ed6..27c5b1f 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -15,6 +15,18 @@ struct svc_xprt { const char *xpt_name; int (*xpt_recvfrom)(struct svc_rqst *rqstp); int (*xpt_sendto)(struct svc_rqst *rqstp); + /* + * Detach the svc_sock from it's socket, so that the + * svc_sock will not be enqueued any more. This is + * the first stage in the destruction of a svc_sock. + */ + void (*xpt_detach)(struct svc_sock *); + /* + * Release all network-level resources held by the svc_sock, + * and the svc_sock itself. This is the final stage in the + * destruction of a svc_sock. + */ + void (*xpt_free)(struct svc_sock *); }; /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 789d94a..4956c88 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -84,6 +84,8 @@ static void svc_udp_data_ready(struct s static int svc_udp_recvfrom(struct svc_rqst *); static int svc_udp_sendto(struct svc_rqst *); static void svc_close_socket(struct svc_sock *svsk); +static void svc_sock_detach(struct svc_sock *); +static void svc_sock_free(struct svc_sock *); static struct svc_deferred_req *svc_deferred_dequeue(struct svc_sock *svsk); static int svc_deferred_recv(struct svc_rqst *rqstp); @@ -378,14 +380,9 @@ svc_sock_put(struct svc_sock *svsk) if (atomic_dec_and_test(&svsk->sk_inuse)) { BUG_ON(! test_bit(SK_DEAD, &svsk->sk_flags)); - dprintk("svc: releasing dead socket\n"); - if (svsk->sk_sock->file) - sockfd_put(svsk->sk_sock); - else - sock_release(svsk->sk_sock); if (svsk->sk_info_authunix != NULL) svcauth_unix_info_release(svsk->sk_info_authunix); - kfree(svsk); + svsk->sk_xprt->xpt_free(svsk); } } @@ -889,6 +886,8 @@ static const struct svc_xprt svc_udp_xpr .xpt_name = "udp", .xpt_recvfrom = svc_udp_recvfrom, .xpt_sendto = svc_udp_sendto, + .xpt_detach = svc_sock_detach, + .xpt_free = svc_sock_free, }; static void @@ -1331,6 +1330,8 @@ static const struct svc_xprt svc_tcp_xpr .xpt_name = "tcp", .xpt_recvfrom = svc_tcp_recvfrom, .xpt_sendto = svc_tcp_sendto, + .xpt_detach = svc_sock_detach, + .xpt_free = svc_sock_free, }; static void @@ -1770,6 +1771,38 @@ bummer: } /* + * Detach the svc_sock from the socket so that no + * more callbacks occur. + */ +static void +svc_sock_detach(struct svc_sock *svsk) +{ + struct sock *sk = svsk->sk_sk; + + dprintk("svc: svc_sock_detach(%p)\n", svsk); + + /* put back the old socket callbacks */ + sk->sk_state_change = svsk->sk_ostate; + sk->sk_data_ready = svsk->sk_odata; + sk->sk_write_space = svsk->sk_owspace; +} + +/* + * Free the svc_sock's socket resources and the svc_sock itself. + */ +static void +svc_sock_free(struct svc_sock *svsk) +{ + dprintk("svc: svc_sock_free(%p)\n", svsk); + + if (svsk->sk_sock->file) + sockfd_put(svsk->sk_sock); + else + sock_release(svsk->sk_sock); + kfree(svsk); +} + +/* * Remove a dead socket */ static void @@ -1783,9 +1816,8 @@ svc_delete_socket(struct svc_sock *svsk) serv = svsk->sk_server; sk = svsk->sk_sk; - sk->sk_state_change = svsk->sk_ostate; - sk->sk_data_ready = svsk->sk_odata; - sk->sk_write_space = svsk->sk_owspace; + if (svsk->sk_xprt->xpt_detach) + svsk->sk_xprt->xpt_detach(svsk); spin_lock_bh(&serv->sv_lock); From tom at opengridcomputing.com Mon Aug 20 11:57:28 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:28 -0500 Subject: [ofa-general] [RFC,PATCH 03/20] svc: xpt_prep_reply_hdr In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185727.18466.9634.stgit@dell3.ogc.int> Add a transport function that prepares the transport specific header for RPC replies. UDP has none, TCP has a 4B record length. This will allow the RDMA transport to prepare it's variable length reply header as well. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 4 ++++ net/sunrpc/svc.c | 8 +++++--- net/sunrpc/svcsock.c | 15 +++++++++++++++ 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 27c5b1f..1da42c2 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -27,6 +27,10 @@ struct svc_xprt { * destruction of a svc_sock. */ void (*xpt_free)(struct svc_sock *); + /* + * Prepare any transport-specific RPC header. + */ + int (*xpt_prep_reply_hdr)(struct svc_rqst *); }; /* diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index e673ef9..72a900f 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -815,9 +815,11 @@ svc_process(struct svc_rqst *rqstp) rqstp->rq_res.tail[0].iov_len = 0; /* Will be turned off only in gss privacy case: */ rqstp->rq_sendfile_ok = 1; - /* tcp needs a space for the record length... */ - if (rqstp->rq_prot == IPPROTO_TCP) - svc_putnl(resv, 0); + + /* setup response header. */ + if (rqstp->rq_sock->sk_xprt->xpt_prep_reply_hdr && + rqstp->rq_sock->sk_xprt->xpt_prep_reply_hdr(rqstp)) + goto dropit; rqstp->rq_xid = svc_getu32(argv); svc_putu32(resv, rqstp->rq_xid); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 4956c88..ca473ee 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1326,12 +1326,27 @@ svc_tcp_sendto(struct svc_rqst *rqstp) return sent; } +/* + * Setup response header. TCP has a 4B record length field. + */ +static int +svc_tcp_prep_reply_hdr(struct svc_rqst *rqstp) +{ + struct kvec *resv = &rqstp->rq_res.head[0]; + + /* tcp needs a space for the record length... */ + svc_putnl(resv, 0); + + return 0; +} + static const struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", .xpt_recvfrom = svc_tcp_recvfrom, .xpt_sendto = svc_tcp_sendto, .xpt_detach = svc_sock_detach, .xpt_free = svc_sock_free, + .xpt_prep_reply_hdr = svc_tcp_prep_reply_hdr, }; static void From tom at opengridcomputing.com Mon Aug 20 11:57:30 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:30 -0500 Subject: [ofa-general] [RFC,PATCH 04/20] svc: xpt_has_wspace In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185730.18466.85138.stgit@dell3.ogc.int> Move the code that checks for available write space on the socket, into a new transport function. This will allow transports flexibility when determining if enough space/memory is available to process the reply. The role of this function for RDMA is to avoid stalling an knfsd thread when SQ space is not available. Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 4 ++ net/sunrpc/svcsock.c | 75 ++++++++++++++++++++++++++-------------- 2 files changed, 52 insertions(+), 27 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 1da42c2..3faa95c 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -31,6 +31,10 @@ struct svc_xprt { * Prepare any transport-specific RPC header. */ int (*xpt_prep_reply_hdr)(struct svc_rqst *); + /* + * Return 1 if sufficient space to write reply to network. + */ + int (*xpt_has_wspace)(struct svc_sock *); }; /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index ca473ee..b16dad4 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -205,22 +205,6 @@ svc_release_skb(struct svc_rqst *rqstp) } /* - * Any space to write? - */ -static inline unsigned long -svc_sock_wspace(struct svc_sock *svsk) -{ - int wspace; - - if (svsk->sk_sock->type == SOCK_STREAM) - wspace = sk_stream_wspace(svsk->sk_sk); - else - wspace = sock_wspace(svsk->sk_sk); - - return wspace; -} - -/* * Queue up a socket with data pending. If there are idle nfsd * processes, wake 'em up. * @@ -269,21 +253,13 @@ svc_sock_enqueue(struct svc_sock *svsk) BUG_ON(svsk->sk_pool != NULL); svsk->sk_pool = pool; - set_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); - if (((atomic_read(&svsk->sk_reserved) + serv->sv_max_mesg)*2 - > svc_sock_wspace(svsk)) - && !test_bit(SK_CLOSE, &svsk->sk_flags) - && !test_bit(SK_CONN, &svsk->sk_flags)) { - /* Don't enqueue while not enough space for reply */ - dprintk("svc: socket %p no space, %d*2 > %ld, not enqueued\n", - svsk->sk_sk, atomic_read(&svsk->sk_reserved)+serv->sv_max_mesg, - svc_sock_wspace(svsk)); + if (!test_bit(SK_CLOSE, &svsk->sk_flags) + && !test_bit(SK_CONN, &svsk->sk_flags) + && !svsk->sk_xprt->xpt_has_wspace(svsk)) { svsk->sk_pool = NULL; clear_bit(SK_BUSY, &svsk->sk_flags); goto out_unlock; } - clear_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); - if (!list_empty(&pool->sp_threads)) { rqstp = list_entry(pool->sp_threads.next, @@ -882,12 +858,45 @@ svc_udp_sendto(struct svc_rqst *rqstp) return error; } +/** + * svc_sock_has_write_space - Checks if there is enough space + * to send the reply on the socket. + * @svsk: the svc_sock to write on + * @wspace: the number of bytes available for writing + */ +static int svc_sock_has_write_space(struct svc_sock *svsk, int wspace) +{ + struct svc_serv *serv = svsk->sk_server; + int required = atomic_read(&svsk->sk_reserved) + serv->sv_max_mesg; + + if (required*2 > wspace) { + /* Don't enqueue while not enough space for reply */ + dprintk("svc: socket %p no space, %d*2 > %d, not enqueued\n", + svsk->sk_sk, required, wspace); + return 0; + } + clear_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); + return 1; +} + +static int +svc_udp_has_wspace(struct svc_sock *svsk) +{ + /* + * Set the SOCK_NOSPACE flag before checking the available + * sock space. + */ + set_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); + return svc_sock_has_write_space(svsk, sock_wspace(svsk->sk_sk)); +} + static const struct svc_xprt svc_udp_xprt = { .xpt_name = "udp", .xpt_recvfrom = svc_udp_recvfrom, .xpt_sendto = svc_udp_sendto, .xpt_detach = svc_sock_detach, .xpt_free = svc_sock_free, + .xpt_has_wspace = svc_udp_has_wspace, }; static void @@ -1340,6 +1349,17 @@ svc_tcp_prep_reply_hdr(struct svc_rqst * return 0; } +static int +svc_tcp_has_wspace(struct svc_sock *svsk) +{ + /* + * Set the SOCK_NOSPACE flag before checking the available + * sock space. + */ + set_bit(SOCK_NOSPACE, &svsk->sk_sock->flags); + return svc_sock_has_write_space(svsk, sk_stream_wspace(svsk->sk_sk)); +} + static const struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", .xpt_recvfrom = svc_tcp_recvfrom, @@ -1347,6 +1367,7 @@ static const struct svc_xprt svc_tcp_xpr .xpt_detach = svc_sock_detach, .xpt_free = svc_sock_free, .xpt_prep_reply_hdr = svc_tcp_prep_reply_hdr, + .xpt_has_wspace = svc_tcp_has_wspace, }; static void From tom at opengridcomputing.com Mon Aug 20 11:57:34 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:34 -0500 Subject: [ofa-general] [RFC, PATCH 06/20] svc: export svc_sock_enqueue, svc_sock_received In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185734.18466.31124.stgit@dell3.ogc.int> Export svc_sock_enqueue() and svc_sock_received() so they can be used by sunrpc server transport implementations (even future modular ones). Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 2 ++ net/sunrpc/svcsock.c | 7 ++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 4e24e6d..0145057 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -108,6 +108,8 @@ int svc_addsock(struct svc_serv *serv, int fd, char *name_return, int *proto); +void svc_sock_enqueue(struct svc_sock *svsk); +void svc_sock_received(struct svc_sock *svsk); /* * svc_makesock socket characteristics diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 0dc94a8..8fad53d 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -209,7 +209,7 @@ svc_release_skb(struct svc_rqst *rqstp) * processes, wake 'em up. * */ -static void +void svc_sock_enqueue(struct svc_sock *svsk) { struct svc_serv *serv = svsk->sk_server; @@ -287,6 +287,7 @@ svc_sock_enqueue(struct svc_sock *svsk) out_unlock: spin_unlock_bh(&pool->sp_lock); } +EXPORT_SYMBOL_GPL(svc_sock_enqueue); /* * Dequeue the first socket. Must be called with the pool->sp_lock held. @@ -315,14 +316,14 @@ svc_sock_dequeue(struct svc_pool *pool) * Note: SK_DATA only gets cleared when a read-attempt finds * no (or insufficient) data. */ -static inline void +void svc_sock_received(struct svc_sock *svsk) { svsk->sk_pool = NULL; clear_bit(SK_BUSY, &svsk->sk_flags); svc_sock_enqueue(svsk); } - +EXPORT_SYMBOL_GPL(svc_sock_received); /** * svc_reserve - change the space reserved for the reply to a request. From tom at opengridcomputing.com Mon Aug 20 11:57:37 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:37 -0500 Subject: [ofa-general] [RFC,PATCH 07/20] svc: centralise close handling In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185736.18466.5071.stgit@dell3.ogc.int> Centralise the handling of the SK_CLOSE bit so that future server transport implementations will be easier to write correctly. The xpt_recvfrom method does not need to check for SK_CLOSE anymore, that's handled in core code. Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- net/sunrpc/svcsock.c | 28 +++++++++++++--------------- 1 files changed, 13 insertions(+), 15 deletions(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 8fad53d..5c3a794 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -756,11 +756,6 @@ svc_udp_recvfrom(struct svc_rqst *rqstp) return svc_deferred_recv(rqstp); } - if (test_bit(SK_CLOSE, &svsk->sk_flags)) { - svc_delete_socket(svsk); - return 0; - } - clear_bit(SK_DATA, &svsk->sk_flags); skb = NULL; err = kernel_recvmsg(svsk->sk_sock, &msg, NULL, @@ -1158,11 +1153,6 @@ svc_tcp_recvfrom(struct svc_rqst *rqstp) return svc_deferred_recv(rqstp); } - if (test_bit(SK_CLOSE, &svsk->sk_flags)) { - svc_delete_socket(svsk); - return 0; - } - if (svsk->sk_sk->sk_state == TCP_LISTEN) { svc_tcp_accept(svsk); svc_sock_received(svsk); @@ -1406,8 +1396,10 @@ svc_tcp_init(struct svc_sock *svsk) set_bit(SK_CHNGBUF, &svsk->sk_flags); set_bit(SK_DATA, &svsk->sk_flags); - if (sk->sk_state != TCP_ESTABLISHED) + if (sk->sk_state != TCP_ESTABLISHED) { + /* note: caller calls svc_sock_enqueue() */ set_bit(SK_CLOSE, &svsk->sk_flags); + } } } @@ -1525,10 +1517,16 @@ svc_recv(struct svc_rqst *rqstp, long ti } spin_unlock_bh(&pool->sp_lock); - dprintk("svc: server %p, pool %u, socket %p, inuse=%d\n", - rqstp, pool->sp_id, svsk, atomic_read(&svsk->sk_inuse)); - len = svsk->sk_xprt->xpt_recvfrom(rqstp); - dprintk("svc: got len=%d\n", len); + len = 0; + if (test_bit(SK_CLOSE, &svsk->sk_flags)) { + dprintk("svc_recv: found SK_CLOSE\n"); + svc_delete_socket(svsk); + } else { + dprintk("svc: server %p, pool %u, socket %p, inuse=%d\n", + rqstp, pool->sp_id, svsk, atomic_read(&svsk->sk_inuse)); + len = svsk->sk_xprt->xpt_recvfrom(rqstp); + dprintk("svc: got len=%d\n", len); + } /* No data, incomplete (TCP) read, or accept() */ if (len == 0 || len == -EAGAIN) { From tom at opengridcomputing.com Mon Aug 20 11:57:39 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:39 -0500 Subject: [ofa-general] [RFC,PATCH 08/20] svc: centralise accept handling In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185739.18466.81457.stgit@dell3.ogc.int> Centralise the handling of the SK_CONN bit so that future server transport implementations will be easier to write correctly. Also, the xpt_recvfrom method does not need to check for SK_CONN anymore, that's handled in core code which calls a new xpt_accept method. Signed-off-by: Greg Banks Signed-off-by: Peter Leckie Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 4 ++++ net/sunrpc/svcsock.c | 24 +++++++++++------------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 0145057..7663578 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -41,6 +41,10 @@ struct svc_xprt { * svc_sock. */ u32 xpt_max_payload; + /* + * Accept a pending connection, for connection-oriented transports + */ + int (*xpt_accept)(struct svc_sock *svsk); }; /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 5c3a794..94eb921 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1012,7 +1012,7 @@ static inline int svc_port_is_privileged /* * Accept a TCP connection */ -static void +static int svc_tcp_accept(struct svc_sock *svsk) { struct sockaddr_storage addr; @@ -1021,12 +1021,12 @@ svc_tcp_accept(struct svc_sock *svsk) struct socket *sock = svsk->sk_sock; struct socket *newsock; struct svc_sock *newsvsk; - int err, slen; + int err = 0, slen; char buf[RPC_MAX_ADDRBUFLEN]; dprintk("svc: tcp_accept %p sock %p\n", svsk, sock); if (!sock) - return; + return -EINVAL; clear_bit(SK_CONN, &svsk->sk_flags); err = kernel_accept(sock, &newsock, O_NONBLOCK); @@ -1037,9 +1037,8 @@ svc_tcp_accept(struct svc_sock *svsk) else if (err != -EAGAIN && net_ratelimit()) printk(KERN_WARNING "%s: accept failed (err %d)!\n", serv->sv_name, -err); - return; + return err; } - set_bit(SK_CONN, &svsk->sk_flags); svc_sock_enqueue(svsk); @@ -1124,11 +1123,11 @@ svc_tcp_accept(struct svc_sock *svsk) if (serv->sv_stats) serv->sv_stats->nettcpconn++; - return; + return 0; failed: sock_release(newsock); - return; + return err; } /* @@ -1153,12 +1152,6 @@ svc_tcp_recvfrom(struct svc_rqst *rqstp) return svc_deferred_recv(rqstp); } - if (svsk->sk_sk->sk_state == TCP_LISTEN) { - svc_tcp_accept(svsk); - svc_sock_received(svsk); - return 0; - } - if (test_and_clear_bit(SK_CHNGBUF, &svsk->sk_flags)) /* sndbuf needs to have room for one request * per thread, otherwise we can stall even when the @@ -1361,6 +1354,7 @@ static const struct svc_xprt svc_tcp_xpr .xpt_prep_reply_hdr = svc_tcp_prep_reply_hdr, .xpt_has_wspace = svc_tcp_has_wspace, .xpt_max_payload = RPCSVC_MAXPAYLOAD_TCP, + .xpt_accept = svc_tcp_accept, }; static void @@ -1521,6 +1515,9 @@ svc_recv(struct svc_rqst *rqstp, long ti if (test_bit(SK_CLOSE, &svsk->sk_flags)) { dprintk("svc_recv: found SK_CLOSE\n"); svc_delete_socket(svsk); + } else if (svsk->sk_sk->sk_state == TCP_LISTEN) { + svsk->sk_xprt->xpt_accept(svsk); + svc_sock_received(svsk); } else { dprintk("svc: server %p, pool %u, socket %p, inuse=%d\n", rqstp, pool->sp_id, svsk, atomic_read(&svsk->sk_inuse)); @@ -1660,6 +1657,7 @@ static struct svc_sock *svc_setup_socket int is_temporary = flags & SVC_SOCK_TEMPORARY; dprintk("svc: svc_setup_socket %p\n", sock); + *errp = 0; if (!(svsk = kzalloc(sizeof(*svsk), GFP_KERNEL))) { *errp = -ENOMEM; return NULL; From tom at opengridcomputing.com Mon Aug 20 11:57:45 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:45 -0500 Subject: [ofa-general] [RFC,PATCH 10/20] svc: Add generic refcount services In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185744.18466.49815.stgit@dell3.ogc.int> Add inline svc_sock_get() so that service transport code will not need to manipulate sk_inuse directly. Also, make svc_sock_put() available so that transport code outside svcsock.c can use it. Signed-off-by: Greg Banks Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 15 +++++++++++++++ net/sunrpc/svcsock.c | 29 ++++++++++++++--------------- 2 files changed, 29 insertions(+), 15 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index ea8b62b..9f37f30 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -115,6 +115,7 @@ int svc_addsock(struct svc_serv *serv, int *proto); void svc_sock_enqueue(struct svc_sock *svsk); void svc_sock_received(struct svc_sock *svsk); +void __svc_sock_put(struct svc_sock *svsk); /* * svc_makesock socket characteristics @@ -123,4 +124,18 @@ #define SVC_SOCK_DEFAULTS (0U) #define SVC_SOCK_ANONYMOUS (1U << 0) /* don't register with pmap */ #define SVC_SOCK_TEMPORARY (1U << 1) /* flag socket as temporary */ +/* + * Take and drop a temporary reference count on the svc_sock. + */ +static inline void svc_sock_get(struct svc_sock *svsk) +{ + atomic_inc(&svsk->sk_inuse); +} + +static inline void svc_sock_put(struct svc_sock *svsk) +{ + if (atomic_dec_and_test(&svsk->sk_inuse)) + __svc_sock_put(svsk); +} + #endif /* SUNRPC_SVCSOCK_H */ diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index dcb5c7a..02f682a 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -273,7 +273,7 @@ svc_sock_enqueue(struct svc_sock *svsk) "svc_sock_enqueue: server %p, rq_sock=%p!\n", rqstp, rqstp->rq_sock); rqstp->rq_sock = svsk; - atomic_inc(&svsk->sk_inuse); + svc_sock_get(svsk); rqstp->rq_reserved = serv->sv_max_mesg; atomic_add(rqstp->rq_reserved, &svsk->sk_reserved); BUG_ON(svsk->sk_pool != pool); @@ -351,17 +351,16 @@ void svc_reserve(struct svc_rqst *rqstp, /* * Release a socket after use. */ -static inline void -svc_sock_put(struct svc_sock *svsk) +void +__svc_sock_put(struct svc_sock *svsk) { - if (atomic_dec_and_test(&svsk->sk_inuse)) { - BUG_ON(! test_bit(SK_DEAD, &svsk->sk_flags)); + BUG_ON(! test_bit(SK_DEAD, &svsk->sk_flags)); - if (svsk->sk_info_authunix != NULL) - svcauth_unix_info_release(svsk->sk_info_authunix); - svsk->sk_xprt->xpt_free(svsk); - } + if (svsk->sk_info_authunix != NULL) + svcauth_unix_info_release(svsk->sk_info_authunix); + svsk->sk_xprt->xpt_free(svsk); } +EXPORT_SYMBOL_GPL(__svc_sock_put); static void svc_sock_release(struct svc_rqst *rqstp) @@ -1109,7 +1108,7 @@ svc_tcp_accept(struct svc_sock *svsk) struct svc_sock, sk_list); set_bit(SK_CLOSE, &svsk->sk_flags); - atomic_inc(&svsk->sk_inuse); + svc_sock_get(svsk); } spin_unlock_bh(&serv->sv_lock); @@ -1481,7 +1480,7 @@ svc_recv(struct svc_rqst *rqstp, long ti spin_lock_bh(&pool->sp_lock); if ((svsk = svc_sock_dequeue(pool)) != NULL) { rqstp->rq_sock = svsk; - atomic_inc(&svsk->sk_inuse); + svc_sock_get(svsk); rqstp->rq_reserved = serv->sv_max_mesg; atomic_add(rqstp->rq_reserved, &svsk->sk_reserved); } else { @@ -1620,7 +1619,7 @@ svc_age_temp_sockets(unsigned long closu continue; if (atomic_read(&svsk->sk_inuse) || test_bit(SK_BUSY, &svsk->sk_flags)) continue; - atomic_inc(&svsk->sk_inuse); + svc_sock_get(svsk); list_move(le, &to_be_aged); set_bit(SK_CLOSE, &svsk->sk_flags); set_bit(SK_DETACHED, &svsk->sk_flags); @@ -1868,7 +1867,7 @@ svc_delete_socket(struct svc_sock *svsk) */ if (!test_and_set_bit(SK_DEAD, &svsk->sk_flags)) { BUG_ON(atomic_read(&svsk->sk_inuse)<2); - atomic_dec(&svsk->sk_inuse); + svc_sock_put(svsk); if (test_bit(SK_TEMP, &svsk->sk_flags)) serv->sv_tmpcnt--; } @@ -1883,7 +1882,7 @@ static void svc_close_socket(struct svc_ /* someone else will have to effect the close */ return; - atomic_inc(&svsk->sk_inuse); + svc_sock_get(svsk); svc_delete_socket(svsk); clear_bit(SK_BUSY, &svsk->sk_flags); svc_sock_put(svsk); @@ -1976,7 +1975,7 @@ svc_defer(struct cache_req *req) dr->argslen = rqstp->rq_arg.len >> 2; memcpy(dr->args, rqstp->rq_arg.head[0].iov_base-skip, dr->argslen<<2); } - atomic_inc(&rqstp->rq_sock->sk_inuse); + svc_sock_get(rqstp->rq_sock); dr->svsk = rqstp->rq_sock; dr->handle.revisit = svc_revisit; From tom at opengridcomputing.com Mon Aug 20 11:57:47 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:47 -0500 Subject: [ofa-general] [RFC, PATCH 11/20] svc: cleanup svc_sock initialization In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185747.18466.61852.stgit@dell3.ogc.int> Reorganise the svc_sock initialisation code so that new service transport code can use it without duplicating lots of code that futzes with internal transport details (for example the SK_BUSY bit). Transport code should now call svc_sock_init() to initialise the svc_sock structure, then one of svc_sock_add_listener sock_add_connection or svc_sock_add_connectionless, and finally svc_sock_received. Signed-off-by: Greg Banks Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 10 +++ net/sunrpc/svcsock.c | 143 ++++++++++++++++++++++++++-------------- 2 files changed, 103 insertions(+), 50 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 9f37f30..7def951 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -116,6 +116,16 @@ int svc_addsock(struct svc_serv *serv, void svc_sock_enqueue(struct svc_sock *svsk); void svc_sock_received(struct svc_sock *svsk); void __svc_sock_put(struct svc_sock *svsk); +/* Initialise a newly allocated svc_sock. The transport code needs + * to call svc_sock_received() when transport-specific initialisation + * is complete and one of the svc_add_*() functions has been called. */ +void svc_sock_init(struct svc_sock *, struct svc_serv *); +/* Add an initialised connection svc_sock to the server */ +void svc_sock_add_connection(struct svc_sock *); +/* Add an initialised listener svc_sock to the server */ +void svc_sock_add_listener(struct svc_sock *); +/* Add an initialised connectionless svc_sock to the server */ +void svc_sock_add_connectionless(struct svc_sock *); /* * svc_makesock socket characteristics diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 02f682a..7d219de 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1357,44 +1357,49 @@ static const struct svc_xprt svc_tcp_xpr }; static void -svc_tcp_init(struct svc_sock *svsk) +svc_tcp_init_listener(struct svc_sock *svsk) +{ + struct sock *sk = svsk->sk_sk; + + svsk->sk_xprt = &svc_tcp_xprt; + + dprintk("setting up TCP socket for listening\n"); + sk->sk_data_ready = svc_tcp_listen_data_ready; + set_bit(SK_LISTENER, &svsk->sk_flags); + set_bit(SK_CONN, &svsk->sk_flags); +} + +static void +svc_tcp_init_connection(struct svc_sock *svsk) { struct sock *sk = svsk->sk_sk; struct tcp_sock *tp = tcp_sk(sk); svsk->sk_xprt = &svc_tcp_xprt; - if (sk->sk_state == TCP_LISTEN) { - dprintk("setting up TCP socket for listening\n"); - sk->sk_data_ready = svc_tcp_listen_data_ready; - set_bit(SK_LISTENER, &svsk->sk_flags); - set_bit(SK_CONN, &svsk->sk_flags); - } else { - dprintk("setting up TCP socket for reading\n"); - sk->sk_state_change = svc_tcp_state_change; - sk->sk_data_ready = svc_tcp_data_ready; - sk->sk_write_space = svc_write_space; + dprintk("setting up TCP socket for reading\n"); + sk->sk_state_change = svc_tcp_state_change; + sk->sk_data_ready = svc_tcp_data_ready; + sk->sk_write_space = svc_write_space; - svsk->sk_reclen = 0; - svsk->sk_tcplen = 0; + svsk->sk_reclen = 0; + svsk->sk_tcplen = 0; - tp->nonagle = 1; /* disable Nagle's algorithm */ + tp->nonagle = 1; /* disable Nagle's algorithm */ - /* initialise setting must have enough space to - * receive and respond to one request. - * svc_tcp_recvfrom will re-adjust if necessary - */ - svc_sock_setbufsize(svsk->sk_sock, - 3 * svsk->sk_server->sv_max_mesg, - 3 * svsk->sk_server->sv_max_mesg); + /* + * Initialise setting must have enough space to receive and + * respond to one request. svc_tcp_recvfrom will re-adjust if + * necessary + */ + svc_sock_setbufsize(svsk->sk_sock, + 3 * svsk->sk_server->sv_max_mesg, + 3 * svsk->sk_server->sv_max_mesg); - set_bit(SK_CHNGBUF, &svsk->sk_flags); - set_bit(SK_DATA, &svsk->sk_flags); - if (sk->sk_state != TCP_ESTABLISHED) { - /* note: caller calls svc_sock_enqueue() */ - set_bit(SK_CLOSE, &svsk->sk_flags); - } - } + set_bit(SK_CHNGBUF, &svsk->sk_flags); + set_bit(SK_DATA, &svsk->sk_flags); + if (sk->sk_state != TCP_ESTABLISHED) + set_bit(SK_CLOSE, &svsk->sk_flags); } void @@ -1682,6 +1687,29 @@ static struct svc_sock *svc_setup_socket svsk->sk_ostate = inet->sk_state_change; svsk->sk_odata = inet->sk_data_ready; svsk->sk_owspace = inet->sk_write_space; + svc_sock_init(svsk, serv); + + /* Initialize the socket */ + if (sock->type == SOCK_DGRAM) { + svc_udp_init(svsk); + svc_sock_add_connectionless(svsk); + } else if (inet->sk_state == TCP_LISTEN) { + BUG_ON(is_temporary); + svc_tcp_init_listener(svsk); + svc_sock_add_listener(svsk); + } else { + BUG_ON(!is_temporary); + svc_tcp_init_connection(svsk); + svc_sock_add_connection(svsk); + } + + dprintk("svc: svc_setup_socket created %p (inet %p)\n", + svsk, svsk->sk_sk); + return svsk; +} + +void svc_sock_init(struct svc_sock *svsk, struct svc_serv *serv) +{ svsk->sk_server = serv; atomic_set(&svsk->sk_inuse, 1); svsk->sk_lastrecv = get_seconds(); @@ -1689,36 +1717,51 @@ static struct svc_sock *svc_setup_socket INIT_LIST_HEAD(&svsk->sk_deferred); INIT_LIST_HEAD(&svsk->sk_ready); mutex_init(&svsk->sk_mutex); +} +EXPORT_SYMBOL_GPL(svc_sock_init); - /* Initialize the socket */ - if (sock->type == SOCK_DGRAM) - svc_udp_init(svsk); - else - svc_tcp_init(svsk); +void svc_sock_add_connection(struct svc_sock *svsk) +{ + struct svc_serv *serv = svsk->sk_server; spin_lock_bh(&serv->sv_lock); - if (is_temporary) { - set_bit(SK_TEMP, &svsk->sk_flags); - list_add(&svsk->sk_list, &serv->sv_tempsocks); - serv->sv_tmpcnt++; - if (serv->sv_temptimer.function == NULL) { - /* setup timer to age temp sockets */ - setup_timer(&serv->sv_temptimer, svc_age_temp_sockets, - (unsigned long)serv); - mod_timer(&serv->sv_temptimer, - jiffies + svc_conn_age_period * HZ); - } - } else { - clear_bit(SK_TEMP, &svsk->sk_flags); - list_add(&svsk->sk_list, &serv->sv_permsocks); + + set_bit(SK_TEMP, &svsk->sk_flags); + list_add(&svsk->sk_list, &serv->sv_tempsocks); + serv->sv_tmpcnt++; + if (serv->sv_temptimer.function == NULL) { + /* setup timer to age temp sockets */ + setup_timer(&serv->sv_temptimer, svc_age_temp_sockets, + (unsigned long)serv); + mod_timer(&serv->sv_temptimer, + jiffies + svc_conn_age_period * HZ); } + spin_unlock_bh(&serv->sv_lock); +} +EXPORT_SYMBOL_GPL(svc_sock_add_connection); - dprintk("svc: svc_setup_socket created %p (inet %p)\n", - svsk, svsk->sk_sk); +static void svc_sock_add_permanent(struct svc_sock *svsk) +{ + struct svc_serv *serv = svsk->sk_server; - return svsk; + BUG_ON(test_bit(SK_TEMP, &svsk->sk_flags)); + spin_lock_bh(&serv->sv_lock); + list_add(&svsk->sk_list, &serv->sv_permsocks); + spin_unlock_bh(&serv->sv_lock); +} + +void svc_sock_add_listener(struct svc_sock *svsk) +{ + svc_sock_add_permanent(svsk); +} +EXPORT_SYMBOL_GPL(svc_sock_add_listener); + +void svc_sock_add_connectionless(struct svc_sock *svsk) +{ + svc_sock_add_permanent(svsk); } +EXPORT_SYMBOL_GPL(svc_sock_add_connectionless); int svc_addsock(struct svc_serv *serv, int fd, From tom at opengridcomputing.com Mon Aug 20 11:57:53 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:53 -0500 Subject: [ofa-general] [RFC,PATCH 13/20] svc: Add svc_[un]register_transport In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185752.18466.82074.stgit@dell3.ogc.int> Add an exported function for transport modules to [un]register themselves with the sunrpc server side transport switch. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 6 +++++ net/sunrpc/svcsock.c | 50 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 0 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 7def951..cc911ab 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -13,6 +13,7 @@ #include struct svc_xprt { const char *xpt_name; + struct module *xpt_owner; int (*xpt_recvfrom)(struct svc_rqst *rqstp); int (*xpt_sendto)(struct svc_rqst *rqstp); /* @@ -45,7 +46,10 @@ struct svc_xprt { * Accept a pending connection, for connection-oriented transports */ int (*xpt_accept)(struct svc_sock *svsk); + /* Transport list link */ + struct list_head xpt_list; }; +extern struct list_head svc_transport_list; /* * RPC server socket. @@ -102,6 +106,8 @@ #define SK_LISTENER 11 /* listener (e. /* * Function prototypes. */ +int svc_register_transport(struct svc_xprt *xprt); +int svc_unregister_transport(struct svc_xprt *xprt); int svc_makesock(struct svc_serv *, int, unsigned short, int flags); void svc_force_close_socket(struct svc_sock *); int svc_recv(struct svc_rqst *, long); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 6acf22f..6183951 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -91,6 +91,54 @@ static struct svc_deferred_req *svc_defe static int svc_deferred_recv(struct svc_rqst *rqstp); static struct cache_deferred_req *svc_defer(struct cache_req *req); +/* List of registered transports */ +static spinlock_t svc_transport_lock = SPIN_LOCK_UNLOCKED; +LIST_HEAD(svc_transport_list); + +int svc_register_transport(struct svc_xprt *xprt) +{ + struct svc_xprt *ops; + int res; + + dprintk("svc: Adding svc transport '%s'\n", + xprt->xpt_name); + + res = -EEXIST; + INIT_LIST_HEAD(&xprt->xpt_list); + spin_lock(&svc_transport_lock); + list_for_each_entry(ops, &svc_transport_list, xpt_list) { + if (xprt == ops) + goto out; + } + list_add_tail(&xprt->xpt_list, &svc_transport_list); + res = 0; +out: + spin_unlock(&svc_transport_lock); + return res; +} +EXPORT_SYMBOL_GPL(svc_register_transport); + +int svc_unregister_transport(struct svc_xprt *xprt) +{ + struct svc_xprt *ops; + int res = 0; + + dprintk("svc: Removing svc transport '%s'\n", xprt->xpt_name); + + spin_lock(&svc_transport_lock); + list_for_each_entry(ops, &svc_transport_list, xpt_list) { + if (xprt == ops) { + list_del_init(&ops->xpt_list); + goto out; + } + } + res = -ENOENT; + out: + spin_unlock(&svc_transport_lock); + return res; +} +EXPORT_SYMBOL_GPL(svc_unregister_transport); + /* apparently the "standard" is that clients close * idle connections after 5 minutes, servers after * 6 minutes @@ -887,6 +935,7 @@ svc_udp_has_wspace(struct svc_sock *svsk static const struct svc_xprt svc_udp_xprt = { .xpt_name = "udp", + .xpt_owner = THIS_MODULE, .xpt_recvfrom = svc_udp_recvfrom, .xpt_sendto = svc_udp_sendto, .xpt_detach = svc_sock_detach, @@ -1346,6 +1395,7 @@ svc_tcp_has_wspace(struct svc_sock *svsk static const struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", + .xpt_owner = THIS_MODULE, .xpt_recvfrom = svc_tcp_recvfrom, .xpt_sendto = svc_tcp_sendto, .xpt_detach = svc_sock_detach, From tom at opengridcomputing.com Mon Aug 20 11:57:56 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:56 -0500 Subject: [ofa-general] [RFC,PATCH 14/20] svc: Register TCP/UDP Transports In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185755.18466.54658.stgit@dell3.ogc.int> Add a call to svc_register_transport for the built in transports UDP and TCP. The registration is done in the sunrpc module initialization logic. Signed-off-by: Tom Tucker --- net/sunrpc/sunrpc_syms.c | 2 ++ net/sunrpc/svcsock.c | 10 ++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/sunrpc_syms.c b/net/sunrpc/sunrpc_syms.c index 73075de..c68577b 100644 --- a/net/sunrpc/sunrpc_syms.c +++ b/net/sunrpc/sunrpc_syms.c @@ -134,6 +134,7 @@ EXPORT_SYMBOL(nfsd_debug); EXPORT_SYMBOL(nlm_debug); #endif +extern void init_svc_xprt(void); extern struct cache_detail ip_map_cache, unix_gid_cache; static int __init @@ -156,6 +157,7 @@ #endif cache_register(&ip_map_cache); cache_register(&unix_gid_cache); init_socket_xprt(); + init_svc_xprt(); out: return err; } diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 6183951..d6443e8 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -933,7 +933,7 @@ svc_udp_has_wspace(struct svc_sock *svsk return svc_sock_has_write_space(svsk, sock_wspace(svsk->sk_sk)); } -static const struct svc_xprt svc_udp_xprt = { +static struct svc_xprt svc_udp_xprt = { .xpt_name = "udp", .xpt_owner = THIS_MODULE, .xpt_recvfrom = svc_udp_recvfrom, @@ -1393,7 +1393,7 @@ svc_tcp_has_wspace(struct svc_sock *svsk return svc_sock_has_write_space(svsk, sk_stream_wspace(svsk->sk_sk)); } -static const struct svc_xprt svc_tcp_xprt = { +static struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", .xpt_owner = THIS_MODULE, .xpt_recvfrom = svc_tcp_recvfrom, @@ -1406,6 +1406,12 @@ static const struct svc_xprt svc_tcp_xpr .xpt_accept = svc_tcp_accept, }; +void init_svc_xprt(void) +{ + svc_register_transport(&svc_udp_xprt); + svc_register_transport(&svc_tcp_xprt); +} + static void svc_tcp_init_listener(struct svc_sock *svsk) { From tom at opengridcomputing.com Mon Aug 20 11:57:58 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:57:58 -0500 Subject: [ofa-general] [RFC,PATCH 15/20] svc: transport file implementation In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185758.18466.94522.stgit@dell3.ogc.int> Create a proc/sys/sunrpc/transport file that contains information about the currently registered transports. Signed-off-by: Tom Tucker --- include/linux/sunrpc/debug.h | 1 + net/sunrpc/svcsock.c | 28 ++++++++++++++++++++++++++++ net/sunrpc/sysctl.c | 40 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 68 insertions(+), 1 deletions(-) diff --git a/include/linux/sunrpc/debug.h b/include/linux/sunrpc/debug.h index 10709cb..89458df 100644 --- a/include/linux/sunrpc/debug.h +++ b/include/linux/sunrpc/debug.h @@ -88,6 +88,7 @@ enum { CTL_SLOTTABLE_TCP, CTL_MIN_RESVPORT, CTL_MAX_RESVPORT, + CTL_TRANSPORTS, }; #endif /* _LINUX_SUNRPC_DEBUG_H_ */ diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index d6443e8..276737e 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -139,6 +139,34 @@ int svc_unregister_transport(struct svc_ } EXPORT_SYMBOL_GPL(svc_unregister_transport); +/* + * Format the transport list for printing + */ +int svc_print_transports(char *buf, int maxlen) +{ + struct list_head *le; + char tmpstr[80]; + int len = 0; + buf[0] = '\0'; + + spin_lock(&svc_transport_lock); + list_for_each(le, &svc_transport_list) { + int slen; + struct svc_xprt *xprt = + list_entry(le, struct svc_xprt, xpt_list); + + sprintf(tmpstr, "%s %d\n", xprt->xpt_name, xprt->xpt_max_payload); + slen = strlen(tmpstr); + if (len + slen > maxlen) + break; + len += slen; + strcat(buf, tmpstr); + } + spin_unlock(&svc_transport_lock); + + return len; +} + /* apparently the "standard" is that clients close * idle connections after 5 minutes, servers after * 6 minutes diff --git a/net/sunrpc/sysctl.c b/net/sunrpc/sysctl.c index 738db32..683cf90 100644 --- a/net/sunrpc/sysctl.c +++ b/net/sunrpc/sysctl.c @@ -27,6 +27,9 @@ unsigned int nfs_debug; unsigned int nfsd_debug; unsigned int nlm_debug; +/* Transport string */ +char xprt_buf[128]; + #ifdef RPC_DEBUG static struct ctl_table_header *sunrpc_table_header; @@ -48,6 +51,34 @@ rpc_unregister_sysctl(void) } } +int svc_print_transports(char *buf, int maxlen); +static int proc_do_xprt(ctl_table *table, int write, struct file *file, + void __user *buffer, size_t *lenp, loff_t *ppos) +{ + char tmpbuf[128]; + int len; + if ((*ppos && !write) || !*lenp) { + *lenp = 0; + return 0; + } + + if (write) + return -EINVAL; + else { + + len = svc_print_transports(tmpbuf, 128); + if (!access_ok(VERIFY_WRITE, buffer, len)) + return -EFAULT; + + if (__copy_to_user(buffer, tmpbuf, len)) + return -EFAULT; + } + + *lenp -= len; + *ppos += len; + return 0; +} + static int proc_dodebug(ctl_table *table, int write, struct file *file, void __user *buffer, size_t *lenp, loff_t *ppos) @@ -111,7 +142,6 @@ done: return 0; } - static ctl_table debug_table[] = { { .ctl_name = CTL_RPCDEBUG, @@ -145,6 +175,14 @@ static ctl_table debug_table[] = { .mode = 0644, .proc_handler = &proc_dodebug }, + { + .ctl_name = CTL_TRANSPORTS, + .procname = "transports", + .data = xprt_buf, + .maxlen = sizeof(xprt_buf), + .mode = 0444, + .proc_handler = &proc_do_xprt, + }, { .ctl_name = 0 } }; From tom at opengridcomputing.com Mon Aug 20 11:58:01 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:58:01 -0500 Subject: [ofa-general] [RFC,PATCH 16/20] svc: xpt_create_svc In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185800.18466.13401.stgit@dell3.ogc.int> Add transport function that makes the creation of a listening endpoint transport independent. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 5 +++ net/sunrpc/svcsock.c | 65 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 65 insertions(+), 5 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index cc911ab..e2d0256 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -14,6 +14,10 @@ #include struct svc_xprt { const char *xpt_name; struct module *xpt_owner; + /* Create an svc socket for this transport */ + int (*xpt_create_svc)(struct svc_serv *, + struct sockaddr *, + int); int (*xpt_recvfrom)(struct svc_rqst *rqstp); int (*xpt_sendto)(struct svc_rqst *rqstp); /* @@ -109,6 +113,7 @@ #define SK_LISTENER 11 /* listener (e. int svc_register_transport(struct svc_xprt *xprt); int svc_unregister_transport(struct svc_xprt *xprt); int svc_makesock(struct svc_serv *, int, unsigned short, int flags); +int svc_create_svcsock(struct svc_serv *, char *, unsigned short, int); void svc_force_close_socket(struct svc_sock *); int svc_recv(struct svc_rqst *, long); int svc_send(struct svc_rqst *); diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 276737e..44d6484 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -87,6 +87,8 @@ static void svc_close_socket(struct svc static void svc_sock_detach(struct svc_sock *); static void svc_sock_free(struct svc_sock *); +static int +svc_create_socket(struct svc_serv *, int, struct sockaddr *, int, int); static struct svc_deferred_req *svc_deferred_dequeue(struct svc_sock *svsk); static int svc_deferred_recv(struct svc_rqst *rqstp); static struct cache_deferred_req *svc_defer(struct cache_req *req); @@ -434,6 +436,7 @@ __svc_sock_put(struct svc_sock *svsk) if (svsk->sk_info_authunix != NULL) svcauth_unix_info_release(svsk->sk_info_authunix); + module_put(svsk->sk_xprt->xpt_owner); svsk->sk_xprt->xpt_free(svsk); } EXPORT_SYMBOL_GPL(__svc_sock_put); @@ -961,9 +964,17 @@ svc_udp_has_wspace(struct svc_sock *svsk return svc_sock_has_write_space(svsk, sock_wspace(svsk->sk_sk)); } +static int +svc_udp_create_svc(struct svc_serv *serv, struct sockaddr *sa, int flags) +{ + return svc_create_socket(serv, IPPROTO_UDP, sa, + sizeof(struct sockaddr_in), flags); +} + static struct svc_xprt svc_udp_xprt = { .xpt_name = "udp", .xpt_owner = THIS_MODULE, + .xpt_create_svc = svc_udp_create_svc, .xpt_recvfrom = svc_udp_recvfrom, .xpt_sendto = svc_udp_sendto, .xpt_detach = svc_sock_detach, @@ -1421,9 +1432,17 @@ svc_tcp_has_wspace(struct svc_sock *svsk return svc_sock_has_write_space(svsk, sk_stream_wspace(svsk->sk_sk)); } +static int +svc_tcp_create_svc(struct svc_serv *serv, struct sockaddr *sa, int flags) +{ + return svc_create_socket(serv, IPPROTO_TCP, sa, + sizeof(struct sockaddr_in), flags); +} + static struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", .xpt_owner = THIS_MODULE, + .xpt_create_svc = svc_tcp_create_svc, .xpt_recvfrom = svc_tcp_recvfrom, .xpt_sendto = svc_tcp_sendto, .xpt_detach = svc_sock_detach, @@ -1606,6 +1625,7 @@ svc_recv(struct svc_rqst *rqstp, long ti svc_delete_socket(svsk); } else if (test_bit(SK_LISTENER, &svsk->sk_flags)) { svsk->sk_xprt->xpt_accept(svsk); + __module_get(svsk->sk_xprt->xpt_owner); svc_sock_received(svsk); } else { dprintk("svc: server %p, pool %u, socket %p, inuse=%d\n", @@ -1885,7 +1905,7 @@ EXPORT_SYMBOL_GPL(svc_addsock); * Create socket for RPC service. */ static int svc_create_socket(struct svc_serv *serv, int protocol, - struct sockaddr *sin, int len, int flags) + struct sockaddr *sin, int len, int flags) { struct svc_sock *svsk; struct socket *sock; @@ -2037,18 +2057,53 @@ void svc_force_close_socket(struct svc_s * */ int svc_makesock(struct svc_serv *serv, int protocol, unsigned short port, - int flags) + int flags) { + dprintk("svc: creating socket proto = %d\n", protocol); + switch (protocol) { + case IPPROTO_TCP: + return svc_create_svcsock(serv, "tcp", port, flags); + case IPPROTO_UDP: + return svc_create_svcsock(serv, "udp", port, flags); + default: + return -EINVAL; + } +} + +int svc_create_svcsock(struct svc_serv *serv, char *transport, unsigned short port, + int flags) +{ + int ret = -ENOENT; + struct list_head *le; struct sockaddr_in sin = { .sin_family = AF_INET, .sin_addr.s_addr = INADDR_ANY, .sin_port = htons(port), }; + dprintk("svc: creating transport socket %s[%d]\n", transport, port); + spin_lock(&svc_transport_lock); + list_for_each(le, &svc_transport_list) { + struct svc_xprt *xprt = + list_entry(le, struct svc_xprt, xpt_list); - dprintk("svc: creating socket proto = %d\n", protocol); - return svc_create_socket(serv, protocol, (struct sockaddr *) &sin, - sizeof(sin), flags); + if (strcmp(transport, xprt->xpt_name)==0) { + spin_unlock(&svc_transport_lock); + if (try_module_get(xprt->xpt_owner)) { + ret = xprt->xpt_create_svc(serv, + (struct sockaddr*)&sin, + flags); + if (ret < 0) + module_put(xprt->xpt_owner); + goto out; + } + } + } + spin_unlock(&svc_transport_lock); + dprintk("svc: transport %s not found\n", transport); + out: + return ret; } +EXPORT_SYMBOL_GPL(svc_create_svcsock); /* * Handle defer and revisit of requests From tom at opengridcomputing.com Mon Aug 20 11:58:05 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:58:05 -0500 Subject: [ofa-general] [RFC, PATCH 18/20] svc: Add xpt_defer transport function In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185805.18466.78815.stgit@dell3.ogc.int> The RDMA transport includes an ONCRDMA header that precedes the RPC message. This header needs to be saved in addition to the RPC message itself. The RPC transport uses page swapping to implement copy avoidance. These transport dependencies are hidden in the xpt_defer routine allowing the bulk of the deferral processing to remain in transport independent code. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 5 +++++ net/sunrpc/svcsock.c | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index a920e9b..145c82b 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -51,6 +51,10 @@ struct svc_xprt { * Accept a pending connection, for connection-oriented transports */ int (*xpt_accept)(struct svc_sock *svsk); + + /* RPC defer routine. */ + struct cache_deferred_req *(*xpt_defer)(struct cache_req *req); + /* Transport list link */ struct list_head xpt_list; }; @@ -138,6 +142,7 @@ void svc_sock_add_connection(struct svc void svc_sock_add_listener(struct svc_sock *); /* Add an initialised connectionless svc_sock to the server */ void svc_sock_add_connectionless(struct svc_sock *); +void svc_revisit(struct cache_deferred_req *dreq, int too_many); /* * svc_makesock socket characteristics diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 03ce7e9..b89c577 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1651,7 +1651,7 @@ svc_recv(struct svc_rqst *rqstp, long ti clear_bit(SK_OLD, &svsk->sk_flags); rqstp->rq_secure = svc_port_is_privileged(svc_addr(rqstp)); - rqstp->rq_chandle.defer = svc_defer; + rqstp->rq_chandle.defer = svsk->sk_xprt->xpt_defer; if (serv->sv_stats) serv->sv_stats->netcnt++; @@ -2116,7 +2116,7 @@ EXPORT_SYMBOL_GPL(svc_create_svcsock); * Handle defer and revisit of requests */ -static void svc_revisit(struct cache_deferred_req *dreq, int too_many) +void svc_revisit(struct cache_deferred_req *dreq, int too_many) { struct svc_deferred_req *dr = container_of(dreq, struct svc_deferred_req, handle); struct svc_sock *svsk; @@ -2136,6 +2136,7 @@ static void svc_revisit(struct cache_def svc_sock_enqueue(svsk); svc_sock_put(svsk); } +EXPORT_SYMBOL_GPL(svc_revisit); static struct cache_deferred_req * svc_defer(struct cache_req *req) From tom at opengridcomputing.com Mon Aug 20 11:58:03 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:58:03 -0500 Subject: [ofa-general] [RFC,PATCH 17/20] svc: Add xpt_get_name service In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185803.18466.87448.stgit@dell3.ogc.int> Add an transport function to return a formatted socket name. This is used by knfsd when satisfying reads to the portlist file. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svcsock.h | 1 + net/sunrpc/svcsock.c | 17 ++++++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index e2d0256..a920e9b 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -18,6 +18,7 @@ struct svc_xprt { int (*xpt_create_svc)(struct svc_serv *, struct sockaddr *, int); + int (*xpt_get_name)(char *, struct svc_sock*); int (*xpt_recvfrom)(struct svc_rqst *rqstp); int (*xpt_sendto)(struct svc_rqst *rqstp); /* diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 44d6484..03ce7e9 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -622,10 +622,7 @@ out: return len; } -/* - * Report socket names for nfsdfs - */ -static int one_sock_name(char *buf, struct svc_sock *svsk) +static int svc_sock_get_name(char *buf, struct svc_sock *svsk) { int len; @@ -639,11 +636,19 @@ static int one_sock_name(char *buf, stru break; default: len = sprintf(buf, "*unknown-%d*\n", - svsk->sk_sk->sk_family); + svsk->sk_sk->sk_family); } return len; } +/* + * Report socket names for nfsdfs + */ +static int one_sock_name(char *buf, struct svc_sock *svsk) +{ + return svsk->sk_xprt->xpt_get_name(buf,svsk); +} + int svc_sock_names(char *buf, struct svc_serv *serv, char *toclose) { @@ -975,6 +980,7 @@ static struct svc_xprt svc_udp_xprt = { .xpt_name = "udp", .xpt_owner = THIS_MODULE, .xpt_create_svc = svc_udp_create_svc, + .xpt_get_name = svc_sock_get_name, .xpt_recvfrom = svc_udp_recvfrom, .xpt_sendto = svc_udp_sendto, .xpt_detach = svc_sock_detach, @@ -1443,6 +1449,7 @@ static struct svc_xprt svc_tcp_xprt = { .xpt_name = "tcp", .xpt_owner = THIS_MODULE, .xpt_create_svc = svc_tcp_create_svc, + .xpt_get_name = svc_sock_get_name, .xpt_recvfrom = svc_tcp_recvfrom, .xpt_sendto = svc_tcp_sendto, .xpt_detach = svc_sock_detach, From tom at opengridcomputing.com Mon Aug 20 11:58:09 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 13:58:09 -0500 Subject: [ofa-general] [RFC, PATCH 20/20] knfsd: create listener via portlist write In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <20070820185809.18466.82600.stgit@dell3.ogc.int> Update the write handler for the portlist file to allow creating new listening endpoints on a transport. The general form of the string is: For example: tcp 2049 This is intended to support the creation of a listening endpoint for RDMA transports without adding #ifdef code to the nfssvc.c file. The "built-in" transports UDP/TCP were left in the nfssvc initialization code to avoid having to change rpc.nfsd, etc... Signed-off-by: Tom Tucker --- fs/nfsd/nfsctl.c | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 71c686d..da2abda 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -555,6 +555,23 @@ static ssize_t write_ports(struct file * kfree(toclose); return len; } + /* This implements the ability to add a transport by writing + * it's transport name to the portlist file + */ + if (isalnum(buf[0])) { + int err; + char transport[16]; + int port; + if (sscanf(buf, "%15s %4d", transport, &port) == 2) { + err = nfsd_create_serv(); + if (!err) + err = svc_create_svcsock(nfsd_serv, + transport, port, + SVC_SOCK_ANONYMOUS); + return err < 0 ? err : 0; + } + } + return -EINVAL; } From tom at opengridcomputing.com Mon Aug 20 12:01:26 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:26 -0500 Subject: [ofa-general] [RFC,PATCH 01/10] rdma: ONCRPC RDMA Header File In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190126.18615.8406.stgit@dell3.ogc.int> These are the core data types that are used to process the ONCRPC protocol in the NFS-RDMA client and server. Signed-off-by: Tom Talpey --- include/linux/sunrpc/rpc_rdma.h | 116 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 116 insertions(+), 0 deletions(-) diff --git a/include/linux/sunrpc/rpc_rdma.h b/include/linux/sunrpc/rpc_rdma.h new file mode 100644 index 0000000..0013a0d --- /dev/null +++ b/include/linux/sunrpc/rpc_rdma.h @@ -0,0 +1,116 @@ +/* + * Copyright (c) 2003-2007 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _LINUX_SUNRPC_RPC_RDMA_H +#define _LINUX_SUNRPC_RPC_RDMA_H + +struct rpcrdma_segment { + uint32_t rs_handle; /* Registered memory handle */ + uint32_t rs_length; /* Length of the chunk in bytes */ + uint64_t rs_offset; /* Chunk virtual address or offset */ +}; + +/* + * read chunk(s), encoded as a linked list. + */ +struct rpcrdma_read_chunk { + uint32_t rc_discrim; /* 1 indicates presence */ + uint32_t rc_position; /* Position in XDR stream */ + struct rpcrdma_segment rc_target; +}; + +/* + * write chunk, and reply chunk. + */ +struct rpcrdma_write_chunk { + struct rpcrdma_segment wc_target; +}; + +/* + * write chunk(s), encoded as a counted array. + */ +struct rpcrdma_write_array { + uint32_t wc_discrim; /* 1 indicates presence */ + uint32_t wc_nchunks; /* Array count */ + struct rpcrdma_write_chunk wc_array[0]; +}; + +struct rpcrdma_msg { + uint32_t rm_xid; /* Mirrors the RPC header xid */ + uint32_t rm_vers; /* Version of this protocol */ + uint32_t rm_credit; /* Buffers requested/granted */ + uint32_t rm_type; /* Type of message (enum rpcrdma_proc) */ + union { + + struct { /* no chunks */ + uint32_t rm_empty[3]; /* 3 empty chunk lists */ + } rm_nochunks; + + struct { /* no chunks and padded */ + uint32_t rm_align; /* Padding alignment */ + uint32_t rm_thresh; /* Padding threshold */ + uint32_t rm_pempty[3]; /* 3 empty chunk lists */ + } rm_padded; + + uint32_t rm_chunks[0]; /* read, write and reply chunks */ + + } rm_body; +}; + +#define RPCRDMA_HDRLEN_MIN 28 + +enum rpcrdma_errcode { + ERR_VERS = 1, + ERR_CHUNK = 2 +}; + +struct rpcrdma_err_vers { + uint32_t rdma_vers_low; /* Version range supported by peer */ + uint32_t rdma_vers_high; +}; + +enum rpcrdma_proc { + RDMA_MSG = 0, /* An RPC call or reply msg */ + RDMA_NOMSG = 1, /* An RPC call or reply msg - separate body */ + RDMA_MSGP = 2, /* An RPC call or reply msg with padding */ + RDMA_DONE = 3, /* Client signals reply completion */ + RDMA_ERROR = 4 /* An RPC RDMA encoding error */ +}; + +#endif /* _LINUX_SUNRPC_RPC_RDMA_H */ From tom at opengridcomputing.com Mon Aug 20 12:01:30 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:30 -0500 Subject: [ofa-general] [RFC,PATCH 03/10] rdma: SVCRMDA Header File In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190130.18615.28188.stgit@dell3.ogc.int> This file defines the data types used by the SVCRDMA transport module. The principle data structure is the transport specific extension to the svcxprt structure. Signed-off-by: Tom Tucker --- include/linux/sunrpc/svc_rdma.h | 261 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 261 insertions(+), 0 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h new file mode 100644 index 0000000..0bad94b --- /dev/null +++ b/include/linux/sunrpc/svc_rdma.h @@ -0,0 +1,261 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ + +#ifndef SVC_RDMA_H +#define SVC_RDMA_H +#include +#include +#include +#include +#include +#define SVCRDMA_DEBUG + +/* RPC/RDMA parameters */ +extern unsigned int svcrdma_ord; +extern unsigned int svcrdma_max_requests; +extern unsigned int svcrdma_max_req_size; +extern unsigned int rdma_stat_recv; +extern unsigned int rdma_stat_read; +extern unsigned int rdma_stat_write; +extern unsigned int rdma_stat_sq_starve; +extern unsigned int rdma_stat_rq_starve; +extern unsigned int rdma_stat_rq_poll; +extern unsigned int rdma_stat_rq_prod; +extern unsigned int rdma_stat_sq_poll; +extern unsigned int rdma_stat_sq_prod; + +#define RPCRDMA_VERSION 1 + +/* + * Contexts are built when an RDMA request is created and are a + * record of the resources that can be recovered when the request + * completes. + */ +struct svc_rdma_op_ctxt { + struct svc_rdma_op_ctxt *next; + struct xdr_buf arg; + struct list_head dto_q; + enum ib_wr_opcode wr_op; + enum ib_wc_status wc_status; + u32 byte_len; + struct svcxprt_rdma *xprt; + unsigned long flags; + enum dma_data_direction direction; + int count; + struct ib_sge sge[RPCSVC_MAXPAGES]; + struct page *pages[RPCSVC_MAXPAGES]; +}; + +#define RDMACTXT_F_READ_DONE 1 +#define RDMACTXT_F_LAST_CTXT 2 + +struct svc_rdma_deferred_req { + struct svc_deferred_req req; + struct page *arg_page; + int arg_len; +}; + +struct svcxprt_rdma { + struct svc_sock sc_xprt; /* SVC transport structure */ + struct rdma_cm_id *sc_cm_id; /* RDMA connection id */ + struct list_head sc_accept_q; /* Conn. waiting accept */ + int sc_ord; /* RDMA read limit */ + wait_queue_head_t sc_read_wait; + int sc_max_sge; + + int sc_sq_depth; /* Depth of SQ */ + atomic_t sc_sq_count; /* Number of SQ WR on queue */ + + int sc_max_requests; /* Depth of RQ */ + int sc_max_req_size; /* Size of each RQ WR buf */ + + struct ib_pd *sc_pd; + + struct svc_rdma_op_ctxt *sc_ctxt_head; + int sc_ctxt_cnt; + int sc_ctxt_bump; + int sc_ctxt_max; + spinlock_t sc_ctxt_lock; + struct list_head sc_rq_dto_q; + spinlock_t sc_rq_dto_lock; + struct ib_qp *sc_qp; + struct ib_cq *sc_rq_cq; + struct ib_cq *sc_sq_cq; + struct ib_mr *sc_phys_mr; /* MR for server memory */ + + spinlock_t sc_lock; + + wait_queue_head_t sc_send_wait; /* SQ exhaustion waitlist */ + int sc_flags; + struct list_head sc_dto_q; /* DTO tasklet I/O pending */ + struct list_head sc_read_complete_q; + spinlock_t sc_read_complete_lock; +}; +/* sc_flags */ +#define RDMAXPRT_RQ_PENDING 1 +#define RDMAXPRT_SQ_PENDING 2 + +#define RPCRDMA_LISTEN_BACKLOG 10 +/* The default ORD value is based on two outstanding full-size writes with a + * page size of 4k, or 32k * 2 ops / 4k = 16 outstanding RDMA_READ. */ +#define RPCRDMA_ORD (64/4) +#define RPCRDMA_SQ_DEPTH_MULT 8 +#define RPCRDMA_MAX_THREADS 16 +#define RPCRDMA_MAX_REQUESTS 16 +#define RPCRDMA_MAX_REQ_SIZE 4096 + +/* svc_rdma_marshal.c */ +extern void svc_rdma_rcl_chunk_counts(struct rpcrdma_read_chunk *, int *, int *); +extern int svc_rdma_xdr_decode_req(struct rpcrdma_msg**, struct svc_rqst *); +extern int svc_rdma_xdr_decode_deferred_req(struct svc_rqst *); +extern int svc_rdma_xdr_encode_error(struct svcxprt_rdma*, struct rpcrdma_msg*, + enum rpcrdma_errcode, u32*); +extern void svc_rdma_xdr_encode_write_list(struct rpcrdma_msg *, int); +extern void svc_rdma_xdr_encode_reply_array(struct rpcrdma_write_array *, int); +extern void svc_rdma_xdr_encode_array_chunk(struct rpcrdma_write_array *, int, + u32, u64, u32); +extern void svc_rdma_xdr_encode_reply_header(struct svcxprt_rdma *, + struct rpcrdma_msg *, + struct rpcrdma_msg *, + enum rpcrdma_proc); +extern int svc_rdma_xdr_get_reply_hdr_len(struct rpcrdma_msg *); + +/* svc_rdma_recvfrom.c */ +extern int svc_rdma_recvfrom(struct svc_rqst*); + +/* svc_rdma_sendto.c */ +extern int svc_rdma_sendto(struct svc_rqst*); + +/* svc_rdma_transport.c */ +extern int svc_rdma_send(struct svcxprt_rdma*, struct ib_send_wr*); +extern int svc_rdma_send_error(struct svcxprt_rdma*, struct rpcrdma_msg*, + enum rpcrdma_errcode); +struct page* svc_rdma_get_page(void); +extern int svc_rdma_post_recv(struct svcxprt_rdma*); +extern int svc_rdma_create_listen(struct svc_serv*, int, struct sockaddr*); +extern struct svc_rdma_op_ctxt *svc_rdma_get_context(struct svcxprt_rdma *); +extern void svc_rdma_put_context(struct svc_rdma_op_ctxt *, int); +extern void svc_sq_reap(struct svcxprt_rdma*); +extern void svc_rq_reap(struct svcxprt_rdma*); + +/* svc_rdma.c */ +extern int svc_rdma_init(void); +extern void svc_rdma_cleanup(void); + +/* + * Returns the address of the first read chunk or if no read chunk is + * present + */ +static inline struct rpcrdma_read_chunk * +svc_rdma_get_read_chunk(struct rpcrdma_msg *rmsgp) +{ + struct rpcrdma_read_chunk *ch = + (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; + + if (ch->rc_discrim == 0) + return NULL; + + return ch; +} + +/* + * Returns the address of the first read write array element or if no + * write array list is present + */ +static inline struct rpcrdma_write_array * +svc_rdma_get_write_array(struct rpcrdma_msg *rmsgp) +{ + if (rmsgp->rm_body.rm_chunks[0] != 0 + || rmsgp->rm_body.rm_chunks[1] == 0) + return NULL; + + return (struct rpcrdma_write_array *)&rmsgp->rm_body.rm_chunks[1]; +} + +/* + * Returns the address of the first reply array element or if no + * reply array is present + */ +static inline struct rpcrdma_write_array * +svc_rdma_get_reply_array(struct rpcrdma_msg *rmsgp) +{ + struct rpcrdma_read_chunk *rch; + struct rpcrdma_write_array *wr_ary; + struct rpcrdma_write_array *rp_ary; + + /* XXX: Need to fix when reply list may occur with read-list and/or + * write list */ + if (rmsgp->rm_body.rm_chunks[0] != 0 || + rmsgp->rm_body.rm_chunks[1] != 0) + return NULL; + + rch = svc_rdma_get_read_chunk(rmsgp); + if (rch) { + while (rch->rc_discrim) + rch ++; + + /* The reply list follows an empty write array located + * at 'rc_position' here. The reply array is at rc_target. + */ + rp_ary = (struct rpcrdma_write_array*)&rch->rc_target; + + goto found_it; + } + + wr_ary = svc_rdma_get_write_array(rmsgp); + if (wr_ary) { + rp_ary = (struct rpcrdma_write_array*) + &wr_ary->wc_array[wr_ary->wc_nchunks].wc_target.rs_length; + + goto found_it; + } + + /* No read list, no write list */ + rp_ary = (struct rpcrdma_write_array*) + &rmsgp->rm_body.rm_chunks[2]; + + found_it: + if (rp_ary->wc_discrim == 0) + return NULL; + + return rp_ary; +} +#endif From tom at opengridcomputing.com Mon Aug 20 12:01:32 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:32 -0500 Subject: [ofa-general] [RFC,PATCH 04/10] rdma: SVCRDMA Transport Module In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190132.18615.13961.stgit@dell3.ogc.int> This file implements the RDMA transport module initialization and termination logic and registers the transport sysctl variables. Signed-off-by: Tom Tucker --- net/sunrpc/svc_rdma.c | 270 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 270 insertions(+), 0 deletions(-) diff --git a/net/sunrpc/svc_rdma.c b/net/sunrpc/svc_rdma.c new file mode 100644 index 0000000..620249d --- /dev/null +++ b/net/sunrpc/svc_rdma.c @@ -0,0 +1,270 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ +#include +#include +#include +#include +#include +#include +#include + +#define RPCDBG_FACILITY RPCDBG_SVCXPRT + +/* RPC/RDMA parameters */ +unsigned int svcrdma_ord = RPCRDMA_ORD; +static unsigned int min_ord = 1; +static unsigned int max_ord = 4096; +unsigned int svcrdma_max_requests = RPCRDMA_MAX_REQUESTS; +static unsigned int min_max_requests = 4; +static unsigned int max_max_requests = 16384; +unsigned int svcrdma_max_req_size = RPCRDMA_MAX_REQ_SIZE; +static unsigned int min_max_inline = 4096; +static unsigned int max_max_inline = 65536; +static unsigned int zero = 0; +static unsigned int one = 1; + +unsigned int rdma_stat_recv = 0; +unsigned int rdma_stat_read = 0; +unsigned int rdma_stat_write = 0; +unsigned int rdma_stat_sq_starve = 0; +unsigned int rdma_stat_rq_starve = 0; +unsigned int rdma_stat_rq_poll = 0; +unsigned int rdma_stat_rq_prod = 0; +unsigned int rdma_stat_sq_poll = 0; +unsigned int rdma_stat_sq_prod = 0; + +extern struct svc_xprt svc_rdma_xprt; + +static struct ctl_table_header *svcrdma_table_header; +static ctl_table svcrdma_parm_table[] = { + { + .ctl_name = CTL_RDMA_MAX_REQUESTS, + .procname = "max_requests", + .data = &svcrdma_max_requests, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &min_max_requests, + .extra2 = &max_max_requests + }, + { + .ctl_name = CTL_RDMA_MAX_REQ_SIZE, + .procname = "max_req_size", + .data = &svcrdma_max_req_size, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &min_max_inline, + .extra2 = &max_max_inline + }, + { + .ctl_name = CTL_RDMA_ORD, + .procname = "max_outbound_read_requests", + .data = &svcrdma_ord, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &min_ord, + .extra2 = &max_ord, + }, + + { + .ctl_name = CTL_RDMA_STAT_READ, + .procname = "rdma_stat_read", + .data = &rdma_stat_read, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_RECV, + .procname = "rdma_stat_recv", + .data = &rdma_stat_recv, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_WRITE, + .procname = "rdma_stat_write", + .data = &rdma_stat_write, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_SQ_STARVE, + .procname = "rdma_stat_sq_starve", + .data = &rdma_stat_sq_starve, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_RQ_STARVE, + .procname = "rdma_stat_rq_starve", + .data = &rdma_stat_rq_starve, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_RQ_POLL, + .procname = "rdma_stat_rq_poll", + .data = &rdma_stat_rq_poll, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_RQ_PROD, + .procname = "rdma_stat_rq_prod", + .data = &rdma_stat_rq_prod, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_SQ_POLL, + .procname = "rdma_stat_sq_poll", + .data = &rdma_stat_sq_poll, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = CTL_RDMA_STAT_SQ_PROD, + .procname = "rdma_stat_sq_prod", + .data = &rdma_stat_sq_prod, + .maxlen = sizeof(unsigned int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .strategy = &sysctl_intvec, + .extra1 = &zero, + .extra2 = &one, + }, + { + .ctl_name = 0, + }, +}; + +static ctl_table svcrdma_table[] = { + { + .ctl_name = CTL_SVCRDMA, + .procname = "svc_rdma", + .mode = 0555, + .child = svcrdma_parm_table + }, + { + .ctl_name = 0, + }, +}; + +static ctl_table svcrdma_root_table[] = { + { + .ctl_name = CTL_SUNRPC, + .procname = "sunrpc", + .mode = 0555, + .child = svcrdma_table + }, + { + .ctl_name = 0, + }, +}; + +void svc_rdma_cleanup(void) +{ + dprintk("SVCRDMA Module Removed, deregister RPC RDMA transport\n"); + if (svcrdma_table_header) { + unregister_sysctl_table(svcrdma_table_header); + svcrdma_table_header = NULL; + } + svc_unregister_transport(&svc_rdma_xprt); +} + +int svc_rdma_init(void) +{ + dprintk("SVCRDMA Module Init, register RPC RDMA transport\n"); + dprintk("\tsvcrdma_ord : %d\n", svcrdma_ord); + dprintk("\tmax_requests : %d\n", svcrdma_max_requests); + dprintk("\tsq_depth : %d\n", + svcrdma_max_requests * RPCRDMA_SQ_DEPTH_MULT); + dprintk("\tmax_inline : %d\n", svcrdma_max_req_size); + if (!svcrdma_table_header) { + svcrdma_table_header = + register_sysctl_table(svcrdma_root_table); + } + /* Register RDMA with the SVC transport switch */ + svc_register_transport(&svc_rdma_xprt); + return 0; +} +MODULE_AUTHOR("Tom Tucker "); +MODULE_DESCRIPTION("SVC RDMA Transport"); +MODULE_LICENSE("Dual BSD/GPL"); +module_init(svc_rdma_init); +module_exit(svc_rdma_cleanup); From tom at opengridcomputing.com Mon Aug 20 12:01:34 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:34 -0500 Subject: [ofa-general] [RFC, PATCH 05/10] rdma: SVCRDMA Core Transport Services In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190134.18615.8463.stgit@dell3.ogc.int> This file implements the core transport data management and I/O path. The I/O path for RDMA involves receiving callbacks on interrupt context. Since all the svc transport locks are _bh locks we enqueue the transport on a list, schedule a tasklet to dequeue data indications from the RDMA completion queue. The tasklet in turn takes _bh locks to enqueue receive data indications on a list for the transport. The svc_rdma_recvfrom transport function dequeues data from this list in an NFSD thread context. Signed-off-by: Tom Tucker --- net/sunrpc/svc_rdma_transport.c | 1207 +++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcauth_unix.c | 3 2 files changed, 1208 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/svc_rdma_transport.c b/net/sunrpc/svc_rdma_transport.c new file mode 100644 index 0000000..3f1f251 --- /dev/null +++ b/net/sunrpc/svc_rdma_transport.c @@ -0,0 +1,1207 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include /* num_physpages */ +#include +#include +#include +#include +#include +#include +#include +#include + +#define RPCDBG_FACILITY RPCDBG_SVCXPRT + +int svc_rdma_create_svc(struct svc_serv *serv, struct sockaddr *sa, int flags); +static int svc_rdma_accept(struct svc_sock *xprt); +static void rdma_destroy_xprt(struct svcxprt_rdma *xprt); +static void dto_tasklet_func(unsigned long data); +static struct cache_deferred_req *svc_rdma_defer(struct cache_req *req); +static void svc_rdma_detach(struct svc_sock *svsk); +static void svc_rdma_free(struct svc_sock *svsk); +static int svc_rdma_has_wspace(struct svc_sock *svsk); +static int svc_rdma_get_name(char *buf, struct svc_sock *svsk); + +static void rq_cq_reap(struct svcxprt_rdma *xprt); +static void sq_cq_reap(struct svcxprt_rdma *xprt); + +DECLARE_TASKLET(dto_tasklet, dto_tasklet_func, 0UL); +static spinlock_t dto_lock = SPIN_LOCK_UNLOCKED; +static LIST_HEAD(dto_xprt_q); + +struct svc_xprt svc_rdma_xprt = { + .xpt_name = "rdma", + .xpt_owner = THIS_MODULE, + .xpt_create_svc = svc_rdma_create_svc, + .xpt_get_name = svc_rdma_get_name, + .xpt_recvfrom = svc_rdma_recvfrom, + .xpt_sendto = svc_rdma_sendto, + .xpt_detach = svc_rdma_detach, + .xpt_free = svc_rdma_free, + .xpt_has_wspace = svc_rdma_has_wspace, + .xpt_max_payload = RPCSVC_MAXPAYLOAD_TCP, + .xpt_accept = svc_rdma_accept, + .xpt_defer = svc_rdma_defer +}; + +static int rdma_bump_context_cache(struct svcxprt_rdma *xprt) +{ + int target; + int at_least_one = 0; + struct svc_rdma_op_ctxt *ctxt; + + target = min(xprt->sc_ctxt_cnt + xprt->sc_ctxt_bump, + xprt->sc_ctxt_max); + + spin_lock_bh(&xprt->sc_ctxt_lock); + while (xprt->sc_ctxt_cnt < target) { + xprt->sc_ctxt_cnt ++; + spin_unlock_bh(&xprt->sc_ctxt_lock); + + ctxt = kmalloc(sizeof(*ctxt), GFP_KERNEL); + + spin_lock_bh(&xprt->sc_ctxt_lock); + if (ctxt) { + at_least_one = 1; + ctxt->next = xprt->sc_ctxt_head; + xprt->sc_ctxt_head = ctxt; + } else { + /* kmalloc failed...give up for now */ + xprt->sc_ctxt_cnt --; + break; + } + } + spin_unlock_bh(&xprt->sc_ctxt_lock); + dprintk("svcrdma: sc_ctxt_max=%d, sc_ctxt_cnt=%d\n", + xprt->sc_ctxt_max,xprt->sc_ctxt_cnt); + return at_least_one; +} + +struct svc_rdma_op_ctxt *svc_rdma_get_context(struct svcxprt_rdma *xprt) +{ + struct svc_rdma_op_ctxt *ctxt; + + while (1) { + spin_lock_bh(&xprt->sc_ctxt_lock); + if (unlikely(xprt->sc_ctxt_head == NULL)) { + /* Try to bump my cache. */ + spin_unlock_bh(&xprt->sc_ctxt_lock); + + if (rdma_bump_context_cache(xprt)) + continue; + + printk(KERN_INFO "svcrdma: sleeping waiting for context " + "memory on xprt=%p\n", + xprt); + schedule_timeout_uninterruptible(msecs_to_jiffies(500)); + continue; + } + ctxt = xprt->sc_ctxt_head; + xprt->sc_ctxt_head = ctxt->next; + spin_unlock_bh(&xprt->sc_ctxt_lock); + ctxt->xprt = xprt; + INIT_LIST_HEAD(&ctxt->dto_q); + ctxt->count = 0; + break; + } + return ctxt; +} + +void svc_rdma_put_context(struct svc_rdma_op_ctxt *ctxt, int free_pages) +{ + struct svcxprt_rdma *xprt; + int i; + + BUG_ON(!ctxt); + xprt = ctxt->xprt; + if (free_pages) { + for (i=0; i < ctxt->count; i++) + put_page(ctxt->pages[i]); + } + + for (i=0; i < ctxt->count; i++) { + dma_unmap_single(xprt->sc_cm_id->device->dma_device, + ctxt->sge[i].addr, + ctxt->sge[i].length, + ctxt->direction); + } + spin_lock_bh(&xprt->sc_ctxt_lock); + ctxt->next = xprt->sc_ctxt_head; + xprt->sc_ctxt_head = ctxt; + spin_unlock_bh(&xprt->sc_ctxt_lock); +} + +/* ib_cq event handler */ +static void cq_event_handler(struct ib_event *event, void *context) +{ + struct svcxprt_rdma *xprt = (struct svcxprt_rdma *)context; + printk(KERN_INFO "svcrdma: received CQ event id=%d, context=%p\n", + event->event, context); + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); +} + +/* QP event handler */ +static void qp_event_handler(struct ib_event *event, void *context) +{ + struct svcxprt_rdma *xprt = context; + + switch (event->event) { + /* These are considered benign events */ + case IB_EVENT_PATH_MIG: + case IB_EVENT_COMM_EST: + case IB_EVENT_SQ_DRAINED: + case IB_EVENT_QP_LAST_WQE_REACHED: + printk(KERN_INFO "svcrdma: QP event %d received for QP=%p\n", + event->event, event->element.qp); + break; + /* These are considered fatal events */ + case IB_EVENT_PATH_MIG_ERR: + case IB_EVENT_QP_FATAL: + case IB_EVENT_QP_REQ_ERR: + case IB_EVENT_QP_ACCESS_ERR: + case IB_EVENT_DEVICE_FATAL: + default: + printk(KERN_ERR "svcrdma: QP ERROR event %d received for QP=%p, " + "closing transport\n", + event->event, event->element.qp); + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + break; + } +} + +/* + * Data Transfer Operation Tasklet + * + * Walks a list of transports with I/O pending, removing entries as + * they are added to the server's I/O pending list. Two bits indicate + * if SQ, RQ, or both have I/O pending. The dto_lock is an irqsave + * spinlock that serializes access to the transport list with the RQ + * and SQ interrupt handlers. + */ +static void dto_tasklet_func(unsigned long data) +{ + struct svcxprt_rdma *xprt; + unsigned long flags; + + spin_lock_irqsave(&dto_lock, flags); + while (!list_empty(&dto_xprt_q)) { + xprt = list_entry(dto_xprt_q.next, struct svcxprt_rdma, sc_dto_q); + list_del_init(&xprt->sc_dto_q); + spin_unlock_irqrestore(&dto_lock, flags); + + if (test_and_clear_bit(RDMAXPRT_RQ_PENDING, &xprt->sc_flags)) { + ib_req_notify_cq(xprt->sc_rq_cq, IB_CQ_NEXT_COMP); + rq_cq_reap(xprt); + set_bit(SK_DATA, &xprt->sc_xprt.sk_flags); + svc_sock_enqueue(&xprt->sc_xprt); + } + + if (test_and_clear_bit(RDMAXPRT_SQ_PENDING, &xprt->sc_flags)) { + ib_req_notify_cq(xprt->sc_sq_cq, IB_CQ_NEXT_COMP); + sq_cq_reap(xprt); + } + + spin_lock_irqsave(&dto_lock, flags); + } + spin_unlock_irqrestore(&dto_lock, flags); +} + +/* + * Receive Queue Completion Handler - potentially called on interrupt context. + * + * svc_sock_enqueue and the remainder of the svc core assumes + * uses _bh locks. Since the rq_comp_handler is called on interrupt + * context, we need to refer the handling of the I/O to a tasklet + */ +static void +rq_comp_handler(struct ib_cq *cq, void *cq_context) +{ + struct svcxprt_rdma *xprt = cq_context; + unsigned long flags; + + /* + * Set the bit regardless of whether or not it's on the list + * because it may be on the list already due to an SQ + * completion. + */ + set_bit(RDMAXPRT_RQ_PENDING, &xprt->sc_flags); + + /* + * If this transport is not already on the DTO transport queue, + * add it + */ + spin_lock_irqsave(&dto_lock, flags); + if (list_empty(&xprt->sc_dto_q)) + list_add_tail(&xprt->sc_dto_q, &dto_xprt_q); + spin_unlock_irqrestore(&dto_lock, flags); + + /* Tasklet does all the work to avoid irqsave locks. */ + tasklet_schedule(&dto_tasklet); +} + +/* + * rq_cq_reap - Process the RQ CQ. + * + * Take all completing WC off the CQE and enqueue the associated DTO context + * on the dto_q for the transport. + */ +static void +rq_cq_reap(struct svcxprt_rdma *xprt) +{ + int ret; + struct ib_wc wc; + struct svc_rdma_op_ctxt *ctxt = NULL; + + rdma_stat_rq_poll ++; + + spin_lock_bh(&xprt->sc_rq_dto_lock); + while ((ret = ib_poll_cq(xprt->sc_rq_cq, 1, &wc)) > 0) { + ctxt = (struct svc_rdma_op_ctxt*)(unsigned long)wc.wr_id; + ctxt->wc_status = wc.status; + ctxt->byte_len = wc.byte_len; + if (wc.status != IB_WC_SUCCESS) { + /* Close the transport */ + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + svc_rdma_put_context(ctxt, 1); + continue; + } + list_add_tail(&ctxt->dto_q, &xprt->sc_rq_dto_q); + } + spin_unlock_bh(&xprt->sc_rq_dto_lock); + + if (ctxt) + rdma_stat_rq_prod ++; +} + +/* + * Send Queue Completion Handler - potentially called on interrupt context. + */ +static void +sq_cq_reap(struct svcxprt_rdma *xprt) +{ + struct svc_rdma_op_ctxt *ctxt = NULL; + struct ib_wc wc; + struct ib_cq *cq = xprt->sc_sq_cq; + int ret; + + rdma_stat_sq_poll ++; + while ((ret = ib_poll_cq(cq, 1, &wc)) > 0) { + ctxt = (struct svc_rdma_op_ctxt*)(unsigned long)wc.wr_id; + xprt = ctxt->xprt; + + if (wc.status != IB_WC_SUCCESS) + /* Close the transport */ + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + + /* Decrement used SQ WR count */ + atomic_dec(&xprt->sc_sq_count); + wake_up(&xprt->sc_send_wait); + + switch (ctxt->wr_op) { + case IB_WR_SEND: + case IB_WR_RDMA_WRITE: + svc_rdma_put_context(ctxt,1); + break; + + case IB_WR_RDMA_READ: + if (test_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags)) { + set_bit(SK_DATA, &xprt->sc_xprt.sk_flags); + set_bit(RDMACTXT_F_READ_DONE, &ctxt->flags); + spin_lock_bh(&xprt->sc_read_complete_lock); + list_add_tail(&ctxt->dto_q, &xprt->sc_read_complete_q); + spin_unlock_bh(&xprt->sc_read_complete_lock); + svc_sock_enqueue(&xprt->sc_xprt); + } + break; + + default: + printk(KERN_ERR "svcrdma: unexpected completion type, " + "opcode=%d, status=%d\n", + wc.opcode, wc.status); + break; + } + } + + if (ctxt) + rdma_stat_sq_prod ++; +} + +static void +sq_comp_handler(struct ib_cq *cq, void *cq_context) +{ + struct svcxprt_rdma *xprt = cq_context; + unsigned long flags; + + /* + * Set the bit regardless of whether or not it's on the list + * because it may be on the list already due to an RQ + * completion. + */ + set_bit(RDMAXPRT_SQ_PENDING,&xprt->sc_flags); + + /* + * If this transport is not already on the DTO transport queue, + * add it + */ + spin_lock_irqsave(&dto_lock, flags); + if (list_empty(&xprt->sc_dto_q)) + list_add_tail(&xprt->sc_dto_q, &dto_xprt_q); + spin_unlock_irqrestore(&dto_lock, flags); + + /* Tasklet does all the work to avoid irqsave locks. */ + tasklet_schedule(&dto_tasklet); +} + +static void +create_context_cache(struct svcxprt_rdma *xprt, + int ctxt_count, int ctxt_bump, int ctxt_max) +{ + struct svc_rdma_op_ctxt *ctxt; + int i; + + xprt->sc_ctxt_max = ctxt_max; + xprt->sc_ctxt_bump = ctxt_bump; + xprt->sc_ctxt_cnt = 0; + xprt->sc_ctxt_head = NULL; + for (i=0; i < ctxt_count; i++) { + ctxt = kmalloc(sizeof(*ctxt), GFP_KERNEL); + if (ctxt) { + ctxt->next = xprt->sc_ctxt_head; + xprt->sc_ctxt_head = ctxt; + xprt->sc_ctxt_cnt ++; + } + } +} + +static void destroy_context_cache(struct svc_rdma_op_ctxt *ctxt) +{ + struct svc_rdma_op_ctxt *next; + if (!ctxt) + return; + + do { + next = ctxt->next; + kfree(ctxt); + ctxt = next; + } while (next); +} + +static struct svcxprt_rdma *rdma_create_xprt(int listener) +{ + struct svcxprt_rdma *cma_xprt = kzalloc(sizeof *cma_xprt, GFP_KERNEL); + + if (!cma_xprt) + return NULL; + + INIT_LIST_HEAD(&cma_xprt->sc_accept_q); + INIT_LIST_HEAD(&cma_xprt->sc_dto_q); + INIT_LIST_HEAD(&cma_xprt->sc_rq_dto_q); + INIT_LIST_HEAD(&cma_xprt->sc_read_complete_q); + init_waitqueue_head(&cma_xprt->sc_send_wait); + + spin_lock_init(&cma_xprt->sc_lock); + spin_lock_init(&cma_xprt->sc_read_complete_lock); + spin_lock_init(&cma_xprt->sc_ctxt_lock); + spin_lock_init(&cma_xprt->sc_rq_dto_lock); + + cma_xprt->sc_ord = svcrdma_ord; + + cma_xprt->sc_max_req_size = svcrdma_max_req_size; + cma_xprt->sc_max_requests = svcrdma_max_requests; + cma_xprt->sc_sq_depth = svcrdma_max_requests * RPCRDMA_SQ_DEPTH_MULT; + atomic_set(&cma_xprt->sc_sq_count,0); + + if (!listener) { + int reqs = cma_xprt->sc_max_requests; + create_context_cache(cma_xprt, + reqs << 1, /* starting size */ + reqs, /* bump amount */ + reqs + + cma_xprt->sc_sq_depth + + RPCRDMA_MAX_THREADS + 1); /* max */ + if (!cma_xprt->sc_ctxt_head) { + kfree(cma_xprt); + return NULL; + } + } + + return cma_xprt; +} + +/* + * Create a string for presentation in the portlist file. The RDMA + * listener presents an issue because it is a) transport independent, + * and b) not bound to a local interface yet. That is, it's listening + * on INADDR_ANY. This means that the endpoint 'cm_id' is not yet + * bound to a device and therefore, we dont' know if it's IP or + * IB. The string created here, therefore, is a fabrication unless the + * service has bound to a specific local endpoint. + */ +static int svc_rdma_get_name(char *buf, struct svc_sock *svsk) +{ + int len; + struct svcxprt_rdma *rdma = (struct svcxprt_rdma *)svsk; + + BUG_ON(rdma->sc_cm_id==NULL); + + /* Check if we're bound to a device. If not, */ + if (!rdma->sc_cm_id->device) { + /* fabricate a string */ + len = sprintf(buf, "ofa rdma 0.0.0.0 %d\n", + ntohs(((struct sockaddr_in*)&rdma->sc_cm_id-> + route.addr.src_addr)->sin_port)); + return len; + } + + switch (rdma_node_get_transport(rdma->sc_cm_id->device->node_type)) { + case RDMA_TRANSPORT_IB: + len = sprintf(buf, "ib rdma %u.%u.%u.%u %d\n", + NIPQUAD(((struct sockaddr_in*)&rdma->sc_cm_id-> + route.addr.src_addr)->sin_addr.s_addr), + ntohs(((struct sockaddr_in*)&rdma->sc_cm_id-> + route.addr.src_addr)->sin_port)); + break; + case RDMA_TRANSPORT_IWARP: + len = sprintf(buf, "ipv4 rdma %u.%u.%u.%u %d\n", + NIPQUAD(((struct sockaddr_in*)&rdma->sc_cm_id-> + route.addr.src_addr)->sin_addr.s_addr), + ntohs(((struct sockaddr_in*)&rdma->sc_cm_id-> + route.addr.src_addr)->sin_port)); + break; + default: + len = sprintf(buf, "*unknown-%d*\n", + rdma->sc_cm_id->device->node_type); + } + + return len; +} + +struct page *svc_rdma_get_page(void) +{ + struct page *page; + + while ((page = alloc_page(GFP_KERNEL))==NULL) { + /* If we can't get memory, wait a bit and try again */ + printk(KERN_INFO "svcrdma: out of memory...retrying in 1000 jiffies.\n"); + schedule_timeout_uninterruptible(msecs_to_jiffies(1000)); + } + return page; +} + +int svc_rdma_post_recv(struct svcxprt_rdma *xprt) +{ + struct ib_recv_wr recv_wr, *bad_recv_wr; + struct svc_rdma_op_ctxt *ctxt; + struct page *page; + unsigned long pa; + int sge_no; + int buflen; + int ret; + + ctxt = svc_rdma_get_context(xprt); + buflen = 0; + ctxt->direction = DMA_FROM_DEVICE; + for (sge_no=0; buflen < xprt->sc_max_req_size; sge_no++) { + BUG_ON(sge_no >= xprt->sc_max_sge); + page = svc_rdma_get_page(); + ctxt->pages[sge_no] = page; + pa = ib_dma_map_page(xprt->sc_cm_id->device, + page, 0, PAGE_SIZE, + DMA_FROM_DEVICE); + ctxt->sge[sge_no].addr = pa; + ctxt->sge[sge_no].length = PAGE_SIZE; + ctxt->sge[sge_no].lkey = xprt->sc_phys_mr->lkey; + buflen += PAGE_SIZE; + } + ctxt->count = sge_no; + recv_wr.next = NULL; + recv_wr.sg_list = &ctxt->sge[0]; + recv_wr.num_sge = ctxt->count; + recv_wr.wr_id = (u64)(unsigned long)ctxt; + + ret = ib_post_recv(xprt->sc_qp, &recv_wr, &bad_recv_wr); + return ret; +} + +static void init_sock(struct svc_sock *svsk, struct svc_serv* serv) +{ + svsk->sk_xprt = &svc_rdma_xprt; + INIT_LIST_HEAD(&svsk->sk_list); + svc_sock_init(svsk, serv); +} + +/* + * This function handles the CONNECT_REQUEST event on a listening + * endpoint. It is passed the cma_id for the _new_ connection. The context in + * this cma_id is inherited from the listening cma_id and is the svc_sock + * structure for the listening endpoint. + * + * This function creates a new xprt for the new connection and enqueues it on + * the accept queue for the listent xprt. When the listen thread is kicked, it + * will call the recvfrom method on the listen xprt which will accept the new + * connection. + */ +static void handle_connect_req(struct rdma_cm_id *new_cma_id) +{ + struct svcxprt_rdma *listen_xprt = new_cma_id->context; + struct svcxprt_rdma *newxprt; + + /* Create a new transport */ + newxprt = rdma_create_xprt(0); + if (!newxprt) { + dprintk("svcrdma: failed to create new transport\n"); + return; + } + newxprt->sc_cm_id = new_cma_id; + new_cma_id->context = newxprt; + dprintk("svcrdma: Creating newxprt=%p, cm_id=%p, listenxprt=%p\n", + newxprt, newxprt->sc_cm_id, listen_xprt); + + /* Initialize the new transport */ + init_sock(&newxprt->sc_xprt, listen_xprt->sc_xprt.sk_server); + + /* Enqueue the new transport on the accept queue of the listening + * transport */ + spin_lock_bh(&listen_xprt->sc_lock); + list_add_tail(&newxprt->sc_accept_q, &listen_xprt->sc_accept_q); + spin_unlock_bh(&listen_xprt->sc_lock); + + listen_xprt->sc_xprt.sk_pool = NULL; + set_bit(SK_CONN, &listen_xprt->sc_xprt.sk_flags); + svc_sock_enqueue(&listen_xprt->sc_xprt); +} + +/* + * Handles events generated on the listening endpoint. These events will be + * either be incoming connect requests or adapter removal events. + * @param cma_id The CMA ID for the listening endpoint + * @event the event being delivered. + */ +static int +rdma_listen_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event) +{ + struct svcxprt_rdma *xprt = cma_id->context; + int ret = 0; + + switch (event->event) { + case RDMA_CM_EVENT_CONNECT_REQUEST: + dprintk("svcrdma: Connect request on cma_id=%p, xprt = %p, event=%d\n", + cma_id, cma_id->context, event->event); + handle_connect_req(cma_id); + break; + + case RDMA_CM_EVENT_ESTABLISHED: + /* Accept complete */ + dprintk("svcrdma: Connection completed on LISTEN xprt=%p, cm_id=%p\n", + xprt, cma_id); + break; + + case RDMA_CM_EVENT_DEVICE_REMOVAL: + dprintk("svcrdma: Device removal xprt=%p, cm_id=%p\n", + xprt, cma_id); + if (xprt) + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + break; + + default: + dprintk("svcrdma: Unexpected event on listening endpoint %p, event=%d\n", + cma_id, event->event); + break; + } + + return ret; +} + +static int +rdma_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event) +{ + struct svcxprt_rdma *xprt = cma_id->context; + int ret = 0; + + switch (event->event) { + case RDMA_CM_EVENT_ESTABLISHED: + /* Accept complete */ + dprintk("svcrdma: Connection completed on DTO xprt=%p, cm_id=%p\n", + xprt, cma_id); + break; + + case RDMA_CM_EVENT_DISCONNECTED: + dprintk("svcrdma: Disconnect on DTO xprt=%p, cm_id=%p\n", + xprt, cma_id); + if (xprt) { + xprt->sc_xprt.sk_pool = NULL; + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + svc_sock_enqueue(&xprt->sc_xprt); + } + break; + + case RDMA_CM_EVENT_DEVICE_REMOVAL: + dprintk("svcrdma: Device removal cma_id=%p, xprt = %p, event=%d\n", + cma_id, cma_id->context, event->event); + if (xprt) { + xprt->sc_xprt.sk_pool = NULL; + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + svc_sock_enqueue(&xprt->sc_xprt); + } + break; + + default: + dprintk("svcrdma: Unexpected event on DTO endpoint %p, event=%d\n", + cma_id, event->event); + break; + } + + return ret; +} + +/* + * Create a listening RDMA service endpoint + * @param serv the RPC service this instance will belong to + * @param protocol the protocol for the instance + * @param sa the address to bind the local interface to + * @return 0 on success, negative value for errors + */ +int svc_rdma_create_svc(struct svc_serv *serv, struct sockaddr *sa, + int flags) +{ + struct rdma_cm_id *listen_id; + struct svcxprt_rdma *cma_xprt; + struct svc_sock *xprt; + int ret; + + dprintk("svcrdma: Creating RDMA socket\n"); + + cma_xprt = rdma_create_xprt(1); + if (!cma_xprt) + return -ENOMEM; + + xprt = &cma_xprt->sc_xprt; + init_sock(xprt, serv); + set_bit(SK_LISTENER, &xprt->sk_flags); + + /* + * We shouldn't receive any events (except device removal) on + * the id until we submit the listen request. Any events that + * we do receive will get logged as errors and ignored + */ + listen_id = rdma_create_id(rdma_listen_handler, cma_xprt, RDMA_PS_TCP); + if (IS_ERR(listen_id)) { + ret = PTR_ERR(listen_id); + rdma_destroy_xprt(cma_xprt); + dprintk("svcrdma: rdma_create_id failed = %d\n", ret); + return ret; + } + ret = rdma_bind_addr(listen_id, sa); + if (ret) { + ret = PTR_ERR(listen_id); + rdma_destroy_xprt(cma_xprt); + rdma_destroy_id(listen_id); + dprintk("svcrdma: rdma_bind_addr failed = %d\n", ret); + return ret; + } + cma_xprt->sc_cm_id = listen_id; + + /* The xprt is ready to process events at this point */ + ret = rdma_listen(listen_id, RPCRDMA_LISTEN_BACKLOG); + if (ret) { + ret = PTR_ERR(listen_id); + rdma_destroy_id(listen_id); + rdma_destroy_xprt(cma_xprt); + dprintk("svcrdma: rdma_listen failed = %d\n", ret); + return ret; + } + + /* Add to list of permanent (listening/unconnected) sockets */ + svc_sock_add_listener(xprt); + clear_bit(SK_BUSY, &xprt->sk_flags); + + return 0; +} + +/* + * This is the sk_recvfrom function for listening endpoints. It's purpose is + * to accept incoming connections. The CMA callback handler has already + * created a new transport and attached the new CMA ID. + * + * There is a queue of pending connections hung on the listening + * transport. This queue contains the new svc_sock structure. This function + * takes svc_sock structures off the accept_q and completes the + * connection. + */ +static int +svc_rdma_accept(struct svc_sock *xprt) +{ + struct svcxprt_rdma *listen_xprt; + struct svcxprt_rdma *newxprt; + struct rdma_conn_param conn_param; + struct ib_qp_init_attr qp_attr; + struct ib_device_attr devattr; + int ret; + int i; + + listen_xprt = (struct svcxprt_rdma*)xprt; + if (list_empty(&listen_xprt->sc_accept_q)) { + printk(KERN_INFO + "svcrdma: woke-up with no pending connection!\n"); + clear_bit(SK_CONN, &listen_xprt->sc_xprt.sk_flags); + BUG_ON(test_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags)==0); + clear_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags); + return 0; + } + + /* Get the next entry off the accept list */ + spin_lock_bh(&listen_xprt->sc_lock); + newxprt = list_entry(listen_xprt->sc_accept_q.next, + struct svcxprt_rdma, sc_accept_q); + list_del_init(&newxprt->sc_accept_q); + spin_unlock_bh(&listen_xprt->sc_lock); + + dprintk("svcrdma: newxprt from accept queue = %p, cm_id=%p\n", + newxprt, newxprt->sc_cm_id); + + ret = ib_query_device(newxprt->sc_cm_id->device, &devattr); + if (ret) { + printk(KERN_ERR + "svcrdma: could not query device attributes on " + "device %p, rc=%d\n", + newxprt->sc_cm_id->device, ret); + goto errout; + } + + /* Qualify the transport resource defaults with the + * capabilities of this particular device */ + newxprt->sc_max_sge = min((size_t)devattr.max_sge, + (size_t)RPCSVC_MAXPAGES); + newxprt->sc_max_requests = min((size_t)devattr.max_qp_wr, + (size_t)svcrdma_max_requests); + newxprt->sc_sq_depth = RPCRDMA_SQ_DEPTH_MULT * newxprt->sc_max_requests; + + newxprt->sc_ord = min((size_t)devattr.max_qp_rd_atom, + (size_t)svcrdma_ord); + + newxprt->sc_pd = ib_alloc_pd(newxprt->sc_cm_id->device); + if (IS_ERR(newxprt->sc_pd)) { + printk(KERN_ERR + "svcrdma: error creating PD for connect request\n"); + ret = PTR_ERR(newxprt->sc_pd); + goto errout; + } + newxprt->sc_sq_cq = ib_create_cq(newxprt->sc_cm_id->device, + sq_comp_handler, + cq_event_handler, + newxprt, + newxprt->sc_sq_depth, + 0); + if (IS_ERR(newxprt->sc_sq_cq)) { + printk(KERN_ERR + "svcrdma: error creating SQ CQ for connect request\n"); + ret = PTR_ERR(newxprt->sc_sq_cq); + goto errout; + } + newxprt->sc_rq_cq = ib_create_cq(newxprt->sc_cm_id->device, + rq_comp_handler, + cq_event_handler, + newxprt, + newxprt->sc_max_requests, + 0); + if (IS_ERR(newxprt->sc_rq_cq)) { + printk(KERN_ERR + "svcrdma: error creating RQ CQ for connect request\n"); + ret = PTR_ERR(newxprt->sc_rq_cq); + goto errout; + } + + memset(&qp_attr, 0, sizeof qp_attr); + qp_attr.event_handler = qp_event_handler; + qp_attr.qp_context = newxprt; + qp_attr.cap.max_send_wr = newxprt->sc_sq_depth; + qp_attr.cap.max_recv_wr = newxprt->sc_max_requests; + qp_attr.cap.max_send_sge = newxprt->sc_max_sge; + qp_attr.cap.max_recv_sge = newxprt->sc_max_sge; + qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR; + qp_attr.qp_type = IB_QPT_RC; + qp_attr.send_cq = newxprt->sc_sq_cq; + qp_attr.recv_cq = newxprt->sc_rq_cq; + dprintk("newxprt->sc_cm_id=%p, newxprt->sc_pd=%p\n" + "cm_id->device=%p, sc_pd->device=%p\n" + "qp_attr.cap.max_send_wr = %d\n" + "qp_attr.cap.max_recv_wr = %d\n" + "qp_attr.cap.max_send_sge = %d\n" + "qp_attr.cap.max_recv_sge = %d\n", + newxprt->sc_cm_id, newxprt->sc_pd, + newxprt->sc_cm_id->device, newxprt->sc_pd->device, + qp_attr.cap.max_send_wr, + qp_attr.cap.max_recv_wr, + qp_attr.cap.max_send_sge, + qp_attr.cap.max_recv_sge); + + ret = rdma_create_qp(newxprt->sc_cm_id, newxprt->sc_pd, &qp_attr); + if (ret) { + /* + * XXX: This is a hack. We need a xx_request_qp interface + * that will adjust the qp_attr's with a best-effort + * number + */ + qp_attr.cap.max_send_sge -= 2; + qp_attr.cap.max_recv_sge -= 2; + ret = rdma_create_qp(newxprt->sc_cm_id, newxprt->sc_pd, &qp_attr); + if (ret) { + printk(KERN_ERR "svcrdma: failed to create QP, ret=%d\n", ret); + goto errout; + } + newxprt->sc_max_sge = qp_attr.cap.max_send_sge; + newxprt->sc_max_sge = qp_attr.cap.max_recv_sge; + newxprt->sc_sq_depth = qp_attr.cap.max_send_wr; + newxprt->sc_max_requests = qp_attr.cap.max_recv_wr; + } + newxprt->sc_qp = newxprt->sc_cm_id->qp; + + /* Register all of physical memory */ + newxprt->sc_phys_mr = ib_get_dma_mr(newxprt->sc_pd, + IB_ACCESS_LOCAL_WRITE | + IB_ACCESS_REMOTE_WRITE); + if (IS_ERR(newxprt->sc_phys_mr)) { + ret = PTR_ERR(newxprt->sc_phys_mr); + printk(KERN_ERR + "svcrdma: Failed to create DMA MR ret=%d\n", ret); + goto errout; + } + + /* Post receive buffers */ + for (i=0; i < newxprt->sc_max_requests; i++) + if ((ret = svc_rdma_post_recv(newxprt))) { + printk(KERN_ERR + "svcrdma: failure posting receive buffers\n"); + goto errout; + } + + /* Swap out the handler */ + newxprt->sc_cm_id->event_handler = rdma_cma_handler; + + /* We will get a getattr request from the client before we see + * the connect complete event because DTO's run on tasklets, + * and connection events run on threads + */ + clear_bit(SK_BUSY, &newxprt->sc_xprt.sk_flags); + + /* Accept Connection */ + memset(&conn_param, 0, sizeof conn_param); + conn_param.responder_resources = 0; + conn_param.initiator_depth = newxprt->sc_ord; + ret = rdma_accept(newxprt->sc_cm_id, &conn_param); + if (ret) { + printk(KERN_ERR + "svcrdma: failed to accept new connection, ret=%d\n", + ret); + goto errout; + } + + dprintk("svcrdma: new connection %p accepted with the following " + "attributes:\n" + "\tlocal_ip : %d.%d.%d.%d\n" + "\tlocal_port : %d\n" + "\tremote_ip : %d.%d.%d.%d\n" + "\tremote_port : %d\n" + "\tmax_sge : %d\n" + "\tsq_depth : %d\n" + "\tmax_requests : %d\n" + "\tord : %d\n", + newxprt, + NIPQUAD(((struct sockaddr_in*)&newxprt->sc_cm_id-> + route.addr.src_addr)->sin_addr.s_addr), + ntohs(((struct sockaddr_in*)&newxprt->sc_cm_id-> + route.addr.src_addr)->sin_port), + NIPQUAD(((struct sockaddr_in*)&newxprt->sc_cm_id-> + route.addr.dst_addr)->sin_addr.s_addr), + ntohs(((struct sockaddr_in*)&newxprt->sc_cm_id-> + route.addr.dst_addr)->sin_port), + newxprt->sc_max_sge, + newxprt->sc_sq_depth, + newxprt->sc_max_requests, + newxprt->sc_ord); + + spin_lock_bh(&listen_xprt->sc_lock); + if (list_empty(&listen_xprt->sc_accept_q)) + clear_bit(SK_CONN, &listen_xprt->sc_xprt.sk_flags); + spin_unlock_bh(&listen_xprt->sc_lock); + listen_xprt->sc_xprt.sk_pool = NULL; + BUG_ON(test_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags)==0); + clear_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags); + svc_sock_enqueue(&listen_xprt->sc_xprt); + + /* Add to the server's temporary (connected) socket list */ + svc_sock_add_connection(&newxprt->sc_xprt); + + ib_req_notify_cq(newxprt->sc_sq_cq, IB_CQ_NEXT_COMP); + ib_req_notify_cq(newxprt->sc_rq_cq, IB_CQ_NEXT_COMP); + return ret; + + errout: + printk(KERN_ERR "svcrdma: failure accepting new connection rc=%d.\n", + ret); + BUG_ON(test_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags)==0); + clear_bit(SK_BUSY, &listen_xprt->sc_xprt.sk_flags); + clear_bit(SK_CONN, &listen_xprt->sc_xprt.sk_flags); + rdma_destroy_id(newxprt->sc_cm_id); + rdma_destroy_xprt(newxprt); + return 0; /* ret; */ +} + +/* Disable data ready events for this connection */ +static void svc_rdma_detach(struct svc_sock *svsk) +{ + struct svcxprt_rdma *rdma = (struct svcxprt_rdma*)svsk; + unsigned long flags; + + dprintk("svc: svc_rdma_detach(%p)\n", svsk); + + /* + * Shutdown the connection. This will ensure we don't get any + * more events from the provider. + */ + rdma_disconnect(rdma->sc_cm_id); + rdma_destroy_id(rdma->sc_cm_id); + + /* We may already be on the DTO list, however */ + spin_lock_irqsave(&dto_lock, flags); + if (!list_empty(&rdma->sc_dto_q)) + list_del_init(&rdma->sc_dto_q); + spin_unlock_irqrestore(&dto_lock, flags); +} + +static void svc_rdma_free(struct svc_sock *svsk) +{ + struct svcxprt_rdma *xprt = (struct svcxprt_rdma *)svsk; + dprintk("svcrdma: svc_rdma_free(%p)\n", svsk); + + rdma_destroy_xprt(xprt); + kfree(svsk); +} + +static void rdma_destroy_xprt(struct svcxprt_rdma *xprt) +{ + if (xprt->sc_qp) + ib_destroy_qp(xprt->sc_qp); + + if (xprt->sc_sq_cq) + ib_destroy_cq(xprt->sc_sq_cq); + + if (xprt->sc_rq_cq) + ib_destroy_cq(xprt->sc_rq_cq); + + if (xprt->sc_pd) + ib_dealloc_pd(xprt->sc_pd); + + destroy_context_cache(xprt->sc_ctxt_head); +} + +static int svc_rdma_has_wspace(struct svc_sock *svsk) +{ + struct svcxprt_rdma *xprt = (struct svcxprt_rdma *)svsk; + /* + * If there are fewer SQ WR available than required to send a + * simple response, return false. + */ + if ((xprt->sc_sq_depth - atomic_read(&xprt->sc_sq_count) < 3)) + return 0; + + /* + * ...or there are already waiters on the SQ, + * return false. + */ + if (waitqueue_active(&xprt->sc_send_wait)) + return 0; + + /* Otherwise return true. */ + return 1; +} + +int svc_rdma_send(struct svcxprt_rdma *xprt, struct ib_send_wr *wr) +{ + struct ib_send_wr *bad_wr; + int ret; + + if (test_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags)) + return 0; + + BUG_ON(wr->send_flags != IB_SEND_SIGNALED); + BUG_ON(((struct svc_rdma_op_ctxt*)(unsigned long)wr->wr_id)->wr_op != + wr->opcode); + /* If the SQ is full, wait until an SQ entry is available */ + while (1) { + spin_lock_bh(&xprt->sc_lock); + if (xprt->sc_sq_depth == atomic_read(&xprt->sc_sq_count)) { + spin_unlock_bh(&xprt->sc_lock); + rdma_stat_sq_starve ++; + /* First see if we can opportunistically reap some SQ WR */ + sq_cq_reap(xprt); + + /* Wait until SQ WR available if SQ still full */ + wait_event(xprt->sc_send_wait, + atomic_read(&xprt->sc_sq_count) < xprt->sc_sq_depth); + continue; + } + /* Bumped used SQ WR count and post */ + ret = ib_post_send(xprt->sc_qp, wr, &bad_wr); + if (!ret) + atomic_inc(&xprt->sc_sq_count); + else { + printk(KERN_ERR "svcrdma: failed to post SQ WR rc=%d, " + "sc_sq_count=%d, sc_sq_depth=%d\n", + ret, atomic_read(&xprt->sc_sq_count), + xprt->sc_sq_depth); + } + spin_unlock_bh(&xprt->sc_lock); + break; + } + + return ret; +} + +int svc_rdma_send_error(struct svcxprt_rdma *xprt, struct rpcrdma_msg *rmsgp, + enum rpcrdma_errcode err) +{ + struct ib_send_wr err_wr; + struct ib_sge sge; + struct page *p; + struct svc_rdma_op_ctxt *ctxt; + u32 *va; + int length; + int ret; + + p = svc_rdma_get_page(); + va = page_address(p); + + /* XDR encode error */ + length = svc_rdma_xdr_encode_error(xprt, rmsgp, err, va); + + /* Prepare SGE for local address */ + sge.addr = ib_dma_map_page(xprt->sc_cm_id->device, + p, 0, PAGE_SIZE, DMA_FROM_DEVICE); + sge.lkey = xprt->sc_phys_mr->lkey; + sge.length = length; + + ctxt = svc_rdma_get_context(xprt); + ctxt->count = 1; + ctxt->pages[0] = p; + + /* Prepare SEND WR */ + memset(&err_wr, 0, sizeof err_wr); + ctxt->wr_op = IB_WR_SEND; + err_wr.wr_id = (unsigned long)ctxt; + err_wr.sg_list = &sge; + err_wr.num_sge = 1; + err_wr.opcode = IB_WR_SEND; + err_wr.send_flags = IB_SEND_SIGNALED; + + /* Post It */ + ret = svc_rdma_send(xprt, &err_wr); + if (ret) { + dprintk("svcrdma: Error posting send = %d\n", ret); + svc_rdma_put_context(ctxt,1); + } + + return ret; +} + +/* + * This request cannot be handled right now. Allocate a structure to + * keep it's state pending completion processing. To accomplish this, the + * function creates an svc_rdma_op_ctxt that looks like a receive completion and + * enqueues it on the svc_sock's deferred request list. When* + * svc_rdma_recvfrom is subsequently called, it first checks if there is a + * deferred RPC and if there is: + * - Takes the deferred request off the deferred request queue + * - Extracts the svc_rdma_op_ctxt from the deferred request structure + * - Frees the deferred request structure + * - Skips the ib_cq_poll call and processes the svc_rdma_op_ctxt as if it had + * just come out of an WR pulled from the CQ. + */ +static struct cache_deferred_req * +svc_rdma_defer(struct cache_req *req) +{ + struct svc_rqst *rqstp = container_of(req, struct svc_rqst, rq_chandle); + struct svcxprt_rdma *xprt; + struct svc_rdma_deferred_req *dr; + + dprintk("svcrdma: deferring request on \n" + " rqstp=%p\n" + " rqstp->rq_arg.len=%d\n", + rqstp, + rqstp->rq_arg.len); + + /* if more than a page, give up FIXME */ + if (rqstp->rq_arg.page_len) + return NULL; + + BUG_ON(rqstp->rq_deferred); + xprt = (struct svcxprt_rdma*)rqstp->rq_sock; + dr = kmalloc(sizeof(struct svc_rdma_deferred_req), GFP_KERNEL); + if (!dr) + return NULL; + + dr->req.handle.owner = rqstp->rq_server; + dr->req.prot = rqstp->rq_prot; + dr->req.addr = rqstp->rq_addr; + dr->req.daddr = rqstp->rq_daddr; + dr->req.argslen = rqstp->rq_arg.len >> 2; + dr->arg_page = rqstp->rq_pages[0]; + dr->arg_len = rqstp->rq_arg.len; + rqstp->rq_pages[0] = svc_rdma_get_page(); + + svc_sock_get(rqstp->rq_sock); + dr->req.svsk = rqstp->rq_sock; + dr->req.handle.revisit = svc_revisit; + + return &dr->req.handle; +} + diff --git a/net/sunrpc/svcauth_unix.c b/net/sunrpc/svcauth_unix.c index 07dcd20..4029924 100644 --- a/net/sunrpc/svcauth_unix.c +++ b/net/sunrpc/svcauth_unix.c @@ -409,9 +409,8 @@ static inline void ip_map_cached_put(struct svc_rqst *rqstp, struct ip_map *ipm) { struct svc_sock *svsk = rqstp->rq_sock; - spin_lock(&svsk->sk_lock); - if (svsk->sk_sock->type == SOCK_STREAM && + if (test_bit(SK_TEMP, &svsk->sk_flags) && svsk->sk_info_authunix == NULL) { /* newly cached, keep the reference */ svsk->sk_info_authunix = ipm; From tom at opengridcomputing.com Mon Aug 20 12:01:36 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:36 -0500 Subject: [ofa-general] [RFC,PATCH 06/10] rdma: SVCRDMA recvfrom In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190136.18615.1863.stgit@dell3.ogc.int> This file implements the RDMA transport recvfrom function. The function dequeues work reqeust completion contexts from an I/O list that it shares with the I/O tasklet in svc_rdma_transport.c. For ONCRPC RDMA, an RPC may not be complete when it is received. Instead, the RDMA header that precedes the RPC message informs the transport where to get the RPC data from on the client and where to place it in the RPC message before it is delivered to the server. The svc_rdma_recvfrom function therefore, parses this RDMA header and issues any necessary RDMA operations to fetch the remainder of the RPC from the client. Special handling is required when the request involves an RDMA_READ in this case, submits all of the RDMA_READ requests to the underlying transport driver and then returns 0 (EAGAIN). When the transport completes the last RDMA_READ for the request, it enqueues it on an read completion queue and enqueues the transport. The recvfrom code favors this queue over the regular DTO queue when satisfying reads. Signed-off-by: Tom Tucker --- net/sunrpc/svc_rdma_recvfrom.c | 664 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 664 insertions(+), 0 deletions(-) diff --git a/net/sunrpc/svc_rdma_recvfrom.c b/net/sunrpc/svc_rdma_recvfrom.c new file mode 100644 index 0000000..681f25a --- /dev/null +++ b/net/sunrpc/svc_rdma_recvfrom.c @@ -0,0 +1,664 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include /* num_physpages */ +#include +#include +#include +#include +#include +#include +#include +#include + +#define RPCDBG_FACILITY RPCDBG_SVCXPRT + +/* + * Replace the pages in the rq_argpages array with the pages from the SGE in + * the RDMA_RECV completion. The SGL should contain full pages up until the + * last one. + */ +static void rdma_build_arg_xdr(struct svc_rqst *rqstp, + struct svc_rdma_op_ctxt *ctxt, + u32 byte_count) +{ + struct page *page; + u32 bc; + int sge_no; + + /* Swap the page in the SGE with the page in argpages */ + page = ctxt->pages[0]; + put_page(rqstp->rq_pages[0]); + rqstp->rq_pages[0] = page; + + /* Set up the XDR head */ + rqstp->rq_arg.head[0].iov_base = page_address(page); + rqstp->rq_arg.head[0].iov_len = min(byte_count, ctxt->sge[0].length); + rqstp->rq_arg.len = byte_count; + rqstp->rq_arg.buflen = byte_count; + + /* Compute bytes past head in the SGL */ + bc = byte_count - rqstp->rq_arg.head[0].iov_len; + + /* If data remains, store it in the pagelist */ + rqstp->rq_arg.page_len = bc; + rqstp->rq_arg.page_base = 0; + rqstp->rq_arg.pages = &rqstp->rq_pages[1]; + sge_no = 1; + while (bc && sge_no < ctxt->count) { + page = ctxt->pages[sge_no]; + put_page(rqstp->rq_pages[sge_no]); + rqstp->rq_pages[sge_no] = page; + bc -= min(bc, ctxt->sge[sge_no].length); + rqstp->rq_arg.buflen += ctxt->sge[sge_no].length; + sge_no ++; + } + rqstp->rq_respages = &rqstp->rq_pages[sge_no]; + + /* We should never run out of SGE because the limit is defined to + * support the max allowed RPC data length + */ + BUG_ON(bc && (sge_no == ctxt->count)); + BUG_ON((rqstp->rq_arg.head[0].iov_len + rqstp->rq_arg.page_len) + != byte_count); + BUG_ON(rqstp->rq_arg.len != byte_count); + + /* If not all pages were used from the SGL, free the remaining ones */ + bc = sge_no; + while (sge_no < ctxt->count) { + page = ctxt->pages[sge_no++]; + put_page(page); + } + ctxt->count = bc; + + /* Set up tail */ + rqstp->rq_arg.tail[0].iov_base = NULL; + rqstp->rq_arg.tail[0].iov_len = 0; +} + +struct chunk_sge { + int start; /* sge no for this chunk */ + int count; /* sge count for this chunk */ +}; + +/* Encode a read-chunk-list as an array of IB SGE + * + * Assumptions: + * - chunk[0]->position points to pages[0] at an offset of 0 + * - pages[] is not physically or virtually contigous and consists of + * PAGE_SIZE elements. + * - The context at 'head' will be used to create a copy of + * rq_arg.pages for deferred processing in recvfrom + * + * Output: + * - sge array pointing into pages[] array. + * - chunk_sge array specifying sge index and count for each + * chunk in the read list + * + */ +static int rdma_rcl_to_sge(struct svcxprt_rdma *xprt, + struct svc_rqst *rqstp, + struct svc_rdma_op_ctxt *head, + struct rpcrdma_msg *rmsgp, + struct ib_sge *sge, + struct chunk_sge *ch_sge_ary, + int ch_count, + int byte_count) +{ + int sge_no; + int sge_bytes; + int page_off; + int page_no; + int ch_bytes; + int ch_no; + struct rpcrdma_read_chunk *ch; + + sge_no = 0; + page_no = 0; + page_off = 0; + ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; + ch_no = 0; + ch_bytes = ch->rc_target.rs_length; + head->arg.head[0] = rqstp->rq_arg.head[0]; + head->arg.tail[0] = rqstp->rq_arg.tail[0]; + head->arg.pages = &head->pages[head->count]; + head->sge[0].length = head->count; /* save count of hdr pages */ + head->arg.page_base = 0; + head->arg.page_len = ch_bytes; + head->arg.len = rqstp->rq_arg.len + ch_bytes; + head->arg.buflen = rqstp->rq_arg.buflen + ch_bytes; + head->count ++; + ch_sge_ary[0].start = 0; + while (byte_count) { + sge_bytes = min_t(int, PAGE_SIZE-page_off, ch_bytes); + sge[sge_no].addr = + ib_dma_map_page(xprt->sc_cm_id->device, + rqstp->rq_arg.pages[page_no], + page_off, sge_bytes, + DMA_FROM_DEVICE); + sge[sge_no].length = sge_bytes; + sge[sge_no].lkey = xprt->sc_phys_mr->lkey; + /* + * Don't bump head->count here because the same page + * may be used by multiple SGE. + */ + head->arg.pages[page_no] = rqstp->rq_arg.pages[page_no]; + rqstp->rq_respages = &rqstp->rq_arg.pages[page_no+1]; + + byte_count -= sge_bytes; + ch_bytes -= sge_bytes; + sge_no ++; + /* + * If all bytes for this chunk have been mapped to an + * SGE, move to the next SGE + */ + if (ch_bytes == 0) { + ch_sge_ary[ch_no].count = + sge_no - ch_sge_ary[ch_no].start; + ch_no ++; + ch++; + ch_sge_ary[ch_no].start = sge_no; + ch_bytes = ch->rc_target.rs_length; + /* If bytes remaining account for next chunk */ + if (byte_count) { + head->arg.page_len += ch_bytes; + head->arg.len += ch_bytes; + head->arg.buflen += ch_bytes; + } + } + /* + * If this SGE consumed all of the page, move to the + * next page + */ + if ((sge_bytes + page_off) == PAGE_SIZE) { + page_no ++; + page_off = 0; + /* + * If there are still bytes left to map, bump + * the page count + */ + if (byte_count) + head->count ++; + } else + page_off += sge_bytes; + } + BUG_ON(byte_count != 0); + return sge_no; +} + +static void rdma_set_ctxt_sge(struct svc_rdma_op_ctxt *ctxt, + struct ib_sge *sge, + u64 *sgl_offset, + int count) +{ + int i; + + ctxt->count = count; + for (i = 0; i < count; i++) { + ctxt->sge[i].addr = sge[i].addr; + ctxt->sge[i].length = sge[i].length; + *sgl_offset = *sgl_offset + sge[i].length; + } +} + +static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count) +{ +#ifdef RDMA_TRANSPORT_IWARP + if ((RDMA_TRANSPORT_IWARP == + rdma_node_get_transport(xprt->sc_cm_id-> + device->node_type)) + && sge_count > 1) + return 1; + else +#endif + return min_t(int, sge_count, xprt->sc_max_sge); +} + +/* + * Use RDMA_READ to read data from the advertised client buffer into the + * XDR stream starting at rq_arg.head[0].iov_base. + * Each chunk in the array + * contains the following fields: + * discrim - '1', This isn't used for data placement + * position - The xdr stream offset (the same for every chunk) + * handle - RMR for client memory region + * length - data transfer length + * offset - 64 bit tagged offset in remote memory region + * + * On our side, we need to read into a pagelist. The first page immediately + * follows the RPC header. + * + * This function returns 1 to indicate success. The data is not yet in + * the pagelist and therefore the RPC request must be deferred. The + * I/O completion will enqueue the transport again and + * svc_rdma_recvfrom will complete the request. + * + * NOTE: The ctxt must not be touched after the last WR has been posted + * because the I/O completion processing may occur on another + * processor and free / modify the context. Ne touche pas! + */ +static int +rdma_read_xdr(struct svcxprt_rdma *xprt, + struct rpcrdma_msg *rmsgp, + struct svc_rqst *rqstp, + struct svc_rdma_op_ctxt *hdr_ctxt) +{ + struct ib_send_wr read_wr; + int err = 0; + int ch_no; + struct ib_sge *sge; + int ch_count; + int byte_count; + int sge_count; + u64 sgl_offset; + struct rpcrdma_read_chunk *ch; + struct svc_rdma_op_ctxt *ctxt = NULL; + struct svc_rdma_op_ctxt *head; + struct svc_rdma_op_ctxt *tmp_sge_ctxt; + struct svc_rdma_op_ctxt *tmp_ch_ctxt; + struct chunk_sge *ch_sge_ary; + + /* If no read list is present, return 0 */ + ch = svc_rdma_get_read_chunk(rmsgp); + if (!ch) + return 0; + + /* Allocate temporary contexts to keep SGE */ + BUG_ON(sizeof(struct ib_sge) < sizeof(struct chunk_sge)); + tmp_sge_ctxt = svc_rdma_get_context(xprt); + sge = tmp_sge_ctxt->sge; + tmp_ch_ctxt = svc_rdma_get_context(xprt); + ch_sge_ary = (struct chunk_sge*)tmp_ch_ctxt->sge; + + svc_rdma_rcl_chunk_counts(ch, &ch_count, &byte_count); + sge_count = rdma_rcl_to_sge(xprt, rqstp, hdr_ctxt, rmsgp, + sge, ch_sge_ary, + ch_count, byte_count); + head = svc_rdma_get_context(xprt); + sgl_offset = 0; + ch_no = 0; + + for (ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; + ch->rc_discrim != 0; ch++, ch_no++) { + next_sge: + if (!ctxt) + ctxt = head; + else { + ctxt->next = svc_rdma_get_context(xprt); + ctxt = ctxt->next; + } + ctxt->next = NULL; + ctxt->direction = DMA_FROM_DEVICE; + clear_bit(RDMACTXT_F_READ_DONE, &ctxt->flags); + clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags); + if ((ch+1)->rc_discrim == 0) { + /* + * Checked in sq_cq_reap to see if we need to + * be enqueued + */ + set_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags); + ctxt->next = hdr_ctxt; + hdr_ctxt->next = head; + } + + /* Prepare READ WR */ + memset(&read_wr, 0, sizeof read_wr); + ctxt->wr_op = IB_WR_RDMA_READ; + read_wr.wr_id = (unsigned long)ctxt; + read_wr.opcode = IB_WR_RDMA_READ; + read_wr.send_flags = IB_SEND_SIGNALED; + read_wr.wr.rdma.rkey = ch->rc_target.rs_handle; + read_wr.wr.rdma.remote_addr = + get_unaligned(&(ch->rc_target.rs_offset)) + + sgl_offset; + read_wr.sg_list = &sge[ch_sge_ary[ch_no].start]; + read_wr.num_sge = + rdma_read_max_sge(xprt, ch_sge_ary[ch_no].count); + rdma_set_ctxt_sge(ctxt, &sge[ch_sge_ary[ch_no].start], + &sgl_offset, + read_wr.num_sge); + + /* Post the read */ + err = svc_rdma_send(xprt, &read_wr); + if (err) { + printk(KERN_ERR "svcrdma: Error posting send =%d\n", err); + /* + * Break the circular list so free knows when + * to stop if the error happened to occur on + * the last read + */ + ctxt->next = NULL; + goto out; + } + rdma_stat_read ++; + + if (read_wr.num_sge < ch_sge_ary[ch_no].count) { + ch_sge_ary[ch_no].count -= read_wr.num_sge; + ch_sge_ary[ch_no].start += read_wr.num_sge; + goto next_sge; + } + sgl_offset = 0; + err = 0; + } + + out: + svc_rdma_put_context(tmp_sge_ctxt, 0); + svc_rdma_put_context(tmp_ch_ctxt, 0); + + /* Detach arg pages. svc_recv will replenish them */ + for (ch_no=0; &rqstp->rq_pages[ch_no] < rqstp->rq_respages; ch_no++) + rqstp->rq_pages[ch_no] = NULL; + + /* + * Detach res pages. svc_release must see a resused count of + * zero or it will attempt to put them. + */ + while (rqstp->rq_resused) + rqstp->rq_respages[--rqstp->rq_resused] = NULL; + + if (err) { + printk(KERN_ERR "svcrdma : RDMA_READ error = %d\n", err); + set_bit(SK_CLOSE, &xprt->sc_xprt.sk_flags); + /* Free the linked list of read contexts */ + while (head != NULL) { + ctxt = head->next; + svc_rdma_put_context(head, 1); + head = ctxt; + } + return 0; + } + + return 1; +} + +static struct svc_deferred_req * +svc_rdma_deferred_dequeue(struct svc_sock *xprt) +{ + struct svc_deferred_req *dr = NULL; + spin_lock_bh(&xprt->sk_server->sv_lock); + clear_bit(SK_DEFERRED, &xprt->sk_flags); + if (!list_empty(&xprt->sk_deferred)) { + dr = list_entry(xprt->sk_deferred.next, + struct svc_deferred_req, + handle.recent); + list_del_init(&dr->handle.recent); + set_bit(SK_DEFERRED, &xprt->sk_flags); + } + spin_unlock_bh(&xprt->sk_server->sv_lock); + return dr; +} + +static int svc_rdma_deferred_recv(struct svc_rqst *rqstp) +{ + struct svc_sock *xprt = rqstp->rq_sock; + struct svcxprt_rdma *rdma_xprt = (struct svcxprt_rdma*)xprt; + struct svc_rdma_deferred_req *dr = + (struct svc_rdma_deferred_req *)rqstp->rq_deferred; + void *page_va; + int ret; + int len; + + /* Rebuild the thread context */ + rqstp->rq_prot = IPPROTO_MAX; + memcpy(&rqstp->rq_addr, + &rdma_xprt->sc_cm_id->route.addr.dst_addr, + sizeof(rqstp->rq_addr)); + rqstp->rq_addrlen = sizeof(rqstp->rq_addr); + + page_va = page_address(dr->arg_page); + dprintk("svcrdma: satisfying receive from deferred req=%p\n" + "\tpage=%p, page va=%p\n\targ_len=%d\n", + dr, dr->arg_page, page_va, + dr->arg_len); + + /* Replace page in rq_arg.head */ + put_page(rqstp->rq_pages[0]); + rqstp->rq_pages[0] = dr->arg_page; + rqstp->rq_arg.head[0].iov_base = page_address(rqstp->rq_pages[0]); + rqstp->rq_arg.head[0].iov_len = dr->arg_len; + rqstp->rq_arg.tail[0].iov_base = NULL; + rqstp->rq_arg.tail[0].iov_len = 0; + + /* There are no additional pages */ + rqstp->rq_arg.pages = NULL; + rqstp->rq_arg.page_base = 0; + rqstp->rq_arg.page_len = 0; + rqstp->rq_arg.len = dr->arg_len; + + /* Prep the response pages */ + rqstp->rq_respages = &rqstp->rq_pages[1]; + + len = svc_rdma_xdr_decode_deferred_req(rqstp); + + ret = rqstp->rq_arg.head[0].iov_len + + rqstp->rq_arg.page_len + + rqstp->rq_arg.tail[0].iov_len; + + rqstp->rq_deferred = NULL; + kfree(dr); + svc_sock_received(xprt); + return ret; +} + +static int +rdma_read_complete(struct svc_rqst *rqstp, struct svc_rdma_op_ctxt *data) +{ + struct svc_rdma_op_ctxt *head = data->next; + struct svcxprt_rdma *rdma_xprt = (struct svcxprt_rdma*)rqstp->rq_sock; + int page_no; + int ret; + + BUG_ON(!head); + + /* Copy RPC pages */ + for (page_no=0; page_no < head->count; page_no++) { + put_page(rqstp->rq_pages[page_no]); + rqstp->rq_pages[page_no] = head->pages[page_no]; + } + /* Point rq_arg.pages past header */ + rqstp->rq_arg.pages = &rqstp->rq_pages[head->sge[0].length]; + rqstp->rq_arg.page_len = head->arg.page_len; + rqstp->rq_arg.page_base = head->arg.page_base; + + /* rq_respages starts after the last arg page */ + rqstp->rq_respages = &rqstp->rq_arg.pages[page_no]; + rqstp->rq_resused = 0; + + /* Rebuild rq_arg head and tail. */ + rqstp->rq_arg.head[0] = head->arg.head[0]; + rqstp->rq_arg.tail[0] = head->arg.tail[0]; + rqstp->rq_arg.len = head->arg.len; + rqstp->rq_arg.buflen = head->arg.buflen; + + rqstp->rq_prot = IPPROTO_MAX; + memcpy(&rqstp->rq_addr, + &rdma_xprt->sc_cm_id->route.addr.dst_addr, + sizeof(rqstp->rq_addr)); + rqstp->rq_addrlen = sizeof(rqstp->rq_addr); + + /* + * Free the contexts we used to build the RDMA_READ. We have + * to be careful here because the context list uses the same + * next pointer used to chain the contexts associated with the + * RDMA_READ + */ + data->next = NULL; /* terminate circular list */ + do { + data = head->next; + svc_rdma_put_context(head, 0); + head = data; + } while (head != NULL); + + svc_sock_received(rqstp->rq_sock); + ret = rqstp->rq_arg.head[0].iov_len + + rqstp->rq_arg.page_len + + rqstp->rq_arg.tail[0].iov_len; + dprintk("svcrdma: deferred read ret=%d, rq_arg.len =%d, " + "rq_arg.head[0].iov_base=%p, rq_arg.head[0].iov_len = %zd\n", + ret, + rqstp->rq_arg.len, + rqstp->rq_arg.head[0].iov_base, + rqstp->rq_arg.head[0].iov_len); + return ret; +} + +/* + * Set up the rqstp thread context to point to the RQ buffer. If + * necessary, pull additional data from the client with an RDMA_READ + * request. + */ +int svc_rdma_recvfrom(struct svc_rqst *rqstp) +{ + struct svc_sock *xprt = rqstp->rq_sock; + struct svcxprt_rdma *rdma_xprt = (struct svcxprt_rdma*)xprt; + struct svc_rdma_op_ctxt *ctxt = NULL; + struct rpcrdma_msg *rmsgp; + int ret = 0; + int len; + + dprintk("svcrdma: rqstp=%p\n", rqstp); + if (unlikely(test_bit(SK_DEFERRED, &xprt->sk_flags))) + if ((rqstp->rq_deferred = svc_rdma_deferred_dequeue(xprt))) + return svc_rdma_deferred_recv(rqstp); + + spin_lock_bh(&rdma_xprt->sc_read_complete_lock); + if (!list_empty(&rdma_xprt->sc_read_complete_q)) { + ctxt = list_entry(rdma_xprt->sc_read_complete_q.next, + struct svc_rdma_op_ctxt, + dto_q); + list_del_init(&ctxt->dto_q); + } + spin_unlock_bh(&rdma_xprt->sc_read_complete_lock); + if (ctxt) + return rdma_read_complete(rqstp, ctxt); + + spin_lock_bh(&rdma_xprt->sc_rq_dto_lock); + if (!list_empty(&rdma_xprt->sc_rq_dto_q)) { + ctxt = list_entry(rdma_xprt->sc_rq_dto_q.next, + struct svc_rdma_op_ctxt, + dto_q); + list_del_init(&ctxt->dto_q); + set_bit(SK_DATA, &xprt->sk_flags); + } else { + rdma_stat_rq_starve ++; + clear_bit(SK_DATA, &xprt->sk_flags); + ctxt = NULL; + } + spin_unlock_bh(&rdma_xprt->sc_rq_dto_lock); + if (!ctxt) { + /* This is the EAGAIN path. The svc_recv routine will + * return -EAGAIN, the nfsd thread will go to call into + * svc_recv again and we shouldn't be on the active + * transport list + */ + if (test_bit(SK_CLOSE, &xprt->sk_flags)) + goto close_out; + + BUG_ON(ret); + svc_sock_received(xprt); + goto out; + } + dprintk("svcrdma: processing ctxt=%p on xprt=%p, rqstp=%p, status=%d\n", + ctxt, rdma_xprt, rqstp, ctxt->wc_status); + BUG_ON(ctxt->wc_status != IB_WC_SUCCESS); + rdma_stat_recv ++; + + /* Kick another thread for the next RPC */ + svc_sock_received(xprt); + + /* rqstp struct expects transport to fill in peer address */ + rqstp->rq_prot = IPPROTO_MAX; + memcpy(&rqstp->rq_addr, + &rdma_xprt->sc_cm_id->route.addr.dst_addr, + sizeof(rqstp->rq_addr)); + rqstp->rq_addrlen = sizeof(rqstp->rq_addr); + + /* Build up the XDR from the receive buffers. */ + rdma_build_arg_xdr(rqstp, ctxt, ctxt->byte_len); + + /* Decode the RDMA header. */ + len = svc_rdma_xdr_decode_req(&rmsgp, rqstp); + + /* If the request is invalid, reply with an error */ + if (len < 0) { + if (len == -ENOSYS) + (void)svc_rdma_send_error(rdma_xprt, rmsgp, ERR_VERS); + goto close_out; + } + + /* Read read-list data. If we would need to wait, defer it */ + if (rdma_read_xdr(rdma_xprt, rmsgp, rqstp, ctxt)) + return 0; + + ret = rqstp->rq_arg.head[0].iov_len + + rqstp->rq_arg.page_len + + rqstp->rq_arg.tail[0].iov_len; + svc_rdma_put_context(ctxt, 0); + out: + dprintk("svcrdma: ret = %d, rq_arg.len =%d, " + "rq_arg.head[0].iov_base=%p, rq_arg.head[0].iov_len = %zd\n", + ret, rqstp->rq_arg.len, + rqstp->rq_arg.head[0].iov_base, + rqstp->rq_arg.head[0].iov_len); + return ret; + + close_out: + if (ctxt) + svc_rdma_put_context(ctxt, 1); + dprintk("svcrdma: transport %p is closing\n", xprt); + /* + * Set the close bit and enqueue it. svc_recv will see the + * close bit and call svc_sock_delete + */ + set_bit(SK_CLOSE, &xprt->sk_flags); + svc_sock_received(xprt); + + return 0; +} From tom at opengridcomputing.com Mon Aug 20 12:01:39 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:39 -0500 Subject: [ofa-general] [RFC,PATCH 07/10] rdma: SVCRDMA sendto In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190138.18615.68952.stgit@dell3.ogc.int> This file implements the RDMA transport sendto function. A RPC reply on an RDMA transport consists of some number of RDMA_WRITE requests followed by an RDMA_SEND request. The sendto function parses the ONCRPC RDMA reply header to determine how to send the reply back to the client. The send queue is sized so as to be able to send complete replies for requests in most cases. In the event that there are not enough SQ WR slots to reply, e.g. big data, the send will block the NFSD thread. The I/O callback functions in svc_rdma_transport.c that reap WR completions wake any waiters blocked on the SQ. In general, the goal is not to block NFSD threads and the has_wspace method stall requests when the SQ is nearly full. Signed-off-by: Tom Tucker --- net/sunrpc/svc_rdma_sendto.c | 515 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 515 insertions(+), 0 deletions(-) diff --git a/net/sunrpc/svc_rdma_sendto.c b/net/sunrpc/svc_rdma_sendto.c new file mode 100644 index 0000000..cd4b5ac --- /dev/null +++ b/net/sunrpc/svc_rdma_sendto.c @@ -0,0 +1,515 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include /* num_physpages */ +#include +#include +#include +#include +#include +#include +#include +#include + +#define RPCDBG_FACILITY RPCDBG_SVCXPRT + +/* Encode an XDR as an array of IB SGE + * + * Assumptions: + * - head[0] is physically contiguous. + * - tail[0] is physically contiguous. + * - pages[] is not physically or virtually contigous and consists of + * PAGE_SIZE elements. + * + * Output: + * SGE[0] reserved for RCPRDMA header + * SGE[1] data from xdr->head[] + * SGE[2..sge_count-2] data from xdr->pages[] + * SGE[sge_count-1] data from xdr->tail. + * + */ +static struct ib_sge *xdr_to_sge(struct svcxprt_rdma *xprt, + struct xdr_buf *xdr, + struct ib_sge *sge, + int *sge_count) +{ + /* Max we need is the length of the XDR / pagesize + one for + * head + one for tail + one for RPCRDMA header + */ + int sge_max = (xdr->len+PAGE_SIZE-1) / PAGE_SIZE + 3; +int sge_no; + u32 byte_count = xdr->len; + u32 sge_bytes; + u32 page_bytes; + int page_off; + int page_no; + + /* Skip the first sge, this is for the RPCRDMA header */ + sge_no = 1; + + /* Head SGE */ + sge[sge_no].addr = ib_dma_map_single(xprt->sc_cm_id->device, + xdr->head[0].iov_base, + xdr->head[0].iov_len, + DMA_TO_DEVICE); + sge_bytes = min_t(u32, byte_count, xdr->head[0].iov_len); + byte_count -= sge_bytes; + sge[sge_no].length = sge_bytes; + sge[sge_no].lkey = xprt->sc_phys_mr->lkey; + sge_no ++; + + /* pages SGE */ + page_no = 0; + page_bytes = xdr->page_len; + page_off = xdr->page_base; + while (byte_count && page_bytes) { + sge_bytes = min_t(u32, byte_count, (PAGE_SIZE-page_off)); + sge[sge_no].addr = + ib_dma_map_page(xprt->sc_cm_id->device, + xdr->pages[page_no], page_off, sge_bytes, + DMA_TO_DEVICE); + sge_bytes = min(sge_bytes, page_bytes); + byte_count -= sge_bytes; + page_bytes -= sge_bytes; + sge[sge_no].length = sge_bytes; + sge[sge_no].lkey = xprt->sc_phys_mr->lkey; + + sge_no ++; + page_no ++; + page_off = 0; /* reset for next time through loop */ + } + + /* Tail SGE */ + if (byte_count && xdr->tail[0].iov_len) { + sge[sge_no].addr = + ib_dma_map_single(xprt->sc_cm_id->device, + xdr->tail[0].iov_base, + xdr->tail[0].iov_len, + DMA_TO_DEVICE); + sge_bytes = min_t(u32, byte_count, xdr->tail[0].iov_len); + byte_count -= sge_bytes; + sge[sge_no].length = sge_bytes; + sge[sge_no].lkey = xprt->sc_phys_mr->lkey; + sge_no ++; + } + + BUG_ON(sge_no > sge_max); + BUG_ON(byte_count != 0); + + *sge_count = sge_no; + return sge; +} + + +/* Assumptions: + * - The specified write_len can be represented in sc_max_sge * PAGE_SIZE + */ +static int send_write(struct svcxprt_rdma *xprt, struct svc_rqst *rqstp, + u32 rmr, u64 to, + u32 xdr_off, int write_len, + struct ib_sge *xdr_sge, int sge_count) +{ + struct svc_rdma_op_ctxt *tmp_sge_ctxt; + struct ib_send_wr write_wr; + struct ib_sge *sge; + int xdr_sge_no; + int sge_no; + int sge_bytes; + int sge_off; + int bc; + struct svc_rdma_op_ctxt *ctxt; + int ret = 0; + + BUG_ON(sge_count >= 32); + dprintk("svcrdma: RDMA_WRITE rmr=%x, to=%llx, xdr_off=%d, write_len=%d, " + "xdr_sge=%p, sge_count=%d\n", + rmr, to, xdr_off, write_len, xdr_sge, sge_count); + + ctxt = svc_rdma_get_context(xprt); + ctxt->count = 0; + tmp_sge_ctxt = svc_rdma_get_context(xprt); + sge = tmp_sge_ctxt->sge; + + /* Find the SGE associated with xdr_off */ + for (bc = xdr_off, xdr_sge_no=1; bc && xdr_sge_no < sge_count; + xdr_sge_no++) { + if (xdr_sge[xdr_sge_no].length > bc) + break; + bc -= xdr_sge[xdr_sge_no].length; + } + + sge_off = bc; + bc = write_len; + sge_no = 0; + + /* Copy the remaining SGE */ + while (bc != 0 && xdr_sge_no < sge_count) { + sge[sge_no].addr = xdr_sge[xdr_sge_no].addr + sge_off; + sge[sge_no].lkey = xdr_sge[xdr_sge_no].lkey; + sge_bytes = min((size_t)bc, + (size_t)(xdr_sge[xdr_sge_no].length-sge_off)); + sge[sge_no].length = sge_bytes; + + sge_off = 0; + sge_no ++; + xdr_sge_no ++; + bc -= sge_bytes; + } + + BUG_ON(bc != 0); + BUG_ON(xdr_sge_no > sge_count); + + /* Prepare WRITE WR */ + memset(&write_wr, 0, sizeof write_wr); + ctxt->wr_op = IB_WR_RDMA_WRITE; + write_wr.wr_id = (unsigned long)ctxt; + write_wr.sg_list = &sge[0]; + write_wr.num_sge = sge_no; + write_wr.opcode = IB_WR_RDMA_WRITE; + write_wr.send_flags = IB_SEND_SIGNALED; + write_wr.wr.rdma.rkey = rmr; + write_wr.wr.rdma.remote_addr = to; + + /* Post It */ + rdma_stat_write ++; + if (svc_rdma_send(xprt, &write_wr)) { + svc_rdma_put_context(ctxt,1); + /* Fatal error, close transport */ + ret = -EIO; + } + svc_rdma_put_context(tmp_sge_ctxt, 0); + return ret; +} + +static int send_write_chunks(struct svcxprt_rdma *xprt, + struct rpcrdma_msg *rdma_argp, + struct rpcrdma_msg *rdma_resp, + struct svc_rqst *rqstp, + struct ib_sge *sge, + int sge_count) +{ + u32 xfer_len = rqstp->rq_res.page_len + rqstp->rq_res.tail[0].iov_len; + int write_len; + int max_write; + u32 xdr_off; + int chunk_off; + int chunk_no; + struct rpcrdma_write_array *arg_ary; + struct rpcrdma_write_array *res_ary; + int ret; + + arg_ary = svc_rdma_get_write_array(rdma_argp); + if (!arg_ary) + return 0; + res_ary = (struct rpcrdma_write_array*) + &rdma_resp->rm_body.rm_chunks[1]; + + max_write = xprt->sc_max_sge * PAGE_SIZE; + + /* Write chunks start at the pagelist */ + for (xdr_off = rqstp->rq_res.head[0].iov_len, chunk_no = 0; + xfer_len && chunk_no < arg_ary->wc_nchunks; + chunk_no++) { + struct rpcrdma_segment *arg_ch; + u64 rs_offset; + + arg_ch = &arg_ary->wc_array[chunk_no].wc_target; + write_len = min(xfer_len, arg_ch->rs_length); + + /* Prepare the response chunk given the length actually + * written */ + rs_offset = get_unaligned(&(arg_ch->rs_offset)); + svc_rdma_xdr_encode_array_chunk(res_ary, chunk_no, + arg_ch->rs_handle, + rs_offset, + write_len); + chunk_off = 0; + while (write_len) { + int this_write; + this_write = min(write_len, max_write); + ret = send_write(xprt, rqstp, + arg_ch->rs_handle, + rs_offset + chunk_off, + xdr_off, + this_write, + sge, + sge_count); + if (ret) { + dprintk("svcrdma: RDMA_WRITE failed, ret=%d\n", ret); + return -EIO; + } + chunk_off += this_write; + xdr_off += this_write; + xfer_len -= this_write; + write_len -= this_write; + } + } + /* Update the req with the number of chunks actually used */ + svc_rdma_xdr_encode_write_list(rdma_resp, chunk_no); + + return rqstp->rq_res.page_len + rqstp->rq_res.tail[0].iov_len; +} + +static int send_reply_chunks(struct svcxprt_rdma *xprt, + struct rpcrdma_msg *rdma_argp, + struct rpcrdma_msg *rdma_resp, + struct svc_rqst *rqstp, + struct ib_sge *sge, + int sge_count) +{ + u32 xfer_len = rqstp->rq_res.len; + int write_len; + int max_write; + u32 xdr_off; + int chunk_no; + int chunk_off; + struct rpcrdma_segment *ch; + struct rpcrdma_write_array *arg_ary; + struct rpcrdma_write_array *res_ary; + int ret; + + arg_ary = svc_rdma_get_reply_array(rdma_argp); + if (!arg_ary) + return 0; + /* XXX: need to fix when reply lists occur with read-list and or + * write-list */ + res_ary = (struct rpcrdma_write_array*) + &rdma_resp->rm_body.rm_chunks[2]; + + max_write = xprt->sc_max_sge * PAGE_SIZE; + + /* xdr offset starts at RPC message */ + for (xdr_off = 0, chunk_no = 0; + xfer_len && chunk_no < arg_ary->wc_nchunks; + chunk_no++) { + u64 rs_offset; + ch = &arg_ary->wc_array[chunk_no].wc_target; + write_len = min(xfer_len, ch->rs_length); + + + /* Prepare the reply chunk given the length actually + * written */ + rs_offset = get_unaligned(&(ch->rs_offset)); + svc_rdma_xdr_encode_array_chunk(res_ary, chunk_no, + ch->rs_handle,rs_offset, + write_len); + chunk_off = 0; + while (write_len) { + int this_write; + + this_write = min(write_len, max_write); + ret = send_write(xprt, rqstp, + ch->rs_handle, + rs_offset + chunk_off, + xdr_off, + this_write, + sge, + sge_count); + if (ret) { + dprintk("svcrdma: RDMA_WRITE failed, ret=%d\n", ret); + return -EIO; + } + chunk_off += this_write; + xdr_off += this_write; + xfer_len -= this_write; + write_len -= this_write; + } + } + /* Update the req with the number of chunks actually used */ + svc_rdma_xdr_encode_reply_array(res_ary, chunk_no); + + return rqstp->rq_res.len; +} + +/* This function prepares the portion of the RPCRDMA message to be + * sent in the RDMA_SEND. This function is called after data sent via + * RDMA has already been transmitted. There are three cases: + * - The RPCRDMA header, RPC header, and payload are all sent in a + * single RDMA_SEND. This is the "inline" case. + * - The RPCRDMA header and some portion of the RPC header and data + * are sent via this RDMA_SEND and another portion of the data is + * sent via RDMA. + * - The RPCRDMA header [NOMSG] is sent in this RDMA_SEND and the RPC + * header and data are all transmitted via RDMA. + * In all three cases, this function prepares the RPCRDMA header in + * sge[0], the 'type' parameter indicates the type to place in the + * RPCRDMA header, and the 'byte_count' field indicates how much of + * the XDR to include in this RDMA_SEND. + */ +static int send_reply(struct svcxprt_rdma *rdma, + struct svc_rqst *rqstp, + struct page *page, + struct rpcrdma_msg *rdma_resp, + struct svc_rdma_op_ctxt *ctxt, + int sge_count, + int byte_count) +{ + struct ib_send_wr send_wr; + int sge_no; + int sge_bytes; + int page_no; + int ret; + + /* Prepare the context */ + ctxt->pages[0] = page; + ctxt->count = 1; + + /* Prepare the SGE for the RPCRDMA Header */ + ctxt->sge[0].addr = + ib_dma_map_page(rdma->sc_cm_id->device, + page, 0, PAGE_SIZE, DMA_TO_DEVICE); + ctxt->direction = DMA_TO_DEVICE; + ctxt->sge[0].length = svc_rdma_xdr_get_reply_hdr_len(rdma_resp); + ctxt->sge[0].lkey = rdma->sc_phys_mr->lkey; + + /* Determine how many of our SGE are to be transmitted */ + for (sge_no = 1; byte_count && sge_no < sge_count; sge_no++) { + sge_bytes = min((size_t)ctxt->sge[sge_no].length, (size_t)byte_count); + byte_count -= sge_bytes; + } + BUG_ON(byte_count != 0); + + /* Save all respages in the ctxt and remove them from the + * respages array. They are our pages until the I/O + * completes. + */ + for (page_no=0; page_no < rqstp->rq_resused; page_no++) { + ctxt->pages[page_no+1] = rqstp->rq_respages[page_no]; + ctxt->count ++; + rqstp->rq_respages[page_no] = NULL; + } + + BUG_ON(sge_no > rdma->sc_max_sge); + memset(&send_wr, 0, sizeof send_wr); + ctxt->wr_op = IB_WR_SEND; + send_wr.wr_id = (unsigned long)ctxt; + send_wr.sg_list = ctxt->sge; + send_wr.num_sge = sge_no; + send_wr.opcode = IB_WR_SEND; + send_wr.send_flags = IB_SEND_SIGNALED; + + ret = svc_rdma_send(rdma, &send_wr); + if (ret) + svc_rdma_put_context(ctxt,1); + + return ret; +} + +int svc_rdma_sendto(struct svc_rqst *rqstp) +{ + struct svc_sock *xprt = rqstp->rq_sock; + struct svcxprt_rdma *rdma = (struct svcxprt_rdma *)xprt; + struct rpcrdma_msg *rdma_argp; + struct rpcrdma_msg *rdma_resp; + struct rpcrdma_write_array *reply_ary; + enum rpcrdma_proc reply_type; + int ret; + int inline_bytes; + struct ib_sge *sge; + int sge_count = 0; + struct page *res_page; + struct svc_rdma_op_ctxt *ctxt; + + dprintk("svcrdma: sending response for rqstp=%p\n", rqstp); + + /* Return the receive WR to the RQ. Any error posting to the + * RQ will surface on the SQ post below + */ + (void)svc_rdma_post_recv(rdma); + + /* Get the RDMA request header. */ + rdma_argp = page_address(rqstp->rq_pages[0]); + + /* Build an SGE for the XDR */ + ctxt = svc_rdma_get_context(rdma); + ctxt->direction = DMA_TO_DEVICE; + sge = xdr_to_sge(rdma, &rqstp->rq_res, ctxt->sge, &sge_count); + + inline_bytes = rqstp->rq_res.len; + + /* Create the RDMA response header */ + res_page = svc_rdma_get_page(); + rdma_resp = page_address(res_page); + reply_ary=svc_rdma_get_reply_array(rdma_argp); + if (reply_ary) + reply_type = RDMA_NOMSG; + else + reply_type = RDMA_MSG; + svc_rdma_xdr_encode_reply_header(rdma, rdma_argp, + rdma_resp, reply_type); + + /* Send any write-chunk data and build resp write-list */ + ret = send_write_chunks(rdma, rdma_argp, rdma_resp, + rqstp, sge, sge_count); + if (ret < 0) { + printk(KERN_ERR "svcrdma: failed to send write chunks, rc=%d\n", ret); + goto error; + } + inline_bytes -= ret; + + /* Send any reply-list data and update resp reply-list */ + ret = send_reply_chunks(rdma, rdma_argp, rdma_resp, + rqstp, sge, sge_count); + if (ret < 0) { + printk(KERN_ERR "svcrdma: failed to send reply chunks, rc=%d\n", ret); + goto error; + } + inline_bytes -= ret; + + ret = send_reply(rdma, rqstp, res_page, rdma_resp, ctxt, sge_count, + inline_bytes); + dprintk("svcrdma: send_reply returns %d\n", ret); + return ret; + error: + svc_rdma_put_context(ctxt, 0); + put_page(res_page); + return ret; +} + From tom at opengridcomputing.com Mon Aug 20 12:01:41 2007 From: tom at opengridcomputing.com (Tom Tucker) Date: Mon, 20 Aug 2007 14:01:41 -0500 Subject: [ofa-general] [RFC, PATCH 08/10] rdma: ONCRPC RDMA protocol marshalling In-Reply-To: <20070820185849.18615.23667.stgit@dell3.ogc.int> References: <20070820185849.18615.23667.stgit@dell3.ogc.int> Message-ID: <20070820190141.18615.91355.stgit@dell3.ogc.int> This logic parses the ONCRDMA protocol headers that precede the actual RPC header. It is placed in a separate file to keep all protocol aware code in a single place. Signed-off-by: Tom Tucker --- net/sunrpc/svc_rdma_marshal.c | 424 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 424 insertions(+), 0 deletions(-) diff --git a/net/sunrpc/svc_rdma_marshal.c b/net/sunrpc/svc_rdma_marshal.c new file mode 100644 index 0000000..feebabd --- /dev/null +++ b/net/sunrpc/svc_rdma_marshal.c @@ -0,0 +1,424 @@ +/* + * Copyright (c) 2005-2006 Network Appliance, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the BSD-type + * license below: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * Neither the name of the Network Appliance, Inc. nor the names of + * its contributors may be used to endorse or promote products + * derived from this software without specific prior written + * permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Author: Tom Tucker + */ + +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define RPCDBG_FACILITY RPCDBG_SVCXPRT + +/* + * Decodes a read chunk list. The expected format is as follows: + * descrim : xdr_one + * position : u32 offset into XDR stream + * handle : u32 RKEY + * . . . + * end-of-list: xdr_zero + */ +static u32 *decode_read_list(u32 *va, u32 *vaend) +{ + struct rpcrdma_read_chunk *ch = (struct rpcrdma_read_chunk*)va; + + while (ch->rc_discrim != xdr_zero) { + u64 ch_offset; + + if (((unsigned long)ch + sizeof(struct rpcrdma_read_chunk)) > + (unsigned long)vaend) { + dprintk("svcrdma: vaend=%p, ch=%p\n", vaend, ch); + return NULL; + } + + ch->rc_discrim = ntohl(ch->rc_discrim); + ch->rc_position = ntohl(ch->rc_position); + ch->rc_target.rs_handle = ntohl(ch->rc_target.rs_handle); + ch->rc_target.rs_length = ntohl(ch->rc_target.rs_length); + va = (u32*)&ch->rc_target.rs_offset; + xdr_decode_hyper(va, &ch_offset); + put_unaligned(ch_offset, (u64*)va); + ch++; + } + return (u32*)&ch->rc_position; +} + +/* + * Determine number of chunks and total bytes in chunk list. The chunk + * list has already been verified to fit within the RPCRDMA header. + */ +void svc_rdma_rcl_chunk_counts(struct rpcrdma_read_chunk *ch, + int *ch_count, int *byte_count) +{ + /* compute the number of bytes represented by read chunks */ + *byte_count = 0; + *ch_count = 0; + for (; ch->rc_discrim != 0; ch++) { + *byte_count = *byte_count + ch->rc_target.rs_length; + *ch_count = *ch_count + 1; + } +} + +/* + * Decodes a write chunk list. The expected format is as follows: + * descrim : xdr_one + * nchunks : + * handle : u32 RKEY ---+ + * length : u32 | + * offset : remove va + + * . . . | + * ---+ + */ +static u32 *decode_write_list(u32 *va, u32 *vaend) +{ + int ch_no; + struct rpcrdma_write_array *ary = + (struct rpcrdma_write_array*)va; + + /* Check for not write-array */ + if (ary->wc_discrim == xdr_zero) + return (u32*)&ary->wc_nchunks; + + if ((unsigned long)ary + sizeof(struct rpcrdma_write_array) > + (unsigned long)vaend) { + dprintk("svcrdma: ary=%p, vaend=%p\n", ary, vaend); + return NULL; + } + ary->wc_discrim = ntohl(ary->wc_discrim); + ary->wc_nchunks = ntohl(ary->wc_nchunks); + if (((unsigned long)&ary->wc_array[0] + + (sizeof(struct rpcrdma_write_chunk) * ary->wc_nchunks)) > + (unsigned long)vaend) { + dprintk("svcrdma: ary=%p, wc_nchunks=%d, vaend=%p\n", + ary, ary->wc_nchunks, vaend); + return NULL; + } + for (ch_no = 0; ch_no < ary->wc_nchunks; ch_no++) { + u64 ch_offset; + + ary->wc_array[ch_no].wc_target.rs_handle = + ntohl(ary->wc_array[ch_no].wc_target.rs_handle); + ary->wc_array[ch_no].wc_target.rs_length = + ntohl(ary->wc_array[ch_no].wc_target.rs_length); + va = (u32*)&ary->wc_array[ch_no].wc_target.rs_offset; + xdr_decode_hyper(va, &ch_offset); + put_unaligned(ch_offset, (u64*)va); + } + + /* + * rs_length is the 2nd 4B field in wc_target and taking its + * address skips the list terminator + */ + return (u32*)&ary->wc_array[ch_no].wc_target.rs_length; +} + +static u32 *decode_reply_array(u32 *va, u32 *vaend) +{ + int ch_no; + struct rpcrdma_write_array *ary = + (struct rpcrdma_write_array*)va; + + /* Check for no reply-array */ + if (ary->wc_discrim == xdr_zero) + return (u32*)&ary->wc_nchunks; + + if ((unsigned long)ary + sizeof(struct rpcrdma_write_array) > + (unsigned long)vaend) { + dprintk("svcrdma: ary=%p, vaend=%p\n", ary, vaend); + return NULL; + } + ary->wc_discrim = ntohl(ary->wc_discrim); + ary->wc_nchunks = ntohl(ary->wc_nchunks); + if (((unsigned long)&ary->wc_array[0] + + (sizeof(struct rpcrdma_write_chunk) * ary->wc_nchunks)) > + (unsigned long)vaend) { + dprintk("svcrdma: ary=%p, wc_nchunks=%d, vaend=%p\n", + ary, ary->wc_nchunks, vaend); + return NULL; + } + for (ch_no = 0; ch_no < ary->wc_nchunks; ch_no++) { + u64 ch_offset; + + ary->wc_array[ch_no].wc_target.rs_handle = + ntohl(ary->wc_array[ch_no].wc_target.rs_handle); + ary->wc_array[ch_no].wc_target.rs_length = + ntohl(ary->wc_array[ch_no].wc_target.rs_length); + va = (u32*)&ary->wc_array[ch_no].wc_target.rs_offset; + xdr_decode_hyper(va, &ch_offset); + put_unaligned(ch_offset, (u64*)va); + } + + return (u32*)&ary->wc_array[ch_no]; +} + +int svc_rdma_xdr_decode_req(struct rpcrdma_msg **rdma_req, struct svc_rqst *rqstp) +{ + struct rpcrdma_msg *rmsgp = NULL; + u32 *va; + u32 *vaend; + u32 hdr_len; + + rmsgp = (struct rpcrdma_msg*)rqstp->rq_arg.head[0].iov_base; + + /* Verify that there's enough bytes for header + something */ + if (rqstp->rq_arg.len <= RPCRDMA_HDRLEN_MIN) { + dprintk("svcrdma: header too short = %d\n", + rqstp->rq_arg.len); + return -EINVAL; + } + + /* Decode the header */ + rmsgp->rm_xid = ntohl(rmsgp->rm_xid); + rmsgp->rm_vers = ntohl(rmsgp->rm_vers); + rmsgp->rm_credit = ntohl(rmsgp->rm_credit); + rmsgp->rm_type = ntohl(rmsgp->rm_type); + + if (rmsgp->rm_vers != RPCRDMA_VERSION) + return -ENOSYS; + + /* Pull in the extra for the padded case and bump our pointer */ + if (rmsgp->rm_type == RDMA_MSGP) { + int hdrlen; + rmsgp->rm_body.rm_padded.rm_align = + ntohl(rmsgp->rm_body.rm_padded.rm_align); + rmsgp->rm_body.rm_padded.rm_thresh = + ntohl(rmsgp->rm_body.rm_padded.rm_thresh); + + va = &rmsgp->rm_body.rm_padded.rm_pempty[4]; + rqstp->rq_arg.head[0].iov_base = va; + hdrlen = (u32)((unsigned long)va - (unsigned long)rmsgp); + rqstp->rq_arg.head[0].iov_len -= hdrlen; + if (hdrlen > rqstp->rq_arg.len) + return -EINVAL; + return hdrlen; + } + + /* The chunk list may contain either a read chunk list or a write + * chunk list and a reply chunk list. + */ + va = &rmsgp->rm_body.rm_chunks[0]; + vaend = (u32*)((unsigned long)rmsgp + rqstp->rq_arg.len); + va = decode_read_list(va,vaend); + if (!va) + return -EINVAL; + va = decode_write_list(va, vaend); + if (!va) + return -EINVAL; + va = decode_reply_array(va,vaend); + if (!va) + return -EINVAL; + + rqstp->rq_arg.head[0].iov_base = va; + hdr_len = (unsigned long)va - (unsigned long)rmsgp; + rqstp->rq_arg.head[0].iov_len -= hdr_len; + + *rdma_req = rmsgp; + return hdr_len; +} + +int svc_rdma_xdr_decode_deferred_req(struct svc_rqst *rqstp) +{ + struct rpcrdma_msg *rmsgp = NULL; + struct rpcrdma_read_chunk *ch; + struct rpcrdma_write_array *ary; + u32 *va; + u32 hdrlen; + + dprintk("svcrdma: processing deferred RDMA header on rqstp=%p\n", + rqstp); + rmsgp = (struct rpcrdma_msg*)rqstp->rq_arg.head[0].iov_base; + + /* Pull in the extra for the padded case and bump our pointer */ + if (rmsgp->rm_type == RDMA_MSGP) { + va = &rmsgp->rm_body.rm_padded.rm_pempty[4]; + rqstp->rq_arg.head[0].iov_base = va; + hdrlen = (u32)((unsigned long)va - (unsigned long)rmsgp); + rqstp->rq_arg.head[0].iov_len -= hdrlen; + return hdrlen; + } + + /* + * Skip all chunks to find RPC msg. These were previously processed + */ + va = &rmsgp->rm_body.rm_chunks[0]; + + /* Skip read-list */ + for (ch = (struct rpcrdma_read_chunk*)va; + ch->rc_discrim != xdr_zero; ch++); + va = (u32*)&ch->rc_position; + + /* Skip write-list */ + ary = (struct rpcrdma_write_array*)va; + if (ary->wc_discrim == xdr_zero) + va = (u32*)&ary->wc_nchunks; + else + /* + * rs_length is the 2nd 4B field in wc_target and taking its + * address skips the list terminator + */ + va = (u32*)&ary->wc_array[ary->wc_nchunks].wc_target.rs_length; + + /* Skip reply-array */ + ary = (struct rpcrdma_write_array*)va; + if (ary->wc_discrim == xdr_zero) + va = (u32*)&ary->wc_nchunks; + else + va = (u32*)&ary->wc_array[ary->wc_nchunks]; + + rqstp->rq_arg.head[0].iov_base = va; + hdrlen = (unsigned long)va - (unsigned long)rmsgp; + rqstp->rq_arg.head[0].iov_len -= hdrlen; + + return hdrlen; +} + +int svc_rdma_xdr_encode_error(struct svcxprt_rdma *xprt, struct rpcrdma_msg *rmsgp, + enum rpcrdma_errcode err, u32 *va) +{ + u32* startp = va; + + *va++ = htonl(rmsgp->rm_xid); + *va++ = htonl(rmsgp->rm_vers); + *va++ = htonl(xprt->sc_max_requests); + *va++ = htonl(RDMA_ERROR); + *va++ = htonl(err); + if (err == ERR_VERS) { + *va++ = htonl(RPCRDMA_VERSION); + *va++ = htonl(RPCRDMA_VERSION); + } + + return (int)((unsigned long)va - (unsigned long)startp); +} + +int svc_rdma_xdr_get_reply_hdr_len(struct rpcrdma_msg *rmsgp) +{ + struct rpcrdma_write_array *wr_ary; + + /* There is no read-list in a reply */ + + /* skip write list */ + wr_ary = (struct rpcrdma_write_array *) + &rmsgp->rm_body.rm_chunks[1]; + if (wr_ary->wc_discrim) + wr_ary = (struct rpcrdma_write_array *) + &wr_ary->wc_array[ntohl(wr_ary->wc_nchunks)]. + wc_target.rs_length; + else + wr_ary = (struct rpcrdma_write_array *) + &wr_ary->wc_nchunks; + + /* skip reply array */ + if (wr_ary->wc_discrim) + wr_ary = (struct rpcrdma_write_array *) + &wr_ary->wc_array[ntohl(wr_ary->wc_nchunks)]; + else + wr_ary = (struct rpcrdma_write_array *) + &wr_ary->wc_nchunks; + + return (unsigned long) wr_ary - (unsigned long) rmsgp; +} + +void svc_rdma_xdr_encode_write_list(struct rpcrdma_msg *rmsgp, int chunks) +{ + struct rpcrdma_write_array *ary; + + /* no read-list */ + rmsgp->rm_body.rm_chunks[0] = xdr_zero; + + /* write-array discrim */ + ary = (struct rpcrdma_write_array *) + &rmsgp->rm_body.rm_chunks[1]; + ary->wc_discrim = xdr_one; + ary->wc_nchunks = htonl(chunks); + + /* write-list terminator */ + ary->wc_array[chunks].wc_target.rs_handle = xdr_zero; + + /* reply-array discriminator */ + ary->wc_array[chunks].wc_target.rs_length = xdr_zero; +} + +void svc_rdma_xdr_encode_reply_array(struct rpcrdma_write_array *ary, + int chunks) +{ + ary->wc_discrim = xdr_one; + ary->wc_nchunks = htonl(chunks); +} + +void svc_rdma_xdr_encode_array_chunk(struct rpcrdma_write_array *ary, int chunk_no, + u32 rs_handle, u64 rs_offset, u32 write_len) +{ + struct rpcrdma_segment *seg = &ary->wc_array[chunk_no].wc_target; + seg->rs_handle = htonl(rs_handle); + seg->rs_length = htonl(write_len); + xdr_encode_hyper((u32*) &seg->rs_offset, rs_offset); +} + +void svc_rdma_xdr_encode_reply_header(struct svcxprt_rdma *xprt, + struct rpcrdma_msg *rdma_argp, + struct rpcrdma_msg *rdma_resp, + enum rpcrdma_proc rdma_type) +{ + rdma_resp->rm_xid = htonl(rdma_argp->rm_xid); + rdma_resp->rm_vers = htonl(rdma_argp->rm_vers); + rdma_resp->rm_credit = htonl(xprt->sc_max_requests); + rdma_resp->rm_type = htonl(rdma_type); + + /* Encode chunks lists */ + rdma_resp->rm_body.rm_chunks[0] = xdr_zero; + rdma_resp->rm_body.rm_chunks[1] = xdr_zero; + rdma_resp->rm_body.rm_chunks[2] = xdr_zero; +} + From felix at chelsio.com Mon Aug 20 12:02:25 2007 From: felix at chelsio.com (Felix Marti) Date: Mon, 20 Aug 2007 12:02:25 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <20070819.174017.77241227.davem@davemloft.net><8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com><20070820094317.GA14817@2ka.mipt.ru><8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> Message-ID: <8A71B368A89016469F72CD08050AD334018E2149@maui.asicdesigners.com> > -----Original Message----- > From: ak at suse.de [mailto:ak at suse.de] On Behalf Of Andi Kleen > Sent: Monday, August 20, 2007 11:11 AM > To: Felix Marti > Cc: Evgeniy Polyakov; jeff at garzik.org; netdev at vger.kernel.org; > rdreier at cisco.com; linux-kernel at vger.kernel.org; > general at lists.openfabrics.org; David Miller > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > "Felix Marti" writes: > > > What I was referring to is that TSO(/LRO) have their own > > issues, some eluded to by Roland and me. In fact, customers working > on > > the LSR couldn't use TSO due to the burstiness it introduces > > That was in old kernels where TSO didn't honor the initial cwnd > correctly, > right? I assume it's long fixed. > > If not please clarify what the problem was. The problem is that is that Ethernet is about the only technology that discloses 'useable' throughput while everybody else talks about signaling rates ;) - OC-192 can carry about 9.128Gbps (or close to that number) and hence 10Gbps Ethernet was overwhelming the OC-192 network. The customer needed to schedule packets at about 98% of OC-192 throughput in order to avoid packet drop. The scheduling needed to be done on a per packet basis and not per 'burst of packets' basis in order to avoid packet drop. > > > have a look at graphics. > > Graphics used to be done by the host CPU and now we have dedicated > > graphics adapters that do a much better job... > > Is your off load device as programable as a modern GPU? It has a lot of knobs to turn. > > > farfetched that offload devices can do a better job at a data-flow > > problem? > > One big difference is that there is no potentially adverse and > always varying internet between the graphics card and your monitor. These graphic adapters provide a wealth of features that you can take advantage of to bring these amazing graphics to life. General purpose CPUs cannot keep up. Chelsio offload devices do the same thing in the realm of networking. - Will there be things you can't do, probably yes, but as I said, there are lots of knobs to turn (and the latest and greatest feature that gets hyped up might not always be the best thing since sliced bread anyway; what happened to BIC love? ;) > > -Andi From rick.jones2 at hp.com Mon Aug 20 12:16:41 2007 From: rick.jones2 at hp.com (Rick Jones) Date: Mon, 20 Aug 2007 12:16:41 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: References: <8A71B368A89016469F72CD08050AD334018E20BC@maui.asicdesigners.com> <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070819.180540.74750322.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20C1@maui.asicdesigners.com> Message-ID: <46C9E899.9020609@hp.com> Andi Kleen wrote: > TSO is beneficial for the software again. The linux code currently > takes several locks and does quite a few function calls for each > packet and using larger packets lowers this overhead. At least with > 10GbE saving CPU cycles is still quite important. Some quick netperf TCP_RR tests between a pair of dual-core rx6600's running 2.6.23-rc3. the NICs are dual-core e1000's connected back-to-back with the interrupt throttle disabled. I like using TCP_RR to tickle path-length questions because it rarely runs into bandwidth limitations regardless of the link-type. First, with TSO enabled on both sides, then with it disabled, netperf/netserver bound to the same CPU as takes interrupts, which is the "best" place to be for a TCP_RR test (although not always for a TCP_STREAM test...): :~# netperf -T 1 -t TCP_RR -H 192.168.2.105 -I 99,1 -c -C TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.105 (192.168.2.105) port 0 AF_INET : +/-0.5% @ 99% conf. : first burst 0 : cpu bind !!! WARNING !!! Desired confidence was not achieved within the specified iterations. !!! This implies that there was variability in the test environment that !!! must be investigated before going further. !!! Confidence intervals: Throughput : 0.3% !!! Local CPU util : 39.3% !!! Remote CPU util : 40.6% Local /Remote Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem Send Recv Size Size Time Rate local remote local remote bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr 16384 87380 1 1 10.01 18611.32 20.96 22.35 22.522 24.017 16384 87380 :~# ethtool -K eth2 tso off e1000: eth2: e1000_set_tso: TSO is Disabled :~# netperf -T 1 -t TCP_RR -H 192.168.2.105 -I 99,1 -c -C TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.105 (192.168.2.105) port 0 AF_INET : +/-0.5% @ 99% conf. : first burst 0 : cpu bind !!! WARNING !!! Desired confidence was not achieved within the specified iterations. !!! This implies that there was variability in the test environment that !!! must be investigated before going further. !!! Confidence intervals: Throughput : 0.4% !!! Local CPU util : 21.0% !!! Remote CPU util : 25.2% Local /Remote Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem Send Recv Size Size Time Rate local remote local remote bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr 16384 87380 1 1 10.01 19812.51 17.81 17.19 17.983 17.358 16384 87380 While the confidence intervals for CPU util weren't hit, I suspect the differences in service demand were still real. On throughput we are talking about +/- 0.2%, for CPU util we are talking about +/- 20% (percent not percentage points) in the first test and 12.5% in the second. So, in broad handwaving terms, TSO increased the per-transaction service demand by something along the lines of (23.27 - 17.67)/17.67 or ~30% and the transaction rate decreased by ~6%. rick jones bitrate blindless is a constant concern From vu at mellanox.com Mon Aug 20 12:43:48 2007 From: vu at mellanox.com (Vu Pham) Date: Mon, 20 Aug 2007 12:43:48 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: <1186200658.6031.44.camel@gentoo-linux.localdomain> References: <1186200658.6031.44.camel@gentoo-linux.localdomain> Message-ID: <46C9EEF4.6000407@mellanox.com> Thanks. I checked in with a few other changes/bug fixes. Please test and report any problem > SCST redefined the scst_tgt_template and it no longer contains the > tm_sync_reply field. Also an extra arg is required for scst_register. > Consequently, the build fails. > > I searched through past SCST versions and docs but could not find what > this was for and it seems to be set to zero (not used?) in this struct > anyhow. > > > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From vu at mellanox.com Mon Aug 20 12:44:32 2007 From: vu at mellanox.com (Vu Pham) Date: Mon, 20 Aug 2007 12:44:32 -0700 Subject: [ofa-general] [PATCH] srpt_inc branch README update In-Reply-To: <1186200777.6031.47.camel@gentoo-linux.localdomain> References: <1186200777.6031.47.camel@gentoo-linux.localdomain> Message-ID: <46C9EF20.7070508@mellanox.com> Thanks. I checked in with few other changes/fixes > Bring the srpt_inc branch README up to date with online version ( > http://www.mellanox.com/pdf/products/software/Gen2_SRPT_README.txt ) > > > ------------------------------------------------------------------------ > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From tgraf at suug.ch Mon Aug 20 13:18:08 2007 From: tgraf at suug.ch (Thomas Graf) Date: Mon, 20 Aug 2007 22:18:08 +0200 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E2149@maui.asicdesigners.com> References: <8A71B368A89016469F72CD08050AD334018E2149@maui.asicdesigners.com> Message-ID: <20070820201808.GM32236@postel.suug.ch> * Felix Marti 2007-08-20 12:02 > These graphic adapters provide a wealth of features that you can take > advantage of to bring these amazing graphics to life. General purpose > CPUs cannot keep up. Chelsio offload devices do the same thing in the > realm of networking. - Will there be things you can't do, probably yes, > but as I said, there are lots of knobs to turn (and the latest and > greatest feature that gets hyped up might not always be the best thing > since sliced bread anyway; what happened to BIC love? ;) GPUs have almost no influence on system security, the network stack OTOH is probably the most vulnerable part of an operating system. Even if all vendors would implement all the features collected over the last years properly which seems unlikely. Having such an essential and critical part depend on the vendor of my network card without being able to even verify it properly is truly frightening. From andi at firstfloor.org Mon Aug 20 13:33:55 2007 From: andi at firstfloor.org (Andi Kleen) Date: Mon, 20 Aug 2007 22:33:55 +0200 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <20070820201808.GM32236@postel.suug.ch> References: <8A71B368A89016469F72CD08050AD334018E2149@maui.asicdesigners.com> <20070820201808.GM32236@postel.suug.ch> Message-ID: <20070820203355.GA21244@one.firstfloor.org> > GPUs have almost no influence on system security, Unless you use direct rendering from user space. -Andi From patrick at myri.com Mon Aug 20 13:33:56 2007 From: patrick at myri.com (Patrick Geoffray) Date: Mon, 20 Aug 2007 16:33:56 -0400 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. In-Reply-To: <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> References: <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070820094317.GA14817@2ka.mipt.ru> <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> Message-ID: <46C9FAB4.5020609@myri.com> Felix Marti wrote: > Yes, the app will take the cache hits when accessing the data. However, > the fact remains that if there is a copy in the receive path, you > require and additional 3x memory BW (which is very significant at these > high rates and most likely the bottleneck for most current systems)... > and somebody always has to take the cache miss be it the copy_to_user or > the app. The cache miss is going to cost you half the memory bandwidth of a full copy. If the data is already in cache, then the copy is cheaper. However, removing the copy removes the kernel from the picture on the receive side, so you lose demultiplexing, asynchronism, security, accounting, flow-control, swapping, etc. If it's ok with you to not use the kernel stack, then why expect to fit in the existing infrastructure anyway ? > Yes, RDMA support is there... but we could make it better and easier to What do you need from the kernel for RDMA support beyond HW drivers ? A fast way to pin and translate user memory (ie registration). That is pretty much the sandbox that David referred to. Eventually, it would be useful to be able to track the VM space to implement a registration cache instead of using ugly hacks in user-space to hijack malloc, but this is completely independent from the net stack. > use. We have a problem today with port sharing and there was a proposal The port spaces are either totally separate and there is no issue, or completely identical and you should then run your connection manager in user-space or fix your middlewares. > and not for technical reasons. I believe this email threads shows in > detail how RDMA (a network technology) is treated as bastard child by > the network folks, well at least by one of them. I don't think it's fair. This thread actually show how pushy some RDMA folks are about not acknowledging that the current infrastructure is here for a reason, and about mistaking zero-copy and RDMA. This is a similar argument than the TOE discussion, and it was definitively a good decision to not mess up the Linux stack with TOEs. Patrick From rdreier at cisco.com Mon Aug 20 13:56:57 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 20 Aug 2007 13:56:57 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 1] In-Reply-To: <46C38C3A.4020704@linux.vnet.ibm.com> (Pradeep Satyanarayana's message of "Wed, 15 Aug 2007 16:28:58 -0700") References: <46C38C3A.4020704@linux.vnet.ibm.com> Message-ID: I can't find that much to object to here... the static table data structure is pretty ugly but oh well. I think with a few more fixes this could be mergable. Anyone else have an opinion? Anyway: > +extern int max_rc_qp ; > + index = id & NOSRQ_INDEX_MASK ; > + index = id & NOSRQ_INDEX_MASK ; > + index = (wc->wr_id & ~IPOIB_CM_OP_RECV) & NOSRQ_INDEX_MASK ; since other fixes are needed please get rid of the spaces before semicolons. > +module_param_named(nosrq_max_rc_qp, max_rc_qp, int, 0644); Seems like making this writable is a bad idea -- I can't see how it could possibly work if this changes after the driver starts. > +static void timer_check_srq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) > +{ > + unsigned long flags; > + > + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { > + spin_lock_irqsave(&priv->lock, flags); > + p->jiffies = jiffies; > + /* Move this entry to list head, but do > + * not re-add it if it has been removed. > + */ > + if (p->state == IPOIB_CM_RX_LIVE) > + list_move(&p->list, &priv->cm.passive_ids); > + spin_unlock_irqrestore(&priv->lock, flags); > + } > +} > + > +static void timer_check_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) > +{ > + unsigned long flags; > + > + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { > + spin_lock_irqsave(&priv->lock, flags); > + p->jiffies = jiffies; > + /* Move this entry to list head, but do > + * not re-add it if it has been removed. */ > + if (!list_empty(&p->list)) > + list_move(&p->list, &priv->cm.passive_ids); > + spin_unlock_irqrestore(&priv->lock, flags); > + } > +} I need to read more closely to understand this, but maybe you could give a hint why you need two nearly identical copies of the same function (which does not appear to be in the fast path)? - R. From rdreier at cisco.com Mon Aug 20 13:58:31 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 20 Aug 2007 13:58:31 -0700 Subject: [ofa-general] On vacation starting tomorrow... Message-ID: I'll be away starting Tuesday until next Sunday (the 21st through the 26th) so please don't expect me to do anything during that time... From rdreier at cisco.com Mon Aug 20 14:32:22 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 20 Aug 2007 14:32:22 -0700 Subject: [ofa-general] ib_umad method mask problems on big-endian 64-bit archs Message-ID: It turns out there is a problem with how user_mad copies the method_mask from the userspace agent registration request, which has the member __u32 method_mask[4]; into the kernel agent registration request, which has the member DECLARE_BITMAP(method_mask, IB_MGMT_MAX_METHODS); which defines method_mask to be an array of longs. has a good pictorial representation of the problem: * The bitop functions are defined to work on unsigned longs, so for a * ppc64 system the bits end up numbered: * |63..............0|127............64|191...........128|255...........196| * and on ppc32: * |31.....0|63....31|95....64|127...96|159..128|191..160|223..192|255..224| ie if userspace fills in the method_mask in the natural way then things end up being wrong when the kernel memcpys the userspace request into the kernel registration request. Userspace could try and work around this and match the kernel's layout, but unfortunately 32-bit userspace can't do that in any sane way: on a 32-bit kernel there is one layout and on a 64-bit kernel there is another and so userspace is stuck. The patch below fixes the kernel to fix the bit layour when copying the method_mask. I'd like to merge this, but unfortunately there is the complication that opensm/libvendor/osm_vendor_ibumad.c has this: static int set_bit(int nr, void *method_mask) { int mask, retval; long *addr = method_mask; addr += nr >> 5; mask = 1 << (nr & 0x1f); retval = (mask & *addr) != 0; *addr |= mask; return retval; } which is actually wrong in general (if sizeof long is 8, then offsetting by nr>>5 ends up making the subscript twice what it should be), but all defined IB MAD methods are less than 32 so it ends up being OK, and actually works for 64-bit big-endian userspace on a 64-bit kernel. So the patch below would mean that 64-bit osm binaries on ppc64 would break (32-bit osm binaries on powerpc are of course broken with a 64-bit kernel). Still, perhaps the right thing to do is to just fix the opensm code at the same time? Opinions? diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c index d97ded2..c2c58c9 100644 --- a/drivers/infiniband/core/user_mad.c +++ b/drivers/infiniband/core/user_mad.c @@ -596,6 +596,7 @@ static int ib_umad_reg_agent(struct ib_umad_file *file, unsigned long arg) struct ib_mad_agent *agent; int agent_id; int ret; + int i, j; down_write(&file->port->mutex); @@ -625,8 +626,16 @@ found: if (ureq.mgmt_class) { req.mgmt_class = ureq.mgmt_class; req.mgmt_class_version = ureq.mgmt_class_version; - memcpy(req.method_mask, ureq.method_mask, sizeof req.method_mask); - memcpy(req.oui, ureq.oui, sizeof req.oui); + + for (i = 0; i < BITS_TO_LONGS(IB_MGMT_MAX_METHODS); ++i) { + req.method_mask[i] = 0; + for (j = 0; j < sizeof (long) / 4; ++j) + req.method_mask[i] |= + ureq.method_mask[i * sizeof (long) / 4 + + j] << (j * 32); + } + + memcpy(req.oui, ureq.oui, sizeof req.oui); } agent = ib_register_mad_agent(file->port->ib_dev, file->port->port_num, From pradeeps at linux.vnet.ibm.com Mon Aug 20 14:36:41 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Mon, 20 Aug 2007 14:36:41 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 1] In-Reply-To: References: <46C38C3A.4020704@linux.vnet.ibm.com> Message-ID: <46CA0969.4020601@linux.vnet.ibm.com> Roland Dreier wrote: > I can't find that much to object to here... the static table data > structure is pretty ugly but oh well. > > I think with a few more fixes this could be mergable. Anyone else > have an opinion? > > Anyway: > > > +extern int max_rc_qp ; > > + index = id & NOSRQ_INDEX_MASK ; > > + index = id & NOSRQ_INDEX_MASK ; > > + index = (wc->wr_id & ~IPOIB_CM_OP_RECV) & NOSRQ_INDEX_MASK ; > > since other fixes are needed please get rid of the spaces before > semicolons. OK. > > > +module_param_named(nosrq_max_rc_qp, max_rc_qp, int, 0644); > > Seems like making this writable is a bad idea -- I can't see how it > could possibly work if this changes after the driver starts. Yes, it should only be readable. > > > +static void timer_check_srq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) > > +{ > > + unsigned long flags; > > + > > + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { > > + spin_lock_irqsave(&priv->lock, flags); > > + p->jiffies = jiffies; > > + /* Move this entry to list head, but do > > + * not re-add it if it has been removed. > > + */ > > + if (p->state == IPOIB_CM_RX_LIVE) > > + list_move(&p->list, &priv->cm.passive_ids); > > + spin_unlock_irqrestore(&priv->lock, flags); > > + } > > +} > > + > > +static void timer_check_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) > > +{ > > + unsigned long flags; > > + > > + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { > > + spin_lock_irqsave(&priv->lock, flags); > > + p->jiffies = jiffies; > > + /* Move this entry to list head, but do > > + * not re-add it if it has been removed. */ > > + if (!list_empty(&p->list)) > > + list_move(&p->list, &priv->cm.passive_ids); > > + spin_unlock_irqrestore(&priv->lock, flags); > > + } > > +} > > I need to read more closely to understand this, but maybe you could > give a hint why you need two nearly identical copies of the same > function (which does not appear to be in the fast path)? These are in fact invoked by the receive handlers and that is why I had them as two separate functions. Pradeep From sean.hefty at intel.com Mon Aug 20 15:15:09 2007 From: sean.hefty at intel.com (Sean Hefty) Date: Mon, 20 Aug 2007 15:15:09 -0700 Subject: [ofa-general] On vacation starting tomorrow... In-Reply-To: References: Message-ID: <000501c7e377$9218a310$ff0da8c0@amr.corp.intel.com> >I'll be away starting Tuesday until next Sunday (the 21st through the >26th) so please don't expect me to do anything during that time... I too will be on vacation starting tomorrow through next week. - Sean From pradeeps at linux.vnet.ibm.com Mon Aug 20 15:44:19 2007 From: pradeeps at linux.vnet.ibm.com (Pradeep Satyanarayana) Date: Mon, 20 Aug 2007 15:44:19 -0700 Subject: [ofa-general] IPoIB CM (NOSRQ) [PATCH 1] In-Reply-To: References: <46C38C3A.4020704@linux.vnet.ibm.com> Message-ID: <46CA1943.1030309@linux.vnet.ibm.com> Resubmitting the patch after incorporating Roland's comments and also merged in the small event handler patch into this one. Signed-off-by: Pradeep Satyanarayana --- --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-20 17:39:25.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-20 17:49:14.000000000 -0400 @@ -95,11 +95,14 @@ enum { IPOIB_MCAST_FLAG_ATTACHED = 3, }; +#define CM_PACKET_SIZE (ALIGN(IPOIB_CM_MTU, PAGE_SIZE)) #define IPOIB_OP_RECV (1ul << 31) #ifdef CONFIG_INFINIBAND_IPOIB_CM -#define IPOIB_CM_OP_SRQ (1ul << 30) +#define IPOIB_CM_OP_RECV (1ul << 30) + +#define NOSRQ_INDEX_TABLE_SIZE 128 #else -#define IPOIB_CM_OP_SRQ (0) +#define IPOIB_CM_OP_RECV (0) #endif /* structs */ @@ -166,11 +169,14 @@ enum ipoib_cm_state { }; struct ipoib_cm_rx { - struct ib_cm_id *id; - struct ib_qp *qp; - struct list_head list; - struct net_device *dev; - unsigned long jiffies; + struct ib_cm_id *id; + struct ib_qp *qp; + struct ipoib_cm_rx_buf *rx_ring; /* Used by NOSRQ only */ + struct list_head list; + struct net_device *dev; + unsigned long jiffies; + u32 index; /* wr_ids are distinguished by index + * to identify the QP -NOSRQ only */ enum ipoib_cm_state state; }; @@ -215,6 +221,8 @@ struct ipoib_cm_dev_priv { struct ib_wc ibwc[IPOIB_NUM_WC]; struct ib_sge rx_sge[IPOIB_CM_RX_SG]; struct ib_recv_wr rx_wr; + struct ipoib_cm_rx **rx_index_table; /* See ipoib_cm_dev_init() + *for usage of this element */ }; /* @@ -438,6 +446,7 @@ void ipoib_drain_cq(struct net_device *d /* We don't support UC connections at the moment */ #define IPOIB_CM_SUPPORTED(ha) (ha[0] & (IPOIB_FLAGS_RC)) +extern int max_rc_qp; static inline int ipoib_cm_admin_enabled(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-20 17:39:25.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-20 17:51:46.000000000 -0400 @@ -49,6 +49,18 @@ MODULE_PARM_DESC(cm_data_debug_level, #include "ipoib.h" +int max_rc_qp = NOSRQ_INDEX_TABLE_SIZE; +static int max_recv_buf = 1024; /* Default is 1024 MB */ + +module_param_named(nosrq_max_rc_qp, max_rc_qp, int, 0444); +MODULE_PARM_DESC(nosrq_max_rc_qp, "Max number of NOSRQ RC QPs supported; must be a power of 2"); + +module_param_named(max_receive_buffer, max_recv_buf, int, 0644); +MODULE_PARM_DESC(max_receive_buffer, "Max Receive Buffer Size in MB"); + +static atomic_t current_rc_qp = ATOMIC_INIT(0); /* Active number of RC QPs for NOSRQ */ + +#define NOSRQ_INDEX_MASK (max_rc_qp -1) #define IPOIB_CM_IETF_ID 0x1000000000000000ULL #define IPOIB_CM_RX_UPDATE_TIME (256 * HZ) @@ -81,20 +93,21 @@ static void ipoib_cm_dma_unmap_rx(struct ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); } -static int ipoib_cm_post_receive(struct net_device *dev, int id) +static int post_receive_srq(struct net_device *dev, u64 id) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ib_recv_wr *bad_wr; int i, ret; - priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_SRQ; + priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_RECV; for (i = 0; i < IPOIB_CM_RX_SG; ++i) priv->cm.rx_sge[i].addr = priv->cm.srq_ring[id].mapping[i]; ret = ib_post_srq_recv(priv->cm.srq, &priv->cm.rx_wr, &bad_wr); if (unlikely(ret)) { - ipoib_warn(priv, "post srq failed for buf %d (%d)\n", id, ret); + ipoib_warn(priv, "post srq failed for buf %lld (%d)\n", + (unsigned long long)id, ret); ipoib_cm_dma_unmap_rx(priv, IPOIB_CM_RX_SG - 1, priv->cm.srq_ring[id].mapping); dev_kfree_skb_any(priv->cm.srq_ring[id].skb); @@ -104,12 +117,47 @@ static int ipoib_cm_post_receive(struct return ret; } -static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev, int id, int frags, +static int post_receive_nosrq(struct net_device *dev, u64 id) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct ib_recv_wr *bad_wr; + int i, ret; + u32 index; + u32 wr_id; + struct ipoib_cm_rx *rx_ptr; + + index = id & NOSRQ_INDEX_MASK; + wr_id = id >> 32; + + rx_ptr = priv->cm.rx_index_table[index]; + + priv->cm.rx_wr.wr_id = id | IPOIB_CM_OP_RECV; + + for (i = 0; i < IPOIB_CM_RX_SG; ++i) + priv->cm.rx_sge[i].addr = rx_ptr->rx_ring[wr_id].mapping[i]; + + ret = ib_post_recv(rx_ptr->qp, &priv->cm.rx_wr, &bad_wr); + if (unlikely(ret)) { + ipoib_warn(priv, "post recv failed for buf %d (%d)\n", + wr_id, ret); + ipoib_cm_dma_unmap_rx(priv, IPOIB_CM_RX_SG - 1, + rx_ptr->rx_ring[wr_id].mapping); + dev_kfree_skb_any(rx_ptr->rx_ring[wr_id].skb); + rx_ptr->rx_ring[wr_id].skb = NULL; + } + + return ret; +} + +static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev, u64 id, + int frags, u64 mapping[IPOIB_CM_RX_SG]) { struct ipoib_dev_priv *priv = netdev_priv(dev); struct sk_buff *skb; int i; + struct ipoib_cm_rx *rx_ptr; + u32 index, wr_id; skb = dev_alloc_skb(IPOIB_CM_HEAD_SIZE + 12); if (unlikely(!skb)) @@ -141,7 +189,14 @@ static struct sk_buff *ipoib_cm_alloc_rx goto partial_error; } - priv->cm.srq_ring[id].skb = skb; + if (priv->cm.srq) + priv->cm.srq_ring[id].skb = skb; + else { + index = id & NOSRQ_INDEX_MASK; + wr_id = id >> 32; + rx_ptr = priv->cm.rx_index_table[index]; + rx_ptr->rx_ring[wr_id].skb = skb; + } return skb; partial_error: @@ -203,11 +258,14 @@ static struct ib_qp *ipoib_cm_create_rx_ .recv_cq = priv->cq, .srq = priv->cm.srq, .cap.max_send_wr = 1, /* For drain WR */ + .cap.max_recv_wr = ipoib_recvq_size + 1, .cap.max_send_sge = 1, /* FIXME: 0 Seems not to work */ .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_RC, .qp_context = p, }; + if (!priv->cm.srq) + attr.cap.max_recv_sge = IPOIB_CM_RX_SG; return ib_create_qp(priv->pd, &attr); } @@ -281,12 +339,130 @@ static int ipoib_cm_send_rep(struct net_ rep.private_data_len = sizeof data; rep.flow_control = 0; rep.rnr_retry_count = req->rnr_retry_count; - rep.srq = 1; rep.qp_num = qp->qp_num; rep.starting_psn = psn; + rep.srq = !!priv->cm.srq; return ib_send_cm_rep(cm_id, &rep); } +static void init_context_and_add_list(struct ib_cm_id *cm_id, + struct ipoib_cm_rx *p, + struct ipoib_dev_priv *priv) +{ + cm_id->context = p; + p->jiffies = jiffies; + spin_lock_irq(&priv->lock); + if (list_empty(&priv->cm.passive_ids)) + queue_delayed_work(ipoib_workqueue, + &priv->cm.stale_task, IPOIB_CM_RX_DELAY); + if (priv->cm.srq) { + /* Add this entry to passive ids list head, but do not re-add + * it if IB_EVENT_QP_LAST_WQE_REACHED has moved it to flush + * list. + */ + if (p->state == IPOIB_CM_RX_LIVE) + list_move(&p->list, &priv->cm.passive_ids); + } + spin_unlock_irq(&priv->lock); +} + +static int allocate_and_post_rbuf_nosrq(struct ib_cm_id *cm_id, + struct ipoib_cm_rx *p, unsigned psn) +{ + struct net_device *dev = cm_id->context; + struct ipoib_dev_priv *priv = netdev_priv(dev); + int ret; + u32 qp_num, index; + u64 i, recv_mem_used; + + qp_num = p->qp->qp_num; + + /* In the SRQ case there is a common rx buffer called the srq_ring. + * However, for the NOSRQ case we create an rx_ring for every + * struct ipoib_cm_rx. + */ + p->rx_ring = kzalloc(ipoib_recvq_size * sizeof *p->rx_ring, GFP_KERNEL); + if (!p->rx_ring) { + printk(KERN_WARNING "Failed to allocate rx_ring for 0x%x\n", + qp_num); + return -ENOMEM; + } + + spin_lock_irq(&priv->lock); + list_add(&p->list, &priv->cm.passive_ids); + spin_unlock_irq(&priv->lock); + + init_context_and_add_list(cm_id, p, priv); + spin_lock_irq(&priv->lock); + + for (index = 0; index < max_rc_qp; index++) + if (priv->cm.rx_index_table[index] == NULL) + break; + + recv_mem_used = (u64)ipoib_recvq_size * + (u64)atomic_inc_return(¤t_rc_qp) * CM_PACKET_SIZE; + if ((index == max_rc_qp) || + (recv_mem_used >= max_recv_buf * (1ul << 20))) { + spin_unlock_irq(&priv->lock); + ipoib_warn(priv, "NOSRQ has reached the configurable limit " + "of either %d RC QPs or, max recv buf size of " + "0x%x MB\n", max_rc_qp, max_recv_buf); + + /* We send a REJ to the remote side indicating that we + * have no more free RC QPs and leave it to the remote side + * to take appropriate action. This should leave the + * current set of QPs unaffected and any subsequent REQs + * will be able to use RC QPs if they are available. + */ + ib_send_cm_rej(cm_id, IB_CM_REJ_NO_QP, NULL, 0, NULL, 0); + ret = -EINVAL; + goto err_alloc_and_post; + } + + priv->cm.rx_index_table[index] = p; + spin_unlock_irq(&priv->lock); + + /* We will subsequently use this stored pointer while freeing + * resources in stale task + */ + p->index = index; + + ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); + if (ret) { + ipoib_warn(priv, "ipoib_cm_modify_rx_qp() failed %d\n", ret); + ipoib_cm_dev_cleanup(dev); + goto err_alloc_and_post; + } + + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_cm_alloc_rx_skb(dev, i << 32 | index, + IPOIB_CM_RX_SG - 1, + p->rx_ring[i].mapping)) { + ipoib_warn(priv, "failed to allocate receive " + "buffer %d\n", (int)i); + ipoib_cm_dev_cleanup(dev); + ret = -ENOMEM; + goto err_alloc_and_post; + } + + if (post_receive_nosrq(dev, i << 32 | index)) { + ipoib_warn(priv, "post_receive_nosrq " + "failed for buf %lld\n", (unsigned long long)i); + ipoib_cm_dev_cleanup(dev); + ret = -EIO; + goto err_alloc_and_post; + } + } + + return 0; + +err_alloc_and_post: + atomic_dec(¤t_rc_qp); + kfree(p->rx_ring); + list_del_init(&p->list); + return ret; +} + static int ipoib_cm_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event) { struct net_device *dev = cm_id->context; @@ -301,9 +477,6 @@ static int ipoib_cm_req_handler(struct i return -ENOMEM; p->dev = dev; p->id = cm_id; - cm_id->context = p; - p->state = IPOIB_CM_RX_LIVE; - p->jiffies = jiffies; INIT_LIST_HEAD(&p->list); p->qp = ipoib_cm_create_rx_qp(dev, p); @@ -313,19 +486,21 @@ static int ipoib_cm_req_handler(struct i } psn = random32() & 0xffffff; - ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); - if (ret) - goto err_modify; + if (!priv->cm.srq) { + ret = allocate_and_post_rbuf_nosrq(cm_id, p, psn); + if (ret) + goto err_modify; + } else { + p->rx_ring = NULL; + ret = ipoib_cm_modify_rx_qp(dev, cm_id, p->qp, psn); + if (ret) + goto err_modify; + } - spin_lock_irq(&priv->lock); - queue_delayed_work(ipoib_workqueue, - &priv->cm.stale_task, IPOIB_CM_RX_DELAY); - /* Add this entry to passive ids list head, but do not re-add it - * if IB_EVENT_QP_LAST_WQE_REACHED has moved it to flush list. */ - p->jiffies = jiffies; - if (p->state == IPOIB_CM_RX_LIVE) - list_move(&p->list, &priv->cm.passive_ids); - spin_unlock_irq(&priv->lock); + if (priv->cm.srq) { + p->state = IPOIB_CM_RX_LIVE; + init_context_and_add_list(cm_id, p, priv); + } ret = ipoib_cm_send_rep(dev, cm_id, p->qp, &event->param.req_rcvd, psn); if (ret) { @@ -398,29 +573,60 @@ static void skb_put_frags(struct sk_buff } } -void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +static void timer_check_srq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + unsigned long flags; + + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { + spin_lock_irqsave(&priv->lock, flags); + p->jiffies = jiffies; + /* Move this entry to list head, but do + * not re-add it if it has been removed. + */ + if (p->state == IPOIB_CM_RX_LIVE) + list_move(&p->list, &priv->cm.passive_ids); + spin_unlock_irqrestore(&priv->lock, flags); + } +} + +static void timer_check_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + unsigned long flags; + + if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { + spin_lock_irqsave(&priv->lock, flags); + p->jiffies = jiffies; + /* Move this entry to list head, but do + * not re-add it if it has been removed. */ + if (!list_empty(&p->list)) + list_move(&p->list, &priv->cm.passive_ids); + spin_unlock_irqrestore(&priv->lock, flags); + } +} + +void handle_rx_wc_srq(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); - unsigned int wr_id = wc->wr_id & ~IPOIB_CM_OP_SRQ; + u64 wr_id = wc->wr_id & ~IPOIB_CM_OP_RECV; struct sk_buff *skb, *newskb; struct ipoib_cm_rx *p; unsigned long flags; u64 mapping[IPOIB_CM_RX_SG]; - int frags; + int frags, ret; - ipoib_dbg_data(priv, "cm recv completion: id %d, status: %d\n", - wr_id, wc->status); + ipoib_dbg_data(priv, "cm recv completion: id %lld, status: %d\n", + (unsigned long long)wr_id, wc->status); if (unlikely(wr_id >= ipoib_recvq_size)) { - if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~IPOIB_CM_OP_SRQ)) { + if (wr_id == (IPOIB_CM_RX_DRAIN_WRID & ~IPOIB_CM_OP_RECV)) { spin_lock_irqsave(&priv->lock, flags); list_splice_init(&priv->cm.rx_drain_list, &priv->cm.rx_reap_list); ipoib_cm_start_rx_drain(priv); queue_work(ipoib_workqueue, &priv->cm.rx_reap_task); spin_unlock_irqrestore(&priv->lock, flags); } else - ipoib_warn(priv, "cm recv completion event with wrid %d (> %d)\n", - wr_id, ipoib_recvq_size); + ipoib_warn(priv, "cm recv completion event with wrid %lld (> %d)\n", + (unsigned long long)wr_id, ipoib_recvq_size); return; } @@ -428,23 +634,15 @@ void ipoib_cm_handle_rx_wc(struct net_de if (unlikely(wc->status != IB_WC_SUCCESS)) { ipoib_dbg(priv, "cm recv error " - "(status=%d, wrid=%d vend_err %x)\n", - wc->status, wr_id, wc->vendor_err); + "(status=%d, wrid=%lld vend_err %x)\n", + wc->status, (unsigned long long)wr_id, wc->vendor_err); ++priv->stats.rx_dropped; - goto repost; + goto repost_srq; } if (!likely(wr_id & IPOIB_CM_RX_UPDATE_MASK)) { p = wc->qp->qp_context; - if (p && time_after_eq(jiffies, p->jiffies + IPOIB_CM_RX_UPDATE_TIME)) { - spin_lock_irqsave(&priv->lock, flags); - p->jiffies = jiffies; - /* Move this entry to list head, but do not re-add it - * if it has been moved out of list. */ - if (p->state == IPOIB_CM_RX_LIVE) - list_move(&p->list, &priv->cm.passive_ids); - spin_unlock_irqrestore(&priv->lock, flags); - } + timer_check_srq(priv, p); } frags = PAGE_ALIGN(wc->byte_len - min(wc->byte_len, @@ -456,13 +654,112 @@ void ipoib_cm_handle_rx_wc(struct net_de * If we can't allocate a new RX buffer, dump * this packet and reuse the old buffer. */ - ipoib_dbg(priv, "failed to allocate receive buffer %d\n", wr_id); + ipoib_dbg(priv, "failed to allocate receive buffer %lld\n", + (unsigned long long)wr_id); ++priv->stats.rx_dropped; - goto repost; + goto repost_srq; } - ipoib_cm_dma_unmap_rx(priv, frags, priv->cm.srq_ring[wr_id].mapping); - memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, (frags + 1) * sizeof *mapping); + ipoib_cm_dma_unmap_rx(priv, frags, + priv->cm.srq_ring[wr_id].mapping); + memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, + (frags + 1) * sizeof *mapping); + ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", + wc->byte_len, wc->slid); + + skb_put_frags(skb, IPOIB_CM_HEAD_SIZE, wc->byte_len, newskb); + + skb->protocol = ((struct ipoib_header *) skb->data)->proto; + skb_reset_mac_header(skb); + skb_pull(skb, IPOIB_ENCAP_LEN); + + dev->last_rx = jiffies; + ++priv->stats.rx_packets; + priv->stats.rx_bytes += skb->len; + + skb->dev = dev; + /* XXX get correct PACKET_ type here */ + skb->pkt_type = PACKET_HOST; + netif_receive_skb(skb); + +repost_srq: + ret = post_receive_srq(dev, wr_id); + + if (unlikely(ret)) + ipoib_warn(priv, "post_receive_srq failed for buf %lld\n", + (unsigned long long)wr_id); + +} + +static void handle_rx_wc_nosrq(struct net_device *dev, struct ib_wc *wc) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + struct sk_buff *skb, *newskb; + u64 mapping[IPOIB_CM_RX_SG], wr_id = wc->wr_id >> 32; + u32 index; + struct ipoib_cm_rx *rx_ptr; + int frags, ret; + + + ipoib_dbg_data(priv, "cm recv completion: id %lld, status: %d\n", + (unsigned long long)wr_id, wc->status); + + if (unlikely(wr_id >= ipoib_recvq_size)) { + ipoib_warn(priv, "cm recv completion event with wrid %lld (> %d)\n", + (unsigned long long)wr_id, ipoib_recvq_size); + return; + } + + index = (wc->wr_id & ~IPOIB_CM_OP_RECV) & NOSRQ_INDEX_MASK; + + /* This is the only place where rx_ptr could be a NULL - could + * have just received a packet from a connection that has become + * stale and so is going away. We will simply drop the packet and + * let the hardware (it s IB_QPT_RC) handle the dropped packet. + * In the timer_check() function below, p->jiffies is updated and + * hence the connection will not be stale after that. + */ + rx_ptr = priv->cm.rx_index_table[index]; + if (unlikely(!rx_ptr)) { + ipoib_warn(priv, "Received packet from a connection " + "that is going away. Hardware will handle it.\n"); + return; + } + + skb = rx_ptr->rx_ring[wr_id].skb; + + if (unlikely(wc->status != IB_WC_SUCCESS)) { + ipoib_dbg(priv, "cm recv error " + "(status=%d, wrid=%lld vend_err %x)\n", + wc->status, (unsigned long long)wr_id, wc->vendor_err); + ++priv->stats.rx_dropped; + goto repost_nosrq; + } + + if (!likely(wr_id & IPOIB_CM_RX_UPDATE_MASK)) + /* There are no guarantees that wc->qp is not NULL for HCAs + * that do not support SRQ. */ + timer_check_nosrq(priv, rx_ptr); + + frags = PAGE_ALIGN(wc->byte_len - min(wc->byte_len, + (unsigned)IPOIB_CM_HEAD_SIZE)) / PAGE_SIZE; + + newskb = ipoib_cm_alloc_rx_skb(dev, wr_id << 32 | index, frags, + mapping); + if (unlikely(!newskb)) { + /* + * If we can't allocate a new RX buffer, dump + * this packet and reuse the old buffer. + */ + ipoib_dbg(priv, "failed to allocate receive buffer %lld\n", + (unsigned long long)wr_id); + ++priv->stats.rx_dropped; + goto repost_nosrq; + } + + ipoib_cm_dma_unmap_rx(priv, frags, rx_ptr->rx_ring[wr_id].mapping); + memcpy(rx_ptr->rx_ring[wr_id].mapping, mapping, + (frags + 1) * sizeof *mapping); ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", wc->byte_len, wc->slid); @@ -482,10 +779,22 @@ void ipoib_cm_handle_rx_wc(struct net_de skb->pkt_type = PACKET_HOST; netif_receive_skb(skb); -repost: - if (unlikely(ipoib_cm_post_receive(dev, wr_id))) - ipoib_warn(priv, "ipoib_cm_post_receive failed " - "for buf %d\n", wr_id); +repost_nosrq: + ret = post_receive_nosrq(dev, wr_id << 32 | index); + + if (unlikely(ret)) + ipoib_warn(priv, "post_receive_nosrq failed for buf %lld\n", + (unsigned long long)wr_id); +} + +void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + + if (priv->cm.srq) + handle_rx_wc_srq(dev, wc); + else + handle_rx_wc_nosrq(dev, wc); } static inline int post_send(struct ipoib_dev_priv *priv, @@ -677,6 +986,43 @@ err_cm: return ret; } +static void free_resources_nosrq(struct ipoib_dev_priv *priv, struct ipoib_cm_rx *p) +{ + int i; + + for (i = 0; i < ipoib_recvq_size; ++i) + if (p->rx_ring[i].skb) { + ipoib_cm_dma_unmap_rx(priv, + IPOIB_CM_RX_SG - 1, + p->rx_ring[i].mapping); + dev_kfree_skb_any(p->rx_ring[i].skb); + p->rx_ring[i].skb = NULL; + } + kfree(p->rx_ring); +} + +void dev_stop_nosrq(struct ipoib_dev_priv *priv) +{ + struct ipoib_cm_rx *p; + + spin_lock_irq(&priv->lock); + while (!list_empty(&priv->cm.passive_ids)) { + p = list_entry(priv->cm.passive_ids.next, typeof(*p), list); + free_resources_nosrq(priv, p); + list_del(&p->list); + spin_unlock_irq(&priv->lock); + ib_destroy_cm_id(p->id); + ib_destroy_qp(p->qp); + atomic_dec(¤t_rc_qp); + kfree(p); + spin_lock_irq(&priv->lock); + } + spin_unlock_irq(&priv->lock); + + cancel_delayed_work(&priv->cm.stale_task); + kfree(priv->cm.rx_index_table); +} + void ipoib_cm_dev_stop(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -691,6 +1037,11 @@ void ipoib_cm_dev_stop(struct net_device ib_destroy_cm_id(priv->cm.id); priv->cm.id = NULL; + if (!priv->cm.srq) { + dev_stop_nosrq(priv); + return; + } + spin_lock_irq(&priv->lock); while (!list_empty(&priv->cm.passive_ids)) { p = list_entry(priv->cm.passive_ids.next, typeof(*p), list); @@ -814,7 +1165,9 @@ static struct ib_qp *ipoib_cm_create_tx_ attr.recv_cq = priv->cq; attr.srq = priv->cm.srq; attr.cap.max_send_wr = ipoib_sendq_size; + attr.cap.max_recv_wr = 0; attr.cap.max_send_sge = 1; + attr.cap.max_recv_sge = 0; attr.sq_sig_type = IB_SIGNAL_ALL_WR; attr.qp_type = IB_QPT_RC; attr.send_cq = cq; @@ -854,7 +1207,7 @@ static int ipoib_cm_send_req(struct net_ req.retry_count = 0; /* RFC draft warns against retries */ req.rnr_retry_count = 0; /* RFC draft warns against retries */ req.max_cm_retries = 15; - req.srq = 1; + req.srq = !!priv->cm.srq; return ib_send_cm_req(id, &req); } @@ -1198,6 +1551,8 @@ static void ipoib_cm_rx_reap(struct work list_for_each_entry_safe(p, n, &list, list) { ib_destroy_cm_id(p->id); ib_destroy_qp(p->qp); + if (!priv->cm.srq) + atomic_dec(¤t_rc_qp); kfree(p); } } @@ -1216,12 +1571,19 @@ static void ipoib_cm_stale_task(struct w p = list_entry(priv->cm.passive_ids.prev, typeof(*p), list); if (time_before_eq(jiffies, p->jiffies + IPOIB_CM_RX_TIMEOUT)) break; - list_move(&p->list, &priv->cm.rx_error_list); - p->state = IPOIB_CM_RX_ERROR; - spin_unlock_irq(&priv->lock); - ret = ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE); - if (ret) - ipoib_warn(priv, "unable to move qp to error state: %d\n", ret); + if (!priv->cm.srq) { + free_resources_nosrq(priv, p); + list_del_init(&p->list); + priv->cm.rx_index_table[p->index] = NULL; + spin_unlock_irq(&priv->lock); + } else { + list_move(&p->list, &priv->cm.rx_error_list); + p->state = IPOIB_CM_RX_ERROR; + spin_unlock_irq(&priv->lock); + ret = ib_modify_qp(p->qp, &ipoib_cm_err_attr, IB_QP_STATE); + if (ret) + ipoib_warn(priv, "unable to move qp to error state: %d\n", ret); + } spin_lock_irq(&priv->lock); } @@ -1275,16 +1637,40 @@ int ipoib_cm_add_mode_attr(struct net_de return device_create_file(&dev->dev, &dev_attr_mode); } +static int create_srq(struct net_device *dev, struct ipoib_dev_priv *priv) +{ + struct ib_srq_init_attr srq_init_attr; + int ret; + + srq_init_attr.attr.max_wr = ipoib_recvq_size; + srq_init_attr.attr.max_sge = IPOIB_CM_RX_SG; + + priv->cm.srq = ib_create_srq(priv->pd, &srq_init_attr); + if (IS_ERR(priv->cm.srq)) { + ret = PTR_ERR(priv->cm.srq); + priv->cm.srq = NULL; + return ret; + } + + priv->cm.srq_ring = kzalloc(ipoib_recvq_size * + sizeof *priv->cm.srq_ring, + GFP_KERNEL); + if (!priv->cm.srq_ring) { + printk(KERN_WARNING "%s: failed to allocate CM ring " + "(%d entries)\n", + priv->ca->name, ipoib_recvq_size); + ipoib_cm_dev_cleanup(dev); + return -ENOMEM; + } + + return 0; +} + int ipoib_cm_dev_init(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ib_srq_init_attr srq_init_attr = { - .attr = { - .max_wr = ipoib_recvq_size, - .max_sge = IPOIB_CM_RX_SG - } - }; int ret, i; + struct ib_device_attr attr; INIT_LIST_HEAD(&priv->cm.passive_ids); INIT_LIST_HEAD(&priv->cm.reap_list); @@ -1301,20 +1687,32 @@ int ipoib_cm_dev_init(struct net_device skb_queue_head_init(&priv->cm.skb_queue); - priv->cm.srq = ib_create_srq(priv->pd, &srq_init_attr); - if (IS_ERR(priv->cm.srq)) { - ret = PTR_ERR(priv->cm.srq); - priv->cm.srq = NULL; + ret = ib_query_device(priv->ca, &attr); + if (ret) return ret; - } - priv->cm.srq_ring = kzalloc(ipoib_recvq_size * sizeof *priv->cm.srq_ring, - GFP_KERNEL); - if (!priv->cm.srq_ring) { - printk(KERN_WARNING "%s: failed to allocate CM ring (%d entries)\n", - priv->ca->name, ipoib_recvq_size); - ipoib_cm_dev_cleanup(dev); - return -ENOMEM; + if (attr.max_srq) { + /* This device supports SRQ */ + ret = create_srq(dev, priv); + if (ret) + return ret; + priv->cm.rx_index_table = NULL; + } else { + priv->cm.srq = NULL; + priv->cm.srq_ring = NULL; + + /* Every new REQ that arrives creates a struct ipoib_cm_rx. + * These structures form a link list starting with the + * passive_ids. For quick and easy access we maintain a table + * of pointers to struct ipoib_cm_rx called the rx_index_table + */ + priv->cm.rx_index_table = kcalloc(max_rc_qp, + sizeof *priv->cm.rx_index_table, + GFP_KERNEL); + if (!priv->cm.rx_index_table) { + printk(KERN_WARNING "Failed to allocate rx_index_table\n"); + return -ENOMEM; + } } for (i = 0; i < IPOIB_CM_RX_SG; ++i) @@ -1327,17 +1725,24 @@ int ipoib_cm_dev_init(struct net_device priv->cm.rx_wr.sg_list = priv->cm.rx_sge; priv->cm.rx_wr.num_sge = IPOIB_CM_RX_SG; - for (i = 0; i < ipoib_recvq_size; ++i) { - if (!ipoib_cm_alloc_rx_skb(dev, i, IPOIB_CM_RX_SG - 1, + /* One can post receive buffers even before the RX QP is created + * only in the SRQ case. Therefore for NOSRQ we skip the rest of init + * and do that in ipoib_cm_req_handler() + */ + + if (priv->cm.srq) { + for (i = 0; i < ipoib_recvq_size; ++i) { + if (!ipoib_cm_alloc_rx_skb(dev, i, IPOIB_CM_RX_SG - 1, priv->cm.srq_ring[i].mapping)) { - ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); - ipoib_cm_dev_cleanup(dev); - return -ENOMEM; - } - if (ipoib_cm_post_receive(dev, i)) { - ipoib_warn(priv, "ipoib_ib_post_receive failed for buf %d\n", i); - ipoib_cm_dev_cleanup(dev); - return -EIO; + ipoib_warn(priv, "failed to allocate receive buffer %d\n", i); + ipoib_cm_dev_cleanup(dev); + return -ENOMEM; + } + if (post_receive_srq(dev, i)) { + ipoib_warn(priv, "post_receive_srq failed for buf %d\n", i); + ipoib_cm_dev_cleanup(dev); + return -EIO; + } } } --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-20 17:39:25.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-14 19:53:16.000000000 -0400 @@ -300,7 +300,7 @@ int ipoib_poll(struct net_device *dev, i for (i = 0; i < n; ++i) { struct ib_wc *wc = priv->ibwc + i; - if (wc->wr_id & IPOIB_CM_OP_SRQ) { + if (wc->wr_id & IPOIB_CM_OP_RECV) { ++done; --max; ipoib_cm_handle_rx_wc(dev, wc); @@ -558,7 +558,7 @@ void ipoib_drain_cq(struct net_device *d do { n = ib_poll_cq(priv->cq, IPOIB_NUM_WC, priv->ibwc); for (i = 0; i < n; ++i) { - if (priv->ibwc[i].wr_id & IPOIB_CM_OP_SRQ) + if (priv->ibwc[i].wr_id & IPOIB_CM_OP_RECV) ipoib_cm_handle_rx_wc(dev, priv->ibwc + i); else if (priv->ibwc[i].wr_id & IPOIB_OP_RECV) ipoib_ib_handle_rx_wc(dev, priv->ibwc + i); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-20 17:39:25.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-14 19:53:16.000000000 -0400 @@ -175,6 +175,18 @@ int ipoib_transport_dev_init(struct net_ if (!ret) size += ipoib_recvq_size + 1 /* 1 extra for rx_drain_qp */; +#ifdef CONFIG_INFINIBAND_IPOIB_CM + + /* We increase the size of the CQ in the NOSRQ case to prevent CQ + * overflow. Every new REQ creates a new RX QP and each QP has an + * RX ring associated with it. Therefore we could have + * max_rc_qp*ipoib_recvq_size + ipoib_sendq_size CQEs + * in a CQ. + */ + if (!priv->cm.srq) + size += (max_rc_qp - 1) * ipoib_recvq_size; +#endif + priv->cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL, dev, size, 0); if (IS_ERR(priv->cq)) { printk(KERN_WARNING "%s: failed to create CQ\n", ca->name); --- a/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-20 17:39:25.000000000 -0400 +++ b/linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-20 17:42:13.000000000 -0400 @@ -1227,6 +1227,7 @@ static int __init ipoib_init_module(void ipoib_sendq_size = roundup_pow_of_two(ipoib_sendq_size); ipoib_sendq_size = min(ipoib_sendq_size, IPOIB_MAX_QUEUE_SIZE); ipoib_sendq_size = max(ipoib_sendq_size, IPOIB_MIN_QUEUE_SIZE); + max_rc_qp = roundup_pow_of_two(max_rc_qp); ret = ipoib_register_debugfs(); if (ret) From rdreier at cisco.com Mon Aug 20 18:16:54 2007 From: rdreier at cisco.com (Roland Dreier) Date: Mon, 20 Aug 2007 18:16:54 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070817.234405.66176298.davem@davemloft.net> (David Miller's message of "Fri, 17 Aug 2007 23:44:05 -0700 (PDT)") References: <20070817.170033.63993876.davem@davemloft.net> <20070817.234405.66176298.davem@davemloft.net> Message-ID: [TSO / LRO discussion snipped -- it's not the main point so no sense spending energy arguing about it] > Just be realistic and accept that RDMA is a point in time solution, > and like any other such technology takes flexibility away from users. > > Horizontal scaling of cpus up to huge arity cores, network devices > using large numbers of transmit and receive queues and classification > based queue selection, are all going to work to make things like RDMA > even more irrelevant than they already are. To me there is a real fundamental difference between RDMA and traditional SOCK_STREAM / SOCK_DATAGRAM networking, namely that messages can carry the address where they're supposed to be delivered (what the IETF calls "direct data placement"). And on top of that you can build one-sided operations aka put/get aka RDMA. And direct data placement really does give you a factor of two at least, because otherwise you're stuck receiving the data in one buffer, looking at some of the data at least, and then figuring out where to copy it. And memory bandwidth is if anything becoming more valuable; maybe LRO + header splitting + page remapping tricks can get you somewhere but as NCPUS grows then it seems the TLB shootdown cost of page flipping is only going to get worse. Don't get too hung up on the fact that current iWARP (RDMA over IP) implementations are using TCP offload -- to me that is just a side effect of doing enough processing on the NIC side of the PCI bus to be able to do direct data placement. InfiniBand with competely different transport, link and physical layers is one way to implement RDMA without TCP offload and I'm sure there will be others -- eg Intel's IOAT stuff could probably evolve to the point where you could implement iWARP with software TCP and the data placement offloaded to some DMA engine. - R. From felix at chelsio.com Mon Aug 20 21:21:18 2007 From: felix at chelsio.com (Felix Marti) Date: Mon, 20 Aug 2007 21:21:18 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCPportsfrom the host TCP port space. References: <20070819.174017.77241227.davem@davemloft.net> <8A71B368A89016469F72CD08050AD334018E20BE@maui.asicdesigners.com> <20070820094317.GA14817@2ka.mipt.ru> <8A71B368A89016469F72CD08050AD334018E2115@maui.asicdesigners.com> <46C9FAB4.5020609@myri.com> Message-ID: <8A71B368A89016469F72CD08050AD334018E21C6@maui.asicdesigners.com> > -----Original Message----- > From: Patrick Geoffray [mailto:patrick at myri.com] > Sent: Monday, August 20, 2007 1:34 PM > To: Felix Marti > Cc: Evgeniy Polyakov; David Miller; sean.hefty at intel.com; > netdev at vger.kernel.org; rdreier at cisco.com; > general at lists.openfabrics.org; linux-kernel at vger.kernel.org; > jeff at garzik.org > Subject: Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate > PS_TCPportsfrom the host TCP port space. > > Felix Marti wrote: > > Yes, the app will take the cache hits when accessing the data. > However, > > the fact remains that if there is a copy in the receive path, you > > require and additional 3x memory BW (which is very significant at > these > > high rates and most likely the bottleneck for most current > systems)... > > and somebody always has to take the cache miss be it the copy_to_user > or > > the app. > > The cache miss is going to cost you half the memory bandwidth of a full > copy. If the data is already in cache, then the copy is cheaper. > > However, removing the copy removes the kernel from the picture on the > receive side, so you lose demultiplexing, asynchronism, security, > accounting, flow-control, swapping, etc. If it's ok with you to not use > the kernel stack, then why expect to fit in the existing infrastructure > anyway ? Many of the things you're referring to are moved to the offload adapter but from an ease of use point of view, it would be great if the user could still collect stats the same way, i.e. netstat reports the 4-tuple in use and other network stats. In addition, security features and packet scheduling could be integrated so that the user configures them the same way as the network stack. > > > Yes, RDMA support is there... but we could make it better and easier > to > > What do you need from the kernel for RDMA support beyond HW drivers ? A > fast way to pin and translate user memory (ie registration). That is > pretty much the sandbox that David referred to. > > Eventually, it would be useful to be able to track the VM space to > implement a registration cache instead of using ugly hacks in user- > space > to hijack malloc, but this is completely independent from the net > stack. > > > use. We have a problem today with port sharing and there was a > proposal > > The port spaces are either totally separate and there is no issue, or > completely identical and you should then run your connection manager in > user-space or fix your middlewares. When running on an iWarp device (and hence on top of TCP) I believe that the port space should shared and i.e. netstat reports the 4-tuple in use. > > > and not for technical reasons. I believe this email threads shows in > > detail how RDMA (a network technology) is treated as bastard child by > > the network folks, well at least by one of them. > > I don't think it's fair. This thread actually show how pushy some RDMA > folks are about not acknowledging that the current infrastructure is > here for a reason, and about mistaking zero-copy and RDMA. Zero-copy and RDMA are not the same but in the context of this discussion I referred to RDMA as a superset (zero-copy is implied). > > This is a similar argument than the TOE discussion, and it was > definitively a good decision to not mess up the Linux stack with TOEs. > > Patrick From kliteyn at mellanox.co.il Mon Aug 20 21:39:11 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 21 Aug 2007 07:39:11 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-21:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_20_12:20:05_2007 [a6cfde5efb96f6b3858c26e6934554ded57f89cb] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From davem at davemloft.net Mon Aug 20 23:58:04 2007 From: davem at davemloft.net (David Miller) Date: Mon, 20 Aug 2007 23:58:04 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: References: <20070817.234405.66176298.davem@davemloft.net> Message-ID: <20070820.235804.85409183.davem@davemloft.net> From: Roland Dreier Date: Mon, 20 Aug 2007 18:16:54 -0700 > And direct data placement really does give you a factor of two at > least, because otherwise you're stuck receiving the data in one > buffer, looking at some of the data at least, and then figuring out > where to copy it. And memory bandwidth is if anything becoming more > valuable; maybe LRO + header splitting + page remapping tricks can get > you somewhere but as NCPUS grows then it seems the TLB shootdown cost > of page flipping is only going to get worse. As Herbert has said already, people can code for this just like they have to code for RDMA. There is no fundamental difference from converting an application to sendfile or similar. The only thing this needs is a "recvmsg_I_dont_care_where_the_data_is()" call. There are no alignment issues unless you are trying to push this data directly into the page cache. Couple this with a card that makes sure that on a per-page basis, only data for a particular flow (or group of flows) will accumulate. People already make cards that can do stuff like this, it can be done statelessly with an on-chip dynamically maintained flow table. And best yet it doesn't turn off every feature in the networking nor bypass it for the actual protocol processing. From davem at davemloft.net Tue Aug 21 00:18:24 2007 From: davem at davemloft.net (David Miller) Date: Tue, 21 Aug 2007 00:18:24 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: Message-ID: <20070821.001824.105429474.davem@davemloft.net> From: Krishna Kumar2 Date: Fri, 17 Aug 2007 11:36:03 +0530 > > I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will > > run a longer one tonight). The results are (results in KB/s, and %): > > I ran a 8.5 hours run with no batching + another 8.5 hours run with > batching (Buffer sizes: "32 128 512 4096 16384", Threads: "1 8 32", > Each test run time: 3 minutes, Iterations to average: 5). TCP seems > to get a small improvement. Using 16K buffer size really isn't going to keep the pipe full enough for TSO. And realistically applications queue much more data at a time. Also, with smaller buffer sizes can have negative effects for the dynamic receive and send buffer growth algorithm the kernel uses, it might consider the connection application limited for too long. I would really prefer to see numbers that use buffer sizes more in line with the amount of data that is typically inflight on a 1G connection on a local network. Do a tcpdump during the height of the transfer to see about what this value is. When an ACK comes in, compare the sequence number it's ACK'ing with the sequence number of the most recently sent frame. The difference is approximately the pipe size at maximum congestion window assuming a loss free local network. From a-3-9-6-2 at accessology.com Tue Aug 21 09:18:15 2007 From: a-3-9-6-2 at accessology.com (Kyle Barnes) Date: Tue, 21 Aug 2007 08:18:15 -0800 Subject: [ofa-general] Pics Message-ID: <01c7e3cb$d2e3f710$484e373b@a-3-9-6-2> Hello! I am tired this afternoon. I am nice girl that would like to chat with you. Email me at eh at mailmessageonline.info only, because I am writing not from my personal email. Don't miss some of my naughty pictures. From kliteyn at dev.mellanox.co.il Tue Aug 21 01:21:48 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 21 Aug 2007 11:21:48 +0300 Subject: [ofa-general] Re: [PATCH 5/7] osm: QoS - compiling policy file parser In-Reply-To: <20070820124049.GC25387@mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <20070820123716.GB25387@mellanox.co.il> <20070820124049.GC25387@mellanox.co.il> Message-ID: <46CAA09C.7000304@dev.mellanox.co.il> Michael S. Tsirkin wrote: >> Quoting Michael S. Tsirkin : >> Subject: Re: [PATCH 5/7] osm: QoS - compiling policy file parser >> >>> Quoting Yevgeny Kliteynik : >>> Subject: [PATCH 5/7] osm: QoS - compiling policy file parser >>> >>> Compiling QoS policy file parser. >>> >>> C & H files are generated from Lex & Yacc files and the >>> generated files are compiled. >>> Generated files are not included in git, but they do get >>> included in RPM as they are also created by 'make dist'. >>> >>> Signed-off-by: Yevgeny Kliteynik >>> --- >>> opensm/include/Makefile.am | 2 ++ >>> opensm/opensm/Makefile.am | 28 +++++++++++++++++++++++++++- >>> opensm/opensm/configure.in | 2 ++ >>> 3 files changed, 31 insertions(+), 1 deletions(-) >>> >>> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am >>> index ea62507..b83205f 100644 >>> --- a/opensm/include/Makefile.am >>> +++ b/opensm/include/Makefile.am >>> @@ -90,6 +90,8 @@ EXTRA_DIST = \ >>> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ >>> $(srcdir)/opensm/osm_perfmgr.h \ >>> $(srcdir)/opensm/osm_perfmgr_db.h \ >>> + $(srcdir)/opensm/osm_qos_policy.h \ >>> + $(srcdir)/opensm/osm_qos_parser_y.h \ >>> $(srcdir)/complib/cl_thread_osd.h \ >>> $(srcdir)/complib/cl_packon.h \ >>> $(srcdir)/complib/cl_atomic_osd.h \ >>> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >>> index 6dfa824..6c7208d 100644 >>> --- a/opensm/opensm/Makefile.am >>> +++ b/opensm/opensm/Makefile.am >>> @@ -56,7 +56,21 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >>> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ >>> osm_vl15intf.c osm_vl_arb_rcv.c \ >>> st.c osm_perfmgr.c osm_perfmgr_db.c \ >>> - osm_event_plugin.c osm_dump.c >>> + osm_event_plugin.c osm_dump.c \ >>> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >>> + >>> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h >>> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y >>> + mv y.tab.c osm_qos_parser_y.c >>> + mv y.tab.h osm_qos_parser_y.h >>> + cp -f osm_qos_parser_y.c $(srcdir)/ >>> + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ >>> + >>> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h >>> + $(LEX) $(srcdir)/osm_qos_parser.l >>> + mv lex.yy.c osm_qos_parser_l.c >>> + cp -f osm_qos_parser_l.c $(srcdir)/ >>> + >> Why do you use these mv tricks? >> Can you not use the -o flag to generate the output file with the proper name? Good idea > Alternatively, if you do not want to use flex/bison extensions, > you can use -t flag to have lex output to standard output, > and -b flag to supply prefix to yacc. > > BTW, I think it's a good idea to also use -p prefix to reduce > the global namespace pollution by yacc. OK. It affects patches 2/7 and 5/7 - I'll repost them. -- Yevgeny > From vlad at lists.openfabrics.org Tue Aug 21 01:41:44 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 21 Aug 2007 01:41:44 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070821-0100 daily build status Message-ID: <20070821084145.1666FE60836@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: From koen.segers at vrt.be Tue Aug 21 02:40:11 2007 From: koen.segers at vrt.be (Koen Segers) Date: Tue, 21 Aug 2007 11:40:11 +0200 Subject: [ofa-general] SDP overhead Message-ID: <1187689211.7505.7.camel@koenVRT> Hello, Can somebody tell me what the overhead is of an SDP packet when the packet is completely filled with data? The numbers I want to have are: max data length of 1 packet total header length of 1 packet Regards, Koen Segers *** Disclaimer *** Vlaamse Radio- en Televisieomroep Auguste Reyerslaan 52, 1043 Brussel nv van publiek recht BTW BE 0244.142.664 RPR Brussel http://www.vrt.be/disclaimer From ogerlitz at voltaire.com Tue Aug 21 02:51:09 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 21 Aug 2007 12:51:09 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <1187627087.30342.38.camel@mtls03> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> Message-ID: <46CAB58D.5090705@voltaire.com> Eli Cohen wrote: > On Mon, 2007-08-20 at 18:48 +0300, Or Gerlitz wrote: > >> I see that the patch adds the NETIF_F_TSO flag to the device features >> but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? >> if yes, what would it take SW wise to support it? > > UDP LSO is not supported by the HW. I see. Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and thinking about the connectx TCP segmentation offloading a little further I am somehow confused: With the ipoib connected mode a typical MTU exposed to the OS is 64K, where data goes over IB RC (soon to be UC) transport meaning that the HCA does the "fragmentation / reassembly" at the QP (IB L4) level allowing to send one 64K IB packet over a path whose Link layer (IB L2) MTU is 2k. So TCP can use segments size upto 64K and UDP can send datagram's whose size is upto 64K without IP fragmentation. With all this at hand, what does LSO buys you at all? Or. From vlad at lists.openfabrics.org Tue Aug 21 02:53:32 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 21 Aug 2007 02:53:32 -0700 (PDT) Subject: [ofa-general] ofa_1_2_c_kernel 20070821-0200 daily build status Message-ID: <20070821095332.EE3D9E603CA@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of ‘->’ /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of ‘->’ make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From mst at dev.mellanox.co.il Tue Aug 21 03:13:01 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 21 Aug 2007 13:13:01 +0300 Subject: [ofa-general] Re: [PATCH - 11] ipoib - add LSO support In-Reply-To: <46CAB58D.5090705@voltaire.com> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> Message-ID: <20070821101300.GC9906@mellanox.co.il> > Quoting Or Gerlitz : > Subject: Re: [PATCH - 11] ipoib - add LSO support > > Eli Cohen wrote: > >On Mon, 2007-08-20 at 18:48 +0300, Or Gerlitz wrote: > > > >>I see that the patch adds the NETIF_F_TSO flag to the device features > >>but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? > >>if yes, what would it take SW wise to support it? > > > >UDP LSO is not supported by the HW. > > I see. > > > Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and > thinking about the connectx TCP segmentation offloading a little further > I am somehow confused: > > With the ipoib connected mode a typical MTU exposed to the OS is 64K, > where data goes over IB RC (soon to be UC) transport meaning that the > HCA does the "fragmentation / reassembly" at the QP (IB L4) level > allowing to send one 64K IB packet over a path whose Link layer (IB L2) > MTU is 2k. > > So TCP can use segments size upto 64K and UDP can send datagram's whose > size is upto 64K without IP fragmentation. > > With all this at hand, what does LSO buys you at all? If you read these patches closely, you'll see that LSO is disabled for connected mode. -- MST From eli at mellanox.co.il Tue Aug 21 03:33:52 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 21 Aug 2007 13:33:52 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <46CAB58D.5090705@voltaire.com> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> Message-ID: <1187692432.30342.57.camel@mtls03> On Tue, 2007-08-21 at 12:51 +0300, Or Gerlitz wrote: > Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and > thinking about the connectx TCP segmentation offloading a little further > I am somehow confused: > > With the ipoib connected mode a typical MTU exposed to the OS is 64K, > where data goes over IB RC (soon to be UC) transport meaning that the > HCA does the "fragmentation / reassembly" at the QP (IB L4) level > allowing to send one 64K IB packet over a path whose Link layer (IB L2) > MTU is 2k. > > So TCP can use segments size upto 64K and UDP can send datagram's whose > size is upto 64K without IP fragmentation. > > With all this at hand, what does LSO buys you at all? > You have the option to work in either connected or datagram mode, and some might prefer to use datagram mode for various reasons. In these cases you will want the benefits that LSO provides. From ogerlitz at voltaire.com Tue Aug 21 03:45:54 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 21 Aug 2007 13:45:54 +0300 Subject: [ofa-general] Re: [PATCH - 11] ipoib - add LSO support In-Reply-To: <20070821101300.GC9906@mellanox.co.il> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> <20070821101300.GC9906@mellanox.co.il> Message-ID: <46CAC262.1040509@voltaire.com> Michael S. Tsirkin wrote: >> Quoting Or Gerlitz : >> With the ipoib connected mode a typical MTU exposed to the OS is 64K, >> where data goes over IB RC (soon to be UC) transport meaning that the >> HCA does the "fragmentation / reassembly" at the QP (IB L4) level >> allowing to send one 64K IB packet over a path whose Link layer (IB L2) >> MTU is 2k. >> So TCP can use segments size upto 64K and UDP can send datagram's whose >> size is upto 64K without IP fragmentation. >> With all this at hand, what does LSO buys you at all? > If you read these patches closely, you'll see that LSO > is disabled for connected mode. This makes me more confused: if someone buys connectX and uses IPoIB, since connected mode provides the best performance for both TCP and UDP unicast traffic they would not bother to use the datagram mode. And if they have UDP multicast traffic, they would use the UD QP of IPoIB (which means they use datagram mode for multicast). So the only case where I see usage of the LSO stateless offloading is when working against device which does not support connected mode, such as IB/Ethernet IP gateways, but as you may know, very much of the traffic over these gateways is IP multicast based... Or. From ogerlitz at voltaire.com Tue Aug 21 03:52:31 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Tue, 21 Aug 2007 13:52:31 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <1187692432.30342.57.camel@mtls03> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> <1187692432.30342.57.camel@mtls03> Message-ID: <46CAC3EF.6000509@voltaire.com> Eli Cohen wrote: > You have the option to work in either connected or datagram mode, and > some might prefer to use datagram mode for various reasons. In these > cases you will want the benefits that LSO provides. Can you provide a pointer to some Mellanox high level doc/faq explaining the possible benefits from the stateless offloading features of connectX? Or. From kliteyn at dev.mellanox.co.il Tue Aug 21 04:11:59 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 21 Aug 2007 14:11:59 +0300 Subject: [ofa-general] Re: [PATCH 1/7] osm: QoS - adding new PathRecord fields In-Reply-To: <20070820165831.GG27000@sashak.voltaire.com> References: <46C982FA.2040601@dev.mellanox.co.il> <20070820165831.GG27000@sashak.voltaire.com> Message-ID: <46CAC87F.8010702@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > Hi Yevgeny! > > On 15:03 Mon 20 Aug , Yevgeny Kliteynik wrote: >> Adding QoS fields to PathRecord. >> >> Signed-off-by: Yevgeny Kliteynik > > Some comments are below. > >> --- >> infiniband-diags/src/saquery.c | 8 +- >> opensm/include/iba/ib_types.h | 158 +++++++++++++++++++++++++++---- >> opensm/opensm/osm_helper.c | 8 +- >> opensm/opensm/osm_sa_multipath_record.c | 2 +- >> opensm/opensm/osm_sa_path_record.c | 4 +- >> opensm/osmtest/osmtest.c | 2 +- >> 6 files changed, 154 insertions(+), 28 deletions(-) >> >> diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c >> index 522399e..f085854 100644 >> --- a/infiniband-diags/src/saquery.c >> +++ b/infiniband-diags/src/saquery.c >> @@ -188,7 +188,7 @@ static void >> print_path_record(ib_path_rec_t *p_pr) >> { >> printf("PathRecord dump:\n" >> - "\t\tresv0...................0x%016" PRIx64 "\n" >> + "\t\tservice_id..............0x%016" PRIx64 "\n" >> "\t\tdgid....................0x%016" PRIx64 " : " >> "0x%016" PRIx64 "\n" >> "\t\tsgid....................0x%016" PRIx64 " : " >> @@ -199,7 +199,7 @@ print_path_record(ib_path_rec_t *p_pr) >> "\t\ttclass..................0x%X\n" >> "\t\tnum_path_revers.........0x%X\n" >> "\t\tpkey....................0x%X\n" >> - "\t\tsl......................0x%X\n" >> + "\t\tqos_class_sl............0x%X\n" > > I think it is more useful to have separate printouts for sl and > qos_class. Agree >> "\t\tmtu.....................0x%X\n" >> "\t\trate....................0x%X\n" >> "\t\tpkt_life................0x%X\n" >> @@ -207,7 +207,7 @@ print_path_record(ib_path_rec_t *p_pr) >> "\t\tresv2...................0x%X\n" >> "\t\tresv3...................0x%X\n" >> "", >> - *(uint64_t*)p_pr->resv0, >> + cl_ntoh64( p_pr->service_id ), >> cl_ntoh64( p_pr->dgid.unicast.prefix ), >> cl_ntoh64( p_pr->dgid.unicast.interface_id ), >> cl_ntoh64( p_pr->sgid.unicast.prefix ), >> @@ -218,7 +218,7 @@ print_path_record(ib_path_rec_t *p_pr) >> p_pr->tclass, >> p_pr->num_path, >> cl_ntoh16( p_pr->pkey ), >> - cl_ntoh16( p_pr->sl ), >> + cl_ntoh16( p_pr->qos_class_sl ), >> p_pr->mtu, >> p_pr->rate, >> p_pr->pkt_life, >> diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h >> index 490e3fc..f96893a 100644 >> --- a/opensm/include/iba/ib_types.h >> +++ b/opensm/include/iba/ib_types.h >> @@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN const uint8_t class_code) >> #define IB_SMINFO_STATE_MASTER 3 >> /**********/ >> >> +/****d* IBA Base: Constants/IB_PATH_REC_SL_MASK >> +* NAME >> +* IB_PATH_REC_SL_MASK >> +* >> +* DESCRIPTION >> +* Mask for the sl field for path record >> +* >> +* SOURCE >> +*/ >> +#define IB_PATH_REC_SL_MASK 0x000F >> + >> +/****d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK >> +* NAME >> +* IB_PATH_REC_QOS_CLASS_MASK >> +* >> +* DESCRIPTION >> +* Mask for the QoS class field for path record >> +* >> +* SOURCE >> +*/ >> +#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0 >> + >> /****d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK >> * NAME >> * IB_PATH_REC_SELECTOR_MASK >> @@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid) >> */ >> #include >> typedef struct _ib_path_rec { >> - uint8_t resv0[8]; >> + ib_net64_t service_id; >> ib_gid_t dgid; >> ib_gid_t sgid; >> ib_net16_t dlid; >> @@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec { >> uint8_t tclass; >> uint8_t num_path; >> ib_net16_t pkey; >> - ib_net16_t sl; >> + ib_net16_t qos_class_sl; >> uint8_t mtu; >> uint8_t rate; >> uint8_t pkt_life; >> @@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec { >> #include >> /* >> * FIELDS >> -* resv0 >> -* Reserved bytes. >> +* service_id >> +* Service ID for QoS. >> * >> * dgid >> * GID of destination port. >> @@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec { >> * pkey >> * Partition key (P_Key) to use on this path. >> * >> -* resv1 >> -* Reserved byte. >> -* >> -* sl >> -* Service level to use on this path. >> +* qos_class_sl >> +* QoS class and service level to use on this path. >> * >> * mtu >> * MTU and MTU selector fields to use on this path >> @@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec { >> *********/ >> >> /* Path Record Component Masks */ >> +#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)<<1)) >> #define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)<<2)) >> #define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)<<3)) >> #define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)<<4)) >> @@ -2372,7 +2392,7 @@ typedef struct _ib_path_rec { >> #define IB_PR_COMPMASK_REVERSIBLE (CL_HTON64(((uint64_t)1)<<11)) >> #define IB_PR_COMPMASK_NUMBPATH (CL_HTON64(((uint64_t)1)<<12)) >> #define IB_PR_COMPMASK_PKEY (CL_HTON64(((uint64_t)1)<<13)) >> -#define IB_PR_COMPMASK_RESV1 (CL_HTON64(((uint64_t)1)<<14)) >> +#define IB_PR_COMPMASK_QOS_CLASS (CL_HTON64(((uint64_t)1)<<14)) >> #define IB_PR_COMPMASK_SL (CL_HTON64(((uint64_t)1)<<15)) >> #define IB_PR_COMPMASK_MTUSELEC (CL_HTON64(((uint64_t)1)<<16)) >> #define IB_PR_COMPMASK_MTU (CL_HTON64(((uint64_t)1)<<17)) >> @@ -2630,6 +2650,7 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, >> IN uint8_t num_path, >> IN ib_net16_t pkey, >> IN uint8_t sl, >> + IN uint16_t qos_class, >> IN uint8_t mtu_selector, >> IN uint8_t mtu, >> IN uint8_t rate_selector, >> @@ -2643,8 +2664,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, >> p_rec->slid = slid; >> p_rec->num_path = num_path; >> p_rec->pkey = pkey; >> - /* Lower 4 bits of path rec's SL are reserved. */ >> - p_rec->sl = cl_ntoh16(sl); >> + p_rec->qos_class_sl = cl_hton16( (sl & IB_PATH_REC_SL_MASK) | >> + (qos_class << 4) ); >> p_rec->mtu = (uint8_t) ((mtu & IB_PATH_REC_BASE_MASK) | >> (uint8_t) (mtu_selector << 6)); >> p_rec->rate = (uint8_t) ((rate & IB_PATH_REC_BASE_MASK) | >> @@ -2656,8 +2677,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, >> /* Clear global routing fields for local path records */ >> p_rec->hop_flow_raw = 0; >> p_rec->tclass = 0; >> + p_rec->service_id = 0; >> >> - *((uint64_t *) p_rec->resv0) = 0; >> *((uint32_t *) p_rec->resv2) = 0; >> *((uint16_t *) p_rec->resv2 + 2) = 0; >> } >> @@ -2687,6 +2708,9 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, >> * pkey >> * [in] Partition key (P_Key) to use on this path. >> * >> +* qos_class >> +* [in] QoS class to use on this path. Lower 12-bits are valid. >> +* >> * sl >> * [in] Service level to use on this path. Lower 4-bits are valid. >> * >> @@ -2750,6 +2774,41 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) >> * ib_path_rec_t >> *********/ >> >> +/****f* IBA Base: Types/ib_path_rec_set_sl >> +* NAME >> +* ib_path_rec_set_sl >> +* >> +* DESCRIPTION >> +* Set path service level. >> +* >> +* SYNOPSIS >> +*/ >> +static inline void OSM_API >> +ib_path_rec_set_sl( >> + IN ib_path_rec_t* const p_rec, >> + IN const uint8_t sl ) >> +{ >> + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & >> + IB_PATH_REC_QOS_CLASS_MASK ) | >> + ( sl & IB_PATH_REC_SL_MASK) ); > > You could do something like: > > (p_rec->qos_class_sl & CL_HTON16(IB_PATH_REC_QOS_CLASS_MASK)) | > cl_hton16(sl & IB_PATH_REC_SL_MASK) Sure, why not > CL_HTON16(IB_PATH_REC_QOS_CLASS_MASK) is calculated at compile time, so > one cl_hton16() is saved. > >> +} >> +/* >> +* PARAMETERS >> +* p_rec >> +* [in] Pointer to the path record object. >> +* >> +* sl >> +* [in] Service level to set. >> +* >> +* RETURN VALUES >> +* None >> +* >> +* NOTES >> +* >> +* SEE ALSO >> +* ib_path_rec_t >> +*********/ >> + >> /****f* IBA Base: Types/ib_path_rec_sl >> * NAME >> * ib_path_rec_sl >> @@ -2762,7 +2821,7 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) >> static inline uint8_t OSM_API >> ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) >> { >> - return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); >> + return ((uint8_t) ((cl_ntoh16(p_rec->qos_class_sl)) & IB_PATH_REC_SL_MASK)); >> } >> >> /* >> @@ -2779,6 +2838,70 @@ ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) >> * ib_path_rec_t >> *********/ >> >> +/****f* IBA Base: Types/ib_path_rec_set_qos_class >> +* NAME >> +* ib_path_rec_set_qos_class >> +* >> +* DESCRIPTION >> +* Set path QoS class. >> +* >> +* SYNOPSIS >> +*/ >> +static inline void OSM_API >> +ib_path_rec_set_qos_class( >> + IN ib_path_rec_t* const p_rec, >> + IN const uint16_t qos_class ) >> +{ >> + p_rec->qos_class_sl = cl_hton16( ( cl_ntoh16(p_rec->qos_class_sl) & >> + IB_PATH_REC_SL_MASK ) | >> + ( qos_class << 4) ); > > Similar to above. Right >> +} >> +/* >> +* PARAMETERS >> +* p_rec >> +* [in] Pointer to the path record object. >> +* >> +* qos_class >> +* [in] QoS class to set. >> +* >> +* RETURN VALUES >> +* None >> +* >> +* NOTES >> +* >> +* SEE ALSO >> +* ib_path_rec_t >> +*********/ >> + >> +/****f* IBA Base: Types/ib_path_rec_qos_class >> +* NAME >> +* ib_path_rec_qos_class >> +* >> +* DESCRIPTION >> +* Get QoS class. >> +* >> +* SYNOPSIS >> +*/ >> +static inline uint16_t OSM_API >> +ib_path_rec_qos_class( >> + IN const ib_path_rec_t* const p_rec ) >> +{ >> + return (cl_ntoh16( p_rec->qos_class_sl ) >> 4); >> +} >> +/* >> +* PARAMETERS >> +* p_rec >> +* [in] Pointer to the path record object. >> +* >> +* RETURN VALUES >> +* QoS class of the path record. >> +* >> +* NOTES >> +* >> +* SEE ALSO >> +* ib_path_rec_t >> +*********/ >> + >> /****f* IBA Base: Types/ib_path_rec_mtu >> * NAME >> * ib_path_rec_mtu >> @@ -2940,7 +3063,7 @@ ib_path_rec_pkt_life(IN const ib_path_rec_t * const p_rec) >> * [in] Pointer to the path record object. >> * >> * RETURN VALUES >> -* Encoded path pkt_life = 4.096 ?sec * 2 ** PacketLifeTime. >> +* Encoded path pkt_life = 4.096 ??sec * 2 ** PacketLifeTime. > > It is not ascii. What is the change here? I think my diff tool has troubles handling unusual chars. There's a 'µ' character there - it wasn't changed. I guess it would be better to replace it with "micro" in the future - less elegant, but also less error prone. >> * >> * NOTES >> * >> @@ -2988,7 +3111,8 @@ ib_path_rec_pkt_life_sel(IN const ib_path_rec_t * const p_rec) >> * DESCRIPTION >> * Get flow label. >> * >> -* SYNOPSIS >> +* SYNOPSIS p_rec->tclass = 0; > > What do you mean? Beats me... >> + >> */ >> static inline uint32_t OSM_API >> ib_path_rec_flow_lbl(IN const ib_path_rec_t * const p_rec) >> @@ -5999,7 +6123,7 @@ ib_multipath_rec_pkt_life(IN const ib_multipath_rec_t * const p_rec) >> * [in] Pointer to the multipath record object. >> * >> * RETURN VALUES >> -* Encoded multipath pkt_life = 4.096 ?sec * 2 ** PacketLifeTime. >> +* Encoded multipath pkt_life = 4.096 ??sec * 2 ** PacketLifeTime. > > ascii? Same as above >> * >> * NOTES >> * >> diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c >> index c428823..04e52a8 100644 >> --- a/opensm/opensm/osm_helper.c >> +++ b/opensm/opensm/osm_helper.c >> @@ -1044,7 +1044,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, >> if (osm_log_is_active(p_log, log_level)) { >> osm_log(p_log, log_level, >> "PathRecord dump:\n" >> - "\t\t\t\tresv0...................0x%016" PRIx64 "\n" >> + "\t\t\t\tservice_id..............0x%016" PRIx64 "\n" >> "\t\t\t\tdgid....................0x%016" PRIx64 " : " >> "0x%016" PRIx64 "\n" >> "\t\t\t\tsgid....................0x%016" PRIx64 " : " >> @@ -1055,6 +1055,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, >> "\t\t\t\ttclass..................0x%X\n" >> "\t\t\t\tnum_path_revers.........0x%X\n" >> "\t\t\t\tpkey....................0x%X\n" >> + "\t\t\t\tqos_class...............0x%X\n" >> "\t\t\t\tsl......................0x%X\n" >> "\t\t\t\tmtu.....................0x%X\n" >> "\t\t\t\trate....................0x%X\n" >> @@ -1063,7 +1064,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, >> "\t\t\t\tresv2...................0x%X\n" >> "\t\t\t\tresv3...................0x%X\n" >> "", >> - *(uint64_t *) p_pr->resv0, >> + cl_ntoh64(p_pr->service_id), >> cl_ntoh64(p_pr->dgid.unicast.prefix), >> cl_ntoh64(p_pr->dgid.unicast.interface_id), >> cl_ntoh64(p_pr->sgid.unicast.prefix), >> @@ -1074,7 +1075,8 @@ osm_dump_path_record(IN osm_log_t * const p_log, >> p_pr->tclass, >> p_pr->num_path, >> cl_ntoh16(p_pr->pkey), >> - cl_ntoh16(p_pr->sl), >> + ib_path_rec_qos_class(p_pr), >> + ib_path_rec_sl(p_pr), >> p_pr->mtu, >> p_pr->rate, >> p_pr->pkt_life, >> diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c >> index b3f42e2..3c79de7 100644 >> --- a/opensm/opensm/osm_sa_multipath_record.c >> +++ b/opensm/opensm/osm_sa_multipath_record.c >> @@ -724,7 +724,7 @@ __osm_mpr_rcv_build_pr(IN osm_mpr_rcv_t * const p_rcv, >> p_pr->hop_flow_raw &= cl_hton32(1 << 31); >> >> p_pr->pkey = p_parms->pkey; >> - p_pr->sl = cl_hton16(p_parms->sl); >> + ib_path_rec_set_sl(p_pr, p_parms->sl); > > Now when ib_path_rec_set_sl() sets only low 4 bits (and all 16 as > before), shouldn't upper 12 bits (qos_class) be initialized here? Right >> p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); >> p_pr->rate = (uint8_t) (p_parms->rate | 0x80); >> >> diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c >> index 602fd2a..e8b24d0 100644 >> --- a/opensm/opensm/osm_sa_path_record.c >> +++ b/opensm/opensm/osm_sa_path_record.c >> @@ -740,7 +740,7 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv, >> #endif >> >> p_pr->pkey = p_parms->pkey; >> - p_pr->sl = cl_hton16(p_parms->sl); >> + ib_path_rec_set_sl(p_pr, p_parms->sl); > > Ditto. OK >> p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); >> p_pr->rate = (uint8_t) (p_parms->rate | 0x80); >> >> @@ -1968,7 +1968,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data) >> mcmember_rec. >> sl_flow_hop, >> &sl, &flow_label, &hop_limit); >> - p_pr_item->path_rec.sl = cl_hton16(sl); >> + ib_path_rec_set_sl(&p_pr_item->path_rec, sl); > > Ditto. OK >> #ifndef ROUTER_EXP >> p_pr_item->path_rec.hop_flow_raw = >> cl_hton32(hop_limit) | (flow_label << 8); >> diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c >> index 36cb825..5bf11ea 100644 >> --- a/opensm/osmtest/osmtest.c >> +++ b/opensm/osmtest/osmtest.c >> @@ -1918,7 +1918,7 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, >> cl_ntoh64(p_rec->sgid.unicast.interface_id), >> cl_ntoh16(p_rec->dlid), cl_ntoh16(p_rec->slid), >> cl_ntoh32(p_rec->hop_flow_raw), p_rec->tclass, >> - p_rec->num_path, cl_ntoh16(p_rec->pkey), p_rec->sl, >> + p_rec->num_path, cl_ntoh16(p_rec->pkey), ib_path_rec_sl(p_rec), > > Why to not print qos_class too? Good idea. >> p_rec->mtu, p_rec->rate, p_rec->pkt_life, >> p_rec->preference); >> >> -- >> 1.5.1.4 >> > > Also it would be nice if newly added code will be formatted with > opensm/osm_indent. Thanks. To tell you the truth, I'm not a big fan of this script. Sometimes it does stuff it shouldn't. For instance, I run it on osm_qos_policy.c, and this is what it does: Before: static osm_qos_match_rule_t * __qos_policy_get_match_rule_by_pr( const osm_pr_rcv_t * p_rcv, const ib_path_rec_t * p_pr, const osm_physp_t * p_src_physp, const osm_physp_t * p_dest_physp, ib_net64_t comp_mask) After: static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr(const osm_pr_rcv_t * p_rcv, const ib_path_rec_t * p_pr, const osm_physp_t * p_src_physp, const osm_physp_t * p_dest_physp, ib_net64_t comp_mask) Correct me if I'm wrong, but I think that the 'before' example is closer to our coding style than the 'after'. But you're right in general. It affects only patch 3/7 - adding QoS policy c & h files. I'll repost this patch with rest of the fixed patches. -- Yevgeny > Sasha > From ahsma008 at planet.nl Tue Aug 21 04:22:39 2007 From: ahsma008 at planet.nl (ahsma008 at planet.nl) Date: Tue, 21 Aug 2007 13:22:39 +0200 Subject: [ofa-general] WINNING NOTIFICATION: Message-ID: UK Lottery Headquarters: Customer Service 28 TANFIELD ROAD, CROYDON.LONDON Ref: BTL/491OXI/04 Batch: 12/25/0304 WINNING NOTIFICATION: Dear E-mail User This E mail is to inform you that you emerged a winner on our online draw which was drawn on the 6th August, 2007 from a world wide reference of existing e-mail addresses.You Have Won £1,850,000.00 (One Million Eight hundred & Fifty Thousand Pounds sterling). For further Information about your winnings, contact your Fiduciary Agent with the information below: Name: WILLIAMS BLAKE, Telephone: (+44) 7045 7068 75 e-mail address: fiduciaryagentwilliamsblake007 at yahoo.co.uk Warm Regards, Mr. Mark Richards Online coordinator -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Tue Aug 21 04:44:58 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 21 Aug 2007 14:44:58 +0300 Subject: [ofa-general] [PATCH 1/7 V2] osm: QoS - adding new PathRecord fields In-Reply-To: <46C982FA.2040601@dev.mellanox.co.il> References: <46C982FA.2040601@dev.mellanox.co.il> Message-ID: <46CAD03A.9060004@dev.mellanox.co.il> Adding QoS fields to PathRecord. Signed-off-by: Yevgeny Kliteynik --- infiniband-diags/src/saquery.c | 8 +- opensm/include/iba/ib_types.h | 149 ++++++++++++++++++++++++++++--- opensm/opensm/osm_helper.c | 8 +- opensm/opensm/osm_sa_multipath_record.c | 3 +- opensm/opensm/osm_sa_path_record.c | 6 +- opensm/osmtest/osmtest.c | 4 +- 6 files changed, 154 insertions(+), 24 deletions(-) diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c index 522399e..a9a8da4 100644 --- a/infiniband-diags/src/saquery.c +++ b/infiniband-diags/src/saquery.c @@ -188,7 +188,7 @@ static void print_path_record(ib_path_rec_t *p_pr) { printf("PathRecord dump:\n" - "\t\tresv0...................0x%016" PRIx64 "\n" + "\t\tservice_id..............0x%016" PRIx64 "\n" "\t\tdgid....................0x%016" PRIx64 " : " "0x%016" PRIx64 "\n" "\t\tsgid....................0x%016" PRIx64 " : " @@ -199,6 +199,7 @@ print_path_record(ib_path_rec_t *p_pr) "\t\ttclass..................0x%X\n" "\t\tnum_path_revers.........0x%X\n" "\t\tpkey....................0x%X\n" + "\t\tqos_class...............0x%X\n" "\t\tsl......................0x%X\n" "\t\tmtu.....................0x%X\n" "\t\trate....................0x%X\n" @@ -207,7 +208,7 @@ print_path_record(ib_path_rec_t *p_pr) "\t\tresv2...................0x%X\n" "\t\tresv3...................0x%X\n" "", - *(uint64_t*)p_pr->resv0, + cl_ntoh64( p_pr->service_id ), cl_ntoh64( p_pr->dgid.unicast.prefix ), cl_ntoh64( p_pr->dgid.unicast.interface_id ), cl_ntoh64( p_pr->sgid.unicast.prefix ), @@ -218,7 +219,8 @@ print_path_record(ib_path_rec_t *p_pr) p_pr->tclass, p_pr->num_path, cl_ntoh16( p_pr->pkey ), - cl_ntoh16( p_pr->sl ), + ib_path_rec_qos_class( p_pr ), + ib_path_rec_sl( p_pr ), p_pr->mtu, p_pr->rate, p_pr->pkt_life, diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h index 490e3fc..ada4428 100644 --- a/opensm/include/iba/ib_types.h +++ b/opensm/include/iba/ib_types.h @@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN const uint8_t class_code) #define IB_SMINFO_STATE_MASTER 3 /**********/ +/****d* IBA Base: Constants/IB_PATH_REC_SL_MASK +* NAME +* IB_PATH_REC_SL_MASK +* +* DESCRIPTION +* Mask for the sl field for path record +* +* SOURCE +*/ +#define IB_PATH_REC_SL_MASK 0x000F + +/****d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK +* NAME +* IB_PATH_REC_QOS_CLASS_MASK +* +* DESCRIPTION +* Mask for the QoS class field for path record +* +* SOURCE +*/ +#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0 + /****d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK * NAME * IB_PATH_REC_SELECTOR_MASK @@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid) */ #include typedef struct _ib_path_rec { - uint8_t resv0[8]; + ib_net64_t service_id; ib_gid_t dgid; ib_gid_t sgid; ib_net16_t dlid; @@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec { uint8_t tclass; uint8_t num_path; ib_net16_t pkey; - ib_net16_t sl; + ib_net16_t qos_class_sl; uint8_t mtu; uint8_t rate; uint8_t pkt_life; @@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec { #include /* * FIELDS -* resv0 -* Reserved bytes. +* service_id +* Service ID for QoS. * * dgid * GID of destination port. @@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec { * pkey * Partition key (P_Key) to use on this path. * -* resv1 -* Reserved byte. -* -* sl -* Service level to use on this path. +* qos_class_sl +* QoS class and service level to use on this path. * * mtu * MTU and MTU selector fields to use on this path @@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec { *********/ /* Path Record Component Masks */ +#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)<<1)) #define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)<<2)) #define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)<<3)) #define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)<<4)) @@ -2372,7 +2392,7 @@ typedef struct _ib_path_rec { #define IB_PR_COMPMASK_REVERSIBLE (CL_HTON64(((uint64_t)1)<<11)) #define IB_PR_COMPMASK_NUMBPATH (CL_HTON64(((uint64_t)1)<<12)) #define IB_PR_COMPMASK_PKEY (CL_HTON64(((uint64_t)1)<<13)) -#define IB_PR_COMPMASK_RESV1 (CL_HTON64(((uint64_t)1)<<14)) +#define IB_PR_COMPMASK_QOS_CLASS (CL_HTON64(((uint64_t)1)<<14)) #define IB_PR_COMPMASK_SL (CL_HTON64(((uint64_t)1)<<15)) #define IB_PR_COMPMASK_MTUSELEC (CL_HTON64(((uint64_t)1)<<16)) #define IB_PR_COMPMASK_MTU (CL_HTON64(((uint64_t)1)<<17)) @@ -2630,6 +2650,7 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, IN uint8_t num_path, IN ib_net16_t pkey, IN uint8_t sl, + IN uint16_t qos_class, IN uint8_t mtu_selector, IN uint8_t mtu, IN uint8_t rate_selector, @@ -2643,8 +2664,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, p_rec->slid = slid; p_rec->num_path = num_path; p_rec->pkey = pkey; - /* Lower 4 bits of path rec's SL are reserved. */ - p_rec->sl = cl_ntoh16(sl); + p_rec->qos_class_sl = cl_hton16( (sl & IB_PATH_REC_SL_MASK) | + (qos_class << 4) ); p_rec->mtu = (uint8_t) ((mtu & IB_PATH_REC_BASE_MASK) | (uint8_t) (mtu_selector << 6)); p_rec->rate = (uint8_t) ((rate & IB_PATH_REC_BASE_MASK) | @@ -2656,8 +2677,8 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, /* Clear global routing fields for local path records */ p_rec->hop_flow_raw = 0; p_rec->tclass = 0; + p_rec->service_id = 0; - *((uint64_t *) p_rec->resv0) = 0; *((uint32_t *) p_rec->resv2) = 0; *((uint16_t *) p_rec->resv2 + 2) = 0; } @@ -2687,6 +2708,9 @@ ib_path_rec_init_local(IN ib_path_rec_t * const p_rec, * pkey * [in] Partition key (P_Key) to use on this path. * +* qos_class +* [in] QoS class to use on this path. Lower 12-bits are valid. +* * sl * [in] Service level to use on this path. Lower 4-bits are valid. * @@ -2750,6 +2774,40 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) * ib_path_rec_t *********/ +/****f* IBA Base: Types/ib_path_rec_set_sl +* NAME +* ib_path_rec_set_sl +* +* DESCRIPTION +* Set path service level. +* +* SYNOPSIS +*/ +static inline void OSM_API +ib_path_rec_set_sl( + IN ib_path_rec_t* const p_rec, + IN const uint8_t sl ) +{ + p_rec->qos_class_sl = (p_rec->qos_class_sl & CL_HTON16(IB_PATH_REC_QOS_CLASS_MASK)) | + cl_hton16(sl & IB_PATH_REC_SL_MASK); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* sl +* [in] Service level to set. +* +* RETURN VALUES +* None +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + /****f* IBA Base: Types/ib_path_rec_sl * NAME * ib_path_rec_sl @@ -2762,7 +2820,7 @@ ib_path_rec_num_path(IN const ib_path_rec_t * const p_rec) static inline uint8_t OSM_API ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) { - return ((uint8_t) ((cl_ntoh16(p_rec->sl)) & 0xF)); + return ((uint8_t) ((cl_ntoh16(p_rec->qos_class_sl)) & IB_PATH_REC_SL_MASK)); } /* @@ -2779,6 +2837,69 @@ ib_path_rec_sl(IN const ib_path_rec_t * const p_rec) * ib_path_rec_t *********/ +/****f* IBA Base: Types/ib_path_rec_set_qos_class +* NAME +* ib_path_rec_set_qos_class +* +* DESCRIPTION +* Set path QoS class. +* +* SYNOPSIS +*/ +static inline void OSM_API +ib_path_rec_set_qos_class( + IN ib_path_rec_t* const p_rec, + IN const uint16_t qos_class ) +{ + p_rec->qos_class_sl = (p_rec->qos_class_sl & CL_HTON16(IB_PATH_REC_SL_MASK)) | + cl_hton16(qos_class << 4); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* qos_class +* [in] QoS class to set. +* +* RETURN VALUES +* None +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + +/****f* IBA Base: Types/ib_path_rec_qos_class +* NAME +* ib_path_rec_qos_class +* +* DESCRIPTION +* Get QoS class. +* +* SYNOPSIS +*/ +static inline uint16_t OSM_API +ib_path_rec_qos_class( + IN const ib_path_rec_t* const p_rec ) +{ + return (cl_ntoh16( p_rec->qos_class_sl ) >> 4); +} +/* +* PARAMETERS +* p_rec +* [in] Pointer to the path record object. +* +* RETURN VALUES +* QoS class of the path record. +* +* NOTES +* +* SEE ALSO +* ib_path_rec_t +*********/ + /****f* IBA Base: Types/ib_path_rec_mtu * NAME * ib_path_rec_mtu diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index c428823..04e52a8 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -1044,7 +1044,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, if (osm_log_is_active(p_log, log_level)) { osm_log(p_log, log_level, "PathRecord dump:\n" - "\t\t\t\tresv0...................0x%016" PRIx64 "\n" + "\t\t\t\tservice_id..............0x%016" PRIx64 "\n" "\t\t\t\tdgid....................0x%016" PRIx64 " : " "0x%016" PRIx64 "\n" "\t\t\t\tsgid....................0x%016" PRIx64 " : " @@ -1055,6 +1055,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, "\t\t\t\ttclass..................0x%X\n" "\t\t\t\tnum_path_revers.........0x%X\n" "\t\t\t\tpkey....................0x%X\n" + "\t\t\t\tqos_class...............0x%X\n" "\t\t\t\tsl......................0x%X\n" "\t\t\t\tmtu.....................0x%X\n" "\t\t\t\trate....................0x%X\n" @@ -1063,7 +1064,7 @@ osm_dump_path_record(IN osm_log_t * const p_log, "\t\t\t\tresv2...................0x%X\n" "\t\t\t\tresv3...................0x%X\n" "", - *(uint64_t *) p_pr->resv0, + cl_ntoh64(p_pr->service_id), cl_ntoh64(p_pr->dgid.unicast.prefix), cl_ntoh64(p_pr->dgid.unicast.interface_id), cl_ntoh64(p_pr->sgid.unicast.prefix), @@ -1074,7 +1075,8 @@ osm_dump_path_record(IN osm_log_t * const p_log, p_pr->tclass, p_pr->num_path, cl_ntoh16(p_pr->pkey), - cl_ntoh16(p_pr->sl), + ib_path_rec_qos_class(p_pr), + ib_path_rec_sl(p_pr), p_pr->mtu, p_pr->rate, p_pr->pkt_life, diff --git a/opensm/opensm/osm_sa_multipath_record.c b/opensm/opensm/osm_sa_multipath_record.c index b3f42e2..889d7c6 100644 --- a/opensm/opensm/osm_sa_multipath_record.c +++ b/opensm/opensm/osm_sa_multipath_record.c @@ -724,7 +724,8 @@ __osm_mpr_rcv_build_pr(IN osm_mpr_rcv_t * const p_rcv, p_pr->hop_flow_raw &= cl_hton32(1 << 31); p_pr->pkey = p_parms->pkey; - p_pr->sl = cl_hton16(p_parms->sl); + ib_path_rec_set_qos_class(p_pr, 0); + ib_path_rec_set_sl(p_pr, p_parms->sl); p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); p_pr->rate = (uint8_t) (p_parms->rate | 0x80); diff --git a/opensm/opensm/osm_sa_path_record.c b/opensm/opensm/osm_sa_path_record.c index 602fd2a..1b781f0 100644 --- a/opensm/opensm/osm_sa_path_record.c +++ b/opensm/opensm/osm_sa_path_record.c @@ -740,7 +740,8 @@ __osm_pr_rcv_build_pr(IN osm_pr_rcv_t * const p_rcv, #endif p_pr->pkey = p_parms->pkey; - p_pr->sl = cl_hton16(p_parms->sl); + ib_path_rec_set_sl(p_pr, p_parms->sl); + ib_path_rec_set_qos_class(p_pr, 0); p_pr->mtu = (uint8_t) (p_parms->mtu | 0x80); p_pr->rate = (uint8_t) (p_parms->rate | 0x80); @@ -1968,7 +1969,8 @@ void osm_pr_rcv_process(IN void *context, IN void *data) mcmember_rec. sl_flow_hop, &sl, &flow_label, &hop_limit); - p_pr_item->path_rec.sl = cl_hton16(sl); + ib_path_rec_set_sl(&p_pr_item->path_rec, sl); + ib_path_rec_set_qos_class(&p_pr_item->path_rec, 0); #ifndef ROUTER_EXP p_pr_item->path_rec.hop_flow_raw = cl_hton32(hop_limit) | (flow_label << 8); diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c index 36cb825..73933a3 100644 --- a/opensm/osmtest/osmtest.c +++ b/opensm/osmtest/osmtest.c @@ -1908,6 +1908,7 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, "# num_path 0x%X\n" "pkey 0x%X\n" "# sl 0x%X\n" + "# qos_class 0x%X\n" "# mtu 0x%X\n" "# rate 0x%X\n" "# pkt_life 0x%X\n" @@ -1918,7 +1919,8 @@ osmtest_write_path_info(IN osmtest_t * const p_osmt, cl_ntoh64(p_rec->sgid.unicast.interface_id), cl_ntoh16(p_rec->dlid), cl_ntoh16(p_rec->slid), cl_ntoh32(p_rec->hop_flow_raw), p_rec->tclass, - p_rec->num_path, cl_ntoh16(p_rec->pkey), p_rec->sl, + p_rec->num_path, cl_ntoh16(p_rec->pkey), + ib_path_rec_sl(p_rec), ib_path_rec_qos_class(p_rec), p_rec->mtu, p_rec->rate, p_rec->pkt_life, p_rec->preference); -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Aug 21 04:46:36 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 21 Aug 2007 14:46:36 +0300 Subject: [ofa-general] [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <46C983BB.4060007@dev.mellanox.co.il> References: <46C983BB.4060007@dev.mellanox.co.il> Message-ID: <46CAD09C.4030400@dev.mellanox.co.il> QoS policy data structures and functions Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_qos_policy.h | 188 +++++++ opensm/opensm/osm_qos_policy.c | 901 ++++++++++++++++++++++++++++++++ 2 files changed, 1089 insertions(+), 0 deletions(-) create mode 100644 opensm/include/opensm/osm_qos_policy.h create mode 100644 opensm/opensm/osm_qos_policy.c diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h new file mode 100644 index 0000000..05b8dc9 --- /dev/null +++ b/opensm/include/opensm/osm_qos_policy.h @@ -0,0 +1,188 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Declaration of OSM QoS Policy data types and functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#ifndef OSM_QOS_POLICY_H +#define OSM_QOS_POLICY_H + +#include +#include +#include +#include +#include +#include +#include +#include + +#define YYSTYPE char * +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" + +/***************************************************/ + +typedef struct osm_qos_port_group_t_ { + char *name; /* single string (this port group name) */ + char *use; /* single string (description) */ + cl_list_t port_name_list; /* list of port names (.../.../...) */ + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ + unsigned guid_range_len; /* num of guid ranges in the array */ + cl_list_t partition_list; /* list of partition names */ + boolean_t node_type_ca; + boolean_t node_type_switch; + boolean_t node_type_router; + boolean_t node_type_self; +} osm_qos_port_group_t; + +osm_qos_port_group_t *osm_qos_policy_port_group_create(); +void osm_qos_policy_port_group_destroy(); + +/***************************************************/ + +typedef struct osm_qos_vlarb_scope_t_ { + cl_list_t group_list; /* list of group names (strings) */ + cl_list_t across_list; /* list of 'across' group names (strings) */ + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ + uint32_t vl_high_limit; /* single integer */ + boolean_t vl_high_limit_set; +} osm_qos_vlarb_scope_t; + +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); +void osm_qos_policy_vlarb_scope_destroy(); + +/***************************************************/ + +typedef struct osm_qos_sl2vl_scope_t_ { + cl_list_t group_list; /* list of strings (port group names) */ + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + cl_list_t across_from_list; /* list of strings (port group names) */ + cl_list_t across_to_list; /* list of strings (port group names) */ + uint8_t sl2vl_table[16]; /* array of sl2vl values */ + boolean_t sl2vl_table_set; +} osm_qos_sl2vl_scope_t; + +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); +void osm_qos_policy_sl2vl_scope_destroy(); + +/***************************************************/ + +typedef struct osm_qos_level_t_ { + char *use; + char *name; + uint8_t sl; + boolean_t sl_set; + uint8_t mtu_limit; + boolean_t mtu_limit_set; + uint8_t rate_limit; + boolean_t rate_limit_set; + uint8_t pkt_life; + boolean_t pkt_life_set; + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ + unsigned path_bits_range_len; /* num of bit ranges in the array */ + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_level_t; + +osm_qos_level_t *osm_qos_policy_qos_level_create(); +void osm_qos_policy_qos_level_destroy(); + +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey); + +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp); + +/***************************************************/ + +typedef struct osm_qos_match_rule_t_ { + char *use; + cl_list_t source_list; /* list of strings */ + cl_list_t source_group_list; /* list of pointers to relevant port-group */ + cl_list_t destination_list; /* list of strings */ + cl_list_t destination_group_list; /* list of pointers to relevant port-group */ + char *qos_level_name; + osm_qos_level_t *p_qos_level; + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) */ + unsigned service_id_range_len; + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ + unsigned qos_class_range_len; + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_match_rule_t; + +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); +void osm_qos_policy_match_rule_destroy(); + +/***************************************************/ + +typedef struct osm_qos_policy_t_ { + cl_list_t port_groups; /* list of osm_qos_port_group_t */ + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ + cl_list_t qos_levels; /* list of osm_qos_level_t */ + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ + osm_qos_level_t *p_default_qos_level; /* default QoS level */ +} osm_qos_policy_t; + +void osm_qos_policy_create(); +void osm_qos_policy_destroy(); +int osm_qos_policy_validate(); + +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level); + +/***************************************************/ + +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char *policy_file); + +/***************************************************/ + +#endif /* ifndef OSM_QOS_POLICY_H */ diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c new file mode 100644 index 0000000..bc2aa68 --- /dev/null +++ b/opensm/opensm/osm_qos_policy.c @@ -0,0 +1,901 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * OSM QoS Policy functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include +#include + +extern void yyerror(char *s); +osm_log_t *p_qos_parser_osm_log = NULL; +osm_qos_policy_t *p_qos_policy = NULL; + +/*************************************************** + ***************************************************/ + +static boolean_t +__is_num_in_range_arr(uint64_t ** range_arr, + unsigned range_arr_len, uint64_t num) +{ + unsigned i; + + /* + * TODO: although this array should be rather short, + * I should implement a binary search here + */ + + if (num < range_arr[0][0] || num > range_arr[range_arr_len - 1][1]) + return FALSE; + + for (i = 0; i < range_arr_len; i++) { + if (num >= range_arr[i][0] && num <= range_arr[i][1]) + return TRUE; + if (num < range_arr[i][0]) + return FALSE; + } + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static void __free_single_element(void *p_element, void *context) +{ + if (p_element) + free(p_element); +} + +/*************************************************** + ***************************************************/ + +osm_qos_port_group_t *osm_qos_policy_port_group_create() +{ + osm_qos_port_group_t *p = + (osm_qos_port_group_t *) malloc(sizeof(osm_qos_port_group_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_port_group_t)); + + cl_list_init(&p->port_name_list, 10); + cl_list_init(&p->partition_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->name) + free(p->name); + if (p->use) + free(p->use); + + for (i = 0; i < p->guid_range_len; i++) + free(p->guid_range_arr[i]); + if (p->guid_range_arr) + free(p->guid_range_arr); + + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); + cl_list_remove_all(&p->port_name_list); + cl_list_destroy(&p->port_name_list); + + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); + cl_list_remove_all(&p->partition_list); + cl_list_destroy(&p->partition_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create() +{ + osm_qos_vlarb_scope_t *p = + (osm_qos_vlarb_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list, 10); + cl_list_init(&p->across_list, 10); + cl_list_init(&p->vlarb_high_list, 10); + cl_list_init(&p->vlarb_low_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_list); + cl_list_remove_all(&p->vlarb_high_list); + cl_list_remove_all(&p->vlarb_low_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_list); + cl_list_destroy(&p->vlarb_high_list); + cl_list_destroy(&p->vlarb_low_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create() +{ + osm_qos_sl2vl_scope_t *p = + (osm_qos_sl2vl_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list, 10); + cl_list_init(&p->across_from_list, 10); + cl_list_init(&p->across_to_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_from_list); + cl_list_remove_all(&p->across_to_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_from_list); + cl_list_destroy(&p->across_to_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_level_t *osm_qos_policy_qos_level_create() +{ + osm_qos_level_t *p = + (osm_qos_level_t *) malloc(sizeof(osm_qos_level_t)); + if (!p) + return NULL; + memset(p, 0, sizeof(osm_qos_level_t)); + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->use) + free(p->use); + + for (i = 0; i < p->path_bits_range_len; i++) + free(p->path_bits_range_arr[i]); + if (p->path_bits_range_arr) + free(p->path_bits_range_arr); + + free(p); +} + +/*************************************************** + ***************************************************/ + +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey) +{ + if (!p_qos_level || !p_qos_level->pkey_range_len) + return FALSE; + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, + p_qos_level->pkey_range_len, + cl_ntoh16(pkey)); +} + +/*************************************************** + ***************************************************/ + +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp) +{ + unsigned i; + uint16_t pkey_ho = 0; + + if (!p_qos_level || !p_qos_level->pkey_range_len) + return 0; + + /* + * ToDo: This approach is not optimal. + * Think how to find shared pkey that also exists + * in QoS level in less runtime. + */ + + for (i = 0; i < p_qos_level->pkey_range_len; i++) { + for (pkey_ho = p_qos_level->pkey_range_arr[i][0]; + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; pkey_ho++) { + if (osm_physp_share_this_pkey + (p_src_physp, p_dest_physp, cl_hton16(pkey_ho))) + return cl_hton16(pkey_ho); + } + } + + return 0; +} + +/*************************************************** + ***************************************************/ + +osm_qos_match_rule_t *osm_qos_policy_match_rule_create() +{ + osm_qos_match_rule_t *p = + (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_match_rule_t)); + + cl_list_init(&p->source_list, 10); + cl_list_init(&p->source_group_list, 10); + cl_list_init(&p->destination_list, 10); + cl_list_init(&p->destination_group_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->qos_level_name) + free(p->qos_level_name); + if (p->use) + free(p->use); + + for (i = 0; i < p->service_id_range_len; i++) + free(p->service_id_range_arr[i]); + if (p->service_id_range_arr) + free(p->service_id_range_arr); + + for (i = 0; i < p->qos_class_range_len; i++) + free(p->qos_class_range_arr[i]); + if (p->qos_class_range_arr) + free(p->qos_class_range_arr); + + cl_list_apply_func(&p->source_list, __free_single_element, NULL); + cl_list_remove_all(&p->source_list); + cl_list_destroy(&p->source_list); + + cl_list_remove_all(&p->source_group_list); + cl_list_destroy(&p->source_group_list); + + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); + cl_list_remove_all(&p->destination_list); + cl_list_destroy(&p->destination_list); + + cl_list_remove_all(&p->destination_group_list); + cl_list_destroy(&p->destination_group_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_create() +{ + if (p_qos_policy) + osm_qos_policy_destroy(); + + p_qos_policy = (osm_qos_policy_t *) malloc(sizeof(osm_qos_policy_t)); + CL_ASSERT(p_qos_policy); + + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); + + cl_list_construct(&p_qos_policy->port_groups); + cl_list_init(&p_qos_policy->port_groups, 10); + + cl_list_construct(&p_qos_policy->vlarb_tables); + cl_list_init(&p_qos_policy->vlarb_tables, 10); + + cl_list_construct(&p_qos_policy->sl2vl_tables); + cl_list_init(&p_qos_policy->sl2vl_tables, 10); + + cl_list_construct(&p_qos_policy->qos_levels); + cl_list_init(&p_qos_policy->qos_levels, 10); + + cl_list_construct(&p_qos_policy->qos_match_rules); + cl_list_init(&p_qos_policy->qos_match_rules, 10); +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_destroy() +{ + cl_list_iterator_t list_iterator; + osm_qos_port_group_t *p_port_group = NULL; + osm_qos_vlarb_scope_t *p_vlarb_scope = NULL; + osm_qos_sl2vl_scope_t *p_sl2vl_scope = NULL; + osm_qos_level_t *p_qos_level = NULL; + osm_qos_match_rule_t *p_qos_match_rule = NULL; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (p_port_group) + osm_qos_policy_port_group_destroy(p_port_group); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->port_groups); + cl_list_destroy(&p_qos_policy->port_groups); + + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); + while (list_iterator != cl_list_end(&p_qos_policy->vlarb_tables)) { + p_vlarb_scope = + (osm_qos_vlarb_scope_t *) cl_list_obj(list_iterator); + if (p_vlarb_scope) + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->vlarb_tables); + cl_list_destroy(&p_qos_policy->vlarb_tables); + + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); + while (list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables)) { + p_sl2vl_scope = + (osm_qos_sl2vl_scope_t *) cl_list_obj(list_iterator); + if (p_sl2vl_scope) + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->sl2vl_tables); + cl_list_destroy(&p_qos_policy->sl2vl_tables); + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); + if (p_qos_level) + osm_qos_policy_qos_level_destroy(p_qos_level); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_levels); + cl_list_destroy(&p_qos_policy->qos_levels); + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); + if (p_qos_match_rule) + osm_qos_policy_match_rule_destroy(p_qos_match_rule); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_match_rules); + cl_list_destroy(&p_qos_policy->qos_match_rules); + + free(p_qos_policy); + + p_qos_policy = NULL; +} + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group(osm_subn_t * p_subn, + const osm_physp_t * p_physp, + osm_qos_port_group_t * p_port_group) +{ + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); + osm_prtn_t *p_prtn = NULL; + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); + uint64_t port_guid_ho = cl_ntoh64(port_guid); + uint8_t node_type = osm_node_get_type(p_node); + cl_list_iterator_t list_iterator; + char *partition_name; + + /* check whether this port's type matches any of group's types */ + + if ((node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) + || (node_type == IB_NODE_TYPE_ROUTER + && p_port_group->node_type_router)) + return TRUE; + + /* check whether this port's guid is in range of this group's guids */ + + if (__is_num_in_range_arr(p_port_group->guid_range_arr, + p_port_group->guid_range_len, port_guid_ho)) + return TRUE; + + /* check whether this port is member of this group's partitions */ + + list_iterator = cl_list_head(&p_port_group->partition_list); + while (list_iterator != cl_list_end(&p_port_group->partition_list)) { + partition_name = (char *)cl_list_obj(list_iterator); + if (partition_name && strlen(partition_name)) { + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); + if (p_prtn) { + if (osm_prtn_is_guid(p_prtn, port_guid)) + return TRUE; + } + } + list_iterator = cl_list_next(list_iterator); + } + + /* check whether this port's name matches any of group's names */ + + /* + * TODO: check port names + * + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; + */ + + return FALSE; +} /* __qos_policy_is_port_in_group() */ + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group_list(const osm_pr_rcv_t * p_rcv, + const osm_physp_t * p_physp, + cl_list_t * p_port_group_list) +{ + osm_qos_port_group_t *p_port_group; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(p_port_group_list); + while (list_iterator != cl_list_end(p_port_group_list)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (p_port_group) { + if (__qos_policy_is_port_in_group + (p_rcv->p_subn, p_physp, p_port_group)) + return TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr( + const osm_pr_rcv_t * p_rcv, + const ib_path_rec_t * p_pr, + const osm_physp_t * p_src_physp, + const osm_physp_t * p_dest_physp, + ib_net64_t comp_mask) +{ + osm_qos_match_rule_t *p_qos_match_rule = NULL; + cl_list_iterator_t list_iterator; + + if (!cl_list_count(&p_qos_policy->qos_match_rules)) + return NULL; + + /* Go over all QoS match rules and find the one that matches the request */ + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); + if (!p_qos_match_rule) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + /* If a match rule has Source groups, PR request source has to be in this list */ + + if (cl_list_count(&p_qos_match_rule->source_group_list)) { + if (!__qos_policy_is_port_in_group_list(p_rcv, + p_src_physp, + &p_qos_match_rule-> + source_group_list)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has Destination groups, PR request dest. has to be in this list */ + + if (cl_list_count(&p_qos_match_rule->destination_group_list)) { + if (!__qos_policy_is_port_in_group_list(p_rcv, + p_dest_physp, + &p_qos_match_rule-> + destination_group_list)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has QoS classes, PR request HAS + to have a matching QoS class to match the rule */ + + if (p_qos_match_rule->qos_class_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->qos_class_range_arr, + p_qos_match_rule->qos_class_range_len, + ib_path_rec_qos_class(p_pr))) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has Service IDs, PR request HAS + to have a matching Service ID to match the rule */ + + if (p_qos_match_rule->service_id_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->service_id_range_arr, + p_qos_match_rule->service_id_range_len, + p_pr->service_id)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has PKeys, PR request HAS + to have a matching PKey to match the rule */ + + if (p_qos_match_rule->pkey_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->pkey_range_arr, + p_qos_match_rule->pkey_range_len, + ib_path_rec_qos_class(p_pr))) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* if we got here, then this match-rule matched this PR request */ + break; + } + + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) + return NULL; + + return p_qos_match_rule; +} /* __qos_policy_get_match_rule_by_pr() */ + +/*************************************************** + ***************************************************/ + +static osm_qos_level_t *__qos_policy_get_qos_level_by_name(char *name) +{ + osm_qos_level_t *p_qos_level = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); + if (!p_qos_level) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name, p_qos_level->name) == 0) + return p_qos_level; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_port_group_t *__qos_policy_get_port_group_by_name(const char + *const name) +{ + osm_qos_port_group_t *p_port_group = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (!p_port_group) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name, p_port_group->name) == 0) + return p_port_group; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +int osm_qos_policy_validate() +{ + cl_list_iterator_t match_rules_list_iterator; + cl_list_iterator_t list_iterator; + osm_qos_port_group_t *p_port_group = NULL; + osm_qos_match_rule_t *p_qos_match_rule = NULL; + char *str; + unsigned i; + int res = 0; + + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_policy_validate); + + /* set default qos level */ + + p_qos_policy->p_default_qos_level = + __qos_policy_get_qos_level_by_name + (OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + if (!p_qos_policy->p_default_qos_level) { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC10: " + "Default qos-level (%s) not defined.\n", + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + res = 1; + goto Exit; + } + + /* scan all the match rules, and fill the lists of pointers to + relevant qos levels and port groups to speed up PR matching */ + + i = 1; + match_rules_list_iterator = + cl_list_head(&p_qos_policy->qos_match_rules); + while (match_rules_list_iterator != + cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) + cl_list_obj(match_rules_list_iterator); + CL_ASSERT(p_qos_match_rule); + + /* find the matching qos-level for each match-rule */ + + p_qos_match_rule->p_qos_level = + __qos_policy_get_qos_level_by_name(p_qos_match_rule-> + qos_level_name); + + if (!p_qos_match_rule->p_qos_level) { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC11: " + "qos-match-rule num %u: qos-level '%s' not found\n", + i, p_qos_match_rule->qos_level_name); + res = 1; + goto Exit; + } + + /* find the matching port-group for element of source_list */ + + if (cl_list_count(&p_qos_match_rule->source_list)) { + list_iterator = + cl_list_head(&p_qos_match_rule->source_list); + while (list_iterator != + cl_list_end(&p_qos_match_rule->source_list)) { + str = (char *)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = + __qos_policy_get_port_group_by_name(str); + if (!p_port_group) { + osm_log(p_qos_parser_osm_log, + OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC12: " + "qos-match-rule num %u: source port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule-> + source_group_list, + p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* find the matching port-group for element of destination_list */ + + if (cl_list_count(&p_qos_match_rule->destination_list)) { + list_iterator = + cl_list_head(&p_qos_match_rule->destination_list); + while (list_iterator != + cl_list_end(&p_qos_match_rule-> + destination_list)) { + str = (char *)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = + __qos_policy_get_port_group_by_name(str); + if (!p_port_group) { + osm_log(p_qos_parser_osm_log, + OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC13: " + "qos-match-rule num %u: destination port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule-> + destination_group_list, + p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* done with the current match-rule */ + + match_rules_list_iterator = + cl_list_next(match_rules_list_iterator); + i++; + } + + Exit: + OSM_LOG_EXIT(p_qos_parser_osm_log); + return res; +} /* osm_qos_policy_validate() */ + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level) +{ + osm_qos_match_rule_t *p_qos_match_rule = NULL; + osm_qos_level_t *p_qos_level = NULL; + + OSM_LOG_ENTER(p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr); + + *pp_qos_level = NULL; + + if (!p_qos_policy) + goto Exit; + + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_rcv, + p_pr, + p_src_physp, + p_dest_physp, + comp_mask); + + if (p_qos_match_rule) + p_qos_level = p_qos_match_rule->p_qos_level; + else + p_qos_level = p_qos_policy->p_default_qos_level; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "PathRecord request:" + "Src port 0x%016" PRIx64 ", " + "Dst port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp))); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "Applying QoS Level %s (%s)\n", + p_qos_level->name, + (p_qos_level->use) ? p_qos_level->use : "no description"); + + *pp_qos_level = p_qos_level; + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); +} /* osm_qos_policy_get_qos_level_by_pr() */ + +/*************************************************** + ***************************************************/ -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Aug 21 04:53:52 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 21 Aug 2007 14:53:52 +0300 Subject: [ofa-general] [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <46C98428.1010607@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> Message-ID: <46CAD250.6000603@dev.mellanox.co.il> Compiling QoS policy file parser. C & H files are generated from Lex & Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. Signed-off-by: Yevgeny Kliteynik --- opensm/include/Makefile.am | 2 ++ opensm/opensm/Makefile.am | 16 +++++++++++++++- opensm/opensm/configure.in | 2 ++ 3 files changed, 19 insertions(+), 1 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..b83205f 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -90,6 +90,8 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ + $(srcdir)/opensm/osm_qos_policy.h \ + $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/complib/cl_thread_osd.h \ $(srcdir)/complib/cl_packon.h \ $(srcdir)/complib/cl_atomic_osd.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..31e09f7 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ osm_vl15intf.c osm_vl_arb_rcv.c \ st.c osm_perfmgr.c osm_perfmgr_db.c \ - osm_event_plugin.c osm_dump.c + osm_event_plugin.c osm_dump.c \ + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c + +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ + +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l + if OSMV_OPENIB opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 @@ -104,3 +112,9 @@ install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so + +# generate c and h files from the lex and yacc files +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l + diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in index afb20b3..a49538d 100644 --- a/opensm/opensm/configure.in +++ b/opensm/opensm/configure.in @@ -22,6 +22,8 @@ AC_PROG_INSTALL AC_PROG_LN_S AC_PROG_MAKE_SET AC_PROG_LIBTOOL +AM_PROG_LEX +AC_PROG_YACC dnl Checks for libraries -- 1.5.1.4 From jim at mellanox.com Tue Aug 21 05:19:12 2007 From: jim at mellanox.com (Jim Mott) Date: Tue, 21 Aug 2007 05:19:12 -0700 Subject: [ofa-general] SDP overhead In-Reply-To: <1187689211.7505.7.camel@koenVRT> References: <1187689211.7505.7.camel@koenVRT> Message-ID: Every SDP data frame has a fixed size 16 byte header. Largest SDP frame is 32K + 16 byte header in OFED 1.2 implementation. Ack overhead for credit replenish is just the header (16 bytes), but can be carried 'for free' in bidirectional sessions. Thanks, JIm Jim Mott Mellanox Technologies Ltd. mail: jim at mellanox.com -----Original Message----- From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Koen Segers Sent: Tuesday, August 21, 2007 4:40 AM To: openib-general at openib.org Subject: [ofa-general] SDP overhead Hello, Can somebody tell me what the overhead is of an SDP packet when the packet is completely filled with data? The numbers I want to have are: max data length of 1 packet total header length of 1 packet Regards, Koen Segers *** Disclaimer *** Vlaamse Radio- en Televisieomroep Auguste Reyerslaan 52, 1043 Brussel nv van publiek recht BTW BE 0244.142.664 RPR Brussel http://www.vrt.be/disclaimer _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From hadi at cyberus.ca Tue Aug 21 05:30:22 2007 From: hadi at cyberus.ca (jamal) Date: Tue, 21 Aug 2007 08:30:22 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.001824.105429474.davem@davemloft.net> References: <20070821.001824.105429474.davem@davemloft.net> Message-ID: <1187699422.5324.41.camel@localhost> On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: > Using 16K buffer size really isn't going to keep the pipe full enough > for TSO. Why the comparison with TSO (or GSO for that matter)? Seems to me that is only valid/fair if you have a single flow. Batching is multi-flow focused (or i should say flow-unaware). cheers, jamal From swise at opengridcomputing.com Tue Aug 21 06:31:48 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 21 Aug 2007 08:31:48 -0500 Subject: [ofa-general] RE: OFA website edits In-Reply-To: <795c49870708081047v2d5598b6q6c7ef5a91063897c@mail.gmail.com> References: <46956FF9.50102@ichips.intel.com> <46968448.2000401@ichips.intel.com> <46A798F0.5070902@ichips.intel.com> <795c49870708081047v2d5598b6q6c7ef5a91063897c@mail.gmail.com> Message-ID: <46CAE944.3090902@opengridcomputing.com> Hey Jeff, Can I get ownership of /var/www/openfabrics.org/downloads/cxgb3? That way I can publish libcxgb3 releases, which I maintain. Right now ralfc owns this instead of swise. Thanks, Steve. Jeff Becker wrote: > Hi. I created most of the requested directory/owner pairs in > /var/www/openfabrics.org/downloads. I left out the various MPI > directories, figuring the appropriate web pages will be linked from > somewhere (possibly the downloads web page). I gave Stan Smith an > account. Stan, please contact me to get the account info. > > I'm still working out how to do the dynamic web page stuff, but at > least people can start populating their directories. > > Thanks. > > -jeff > > On 7/25/07, Arlin Davis wrote: >>> I would like to propose adding project directories under >>> http://www.openfabrics.org/downloads/ where appropriate and give >>> maintainers access. For example: >>> >> Jeff, please add the following directories with maintainer access as >> follow (or grant access at a maintainer group level): >> >> http://www.openfabrics.org/downloads/verbs (rdreier) >> http://www.openfabrics.org/downloads/rdmacm (shefty) >> http://www.openfabrics.org/downloads/dapl (ardavis) >> http://www.openfabrics.org/downloads/sdp (eitan) >> http://www.openfabrics.org/downloads/utils (eitan) >> http://www.openfabrics.org/downloads/management (sashak) >> http://www.openfabrics.org/downloads/OFED (vlad) >> http://www.openfabrics.org/downloads/archives (vlad) >> http://www.openfabrics.org/downloads/WinOF (ssmith) (Stan Smith will >> need an account) >> http://www.openfabrics.org/downloads/hw/mthca (rdreir) >> http://www.openfabrics.org/downloads/hw/mlx4 (rdreir) >> http://www.openfabrics.org/downloads/hw/ehca (raisch) >> http://www.openfabrics.org/downloads/hw/ipath (ralphc) >> http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc) >> http://www.openfabrics.org/downloads/mpi/mvapich (pasha) >> http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland) >> http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres) >> >> Let us know when these directories are created and the maintainers, who >> want to expose their packages via the webpage, will create a README that >> details the contents of the directory along with WEB_README that >> provides a short description for the webpage. >> >> Will this format allow you to auto configure the download webpage >> sufficiently? The idea is to only add links/descriptions to those >> project sub-directories with WEB_README files present. >> >> Please advise if something on the list is wrong or we missed a project. >> >> Thanks, >> >> -arlin >> >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From tziporet at dev.mellanox.co.il Tue Aug 21 07:46:33 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 21 Aug 2007 17:46:33 +0300 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <20070820170117.GH27000@sashak.voltaire.com> References: <20070820133530.GE27000@sashak.voltaire.com> <20070820093430.02c1706e.weiny2@llnl.gov> <20070820170117.GH27000@sashak.voltaire.com> Message-ID: <46CAFAC9.8020804@mellanox.co.il> Sasha Khapyorsky wrote: > Hi Ira, > > On 09:34 Mon 20 Aug , Ira Weiny wrote: > >> Sasha, >> >> Should this be applied to 1.2? >> > > Yes, I think it should, although don't know when next 1.2.x release will > be. > > Sasha > > Ira, Till (and if) we have another 1.2.x release you can apply the patch on your local OFED copy using the ofed_patch.sh script that is located under OFED docs. Tziporet From swise at opengridcomputing.com Tue Aug 21 07:52:12 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Tue, 21 Aug 2007 09:52:12 -0500 Subject: [ofa-general] [Fwd: [GIT PULL] ofed-1.3 - libcxgb3 refresh] Message-ID: <46CAFC1C.6060103@opengridcomputing.com> resending to lists... -------- Original Message -------- Subject: [GIT PULL] ofed-1.3 - libcxgb3 refresh Date: Tue, 21 Aug 2007 09:17:23 -0500 From: Steve Wise To: Vladimir Sokolovsky CC: OpenFabricsEWG , OpenFabrics General Vlad, I have a new release of libcxgb3 that I want included in ofed-1.3. Changes since ofed-1.2 are 1 bug fix (bug 703), some cleanup on the spec file as well as adding a ChangeLog file (bug 707). This is release 1.0.1 of libcxgb3... Pull from git://git.openfabrics.org/~swise/libcxgb3 ofed_1_3 Thanks, Steve. From dotanb at dev.mellanox.co.il Tue Aug 21 08:18:48 2007 From: dotanb at dev.mellanox.co.il (Dotan Barak) Date: Tue, 21 Aug 2007 18:18:48 +0300 Subject: [ofa-general] Query on ibv_post_recv and IBV_SEND_INLINE In-Reply-To: <20070820175139.GA8604@vt.edu> References: <20070820175139.GA8604@vt.edu> Message-ID: <46CB0258.7040706@dev.mellanox.co.il> Hi. Bharath Ramesh wrote: > I have a question on using IBV_SEND_INLINE. From what I understand if I am > using IBV_SEND_INLINE I technically neednt register the buffer with > ibv_reg_mr as I dont need the L_Key. But still in the receiver end I > would have to have the buffer registered to receive the inline data that > has been posted. If I have an application where I am using a lot of > small messages, then memory registration would be expensive. Is there > any other option for receiving inline messages with out actually > registering the receive buffer. > The flag IBV_SEND_INLINE describe to the low level driver how the data will be read from the memory buffer (the driver will read the data and give it to the HCA or the HCA will fetch the data from the memory using the gather list). The data on the wire for inline messages looks the same like data which was fetched by the HCA. The receive side don't have any feature which can work with memory buffer without any registration. thanks Dotan From eli at mellanox.co.il Tue Aug 21 08:15:49 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 21 Aug 2007 18:15:49 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <46CAC3EF.6000509@voltaire.com> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> <1187692432.30342.57.camel@mtls03> <46CAC3EF.6000509@voltaire.com> Message-ID: <1187709349.30342.103.camel@mtls03> > Can you provide a pointer to some Mellanox high level doc/faq explaining > the possible benefits from the stateless offloading features of connectX? The benefits of the stateless offload features can best be described by numbers. I have been able to push throughput in datagram mode to 814 MB/s. Details of the offload features are in the ConnectX prm. From eli at mellanox.co.il Tue Aug 21 08:46:10 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Tue, 21 Aug 2007 18:46:10 +0300 Subject: [ofa-general] [PATCH] ipoib - possible memory leak Message-ID: <1187711170.30342.106.camel@mtls03> Fix cleanup in case of failure - prevent memory leak in case of failure to query pkey ot gid. Signed-off-by: Eli Cohen --- Index: linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-21 18:25:07.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-21 18:38:34.000000000 +0300 @@ -1083,7 +1083,7 @@ static struct net_device *ipoib_add_port if (result) { printk(KERN_WARNING "%s: ib_query_pkey port %d failed (ret = %d)\n", hca->name, port, result); - goto alloc_mem_failed; + goto device_init_failed; } /* @@ -1099,7 +1099,7 @@ static struct net_device *ipoib_add_port if (result) { printk(KERN_WARNING "%s: ib_query_gid port %d failed (ret = %d)\n", hca->name, port, result); - goto alloc_mem_failed; + goto device_init_failed; } else memcpy(priv->dev->dev_addr + 4, priv->local_gid.raw, sizeof (union ib_gid)); From changquing.tang at hp.com Tue Aug 21 08:58:53 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Tue, 21 Aug 2007 15:58:53 -0000 Subject: [ofa-general] does ibv_modify_qp() check error ? In-Reply-To: <20070820175139.GA8604@vt.edu> References: <20070820175139.GA8604@vt.edu> Message-ID: <349DCDA352EACF42A0C49FA6DCEA840302190ABB@G3W0634.americas.hpqcorp.net> Hi, if anyone can answer, I have a question about ibv_modify_qp() behavior. I use ibv_modify_qp() to move QP state from INIT to RTR and then RTS. when INIT-->RTR, I need to provide peer's qp number and port lid. If qp number and port lid are not correct (for example, both QPs are connected to different switch, and there is no bridge link between the switches), my code is silently hanging. Does ibv_modify_qp() check such error, or is there other way to detect such error ? so my code can error out, not hanging ? Thanks. --CQ, HP MPI From weiny2 at llnl.gov Tue Aug 21 09:41:23 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Tue, 21 Aug 2007 09:41:23 -0700 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <46CAFAC9.8020804@mellanox.co.il> References: <20070820133530.GE27000@sashak.voltaire.com> <20070820093430.02c1706e.weiny2@llnl.gov> <20070820170117.GH27000@sashak.voltaire.com> <46CAFAC9.8020804@mellanox.co.il> Message-ID: <20070821094123.0114c873.weiny2@llnl.gov> On Tue, 21 Aug 2007 17:46:33 +0300 Tziporet Koren wrote: > Sasha Khapyorsky wrote: > > Hi Ira, > > > > > > Yes, I think it should, although don't know when next 1.2.x release will > > be. > > Ira, > Till (and if) we have another 1.2.x release you can apply the patch on > your local OFED copy using the ofed_patch.sh script that is located > under OFED docs. > Thanks, we have already applied using patch to our local repo here. I think we have seen this issue with our 1.2 here and I wondered if this patch would be applicable to 1.2. Sasha, I will let you know if there are issues. Thanks, Ira From sashak at voltaire.com Tue Aug 21 09:55:22 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 21 Aug 2007 19:55:22 +0300 Subject: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <20070821094123.0114c873.weiny2@llnl.gov> References: <20070820133530.GE27000@sashak.voltaire.com> <20070820093430.02c1706e.weiny2@llnl.gov> <20070820170117.GH27000@sashak.voltaire.com> <46CAFAC9.8020804@mellanox.co.il> <20070821094123.0114c873.weiny2@llnl.gov> Message-ID: <20070821165522.GA1397@sashak.voltaire.com> On 09:41 Tue 21 Aug , Ira Weiny wrote: > On Tue, 21 Aug 2007 17:46:33 +0300 > Tziporet Koren wrote: > > > Sasha Khapyorsky wrote: > > > Hi Ira, > > > > > > > > > > > > Yes, I think it should, although don't know when next 1.2.x release will > > > be. > > > > Ira, > > Till (and if) we have another 1.2.x release you can apply the patch on > > your local OFED copy using the ofed_patch.sh script that is located > > under OFED docs. > > > > Thanks, we have already applied using patch to our local repo here. > > I think we have seen this issue with our 1.2 here and I wondered if this patch > would be applicable to 1.2. > > Sasha, I will let you know if there are issues. Thanks. Sasha From mst at dev.mellanox.co.il Tue Aug 21 10:38:07 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 21 Aug 2007 20:38:07 +0300 Subject: [ofa-general] Re: does ibv_modify_qp() check error ? In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA840302190ABB@G3W0634.americas.hpqcorp.net> References: <20070820175139.GA8604@vt.edu> <349DCDA352EACF42A0C49FA6DCEA840302190ABB@G3W0634.americas.hpqcorp.net> Message-ID: <20070821173807.GB8192@mellanox.co.il> > Quoting Tang, Changqing : > Subject: does ibv_modify_qp() check error ? > > > Hi, if anyone can answer, > > I have a question about ibv_modify_qp() behavior. I use > ibv_modify_qp() to move QP state from INIT to RTR and then RTS. > when INIT-->RTR, I need to provide peer's qp number and port lid. If qp > number and port lid are not correct (for example, both > QPs are connected to different switch, and there is no bridge link > between the switches), my code is silently hanging. > > Does ibv_modify_qp() check such error, or is there other way to > detect such error ? so my code can error out, not hanging ? > > Thanks. Try posting a WR, e.g. a 0 size RDMA write. You'll get a CQE with error if the QPN is wrong. -- MST From davem at davemloft.net Tue Aug 21 11:51:43 2007 From: davem at davemloft.net (David Miller) Date: Tue, 21 Aug 2007 11:51:43 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187699422.5324.41.camel@localhost> References: <20070821.001824.105429474.davem@davemloft.net> <1187699422.5324.41.camel@localhost> Message-ID: <20070821.115143.23013721.davem@davemloft.net> From: jamal Date: Tue, 21 Aug 2007 08:30:22 -0400 > On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: > > > Using 16K buffer size really isn't going to keep the pipe full enough > > for TSO. > > Why the comparison with TSO (or GSO for that matter)? Because TSO does batching already, so it's a very good "tit for tat" comparison of the new batching scheme vs. an existing one. From tziporet at dev.mellanox.co.il Tue Aug 21 12:01:43 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Tue, 21 Aug 2007 22:01:43 +0300 Subject: [ofa-general] Do you think the QoS new code will make it for 2.6.24? Message-ID: <46CB3697.9000206@mellanox.co.il> Thanks Tziporet From hadi at cyberus.ca Tue Aug 21 14:09:12 2007 From: hadi at cyberus.ca (jamal) Date: Tue, 21 Aug 2007 17:09:12 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.115143.23013721.davem@davemloft.net> References: <20070821.001824.105429474.davem@davemloft.net> <1187699422.5324.41.camel@localhost> <20070821.115143.23013721.davem@davemloft.net> Message-ID: <1187730552.5324.76.camel@localhost> On Tue, 2007-21-08 at 11:51 -0700, David Miller wrote: > Because TSO does batching already, so it's a very good > "tit for tat" comparison of the new batching scheme > vs. an existing one. Fair enough - I may have read too much into your email then;-> For bulk type of apps (where TSO will make a difference) this a fair test. Hence i agree the 16KB buffer size is not sensible if the goal is to simulate such an app. However (and this is where i read too much into what you were saying) that the test by itself is insufficient comparison. You gotta look at the other side of the coin i.e. at apps where TSO wont buy much. Examples, a busy ssh or irc server and you could go as far as looking at the most pre-dominant app on the wild west, http (average page size from a few years back was in the range of 10-20K and can be simulated with good ole netperf/iperf). cheers, jamal From rvm at obsidianresearch.com Tue Aug 21 16:42:20 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Tue, 21 Aug 2007 17:42:20 -0600 Subject: [ofa-general] IPOIB subnet on IB fabric with routers Message-ID: In order for an ipoib subnet to work on an IB fabric consisting of multiple IB subnets, the MGIDs used for multicast can't have link-local scope otherwise packets will get dropped by the first IB router. Currently, the scope is hard-coded to link-local in the ip*_ib_mc_map() routines. A reasonable first step is to modify said routines to use the scope from the link level broadcast address of the ipoib device in question, and I have a proposed patch which I will post right away. The second step is to make the scope in the link level broadcast address configurable, but I'm not sure what the best way to accomplish that would be. The ip command supports changing the link level broadcast address, but it uses an ioctl which limits the hardware address to 14 bytes (six bytes short) ... so it fails. Even if that approach would work it would be a little error prone, we only care about four bits out of a 20 byte value. My current favourite approach is to add a "broadcast_scope" attribute under /sys/class/net/ibX/ that could be configured before bringing up the link. Does anybody have a better solution? Rolf From ssufficool at rov.sbcounty.gov Tue Aug 21 16:50:22 2007 From: ssufficool at rov.sbcounty.gov (Sufficool, Stanley) Date: Tue, 21 Aug 2007 16:50:22 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: <46C9EEF4.6000407@mellanox.com> References: <1186200658.6031.44.camel@gentoo-linux.localdomain> <46C9EEF4.6000407@mellanox.com> Message-ID: This is what I am getting whe connecting from a Win SRP Intiator on the Target. I added a few more debug messages to the target code. I believe the Windows x64 boxes are using memfree adapters and the Linux x64, a 128MB adapter. ib_srpt: Host i_port_id=0x44d00cffff4b1a00:0x501ed0ffff081700 login with t_port_id=0x44d00cffff4b1a00:0x44d00cffff4b1a00 it_iu_len=340 ib_srpt: srpt_create_ch_ib[1143] max_cqe= 255 max_sge= 30 cm_id= ffff81003e1dce00 ib_mthca 0000:10:00.0: Max rdma_atomic as responder 15 too large (max 4) ib_srpt: srpt_ch_qp_rtr_rts[683]: ib_modify_qp returned EINVAL ib_srpt: srpt_cm_req_recv[1403]: srpt_ch_qp_rtr_rts returned error ib_srpt: Rejected connection I'm still researching where the rdma_atomic is getting the invalid attribute. I will give an update if I find anything else. -----Original Message----- From: Vu Pham [mailto:vu at mellanox.com] Sent: Monday, August 20, 2007 12:44 PM To: Stanley Sufficool Cc: general Subject: Re: [ofa-general] [PATCH] SRPT for current SCST Thanks. I checked in with a few other changes/bug fixes. Please test and report any problem > SCST redefined the scst_tgt_template and it no longer contains the > tm_sync_reply field. Also an extra arg is required for scst_register. > Consequently, the build fails. > > I searched through past SCST versions and docs but could not find what > this was for and it seems to be set to zero (not used?) in this struct > anyhow. > > > ---------------------------------------------------------------------- > -- > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general From rvm at obsidianresearch.com Tue Aug 21 16:50:43 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Tue, 21 Aug 2007 17:50:43 -0600 Subject: [ofa-general] [PATCH] use scope from ipoib device link-level broadcast address in MGIDs Message-ID: An ipoib subnet on an IB fabric that spans multiple IB subnets can't use link-local scope in multicast GIDs. This patch takes the scope from the link level broadcast address of the ipoib device when mapping to an IB multicast address. Signed-off-by: Rolf Manderscheid --- diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 9ffb998..2fd1b30 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -2560,7 +2560,7 @@ static void cma_set_mgid(struct rdma_id_private *id_priv, /* IPv6 address is an SA assigned MGID. */ memcpy(mgid, &sin6->sin6_addr, sizeof *mgid); } else { - ip_ib_mc_map(sin->sin_addr.s_addr, mc_map); + ip_ib_mc_map(sin->sin_addr.s_addr, dev_addr->broadcast, mc_map); if (id_priv->id.ps == RDMA_PS_UDP) mc_map[7] = 0x01; /* Use RDMA CM signature */ mc_map[8] = ib_addr_get_pkey(dev_addr) >> 8; diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h index 3ec7d07..b636221 100644 --- a/include/net/if_inet6.h +++ b/include/net/if_inet6.h @@ -268,14 +268,17 @@ static inline void ipv6_arcnet_mc_map(const struct in6_addr *addr, char *buf) buf[0] = 0x00; } -static inline void ipv6_ib_mc_map(struct in6_addr *addr, char *buf) +static inline void ipv6_ib_mc_map(const struct in6_addr *addr, + const unsigned char *broadcast, char *buf) { + unsigned char scope = broadcast[5] & 0xF; + buf[0] = 0; /* Reserved */ buf[1] = 0xff; /* Multicast QPN */ buf[2] = 0xff; buf[3] = 0xff; buf[4] = 0xff; - buf[5] = 0x12; /* link local scope */ + buf[5] = 0x10 | scope; /* scope from broadcast address */ buf[6] = 0x60; /* IPv6 signature */ buf[7] = 0x1b; buf[8] = 0; /* P_Key */ diff --git a/include/net/ip.h b/include/net/ip.h index abf2820..fbb4dd0 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -266,16 +266,18 @@ static inline void ip_eth_mc_map(__be32 naddr, char *buf) * Leave P_Key as 0 to be filled in by driver. */ -static inline void ip_ib_mc_map(__be32 naddr, char *buf) +static inline void ip_ib_mc_map(__be32 naddr, const unsigned char *broadcast, char *buf) { __u32 addr; + unsigned char scope = broadcast[5] & 0xF; + buf[0] = 0; /* Reserved */ buf[1] = 0xff; /* Multicast QPN */ buf[2] = 0xff; buf[3] = 0xff; addr = ntohl(naddr); buf[4] = 0xff; - buf[5] = 0x12; /* link local scope */ + buf[5] = 0x10 | scope; /* scope from broadcast address */ buf[6] = 0x40; /* IPv4 signature */ buf[7] = 0x1b; buf[8] = 0; /* P_Key */ diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c index 9ab9d53..feb643c 100644 --- a/net/ipv4/arp.c +++ b/net/ipv4/arp.c @@ -214,7 +214,7 @@ int arp_mc_map(__be32 addr, u8 *haddr, struct net_device *dev, int dir) ip_tr_mc_map(addr, haddr); return 0; case ARPHRD_INFINIBAND: - ip_ib_mc_map(addr, haddr); + ip_ib_mc_map(addr, dev->broadcast, haddr); return 0; default: if (dir) { diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index 0358e60..8275625 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -308,7 +308,7 @@ int ndisc_mc_map(struct in6_addr *addr, char *buf, struct net_device *dev, int d ipv6_arcnet_mc_map(addr, buf); return 0; case ARPHRD_INFINIBAND: - ipv6_ib_mc_map(addr, buf); + ipv6_ib_mc_map(addr, dev->broadcast, buf); return 0; default: if (dir) { From yppsatjbe at isoa.net Tue Aug 21 22:10:36 2007 From: yppsatjbe at isoa.net (Boy) Date: Wed, 22 Aug 2007 10:10:36 +0500 Subject: [ofa-general] Re: Message-ID: <0D76F9CE4C59B77.7A0728FBD0@isoa.net> Strategy INF files. inform how AGP USB Core PCI ISA PnP that we took crawled web.The rich Patches Updates NES SNES Game Boy Advance Sega Master Megadrive Engine Neo Geo grabber CDROM -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bgimg.gif Type: image/gif Size: 5018 bytes Desc: not available URL: From krkumar2 at in.ibm.com Tue Aug 21 21:11:52 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 22 Aug 2007 09:41:52 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.115143.23013721.davem@davemloft.net> Message-ID: David Miller wrote on 08/22/2007 12:21:43 AM: > From: jamal > Date: Tue, 21 Aug 2007 08:30:22 -0400 > > > On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: > > > > > Using 16K buffer size really isn't going to keep the pipe full enough > > > for TSO. > > > > Why the comparison with TSO (or GSO for that matter)? > > Because TSO does batching already, so it's a very good > "tit for tat" comparison of the new batching scheme > vs. an existing one. I am planning to do more testing on your suggestion over the weekend, but I had a comment. Are you saying that TSO and batching should be mutually exclusive so hardware that doesn't support TSO (like IB) only would benefit? But even if they can co-exist, aren't cases like sending multiple small skbs better handled with batching? Thanks, - KK From davem at davemloft.net Tue Aug 21 21:22:29 2007 From: davem at davemloft.net (David Miller) Date: Tue, 21 Aug 2007 21:22:29 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070821.115143.23013721.davem@davemloft.net> Message-ID: <20070821.212229.82050253.davem@davemloft.net> From: Krishna Kumar2 Date: Wed, 22 Aug 2007 09:41:52 +0530 > David Miller wrote on 08/22/2007 12:21:43 AM: > > > From: jamal > > Date: Tue, 21 Aug 2007 08:30:22 -0400 > > > > > On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: > > > > > > > Using 16K buffer size really isn't going to keep the pipe full enough > > > > for TSO. > > > > > > Why the comparison with TSO (or GSO for that matter)? > > > > Because TSO does batching already, so it's a very good > > "tit for tat" comparison of the new batching scheme > > vs. an existing one. > > I am planning to do more testing on your suggestion over the > weekend, but I had a comment. Are you saying that TSO and > batching should be mutually exclusive so hardware that doesn't > support TSO (like IB) only would benefit? > > But even if they can co-exist, aren't cases like sending > multiple small skbs better handled with batching? I'm not making any suggestions, so don't read that into anything I've said :-) I think the jury is still out, but seeing TSO perform even slightly worse with the batching changes in place would be very worrysome. This applies to both throughput and cpu utilization. From kliteyn at mellanox.co.il Tue Aug 21 21:36:02 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 22 Aug 2007 07:36:02 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-22:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_20_12:20:05_2007 [a6cfde5efb96f6b3858c26e6934554ded57f89cb] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=559 Fail=1 Pass: 42 Stability IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 41 Pkey IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 1 Pkey IS1-16.topo From krkumar2 at in.ibm.com Wed Aug 22 00:03:04 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 22 Aug 2007 12:33:04 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.212229.82050253.davem@davemloft.net> Message-ID: Hi Dave, David Miller wrote on 08/22/2007 09:52:29 AM: > From: Krishna Kumar2 > Date: Wed, 22 Aug 2007 09:41:52 +0530 > > > > Because TSO does batching already, so it's a very good > > > "tit for tat" comparison of the new batching scheme > > > vs. an existing one. > > > > I am planning to do more testing on your suggestion over the > > weekend, but I had a comment. Are you saying that TSO and > > batching should be mutually exclusive so hardware that doesn't > > support TSO (like IB) only would benefit? > > > > But even if they can co-exist, aren't cases like sending > > multiple small skbs better handled with batching? > > I'm not making any suggestions, so don't read that into anything I've > said :-) > > I think the jury is still out, but seeing TSO perform even slightly > worse with the batching changes in place would be very worrysome. > This applies to both throughput and cpu utilization. Does turning off batching solve that problem? What I mean by that is: batching can be disabled if a TSO device is worse for some cases. Infact something that I had changed my latest code is to not enable batching in register_netdevice (in Rev4 which I am sending in a few mins), rather the user has to explicitly turn 'on' batching. Wondering if that is what you are concerned about. In any case, I will test your case on Monday (I am on vacation for next couple of days). Thanks, - KK From krkumar2 at in.ibm.com Wed Aug 22 01:28:58 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 13:58:58 +0530 Subject: [ofa-general] [PATCH 1/10 Rev4] [Doc] HOWTO Documentation for batching In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822082858.11964.87377.sendpatchset@localhost.localdomain> Add Documentation describing batching skb xmit capability. Signed-off-by: Krishna Kumar --- batching_skb_xmit.txt | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 78 insertions(+) diff -ruNp org/Documentation/networking/batching_skb_xmit.txt new/Documentation/networking/batching_skb_xmit.txt --- org/Documentation/networking/batching_skb_xmit.txt 1970-01-01 05:30:00.000000000 +0530 +++ new/Documentation/networking/batching_skb_xmit.txt 2007-08-22 10:21:19.000000000 +0530 @@ -0,0 +1,78 @@ + HOWTO for batching skb xmit support + ----------------------------------- + +Section 1: What is batching skb xmit +Section 2: How batching xmit works vs the regular xmit +Section 3: How drivers can support batching +Section 4: How users can work with batching + + +Introduction: Kernel support for batching skb +---------------------------------------------- + +A new capability to support xmit of multiple skbs is provided in the netdevice +layer. Drivers which enable this capability should be able to process multiple +skbs in a single call to their xmit handler. + + +Section 1: What is batching skb xmit +------------------------------------- + + This capability is optionally enabled by a driver by setting the + NETIF_F_BATCH_SKBS bit in dev->features. The pre-requisite for a + driver to use this capability is that it should have a reasonably + sized hardware queue that can process multiple skbs. + + +Section 2: How batching xmit works vs the regular xmit +------------------------------------------------------- + + The network stack gets called from upper layer protocols with a single + skb to transmit. This skb is first enqueue'd and an attempt is made to + transmit it immediately (via qdisc_run). However, events like tx lock + contention, tx queue stopped, etc, can result in the skb not getting + sent out and it remains in the queue. When the next xmit is called or + when the queue is re-enabled, qdisc_run could potentially find + multiple packets in the queue, and iteratively send them all out + one-by-one. + + Batching skb xmit is a mechanism to exploit this situation where all + skbs can be passed in one shot to the device. This reduces driver + processing, locking at the driver (or in stack for ~LLTX drivers) + gets amortized over multiple skbs, and in case of specific drivers + where every xmit results in a completion processing (like IPoIB) - + optimizations can be made in the driver to request a completion for + only the last skb that was sent which results in saving interrupts + for every (but the last) skb that was sent in the same batch. + + Batching can result in significant performance gains for systems that + have multiple data stream paths over the same network interface card. + + +Section 3: How drivers can support batching +--------------------------------------------- + + Batching requires the driver to set the NETIF_F_BATCH_SKBS bit in + dev->features. + + The driver's xmit handler should be modified to process multiple skbs + instead of one skb. The driver's xmit handler is called either with a + skb to transmit or NULL skb, where the latter case should be handled + as a call to xmit multiple skbs. This is done by sending out all skbs + in the dev->skb_blist list (where it was added by the core stack). + + +Section 4: How users can work with batching +--------------------------------------------- + + Batching can be disabled for a particular device, e.g. on desktop + systems if only one stream of network activity for that device is + taking place, since performance could be slightly affected due to + extra processing that batching adds (unless packets are getting + sent fast resulting in stopped queue's). Batching can be enabled if + more than one stream of network activity per device is being done, + e.g. on servers; or even desktop usage with multiple browser, chat, + file transfer sessions, etc. + + Per device batching can be enabled/disabled by passing 'on' or 'off' + respectively to ethtool. From krkumar2 at in.ibm.com Wed Aug 22 01:29:17 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 13:59:17 +0530 Subject: [ofa-general] [PATCH 2/10 Rev4] [core] Add skb_blist & support for batching In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822082917.11964.52051.sendpatchset@localhost.localdomain> Introduce skb_blist, NETIF_F_BATCH_SKBS, use single API for batching/no-batching, etc. Signed-off-by: Krishna Kumar --- include/linux/netdevice.h | 4 ++++ net/core/dev.c | 21 ++++++++++++++++++--- 2 files changed, 22 insertions(+), 3 deletions(-) diff -ruNp org/include/linux/netdevice.h new/include/linux/netdevice.h --- org/include/linux/netdevice.h 2007-08-20 14:26:36.000000000 +0530 +++ new/include/linux/netdevice.h 2007-08-22 08:42:10.000000000 +0530 @@ -399,6 +399,7 @@ struct net_device #define NETIF_F_VLAN_CHALLENGED 1024 /* Device cannot handle VLAN packets */ #define NETIF_F_GSO 2048 /* Enable software GSO. */ #define NETIF_F_LLTX 4096 /* LockLess TX */ +#define NETIF_F_BATCH_SKBS 8192 /* Driver supports multiple skbs/xmit */ #define NETIF_F_MULTI_QUEUE 16384 /* Has multiple TX/RX queues */ #define NETIF_F_LRO 32768 /* large receive offload */ @@ -510,6 +511,9 @@ struct net_device /* Partially transmitted GSO packet. */ struct sk_buff *gso_skb; + /* List of batch skbs (optional, used if driver supports skb batching */ + struct sk_buff_head *skb_blist; + /* ingress path synchronizer */ spinlock_t ingress_lock; struct Qdisc *qdisc_ingress; diff -ruNp org/net/core/dev.c new/net/core/dev.c --- org/net/core/dev.c 2007-08-20 14:26:37.000000000 +0530 +++ new/net/core/dev.c 2007-08-22 10:49:22.000000000 +0530 @@ -898,6 +898,16 @@ void netdev_state_change(struct net_devi } } +static void free_batching(struct net_device *dev) +{ + if (dev->skb_blist) { + if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + kfree(dev->skb_blist); + dev->skb_blist = NULL; + } +} + /** * dev_load - load a network module * @name: name of interface @@ -1458,7 +1468,9 @@ static int dev_gso_segment(struct sk_buf int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev) { - if (likely(!skb->next)) { + if (likely(skb)) { + if (unlikely(skb->next)) + goto gso; if (!list_empty(&ptype_all)) dev_queue_xmit_nit(skb, dev); @@ -1468,10 +1480,10 @@ int dev_hard_start_xmit(struct sk_buff * if (skb->next) goto gso; } - - return dev->hard_start_xmit(skb, dev); } + return dev->hard_start_xmit(skb, dev); + gso: do { struct sk_buff *nskb = skb->next; @@ -3791,6 +3803,9 @@ void unregister_netdevice(struct net_dev synchronize_net(); + /* Deallocate batching structure */ + free_batching(dev); + /* Shutdown queueing discipline. */ dev_shutdown(dev); From krkumar2 at in.ibm.com Wed Aug 22 01:29:36 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 13:59:36 +0530 Subject: [ofa-general] [PATCH 3/10 Rev4] [sched] Modify qdisc_run to support batching In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822082936.11964.25.sendpatchset@localhost.localdomain> Modify qdisc_run() to support batching. Modify callers of qdisc_run to use batching, modify qdisc_restart to implement batching. Signed-off-by: Krishna Kumar --- include/linux/netdevice.h | 2 + include/net/pkt_sched.h | 6 +-- net/core/dev.c | 44 +++++++++++++++++++++++++++- net/sched/sch_generic.c | 70 ++++++++++++++++++++++++++++++++++++++-------- 4 files changed, 105 insertions(+), 17 deletions(-) diff -ruNp org/include/net/pkt_sched.h new/include/net/pkt_sched.h --- org/include/net/pkt_sched.h 2007-08-20 14:26:36.000000000 +0530 +++ new/include/net/pkt_sched.h 2007-08-22 09:23:57.000000000 +0530 @@ -80,13 +80,13 @@ extern struct qdisc_rate_table *qdisc_ge struct rtattr *tab); extern void qdisc_put_rtab(struct qdisc_rate_table *tab); -extern void __qdisc_run(struct net_device *dev); +extern void __qdisc_run(struct net_device *dev, struct sk_buff_head *blist); -static inline void qdisc_run(struct net_device *dev) +static inline void qdisc_run(struct net_device *dev, struct sk_buff_head *blist) { if (!netif_queue_stopped(dev) && !test_and_set_bit(__LINK_STATE_QDISC_RUNNING, &dev->state)) - __qdisc_run(dev); + __qdisc_run(dev, blist); } extern int tc_classify_compat(struct sk_buff *skb, struct tcf_proto *tp, diff -ruNp org/include/linux/netdevice.h new/include/linux/netdevice.h --- org/include/linux/netdevice.h 2007-08-20 14:26:36.000000000 +0530 +++ new/include/linux/netdevice.h 2007-08-22 08:42:10.000000000 +0530 @@ -892,6 +896,8 @@ extern int dev_set_mac_address(struct n struct sockaddr *); extern int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev); +extern int dev_add_skb_to_blist(struct sk_buff *skb, + struct net_device *dev); extern void dev_init(void); diff -ruNp org/net/sched/sch_generic.c new/net/sched/sch_generic.c --- org/net/sched/sch_generic.c 2007-08-20 14:26:37.000000000 +0530 +++ new/net/sched/sch_generic.c 2007-08-22 08:49:55.000000000 +0530 @@ -59,10 +59,12 @@ static inline int qdisc_qlen(struct Qdis static inline int dev_requeue_skb(struct sk_buff *skb, struct net_device *dev, struct Qdisc *q) { - if (unlikely(skb->next)) - dev->gso_skb = skb; - else - q->ops->requeue(skb, q); + if (likely(skb)) { + if (unlikely(skb->next)) + dev->gso_skb = skb; + else + q->ops->requeue(skb, q); + } netif_schedule(dev); return 0; @@ -91,10 +93,15 @@ static inline int handle_dev_cpu_collisi /* * Same CPU holding the lock. It may be a transient * configuration error, when hard_start_xmit() recurses. We - * detect it by checking xmit owner and drop the packet when - * deadloop is detected. Return OK to try the next skb. + * detect it by checking xmit owner and drop the packet (or + * all packets in batching case) when deadloop is detected. + * Return OK to try the next skb. */ - kfree_skb(skb); + if (likely(skb)) + kfree_skb(skb); + else if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + if (net_ratelimit()) printk(KERN_WARNING "Dead loop on netdevice %s, " "fix it urgently!\n", dev->name); @@ -112,6 +119,38 @@ static inline int handle_dev_cpu_collisi } /* + * Algorithm to get skb(s) is: + * - Non batching drivers, or if the batch list is empty and there is + * 1 skb in the queue - dequeue skb and put it in *skbp to tell the + * caller to use the single xmit API. + * - Batching drivers where the batch list already contains atleast one + * skb, or if there are multiple skbs in the queue: keep dequeue'ing + * skb's upto a limit and set *skbp to NULL to tell the caller to use + * the multiple xmit API. + * + * Returns: + * 1 - atleast one skb is to be sent out, *skbp contains skb or NULL + * (in case >1 skbs present in blist for batching) + * 0 - no skbs to be sent. + */ +static inline int get_skb(struct net_device *dev, struct Qdisc *q, + struct sk_buff_head *blist, struct sk_buff **skbp) +{ + if (likely(!blist || (!skb_queue_len(blist) && qdisc_qlen(q) <= 1))) { + return likely((*skbp = dev_dequeue_skb(dev, q)) != NULL); + } else { + struct sk_buff *skb; + int max = dev->tx_queue_len - skb_queue_len(blist); + + while (max > 0 && (skb = dev_dequeue_skb(dev, q)) != NULL) + max -= dev_add_skb_to_blist(skb, dev); + + *skbp = NULL; + return 1; /* there is atleast one skb in skb_blist */ + } +} + +/* * NOTE: Called under dev->queue_lock with locally disabled BH. * * __LINK_STATE_QDISC_RUNNING guarantees only one CPU can process this @@ -130,7 +169,8 @@ static inline int handle_dev_cpu_collisi * >0 - queue is not empty. * */ -static inline int qdisc_restart(struct net_device *dev) +static inline int qdisc_restart(struct net_device *dev, + struct sk_buff_head *blist) { struct Qdisc *q = dev->qdisc; struct sk_buff *skb; @@ -138,7 +178,7 @@ static inline int qdisc_restart(struct n int ret; /* Dequeue packet */ - if (unlikely((skb = dev_dequeue_skb(dev, q)) == NULL)) + if (unlikely(!get_skb(dev, q, blist, &skb))) return 0; /* @@ -168,7 +208,7 @@ static inline int qdisc_restart(struct n switch (ret) { case NETDEV_TX_OK: - /* Driver sent out skb successfully */ + /* Driver sent out skb (or entire skb_blist) successfully */ ret = qdisc_qlen(q); break; @@ -190,10 +230,10 @@ static inline int qdisc_restart(struct n return ret; } -void __qdisc_run(struct net_device *dev) +void __qdisc_run(struct net_device *dev, struct sk_buff_head *blist) { do { - if (!qdisc_restart(dev)) + if (!qdisc_restart(dev, blist)) break; } while (!netif_queue_stopped(dev)); @@ -563,6 +603,12 @@ void dev_deactivate(struct net_device *d qdisc = dev->qdisc; dev->qdisc = &noop_qdisc; + if (dev->skb_blist) { + /* Release skbs on batch list */ + if (!skb_queue_empty(dev->skb_blist)) + skb_queue_purge(dev->skb_blist); + } + qdisc_reset(qdisc); skb = dev->gso_skb; diff -ruNp org/net/core/dev.c new/net/core/dev.c --- org/net/core/dev.c 2007-08-20 14:26:37.000000000 +0530 +++ new/net/core/dev.c 2007-08-22 10:49:22.000000000 +0530 @@ -1466,6 +1466,45 @@ static int dev_gso_segment(struct sk_buf return 0; } +/* + * Add skb (skbs in case segmentation is required) to dev->skb_blist. No one + * can add to this list simultaneously since we are holding QDISC RUNNING + * bit. Also list is safe from simultaneous deletes too since skbs are + * dequeued only when the driver is invoked. + * + * Returns count of successful skb(s) added to skb_blist. + */ +int dev_add_skb_to_blist(struct sk_buff *skb, struct net_device *dev) +{ + if (!list_empty(&ptype_all)) + dev_queue_xmit_nit(skb, dev); + + if (netif_needs_gso(dev, skb)) { + if (unlikely(dev_gso_segment(skb))) { + kfree(skb); + return 0; + } + + if (skb->next) { + int count = 0; + + do { + struct sk_buff *nskb = skb->next; + + skb->next = nskb->next; + __skb_queue_tail(dev->skb_blist, nskb); + count++; + } while (skb->next); + + skb->destructor = DEV_GSO_CB(skb)->destructor; + kfree_skb(skb); + return count; + } + } + __skb_queue_tail(dev->skb_blist, skb); + return 1; +} + int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev) { if (likely(skb)) { @@ -1620,7 +1659,7 @@ gso: /* reset queue_mapping to zero */ skb->queue_mapping = 0; rc = q->enqueue(skb, q); - qdisc_run(dev); + qdisc_run(dev, NULL); spin_unlock(&dev->queue_lock); rc = rc == NET_XMIT_BYPASS ? NET_XMIT_SUCCESS : rc; @@ -1818,7 +1857,8 @@ static void net_tx_action(struct softirq clear_bit(__LINK_STATE_SCHED, &dev->state); if (spin_trylock(&dev->queue_lock)) { - qdisc_run(dev); + /* Send all skbs if driver supports batching */ + qdisc_run(dev, dev->skb_blist); spin_unlock(&dev->queue_lock); } else { netif_schedule(dev); From krkumar2 at in.ibm.com Wed Aug 22 01:28:39 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 13:58:39 +0530 Subject: [ofa-general] [PATCH 0/10 Rev4] Implement skb batching and support in IPoIB Message-ID: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> This set of patches implements the batching xmit capability (changed from API), and adds support for batching in IPoIB. Also included is a sample patch for E1000 (ported - thanks to Jamal's E1000 changes from earlier kernel). I will use this patch for testing E1000 TSO vs batching after the weekend. List of changes from previous revision: ---------------------------------------- 1. [Dave/Patrick] Remove new xmit API altogether (and add a capabilities flag in dev->features). Modify documentation to remove API, etc. 2. [Evgeniy] Remove bogus checks for <0, and use spin_lock_bh. 3. [Jamal] Ported Jamal's E1000 driver changes for using batching xmit. 5. [KK] Fix out-of-order sending of skbs bug resulting in re-transmissions by a fix in IPoIB [see XXX]. 6. [KK] Do not force device to use batching as default, instead let user enable batching if required. This is useful in case users are not aware that batching is taking place. 4. [KK] IPoIB: Remove multiple xmit handlers and convert to use one. 7. [KK] IPoIB: Removed overkill - poll handler can be called on one CPU, so there is no need to take a new lock against parallel WC's. Extras that I can do later: --------------------------- 1. [Patrick] Use skb_blist statically in netdevice. This could also be used to integrate GSO and batching. 2. [Evgeniy] Useful to splice lists dev_add_skb_to_blist (and this can be done for regular xmit's of GSO skbs too for #1 above). Patches are described as: Mail 0/10: This mail Mail 1/10: HOWTO documentation Mail 2/10: Introduce skb_blist, NETIF_F_BATCH_SKBS, use single API for batching/no-batching, etc. Mail 3/10: Modify qdisc_run() to support batching Mail 4/10: Add ethtool support to enable/disable batching Mail 5/10: IPoIB: Header file changes to use batching Mail 6/10: IPoIB: CM & Multicast changes Mail 7/10: IPoIB: Verbs changes to use batching Mail 8/10: IPoIB: Internal post and work completion handler Mail 9/10: IPoIB: Implement the new batching capability Mail 10/10: E1000: Implement the new batching capability Issues: -------- I am getting a huge amount of retransmissions for both TCP and TCP No Delay cases for IPoIB (which explains the slight degradation for some test cases mentioned in previous mail). After a full test run, there were 18500 retransmissions for every 1 in regular code. But there is 20.7% overall improvement in BW even with this huge amount of retransmissions (which implies batching could improve results even more if this problem is fixed). Results of experiments are: a. With batching set to maximum 2 skbs, I get almost the same number of retransmissions (implies receiver probably is not dropping skbs). ifconfig/netstat on receiver gives no clue (drop/errors, etc). b. Making the IPoIB xmit create single work requests for each skb on blist reduces retrans to same as in regular code. c. Similar retransmission increase is not seen for E1000. Please review and provide feedback; and consider for inclusion. Thanks, - KK [XXX] Dave had suggested to use batching only in the net_tx_action case. When I implemented that in earlier revisions, there were lots of TCP retransmissions (about 18,000 to every 1 in regular code). I found the reason for part of that problem as: skbs get queue'd up in dev->qdisc (when tx lock was not got or queue blocked); when net_tx_action is called later, it passes the batch list as argument to qdisc_run and this results in skbs being moved to the batch list; then batching xmit also fails due to tx lock failure; the next many regular xmit of a single skb will go through the fast path (pass NULL batch list to qdisc_run) and send those skbs out to the device while previous skbs are cooling their heels in the batch list. The first fix was to not pass NULL/batch-list to qdisc_run() but to always check whether skbs are present in batch list when trying to xmit. This reduced retransmissions by a third (from 18,000 to around 12,000), but led to another problem while testing - iperf transmit almost zero data for higher # of parallel flows like 64 or more (and when I run iperf for a 2 min run, it takes about 5-6 mins, and reports that it ran 0 secs and the amount of data transfered is a few MB's). I don't know why this happens with this being the only change (any ideas is very appreciated). The second fix that resolved this was to revert back to Dave's suggestion to use batching only in net_tx_action case, and modify the driver to see if skbs are present in batch list and to send them out first before sending the current skb. I still see huge retransmission for IPoIB (but not for E1000), though it has reduced to 12,000 from the earlier 18,000 number. From krkumar2 at in.ibm.com Wed Aug 22 01:30:32 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:00:32 +0530 Subject: [ofa-general] [PATCH 6/10 Rev4] [IPoIB] CM & Multicast changes In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083032.11964.97171.sendpatchset@localhost.localdomain> IPoIB CM & Multicast changes based on header file changes. Signed-off-by: Krishna Kumar --- ipoib_cm.c | 13 +++++++++---- ipoib_multicast.c | 4 ++-- 2 files changed, 11 insertions(+), 6 deletions(-) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib_cm.c new/drivers/infiniband/ulp/ipoib/ipoib_cm.c --- org/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib_cm.c 2007-08-22 08:33:51.000000000 +0530 @@ -493,14 +493,19 @@ static inline int post_send(struct ipoib unsigned int wr_id, u64 addr, int len) { + int ret; struct ib_send_wr *bad_wr; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + priv->tx_sge[0].addr = addr; + priv->tx_sge[0].length = len; + + priv->tx_wr[0].wr_id = wr_id; - priv->tx_wr.wr_id = wr_id; + priv->tx_wr[0].next = NULL; + ret = ib_post_send(tx->qp, priv->tx_wr, &bad_wr); + priv->tx_wr[0].next = &priv->tx_wr[1]; - return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); + return ret; } void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_tx *tx) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib_multicast.c new/drivers/infiniband/ulp/ipoib/ipoib_multicast.c --- org/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2007-08-22 08:33:51.000000000 +0530 @@ -217,7 +217,7 @@ static int ipoib_mcast_join_finish(struc if (!memcmp(mcast->mcmember.mgid.raw, priv->dev->broadcast + 4, sizeof (union ib_gid))) { priv->qkey = be32_to_cpu(priv->broadcast->mcmember.qkey); - priv->tx_wr.wr.ud.remote_qkey = priv->qkey; + priv->tx_wr[0].wr.ud.remote_qkey = priv->qkey; } if (!test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) { @@ -736,7 +736,7 @@ out: } } - ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN); + ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN, 1); } unlock: From krkumar2 at in.ibm.com Wed Aug 22 01:31:10 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:01:10 +0530 Subject: [ofa-general] [PATCH 8/10 Rev4] [IPoIB] Post and work completion handler changes In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083110.11964.81864.sendpatchset@localhost.localdomain> IPoIB internal post and work completion handler changes. Signed-off-by: Krishna Kumar --- ipoib_ib.c | 207 ++++++++++++++++++++++++++++++++++++++++++++++++------------- 1 files changed, 163 insertions(+), 44 deletions(-) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib_ib.c new/drivers/infiniband/ulp/ipoib/ipoib_ib.c --- org/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2007-08-22 08:33:51.000000000 +0530 @@ -242,6 +242,8 @@ repost: static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) { struct ipoib_dev_priv *priv = netdev_priv(dev); + int i, num_completions; + unsigned int tx_ring_index; unsigned int wr_id = wc->wr_id; struct ipoib_tx_buf *tx_req; unsigned long flags; @@ -255,18 +257,46 @@ static void ipoib_ib_handle_tx_wc(struct return; } - tx_req = &priv->tx_ring[wr_id]; + /* Get first WC to process (no one can update tx_tail at this time) */ + tx_ring_index = priv->tx_tail & (ipoib_sendq_size - 1); - ib_dma_unmap_single(priv->ca, tx_req->mapping, - tx_req->skb->len, DMA_TO_DEVICE); + /* Find number of WC's */ + num_completions = wr_id - tx_ring_index + 1; + if (unlikely(num_completions <= 0)) + num_completions += ipoib_sendq_size; - ++priv->stats.tx_packets; - priv->stats.tx_bytes += tx_req->skb->len; + /* + * Handle WC's from earlier (possibly multiple) post_sends in this + * iteration as we move from tx_tail to wr_id, since if the last WR + * (which is the one which requested completion notification) failed + * to be sent for any of those earlier request(s), no completion + * notification is generated for successful WR's of those earlier + * request(s). + */ + tx_req = &priv->tx_ring[tx_ring_index]; + for (i = 0; i < num_completions; i++) { + if (likely(tx_req->skb)) { + ib_dma_unmap_single(priv->ca, tx_req->mapping, + tx_req->skb->len, DMA_TO_DEVICE); + + ++priv->stats.tx_packets; + priv->stats.tx_bytes += tx_req->skb->len; - dev_kfree_skb_any(tx_req->skb); + dev_kfree_skb_any(tx_req->skb); + } + /* + * else this skb failed synchronously when posted and was + * freed immediately. + */ + + if (likely(++tx_ring_index != ipoib_sendq_size)) + tx_req++; + else + tx_req = &priv->tx_ring[0]; + } spin_lock_irqsave(&priv->tx_lock, flags); - ++priv->tx_tail; + priv->tx_tail += num_completions; if (unlikely(test_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags)) && priv->tx_head - priv->tx_tail <= ipoib_sendq_size >> 1) { clear_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); @@ -335,29 +365,57 @@ void ipoib_ib_completion(struct ib_cq *c netif_rx_schedule(dev, &priv->napi); } -static inline int post_send(struct ipoib_dev_priv *priv, - unsigned int wr_id, - struct ib_ah *address, u32 qpn, - u64 addr, int len) +/* + * post_send : Post WR(s) to the device. + * + * num_skbs is the number of WR's, first_wr is the first slot in tx_wr[] (or + * tx_sge[]). first_wr is normally zero unless a previous post_send returned + * error and we are trying to post the untried WR's, in which case first_wr + * is the index to the first untried WR. + * + * Break the WR link before posting so that provider knows how many WR's to + * process, and this is set back after the post. + */ +static inline int post_send(struct ipoib_dev_priv *priv, u32 qpn, + int first_wr, int num_skbs, + struct ib_send_wr **bad_wr) { - struct ib_send_wr *bad_wr; + int ret; + struct ib_send_wr *last_wr, *next_wr; + + last_wr = &priv->tx_wr[first_wr + num_skbs - 1]; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; + /* Set Completion Notification for last WR */ + last_wr->send_flags = IB_SEND_SIGNALED; - priv->tx_wr.wr_id = wr_id; - priv->tx_wr.wr.ud.remote_qpn = qpn; - priv->tx_wr.wr.ud.ah = address; + /* Terminate the last WR */ + next_wr = last_wr->next; + last_wr->next = NULL; - return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); + /* Send all the WR's in one doorbell */ + ret = ib_post_send(priv->qp, &priv->tx_wr[first_wr], bad_wr); + + /* Restore send_flags & WR chain */ + last_wr->send_flags = 0; + last_wr->next = next_wr; + + return ret; } -void ipoib_send(struct net_device *dev, struct sk_buff *skb, - struct ipoib_ah *address, u32 qpn) +/* + * Map skb & store skb/mapping in tx_ring; and details of the WR in tx_wr + * to pass to the provider. + * + * Returns: + * 1: Error and the skb is freed. + * 0 skb processed successfully. + */ +int ipoib_process_skb(struct net_device *dev, struct sk_buff *skb, + struct ipoib_dev_priv *priv, struct ipoib_ah *address, + u32 qpn, int wr_num) { - struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ipoib_tx_buf *tx_req; u64 addr; + unsigned int tx_ring_index; if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -365,7 +423,7 @@ void ipoib_send(struct net_device *dev, ++priv->stats.tx_dropped; ++priv->stats.tx_errors; ipoib_cm_skb_too_long(dev, skb, priv->mcast_mtu); - return; + return 1; } ipoib_dbg_data(priv, "sending packet, length=%d address=%p qpn=0x%06x\n", @@ -378,35 +436,96 @@ void ipoib_send(struct net_device *dev, * means we have to make sure everything is properly recorded and * our state is consistent before we call post_send(). */ - tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; - tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, - DMA_TO_DEVICE); + addr = ib_dma_map_single(priv->ca, skb->data, skb->len, DMA_TO_DEVICE); if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); - return; + return 1; } - tx_req->mapping = addr; - if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, addr, skb->len))) { - ipoib_warn(priv, "post_send failed\n"); - ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); - dev_kfree_skb_any(skb); - } else { - dev->trans_start = jiffies; + tx_ring_index = priv->tx_head & (ipoib_sendq_size - 1); - address->last_send = priv->tx_head; - ++priv->tx_head; + /* Save till completion handler executes */ + priv->tx_ring[tx_ring_index].skb = skb; + priv->tx_ring[tx_ring_index].mapping = addr; + + /* Set WR values for the provider to use */ + priv->tx_sge[wr_num].addr = addr; + priv->tx_sge[wr_num].length = skb->len; + + priv->tx_wr[wr_num].wr_id = tx_ring_index; + priv->tx_wr[wr_num].wr.ud.remote_qpn = qpn; + priv->tx_wr[wr_num].wr.ud.ah = address->ah; + + priv->tx_head++; + + if (unlikely(priv->tx_head - priv->tx_tail == ipoib_sendq_size)) { + ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); + netif_stop_queue(dev); + set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + } - if (priv->tx_head - priv->tx_tail == ipoib_sendq_size) { - ipoib_dbg(priv, "TX ring full, stopping kernel net queue\n"); - netif_stop_queue(dev); - set_bit(IPOIB_FLAG_NETIF_STOPPED, &priv->flags); + return 0; +} + +/* + * Send num_skbs to the device. If an skb is passed to this function, it is + * single, unprocessed skb send case; otherwise it means that all skbs are + * already processed and put on priv->tx_wr,tx_sge,tx_ring, etc. + */ +void ipoib_send(struct net_device *dev, struct sk_buff *skb, + struct ipoib_ah *address, u32 qpn, int num_skbs) +{ + struct ipoib_dev_priv *priv = netdev_priv(dev); + int first_wr = 0; + + if (skb && ipoib_process_skb(dev, skb, priv, address, qpn, 0)) + return; + + /* Send all skb's in one post */ + do { + struct ib_send_wr *bad_wr; + + if (unlikely((post_send(priv, qpn, first_wr, num_skbs, + &bad_wr)))) { + int done; + + ipoib_warn(priv, "post_send failed\n"); + + /* Get number of WR's that finished successfully */ + done = bad_wr - &priv->tx_wr[first_wr]; + + /* Handle 1 error */ + priv->stats.tx_errors++; + ib_dma_unmap_single(priv->ca, + priv->tx_sge[first_wr + done].addr, + priv->tx_sge[first_wr + done].length, + DMA_TO_DEVICE); + + /* Free failed WR & reset for WC handler to recognize */ + dev_kfree_skb_any(priv->tx_ring[bad_wr->wr_id].skb); + priv->tx_ring[bad_wr->wr_id].skb = NULL; + + /* Handle 'n' successes */ + if (done) { + dev->trans_start = jiffies; + address->last_send = priv->tx_head - (num_skbs - + done) - 1; + } + + /* Get count of skbs that were not tried */ + num_skbs -= (done + 1); + /* + 1 for WR that was tried & failed */ + + /* Get start index for next iteration */ + first_wr += (done + 1); + } else { + dev->trans_start = jiffies; + + address->last_send = priv->tx_head - 1; + num_skbs = 0; } - } + } while (num_skbs); } static void __ipoib_reap_ah(struct net_device *dev) From krkumar2 at in.ibm.com Wed Aug 22 01:31:48 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:01:48 +0530 Subject: [ofa-general] [PATCH 10/10 Rev4] [E1000] Implement batching In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083148.11964.50427.sendpatchset@localhost.localdomain> E1000: Implement batching capability (ported thanks to changes taken from Jamal). Not all changes are made in this as in IPoIB, eg, handling out of order skbs (see XXX in the first mail). Signed-off-by: Krishna Kumar --- e1000_main.c | 150 +++++++++++++++++++++++++++++++++++++++++++++++------------ 1 files changed, 121 insertions(+), 29 deletions(-) diff -ruNp org/drivers/net/e1000/e1000_main.c new/drivers/net/e1000/e1000_main.c --- org/drivers/net/e1000/e1000_main.c 2007-08-20 14:26:29.000000000 +0530 +++ new/drivers/net/e1000/e1000_main.c 2007-08-22 08:33:51.000000000 +0530 @@ -157,6 +157,7 @@ static void e1000_update_phy_info(unsign static void e1000_watchdog(unsigned long data); static void e1000_82547_tx_fifo_stall(unsigned long data); static int e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev); +static int e1000_xmit_frames(struct net_device *dev); static struct net_device_stats * e1000_get_stats(struct net_device *netdev); static int e1000_change_mtu(struct net_device *netdev, int new_mtu); static int e1000_set_mac(struct net_device *netdev, void *p); @@ -990,7 +991,7 @@ e1000_probe(struct pci_dev *pdev, if (pci_using_dac) netdev->features |= NETIF_F_HIGHDMA; - netdev->features |= NETIF_F_LLTX; + netdev->features |= NETIF_F_LLTX | NETIF_F_BATCH_SKBS; adapter->en_mng_pt = e1000_enable_mng_pass_thru(&adapter->hw); @@ -3098,6 +3099,18 @@ e1000_tx_map(struct e1000_adapter *adapt return count; } +static void e1000_kick_DMA(struct e1000_adapter *adapter, + struct e1000_tx_ring *tx_ring, int i) +{ + wmb(); + + writel(i, adapter->hw.hw_addr + tx_ring->tdt); + /* we need this if more than one processor can write to our tail + * at a time, it syncronizes IO on IA64/Altix systems */ + mmiowb(); +} + + static void e1000_tx_queue(struct e1000_adapter *adapter, struct e1000_tx_ring *tx_ring, int tx_flags, int count) @@ -3144,13 +3157,7 @@ e1000_tx_queue(struct e1000_adapter *ada * know there are new descriptors to fetch. (Only * applicable for weak-ordered memory model archs, * such as IA-64). */ - wmb(); - tx_ring->next_to_use = i; - writel(i, adapter->hw.hw_addr + tx_ring->tdt); - /* we need this if more than one processor can write to our tail - * at a time, it syncronizes IO on IA64/Altix systems */ - mmiowb(); } /** @@ -3257,21 +3264,31 @@ static int e1000_maybe_stop_tx(struct ne } #define TXD_USE_COUNT(S, X) (((S) >> (X)) + 1 ) + +struct e1000_tx_cbdata { + int count; + unsigned int max_per_txd; + unsigned int nr_frags; + unsigned int mss; +}; + +#define E1000_SKB_CB(__skb) ((struct e1000_tx_cbdata *)&((__skb)->cb[0])) +#define NETDEV_TX_DROPPED -5 + static int -e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev) +e1000_prep_queue_frame(struct sk_buff *skb, struct net_device *netdev) { struct e1000_adapter *adapter = netdev_priv(netdev); struct e1000_tx_ring *tx_ring; - unsigned int first, max_per_txd = E1000_MAX_DATA_PER_TXD; + unsigned int max_per_txd = E1000_MAX_DATA_PER_TXD; unsigned int max_txd_pwr = E1000_MAX_TXD_PWR; - unsigned int tx_flags = 0; unsigned int len = skb->len; - unsigned long flags; - unsigned int nr_frags = 0; - unsigned int mss = 0; + unsigned int nr_frags; + unsigned int mss; int count = 0; - int tso; unsigned int f; + struct e1000_tx_cbdata *cb = E1000_SKB_CB(skb); + len -= skb->data_len; /* This goes back to the question of how to logically map a tx queue @@ -3282,7 +3299,7 @@ e1000_xmit_frame(struct sk_buff *skb, st if (unlikely(skb->len <= 0)) { dev_kfree_skb_any(skb); - return NETDEV_TX_OK; + return NETDEV_TX_DROPPED; } /* 82571 and newer doesn't need the workaround that limited descriptor @@ -3328,7 +3345,7 @@ e1000_xmit_frame(struct sk_buff *skb, st DPRINTK(DRV, ERR, "__pskb_pull_tail failed.\n"); dev_kfree_skb_any(skb); - return NETDEV_TX_OK; + return NETDEV_TX_DROPPED; } len = skb->len - skb->data_len; break; @@ -3372,22 +3389,32 @@ e1000_xmit_frame(struct sk_buff *skb, st (adapter->hw.mac_type == e1000_82573)) e1000_transfer_dhcp_info(adapter, skb); - if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) - /* Collision - tell upper layer to requeue */ - return NETDEV_TX_LOCKED; + cb->count = count; + cb->max_per_txd = max_per_txd; + cb->nr_frags = nr_frags; + cb->mss = mss; + + return NETDEV_TX_OK; +} + +static int e1000_queue_frame(struct sk_buff *skb, struct net_device *netdev) +{ + struct e1000_adapter *adapter = netdev_priv(netdev); + struct e1000_tx_ring *tx_ring = adapter->tx_ring; + int tso; + unsigned int first; + unsigned int tx_flags = 0; + struct e1000_tx_cbdata *cb = E1000_SKB_CB(skb); /* need: count + 2 desc gap to keep tail from touching * head, otherwise try next time */ - if (unlikely(e1000_maybe_stop_tx(netdev, tx_ring, count + 2))) { - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); + if (unlikely(e1000_maybe_stop_tx(netdev, tx_ring, cb->count + 2))) return NETDEV_TX_BUSY; - } if (unlikely(adapter->hw.mac_type == e1000_82547)) { if (unlikely(e1000_82547_fifo_workaround(adapter, skb))) { netif_stop_queue(netdev); mod_timer(&adapter->tx_fifo_stall_timer, jiffies + 1); - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); return NETDEV_TX_BUSY; } } @@ -3402,8 +3429,7 @@ e1000_xmit_frame(struct sk_buff *skb, st tso = e1000_tso(adapter, tx_ring, skb); if (tso < 0) { dev_kfree_skb_any(skb); - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); - return NETDEV_TX_OK; + return NETDEV_TX_DROPPED; } if (likely(tso)) { @@ -3420,12 +3446,78 @@ e1000_xmit_frame(struct sk_buff *skb, st e1000_tx_queue(adapter, tx_ring, tx_flags, e1000_tx_map(adapter, tx_ring, skb, first, - max_per_txd, nr_frags, mss)); + cb->max_per_txd, cb->nr_frags, cb->mss)); + + return NETDEV_TX_OK; +} - netdev->trans_start = jiffies; +static inline int e1000_xmit_frames(struct net_device *netdev) +{ + struct e1000_adapter *adapter = netdev->priv; + struct e1000_tx_ring *tx_ring = adapter->tx_ring; + int ret = NETDEV_TX_OK; + int skbs_done = 0; + struct sk_buff *skb; + unsigned long flags; + + if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) { + /* Collision - tell upper layer to requeue */ + return NETDEV_TX_LOCKED; + } - /* Make sure there is space in the ring for the next send. */ - e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); + while ((skb = __skb_dequeue(netdev->skb_blist)) != NULL) { + if ((ret = e1000_prep_queue_frame(skb, netdev)) != NETDEV_TX_OK) + continue; + + ret = e1000_queue_frame(skb, netdev); + if (ret == NETDEV_TX_OK) { + skbs_done++; + } else { + if (ret == NETDEV_TX_BUSY) + __skb_queue_head(netdev->skb_blist, skb); + break; + } + } + + if (skbs_done) { + e1000_kick_DMA(adapter, tx_ring, adapter->tx_ring->next_to_use); + netdev->trans_start = jiffies; + e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); + } + + spin_unlock_irqrestore(&tx_ring->tx_lock, flags); + + if (ret == NETDEV_TX_DROPPED) + ret = NETDEV_TX_OK; + + return ret; +} + +static int e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev) +{ + struct e1000_adapter *adapter = netdev_priv(netdev); + struct e1000_tx_ring *tx_ring = adapter->tx_ring; + unsigned long flags; + + if (!skb) + return e1000_xmit_frames(netdev); + + if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) { + /* Collision - tell upper layer to requeue */ + return NETDEV_TX_LOCKED; + } + + if (unlikely(e1000_prep_queue_frame(skb, netdev) != NETDEV_TX_OK)) { + spin_unlock_irqrestore(&tx_ring->tx_lock, flags); + return NETDEV_TX_OK; + } + + if (e1000_queue_frame(skb, netdev) == NETDEV_TX_OK) { + e1000_kick_DMA(adapter, tx_ring, adapter->tx_ring->next_to_use); + netdev->trans_start = jiffies; + /* Make sure there is space in the ring for the next send. */ + e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); + } spin_unlock_irqrestore(&tx_ring->tx_lock, flags); return NETDEV_TX_OK; From krkumar2 at in.ibm.com Wed Aug 22 01:30:13 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:00:13 +0530 Subject: [ofa-general] [PATCH 5/10 Rev4] [IPoIB] Header file changes In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083013.11964.62026.sendpatchset@localhost.localdomain> IPoIB header file changes to use batching. Signed-off-by: Krishna Kumar --- ipoib.h | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib.h new/drivers/infiniband/ulp/ipoib/ipoib.h --- org/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib.h 2007-08-22 08:33:51.000000000 +0530 @@ -271,8 +271,8 @@ struct ipoib_dev_priv { struct ipoib_tx_buf *tx_ring; unsigned tx_head; unsigned tx_tail; - struct ib_sge tx_sge; - struct ib_send_wr tx_wr; + struct ib_sge *tx_sge; + struct ib_send_wr *tx_wr; struct ib_wc ibwc[IPOIB_NUM_WC]; @@ -367,8 +367,11 @@ static inline void ipoib_put_ah(struct i int ipoib_open(struct net_device *dev); int ipoib_add_pkey_attr(struct net_device *dev); +int ipoib_process_skb(struct net_device *dev, struct sk_buff *skb, + struct ipoib_dev_priv *priv, struct ipoib_ah *address, + u32 qpn, int wr_num); void ipoib_send(struct net_device *dev, struct sk_buff *skb, - struct ipoib_ah *address, u32 qpn); + struct ipoib_ah *address, u32 qpn, int num_skbs); void ipoib_reap_ah(struct work_struct *work); void ipoib_flush_paths(struct net_device *dev); From krkumar2 at in.ibm.com Wed Aug 22 01:29:54 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 13:59:54 +0530 Subject: [ofa-general] [PATCH 4/10 Rev4] [ethtool] Add ethtool support In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822082954.11964.9626.sendpatchset@localhost.localdomain> Add ethtool support to enable/disable batching. Signed-off-by: Krishna Kumar --- include/linux/ethtool.h | 2 ++ include/linux/netdevice.h | 2 ++ net/core/dev.c | 36 ++++++++++++++++++++++++++++++++++++ net/core/ethtool.c | 27 +++++++++++++++++++++++++++ 4 files changed, 67 insertions(+) diff -ruNp org/include/linux/ethtool.h new/include/linux/ethtool.h --- org/include/linux/ethtool.h 2007-08-20 14:26:35.000000000 +0530 +++ new/include/linux/ethtool.h 2007-08-22 08:37:35.000000000 +0530 @@ -440,6 +440,8 @@ struct ethtool_ops { #define ETHTOOL_SFLAGS 0x00000026 /* Set flags bitmap(ethtool_value) */ #define ETHTOOL_GPFLAGS 0x00000027 /* Get driver-private flags bitmap */ #define ETHTOOL_SPFLAGS 0x00000028 /* Set driver-private flags bitmap */ +#define ETHTOOL_GBATCH 0x00000029 /* Get Batching (ethtool_value) */ +#define ETHTOOL_SBATCH 0x00000030 /* Set Batching (ethtool_value) */ /* compatibility with older code */ #define SPARC_ETH_GSET ETHTOOL_GSET diff -ruNp org/include/linux/netdevice.h new/include/linux/netdevice.h --- org/include/linux/netdevice.h 2007-08-20 14:26:36.000000000 +0530 +++ new/include/linux/netdevice.h 2007-08-22 08:42:10.000000000 +0530 @@ -1152,6 +1152,8 @@ extern void dev_set_promiscuity(struct extern void dev_set_allmulti(struct net_device *dev, int inc); extern void netdev_state_change(struct net_device *dev); extern void netdev_features_change(struct net_device *dev); +extern int dev_change_tx_batch_skb(struct net_device *dev, + unsigned long new_batch_skb); /* Load a device via the kmod */ extern void dev_load(const char *name); extern void dev_mcast_init(void); diff -ruNp org/net/core/dev.c new/net/core/dev.c --- org/net/core/dev.c 2007-08-20 14:26:37.000000000 +0530 +++ new/net/core/dev.c 2007-08-22 10:49:22.000000000 +0530 @@ -908,6 +908,42 @@ static void free_batching(struct net_dev } } +int dev_change_tx_batch_skb(struct net_device *dev, unsigned long new_batch_skb) +{ + int ret = 0; + struct sk_buff_head *blist; + + if (!(dev->features & NETIF_F_BATCH_SKBS)) { + /* Driver doesn't support batching skb API */ + ret = -ENOTSUPP; + goto out; + } + + /* + * Check if new value is same as the current (paranoia to use !! for + * new_batch_skb as that should always be boolean). + */ + if (!!dev->skb_blist == !!new_batch_skb) + goto out; + + if (new_batch_skb && + (blist = kmalloc(sizeof *blist, GFP_KERNEL)) == NULL) { + ret = -ENOMEM; + goto out; + } + + spin_lock_bh(&dev->queue_lock); + if (new_batch_skb) { + skb_queue_head_init(blist); + dev->skb_blist = blist; + } else + free_batching(dev); + spin_unlock_bh(&dev->queue_lock); + +out: + return ret; +} + /** * dev_load - load a network module * @name: name of interface diff -ruNp org/net/core/ethtool.c new/net/core/ethtool.c --- org/net/core/ethtool.c 2007-08-20 14:26:37.000000000 +0530 +++ new/net/core/ethtool.c 2007-08-22 08:36:07.000000000 +0530 @@ -556,6 +556,26 @@ static int ethtool_set_gso(struct net_de return 0; } +static int ethtool_get_batch(struct net_device *dev, char __user *useraddr) +{ + struct ethtool_value edata = { ETHTOOL_GBATCH }; + + edata.data = dev->skb_blist != NULL; + if (copy_to_user(useraddr, &edata, sizeof(edata))) + return -EFAULT; + return 0; +} + +static int ethtool_set_batch(struct net_device *dev, char __user *useraddr) +{ + struct ethtool_value edata; + + if (copy_from_user(&edata, useraddr, sizeof(edata))) + return -EFAULT; + + return dev_change_tx_batch_skb(dev, edata.data); +} + static int ethtool_self_test(struct net_device *dev, char __user *useraddr) { struct ethtool_test test; @@ -813,6 +833,7 @@ int dev_ethtool(struct ifreq *ifr) case ETHTOOL_GGSO: case ETHTOOL_GFLAGS: case ETHTOOL_GPFLAGS: + case ETHTOOL_GBATCH: break; default: if (!capable(CAP_NET_ADMIN)) @@ -956,6 +977,12 @@ int dev_ethtool(struct ifreq *ifr) rc = ethtool_set_value(dev, useraddr, dev->ethtool_ops->set_priv_flags); break; + case ETHTOOL_GBATCH: + rc = ethtool_get_batch(dev, useraddr); + break; + case ETHTOOL_SBATCH: + rc = ethtool_set_batch(dev, useraddr); + break; default: rc = -EOPNOTSUPP; } From krkumar2 at in.ibm.com Wed Aug 22 01:30:51 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:00:51 +0530 Subject: [ofa-general] [PATCH 7/10 Rev4] [IPoIB] Verbs changes In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083051.11964.70799.sendpatchset@localhost.localdomain> IPoIB verb changes to use batching. Signed-off-by: Krishna Kumar --- ipoib_verbs.c | 23 ++++++++++++++--------- 1 files changed, 14 insertions(+), 9 deletions(-) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib_verbs.c new/drivers/infiniband/ulp/ipoib/ipoib_verbs.c --- org/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib_verbs.c 2007-08-22 08:33:51.000000000 +0530 @@ -152,11 +152,11 @@ int ipoib_transport_dev_init(struct net_ .max_send_sge = 1, .max_recv_sge = 1 }, - .sq_sig_type = IB_SIGNAL_ALL_WR, + .sq_sig_type = IB_SIGNAL_REQ_WR, /* 11.2.4.1 */ .qp_type = IB_QPT_UD }; - - int ret, size; + struct ib_send_wr *next_wr = NULL; + int i, ret, size; priv->pd = ib_alloc_pd(priv->ca); if (IS_ERR(priv->pd)) { @@ -197,12 +197,17 @@ int ipoib_transport_dev_init(struct net_ priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - priv->tx_sge.lkey = priv->mr->lkey; - - priv->tx_wr.opcode = IB_WR_SEND; - priv->tx_wr.sg_list = &priv->tx_sge; - priv->tx_wr.num_sge = 1; - priv->tx_wr.send_flags = IB_SEND_SIGNALED; + for (i = ipoib_sendq_size - 1; i >= 0; i--) { + priv->tx_sge[i].lkey = priv->mr->lkey; + priv->tx_wr[i].opcode = IB_WR_SEND; + priv->tx_wr[i].sg_list = &priv->tx_sge[i]; + priv->tx_wr[i].num_sge = 1; + priv->tx_wr[i].send_flags = 0; + + /* Link the list properly for provider to use */ + priv->tx_wr[i].next = next_wr; + next_wr = &priv->tx_wr[i]; + } return 0; From krkumar2 at in.ibm.com Wed Aug 22 01:31:29 2007 From: krkumar2 at in.ibm.com (Krishna Kumar) Date: Wed, 22 Aug 2007 14:01:29 +0530 Subject: [ofa-general] [PATCH 9/10 Rev4] [IPoIB] Implement batching In-Reply-To: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> Message-ID: <20070822083129.11964.88364.sendpatchset@localhost.localdomain> IPoIB: implement the new batching API. Signed-off-by: Krishna Kumar --- ipoib_main.c | 251 ++++++++++++++++++++++++++++++++++++++++------------------- 1 files changed, 171 insertions(+), 80 deletions(-) diff -ruNp org/drivers/infiniband/ulp/ipoib/ipoib_main.c new/drivers/infiniband/ulp/ipoib/ipoib_main.c --- org/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-20 14:26:26.000000000 +0530 +++ new/drivers/infiniband/ulp/ipoib/ipoib_main.c 2007-08-22 08:33:51.000000000 +0530 @@ -560,7 +560,8 @@ static void neigh_add_path(struct sk_buf goto err_drop; } } else - ipoib_send(dev, skb, path->ah, IPOIB_QPN(skb->dst->neighbour->ha)); + ipoib_send(dev, skb, path->ah, + IPOIB_QPN(skb->dst->neighbour->ha), 1); } else { neigh->ah = NULL; @@ -640,7 +641,7 @@ static void unicast_arp_send(struct sk_b ipoib_dbg(priv, "Send unicast ARP to %04x\n", be16_to_cpu(path->pathrec.dlid)); - ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr)); + ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr), 1); } else if ((path->query || !path_rec_start(dev, path)) && skb_queue_len(&path->queue) < IPOIB_MAX_PATH_REC_QUEUE) { /* put pseudoheader back on for next time */ @@ -654,105 +655,166 @@ static void unicast_arp_send(struct sk_b spin_unlock(&priv->lock); } +#define XMIT_PROCESSED_SKBS() \ + do { \ + if (wr_num) { \ + ipoib_send(dev, NULL, old_neigh->ah, old_qpn, \ + wr_num); \ + wr_num = 0; \ + } \ + } while (0) + static int ipoib_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); - struct ipoib_neigh *neigh; + struct sk_buff_head *blist; + int max_skbs, wr_num = 0; + u32 qpn, old_qpn = 0; + struct ipoib_neigh *neigh, *old_neigh = NULL; unsigned long flags; if (unlikely(!spin_trylock_irqsave(&priv->tx_lock, flags))) return NETDEV_TX_LOCKED; - /* - * Check if our queue is stopped. Since we have the LLTX bit - * set, we can't rely on netif_stop_queue() preventing our - * xmit function from being called with a full queue. - */ - if (unlikely(netif_queue_stopped(dev))) { - spin_unlock_irqrestore(&priv->tx_lock, flags); - return NETDEV_TX_BUSY; - } - - if (likely(skb->dst && skb->dst->neighbour)) { - if (unlikely(!*to_ipoib_neigh(skb->dst->neighbour))) { - ipoib_path_lookup(skb, dev); - goto out; - } + blist = dev->skb_blist; - neigh = *to_ipoib_neigh(skb->dst->neighbour); + if (!skb || (blist && skb_queue_len(blist))) { + /* + * Either batching xmit call, or single skb case but there are + * skbs already in the batch list from previous failure to + * xmit - send the earlier skbs first to avoid out of order. + */ + + if (skb) + __skb_queue_tail(blist, skb); + + /* + * Figure out how many skbs can be sent. This prevents the + * device getting full and avoids checking for stopped queue + * after each iteration. Now the queue can get stopped atmost + * after xmit of the last skb. + */ + max_skbs = ipoib_sendq_size - (priv->tx_head - priv->tx_tail); + skb = __skb_dequeue(blist); + } else { + blist = NULL; + max_skbs = 1; + } - if (ipoib_cm_get(neigh)) { - if (ipoib_cm_up(neigh)) { - ipoib_cm_send(dev, skb, ipoib_cm_get(neigh)); - goto out; - } - } else if (neigh->ah) { - if (unlikely(memcmp(&neigh->dgid.raw, - skb->dst->neighbour->ha + 4, - sizeof(union ib_gid)))) { - spin_lock(&priv->lock); - /* - * It's safe to call ipoib_put_ah() inside - * priv->lock here, because we know that - * path->ah will always hold one more reference, - * so ipoib_put_ah() will never do more than - * decrement the ref count. - */ - ipoib_put_ah(neigh->ah); - list_del(&neigh->list); - ipoib_neigh_free(dev, neigh); - spin_unlock(&priv->lock); + do { + if (likely(skb->dst && skb->dst->neighbour)) { + if (unlikely(!*to_ipoib_neigh(skb->dst->neighbour))) { + XMIT_PROCESSED_SKBS(); ipoib_path_lookup(skb, dev); - goto out; + continue; } - ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(skb->dst->neighbour->ha)); - goto out; - } - - if (skb_queue_len(&neigh->queue) < IPOIB_MAX_PATH_REC_QUEUE) { - spin_lock(&priv->lock); - __skb_queue_tail(&neigh->queue, skb); - spin_unlock(&priv->lock); - } else { - ++priv->stats.tx_dropped; - dev_kfree_skb_any(skb); - } - } else { - struct ipoib_pseudoheader *phdr = - (struct ipoib_pseudoheader *) skb->data; - skb_pull(skb, sizeof *phdr); + neigh = *to_ipoib_neigh(skb->dst->neighbour); - if (phdr->hwaddr[4] == 0xff) { - /* Add in the P_Key for multicast*/ - phdr->hwaddr[8] = (priv->pkey >> 8) & 0xff; - phdr->hwaddr[9] = priv->pkey & 0xff; + if (ipoib_cm_get(neigh)) { + if (ipoib_cm_up(neigh)) { + XMIT_PROCESSED_SKBS(); + ipoib_cm_send(dev, skb, + ipoib_cm_get(neigh)); + continue; + } + } else if (neigh->ah) { + if (unlikely(memcmp(&neigh->dgid.raw, + skb->dst->neighbour->ha + 4, + sizeof(union ib_gid)))) { + spin_lock(&priv->lock); + /* + * It's safe to call ipoib_put_ah() + * inside priv->lock here, because we + * know that path->ah will always hold + * one more reference, so ipoib_put_ah() + * will never do more than decrement + * the ref count. + */ + ipoib_put_ah(neigh->ah); + list_del(&neigh->list); + ipoib_neigh_free(dev, neigh); + spin_unlock(&priv->lock); + XMIT_PROCESSED_SKBS(); + ipoib_path_lookup(skb, dev); + continue; + } + + qpn = IPOIB_QPN(skb->dst->neighbour->ha); + if (neigh != old_neigh || qpn != old_qpn) { + /* + * Sending to a different destination + * from earlier skb's (or this is the + * first skb) - send all existing skbs. + */ + XMIT_PROCESSED_SKBS(); + old_neigh = neigh; + old_qpn = qpn; + } + + if (likely(!ipoib_process_skb(dev, skb, priv, + neigh->ah, qpn, + wr_num))) + wr_num++; - ipoib_mcast_send(dev, phdr->hwaddr + 4, skb); - } else { - /* unicast GID -- should be ARP or RARP reply */ + continue; + } - if ((be16_to_cpup((__be16 *) skb->data) != ETH_P_ARP) && - (be16_to_cpup((__be16 *) skb->data) != ETH_P_RARP)) { - ipoib_warn(priv, "Unicast, no %s: type %04x, QPN %06x " - IPOIB_GID_FMT "\n", - skb->dst ? "neigh" : "dst", - be16_to_cpup((__be16 *) skb->data), - IPOIB_QPN(phdr->hwaddr), - IPOIB_GID_RAW_ARG(phdr->hwaddr + 4)); + if (skb_queue_len(&neigh->queue) < + IPOIB_MAX_PATH_REC_QUEUE) { + spin_lock(&priv->lock); + __skb_queue_tail(&neigh->queue, skb); + spin_unlock(&priv->lock); + } else { dev_kfree_skb_any(skb); ++priv->stats.tx_dropped; - goto out; + ++max_skbs; + } + } else { + struct ipoib_pseudoheader *phdr = + (struct ipoib_pseudoheader *) skb->data; + skb_pull(skb, sizeof *phdr); + + if (phdr->hwaddr[4] == 0xff) { + /* Add in the P_Key for multicast*/ + phdr->hwaddr[8] = (priv->pkey >> 8) & 0xff; + phdr->hwaddr[9] = priv->pkey & 0xff; + + XMIT_PROCESSED_SKBS(); + ipoib_mcast_send(dev, phdr->hwaddr + 4, skb); + } else { + /* unicast GID -- should be ARP or RARP reply */ + + if ((be16_to_cpup((__be16 *) skb->data) != + ETH_P_ARP) && + (be16_to_cpup((__be16 *) skb->data) != + ETH_P_RARP)) { + ipoib_warn(priv, "Unicast, no %s: type %04x, QPN %06x " + IPOIB_GID_FMT "\n", + skb->dst ? "neigh" : "dst", + be16_to_cpup((__be16 *) + skb->data), + IPOIB_QPN(phdr->hwaddr), + IPOIB_GID_RAW_ARG(phdr->hwaddr + + 4)); + dev_kfree_skb_any(skb); + ++priv->stats.tx_dropped; + ++max_skbs; + continue; + } + XMIT_PROCESSED_SKBS(); + unicast_arp_send(skb, dev, phdr); } - - unicast_arp_send(skb, dev, phdr); } - } + } while (--max_skbs && (skb = __skb_dequeue(blist)) != NULL); + + /* Send out last packets (if any) */ + XMIT_PROCESSED_SKBS(); -out: spin_unlock_irqrestore(&priv->tx_lock, flags); - return NETDEV_TX_OK; + return (!blist || !skb_queue_len(blist)) ? NETDEV_TX_OK : + NETDEV_TX_BUSY; } static struct net_device_stats *ipoib_get_stats(struct net_device *dev) @@ -900,11 +962,35 @@ int ipoib_dev_init(struct net_device *de /* priv->tx_head & tx_tail are already 0 */ - if (ipoib_ib_dev_init(dev, ca, port)) + /* Allocate tx_sge */ + priv->tx_sge = kmalloc(ipoib_sendq_size * sizeof *priv->tx_sge, + GFP_KERNEL); + if (!priv->tx_sge) { + printk(KERN_WARNING "%s: failed to allocate TX sge (%d entries)\n", + ca->name, ipoib_sendq_size); goto out_tx_ring_cleanup; + } + + /* Allocate tx_wr */ + priv->tx_wr = kmalloc(ipoib_sendq_size * sizeof *priv->tx_wr, + GFP_KERNEL); + if (!priv->tx_wr) { + printk(KERN_WARNING "%s: failed to allocate TX wr (%d entries)\n", + ca->name, ipoib_sendq_size); + goto out_tx_sge_cleanup; + } + + if (ipoib_ib_dev_init(dev, ca, port)) + goto out_tx_wr_cleanup; return 0; +out_tx_wr_cleanup: + kfree(priv->tx_wr); + +out_tx_sge_cleanup: + kfree(priv->tx_sge); + out_tx_ring_cleanup: kfree(priv->tx_ring); @@ -932,9 +1018,13 @@ void ipoib_dev_cleanup(struct net_device kfree(priv->rx_ring); kfree(priv->tx_ring); + kfree(priv->tx_sge); + kfree(priv->tx_wr); priv->rx_ring = NULL; priv->tx_ring = NULL; + priv->tx_sge = NULL; + priv->tx_wr = NULL; } static void ipoib_setup(struct net_device *dev) @@ -965,7 +1055,8 @@ static void ipoib_setup(struct net_devic dev->addr_len = INFINIBAND_ALEN; dev->type = ARPHRD_INFINIBAND; dev->tx_queue_len = ipoib_sendq_size * 2; - dev->features = NETIF_F_VLAN_CHALLENGED | NETIF_F_LLTX; + dev->features = NETIF_F_VLAN_CHALLENGED | NETIF_F_LLTX | + NETIF_F_BATCH_SKBS; /* MTU will be reset when mcast join happens */ dev->mtu = IPOIB_PACKET_SIZE - IPOIB_ENCAP_LEN; From ossrosch at linux.vnet.ibm.com Wed Aug 22 01:44:48 2007 From: ossrosch at linux.vnet.ibm.com (Stefan Roscher) Date: Wed, 22 Aug 2007 10:44:48 +0200 Subject: [ofa-general] Re: [GIT PULL] please pull infiniband.git for-linus branch In-Reply-To: References: <20070816064826.GB19432@mellanox.co.il> Message-ID: <200708221044.48933.ossrosch@linux.vnet.ibm.com> Hi Roland, On Thursday 16 August 2007 18:19, Roland Dreier wrote: > > Hmm, if these things are appropriate for 2.6.23, maybe my patch > > adding linux/mutex.h to ehca_classes.h can go in too? > > Actually I queued Dotan's patches quite a while ago, although Linus > seems to be ignoring my pull requests. > > I don't see any urgency in adding more similar patches to 2.6.23, > since AFAIK 2.6.23 builds fine, right? Yes, there is no build issue with 2.6.23. We have realized on rhel4.5 that linux/mutex.h needs to be included explicitly. Thus, I'm fine with your decision. > > Also I would prefer to get ehca patches from the ehca people at IBM, > or at least an ack from them. Acked-by: Stefan Roscher regards Stefan From vlad at lists.openfabrics.org Wed Aug 22 01:46:02 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 22 Aug 2007 01:46:02 -0700 (PDT) Subject: [ofa-general] ofa_1_2_kernel 20070822-0100 daily build status Message-ID: <20070822084602.6B5F0E60885@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.15 Passed on powerpc with linux-2.6.19 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Failed: From davem at davemloft.net Wed Aug 22 02:14:40 2007 From: davem at davemloft.net (David Miller) Date: Wed, 22 Aug 2007 02:14:40 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070821.212229.82050253.davem@davemloft.net> Message-ID: <20070822.021440.123973929.davem@davemloft.net> From: Krishna Kumar2 Date: Wed, 22 Aug 2007 12:33:04 +0530 > Does turning off batching solve that problem? What I mean by that is: > batching can be disabled if a TSO device is worse for some cases. This new batching stuff isn't going to be enabled or disabled on a per-device basis just to get "parity" with how things are now. It should be enabled by default, and give at least as good performance as what can be obtained right now. Otherwise it's a clear regression. From vlad at lists.openfabrics.org Wed Aug 22 02:46:57 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 22 Aug 2007 02:46:57 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070822-0200 daily build status Message-ID: <20070822094657.A7748E60876@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.14 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.22 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1294: error: 'struct net_device' has no member named 'dev' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1296: error: 'struct net_device' has no member named 'dev' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1298: error: 'struct net_device' has no member named 'dev' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband/ulp/ipoib] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-22.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:739: error: implicit declaration of function 'skb_is_gso' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c: In function 'ipoib_neigh_cleanup': /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:872: warning: ISO C90 forbids mixed declarations and code make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-22.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-22.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-34.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070822-0200_linux-2.6.9-34.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-34.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ogerlitz at voltaire.com Wed Aug 22 03:43:55 2007 From: ogerlitz at voltaire.com (Or Gerlitz) Date: Wed, 22 Aug 2007 13:43:55 +0300 Subject: [ofa-general] [PATCH - 11] ipoib - add LSO support In-Reply-To: <1187709349.30342.103.camel@mtls03> References: <1187202159.16253.195.camel@mtls03> <46C9B7B6.4060609@voltaire.com> <1187627087.30342.38.camel@mtls03> <46CAB58D.5090705@voltaire.com> <1187692432.30342.57.camel@mtls03> <46CAC3EF.6000509@voltaire.com> <1187709349.30342.103.camel@mtls03> Message-ID: <46CC136B.7050903@voltaire.com> Eli Cohen wrote: > > > Can you provide a pointer to some Mellanox high level doc/faq explaining > > the possible benefits from the stateless offloading features of connectX? > > The benefits of the stateless offload features can best be described by > numbers. I have been able to push throughput in datagram mode to 814 > MB/s. Details of the offload features are in the ConnectX prm. Hi Eli, Thanks for the feedback, the 814 MB/s number is quite impressive. Dror/Eitan - The question here is that on what scenario this code would in use when taking into account that the connected mode is now available and can provide ever higher throughput. My thinking was that this --can-- helpful when there is UDP multicast traffic of big packets, but Eli says you don't support it (UDP UFO) Again, I would be happy to get from Mellanox pointer to document which is not under NDA and describes the feature (eg Marketing stuff) Or. From chas at cmf.nrl.navy.mil Wed Aug 22 04:28:25 2007 From: chas at cmf.nrl.navy.mil (chas williams - CONTRACTOR) Date: Wed, 22 Aug 2007 07:28:25 -0400 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: Message-ID: <200708221128.l7MBSQ2H004759@cmf.nrl.navy.mil> In message ,"Su fficool, Stanley" writes: >I'm still researching where the rdma_atomic is getting the invalid >attribute. I will give an update if I find anything else. srp clients set this value as the responder_resources during login i believe. this seems to be done in srp_get_responder_resources() on the windows srp client. its taken from the qp attributes instead of picking a fixed number, like 4 in the linux version of the srp client. From swise at opengridcomputing.com Wed Aug 22 07:37:26 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Wed, 22 Aug 2007 09:37:26 -0500 Subject: [ofa-general] [Fwd: [GIT PULL] ofed-1.3 - libcxgb3 refresh] In-Reply-To: <46CAFC1C.6060103@opengridcomputing.com> References: <46CAFC1C.6060103@opengridcomputing.com> Message-ID: <46CC4A26.8020804@opengridcomputing.com> Vlad, I would also like this libcxgb3 release pulled into the ofed-1.2.5 libcxgb3 tree so subsequent weekly builds will include the bug fix. For ofed-1.2.5, please pull from git://git.openfabrics.org/~swise/libcxgb3 ofed_1_2_5 Thanks, Steve. Steve Wise wrote: > > > -------- Original Message -------- > Subject: [GIT PULL] ofed-1.3 - libcxgb3 refresh > Date: Tue, 21 Aug 2007 09:17:23 -0500 > From: Steve Wise > To: Vladimir Sokolovsky > CC: OpenFabricsEWG , OpenFabrics General > > > Vlad, > > I have a new release of libcxgb3 that I want included in ofed-1.3. > Changes since ofed-1.2 are 1 bug fix (bug 703), some cleanup on the spec > file as well as adding a ChangeLog file (bug 707). > > This is release 1.0.1 of libcxgb3... > > Pull from git://git.openfabrics.org/~swise/libcxgb3 ofed_1_3 > > Thanks, > > Steve. > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general From auke-jan.h.kok at intel.com Wed Aug 22 07:39:31 2007 From: auke-jan.h.kok at intel.com (Kok, Auke) Date: Wed, 22 Aug 2007 07:39:31 -0700 Subject: [ofa-general] Re: [PATCH 10/10 Rev4] [E1000] Implement batching In-Reply-To: <20070822083148.11964.50427.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> <20070822083148.11964.50427.sendpatchset@localhost.localdomain> Message-ID: <46CC4AA3.5030607@intel.com> Krishna Kumar wrote: > E1000: Implement batching capability (ported thanks to changes taken from > Jamal). Not all changes are made in this as in IPoIB, eg, handling > out of order skbs (see XXX in the first mail). > > Signed-off-by: Krishna Kumar > --- > e1000_main.c | 150 +++++++++++++++++++++++++++++++++++++++++++++++------------ > 1 files changed, 121 insertions(+), 29 deletions(-) Krishna, while I appreciate the patch I would have preferred a patch to e1000e. Not only does the e1000e driver remove a lot of the workarounds for old silicon, it is also a good way for us to move the current e1000 driver into a bit more stable maintenance mode. Do you think you can write this patch for e1000e instead? code-wise a lot of things are still the same, so your patch should be relatively easy to generate. e1000e currently lives in a branch from jeff garzik's netdev-2.6 tree Thanks, Auke > > diff -ruNp org/drivers/net/e1000/e1000_main.c new/drivers/net/e1000/e1000_main.c > --- org/drivers/net/e1000/e1000_main.c 2007-08-20 14:26:29.000000000 +0530 > +++ new/drivers/net/e1000/e1000_main.c 2007-08-22 08:33:51.000000000 +0530 > @@ -157,6 +157,7 @@ static void e1000_update_phy_info(unsign > static void e1000_watchdog(unsigned long data); > static void e1000_82547_tx_fifo_stall(unsigned long data); > static int e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev); > +static int e1000_xmit_frames(struct net_device *dev); > static struct net_device_stats * e1000_get_stats(struct net_device *netdev); > static int e1000_change_mtu(struct net_device *netdev, int new_mtu); > static int e1000_set_mac(struct net_device *netdev, void *p); > @@ -990,7 +991,7 @@ e1000_probe(struct pci_dev *pdev, > if (pci_using_dac) > netdev->features |= NETIF_F_HIGHDMA; > > - netdev->features |= NETIF_F_LLTX; > + netdev->features |= NETIF_F_LLTX | NETIF_F_BATCH_SKBS; > > adapter->en_mng_pt = e1000_enable_mng_pass_thru(&adapter->hw); > > @@ -3098,6 +3099,18 @@ e1000_tx_map(struct e1000_adapter *adapt > return count; > } > > +static void e1000_kick_DMA(struct e1000_adapter *adapter, > + struct e1000_tx_ring *tx_ring, int i) > +{ > + wmb(); > + > + writel(i, adapter->hw.hw_addr + tx_ring->tdt); > + /* we need this if more than one processor can write to our tail > + * at a time, it syncronizes IO on IA64/Altix systems */ > + mmiowb(); > +} > + > + > static void > e1000_tx_queue(struct e1000_adapter *adapter, struct e1000_tx_ring *tx_ring, > int tx_flags, int count) > @@ -3144,13 +3157,7 @@ e1000_tx_queue(struct e1000_adapter *ada > * know there are new descriptors to fetch. (Only > * applicable for weak-ordered memory model archs, > * such as IA-64). */ > - wmb(); > - > tx_ring->next_to_use = i; > - writel(i, adapter->hw.hw_addr + tx_ring->tdt); > - /* we need this if more than one processor can write to our tail > - * at a time, it syncronizes IO on IA64/Altix systems */ > - mmiowb(); > } > > /** > @@ -3257,21 +3264,31 @@ static int e1000_maybe_stop_tx(struct ne > } > > #define TXD_USE_COUNT(S, X) (((S) >> (X)) + 1 ) > + > +struct e1000_tx_cbdata { > + int count; > + unsigned int max_per_txd; > + unsigned int nr_frags; > + unsigned int mss; > +}; > + > +#define E1000_SKB_CB(__skb) ((struct e1000_tx_cbdata *)&((__skb)->cb[0])) > +#define NETDEV_TX_DROPPED -5 > + > static int > -e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev) > +e1000_prep_queue_frame(struct sk_buff *skb, struct net_device *netdev) > { > struct e1000_adapter *adapter = netdev_priv(netdev); > struct e1000_tx_ring *tx_ring; > - unsigned int first, max_per_txd = E1000_MAX_DATA_PER_TXD; > + unsigned int max_per_txd = E1000_MAX_DATA_PER_TXD; > unsigned int max_txd_pwr = E1000_MAX_TXD_PWR; > - unsigned int tx_flags = 0; > unsigned int len = skb->len; > - unsigned long flags; > - unsigned int nr_frags = 0; > - unsigned int mss = 0; > + unsigned int nr_frags; > + unsigned int mss; > int count = 0; > - int tso; > unsigned int f; > + struct e1000_tx_cbdata *cb = E1000_SKB_CB(skb); > + > len -= skb->data_len; > > /* This goes back to the question of how to logically map a tx queue > @@ -3282,7 +3299,7 @@ e1000_xmit_frame(struct sk_buff *skb, st > > if (unlikely(skb->len <= 0)) { > dev_kfree_skb_any(skb); > - return NETDEV_TX_OK; > + return NETDEV_TX_DROPPED; > } > > /* 82571 and newer doesn't need the workaround that limited descriptor > @@ -3328,7 +3345,7 @@ e1000_xmit_frame(struct sk_buff *skb, st > DPRINTK(DRV, ERR, > "__pskb_pull_tail failed.\n"); > dev_kfree_skb_any(skb); > - return NETDEV_TX_OK; > + return NETDEV_TX_DROPPED; > } > len = skb->len - skb->data_len; > break; > @@ -3372,22 +3389,32 @@ e1000_xmit_frame(struct sk_buff *skb, st > (adapter->hw.mac_type == e1000_82573)) > e1000_transfer_dhcp_info(adapter, skb); > > - if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) > - /* Collision - tell upper layer to requeue */ > - return NETDEV_TX_LOCKED; > + cb->count = count; > + cb->max_per_txd = max_per_txd; > + cb->nr_frags = nr_frags; > + cb->mss = mss; > + > + return NETDEV_TX_OK; > +} > + > +static int e1000_queue_frame(struct sk_buff *skb, struct net_device *netdev) > +{ > + struct e1000_adapter *adapter = netdev_priv(netdev); > + struct e1000_tx_ring *tx_ring = adapter->tx_ring; > + int tso; > + unsigned int first; > + unsigned int tx_flags = 0; > + struct e1000_tx_cbdata *cb = E1000_SKB_CB(skb); > > /* need: count + 2 desc gap to keep tail from touching > * head, otherwise try next time */ > - if (unlikely(e1000_maybe_stop_tx(netdev, tx_ring, count + 2))) { > - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > + if (unlikely(e1000_maybe_stop_tx(netdev, tx_ring, cb->count + 2))) > return NETDEV_TX_BUSY; > - } > > if (unlikely(adapter->hw.mac_type == e1000_82547)) { > if (unlikely(e1000_82547_fifo_workaround(adapter, skb))) { > netif_stop_queue(netdev); > mod_timer(&adapter->tx_fifo_stall_timer, jiffies + 1); > - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > return NETDEV_TX_BUSY; > } > } > @@ -3402,8 +3429,7 @@ e1000_xmit_frame(struct sk_buff *skb, st > tso = e1000_tso(adapter, tx_ring, skb); > if (tso < 0) { > dev_kfree_skb_any(skb); > - spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > - return NETDEV_TX_OK; > + return NETDEV_TX_DROPPED; > } > > if (likely(tso)) { > @@ -3420,12 +3446,78 @@ e1000_xmit_frame(struct sk_buff *skb, st > > e1000_tx_queue(adapter, tx_ring, tx_flags, > e1000_tx_map(adapter, tx_ring, skb, first, > - max_per_txd, nr_frags, mss)); > + cb->max_per_txd, cb->nr_frags, cb->mss)); > + > + return NETDEV_TX_OK; > +} > > - netdev->trans_start = jiffies; > +static inline int e1000_xmit_frames(struct net_device *netdev) > +{ > + struct e1000_adapter *adapter = netdev->priv; > + struct e1000_tx_ring *tx_ring = adapter->tx_ring; > + int ret = NETDEV_TX_OK; > + int skbs_done = 0; > + struct sk_buff *skb; > + unsigned long flags; > + > + if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) { > + /* Collision - tell upper layer to requeue */ > + return NETDEV_TX_LOCKED; > + } > > - /* Make sure there is space in the ring for the next send. */ > - e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); > + while ((skb = __skb_dequeue(netdev->skb_blist)) != NULL) { > + if ((ret = e1000_prep_queue_frame(skb, netdev)) != NETDEV_TX_OK) > + continue; > + > + ret = e1000_queue_frame(skb, netdev); > + if (ret == NETDEV_TX_OK) { > + skbs_done++; > + } else { > + if (ret == NETDEV_TX_BUSY) > + __skb_queue_head(netdev->skb_blist, skb); > + break; > + } > + } > + > + if (skbs_done) { > + e1000_kick_DMA(adapter, tx_ring, adapter->tx_ring->next_to_use); > + netdev->trans_start = jiffies; > + e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); > + } > + > + spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > + > + if (ret == NETDEV_TX_DROPPED) > + ret = NETDEV_TX_OK; > + > + return ret; > +} > + > +static int e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev) > +{ > + struct e1000_adapter *adapter = netdev_priv(netdev); > + struct e1000_tx_ring *tx_ring = adapter->tx_ring; > + unsigned long flags; > + > + if (!skb) > + return e1000_xmit_frames(netdev); > + > + if (!spin_trylock_irqsave(&tx_ring->tx_lock, flags)) { > + /* Collision - tell upper layer to requeue */ > + return NETDEV_TX_LOCKED; > + } > + > + if (unlikely(e1000_prep_queue_frame(skb, netdev) != NETDEV_TX_OK)) { > + spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > + return NETDEV_TX_OK; > + } > + > + if (e1000_queue_frame(skb, netdev) == NETDEV_TX_OK) { > + e1000_kick_DMA(adapter, tx_ring, adapter->tx_ring->next_to_use); > + netdev->trans_start = jiffies; > + /* Make sure there is space in the ring for the next send. */ > + e1000_maybe_stop_tx(netdev, tx_ring, MAX_SKB_FRAGS + 2); > + } > > spin_unlock_irqrestore(&tx_ring->tx_lock, flags); > return NETDEV_TX_OK; > - > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From vlad at mellanox.co.il Wed Aug 22 08:22:55 2007 From: vlad at mellanox.co.il (Vladimir Sokolovsky) Date: Wed, 22 Aug 2007 18:22:55 +0300 Subject: [ofa-general] [Fwd: [GIT PULL] ofed-1.3 - libcxgb3 refresh] In-Reply-To: <46CC4A26.8020804@opengridcomputing.com> References: <46CAFC1C.6060103@opengridcomputing.com> <46CC4A26.8020804@opengridcomputing.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9021EB317@mtlexch01.mtl.com> Done, See git://git.openfabrics.org/ofed_1_2_5/libcxgb3.git Included in http://www.openfabrics.org/builds/connectx/OFED-1.2.5-20070822-0809.tgz Regards, Vladimir > -----Original Message----- > From: Steve Wise [mailto:swise at opengridcomputing.com] > Sent: Wednesday, August 22, 2007 5:37 PM > To: Vladimir Sokolovsky > Cc: OpenFabricsEWG; OpenFabrics General > Subject: Re: [ofa-general] [Fwd: [GIT PULL] ofed-1.3 - libcxgb3 > refresh] > > Vlad, > > I would also like this libcxgb3 release pulled into the ofed-1.2.5 > libcxgb3 tree so subsequent weekly builds will include the bug fix. > For > ofed-1.2.5, please pull from > > git://git.openfabrics.org/~swise/libcxgb3 ofed_1_2_5 > > Thanks, > > Steve. > > Steve Wise wrote: > > > > > > -------- Original Message -------- > > Subject: [GIT PULL] ofed-1.3 - libcxgb3 refresh > > Date: Tue, 21 Aug 2007 09:17:23 -0500 > > From: Steve Wise > > To: Vladimir Sokolovsky > > CC: OpenFabricsEWG , OpenFabrics General > > > > > > Vlad, > > > > I have a new release of libcxgb3 that I want included in ofed-1.3. > > Changes since ofed-1.2 are 1 bug fix (bug 703), some cleanup on the > spec > > file as well as adding a ChangeLog file (bug 707). > > > > This is release 1.0.1 of libcxgb3... > > > > Pull from git://git.openfabrics.org/~swise/libcxgb3 ofed_1_3 > > > > Thanks, > > > > Steve. > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general From randy.dunlap at oracle.com Wed Aug 22 08:50:13 2007 From: randy.dunlap at oracle.com (Randy Dunlap) Date: Wed, 22 Aug 2007 08:50:13 -0700 Subject: [ofa-general] Re: [PATCH 1/10 Rev4] [Doc] HOWTO Documentation for batching In-Reply-To: <20070822082858.11964.87377.sendpatchset@localhost.localdomain> References: <20070822082839.11964.63503.sendpatchset@localhost.localdomain> <20070822082858.11964.87377.sendpatchset@localhost.localdomain> Message-ID: <20070822085013.d20e1fd4.randy.dunlap@oracle.com> On Wed, 22 Aug 2007 13:58:58 +0530 Krishna Kumar wrote: > Add Documentation describing batching skb xmit capability. > > Signed-off-by: Krishna Kumar > --- > batching_skb_xmit.txt | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 78 insertions(+) > > diff -ruNp org/Documentation/networking/batching_skb_xmit.txt new/Documentation/networking/batching_skb_xmit.txt > --- org/Documentation/networking/batching_skb_xmit.txt 1970-01-01 05:30:00.000000000 +0530 > +++ new/Documentation/networking/batching_skb_xmit.txt 2007-08-22 10:21:19.000000000 +0530 > @@ -0,0 +1,78 @@ > + HOWTO for batching skb xmit support > + ----------------------------------- > + > +Section 1: What is batching skb xmit > +Section 2: How batching xmit works vs the regular xmit > +Section 3: How drivers can support batching > +Section 4: How users can work with batching > + > + > +Introduction: Kernel support for batching skb > +---------------------------------------------- > + > +A new capability to support xmit of multiple skbs is provided in the netdevice > +layer. Drivers which enable this capability should be able to process multiple > +skbs in a single call to their xmit handler. > + > + > +Section 1: What is batching skb xmit > +------------------------------------- > + > + This capability is optionally enabled by a driver by setting the > + NETIF_F_BATCH_SKBS bit in dev->features. The pre-requisite for a prerequisite > + driver to use this capability is that it should have a reasonably I would say "reasonably-sized". > + sized hardware queue that can process multiple skbs. > + > + > +Section 2: How batching xmit works vs the regular xmit > +------------------------------------------------------- > + > + The network stack gets called from upper layer protocols with a single > + skb to transmit. This skb is first enqueue'd and an attempt is made to enqueued > + transmit it immediately (via qdisc_run). However, events like tx lock > + contention, tx queue stopped, etc, can result in the skb not getting etc., > + sent out and it remains in the queue. When the next xmit is called or > + when the queue is re-enabled, qdisc_run could potentially find > + multiple packets in the queue, and iteratively send them all out > + one-by-one. > + > + Batching skb xmit is a mechanism to exploit this situation where all > + skbs can be passed in one shot to the device. This reduces driver > + processing, locking at the driver (or in stack for ~LLTX drivers) > + gets amortized over multiple skbs, and in case of specific drivers > + where every xmit results in a completion processing (like IPoIB) - > + optimizations can be made in the driver to request a completion for > + only the last skb that was sent which results in saving interrupts > + for every (but the last) skb that was sent in the same batch. > + > + Batching can result in significant performance gains for systems that > + have multiple data stream paths over the same network interface card. > + > + > +Section 3: How drivers can support batching > +--------------------------------------------- > + > + Batching requires the driver to set the NETIF_F_BATCH_SKBS bit in > + dev->features. > + > + The driver's xmit handler should be modified to process multiple skbs > + instead of one skb. The driver's xmit handler is called either with a an > + skb to transmit or NULL skb, where the latter case should be handled > + as a call to xmit multiple skbs. This is done by sending out all skbs > + in the dev->skb_blist list (where it was added by the core stack). > + > + > +Section 4: How users can work with batching > +--------------------------------------------- > + > + Batching can be disabled for a particular device, e.g. on desktop > + systems if only one stream of network activity for that device is > + taking place, since performance could be slightly affected due to > + extra processing that batching adds (unless packets are getting > + sent fast resulting in stopped queue's). Batching can be enabled if queues). > + more than one stream of network activity per device is being done, > + e.g. on servers; or even desktop usage with multiple browser, chat, > + file transfer sessions, etc. > + > + Per device batching can be enabled/disabled by passing 'on' or 'off' > + respectively to ethtool. with what other parameter(s), e.g., ethtool batching on/off ? --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** From ssufficool at rov.sbcounty.gov Wed Aug 22 09:20:51 2007 From: ssufficool at rov.sbcounty.gov (Sufficool, Stanley) Date: Wed, 22 Aug 2007 09:20:51 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: <200708221128.l7MBSQ2H004759@cmf.nrl.navy.mil> References: <200708221128.l7MBSQ2H004759@cmf.nrl.navy.mil> Message-ID: So the client is requesting a QP with a max rdma_atomic of 15, but the target initializes the QP with a max of 4. Is this max rdma_atomic of 4 based on hardware capabilities (Memory vs MemFree adapters), or a target QP initialization preset value? I'll swap adapters if needed. -----Original Message----- From: chas williams - CONTRACTOR [mailto:chas at cmf.nrl.navy.mil] Sent: Wednesday, August 22, 2007 4:28 AM To: Sufficool, Stanley Cc: Vu Pham; Stanley Sufficool; general Subject: Re: [ofa-general] [PATCH] SRPT for current SCST In message ,"Su fficool, Stanley" writes: >I'm still researching where the rdma_atomic is getting the invalid >attribute. I will give an update if I find anything else. srp clients set this value as the responder_resources during login i believe. this seems to be done in srp_get_responder_resources() on the windows srp client. its taken from the qp attributes instead of picking a fixed number, like 4 in the linux version of the srp client. From rick.jones2 at hp.com Wed Aug 22 10:09:37 2007 From: rick.jones2 at hp.com (Rick Jones) Date: Wed, 22 Aug 2007 10:09:37 -0700 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.212229.82050253.davem@davemloft.net> References: <20070821.115143.23013721.davem@davemloft.net> <20070821.212229.82050253.davem@davemloft.net> Message-ID: <46CC6DD1.5020105@hp.com> David Miller wrote: > I think the jury is still out, but seeing TSO perform even slightly > worse with the batching changes in place would be very worrysome. > This applies to both throughput and cpu utilization. Should it be any more or less worrysome than small packet performance (eg the TCP_RR stuff I posted recently) being rather worse with TSO enabled than with it disabled? rick jones From sashak at voltaire.com Wed Aug 22 12:05:19 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 22 Aug 2007 22:05:19 +0300 Subject: [ofa-general] ib_umad method mask problems on big-endian 64-bit archs In-Reply-To: References: Message-ID: <20070822190519.GD1397@sashak.voltaire.com> On 14:32 Mon 20 Aug , Roland Dreier wrote: > > I'd like to merge this, but unfortunately there is the complication that > opensm/libvendor/osm_vendor_ibumad.c has this: > > static int set_bit(int nr, void *method_mask) > { > int mask, retval; > long *addr = method_mask; > > addr += nr >> 5; > mask = 1 << (nr & 0x1f); > retval = (mask & *addr) != 0; > *addr |= mask; > return retval; > } > > which is actually wrong in general (if sizeof long is 8, then > offsetting by nr>>5 ends up making the subscript twice what it should > be), but all defined IB MAD methods are less than 32 so it ends up > being OK, and actually works for 64-bit big-endian userspace on a > 64-bit kernel. So the patch below would mean that 64-bit osm binaries > on ppc64 would break (32-bit osm binaries on powerpc are of course > broken with a 64-bit kernel). Still, perhaps the right thing to do is > to just fix the opensm code at the same time? > > Opinions? It looks that using uint32_t for addr in set_bit() function is sufficient fix. But for ppc64 this means that new OpenSM will break with old kernels, probably we will need to put some ugly #ifdef in osm_vendor_ibumad.c... Sasha From john.leidel at gmail.com Wed Aug 22 13:02:58 2007 From: john.leidel at gmail.com (John Leidel) Date: Wed, 22 Aug 2007 17:02:58 -0300 Subject: [ofa-general] MPI IB Errors Message-ID: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> All, in adding two new blade centers full of machines to my existing cluster install, I'm getting the following errors in trying to run MPI jobs over MVAPICH :: libibverbs: Fatal: no infiniband class devices found. No IB device found All machines are essentially the same archticture... 2.8Ghz opterons. All are running TopSpin/Cisco IB gear. The blade centers both has internal IB switches, to which I have connected to our existing TopSpin core switch. I disabled the subnet managers on both blade chassis IB switches as I'm running the subnet manager from the IB core switch. Any thoughts on what is going on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From davem at davemloft.net Wed Aug 22 13:21:45 2007 From: davem at davemloft.net (David Miller) Date: Wed, 22 Aug 2007 13:21:45 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <46CC6DD1.5020105@hp.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> Message-ID: <20070822.132145.90824527.davem@davemloft.net> From: Rick Jones Date: Wed, 22 Aug 2007 10:09:37 -0700 > Should it be any more or less worrysome than small packet > performance (eg the TCP_RR stuff I posted recently) being rather > worse with TSO enabled than with it disabled? That, like any such thing shown by the batching changes, is a bug to fix. From jsquyres at cisco.com Wed Aug 22 13:47:32 2007 From: jsquyres at cisco.com (Jeff Squyres) Date: Wed, 22 Aug 2007 16:47:32 -0400 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> Message-ID: <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> This typically means that the IB kernel drivers are not loaded. Are you running the same version of OFED on all of your blades? On Aug 22, 2007, at 4:02 PM, John Leidel wrote: > All, in adding two new blade centers full of machines to my > existing cluster install, I'm getting the following errors in > trying to run MPI jobs over MVAPICH :: > > libibverbs: Fatal: no infiniband class devices found. > No IB device found > > All machines are essentially the same archticture... 2.8Ghz > opterons. All are running TopSpin/Cisco IB gear. The blade > centers both has internal IB switches, to which I have connected to > our existing TopSpin core switch. I disabled the subnet managers > on both blade chassis IB switches as I'm running the subnet manager > from the IB core switch. > > > Any thoughts on what is going on? > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/ > openib-general -- Jeff Squyres Cisco Systems From sashak at voltaire.com Wed Aug 22 13:52:06 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 22 Aug 2007 23:52:06 +0300 Subject: [ofa-general] Re: [PATCH 1/7] osm: QoS - adding new PathRecord fields In-Reply-To: <46CAC87F.8010702@dev.mellanox.co.il> References: <46C982FA.2040601@dev.mellanox.co.il> <20070820165831.GG27000@sashak.voltaire.com> <46CAC87F.8010702@dev.mellanox.co.il> Message-ID: <20070822205206.GE1397@sashak.voltaire.com> Hi Yevgeny, On 14:11 Tue 21 Aug , Yevgeny Kliteynik wrote: > >> * > >> * RETURN VALUES > >> -* Encoded path pkt_life = 4.096 ?sec * 2 ** PacketLifeTime. > >> +* Encoded path pkt_life = 4.096 ??sec * 2 ** PacketLifeTime. > > It is not ascii. What is the change here? > > I think my diff tool has troubles handling unusual chars. > There's a '?' character there - it wasn't changed. > I guess it would be better to replace it with "micro" in > the future - less elegant, but also less error prone. Yes, just micro or usec look fine. > > Also it would be nice if newly added code will be formatted with > > opensm/osm_indent. Thanks. > > To tell you the truth, I'm not a big fan of this script. > Sometimes it does stuff it shouldn't. For instance, > I run it on osm_qos_policy.c, and this is what it does: > > Before: > > static osm_qos_match_rule_t * > __qos_policy_get_match_rule_by_pr( > const osm_pr_rcv_t * p_rcv, > const ib_path_rec_t * p_pr, > const osm_physp_t * p_src_physp, > const osm_physp_t * p_dest_physp, > ib_net64_t comp_mask) > > > After: > > static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr(const > osm_pr_rcv_t * > p_rcv, > const > ib_path_rec_t * > p_pr, > const osm_physp_t > * p_src_physp, > const osm_physp_t > * p_dest_physp, > ib_net64_t > comp_mask) > > Correct me if I'm wrong, but I think that the 'before' example > is closer to our coding style than the 'after'. Sure, indent program can do stupid things sometimes (mostly when fooled by long names). But main idea was to have consistent and readable source code. So both examples are bad from this point of view. Sasha From john.leidel at gmail.com Wed Aug 22 14:32:12 2007 From: john.leidel at gmail.com (John Leidel) Date: Wed, 22 Aug 2007 18:32:12 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> Message-ID: <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> Yeah, I'm sure the OFED release is the same. I'm running ROCKS 4.2.1, so all the node images are identical regarding the package selection. There could possibly be [well, probably] a difference in the firmware releases of the HCAs and switches from the older machines and the latest delivery. On 8/22/07, Jeff Squyres wrote: > > This typically means that the IB kernel drivers are not loaded. Are > you running the same version of OFED on all of your blades? > > > On Aug 22, 2007, at 4:02 PM, John Leidel wrote: > > > All, in adding two new blade centers full of machines to my > > existing cluster install, I'm getting the following errors in > > trying to run MPI jobs over MVAPICH :: > > > > libibverbs: Fatal: no infiniband class devices found. > > No IB device found > > > > All machines are essentially the same archticture... 2.8Ghz > > opterons. All are running TopSpin/Cisco IB gear. The blade > > centers both has internal IB switches, to which I have connected to > > our existing TopSpin core switch. I disabled the subnet managers > > on both blade chassis IB switches as I'm running the subnet manager > > from the IB core switch. > > > > > > Any thoughts on what is going on? > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/ > > openib-general > > > -- > Jeff Squyres > Cisco Systems > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at cisco.com Wed Aug 22 14:39:43 2007 From: jsquyres at cisco.com (Jeff Squyres) Date: Wed, 22 Aug 2007 17:39:43 -0400 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> Message-ID: <35ACDE37-CE27-48C8-B294-8E78B53C8015@cisco.com> On Aug 22, 2007, at 5:32 PM, John Leidel wrote: > Yeah, I'm sure the OFED release is the same. I'm running ROCKS > 4.2.1, so all the node images are identical regarding the package > selection. There could possibly be [well, probably] a difference > in the firmware releases of the HCAs and switches from the older > machines and the latest delivery. Try running ibv_devinfo on your new nodes, which should show you the HCA(s) on your host. I suspect that it will fail with a similar error (but am not 100% sure -- I'm the MPI guy, not the verbs guy :-) ). If this is the case, then you've got a bigger issue that your IB drivers are not loading. This will need to be fixed before you investigate the firmware level on your HCAs across the cluster. (other IB stack experts feel free to chime in...) -- Jeff Squyres Cisco Systems From john.leidel at gmail.com Wed Aug 22 14:45:36 2007 From: john.leidel at gmail.com (John Leidel) Date: Wed, 22 Aug 2007 18:45:36 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <35ACDE37-CE27-48C8-B294-8E78B53C8015@cisco.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> <35ACDE37-CE27-48C8-B294-8E78B53C8015@cisco.com> Message-ID: <27f776af0708221445w583004cat8379080abcad060f@mail.gmail.com> Err.... I don't seem to have any of the typical IB diagnostics tools. All they [ie, i didn't originally install this machine] had running in the first place were the IB modules resident with the 2.6.9-42 redhat kernel. I'm debating on whether I should go install the CISCO IB Roll to make sure I have all the necessary tools in place. On 8/22/07, Jeff Squyres wrote: > > On Aug 22, 2007, at 5:32 PM, John Leidel wrote: > > > Yeah, I'm sure the OFED release is the same. I'm running ROCKS > > 4.2.1, so all the node images are identical regarding the package > > selection. There could possibly be [well, probably] a difference > > in the firmware releases of the HCAs and switches from the older > > machines and the latest delivery. > > Try running ibv_devinfo on your new nodes, which should show you the > HCA(s) on your host. I suspect that it will fail with a similar > error (but am not 100% sure -- I'm the MPI guy, not the verbs guy :-) ). > > If this is the case, then you've got a bigger issue that your IB > drivers are not loading. This will need to be fixed before you > investigate the firmware level on your HCAs across the cluster. > > (other IB stack experts feel free to chime in...) > > -- > Jeff Squyres > Cisco Systems > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at cisco.com Wed Aug 22 14:52:58 2007 From: jsquyres at cisco.com (Jeff Squyres) Date: Wed, 22 Aug 2007 17:52:58 -0400 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708221445w583004cat8379080abcad060f@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> <35ACDE37-CE27-48C8-B294-8E78B53C8015@cisco.com> <27f776af0708221445w583004cat8379080abcad060f@mail.gmail.com> Message-ID: <3670FA04-1042-430D-9E77-CF3874421CF2@cisco.com> On Aug 22, 2007, at 5:45 PM, John Leidel wrote: > Err.... I don't seem to have any of the typical IB diagnostics > tools. All they [ie, i didn't originally install this machine] had > running in the first place were the IB modules resident with the > 2.6.9-42 redhat kernel. Is that RHEL4U4? I'm afraid I don't know what ships with RHEL4U4 by default; I always install the Cisco OFED distribution (which uninstalls the OFED that ships with RHEL). It might be OFED 1.1...? Try a "locate" for ibv_devinfo to see if it's installed where. > I'm debating on whether I should go install the CISCO IB Roll to > make sure I have all the necessary tools in place. I'm not entirely sure what's in there (again, I'm the MPI guy, not the IB stack guy :-)); it may be the Cisco proprietary IB stack, not the OpenFabrics stack. -- Jeff Squyres Cisco Systems From john.leidel at gmail.com Wed Aug 22 15:03:34 2007 From: john.leidel at gmail.com (John Leidel) Date: Wed, 22 Aug 2007 19:03:34 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <3670FA04-1042-430D-9E77-CF3874421CF2@cisco.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <88522618-51FF-471A-907A-3A2895DCEFB4@cisco.com> <27f776af0708221432r4b3cc49eq22a0d36c648a7e65@mail.gmail.com> <35ACDE37-CE27-48C8-B294-8E78B53C8015@cisco.com> <27f776af0708221445w583004cat8379080abcad060f@mail.gmail.com> <3670FA04-1042-430D-9E77-CF3874421CF2@cisco.com> Message-ID: <27f776af0708221503p573ff9c5sdf1889f270a4bf88@mail.gmail.com> I tried `finding` and `locating` the tools without any luck... looks like they were running on borrowed time regarding their IB install. I'm a bit hesitant to install the Roll, but I don't think I have much choice at this point. On 8/22/07, Jeff Squyres wrote: > > On Aug 22, 2007, at 5:45 PM, John Leidel wrote: > > > Err.... I don't seem to have any of the typical IB diagnostics > > tools. All they [ie, i didn't originally install this machine] had > > running in the first place were the IB modules resident with the > > 2.6.9-42 redhat kernel. > > Is that RHEL4U4? I'm afraid I don't know what ships with RHEL4U4 by > default; I always install the Cisco OFED distribution (which > uninstalls the OFED that ships with RHEL). It might be OFED 1.1...? > Try a "locate" for ibv_devinfo to see if it's installed where. > > > I'm debating on whether I should go install the CISCO IB Roll to > > make sure I have all the necessary tools in place. > > I'm not entirely sure what's in there (again, I'm the MPI guy, not > the IB stack guy :-)); it may be the Cisco proprietary IB stack, not > the OpenFabrics stack. > > -- > Jeff Squyres > Cisco Systems > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ssufficool at rov.sbcounty.gov Wed Aug 22 15:28:55 2007 From: ssufficool at rov.sbcounty.gov (Sufficool, Stanley) Date: Wed, 22 Aug 2007 15:28:55 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: <46C9EEF4.6000407@mellanox.com> References: <1186200658.6031.44.camel@gentoo-linux.localdomain> <46C9EEF4.6000407@mellanox.com> Message-ID: How did you get the SCST to compile against the current ofed 1.2.5 kernel tree? SCST svn fails with ofed 1.2.5 kernel 2.6.22 due to changes in the SLAB routines. Tried pruning the extra NULL arg to kmem_cache_create, and got the attached oopses when the target tried to register the SCST template. -----Original Message----- From: Vu Pham [mailto:vu at mellanox.com] Sent: Monday, August 20, 2007 12:44 PM To: Stanley Sufficool Cc: general Subject: Re: [ofa-general] [PATCH] SRPT for current SCST Thanks. I checked in with a few other changes/bug fixes. Please test and report any problem > SCST redefined the scst_tgt_template and it no longer contains the > tm_sync_reply field. Also an extra arg is required for scst_register. > Consequently, the build fails. > > I searched through past SCST versions and docs but could not find what > this was for and it seems to be set to zero (not used?) in this struct > anyhow. > > > ---------------------------------------------------------------------- > -- > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: crashnburn.txt URL: From weiny2 at llnl.gov Wed Aug 22 17:09:47 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Wed, 22 Aug 2007 17:09:47 -0700 Subject: [ofa-general] [PATCH] opensm configure: add perf-mgr-profile option and clean up "#if 0" debug code Message-ID: <20070822170947.662aab74.weiny2@llnl.gov> >From 5b22095ed47d62355c14441c3aaf7e9db7829f95 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Fri, 17 Aug 2007 14:47:58 -0700 Subject: [PATCH] opensm configure: add perf-mgr-profile option and clean up "#if 0" debug code This option does some time stamping on the perfmgr data colection and reports it under OSM_LOG_INFO level. Signed-off-by: Ira K. Weiny --- opensm/config/osmvsel.m4 | 16 ++++++- opensm/include/opensm/osm_madw.h | 2 +- opensm/opensm/osm_perfmgr.c | 97 ++++++++++++++++++++++++++++++-------- 3 files changed, 92 insertions(+), 23 deletions(-) diff --git a/opensm/config/osmvsel.m4 b/opensm/config/osmvsel.m4 index 3cdccc6..47ad36f 100644 --- a/opensm/config/osmvsel.m4 +++ b/opensm/config/osmvsel.m4 @@ -215,10 +215,22 @@ AC_ARG_ENABLE(perf-mgr, no) perf_mgr=no ;; esac], perf_mgr=no) +AC_ARG_ENABLE(perf-mgr-profile, +[ --enable-perf-mgr-profile Enable the performance manager profiling (default no)], + [case $enableval in + yes) perf_mgr_profile=yes ;; + no) perf_mgr_profile=no ;; + esac], + perf_mgr_profile=no) if test $perf_mgr = yes; then AC_DEFINE(ENABLE_OSM_PERF_MGR, - 1, - [Define as 1 if you want to enable the performance manager]) + 1, + [Define as 1 if you want to enable the performance manager]) + if test $perf_mgr_profile = yes; then + AC_DEFINE(ENABLE_OSM_PERF_MGR_PROFILE, + 1, + [Define as 1 if you want to enable the performance manager profiling code]) + fi fi # --- END OPENIB_OSM_PERF_MGR_SEL --- ]) dnl OPENIB_OSM_PERF_MGR_SEL diff --git a/opensm/include/opensm/osm_madw.h b/opensm/include/opensm/osm_madw.h index b996037..bdaa7bc 100644 --- a/opensm/include/opensm/osm_madw.h +++ b/opensm/include/opensm/osm_madw.h @@ -314,7 +314,7 @@ typedef struct _osm_perfmgr_context { uint64_t node_guid; uint16_t port; uint8_t mad_method; /* was this a get or a set */ -#if 0 +#if ENABLE_OSM_PERF_MGR_PROFILE struct timeval query_start; #endif } osm_perfmgr_context_t; diff --git a/opensm/opensm/osm_perfmgr.c b/opensm/opensm/osm_perfmgr.c index f1c6ca0..37baf7c 100644 --- a/opensm/opensm/osm_perfmgr.c +++ b/opensm/opensm/osm_perfmgr.c @@ -61,9 +61,62 @@ #include #include #include +#include #define OSM_PERFMGR_INITIAL_TID_VALUE 0xcafe +#if ENABLE_OSM_PERF_MGR_PROFILE +struct { + double fastest_us; + double slowest_us; + double avg_us; + uint64_t num; +} perfmgr_mad_stats = +{ + fastest_us: DBL_MAX, + slowest_us: DBL_MIN, + avg_us: 0, + num: 0 +}; + +/* diff must be something which can fit in a susecond_t */ +static inline void update_mad_stats(struct timeval *diff) +{ + double new = (diff->tv_sec * 1000000) + diff->tv_usec; + if (new < perfmgr_mad_stats.fastest_us) + perfmgr_mad_stats.fastest_us = new; + if (new > perfmgr_mad_stats.slowest_us) + perfmgr_mad_stats.slowest_us = new; + + perfmgr_mad_stats.avg_us = ((perfmgr_mad_stats.avg_us * perfmgr_mad_stats.num) + new) + /(perfmgr_mad_stats.num+1); + perfmgr_mad_stats.num++; +} + +static inline void perfmgr_clear_mad_stats(void) +{ + perfmgr_mad_stats.fastest_us = DBL_MAX; + perfmgr_mad_stats.slowest_us = DBL_MIN; + perfmgr_mad_stats.avg_us = 0; + perfmgr_mad_stats.num = 0; +} + +/* after and diff can be the same struct */ +static inline void diff_time(struct timeval *before, + struct timeval *after, + struct timeval *diff) +{ + struct timeval tmp = *after; + if (tmp.tv_usec < before->tv_usec) { + tmp.tv_sec--; + tmp.tv_usec += 1000000; + } + diff->tv_sec = tmp.tv_sec - before->tv_sec; + diff->tv_usec = tmp.tv_usec - before->tv_usec; +} + +#endif + /********************************************************************** * Internal helper functions. **********************************************************************/ @@ -130,16 +183,6 @@ osm_perfmgr_mad_recv_callback(osm_madw_t * p_madw, void *bind_context, "PerfMgr Dispatcher post failed\n"); osm_mad_pool_put(pm->mad_pool, p_madw); } -#if 0 - do { - struct timeval rcv_time; - gettimeofday(&rcv_time, NULL); - osm_log(pm->log, OSM_LOG_INFO, - "perfmgr rcv time %ld\n", - rcv_time.tv_usec - - p_madw->context.perfmgr_context.query_start.tv_usec); - } while (0); -#endif OSM_LOG_EXIT(pm->log); } @@ -506,7 +549,7 @@ __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) mad_context.perfmgr_context.node_guid = node_guid; mad_context.perfmgr_context.port = port; mad_context.perfmgr_context.mad_method = IB_MAD_METHOD_GET; -#if 0 +#if ENABLE_OSM_PERF_MGR_PROFILE gettimeofday(&(mad_context.perfmgr_context.query_start), NULL); #endif osm_log(pm->log, OSM_LOG_VERBOSE, @@ -554,7 +597,7 @@ void __osm_perfmgr_sweeper(void *p_ptr) */ if (pm->subn->sm_state == IB_SMINFO_STATE_MASTER && pm->state == PERFMGR_STATE_ENABLED) { -#if 0 +#if ENABLE_OSM_PERF_MGR_PROFILE struct timeval before, after; gettimeofday(&before, NULL); #endif @@ -579,11 +622,25 @@ void __osm_perfmgr_sweeper(void *p_ptr) * sweep */ __remove_marked_nodes(pm); -#if 0 + +#if ENABLE_OSM_PERF_MGR_PROFILE + /* spin on outstanding queries */ + while (pm->outstanding_queries > 0) + cl_event_wait_on(&pm->sig_sweep, 1000, TRUE); + gettimeofday(&after, NULL); + diff_time(&before, &after, &after); osm_log(pm->log, OSM_LOG_INFO, - "PerfMgr total sweep time : %ld us\n", - after.tv_usec - before.tv_usec); + "PerfMgr total sweep time : %ld.%06ld s\n" + " fastest mad : %g us\n" + " slowest mad : %g us\n" + " average mad : %g us\n" + , + after.tv_sec, after.tv_usec, + perfmgr_mad_stats.fastest_us, + perfmgr_mad_stats.slowest_us, + perfmgr_mad_stats.avg_us); + perfmgr_clear_mad_stats(); #endif } @@ -950,14 +1007,14 @@ static void osm_pc_rcv_process(void *context, void *data) osm_perfmgr_check_overflow(pm, node_guid, port, wire_read); -#if 0 +#if ENABLE_OSM_PERF_MGR_PROFILE do { struct timeval proc_time; gettimeofday(&proc_time, NULL); - osm_log(pm->log, OSM_LOG_INFO, - "PerfMgr done: processing time %ld\n", - proc_time.tv_usec - - p_madw->context.perfmgr_context.query_start.tv_usec); + diff_time(&(p_madw->context.perfmgr_context.query_start), + &proc_time, + &proc_time); + update_mad_stats(&proc_time); } while (0); #endif -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-opensm-configure-add-perf-mgr-profile-option-and-cl.patch Type: application/octet-stream Size: 6431 bytes Desc: not available URL: From krkumar2 at in.ibm.com Wed Aug 22 19:43:20 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 23 Aug 2007 08:13:20 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070822.021440.123973929.davem@davemloft.net> Message-ID: David Miller wrote on 08/22/2007 02:44:40 PM: > From: Krishna Kumar2 > Date: Wed, 22 Aug 2007 12:33:04 +0530 > > > Does turning off batching solve that problem? What I mean by that is: > > batching can be disabled if a TSO device is worse for some cases. > > This new batching stuff isn't going to be enabled or disabled > on a per-device basis just to get "parity" with how things are > now. > > It should be enabled by default, and give at least as good > performance as what can be obtained right now. That was how it was in earlier revisions. In revision4 I coded it so that it is enabled only if explicitly set by the user. I can revert that change. > Otherwise it's a clear regression. Definitely. For drivers that support it, it should not reduce performance. Thanks, - KK From krkumar2 at in.ibm.com Wed Aug 22 19:44:32 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 23 Aug 2007 08:14:32 +0530 Subject: [ofa-general] Re: [PATCH 10/10 Rev4] [E1000] Implement batching In-Reply-To: <46CC4AA3.5030607@intel.com> Message-ID: Hi Auke, "Kok, Auke" wrote on 08/22/2007 08:09:31 PM: > Krishna, > > while I appreciate the patch I would have preferred a patch to e1000e. Not only > does the e1000e driver remove a lot of the workarounds for old silicon, it is > also a good way for us to move the current e1000 driver into a bit more stable > maintenance mode. > > Do you think you can write this patch for e1000e instead? code-wise a lot of > things are still the same, so your patch should be relatively easy to generate. > > e1000e currently lives in a branch from jeff garzik's netdev-2.6 tree Definitely, I will pick it up and generate a patch. Thanks, - KK From krkumar2 at in.ibm.com Wed Aug 22 19:48:38 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Thu, 23 Aug 2007 08:18:38 +0530 Subject: [ofa-general] Re: [PATCH 1/10 Rev4] [Doc] HOWTO Documentation for batching In-Reply-To: <20070822085013.d20e1fd4.randy.dunlap@oracle.com> Message-ID: Hi Randy, Thanks for your suggestions. Will clean up those changes. - KK Randy Dunlap wrote on 08/22/2007 09:20:13 PM: > On Wed, 22 Aug 2007 13:58:58 +0530 Krishna Kumar wrote: > > > Add Documentation describing batching skb xmit capability. > > > > Signed-off-by: Krishna Kumar > > --- > > batching_skb_xmit.txt | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++ > > 1 files changed, 78 insertions(+) > > > > diff -ruNp org/Documentation/networking/batching_skb_xmit.txt > new/Documentation/networking/batching_skb_xmit.txt > > --- org/Documentation/networking/batching_skb_xmit.txt 1970-01-01 05:30: > 00.000000000 +0530 > > +++ new/Documentation/networking/batching_skb_xmit.txt 2007-08-22 10:21: > 19.000000000 +0530 > > @@ -0,0 +1,78 @@ > > + HOWTO for batching skb xmit support > > + ----------------------------------- > > + > > +Section 1: What is batching skb xmit > > +Section 2: How batching xmit works vs the regular xmit > > +Section 3: How drivers can support batching > > +Section 4: How users can work with batching > > + > > + > > +Introduction: Kernel support for batching skb > > +---------------------------------------------- > > + > > +A new capability to support xmit of multiple skbs is provided in the netdevice > > +layer. Drivers which enable this capability should be able to process multiple > > +skbs in a single call to their xmit handler. > > + > > + > > +Section 1: What is batching skb xmit > > +------------------------------------- > > + > > + This capability is optionally enabled by a driver by setting the > > + NETIF_F_BATCH_SKBS bit in dev->features. The pre-requisite for a > > prerequisite > > > + driver to use this capability is that it should have a reasonably > > I would say "reasonably-sized". > > > + sized hardware queue that can process multiple skbs. > > + > > + > > +Section 2: How batching xmit works vs the regular xmit > > +------------------------------------------------------- > > + > > + The network stack gets called from upper layer protocols with a single > > + skb to transmit. This skb is first enqueue'd and an attempt is made to > > enqueued > > > + transmit it immediately (via qdisc_run). However, events like tx lock > > + contention, tx queue stopped, etc, can result in the skb not getting > > etc., > > > + sent out and it remains in the queue. When the next xmit is called or > > + when the queue is re-enabled, qdisc_run could potentially find > > + multiple packets in the queue, and iteratively send them all out > > + one-by-one. > > + > > + Batching skb xmit is a mechanism to exploit this situation where all > > + skbs can be passed in one shot to the device. This reduces driver > > + processing, locking at the driver (or in stack for ~LLTX drivers) > > + gets amortized over multiple skbs, and in case of specific drivers > > + where every xmit results in a completion processing (like IPoIB) - > > + optimizations can be made in the driver to request a completion for > > + only the last skb that was sent which results in saving interrupts > > + for every (but the last) skb that was sent in the same batch. > > + > > + Batching can result in significant performance gains for systems that > > + have multiple data stream paths over the same network interface card. > > + > > + > > +Section 3: How drivers can support batching > > +--------------------------------------------- > > + > > + Batching requires the driver to set the NETIF_F_BATCH_SKBS bit in > > + dev->features. > > + > > + The driver's xmit handler should be modified to process multiple skbs > > + instead of one skb. The driver's xmit handler is called either with a > > an > > > + skb to transmit or NULL skb, where the latter case should be handled > > + as a call to xmit multiple skbs. This is done by sending out all skbs > > + in the dev->skb_blist list (where it was added by the core stack). > > + > > + > > +Section 4: How users can work with batching > > +--------------------------------------------- > > + > > + Batching can be disabled for a particular device, e.g. on desktop > > + systems if only one stream of network activity for that device is > > + taking place, since performance could be slightly affected due to > > + extra processing that batching adds (unless packets are getting > > + sent fast resulting in stopped queue's). Batching can be enabled if > > queues). > > > + more than one stream of network activity per device is being done, > > + e.g. on servers; or even desktop usage with multiple browser, chat, > > + file transfer sessions, etc. > > + > > + Per device batching can be enabled/disabled by passing 'on' or 'off' > > + respectively to ethtool. > > with what other parameter(s), e.g., > > ethtool batching on/off ? > > --- > ~Randy > *** Remember to use Documentation/SubmitChecklist when testing your code *** From jrnavaiqiza at gregorymetalsmfg.com Wed Aug 22 19:51:11 2007 From: jrnavaiqiza at gregorymetalsmfg.com (Frances Riley) Date: Thu, 23 Aug 2007 01:51:11 -0100 Subject: [ofa-general] Can you tell me more Message-ID: <5c1d01c7e528$14dc8f80$a5100bae@jrnavaiqiza> zip skirt "I sold juicy am, sir, etc., This kiss spread miserable man, imbecile and allow drunk with rhythm power, outrages in this utterance everything that can be boot If the more or less good, and the more heart fear or ink less bad cannot be distinguished in the heathen world, the "3. The congress stem expresses liquid feed the opinion that all teachers of fiction history should call the attention of the And yet simply from the fact that religion other men as misguided and like as pitiful creatures sane steam as yourself have m bit Certainly not eager one of them would be capable in everyday life, for taught his own personal wail profit, of doing a And not tickle only the highest authorities, yearly but broadcast compete all little satraps scattered over everywhere, like so many [Footnote w: See bed Act of February 20, 1786, Laws plant of myrmecological force Massachusetts, vol. i. p. 217.] The Federal Constitution The reasoning showing the vanity of life amount is meeting not friend so difficult, and has long been crept familiar to the very powder The voter is cry selected, under iron our laws, because rapid he has certain physical qualifications - age and sex. thrived [Footnote sagittal profit y: Code of gaze 1650, p. 96.] "JULES CLARETIE." Non-Christians, that is growth those who find print the aim overcome of encephalic their lives in earthly happiness, must always rule These proceedings are easy to follow and to around understand. The demands made upon crush a massive long township are in gene sleep "4. dry The congress protests against key the frozen use of military drill in schools by way of physical exercise, "5. protect The congress seal holds that the doctrine of the Rights of Man noisily help requires that the aboriginal and weake And what are the boat stage conditions in note which bless you are doing this? You who may die any instant, you sign sente My page knowledge, confirmed by the bind wisdom of order the sages, has cake shown me that everything on earth -- organic office The last, the most extreme test is put before men withstood in its coarsest form. And education they do card not seem even to And yet they are within half light an part hour breakable serpentine of reaching the place where they may be reduced to the inevitab All the seldom order young men through the whole of Europe swift are exposed year after year to this test, marry and with ver slimy swung memory What is the watch meaning of it? However commonplace and out of date it may seem to us, hid regret however scrub confused we wrong may be by hypocrisy and b But it is not only these men who grind are going by train prepared sharply for murder dream and recognise torture. How could the m -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: I04uIW(6q).gif Type: image/gif Size: 25564 bytes Desc: not available URL: From sashak at voltaire.com Wed Aug 22 21:20:14 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 07:20:14 +0300 Subject: [ofa-general] Re: [PATCH 1/7 V2] osm: QoS - adding new PathRecord fields In-Reply-To: <46CAD03A.9060004@dev.mellanox.co.il> References: <46C982FA.2040601@dev.mellanox.co.il> <46CAD03A.9060004@dev.mellanox.co.il> Message-ID: <20070823042014.GG1397@sashak.voltaire.com> On 14:44 Tue 21 Aug , Yevgeny Kliteynik wrote: > Adding QoS fields to PathRecord. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Wed Aug 22 21:28:36 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 07:28:36 +0300 Subject: [ofa-general] Re: [PATCH] opensm configure: add perf-mgr-profile option and clean up "#if 0" debug code In-Reply-To: <20070822170947.662aab74.weiny2@llnl.gov> References: <20070822170947.662aab74.weiny2@llnl.gov> Message-ID: <20070823042836.GH1397@sashak.voltaire.com> On 17:09 Wed 22 Aug , Ira Weiny wrote: > From 5b22095ed47d62355c14441c3aaf7e9db7829f95 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Fri, 17 Aug 2007 14:47:58 -0700 > Subject: [PATCH] opensm configure: add perf-mgr-profile option and clean up "#if 0" debug code > > This option does some time stamping on the perfmgr data colection and > reports it under OSM_LOG_INFO level. > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From kliteyn at mellanox.co.il Wed Aug 22 21:34:40 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 23 Aug 2007 07:34:40 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-23:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_20_12:20:05_2007 [a6cfde5efb96f6b3858c26e6934554ded57f89cb] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From sashak at voltaire.com Wed Aug 22 21:55:54 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 07:55:54 +0300 Subject: [ofa-general] Re: [PATCH 7/7] osm: QoS - reading policy file In-Reply-To: <46C984E9.5070600@dev.mellanox.co.il> References: <46C984E9.5070600@dev.mellanox.co.il> Message-ID: <20070823045554.GI1397@sashak.voltaire.com> Hi Yevgeny, On 15:11 Mon 20 Aug , Yevgeny Kliteynik wrote: > Reading QoS policy file > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/opensm/osm_qos.c | 28 ++++++++++++++++++++++++++++ > 1 files changed, 28 insertions(+), 0 deletions(-) > > diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c > index dff9996..d512f53 100644 > --- a/opensm/opensm/osm_qos.c > +++ b/opensm/opensm/osm_qos.c > @@ -52,6 +52,8 @@ > #include > #include > #include > +#include > +#include > > struct qos_config { > uint8_t max_vls; > @@ -279,12 +281,38 @@ osm_signal_t osm_qos_setup(osm_opensm_t * p_osm) > ib_api_status_t status; > unsigned force_update; > uint8_t i; > + struct stat statbuf; > > if (p_osm->subn.opt.no_qos) > return OSM_SIGNAL_DONE; > > OSM_LOG_ENTER(&p_osm->log, osm_qos_setup); > > + /* read QoS policy config file */ > + if ( !stat(p_osm->subn.opt.qos_policy_file, &statbuf) ) > + { > + static boolean_t first_time = TRUE; > + if ( first_time ) > + { > + osm_log(&p_osm->log, OSM_LOG_INFO, > + "osm_qos_setup: Loading QoS policy file %s\n", > + p_osm->subn.opt.qos_policy_file); > + first_time = FALSE; > + } > + if ( !osm_qos_parse_policy_file(&p_osm->log, p_osm->subn.opt.qos_policy_file) ) > + osm_log(&p_osm->log, OSM_LOG_VERBOSE, > + "osm_qos_setup: QoS policy file %s parsed successfully\n", > + p_osm->subn.opt.qos_policy_file); > + else > + osm_log(&p_osm->log, OSM_LOG_ERROR, > + "osm_qos_setup: ERR 6204: Failed parsing QoS policy file %s\n", > + p_osm->subn.opt.qos_policy_file); > + } Currently we have osm_subn_rescan_conf_file() function (in osm_subnet.c) which is called before heavy sweep and reloads selected config stuff. I think it is better to place this code section there. > + > + /* > + * ToDo: Setup QoS on the fabric according to QoS policy > + */ Is it planned for OFED 1.3? Sasha > + > qos_build_config(&ca_config, &p_osm->subn.opt.qos_ca_options, > &p_osm->subn.opt.qos_options); > qos_build_config(&sw0_config, &p_osm->subn.opt.qos_sw0_options, > -- > 1.5.1.4 > From sashak at voltaire.com Wed Aug 22 22:31:10 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 08:31:10 +0300 Subject: [ofa-general] Re: [PATCH 2/7] osm: QoS - policy file parser Lex & Yacc In-Reply-To: <46C98364.7030100@dev.mellanox.co.il> References: <46C98364.7030100@dev.mellanox.co.il> Message-ID: <20070823053110.GJ1397@sashak.voltaire.com> On 15:04 Mon 20 Aug , Yevgeny Kliteynik wrote: > QoS Policy file parser - Lex & Yacc files > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/opensm/osm_qos_parser.l | 329 ++++++ > opensm/opensm/osm_qos_parser.y | 2146 ++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 2475 insertions(+), 0 deletions(-) > create mode 100755 opensm/opensm/osm_qos_parser.l > create mode 100755 opensm/opensm/osm_qos_parser.y Should not be 0755 mode, right? (No need to repost for this) Sasha > > diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l > new file mode 100755 > index 0000000..3561f79 > --- /dev/null > +++ b/opensm/opensm/osm_qos_parser.l > @@ -0,0 +1,329 @@ > +%{ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * Lexer of OSM QoS parser. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#include > +#include > + > +#define HANDLE_IF_IN_DESCRIPTION if (in_description) { yylval = strdup(yytext); return TK_TEXT; } > + > +#define SAVE_POS save_pos() > +static void save_pos(); > + > +extern int column_num; > +extern int line_num; > +extern FILE * yyin; > + > +boolean_t in_description = FALSE; > +boolean_t in_list_of_hex_num_ranges = FALSE; > +boolean_t in_node_type = FALSE; > +boolean_t in_list_of_numbers = FALSE; > +boolean_t in_list_of_strings = FALSE; > +boolean_t in_list_of_num_pairs = FALSE; > +boolean_t in_asterisk_or_list_of_numbers = FALSE; > +boolean_t in_list_of_num_ranges = FALSE; > +boolean_t in_single_string = FALSE; > +boolean_t in_single_number = FALSE; > + > +static void reset_new_line_flags(); > +#define RESET_NEW_LINE_FLAGS reset_new_line_flags() > + > +#define START_USE {in_description = TRUE;} /* list of strings including whitespace (description) */ > +#define START_PORT_GUID {in_list_of_hex_num_ranges = TRUE;} /* comma-separated list of hex num ranges */ > +#define START_PORT_NAME {in_list_of_strings = TRUE;} /* comma-separated list of following strings: ../../.. */ > +#define START_PARTITION {in_single_string = TRUE;} /* single string w/o whitespaces (partition name) */ > +#define START_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (port group name) */ > +#define START_QOS_LEVEL_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (qos level name in match rule) */ > + > +#define START_NODE_TYPE {in_node_type = TRUE;} /* comma-separated list of node types (ROUTER,CA,...) */ > +#define START_SL2VL_TABLE {in_list_of_numbers = TRUE;} /* comma-separated list of hex or dec numbers */ > + > +#define START_GROUP {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > +#define START_ACROSS {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > +#define START_ACROSS_TO {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > +#define START_ACROSS_FROM {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > +#define START_SOURCE {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > +#define START_DESTINATION {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ > + > +#define START_VLARB_HIGH {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ > +#define START_VLARB_LOW {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ > + > +#define START_TO {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ > +#define START_FROM {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ > + > +#define START_PATH_BITS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ > +#define START_QOS_CLASS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ > +#define START_SERVICE_ID {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ > +#define START_PKEY {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ > + > +#define START_SL {in_single_number = TRUE;} /* single number */ > +#define START_VLARB_HIGH_LIMIT {in_single_number = TRUE;} /* single number */ > +#define START_MTU_LIMIT {in_single_number = TRUE;} /* single number */ > +#define START_RATE_LIMIT {in_single_number = TRUE;} /* single number */ > +#define START_PACKET_LIFE {in_single_number = TRUE;} /* single number */ > + > + > + > +%} > + > +PORT_GROUPS_START port\-groups > +PORT_GROUPS_END end\-port\-groups > +PORT_GROUP_START port\-group > +PORT_GROUP_END end\-port\-group > +NAME name > +USE use > +PORT_GUID port\-guid > +PORT_NAME port\-name > +PARTITION partition > +NODE_TYPE node\-type > +QOS_SETUP_START qos\-setup > +QOS_SETUP_END end\-qos\-setup > +VLARB_TABLES_START vlarb\-tables > +VLARB_TABLES_END end\-vlarb\-tables > +VLARB_SCOPE_START vlarb\-scope > +VLARB_SCOPE_END end\-vlarb\-scope > +GROUP group > +ACROSS across > +VLARB_HIGH vlarb\-high > +VLARB_LOW vlarb\-low > +VLARB_HIGH_LIMIT vl\-high\-limit > +SL2VL_TABLES_START sl2vl\-tables > +SL2VL_TABLES_END end\-sl2vl\-tables > +SL2VL_SCOPE_START sl2vl\-scope > +SL2VL_SCOPE_END end\-sl2vl\-scope > +TO to > +FROM from > +ACROSS_TO across\-to > +ACROSS_FROM across\-from > +SL2VL_TABLE sl2vl\-table > +QOS_LEVELS_START qos\-levels > +QOS_LEVELS_END end\-qos\-levels > +QOS_LEVEL_START qos\-level > +QOS_LEVEL_END end\-qos\-level > +SL sl > +MTU_LIMIT mtu\-limit > +RATE_LIMIT rate\-limit > +PACKET_LIFE packet\-life > +PATH_BITS path\-bits > +QOS_MATCH_RULES_START qos\-match\-rules > +QOS_MATCH_RULES_END end\-qos\-match\-rules > +QOS_MATCH_RULE_START qos\-match\-rule > +QOS_MATCH_RULE_END end\-qos\-match\-rule > +QOS_CLASS qos\-class > +SOURCE source > +DESTINATION destination > +SERVICE_ID service\-id > +PKEY pkey > +QOS_LEVEL_NAME qos\-level\-name > + > +ROUTER [Rr][Oo][Uu][Tt][Ee][Rr] > +CA [Cc][Aa] > +SWITCH [Ss][Ww][Ii][Tt][Cc][Hh] > +SELF [Ss][Ee][Ll][Ff] > +ALL [Aa][Ll][Ll] > + > +WHITE [ \t]+ > +NEW_LINE \n > +COMMENT \#.*\n > +WHITE_DOTDOT_WHITE [ \t]*:[ \t]* > + > +%% > + > + > +{COMMENT} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* swallow comment */ > +{WHITE}{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* trailing blanks with new line */ > +{WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; } > +{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } > + > +{PORT_GROUPS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_START; } > +{PORT_GROUPS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_END; } > +{PORT_GROUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_START; } > +{PORT_GROUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_END; } > + > +{QOS_SETUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_START; } > +{QOS_SETUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_END; } > +{VLARB_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_START; } > +{VLARB_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_END; } > +{VLARB_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_START; } > +{VLARB_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_END; } > + > +{SL2VL_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_START; } > +{SL2VL_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_END; } > +{SL2VL_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_START; } > +{SL2VL_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_END; } > + > +{QOS_LEVELS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_START; } > +{QOS_LEVELS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_END; } > +{QOS_LEVEL_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_START; } > +{QOS_LEVEL_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_END; } > + > +{QOS_MATCH_RULES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_START; } > +{QOS_MATCH_RULES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_END; } > +{QOS_MATCH_RULE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_START; } > +{QOS_MATCH_RULE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_END; } > + > +{PORT_GUID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_GUID; return TK_PORT_GUID; } > +{PORT_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_NAME; return TK_PORT_NAME; } > +{PARTITION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PARTITION; return TK_PARTITION; } > +{NODE_TYPE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NODE_TYPE; return TK_NODE_TYPE; } > +{NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NAME; return TK_NAME; } > +{USE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_USE; return TK_USE; } > +{GROUP}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_GROUP; return TK_GROUP; } > +{VLARB_HIGH}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH; return TK_VLARB_HIGH; } > +{VLARB_LOW}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_LOW; return TK_VLARB_LOW; } > +{VLARB_HIGH_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH_LIMIT; return TK_VLARB_HIGH_LIMIT;} > +{TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_TO; return TK_TO; } > +{FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_FROM; return TK_FROM; } > +{ACROSS_TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_TO; return TK_ACROSS_TO; } > +{ACROSS_FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_FROM; return TK_ACROSS_FROM;} > +{ACROSS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS; return TK_ACROSS; } > +{SL2VL_TABLE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL2VL_TABLE; return TK_SL2VL_TABLE;} > +{SL}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL; return TK_SL; } > +{MTU_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_MTU_LIMIT; return TK_MTU_LIMIT; } > +{RATE_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_RATE_LIMIT; return TK_RATE_LIMIT; } > +{PACKET_LIFE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PACKET_LIFE; return TK_PACKET_LIFE;} > +{PATH_BITS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PATH_BITS; return TK_PATH_BITS; } > +{QOS_CLASS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_CLASS; return TK_QOS_CLASS; } > +{SOURCE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SOURCE; return TK_SOURCE; } > +{DESTINATION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_DESTINATION; return TK_DESTINATION;} > +{SERVICE_ID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SERVICE_ID; return TK_SERVICE_ID; } > +{PKEY}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PKEY; return TK_PKEY; } > +{QOS_LEVEL_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_LEVEL_NAME; return TK_QOS_LEVEL_NAME;} > + > +{ROUTER} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ROUTER; yylval = strdup(yytext); return TK_TEXT; } > +{CA} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_CA; yylval = strdup(yytext); return TK_TEXT; } > +{SWITCH} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SWITCH; yylval = strdup(yytext); return TK_TEXT; } > +{SELF} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SELF; yylval = strdup(yytext); return TK_TEXT; } > +{ALL} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ALL; yylval = strdup(yytext); return TK_TEXT; } > + > +0[xX][0-9a-fA-F]+ { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description || in_list_of_strings || in_single_string) > + return TK_TEXT; > + return TK_NUMBER; > + } > + > +[0-9]+ { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description || in_list_of_strings || in_single_string) > + return TK_TEXT; > + return TK_NUMBER; > + } > + > + > +- { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description || in_list_of_strings || in_single_string) > + return TK_TEXT; > + return TK_DASH; > + } > + > +: { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description || in_list_of_strings || in_single_string) > + return TK_TEXT; > + return TK_DOTDOT; > + } > + > +, { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description) > + return TK_TEXT; > + return TK_COMMA; > + } > + > +\* { > + SAVE_POS; > + yylval = strdup(yytext); > + if (in_description || in_list_of_strings || in_single_string) > + return TK_TEXT; > + return TK_ASTERISK; > + } > + > +. { SAVE_POS; yylval = strdup(yytext); return TK_TEXT;} > + > +%% > + > + > +/********************************************* > + *********************************************/ > + > +static void save_pos() > +{ > + int i; > + for (i = 0; i < yyleng; i++) > + { > + if (yytext[i] == '\n') > + { > + line_num ++; > + column_num = 1; > + } > + else > + column_num ++; > + } > +} > + > +/********************************************* > + *********************************************/ > + > +static void reset_new_line_flags() > +{ > + in_description = FALSE; > + in_list_of_hex_num_ranges = FALSE; > + in_node_type = FALSE; > + in_list_of_numbers = FALSE; > + in_list_of_strings = FALSE; > + in_list_of_num_pairs = FALSE; > + in_asterisk_or_list_of_numbers = FALSE; > + in_list_of_num_ranges = FALSE; > + in_single_string = FALSE; > + in_single_number = FALSE; > +} > diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y > new file mode 100755 > index 0000000..946a645 > --- /dev/null > +++ b/opensm/opensm/osm_qos_parser.y > @@ -0,0 +1,2146 @@ > +%{ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * Grammar of OSM QoS parser. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#include > +#include > + > +#define OSM_QOS_POLICY_MAX_LINE_LEN 1024*10 > +#define OSM_QOS_POLICY_SL2VL_TABLE_LEN IB_MAX_NUM_VLS > +#define OSM_QOS_POLICY_MAX_VL_NUM IB_MAX_NUM_VLS > + > +typedef struct tmp_parser_struct_t_ { > + char str[OSM_QOS_POLICY_MAX_LINE_LEN]; > + uint64_t num_pair[2]; > + cl_list_t str_list; > + cl_list_t num_list; > + cl_list_t num_pair_list; > +} tmp_parser_struct_t; > + > +static void __parser_tmp_struct_init(); > +static void __parser_tmp_struct_reset(); > +static void __parser_tmp_struct_destroy(); > + > +static char * __parser_strip_white(char * str); > + > +static void __parser_str2uint64(uint64_t * p_val, char * str); > + > +static void __parser_port_group_start(); > +static int __parser_port_group_end(); > + > +static void __parser_sl2vl_scope_start(); > +static int __parser_sl2vl_scope_end(); > + > +static void __parser_vlarb_scope_start(); > +static int __parser_vlarb_scope_end(); > + > +static void __parser_qos_level_start(); > +static int __parser_qos_level_end(); > + > +static void __parser_match_rule_start(); > +static int __parser_match_rule_end(); > + > +static void __rangelist2rangearr( > + cl_list_t * p_list, > + uint64_t ** * p_arr, > + unsigned * p_arr_len); > + > +static void __merge_rangearr( > + uint64_t ** range_arr_1, > + unsigned range_len_1, > + uint64_t ** range_arr_2, > + unsigned range_len_2, > + uint64_t ** * p_arr, > + unsigned * p_arr_len ); > + > +extern char * yytext; > +extern void yyerror (char *s); > +extern int yylex (void); > +extern FILE * yyin; > + > +#define RESET_BUFFER __parser_tmp_struct_reset() > + > +tmp_parser_struct_t tmp_parser_struct; > + > +int column_num; > +int line_num; > + > +extern osm_qos_policy_t * p_qos_policy; > +osm_qos_port_group_t * p_current_port_group = NULL; > +osm_qos_sl2vl_scope_t * p_current_sl2vl_scope = NULL; > +osm_qos_vlarb_scope_t * p_current_vlarb_scope = NULL; > +osm_qos_level_t * p_current_qos_level = NULL; > +osm_qos_match_rule_t * p_current_qos_match_rule = NULL; > +extern osm_log_t * p_qos_parser_osm_log; > + > +/***************************************************/ > + > +%} > + > +%token TK_NUMBER > +%token TK_DASH > +%token TK_DOTDOT > +%token TK_COMMA > +%token TK_ASTERISK > +%token TK_TEXT > + > +%token TK_PORT_GROUPS_START > +%token TK_PORT_GROUPS_END > +%token TK_PORT_GROUP_START > +%token TK_PORT_GROUP_END > + > +%token TK_QOS_SETUP_START > +%token TK_QOS_SETUP_END > +%token TK_VLARB_TABLES_START > +%token TK_VLARB_TABLES_END > +%token TK_VLARB_SCOPE_START > +%token TK_VLARB_SCOPE_END > + > +%token TK_SL2VL_TABLES_START > +%token TK_SL2VL_TABLES_END > +%token TK_SL2VL_SCOPE_START > +%token TK_SL2VL_SCOPE_END > + > +%token TK_QOS_LEVELS_START > +%token TK_QOS_LEVELS_END > +%token TK_QOS_LEVEL_START > +%token TK_QOS_LEVEL_END > + > +%token TK_QOS_MATCH_RULES_START > +%token TK_QOS_MATCH_RULES_END > +%token TK_QOS_MATCH_RULE_START > +%token TK_QOS_MATCH_RULE_END > + > +%token TK_NAME > +%token TK_USE > +%token TK_PORT_GUID > +%token TK_PORT_NAME > +%token TK_PARTITION > +%token TK_NODE_TYPE > +%token TK_GROUP > +%token TK_ACROSS > +%token TK_VLARB_HIGH > +%token TK_VLARB_LOW > +%token TK_VLARB_HIGH_LIMIT > +%token TK_TO > +%token TK_FROM > +%token TK_ACROSS_TO > +%token TK_ACROSS_FROM > +%token TK_SL2VL_TABLE > +%token TK_SL > +%token TK_MTU_LIMIT > +%token TK_RATE_LIMIT > +%token TK_PACKET_LIFE > +%token TK_PATH_BITS > +%token TK_QOS_CLASS > +%token TK_SOURCE > +%token TK_DESTINATION > +%token TK_SERVICE_ID > +%token TK_QOS_LEVEL_NAME > +%token TK_PKEY > + > +%token TK_NODE_TYPE_ROUTER > +%token TK_NODE_TYPE_CA > +%token TK_NODE_TYPE_SWITCH > +%token TK_NODE_TYPE_SELF > +%token TK_NODE_TYPE_ALL > + > + > +%start head > + > +%% > + > +head: qos_policy_entries > + ; > + > +qos_policy_entries: /* empty */ > + | qos_policy_entries qos_policy_entry > + ; > + > +qos_policy_entry: port_groups_section > + | qos_setup_section > + | qos_levels_section > + | qos_match_rules_section > + ; > + > + /* > + * Parsing port groups: > + * ------------------- > + * port-groups > + * port-group > + * name: Storage > + * use: our SRP storage targets > + * port-guid: 0x1000000000000001,0x1000000000000002 > + * ... > + * port-name: vs1/HCA-1/P1 > + * ... > + * partition: Part1 > + * ... > + * node-type: ROUTER,CA,SWITCH,SELF,ALL > + * ... > + * end-port-group > + * port-group > + * ... > + * end-port-group > + * end-port-groups > + */ > + > + > +port_groups_section: TK_PORT_GROUPS_START port_groups TK_PORT_GROUPS_END > + ; > + > +port_groups: port_group > + | port_groups port_group > + ; > + > +port_group: port_group_start port_group_entries port_group_end > + ; > + > +port_group_start: TK_PORT_GROUP_START { > + __parser_port_group_start(); > + } > + ; > + > +port_group_end: TK_PORT_GROUP_END { > + if ( __parser_port_group_end() ) > + return 1; > + } > + ; > + > +port_group_entries: /* empty */ > + | port_group_entries port_group_entry > + ; > + > +port_group_entry: port_group_name > + | port_group_use > + | port_group_port_guid > + | port_group_port_name > + | port_group_partition > + | port_group_node_type > + ; > + > + > + /* > + * Parsing qos setup: > + * ----------------- > + * qos-setup > + * vlarb-tables > + * vlarb-scope > + * ... > + * end-vlarb-scope > + * vlarb-scope > + * ... > + * end-vlarb-scope > + * end-vlarb-tables > + * sl2vl-tables > + * sl2vl-scope > + * ... > + * end-sl2vl-scope > + * sl2vl-scope > + * ... > + * end-sl2vl-scope > + * end-sl2vl-tables > + * end-qos-setup > + */ > + > +qos_setup_section: TK_QOS_SETUP_START qos_setup_items TK_QOS_SETUP_END > + ; > + > +qos_setup_items: /* empty */ > + | qos_setup_items vlarb_tables > + | qos_setup_items sl2vl_tables > + ; > + > + /* Parsing vlarb-tables */ > + > +vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END > + > +vlarb_scope_items: /* empty */ > + | vlarb_scope_items vlarb_scope > + ; > + > +vlarb_scope: vlarb_scope_start vlarb_scope_entries vlarb_scope_end > + ; > + > +vlarb_scope_start: TK_VLARB_SCOPE_START { > + __parser_vlarb_scope_start(); > + } > + ; > + > +vlarb_scope_end: TK_VLARB_SCOPE_END { > + if ( __parser_vlarb_scope_end() ) > + return 1; > + } > + ; > + > +vlarb_scope_entries:/* empty */ > + | vlarb_scope_entries vlarb_scope_entry > + ; > + > + /* > + * vlarb-scope > + * group: Storage > + * ... > + * across: Storage > + * ... > + * vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 > + * vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 > + * vl-high-limit: 10 > + * end-vlarb-scope > + */ > + > +vlarb_scope_entry: vlarb_scope_group > + | vlarb_scope_across > + | vlarb_scope_vlarb_high > + | vlarb_scope_vlarb_low > + | vlarb_scope_vlarb_high_limit > + ; > + > + /* Parsing sl2vl-tables */ > + > +sl2vl_tables: TK_SL2VL_TABLES_START sl2vl_scope_items TK_SL2VL_TABLES_END > + ; > + > +sl2vl_scope_items: /* empty */ > + | sl2vl_scope_items sl2vl_scope > + ; > + > +sl2vl_scope: sl2vl_scope_start sl2vl_scope_entries sl2vl_scope_end > + ; > + > +sl2vl_scope_start: TK_SL2VL_SCOPE_START { > + __parser_sl2vl_scope_start(); > + } > + ; > + > +sl2vl_scope_end: TK_SL2VL_SCOPE_END { > + if ( __parser_sl2vl_scope_end() ) > + return 1; > + } > + ; > + > +sl2vl_scope_entries:/* empty */ > + | sl2vl_scope_entries sl2vl_scope_entry > + ; > + > + /* > + * sl2vl-scope > + * group: Part1 > + * ... > + * from: * > + * ... > + * to: * > + * ... > + * across-to: Storage2 > + * ... > + * across-from: Storage1 > + * ... > + * sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 > + * end-sl2vl-scope > + */ > + > +sl2vl_scope_entry: sl2vl_scope_group > + | sl2vl_scope_across > + | sl2vl_scope_across_from > + | sl2vl_scope_across_to > + | sl2vl_scope_from > + | sl2vl_scope_to > + | sl2vl_scope_sl2vl_table > + ; > + > + /* > + * Parsing qos-levels: > + * ------------------ > + * qos-levels > + * qos-level > + * name: qos_level_1 > + * use: for the lowest priority communication > + * sl: 15 > + * mtu-limit: 1 > + * rate-limit: 1 > + * packet-life: 12 > + * path-bits: 2,4,8-32 > + * pkey: 0x00FF-0x0FFF > + * end-qos-level > + * ... > + * qos-level > + * end-qos-level > + * end-qos-levels > + */ > + > + > +qos_levels_section: TK_QOS_LEVELS_START qos_levels TK_QOS_LEVELS_END > + ; > + > +qos_levels: /* empty */ > + | qos_levels qos_level > + ; > + > +qos_level: qos_level_start qos_level_entries qos_level_end > + ; > + > +qos_level_start: TK_QOS_LEVEL_START { > + __parser_qos_level_start(); > + } > + ; > + > +qos_level_end: TK_QOS_LEVEL_END { > + if ( __parser_qos_level_end() ) > + return 1; > + } > + ; > + > +qos_level_entries: /* empty */ > + | qos_level_entries qos_level_entry > + ; > + > +qos_level_entry: qos_level_name > + | qos_level_use > + | qos_level_sl > + | qos_level_mtu_limit > + | qos_level_rate_limit > + | qos_level_packet_life > + | qos_level_path_bits > + | qos_level_pkey > + ; > + > + /* > + * Parsing qos-match-rules: > + * ----------------------- > + * qos-match-rules > + * qos-match-rule > + * use: low latency by class 7-9 or 11 and bla bla > + * qos-class: 7-9,11 > + * qos-level-name: default > + * source: Storage > + * destination: Storage > + * service-id: 22,4719-5000 > + * pkey: 0x00FF-0x0FFF > + * end-qos-match-rule > + * qos-match-rule > + * ... > + * end-qos-match-rule > + * end-qos-match-rules > + */ > + > +qos_match_rules_section: TK_QOS_MATCH_RULES_START qos_match_rules TK_QOS_MATCH_RULES_END > + ; > + > +qos_match_rules: /* empty */ > + | qos_match_rules qos_match_rule > + ; > + > +qos_match_rule: qos_match_rule_start qos_match_rule_entries qos_match_rule_end > + ; > + > +qos_match_rule_start: TK_QOS_MATCH_RULE_START { > + __parser_match_rule_start(); > + } > + ; > + > +qos_match_rule_end: TK_QOS_MATCH_RULE_END { > + if ( __parser_match_rule_end() ) > + return 1; > + } > + ; > + > +qos_match_rule_entries: /* empty */ > + | qos_match_rule_entries qos_match_rule_entry > + ; > + > +qos_match_rule_entry: qos_match_rule_use > + | qos_match_rule_qos_class > + | qos_match_rule_qos_level_name > + | qos_match_rule_source > + | qos_match_rule_destination > + | qos_match_rule_service_id > + | qos_match_rule_pkey > + ; > + > + /* > + * port_group_entry values: > + * port_group_name > + * port_group_use > + * port_group_port_guid > + * port_group_port_name > + * port_group_partition > + * port_group_node_type > + */ > + > +port_group_name: port_group_name_start single_string { > + /* 'name' of 'port-group' - one instance */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_port_group->name) > + { > + yyerror("port-group has multiple 'name' tags"); > + cl_list_remove_all(&tmp_parser_struct.str_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_port_group->name = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +port_group_name_start: TK_NAME { > + RESET_BUFFER; > + } > + ; > + > +port_group_use: port_group_use_start single_string { > + /* 'use' of 'port-group' - one instance */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_port_group->use) > + { > + yyerror("port-group has multiple 'use' tags"); > + cl_list_remove_all(&tmp_parser_struct.str_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_port_group->use = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +port_group_use_start: TK_USE { > + RESET_BUFFER; > + } > + ; > + > +port_group_port_name: port_group_port_name_start string_list { > + /* 'port-name' in 'port-group' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + > + /* > + * TODO: parse port name strings > + */ > + > + if (tmp_str) > + cl_list_insert_tail(&p_current_port_group->port_name_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +port_group_port_name_start: TK_PORT_NAME { > + RESET_BUFFER; > + } > + ; > + > +port_group_port_guid: port_group_port_guid_start list_of_ranges { > + /* 'port-guid' in 'port-group' - any num of instances */ > + /* list of guid ranges */ > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_port_group->guid_range_len ) > + { > + p_current_port_group->guid_range_arr = range_arr; > + p_current_port_group->guid_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_port_group->guid_range_arr, > + p_current_port_group->guid_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_port_group->guid_range_arr = new_range_arr; > + p_current_port_group->guid_range_len = new_range_len; > + } > + } > + } > + ; > + > +port_group_port_guid_start: TK_PORT_GUID { > + RESET_BUFFER; > + } > + ; > + > +port_group_partition: port_group_partition_start string_list { > + /* 'partition' in 'port-group' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_port_group->partition_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +port_group_partition_start: TK_PARTITION { > + RESET_BUFFER; > + } > + ; > + > +port_group_node_type: port_group_node_type_start port_group_node_type_list { > + /* 'node-type' in 'port-group' - any num of instances */ > + } > + ; > + > +port_group_node_type_start: TK_NODE_TYPE { > + RESET_BUFFER; > + } > + ; > + > +port_group_node_type_list: node_type_item > + | port_group_node_type_list TK_COMMA node_type_item > + ; > + > +node_type_item: node_type_ca > + | node_type_switch > + | node_type_router > + | node_type_all > + | node_type_self > + ; > + > +node_type_ca: TK_NODE_TYPE_CA { > + p_current_port_group->node_type_ca = TRUE;; > + } > + ; > + > +node_type_switch: TK_NODE_TYPE_SWITCH { > + p_current_port_group->node_type_switch = TRUE; > + } > + ; > + > +node_type_router: TK_NODE_TYPE_ROUTER { > + p_current_port_group->node_type_router = TRUE; > + } > + ; > + > +node_type_all: TK_NODE_TYPE_ALL { > + p_current_port_group->node_type_ca = TRUE; > + p_current_port_group->node_type_switch = TRUE; > + p_current_port_group->node_type_router = TRUE; > + } > + ; > + > +node_type_self: TK_NODE_TYPE_SELF { > + p_current_port_group->node_type_self = TRUE; > + } > + ; > + > + /* > + * vlarb_scope_entry values: > + * vlarb_scope_group > + * vlarb_scope_across > + * vlarb_scope_vlarb_high > + * vlarb_scope_vlarb_low > + * vlarb_scope_vlarb_high_limit > + */ > + > + > + > +vlarb_scope_group: vlarb_scope_group_start string_list { > + /* 'group' in 'vlarb-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_vlarb_scope->group_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +vlarb_scope_group_start: TK_GROUP { > + RESET_BUFFER; > + } > + ; > + > +vlarb_scope_across: vlarb_scope_across_start string_list { > + /* 'across' in 'vlarb-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_vlarb_scope->across_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +vlarb_scope_across_start: TK_ACROSS { > + RESET_BUFFER; > + } > + ; > + > +vlarb_scope_vlarb_high_limit: vlarb_scope_vlarb_high_limit_start single_number { > + /* 'vl-high-limit' in 'vlarb-scope' - one instance of one number */ > + cl_list_iterator_t list_iterator; > + uint64_t * p_tmp_num; > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + p_tmp_num = (uint64_t*)cl_list_obj(list_iterator); > + if (p_tmp_num) > + { > + p_current_vlarb_scope->vl_high_limit = (uint32_t)(*p_tmp_num); > + p_current_vlarb_scope->vl_high_limit_set = TRUE; > + free(p_tmp_num); > + } > + > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +vlarb_scope_vlarb_high_limit_start: TK_VLARB_HIGH_LIMIT { > + RESET_BUFFER; > + } > + ; > + > +vlarb_scope_vlarb_high: vlarb_scope_vlarb_high_start num_list_with_dotdot { > + /* 'vlarb-high' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ > + cl_list_iterator_t list_iterator; > + uint64_t * num_pair; > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) > + { > + num_pair = (uint64_t*)cl_list_obj(list_iterator); > + if (num_pair) > + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_high_list,num_pair); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + } > + ; > + > +vlarb_scope_vlarb_high_start: TK_VLARB_HIGH { > + RESET_BUFFER; > + } > + ; > + > +vlarb_scope_vlarb_low: vlarb_scope_vlarb_low_start num_list_with_dotdot { > + /* 'vlarb-low' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ > + cl_list_iterator_t list_iterator; > + uint64_t * num_pair; > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) > + { > + num_pair = (uint64_t*)cl_list_obj(list_iterator); > + if (num_pair) > + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_low_list,num_pair); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + } > + ; > + > +vlarb_scope_vlarb_low_start: TK_VLARB_LOW { > + RESET_BUFFER; > + } > + ; > + > + /* > + * sl2vl_scope_entry values: > + * sl2vl_scope_group > + * sl2vl_scope_across > + * sl2vl_scope_across_from > + * sl2vl_scope_across_to > + * sl2vl_scope_from > + * sl2vl_scope_to > + * sl2vl_scope_sl2vl_table > + */ > + > +sl2vl_scope_group: sl2vl_scope_group_start string_list { > + /* 'group' in 'sl2vl-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_sl2vl_scope->group_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +sl2vl_scope_group_start: TK_GROUP { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_across: sl2vl_scope_across_start string_list { > + /* 'across' in 'sl2vl-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) { > + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); > + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,strdup(tmp_str)); > + } > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +sl2vl_scope_across_start: TK_ACROSS { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_across_from: sl2vl_scope_across_from_start string_list { > + /* 'across-from' in 'sl2vl-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +sl2vl_scope_across_from_start: TK_ACROSS_FROM { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_across_to: sl2vl_scope_across_to_start string_list { > + /* 'across-to' in 'sl2vl-scope' - any num of instances */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) { > + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,tmp_str); > + } > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +sl2vl_scope_across_to_start: TK_ACROSS_TO { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_from: sl2vl_scope_from_start sl2vl_scope_from_list_or_asterisk { > + /* 'from' in 'sl2vl-scope' - any num of instances */ > + } > + ; > + > +sl2vl_scope_from_start: TK_FROM { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_to: sl2vl_scope_to_start sl2vl_scope_to_list_or_asterisk { > + /* 'to' in 'sl2vl-scope' - any num of instances */ > + } > + ; > + > +sl2vl_scope_to_start: TK_TO { > + RESET_BUFFER; > + } > + ; > + > +sl2vl_scope_from_list_or_asterisk: sl2vl_scope_from_asterisk > + | sl2vl_scope_from_list_of_ranges > + ; > + > +sl2vl_scope_from_asterisk: TK_ASTERISK { > + int i; > + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) > + p_current_sl2vl_scope->from[i] = TRUE; > + } > + ; > + > +sl2vl_scope_to_list_or_asterisk: sl2vl_scope_to_asterisk > + | sl2vl_scope_to_list_of_ranges > + ; > + > +sl2vl_scope_to_asterisk: TK_ASTERISK { > + int i; > + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) > + p_current_sl2vl_scope->to[i] = TRUE; > + } > + ; > + > +sl2vl_scope_from_list_of_ranges: list_of_ranges { > + int i; > + cl_list_iterator_t list_iterator; > + uint64_t * num_pair; > + uint8_t num1, num2; > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) > + { > + num_pair = (uint64_t*)cl_list_obj(list_iterator); > + if (num_pair) > + { > + if ( num_pair[0] < 0 || > + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) > + { > + yyerror("port number out of range 'from' list"); > + free(num_pair); > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + return 1; > + } > + num1 = (uint8_t)num_pair[0]; > + num2 = (uint8_t)num_pair[1]; > + free(num_pair); > + for (i = num1; i <= num2; i++) > + p_current_sl2vl_scope->from[i] = TRUE; > + } > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + } > + ; > + > +sl2vl_scope_to_list_of_ranges: list_of_ranges { > + int i; > + cl_list_iterator_t list_iterator; > + uint64_t * num_pair; > + uint8_t num1, num2; > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) > + { > + num_pair = (uint64_t*)cl_list_obj(list_iterator); > + if (num_pair) > + { > + if ( num_pair[0] < 0 || > + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) > + { > + yyerror("port number out of range 'to' list"); > + free(num_pair); > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + return 1; > + } > + num1 = (uint8_t)num_pair[0]; > + num2 = (uint8_t)num_pair[1]; > + free(num_pair); > + for (i = num1; i <= num2; i++) > + p_current_sl2vl_scope->to[i] = TRUE; > + } > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > + } > + ; > + > + > +sl2vl_scope_sl2vl_table: sl2vl_scope_sl2vl_table_start num_list { > + /* 'sl2vl-table' - one instance of exactly > + OSM_QOS_POLICY_SL2VL_TABLE_LEN numbers */ > + cl_list_iterator_t list_iterator; > + uint64_t num; > + uint64_t * p_num; > + int i = 0; > + > + if (p_current_sl2vl_scope->sl2vl_table_set) > + { > + yyerror("sl2vl-scope has more than one sl2vl-table"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + > + if (cl_list_count(&tmp_parser_struct.num_list) != OSM_QOS_POLICY_SL2VL_TABLE_LEN) > + { > + yyerror("wrong number of values in 'sl2vl-table' (should be 16)"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.num_list) ) > + { > + p_num = (uint64_t*)cl_list_obj(list_iterator); > + num = *p_num; > + free(p_num); > + if (num >= OSM_QOS_POLICY_MAX_VL_NUM) > + { > + yyerror("wrong VL value in 'sl2vl-table' (should be 0 to 15)"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + > + p_current_sl2vl_scope->sl2vl_table[i++] = (uint8_t)num; > + list_iterator = cl_list_next(list_iterator); > + } > + p_current_sl2vl_scope->sl2vl_table_set = TRUE; > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +sl2vl_scope_sl2vl_table_start: TK_SL2VL_TABLE { > + RESET_BUFFER; > + } > + ; > + > + /* > + * qos_level_entry values: > + * qos_level_name > + * qos_level_use > + * qos_level_sl > + * qos_level_mtu_limit > + * qos_level_rate_limit > + * qos_level_packet_life > + * qos_level_path_bits > + * qos_level_pkey > + */ > + > +qos_level_name: qos_level_name_start single_string { > + /* 'name' of 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_qos_level->name) > + { > + yyerror("qos-level has multiple 'name' tags"); > + cl_list_remove_all(&tmp_parser_struct.str_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_qos_level->name = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_level_name_start: TK_NAME { > + RESET_BUFFER; > + } > + ; > + > +qos_level_use: qos_level_use_start single_string { > + /* 'use' of 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_qos_level->use) > + { > + yyerror("qos-level has multiple 'use' tags"); > + cl_list_remove_all(&tmp_parser_struct.str_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_qos_level->use = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_level_use_start: TK_USE { > + RESET_BUFFER; > + } > + ; > + > +qos_level_sl: qos_level_sl_start single_number { > + /* 'sl' in 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + uint64_t * p_num; > + > + if (p_current_qos_level->sl_set) > + { > + yyerror("'qos-level' has multiple 'sl' tags"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + p_num = (uint64_t*)cl_list_obj(list_iterator); > + p_current_qos_level->sl = (uint8_t)(*p_num); > + free(p_num); > + p_current_qos_level->sl_set = TRUE; > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +qos_level_sl_start: TK_SL { > + RESET_BUFFER; > + } > + ; > + > +qos_level_mtu_limit: qos_level_mtu_limit_start single_number { > + /* 'mtu-limit' in 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + uint64_t * p_num; > + > + if (p_current_qos_level->mtu_limit_set) > + { > + yyerror("'qos-level' has multiple 'mtu-limit' tags"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + p_num = (uint64_t*)cl_list_obj(list_iterator); > + p_current_qos_level->mtu_limit = (uint8_t)(*p_num); > + free(p_num); > + p_current_qos_level->mtu_limit_set = TRUE; > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +qos_level_mtu_limit_start: TK_MTU_LIMIT { > + /* 'mtu-limit' in 'qos-level' - one instance */ > + RESET_BUFFER; > + } > + ; > + > +qos_level_rate_limit: qos_level_rate_limit_start single_number { > + /* 'rate-limit' in 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + uint64_t * p_num; > + > + if (p_current_qos_level->rate_limit_set) > + { > + yyerror("'qos-level' has multiple 'rate-limit' tags"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + p_num = (uint64_t*)cl_list_obj(list_iterator); > + p_current_qos_level->rate_limit = (uint8_t)(*p_num); > + free(p_num); > + p_current_qos_level->rate_limit_set = TRUE; > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +qos_level_rate_limit_start: TK_RATE_LIMIT { > + /* 'rate-limit' in 'qos-level' - one instance */ > + RESET_BUFFER; > + } > + ; > + > +qos_level_packet_life: qos_level_packet_life_start single_number { > + /* 'packet-life' in 'qos-level' - one instance */ > + cl_list_iterator_t list_iterator; > + uint64_t * p_num; > + > + if (p_current_qos_level->pkt_life_set) > + { > + yyerror("'qos-level' has multiple 'packet-life' tags"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + list_iterator = cl_list_head(&tmp_parser_struct.num_list); > + p_num = (uint64_t*)cl_list_obj(list_iterator); > + p_current_qos_level->pkt_life = (uint8_t)(*p_num); > + free(p_num); > + p_current_qos_level->pkt_life_set= TRUE; > + cl_list_remove_all(&tmp_parser_struct.num_list); > + } > + ; > + > +qos_level_packet_life_start: TK_PACKET_LIFE { > + /* 'packet-life' in 'qos-level' - one instance */ > + RESET_BUFFER; > + } > + ; > + > +qos_level_path_bits: qos_level_path_bits_start list_of_ranges { > + /* 'path-bits' in 'qos-level' - any num of instances */ > + /* list of path bit ranges */ > + > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_qos_level->path_bits_range_len ) > + { > + p_current_qos_level->path_bits_range_arr = range_arr; > + p_current_qos_level->path_bits_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_qos_level->path_bits_range_arr, > + p_current_qos_level->path_bits_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_qos_level->path_bits_range_arr = new_range_arr; > + p_current_qos_level->path_bits_range_len = new_range_len; > + } > + } > + } > + ; > + > +qos_level_path_bits_start: TK_PATH_BITS { > + RESET_BUFFER; > + } > + ; > + > +qos_level_pkey: qos_level_pkey_start list_of_ranges { > + /* 'pkey' in 'qos-level' - num of instances of list of ranges */ > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_qos_level->pkey_range_len ) > + { > + p_current_qos_level->pkey_range_arr = range_arr; > + p_current_qos_level->pkey_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_qos_level->pkey_range_arr, > + p_current_qos_level->pkey_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_qos_level->pkey_range_arr = new_range_arr; > + p_current_qos_level->pkey_range_len = new_range_len; > + } > + } > + } > + ; > + > +qos_level_pkey_start: TK_PKEY { > + RESET_BUFFER; > + } > + ; > + > + /* > + * qos_match_rule_entry values: > + * qos_match_rule_use > + * qos_match_rule_qos_class > + * qos_match_rule_qos_level_name > + * qos_match_rule_source > + * qos_match_rule_destination > + * qos_match_rule_service_id > + * qos_match_rule_pkey > + */ > + > + > +qos_match_rule_use: qos_match_rule_use_start single_string { > + /* 'use' of 'qos-match-rule' - one instance */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_qos_match_rule->use) > + { > + yyerror("'qos-match-rule' has multiple 'use' tags"); > + cl_list_remove_all(&tmp_parser_struct.str_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_qos_match_rule->use = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_match_rule_use_start: TK_USE { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_qos_class: qos_match_rule_qos_class_start list_of_ranges { > + /* 'qos-class' in 'qos-match-rule' - num of instances of list of ranges */ > + /* list of class ranges (QoS Class is 12-bit value) */ > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_qos_match_rule->qos_class_range_len ) > + { > + p_current_qos_match_rule->qos_class_range_arr = range_arr; > + p_current_qos_match_rule->qos_class_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_qos_match_rule->qos_class_range_arr, > + p_current_qos_match_rule->qos_class_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_qos_match_rule->qos_class_range_arr = new_range_arr; > + p_current_qos_match_rule->qos_class_range_len = new_range_len; > + } > + } > + } > + ; > + > +qos_match_rule_qos_class_start: TK_QOS_CLASS { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_source: qos_match_rule_source_start string_list { > + /* 'source' in 'qos-match-rule' - text */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_qos_match_rule->source_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_match_rule_source_start: TK_SOURCE { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_destination: qos_match_rule_destination_start string_list { > + /* 'destination' in 'qos-match-rule' - text */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + cl_list_insert_tail(&p_current_qos_match_rule->destination_list,tmp_str); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_match_rule_destination_start: TK_DESTINATION { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_qos_level_name: qos_match_rule_qos_level_name_start single_string { > + /* 'qos-level-name' in 'qos-match-rule' - single string */ > + cl_list_iterator_t list_iterator; > + char * tmp_str; > + > + if (p_current_qos_match_rule->qos_level_name) > + { > + yyerror("qos-match-rule has multiple 'qos-level-name' tags"); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + return 1; > + } > + > + list_iterator = cl_list_head(&tmp_parser_struct.str_list); > + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) > + { > + tmp_str = (char*)cl_list_obj(list_iterator); > + if (tmp_str) > + p_current_qos_match_rule->qos_level_name = tmp_str; > + } > + cl_list_remove_all(&tmp_parser_struct.str_list); > + } > + ; > + > +qos_match_rule_qos_level_name_start: TK_QOS_LEVEL_NAME { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_service_id: qos_match_rule_service_id_start list_of_ranges { > + /* 'service-id' in 'qos-match-rule' - num of instances of list of ranges */ > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_qos_match_rule->service_id_range_len ) > + { > + p_current_qos_match_rule->service_id_range_arr = range_arr; > + p_current_qos_match_rule->service_id_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_qos_match_rule->service_id_range_arr, > + p_current_qos_match_rule->service_id_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_qos_match_rule->service_id_range_arr = new_range_arr; > + p_current_qos_match_rule->service_id_range_len = new_range_len; > + } > + } > + } > + ; > + > +qos_match_rule_service_id_start: TK_SERVICE_ID { > + RESET_BUFFER; > + } > + ; > + > +qos_match_rule_pkey: qos_match_rule_pkey_start list_of_ranges { > + /* 'pkey' in 'qos-match-rule' - num of instances of list of ranges */ > + if (cl_list_count(&tmp_parser_struct.num_pair_list)) > + { > + uint64_t ** range_arr; > + unsigned range_len; > + > + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, > + &range_arr, > + &range_len ); > + > + if ( !p_current_qos_match_rule->pkey_range_len ) > + { > + p_current_qos_match_rule->pkey_range_arr = range_arr; > + p_current_qos_match_rule->pkey_range_len = range_len; > + } > + else > + { > + uint64_t ** new_range_arr; > + unsigned new_range_len; > + __merge_rangearr( p_current_qos_match_rule->pkey_range_arr, > + p_current_qos_match_rule->pkey_range_len, > + range_arr, > + range_len, > + &new_range_arr, > + &new_range_len ); > + p_current_qos_match_rule->pkey_range_arr = new_range_arr; > + p_current_qos_match_rule->pkey_range_len = new_range_len; > + } > + } > + } > + ; > + > +qos_match_rule_pkey_start: TK_PKEY { > + RESET_BUFFER; > + } > + ; > + > + > + /* > + * Common part > + */ > + > + > +single_string: single_string_elems { > + cl_list_insert_tail(&tmp_parser_struct.str_list, > + strdup(__parser_strip_white(tmp_parser_struct.str))); > + tmp_parser_struct.str[0] = '\0'; > + } > + ; > + > +single_string_elems: single_string_element > + | single_string_elems single_string_element > + ; > + > +single_string_element: TK_TEXT { > + strcat(tmp_parser_struct.str,$1); > + free($1); > + } > + ; > + > + > +string_list: single_string > + | string_list TK_COMMA single_string > + ; > + > + > + > +single_number: number > + ; > + > +num_list: number > + | num_list TK_COMMA number > + ; > + > +number: TK_NUMBER { > + uint64_t * p_num = (uint64_t*)malloc(sizeof(uint64_t)); > + __parser_str2uint64(p_num,$1); > + free($1); > + cl_list_insert_tail(&tmp_parser_struct.num_list, p_num); > + } > + ; > + > +num_list_with_dotdot: number_from_pair_1 TK_DOTDOT number_from_pair_2 { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + | num_list_with_dotdot TK_COMMA number_from_pair_1 TK_DOTDOT number_from_pair_2 { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + ; > + > +number_from_pair_1: TK_NUMBER { > + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); > + free($1); > + } > + ; > + > +number_from_pair_2: TK_NUMBER { > + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); > + free($1); > + } > + ; > + > +list_of_ranges: num_list_with_dash > + ; > + > +num_list_with_dash: single_number_from_range { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + | number_from_range_1 TK_DASH number_from_range_2 { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + } > + else { > + num_pair[1] = tmp_parser_struct.num_pair[0]; > + num_pair[0] = tmp_parser_struct.num_pair[1]; > + } > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + | num_list_with_dash TK_COMMA number_from_range_1 TK_DASH number_from_range_2 { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + } > + else { > + num_pair[1] = tmp_parser_struct.num_pair[0]; > + num_pair[0] = tmp_parser_struct.num_pair[1]; > + } > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + | num_list_with_dash TK_COMMA single_number_from_range { > + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); > + num_pair[0] = tmp_parser_struct.num_pair[0]; > + num_pair[1] = tmp_parser_struct.num_pair[1]; > + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); > + } > + ; > + > +single_number_from_range: TK_NUMBER { > + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); > + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); > + free($1); > + } > + ; > + > +number_from_range_1: TK_NUMBER { > + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); > + free($1); > + } > + ; > + > +number_from_range_2: TK_NUMBER { > + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); > + free($1); > + } > + ; > + > +%% > + > +/*************************************************** > + ***************************************************/ > + > +int osm_qos_parse_policy_file( > + IN osm_log_t * p_log, > + IN const char * policy_file) > +{ > + int res = 0; > + p_qos_parser_osm_log = p_log; > + > + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_parse_policy_file); > + > + p_qos_policy = NULL; > + yyin = fopen (policy_file, "r"); > + if (!yyin) > + { > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "osm_qos_parse: ERR AC01: " > + "Failed opening QoS policy file (%s)\n", > + policy_file); > + res = 1; > + goto Exit; > + } > + column_num = 1; > + line_num = 1; > + osm_qos_policy_create(); > + __parser_tmp_struct_init(); > + > + res = yyparse(); > + > + __parser_tmp_struct_destroy(); > + > + if (res != 0) > + { > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "osm_qos_parse: ERR AC02: " > + "Failed parsing QoS policy file (%s)\n", > + policy_file); > + p_qos_policy = NULL; > + res = 1; > + goto Exit; > + } > + > + if (osm_qos_policy_validate()) > + { > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "osm_qos_parse: ERR AC03: " > + "Error(s) in QoS policy file (%s)\n", > + policy_file); > + res = 1; > + goto Exit; > + } > + > + Exit: > + if (yyin) > + fclose(yyin); > + OSM_LOG_EXIT(p_qos_parser_osm_log); > + return res; > +} > + > +/*************************************************** > + ***************************************************/ > + > +int yywrap() > +{ > + return(1); > +} > + > +/*************************************************** > + ***************************************************/ > + > +void yyerror (char *s) > +{ > + OSM_LOG_ENTER(p_qos_parser_osm_log, yyerror); > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "yyerror: ERR AC03: " > + "Syntax error (line %d:%d): %s. " > + "Last text read: \"%s\"\n", > + line_num, column_num, s, __parser_strip_white(yytext)); > + OSM_LOG_EXIT(p_qos_parser_osm_log); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static char * __parser_strip_white(char * str) > +{ > + int i; > + for (i = (strlen(str)-1); i >= 0; i--) > + { > + if (isspace(str[i])) > + str[i] = '\0'; > + else > + break; > + } > + for (i = 0; i < strlen(str); i++) > + { > + if (!isspace(str[i])) > + break; > + } > + return &(str[i]); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_str2uint64(uint64_t * p_val, char * str) > +{ > +#if __WORDSIZE == 64 > + *p_val = strtoul(str, NULL, 0); > +#else > + *p_val = strtoull(str, NULL, 0); > +#endif > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_port_group_start() > +{ > + p_current_port_group = osm_qos_policy_port_group_create(); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int __parser_port_group_end() > +{ > + if(!p_current_port_group->name) > + { > + yyerror("port-group validation failed - no port group name specified"); > + return -1; > + } > + > + cl_list_insert_tail(&p_qos_policy->port_groups, > + p_current_port_group); > + p_current_port_group = NULL; > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_vlarb_scope_start() > +{ > + p_current_vlarb_scope = osm_qos_policy_vlarb_scope_create(); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int __parser_vlarb_scope_end() > +{ > + if ( !cl_list_count(&p_current_vlarb_scope->group_list) && > + !cl_list_count(&p_current_vlarb_scope->across_list) ) > + { > + yyerror("vlarb-scope validation failed - no port groups specified by 'group' or by 'across'"); > + return -1; > + } > + > + cl_list_insert_tail(&p_qos_policy->vlarb_tables, > + p_current_vlarb_scope); > + p_current_vlarb_scope = NULL; > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_sl2vl_scope_start() > +{ > + p_current_sl2vl_scope = osm_qos_policy_sl2vl_scope_create(); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int __parser_sl2vl_scope_end() > +{ > + if (!p_current_sl2vl_scope->sl2vl_table_set) > + { > + yyerror("sl2vl-scope validation failed - no sl2vl table specified"); > + return -1; > + } > + if ( !cl_list_count(&p_current_sl2vl_scope->group_list) && > + !cl_list_count(&p_current_sl2vl_scope->across_to_list) && > + !cl_list_count(&p_current_sl2vl_scope->across_from_list) ) > + { > + yyerror("sl2vl-scope validation failed - no port groups specified by 'group', 'across-to' or 'across-from'"); > + return -1; > + } > + > + cl_list_insert_tail(&p_qos_policy->sl2vl_tables, > + p_current_sl2vl_scope); > + p_current_sl2vl_scope = NULL; > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_qos_level_start() > +{ > + p_current_qos_level = osm_qos_policy_qos_level_create(); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int __parser_qos_level_end() > +{ > + if (!p_current_qos_level->sl_set) > + { > + yyerror("qos-level validation failed - no 'sl' specified"); > + return -1; > + } > + if (!p_current_qos_level->name) > + { > + yyerror("qos-level validation failed - no 'name' specified"); > + return -1; > + } > + > + cl_list_insert_tail(&p_qos_policy->qos_levels, > + p_current_qos_level); > + p_current_qos_level = NULL; > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_match_rule_start() > +{ > + p_current_qos_match_rule = osm_qos_policy_match_rule_create(); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int __parser_match_rule_end() > +{ > + if (!p_current_qos_match_rule->qos_level_name) > + { > + yyerror("match-rule validation failed - no 'qos-level-name' specified"); > + return -1; > + } > + > + cl_list_insert_tail(&p_qos_policy->qos_match_rules, > + p_current_qos_match_rule); > + p_current_qos_match_rule = NULL; > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_tmp_struct_init() > +{ > + tmp_parser_struct.str[0] = '\0'; > + cl_list_construct(&tmp_parser_struct.str_list); > + cl_list_init(&tmp_parser_struct.str_list, 10); > + cl_list_construct(&tmp_parser_struct.num_list); > + cl_list_init(&tmp_parser_struct.num_list, 10); > + cl_list_construct(&tmp_parser_struct.num_pair_list); > + cl_list_init(&tmp_parser_struct.num_pair_list, 10); > +} > + > +/*************************************************** > + ***************************************************/ > + > +/* > + * Do NOT free objects from the temp struct. > + * Either they are inserted into the parse tree data > + * structure, or they are already freed when copying > + * their values to the parse tree data structure. > + */ > +static void __parser_tmp_struct_reset() > +{ > + tmp_parser_struct.str[0] = '\0'; > + cl_list_remove_all(&tmp_parser_struct.str_list); > + cl_list_remove_all(&tmp_parser_struct.num_list); > + cl_list_remove_all(&tmp_parser_struct.num_pair_list); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __parser_tmp_struct_destroy() > +{ > + __parser_tmp_struct_reset(); > + cl_list_destroy(&tmp_parser_struct.str_list); > + cl_list_destroy(&tmp_parser_struct.num_list); > + cl_list_destroy(&tmp_parser_struct.num_pair_list); > +} > + > +/*************************************************** > + ***************************************************/ > + > +static int OSM_CDECL > +__cmp_num_range( > + const void * p1, > + const void * p2) > +{ > + uint64_t * pair1 = *((uint64_t **)p1); > + uint64_t * pair2 = *((uint64_t **)p2); > + > + if (pair1[0] < pair2[0]) > + return -1; > + if (pair1[0] > pair2[0]) > + return 1; > + > + if (pair1[1] < pair2[1]) > + return -1; > + if (pair1[1] > pair2[1]) > + return 1; > + > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __sort_reduce_rangearr( > + uint64_t ** arr, > + unsigned arr_len, > + uint64_t ** * p_res_arr, > + unsigned * p_res_arr_len ) > +{ > + unsigned i = 0; > + unsigned j = 0; > + unsigned last_valid_ind = 0; > + unsigned valid_cnt = 0; > + uint64_t ** res_arr; > + boolean_t * is_valir_arr; > + > + *p_res_arr = NULL; > + *p_res_arr_len = 0; > + > + qsort(arr, arr_len, sizeof(uint64_t*), __cmp_num_range); > + > + is_valir_arr = (boolean_t *)malloc(arr_len * sizeof(boolean_t)); > + is_valir_arr[last_valid_ind] = TRUE; > + valid_cnt++; > + for (i = 1; i < arr_len; i++) > + { > + if (arr[i][0] <= arr[last_valid_ind][1]) > + { > + if (arr[i][1] > arr[last_valid_ind][1]) > + arr[last_valid_ind][1] = arr[i][1]; > + free(arr[i]); > + arr[i] = NULL; > + is_valir_arr[i] = FALSE; > + } > + else if ((arr[i][0] - 1) == arr[last_valid_ind][1]) > + { > + arr[last_valid_ind][1] = arr[i][1]; > + free(arr[i]); > + arr[i] = NULL; > + is_valir_arr[i] = FALSE; > + } > + else > + { > + is_valir_arr[i] = TRUE; > + last_valid_ind = i; > + valid_cnt++; > + } > + } > + > + res_arr = (uint64_t **)malloc(valid_cnt * sizeof(uint64_t *)); > + for (i = 0; i < arr_len; i++) > + { > + if (is_valir_arr[i]) > + res_arr[j++] = arr[i]; > + } > + free(arr); > + > + *p_res_arr = res_arr; > + *p_res_arr_len = valid_cnt; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __rangelist2rangearr( > + cl_list_t * p_list, > + uint64_t ** * p_arr, > + unsigned * p_arr_len) > +{ > + cl_list_iterator_t list_iterator; > + unsigned len = cl_list_count(p_list); > + unsigned i = 0; > + uint64_t ** tmp_arr; > + uint64_t ** res_arr = NULL; > + unsigned res_arr_len = 0; > + > + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); > + > + list_iterator = cl_list_head(p_list); > + while( list_iterator != cl_list_end(p_list) ) > + { > + tmp_arr[i++] = (uint64_t *)cl_list_obj(list_iterator); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(p_list); > + > + __sort_reduce_rangearr( tmp_arr, > + len, > + &res_arr, > + &res_arr_len ); > + *p_arr = res_arr; > + *p_arr_len = res_arr_len; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __merge_rangearr( > + uint64_t ** range_arr_1, > + unsigned range_len_1, > + uint64_t ** range_arr_2, > + unsigned range_len_2, > + uint64_t ** * p_arr, > + unsigned * p_arr_len ) > +{ > + unsigned i = 0; > + unsigned j = 0; > + unsigned len = range_len_1 + range_len_2; > + uint64_t ** tmp_arr; > + uint64_t ** res_arr = NULL; > + unsigned res_arr_len = 0; > + > + *p_arr = NULL; > + *p_arr_len = 0; > + > + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); > + > + for (i = 0; i < range_len_1; i++) > + tmp_arr[j++] = range_arr_1[i]; > + for (i = 0; i < range_len_2; i++) > + tmp_arr[j++] = range_arr_2[i]; > + free(range_arr_1); > + free(range_arr_2); > + > + __sort_reduce_rangearr( tmp_arr, > + len, > + &res_arr, > + &res_arr_len ); > + *p_arr = res_arr; > + *p_arr_len = res_arr_len; > +} > + > +/*************************************************** > + ***************************************************/ > + > -- > 1.5.1.4 > From sashak at voltaire.com Wed Aug 22 22:40:27 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 08:40:27 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <46CAD250.6000603@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> Message-ID: <20070823054027.GK1397@sashak.voltaire.com> On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index 6dfa824..31e09f7 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > osm_vl15intf.c osm_vl_arb_rcv.c \ > st.c osm_perfmgr.c osm_perfmgr_db.c \ > - osm_event_plugin.c osm_dump.c > + osm_event_plugin.c osm_dump.c \ > + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > + > +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ This is what I have now. Any idea? make all-am make[1]: Entering directory `/home/sashak/src/m/opensm/opensm' flex --prefix=__qos_parser_ -o ./osm_qos_parser_l.c ./osm_qos_parser.l gcc -DHAVE_CONFIG_H -I. -I./../include -I./../../libibcommon/include/infiniband -I./../../libibumad/include/infiniband -Wall -DOSM_VENDOR_INTF_OPENIB -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -g -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 -MT opensm-osm_qos_parser_l.o -MD -MP -MF .deps/opensm-osm_qos_parser_l.Tpo -c -o opensm-osm_qos_parser_l.o `test -f 'osm_qos_parser_l.c' || echo './'`osm_qos_parser_l.c ./osm_qos_parser.l: In function '__qos_parser_lex': ./osm_qos_parser.l:178: error: 'yylval' undeclared (first use in this function) ./osm_qos_parser.l:178: error: (Each undeclared identifier is reported only once ./osm_qos_parser.l:178: error: for each function it appears in.) make[1]: *** [opensm-osm_qos_parser_l.o] Error 1 make[1]: Leaving directory `/home/sashak/src/m/opensm/opensm' make: *** [all] Error 2 Sasha From sashak at voltaire.com Wed Aug 22 22:53:00 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 08:53:00 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <46CAD250.6000603@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> Message-ID: <20070823055300.GL1397@sashak.voltaire.com> Yevgeny, Hi again, On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: > Compiling QoS policy file parser. > > C & H files are generated from Lex & Yacc files and the > generated files are compiled. > Generated files are not included in git, but they do get > included in RPM as they are also created by 'make dist'. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/Makefile.am | 2 ++ > opensm/opensm/Makefile.am | 16 +++++++++++++++- > opensm/opensm/configure.in | 2 ++ > 3 files changed, 19 insertions(+), 1 deletions(-) > > diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > index ea62507..b83205f 100644 > --- a/opensm/include/Makefile.am > +++ b/opensm/include/Makefile.am > @@ -90,6 +90,8 @@ EXTRA_DIST = \ > $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > $(srcdir)/opensm/osm_perfmgr.h \ > $(srcdir)/opensm/osm_perfmgr_db.h \ > + $(srcdir)/opensm/osm_qos_policy.h \ > + $(srcdir)/opensm/osm_qos_parser_y.h \ > $(srcdir)/complib/cl_thread_osd.h \ > $(srcdir)/complib/cl_packon.h \ > $(srcdir)/complib/cl_atomic_osd.h \ > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index 6dfa824..31e09f7 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > osm_vl15intf.c osm_vl_arb_rcv.c \ > st.c osm_perfmgr.c osm_perfmgr_db.c \ > - osm_event_plugin.c osm_dump.c > + osm_event_plugin.c osm_dump.c \ > + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > + > +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > + > +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h > + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l > + Should generated files be cleaned somewhere? > if OSMV_OPENIB > opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > @@ -104,3 +112,9 @@ install-exec-hook: > if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ > lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ > ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so > + > +# generate c and h files from the lex and yacc files > +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l > + Will simpler 'dist-hook' like this dist-hook: osm_qos_parser_y.c osm_qos_parser_l.c be fine here? > diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in > index afb20b3..a49538d 100644 > --- a/opensm/opensm/configure.in > +++ b/opensm/opensm/configure.in > @@ -22,6 +22,8 @@ AC_PROG_INSTALL > AC_PROG_LN_S > AC_PROG_MAKE_SET > AC_PROG_LIBTOOL > +AM_PROG_LEX > +AC_PROG_YACC With tarball where *.c files are generated already we will not need lex and yacc (OTOH it is needed for "regular" (or "maintainer-mode") build). Any idea how to avoid this dependencies for tarball builds? Sasha From vu at mellanox.com Wed Aug 22 23:14:33 2007 From: vu at mellanox.com (Vu Pham) Date: Wed, 22 Aug 2007 23:14:33 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: References: <1186200658.6031.44.camel@gentoo-linux.localdomain> <46C9EEF4.6000407@mellanox.com> Message-ID: <46CD25C9.3080104@mellanox.com> ofed-1.2.5 is package - it can be build and run on many kernel (with appropriate patches applied) ofed_kernel kernel git tree (ofa_kernel-1.2.5 kernel ib package is from this git tree) is 2.6.23-rc3. I have not tested scst svn with 2.6.23-rcX - I don't know if anyone on scst community test scst with 2.6.23-rcX or not -vu > How did you get the SCST to compile against the current ofed 1.2.5 > kernel tree? SCST svn fails with ofed 1.2.5 kernel 2.6.22 due to changes > in the SLAB routines. > > Tried pruning the extra NULL arg to kmem_cache_create, and got the > attached oopses when the target tried to register the SCST template. > > -----Original Message----- > From: Vu Pham [mailto:vu at mellanox.com] > Sent: Monday, August 20, 2007 12:44 PM > To: Stanley Sufficool > Cc: general > Subject: Re: [ofa-general] [PATCH] SRPT for current SCST > > Thanks. I checked in with a few other changes/bug fixes. > Please test and report any problem > >> SCST redefined the scst_tgt_template and it no longer contains the >> tm_sync_reply field. Also an extra arg is required for scst_register. >> Consequently, the build fails. >> >> I searched through past SCST versions and docs but could not find what > >> this was for and it seems to be set to zero (not used?) in this struct > >> anyhow. >> >> >> ---------------------------------------------------------------------- >> -- >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general > From vu at mellanox.com Wed Aug 22 23:40:04 2007 From: vu at mellanox.com (Vu Pham) Date: Wed, 22 Aug 2007 23:40:04 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: References: <200708221128.l7MBSQ2H004759@cmf.nrl.navy.mil> Message-ID: <46CD2BC4.5020803@mellanox.com> You can get hardware max rdma atomic by doing ib_query_qp CM save the initiator's req->init_depth to its cm_id_priv->responder_resources in cm_req_handler (Win's srp initiator set it at 16) srpt call ib_cm_init_qp_attr() which will initialize qp_attr->max_dest_rd_atomic = cm_id_priv->responder_resources = 16; then srpt call ib_modify_qp() without checking/reseting the hw max rdma atomic in qp_attr; therefore, ib_modify_qp() failed You can try a quick dirty fix by applying this patch to ib_srpt.c -vu > So the client is requesting a QP with a max rdma_atomic of 15, but the > target initializes the QP with a max of 4. Is this max rdma_atomic of 4 > based on hardware capabilities (Memory vs MemFree adapters), or a target > QP initialization preset value? I'll swap adapters if needed. > > -----Original Message----- > From: chas williams - CONTRACTOR [mailto:chas at cmf.nrl.navy.mil] > Sent: Wednesday, August 22, 2007 4:28 AM > To: Sufficool, Stanley > Cc: Vu Pham; Stanley Sufficool; general > Subject: Re: [ofa-general] [PATCH] SRPT for current SCST > > In message > ,"Su > fficool, Stanley" writes: >> I'm still researching where the rdma_atomic is getting the invalid >> attribute. I will give an update if I find anything else. > > srp clients set this value as the responder_resources during login i > believe. this seems to be done in srp_get_responder_resources() on the > windows srp client. its taken from the qp attributes instead of picking > a fixed number, like 4 in the linux version of the srp client. -------------- next part -------------- A non-text attachment was scrubbed... Name: qp_max_atomic_res.patch Type: text/x-diff Size: 917 bytes Desc: not available URL: From sashak at voltaire.com Wed Aug 22 23:46:48 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 09:46:48 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <46CAD09C.4030400@dev.mellanox.co.il> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> Message-ID: <20070823064648.GM1397@sashak.voltaire.com> Hi Yevgeny, Some initial comments below. Sasha On 14:46 Tue 21 Aug , Yevgeny Kliteynik wrote: > QoS policy data structures and functions > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/opensm/osm_qos_policy.h | 188 +++++++ > opensm/opensm/osm_qos_policy.c | 901 ++++++++++++++++++++++++++++++++ > 2 files changed, 1089 insertions(+), 0 deletions(-) > create mode 100644 opensm/include/opensm/osm_qos_policy.h > create mode 100644 opensm/opensm/osm_qos_policy.c > > diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h > new file mode 100644 > index 0000000..05b8dc9 > --- /dev/null > +++ b/opensm/include/opensm/osm_qos_policy.h > @@ -0,0 +1,188 @@ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * Declaration of OSM QoS Policy data types and functions. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#ifndef OSM_QOS_POLICY_H > +#define OSM_QOS_POLICY_H > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define YYSTYPE char * > +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 > +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" > + > +/***************************************************/ > + > +typedef struct osm_qos_port_group_t_ { > + char *name; /* single string (this port group name) */ > + char *use; /* single string (description) */ > + cl_list_t port_name_list; /* list of port names (.../.../...) */ > + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ > + unsigned guid_range_len; /* num of guid ranges in the array */ > + cl_list_t partition_list; /* list of partition names */ > + boolean_t node_type_ca; > + boolean_t node_type_switch; > + boolean_t node_type_router; > + boolean_t node_type_self; > +} osm_qos_port_group_t; I see you are using this in "run-time", not just during the parsing. Instead of having all this config features you can just resolve port guids in parse time and keep it here in cl_map() for fast searches. > + > +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > +void osm_qos_policy_port_group_destroy(); Would be nice to have function prototypes in one place. > + > +/***************************************************/ > + > +typedef struct osm_qos_vlarb_scope_t_ { > + cl_list_t group_list; /* list of group names (strings) */ > + cl_list_t across_list; /* list of 'across' group names (strings) */ > + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ > + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ Why cl_list for VLArb? it should be short fixed length arrays? > + uint32_t vl_high_limit; /* single integer */ > + boolean_t vl_high_limit_set; > +} osm_qos_vlarb_scope_t; > + > +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); > +void osm_qos_policy_vlarb_scope_destroy(); > + > +/***************************************************/ > + > +typedef struct osm_qos_sl2vl_scope_t_ { > + cl_list_t group_list; /* list of strings (port group names) */ > + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > + cl_list_t across_from_list; /* list of strings (port group names) */ > + cl_list_t across_to_list; /* list of strings (port group names) */ > + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > + boolean_t sl2vl_table_set; > +} osm_qos_sl2vl_scope_t; This will be used for sl2vl setup? Same as above - why not to generate final port guid list just in parse time? > + > +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); > +void osm_qos_policy_sl2vl_scope_destroy(); > + > +/***************************************************/ > + > +typedef struct osm_qos_level_t_ { > + char *use; > + char *name; > + uint8_t sl; > + boolean_t sl_set; > + uint8_t mtu_limit; > + boolean_t mtu_limit_set; > + uint8_t rate_limit; > + boolean_t rate_limit_set; > + uint8_t pkt_life; > + boolean_t pkt_life_set; > + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ > + unsigned path_bits_range_len; /* num of bit ranges in the array */ > + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ > + unsigned pkey_range_len; > +} osm_qos_level_t; > + > +osm_qos_level_t *osm_qos_policy_qos_level_create(); > +void osm_qos_policy_qos_level_destroy(); > + > +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > + IN ib_net16_t pkey); > + > +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp); > + > +/***************************************************/ > + > +typedef struct osm_qos_match_rule_t_ { > + char *use; > + cl_list_t source_list; /* list of strings */ > + cl_list_t source_group_list; /* list of pointers to relevant port-group */ > + cl_list_t destination_list; /* list of strings */ > + cl_list_t destination_group_list; /* list of pointers to relevant port-group */ I think you should only keep port guids there (mapped for fast searches). > + char *qos_level_name; > + osm_qos_level_t *p_qos_level; Why do you need qos_level_name if you keep the pointer to this qos_level struct? > + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) */ > + unsigned service_id_range_len; > + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ > + unsigned qos_class_range_len; > + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ > + unsigned pkey_range_len; > +} osm_qos_match_rule_t; > + > +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > +void osm_qos_policy_match_rule_destroy(); > + > +/***************************************************/ > + > +typedef struct osm_qos_policy_t_ { > + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > + cl_list_t qos_levels; /* list of osm_qos_level_t */ > + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ > + osm_qos_level_t *p_default_qos_level; /* default QoS level */ > +} osm_qos_policy_t; > + > +void osm_qos_policy_create(); > +void osm_qos_policy_destroy(); > +int osm_qos_policy_validate(); > + > +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, > + IN const ib_path_rec_t * p_pr, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp, > + IN ib_net64_t comp_mask, > + OUT osm_qos_level_t ** pp_qos_level); > + > +/***************************************************/ > + > +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char *policy_file); > + > +/***************************************************/ > + > +#endif /* ifndef OSM_QOS_POLICY_H */ > diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c > new file mode 100644 > index 0000000..bc2aa68 > --- /dev/null > +++ b/opensm/opensm/osm_qos_policy.c > @@ -0,0 +1,901 @@ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * OSM QoS Policy functions. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#include > +#include > +#include > + > +extern void yyerror(char *s); > +osm_log_t *p_qos_parser_osm_log = NULL; > +osm_qos_policy_t *p_qos_policy = NULL; Please try to avoid globals - keep it as part of osm_opensm_t or osm_subn_t structures. > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__is_num_in_range_arr(uint64_t ** range_arr, > + unsigned range_arr_len, uint64_t num) > +{ > + unsigned i; > + > + /* > + * TODO: although this array should be rather short, > + * I should implement a binary search here > + */ > + > + if (num < range_arr[0][0] || num > range_arr[range_arr_len - 1][1]) > + return FALSE; > + > + for (i = 0; i < range_arr_len; i++) { > + if (num >= range_arr[i][0] && num <= range_arr[i][1]) > + return TRUE; > + if (num < range_arr[i][0]) > + return FALSE; > + } > + return FALSE; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __free_single_element(void *p_element, void *context) > +{ > + if (p_element) > + free(p_element); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_port_group_t *osm_qos_policy_port_group_create() > +{ > + osm_qos_port_group_t *p = > + (osm_qos_port_group_t *) malloc(sizeof(osm_qos_port_group_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_port_group_t)); > + > + cl_list_init(&p->port_name_list, 10); > + cl_list_init(&p->partition_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->name) > + free(p->name); > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->guid_range_len; i++) > + free(p->guid_range_arr[i]); > + if (p->guid_range_arr) > + free(p->guid_range_arr); > + > + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); > + cl_list_remove_all(&p->port_name_list); > + cl_list_destroy(&p->port_name_list); > + > + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); > + cl_list_remove_all(&p->partition_list); > + cl_list_destroy(&p->partition_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create() > +{ > + osm_qos_vlarb_scope_t *p = > + (osm_qos_vlarb_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); > + > + cl_list_init(&p->group_list, 10); > + cl_list_init(&p->across_list, 10); > + cl_list_init(&p->vlarb_high_list, 10); > + cl_list_init(&p->vlarb_low_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) > +{ > + if (!p) > + return; > + > + cl_list_apply_func(&p->group_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_list, __free_single_element, NULL); > + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); > + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); > + > + cl_list_remove_all(&p->group_list); > + cl_list_remove_all(&p->across_list); > + cl_list_remove_all(&p->vlarb_high_list); > + cl_list_remove_all(&p->vlarb_low_list); > + > + cl_list_destroy(&p->group_list); > + cl_list_destroy(&p->across_list); > + cl_list_destroy(&p->vlarb_high_list); > + cl_list_destroy(&p->vlarb_low_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create() > +{ > + osm_qos_sl2vl_scope_t *p = > + (osm_qos_sl2vl_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); > + > + cl_list_init(&p->group_list, 10); > + cl_list_init(&p->across_from_list, 10); > + cl_list_init(&p->across_to_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) > +{ > + if (!p) > + return; > + > + cl_list_apply_func(&p->group_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); > + > + cl_list_remove_all(&p->group_list); > + cl_list_remove_all(&p->across_from_list); > + cl_list_remove_all(&p->across_to_list); > + > + cl_list_destroy(&p->group_list); > + cl_list_destroy(&p->across_from_list); > + cl_list_destroy(&p->across_to_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_level_t *osm_qos_policy_qos_level_create() > +{ > + osm_qos_level_t *p = > + (osm_qos_level_t *) malloc(sizeof(osm_qos_level_t)); > + if (!p) > + return NULL; > + memset(p, 0, sizeof(osm_qos_level_t)); > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->path_bits_range_len; i++) > + free(p->path_bits_range_arr[i]); > + if (p->path_bits_range_arr) > + free(p->path_bits_range_arr); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > + IN ib_net16_t pkey) > +{ > + if (!p_qos_level || !p_qos_level->pkey_range_len) > + return FALSE; > + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, > + p_qos_level->pkey_range_len, > + cl_ntoh16(pkey)); > +} > + > +/*************************************************** > + ***************************************************/ > + > +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp) > +{ > + unsigned i; > + uint16_t pkey_ho = 0; > + > + if (!p_qos_level || !p_qos_level->pkey_range_len) > + return 0; > + > + /* > + * ToDo: This approach is not optimal. > + * Think how to find shared pkey that also exists > + * in QoS level in less runtime. > + */ > + > + for (i = 0; i < p_qos_level->pkey_range_len; i++) { > + for (pkey_ho = p_qos_level->pkey_range_arr[i][0]; > + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; pkey_ho++) { > + if (osm_physp_share_this_pkey > + (p_src_physp, p_dest_physp, cl_hton16(pkey_ho))) > + return cl_hton16(pkey_ho); > + } > + } > + > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_match_rule_t *osm_qos_policy_match_rule_create() > +{ > + osm_qos_match_rule_t *p = > + (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_match_rule_t)); > + > + cl_list_init(&p->source_list, 10); > + cl_list_init(&p->source_group_list, 10); > + cl_list_init(&p->destination_list, 10); > + cl_list_init(&p->destination_group_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->qos_level_name) > + free(p->qos_level_name); > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->service_id_range_len; i++) > + free(p->service_id_range_arr[i]); > + if (p->service_id_range_arr) > + free(p->service_id_range_arr); > + > + for (i = 0; i < p->qos_class_range_len; i++) > + free(p->qos_class_range_arr[i]); > + if (p->qos_class_range_arr) > + free(p->qos_class_range_arr); > + > + cl_list_apply_func(&p->source_list, __free_single_element, NULL); > + cl_list_remove_all(&p->source_list); > + cl_list_destroy(&p->source_list); > + > + cl_list_remove_all(&p->source_group_list); > + cl_list_destroy(&p->source_group_list); > + > + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); > + cl_list_remove_all(&p->destination_list); > + cl_list_destroy(&p->destination_list); > + > + cl_list_remove_all(&p->destination_group_list); > + cl_list_destroy(&p->destination_group_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_create() > +{ > + if (p_qos_policy) > + osm_qos_policy_destroy(); > + > + p_qos_policy = (osm_qos_policy_t *) malloc(sizeof(osm_qos_policy_t)); > + CL_ASSERT(p_qos_policy); > + > + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); > + > + cl_list_construct(&p_qos_policy->port_groups); > + cl_list_init(&p_qos_policy->port_groups, 10); > + > + cl_list_construct(&p_qos_policy->vlarb_tables); > + cl_list_init(&p_qos_policy->vlarb_tables, 10); > + > + cl_list_construct(&p_qos_policy->sl2vl_tables); > + cl_list_init(&p_qos_policy->sl2vl_tables, 10); > + > + cl_list_construct(&p_qos_policy->qos_levels); > + cl_list_init(&p_qos_policy->qos_levels, 10); > + > + cl_list_construct(&p_qos_policy->qos_match_rules); > + cl_list_init(&p_qos_policy->qos_match_rules, 10); > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_destroy() > +{ > + cl_list_iterator_t list_iterator; > + osm_qos_port_group_t *p_port_group = NULL; > + osm_qos_vlarb_scope_t *p_vlarb_scope = NULL; > + osm_qos_sl2vl_scope_t *p_sl2vl_scope = NULL; > + osm_qos_level_t *p_qos_level = NULL; > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + > + list_iterator = cl_list_head(&p_qos_policy->port_groups); > + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (p_port_group) > + osm_qos_policy_port_group_destroy(p_port_group); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->port_groups); > + cl_list_destroy(&p_qos_policy->port_groups); > + > + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); > + while (list_iterator != cl_list_end(&p_qos_policy->vlarb_tables)) { > + p_vlarb_scope = > + (osm_qos_vlarb_scope_t *) cl_list_obj(list_iterator); > + if (p_vlarb_scope) > + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->vlarb_tables); > + cl_list_destroy(&p_qos_policy->vlarb_tables); > + > + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); > + while (list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables)) { > + p_sl2vl_scope = > + (osm_qos_sl2vl_scope_t *) cl_list_obj(list_iterator); > + if (p_sl2vl_scope) > + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->sl2vl_tables); > + cl_list_destroy(&p_qos_policy->sl2vl_tables); > + > + list_iterator = cl_list_head(&p_qos_policy->qos_levels); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { > + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); > + if (p_qos_level) > + osm_qos_policy_qos_level_destroy(p_qos_level); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->qos_levels); > + cl_list_destroy(&p_qos_policy->qos_levels); > + > + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); > + if (p_qos_match_rule) > + osm_qos_policy_match_rule_destroy(p_qos_match_rule); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->qos_match_rules); > + cl_list_destroy(&p_qos_policy->qos_match_rules); > + > + free(p_qos_policy); > + > + p_qos_policy = NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__qos_policy_is_port_in_group(osm_subn_t * p_subn, > + const osm_physp_t * p_physp, > + osm_qos_port_group_t * p_port_group) > +{ > + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); > + osm_prtn_t *p_prtn = NULL; > + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); > + uint64_t port_guid_ho = cl_ntoh64(port_guid); > + uint8_t node_type = osm_node_get_type(p_node); > + cl_list_iterator_t list_iterator; > + char *partition_name; > + > + /* check whether this port's type matches any of group's types */ > + > + if ((node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || > + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) > + || (node_type == IB_NODE_TYPE_ROUTER > + && p_port_group->node_type_router)) > + return TRUE; > + > + /* check whether this port's guid is in range of this group's guids */ > + > + if (__is_num_in_range_arr(p_port_group->guid_range_arr, > + p_port_group->guid_range_len, port_guid_ho)) > + return TRUE; > + > + /* check whether this port is member of this group's partitions */ > + > + list_iterator = cl_list_head(&p_port_group->partition_list); > + while (list_iterator != cl_list_end(&p_port_group->partition_list)) { > + partition_name = (char *)cl_list_obj(list_iterator); > + if (partition_name && strlen(partition_name)) { > + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); > + if (p_prtn) { > + if (osm_prtn_is_guid(p_prtn, port_guid)) > + return TRUE; > + } > + } > + list_iterator = cl_list_next(list_iterator); > + } > + > + /* check whether this port's name matches any of group's names */ > + > + /* > + * TODO: check port names > + * > + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; > + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); > + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; > + */ > + > + return FALSE; > +} /* __qos_policy_is_port_in_group() */ > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__qos_policy_is_port_in_group_list(const osm_pr_rcv_t * p_rcv, > + const osm_physp_t * p_physp, > + cl_list_t * p_port_group_list) > +{ > + osm_qos_port_group_t *p_port_group; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(p_port_group_list); > + while (list_iterator != cl_list_end(p_port_group_list)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (p_port_group) { > + if (__qos_policy_is_port_in_group > + (p_rcv->p_subn, p_physp, p_port_group)) > + return TRUE; > + } > + list_iterator = cl_list_next(list_iterator); > + } > + return FALSE; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr( > + const osm_pr_rcv_t * p_rcv, > + const ib_path_rec_t * p_pr, > + const osm_physp_t * p_src_physp, > + const osm_physp_t * p_dest_physp, > + ib_net64_t comp_mask) > +{ > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + cl_list_iterator_t list_iterator; > + > + if (!cl_list_count(&p_qos_policy->qos_match_rules)) > + return NULL; > + > + /* Go over all QoS match rules and find the one that matches the request */ > + > + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); > + if (!p_qos_match_rule) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + /* If a match rule has Source groups, PR request source has to be in this list */ > + > + if (cl_list_count(&p_qos_match_rule->source_group_list)) { > + if (!__qos_policy_is_port_in_group_list(p_rcv, > + p_src_physp, > + &p_qos_match_rule-> > + source_group_list)) > + { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + } > + > + /* If a match rule has Destination groups, PR request dest. has to be in this list */ > + > + if (cl_list_count(&p_qos_match_rule->destination_group_list)) { > + if (!__qos_policy_is_port_in_group_list(p_rcv, > + p_dest_physp, > + &p_qos_match_rule-> > + destination_group_list)) > + { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + } > + > + /* If a match rule has QoS classes, PR request HAS > + to have a matching QoS class to match the rule */ > + > + if (p_qos_match_rule->qos_class_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->qos_class_range_arr, > + p_qos_match_rule->qos_class_range_len, > + ib_path_rec_qos_class(p_pr))) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* If a match rule has Service IDs, PR request HAS > + to have a matching Service ID to match the rule */ > + > + if (p_qos_match_rule->service_id_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->service_id_range_arr, > + p_qos_match_rule->service_id_range_len, > + p_pr->service_id)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* If a match rule has PKeys, PR request HAS > + to have a matching PKey to match the rule */ > + > + if (p_qos_match_rule->pkey_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->pkey_range_arr, > + p_qos_match_rule->pkey_range_len, > + ib_path_rec_qos_class(p_pr))) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* if we got here, then this match-rule matched this PR request */ > + break; > + } > + > + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) > + return NULL; > + > + return p_qos_match_rule; > +} /* __qos_policy_get_match_rule_by_pr() */ > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_level_t *__qos_policy_get_qos_level_by_name(char *name) > +{ > + osm_qos_level_t *p_qos_level = NULL; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(&p_qos_policy->qos_levels); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { > + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); > + if (!p_qos_level) > + continue; > + > + /* names are case INsensitive */ > + if (strcasecmp(name, p_qos_level->name) == 0) > + return p_qos_level; > + > + list_iterator = cl_list_next(list_iterator); > + } > + > + return NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_port_group_t *__qos_policy_get_port_group_by_name(const char > + *const name) > +{ > + osm_qos_port_group_t *p_port_group = NULL; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(&p_qos_policy->port_groups); > + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (!p_port_group) > + continue; > + > + /* names are case INsensitive */ > + if (strcasecmp(name, p_port_group->name) == 0) > + return p_port_group; > + > + list_iterator = cl_list_next(list_iterator); > + } > + > + return NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +int osm_qos_policy_validate() > +{ > + cl_list_iterator_t match_rules_list_iterator; > + cl_list_iterator_t list_iterator; > + osm_qos_port_group_t *p_port_group = NULL; > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + char *str; > + unsigned i; > + int res = 0; > + > + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_policy_validate); > + > + /* set default qos level */ > + > + p_qos_policy->p_default_qos_level = > + __qos_policy_get_qos_level_by_name > + (OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); > + if (!p_qos_policy->p_default_qos_level) { > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC10: " > + "Default qos-level (%s) not defined.\n", > + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); > + res = 1; > + goto Exit; > + } > + > + /* scan all the match rules, and fill the lists of pointers to > + relevant qos levels and port groups to speed up PR matching */ > + > + i = 1; > + match_rules_list_iterator = > + cl_list_head(&p_qos_policy->qos_match_rules); > + while (match_rules_list_iterator != > + cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) > + cl_list_obj(match_rules_list_iterator); > + CL_ASSERT(p_qos_match_rule); > + > + /* find the matching qos-level for each match-rule */ > + > + p_qos_match_rule->p_qos_level = > + __qos_policy_get_qos_level_by_name(p_qos_match_rule-> > + qos_level_name); > + > + if (!p_qos_match_rule->p_qos_level) { > + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC11: " > + "qos-match-rule num %u: qos-level '%s' not found\n", > + i, p_qos_match_rule->qos_level_name); > + res = 1; > + goto Exit; > + } > + > + /* find the matching port-group for element of source_list */ > + > + if (cl_list_count(&p_qos_match_rule->source_list)) { > + list_iterator = > + cl_list_head(&p_qos_match_rule->source_list); > + while (list_iterator != > + cl_list_end(&p_qos_match_rule->source_list)) { > + str = (char *)cl_list_obj(list_iterator); > + CL_ASSERT(str); > + > + p_port_group = > + __qos_policy_get_port_group_by_name(str); > + if (!p_port_group) { > + osm_log(p_qos_parser_osm_log, > + OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC12: " > + "qos-match-rule num %u: source port-group '%s' not found\n", > + i, str); > + res = 1; > + goto Exit; > + } > + > + cl_list_insert_tail(&p_qos_match_rule-> > + source_group_list, > + p_port_group); > + > + list_iterator = cl_list_next(list_iterator); > + } > + } > + > + /* find the matching port-group for element of destination_list */ > + > + if (cl_list_count(&p_qos_match_rule->destination_list)) { > + list_iterator = > + cl_list_head(&p_qos_match_rule->destination_list); > + while (list_iterator != > + cl_list_end(&p_qos_match_rule-> > + destination_list)) { > + str = (char *)cl_list_obj(list_iterator); > + CL_ASSERT(str); > + > + p_port_group = > + __qos_policy_get_port_group_by_name(str); > + if (!p_port_group) { > + osm_log(p_qos_parser_osm_log, > + OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC13: " > + "qos-match-rule num %u: destination port-group '%s' not found\n", > + i, str); > + res = 1; > + goto Exit; > + } > + > + cl_list_insert_tail(&p_qos_match_rule-> > + destination_group_list, > + p_port_group); > + > + list_iterator = cl_list_next(list_iterator); > + } > + } > + > + /* done with the current match-rule */ > + > + match_rules_list_iterator = > + cl_list_next(match_rules_list_iterator); > + i++; > + } > + > + Exit: > + OSM_LOG_EXIT(p_qos_parser_osm_log); > + return res; > +} /* osm_qos_policy_validate() */ > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, > + IN const ib_path_rec_t * p_pr, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp, > + IN ib_net64_t comp_mask, > + OUT osm_qos_level_t ** pp_qos_level) > +{ > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + osm_qos_level_t *p_qos_level = NULL; > + > + OSM_LOG_ENTER(p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr); > + > + *pp_qos_level = NULL; > + > + if (!p_qos_policy) > + goto Exit; > + > + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_rcv, > + p_pr, > + p_src_physp, > + p_dest_physp, > + comp_mask); > + > + if (p_qos_match_rule) > + p_qos_level = p_qos_match_rule->p_qos_level; > + else > + p_qos_level = p_qos_policy->p_default_qos_level; > + > + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, > + "osm_qos_policy_get_qos_level_by_pr: " > + "PathRecord request:" > + "Src port 0x%016" PRIx64 ", " > + "Dst port 0x%016" PRIx64 "\n", > + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), > + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp))); > + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, > + "osm_qos_policy_get_qos_level_by_pr: " > + "Applying QoS Level %s (%s)\n", > + p_qos_level->name, > + (p_qos_level->use) ? p_qos_level->use : "no description"); > + > + *pp_qos_level = p_qos_level; > + > + Exit: > + OSM_LOG_EXIT(p_rcv->p_log); > +} /* osm_qos_policy_get_qos_level_by_pr() */ > + > +/*************************************************** > + ***************************************************/ > -- > 1.5.1.4 > From kliteyn at dev.mellanox.co.il Thu Aug 23 00:34:15 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 10:34:15 +0300 Subject: [ofa-general] Re: [PATCH 2/7] osm: QoS - policy file parser Lex & Yacc In-Reply-To: <20070823053110.GJ1397@sashak.voltaire.com> References: <46C98364.7030100@dev.mellanox.co.il> <20070823053110.GJ1397@sashak.voltaire.com> Message-ID: <46CD3877.6040401@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 15:04 Mon 20 Aug , Yevgeny Kliteynik wrote: >> QoS Policy file parser - Lex & Yacc files >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/opensm/osm_qos_parser.l | 329 ++++++ >> opensm/opensm/osm_qos_parser.y | 2146 ++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 2475 insertions(+), 0 deletions(-) >> create mode 100755 opensm/opensm/osm_qos_parser.l >> create mode 100755 opensm/opensm/osm_qos_parser.y > > Should not be 0755 mode, right? (No need to repost for this) Right. These two files are "-rwxr-xr-x", while the rest are "-rw-r--r--". -- Yevgeny > Sasha > >> diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l >> new file mode 100755 >> index 0000000..3561f79 >> --- /dev/null >> +++ b/opensm/opensm/osm_qos_parser.l >> @@ -0,0 +1,329 @@ >> +%{ >> +/* >> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. >> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. >> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. >> + * >> + * This software is available to you under a choice of one of two >> + * licenses. You may choose to be licensed under the terms of the GNU >> + * General Public License (GPL) Version 2, available from the file >> + * COPYING in the main directory of this source tree, or the >> + * OpenIB.org BSD license below: >> + * >> + * Redistribution and use in source and binary forms, with or >> + * without modification, are permitted provided that the following >> + * conditions are met: >> + * >> + * - Redistributions of source code must retain the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer. >> + * >> + * - Redistributions in binary form must reproduce the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer in the documentation and/or other materials >> + * provided with the distribution. >> + * >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE >> + * SOFTWARE. >> + * >> + */ >> + >> +/* >> + * Abstract: >> + * Lexer of OSM QoS parser. >> + * >> + * Environment: >> + * Linux User Mode >> + * >> + * Author: >> + * Yevgeny Kliteynik, Mellanox >> + */ >> + >> +#include >> +#include >> + >> +#define HANDLE_IF_IN_DESCRIPTION if (in_description) { yylval = strdup(yytext); return TK_TEXT; } >> + >> +#define SAVE_POS save_pos() >> +static void save_pos(); >> + >> +extern int column_num; >> +extern int line_num; >> +extern FILE * yyin; >> + >> +boolean_t in_description = FALSE; >> +boolean_t in_list_of_hex_num_ranges = FALSE; >> +boolean_t in_node_type = FALSE; >> +boolean_t in_list_of_numbers = FALSE; >> +boolean_t in_list_of_strings = FALSE; >> +boolean_t in_list_of_num_pairs = FALSE; >> +boolean_t in_asterisk_or_list_of_numbers = FALSE; >> +boolean_t in_list_of_num_ranges = FALSE; >> +boolean_t in_single_string = FALSE; >> +boolean_t in_single_number = FALSE; >> + >> +static void reset_new_line_flags(); >> +#define RESET_NEW_LINE_FLAGS reset_new_line_flags() >> + >> +#define START_USE {in_description = TRUE;} /* list of strings including whitespace (description) */ >> +#define START_PORT_GUID {in_list_of_hex_num_ranges = TRUE;} /* comma-separated list of hex num ranges */ >> +#define START_PORT_NAME {in_list_of_strings = TRUE;} /* comma-separated list of following strings: ../../.. */ >> +#define START_PARTITION {in_single_string = TRUE;} /* single string w/o whitespaces (partition name) */ >> +#define START_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (port group name) */ >> +#define START_QOS_LEVEL_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (qos level name in match rule) */ >> + >> +#define START_NODE_TYPE {in_node_type = TRUE;} /* comma-separated list of node types (ROUTER,CA,...) */ >> +#define START_SL2VL_TABLE {in_list_of_numbers = TRUE;} /* comma-separated list of hex or dec numbers */ >> + >> +#define START_GROUP {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> +#define START_ACROSS {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> +#define START_ACROSS_TO {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> +#define START_ACROSS_FROM {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> +#define START_SOURCE {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> +#define START_DESTINATION {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ >> + >> +#define START_VLARB_HIGH {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ >> +#define START_VLARB_LOW {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ >> + >> +#define START_TO {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ >> +#define START_FROM {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ >> + >> +#define START_PATH_BITS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ >> +#define START_QOS_CLASS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ >> +#define START_SERVICE_ID {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ >> +#define START_PKEY {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ >> + >> +#define START_SL {in_single_number = TRUE;} /* single number */ >> +#define START_VLARB_HIGH_LIMIT {in_single_number = TRUE;} /* single number */ >> +#define START_MTU_LIMIT {in_single_number = TRUE;} /* single number */ >> +#define START_RATE_LIMIT {in_single_number = TRUE;} /* single number */ >> +#define START_PACKET_LIFE {in_single_number = TRUE;} /* single number */ >> + >> + >> + >> +%} >> + >> +PORT_GROUPS_START port\-groups >> +PORT_GROUPS_END end\-port\-groups >> +PORT_GROUP_START port\-group >> +PORT_GROUP_END end\-port\-group >> +NAME name >> +USE use >> +PORT_GUID port\-guid >> +PORT_NAME port\-name >> +PARTITION partition >> +NODE_TYPE node\-type >> +QOS_SETUP_START qos\-setup >> +QOS_SETUP_END end\-qos\-setup >> +VLARB_TABLES_START vlarb\-tables >> +VLARB_TABLES_END end\-vlarb\-tables >> +VLARB_SCOPE_START vlarb\-scope >> +VLARB_SCOPE_END end\-vlarb\-scope >> +GROUP group >> +ACROSS across >> +VLARB_HIGH vlarb\-high >> +VLARB_LOW vlarb\-low >> +VLARB_HIGH_LIMIT vl\-high\-limit >> +SL2VL_TABLES_START sl2vl\-tables >> +SL2VL_TABLES_END end\-sl2vl\-tables >> +SL2VL_SCOPE_START sl2vl\-scope >> +SL2VL_SCOPE_END end\-sl2vl\-scope >> +TO to >> +FROM from >> +ACROSS_TO across\-to >> +ACROSS_FROM across\-from >> +SL2VL_TABLE sl2vl\-table >> +QOS_LEVELS_START qos\-levels >> +QOS_LEVELS_END end\-qos\-levels >> +QOS_LEVEL_START qos\-level >> +QOS_LEVEL_END end\-qos\-level >> +SL sl >> +MTU_LIMIT mtu\-limit >> +RATE_LIMIT rate\-limit >> +PACKET_LIFE packet\-life >> +PATH_BITS path\-bits >> +QOS_MATCH_RULES_START qos\-match\-rules >> +QOS_MATCH_RULES_END end\-qos\-match\-rules >> +QOS_MATCH_RULE_START qos\-match\-rule >> +QOS_MATCH_RULE_END end\-qos\-match\-rule >> +QOS_CLASS qos\-class >> +SOURCE source >> +DESTINATION destination >> +SERVICE_ID service\-id >> +PKEY pkey >> +QOS_LEVEL_NAME qos\-level\-name >> + >> +ROUTER [Rr][Oo][Uu][Tt][Ee][Rr] >> +CA [Cc][Aa] >> +SWITCH [Ss][Ww][Ii][Tt][Cc][Hh] >> +SELF [Ss][Ee][Ll][Ff] >> +ALL [Aa][Ll][Ll] >> + >> +WHITE [ \t]+ >> +NEW_LINE \n >> +COMMENT \#.*\n >> +WHITE_DOTDOT_WHITE [ \t]*:[ \t]* >> + >> +%% >> + >> + >> +{COMMENT} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* swallow comment */ >> +{WHITE}{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* trailing blanks with new line */ >> +{WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; } >> +{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } >> + >> +{PORT_GROUPS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_START; } >> +{PORT_GROUPS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_END; } >> +{PORT_GROUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_START; } >> +{PORT_GROUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_END; } >> + >> +{QOS_SETUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_START; } >> +{QOS_SETUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_END; } >> +{VLARB_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_START; } >> +{VLARB_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_END; } >> +{VLARB_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_START; } >> +{VLARB_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_END; } >> + >> +{SL2VL_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_START; } >> +{SL2VL_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_END; } >> +{SL2VL_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_START; } >> +{SL2VL_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_END; } >> + >> +{QOS_LEVELS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_START; } >> +{QOS_LEVELS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_END; } >> +{QOS_LEVEL_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_START; } >> +{QOS_LEVEL_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_END; } >> + >> +{QOS_MATCH_RULES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_START; } >> +{QOS_MATCH_RULES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_END; } >> +{QOS_MATCH_RULE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_START; } >> +{QOS_MATCH_RULE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_END; } >> + >> +{PORT_GUID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_GUID; return TK_PORT_GUID; } >> +{PORT_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_NAME; return TK_PORT_NAME; } >> +{PARTITION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PARTITION; return TK_PARTITION; } >> +{NODE_TYPE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NODE_TYPE; return TK_NODE_TYPE; } >> +{NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NAME; return TK_NAME; } >> +{USE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_USE; return TK_USE; } >> +{GROUP}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_GROUP; return TK_GROUP; } >> +{VLARB_HIGH}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH; return TK_VLARB_HIGH; } >> +{VLARB_LOW}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_LOW; return TK_VLARB_LOW; } >> +{VLARB_HIGH_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH_LIMIT; return TK_VLARB_HIGH_LIMIT;} >> +{TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_TO; return TK_TO; } >> +{FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_FROM; return TK_FROM; } >> +{ACROSS_TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_TO; return TK_ACROSS_TO; } >> +{ACROSS_FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_FROM; return TK_ACROSS_FROM;} >> +{ACROSS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS; return TK_ACROSS; } >> +{SL2VL_TABLE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL2VL_TABLE; return TK_SL2VL_TABLE;} >> +{SL}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL; return TK_SL; } >> +{MTU_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_MTU_LIMIT; return TK_MTU_LIMIT; } >> +{RATE_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_RATE_LIMIT; return TK_RATE_LIMIT; } >> +{PACKET_LIFE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PACKET_LIFE; return TK_PACKET_LIFE;} >> +{PATH_BITS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PATH_BITS; return TK_PATH_BITS; } >> +{QOS_CLASS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_CLASS; return TK_QOS_CLASS; } >> +{SOURCE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SOURCE; return TK_SOURCE; } >> +{DESTINATION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_DESTINATION; return TK_DESTINATION;} >> +{SERVICE_ID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SERVICE_ID; return TK_SERVICE_ID; } >> +{PKEY}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PKEY; return TK_PKEY; } >> +{QOS_LEVEL_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_LEVEL_NAME; return TK_QOS_LEVEL_NAME;} >> + >> +{ROUTER} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ROUTER; yylval = strdup(yytext); return TK_TEXT; } >> +{CA} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_CA; yylval = strdup(yytext); return TK_TEXT; } >> +{SWITCH} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SWITCH; yylval = strdup(yytext); return TK_TEXT; } >> +{SELF} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SELF; yylval = strdup(yytext); return TK_TEXT; } >> +{ALL} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ALL; yylval = strdup(yytext); return TK_TEXT; } >> + >> +0[xX][0-9a-fA-F]+ { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description || in_list_of_strings || in_single_string) >> + return TK_TEXT; >> + return TK_NUMBER; >> + } >> + >> +[0-9]+ { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description || in_list_of_strings || in_single_string) >> + return TK_TEXT; >> + return TK_NUMBER; >> + } >> + >> + >> +- { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description || in_list_of_strings || in_single_string) >> + return TK_TEXT; >> + return TK_DASH; >> + } >> + >> +: { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description || in_list_of_strings || in_single_string) >> + return TK_TEXT; >> + return TK_DOTDOT; >> + } >> + >> +, { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description) >> + return TK_TEXT; >> + return TK_COMMA; >> + } >> + >> +\* { >> + SAVE_POS; >> + yylval = strdup(yytext); >> + if (in_description || in_list_of_strings || in_single_string) >> + return TK_TEXT; >> + return TK_ASTERISK; >> + } >> + >> +. { SAVE_POS; yylval = strdup(yytext); return TK_TEXT;} >> + >> +%% >> + >> + >> +/********************************************* >> + *********************************************/ >> + >> +static void save_pos() >> +{ >> + int i; >> + for (i = 0; i < yyleng; i++) >> + { >> + if (yytext[i] == '\n') >> + { >> + line_num ++; >> + column_num = 1; >> + } >> + else >> + column_num ++; >> + } >> +} >> + >> +/********************************************* >> + *********************************************/ >> + >> +static void reset_new_line_flags() >> +{ >> + in_description = FALSE; >> + in_list_of_hex_num_ranges = FALSE; >> + in_node_type = FALSE; >> + in_list_of_numbers = FALSE; >> + in_list_of_strings = FALSE; >> + in_list_of_num_pairs = FALSE; >> + in_asterisk_or_list_of_numbers = FALSE; >> + in_list_of_num_ranges = FALSE; >> + in_single_string = FALSE; >> + in_single_number = FALSE; >> +} >> diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y >> new file mode 100755 >> index 0000000..946a645 >> --- /dev/null >> +++ b/opensm/opensm/osm_qos_parser.y >> @@ -0,0 +1,2146 @@ >> +%{ >> +/* >> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. >> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. >> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. >> + * >> + * This software is available to you under a choice of one of two >> + * licenses. You may choose to be licensed under the terms of the GNU >> + * General Public License (GPL) Version 2, available from the file >> + * COPYING in the main directory of this source tree, or the >> + * OpenIB.org BSD license below: >> + * >> + * Redistribution and use in source and binary forms, with or >> + * without modification, are permitted provided that the following >> + * conditions are met: >> + * >> + * - Redistributions of source code must retain the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer. >> + * >> + * - Redistributions in binary form must reproduce the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer in the documentation and/or other materials >> + * provided with the distribution. >> + * >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE >> + * SOFTWARE. >> + * >> + */ >> + >> +/* >> + * Abstract: >> + * Grammar of OSM QoS parser. >> + * >> + * Environment: >> + * Linux User Mode >> + * >> + * Author: >> + * Yevgeny Kliteynik, Mellanox >> + */ >> + >> +#include >> +#include >> + >> +#define OSM_QOS_POLICY_MAX_LINE_LEN 1024*10 >> +#define OSM_QOS_POLICY_SL2VL_TABLE_LEN IB_MAX_NUM_VLS >> +#define OSM_QOS_POLICY_MAX_VL_NUM IB_MAX_NUM_VLS >> + >> +typedef struct tmp_parser_struct_t_ { >> + char str[OSM_QOS_POLICY_MAX_LINE_LEN]; >> + uint64_t num_pair[2]; >> + cl_list_t str_list; >> + cl_list_t num_list; >> + cl_list_t num_pair_list; >> +} tmp_parser_struct_t; >> + >> +static void __parser_tmp_struct_init(); >> +static void __parser_tmp_struct_reset(); >> +static void __parser_tmp_struct_destroy(); >> + >> +static char * __parser_strip_white(char * str); >> + >> +static void __parser_str2uint64(uint64_t * p_val, char * str); >> + >> +static void __parser_port_group_start(); >> +static int __parser_port_group_end(); >> + >> +static void __parser_sl2vl_scope_start(); >> +static int __parser_sl2vl_scope_end(); >> + >> +static void __parser_vlarb_scope_start(); >> +static int __parser_vlarb_scope_end(); >> + >> +static void __parser_qos_level_start(); >> +static int __parser_qos_level_end(); >> + >> +static void __parser_match_rule_start(); >> +static int __parser_match_rule_end(); >> + >> +static void __rangelist2rangearr( >> + cl_list_t * p_list, >> + uint64_t ** * p_arr, >> + unsigned * p_arr_len); >> + >> +static void __merge_rangearr( >> + uint64_t ** range_arr_1, >> + unsigned range_len_1, >> + uint64_t ** range_arr_2, >> + unsigned range_len_2, >> + uint64_t ** * p_arr, >> + unsigned * p_arr_len ); >> + >> +extern char * yytext; >> +extern void yyerror (char *s); >> +extern int yylex (void); >> +extern FILE * yyin; >> + >> +#define RESET_BUFFER __parser_tmp_struct_reset() >> + >> +tmp_parser_struct_t tmp_parser_struct; >> + >> +int column_num; >> +int line_num; >> + >> +extern osm_qos_policy_t * p_qos_policy; >> +osm_qos_port_group_t * p_current_port_group = NULL; >> +osm_qos_sl2vl_scope_t * p_current_sl2vl_scope = NULL; >> +osm_qos_vlarb_scope_t * p_current_vlarb_scope = NULL; >> +osm_qos_level_t * p_current_qos_level = NULL; >> +osm_qos_match_rule_t * p_current_qos_match_rule = NULL; >> +extern osm_log_t * p_qos_parser_osm_log; >> + >> +/***************************************************/ >> + >> +%} >> + >> +%token TK_NUMBER >> +%token TK_DASH >> +%token TK_DOTDOT >> +%token TK_COMMA >> +%token TK_ASTERISK >> +%token TK_TEXT >> + >> +%token TK_PORT_GROUPS_START >> +%token TK_PORT_GROUPS_END >> +%token TK_PORT_GROUP_START >> +%token TK_PORT_GROUP_END >> + >> +%token TK_QOS_SETUP_START >> +%token TK_QOS_SETUP_END >> +%token TK_VLARB_TABLES_START >> +%token TK_VLARB_TABLES_END >> +%token TK_VLARB_SCOPE_START >> +%token TK_VLARB_SCOPE_END >> + >> +%token TK_SL2VL_TABLES_START >> +%token TK_SL2VL_TABLES_END >> +%token TK_SL2VL_SCOPE_START >> +%token TK_SL2VL_SCOPE_END >> + >> +%token TK_QOS_LEVELS_START >> +%token TK_QOS_LEVELS_END >> +%token TK_QOS_LEVEL_START >> +%token TK_QOS_LEVEL_END >> + >> +%token TK_QOS_MATCH_RULES_START >> +%token TK_QOS_MATCH_RULES_END >> +%token TK_QOS_MATCH_RULE_START >> +%token TK_QOS_MATCH_RULE_END >> + >> +%token TK_NAME >> +%token TK_USE >> +%token TK_PORT_GUID >> +%token TK_PORT_NAME >> +%token TK_PARTITION >> +%token TK_NODE_TYPE >> +%token TK_GROUP >> +%token TK_ACROSS >> +%token TK_VLARB_HIGH >> +%token TK_VLARB_LOW >> +%token TK_VLARB_HIGH_LIMIT >> +%token TK_TO >> +%token TK_FROM >> +%token TK_ACROSS_TO >> +%token TK_ACROSS_FROM >> +%token TK_SL2VL_TABLE >> +%token TK_SL >> +%token TK_MTU_LIMIT >> +%token TK_RATE_LIMIT >> +%token TK_PACKET_LIFE >> +%token TK_PATH_BITS >> +%token TK_QOS_CLASS >> +%token TK_SOURCE >> +%token TK_DESTINATION >> +%token TK_SERVICE_ID >> +%token TK_QOS_LEVEL_NAME >> +%token TK_PKEY >> + >> +%token TK_NODE_TYPE_ROUTER >> +%token TK_NODE_TYPE_CA >> +%token TK_NODE_TYPE_SWITCH >> +%token TK_NODE_TYPE_SELF >> +%token TK_NODE_TYPE_ALL >> + >> + >> +%start head >> + >> +%% >> + >> +head: qos_policy_entries >> + ; >> + >> +qos_policy_entries: /* empty */ >> + | qos_policy_entries qos_policy_entry >> + ; >> + >> +qos_policy_entry: port_groups_section >> + | qos_setup_section >> + | qos_levels_section >> + | qos_match_rules_section >> + ; >> + >> + /* >> + * Parsing port groups: >> + * ------------------- >> + * port-groups >> + * port-group >> + * name: Storage >> + * use: our SRP storage targets >> + * port-guid: 0x1000000000000001,0x1000000000000002 >> + * ... >> + * port-name: vs1/HCA-1/P1 >> + * ... >> + * partition: Part1 >> + * ... >> + * node-type: ROUTER,CA,SWITCH,SELF,ALL >> + * ... >> + * end-port-group >> + * port-group >> + * ... >> + * end-port-group >> + * end-port-groups >> + */ >> + >> + >> +port_groups_section: TK_PORT_GROUPS_START port_groups TK_PORT_GROUPS_END >> + ; >> + >> +port_groups: port_group >> + | port_groups port_group >> + ; >> + >> +port_group: port_group_start port_group_entries port_group_end >> + ; >> + >> +port_group_start: TK_PORT_GROUP_START { >> + __parser_port_group_start(); >> + } >> + ; >> + >> +port_group_end: TK_PORT_GROUP_END { >> + if ( __parser_port_group_end() ) >> + return 1; >> + } >> + ; >> + >> +port_group_entries: /* empty */ >> + | port_group_entries port_group_entry >> + ; >> + >> +port_group_entry: port_group_name >> + | port_group_use >> + | port_group_port_guid >> + | port_group_port_name >> + | port_group_partition >> + | port_group_node_type >> + ; >> + >> + >> + /* >> + * Parsing qos setup: >> + * ----------------- >> + * qos-setup >> + * vlarb-tables >> + * vlarb-scope >> + * ... >> + * end-vlarb-scope >> + * vlarb-scope >> + * ... >> + * end-vlarb-scope >> + * end-vlarb-tables >> + * sl2vl-tables >> + * sl2vl-scope >> + * ... >> + * end-sl2vl-scope >> + * sl2vl-scope >> + * ... >> + * end-sl2vl-scope >> + * end-sl2vl-tables >> + * end-qos-setup >> + */ >> + >> +qos_setup_section: TK_QOS_SETUP_START qos_setup_items TK_QOS_SETUP_END >> + ; >> + >> +qos_setup_items: /* empty */ >> + | qos_setup_items vlarb_tables >> + | qos_setup_items sl2vl_tables >> + ; >> + >> + /* Parsing vlarb-tables */ >> + >> +vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END >> + >> +vlarb_scope_items: /* empty */ >> + | vlarb_scope_items vlarb_scope >> + ; >> + >> +vlarb_scope: vlarb_scope_start vlarb_scope_entries vlarb_scope_end >> + ; >> + >> +vlarb_scope_start: TK_VLARB_SCOPE_START { >> + __parser_vlarb_scope_start(); >> + } >> + ; >> + >> +vlarb_scope_end: TK_VLARB_SCOPE_END { >> + if ( __parser_vlarb_scope_end() ) >> + return 1; >> + } >> + ; >> + >> +vlarb_scope_entries:/* empty */ >> + | vlarb_scope_entries vlarb_scope_entry >> + ; >> + >> + /* >> + * vlarb-scope >> + * group: Storage >> + * ... >> + * across: Storage >> + * ... >> + * vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 >> + * vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 >> + * vl-high-limit: 10 >> + * end-vlarb-scope >> + */ >> + >> +vlarb_scope_entry: vlarb_scope_group >> + | vlarb_scope_across >> + | vlarb_scope_vlarb_high >> + | vlarb_scope_vlarb_low >> + | vlarb_scope_vlarb_high_limit >> + ; >> + >> + /* Parsing sl2vl-tables */ >> + >> +sl2vl_tables: TK_SL2VL_TABLES_START sl2vl_scope_items TK_SL2VL_TABLES_END >> + ; >> + >> +sl2vl_scope_items: /* empty */ >> + | sl2vl_scope_items sl2vl_scope >> + ; >> + >> +sl2vl_scope: sl2vl_scope_start sl2vl_scope_entries sl2vl_scope_end >> + ; >> + >> +sl2vl_scope_start: TK_SL2VL_SCOPE_START { >> + __parser_sl2vl_scope_start(); >> + } >> + ; >> + >> +sl2vl_scope_end: TK_SL2VL_SCOPE_END { >> + if ( __parser_sl2vl_scope_end() ) >> + return 1; >> + } >> + ; >> + >> +sl2vl_scope_entries:/* empty */ >> + | sl2vl_scope_entries sl2vl_scope_entry >> + ; >> + >> + /* >> + * sl2vl-scope >> + * group: Part1 >> + * ... >> + * from: * >> + * ... >> + * to: * >> + * ... >> + * across-to: Storage2 >> + * ... >> + * across-from: Storage1 >> + * ... >> + * sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 >> + * end-sl2vl-scope >> + */ >> + >> +sl2vl_scope_entry: sl2vl_scope_group >> + | sl2vl_scope_across >> + | sl2vl_scope_across_from >> + | sl2vl_scope_across_to >> + | sl2vl_scope_from >> + | sl2vl_scope_to >> + | sl2vl_scope_sl2vl_table >> + ; >> + >> + /* >> + * Parsing qos-levels: >> + * ------------------ >> + * qos-levels >> + * qos-level >> + * name: qos_level_1 >> + * use: for the lowest priority communication >> + * sl: 15 >> + * mtu-limit: 1 >> + * rate-limit: 1 >> + * packet-life: 12 >> + * path-bits: 2,4,8-32 >> + * pkey: 0x00FF-0x0FFF >> + * end-qos-level >> + * ... >> + * qos-level >> + * end-qos-level >> + * end-qos-levels >> + */ >> + >> + >> +qos_levels_section: TK_QOS_LEVELS_START qos_levels TK_QOS_LEVELS_END >> + ; >> + >> +qos_levels: /* empty */ >> + | qos_levels qos_level >> + ; >> + >> +qos_level: qos_level_start qos_level_entries qos_level_end >> + ; >> + >> +qos_level_start: TK_QOS_LEVEL_START { >> + __parser_qos_level_start(); >> + } >> + ; >> + >> +qos_level_end: TK_QOS_LEVEL_END { >> + if ( __parser_qos_level_end() ) >> + return 1; >> + } >> + ; >> + >> +qos_level_entries: /* empty */ >> + | qos_level_entries qos_level_entry >> + ; >> + >> +qos_level_entry: qos_level_name >> + | qos_level_use >> + | qos_level_sl >> + | qos_level_mtu_limit >> + | qos_level_rate_limit >> + | qos_level_packet_life >> + | qos_level_path_bits >> + | qos_level_pkey >> + ; >> + >> + /* >> + * Parsing qos-match-rules: >> + * ----------------------- >> + * qos-match-rules >> + * qos-match-rule >> + * use: low latency by class 7-9 or 11 and bla bla >> + * qos-class: 7-9,11 >> + * qos-level-name: default >> + * source: Storage >> + * destination: Storage >> + * service-id: 22,4719-5000 >> + * pkey: 0x00FF-0x0FFF >> + * end-qos-match-rule >> + * qos-match-rule >> + * ... >> + * end-qos-match-rule >> + * end-qos-match-rules >> + */ >> + >> +qos_match_rules_section: TK_QOS_MATCH_RULES_START qos_match_rules TK_QOS_MATCH_RULES_END >> + ; >> + >> +qos_match_rules: /* empty */ >> + | qos_match_rules qos_match_rule >> + ; >> + >> +qos_match_rule: qos_match_rule_start qos_match_rule_entries qos_match_rule_end >> + ; >> + >> +qos_match_rule_start: TK_QOS_MATCH_RULE_START { >> + __parser_match_rule_start(); >> + } >> + ; >> + >> +qos_match_rule_end: TK_QOS_MATCH_RULE_END { >> + if ( __parser_match_rule_end() ) >> + return 1; >> + } >> + ; >> + >> +qos_match_rule_entries: /* empty */ >> + | qos_match_rule_entries qos_match_rule_entry >> + ; >> + >> +qos_match_rule_entry: qos_match_rule_use >> + | qos_match_rule_qos_class >> + | qos_match_rule_qos_level_name >> + | qos_match_rule_source >> + | qos_match_rule_destination >> + | qos_match_rule_service_id >> + | qos_match_rule_pkey >> + ; >> + >> + /* >> + * port_group_entry values: >> + * port_group_name >> + * port_group_use >> + * port_group_port_guid >> + * port_group_port_name >> + * port_group_partition >> + * port_group_node_type >> + */ >> + >> +port_group_name: port_group_name_start single_string { >> + /* 'name' of 'port-group' - one instance */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_port_group->name) >> + { >> + yyerror("port-group has multiple 'name' tags"); >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_port_group->name = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +port_group_name_start: TK_NAME { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_use: port_group_use_start single_string { >> + /* 'use' of 'port-group' - one instance */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_port_group->use) >> + { >> + yyerror("port-group has multiple 'use' tags"); >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_port_group->use = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +port_group_use_start: TK_USE { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_port_name: port_group_port_name_start string_list { >> + /* 'port-name' in 'port-group' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + >> + /* >> + * TODO: parse port name strings >> + */ >> + >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_port_group->port_name_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +port_group_port_name_start: TK_PORT_NAME { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_port_guid: port_group_port_guid_start list_of_ranges { >> + /* 'port-guid' in 'port-group' - any num of instances */ >> + /* list of guid ranges */ >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_port_group->guid_range_len ) >> + { >> + p_current_port_group->guid_range_arr = range_arr; >> + p_current_port_group->guid_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_port_group->guid_range_arr, >> + p_current_port_group->guid_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_port_group->guid_range_arr = new_range_arr; >> + p_current_port_group->guid_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +port_group_port_guid_start: TK_PORT_GUID { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_partition: port_group_partition_start string_list { >> + /* 'partition' in 'port-group' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_port_group->partition_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +port_group_partition_start: TK_PARTITION { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_node_type: port_group_node_type_start port_group_node_type_list { >> + /* 'node-type' in 'port-group' - any num of instances */ >> + } >> + ; >> + >> +port_group_node_type_start: TK_NODE_TYPE { >> + RESET_BUFFER; >> + } >> + ; >> + >> +port_group_node_type_list: node_type_item >> + | port_group_node_type_list TK_COMMA node_type_item >> + ; >> + >> +node_type_item: node_type_ca >> + | node_type_switch >> + | node_type_router >> + | node_type_all >> + | node_type_self >> + ; >> + >> +node_type_ca: TK_NODE_TYPE_CA { >> + p_current_port_group->node_type_ca = TRUE;; >> + } >> + ; >> + >> +node_type_switch: TK_NODE_TYPE_SWITCH { >> + p_current_port_group->node_type_switch = TRUE; >> + } >> + ; >> + >> +node_type_router: TK_NODE_TYPE_ROUTER { >> + p_current_port_group->node_type_router = TRUE; >> + } >> + ; >> + >> +node_type_all: TK_NODE_TYPE_ALL { >> + p_current_port_group->node_type_ca = TRUE; >> + p_current_port_group->node_type_switch = TRUE; >> + p_current_port_group->node_type_router = TRUE; >> + } >> + ; >> + >> +node_type_self: TK_NODE_TYPE_SELF { >> + p_current_port_group->node_type_self = TRUE; >> + } >> + ; >> + >> + /* >> + * vlarb_scope_entry values: >> + * vlarb_scope_group >> + * vlarb_scope_across >> + * vlarb_scope_vlarb_high >> + * vlarb_scope_vlarb_low >> + * vlarb_scope_vlarb_high_limit >> + */ >> + >> + >> + >> +vlarb_scope_group: vlarb_scope_group_start string_list { >> + /* 'group' in 'vlarb-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_vlarb_scope->group_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +vlarb_scope_group_start: TK_GROUP { >> + RESET_BUFFER; >> + } >> + ; >> + >> +vlarb_scope_across: vlarb_scope_across_start string_list { >> + /* 'across' in 'vlarb-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_vlarb_scope->across_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +vlarb_scope_across_start: TK_ACROSS { >> + RESET_BUFFER; >> + } >> + ; >> + >> +vlarb_scope_vlarb_high_limit: vlarb_scope_vlarb_high_limit_start single_number { >> + /* 'vl-high-limit' in 'vlarb-scope' - one instance of one number */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * p_tmp_num; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + p_tmp_num = (uint64_t*)cl_list_obj(list_iterator); >> + if (p_tmp_num) >> + { >> + p_current_vlarb_scope->vl_high_limit = (uint32_t)(*p_tmp_num); >> + p_current_vlarb_scope->vl_high_limit_set = TRUE; >> + free(p_tmp_num); >> + } >> + >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +vlarb_scope_vlarb_high_limit_start: TK_VLARB_HIGH_LIMIT { >> + RESET_BUFFER; >> + } >> + ; >> + >> +vlarb_scope_vlarb_high: vlarb_scope_vlarb_high_start num_list_with_dotdot { >> + /* 'vlarb-high' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * num_pair; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) >> + { >> + num_pair = (uint64_t*)cl_list_obj(list_iterator); >> + if (num_pair) >> + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_high_list,num_pair); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + } >> + ; >> + >> +vlarb_scope_vlarb_high_start: TK_VLARB_HIGH { >> + RESET_BUFFER; >> + } >> + ; >> + >> +vlarb_scope_vlarb_low: vlarb_scope_vlarb_low_start num_list_with_dotdot { >> + /* 'vlarb-low' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * num_pair; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) >> + { >> + num_pair = (uint64_t*)cl_list_obj(list_iterator); >> + if (num_pair) >> + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_low_list,num_pair); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + } >> + ; >> + >> +vlarb_scope_vlarb_low_start: TK_VLARB_LOW { >> + RESET_BUFFER; >> + } >> + ; >> + >> + /* >> + * sl2vl_scope_entry values: >> + * sl2vl_scope_group >> + * sl2vl_scope_across >> + * sl2vl_scope_across_from >> + * sl2vl_scope_across_to >> + * sl2vl_scope_from >> + * sl2vl_scope_to >> + * sl2vl_scope_sl2vl_table >> + */ >> + >> +sl2vl_scope_group: sl2vl_scope_group_start string_list { >> + /* 'group' in 'sl2vl-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_sl2vl_scope->group_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +sl2vl_scope_group_start: TK_GROUP { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_across: sl2vl_scope_across_start string_list { >> + /* 'across' in 'sl2vl-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) { >> + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); >> + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,strdup(tmp_str)); >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +sl2vl_scope_across_start: TK_ACROSS { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_across_from: sl2vl_scope_across_from_start string_list { >> + /* 'across-from' in 'sl2vl-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +sl2vl_scope_across_from_start: TK_ACROSS_FROM { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_across_to: sl2vl_scope_across_to_start string_list { >> + /* 'across-to' in 'sl2vl-scope' - any num of instances */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) { >> + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,tmp_str); >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +sl2vl_scope_across_to_start: TK_ACROSS_TO { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_from: sl2vl_scope_from_start sl2vl_scope_from_list_or_asterisk { >> + /* 'from' in 'sl2vl-scope' - any num of instances */ >> + } >> + ; >> + >> +sl2vl_scope_from_start: TK_FROM { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_to: sl2vl_scope_to_start sl2vl_scope_to_list_or_asterisk { >> + /* 'to' in 'sl2vl-scope' - any num of instances */ >> + } >> + ; >> + >> +sl2vl_scope_to_start: TK_TO { >> + RESET_BUFFER; >> + } >> + ; >> + >> +sl2vl_scope_from_list_or_asterisk: sl2vl_scope_from_asterisk >> + | sl2vl_scope_from_list_of_ranges >> + ; >> + >> +sl2vl_scope_from_asterisk: TK_ASTERISK { >> + int i; >> + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) >> + p_current_sl2vl_scope->from[i] = TRUE; >> + } >> + ; >> + >> +sl2vl_scope_to_list_or_asterisk: sl2vl_scope_to_asterisk >> + | sl2vl_scope_to_list_of_ranges >> + ; >> + >> +sl2vl_scope_to_asterisk: TK_ASTERISK { >> + int i; >> + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) >> + p_current_sl2vl_scope->to[i] = TRUE; >> + } >> + ; >> + >> +sl2vl_scope_from_list_of_ranges: list_of_ranges { >> + int i; >> + cl_list_iterator_t list_iterator; >> + uint64_t * num_pair; >> + uint8_t num1, num2; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) >> + { >> + num_pair = (uint64_t*)cl_list_obj(list_iterator); >> + if (num_pair) >> + { >> + if ( num_pair[0] < 0 || >> + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) >> + { >> + yyerror("port number out of range 'from' list"); >> + free(num_pair); >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + return 1; >> + } >> + num1 = (uint8_t)num_pair[0]; >> + num2 = (uint8_t)num_pair[1]; >> + free(num_pair); >> + for (i = num1; i <= num2; i++) >> + p_current_sl2vl_scope->from[i] = TRUE; >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + } >> + ; >> + >> +sl2vl_scope_to_list_of_ranges: list_of_ranges { >> + int i; >> + cl_list_iterator_t list_iterator; >> + uint64_t * num_pair; >> + uint8_t num1, num2; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) >> + { >> + num_pair = (uint64_t*)cl_list_obj(list_iterator); >> + if (num_pair) >> + { >> + if ( num_pair[0] < 0 || >> + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) >> + { >> + yyerror("port number out of range 'to' list"); >> + free(num_pair); >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + return 1; >> + } >> + num1 = (uint8_t)num_pair[0]; >> + num2 = (uint8_t)num_pair[1]; >> + free(num_pair); >> + for (i = num1; i <= num2; i++) >> + p_current_sl2vl_scope->to[i] = TRUE; >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> + } >> + ; >> + >> + >> +sl2vl_scope_sl2vl_table: sl2vl_scope_sl2vl_table_start num_list { >> + /* 'sl2vl-table' - one instance of exactly >> + OSM_QOS_POLICY_SL2VL_TABLE_LEN numbers */ >> + cl_list_iterator_t list_iterator; >> + uint64_t num; >> + uint64_t * p_num; >> + int i = 0; >> + >> + if (p_current_sl2vl_scope->sl2vl_table_set) >> + { >> + yyerror("sl2vl-scope has more than one sl2vl-table"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + >> + if (cl_list_count(&tmp_parser_struct.num_list) != OSM_QOS_POLICY_SL2VL_TABLE_LEN) >> + { >> + yyerror("wrong number of values in 'sl2vl-table' (should be 16)"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.num_list) ) >> + { >> + p_num = (uint64_t*)cl_list_obj(list_iterator); >> + num = *p_num; >> + free(p_num); >> + if (num >= OSM_QOS_POLICY_MAX_VL_NUM) >> + { >> + yyerror("wrong VL value in 'sl2vl-table' (should be 0 to 15)"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + >> + p_current_sl2vl_scope->sl2vl_table[i++] = (uint8_t)num; >> + list_iterator = cl_list_next(list_iterator); >> + } >> + p_current_sl2vl_scope->sl2vl_table_set = TRUE; >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +sl2vl_scope_sl2vl_table_start: TK_SL2VL_TABLE { >> + RESET_BUFFER; >> + } >> + ; >> + >> + /* >> + * qos_level_entry values: >> + * qos_level_name >> + * qos_level_use >> + * qos_level_sl >> + * qos_level_mtu_limit >> + * qos_level_rate_limit >> + * qos_level_packet_life >> + * qos_level_path_bits >> + * qos_level_pkey >> + */ >> + >> +qos_level_name: qos_level_name_start single_string { >> + /* 'name' of 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_qos_level->name) >> + { >> + yyerror("qos-level has multiple 'name' tags"); >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_qos_level->name = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_level_name_start: TK_NAME { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_use: qos_level_use_start single_string { >> + /* 'use' of 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_qos_level->use) >> + { >> + yyerror("qos-level has multiple 'use' tags"); >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_qos_level->use = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_level_use_start: TK_USE { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_sl: qos_level_sl_start single_number { >> + /* 'sl' in 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * p_num; >> + >> + if (p_current_qos_level->sl_set) >> + { >> + yyerror("'qos-level' has multiple 'sl' tags"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + p_num = (uint64_t*)cl_list_obj(list_iterator); >> + p_current_qos_level->sl = (uint8_t)(*p_num); >> + free(p_num); >> + p_current_qos_level->sl_set = TRUE; >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +qos_level_sl_start: TK_SL { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_mtu_limit: qos_level_mtu_limit_start single_number { >> + /* 'mtu-limit' in 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * p_num; >> + >> + if (p_current_qos_level->mtu_limit_set) >> + { >> + yyerror("'qos-level' has multiple 'mtu-limit' tags"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + p_num = (uint64_t*)cl_list_obj(list_iterator); >> + p_current_qos_level->mtu_limit = (uint8_t)(*p_num); >> + free(p_num); >> + p_current_qos_level->mtu_limit_set = TRUE; >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +qos_level_mtu_limit_start: TK_MTU_LIMIT { >> + /* 'mtu-limit' in 'qos-level' - one instance */ >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_rate_limit: qos_level_rate_limit_start single_number { >> + /* 'rate-limit' in 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * p_num; >> + >> + if (p_current_qos_level->rate_limit_set) >> + { >> + yyerror("'qos-level' has multiple 'rate-limit' tags"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + p_num = (uint64_t*)cl_list_obj(list_iterator); >> + p_current_qos_level->rate_limit = (uint8_t)(*p_num); >> + free(p_num); >> + p_current_qos_level->rate_limit_set = TRUE; >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +qos_level_rate_limit_start: TK_RATE_LIMIT { >> + /* 'rate-limit' in 'qos-level' - one instance */ >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_packet_life: qos_level_packet_life_start single_number { >> + /* 'packet-life' in 'qos-level' - one instance */ >> + cl_list_iterator_t list_iterator; >> + uint64_t * p_num; >> + >> + if (p_current_qos_level->pkt_life_set) >> + { >> + yyerror("'qos-level' has multiple 'packet-life' tags"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + list_iterator = cl_list_head(&tmp_parser_struct.num_list); >> + p_num = (uint64_t*)cl_list_obj(list_iterator); >> + p_current_qos_level->pkt_life = (uint8_t)(*p_num); >> + free(p_num); >> + p_current_qos_level->pkt_life_set= TRUE; >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + } >> + ; >> + >> +qos_level_packet_life_start: TK_PACKET_LIFE { >> + /* 'packet-life' in 'qos-level' - one instance */ >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_path_bits: qos_level_path_bits_start list_of_ranges { >> + /* 'path-bits' in 'qos-level' - any num of instances */ >> + /* list of path bit ranges */ >> + >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_qos_level->path_bits_range_len ) >> + { >> + p_current_qos_level->path_bits_range_arr = range_arr; >> + p_current_qos_level->path_bits_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_qos_level->path_bits_range_arr, >> + p_current_qos_level->path_bits_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_qos_level->path_bits_range_arr = new_range_arr; >> + p_current_qos_level->path_bits_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +qos_level_path_bits_start: TK_PATH_BITS { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_level_pkey: qos_level_pkey_start list_of_ranges { >> + /* 'pkey' in 'qos-level' - num of instances of list of ranges */ >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_qos_level->pkey_range_len ) >> + { >> + p_current_qos_level->pkey_range_arr = range_arr; >> + p_current_qos_level->pkey_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_qos_level->pkey_range_arr, >> + p_current_qos_level->pkey_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_qos_level->pkey_range_arr = new_range_arr; >> + p_current_qos_level->pkey_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +qos_level_pkey_start: TK_PKEY { >> + RESET_BUFFER; >> + } >> + ; >> + >> + /* >> + * qos_match_rule_entry values: >> + * qos_match_rule_use >> + * qos_match_rule_qos_class >> + * qos_match_rule_qos_level_name >> + * qos_match_rule_source >> + * qos_match_rule_destination >> + * qos_match_rule_service_id >> + * qos_match_rule_pkey >> + */ >> + >> + >> +qos_match_rule_use: qos_match_rule_use_start single_string { >> + /* 'use' of 'qos-match-rule' - one instance */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_qos_match_rule->use) >> + { >> + yyerror("'qos-match-rule' has multiple 'use' tags"); >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_qos_match_rule->use = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_match_rule_use_start: TK_USE { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_qos_class: qos_match_rule_qos_class_start list_of_ranges { >> + /* 'qos-class' in 'qos-match-rule' - num of instances of list of ranges */ >> + /* list of class ranges (QoS Class is 12-bit value) */ >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_qos_match_rule->qos_class_range_len ) >> + { >> + p_current_qos_match_rule->qos_class_range_arr = range_arr; >> + p_current_qos_match_rule->qos_class_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_qos_match_rule->qos_class_range_arr, >> + p_current_qos_match_rule->qos_class_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_qos_match_rule->qos_class_range_arr = new_range_arr; >> + p_current_qos_match_rule->qos_class_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +qos_match_rule_qos_class_start: TK_QOS_CLASS { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_source: qos_match_rule_source_start string_list { >> + /* 'source' in 'qos-match-rule' - text */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_qos_match_rule->source_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_match_rule_source_start: TK_SOURCE { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_destination: qos_match_rule_destination_start string_list { >> + /* 'destination' in 'qos-match-rule' - text */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + cl_list_insert_tail(&p_current_qos_match_rule->destination_list,tmp_str); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_match_rule_destination_start: TK_DESTINATION { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_qos_level_name: qos_match_rule_qos_level_name_start single_string { >> + /* 'qos-level-name' in 'qos-match-rule' - single string */ >> + cl_list_iterator_t list_iterator; >> + char * tmp_str; >> + >> + if (p_current_qos_match_rule->qos_level_name) >> + { >> + yyerror("qos-match-rule has multiple 'qos-level-name' tags"); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + return 1; >> + } >> + >> + list_iterator = cl_list_head(&tmp_parser_struct.str_list); >> + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) >> + { >> + tmp_str = (char*)cl_list_obj(list_iterator); >> + if (tmp_str) >> + p_current_qos_match_rule->qos_level_name = tmp_str; >> + } >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + } >> + ; >> + >> +qos_match_rule_qos_level_name_start: TK_QOS_LEVEL_NAME { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_service_id: qos_match_rule_service_id_start list_of_ranges { >> + /* 'service-id' in 'qos-match-rule' - num of instances of list of ranges */ >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_qos_match_rule->service_id_range_len ) >> + { >> + p_current_qos_match_rule->service_id_range_arr = range_arr; >> + p_current_qos_match_rule->service_id_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_qos_match_rule->service_id_range_arr, >> + p_current_qos_match_rule->service_id_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_qos_match_rule->service_id_range_arr = new_range_arr; >> + p_current_qos_match_rule->service_id_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +qos_match_rule_service_id_start: TK_SERVICE_ID { >> + RESET_BUFFER; >> + } >> + ; >> + >> +qos_match_rule_pkey: qos_match_rule_pkey_start list_of_ranges { >> + /* 'pkey' in 'qos-match-rule' - num of instances of list of ranges */ >> + if (cl_list_count(&tmp_parser_struct.num_pair_list)) >> + { >> + uint64_t ** range_arr; >> + unsigned range_len; >> + >> + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, >> + &range_arr, >> + &range_len ); >> + >> + if ( !p_current_qos_match_rule->pkey_range_len ) >> + { >> + p_current_qos_match_rule->pkey_range_arr = range_arr; >> + p_current_qos_match_rule->pkey_range_len = range_len; >> + } >> + else >> + { >> + uint64_t ** new_range_arr; >> + unsigned new_range_len; >> + __merge_rangearr( p_current_qos_match_rule->pkey_range_arr, >> + p_current_qos_match_rule->pkey_range_len, >> + range_arr, >> + range_len, >> + &new_range_arr, >> + &new_range_len ); >> + p_current_qos_match_rule->pkey_range_arr = new_range_arr; >> + p_current_qos_match_rule->pkey_range_len = new_range_len; >> + } >> + } >> + } >> + ; >> + >> +qos_match_rule_pkey_start: TK_PKEY { >> + RESET_BUFFER; >> + } >> + ; >> + >> + >> + /* >> + * Common part >> + */ >> + >> + >> +single_string: single_string_elems { >> + cl_list_insert_tail(&tmp_parser_struct.str_list, >> + strdup(__parser_strip_white(tmp_parser_struct.str))); >> + tmp_parser_struct.str[0] = '\0'; >> + } >> + ; >> + >> +single_string_elems: single_string_element >> + | single_string_elems single_string_element >> + ; >> + >> +single_string_element: TK_TEXT { >> + strcat(tmp_parser_struct.str,$1); >> + free($1); >> + } >> + ; >> + >> + >> +string_list: single_string >> + | string_list TK_COMMA single_string >> + ; >> + >> + >> + >> +single_number: number >> + ; >> + >> +num_list: number >> + | num_list TK_COMMA number >> + ; >> + >> +number: TK_NUMBER { >> + uint64_t * p_num = (uint64_t*)malloc(sizeof(uint64_t)); >> + __parser_str2uint64(p_num,$1); >> + free($1); >> + cl_list_insert_tail(&tmp_parser_struct.num_list, p_num); >> + } >> + ; >> + >> +num_list_with_dotdot: number_from_pair_1 TK_DOTDOT number_from_pair_2 { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + | num_list_with_dotdot TK_COMMA number_from_pair_1 TK_DOTDOT number_from_pair_2 { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + ; >> + >> +number_from_pair_1: TK_NUMBER { >> + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); >> + free($1); >> + } >> + ; >> + >> +number_from_pair_2: TK_NUMBER { >> + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); >> + free($1); >> + } >> + ; >> + >> +list_of_ranges: num_list_with_dash >> + ; >> + >> +num_list_with_dash: single_number_from_range { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + | number_from_range_1 TK_DASH number_from_range_2 { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + } >> + else { >> + num_pair[1] = tmp_parser_struct.num_pair[0]; >> + num_pair[0] = tmp_parser_struct.num_pair[1]; >> + } >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + | num_list_with_dash TK_COMMA number_from_range_1 TK_DASH number_from_range_2 { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + } >> + else { >> + num_pair[1] = tmp_parser_struct.num_pair[0]; >> + num_pair[0] = tmp_parser_struct.num_pair[1]; >> + } >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + | num_list_with_dash TK_COMMA single_number_from_range { >> + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); >> + num_pair[0] = tmp_parser_struct.num_pair[0]; >> + num_pair[1] = tmp_parser_struct.num_pair[1]; >> + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); >> + } >> + ; >> + >> +single_number_from_range: TK_NUMBER { >> + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); >> + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); >> + free($1); >> + } >> + ; >> + >> +number_from_range_1: TK_NUMBER { >> + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); >> + free($1); >> + } >> + ; >> + >> +number_from_range_2: TK_NUMBER { >> + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); >> + free($1); >> + } >> + ; >> + >> +%% >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +int osm_qos_parse_policy_file( >> + IN osm_log_t * p_log, >> + IN const char * policy_file) >> +{ >> + int res = 0; >> + p_qos_parser_osm_log = p_log; >> + >> + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_parse_policy_file); >> + >> + p_qos_policy = NULL; >> + yyin = fopen (policy_file, "r"); >> + if (!yyin) >> + { >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "osm_qos_parse: ERR AC01: " >> + "Failed opening QoS policy file (%s)\n", >> + policy_file); >> + res = 1; >> + goto Exit; >> + } >> + column_num = 1; >> + line_num = 1; >> + osm_qos_policy_create(); >> + __parser_tmp_struct_init(); >> + >> + res = yyparse(); >> + >> + __parser_tmp_struct_destroy(); >> + >> + if (res != 0) >> + { >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "osm_qos_parse: ERR AC02: " >> + "Failed parsing QoS policy file (%s)\n", >> + policy_file); >> + p_qos_policy = NULL; >> + res = 1; >> + goto Exit; >> + } >> + >> + if (osm_qos_policy_validate()) >> + { >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "osm_qos_parse: ERR AC03: " >> + "Error(s) in QoS policy file (%s)\n", >> + policy_file); >> + res = 1; >> + goto Exit; >> + } >> + >> + Exit: >> + if (yyin) >> + fclose(yyin); >> + OSM_LOG_EXIT(p_qos_parser_osm_log); >> + return res; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +int yywrap() >> +{ >> + return(1); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void yyerror (char *s) >> +{ >> + OSM_LOG_ENTER(p_qos_parser_osm_log, yyerror); >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "yyerror: ERR AC03: " >> + "Syntax error (line %d:%d): %s. " >> + "Last text read: \"%s\"\n", >> + line_num, column_num, s, __parser_strip_white(yytext)); >> + OSM_LOG_EXIT(p_qos_parser_osm_log); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static char * __parser_strip_white(char * str) >> +{ >> + int i; >> + for (i = (strlen(str)-1); i >= 0; i--) >> + { >> + if (isspace(str[i])) >> + str[i] = '\0'; >> + else >> + break; >> + } >> + for (i = 0; i < strlen(str); i++) >> + { >> + if (!isspace(str[i])) >> + break; >> + } >> + return &(str[i]); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_str2uint64(uint64_t * p_val, char * str) >> +{ >> +#if __WORDSIZE == 64 >> + *p_val = strtoul(str, NULL, 0); >> +#else >> + *p_val = strtoull(str, NULL, 0); >> +#endif >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_port_group_start() >> +{ >> + p_current_port_group = osm_qos_policy_port_group_create(); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int __parser_port_group_end() >> +{ >> + if(!p_current_port_group->name) >> + { >> + yyerror("port-group validation failed - no port group name specified"); >> + return -1; >> + } >> + >> + cl_list_insert_tail(&p_qos_policy->port_groups, >> + p_current_port_group); >> + p_current_port_group = NULL; >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_vlarb_scope_start() >> +{ >> + p_current_vlarb_scope = osm_qos_policy_vlarb_scope_create(); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int __parser_vlarb_scope_end() >> +{ >> + if ( !cl_list_count(&p_current_vlarb_scope->group_list) && >> + !cl_list_count(&p_current_vlarb_scope->across_list) ) >> + { >> + yyerror("vlarb-scope validation failed - no port groups specified by 'group' or by 'across'"); >> + return -1; >> + } >> + >> + cl_list_insert_tail(&p_qos_policy->vlarb_tables, >> + p_current_vlarb_scope); >> + p_current_vlarb_scope = NULL; >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_sl2vl_scope_start() >> +{ >> + p_current_sl2vl_scope = osm_qos_policy_sl2vl_scope_create(); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int __parser_sl2vl_scope_end() >> +{ >> + if (!p_current_sl2vl_scope->sl2vl_table_set) >> + { >> + yyerror("sl2vl-scope validation failed - no sl2vl table specified"); >> + return -1; >> + } >> + if ( !cl_list_count(&p_current_sl2vl_scope->group_list) && >> + !cl_list_count(&p_current_sl2vl_scope->across_to_list) && >> + !cl_list_count(&p_current_sl2vl_scope->across_from_list) ) >> + { >> + yyerror("sl2vl-scope validation failed - no port groups specified by 'group', 'across-to' or 'across-from'"); >> + return -1; >> + } >> + >> + cl_list_insert_tail(&p_qos_policy->sl2vl_tables, >> + p_current_sl2vl_scope); >> + p_current_sl2vl_scope = NULL; >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_qos_level_start() >> +{ >> + p_current_qos_level = osm_qos_policy_qos_level_create(); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int __parser_qos_level_end() >> +{ >> + if (!p_current_qos_level->sl_set) >> + { >> + yyerror("qos-level validation failed - no 'sl' specified"); >> + return -1; >> + } >> + if (!p_current_qos_level->name) >> + { >> + yyerror("qos-level validation failed - no 'name' specified"); >> + return -1; >> + } >> + >> + cl_list_insert_tail(&p_qos_policy->qos_levels, >> + p_current_qos_level); >> + p_current_qos_level = NULL; >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_match_rule_start() >> +{ >> + p_current_qos_match_rule = osm_qos_policy_match_rule_create(); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int __parser_match_rule_end() >> +{ >> + if (!p_current_qos_match_rule->qos_level_name) >> + { >> + yyerror("match-rule validation failed - no 'qos-level-name' specified"); >> + return -1; >> + } >> + >> + cl_list_insert_tail(&p_qos_policy->qos_match_rules, >> + p_current_qos_match_rule); >> + p_current_qos_match_rule = NULL; >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_tmp_struct_init() >> +{ >> + tmp_parser_struct.str[0] = '\0'; >> + cl_list_construct(&tmp_parser_struct.str_list); >> + cl_list_init(&tmp_parser_struct.str_list, 10); >> + cl_list_construct(&tmp_parser_struct.num_list); >> + cl_list_init(&tmp_parser_struct.num_list, 10); >> + cl_list_construct(&tmp_parser_struct.num_pair_list); >> + cl_list_init(&tmp_parser_struct.num_pair_list, 10); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +/* >> + * Do NOT free objects from the temp struct. >> + * Either they are inserted into the parse tree data >> + * structure, or they are already freed when copying >> + * their values to the parse tree data structure. >> + */ >> +static void __parser_tmp_struct_reset() >> +{ >> + tmp_parser_struct.str[0] = '\0'; >> + cl_list_remove_all(&tmp_parser_struct.str_list); >> + cl_list_remove_all(&tmp_parser_struct.num_list); >> + cl_list_remove_all(&tmp_parser_struct.num_pair_list); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __parser_tmp_struct_destroy() >> +{ >> + __parser_tmp_struct_reset(); >> + cl_list_destroy(&tmp_parser_struct.str_list); >> + cl_list_destroy(&tmp_parser_struct.num_list); >> + cl_list_destroy(&tmp_parser_struct.num_pair_list); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static int OSM_CDECL >> +__cmp_num_range( >> + const void * p1, >> + const void * p2) >> +{ >> + uint64_t * pair1 = *((uint64_t **)p1); >> + uint64_t * pair2 = *((uint64_t **)p2); >> + >> + if (pair1[0] < pair2[0]) >> + return -1; >> + if (pair1[0] > pair2[0]) >> + return 1; >> + >> + if (pair1[1] < pair2[1]) >> + return -1; >> + if (pair1[1] > pair2[1]) >> + return 1; >> + >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __sort_reduce_rangearr( >> + uint64_t ** arr, >> + unsigned arr_len, >> + uint64_t ** * p_res_arr, >> + unsigned * p_res_arr_len ) >> +{ >> + unsigned i = 0; >> + unsigned j = 0; >> + unsigned last_valid_ind = 0; >> + unsigned valid_cnt = 0; >> + uint64_t ** res_arr; >> + boolean_t * is_valir_arr; >> + >> + *p_res_arr = NULL; >> + *p_res_arr_len = 0; >> + >> + qsort(arr, arr_len, sizeof(uint64_t*), __cmp_num_range); >> + >> + is_valir_arr = (boolean_t *)malloc(arr_len * sizeof(boolean_t)); >> + is_valir_arr[last_valid_ind] = TRUE; >> + valid_cnt++; >> + for (i = 1; i < arr_len; i++) >> + { >> + if (arr[i][0] <= arr[last_valid_ind][1]) >> + { >> + if (arr[i][1] > arr[last_valid_ind][1]) >> + arr[last_valid_ind][1] = arr[i][1]; >> + free(arr[i]); >> + arr[i] = NULL; >> + is_valir_arr[i] = FALSE; >> + } >> + else if ((arr[i][0] - 1) == arr[last_valid_ind][1]) >> + { >> + arr[last_valid_ind][1] = arr[i][1]; >> + free(arr[i]); >> + arr[i] = NULL; >> + is_valir_arr[i] = FALSE; >> + } >> + else >> + { >> + is_valir_arr[i] = TRUE; >> + last_valid_ind = i; >> + valid_cnt++; >> + } >> + } >> + >> + res_arr = (uint64_t **)malloc(valid_cnt * sizeof(uint64_t *)); >> + for (i = 0; i < arr_len; i++) >> + { >> + if (is_valir_arr[i]) >> + res_arr[j++] = arr[i]; >> + } >> + free(arr); >> + >> + *p_res_arr = res_arr; >> + *p_res_arr_len = valid_cnt; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __rangelist2rangearr( >> + cl_list_t * p_list, >> + uint64_t ** * p_arr, >> + unsigned * p_arr_len) >> +{ >> + cl_list_iterator_t list_iterator; >> + unsigned len = cl_list_count(p_list); >> + unsigned i = 0; >> + uint64_t ** tmp_arr; >> + uint64_t ** res_arr = NULL; >> + unsigned res_arr_len = 0; >> + >> + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); >> + >> + list_iterator = cl_list_head(p_list); >> + while( list_iterator != cl_list_end(p_list) ) >> + { >> + tmp_arr[i++] = (uint64_t *)cl_list_obj(list_iterator); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(p_list); >> + >> + __sort_reduce_rangearr( tmp_arr, >> + len, >> + &res_arr, >> + &res_arr_len ); >> + *p_arr = res_arr; >> + *p_arr_len = res_arr_len; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __merge_rangearr( >> + uint64_t ** range_arr_1, >> + unsigned range_len_1, >> + uint64_t ** range_arr_2, >> + unsigned range_len_2, >> + uint64_t ** * p_arr, >> + unsigned * p_arr_len ) >> +{ >> + unsigned i = 0; >> + unsigned j = 0; >> + unsigned len = range_len_1 + range_len_2; >> + uint64_t ** tmp_arr; >> + uint64_t ** res_arr = NULL; >> + unsigned res_arr_len = 0; >> + >> + *p_arr = NULL; >> + *p_arr_len = 0; >> + >> + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); >> + >> + for (i = 0; i < range_len_1; i++) >> + tmp_arr[j++] = range_arr_1[i]; >> + for (i = 0; i < range_len_2; i++) >> + tmp_arr[j++] = range_arr_2[i]; >> + free(range_arr_1); >> + free(range_arr_2); >> + >> + __sort_reduce_rangearr( tmp_arr, >> + len, >> + &res_arr, >> + &res_arr_len ); >> + *p_arr = res_arr; >> + *p_arr_len = res_arr_len; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> -- >> 1.5.1.4 >> > From kliteyn at dev.mellanox.co.il Thu Aug 23 01:11:10 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 11:11:10 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070823064648.GM1397@sashak.voltaire.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> Message-ID: <46CD411E.1080809@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > Hi Yevgeny, > > Some initial comments below. > > Sasha > > On 14:46 Tue 21 Aug , Yevgeny Kliteynik wrote: >> QoS policy data structures and functions >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/include/opensm/osm_qos_policy.h | 188 +++++++ >> opensm/opensm/osm_qos_policy.c | 901 ++++++++++++++++++++++++++++++++ >> 2 files changed, 1089 insertions(+), 0 deletions(-) >> create mode 100644 opensm/include/opensm/osm_qos_policy.h >> create mode 100644 opensm/opensm/osm_qos_policy.c >> >> diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h >> new file mode 100644 >> index 0000000..05b8dc9 >> --- /dev/null >> +++ b/opensm/include/opensm/osm_qos_policy.h >> @@ -0,0 +1,188 @@ >> +/* >> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. >> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. >> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. >> + * >> + * This software is available to you under a choice of one of two >> + * licenses. You may choose to be licensed under the terms of the GNU >> + * General Public License (GPL) Version 2, available from the file >> + * COPYING in the main directory of this source tree, or the >> + * OpenIB.org BSD license below: >> + * >> + * Redistribution and use in source and binary forms, with or >> + * without modification, are permitted provided that the following >> + * conditions are met: >> + * >> + * - Redistributions of source code must retain the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer. >> + * >> + * - Redistributions in binary form must reproduce the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer in the documentation and/or other materials >> + * provided with the distribution. >> + * >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE >> + * SOFTWARE. >> + * >> + */ >> + >> +/* >> + * Abstract: >> + * Declaration of OSM QoS Policy data types and functions. >> + * >> + * Environment: >> + * Linux User Mode >> + * >> + * Author: >> + * Yevgeny Kliteynik, Mellanox >> + */ >> + >> +#ifndef OSM_QOS_POLICY_H >> +#define OSM_QOS_POLICY_H >> + >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> + >> +#define YYSTYPE char * >> +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 >> +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_port_group_t_ { >> + char *name; /* single string (this port group name) */ >> + char *use; /* single string (description) */ >> + cl_list_t port_name_list; /* list of port names (.../.../...) */ >> + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ >> + unsigned guid_range_len; /* num of guid ranges in the array */ >> + cl_list_t partition_list; /* list of partition names */ >> + boolean_t node_type_ca; >> + boolean_t node_type_switch; >> + boolean_t node_type_router; >> + boolean_t node_type_self; >> +} osm_qos_port_group_t; > > I see you are using this in "run-time", not just during the parsing. > > Instead of having all this config features you can just resolve port > guids in parse time and keep it here in cl_map() for fast searches. By saying "config features", do you mean the four boolean flags? It looks to me that checking the type of node is as fast as it gets, and it won't hurt to leave these booleans instead of resolving all the guids. Moreover, the guids here are stored in range array, which is IMO better suited for the policy file syntax, because if a user specifies something like this "0x0-0x0FFF" in guids, it will be only one element of the array, which is efficient both in memory and in serch time. (I probably should mention here that the efficient search in the range array is not implemented yet, but it would be a simple binary search - there's a "todo" comment in the search function right now) >> + >> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); >> +void osm_qos_policy_port_group_destroy(); > > Would be nice to have function prototypes in one place. > >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_vlarb_scope_t_ { >> + cl_list_t group_list; /* list of group names (strings) */ >> + cl_list_t across_list; /* list of 'across' group names (strings) */ >> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ >> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ > > Why cl_list for VLArb? it should be short fixed length arrays? Right. Since the actual VLArb setup is not implemented yet, I didn't see this obvious thing. >> + uint32_t vl_high_limit; /* single integer */ >> + boolean_t vl_high_limit_set; >> +} osm_qos_vlarb_scope_t; >> + >> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); >> +void osm_qos_policy_vlarb_scope_destroy(); >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_sl2vl_scope_t_ { >> + cl_list_t group_list; /* list of strings (port group names) */ >> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; >> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; >> + cl_list_t across_from_list; /* list of strings (port group names) */ >> + cl_list_t across_to_list; /* list of strings (port group names) */ >> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ >> + boolean_t sl2vl_table_set; >> +} osm_qos_sl2vl_scope_t; > > This will be used for sl2vl setup? > Same as above - why not to generate final port guid list just in parse time? >> + >> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); >> +void osm_qos_policy_sl2vl_scope_destroy(); >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_level_t_ { >> + char *use; >> + char *name; >> + uint8_t sl; >> + boolean_t sl_set; >> + uint8_t mtu_limit; >> + boolean_t mtu_limit_set; >> + uint8_t rate_limit; >> + boolean_t rate_limit_set; >> + uint8_t pkt_life; >> + boolean_t pkt_life_set; >> + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ >> + unsigned path_bits_range_len; /* num of bit ranges in the array */ >> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ >> + unsigned pkey_range_len; >> +} osm_qos_level_t; >> + >> +osm_qos_level_t *osm_qos_policy_qos_level_create(); >> +void osm_qos_policy_qos_level_destroy(); >> + >> +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, >> + IN ib_net16_t pkey); >> + >> +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, >> + IN const osm_physp_t * p_src_physp, >> + IN const osm_physp_t * p_dest_physp); >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_match_rule_t_ { >> + char *use; >> + cl_list_t source_list; /* list of strings */ >> + cl_list_t source_group_list; /* list of pointers to relevant port-group */ >> + cl_list_t destination_list; /* list of strings */ >> + cl_list_t destination_group_list; /* list of pointers to relevant port-group */ > > I think you should only keep port guids there (mapped for fast searches). Same as above. I think that checking node type and then guid range array is essentially faster than checking guid map. You might say, of course, that there can be many port groups in the same match rule, but I don't see this as a practical example. >> + char *qos_level_name; >> + osm_qos_level_t *p_qos_level; > > Why do you need qos_level_name if you keep the pointer to this qos_level > struct? In policy file the match rule might appear before the QoS levels, so matching qos level names to the actual qos levels is done when the parsing is done. >> + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) */ >> + unsigned service_id_range_len; >> + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ >> + unsigned qos_class_range_len; >> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ >> + unsigned pkey_range_len; >> +} osm_qos_match_rule_t; >> + >> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); >> +void osm_qos_policy_match_rule_destroy(); >> + >> +/***************************************************/ >> + >> +typedef struct osm_qos_policy_t_ { >> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ >> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ >> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ >> + cl_list_t qos_levels; /* list of osm_qos_level_t */ >> + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ >> + osm_qos_level_t *p_default_qos_level; /* default QoS level */ >> +} osm_qos_policy_t; >> + >> +void osm_qos_policy_create(); >> +void osm_qos_policy_destroy(); >> +int osm_qos_policy_validate(); >> + >> +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, >> + IN const ib_path_rec_t * p_pr, >> + IN const osm_physp_t * p_src_physp, >> + IN const osm_physp_t * p_dest_physp, >> + IN ib_net64_t comp_mask, >> + OUT osm_qos_level_t ** pp_qos_level); >> + >> +/***************************************************/ >> + >> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char *policy_file); >> + >> +/***************************************************/ >> + >> +#endif /* ifndef OSM_QOS_POLICY_H */ >> diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c >> new file mode 100644 >> index 0000000..bc2aa68 >> --- /dev/null >> +++ b/opensm/opensm/osm_qos_policy.c >> @@ -0,0 +1,901 @@ >> +/* >> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. >> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. >> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. >> + * >> + * This software is available to you under a choice of one of two >> + * licenses. You may choose to be licensed under the terms of the GNU >> + * General Public License (GPL) Version 2, available from the file >> + * COPYING in the main directory of this source tree, or the >> + * OpenIB.org BSD license below: >> + * >> + * Redistribution and use in source and binary forms, with or >> + * without modification, are permitted provided that the following >> + * conditions are met: >> + * >> + * - Redistributions of source code must retain the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer. >> + * >> + * - Redistributions in binary form must reproduce the above >> + * copyright notice, this list of conditions and the following >> + * disclaimer in the documentation and/or other materials >> + * provided with the distribution. >> + * >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE >> + * SOFTWARE. >> + * >> + */ >> + >> +/* >> + * Abstract: >> + * OSM QoS Policy functions. >> + * >> + * Environment: >> + * Linux User Mode >> + * >> + * Author: >> + * Yevgeny Kliteynik, Mellanox >> + */ >> + >> +#include >> +#include >> +#include >> + >> +extern void yyerror(char *s); >> +osm_log_t *p_qos_parser_osm_log = NULL; >> +osm_qos_policy_t *p_qos_policy = NULL; > > Please try to avoid globals - keep it as part of osm_opensm_t or > osm_subn_t structures. I thought about it, but didn't want to "condaminate" the osm_opensm_t or osm_subn_t structures untill the QoS functionality is ready. -- Yevgeny >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static boolean_t >> +__is_num_in_range_arr(uint64_t ** range_arr, >> + unsigned range_arr_len, uint64_t num) >> +{ >> + unsigned i; >> + >> + /* >> + * TODO: although this array should be rather short, >> + * I should implement a binary search here >> + */ >> + >> + if (num < range_arr[0][0] || num > range_arr[range_arr_len - 1][1]) >> + return FALSE; >> + >> + for (i = 0; i < range_arr_len; i++) { >> + if (num >= range_arr[i][0] && num <= range_arr[i][1]) >> + return TRUE; >> + if (num < range_arr[i][0]) >> + return FALSE; >> + } >> + return FALSE; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static void __free_single_element(void *p_element, void *context) >> +{ >> + if (p_element) >> + free(p_element); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +osm_qos_port_group_t *osm_qos_policy_port_group_create() >> +{ >> + osm_qos_port_group_t *p = >> + (osm_qos_port_group_t *) malloc(sizeof(osm_qos_port_group_t)); >> + if (!p) >> + return NULL; >> + >> + memset(p, 0, sizeof(osm_qos_port_group_t)); >> + >> + cl_list_init(&p->port_name_list, 10); >> + cl_list_init(&p->partition_list, 10); >> + >> + return p; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) >> +{ >> + unsigned i; >> + >> + if (!p) >> + return; >> + >> + if (p->name) >> + free(p->name); >> + if (p->use) >> + free(p->use); >> + >> + for (i = 0; i < p->guid_range_len; i++) >> + free(p->guid_range_arr[i]); >> + if (p->guid_range_arr) >> + free(p->guid_range_arr); >> + >> + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); >> + cl_list_remove_all(&p->port_name_list); >> + cl_list_destroy(&p->port_name_list); >> + >> + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); >> + cl_list_remove_all(&p->partition_list); >> + cl_list_destroy(&p->partition_list); >> + >> + free(p); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create() >> +{ >> + osm_qos_vlarb_scope_t *p = >> + (osm_qos_vlarb_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); >> + if (!p) >> + return NULL; >> + >> + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); >> + >> + cl_list_init(&p->group_list, 10); >> + cl_list_init(&p->across_list, 10); >> + cl_list_init(&p->vlarb_high_list, 10); >> + cl_list_init(&p->vlarb_low_list, 10); >> + >> + return p; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) >> +{ >> + if (!p) >> + return; >> + >> + cl_list_apply_func(&p->group_list, __free_single_element, NULL); >> + cl_list_apply_func(&p->across_list, __free_single_element, NULL); >> + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); >> + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); >> + >> + cl_list_remove_all(&p->group_list); >> + cl_list_remove_all(&p->across_list); >> + cl_list_remove_all(&p->vlarb_high_list); >> + cl_list_remove_all(&p->vlarb_low_list); >> + >> + cl_list_destroy(&p->group_list); >> + cl_list_destroy(&p->across_list); >> + cl_list_destroy(&p->vlarb_high_list); >> + cl_list_destroy(&p->vlarb_low_list); >> + >> + free(p); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create() >> +{ >> + osm_qos_sl2vl_scope_t *p = >> + (osm_qos_sl2vl_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); >> + if (!p) >> + return NULL; >> + >> + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); >> + >> + cl_list_init(&p->group_list, 10); >> + cl_list_init(&p->across_from_list, 10); >> + cl_list_init(&p->across_to_list, 10); >> + >> + return p; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) >> +{ >> + if (!p) >> + return; >> + >> + cl_list_apply_func(&p->group_list, __free_single_element, NULL); >> + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); >> + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); >> + >> + cl_list_remove_all(&p->group_list); >> + cl_list_remove_all(&p->across_from_list); >> + cl_list_remove_all(&p->across_to_list); >> + >> + cl_list_destroy(&p->group_list); >> + cl_list_destroy(&p->across_from_list); >> + cl_list_destroy(&p->across_to_list); >> + >> + free(p); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +osm_qos_level_t *osm_qos_policy_qos_level_create() >> +{ >> + osm_qos_level_t *p = >> + (osm_qos_level_t *) malloc(sizeof(osm_qos_level_t)); >> + if (!p) >> + return NULL; >> + memset(p, 0, sizeof(osm_qos_level_t)); >> + return p; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) >> +{ >> + unsigned i; >> + >> + if (!p) >> + return; >> + >> + if (p->use) >> + free(p->use); >> + >> + for (i = 0; i < p->path_bits_range_len; i++) >> + free(p->path_bits_range_arr[i]); >> + if (p->path_bits_range_arr) >> + free(p->path_bits_range_arr); >> + >> + free(p); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, >> + IN ib_net16_t pkey) >> +{ >> + if (!p_qos_level || !p_qos_level->pkey_range_len) >> + return FALSE; >> + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, >> + p_qos_level->pkey_range_len, >> + cl_ntoh16(pkey)); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, >> + IN const osm_physp_t * p_src_physp, >> + IN const osm_physp_t * p_dest_physp) >> +{ >> + unsigned i; >> + uint16_t pkey_ho = 0; >> + >> + if (!p_qos_level || !p_qos_level->pkey_range_len) >> + return 0; >> + >> + /* >> + * ToDo: This approach is not optimal. >> + * Think how to find shared pkey that also exists >> + * in QoS level in less runtime. >> + */ >> + >> + for (i = 0; i < p_qos_level->pkey_range_len; i++) { >> + for (pkey_ho = p_qos_level->pkey_range_arr[i][0]; >> + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; pkey_ho++) { >> + if (osm_physp_share_this_pkey >> + (p_src_physp, p_dest_physp, cl_hton16(pkey_ho))) >> + return cl_hton16(pkey_ho); >> + } >> + } >> + >> + return 0; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create() >> +{ >> + osm_qos_match_rule_t *p = >> + (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); >> + if (!p) >> + return NULL; >> + >> + memset(p, 0, sizeof(osm_qos_match_rule_t)); >> + >> + cl_list_init(&p->source_list, 10); >> + cl_list_init(&p->source_group_list, 10); >> + cl_list_init(&p->destination_list, 10); >> + cl_list_init(&p->destination_group_list, 10); >> + >> + return p; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) >> +{ >> + unsigned i; >> + >> + if (!p) >> + return; >> + >> + if (p->qos_level_name) >> + free(p->qos_level_name); >> + if (p->use) >> + free(p->use); >> + >> + for (i = 0; i < p->service_id_range_len; i++) >> + free(p->service_id_range_arr[i]); >> + if (p->service_id_range_arr) >> + free(p->service_id_range_arr); >> + >> + for (i = 0; i < p->qos_class_range_len; i++) >> + free(p->qos_class_range_arr[i]); >> + if (p->qos_class_range_arr) >> + free(p->qos_class_range_arr); >> + >> + cl_list_apply_func(&p->source_list, __free_single_element, NULL); >> + cl_list_remove_all(&p->source_list); >> + cl_list_destroy(&p->source_list); >> + >> + cl_list_remove_all(&p->source_group_list); >> + cl_list_destroy(&p->source_group_list); >> + >> + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); >> + cl_list_remove_all(&p->destination_list); >> + cl_list_destroy(&p->destination_list); >> + >> + cl_list_remove_all(&p->destination_group_list); >> + cl_list_destroy(&p->destination_group_list); >> + >> + free(p); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_create() >> +{ >> + if (p_qos_policy) >> + osm_qos_policy_destroy(); >> + >> + p_qos_policy = (osm_qos_policy_t *) malloc(sizeof(osm_qos_policy_t)); >> + CL_ASSERT(p_qos_policy); >> + >> + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); >> + >> + cl_list_construct(&p_qos_policy->port_groups); >> + cl_list_init(&p_qos_policy->port_groups, 10); >> + >> + cl_list_construct(&p_qos_policy->vlarb_tables); >> + cl_list_init(&p_qos_policy->vlarb_tables, 10); >> + >> + cl_list_construct(&p_qos_policy->sl2vl_tables); >> + cl_list_init(&p_qos_policy->sl2vl_tables, 10); >> + >> + cl_list_construct(&p_qos_policy->qos_levels); >> + cl_list_init(&p_qos_policy->qos_levels, 10); >> + >> + cl_list_construct(&p_qos_policy->qos_match_rules); >> + cl_list_init(&p_qos_policy->qos_match_rules, 10); >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_destroy() >> +{ >> + cl_list_iterator_t list_iterator; >> + osm_qos_port_group_t *p_port_group = NULL; >> + osm_qos_vlarb_scope_t *p_vlarb_scope = NULL; >> + osm_qos_sl2vl_scope_t *p_sl2vl_scope = NULL; >> + osm_qos_level_t *p_qos_level = NULL; >> + osm_qos_match_rule_t *p_qos_match_rule = NULL; >> + >> + list_iterator = cl_list_head(&p_qos_policy->port_groups); >> + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { >> + p_port_group = >> + (osm_qos_port_group_t *) cl_list_obj(list_iterator); >> + if (p_port_group) >> + osm_qos_policy_port_group_destroy(p_port_group); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&p_qos_policy->port_groups); >> + cl_list_destroy(&p_qos_policy->port_groups); >> + >> + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); >> + while (list_iterator != cl_list_end(&p_qos_policy->vlarb_tables)) { >> + p_vlarb_scope = >> + (osm_qos_vlarb_scope_t *) cl_list_obj(list_iterator); >> + if (p_vlarb_scope) >> + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&p_qos_policy->vlarb_tables); >> + cl_list_destroy(&p_qos_policy->vlarb_tables); >> + >> + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); >> + while (list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables)) { >> + p_sl2vl_scope = >> + (osm_qos_sl2vl_scope_t *) cl_list_obj(list_iterator); >> + if (p_sl2vl_scope) >> + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&p_qos_policy->sl2vl_tables); >> + cl_list_destroy(&p_qos_policy->sl2vl_tables); >> + >> + list_iterator = cl_list_head(&p_qos_policy->qos_levels); >> + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { >> + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); >> + if (p_qos_level) >> + osm_qos_policy_qos_level_destroy(p_qos_level); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&p_qos_policy->qos_levels); >> + cl_list_destroy(&p_qos_policy->qos_levels); >> + >> + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); >> + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { >> + p_qos_match_rule = >> + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); >> + if (p_qos_match_rule) >> + osm_qos_policy_match_rule_destroy(p_qos_match_rule); >> + list_iterator = cl_list_next(list_iterator); >> + } >> + cl_list_remove_all(&p_qos_policy->qos_match_rules); >> + cl_list_destroy(&p_qos_policy->qos_match_rules); >> + >> + free(p_qos_policy); >> + >> + p_qos_policy = NULL; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static boolean_t >> +__qos_policy_is_port_in_group(osm_subn_t * p_subn, >> + const osm_physp_t * p_physp, >> + osm_qos_port_group_t * p_port_group) >> +{ >> + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); >> + osm_prtn_t *p_prtn = NULL; >> + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); >> + uint64_t port_guid_ho = cl_ntoh64(port_guid); >> + uint8_t node_type = osm_node_get_type(p_node); >> + cl_list_iterator_t list_iterator; >> + char *partition_name; >> + >> + /* check whether this port's type matches any of group's types */ >> + >> + if ((node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || >> + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) >> + || (node_type == IB_NODE_TYPE_ROUTER >> + && p_port_group->node_type_router)) >> + return TRUE; >> + >> + /* check whether this port's guid is in range of this group's guids */ >> + >> + if (__is_num_in_range_arr(p_port_group->guid_range_arr, >> + p_port_group->guid_range_len, port_guid_ho)) >> + return TRUE; >> + >> + /* check whether this port is member of this group's partitions */ >> + >> + list_iterator = cl_list_head(&p_port_group->partition_list); >> + while (list_iterator != cl_list_end(&p_port_group->partition_list)) { >> + partition_name = (char *)cl_list_obj(list_iterator); >> + if (partition_name && strlen(partition_name)) { >> + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); >> + if (p_prtn) { >> + if (osm_prtn_is_guid(p_prtn, port_guid)) >> + return TRUE; >> + } >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + >> + /* check whether this port's name matches any of group's names */ >> + >> + /* >> + * TODO: check port names >> + * >> + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; >> + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); >> + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; >> + */ >> + >> + return FALSE; >> +} /* __qos_policy_is_port_in_group() */ >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static boolean_t >> +__qos_policy_is_port_in_group_list(const osm_pr_rcv_t * p_rcv, >> + const osm_physp_t * p_physp, >> + cl_list_t * p_port_group_list) >> +{ >> + osm_qos_port_group_t *p_port_group; >> + cl_list_iterator_t list_iterator; >> + >> + list_iterator = cl_list_head(p_port_group_list); >> + while (list_iterator != cl_list_end(p_port_group_list)) { >> + p_port_group = >> + (osm_qos_port_group_t *) cl_list_obj(list_iterator); >> + if (p_port_group) { >> + if (__qos_policy_is_port_in_group >> + (p_rcv->p_subn, p_physp, p_port_group)) >> + return TRUE; >> + } >> + list_iterator = cl_list_next(list_iterator); >> + } >> + return FALSE; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr( >> + const osm_pr_rcv_t * p_rcv, >> + const ib_path_rec_t * p_pr, >> + const osm_physp_t * p_src_physp, >> + const osm_physp_t * p_dest_physp, >> + ib_net64_t comp_mask) >> +{ >> + osm_qos_match_rule_t *p_qos_match_rule = NULL; >> + cl_list_iterator_t list_iterator; >> + >> + if (!cl_list_count(&p_qos_policy->qos_match_rules)) >> + return NULL; >> + >> + /* Go over all QoS match rules and find the one that matches the request */ >> + >> + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); >> + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { >> + p_qos_match_rule = >> + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); >> + if (!p_qos_match_rule) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + /* If a match rule has Source groups, PR request source has to be in this list */ >> + >> + if (cl_list_count(&p_qos_match_rule->source_group_list)) { >> + if (!__qos_policy_is_port_in_group_list(p_rcv, >> + p_src_physp, >> + &p_qos_match_rule-> >> + source_group_list)) >> + { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + } >> + >> + /* If a match rule has Destination groups, PR request dest. has to be in this list */ >> + >> + if (cl_list_count(&p_qos_match_rule->destination_group_list)) { >> + if (!__qos_policy_is_port_in_group_list(p_rcv, >> + p_dest_physp, >> + &p_qos_match_rule-> >> + destination_group_list)) >> + { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + } >> + >> + /* If a match rule has QoS classes, PR request HAS >> + to have a matching QoS class to match the rule */ >> + >> + if (p_qos_match_rule->qos_class_range_len) { >> + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + if (!__is_num_in_range_arr >> + (p_qos_match_rule->qos_class_range_arr, >> + p_qos_match_rule->qos_class_range_len, >> + ib_path_rec_qos_class(p_pr))) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + } >> + >> + /* If a match rule has Service IDs, PR request HAS >> + to have a matching Service ID to match the rule */ >> + >> + if (p_qos_match_rule->service_id_range_len) { >> + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + if (!__is_num_in_range_arr >> + (p_qos_match_rule->service_id_range_arr, >> + p_qos_match_rule->service_id_range_len, >> + p_pr->service_id)) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + } >> + >> + /* If a match rule has PKeys, PR request HAS >> + to have a matching PKey to match the rule */ >> + >> + if (p_qos_match_rule->pkey_range_len) { >> + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + if (!__is_num_in_range_arr >> + (p_qos_match_rule->pkey_range_arr, >> + p_qos_match_rule->pkey_range_len, >> + ib_path_rec_qos_class(p_pr))) { >> + list_iterator = cl_list_next(list_iterator); >> + continue; >> + } >> + >> + } >> + >> + /* if we got here, then this match-rule matched this PR request */ >> + break; >> + } >> + >> + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) >> + return NULL; >> + >> + return p_qos_match_rule; >> +} /* __qos_policy_get_match_rule_by_pr() */ >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static osm_qos_level_t *__qos_policy_get_qos_level_by_name(char *name) >> +{ >> + osm_qos_level_t *p_qos_level = NULL; >> + cl_list_iterator_t list_iterator; >> + >> + list_iterator = cl_list_head(&p_qos_policy->qos_levels); >> + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { >> + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); >> + if (!p_qos_level) >> + continue; >> + >> + /* names are case INsensitive */ >> + if (strcasecmp(name, p_qos_level->name) == 0) >> + return p_qos_level; >> + >> + list_iterator = cl_list_next(list_iterator); >> + } >> + >> + return NULL; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +static osm_qos_port_group_t *__qos_policy_get_port_group_by_name(const char >> + *const name) >> +{ >> + osm_qos_port_group_t *p_port_group = NULL; >> + cl_list_iterator_t list_iterator; >> + >> + list_iterator = cl_list_head(&p_qos_policy->port_groups); >> + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { >> + p_port_group = >> + (osm_qos_port_group_t *) cl_list_obj(list_iterator); >> + if (!p_port_group) >> + continue; >> + >> + /* names are case INsensitive */ >> + if (strcasecmp(name, p_port_group->name) == 0) >> + return p_port_group; >> + >> + list_iterator = cl_list_next(list_iterator); >> + } >> + >> + return NULL; >> +} >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +int osm_qos_policy_validate() >> +{ >> + cl_list_iterator_t match_rules_list_iterator; >> + cl_list_iterator_t list_iterator; >> + osm_qos_port_group_t *p_port_group = NULL; >> + osm_qos_match_rule_t *p_qos_match_rule = NULL; >> + char *str; >> + unsigned i; >> + int res = 0; >> + >> + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_policy_validate); >> + >> + /* set default qos level */ >> + >> + p_qos_policy->p_default_qos_level = >> + __qos_policy_get_qos_level_by_name >> + (OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); >> + if (!p_qos_policy->p_default_qos_level) { >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "osm_qos_policy_validate: ERR AC10: " >> + "Default qos-level (%s) not defined.\n", >> + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); >> + res = 1; >> + goto Exit; >> + } >> + >> + /* scan all the match rules, and fill the lists of pointers to >> + relevant qos levels and port groups to speed up PR matching */ >> + >> + i = 1; >> + match_rules_list_iterator = >> + cl_list_head(&p_qos_policy->qos_match_rules); >> + while (match_rules_list_iterator != >> + cl_list_end(&p_qos_policy->qos_match_rules)) { >> + p_qos_match_rule = >> + (osm_qos_match_rule_t *) >> + cl_list_obj(match_rules_list_iterator); >> + CL_ASSERT(p_qos_match_rule); >> + >> + /* find the matching qos-level for each match-rule */ >> + >> + p_qos_match_rule->p_qos_level = >> + __qos_policy_get_qos_level_by_name(p_qos_match_rule-> >> + qos_level_name); >> + >> + if (!p_qos_match_rule->p_qos_level) { >> + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, >> + "osm_qos_policy_validate: ERR AC11: " >> + "qos-match-rule num %u: qos-level '%s' not found\n", >> + i, p_qos_match_rule->qos_level_name); >> + res = 1; >> + goto Exit; >> + } >> + >> + /* find the matching port-group for element of source_list */ >> + >> + if (cl_list_count(&p_qos_match_rule->source_list)) { >> + list_iterator = >> + cl_list_head(&p_qos_match_rule->source_list); >> + while (list_iterator != >> + cl_list_end(&p_qos_match_rule->source_list)) { >> + str = (char *)cl_list_obj(list_iterator); >> + CL_ASSERT(str); >> + >> + p_port_group = >> + __qos_policy_get_port_group_by_name(str); >> + if (!p_port_group) { >> + osm_log(p_qos_parser_osm_log, >> + OSM_LOG_ERROR, >> + "osm_qos_policy_validate: ERR AC12: " >> + "qos-match-rule num %u: source port-group '%s' not found\n", >> + i, str); >> + res = 1; >> + goto Exit; >> + } >> + >> + cl_list_insert_tail(&p_qos_match_rule-> >> + source_group_list, >> + p_port_group); >> + >> + list_iterator = cl_list_next(list_iterator); >> + } >> + } >> + >> + /* find the matching port-group for element of destination_list */ >> + >> + if (cl_list_count(&p_qos_match_rule->destination_list)) { >> + list_iterator = >> + cl_list_head(&p_qos_match_rule->destination_list); >> + while (list_iterator != >> + cl_list_end(&p_qos_match_rule-> >> + destination_list)) { >> + str = (char *)cl_list_obj(list_iterator); >> + CL_ASSERT(str); >> + >> + p_port_group = >> + __qos_policy_get_port_group_by_name(str); >> + if (!p_port_group) { >> + osm_log(p_qos_parser_osm_log, >> + OSM_LOG_ERROR, >> + "osm_qos_policy_validate: ERR AC13: " >> + "qos-match-rule num %u: destination port-group '%s' not found\n", >> + i, str); >> + res = 1; >> + goto Exit; >> + } >> + >> + cl_list_insert_tail(&p_qos_match_rule-> >> + destination_group_list, >> + p_port_group); >> + >> + list_iterator = cl_list_next(list_iterator); >> + } >> + } >> + >> + /* done with the current match-rule */ >> + >> + match_rules_list_iterator = >> + cl_list_next(match_rules_list_iterator); >> + i++; >> + } >> + >> + Exit: >> + OSM_LOG_EXIT(p_qos_parser_osm_log); >> + return res; >> +} /* osm_qos_policy_validate() */ >> + >> +/*************************************************** >> + ***************************************************/ >> + >> +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, >> + IN const ib_path_rec_t * p_pr, >> + IN const osm_physp_t * p_src_physp, >> + IN const osm_physp_t * p_dest_physp, >> + IN ib_net64_t comp_mask, >> + OUT osm_qos_level_t ** pp_qos_level) >> +{ >> + osm_qos_match_rule_t *p_qos_match_rule = NULL; >> + osm_qos_level_t *p_qos_level = NULL; >> + >> + OSM_LOG_ENTER(p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr); >> + >> + *pp_qos_level = NULL; >> + >> + if (!p_qos_policy) >> + goto Exit; >> + >> + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_rcv, >> + p_pr, >> + p_src_physp, >> + p_dest_physp, >> + comp_mask); >> + >> + if (p_qos_match_rule) >> + p_qos_level = p_qos_match_rule->p_qos_level; >> + else >> + p_qos_level = p_qos_policy->p_default_qos_level; >> + >> + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, >> + "osm_qos_policy_get_qos_level_by_pr: " >> + "PathRecord request:" >> + "Src port 0x%016" PRIx64 ", " >> + "Dst port 0x%016" PRIx64 "\n", >> + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), >> + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp))); >> + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, >> + "osm_qos_policy_get_qos_level_by_pr: " >> + "Applying QoS Level %s (%s)\n", >> + p_qos_level->name, >> + (p_qos_level->use) ? p_qos_level->use : "no description"); >> + >> + *pp_qos_level = p_qos_level; >> + >> + Exit: >> + OSM_LOG_EXIT(p_rcv->p_log); >> +} /* osm_qos_policy_get_qos_level_by_pr() */ >> + >> +/*************************************************** >> + ***************************************************/ >> -- >> 1.5.1.4 >> > From tziporet at dev.mellanox.co.il Thu Aug 23 01:13:20 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Thu, 23 Aug 2007 11:13:20 +0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> Message-ID: <46CD41A0.70000@mellanox.co.il> John Leidel wrote: > All, in adding two new blade centers full of machines to my existing > cluster install, I'm getting the following errors in trying to run MPI > jobs over MVAPICH :: > > libibverbs: Fatal: no infiniband class devices found. > No IB device found > > Please send the output of lspci -v for the IB card. Also - to see which kernel modules are loaded please run: lsmod | grep ib If you have OFED installed on the system you can find the prefix running the script: /etc/infiniband/info The binaries are installed under /bin Tziporet From kliteyn at dev.mellanox.co.il Thu Aug 23 01:26:46 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 11:26:46 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <20070823054027.GK1397@sashak.voltaire.com> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> <20070823054027.GK1397@sashak.voltaire.com> Message-ID: <46CD44C6.2030606@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >> index 6dfa824..31e09f7 100644 >> --- a/opensm/opensm/Makefile.am >> +++ b/opensm/opensm/Makefile.am >> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ >> osm_vl15intf.c osm_vl_arb_rcv.c \ >> st.c osm_perfmgr.c osm_perfmgr_db.c \ >> - osm_event_plugin.c osm_dump.c >> + osm_event_plugin.c osm_dump.c \ >> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >> + >> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > > This is what I have now. Any idea? > > make all-am > make[1]: Entering directory `/home/sashak/src/m/opensm/opensm' > flex --prefix=__qos_parser_ -o ./osm_qos_parser_l.c ./osm_qos_parser.l > gcc -DHAVE_CONFIG_H -I. -I./../include -I./../../libibcommon/include/infiniband -I./../../libibumad/include/infiniband -Wall -DOSM_VENDOR_INTF_OPENIB -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -g -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 -MT opensm-osm_qos_parser_l.o -MD -MP -MF .deps/opensm-osm_qos_parser_l.Tpo -c -o opensm-osm_qos_parser_l.o `test -f 'osm_qos_parser_l.c' || echo './'`osm_qos_parser_l.c > ./osm_qos_parser.l: In function '__qos_parser_lex': > ./osm_qos_parser.l:178: error: 'yylval' undeclared (first use in this function) > ./osm_qos_parser.l:178: error: (Each undeclared identifier is reported only once > ./osm_qos_parser.l:178: error: for each function it appears in.) > make[1]: *** [opensm-osm_qos_parser_l.o] Error 1 > make[1]: Leaving directory `/home/sashak/src/m/opensm/opensm' > make: *** [all] Error 2 Looks like I forgot to repost the Lex & Yacc files after changing them to have different prefix in variables - this is patch 2/7. Sorry. I'm sending it now. -- Yevgeny > Sasha > From kliteyn at dev.mellanox.co.il Thu Aug 23 01:59:11 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 11:59:11 +0300 Subject: [ofa-general] [PATCH 2/7 V2] osm: QoS - policy file parser Lex & Yacc Message-ID: <46CD4C5F.4060709@dev.mellanox.co.il> QoS Policy file parser - Lex & Yacc files Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_parser.l | 329 ++++++ opensm/opensm/osm_qos_parser.y | 2146 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 2475 insertions(+), 0 deletions(-) create mode 100644 opensm/opensm/osm_qos_parser.l create mode 100644 opensm/opensm/osm_qos_parser.y diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l new file mode 100644 index 0000000..bc03123 --- /dev/null +++ b/opensm/opensm/osm_qos_parser.l @@ -0,0 +1,329 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Lexer of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include + +#define HANDLE_IF_IN_DESCRIPTION if (in_description) { __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } + +#define SAVE_POS save_pos() +static void save_pos(); + +extern int column_num; +extern int line_num; +extern FILE * __qos_parser_in; + +boolean_t in_description = FALSE; +boolean_t in_list_of_hex_num_ranges = FALSE; +boolean_t in_node_type = FALSE; +boolean_t in_list_of_numbers = FALSE; +boolean_t in_list_of_strings = FALSE; +boolean_t in_list_of_num_pairs = FALSE; +boolean_t in_asterisk_or_list_of_numbers = FALSE; +boolean_t in_list_of_num_ranges = FALSE; +boolean_t in_single_string = FALSE; +boolean_t in_single_number = FALSE; + +static void reset_new_line_flags(); +#define RESET_NEW_LINE_FLAGS reset_new_line_flags() + +#define START_USE {in_description = TRUE;} /* list of strings including whitespace (description) */ +#define START_PORT_GUID {in_list_of_hex_num_ranges = TRUE;} /* comma-separated list of hex num ranges */ +#define START_PORT_NAME {in_list_of_strings = TRUE;} /* comma-separated list of following strings: ../../.. */ +#define START_PARTITION {in_single_string = TRUE;} /* single string w/o whitespaces (partition name) */ +#define START_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (port group name) */ +#define START_QOS_LEVEL_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (qos level name in match rule) */ + +#define START_NODE_TYPE {in_node_type = TRUE;} /* comma-separated list of node types (ROUTER,CA,...) */ +#define START_SL2VL_TABLE {in_list_of_numbers = TRUE;} /* comma-separated list of hex or dec numbers */ + +#define START_GROUP {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_TO {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_FROM {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_SOURCE {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_DESTINATION {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ + +#define START_VLARB_HIGH {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ +#define START_VLARB_LOW {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ + +#define START_TO {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ +#define START_FROM {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ + +#define START_PATH_BITS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_QOS_CLASS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_SERVICE_ID {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_PKEY {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ + +#define START_SL {in_single_number = TRUE;} /* single number */ +#define START_VLARB_HIGH_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_MTU_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_RATE_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_PACKET_LIFE {in_single_number = TRUE;} /* single number */ + + + +%} + +PORT_GROUPS_START port\-groups +PORT_GROUPS_END end\-port\-groups +PORT_GROUP_START port\-group +PORT_GROUP_END end\-port\-group +NAME name +USE use +PORT_GUID port\-guid +PORT_NAME port\-name +PARTITION partition +NODE_TYPE node\-type +QOS_SETUP_START qos\-setup +QOS_SETUP_END end\-qos\-setup +VLARB_TABLES_START vlarb\-tables +VLARB_TABLES_END end\-vlarb\-tables +VLARB_SCOPE_START vlarb\-scope +VLARB_SCOPE_END end\-vlarb\-scope +GROUP group +ACROSS across +VLARB_HIGH vlarb\-high +VLARB_LOW vlarb\-low +VLARB_HIGH_LIMIT vl\-high\-limit +SL2VL_TABLES_START sl2vl\-tables +SL2VL_TABLES_END end\-sl2vl\-tables +SL2VL_SCOPE_START sl2vl\-scope +SL2VL_SCOPE_END end\-sl2vl\-scope +TO to +FROM from +ACROSS_TO across\-to +ACROSS_FROM across\-from +SL2VL_TABLE sl2vl\-table +QOS_LEVELS_START qos\-levels +QOS_LEVELS_END end\-qos\-levels +QOS_LEVEL_START qos\-level +QOS_LEVEL_END end\-qos\-level +SL sl +MTU_LIMIT mtu\-limit +RATE_LIMIT rate\-limit +PACKET_LIFE packet\-life +PATH_BITS path\-bits +QOS_MATCH_RULES_START qos\-match\-rules +QOS_MATCH_RULES_END end\-qos\-match\-rules +QOS_MATCH_RULE_START qos\-match\-rule +QOS_MATCH_RULE_END end\-qos\-match\-rule +QOS_CLASS qos\-class +SOURCE source +DESTINATION destination +SERVICE_ID service\-id +PKEY pkey +QOS_LEVEL_NAME qos\-level\-name + +ROUTER [Rr][Oo][Uu][Tt][Ee][Rr] +CA [Cc][Aa] +SWITCH [Ss][Ww][Ii][Tt][Cc][Hh] +SELF [Ss][Ee][Ll][Ff] +ALL [Aa][Ll][Ll] + +WHITE [ \t]+ +NEW_LINE \n +COMMENT \#.*\n +WHITE_DOTDOT_WHITE [ \t]*:[ \t]* + +%% + + +{COMMENT} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* swallow comment */ +{WHITE}{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* trailing blanks with new line */ +{WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; } +{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } + +{PORT_GROUPS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_START; } +{PORT_GROUPS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_END; } +{PORT_GROUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_START; } +{PORT_GROUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_END; } + +{QOS_SETUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_START; } +{QOS_SETUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_END; } +{VLARB_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_START; } +{VLARB_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_END; } +{VLARB_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_START; } +{VLARB_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_END; } + +{SL2VL_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_START; } +{SL2VL_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_END; } +{SL2VL_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_START; } +{SL2VL_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_END; } + +{QOS_LEVELS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_START; } +{QOS_LEVELS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_END; } +{QOS_LEVEL_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_START; } +{QOS_LEVEL_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_END; } + +{QOS_MATCH_RULES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_START; } +{QOS_MATCH_RULES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_END; } +{QOS_MATCH_RULE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_START; } +{QOS_MATCH_RULE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_END; } + +{PORT_GUID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_GUID; return TK_PORT_GUID; } +{PORT_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_NAME; return TK_PORT_NAME; } +{PARTITION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PARTITION; return TK_PARTITION; } +{NODE_TYPE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NODE_TYPE; return TK_NODE_TYPE; } +{NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NAME; return TK_NAME; } +{USE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_USE; return TK_USE; } +{GROUP}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_GROUP; return TK_GROUP; } +{VLARB_HIGH}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH; return TK_VLARB_HIGH; } +{VLARB_LOW}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_LOW; return TK_VLARB_LOW; } +{VLARB_HIGH_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH_LIMIT; return TK_VLARB_HIGH_LIMIT;} +{TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_TO; return TK_TO; } +{FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_FROM; return TK_FROM; } +{ACROSS_TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_TO; return TK_ACROSS_TO; } +{ACROSS_FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_FROM; return TK_ACROSS_FROM;} +{ACROSS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS; return TK_ACROSS; } +{SL2VL_TABLE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL2VL_TABLE; return TK_SL2VL_TABLE;} +{SL}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL; return TK_SL; } +{MTU_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_MTU_LIMIT; return TK_MTU_LIMIT; } +{RATE_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_RATE_LIMIT; return TK_RATE_LIMIT; } +{PACKET_LIFE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PACKET_LIFE; return TK_PACKET_LIFE;} +{PATH_BITS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PATH_BITS; return TK_PATH_BITS; } +{QOS_CLASS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_CLASS; return TK_QOS_CLASS; } +{SOURCE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SOURCE; return TK_SOURCE; } +{DESTINATION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_DESTINATION; return TK_DESTINATION;} +{SERVICE_ID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SERVICE_ID; return TK_SERVICE_ID; } +{PKEY}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PKEY; return TK_PKEY; } +{QOS_LEVEL_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_LEVEL_NAME; return TK_QOS_LEVEL_NAME;} + +{ROUTER} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ROUTER; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{CA} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_CA; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{SWITCH} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SWITCH; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{SELF} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SELF; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{ALL} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ALL; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } + +0[xX][0-9a-fA-F]+ { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + +[0-9]+ { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + + +- { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DASH; + } + +: { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DOTDOT; + } + +, { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description) + return TK_TEXT; + return TK_COMMA; + } + +\* { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_ASTERISK; + } + +. { SAVE_POS; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT;} + +%% + + +/********************************************* + *********************************************/ + +static void save_pos() +{ + int i; + for (i = 0; i < __qos_parser_leng; i++) + { + if (__qos_parser_text[i] == '\n') + { + line_num ++; + column_num = 1; + } + else + column_num ++; + } +} + +/********************************************* + *********************************************/ + +static void reset_new_line_flags() +{ + in_description = FALSE; + in_list_of_hex_num_ranges = FALSE; + in_node_type = FALSE; + in_list_of_numbers = FALSE; + in_list_of_strings = FALSE; + in_list_of_num_pairs = FALSE; + in_asterisk_or_list_of_numbers = FALSE; + in_list_of_num_ranges = FALSE; + in_single_string = FALSE; + in_single_number = FALSE; +} diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y new file mode 100644 index 0000000..a54e9ae --- /dev/null +++ b/opensm/opensm/osm_qos_parser.y @@ -0,0 +1,2146 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Grammar of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include + +#define OSM_QOS_POLICY_MAX_LINE_LEN 1024*10 +#define OSM_QOS_POLICY_SL2VL_TABLE_LEN IB_MAX_NUM_VLS +#define OSM_QOS_POLICY_MAX_VL_NUM IB_MAX_NUM_VLS + +typedef struct tmp_parser_struct_t_ { + char str[OSM_QOS_POLICY_MAX_LINE_LEN]; + uint64_t num_pair[2]; + cl_list_t str_list; + cl_list_t num_list; + cl_list_t num_pair_list; +} tmp_parser_struct_t; + +static void __parser_tmp_struct_init(); +static void __parser_tmp_struct_reset(); +static void __parser_tmp_struct_destroy(); + +static char * __parser_strip_white(char * str); + +static void __parser_str2uint64(uint64_t * p_val, char * str); + +static void __parser_port_group_start(); +static int __parser_port_group_end(); + +static void __parser_sl2vl_scope_start(); +static int __parser_sl2vl_scope_end(); + +static void __parser_vlarb_scope_start(); +static int __parser_vlarb_scope_end(); + +static void __parser_qos_level_start(); +static int __parser_qos_level_end(); + +static void __parser_match_rule_start(); +static int __parser_match_rule_end(); + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len); + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ); + +extern char * __qos_parser_text; +extern void __qos_parser_error (char *s); +extern int __qos_parser_lex (void); +extern FILE * __qos_parser_in; + +#define RESET_BUFFER __parser_tmp_struct_reset() + +tmp_parser_struct_t tmp_parser_struct; + +int column_num; +int line_num; + +extern osm_qos_policy_t * p_qos_policy; +osm_qos_port_group_t * p_current_port_group = NULL; +osm_qos_sl2vl_scope_t * p_current_sl2vl_scope = NULL; +osm_qos_vlarb_scope_t * p_current_vlarb_scope = NULL; +osm_qos_level_t * p_current_qos_level = NULL; +osm_qos_match_rule_t * p_current_qos_match_rule = NULL; +extern osm_log_t * p_qos_parser_osm_log; + +/***************************************************/ + +%} + +%token TK_NUMBER +%token TK_DASH +%token TK_DOTDOT +%token TK_COMMA +%token TK_ASTERISK +%token TK_TEXT + +%token TK_PORT_GROUPS_START +%token TK_PORT_GROUPS_END +%token TK_PORT_GROUP_START +%token TK_PORT_GROUP_END + +%token TK_QOS_SETUP_START +%token TK_QOS_SETUP_END +%token TK_VLARB_TABLES_START +%token TK_VLARB_TABLES_END +%token TK_VLARB_SCOPE_START +%token TK_VLARB_SCOPE_END + +%token TK_SL2VL_TABLES_START +%token TK_SL2VL_TABLES_END +%token TK_SL2VL_SCOPE_START +%token TK_SL2VL_SCOPE_END + +%token TK_QOS_LEVELS_START +%token TK_QOS_LEVELS_END +%token TK_QOS_LEVEL_START +%token TK_QOS_LEVEL_END + +%token TK_QOS_MATCH_RULES_START +%token TK_QOS_MATCH_RULES_END +%token TK_QOS_MATCH_RULE_START +%token TK_QOS_MATCH_RULE_END + +%token TK_NAME +%token TK_USE +%token TK_PORT_GUID +%token TK_PORT_NAME +%token TK_PARTITION +%token TK_NODE_TYPE +%token TK_GROUP +%token TK_ACROSS +%token TK_VLARB_HIGH +%token TK_VLARB_LOW +%token TK_VLARB_HIGH_LIMIT +%token TK_TO +%token TK_FROM +%token TK_ACROSS_TO +%token TK_ACROSS_FROM +%token TK_SL2VL_TABLE +%token TK_SL +%token TK_MTU_LIMIT +%token TK_RATE_LIMIT +%token TK_PACKET_LIFE +%token TK_PATH_BITS +%token TK_QOS_CLASS +%token TK_SOURCE +%token TK_DESTINATION +%token TK_SERVICE_ID +%token TK_QOS_LEVEL_NAME +%token TK_PKEY + +%token TK_NODE_TYPE_ROUTER +%token TK_NODE_TYPE_CA +%token TK_NODE_TYPE_SWITCH +%token TK_NODE_TYPE_SELF +%token TK_NODE_TYPE_ALL + + +%start head + +%% + +head: qos_policy_entries + ; + +qos_policy_entries: /* empty */ + | qos_policy_entries qos_policy_entry + ; + +qos_policy_entry: port_groups_section + | qos_setup_section + | qos_levels_section + | qos_match_rules_section + ; + + /* + * Parsing port groups: + * ------------------- + * port-groups + * port-group + * name: Storage + * use: our SRP storage targets + * port-guid: 0x1000000000000001,0x1000000000000002 + * ... + * port-name: vs1/HCA-1/P1 + * ... + * partition: Part1 + * ... + * node-type: ROUTER,CA,SWITCH,SELF,ALL + * ... + * end-port-group + * port-group + * ... + * end-port-group + * end-port-groups + */ + + +port_groups_section: TK_PORT_GROUPS_START port_groups TK_PORT_GROUPS_END + ; + +port_groups: port_group + | port_groups port_group + ; + +port_group: port_group_start port_group_entries port_group_end + ; + +port_group_start: TK_PORT_GROUP_START { + __parser_port_group_start(); + } + ; + +port_group_end: TK_PORT_GROUP_END { + if ( __parser_port_group_end() ) + return 1; + } + ; + +port_group_entries: /* empty */ + | port_group_entries port_group_entry + ; + +port_group_entry: port_group_name + | port_group_use + | port_group_port_guid + | port_group_port_name + | port_group_partition + | port_group_node_type + ; + + + /* + * Parsing qos setup: + * ----------------- + * qos-setup + * vlarb-tables + * vlarb-scope + * ... + * end-vlarb-scope + * vlarb-scope + * ... + * end-vlarb-scope + * end-vlarb-tables + * sl2vl-tables + * sl2vl-scope + * ... + * end-sl2vl-scope + * sl2vl-scope + * ... + * end-sl2vl-scope + * end-sl2vl-tables + * end-qos-setup + */ + +qos_setup_section: TK_QOS_SETUP_START qos_setup_items TK_QOS_SETUP_END + ; + +qos_setup_items: /* empty */ + | qos_setup_items vlarb_tables + | qos_setup_items sl2vl_tables + ; + + /* Parsing vlarb-tables */ + +vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END + +vlarb_scope_items: /* empty */ + | vlarb_scope_items vlarb_scope + ; + +vlarb_scope: vlarb_scope_start vlarb_scope_entries vlarb_scope_end + ; + +vlarb_scope_start: TK_VLARB_SCOPE_START { + __parser_vlarb_scope_start(); + } + ; + +vlarb_scope_end: TK_VLARB_SCOPE_END { + if ( __parser_vlarb_scope_end() ) + return 1; + } + ; + +vlarb_scope_entries:/* empty */ + | vlarb_scope_entries vlarb_scope_entry + ; + + /* + * vlarb-scope + * group: Storage + * ... + * across: Storage + * ... + * vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 + * vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 + * vl-high-limit: 10 + * end-vlarb-scope + */ + +vlarb_scope_entry: vlarb_scope_group + | vlarb_scope_across + | vlarb_scope_vlarb_high + | vlarb_scope_vlarb_low + | vlarb_scope_vlarb_high_limit + ; + + /* Parsing sl2vl-tables */ + +sl2vl_tables: TK_SL2VL_TABLES_START sl2vl_scope_items TK_SL2VL_TABLES_END + ; + +sl2vl_scope_items: /* empty */ + | sl2vl_scope_items sl2vl_scope + ; + +sl2vl_scope: sl2vl_scope_start sl2vl_scope_entries sl2vl_scope_end + ; + +sl2vl_scope_start: TK_SL2VL_SCOPE_START { + __parser_sl2vl_scope_start(); + } + ; + +sl2vl_scope_end: TK_SL2VL_SCOPE_END { + if ( __parser_sl2vl_scope_end() ) + return 1; + } + ; + +sl2vl_scope_entries:/* empty */ + | sl2vl_scope_entries sl2vl_scope_entry + ; + + /* + * sl2vl-scope + * group: Part1 + * ... + * from: * + * ... + * to: * + * ... + * across-to: Storage2 + * ... + * across-from: Storage1 + * ... + * sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 + * end-sl2vl-scope + */ + +sl2vl_scope_entry: sl2vl_scope_group + | sl2vl_scope_across + | sl2vl_scope_across_from + | sl2vl_scope_across_to + | sl2vl_scope_from + | sl2vl_scope_to + | sl2vl_scope_sl2vl_table + ; + + /* + * Parsing qos-levels: + * ------------------ + * qos-levels + * qos-level + * name: qos_level_1 + * use: for the lowest priority communication + * sl: 15 + * mtu-limit: 1 + * rate-limit: 1 + * packet-life: 12 + * path-bits: 2,4,8-32 + * pkey: 0x00FF-0x0FFF + * end-qos-level + * ... + * qos-level + * end-qos-level + * end-qos-levels + */ + + +qos_levels_section: TK_QOS_LEVELS_START qos_levels TK_QOS_LEVELS_END + ; + +qos_levels: /* empty */ + | qos_levels qos_level + ; + +qos_level: qos_level_start qos_level_entries qos_level_end + ; + +qos_level_start: TK_QOS_LEVEL_START { + __parser_qos_level_start(); + } + ; + +qos_level_end: TK_QOS_LEVEL_END { + if ( __parser_qos_level_end() ) + return 1; + } + ; + +qos_level_entries: /* empty */ + | qos_level_entries qos_level_entry + ; + +qos_level_entry: qos_level_name + | qos_level_use + | qos_level_sl + | qos_level_mtu_limit + | qos_level_rate_limit + | qos_level_packet_life + | qos_level_path_bits + | qos_level_pkey + ; + + /* + * Parsing qos-match-rules: + * ----------------------- + * qos-match-rules + * qos-match-rule + * use: low latency by class 7-9 or 11 and bla bla + * qos-class: 7-9,11 + * qos-level-name: default + * source: Storage + * destination: Storage + * service-id: 22,4719-5000 + * pkey: 0x00FF-0x0FFF + * end-qos-match-rule + * qos-match-rule + * ... + * end-qos-match-rule + * end-qos-match-rules + */ + +qos_match_rules_section: TK_QOS_MATCH_RULES_START qos_match_rules TK_QOS_MATCH_RULES_END + ; + +qos_match_rules: /* empty */ + | qos_match_rules qos_match_rule + ; + +qos_match_rule: qos_match_rule_start qos_match_rule_entries qos_match_rule_end + ; + +qos_match_rule_start: TK_QOS_MATCH_RULE_START { + __parser_match_rule_start(); + } + ; + +qos_match_rule_end: TK_QOS_MATCH_RULE_END { + if ( __parser_match_rule_end() ) + return 1; + } + ; + +qos_match_rule_entries: /* empty */ + | qos_match_rule_entries qos_match_rule_entry + ; + +qos_match_rule_entry: qos_match_rule_use + | qos_match_rule_qos_class + | qos_match_rule_qos_level_name + | qos_match_rule_source + | qos_match_rule_destination + | qos_match_rule_service_id + | qos_match_rule_pkey + ; + + /* + * port_group_entry values: + * port_group_name + * port_group_use + * port_group_port_guid + * port_group_port_name + * port_group_partition + * port_group_node_type + */ + +port_group_name: port_group_name_start single_string { + /* 'name' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->name) + { + __qos_parser_error("port-group has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +port_group_use: port_group_use_start single_string { + /* 'use' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->use) + { + __qos_parser_error("port-group has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_use_start: TK_USE { + RESET_BUFFER; + } + ; + +port_group_port_name: port_group_port_name_start string_list { + /* 'port-name' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + + /* + * TODO: parse port name strings + */ + + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->port_name_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_port_name_start: TK_PORT_NAME { + RESET_BUFFER; + } + ; + +port_group_port_guid: port_group_port_guid_start list_of_ranges { + /* 'port-guid' in 'port-group' - any num of instances */ + /* list of guid ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_port_group->guid_range_len ) + { + p_current_port_group->guid_range_arr = range_arr; + p_current_port_group->guid_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_port_group->guid_range_arr, + p_current_port_group->guid_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_port_group->guid_range_arr = new_range_arr; + p_current_port_group->guid_range_len = new_range_len; + } + } + } + ; + +port_group_port_guid_start: TK_PORT_GUID { + RESET_BUFFER; + } + ; + +port_group_partition: port_group_partition_start string_list { + /* 'partition' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->partition_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_partition_start: TK_PARTITION { + RESET_BUFFER; + } + ; + +port_group_node_type: port_group_node_type_start port_group_node_type_list { + /* 'node-type' in 'port-group' - any num of instances */ + } + ; + +port_group_node_type_start: TK_NODE_TYPE { + RESET_BUFFER; + } + ; + +port_group_node_type_list: node_type_item + | port_group_node_type_list TK_COMMA node_type_item + ; + +node_type_item: node_type_ca + | node_type_switch + | node_type_router + | node_type_all + | node_type_self + ; + +node_type_ca: TK_NODE_TYPE_CA { + p_current_port_group->node_type_ca = TRUE;; + } + ; + +node_type_switch: TK_NODE_TYPE_SWITCH { + p_current_port_group->node_type_switch = TRUE; + } + ; + +node_type_router: TK_NODE_TYPE_ROUTER { + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_all: TK_NODE_TYPE_ALL { + p_current_port_group->node_type_ca = TRUE; + p_current_port_group->node_type_switch = TRUE; + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_self: TK_NODE_TYPE_SELF { + p_current_port_group->node_type_self = TRUE; + } + ; + + /* + * vlarb_scope_entry values: + * vlarb_scope_group + * vlarb_scope_across + * vlarb_scope_vlarb_high + * vlarb_scope_vlarb_low + * vlarb_scope_vlarb_high_limit + */ + + + +vlarb_scope_group: vlarb_scope_group_start string_list { + /* 'group' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +vlarb_scope_across: vlarb_scope_across_start string_list { + /* 'across' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->across_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high_limit: vlarb_scope_vlarb_high_limit_start single_number { + /* 'vl-high-limit' in 'vlarb-scope' - one instance of one number */ + cl_list_iterator_t list_iterator; + uint64_t * p_tmp_num; + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_tmp_num = (uint64_t*)cl_list_obj(list_iterator); + if (p_tmp_num) + { + p_current_vlarb_scope->vl_high_limit = (uint32_t)(*p_tmp_num); + p_current_vlarb_scope->vl_high_limit_set = TRUE; + free(p_tmp_num); + } + + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +vlarb_scope_vlarb_high_limit_start: TK_VLARB_HIGH_LIMIT { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high: vlarb_scope_vlarb_high_start num_list_with_dotdot { + /* 'vlarb-high' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_high_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_high_start: TK_VLARB_HIGH { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_low: vlarb_scope_vlarb_low_start num_list_with_dotdot { + /* 'vlarb-low' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_low_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_low_start: TK_VLARB_LOW { + RESET_BUFFER; + } + ; + + /* + * sl2vl_scope_entry values: + * sl2vl_scope_group + * sl2vl_scope_across + * sl2vl_scope_across_from + * sl2vl_scope_across_to + * sl2vl_scope_from + * sl2vl_scope_to + * sl2vl_scope_sl2vl_table + */ + +sl2vl_scope_group: sl2vl_scope_group_start string_list { + /* 'group' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +sl2vl_scope_across: sl2vl_scope_across_start string_list { + /* 'across' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,strdup(tmp_str)); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_from: sl2vl_scope_across_from_start string_list { + /* 'across-from' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_from_start: TK_ACROSS_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_to: sl2vl_scope_across_to_start string_list { + /* 'across-to' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,tmp_str); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_to_start: TK_ACROSS_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from: sl2vl_scope_from_start sl2vl_scope_from_list_or_asterisk { + /* 'from' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_from_start: TK_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_to: sl2vl_scope_to_start sl2vl_scope_to_list_or_asterisk { + /* 'to' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_to_start: TK_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from_list_or_asterisk: sl2vl_scope_from_asterisk + | sl2vl_scope_from_list_of_ranges + ; + +sl2vl_scope_from_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + ; + +sl2vl_scope_to_list_or_asterisk: sl2vl_scope_to_asterisk + | sl2vl_scope_to_list_of_ranges + ; + +sl2vl_scope_to_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + ; + +sl2vl_scope_from_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + __qos_parser_error("port number out of range 'from' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +sl2vl_scope_to_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + __qos_parser_error("port number out of range 'to' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + + +sl2vl_scope_sl2vl_table: sl2vl_scope_sl2vl_table_start num_list { + /* 'sl2vl-table' - one instance of exactly + OSM_QOS_POLICY_SL2VL_TABLE_LEN numbers */ + cl_list_iterator_t list_iterator; + uint64_t num; + uint64_t * p_num; + int i = 0; + + if (p_current_sl2vl_scope->sl2vl_table_set) + { + __qos_parser_error("sl2vl-scope has more than one sl2vl-table"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + if (cl_list_count(&tmp_parser_struct.num_list) != OSM_QOS_POLICY_SL2VL_TABLE_LEN) + { + __qos_parser_error("wrong number of values in 'sl2vl-table' (should be 16)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_list) ) + { + p_num = (uint64_t*)cl_list_obj(list_iterator); + num = *p_num; + free(p_num); + if (num >= OSM_QOS_POLICY_MAX_VL_NUM) + { + __qos_parser_error("wrong VL value in 'sl2vl-table' (should be 0 to 15)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + p_current_sl2vl_scope->sl2vl_table[i++] = (uint8_t)num; + list_iterator = cl_list_next(list_iterator); + } + p_current_sl2vl_scope->sl2vl_table_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +sl2vl_scope_sl2vl_table_start: TK_SL2VL_TABLE { + RESET_BUFFER; + } + ; + + /* + * qos_level_entry values: + * qos_level_name + * qos_level_use + * qos_level_sl + * qos_level_mtu_limit + * qos_level_rate_limit + * qos_level_packet_life + * qos_level_path_bits + * qos_level_pkey + */ + +qos_level_name: qos_level_name_start single_string { + /* 'name' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->name) + { + __qos_parser_error("qos-level has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +qos_level_use: qos_level_use_start single_string { + /* 'use' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->use) + { + __qos_parser_error("qos-level has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_level_sl: qos_level_sl_start single_number { + /* 'sl' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->sl_set) + { + __qos_parser_error("'qos-level' has multiple 'sl' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->sl = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->sl_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_sl_start: TK_SL { + RESET_BUFFER; + } + ; + +qos_level_mtu_limit: qos_level_mtu_limit_start single_number { + /* 'mtu-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->mtu_limit_set) + { + __qos_parser_error("'qos-level' has multiple 'mtu-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->mtu_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->mtu_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_mtu_limit_start: TK_MTU_LIMIT { + /* 'mtu-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_rate_limit: qos_level_rate_limit_start single_number { + /* 'rate-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->rate_limit_set) + { + __qos_parser_error("'qos-level' has multiple 'rate-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->rate_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->rate_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_rate_limit_start: TK_RATE_LIMIT { + /* 'rate-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_packet_life: qos_level_packet_life_start single_number { + /* 'packet-life' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->pkt_life_set) + { + __qos_parser_error("'qos-level' has multiple 'packet-life' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->pkt_life = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->pkt_life_set= TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_packet_life_start: TK_PACKET_LIFE { + /* 'packet-life' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_path_bits: qos_level_path_bits_start list_of_ranges { + /* 'path-bits' in 'qos-level' - any num of instances */ + /* list of path bit ranges */ + + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->path_bits_range_len ) + { + p_current_qos_level->path_bits_range_arr = range_arr; + p_current_qos_level->path_bits_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->path_bits_range_arr, + p_current_qos_level->path_bits_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->path_bits_range_arr = new_range_arr; + p_current_qos_level->path_bits_range_len = new_range_len; + } + } + } + ; + +qos_level_path_bits_start: TK_PATH_BITS { + RESET_BUFFER; + } + ; + +qos_level_pkey: qos_level_pkey_start list_of_ranges { + /* 'pkey' in 'qos-level' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->pkey_range_len ) + { + p_current_qos_level->pkey_range_arr = range_arr; + p_current_qos_level->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->pkey_range_arr, + p_current_qos_level->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->pkey_range_arr = new_range_arr; + p_current_qos_level->pkey_range_len = new_range_len; + } + } + } + ; + +qos_level_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + /* + * qos_match_rule_entry values: + * qos_match_rule_use + * qos_match_rule_qos_class + * qos_match_rule_qos_level_name + * qos_match_rule_source + * qos_match_rule_destination + * qos_match_rule_service_id + * qos_match_rule_pkey + */ + + +qos_match_rule_use: qos_match_rule_use_start single_string { + /* 'use' of 'qos-match-rule' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->use) + { + __qos_parser_error("'qos-match-rule' has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_class: qos_match_rule_qos_class_start list_of_ranges { + /* 'qos-class' in 'qos-match-rule' - num of instances of list of ranges */ + /* list of class ranges (QoS Class is 12-bit value) */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->qos_class_range_len ) + { + p_current_qos_match_rule->qos_class_range_arr = range_arr; + p_current_qos_match_rule->qos_class_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->qos_class_range_arr, + p_current_qos_match_rule->qos_class_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->qos_class_range_arr = new_range_arr; + p_current_qos_match_rule->qos_class_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_qos_class_start: TK_QOS_CLASS { + RESET_BUFFER; + } + ; + +qos_match_rule_source: qos_match_rule_source_start string_list { + /* 'source' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->source_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_source_start: TK_SOURCE { + RESET_BUFFER; + } + ; + +qos_match_rule_destination: qos_match_rule_destination_start string_list { + /* 'destination' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->destination_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_destination_start: TK_DESTINATION { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_level_name: qos_match_rule_qos_level_name_start single_string { + /* 'qos-level-name' in 'qos-match-rule' - single string */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->qos_level_name) + { + __qos_parser_error("qos-match-rule has multiple 'qos-level-name' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->qos_level_name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_qos_level_name_start: TK_QOS_LEVEL_NAME { + RESET_BUFFER; + } + ; + +qos_match_rule_service_id: qos_match_rule_service_id_start list_of_ranges { + /* 'service-id' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->service_id_range_len ) + { + p_current_qos_match_rule->service_id_range_arr = range_arr; + p_current_qos_match_rule->service_id_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->service_id_range_arr, + p_current_qos_match_rule->service_id_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->service_id_range_arr = new_range_arr; + p_current_qos_match_rule->service_id_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_service_id_start: TK_SERVICE_ID { + RESET_BUFFER; + } + ; + +qos_match_rule_pkey: qos_match_rule_pkey_start list_of_ranges { + /* 'pkey' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->pkey_range_len ) + { + p_current_qos_match_rule->pkey_range_arr = range_arr; + p_current_qos_match_rule->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->pkey_range_arr, + p_current_qos_match_rule->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->pkey_range_arr = new_range_arr; + p_current_qos_match_rule->pkey_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + + /* + * Common part + */ + + +single_string: single_string_elems { + cl_list_insert_tail(&tmp_parser_struct.str_list, + strdup(__parser_strip_white(tmp_parser_struct.str))); + tmp_parser_struct.str[0] = '\0'; + } + ; + +single_string_elems: single_string_element + | single_string_elems single_string_element + ; + +single_string_element: TK_TEXT { + strcat(tmp_parser_struct.str,$1); + free($1); + } + ; + + +string_list: single_string + | string_list TK_COMMA single_string + ; + + + +single_number: number + ; + +num_list: number + | num_list TK_COMMA number + ; + +number: TK_NUMBER { + uint64_t * p_num = (uint64_t*)malloc(sizeof(uint64_t)); + __parser_str2uint64(p_num,$1); + free($1); + cl_list_insert_tail(&tmp_parser_struct.num_list, p_num); + } + ; + +num_list_with_dotdot: number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dotdot TK_COMMA number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +number_from_pair_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_pair_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +list_of_ranges: num_list_with_dash + ; + +num_list_with_dash: single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +single_number_from_range: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +number_from_range_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_range_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +%% + +/*************************************************** + ***************************************************/ + +int osm_qos_parse_policy_file( + IN osm_log_t * p_log, + IN const char * policy_file) +{ + int res = 0; + p_qos_parser_osm_log = p_log; + + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_parse_policy_file); + + p_qos_policy = NULL; + __qos_parser_in = fopen (policy_file, "r"); + if (!__qos_parser_in) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC01: " + "Failed opening QoS policy file (%s)\n", + policy_file); + res = 1; + goto Exit; + } + column_num = 1; + line_num = 1; + osm_qos_policy_create(); + __parser_tmp_struct_init(); + + res = __qos_parser_parse(); + + __parser_tmp_struct_destroy(); + + if (res != 0) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC02: " + "Failed parsing QoS policy file (%s)\n", + policy_file); + p_qos_policy = NULL; + res = 1; + goto Exit; + } + + if (osm_qos_policy_validate()) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse: ERR AC03: " + "Error(s) in QoS policy file (%s)\n", + policy_file); + res = 1; + goto Exit; + } + + Exit: + if (__qos_parser_in) + fclose(__qos_parser_in); + OSM_LOG_EXIT(p_qos_parser_osm_log); + return res; +} + +/*************************************************** + ***************************************************/ + +int __qos_parser_wrap() +{ + return(1); +} + +/*************************************************** + ***************************************************/ + +void __qos_parser_error (char *s) +{ + OSM_LOG_ENTER(p_qos_parser_osm_log, __qos_parser_error); + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "__qos_parser_error: ERR AC03: " + "Syntax error (line %d:%d): %s. " + "Last text read: \"%s\"\n", + line_num, column_num, s, __parser_strip_white(__qos_parser_text)); + OSM_LOG_EXIT(p_qos_parser_osm_log); +} + +/*************************************************** + ***************************************************/ + +static char * __parser_strip_white(char * str) +{ + int i; + for (i = (strlen(str)-1); i >= 0; i--) + { + if (isspace(str[i])) + str[i] = '\0'; + else + break; + } + for (i = 0; i < strlen(str); i++) + { + if (!isspace(str[i])) + break; + } + return &(str[i]); +} + +/*************************************************** + ***************************************************/ + +static void __parser_str2uint64(uint64_t * p_val, char * str) +{ +#if __WORDSIZE == 64 + *p_val = strtoul(str, NULL, 0); +#else + *p_val = strtoull(str, NULL, 0); +#endif +} + +/*************************************************** + ***************************************************/ + +static void __parser_port_group_start() +{ + p_current_port_group = osm_qos_policy_port_group_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_port_group_end() +{ + if(!p_current_port_group->name) + { + __qos_parser_error("port-group validation failed - no port group name specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->port_groups, + p_current_port_group); + p_current_port_group = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_vlarb_scope_start() +{ + p_current_vlarb_scope = osm_qos_policy_vlarb_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_vlarb_scope_end() +{ + if ( !cl_list_count(&p_current_vlarb_scope->group_list) && + !cl_list_count(&p_current_vlarb_scope->across_list) ) + { + __qos_parser_error("vlarb-scope validation failed - no port groups specified by 'group' or by 'across'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->vlarb_tables, + p_current_vlarb_scope); + p_current_vlarb_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_sl2vl_scope_start() +{ + p_current_sl2vl_scope = osm_qos_policy_sl2vl_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_sl2vl_scope_end() +{ + if (!p_current_sl2vl_scope->sl2vl_table_set) + { + __qos_parser_error("sl2vl-scope validation failed - no sl2vl table specified"); + return -1; + } + if ( !cl_list_count(&p_current_sl2vl_scope->group_list) && + !cl_list_count(&p_current_sl2vl_scope->across_to_list) && + !cl_list_count(&p_current_sl2vl_scope->across_from_list) ) + { + __qos_parser_error("sl2vl-scope validation failed - no port groups specified by 'group', 'across-to' or 'across-from'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->sl2vl_tables, + p_current_sl2vl_scope); + p_current_sl2vl_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_qos_level_start() +{ + p_current_qos_level = osm_qos_policy_qos_level_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_qos_level_end() +{ + if (!p_current_qos_level->sl_set) + { + __qos_parser_error("qos-level validation failed - no 'sl' specified"); + return -1; + } + if (!p_current_qos_level->name) + { + __qos_parser_error("qos-level validation failed - no 'name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_levels, + p_current_qos_level); + p_current_qos_level = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_match_rule_start() +{ + p_current_qos_match_rule = osm_qos_policy_match_rule_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_match_rule_end() +{ + if (!p_current_qos_match_rule->qos_level_name) + { + __qos_parser_error("match-rule validation failed - no 'qos-level-name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_match_rules, + p_current_qos_match_rule); + p_current_qos_match_rule = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_init() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_construct(&tmp_parser_struct.str_list); + cl_list_init(&tmp_parser_struct.str_list, 10); + cl_list_construct(&tmp_parser_struct.num_list); + cl_list_init(&tmp_parser_struct.num_list, 10); + cl_list_construct(&tmp_parser_struct.num_pair_list); + cl_list_init(&tmp_parser_struct.num_pair_list, 10); +} + +/*************************************************** + ***************************************************/ + +/* + * Do NOT free objects from the temp struct. + * Either they are inserted into the parse tree data + * structure, or they are already freed when copying + * their values to the parse tree data structure. + */ +static void __parser_tmp_struct_reset() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_remove_all(&tmp_parser_struct.str_list); + cl_list_remove_all(&tmp_parser_struct.num_list); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_destroy() +{ + __parser_tmp_struct_reset(); + cl_list_destroy(&tmp_parser_struct.str_list); + cl_list_destroy(&tmp_parser_struct.num_list); + cl_list_destroy(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static int OSM_CDECL +__cmp_num_range( + const void * p1, + const void * p2) +{ + uint64_t * pair1 = *((uint64_t **)p1); + uint64_t * pair2 = *((uint64_t **)p2); + + if (pair1[0] < pair2[0]) + return -1; + if (pair1[0] > pair2[0]) + return 1; + + if (pair1[1] < pair2[1]) + return -1; + if (pair1[1] > pair2[1]) + return 1; + + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __sort_reduce_rangearr( + uint64_t ** arr, + unsigned arr_len, + uint64_t ** * p_res_arr, + unsigned * p_res_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned last_valid_ind = 0; + unsigned valid_cnt = 0; + uint64_t ** res_arr; + boolean_t * is_valir_arr; + + *p_res_arr = NULL; + *p_res_arr_len = 0; + + qsort(arr, arr_len, sizeof(uint64_t*), __cmp_num_range); + + is_valir_arr = (boolean_t *)malloc(arr_len * sizeof(boolean_t)); + is_valir_arr[last_valid_ind] = TRUE; + valid_cnt++; + for (i = 1; i < arr_len; i++) + { + if (arr[i][0] <= arr[last_valid_ind][1]) + { + if (arr[i][1] > arr[last_valid_ind][1]) + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else if ((arr[i][0] - 1) == arr[last_valid_ind][1]) + { + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else + { + is_valir_arr[i] = TRUE; + last_valid_ind = i; + valid_cnt++; + } + } + + res_arr = (uint64_t **)malloc(valid_cnt * sizeof(uint64_t *)); + for (i = 0; i < arr_len; i++) + { + if (is_valir_arr[i]) + res_arr[j++] = arr[i]; + } + free(arr); + + *p_res_arr = res_arr; + *p_res_arr_len = valid_cnt; +} + +/*************************************************** + ***************************************************/ + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len) +{ + cl_list_iterator_t list_iterator; + unsigned len = cl_list_count(p_list); + unsigned i = 0; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + list_iterator = cl_list_head(p_list); + while( list_iterator != cl_list_end(p_list) ) + { + tmp_arr[i++] = (uint64_t *)cl_list_obj(list_iterator); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(p_list); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned len = range_len_1 + range_len_2; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + *p_arr = NULL; + *p_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + for (i = 0; i < range_len_1; i++) + tmp_arr[j++] = range_arr_1[i]; + for (i = 0; i < range_len_2; i++) + tmp_arr[j++] = range_arr_2[i]; + free(range_arr_1); + free(range_arr_2); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + -- 1.5.1.4 From sashak at voltaire.com Thu Aug 23 02:56:15 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 12:56:15 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <46CD411E.1080809@dev.mellanox.co.il> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> Message-ID: <20070823095615.GP1397@sashak.voltaire.com> On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_port_group_t_ { > >> + char *name; /* single string (this port group name) */ > >> + char *use; /* single string (description) */ > >> + cl_list_t port_name_list; /* list of port names (.../.../...) */ > >> + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit > >> guids) */ > >> + unsigned guid_range_len; /* num of guid ranges in the array */ > >> + cl_list_t partition_list; /* list of partition names */ > >> + boolean_t node_type_ca; > >> + boolean_t node_type_switch; > >> + boolean_t node_type_router; > >> + boolean_t node_type_self; > >> +} osm_qos_port_group_t; > > I see you are using this in "run-time", not just during the parsing. > > Instead of having all this config features you can just resolve port > > guids in parse time and keep it here in cl_map() for fast searches. > > By saying "config features", do you mean the four boolean flags? No, I mean everything except name and use. > It looks to me that checking the type of node is as fast as it gets, > and it won't hurt to leave these booleans instead of resolving > all the guids. It could be optimization when types are specified, which is not always the case and then you are going to do linear searches over all lists. And how something like "for each guid in this group" (which is needed for QoS port parameters setup) should be resolved? By matching each guid in the subnet against those lists? > Moreover, the guids here are stored in range array, which is IMO > better suited for the policy file syntax, because if a user specifies > something like this "0x0-0x0FFF" in guids, it will be only one element > of the array, which is efficient both in memory and in serch time. And what should be there if user specifies ports in the group as: guid1, guid2, guid3, etc. ? > (I probably should mention here that the efficient search in the range > array is not implemented yet, but it would be a simple binary search - > there's a "todo" comment in the search function right now) > > >> + > >> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > >> +void osm_qos_policy_port_group_destroy(); > > Would be nice to have function prototypes in one place. > >> + > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_vlarb_scope_t_ { > >> + cl_list_t group_list; /* list of group names (strings) */ > >> + cl_list_t across_list; /* list of 'across' group names (strings) */ > >> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values > >> */ > >> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values > >> */ > > Why cl_list for VLArb? it should be short fixed length arrays? > > Right. > Since the actual VLArb setup is not implemented yet, I didn't see > this obvious thing. But it should be implemented. Right? > >> + uint32_t vl_high_limit; /* single integer */ > >> + boolean_t vl_high_limit_set; > >> +} osm_qos_vlarb_scope_t; > >> + > >> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); > >> +void osm_qos_policy_vlarb_scope_destroy(); > >> + > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_sl2vl_scope_t_ { > >> + cl_list_t group_list; /* list of strings (port group names) */ > >> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > >> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > >> + cl_list_t across_from_list; /* list of strings (port group names) */ > >> + cl_list_t across_to_list; /* list of strings (port group names) */ > >> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > >> + boolean_t sl2vl_table_set; > >> +} osm_qos_sl2vl_scope_t; > > This will be used for sl2vl setup? Same as above - why not to generate > > final port guid list just in parse time? > >> + > >> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); > >> +void osm_qos_policy_sl2vl_scope_destroy(); > >> + > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_level_t_ { > >> + char *use; > >> + char *name; > >> + uint8_t sl; > >> + boolean_t sl_set; > >> + uint8_t mtu_limit; > >> + boolean_t mtu_limit_set; > >> + uint8_t rate_limit; > >> + boolean_t rate_limit_set; > >> + uint8_t pkt_life; > >> + boolean_t pkt_life_set; > >> + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are > >> 32bits) */ > >> + unsigned path_bits_range_len; /* num of bit ranges in the array */ > >> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are > >> 16bits) */ > >> + unsigned pkey_range_len; > >> +} osm_qos_level_t; > >> + > >> +osm_qos_level_t *osm_qos_policy_qos_level_create(); > >> +void osm_qos_policy_qos_level_destroy(); > >> + > >> +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > >> + IN ib_net16_t pkey); > >> + > >> +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * > >> p_qos_level, > >> + IN const osm_physp_t * p_src_physp, > >> + IN const osm_physp_t * p_dest_physp); > >> + > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_match_rule_t_ { > >> + char *use; > >> + cl_list_t source_list; /* list of strings */ > >> + cl_list_t source_group_list; /* list of pointers to relevant port-group > >> */ > >> + cl_list_t destination_list; /* list of strings */ > >> + cl_list_t destination_group_list; /* list of pointers to relevant > >> port-group */ > > I think you should only keep port guids there (mapped for fast searches). > > Same as above. > I think that checking node type and then guid range array is > essentially faster than checking guid map. _Only_ for case when the group is specified by type, which is likely will not be typical. > You might say, of course, that there can be many port groups > in the same match rule, but I don't see this as a practical > example. Of course it could (or you must disable multi groups here, which is not good idea I think and not what was presented in the RFC). Also each group still have this "linear searchable" lists: cl_list_t port_name_list; /* list of port names (.../.../...) */ uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit cl_list_t partition_list; /* list of partition names */ > >> + char *qos_level_name; > >> + osm_qos_level_t *p_qos_level; > > Why do you need qos_level_name if you keep the pointer to this qos_level > > struct? > > In policy file the match rule might appear before the QoS levels, > so matching qos level names to the actual qos levels is done when > the parsing is done. > > >> + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) > >> */ > >> + unsigned service_id_range_len; > >> + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real > >> values are 16bits) */ > >> + unsigned qos_class_range_len; > >> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are > >> 16bits) */ > >> + unsigned pkey_range_len; > >> +} osm_qos_match_rule_t; > >> + > >> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > >> +void osm_qos_policy_match_rule_destroy(); > >> + > >> +/***************************************************/ > >> + > >> +typedef struct osm_qos_policy_t_ { > >> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > >> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > >> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > >> + cl_list_t qos_levels; /* list of osm_qos_level_t */ > >> + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ > >> + osm_qos_level_t *p_default_qos_level; /* default QoS level */ > >> +} osm_qos_policy_t; > >> + > >> +void osm_qos_policy_create(); > >> +void osm_qos_policy_destroy(); > >> +int osm_qos_policy_validate(); > >> + > >> +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, > >> + IN const ib_path_rec_t * p_pr, > >> + IN const osm_physp_t * p_src_physp, > >> + IN const osm_physp_t * p_dest_physp, > >> + IN ib_net64_t comp_mask, > >> + OUT osm_qos_level_t ** pp_qos_level); > >> + > >> +/***************************************************/ > >> + > >> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char > >> *policy_file); > >> + > >> +/***************************************************/ > >> + > >> +#endif /* ifndef OSM_QOS_POLICY_H */ > >> diff --git a/opensm/opensm/osm_qos_policy.c > >> b/opensm/opensm/osm_qos_policy.c > >> new file mode 100644 > >> index 0000000..bc2aa68 > >> --- /dev/null > >> +++ b/opensm/opensm/osm_qos_policy.c > >> @@ -0,0 +1,901 @@ > >> +/* > >> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > >> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights > >> reserved. > >> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > >> + * > >> + * This software is available to you under a choice of one of two > >> + * licenses. You may choose to be licensed under the terms of the GNU > >> + * General Public License (GPL) Version 2, available from the file > >> + * COPYING in the main directory of this source tree, or the > >> + * OpenIB.org BSD license below: > >> + * > >> + * Redistribution and use in source and binary forms, with or > >> + * without modification, are permitted provided that the following > >> + * conditions are met: > >> + * > >> + * - Redistributions of source code must retain the above > >> + * copyright notice, this list of conditions and the following > >> + * disclaimer. > >> + * > >> + * - Redistributions in binary form must reproduce the above > >> + * copyright notice, this list of conditions and the following > >> + * disclaimer in the documentation and/or other materials > >> + * provided with the distribution. > >> + * > >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > >> + * SOFTWARE. > >> + * > >> + */ > >> + > >> +/* > >> + * Abstract: > >> + * OSM QoS Policy functions. > >> + * > >> + * Environment: > >> + * Linux User Mode > >> + * > >> + * Author: > >> + * Yevgeny Kliteynik, Mellanox > >> + */ > >> + > >> +#include > >> +#include > >> +#include > >> + > >> +extern void yyerror(char *s); > >> +osm_log_t *p_qos_parser_osm_log = NULL; > >> +osm_qos_policy_t *p_qos_policy = NULL; > > Please try to avoid globals - keep it as part of osm_opensm_t or > > osm_subn_t structures. > > I thought about it, but didn't want to "condaminate" the osm_opensm_t > or osm_subn_t structures untill the QoS functionality is ready. How globals are better in this sense? Sasha From vlad at lists.openfabrics.org Thu Aug 23 03:26:36 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 23 Aug 2007 03:26:36 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070823-0200 daily build status Message-ID: <20070823102636.9B428E60872@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1294: error: 'struct net_device' has no member named 'dev' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1296: error: 'struct net_device' has no member named 'dev' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:1298: error: 'struct net_device' has no member named 'dev' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband/ulp/ipoib/ipoib_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband/ulp/ipoib] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-22.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:739: error: implicit declaration of function 'skb_is_gso' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c: In function 'ipoib_neigh_cleanup': /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.c:872: warning: ISO C90 forbids mixed declarations and code make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib/ipoib_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband/ulp/ipoib] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-22.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-22.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-42.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-42.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-42.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-55.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-55.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-55.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on x86_64 with linux-2.6.9-34.ELsmp Log: /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1845: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1846: error: incompatible type for argument 1 of 'crypto_free_tfm' /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1847: error: used struct type value where scalar is required /home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.c:1848: error: incompatible type for argument 1 of 'crypto_free_tfm' make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi/iscsi_tcp.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/scsi] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070823-0200_linux-2.6.9-34.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-34.ELsmp' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From kliteyn at dev.mellanox.co.il Thu Aug 23 04:17:21 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 14:17:21 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070823095615.GP1397@sashak.voltaire.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> Message-ID: <46CD6CC1.30403@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_port_group_t_ { >>>> + char *name; /* single string (this port group name) */ >>>> + char *use; /* single string (description) */ >>>> + cl_list_t port_name_list; /* list of port names (.../.../...) */ >>>> + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit >>>> guids) */ >>>> + unsigned guid_range_len; /* num of guid ranges in the array */ >>>> + cl_list_t partition_list; /* list of partition names */ >>>> + boolean_t node_type_ca; >>>> + boolean_t node_type_switch; >>>> + boolean_t node_type_router; >>>> + boolean_t node_type_self; >>>> +} osm_qos_port_group_t; >>> I see you are using this in "run-time", not just during the parsing. >>> Instead of having all this config features you can just resolve port >>> guids in parse time and keep it here in cl_map() for fast searches. >> By saying "config features", do you mean the four boolean flags? > > No, I mean everything except name and use. > >> It looks to me that checking the type of node is as fast as it gets, >> and it won't hurt to leave these booleans instead of resolving >> all the guids. > > It could be optimization when types are specified, which is not always > the case and then you are going to do linear searches over all lists. Right, it would be linear scanning of list of partitions. > And how something like "for each guid in this group" (which is needed > for QoS port parameters setup) should be resolved? By matching each guid > in the subnet against those lists? Good point. >> Moreover, the guids here are stored in range array, which is IMO >> better suited for the policy file syntax, because if a user specifies >> something like this "0x0-0x0FFF" in guids, it will be only one element >> of the array, which is efficient both in memory and in serch time. > > And what should be there if user specifies ports in the group as: > guid1, guid2, guid3, etc. ? The range array is sorted and "shrinked". That is, if a user specifies guids as "30,1,2,3-20,15", eventually you would get two elements in the array: 1-20 and 30-30. cl_map implemented as a binary tree, right? And doing binary search in this kind of array is faster than searching a guid in cl_map of all the guids. Worst case - you will get the same performance as in case of cl_map if *all* the guids are "discreet" and can't be groupped in ranges. Nevertheless, I agree that there's a problem if there are many partitions in the list (although I'm not sure it's a practical case) I'll work on this. >> (I probably should mention here that the efficient search in the range >> array is not implemented yet, but it would be a simple binary search - >> there's a "todo" comment in the search function right now) >> >>>> + >>>> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); >>>> +void osm_qos_policy_port_group_destroy(); >>> Would be nice to have function prototypes in one place. >>>> + >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_vlarb_scope_t_ { >>>> + cl_list_t group_list; /* list of group names (strings) */ >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values >>>> */ >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values >>>> */ >>> Why cl_list for VLArb? it should be short fixed length arrays? >> Right. >> Since the actual VLArb setup is not implemented yet, I didn't see >> this obvious thing. > > But it should be implemented. Right? Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. >>>> + uint32_t vl_high_limit; /* single integer */ >>>> + boolean_t vl_high_limit_set; >>>> +} osm_qos_vlarb_scope_t; >>>> + >>>> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); >>>> +void osm_qos_policy_vlarb_scope_destroy(); >>>> + >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_sl2vl_scope_t_ { >>>> + cl_list_t group_list; /* list of strings (port group names) */ >>>> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; >>>> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; >>>> + cl_list_t across_from_list; /* list of strings (port group names) */ >>>> + cl_list_t across_to_list; /* list of strings (port group names) */ >>>> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ >>>> + boolean_t sl2vl_table_set; >>>> +} osm_qos_sl2vl_scope_t; >>> This will be used for sl2vl setup? Same as above - why not to generate >>> final port guid list just in parse time? >>>> + >>>> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); >>>> +void osm_qos_policy_sl2vl_scope_destroy(); >>>> + >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_level_t_ { >>>> + char *use; >>>> + char *name; >>>> + uint8_t sl; >>>> + boolean_t sl_set; >>>> + uint8_t mtu_limit; >>>> + boolean_t mtu_limit_set; >>>> + uint8_t rate_limit; >>>> + boolean_t rate_limit_set; >>>> + uint8_t pkt_life; >>>> + boolean_t pkt_life_set; >>>> + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are >>>> 32bits) */ >>>> + unsigned path_bits_range_len; /* num of bit ranges in the array */ >>>> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are >>>> 16bits) */ >>>> + unsigned pkey_range_len; >>>> +} osm_qos_level_t; >>>> + >>>> +osm_qos_level_t *osm_qos_policy_qos_level_create(); >>>> +void osm_qos_policy_qos_level_destroy(); >>>> + >>>> +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, >>>> + IN ib_net16_t pkey); >>>> + >>>> +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * >>>> p_qos_level, >>>> + IN const osm_physp_t * p_src_physp, >>>> + IN const osm_physp_t * p_dest_physp); >>>> + >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_match_rule_t_ { >>>> + char *use; >>>> + cl_list_t source_list; /* list of strings */ >>>> + cl_list_t source_group_list; /* list of pointers to relevant port-group >>>> */ >>>> + cl_list_t destination_list; /* list of strings */ >>>> + cl_list_t destination_group_list; /* list of pointers to relevant >>>> port-group */ >>> I think you should only keep port guids there (mapped for fast searches). >> Same as above. >> I think that checking node type and then guid range array is >> essentially faster than checking guid map. > > _Only_ for case when the group is specified by type, which is likely > will not be typical. Again, in *worst* case you will get the same performance in searching range array as in searching cl_map (which is a binary tree). >> You might say, of course, that there can be many port groups >> in the same match rule, but I don't see this as a practical >> example. > > Of course it could (or you must disable multi groups here, which is > not good idea I think and not what was presented in the RFC). Can you elaborate on this? Do you think that having multiple groups is not useful at all? > Also each group still have this "linear searchable" lists: > > cl_list_t port_name_list; /* list of port names (.../.../...) */ Port names are not implemented yet :) There's a "todo" comment in the code. This is the only keyword in the parser that is not implemented - I mean it's parsed, but other than creating this list the parser doesn't do anything with it. But It definitely won't stay as a pure list. > uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit This is not just a list, as I've explained before. > cl_list_t partition_list; /* list of partition names */ I can hardly believe than there will be more than one or two partitions in this list. Do you think otherwise? Anyway, I can extract all the guids and prepare a map. This is certainly the easiest implementation. And I still think that on average, guid ranges and partition lists are better, but who knows - perhaps someone would want to define a bunch of partitions in a single port group and ruin my average... :) >>>> + char *qos_level_name; >>>> + osm_qos_level_t *p_qos_level; >>> Why do you need qos_level_name if you keep the pointer to this qos_level >>> struct? >> In policy file the match rule might appear before the QoS levels, >> so matching qos level names to the actual qos levels is done when >> the parsing is done. >> >>>> + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) >>>> */ >>>> + unsigned service_id_range_len; >>>> + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real >>>> values are 16bits) */ >>>> + unsigned qos_class_range_len; >>>> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are >>>> 16bits) */ >>>> + unsigned pkey_range_len; >>>> +} osm_qos_match_rule_t; >>>> + >>>> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); >>>> +void osm_qos_policy_match_rule_destroy(); >>>> + >>>> +/***************************************************/ >>>> + >>>> +typedef struct osm_qos_policy_t_ { >>>> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ >>>> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ >>>> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ >>>> + cl_list_t qos_levels; /* list of osm_qos_level_t */ >>>> + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ >>>> + osm_qos_level_t *p_default_qos_level; /* default QoS level */ >>>> +} osm_qos_policy_t; >>>> + >>>> +void osm_qos_policy_create(); >>>> +void osm_qos_policy_destroy(); >>>> +int osm_qos_policy_validate(); >>>> + >>>> +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, >>>> + IN const ib_path_rec_t * p_pr, >>>> + IN const osm_physp_t * p_src_physp, >>>> + IN const osm_physp_t * p_dest_physp, >>>> + IN ib_net64_t comp_mask, >>>> + OUT osm_qos_level_t ** pp_qos_level); >>>> + >>>> +/***************************************************/ >>>> + >>>> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char >>>> *policy_file); >>>> + >>>> +/***************************************************/ >>>> + >>>> +#endif /* ifndef OSM_QOS_POLICY_H */ >>>> diff --git a/opensm/opensm/osm_qos_policy.c >>>> b/opensm/opensm/osm_qos_policy.c >>>> new file mode 100644 >>>> index 0000000..bc2aa68 >>>> --- /dev/null >>>> +++ b/opensm/opensm/osm_qos_policy.c >>>> @@ -0,0 +1,901 @@ >>>> +/* >>>> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. >>>> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights >>>> reserved. >>>> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. >>>> + * >>>> + * This software is available to you under a choice of one of two >>>> + * licenses. You may choose to be licensed under the terms of the GNU >>>> + * General Public License (GPL) Version 2, available from the file >>>> + * COPYING in the main directory of this source tree, or the >>>> + * OpenIB.org BSD license below: >>>> + * >>>> + * Redistribution and use in source and binary forms, with or >>>> + * without modification, are permitted provided that the following >>>> + * conditions are met: >>>> + * >>>> + * - Redistributions of source code must retain the above >>>> + * copyright notice, this list of conditions and the following >>>> + * disclaimer. >>>> + * >>>> + * - Redistributions in binary form must reproduce the above >>>> + * copyright notice, this list of conditions and the following >>>> + * disclaimer in the documentation and/or other materials >>>> + * provided with the distribution. >>>> + * >>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, >>>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >>>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND >>>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS >>>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN >>>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN >>>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE >>>> + * SOFTWARE. >>>> + * >>>> + */ >>>> + >>>> +/* >>>> + * Abstract: >>>> + * OSM QoS Policy functions. >>>> + * >>>> + * Environment: >>>> + * Linux User Mode >>>> + * >>>> + * Author: >>>> + * Yevgeny Kliteynik, Mellanox >>>> + */ >>>> + >>>> +#include >>>> +#include >>>> +#include >>>> + >>>> +extern void yyerror(char *s); >>>> +osm_log_t *p_qos_parser_osm_log = NULL; >>>> +osm_qos_policy_t *p_qos_policy = NULL; >>> Please try to avoid globals - keep it as part of osm_opensm_t or >>> osm_subn_t structures. >> I thought about it, but didn't want to "condaminate" the osm_opensm_t >> or osm_subn_t structures untill the QoS functionality is ready. > > How globals are better in this sense? They're not :) The difference is that less files are modified. But I agree that the QoS policy should be part of osm_opensm_t or osm_subn_t. -- Yevgeny > Sasha > From john.leidel at gmail.com Thu Aug 23 04:08:01 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 23 Aug 2007 08:08:01 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <46CD41A0.70000@mellanox.co.il> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <46CD41A0.70000@mellanox.co.il> Message-ID: <27f776af0708230408t6766940dqe32a1a35f64ad13@mail.gmail.com> lspci output is :: 07:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev a0) Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) Flags: bus master, fast devsel, latency 0, IRQ 193 Memory at dc000000 (64-bit, non-prefetchable) [size=1M] Memory at d8000000 (64-bit, prefetchable) [size=8M] Memory at d0000000 (64-bit, prefetchable) [size=128M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 Loaded modules are : ib_ipoib 60440 0 ib_mthca 151056 0 ib_umad 19888 0 ib_ucm 21000 0 ib_uverbs 46000 1 ib_ucm ib_cm 40744 1 ib_ucm ib_sa 17684 2 ib_ipoib,ib_cm ib_mad 42408 4 ib_mthca,ib_umad,ib_cm,ib_sa ib_core 60032 8 ib_ipoib,ib_mthca,ib_umad,ib_ucm,ib_uverbs,ib_cm,ib_sa,ib_mad OFED config details from /etc/infiniband/ofed/ :: #!/bin/bash echo prefix=/usr/local/ofed echo Kernel=2.6.9-42.ELsmp echo echo "Configure options: --with-ipoib-mod --with-mthca-mod --with-sdp-mod --with-srp-mod --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-madeye-mod" echo cheers john On 8/23/07, Tziporet Koren wrote: > > John Leidel wrote: > > All, in adding two new blade centers full of machines to my existing > > cluster install, I'm getting the following errors in trying to run MPI > > jobs over MVAPICH :: > > > > libibverbs: Fatal: no infiniband class devices found. > > No IB device found > > > > > Please send the output of lspci -v for the IB card. > > Also - to see which kernel modules are loaded please run: lsmod | grep ib > > If you have OFED installed on the system you can find the prefix running > the script: /etc/infiniband/info > > The binaries are installed under /bin > > Tziporet > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Thu Aug 23 04:39:16 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 14:39:16 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <20070823055300.GL1397@sashak.voltaire.com> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> <20070823055300.GL1397@sashak.voltaire.com> Message-ID: <46CD71E4.5040007@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > Yevgeny, Hi again, > > On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: >> Compiling QoS policy file parser. >> >> C & H files are generated from Lex & Yacc files and the >> generated files are compiled. >> Generated files are not included in git, but they do get >> included in RPM as they are also created by 'make dist'. >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/include/Makefile.am | 2 ++ >> opensm/opensm/Makefile.am | 16 +++++++++++++++- >> opensm/opensm/configure.in | 2 ++ >> 3 files changed, 19 insertions(+), 1 deletions(-) >> >> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am >> index ea62507..b83205f 100644 >> --- a/opensm/include/Makefile.am >> +++ b/opensm/include/Makefile.am >> @@ -90,6 +90,8 @@ EXTRA_DIST = \ >> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ >> $(srcdir)/opensm/osm_perfmgr.h \ >> $(srcdir)/opensm/osm_perfmgr_db.h \ >> + $(srcdir)/opensm/osm_qos_policy.h \ >> + $(srcdir)/opensm/osm_qos_parser_y.h \ >> $(srcdir)/complib/cl_thread_osd.h \ >> $(srcdir)/complib/cl_packon.h \ >> $(srcdir)/complib/cl_atomic_osd.h \ >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >> index 6dfa824..31e09f7 100644 >> --- a/opensm/opensm/Makefile.am >> +++ b/opensm/opensm/Makefile.am >> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ >> osm_vl15intf.c osm_vl_arb_rcv.c \ >> st.c osm_perfmgr.c osm_perfmgr_db.c \ >> - osm_event_plugin.c osm_dump.c >> + osm_event_plugin.c osm_dump.c \ >> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >> + >> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ >> + >> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l >> + > > Should generated files be cleaned somewhere? Good question... Should they? I'm not sure. I wouldn't want someone who took the RPM with these files removing the generated files with something like "make clean", and then failing to compile opensm because he doesn't have lex or yacc. > >> if OSMV_OPENIB >> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 >> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 >> @@ -104,3 +112,9 @@ install-exec-hook: >> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ >> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ >> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so >> + >> +# generate c and h files from the lex and yacc files >> +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l >> + > > Will simpler 'dist-hook' like this > > dist-hook: osm_qos_parser_y.c osm_qos_parser_l.c > > be fine here? Good idea. I'll repost the patch. >> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in >> index afb20b3..a49538d 100644 >> --- a/opensm/opensm/configure.in >> +++ b/opensm/opensm/configure.in >> @@ -22,6 +22,8 @@ AC_PROG_INSTALL >> AC_PROG_LN_S >> AC_PROG_MAKE_SET >> AC_PROG_LIBTOOL >> +AM_PROG_LEX >> +AC_PROG_YACC > > With tarball where *.c files are generated already we will not need lex > and yacc (OTOH it is needed for "regular" (or "maintainer-mode") build). > Any idea how to avoid this dependencies for tarball builds? I read some stuff on the net, and looks like "configure" doesn't fail if AM_PROG_LEX and AC_PROG_YACC fail to find targets. But I didn't check it yet - didn't have time or machine w/o lex and yacc. -- Yevgeny > Sasha > From kliteyn at dev.mellanox.co.il Thu Aug 23 04:43:41 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 14:43:41 +0300 Subject: [ofa-general] [PATCH 5/7 V2] osm: QoS - compiling policy file parser Message-ID: <46CD72ED.5040503@dev.mellanox.co.il> Compiling QoS policy file parser. C & H files are generated from Lex & Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. Signed-off-by: Yevgeny Kliteynik --- opensm/include/Makefile.am | 2 ++ opensm/opensm/Makefile.am | 13 ++++++++++++- opensm/opensm/configure.in | 2 ++ 3 files changed, 16 insertions(+), 1 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..b83205f 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -90,6 +90,8 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ + $(srcdir)/opensm/osm_qos_policy.h \ + $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/complib/cl_thread_osd.h \ $(srcdir)/complib/cl_packon.h \ $(srcdir)/complib/cl_atomic_osd.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..0c181c0 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ osm_vl15intf.c osm_vl_arb_rcv.c \ st.c osm_perfmgr.c osm_perfmgr_db.c \ - osm_event_plugin.c osm_dump.c + osm_event_plugin.c osm_dump.c \ + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c + +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ + +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l + if OSMV_OPENIB opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 @@ -104,3 +112,6 @@ install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so + +# generate c and h files from the lex and yacc files +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in index afb20b3..a49538d 100644 --- a/opensm/opensm/configure.in +++ b/opensm/opensm/configure.in @@ -22,6 +22,8 @@ AC_PROG_INSTALL AC_PROG_LN_S AC_PROG_MAKE_SET AC_PROG_LIBTOOL +AM_PROG_LEX +AC_PROG_YACC dnl Checks for libraries -- 1.5.1.4 From bs at q-leap.de Thu Aug 23 04:45:20 2007 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 23 Aug 2007 13:45:20 +0200 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> Message-ID: <200708231345.20247.bs@q-leap.de> On Wednesday 22 August 2007 22:02:58 John Leidel wrote: > All, in adding two new blade centers full of machines to my existing > cluster install, I'm getting the following errors in trying to run MPI jobs > over MVAPICH :: > > libibverbs: Fatal: no infiniband class devices found. > No IB device found Yesterday I also had this problem, can you try "modprobe rdma_ucm" and/or "modprobe rdma_cm"? Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH From sashak at voltaire.com Thu Aug 23 05:03:25 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 15:03:25 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <46CD6CC1.30403@dev.mellanox.co.il> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> Message-ID: <20070823120325.GA15042@sashak.voltaire.com> On 14:17 Thu 23 Aug , Yevgeny Kliteynik wrote: > >>>> + boolean_t node_type_switch; > >>>> + boolean_t node_type_router; > >>>> + boolean_t node_type_self; > >>>> +} osm_qos_port_group_t; > >>> I see you are using this in "run-time", not just during the parsing. > >>> Instead of having all this config features you can just resolve port > >>> guids in parse time and keep it here in cl_map() for fast searches. > >> By saying "config features", do you mean the four boolean flags? > > No, I mean everything except name and use. > >> It looks to me that checking the type of node is as fast as it gets, > >> and it won't hurt to leave these booleans instead of resolving > >> all the guids. > > It could be optimization when types are specified, which is not always > > the case and then you are going to do linear searches over all lists. > > Right, it would be linear scanning of list of partitions. And each partition by name search is linear over all configured by OpenSM partitions (it is "parser-only" stuff in PM), when partition is resolved port guid will be matched using cl_map anyway. BTW partition name is optional as far as I remember. Not sure this is an issue however search by pkey value will be faster. > > And how something like "for each guid in this group" (which is needed > > for QoS port parameters setup) should be resolved? By matching each guid > > in the subnet against those lists? > > Good point. > > >> Moreover, the guids here are stored in range array, which is IMO > >> better suited for the policy file syntax, because if a user specifies > >> something like this "0x0-0x0FFF" in guids, it will be only one element > >> of the array, which is efficient both in memory and in serch time. > > And what should be there if user specifies ports in the group as: > > guid1, guid2, guid3, etc. ? > > The range array is sorted and "shrinked". > That is, if a user specifies guids as "30,1,2,3-20,15", eventually you > would get two elements in the array: 1-20 and 30-30. Generic case would be: 30, 1, 2, 16, 17, 23, etc. :) > cl_map implemented as a binary tree, right? Yes. > And doing binary search Binary search is not implemented yet... Currently it is linear. > in this kind of array is faster than searching > a guid in cl_map of all the guids. > Worst case - you will get the same performance as in case of cl_map if > *all* the guids are "discreet" and can't be groupped in ranges. Right. But why do you think it is not typical? (we will not request from user to re-burn GUIDs for each desired QoS configuration :)) > > Nevertheless, I agree that there's a problem if there are many partitions > in the list (although I'm not sure it's a practical case) > > I'll work on this. Thanks. > >>>> + cl_list_t group_list; /* list of group names (strings) */ > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > >>>> values */ > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > >>>> values */ > >>> Why cl_list for VLArb? it should be short fixed length arrays? > >> Right. > >> Since the actual VLArb setup is not implemented yet, I didn't see > >> this obvious thing. > > But it should be implemented. Right? > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. Then we will have two QoS managers in parallel, I don't like this too much. OTOH the integration should be easy enough (low level stuff is ready there), so I will not mind if it will be done in the week after feature freeze. > >>>> +typedef struct osm_qos_match_rule_t_ { > >>>> + char *use; > >>>> + cl_list_t source_list; /* list of strings */ > >>>> + cl_list_t source_group_list; /* list of pointers to relevant > >>>> port-group */ > >>>> + cl_list_t destination_list; /* list of strings */ > >>>> + cl_list_t destination_group_list; /* list of pointers to relevant > >>>> port-group */ > >>> I think you should only keep port guids there (mapped for fast searches). > >> Same as above. > >> I think that checking node type and then guid range array is > >> essentially faster than checking guid map. > > _Only_ for case when the group is specified by type, which is likely > > will not be typical. > > Again, in *worst* case you will get the same performance in searching > range array as in searching cl_map (which is a binary tree). > > >> You might say, of course, that there can be many port groups > >> in the same match rule, but I don't see this as a practical > >> example. > > Of course it could (or you must disable multi groups here, which is > > not good idea I think and not what was presented in the RFC). > > Can you elaborate on this? > Do you think that having multiple groups is not useful at all? I think it _is_ useful. I'm trying to say is that we cannot from one side to provide some functionality and from another expect that nobody will use it :). > > Also each group still have this "linear searchable" lists: > > cl_list_t port_name_list; /* list of port names (.../.../...) */ > > Port names are not implemented yet :) > There's a "todo" comment in the code. > This is the only keyword in the parser that is not implemented - I mean it's > parsed, > but other than creating this list the parser doesn't do anything with it. > But It definitely won't stay as a pure list. > > > uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit > > This is not just a list, as I've explained before. > > > cl_list_t partition_list; /* list of partition names */ > > I can hardly believe than there will be more than one or two partitions > in this list. Do you think otherwise? Why not? It is permitted and of course will be useful. Also as I stated above even for one partition by name search you will need to scan all configured partitions (so probably using pkey instead of partition name is better). > Anyway, I can extract all the guids and prepare a map. This is certainly the > easiest implementation. And I still think that on average, guid ranges Maybe with fast search, but not how it is now. > and > partition lists are better, Absolutely not, even with one partition. There are always two parts: Partition search (slow by name or fast by pkey) and guid search in partition (which is cl_map). Sasha From john.leidel at gmail.com Thu Aug 23 04:57:59 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 23 Aug 2007 08:57:59 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <200708231345.20247.bs@q-leap.de> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <200708231345.20247.bs@q-leap.de> Message-ID: <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> what do these modules control? I assume some sort of RDMA function... On 8/23/07, Bernd Schubert wrote: > > On Wednesday 22 August 2007 22:02:58 John Leidel wrote: > > All, in adding two new blade centers full of machines to my existing > > cluster install, I'm getting the following errors in trying to run MPI > jobs > > over MVAPICH :: > > > > libibverbs: Fatal: no infiniband class devices found. > > No IB device found > > Yesterday I also had this problem, can you try "modprobe rdma_ucm" > and/or "modprobe rdma_cm"? > > > Cheers, > Bernd > > -- > Bernd Schubert > Q-Leap Networks GmbH > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.leidel at gmail.com Thu Aug 23 05:02:29 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 23 Aug 2007 09:02:29 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <200708231345.20247.bs@q-leap.de> <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> Message-ID: <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> Unfortunately, the RDMA module load didn't help... a simple "hello_world" application still returns :: libibverbs: Fatal: no infiniband class devices found. No IB device found I went and verified that all the nodes see the HCAs... an lspci on all nodes reports :: 07:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev a0) Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) On 8/23/07, John Leidel wrote: > > what do these modules control? > I assume some sort of RDMA function... > > On 8/23/07, Bernd Schubert wrote: > > > > On Wednesday 22 August 2007 22:02:58 John Leidel wrote: > > > All, in adding two new blade centers full of machines to my existing > > > cluster install, I'm getting the following errors in trying to run MPI > > jobs > > > over MVAPICH :: > > > > > > libibverbs: Fatal: no infiniband class devices found. > > > No IB device found > > > > Yesterday I also had this problem, can you try "modprobe rdma_ucm" > > and/or "modprobe rdma_cm"? > > > > > > Cheers, > > Bernd > > > > -- > > Bernd Schubert > > Q-Leap Networks GmbH > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Aug 23 05:13:55 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 15:13:55 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <46CD71E4.5040007@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> <20070823055300.GL1397@sashak.voltaire.com> <46CD71E4.5040007@dev.mellanox.co.il> Message-ID: <20070823121355.GB15042@sashak.voltaire.com> On 14:39 Thu 23 Aug , Yevgeny Kliteynik wrote: > Hi Sasha, > > Sasha Khapyorsky wrote: > > Yevgeny, Hi again, > > On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: > >> Compiling QoS policy file parser. > >> > >> C & H files are generated from Lex & Yacc files and the generated files > >> are compiled. > >> Generated files are not included in git, but they do get > >> included in RPM as they are also created by 'make dist'. > >> > >> Signed-off-by: Yevgeny Kliteynik > >> --- > >> opensm/include/Makefile.am | 2 ++ > >> opensm/opensm/Makefile.am | 16 +++++++++++++++- > >> opensm/opensm/configure.in | 2 ++ > >> 3 files changed, 19 insertions(+), 1 deletions(-) > >> > >> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > >> index ea62507..b83205f 100644 > >> --- a/opensm/include/Makefile.am > >> +++ b/opensm/include/Makefile.am > >> @@ -90,6 +90,8 @@ EXTRA_DIST = \ > >> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > >> $(srcdir)/opensm/osm_perfmgr.h \ > >> $(srcdir)/opensm/osm_perfmgr_db.h \ > >> + $(srcdir)/opensm/osm_qos_policy.h \ > >> + $(srcdir)/opensm/osm_qos_parser_y.h \ > >> $(srcdir)/complib/cl_thread_osd.h \ > >> $(srcdir)/complib/cl_packon.h \ > >> $(srcdir)/complib/cl_atomic_osd.h \ > >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > >> index 6dfa824..31e09f7 100644 > >> --- a/opensm/opensm/Makefile.am > >> +++ b/opensm/opensm/Makefile.am > >> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > >> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > >> osm_vl15intf.c osm_vl_arb_rcv.c \ > >> st.c osm_perfmgr.c osm_perfmgr_db.c \ > >> - osm_event_plugin.c osm_dump.c > >> + osm_event_plugin.c osm_dump.c \ > >> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > >> + > >> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y > >> $(srcdir)/../include/opensm/osm_qos_policy.h > >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c > >> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h > >> --name-prefix=__qos_parser_ > >> + > >> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l > >> $(srcdir)/../include/opensm/osm_qos_policy.h > >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c > >> $(srcdir)/osm_qos_parser.l > >> + > > Should generated files be cleaned somewhere? > > Good question... > Should they? Of course - any generated stuff should be cleaned somehow. > I'm not sure. I wouldn't want someone who took the RPM with > these files removing the generated files with something like "make clean", > and then failing to compile opensm because he doesn't have lex or yacc. Sure, 'make clean' would be bad choice. But something like MAINTAINERCLEANFILES? > > >> if OSMV_OPENIB > >> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing > >> -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 > >> -D_BSD_SOURCE=1 > >> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT > >> -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > >> @@ -104,3 +112,9 @@ install-exec-hook: > >> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm > >> $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ > >> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ > >> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so > >> + > >> +# generate c and h files from the lex and yacc files > >> +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l > >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c > >> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h > >> --name-prefix=__qos_parser_ > >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c > >> $(srcdir)/osm_qos_parser.l > >> + > > Will simpler 'dist-hook' like this > > dist-hook: osm_qos_parser_y.c osm_qos_parser_l.c > > be fine here? > > Good idea. > I'll repost the patch. > > >> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in > >> index afb20b3..a49538d 100644 > >> --- a/opensm/opensm/configure.in > >> +++ b/opensm/opensm/configure.in > >> @@ -22,6 +22,8 @@ AC_PROG_INSTALL > >> AC_PROG_LN_S > >> AC_PROG_MAKE_SET > >> AC_PROG_LIBTOOL > >> +AM_PROG_LEX > >> +AC_PROG_YACC > > With tarball where *.c files are generated already we will not need lex > > and yacc (OTOH it is needed for "regular" (or "maintainer-mode") build). > > Any idea how to avoid this dependencies for tarball builds? > > I read some stuff on the net, and looks like "configure" doesn't fail if > AM_PROG_LEX and AC_PROG_YACC fail to find targets. > But I didn't check it yet - didn't have time or machine w/o lex and yacc. It is risky if it is not "official" feature. I would prefer to get build failure on my machine due to not installed lex, but not put extra dependencies for tarball builds. Probably it is better just to remove this deps. Sasha From bs at q-leap.de Thu Aug 23 05:18:16 2007 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 23 Aug 2007 14:18:16 +0200 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> Message-ID: <200708231418.16393.bs@q-leap.de> On Thursday 23 August 2007 14:02:29 you wrote: > what do these modules control? > I assume some sort of RDMA function... Don't ask me ;) All I know it that I have to load these modules manually. > Unfortunately, the RDMA module load didn't help... a simple "hello_world" > application still returns :: > > libibverbs: Fatal: no infiniband class devices found. > No IB device found Please try to test using the ofed tools first, 'dtest' always does the job for me. Besides loading these modules I also had to 'chmod a+rw /dev/infiniband/uverbs0', otherwise it only worked as root. Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH From kliteyn at dev.mellanox.co.il Thu Aug 23 05:39:08 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 23 Aug 2007 15:39:08 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <20070823121355.GB15042@sashak.voltaire.com> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> <20070823055300.GL1397@sashak.voltaire.com> <46CD71E4.5040007@dev.mellanox.co.il> <20070823121355.GB15042@sashak.voltaire.com> Message-ID: <46CD7FEC.2020407@dev.mellanox.co.il> Hi Sasha, Sasha Khapyorsky wrote: > On 14:39 Thu 23 Aug , Yevgeny Kliteynik wrote: >> Hi Sasha, >> >> Sasha Khapyorsky wrote: >>> Yevgeny, Hi again, >>> On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: >>>> Compiling QoS policy file parser. >>>> >>>> C & H files are generated from Lex & Yacc files and the generated files >>>> are compiled. >>>> Generated files are not included in git, but they do get >>>> included in RPM as they are also created by 'make dist'. >>>> >>>> Signed-off-by: Yevgeny Kliteynik >>>> --- >>>> opensm/include/Makefile.am | 2 ++ >>>> opensm/opensm/Makefile.am | 16 +++++++++++++++- >>>> opensm/opensm/configure.in | 2 ++ >>>> 3 files changed, 19 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am >>>> index ea62507..b83205f 100644 >>>> --- a/opensm/include/Makefile.am >>>> +++ b/opensm/include/Makefile.am >>>> @@ -90,6 +90,8 @@ EXTRA_DIST = \ >>>> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ >>>> $(srcdir)/opensm/osm_perfmgr.h \ >>>> $(srcdir)/opensm/osm_perfmgr_db.h \ >>>> + $(srcdir)/opensm/osm_qos_policy.h \ >>>> + $(srcdir)/opensm/osm_qos_parser_y.h \ >>>> $(srcdir)/complib/cl_thread_osd.h \ >>>> $(srcdir)/complib/cl_packon.h \ >>>> $(srcdir)/complib/cl_atomic_osd.h \ >>>> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >>>> index 6dfa824..31e09f7 100644 >>>> --- a/opensm/opensm/Makefile.am >>>> +++ b/opensm/opensm/Makefile.am >>>> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >>>> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ >>>> osm_vl15intf.c osm_vl_arb_rcv.c \ >>>> st.c osm_perfmgr.c osm_perfmgr_db.c \ >>>> - osm_event_plugin.c osm_dump.c >>>> + osm_event_plugin.c osm_dump.c \ >>>> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >>>> + >>>> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y >>>> $(srcdir)/../include/opensm/osm_qos_policy.h >>>> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c >>>> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h >>>> --name-prefix=__qos_parser_ >>>> + >>>> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l >>>> $(srcdir)/../include/opensm/osm_qos_policy.h >>>> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c >>>> $(srcdir)/osm_qos_parser.l >>>> + >>> Should generated files be cleaned somewhere? >> Good question... >> Should they? > > Of course - any generated stuff should be cleaned somehow. > >> I'm not sure. I wouldn't want someone who took the RPM with >> these files removing the generated files with something like "make clean", >> and then failing to compile opensm because he doesn't have lex or yacc. > > Sure, 'make clean' would be bad choice. But something like > MAINTAINERCLEANFILES? OK, I'll take care of this - I'll send in in a separate patch later. >>>> if OSMV_OPENIB >>>> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing >>>> -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 >>>> -D_BSD_SOURCE=1 >>>> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT >>>> -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 >>>> @@ -104,3 +112,9 @@ install-exec-hook: >>>> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm >>>> $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ >>>> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ >>>> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so >>>> + >>>> +# generate c and h files from the lex and yacc files >>>> +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l >>>> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c >>>> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h >>>> --name-prefix=__qos_parser_ >>>> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c >>>> $(srcdir)/osm_qos_parser.l >>>> + >>> Will simpler 'dist-hook' like this >>> dist-hook: osm_qos_parser_y.c osm_qos_parser_l.c >>> be fine here? >> Good idea. >> I'll repost the patch. >> >>>> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in >>>> index afb20b3..a49538d 100644 >>>> --- a/opensm/opensm/configure.in >>>> +++ b/opensm/opensm/configure.in >>>> @@ -22,6 +22,8 @@ AC_PROG_INSTALL >>>> AC_PROG_LN_S >>>> AC_PROG_MAKE_SET >>>> AC_PROG_LIBTOOL >>>> +AM_PROG_LEX >>>> +AC_PROG_YACC >>> With tarball where *.c files are generated already we will not need lex >>> and yacc (OTOH it is needed for "regular" (or "maintainer-mode") build). >>> Any idea how to avoid this dependencies for tarball builds? >> I read some stuff on the net, and looks like "configure" doesn't fail if >> AM_PROG_LEX and AC_PROG_YACC fail to find targets. >> But I didn't check it yet - didn't have time or machine w/o lex and yacc. > > It is risky if it is not "official" feature. I would prefer to get build > failure on my machine due to not installed lex, but not put extra > dependencies for tarball builds. Probably it is better just to remove > this deps. OK, agree. We will remove them till I have more info on this. Do you want me to repost the patch or you can just remove these lines as you apply it? -- Yevgeny > Sasha > From sashak at voltaire.com Thu Aug 23 05:45:10 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 15:45:10 +0300 Subject: [ofa-general] [PATCH] opensm: add --perfmgr to usage Message-ID: <20070823124510.GD15042@sashak.voltaire.com> Add PerfMgr options description to usage(). Signed-off-by: Sasha Khapyorsky --- opensm/opensm/main.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index 0827f1c..e2541d8 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -284,6 +284,12 @@ void show_usage(void) printf("-I\n" "--inactive\n" " Start SM in inactive rather than normal init SM state.\n\n"); +#ifdef ENABLE_OSM_PERF_MGR + printf("--perfmgr\n" + " Start with PerfMgr enabled.\n\n"); + printf("--perfmgr_sweep_time_s \n" + " PerfMgr sweep interval in seconds.\n\n"); +#endif printf("-v\n" "--verbose\n" " This option increases the log verbosity level.\n" -- 1.5.3.rc2.38.g11308 From tziporet at mellanox.co.il Thu Aug 23 05:55:16 2007 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Thu, 23 Aug 2007 15:55:16 +0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com><200708231345.20247.bs@q-leap.de><27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C901563C00@mtlexch01.mtl.com> John Leidel wrote: > Unfortunately, the RDMA module load didn't help... a simple "hello_world" application still returns :: > > libibverbs: Fatal: no infiniband class devices found. > No IB device found > > I went and verified that all the nodes see the HCAs... an lspci on all nodes reports :: > > 07:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev a0) > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) Can you run: /etc/init.d/openibd restart and send the /var/log/messages Thanks Tziporet -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Aug 23 05:59:51 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 15:59:51 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V2] osm: QoS - compiling policy file parser In-Reply-To: <46CD7FEC.2020407@dev.mellanox.co.il> References: <46C98428.1010607@dev.mellanox.co.il> <46CAD250.6000603@dev.mellanox.co.il> <20070823055300.GL1397@sashak.voltaire.com> <46CD71E4.5040007@dev.mellanox.co.il> <20070823121355.GB15042@sashak.voltaire.com> <46CD7FEC.2020407@dev.mellanox.co.il> Message-ID: <20070823125951.GE15042@sashak.voltaire.com> On 15:39 Thu 23 Aug , Yevgeny Kliteynik wrote: > Hi Sasha, > > Sasha Khapyorsky wrote: > > On 14:39 Thu 23 Aug , Yevgeny Kliteynik wrote: > >> Hi Sasha, > >> > >> Sasha Khapyorsky wrote: > >>> Yevgeny, Hi again, > >>> On 14:53 Tue 21 Aug , Yevgeny Kliteynik wrote: > >>>> Compiling QoS policy file parser. > >>>> > >>>> C & H files are generated from Lex & Yacc files and the generated files > >>>> are compiled. > >>>> Generated files are not included in git, but they do get > >>>> included in RPM as they are also created by 'make dist'. > >>>> > >>>> Signed-off-by: Yevgeny Kliteynik > >>>> --- > >>>> opensm/include/Makefile.am | 2 ++ > >>>> opensm/opensm/Makefile.am | 16 +++++++++++++++- > >>>> opensm/opensm/configure.in | 2 ++ > >>>> 3 files changed, 19 insertions(+), 1 deletions(-) > >>>> > >>>> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > >>>> index ea62507..b83205f 100644 > >>>> --- a/opensm/include/Makefile.am > >>>> +++ b/opensm/include/Makefile.am > >>>> @@ -90,6 +90,8 @@ EXTRA_DIST = \ > >>>> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > >>>> $(srcdir)/opensm/osm_perfmgr.h \ > >>>> $(srcdir)/opensm/osm_perfmgr_db.h \ > >>>> + $(srcdir)/opensm/osm_qos_policy.h \ > >>>> + $(srcdir)/opensm/osm_qos_parser_y.h \ > >>>> $(srcdir)/complib/cl_thread_osd.h \ > >>>> $(srcdir)/complib/cl_packon.h \ > >>>> $(srcdir)/complib/cl_atomic_osd.h \ > >>>> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > >>>> index 6dfa824..31e09f7 100644 > >>>> --- a/opensm/opensm/Makefile.am > >>>> +++ b/opensm/opensm/Makefile.am > >>>> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c > >>>> \ > >>>> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > >>>> osm_vl15intf.c osm_vl_arb_rcv.c \ > >>>> st.c osm_perfmgr.c osm_perfmgr_db.c \ > >>>> - osm_event_plugin.c osm_dump.c > >>>> + osm_event_plugin.c osm_dump.c \ > >>>> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > >>>> + > >>>> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y > >>>> $(srcdir)/../include/opensm/osm_qos_policy.h > >>>> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o > >>>> $(srcdir)/osm_qos_parser_y.c > >>>> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h > >>>> --name-prefix=__qos_parser_ > >>>> + > >>>> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l > >>>> $(srcdir)/../include/opensm/osm_qos_policy.h > >>>> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c > >>>> $(srcdir)/osm_qos_parser.l > >>>> + > >>> Should generated files be cleaned somewhere? > >> Good question... > >> Should they? > > Of course - any generated stuff should be cleaned somehow. > >> I'm not sure. I wouldn't want someone who took the RPM with > >> these files removing the generated files with something like "make > >> clean", > >> and then failing to compile opensm because he doesn't have lex or yacc. > > Sure, 'make clean' would be bad choice. But something like > > MAINTAINERCLEANFILES? > > OK, I'll take care of this - I'll send in in a separate patch later. > > >>>> if OSMV_OPENIB > >>>> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing > >>>> -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 > >>>> -D_BSD_SOURCE=1 > >>>> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT > >>>> -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > >>>> @@ -104,3 +112,9 @@ install-exec-hook: > >>>> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm > >>>> $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ > >>>> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; > >>>> \ > >>>> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so > >>>> + > >>>> +# generate c and h files from the lex and yacc files > >>>> +dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l > >>>> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o > >>>> $(srcdir)/osm_qos_parser_y.c > >>>> --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h > >>>> --name-prefix=__qos_parser_ > >>>> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c > >>>> $(srcdir)/osm_qos_parser.l > >>>> + > >>> Will simpler 'dist-hook' like this > >>> dist-hook: osm_qos_parser_y.c osm_qos_parser_l.c > >>> be fine here? > >> Good idea. > >> I'll repost the patch. > >> > >>>> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in > >>>> index afb20b3..a49538d 100644 > >>>> --- a/opensm/opensm/configure.in > >>>> +++ b/opensm/opensm/configure.in > >>>> @@ -22,6 +22,8 @@ AC_PROG_INSTALL > >>>> AC_PROG_LN_S > >>>> AC_PROG_MAKE_SET > >>>> AC_PROG_LIBTOOL > >>>> +AM_PROG_LEX > >>>> +AC_PROG_YACC > >>> With tarball where *.c files are generated already we will not need lex > >>> and yacc (OTOH it is needed for "regular" (or "maintainer-mode") build). > >>> Any idea how to avoid this dependencies for tarball builds? > >> I read some stuff on the net, and looks like "configure" doesn't fail if > >> AM_PROG_LEX and AC_PROG_YACC fail to find targets. > >> But I didn't check it yet - didn't have time or machine w/o lex and yacc. > > It is risky if it is not "official" feature. I would prefer to get build > > failure on my machine due to not installed lex, but not put extra > > dependencies for tarball builds. Probably it is better just to remove > > this deps. > > OK, agree. > We will remove them till I have more info on this. > Do you want me to repost the patch or you can just remove these lines > as you apply it? I can remove, but anyway this patch should be applied only after 2,3,4. I think '3' should be improved and content of '2' requires global stuff which is in '3'... '4' looks fine, but useless without the rest. So it is up to you - to repost this patch, drop subsequent patch or just remind me if I will forget to remove :). Sasha From bs at q-leap.de Thu Aug 23 06:46:59 2007 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 23 Aug 2007 15:46:59 +0200 Subject: [ofa-general] UPDN algorithm Message-ID: <200708231546.59345.bs@q-leap.de> Hi, I can't make opensm using the UPDN algorithm. Presently we have 3 switched connected with each other and one is supposed to be the root switch. I created an /etc/osm.root root at sn-1:~# cat /etc/osm.roots 0xb8cffff004377 No idea if the syntax is correct, the man page doesn't say anything about it. opensm is started this way /usr/bin/opensm -p 15 -u -a /etc/osm.roots -R updn -f /var/log/osm-port1.log Aug 23 15:38:24 558096 [41001950] -> osm_report_notice: Reporting Generic Notice type:3 num:66 from LID:0x0000 GID:0xfe8000 0000000000,0x0002c902002350f9 Aug 23 15:38:24 558117 [41001950] -> osm_report_notice: Reporting Generic Notice type:3 num:66 from LID:0x0000 GID:0xfe8000 0000000000,0x0002c902002350f9 Aug 23 15:38:24 606386 [42804950] -> __osm_trap_rcv_process_request: Received Generic Notice type:0x04 num:144 Producer:1 f rom LID:0x0004 TID:0x0000000000000009 Aug 23 15:38:24 606485 [42804950] -> osm_report_notice: Reporting Generic Notice type:4 num:144 from LID:0x0004 GID:0xfe800 00000000000,0x0002c902002350f9 Aug 23 15:38:24 759817 [42003950] -> __osm_updn_call: disabling UPDN algorithm, no root nodes were found Aug 23 15:38:24 759823 [42003950] -> osm_ucast_mgr_process: updn tables configured on all switches Aug 23 15:38:24 777747 [42804950] -> SUBNET UP Log with full debugging enabled is attached. Thanks in advance, Bernd -- Bernd Schubert Q-Leap Networks GmbH -------------- next part -------------- A non-text attachment was scrubbed... Name: osm-port1.log.gz Type: application/x-gzip Size: 343125 bytes Desc: not available URL: From sashak at voltaire.com Thu Aug 23 07:46:16 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 17:46:16 +0300 Subject: [ofa-general] UPDN algorithm In-Reply-To: <200708231546.59345.bs@q-leap.de> References: <200708231546.59345.bs@q-leap.de> Message-ID: <20070823144616.GG15042@sashak.voltaire.com> Hi, On 15:46 Thu 23 Aug , Bernd Schubert wrote: > > I can't make opensm using the UPDN algorithm. Presently we have 3 switched > connected with each other and one is supposed to be the root switch. > > I created an /etc/osm.root > > root at sn-1:~# cat /etc/osm.roots > 0xb8cffff004377 > > No idea if the syntax is correct, the man page doesn't say anything about it. > > opensm is started this way > > /usr/bin/opensm -p 15 -u -a /etc/osm.roots -R updn -f /var/log/osm-port1.log Hmm, what '-u' does? According to the log it looks that OpenSM ignores '-a' option. Sasha > > > > Aug 23 15:38:24 558096 [41001950] -> osm_report_notice: Reporting Generic > Notice type:3 num:66 from LID:0x0000 GID:0xfe8000 > 0000000000,0x0002c902002350f9 > Aug 23 15:38:24 558117 [41001950] -> osm_report_notice: Reporting Generic > Notice type:3 num:66 from LID:0x0000 GID:0xfe8000 > 0000000000,0x0002c902002350f9 > Aug 23 15:38:24 606386 [42804950] -> __osm_trap_rcv_process_request: Received > Generic Notice type:0x04 num:144 Producer:1 f > rom LID:0x0004 TID:0x0000000000000009 > Aug 23 15:38:24 606485 [42804950] -> osm_report_notice: Reporting Generic > Notice type:4 num:144 from LID:0x0004 GID:0xfe800 > 00000000000,0x0002c902002350f9 > Aug 23 15:38:24 759817 [42003950] -> __osm_updn_call: disabling UPDN > algorithm, no root nodes were found > Aug 23 15:38:24 759823 [42003950] -> osm_ucast_mgr_process: updn tables > configured on all switches > Aug 23 15:38:24 777747 [42804950] -> SUBNET UP > > > Log with full debugging enabled is attached. > > > > Thanks in advance, > Bernd > > > -- > Bernd Schubert > Q-Leap Networks GmbH > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From john.leidel at gmail.com Thu Aug 23 08:49:47 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 23 Aug 2007 12:49:47 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563C00@mtlexch01.mtl.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <200708231345.20247.bs@q-leap.de> <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> <6C2C79E72C305246B504CBA17B5500C901563C00@mtlexch01.mtl.com> Message-ID: <27f776af0708230849l6b3ab898uf7f3d114b0206f51@mail.gmail.com> Whats especially odd is that I can get a full bandwidth ping pong test running fine [970MB/s++], then rerun the test and have it fail saying it can't find the IB HCA. On 8/23/07, Tziporet Koren wrote: > > John Leidel wrote: > > > Unfortunately, the RDMA module load didn't help... a simple > "hello_world" application still returns :: > > > > libibverbs: Fatal: no infiniband class devices found. > > No IB device found > > > > I went and verified that all the nodes see the HCAs... an lspci on all > nodes reports :: > > > > 07:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > (Tavor compatibility mode) (rev a0) > > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor > compatibility mode) > > Can you run: > /etc/init.d/openibd restart > and send the /var/log/messages > > Thanks > Tziporet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bs at q-leap.de Thu Aug 23 09:07:11 2007 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 23 Aug 2007 18:07:11 +0200 Subject: [ofa-general] UPDN algorithm In-Reply-To: <20070823144616.GG15042@sashak.voltaire.com> References: <200708231546.59345.bs@q-leap.de> <20070823144616.GG15042@sashak.voltaire.com> Message-ID: <200708231807.11409.bs@q-leap.de> Hello Sasha, On Thursday 23 August 2007 16:46:16 you wrote: > Hi, > > On 15:46 Thu 23 Aug , Bernd Schubert wrote: > > I can't make opensm using the UPDN algorithm. Presently we have 3 > > switched connected with each other and one is supposed to be the root > > switch. > > > > I created an /etc/osm.root > > > > root at sn-1:~# cat /etc/osm.roots > > 0xb8cffff004377 > > > > No idea if the syntax is correct, the man page doesn't say anything about > > it. > > > > opensm is started this way > > > > /usr/bin/opensm -p 15 -u -a /etc/osm.roots -R updn -f > > /var/log/osm-port1.log > > Hmm, what '-u' does? oops, left over from an old example config. > > According to the log it looks that OpenSM ignores '-a' option. Bug? Thanks, Bernd -- Bernd Schubert Q-Leap Networks GmbH From john.leidel at gmail.com Thu Aug 23 10:41:34 2007 From: john.leidel at gmail.com (John Leidel) Date: Thu, 23 Aug 2007 14:41:34 -0300 Subject: [ofa-general] MPI IB Errors In-Reply-To: <27f776af0708230849l6b3ab898uf7f3d114b0206f51@mail.gmail.com> References: <27f776af0708221302k5132a5b2g163ec7f5e723a346@mail.gmail.com> <200708231345.20247.bs@q-leap.de> <27f776af0708230457q70f06963wd33bfaab691d1688@mail.gmail.com> <27f776af0708230502i5752eea1ke034060275cc3e2d@mail.gmail.com> <6C2C79E72C305246B504CBA17B5500C901563C00@mtlexch01.mtl.com> <27f776af0708230849l6b3ab898uf7f3d114b0206f51@mail.gmail.com> Message-ID: <27f776af0708231041s7e0133f9sd0d6a6f11db9551b@mail.gmail.com> I found the error in our machine. We had an intermittent connection in one node's HCA card. I just happened to have looked at that node when the HCA was not found in `lscpi` or `proc`. I reset the card on its bus and kaboom... success. Thanks everyone for all your help. On 8/23/07, John Leidel wrote: > > Whats especially odd is that I can get a full bandwidth ping pong test > running fine [970MB/s++], then rerun the test and have it fail saying it > can't find the IB HCA. > > > On 8/23/07, Tziporet Koren wrote: > > > > John Leidel wrote: > > > > > Unfortunately, the RDMA module load didn't help... a simple > > "hello_world" application still returns :: > > > > > > libibverbs: Fatal: no infiniband class devices found. > > > No IB device found > > > > > > I went and verified that all the nodes see the HCAs... an lspci on > > all nodes reports :: > > > > > > 07:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex > > (Tavor compatibility mode) (rev a0) > > > Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex > > (Tavor compatibility mode) > > > > Can you run: > > /etc/init.d/openibd restart > > and send the /var/log/messages > > > > Thanks > > Tziporet > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sashak at voltaire.com Thu Aug 23 11:05:23 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 21:05:23 +0300 Subject: [ofa-general] UPDN algorithm In-Reply-To: <200708231807.11409.bs@q-leap.de> References: <200708231546.59345.bs@q-leap.de> <20070823144616.GG15042@sashak.voltaire.com> <200708231807.11409.bs@q-leap.de> Message-ID: <20070823180523.GA15933@sashak.voltaire.com> On 18:07 Thu 23 Aug , Bernd Schubert wrote: > > > > > > I created an /etc/osm.root > > > > > > root at sn-1:~# cat /etc/osm.roots > > > 0xb8cffff004377 > > > > > > No idea if the syntax is correct, the man page doesn't say anything about > > > it. > > > > > > opensm is started this way > > > > > > /usr/bin/opensm -p 15 -u -a /etc/osm.roots -R updn -f > > > /var/log/osm-port1.log > > > > Hmm, what '-u' does? > > oops, left over from an old example config. > > > > > According to the log it looks that OpenSM ignores '-a' option. > > Bug? Don't know. Could you rerun without '-u' and with '-V -e' and drop me new log file? Sasha From sashak at voltaire.com Thu Aug 23 11:19:17 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 23 Aug 2007 21:19:17 +0300 Subject: [ofa-general] UPDN algorithm In-Reply-To: <20070823180523.GA15933@sashak.voltaire.com> References: <200708231546.59345.bs@q-leap.de> <20070823144616.GG15042@sashak.voltaire.com> <200708231807.11409.bs@q-leap.de> <20070823180523.GA15933@sashak.voltaire.com> Message-ID: <20070823181917.GB15933@sashak.voltaire.com> On 21:05 Thu 23 Aug , Sasha Khapyorsky wrote: > > > > > > > > According to the log it looks that OpenSM ignores '-a' option. > > > > Bug? > > Don't know. Could you rerun without '-u' and with '-V -e' and drop me > new log file? Also you should see on the terminal where opensm is running something like ' UPDN Guid File: ...'. Sasha From bs at q-leap.de Thu Aug 23 11:25:20 2007 From: bs at q-leap.de (Bernd Schubert) Date: Thu, 23 Aug 2007 20:25:20 +0200 Subject: [ofa-general] UPDN algorithm In-Reply-To: <20070823181917.GB15933@sashak.voltaire.com> References: <200708231546.59345.bs@q-leap.de> <20070823180523.GA15933@sashak.voltaire.com> <20070823181917.GB15933@sashak.voltaire.com> Message-ID: <200708232025.20275.bs@q-leap.de> On Thursday 23 August 2007 20:19:17 Sasha Khapyorsky wrote: > On 21:05 Thu 23 Aug , Sasha Khapyorsky wrote: > > > > According to the log it looks that OpenSM ignores '-a' option. > > > > > > Bug? > > > > Don't know. Could you rerun without '-u' and with '-V -e' and drop me > > new log file? > > Also you should see on the terminal where opensm is running something > like ' UPDN Guid File: ...'. Thanks, we oly just changed all cables to get everything on one switch, since we need to run some urgent benchmarks and we also need to stress test this new switch. So I have to postpone the updn-problem to the next week. Thanks for your help, Bernd -- Bernd Schubert Q-Leap Networks GmbH From swise at opengridcomputing.com Thu Aug 23 14:39:45 2007 From: swise at opengridcomputing.com (Steve Wise) Date: Thu, 23 Aug 2007 16:39:45 -0500 Subject: [ofa-general] [PATCH RFC] iw_cxgb3: Support "iwarp-only" interfaces to avoid 4-tuple conflicts with the host stack. Message-ID: <1187905185.5547.13.camel@stevo-desktop> Roland/All, Here is the first swipe at keeping iwarp connections on their own ip addresses to avoid conflicts with the host stack. - this is a request for comments - it is not yet tested fully (tested a prototype of the initial concept) - still needs serialization/locking - stays in our RDMA sandbox ;-) For background reading (if you dare), see: http://www.mail-archive.com/general at lists.openfabrics.org/msg05162.html and http://www.mail-archive.com/netdev at vger.kernel.org/msg44312.html Also: I'm on vacation starting tomorrow until Tuesday 9/4. I'll address comments when I return... Steve. --- iw_cxgb3: Support "iwarp-only" interfaces to avoid 4-tuple conflicts with the host stack. Design: The sysadmin creates "for iwarp use only" alias interfaces of the form "devname:iw*" where devname is the native interface name (eg eth0) for the iwarp netdev device. The alias label can be anything starting with "iw". The "iw" immediately after the ':' is the key used by the iwarp driver. EG: ifconfig eth0 192.168.70.123 up ifconfig eth0:iw1 192.168.71.123 up ifconfig eth0:iw2 192.168.72.123 up In the above example, 192.168.70/24 is for TCP traffic, while 192.168.71/24 and 192.168.72/24 are for iWARP/RDMA use. The rdma-only interface must be on its own subnet. This allows routing all rdma traffic onto this interface. The iWARP driver must translate all listens on address 0.0.0.0 to the set of rdma-only ip addresses. This prevents incoming connects to the TCP ipaddresses from going up the rdma stack. Implementation Details: - The iwarp driver registers for inetaddr events via register_inetaddr_notifier(). This allows tracking the iwarp-only addresses/subnets as they get added and deleted. The iwarp driver maintains a list of the current iwarp-only addresses. - The iwarp driver builds the list of iwarp-only addresses for its devices at module insert time. This is needed because the inetaddr notifier callbacks don't "replay" address-add events when someone registers. So the driver must build the initial list at module load time. - When a listen is done on address 0.0.0.0, then the iwarp driver must translate that into a set of listens on the iwarp-only addresses. - When a new iwarp-only address is added or removed, the iwarp driver must traverse the set of listening endpoints and update them accordingly. This allows an application to bind to 0.0.0.0 prior to the iwarp-only interfaces being configured. It also allows changing the iwarp-only set of addresses and getting the expected behavior for apps already bound to 0.0.0.0. Signed-off-by: Steve Wise --- drivers/infiniband/hw/cxgb3/iwch.c | 116 +++++++++++++++++ drivers/infiniband/hw/cxgb3/iwch.h | 10 + drivers/infiniband/hw/cxgb3/iwch_cm.c | 229 ++++++++++++++++++++++++++------- drivers/infiniband/hw/cxgb3/iwch_cm.h | 11 +- 4 files changed, 318 insertions(+), 48 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c index 0315c9d..da57b77 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.c +++ b/drivers/infiniband/hw/cxgb3/iwch.c @@ -63,6 +63,115 @@ struct cxgb3_client t3c_client = { static LIST_HEAD(dev_list); static DEFINE_MUTEX(dev_mutex); +static void insert_ifa(struct iwch_dev *rnicp, struct in_ifaddr *ifa) +{ + struct iwch_addrlist *addr; + + addr = kmalloc(sizeof *addr, GFP_KERNEL); + if (!addr) { + printk(KERN_ERR MOD "%s - failed to alloc memory!\n", + __FUNCTION__); + return; + } + addr->ifa = ifa; + list_add_tail(&addr->entry, &rnicp->addrlist); +} + +static void remove_ifa(struct iwch_dev *rnicp, struct in_ifaddr *ifa) +{ + struct iwch_addrlist *addr, *tmp; + + list_for_each_entry_safe(addr, tmp, &rnicp->addrlist, entry) { + if (addr->ifa == ifa) { + list_del_init(&addr->entry); + kfree(addr); + return; + } + } +} + +static int netdev_is_ours(struct iwch_dev *rnicp, struct net_device *netdev) +{ + int i; + + for (i = 0; i < rnicp->rdev.port_info.nports; i++) + if (netdev == rnicp->rdev.port_info.lldevs[i]) + return 1; + return 0; +} + +static inline int is_iwarp_label(char *label) +{ + char *colon; + + colon = strchr(label, ':'); + if (colon && !strncmp(colon+1, "iw", 2)) + return 1; + return 0; +} + +static int nb_callback(struct notifier_block *self, unsigned long event, + void *ctx) +{ + struct in_ifaddr *ifa = ctx; + struct iwch_dev *rnicp = container_of(self, struct iwch_dev, nb); + + printk(KERN_INFO "%s rnicp %p event %lx\n", __FUNCTION__, rnicp, event); + + switch (event) { + case NETDEV_UP: + if (netdev_is_ours(rnicp, ifa->ifa_dev->dev) && + is_iwarp_label(ifa->ifa_label)) { + printk(KERN_INFO "label %s addr 0x%x added\n", + ifa->ifa_label, ifa->ifa_address); + insert_ifa(rnicp, ifa); + iwch_listeners_add_addr(rnicp, ifa->ifa_address); + } + break; + case NETDEV_DOWN: + if (netdev_is_ours(rnicp, ifa->ifa_dev->dev) && + is_iwarp_label(ifa->ifa_label)) { + printk(KERN_INFO "label %s addr 0x%x deleted\n", + ifa->ifa_label, ifa->ifa_address); + iwch_listeners_del_addr(rnicp, ifa->ifa_address); + remove_ifa(rnicp, ifa); + } + break; + default: + break; + } + return 0; +} + +static void delete_addrlist(struct iwch_dev *rnicp) +{ + struct iwch_addrlist *addr, *tmp; + + list_for_each_entry_safe(addr, tmp, &rnicp->addrlist, entry) { + list_del_init(&addr->entry); + kfree(addr); + } +} + +static void populate_addrlist(struct iwch_dev *rnicp) +{ + int i; + struct in_device *indev; + + for (i = 0; i < rnicp->rdev.port_info.nports; i++) { + indev = in_dev_get(rnicp->rdev.port_info.lldevs[i]); + if (!indev) + continue; + for_ifa(indev) + if (is_iwarp_label(ifa->ifa_label)) { + printk(KERN_INFO "label %s addr 0x%x added\n", + ifa->ifa_label, ifa->ifa_address); + insert_ifa(rnicp, ifa); + } + endfor_ifa(indev); + } +} + static void rnic_init(struct iwch_dev *rnicp) { PDBG("%s iwch_dev %p\n", __FUNCTION__, rnicp); @@ -70,6 +179,11 @@ static void rnic_init(struct iwch_dev *r idr_init(&rnicp->qpidr); idr_init(&rnicp->mmidr); spin_lock_init(&rnicp->lock); + INIT_LIST_HEAD(&rnicp->addrlist); + INIT_LIST_HEAD(&rnicp->listeners); + rnicp->nb.notifier_call = nb_callback; + populate_addrlist(rnicp); + register_inetaddr_notifier(&rnicp->nb); rnicp->attr.vendor_id = 0x168; rnicp->attr.vendor_part_id = 7; @@ -148,6 +262,8 @@ static void close_rnic_dev(struct t3cdev mutex_lock(&dev_mutex); list_for_each_entry_safe(dev, tmp, &dev_list, entry) { if (dev->rdev.t3cdev_p == tdev) { + unregister_inetaddr_notifier(&dev->nb); + delete_addrlist(dev); list_del(&dev->entry); iwch_unregister_device(dev); cxio_rdev_close(&dev->rdev); diff --git a/drivers/infiniband/hw/cxgb3/iwch.h b/drivers/infiniband/hw/cxgb3/iwch.h index caf4e60..4f9081e 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.h +++ b/drivers/infiniband/hw/cxgb3/iwch.h @@ -36,6 +36,8 @@ #include #include #include #include +#include +#include #include @@ -101,6 +103,11 @@ struct iwch_rnic_attributes { u32 cq_overflow_detection; }; +struct iwch_addrlist { + struct list_head entry; + struct in_ifaddr *ifa; +}; + struct iwch_dev { struct ib_device ibdev; struct cxio_rdev rdev; @@ -111,6 +118,9 @@ struct iwch_dev { struct idr mmidr; spinlock_t lock; struct list_head entry; + struct notifier_block nb; + struct list_head addrlist; + struct list_head listeners; }; static inline struct iwch_dev *to_iwch_dev(struct ib_device *ibdev) diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index 20ba372..4cca2a2 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -1127,23 +1127,129 @@ static int act_open_rpl(struct t3cdev *t return CPL_RET_BUF_DONE; } -static int listen_start(struct iwch_listen_ep *ep) +static struct iwch_listen_entry *alloc_listener(struct iwch_listen_ep *ep, + __be32 addr) +{ + struct iwch_dev *h = to_iwch_dev(ep->com.cm_id->device); + struct iwch_listen_entry *le; + + le = kmalloc(sizeof *le, GFP_KERNEL); + if (!le) { + printk(KERN_ERR MOD "%s - failed to alloc memory!\n", + __FUNCTION__); + return NULL; + } + le->stid = cxgb3_alloc_stid(h->rdev.t3cdev_p, + &t3c_client, ep); + if (le->stid == -1) { + printk(KERN_ERR MOD "%s - cannot alloc stid.\n", + __FUNCTION__); + kfree(le); + return NULL; + } + return le; +} + +static void dealloc_listener(struct iwch_listen_ep *ep, + struct iwch_listen_entry *le) +{ + cxgb3_free_stid(ep->com.tdev, le->stid); + kfree(le); +} + +static void dealloc_listener_list(struct iwch_listen_ep *ep) +{ + struct iwch_listen_entry *le, *tmp; + + list_for_each_entry_safe(le, tmp, &ep->listeners, entry) { + list_del_init(&le->entry); + dealloc_listener(ep, le); + } +} + +static int alloc_listener_list(struct iwch_listen_ep *ep) +{ + struct iwch_dev *h = to_iwch_dev(ep->com.cm_id->device); + struct iwch_addrlist *addr; + struct iwch_listen_entry *le; + int added=0; + int err = 0; + + list_for_each_entry(addr, &h->addrlist, entry) { + if (ep->com.local_addr.sin_addr.s_addr == 0 || + ep->com.local_addr.sin_addr.s_addr == + addr->ifa->ifa_address) { + le = alloc_listener(ep, addr->ifa->ifa_address); + if (!le) + break; + list_add_tail(&le->entry, &ep->listeners); + added++; + } + } + if (ep->com.local_addr.sin_addr.s_addr != 0 && !added) + err = -EADDRNOTAVAIL; + goto out; + dealloc_listener_list(ep); +out: + return err; +} + +static int listen_stop_one(struct t3cdev *tdev, unsigned int stid) { struct sk_buff *skb; - struct cpl_pass_open_req *req; + struct cpl_close_listserv_req *req; + + PDBG("%s stid %u\n", __FUNCTION__, stid); + skb = get_skb(NULL, sizeof(*req), GFP_KERNEL); + if (!skb) { + printk(KERN_ERR MOD "%s - failed to alloc skb\n", __FUNCTION__); + return -ENOMEM; + } + req = (struct cpl_close_listserv_req *) skb_put(skb, sizeof(*req)); + req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_FORWARD)); + req->cpu_idx = 0; + OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_CLOSE_LISTSRV_REQ, stid)); + skb->priority = 1; + cxgb3_ofld_send(tdev, skb); + return 0; +} + +static void listen_stop(struct iwch_listen_ep *ep) +{ + struct iwch_listen_entry *le, *tmp; + int err; PDBG("%s ep %p\n", __FUNCTION__, ep); + list_for_each_entry_safe(le, tmp, &ep->listeners, entry) { + err = listen_stop_one(ep->com.tdev, le->stid); + if (err) { + ep->com.rpl_err = err; + ep->com.rpl_done = 1; + break; + } + } + return; +} + +static int listen_start_one(struct t3cdev *tdev, unsigned int stid, + __be32 addr, __be16 port) +{ + struct sk_buff *skb; + struct cpl_pass_open_req *req; + + PDBG("%s stid %u addr %x port %x\n", __FUNCTION__, stid, ntohl(addr), + ntohs(port)); skb = get_skb(NULL, sizeof(*req), GFP_KERNEL); if (!skb) { - printk(KERN_ERR MOD "t3c_listen_start failed to alloc skb!\n"); + printk(KERN_ERR MOD "%s - failed to alloc skb\n", __FUNCTION__); return -ENOMEM; } req = (struct cpl_pass_open_req *) skb_put(skb, sizeof(*req)); req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_FORWARD)); - OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_PASS_OPEN_REQ, ep->stid)); - req->local_port = ep->com.local_addr.sin_port; - req->local_ip = ep->com.local_addr.sin_addr.s_addr; + OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_PASS_OPEN_REQ, stid)); + req->local_port = port; + req->local_ip = addr; req->peer_port = 0; req->peer_ip = 0; req->peer_netmask = 0; @@ -1152,10 +1258,30 @@ static int listen_start(struct iwch_list req->opt1 = htonl(V_CONN_POLICY(CPL_CONN_POLICY_ASK)); skb->priority = 1; - cxgb3_ofld_send(ep->com.tdev, skb); + cxgb3_ofld_send(tdev, skb); return 0; } +static void listen_start(struct iwch_listen_ep *ep) +{ + struct iwch_listen_entry *le, *tmp; + int err; + + PDBG("%s ep %p\n", __FUNCTION__, ep); + list_for_each_entry_safe(le, tmp, &ep->listeners, entry) { + err = listen_start_one(ep->com.tdev, le->stid, le->addr, + ep->com.local_addr.sin_port); + if (err) + goto fail; + } + return; +fail: + listen_stop(ep); + ep->com.rpl_err = err; + ep->com.rpl_done = 1; + return; +} + static int pass_open_rpl(struct t3cdev *tdev, struct sk_buff *skb, void *ctx) { struct iwch_listen_ep *ep = ctx; @@ -1170,39 +1296,56 @@ static int pass_open_rpl(struct t3cdev * return CPL_RET_BUF_DONE; } -static int listen_stop(struct iwch_listen_ep *ep) -{ - struct sk_buff *skb; - struct cpl_close_listserv_req *req; - - PDBG("%s ep %p\n", __FUNCTION__, ep); - skb = get_skb(NULL, sizeof(*req), GFP_KERNEL); - if (!skb) { - printk(KERN_ERR MOD "%s - failed to alloc skb\n", __FUNCTION__); - return -ENOMEM; - } - req = (struct cpl_close_listserv_req *) skb_put(skb, sizeof(*req)); - req->wr.wr_hi = htonl(V_WR_OP(FW_WROPCODE_FORWARD)); - req->cpu_idx = 0; - OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_CLOSE_LISTSRV_REQ, ep->stid)); - skb->priority = 1; - cxgb3_ofld_send(ep->com.tdev, skb); - return 0; -} - static int close_listsrv_rpl(struct t3cdev *tdev, struct sk_buff *skb, void *ctx) { struct iwch_listen_ep *ep = ctx; struct cpl_close_listserv_rpl *rpl = cplhdr(skb); - PDBG("%s ep %p\n", __FUNCTION__, ep); - ep->com.rpl_err = status2errno(rpl->status); - ep->com.rpl_done = 1; - wake_up(&ep->com.waitq); + PDBG("%s ep %p stid %u\n", __FUNCTION__, ep, GET_TID(rpl)); + if (!ep->com.rpl_done) { + ep->com.rpl_err = status2errno(rpl->status); + ep->com.rpl_done = 1; + wake_up(&ep->com.waitq); + } return CPL_RET_BUF_DONE; } +void iwch_listeners_add_addr(struct iwch_dev *rnicp, __be32 addr) +{ + struct iwch_listen_ep *listen_ep; + struct iwch_listen_entry *le; + + list_for_each_entry(listen_ep, &rnicp->listeners, entry) { + if (listen_ep->com.local_addr.sin_addr.s_addr) + continue; + le = alloc_listener(listen_ep, addr); + if (le) { + list_add_tail(&le->entry, &listen_ep->listeners); + listen_start_one(listen_ep->com.tdev, le->stid, addr, + listen_ep->com.local_addr.sin_port); + } + } +} + +void iwch_listeners_del_addr(struct iwch_dev *rnicp, __be32 addr) +{ + struct iwch_listen_ep *listen_ep; + struct iwch_listen_entry *le, *tmp; + + list_for_each_entry(listen_ep, &rnicp->listeners, entry) { + if (listen_ep->com.local_addr.sin_addr.s_addr) + continue; + list_for_each_entry_safe(le, tmp, &listen_ep->listeners, + entry) + if (le->addr == addr) { + listen_stop_one(listen_ep->com.tdev, le->stid); + list_del_init(&le->entry); + dealloc_listener(listen_ep, le); + } + } +} + static void accept_cr(struct iwch_ep *ep, __be32 peer_ip, struct sk_buff *skb) { struct cpl_pass_accept_rpl *rpl; @@ -1887,31 +2030,23 @@ int iwch_create_listen(struct iw_cm_id * ep->com.cm_id = cm_id; ep->backlog = backlog; ep->com.local_addr = cm_id->local_addr; + INIT_LIST_HEAD(&ep->listeners); - /* - * Allocate a server TID. - */ - ep->stid = cxgb3_alloc_stid(h->rdev.t3cdev_p, &t3c_client, ep); - if (ep->stid == -1) { - printk(KERN_ERR MOD "%s - cannot alloc atid.\n", __FUNCTION__); - err = -ENOMEM; + err = alloc_listener_list(ep); + if (err) goto fail2; - } state_set(&ep->com, LISTEN); - err = listen_start(ep); - if (err) - goto fail3; + listen_start(ep); - /* wait for pass_open_rpl */ + /* wait for last pass_open_rpl */ wait_event(ep->com.waitq, ep->com.rpl_done); err = ep->com.rpl_err; if (!err) { cm_id->provider_data = ep; goto out; } -fail3: - cxgb3_free_stid(ep->com.tdev, ep->stid); + dealloc_listener_list(ep); fail2: cm_id->rem_ref(cm_id); put_ep(&ep->com); @@ -1931,10 +2066,10 @@ int iwch_destroy_listen(struct iw_cm_id state_set(&ep->com, DEAD); ep->com.rpl_done = 0; ep->com.rpl_err = 0; - err = listen_stop(ep); + listen_stop(ep); wait_event(ep->com.waitq, ep->com.rpl_done); - cxgb3_free_stid(ep->com.tdev, ep->stid); err = ep->com.rpl_err; + dealloc_listener_list(ep); cm_id->rem_ref(cm_id); put_ep(&ep->com); return err; diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.h b/drivers/infiniband/hw/cxgb3/iwch_cm.h index 6107e7c..2aafc07 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.h +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.h @@ -162,9 +162,16 @@ struct iwch_ep_common { int rpl_err; }; +struct iwch_listen_entry { + struct list_head entry; + unsigned int stid; + __be32 addr; +}; + struct iwch_listen_ep { + struct list_head entry; struct iwch_ep_common com; - unsigned int stid; + struct list_head listeners; int backlog; }; @@ -222,6 +229,8 @@ int iwch_resume_tid(struct iwch_ep *ep); void __free_ep(struct kref *kref); void iwch_rearp(struct iwch_ep *ep); int iwch_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry *new, struct l2t_entry *l2t); +void iwch_listeners_add_addr(struct iwch_dev *rnicp, __be32 addr); +void iwch_listeners_del_addr(struct iwch_dev *rnicp, __be32 addr); int __init iwch_cm_init(void); void __exit iwch_cm_term(void); From weiny2 at llnl.gov Thu Aug 23 14:46:48 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 23 Aug 2007 14:46:48 -0700 Subject: [ofa-general] [PATCH] infiniband-diags/src/smpquery.c: Add switch-map support for nodedesc query Message-ID: <20070823144648.2fd1cbcd.weiny2@llnl.gov> >From 5bc16ccf9e476d717c8ad02034086e6cfdae3584 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 23 Aug 2007 14:43:40 -0700 Subject: [PATCH] infiniband-diags/src/smpquery.c: Add switch-map support for nodedesc query Signed-off-by: Ira K. Weiny --- infiniband-diags/src/smpquery.c | 36 +++++++++++++++++++++++++++++++++--- 1 files changed, 33 insertions(+), 3 deletions(-) diff --git a/infiniband-diags/src/smpquery.c b/infiniband-diags/src/smpquery.c index 8b882ed..55e0b3f 100644 --- a/infiniband-diags/src/smpquery.c +++ b/infiniband-diags/src/smpquery.c @@ -84,17 +84,41 @@ static const match_rec_t match_tbl[] = { }; char *argv0 = "smpquery"; +static char *switch_map = NULL; +static FILE *switch_map_fp = NULL; /*******************************************/ static char * node_desc(ib_portid_t *dest, char **argv, int argc) { - char nd[IB_SMP_DATA_SIZE]; + int node_type, l; + uint64_t node_guid; + char nd[IB_SMP_DATA_SIZE]; + char data[IB_SMP_DATA_SIZE]; + char dots[128]; + char *nodename = NULL; + + if (!smp_query(data, dest, IB_ATTR_NODE_INFO, 0, 0)) + return "node info query failed"; + + mad_decode_field(data, IB_NODE_TYPE_F, &node_type); + mad_decode_field(data, IB_NODE_GUID_F, &node_guid); if (!smp_query(nd, dest, IB_ATTR_NODE_DESC, 0, 0)) return "node desc query failed"; - _mad_dump(mad_dump_nodedesc, "Node Description", nd, sizeof nd); + if (node_type == IB_NODE_SWITCH) + nodename = lookup_switch_name(switch_map_fp, node_guid, nd); + else + nodename = clean_nodedesc(nd); + + l = strlen(nodename); + if (l < 32) { + memset(dots, '.', 32 - l); + dots[32 - l] = '\0'; + } + + printf("Node Description:%s%s\n", dots, nodename); return 0; } @@ -379,7 +403,7 @@ usage(void) basename++; fprintf(stderr, "Usage: %s [-d(ebug) -e(rr_show) -v(erbose) -D(irect) -G(uid) -s smlid -V(ersion) -C ca_name -P ca_port " - "-t(imeout) timeout_ms] [op params]\n", + "-t(imeout) timeout_ms --switch-map switch-map] [op params]\n", basename); fprintf(stderr, "\tsupported ops:\n"); for (r = match_tbl ; r->name ; r++) { @@ -419,6 +443,7 @@ main(int argc, char **argv) { "Guid", 0, 0, 'G'}, { "smlid", 1, 0, 's'}, { "timeout", 1, 0, 't'}, + { "switch-map", 1, 0, 1}, { "Version", 0, 0, 'V'}, { "help", 0, 0, 'h'}, { "usage", 0, 0, 'u'}, @@ -432,6 +457,9 @@ main(int argc, char **argv) if ( ch == -1 ) break; switch(ch) { + case 1: + switch_map = strdup(optarg); + break; case 'd': ibdebug++; madrpc_show_errors(1); @@ -486,6 +514,7 @@ main(int argc, char **argv) IBERROR("operation '%s' not supported", argv[0]); madrpc_init(ca, ca_port, mgmt_classes, 3); + switch_map_fp = open_switch_map(switch_map); if (dest_type != IB_DEST_DRSLID) { if (ib_resolve_portid_str(&portid, argv[1], dest_type, sm_id) < 0) @@ -502,5 +531,6 @@ main(int argc, char **argv) if ((err = fn(&portid, argv+3, argc-3))) IBERROR("operation %s: %s", argv[0], err); } + close_switch_map(switch_map_fp); exit(0); } -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-infiniband-diags-src-smpquery.c-Add-switch-map-supp.patch Type: application/octet-stream Size: 3147 bytes Desc: not available URL: From hadi at cyberus.ca Thu Aug 23 15:04:10 2007 From: hadi at cyberus.ca (jamal) Date: Thu, 23 Aug 2007 18:04:10 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070822.132145.90824527.davem@davemloft.net> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> Message-ID: <1187906650.4279.16.camel@localhost> On Wed, 2007-22-08 at 13:21 -0700, David Miller wrote: > From: Rick Jones > Date: Wed, 22 Aug 2007 10:09:37 -0700 > > > Should it be any more or less worrysome than small packet > > performance (eg the TCP_RR stuff I posted recently) being rather > > worse with TSO enabled than with it disabled? > > That, like any such thing shown by the batching changes, is a bug > to fix. Possibly a bug - but you really should turn off TSO if you are doing huge interactive transactions (which is fair because there is a clear demarcation). The litmus test is the same as any change that is supposed to improve net performance - it has to demonstrate it is not intrusive and that it improves (consistently) performance. The standard metrics are {throughput, cpu-utilization, latency} i.e as long as one improves and others remain zero, it would make sense. Yes, i am religious for batching after all the invested sweat (and i continue to work on it hoping to demystify) - the theory makes a lot of sense. cheers, jamal From hadi at cyberus.ca Thu Aug 23 15:25:03 2007 From: hadi at cyberus.ca (jamal) Date: Thu, 23 Aug 2007 18:25:03 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187906650.4279.16.camel@localhost> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> Message-ID: <1187907903.4279.28.camel@localhost> On Thu, 2007-23-08 at 18:04 -0400, jamal wrote: > The litmus test is the same as any change that is supposed to improve > net performance - it has to demonstrate it is not intrusive and that it > improves (consistently) performance. The standard metrics are > {throughput, cpu-utilization, latency} i.e as long as one improves and > others remain zero, it would make sense. Yes, i am religious for > batching after all the invested sweat (and i continue to work on it > hoping to demystify) - the theory makes a lot of sense. Before someone jumps and strangles me ;-> By "litmus test" i meant as applied to batching. [TSO already passed - iirc, it has been demostranted to really not add much to throughput (cant improve much over closeness to wire speed) but improve CPU utilization]. cheers, jamal From davem at davemloft.net Thu Aug 23 15:30:47 2007 From: davem at davemloft.net (David Miller) Date: Thu, 23 Aug 2007 15:30:47 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187906650.4279.16.camel@localhost> References: <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> Message-ID: <20070823.153047.99173582.davem@davemloft.net> From: jamal Date: Thu, 23 Aug 2007 18:04:10 -0400 > Possibly a bug - but you really should turn off TSO if you are doing > huge interactive transactions (which is fair because there is a clear > demarcation). I don't see how this can matter. TSO only ever does anything if you accumulate more than one MSS worth of data. And when that does happen, all it does is take whats in the send queue and send as much as possible at once. The packets are already built in big chunks, so there is no extra work to do. The card is going to send the things back to back and as fast as in the non-TSO case as well. It doesn't change application scheduling, and it absolutely does not penalize small sends by the application unless we have a bug somewhere. So I see no reason to disable TSO for any reason other than hardware implementation deficiencies. And for the drivers I am familiar with they do make smart default TSO enabling decisions based upon how well the chip does TSO. From rick.jones2 at hp.com Thu Aug 23 15:35:13 2007 From: rick.jones2 at hp.com (Rick Jones) Date: Thu, 23 Aug 2007 15:35:13 -0700 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187907903.4279.28.camel@localhost> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> Message-ID: <46CE0BA1.60206@hp.com> jamal wrote: > [TSO already passed - iirc, it has been > demostranted to really not add much to throughput (cant improve much > over closeness to wire speed) but improve CPU utilization]. In the one gig space sure, but in the 10 Gig space, TSO on/off does make a difference for throughput. rick jones From hadi at cyberus.ca Thu Aug 23 15:38:22 2007 From: hadi at cyberus.ca (jamal) Date: Thu, 23 Aug 2007 18:38:22 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070823.153047.99173582.davem@davemloft.net> References: <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <20070823.153047.99173582.davem@davemloft.net> Message-ID: <1187908702.4279.36.camel@localhost> On Thu, 2007-23-08 at 15:30 -0700, David Miller wrote: > From: jamal > Date: Thu, 23 Aug 2007 18:04:10 -0400 > > > Possibly a bug - but you really should turn off TSO if you are doing > > huge interactive transactions (which is fair because there is a clear > > demarcation). > > I don't see how this can matter. > > TSO only ever does anything if you accumulate more than one MSS > worth of data. I stand corrected then. cheers, jamal From hadi at cyberus.ca Thu Aug 23 15:41:00 2007 From: hadi at cyberus.ca (jamal) Date: Thu, 23 Aug 2007 18:41:00 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <46CE0BA1.60206@hp.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> Message-ID: <1187908860.4279.39.camel@localhost> On Thu, 2007-23-08 at 15:35 -0700, Rick Jones wrote: > jamal wrote: > > [TSO already passed - iirc, it has been > > demostranted to really not add much to throughput (cant improve much > > over closeness to wire speed) but improve CPU utilization]. > > In the one gig space sure, but in the 10 Gig space, TSO on/off does make a > difference for throughput. I am still so 1Gige;-> I stand corrected again ;-> cheers, jamal From billfink at mindspring.com Thu Aug 23 20:18:20 2007 From: billfink at mindspring.com (Bill Fink) Date: Thu, 23 Aug 2007 23:18:20 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <46CE0BA1.60206@hp.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> Message-ID: <20070823231820.2ae52cc0.billfink@mindspring.com> On Thu, 23 Aug 2007, Rick Jones wrote: > jamal wrote: > > [TSO already passed - iirc, it has been > > demostranted to really not add much to throughput (cant improve much > > over closeness to wire speed) but improve CPU utilization]. > > In the one gig space sure, but in the 10 Gig space, TSO on/off does make a > difference for throughput. Not too much. TSO enabled: [root at lang2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11813.4375 MB / 10.00 sec = 9906.1644 Mbps 99 %TX 80 %RX TSO disabled: [root at lang2 ~]# ethtool -K eth2 tso off [root at lang2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11818.2500 MB / 10.00 sec = 9910.0176 Mbps 100 %TX 78 %RX Pretty negligible difference it seems. This is with a 2.6.20.7 kernel, Myricom 10-GigE NICs, and 9000 byte jumbo frames, in a LAN environment. For grins, I also did a couple of tests with an MSS of 1460 to emulate a standard 1500 byte Ethernet MTU. TSO enabled: [root at lang2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5102.8503 MB / 10.06 sec = 4253.9124 Mbps 39 %TX 99 %RX TSO disabled: [root at lang2 ~]# ethtool -K eth2 tso off [root at lang2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5399.5625 MB / 10.00 sec = 4527.9070 Mbps 99 %TX 76 %RX Here you can see there is a major difference in the TX CPU utilization (99 % with TSO disabled versus only 39 % with TSO enabled), although the TSO disabled case was able to squeeze out a little extra performance from its extra CPU utilization. Interestingly, with TSO enabled, the receiver actually consumed more CPU than with TSO disabled, so I guess the receiver CPU saturation in that case (99 %) was what restricted its performance somewhat (this was consistent across a few test runs). -Bill From shemminger at linux-foundation.org Thu Aug 23 20:34:25 2007 From: shemminger at linux-foundation.org (Stephen Hemminger) Date: Thu, 23 Aug 2007 20:34:25 -0700 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187908702.4279.36.camel@localhost> References: <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <20070823.153047.99173582.davem@davemloft.net> <1187908702.4279.36.camel@localhost> Message-ID: <20070823203425.2a32fb3f@freepuppy.rosehill.hemminger.net> On Thu, 23 Aug 2007 18:38:22 -0400 jamal wrote: > On Thu, 2007-23-08 at 15:30 -0700, David Miller wrote: > > From: jamal > > Date: Thu, 23 Aug 2007 18:04:10 -0400 > > > > > Possibly a bug - but you really should turn off TSO if you are doing > > > huge interactive transactions (which is fair because there is a clear > > > demarcation). > > > > I don't see how this can matter. > > > > TSO only ever does anything if you accumulate more than one MSS > > worth of data. > > I stand corrected then. > > cheers, > jamal > For most normal Internet TCP connections, you will see only 2 or 3 packets per TSO because of ACK clocking. If you turn off delayed ACK on the receiver it will be even less. A current hot topic of research is reducing the number of ACK's to make TCP work better over asymmetric links like 3G. -- Stephen Hemminger From kliteyn at mellanox.co.il Thu Aug 23 21:26:07 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 24 Aug 2007 07:26:07 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-24:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_23_15:29:02_2007 [08e716a33d68b316611313654de5f6bd61b7dec3] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=520 Pass=520 Fail=0 Pass: 39 Stability IS1-16.topo 39 Pkey IS1-16.topo 39 OsmTest IS1-16.topo 39 OsmStress IS1-16.topo 39 Multicast IS1-16.topo 39 LidMgr IS1-16.topo 13 Stability IS3-loop.topo 13 Stability IS3-128.topo 13 Pkey IS3-128.topo 13 OsmTest IS3-loop.topo 13 OsmTest IS3-128.topo 13 OsmStress IS3-128.topo 13 Multicast IS3-loop.topo 13 Multicast IS3-128.topo 13 LidMgr IS3-128.topo 13 FatTree merge-roots-4-ary-2-tree.topo 13 FatTree merge-root-4-ary-3-tree.topo 13 FatTree gnu-stallion-64.topo 13 FatTree blend-4-ary-2-tree.topo 13 FatTree RhinoDDR.topo 13 FatTree FullGnu.topo 13 FatTree 4-ary-2-tree.topo 13 FatTree 2-ary-4-tree.topo 13 FatTree 12-node-spaced.topo 13 FTreeFail 4-ary-2-tree-missing-sw-link.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 13 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 13 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From Notice553 at satx.rr.com Fri Aug 24 01:43:01 2007 From: Notice553 at satx.rr.com (Notice553 at satx.rr.com) Date: Fri, 24 Aug 2007 09:43:01 +0100 Subject: [ofa-general] CONFIRM YOUR WINNING PRIZE Ref: XYL /26510460037/05 Message-ID: The National Lottery PO Box 1010 Liverpool L70 1NL, United Kingdom. Ref: XYL /26510460037/05 Batch: 24/00319/IPD WINNING NOTIFICATION We happily announce to you the draw (#1071) winner of the UK NATIONAL LOTTERY cash prize of £2,696,385 (Two Million Six Hundred and Ninety held on the 23th of August 2007 in London Uk.The selection process was carried out through random selection in our computerized email selection system(ess) from a database of over 250,000 email addresses drawn from which you were selected. The BRITISH UK. Lottery is approved by the British Gaming Board. To begin the processing of your prize you are to contact our fiduaciary claims department for more infomation as regards procedures to claim your prize. Agents Name: Van Williams Email: claims_uknationalotterydept3 at yahoo.co.uk Tel: +447024096270 + 44 702 402 8482 Fax: + 44 7075767527 1.Name.......................... 2.Address....................... 3.Nationality................... 4.Age........................... 5.Sex........................... 6.Occupation.................... 7.Phone/Fax..................... 8.cOUNTRY..................... YOU ARE TO CHOOSE PAYMENT MODE: OPTIONS 1. BANK TO BANK WIRE TRANSFER. 2. CERTIFIED CHEQUE MADE OUT IN YOUR NAME COURIERED TO YOU VIA OUR AFFILIATE COURIER COMPANY AND WILL BE DELIVERED TO YOUR ADDRESS Cordially, Rose Wood Online Co-ordinator U.K NATIONAL LOTTERY Sweepstakes International Program From vlad at lists.openfabrics.org Fri Aug 24 03:20:08 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 24 Aug 2007 03:20:08 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070824-0200 daily build status Message-ID: <20070824102018.DCC71E60874@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check/kernel_addons/backport/2.6.15_ubuntu606/include/linux/skbuff.h:22: error: conflicting types for '__skb_checksum_complete' include/linux/skbuff.h:1306: error: previous declaration of '__skb_checksum_complete' was here /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070824-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From sashak at voltaire.com Fri Aug 24 03:28:52 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 24 Aug 2007 13:28:52 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/src/smpquery.c: Add switch-map support for nodedesc query In-Reply-To: <20070823144648.2fd1cbcd.weiny2@llnl.gov> References: <20070823144648.2fd1cbcd.weiny2@llnl.gov> Message-ID: <20070824102852.GA18082@sashak.voltaire.com> On 14:46 Thu 23 Aug , Ira Weiny wrote: > From 5bc16ccf9e476d717c8ad02034086e6cfdae3584 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 23 Aug 2007 14:43:40 -0700 > Subject: [PATCH] infiniband-diags/src/smpquery.c: Add switch-map support for nodedesc query > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From hadi at cyberus.ca Fri Aug 24 05:14:16 2007 From: hadi at cyberus.ca (jamal) Date: Fri, 24 Aug 2007 08:14:16 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070823231820.2ae52cc0.billfink@mindspring.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> Message-ID: <1187957657.4255.35.camel@localhost> On Thu, 2007-23-08 at 23:18 -0400, Bill Fink wrote: [..] > Here you can see there is a major difference in the TX CPU utilization > (99 % with TSO disabled versus only 39 % with TSO enabled), although > the TSO disabled case was able to squeeze out a little extra performance > from its extra CPU utilization. Good stuff. What kind of machine? SMP? Seems the receive side of the sender is also consuming a lot more cpu i suspect because receiver is generating a lot more ACKs with TSO. Does the choice of the tcp congestion control algorithm affect results? it would be interesting to see both MTUs with either TCP BIC vs good old reno on sender (probably without changing what the receiver does). BIC seems to be the default lately. > Interestingly, with TSO enabled, the > receiver actually consumed more CPU than with TSO disabled, I would suspect the fact that a lot more packets making it into the receiver for TSO contributes. > so I guess > the receiver CPU saturation in that case (99 %) was what restricted > its performance somewhat (this was consistent across a few test runs). Unfortunately the receiver plays a big role in such tests - if it is bottlenecked then you are not really testing the limits of the transmitter. cheers, jamal From billfink at mindspring.com Fri Aug 24 11:08:42 2007 From: billfink at mindspring.com (Bill Fink) Date: Fri, 24 Aug 2007 14:08:42 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187957657.4255.35.camel@localhost> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> <1187957657.4255.35.camel@localhost> Message-ID: <20070824140842.05bc316e.billfink@mindspring.com> On Fri, 24 Aug 2007, jamal wrote: > On Thu, 2007-23-08 at 23:18 -0400, Bill Fink wrote: > > [..] > > Here you can see there is a major difference in the TX CPU utilization > > (99 % with TSO disabled versus only 39 % with TSO enabled), although > > the TSO disabled case was able to squeeze out a little extra performance > > from its extra CPU utilization. > > Good stuff. What kind of machine? SMP? Tyan Thunder K8WE S2895ANRF motherboard with Nvidia nForce Professional 2200+2050 chipset, 2 AMD Opteron 254 2.8 GHz CPUs, 4 GB PC3200 ECC REG-DDR 400 memory, and 2 PCI-Express x16 slots (2 buses). It is SMP but both the NIC interrupts and nuttcp are bound to CPU 0. And all other non-kernel system processes are bound to CPU 1. > Seems the receive side of the sender is also consuming a lot more cpu > i suspect because receiver is generating a lot more ACKs with TSO. Odd. I just reran the TCP CUBIC "-M1460" tests, and with TSO enabled on the transmitter, there were about 153709 eth2 interrupts on the receiver, while with TSO disabled there was actually a somewhat higher number (164988) of receiver side eth2 interrupts, although the receive side CPU utilization was actually lower in that case. On the transmit side (different test run), the TSO enabled case had about 161773 eth2 interrupts whereas the TSO disabled case had about 165179 eth2 interrupts. > Does the choice of the tcp congestion control algorithm affect results? > it would be interesting to see both MTUs with either TCP BIC vs good old > reno on sender (probably without changing what the receiver does). BIC > seems to be the default lately. These tests were with the default TCP CUBIC (with initial_ssthresh set to 0). With TCP BIC (and initial_ssthresh set to 0): TSO enabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11751.3750 MB / 10.00 sec = 9853.9839 Mbps 100 %TX 83 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 4999.3321 MB / 10.06 sec = 4167.7872 Mbps 38 %TX 100 %RX TSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11818.1875 MB / 10.00 sec = 9910.0682 Mbps 99 %TX 81 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5502.6250 MB / 10.00 sec = 4614.3297 Mbps 100 %TX 84 %RX And with TCP Reno: TSO enabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11782.6250 MB / 10.00 sec = 9880.2613 Mbps 100 %TX 77 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5024.6649 MB / 10.06 sec = 4191.6574 Mbps 38 %TX 99 %RX TSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11818.2500 MB / 10.00 sec = 9910.0860 Mbps 99 %TX 77 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5284.0000 MB / 10.00 sec = 4430.9604 Mbps 99 %TX 79 %RX Very similar results to the original TCP CUBIC tests. > > Interestingly, with TSO enabled, the > > receiver actually consumed more CPU than with TSO disabled, > > I would suspect the fact that a lot more packets making it into the > receiver for TSO contributes. > > > so I guess > > the receiver CPU saturation in that case (99 %) was what restricted > > its performance somewhat (this was consistent across a few test runs). > > Unfortunately the receiver plays a big role in such tests - if it is > bottlenecked then you are not really testing the limits of the > transmitter. It might be interesting to see what affect the LRO changes would have on this. Once they are in a stable released kernel, I might try that out, or maybe even before if I get some spare time (but that's in very short supply right now). -Thanks -Bill From rick.jones2 at hp.com Fri Aug 24 11:46:26 2007 From: rick.jones2 at hp.com (Rick Jones) Date: Fri, 24 Aug 2007 11:46:26 -0700 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070823231820.2ae52cc0.billfink@mindspring.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> Message-ID: <46CF2782.6060006@hp.com> Bill Fink wrote: > On Thu, 23 Aug 2007, Rick Jones wrote: > > >>jamal wrote: >> >>>[TSO already passed - iirc, it has been >>>demostranted to really not add much to throughput (cant improve much >>>over closeness to wire speed) but improve CPU utilization]. >> >>In the one gig space sure, but in the 10 Gig space, TSO on/off does make a >>difference for throughput. > > > Not too much. > > TSO enabled: > > [root at lang2 ~]# ethtool -k eth2 > Offload parameters for eth2: > rx-checksumming: on > tx-checksumming: on > scatter-gather: on > tcp segmentation offload: on > > [root at lang2 ~]# nuttcp -w10m 192.168.88.16 > 11813.4375 MB / 10.00 sec = 9906.1644 Mbps 99 %TX 80 %RX > > TSO disabled: > > [root at lang2 ~]# ethtool -K eth2 tso off > [root at lang2 ~]# ethtool -k eth2 > Offload parameters for eth2: > rx-checksumming: on > tx-checksumming: on > scatter-gather: on > tcp segmentation offload: off > > [root at lang2 ~]# nuttcp -w10m 192.168.88.16 > 11818.2500 MB / 10.00 sec = 9910.0176 Mbps 100 %TX 78 %RX > > Pretty negligible difference it seems. Leaves one wondering how often more than one segment was sent to the card in the 9000 byte case :) rick jones From davem at davemloft.net Fri Aug 24 14:25:03 2007 From: davem at davemloft.net (David Miller) Date: Fri, 24 Aug 2007 14:25:03 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <1187957657.4255.35.camel@localhost> References: <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> <1187957657.4255.35.camel@localhost> Message-ID: <20070824.142503.30177455.davem@davemloft.net> From: jamal Date: Fri, 24 Aug 2007 08:14:16 -0400 > Seems the receive side of the sender is also consuming a lot more cpu > i suspect because receiver is generating a lot more ACKs with TSO. I've seen this behavior before on a low cpu powered receiver and the issue is that batching too much actually hurts a receiver. If the data packets were better spaced out, the receive would handle the load better. This is the thing the TOE guys keep talking about overcoming with their packet pacing algorithms in their on-card TOE stack. My hunch is that even if in the non-TSO case the TX packets were all back to back in the cards TX ring, TSO still spits them out faster on the wire. From jheffner at psc.edu Fri Aug 24 17:42:59 2007 From: jheffner at psc.edu (John Heffner) Date: Fri, 24 Aug 2007 20:42:59 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070823231820.2ae52cc0.billfink@mindspring.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> Message-ID: <46CF7B13.3020701@psc.edu> Bill Fink wrote: > Here you can see there is a major difference in the TX CPU utilization > (99 % with TSO disabled versus only 39 % with TSO enabled), although > the TSO disabled case was able to squeeze out a little extra performance > from its extra CPU utilization. Interestingly, with TSO enabled, the > receiver actually consumed more CPU than with TSO disabled, so I guess > the receiver CPU saturation in that case (99 %) was what restricted > its performance somewhat (this was consistent across a few test runs). One possibility is that I think the receive-side processing tends to do better when receiving into an empty queue. When the (non-TSO) sender is the flow's bottleneck, this is going to be the case. But when you switch to TSO, the receiver becomes the bottleneck and you're always going to have to put the packets at the back of the receive queue. This might help account for the reason why you have both lower throughput and higher CPU utilization -- there's a point of instability right where the receiver becomes the bottleneck and you end up pushing it over to the bad side. :) Just a theory. I'm honestly surprised this effect would be so significant. What do the numbers from netstat -s look like in the two cases? -John From kliteyn at mellanox.co.il Fri Aug 24 21:39:40 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 25 Aug 2007 07:39:40 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-25:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_23_14:46:48_2007 [05cae48f68250346dc02b3583044664d6ca6474e] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From vlad at lists.openfabrics.org Sat Aug 25 02:45:32 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sat, 25 Aug 2007 02:45:32 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070825-0200 daily build status Message-ID: <20070825094533.20290E6088E@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.22 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check/kernel_addons/backport/2.6.15_ubuntu606/include/linux/skbuff.h:22: error: conflicting types for '__skb_checksum_complete' include/linux/skbuff.h:1306: error: previous declaration of '__skb_checksum_complete' was here /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070825-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From ralph.campbell at qlogic.com Sat Aug 25 15:56:09 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Sat, 25 Aug 2007 15:56:09 -0700 Subject: [ofa-general] [PATCH] connected mode leaks pages on receive & reuses old skb Message-ID: <1188082569.3730.107.camel@brick.pathscale.com> This patch fixes a bug in connected mode where the newly allocated skb is not put on the SRQ ring and the unused pages from the old skb are not transferred to the new one. This patch was created in Roland's for-2.6.24 branch but probably applies to 2.6.23 too. Here is the basic order of events in the original code: init: create SRQ with ipoib_recvq_size entries fill SRQ ring with max sized SKBs by calling ipoib_cm_alloc_rx_skb() post all SKBs w/ num_sge = IPOIB_CM_RX_SG receive: allocate new SKB based on msg size, fill mapping 0..frags by doing DMA map DMA unmap mapping[0..frags] for old SKB NOTE that [frags..IPOIB_CM_RX_SG-1] are *not* unmapped (which is OK). replace srq_ring[id].mapping[0..frags] with new mapping from new SKB (data is still in old SKB's frag pointers) NOTE that [frags..IPOIB_CM_RX_SG-1] are *not* copied to newskb and thus leaked. Also, newskb's nr_frags is set based on the message size rather than the full size. NOTE that srq_ring[id].skb is *not* updated to point to newskb and thus will be used again after being passed to the Linux network stack. Also, use ib_dma_unmap_page() for addresses mapped by ib_dma_map_page(). Signed-off-by: Ralph Campbell diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 08b4676..a3434f2 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -78,7 +78,7 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags, ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); for (i = 0; i < frags; ++i) - ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); + ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); } static int ipoib_cm_post_receive(struct net_device *dev, int id) @@ -149,7 +149,7 @@ partial_error: ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); for (; i > 0; --i) - ib_dma_unmap_single(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); + ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); dev_kfree_skb_any(skb); return NULL; @@ -365,7 +365,11 @@ static int ipoib_cm_rx_handler(struct ib_cm_id *cm_id, return 0; } } -/* Adjust length of skb with fragments to match received data */ + +/* + * Adjust the length of skb to match the received data and + * move the unused frag pages in skb to the (empty) frag slots in toskb. + */ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space, unsigned int length, struct sk_buff *toskb) { @@ -378,23 +382,26 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space, skb->len += size; length -= size; - num_frags = skb_shinfo(skb)->nr_frags; + num_frags = skb_shinfo(toskb)->nr_frags; for (i = 0; i < num_frags; i++) { skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - if (length == 0) { - /* don't need this page */ - skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); - --skb_shinfo(skb)->nr_frags; - } else { - size = min(length, (unsigned) PAGE_SIZE); - - frag->size = size; - skb->data_len += size; - skb->truesize += size; - skb->len += size; - length -= size; - } + size = min(length, (unsigned) PAGE_SIZE); + WARN_ON(size == 0); + frag->size = size; + skb->data_len += size; + skb->truesize += size; + skb->len += size; + length -= size; + } + WARN_ON(length != 0); + skb_shinfo(skb)->nr_frags = num_frags; + + /* move unused pages from skb to toskb */ + for (; i < IPOIB_CM_RX_SG - 1; i++) { + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; + + skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); } } @@ -463,6 +470,7 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) ipoib_cm_dma_unmap_rx(priv, frags, priv->cm.srq_ring[wr_id].mapping); memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, (frags + 1) * sizeof *mapping); + priv->cm.srq_ring[wr_id].skb = newskb; ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", wc->byte_len, wc->slid); From kliteyn at mellanox.co.il Sat Aug 25 21:41:54 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 26 Aug 2007 07:41:54 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-26:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_23_14:46:48_2007 [05cae48f68250346dc02b3583044664d6ca6474e] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From eli at mellanox.co.il Sat Aug 25 21:40:38 2007 From: eli at mellanox.co.il (Eli Cohen) Date: Sun, 26 Aug 2007 07:40:38 +0300 Subject: [ofa-general] [PATCH -3] ib_mthca checksum offload support - resend Message-ID: <1188103238.23213.2.camel@mtls03> Add checksum offload support in mthca Signed-off-by: Eli Cohen --- resending because the previous one did not compile Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.c 2007-08-22 14:49:41.000000000 +0300 @@ -1377,6 +1377,9 @@ int mthca_INIT_HCA(struct mthca_dev *dev MTHCA_PUT(inbox, param->uarc_base, INIT_HCA_UAR_CTX_BASE_OFFSET); } + if (dev->device_cap_flags & IB_DEVICE_IP_CSUM) + *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) |= cpu_to_be32(7 << 3); + err = mthca_cmd(dev, mailbox->dma, 0, 0, CMD_INIT_HCA, HZ, status); mthca_free_mailbox(dev, mailbox); Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cmd.h 2007-08-22 14:49:41.000000000 +0300 @@ -103,6 +103,7 @@ enum { DEV_LIM_FLAG_RAW_IPV6 = 1 << 4, DEV_LIM_FLAG_RAW_ETHER = 1 << 5, DEV_LIM_FLAG_SRQ = 1 << 6, + DEV_LIM_FLAG_IPOIB_CSUM = 1 << 7, DEV_LIM_FLAG_BAD_PKEY_CNTR = 1 << 8, DEV_LIM_FLAG_BAD_QKEY_CNTR = 1 << 9, DEV_LIM_FLAG_MW = 1 << 16, Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_cq.c 2007-08-22 15:04:32.000000000 +0300 @@ -119,7 +119,8 @@ struct mthca_cqe { __be32 my_qpn; __be32 my_ee; __be32 rqpn; - __be16 sl_g_mlpath; + u8 sl_ipok; + u8 g_mlpath; __be16 rlid; __be32 imm_etype_pkey_eec; __be32 byte_cnt; @@ -498,6 +499,7 @@ static inline int mthca_poll_one(struct int is_send; int free_cqe = 1; int err = 0; + u16 checksum; cqe = next_cqe_sw(cq); if (!cqe) @@ -640,12 +642,15 @@ static inline int mthca_poll_one(struct break; } entry->slid = be16_to_cpu(cqe->rlid); - entry->sl = be16_to_cpu(cqe->sl_g_mlpath) >> 12; + entry->sl = cqe->sl_ipok >> 4; entry->src_qp = be32_to_cpu(cqe->rqpn) & 0xffffff; - entry->dlid_path_bits = be16_to_cpu(cqe->sl_g_mlpath) & 0x7f; + entry->dlid_path_bits = cqe->g_mlpath & 0x7f; entry->pkey_index = be32_to_cpu(cqe->imm_etype_pkey_eec) >> 16; - entry->wc_flags |= be16_to_cpu(cqe->sl_g_mlpath) & 0x80 ? - IB_WC_GRH : 0; + entry->wc_flags |= cqe->g_mlpath & 0x80 ? IB_WC_GRH : 0; + checksum = (be32_to_cpu(cqe->rqpn) >> 24) | + ((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00); + entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff); + entry->csum = checksum; } entry->status = IB_WC_SUCCESS; Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_main.c 2007-08-22 14:49:41.000000000 +0300 @@ -289,6 +289,10 @@ static int mthca_dev_lim(struct mthca_de if (dev_lim->flags & DEV_LIM_FLAG_SRQ) mdev->mthca_flags |= MTHCA_FLAG_SRQ; + if (mthca_is_memfree(mdev)) + if (dev_lim->flags & DEV_LIM_FLAG_IPOIB_CSUM) + mdev->device_cap_flags |= IB_DEVICE_IP_CSUM; + return 0; } Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_qp.c 2007-08-22 14:49:41.000000000 +0300 @@ -2024,6 +2024,10 @@ int mthca_arbel_post_send(struct ib_qp * cpu_to_be32(MTHCA_NEXT_CQ_UPDATE) : 0) | ((wr->send_flags & IB_SEND_SOLICITED) ? cpu_to_be32(MTHCA_NEXT_SOLICIT) : 0) | + ((wr->send_flags & IB_SEND_IP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_IP_CSUM) : 0) | + ((wr->send_flags & IB_SEND_UDP_TCP_CSUM) ? + cpu_to_be32(MTHCA_NEXT_TCP_UDP_CSUM) : 0) | cpu_to_be32(1); if (wr->opcode == IB_WR_SEND_WITH_IMM || wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) Index: linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h =================================================================== --- linux-2.6.23-rc1.orig/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-22 14:40:24.000000000 +0300 +++ linux-2.6.23-rc1/drivers/infiniband/hw/mthca/mthca_wqe.h 2007-08-22 14:49:41.000000000 +0300 @@ -38,14 +38,15 @@ #include enum { - MTHCA_NEXT_DBD = 1 << 7, - MTHCA_NEXT_FENCE = 1 << 6, - MTHCA_NEXT_CQ_UPDATE = 1 << 3, - MTHCA_NEXT_EVENT_GEN = 1 << 2, - MTHCA_NEXT_SOLICIT = 1 << 1, - - MTHCA_MLX_VL15 = 1 << 17, - MTHCA_MLX_SLR = 1 << 16 + MTHCA_NEXT_DBD = 1 << 7, + MTHCA_NEXT_FENCE = 1 << 6, + MTHCA_NEXT_CQ_UPDATE = 1 << 3, + MTHCA_NEXT_EVENT_GEN = 1 << 2, + MTHCA_NEXT_SOLICIT = 1 << 1, + MTHCA_NEXT_IP_CSUM = 1 << 4, + MTHCA_NEXT_TCP_UDP_CSUM = 1 << 5, + MTHCA_MLX_VL15 = 1 << 17, + MTHCA_MLX_SLR = 1 << 16 }; enum { From mst at dev.mellanox.co.il Sat Aug 25 21:56:10 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 26 Aug 2007 07:56:10 +0300 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <1188082569.3730.107.camel@brick.pathscale.com> References: <1188082569.3730.107.camel@brick.pathscale.com> Message-ID: <20070826045609.GA15362@mellanox.co.il> I haven't gone over all of the patch yet, but at least this part is of the claims you make seems wrong: > NOTE that srq_ring[id].skb is *not* updated to point to newskb and > thus will be used again after being passed to the Linux network stack. It is, too. new skb is allocated with ipoib_cm_alloc_rx_skb, and that has priv->cm.srq_ring[id].skb = skb; return skb; > @@ -463,6 +470,7 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) > > ipoib_cm_dma_unmap_rx(priv, frags, priv->cm.srq_ring[wr_id].mapping); > memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, (frags + 1) * sizeof *mapping); > + priv->cm.srq_ring[wr_id].skb = newskb; > > ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", > wc->byte_len, wc->slid); This assignment is already done as part of ipoib_cm_alloc_rx_skb. What am I missing? -- MST From mst at dev.mellanox.co.il Sat Aug 25 22:03:41 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 26 Aug 2007 08:03:41 +0300 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <1188082569.3730.107.camel@brick.pathscale.com> References: <1188082569.3730.107.camel@brick.pathscale.com> Message-ID: <20070826050341.GB15362@mellanox.co.il> The following claim seems wrong, too: > NOTE that [frags..IPOIB_CM_RX_SG-1] are *not* copied to newskb and > thus leaked. Also, newskb's nr_frags is set based on the message size > rather than the full size. Not really. nr_frags *starts* at the message size. skb_fill_page_desc will fill in the fargment in toskb, and increment nr_frags until it matches the full size. > @@ -378,23 +382,26 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space, > skb->len += size; > length -= size; > > - num_frags = skb_shinfo(skb)->nr_frags; > + num_frags = skb_shinfo(toskb)->nr_frags; > for (i = 0; i < num_frags; i++) { > skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; > > - if (length == 0) { > - /* don't need this page */ > - skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); > - --skb_shinfo(skb)->nr_frags; > - } else { > - size = min(length, (unsigned) PAGE_SIZE); > - > - frag->size = size; > - skb->data_len += size; > - skb->truesize += size; > - skb->len += size; > - length -= size; > - } > + size = min(length, (unsigned) PAGE_SIZE); > + WARN_ON(size == 0); > + frag->size = size; > + skb->data_len += size; > + skb->truesize += size; > + skb->len += size; > + length -= size; > + } > + WARN_ON(length != 0); > + skb_shinfo(skb)->nr_frags = num_frags; > + > + /* move unused pages from skb to toskb */ > + for (; i < IPOIB_CM_RX_SG - 1; i++) { > + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; > + > + skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); > } > } > I don't see how this chunk changes anything. It seems you have basically replaced a single loop with a condition by two loops. -- MST From mst at dev.mellanox.co.il Sat Aug 25 22:08:15 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Sun, 26 Aug 2007 08:08:15 +0300 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <1188082569.3730.107.camel@brick.pathscale.com> References: <1188082569.3730.107.camel@brick.pathscale.com> Message-ID: <20070826050815.GC15362@mellanox.co.il> > Also, use ib_dma_unmap_page() for addresses mapped by ib_dma_map_page(). > > Signed-off-by: Ralph Campbell dma_unmap_page and dma_unmap_single are equivalent for all architectures I cared to check, but I agree this might be cleaner as dma_unmap_page. So the following 2 chunks make some sense to me. > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > index 08b4676..a3434f2 100644 > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > @@ -78,7 +78,7 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags, > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > for (i = 0; i < frags; ++i) > - ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > + ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > } > > static int ipoib_cm_post_receive(struct net_device *dev, int id) > @@ -149,7 +149,7 @@ partial_error: > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > for (; i > 0; --i) > - ib_dma_unmap_single(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > + ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > > dev_kfree_skb_any(skb); > return NULL; Is this a cosmetic patch as it seems, or are the ipath implementations of ib_dma_unmap_page and ib_dma_unmap_single significantly different? -- MST From sashak at voltaire.com Sat Aug 25 23:56:22 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 26 Aug 2007 09:56:22 +0300 Subject: [ofa-general] [PATCH] opensm/libvendor: add complib to the linker line Message-ID: <20070826065622.GD18082@sashak.voltaire.com> This adds libosmcomp to the linker command line. Signed-off-by: Sasha Khapyorsky --- opensm/libvendor/Makefile.am | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/opensm/libvendor/Makefile.am b/opensm/libvendor/Makefile.am index 3be6479..3b8c3af 100644 --- a/opensm/libvendor/Makefile.am +++ b/opensm/libvendor/Makefile.am @@ -75,6 +75,7 @@ libosmvendor_la_SOURCES = osm_vendor_mlx.c \ HDRS =$(COMM_HDRS) $(srcdir)/../include/vendor/osm_vendor_mlx.h endif +libosmvendor_la_LIBADD = -L../complib -losmcomp libosmvendor_la_LDFLAGS = -version-info $(osmvendor_api_version) \ -export-dynamic $(libosmvendor_version_script) libosmvendor_la_DEPENDENCIES = $(srcdir)/libosmvendor.map -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sat Aug 25 23:57:17 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 26 Aug 2007 09:57:17 +0300 Subject: [ofa-general] [PATCH] complib: remove osm_log() calls In-Reply-To: <20070826065622.GD18082@sashak.voltaire.com> References: <20070826065622.GD18082@sashak.voltaire.com> Message-ID: <20070826065717.GE18082@sashak.voltaire.com> This removes all osm_log() calls from complib's cl_event_wheel.c. Finally this prevents cross dependencies between libosmcomp and libopensm. Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_event_wheel.c | 170 ++++++++++--------------------- opensm/include/complib/cl_event_wheel.h | 4 - 2 files changed, 55 insertions(+), 119 deletions(-) diff --git a/opensm/complib/cl_event_wheel.c b/opensm/complib/cl_event_wheel.c index 4ad7817..f40a15d 100644 --- a/opensm/complib/cl_event_wheel.c +++ b/opensm/complib/cl_event_wheel.c @@ -42,7 +42,9 @@ #include #include -cl_status_t +#define CL_DBG(fmt, arg...) + +static cl_status_t __event_will_age_before(IN const cl_list_item_t * const p_list_item, IN void *context) { @@ -58,7 +60,7 @@ __event_will_age_before(IN const cl_list_item_t * const p_list_item, return CL_NOT_FOUND; } -void __cl_event_wheel_callback(IN void *context) +static void __cl_event_wheel_callback(IN void *context) { cl_event_wheel_t *p_event_wheel = (cl_event_wheel_t *) context; cl_list_item_t *p_list_item, *p_prev_event_list_item; @@ -69,30 +71,25 @@ void __cl_event_wheel_callback(IN void *context) uint32_t new_timeout; cl_status_t cl_status; - OSM_LOG_ENTER(p_event_wheel->p_log, __cl_event_wheel_callback); - /* might be during closing ... */ - if (p_event_wheel->closing) { - goto JustExit; - } + if (p_event_wheel->closing) + return; current_time = cl_get_time_stamp(); - if (NULL != p_event_wheel->p_external_lock) { + if (NULL != p_event_wheel->p_external_lock) /* Take care of the order of acquiring locks to avoid the deadlock! * The external lock goes first. */ - CL_SPINLOCK_ACQUIRE(p_event_wheel->p_external_lock); - } + cl_spinlock_acquire(p_event_wheel->p_external_lock); - CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + cl_spinlock_acquire(&p_event_wheel->lock); p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); - if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) { + if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) /* the list is empty - nothing to do */ goto Exit; - } /* we found such an item. get the p_event */ p_event = @@ -100,14 +97,13 @@ void __cl_event_wheel_callback(IN void *context) while (p_event->aging_time <= current_time) { /* this object has aged - invoke it's callback */ - if (p_event->pfn_aged_callback) { + if (p_event->pfn_aged_callback) next_aging_time = p_event->pfn_aged_callback(p_event->key, p_event->num_regs, p_event->context); - } else { + else next_aging_time = 0; - } /* point to the next object in the wheel */ p_list_next_item = cl_qlist_next(p_list_item); @@ -153,10 +149,9 @@ void __cl_event_wheel_callback(IN void *context) /* advance to next event */ p_list_item = p_list_next_item; - if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) { + if (p_list_item == cl_qlist_end(&p_event_wheel->events_wheel)) /* the list is empty - nothing to do */ break; - } /* get the p_event */ p_event = @@ -175,25 +170,20 @@ void __cl_event_wheel_callback(IN void *context) new_timeout = (uint32_t) (((p_event->aging_time - current_time) / 1000) + 0.5); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "__cl_event_wheel_callback : " - "Restart timer in : %u [msec]\n", new_timeout); + CL_DBG("__cl_event_wheel_callback: Restart timer in: " + "%u [msec]\n", new_timeout); cl_status = cl_timer_start(&p_event_wheel->timer, new_timeout); if (cl_status != CL_SUCCESS) { - osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, - "__cl_event_wheel_callback : ERR 6100: " - "Failed to start timer\n"); + CL_DBG("__cl_event_wheel_callback : ERR 6100: " + "Failed to start timer\n"); } } /* release the lock */ Exit: - CL_SPINLOCK_RELEASE(&p_event_wheel->lock); - if (NULL != p_event_wheel->p_external_lock) { - CL_SPINLOCK_RELEASE(p_event_wheel->p_external_lock); - } - JustExit: - OSM_LOG_EXIT(p_event_wheel->p_log); + cl_spinlock_release(&p_event_wheel->lock); + if (NULL != p_event_wheel->p_external_lock) + cl_spinlock_release(p_event_wheel->p_external_lock); } /* @@ -211,34 +201,19 @@ cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, { cl_status_t cl_status = CL_SUCCESS; - OSM_LOG_ENTER(p_log, cl_event_wheel_init); - /* initialize */ - p_event_wheel->p_log = p_log; p_event_wheel->p_external_lock = NULL; p_event_wheel->closing = FALSE; cl_status = cl_spinlock_init(&(p_event_wheel->lock)); - if (cl_status != CL_SUCCESS) { - osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_init : ERR 6101: " - "Failed to initialize cl_spinlock\n"); - goto Exit; - } + if (cl_status != CL_SUCCESS) + return cl_status; cl_qlist_init(&p_event_wheel->events_wheel); cl_qmap_init(&p_event_wheel->events_map); /* init the timer with timeout */ cl_status = cl_timer_init(&p_event_wheel->timer, __cl_event_wheel_callback, p_event_wheel); /* cb context */ - if (cl_status != CL_SUCCESS) { - osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_init : ERR 6102: " - "Failed to initialize cl_timer\n"); - goto Exit; - } - Exit: - OSM_LOG_EXIT(p_event_wheel->p_log); - return (cl_status); + return cl_status; } cl_status_t @@ -247,7 +222,7 @@ cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, { cl_status_t cl_status; - cl_status = cl_event_wheel_init(p_event_wheel, p_log); + cl_status = cl_event_wheel_init(p_event_wheel, NULL); if (CL_SUCCESS != cl_status) { return cl_status; } @@ -261,23 +236,17 @@ void cl_event_wheel_dump(IN cl_event_wheel_t * const p_event_wheel) cl_list_item_t *p_list_item; cl_event_wheel_reg_info_t *p_event; - OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_dump); - p_list_item = cl_qlist_head(&p_event_wheel->events_wheel); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_dump: " "event_wheel ptr:%p\n", p_event_wheel); while (p_list_item != cl_qlist_end(&p_event_wheel->events_wheel)) { p_event = PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_dump: " "Found event key:<0x%" PRIx64 - ">, aging time:%" PRIu64 "\n", p_event->key, - p_event->aging_time); + CL_DBG("cl_event_wheel_dump: Found event key:<0x%" + PRIx64 ">, aging time:%" PRIu64 "\n", + p_event->key, p_event->aging_time); p_list_item = cl_qlist_next(p_list_item); } - OSM_LOG_EXIT(p_event_wheel->p_log); } void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel) @@ -286,10 +255,8 @@ void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel) cl_map_item_t *p_map_item; cl_event_wheel_reg_info_t *p_event; - OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_destroy); - /* we need to get a lock */ - CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + cl_spinlock_acquire(&p_event_wheel->lock); cl_event_wheel_dump(p_event_wheel); @@ -300,10 +267,8 @@ void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel) PARENT_STRUCT(p_list_item, cl_event_wheel_reg_info_t, list_item); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_destroy: " - "Found outstanding event key:<0x%" PRIx64 ">\n", - p_event->key); + CL_DBG("cl_event_wheel_destroy: Found outstanding event" + " key:<0x%" PRIx64 ">\n", p_event->key); /* remove it from the map */ p_map_item = &(p_event->map_item); @@ -318,10 +283,8 @@ void cl_event_wheel_destroy(IN cl_event_wheel_t * const p_event_wheel) /* destroy the lock (this should be done without releasing - we don't want any other run to grab the lock at this point. */ - CL_SPINLOCK_RELEASE(&p_event_wheel->lock); + cl_spinlock_release(&p_event_wheel->lock); cl_spinlock_destroy(&(p_event_wheel->lock)); - - OSM_LOG_EXIT(p_event_wheel->p_log); } cl_status_t @@ -338,19 +301,16 @@ cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, cl_list_item_t *prev_event_list_item; cl_map_item_t *p_map_item; - OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_reg); - /* Get the lock on the manager */ - CL_SPINLOCK_ACQUIRE(&(p_event_wheel->lock)); + cl_spinlock_acquire(&(p_event_wheel->lock)); cl_event_wheel_dump(p_event_wheel); /* Make sure such a key does not exists */ p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_reg: " - "Already exists key:0x%" PRIx64 "\n", key); + CL_DBG("cl_event_wheel_reg: Already exists key:0x%" + PRIx64 "\n", key); /* already there - remove it from the list as it is getting a new time */ p_event = @@ -376,12 +336,10 @@ cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, p_event->context = context; p_event->num_regs++; - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_reg: " - "Registering event key:0x%" PRIx64 " aging in %u [msec]\n", - p_event->key, - (uint32_t) ((p_event->aging_time - - cl_get_time_stamp()) / 1000)); + CL_DBG("cl_event_wheel_reg: Registering event key:0x%" PRIx64 + " aging in %u [msec]\n", p_event->key, + (uint32_t) ((p_event->aging_time - + cl_get_time_stamp()) / 1000)); /* If the list is empty - need to start the timer */ if (cl_is_qlist_empty(&p_event_wheel->events_wheel)) { @@ -413,18 +371,15 @@ cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, to = (uint32_t) timeout; if (timeout > (uint32_t) timeout) { to = 0xffffffff; /* max 32 bit timer */ - osm_log(p_event_wheel->p_log, OSM_LOG_INFO, - "cl_event_wheel_reg: " - "timeout requested is too large. Using timeout: %u\n", - to); + CL_DBG("cl_event_wheel_reg: timeout requested is " + "too large. Using timeout: %u\n", to); } /* start the timer to the timeout [msec] */ cl_status = cl_timer_start(&p_event_wheel->timer, to); if (cl_status != CL_SUCCESS) { - osm_log(p_event_wheel->p_log, OSM_LOG_ERROR, - "cl_event_wheel_reg : ERR 6103: " - "Failed to start timer\n"); + CL_DBG("cl_event_wheel_reg : ERR 6103: " + "Failed to start timer\n"); goto Exit; } } @@ -443,8 +398,7 @@ cl_event_wheel_reg(IN cl_event_wheel_t * const p_event_wheel, cl_qmap_insert(&p_event_wheel->events_map, key, &(p_event->map_item)); Exit: - CL_SPINLOCK_RELEASE(&p_event_wheel->lock); - OSM_LOG_EXIT(p_event_wheel->p_log); + cl_spinlock_release(&p_event_wheel->lock); return cl_status; } @@ -455,12 +409,9 @@ cl_event_wheel_unreg(IN cl_event_wheel_t * const p_event_wheel, IN uint64_t key) cl_event_wheel_reg_info_t *p_event; cl_map_item_t *p_map_item; - OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_unreg); + CL_DBG("cl_event_wheel_unreg: " "Removing key:0x%" PRIx64 "\n", key); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " "Removing key:0x%" PRIx64 "\n", key); - - CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + cl_spinlock_acquire(&p_event_wheel->lock); p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { /* we found such an item. */ @@ -475,21 +426,17 @@ cl_event_wheel_unreg(IN cl_event_wheel_t * const p_event_wheel, IN uint64_t key) cl_qmap_remove_item(&p_event_wheel->events_map, &(p_event->map_item)); - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " - "Removed key:0x%" PRIx64 "\n", key); + CL_DBG("cl_event_wheel_unreg: Removed key:0x%" PRIx64 "\n", + key); /* free the item */ free(p_event); } else { - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_unreg: " - "Did not find key:0x%" PRIx64 "\n", key); + CL_DBG("cl_event_wheel_unreg: did not find key:0x%" PRIx64 + "\n", key); } - CL_SPINLOCK_RELEASE(&p_event_wheel->lock); - OSM_LOG_EXIT(p_event_wheel->p_log); - + cl_spinlock_release(&p_event_wheel->lock); } uint32_t @@ -501,14 +448,11 @@ cl_event_wheel_num_regs(IN cl_event_wheel_t * const p_event_wheel, cl_map_item_t *p_map_item; uint32_t num_regs = 0; - OSM_LOG_ENTER(p_event_wheel->p_log, cl_event_wheel_num_regs); - /* try to find the key in the map */ - osm_log(p_event_wheel->p_log, OSM_LOG_DEBUG, - "cl_event_wheel_num_regs: " - "Looking for key:0x%" PRIx64 "\n", key); + CL_DBG("cl_event_wheel_num_regs: Looking for key:0x%" + PRIx64 "\n", key); - CL_SPINLOCK_ACQUIRE(&p_event_wheel->lock); + cl_spinlock_acquire(&p_event_wheel->lock); p_map_item = cl_qmap_get(&p_event_wheel->events_map, key); if (p_map_item != cl_qmap_end(&p_event_wheel->events_map)) { /* ok so we can simply return it's num_regs */ @@ -518,8 +462,7 @@ cl_event_wheel_num_regs(IN cl_event_wheel_t * const p_event_wheel, num_regs = p_event->num_regs; } - CL_SPINLOCK_RELEASE(&p_event_wheel->lock); - OSM_LOG_EXIT(p_event_wheel->p_log); + cl_spinlock_release(&p_event_wheel->lock); return (num_regs); } @@ -577,17 +520,14 @@ void __test_event_aging(uint64_t key, void *context) int main() { - osm_log_t log; cl_event_wheel_t event_wheel; /* uint64_t key; */ /* construct */ - osm_log_construct(&log); cl_event_wheel_construct(&event_wheel); /* init */ - osm_log_init_v2(&log, TRUE, 0xff, NULL, 0, FALSE); - cl_event_wheel_init(&event_wheel, &log); + cl_event_wheel_init(&event_wheel, NULL); /* Start Playing */ cl_event_wheel_reg(&event_wheel, 1, /* key */ diff --git a/opensm/include/complib/cl_event_wheel.h b/opensm/include/complib/cl_event_wheel.h index 056ce99..ab34bc5 100644 --- a/opensm/include/complib/cl_event_wheel.h +++ b/opensm/include/complib/cl_event_wheel.h @@ -148,7 +148,6 @@ typedef struct _cl_event_wheel { boolean_t closing; cl_qlist_t events_wheel; cl_timer_t timer; - osm_log_t *p_log; } cl_event_wheel_t; /* * FIELDS @@ -172,9 +171,6 @@ typedef struct _cl_event_wheel { * timer * The timer scheduling event time propagation. * -* p_log -* Pointer to opensm log object. -* * SEE ALSO * Event_Wheel *********/ -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Sat Aug 25 23:59:31 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Sun, 26 Aug 2007 09:59:31 +0300 Subject: [ofa-general] [PATCH] opensm/complib: remove osm_log usage from complib's API In-Reply-To: <20070826065717.GE18082@sashak.voltaire.com> References: <20070826065622.GD18082@sashak.voltaire.com> <20070826065717.GE18082@sashak.voltaire.com> Message-ID: <20070826065931.GF18082@sashak.voltaire.com> Remove pointer to osm_log from cl_event_wheel_init() parameters list. This changes API, so complib version is bumped. Signed-off-by: Sasha Khapyorsky --- opensm/complib/cl_event_wheel.c | 12 +++++------- opensm/complib/configure.in | 2 +- opensm/complib/libosmcomp.map | 2 +- opensm/complib/libosmcomp.ver | 2 +- opensm/include/complib/cl_event_wheel.h | 11 +---------- opensm/opensm/osm_trap_rcv.c | 7 ++++++- 6 files changed, 15 insertions(+), 21 deletions(-) diff --git a/opensm/complib/cl_event_wheel.c b/opensm/complib/cl_event_wheel.c index f40a15d..f83c47e 100644 --- a/opensm/complib/cl_event_wheel.c +++ b/opensm/complib/cl_event_wheel.c @@ -196,8 +196,7 @@ void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel) } cl_status_t -cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, - IN osm_log_t * p_log) +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel) { cl_status_t cl_status = CL_SUCCESS; @@ -218,14 +217,13 @@ cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, cl_status_t cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, - IN osm_log_t * p_log, IN cl_spinlock_t * p_external_lock) + IN cl_spinlock_t * p_external_lock) { cl_status_t cl_status; - cl_status = cl_event_wheel_init(p_event_wheel, NULL); - if (CL_SUCCESS != cl_status) { + cl_status = cl_event_wheel_init(p_event_wheel); + if (CL_SUCCESS != cl_status) return cl_status; - } p_event_wheel->p_external_lock = p_external_lock; return cl_status; @@ -527,7 +525,7 @@ int main() cl_event_wheel_construct(&event_wheel); /* init */ - cl_event_wheel_init(&event_wheel, NULL); + cl_event_wheel_init(&event_wheel); /* Start Playing */ cl_event_wheel_reg(&event_wheel, 1, /* key */ diff --git a/opensm/complib/configure.in b/opensm/complib/configure.in index 2e2bd59..6abb264 100644 --- a/opensm/complib/configure.in +++ b/opensm/complib/configure.in @@ -1,7 +1,7 @@ dnl Process this file with autoconf to produce a configure script. AC_PREREQ(2.57) -AC_INIT(complib, 2.2.1, general at lists.openfabrics.org) +AC_INIT(complib, 2.2.2, general at lists.openfabrics.org) AC_CONFIG_SRCDIR([cl_spinlock.c]) AC_CONFIG_AUX_DIR(config) AM_CONFIG_HEADER(config.h) diff --git a/opensm/complib/libosmcomp.map b/opensm/complib/libosmcomp.map index 2a1ba63..6361554 100644 --- a/opensm/complib/libosmcomp.map +++ b/opensm/complib/libosmcomp.map @@ -1,4 +1,4 @@ -OSMCOMP_2.0 { +OSMCOMP_2.1 { global: complib_init; complib_exit; diff --git a/opensm/complib/libosmcomp.ver b/opensm/complib/libosmcomp.ver index fab9776..ab7e34e 100644 --- a/opensm/complib/libosmcomp.ver +++ b/opensm/complib/libosmcomp.ver @@ -6,4 +6,4 @@ # API_REV - advance on any added API # RUNNING_REV - advance any change to the vendor files # AGE - number of backward versions the API still supports -LIBVERSION=2:0:0 +LIBVERSION=2:1:0 diff --git a/opensm/include/complib/cl_event_wheel.h b/opensm/include/complib/cl_event_wheel.h index ab34bc5..ba466c6 100644 --- a/opensm/include/complib/cl_event_wheel.h +++ b/opensm/include/complib/cl_event_wheel.h @@ -51,7 +51,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -261,17 +260,13 @@ void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel); * SYNOPSIS */ cl_status_t -cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, - IN osm_log_t * p_log); +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel); /* * PARAMETERS * p_event_wheel * [in] Pointer to a Event_Wheel. * -* p_log -* [in] Pointer to opensm log object to be used for logging -* * RETURN VALUE * CL_SUCCESS if the operation is successful. * @@ -291,7 +286,6 @@ cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, */ cl_status_t cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, - IN osm_log_t * p_log, IN cl_spinlock_t * p_external_lock); /* @@ -299,9 +293,6 @@ cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, * p_event_wheel * [in] Pointer to a Event_Wheel. * -* p_log -* [in] Pointer to opensm log object to be used for logging -* * p_external_lock * [in] Reference to external spinlock to guard internal structures * if the event wheel is part of a larger object protected by its own lock diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index 86aac94..babc625 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -203,7 +203,12 @@ osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, p_rcv->p_resp = p_resp; p_rcv->p_state_mgr = p_state_mgr; - cl_event_wheel_init(&p_rcv->trap_aging_tracker, p_log); + if(cl_event_wheel_init(&p_rcv->trap_aging_tracker)) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_trap_rcv_init: ERR 3800: " + "Failed to initialize cl_event_wheel\n"); + status = IB_NOT_DONE; + } OSM_LOG_EXIT(p_rcv->p_log); return (status); -- 1.5.3.rc2.38.g11308 From billfink at mindspring.com Sun Aug 26 01:41:34 2007 From: billfink at mindspring.com (Bill Fink) Date: Sun, 26 Aug 2007 04:41:34 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <46CF7B13.3020701@psc.edu> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> <46CF7B13.3020701@psc.edu> Message-ID: <20070826044134.eabd18cf.billfink@mindspring.com> On Fri, 24 Aug 2007, John Heffner wrote: > Bill Fink wrote: > > Here you can see there is a major difference in the TX CPU utilization > > (99 % with TSO disabled versus only 39 % with TSO enabled), although > > the TSO disabled case was able to squeeze out a little extra performance > > from its extra CPU utilization. Interestingly, with TSO enabled, the > > receiver actually consumed more CPU than with TSO disabled, so I guess > > the receiver CPU saturation in that case (99 %) was what restricted > > its performance somewhat (this was consistent across a few test runs). > > One possibility is that I think the receive-side processing tends to do > better when receiving into an empty queue. When the (non-TSO) sender is > the flow's bottleneck, this is going to be the case. But when you > switch to TSO, the receiver becomes the bottleneck and you're always > going to have to put the packets at the back of the receive queue. This > might help account for the reason why you have both lower throughput and > higher CPU utilization -- there's a point of instability right where the > receiver becomes the bottleneck and you end up pushing it over to the > bad side. :) > > Just a theory. I'm honestly surprised this effect would be so > significant. What do the numbers from netstat -s look like in the two > cases? Well, I was going to check this out, but I happened to reboot the system and now I get somewhat different results. Here are the new results, which should hopefully be more accurate since they are on a freshly booted system. TSO enabled and GSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11610.6875 MB / 10.00 sec = 9735.9526 Mbps 100 %TX 75 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5029.6875 MB / 10.06 sec = 4194.6931 Mbps 36 %TX 100 %RX TSO disabled and GSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11817.9375 MB / 10.00 sec = 9909.7773 Mbps 99 %TX 77 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5823.3125 MB / 10.00 sec = 4883.2429 Mbps 100 %TX 82 %RX The TSO disabled case got a little better performance even for 9000 byte jumbo frames. For the "-M1460" case eumalating a standard 1500 byte Ethernet MTU, the performance was significantly better and used less CPU on the receiver (82 % versus 100 %) although it did use significantly more CPU on the transmitter (100 % versus 36 %). TSO disabled and GSO enabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11609.5625 MB / 10.00 sec = 9734.9859 Mbps 99 %TX 75 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5001.4375 MB / 10.06 sec = 4170.6739 Mbps 52 %TX 100 %RX The GSO enabled case is very similar to the TSO enabled case, except that for the "-M1460" test the transmitter used more CPU (52 % versus 36 %), which is to be expected since TSO has hardware assist. Here's the beforeafter delta of the receiver's "netstat -s" statistics for the TSO enabled case: Ip: 3659898 total packets received 3659898 incoming packets delivered 80050 requests sent out Tcp: 2 passive connection openings 3659897 segments received 80050 segments send out TcpExt: 33 packets directly queued to recvmsg prequeue. 104956 packets directly received from backlog 705528 packets directly received from prequeue 3654842 packets header predicted 193 packets header predicted and directly queued to user 4 acknowledgments not containing data received 6 predicted acknowledgments And here it is for the TSO disabled case (GSO also disabled): Ip: 4107083 total packets received 4107083 incoming packets delivered 1401376 requests sent out Tcp: 2 passive connection openings 4107083 segments received 1401376 segments send out TcpExt: 2 TCP sockets finished time wait in fast timer 48486 packets directly queued to recvmsg prequeue. 1056111048 packets directly received from backlog 2273357712 packets directly received from prequeue 1819317 packets header predicted 2287497 packets header predicted and directly queued to user 4 acknowledgments not containing data received 10 predicted acknowledgments For the TSO disabled case, there are a huge amount more TCP segments sent out (1401376 versus 80050), which I assume are ACKs, and which could possibly contribute to the higher throughput for the TSO disabled case due to faster feedback, but not explain the lower CPU utilization. There are many more packets directly queued to recvmsg prequeue (48486 versus 33). The numbers for packets directly received from backlog and prequeue in the TCP disabled case seem bogus to me so I don't know how to interpret that. There are only about half as many packets header predicted (1819317 versus 3654842), but there are many more packets header predicted and directly queued to user (2287497 versus 193). I'll leave the analysis of all this to those who might actually know what it all means. I also ran another set of tests that may be of interest. I changed the rx-usecs/tx-usecs interrupt coalescing parameter from the recommended optimum value of 75 usecs to 0 (no coalescing), but only on the transmitter. The comparison discussions below are relative to the previous tests where rx-usecs/tx-usecs were set to 75 usecs. TSO enabled and GSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11812.8125 MB / 10.00 sec = 9905.6640 Mbps 100 %TX 75 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 7701.8750 MB / 10.00 sec = 6458.5541 Mbps 100 %TX 56 %RX For 9000 byte jumbo frames it now gets a little better performance and almost matches the 10-GigE line rate performance of the TSO disabled case. For the "-M1460" test, it gets substantially better performance (6458.5541 Mbps versus 4194.6931 Mbps) at the expense of much higher transmitter CPU utilization (100 % versus 36 %), although the receiver CPU utilization is much less (56 % versus 100 %). TSO disabled and GSO disabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11817.3125 MB / 10.00 sec = 9909.4058 Mbps 100 %TX 76 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 4081.2500 MB / 10.00 sec = 3422.3994 Mbps 99 %TX 41 %RX For 9000 byte jumbo frames the results are essentially the same. For the "-M1460" test, the performance is significantly worse (3422.3994 Mbps versus 4883.2429 Mbps) even though the transmitter CPU utilization is saturated in both cases, but the receiver CPU utilization is about half (41 % versus 82 %). TSO disabled and GSO enabled: [root at lang2 ~]# nuttcp -w10m 192.168.88.16 11813.3750 MB / 10.00 sec = 9906.1090 Mbps 99 %TX 77 %RX [root at lang2 ~]# nuttcp -M1460 -w10m 192.168.88.16 3939.1875 MB / 10.00 sec = 3303.2814 Mbps 100 %TX 41 %RX For 9000 byte jumbo frames the performance is a little better, again approaching 10-GigE line rate. But for the "-M1460" test, the performance is significantly worse (3303.2814 Mbps versus 4170.6739 Mbps) even though the transmitter consumes much more CPU (100 % versus 52 %). In this case though the receiver has a much lower CPU utilization (41 % versus 100 %). -Bill From vlad at lists.openfabrics.org Sun Aug 26 02:46:14 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Sun, 26 Aug 2007 02:46:14 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070826-0200 daily build status Message-ID: <20070826094614.797E6E6081C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.17 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check/kernel_addons/backport/2.6.15_ubuntu606/include/linux/skbuff.h:22: error: conflicting types for '__skb_checksum_complete' include/linux/skbuff.h:1306: error: previous declaration of '__skb_checksum_complete' was here /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070826-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From jheffner at psc.edu Sun Aug 26 18:32:26 2007 From: jheffner at psc.edu (John Heffner) Date: Sun, 26 Aug 2007 21:32:26 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070826044134.eabd18cf.billfink@mindspring.com> References: <20070821.212229.82050253.davem@davemloft.net> <46CC6DD1.5020105@hp.com> <20070822.132145.90824527.davem@davemloft.net> <1187906650.4279.16.camel@localhost> <1187907903.4279.28.camel@localhost> <46CE0BA1.60206@hp.com> <20070823231820.2ae52cc0.billfink@mindspring.com> <46CF7B13.3020701@psc.edu> <20070826044134.eabd18cf.billfink@mindspring.com> Message-ID: <46D229AA.6020900@psc.edu> Bill Fink wrote: > Here's the beforeafter delta of the receiver's "netstat -s" > statistics for the TSO enabled case: > > Ip: > 3659898 total packets received > 3659898 incoming packets delivered > 80050 requests sent out > Tcp: > 2 passive connection openings > 3659897 segments received > 80050 segments send out > TcpExt: > 33 packets directly queued to recvmsg prequeue. > 104956 packets directly received from backlog > 705528 packets directly received from prequeue > 3654842 packets header predicted > 193 packets header predicted and directly queued to user > 4 acknowledgments not containing data received > 6 predicted acknowledgments > > And here it is for the TSO disabled case (GSO also disabled): > > Ip: > 4107083 total packets received > 4107083 incoming packets delivered > 1401376 requests sent out > Tcp: > 2 passive connection openings > 4107083 segments received > 1401376 segments send out > TcpExt: > 2 TCP sockets finished time wait in fast timer > 48486 packets directly queued to recvmsg prequeue. > 1056111048 packets directly received from backlog > 2273357712 packets directly received from prequeue > 1819317 packets header predicted > 2287497 packets header predicted and directly queued to user > 4 acknowledgments not containing data received > 10 predicted acknowledgments > > For the TSO disabled case, there are a huge amount more TCP segments > sent out (1401376 versus 80050), which I assume are ACKs, and which > could possibly contribute to the higher throughput for the TSO disabled > case due to faster feedback, but not explain the lower CPU utilization. > There are many more packets directly queued to recvmsg prequeue > (48486 versus 33). The numbers for packets directly received from > backlog and prequeue in the TCP disabled case seem bogus to me so > I don't know how to interpret that. There are only about half as > many packets header predicted (1819317 versus 3654842), but there > are many more packets header predicted and directly queued to user > (2287497 versus 193). I'll leave the analysis of all this to those > who might actually know what it all means. There are a few interesting things here. For one, the bursts caused by TSO seem to be causing the receiver to do stretch acks. This may have a negative impact on flow performance, but it's hard to say for sure how much. Interestingly, it will even further reduce the CPU load on the sender, since it has to process fewer acks. As I suspected, in the non-TSO case the receiver gets lots of packets directly queued to user. This should result in somewhat lower CPU utilization on the receiver. I don't know if it can account for all the difference you see. The backlog and prequeue values are probably correct, but netstat's description is wrong. A quick look at the code reveals these values are in units of bytes, not packets. -John From kliteyn at mellanox.co.il Sun Aug 26 21:43:05 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 27 Aug 2007 07:43:05 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-27:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_23_14:46:48_2007 [05cae48f68250346dc02b3583044664d6ca6474e] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From vlad at lists.openfabrics.org Mon Aug 27 02:46:45 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Mon, 27 Aug 2007 02:46:45 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070827-0200 daily build status Message-ID: <20070827094645.2628BE6082C@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.15 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.22 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on i686 with 2.6.15-23-server Log: /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check/kernel_addons/backport/2.6.15_ubuntu606/include/linux/skbuff.h:22: error: conflicting types for '__skb_checksum_complete' include/linux/skbuff.h:1306: error: previous declaration of '__skb_checksum_complete' was here /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_check] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.15-23-server' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070827-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From eitan at mellanox.co.il Mon Aug 27 03:30:58 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Mon, 27 Aug 2007 13:30:58 +0300 Subject: [ofa-general] RE: [PATCH][TRIVIAL] ibdiag/src/ibdebug_if.tcl: Fix typo In-Reply-To: References: Message-ID: <6C2C79E72C305246B504CBA17B5500C902257C1B@mtlexch01.mtl.com> Thanks done. Eitan Zahavi Senior Engineering Director, Software Architect Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -----Original Message----- > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com] > Sent: Monday, August 13, 2007 11:56 PM > To: Eitan Zahavi > Cc: OpenIB > Subject: [PATCH][TRIVIAL] ibdiag/src/ibdebug_if.tcl: Fix typo > > ibdiag/src/ibdebug_if.tcl: Fix typo > > Signed-off-by: Hal Rosenstock > > diff --git a/ibdiag/src/ibdebug_if.tcl b/ibdiag/src/ibdebug_if.tcl > index fa769af..2504874 100644 > --- a/ibdiag/src/ibdebug_if.tcl > +++ b/ibdiag/src/ibdebug_if.tcl > @@ -1726,7 +1726,7 @@ proc inform { msgCode args } { > } > "-W-ibdiagnet:bad.pm.counter.report" { > append msgText "$NODE(0,FullNamePort_Last)%n" > - append msgText " Performence Monitor counter" > + append msgText " Performance Monitor counter" > append msgText "[string repeat " " [expr > [GetLengthMaxWord $G(var:list > append msgText "Value" > From tziporet at dev.mellanox.co.il Mon Aug 27 06:05:14 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 27 Aug 2007 16:05:14 +0300 Subject: [ofa-general] [RFC, PATCH 00/20] svc: Server Side Transport Switch In-Reply-To: <20070820185351.18466.73527.stgit@dell3.ogc.int> References: <20070820185351.18466.73527.stgit@dell3.ogc.int> Message-ID: <46D2CC0A.2030003@mellanox.co.il> Tom Tucker wrote: > This patchest modifies the RPC server side implementation > to support pluggable transports. This was done in order to allow > RPC applications (NFS) to run over RDMA transports like IB and iWARP. > This patchset was also published to nfs at lists.sourceforge.net. > Hi Tom, > There was a request to include NFS over RDMA package in OFED 1.3 Do you have any plans to prepare this package for inclusion in OFED 1.3? If yes please note it should be ready by be end of next week. Thanks, Tziporet From tziporet at dev.mellanox.co.il Mon Aug 27 09:07:38 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Mon, 27 Aug 2007 19:07:38 +0300 Subject: [ofa-general] Agenda for today's meeting Message-ID: <46D2F6CA.2050103@mellanox.co.il> Hi, The agenda for today's meeting: OFED 1.3 development status: I think we need to delay the feature freeze in a week. We can close the exact date in the meeting. QoS patches to the kernel - any update on the status MPI updates - if any Inputs on the new install method Tziporet From ralph.campbell at qlogic.com Mon Aug 27 09:30:03 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Mon, 27 Aug 2007 09:30:03 -0700 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <20070826045609.GA15362@mellanox.co.il> References: <1188082569.3730.107.camel@brick.pathscale.com> <20070826045609.GA15362@mellanox.co.il> Message-ID: <1188232203.3730.128.camel@brick.pathscale.com> On Sun, 2007-08-26 at 07:56 +0300, Michael S. Tsirkin wrote: > I haven't gone over all of the patch yet, but at least this part is > of the claims you make seems wrong: > > > NOTE that srq_ring[id].skb is *not* updated to point to newskb and > > thus will be used again after being passed to the Linux network stack. > > It is, too. new skb is allocated with ipoib_cm_alloc_rx_skb, and that has > priv->cm.srq_ring[id].skb = skb; > return skb; > > > > @@ -463,6 +470,7 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) > > > > ipoib_cm_dma_unmap_rx(priv, frags, priv->cm.srq_ring[wr_id].mapping); > > memcpy(priv->cm.srq_ring[wr_id].mapping, mapping, (frags + 1) * sizeof *mapping); > > + priv->cm.srq_ring[wr_id].skb = newskb; > > > > ipoib_dbg_data(priv, "received %d bytes, SLID 0x%04x\n", > > wc->byte_len, wc->slid); > > This assignment is already done as part of ipoib_cm_alloc_rx_skb. Ah, yes. > What am I missing? Nothing that I can see. I agree with you. From ralph.campbell at qlogic.com Mon Aug 27 09:33:20 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Mon, 27 Aug 2007 09:33:20 -0700 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <20070826050815.GC15362@mellanox.co.il> References: <1188082569.3730.107.camel@brick.pathscale.com> <20070826050815.GC15362@mellanox.co.il> Message-ID: <1188232400.3730.132.camel@brick.pathscale.com> On Sun, 2007-08-26 at 08:08 +0300, Michael S. Tsirkin wrote: > > Also, use ib_dma_unmap_page() for addresses mapped by ib_dma_map_page(). > > > > Signed-off-by: Ralph Campbell > > dma_unmap_page and dma_unmap_single are equivalent for all architectures I cared > to check, but I agree this might be cleaner as dma_unmap_page. So the following > 2 chunks make some sense to me. > > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > index 08b4676..a3434f2 100644 > > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > @@ -78,7 +78,7 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags, > > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > > > for (i = 0; i < frags; ++i) > > - ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > > + ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > > } > > > > static int ipoib_cm_post_receive(struct net_device *dev, int id) > > @@ -149,7 +149,7 @@ partial_error: > > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > > > for (; i > 0; --i) > > - ib_dma_unmap_single(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > > + ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > > > > dev_kfree_skb_any(skb); > > return NULL; > > Is this a cosmetic patch as it seems, or are the ipath implementations of > ib_dma_unmap_page and ib_dma_unmap_single significantly different? The ipath implementation for unmap doesn't differ significantly but for consistency and possible future changes, I would like to see the correct unmap calls being made. From hal.rosenstock at gmail.com Mon Aug 27 09:33:16 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 27 Aug 2007 12:33:16 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <46CD6CC1.30403@dev.mellanox.co.il> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> Message-ID: On 8/23/07, Yevgeny Kliteynik wrote: > Hi Sasha, > > Sasha Khapyorsky wrote: > > On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_port_group_t_ { > >>>> + char *name; /* single string (this port group name) */ > >>>> + char *use; /* single string (description) */ > >>>> + cl_list_t port_name_list; /* list of port names (.../.../...) */ > >>>> + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit > >>>> guids) */ > >>>> + unsigned guid_range_len; /* num of guid ranges in the array */ > >>>> + cl_list_t partition_list; /* list of partition names */ > >>>> + boolean_t node_type_ca; > >>>> + boolean_t node_type_switch; > >>>> + boolean_t node_type_router; > >>>> + boolean_t node_type_self; > >>>> +} osm_qos_port_group_t; > >>> I see you are using this in "run-time", not just during the parsing. > >>> Instead of having all this config features you can just resolve port > >>> guids in parse time and keep it here in cl_map() for fast searches. > >> By saying "config features", do you mean the four boolean flags? > > > > No, I mean everything except name and use. > > > >> It looks to me that checking the type of node is as fast as it gets, > >> and it won't hurt to leave these booleans instead of resolving > >> all the guids. > > > > It could be optimization when types are specified, which is not always > > the case and then you are going to do linear searches over all lists. > > Right, it would be linear scanning of list of partitions. > > > And how something like "for each guid in this group" (which is needed > > for QoS port parameters setup) should be resolved? By matching each guid > > in the subnet against those lists? > > Good point. > > >> Moreover, the guids here are stored in range array, which is IMO > >> better suited for the policy file syntax, because if a user specifies > >> something like this "0x0-0x0FFF" in guids, it will be only one element > >> of the array, which is efficient both in memory and in serch time. > > > > And what should be there if user specifies ports in the group as: > > guid1, guid2, guid3, etc. ? > > The range array is sorted and "shrinked". > That is, if a user specifies guids as "30,1,2,3-20,15", eventually you > would get two elements in the array: 1-20 and 30-30. > cl_map implemented as a binary tree, right? > And doing binary search in this kind of array is faster than searching > a guid in cl_map of all the guids. > Worst case - you will get the same performance as in case of cl_map if > *all* the guids are "discreet" and can't be groupped in ranges. > > Nevertheless, I agree that there's a problem if there are many partitions > in the list (although I'm not sure it's a practical case) > > I'll work on this. > > >> (I probably should mention here that the efficient search in the range > >> array is not implemented yet, but it would be a simple binary search - > >> there's a "todo" comment in the search function right now) > >> > >>>> + > >>>> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > >>>> +void osm_qos_policy_port_group_destroy(); > >>> Would be nice to have function prototypes in one place. > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_vlarb_scope_t_ { > >>>> + cl_list_t group_list; /* list of group names (strings) */ > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values > >>>> */ > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values > >>>> */ > >>> Why cl_list for VLArb? it should be short fixed length arrays? > >> Right. > >> Since the actual VLArb setup is not implemented yet, I didn't see > >> this obvious thing. > > > > But it should be implemented. Right? > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. Without VLArb configuration, where does this come from ? What is the QoS support then: just SL ? -- Hal > > >>>> + uint32_t vl_high_limit; /* single integer */ > >>>> + boolean_t vl_high_limit_set; > >>>> +} osm_qos_vlarb_scope_t; > >>>> + > >>>> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); > >>>> +void osm_qos_policy_vlarb_scope_destroy(); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_sl2vl_scope_t_ { > >>>> + cl_list_t group_list; /* list of strings (port group names) */ > >>>> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > >>>> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > >>>> + cl_list_t across_from_list; /* list of strings (port group names) */ > >>>> + cl_list_t across_to_list; /* list of strings (port group names) */ > >>>> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > >>>> + boolean_t sl2vl_table_set; > >>>> +} osm_qos_sl2vl_scope_t; > >>> This will be used for sl2vl setup? Same as above - why not to generate > >>> final port guid list just in parse time? > >>>> + > >>>> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); > >>>> +void osm_qos_policy_sl2vl_scope_destroy(); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_level_t_ { > >>>> + char *use; > >>>> + char *name; > >>>> + uint8_t sl; > >>>> + boolean_t sl_set; > >>>> + uint8_t mtu_limit; > >>>> + boolean_t mtu_limit_set; > >>>> + uint8_t rate_limit; > >>>> + boolean_t rate_limit_set; > >>>> + uint8_t pkt_life; > >>>> + boolean_t pkt_life_set; > >>>> + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are > >>>> 32bits) */ > >>>> + unsigned path_bits_range_len; /* num of bit ranges in the array */ > >>>> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are > >>>> 16bits) */ > >>>> + unsigned pkey_range_len; > >>>> +} osm_qos_level_t; > >>>> + > >>>> +osm_qos_level_t *osm_qos_policy_qos_level_create(); > >>>> +void osm_qos_policy_qos_level_destroy(); > >>>> + > >>>> +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > >>>> + IN ib_net16_t pkey); > >>>> + > >>>> +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * > >>>> p_qos_level, > >>>> + IN const osm_physp_t * p_src_physp, > >>>> + IN const osm_physp_t * p_dest_physp); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_match_rule_t_ { > >>>> + char *use; > >>>> + cl_list_t source_list; /* list of strings */ > >>>> + cl_list_t source_group_list; /* list of pointers to relevant port-group > >>>> */ > >>>> + cl_list_t destination_list; /* list of strings */ > >>>> + cl_list_t destination_group_list; /* list of pointers to relevant > >>>> port-group */ > >>> I think you should only keep port guids there (mapped for fast searches). > >> Same as above. > >> I think that checking node type and then guid range array is > >> essentially faster than checking guid map. > > > > _Only_ for case when the group is specified by type, which is likely > > will not be typical. > > Again, in *worst* case you will get the same performance in searching > range array as in searching cl_map (which is a binary tree). > > >> You might say, of course, that there can be many port groups > >> in the same match rule, but I don't see this as a practical > >> example. > > > > Of course it could (or you must disable multi groups here, which is > > not good idea I think and not what was presented in the RFC). > > Can you elaborate on this? > Do you think that having multiple groups is not useful at all? > > > Also each group still have this "linear searchable" lists: > > > > cl_list_t port_name_list; /* list of port names (.../.../...) */ > > Port names are not implemented yet :) > There's a "todo" comment in the code. > This is the only keyword in the parser that is not implemented - I mean it's parsed, > but other than creating this list the parser doesn't do anything with it. > But It definitely won't stay as a pure list. > > > uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit > > This is not just a list, as I've explained before. > > > cl_list_t partition_list; /* list of partition names */ > > I can hardly believe than there will be more than one or two partitions > in this list. Do you think otherwise? > > Anyway, I can extract all the guids and prepare a map. This is certainly the > easiest implementation. And I still think that on average, guid ranges and > partition lists are better, but who knows - perhaps someone would want > to define a bunch of partitions in a single port group and ruin my average... :) > > >>>> + char *qos_level_name; > >>>> + osm_qos_level_t *p_qos_level; > >>> Why do you need qos_level_name if you keep the pointer to this qos_level > >>> struct? > >> In policy file the match rule might appear before the QoS levels, > >> so matching qos level names to the actual qos levels is done when > >> the parsing is done. > >> > >>>> + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) > >>>> */ > >>>> + unsigned service_id_range_len; > >>>> + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real > >>>> values are 16bits) */ > >>>> + unsigned qos_class_range_len; > >>>> + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are > >>>> 16bits) */ > >>>> + unsigned pkey_range_len; > >>>> +} osm_qos_match_rule_t; > >>>> + > >>>> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > >>>> +void osm_qos_policy_match_rule_destroy(); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +typedef struct osm_qos_policy_t_ { > >>>> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > >>>> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > >>>> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > >>>> + cl_list_t qos_levels; /* list of osm_qos_level_t */ > >>>> + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ > >>>> + osm_qos_level_t *p_default_qos_level; /* default QoS level */ > >>>> +} osm_qos_policy_t; > >>>> + > >>>> +void osm_qos_policy_create(); > >>>> +void osm_qos_policy_destroy(); > >>>> +int osm_qos_policy_validate(); > >>>> + > >>>> +void osm_qos_policy_get_qos_level_by_pr(IN const osm_pr_rcv_t * p_rcv, > >>>> + IN const ib_path_rec_t * p_pr, > >>>> + IN const osm_physp_t * p_src_physp, > >>>> + IN const osm_physp_t * p_dest_physp, > >>>> + IN ib_net64_t comp_mask, > >>>> + OUT osm_qos_level_t ** pp_qos_level); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const char > >>>> *policy_file); > >>>> + > >>>> +/***************************************************/ > >>>> + > >>>> +#endif /* ifndef OSM_QOS_POLICY_H */ > >>>> diff --git a/opensm/opensm/osm_qos_policy.c > >>>> b/opensm/opensm/osm_qos_policy.c > >>>> new file mode 100644 > >>>> index 0000000..bc2aa68 > >>>> --- /dev/null > >>>> +++ b/opensm/opensm/osm_qos_policy.c > >>>> @@ -0,0 +1,901 @@ > >>>> +/* > >>>> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > >>>> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights > >>>> reserved. > >>>> + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > >>>> + * > >>>> + * This software is available to you under a choice of one of two > >>>> + * licenses. You may choose to be licensed under the terms of the GNU > >>>> + * General Public License (GPL) Version 2, available from the file > >>>> + * COPYING in the main directory of this source tree, or the > >>>> + * OpenIB.org BSD license below: > >>>> + * > >>>> + * Redistribution and use in source and binary forms, with or > >>>> + * without modification, are permitted provided that the following > >>>> + * conditions are met: > >>>> + * > >>>> + * - Redistributions of source code must retain the above > >>>> + * copyright notice, this list of conditions and the following > >>>> + * disclaimer. > >>>> + * > >>>> + * - Redistributions in binary form must reproduce the above > >>>> + * copyright notice, this list of conditions and the following > >>>> + * disclaimer in the documentation and/or other materials > >>>> + * provided with the distribution. > >>>> + * > >>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > >>>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > >>>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > >>>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > >>>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > >>>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > >>>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > >>>> + * SOFTWARE. > >>>> + * > >>>> + */ > >>>> + > >>>> +/* > >>>> + * Abstract: > >>>> + * OSM QoS Policy functions. > >>>> + * > >>>> + * Environment: > >>>> + * Linux User Mode > >>>> + * > >>>> + * Author: > >>>> + * Yevgeny Kliteynik, Mellanox > >>>> + */ > >>>> + > >>>> +#include > >>>> +#include > >>>> +#include > >>>> + > >>>> +extern void yyerror(char *s); > >>>> +osm_log_t *p_qos_parser_osm_log = NULL; > >>>> +osm_qos_policy_t *p_qos_policy = NULL; > >>> Please try to avoid globals - keep it as part of osm_opensm_t or > >>> osm_subn_t structures. > >> I thought about it, but didn't want to "condaminate" the osm_opensm_t > >> or osm_subn_t structures untill the QoS functionality is ready. > > > > How globals are better in this sense? > > They're not :) > The difference is that less files are modified. > But I agree that the QoS policy should be part of osm_opensm_t or osm_subn_t. > > -- Yevgeny > > > Sasha > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Mon Aug 27 09:35:09 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 27 Aug 2007 12:35:09 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070823120325.GA15042@sashak.voltaire.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <20070823120325.GA15042@sashak.voltaire.com> Message-ID: On 8/23/07, Sasha Khapyorsky wrote: > On 14:17 Thu 23 Aug , Yevgeny Kliteynik wrote: > > >>>> + boolean_t node_type_switch; > > >>>> + boolean_t node_type_router; > > >>>> + boolean_t node_type_self; > > >>>> +} osm_qos_port_group_t; > > >>> I see you are using this in "run-time", not just during the parsing. > > >>> Instead of having all this config features you can just resolve port > > >>> guids in parse time and keep it here in cl_map() for fast searches. > > >> By saying "config features", do you mean the four boolean flags? > > > No, I mean everything except name and use. > > >> It looks to me that checking the type of node is as fast as it gets, > > >> and it won't hurt to leave these booleans instead of resolving > > >> all the guids. > > > It could be optimization when types are specified, which is not always > > > the case and then you are going to do linear searches over all lists. > > > > Right, it would be linear scanning of list of partitions. > > And each partition by name search is linear over all configured by > OpenSM partitions (it is "parser-only" stuff in PM), when partition is > resolved port guid will be matched using cl_map anyway. > > BTW partition name is optional as far as I remember. Not sure this is > an issue however search by pkey value will be faster. > > > > And how something like "for each guid in this group" (which is needed > > > for QoS port parameters setup) should be resolved? By matching each guid > > > in the subnet against those lists? > > > > Good point. > > > > >> Moreover, the guids here are stored in range array, which is IMO > > >> better suited for the policy file syntax, because if a user specifies > > >> something like this "0x0-0x0FFF" in guids, it will be only one element > > >> of the array, which is efficient both in memory and in serch time. > > > And what should be there if user specifies ports in the group as: > > > guid1, guid2, guid3, etc. ? > > > > The range array is sorted and "shrinked". > > That is, if a user specifies guids as "30,1,2,3-20,15", eventually you > > would get two elements in the array: 1-20 and 30-30. > > Generic case would be: 30, 1, 2, 16, 17, 23, etc. :) > > > cl_map implemented as a binary tree, right? > > Yes. > > > And doing binary search > > Binary search is not implemented yet... Currently it is linear. > > > in this kind of array is faster than searching > > a guid in cl_map of all the guids. > > Worst case - you will get the same performance as in case of cl_map if > > *all* the guids are "discreet" and can't be groupped in ranges. > > Right. But why do you think it is not typical? (we will not request > from user to re-burn GUIDs for each desired QoS configuration :)) > > > > > Nevertheless, I agree that there's a problem if there are many partitions > > in the list (although I'm not sure it's a practical case) > > > > I'll work on this. > > Thanks. > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > > >>>> values */ > > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > > >>>> values */ > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > >> Right. > > >> Since the actual VLArb setup is not implemented yet, I didn't see > > >> this obvious thing. > > > But it should be implemented. Right? > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > Then we will have two QoS managers in parallel, I don't like this too > much. Isn't this needed ? What about LASH ? How is that supported ? I thought there were extra options or the like to enable the higher level QoS functions (manager) ? -- Hal > OTOH the integration should be easy enough (low level stuff is > ready there), so I will not mind if it will be done in the week after > feature freeze. > > > >>>> +typedef struct osm_qos_match_rule_t_ { > > >>>> + char *use; > > >>>> + cl_list_t source_list; /* list of strings */ > > >>>> + cl_list_t source_group_list; /* list of pointers to relevant > > >>>> port-group */ > > >>>> + cl_list_t destination_list; /* list of strings */ > > >>>> + cl_list_t destination_group_list; /* list of pointers to relevant > > >>>> port-group */ > > >>> I think you should only keep port guids there (mapped for fast searches). > > >> Same as above. > > >> I think that checking node type and then guid range array is > > >> essentially faster than checking guid map. > > > _Only_ for case when the group is specified by type, which is likely > > > will not be typical. > > > > Again, in *worst* case you will get the same performance in searching > > range array as in searching cl_map (which is a binary tree). > > > > >> You might say, of course, that there can be many port groups > > >> in the same match rule, but I don't see this as a practical > > >> example. > > > Of course it could (or you must disable multi groups here, which is > > > not good idea I think and not what was presented in the RFC). > > > > Can you elaborate on this? > > Do you think that having multiple groups is not useful at all? > > I think it _is_ useful. I'm trying to say is that we cannot from one > side to provide some functionality and from another expect that nobody > will use it :). > > > > Also each group still have this "linear searchable" lists: > > > cl_list_t port_name_list; /* list of port names (.../.../...) */ > > > > Port names are not implemented yet :) > > There's a "todo" comment in the code. > > This is the only keyword in the parser that is not implemented - I mean it's > > parsed, > > but other than creating this list the parser doesn't do anything with it. > > But It definitely won't stay as a pure list. > > > > > uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit > > > > This is not just a list, as I've explained before. > > > > > cl_list_t partition_list; /* list of partition names */ > > > > I can hardly believe than there will be more than one or two partitions > > in this list. Do you think otherwise? > > Why not? It is permitted and of course will be useful. > > Also as I stated above even for one partition by name search you will > need to scan all configured partitions (so probably using pkey instead > of partition name is better). > > > Anyway, I can extract all the guids and prepare a map. This is certainly the > > easiest implementation. And I still think that on average, guid ranges > > Maybe with fast search, but not how it is now. > > > and > > partition lists are better, > > Absolutely not, even with one partition. There are always two parts: > Partition search (slow by name or fast by pkey) and guid search in > partition (which is cl_map). > > Sasha > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From parks at lanl.gov Mon Aug 27 09:38:13 2007 From: parks at lanl.gov (Parks Fields) Date: Mon, 27 Aug 2007 10:38:13 -0600 Subject: [ofa-general] Agenda for today's meeting In-Reply-To: <46D2F6CA.2050103@mellanox.co.il> References: <46D2F6CA.2050103@mellanox.co.il> Message-ID: <7.0.1.0.2.20070827103759.028e6498@lanl.gov> > what is the call in # >Hi, > >The agenda for today's meeting: > >OFED 1.3 development status: >I think we need to delay the feature freeze in a week. We can close >the exact date in the meeting. >QoS patches to the kernel - any update on the status >MPI updates - if any >Inputs on the new install method > >Tziporet >_______________________________________________ >general mailing list >general at lists.openfabrics.org >http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ***** Correspondence ***** This email contains no programmatic content that requires independent ADC review From ralph.campbell at qlogic.com Mon Aug 27 09:40:32 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Mon, 27 Aug 2007 09:40:32 -0700 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <20070826050341.GB15362@mellanox.co.il> References: <1188082569.3730.107.camel@brick.pathscale.com> <20070826050341.GB15362@mellanox.co.il> Message-ID: <1188232832.3730.138.camel@brick.pathscale.com> On Sun, 2007-08-26 at 08:03 +0300, Michael S. Tsirkin wrote: > The following claim seems wrong, too: > > > NOTE that [frags..IPOIB_CM_RX_SG-1] are *not* copied to newskb and > > thus leaked. Also, newskb's nr_frags is set based on the message size > > rather than the full size. > > Not really. nr_frags *starts* at the message size. > skb_fill_page_desc will fill in the fargment in toskb, and > increment nr_frags until it matches the full size. > > > @@ -378,23 +382,26 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space, > > skb->len += size; > > length -= size; > > > > - num_frags = skb_shinfo(skb)->nr_frags; > > + num_frags = skb_shinfo(toskb)->nr_frags; > > for (i = 0; i < num_frags; i++) { > > skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; > > > > - if (length == 0) { > > - /* don't need this page */ > > - skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); > > - --skb_shinfo(skb)->nr_frags; > > - } else { > > - size = min(length, (unsigned) PAGE_SIZE); > > - > > - frag->size = size; > > - skb->data_len += size; > > - skb->truesize += size; > > - skb->len += size; > > - length -= size; > > - } > > + size = min(length, (unsigned) PAGE_SIZE); > > + WARN_ON(size == 0); > > + frag->size = size; > > + skb->data_len += size; > > + skb->truesize += size; > > + skb->len += size; > > + length -= size; > > + } > > + WARN_ON(length != 0); > > + skb_shinfo(skb)->nr_frags = num_frags; > > + > > + /* move unused pages from skb to toskb */ > > + for (; i < IPOIB_CM_RX_SG - 1; i++) { > > + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; > > + > > + skb_fill_page_desc(toskb, i, frag->page, 0, PAGE_SIZE); > > } > > } > > > > I don't see how this chunk changes anything. > It seems you have basically replaced a single loop with > a condition by two loops. Looking at the orginial code again (after some more sleep :-) ), I agree that new and old are equivalent. I just found the original a bit confusing. From jsquyres at cisco.com Mon Aug 27 09:46:26 2007 From: jsquyres at cisco.com (Jeff Squyres) Date: Mon, 27 Aug 2007 12:46:26 -0400 Subject: [ofa-general] Agenda for today's meeting In-Reply-To: <7.0.1.0.2.20070827103759.028e6498@lanl.gov> References: <46D2F6CA.2050103@mellanox.co.il> <7.0.1.0.2.20070827103759.028e6498@lanl.gov> Message-ID: I sent the reminder this morning across the EWG list at 8:18am US Eastern time (note that the call is now over; we finished around 12:30pm US Eastern time): http://lists.openfabrics.org/pipermail/ewg/2007-August/004418.html On Aug 27, 2007, at 12:38 PM, Parks Fields wrote: > >> > > > > what is the call in # > >> Hi, >> >> The agenda for today's meeting: >> >> OFED 1.3 development status: >> I think we need to delay the feature freeze in a week. We can >> close the exact date in the meeting. >> QoS patches to the kernel - any update on the status >> MPI updates - if any >> Inputs on the new install method >> >> Tziporet >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/ >> openib-general > > ***** Correspondence ***** > > This email contains no programmatic content that requires > independent ADC review > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/ > openib-general -- Jeff Squyres Cisco Systems From mst at dev.mellanox.co.il Mon Aug 27 10:13:24 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Mon, 27 Aug 2007 20:13:24 +0300 Subject: [ofa-general] Re: [PATCH] connected mode leaks pages on receive & reuses old skb In-Reply-To: <1188232400.3730.132.camel@brick.pathscale.com> References: <1188082569.3730.107.camel@brick.pathscale.com> <20070826050815.GC15362@mellanox.co.il> <1188232400.3730.132.camel@brick.pathscale.com> Message-ID: <20070827171324.GC18695@mellanox.co.il> > Quoting Ralph Campbell : > Subject: Re: [PATCH] connected mode leaks pages on receive & reuses old skb > > On Sun, 2007-08-26 at 08:08 +0300, Michael S. Tsirkin wrote: > > > Also, use ib_dma_unmap_page() for addresses mapped by ib_dma_map_page(). > > > > > > Signed-off-by: Ralph Campbell > > > > dma_unmap_page and dma_unmap_single are equivalent for all architectures I cared > > to check, but I agree this might be cleaner as dma_unmap_page. So the following > > 2 chunks make some sense to me. > > > > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > index 08b4676..a3434f2 100644 > > > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > @@ -78,7 +78,7 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags, > > > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > > > > > for (i = 0; i < frags; ++i) > > > - ib_dma_unmap_single(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > > > + ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE); > > > } > > > > > > static int ipoib_cm_post_receive(struct net_device *dev, int id) > > > @@ -149,7 +149,7 @@ partial_error: > > > ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE); > > > > > > for (; i > 0; --i) > > > - ib_dma_unmap_single(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > > > + ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE); > > > > > > dev_kfree_skb_any(skb); > > > return NULL; > > > > Is this a cosmetic patch as it seems, or are the ipath implementations of > > ib_dma_unmap_page and ib_dma_unmap_single significantly different? > > The ipath implementation for unmap doesn't differ significantly but for > consistency and possible future changes, I would like to see the > correct unmap calls being made. OK, probably 2.6.24 though. -- MST From hakginsbergdog at ginsberg.org Mon Aug 27 14:59:40 2007 From: hakginsbergdog at ginsberg.org (Kenny Woods) Date: Mon, 27 Aug 2007 19:59:40 -0200 Subject: [ofa-general] Software Message-ID: <293929697.54937379986644@ginsberg.org> Searching for the better price in discount software? Now you will find the chance to use the softwares you wanted long since. And the better thing is, all softs are awfully cheap. Check it out by yourself & get the softwares for cheapest rates. http://eknxo.shop-oem.net/?997117338299 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kliteyn at dev.mellanox.co.il Mon Aug 27 16:08:46 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:08:46 +0300 Subject: [ofa-general] [PATCH 0/7] osm: QoS in OpenSM In-Reply-To: <46C9820E.7070002@dev.mellanox.co.il> References: <46C9820E.7070002@dev.mellanox.co.il> Message-ID: <46D3597E.6010602@dev.mellanox.co.il> Hi Sasha, I'm sending the fixed patches, marked as V3. Patch 1/7 has been already applied, so I'm sending patches 2 to 7. Main changes are: - Implemented binary search in range arrays instead of linear search - Global qos_policy was moved to osm_subn_t and updated he parser accordingly - Policy file is read from osm_subn_rescan_conf_files() in osm_subnet.c instead of osm_qos.c - Removing generated files by "make maintainer-clean" -- Yevgeny Yevgeny Kliteynik wrote: > Hi Sasha > > The following series of 7 patches adds new QoS fields > to PathRecord and implements QoS policy file parser: > > [1/7] Adding QoS fields to PathRecord > [2/7] QoS policy file parser - Lex & Yacc files > [3/7] QoS policy data structures and functions > [4/7] Renaming static find_prtn_by_name() to non-static > osm_prtn_find_by_name() - it is used by QoS policy > functions. > [5/7] Compiling policy file parser. > C & H files are generated from Lex & Yacc files and > the generated files are compiled. > Generated files are not included in git, but they do > get included in RPM as they are also created by > 'make dist'. > [6/7] Adding QoS policy file option to OpenSM: > -Y or --qos_policy_file > [7/7] Reading QoS policy file > > Most of the code is rather technical, w/o much logic to > review and discuss. > > Right now I'm doing a fine-tuning of the patch that implements > the actual PathRecord selection according to the matching QoS > level, but I prefer to send this patch separately because IMO > it deserves more thorough review. > > -- Yevgeny > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From kliteyn at dev.mellanox.co.il Mon Aug 27 16:09:06 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:09:06 +0300 Subject: [ofa-general] [PATCH 2/7 V3] osm: QoS - policy file parser Lex & Yacc Message-ID: <46D35992.1030804@dev.mellanox.co.il> QoS Policy file parser - Lex & Yacc files Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_parser.l | 329 ++++++ opensm/opensm/osm_qos_parser.y | 2188 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 2517 insertions(+), 0 deletions(-) create mode 100644 opensm/opensm/osm_qos_parser.l create mode 100644 opensm/opensm/osm_qos_parser.y diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l new file mode 100644 index 0000000..bc03123 --- /dev/null +++ b/opensm/opensm/osm_qos_parser.l @@ -0,0 +1,329 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Lexer of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include + +#define HANDLE_IF_IN_DESCRIPTION if (in_description) { __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } + +#define SAVE_POS save_pos() +static void save_pos(); + +extern int column_num; +extern int line_num; +extern FILE * __qos_parser_in; + +boolean_t in_description = FALSE; +boolean_t in_list_of_hex_num_ranges = FALSE; +boolean_t in_node_type = FALSE; +boolean_t in_list_of_numbers = FALSE; +boolean_t in_list_of_strings = FALSE; +boolean_t in_list_of_num_pairs = FALSE; +boolean_t in_asterisk_or_list_of_numbers = FALSE; +boolean_t in_list_of_num_ranges = FALSE; +boolean_t in_single_string = FALSE; +boolean_t in_single_number = FALSE; + +static void reset_new_line_flags(); +#define RESET_NEW_LINE_FLAGS reset_new_line_flags() + +#define START_USE {in_description = TRUE;} /* list of strings including whitespace (description) */ +#define START_PORT_GUID {in_list_of_hex_num_ranges = TRUE;} /* comma-separated list of hex num ranges */ +#define START_PORT_NAME {in_list_of_strings = TRUE;} /* comma-separated list of following strings: ../../.. */ +#define START_PARTITION {in_single_string = TRUE;} /* single string w/o whitespaces (partition name) */ +#define START_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (port group name) */ +#define START_QOS_LEVEL_NAME {in_single_string = TRUE;} /* single string w/o whitespaces (qos level name in match rule) */ + +#define START_NODE_TYPE {in_node_type = TRUE;} /* comma-separated list of node types (ROUTER,CA,...) */ +#define START_SL2VL_TABLE {in_list_of_numbers = TRUE;} /* comma-separated list of hex or dec numbers */ + +#define START_GROUP {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_TO {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_ACROSS_FROM {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_SOURCE {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ +#define START_DESTINATION {in_list_of_strings = TRUE;} /* list of strings w/o whitespaces (group names) */ + +#define START_VLARB_HIGH {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ +#define START_VLARB_LOW {in_list_of_num_pairs = TRUE;} /* comma-separated list of hex or dec num pairs: "num1:num2" */ + +#define START_TO {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ +#define START_FROM {in_asterisk_or_list_of_numbers = TRUE;} /* (asterisk) or (comma-separated list of hex or dec numbers) */ + +#define START_PATH_BITS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_QOS_CLASS {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_SERVICE_ID {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ +#define START_PKEY {in_list_of_num_ranges = TRUE;} /* comma-separated list of hex or dec num ranges */ + +#define START_SL {in_single_number = TRUE;} /* single number */ +#define START_VLARB_HIGH_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_MTU_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_RATE_LIMIT {in_single_number = TRUE;} /* single number */ +#define START_PACKET_LIFE {in_single_number = TRUE;} /* single number */ + + + +%} + +PORT_GROUPS_START port\-groups +PORT_GROUPS_END end\-port\-groups +PORT_GROUP_START port\-group +PORT_GROUP_END end\-port\-group +NAME name +USE use +PORT_GUID port\-guid +PORT_NAME port\-name +PARTITION partition +NODE_TYPE node\-type +QOS_SETUP_START qos\-setup +QOS_SETUP_END end\-qos\-setup +VLARB_TABLES_START vlarb\-tables +VLARB_TABLES_END end\-vlarb\-tables +VLARB_SCOPE_START vlarb\-scope +VLARB_SCOPE_END end\-vlarb\-scope +GROUP group +ACROSS across +VLARB_HIGH vlarb\-high +VLARB_LOW vlarb\-low +VLARB_HIGH_LIMIT vl\-high\-limit +SL2VL_TABLES_START sl2vl\-tables +SL2VL_TABLES_END end\-sl2vl\-tables +SL2VL_SCOPE_START sl2vl\-scope +SL2VL_SCOPE_END end\-sl2vl\-scope +TO to +FROM from +ACROSS_TO across\-to +ACROSS_FROM across\-from +SL2VL_TABLE sl2vl\-table +QOS_LEVELS_START qos\-levels +QOS_LEVELS_END end\-qos\-levels +QOS_LEVEL_START qos\-level +QOS_LEVEL_END end\-qos\-level +SL sl +MTU_LIMIT mtu\-limit +RATE_LIMIT rate\-limit +PACKET_LIFE packet\-life +PATH_BITS path\-bits +QOS_MATCH_RULES_START qos\-match\-rules +QOS_MATCH_RULES_END end\-qos\-match\-rules +QOS_MATCH_RULE_START qos\-match\-rule +QOS_MATCH_RULE_END end\-qos\-match\-rule +QOS_CLASS qos\-class +SOURCE source +DESTINATION destination +SERVICE_ID service\-id +PKEY pkey +QOS_LEVEL_NAME qos\-level\-name + +ROUTER [Rr][Oo][Uu][Tt][Ee][Rr] +CA [Cc][Aa] +SWITCH [Ss][Ww][Ii][Tt][Cc][Hh] +SELF [Ss][Ee][Ll][Ff] +ALL [Aa][Ll][Ll] + +WHITE [ \t]+ +NEW_LINE \n +COMMENT \#.*\n +WHITE_DOTDOT_WHITE [ \t]*:[ \t]* + +%% + + +{COMMENT} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* swallow comment */ +{WHITE}{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } /* trailing blanks with new line */ +{WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; } +{NEW_LINE} { SAVE_POS; RESET_NEW_LINE_FLAGS; } + +{PORT_GROUPS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_START; } +{PORT_GROUPS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUPS_END; } +{PORT_GROUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_START; } +{PORT_GROUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_PORT_GROUP_END; } + +{QOS_SETUP_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_START; } +{QOS_SETUP_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_SETUP_END; } +{VLARB_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_START; } +{VLARB_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_TABLES_END; } +{VLARB_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_START; } +{VLARB_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_VLARB_SCOPE_END; } + +{SL2VL_TABLES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_START; } +{SL2VL_TABLES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_TABLES_END; } +{SL2VL_SCOPE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_START; } +{SL2VL_SCOPE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_SL2VL_SCOPE_END; } + +{QOS_LEVELS_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_START; } +{QOS_LEVELS_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVELS_END; } +{QOS_LEVEL_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_START; } +{QOS_LEVEL_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_LEVEL_END; } + +{QOS_MATCH_RULES_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_START; } +{QOS_MATCH_RULES_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULES_END; } +{QOS_MATCH_RULE_START} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_START; } +{QOS_MATCH_RULE_END} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; return TK_QOS_MATCH_RULE_END; } + +{PORT_GUID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_GUID; return TK_PORT_GUID; } +{PORT_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PORT_NAME; return TK_PORT_NAME; } +{PARTITION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PARTITION; return TK_PARTITION; } +{NODE_TYPE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NODE_TYPE; return TK_NODE_TYPE; } +{NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_NAME; return TK_NAME; } +{USE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_USE; return TK_USE; } +{GROUP}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_GROUP; return TK_GROUP; } +{VLARB_HIGH}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH; return TK_VLARB_HIGH; } +{VLARB_LOW}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_LOW; return TK_VLARB_LOW; } +{VLARB_HIGH_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_VLARB_HIGH_LIMIT; return TK_VLARB_HIGH_LIMIT;} +{TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_TO; return TK_TO; } +{FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_FROM; return TK_FROM; } +{ACROSS_TO}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_TO; return TK_ACROSS_TO; } +{ACROSS_FROM}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS_FROM; return TK_ACROSS_FROM;} +{ACROSS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_ACROSS; return TK_ACROSS; } +{SL2VL_TABLE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL2VL_TABLE; return TK_SL2VL_TABLE;} +{SL}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SL; return TK_SL; } +{MTU_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_MTU_LIMIT; return TK_MTU_LIMIT; } +{RATE_LIMIT}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_RATE_LIMIT; return TK_RATE_LIMIT; } +{PACKET_LIFE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PACKET_LIFE; return TK_PACKET_LIFE;} +{PATH_BITS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PATH_BITS; return TK_PATH_BITS; } +{QOS_CLASS}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_CLASS; return TK_QOS_CLASS; } +{SOURCE}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SOURCE; return TK_SOURCE; } +{DESTINATION}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_DESTINATION; return TK_DESTINATION;} +{SERVICE_ID}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_SERVICE_ID; return TK_SERVICE_ID; } +{PKEY}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_PKEY; return TK_PKEY; } +{QOS_LEVEL_NAME}{WHITE_DOTDOT_WHITE} { SAVE_POS; HANDLE_IF_IN_DESCRIPTION; START_QOS_LEVEL_NAME; return TK_QOS_LEVEL_NAME;} + +{ROUTER} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ROUTER; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{CA} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_CA; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{SWITCH} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SWITCH; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{SELF} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_SELF; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } +{ALL} { SAVE_POS; if (in_node_type) return TK_NODE_TYPE_ALL; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT; } + +0[xX][0-9a-fA-F]+ { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + +[0-9]+ { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_NUMBER; + } + + +- { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DASH; + } + +: { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_DOTDOT; + } + +, { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description) + return TK_TEXT; + return TK_COMMA; + } + +\* { + SAVE_POS; + __qos_parser_lval = strdup(__qos_parser_text); + if (in_description || in_list_of_strings || in_single_string) + return TK_TEXT; + return TK_ASTERISK; + } + +. { SAVE_POS; __qos_parser_lval = strdup(__qos_parser_text); return TK_TEXT;} + +%% + + +/********************************************* + *********************************************/ + +static void save_pos() +{ + int i; + for (i = 0; i < __qos_parser_leng; i++) + { + if (__qos_parser_text[i] == '\n') + { + line_num ++; + column_num = 1; + } + else + column_num ++; + } +} + +/********************************************* + *********************************************/ + +static void reset_new_line_flags() +{ + in_description = FALSE; + in_list_of_hex_num_ranges = FALSE; + in_node_type = FALSE; + in_list_of_numbers = FALSE; + in_list_of_strings = FALSE; + in_list_of_num_pairs = FALSE; + in_asterisk_or_list_of_numbers = FALSE; + in_list_of_num_ranges = FALSE; + in_single_string = FALSE; + in_single_number = FALSE; +} diff --git a/opensm/opensm/osm_qos_parser.y b/opensm/opensm/osm_qos_parser.y new file mode 100644 index 0000000..fc9aa42 --- /dev/null +++ b/opensm/opensm/osm_qos_parser.y @@ -0,0 +1,2188 @@ +%{ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Grammar of OSM QoS parser. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define OSM_QOS_POLICY_MAX_LINE_LEN 1024*10 +#define OSM_QOS_POLICY_SL2VL_TABLE_LEN IB_MAX_NUM_VLS +#define OSM_QOS_POLICY_MAX_VL_NUM IB_MAX_NUM_VLS + +typedef struct tmp_parser_struct_t_ { + char str[OSM_QOS_POLICY_MAX_LINE_LEN]; + uint64_t num_pair[2]; + cl_list_t str_list; + cl_list_t num_list; + cl_list_t num_pair_list; +} tmp_parser_struct_t; + +static void __parser_tmp_struct_init(); +static void __parser_tmp_struct_reset(); +static void __parser_tmp_struct_destroy(); + +static char * __parser_strip_white(char * str); + +static void __parser_str2uint64(uint64_t * p_val, char * str); + +static void __parser_port_group_start(); +static int __parser_port_group_end(); + +static void __parser_sl2vl_scope_start(); +static int __parser_sl2vl_scope_end(); + +static void __parser_vlarb_scope_start(); +static int __parser_vlarb_scope_end(); + +static void __parser_qos_level_start(); +static int __parser_qos_level_end(); + +static void __parser_match_rule_start(); +static int __parser_match_rule_end(); + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len); + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ); + +extern char * __qos_parser_text; +extern void __qos_parser_error (char *s); +extern int __qos_parser_lex (void); +extern FILE * __qos_parser_in; + +#define RESET_BUFFER __parser_tmp_struct_reset() + +tmp_parser_struct_t tmp_parser_struct; + +int column_num; +int line_num; + +osm_qos_policy_t * p_qos_policy = NULL; +osm_qos_port_group_t * p_current_port_group = NULL; +osm_qos_sl2vl_scope_t * p_current_sl2vl_scope = NULL; +osm_qos_vlarb_scope_t * p_current_vlarb_scope = NULL; +osm_qos_level_t * p_current_qos_level = NULL; +osm_qos_match_rule_t * p_current_qos_match_rule = NULL; +osm_log_t * p_qos_parser_osm_log; + +/***************************************************/ + +%} + +%token TK_NUMBER +%token TK_DASH +%token TK_DOTDOT +%token TK_COMMA +%token TK_ASTERISK +%token TK_TEXT + +%token TK_PORT_GROUPS_START +%token TK_PORT_GROUPS_END +%token TK_PORT_GROUP_START +%token TK_PORT_GROUP_END + +%token TK_QOS_SETUP_START +%token TK_QOS_SETUP_END +%token TK_VLARB_TABLES_START +%token TK_VLARB_TABLES_END +%token TK_VLARB_SCOPE_START +%token TK_VLARB_SCOPE_END + +%token TK_SL2VL_TABLES_START +%token TK_SL2VL_TABLES_END +%token TK_SL2VL_SCOPE_START +%token TK_SL2VL_SCOPE_END + +%token TK_QOS_LEVELS_START +%token TK_QOS_LEVELS_END +%token TK_QOS_LEVEL_START +%token TK_QOS_LEVEL_END + +%token TK_QOS_MATCH_RULES_START +%token TK_QOS_MATCH_RULES_END +%token TK_QOS_MATCH_RULE_START +%token TK_QOS_MATCH_RULE_END + +%token TK_NAME +%token TK_USE +%token TK_PORT_GUID +%token TK_PORT_NAME +%token TK_PARTITION +%token TK_NODE_TYPE +%token TK_GROUP +%token TK_ACROSS +%token TK_VLARB_HIGH +%token TK_VLARB_LOW +%token TK_VLARB_HIGH_LIMIT +%token TK_TO +%token TK_FROM +%token TK_ACROSS_TO +%token TK_ACROSS_FROM +%token TK_SL2VL_TABLE +%token TK_SL +%token TK_MTU_LIMIT +%token TK_RATE_LIMIT +%token TK_PACKET_LIFE +%token TK_PATH_BITS +%token TK_QOS_CLASS +%token TK_SOURCE +%token TK_DESTINATION +%token TK_SERVICE_ID +%token TK_QOS_LEVEL_NAME +%token TK_PKEY + +%token TK_NODE_TYPE_ROUTER +%token TK_NODE_TYPE_CA +%token TK_NODE_TYPE_SWITCH +%token TK_NODE_TYPE_SELF +%token TK_NODE_TYPE_ALL + + +%start head + +%% + +head: qos_policy_entries + ; + +qos_policy_entries: /* empty */ + | qos_policy_entries qos_policy_entry + ; + +qos_policy_entry: port_groups_section + | qos_setup_section + | qos_levels_section + | qos_match_rules_section + ; + + /* + * Parsing port groups: + * ------------------- + * port-groups + * port-group + * name: Storage + * use: our SRP storage targets + * port-guid: 0x1000000000000001,0x1000000000000002 + * ... + * port-name: vs1/HCA-1/P1 + * ... + * partition: Part1 + * ... + * node-type: ROUTER,CA,SWITCH,SELF,ALL + * ... + * end-port-group + * port-group + * ... + * end-port-group + * end-port-groups + */ + + +port_groups_section: TK_PORT_GROUPS_START port_groups TK_PORT_GROUPS_END + ; + +port_groups: port_group + | port_groups port_group + ; + +port_group: port_group_start port_group_entries port_group_end + ; + +port_group_start: TK_PORT_GROUP_START { + __parser_port_group_start(); + } + ; + +port_group_end: TK_PORT_GROUP_END { + if ( __parser_port_group_end() ) + return 1; + } + ; + +port_group_entries: /* empty */ + | port_group_entries port_group_entry + ; + +port_group_entry: port_group_name + | port_group_use + | port_group_port_guid + | port_group_port_name + | port_group_partition + | port_group_node_type + ; + + + /* + * Parsing qos setup: + * ----------------- + * qos-setup + * vlarb-tables + * vlarb-scope + * ... + * end-vlarb-scope + * vlarb-scope + * ... + * end-vlarb-scope + * end-vlarb-tables + * sl2vl-tables + * sl2vl-scope + * ... + * end-sl2vl-scope + * sl2vl-scope + * ... + * end-sl2vl-scope + * end-sl2vl-tables + * end-qos-setup + */ + +qos_setup_section: TK_QOS_SETUP_START qos_setup_items TK_QOS_SETUP_END + ; + +qos_setup_items: /* empty */ + | qos_setup_items vlarb_tables + | qos_setup_items sl2vl_tables + ; + + /* Parsing vlarb-tables */ + +vlarb_tables: TK_VLARB_TABLES_START vlarb_scope_items TK_VLARB_TABLES_END + +vlarb_scope_items: /* empty */ + | vlarb_scope_items vlarb_scope + ; + +vlarb_scope: vlarb_scope_start vlarb_scope_entries vlarb_scope_end + ; + +vlarb_scope_start: TK_VLARB_SCOPE_START { + __parser_vlarb_scope_start(); + } + ; + +vlarb_scope_end: TK_VLARB_SCOPE_END { + if ( __parser_vlarb_scope_end() ) + return 1; + } + ; + +vlarb_scope_entries:/* empty */ + | vlarb_scope_entries vlarb_scope_entry + ; + + /* + * vlarb-scope + * group: Storage + * ... + * across: Storage + * ... + * vlarb-high: 0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1 + * vlarb-low: 8:255,9:127,10:63,11:31,12:15,13:7,14:3 + * vl-high-limit: 10 + * end-vlarb-scope + */ + +vlarb_scope_entry: vlarb_scope_group + | vlarb_scope_across + | vlarb_scope_vlarb_high + | vlarb_scope_vlarb_low + | vlarb_scope_vlarb_high_limit + ; + + /* Parsing sl2vl-tables */ + +sl2vl_tables: TK_SL2VL_TABLES_START sl2vl_scope_items TK_SL2VL_TABLES_END + ; + +sl2vl_scope_items: /* empty */ + | sl2vl_scope_items sl2vl_scope + ; + +sl2vl_scope: sl2vl_scope_start sl2vl_scope_entries sl2vl_scope_end + ; + +sl2vl_scope_start: TK_SL2VL_SCOPE_START { + __parser_sl2vl_scope_start(); + } + ; + +sl2vl_scope_end: TK_SL2VL_SCOPE_END { + if ( __parser_sl2vl_scope_end() ) + return 1; + } + ; + +sl2vl_scope_entries:/* empty */ + | sl2vl_scope_entries sl2vl_scope_entry + ; + + /* + * sl2vl-scope + * group: Part1 + * ... + * from: * + * ... + * to: * + * ... + * across-to: Storage2 + * ... + * across-from: Storage1 + * ... + * sl2vl-table: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7 + * end-sl2vl-scope + */ + +sl2vl_scope_entry: sl2vl_scope_group + | sl2vl_scope_across + | sl2vl_scope_across_from + | sl2vl_scope_across_to + | sl2vl_scope_from + | sl2vl_scope_to + | sl2vl_scope_sl2vl_table + ; + + /* + * Parsing qos-levels: + * ------------------ + * qos-levels + * qos-level + * name: qos_level_1 + * use: for the lowest priority communication + * sl: 15 + * mtu-limit: 1 + * rate-limit: 1 + * packet-life: 12 + * path-bits: 2,4,8-32 + * pkey: 0x00FF-0x0FFF + * end-qos-level + * ... + * qos-level + * end-qos-level + * end-qos-levels + */ + + +qos_levels_section: TK_QOS_LEVELS_START qos_levels TK_QOS_LEVELS_END + ; + +qos_levels: /* empty */ + | qos_levels qos_level + ; + +qos_level: qos_level_start qos_level_entries qos_level_end + ; + +qos_level_start: TK_QOS_LEVEL_START { + __parser_qos_level_start(); + } + ; + +qos_level_end: TK_QOS_LEVEL_END { + if ( __parser_qos_level_end() ) + return 1; + } + ; + +qos_level_entries: /* empty */ + | qos_level_entries qos_level_entry + ; + +qos_level_entry: qos_level_name + | qos_level_use + | qos_level_sl + | qos_level_mtu_limit + | qos_level_rate_limit + | qos_level_packet_life + | qos_level_path_bits + | qos_level_pkey + ; + + /* + * Parsing qos-match-rules: + * ----------------------- + * qos-match-rules + * qos-match-rule + * use: low latency by class 7-9 or 11 and bla bla + * qos-class: 7-9,11 + * qos-level-name: default + * source: Storage + * destination: Storage + * service-id: 22,4719-5000 + * pkey: 0x00FF-0x0FFF + * end-qos-match-rule + * qos-match-rule + * ... + * end-qos-match-rule + * end-qos-match-rules + */ + +qos_match_rules_section: TK_QOS_MATCH_RULES_START qos_match_rules TK_QOS_MATCH_RULES_END + ; + +qos_match_rules: /* empty */ + | qos_match_rules qos_match_rule + ; + +qos_match_rule: qos_match_rule_start qos_match_rule_entries qos_match_rule_end + ; + +qos_match_rule_start: TK_QOS_MATCH_RULE_START { + __parser_match_rule_start(); + } + ; + +qos_match_rule_end: TK_QOS_MATCH_RULE_END { + if ( __parser_match_rule_end() ) + return 1; + } + ; + +qos_match_rule_entries: /* empty */ + | qos_match_rule_entries qos_match_rule_entry + ; + +qos_match_rule_entry: qos_match_rule_use + | qos_match_rule_qos_class + | qos_match_rule_qos_level_name + | qos_match_rule_source + | qos_match_rule_destination + | qos_match_rule_service_id + | qos_match_rule_pkey + ; + + /* + * port_group_entry values: + * port_group_name + * port_group_use + * port_group_port_guid + * port_group_port_name + * port_group_partition + * port_group_node_type + */ + +port_group_name: port_group_name_start single_string { + /* 'name' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->name) + { + __qos_parser_error("port-group has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +port_group_use: port_group_use_start single_string { + /* 'use' of 'port-group' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_port_group->use) + { + __qos_parser_error("port-group has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_port_group->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_use_start: TK_USE { + RESET_BUFFER; + } + ; + +port_group_port_name: port_group_port_name_start string_list { + /* 'port-name' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + + /* + * TODO: parse port name strings + */ + + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->port_name_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_port_name_start: TK_PORT_NAME { + RESET_BUFFER; + } + ; + +port_group_port_guid: port_group_port_guid_start list_of_ranges { + /* 'port-guid' in 'port-group' - any num of instances */ + /* list of guid ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_port_group->guid_range_len ) + { + p_current_port_group->guid_range_arr = range_arr; + p_current_port_group->guid_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_port_group->guid_range_arr, + p_current_port_group->guid_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_port_group->guid_range_arr = new_range_arr; + p_current_port_group->guid_range_len = new_range_len; + } + } + } + ; + +port_group_port_guid_start: TK_PORT_GUID { + RESET_BUFFER; + } + ; + +port_group_partition: port_group_partition_start string_list { + /* 'partition' in 'port-group' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_port_group->partition_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +port_group_partition_start: TK_PARTITION { + RESET_BUFFER; + } + ; + +port_group_node_type: port_group_node_type_start port_group_node_type_list { + /* 'node-type' in 'port-group' - any num of instances */ + } + ; + +port_group_node_type_start: TK_NODE_TYPE { + RESET_BUFFER; + } + ; + +port_group_node_type_list: node_type_item + | port_group_node_type_list TK_COMMA node_type_item + ; + +node_type_item: node_type_ca + | node_type_switch + | node_type_router + | node_type_all + | node_type_self + ; + +node_type_ca: TK_NODE_TYPE_CA { + p_current_port_group->node_type_ca = TRUE;; + } + ; + +node_type_switch: TK_NODE_TYPE_SWITCH { + p_current_port_group->node_type_switch = TRUE; + } + ; + +node_type_router: TK_NODE_TYPE_ROUTER { + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_all: TK_NODE_TYPE_ALL { + p_current_port_group->node_type_ca = TRUE; + p_current_port_group->node_type_switch = TRUE; + p_current_port_group->node_type_router = TRUE; + } + ; + +node_type_self: TK_NODE_TYPE_SELF { + p_current_port_group->node_type_self = TRUE; + } + ; + + /* + * vlarb_scope_entry values: + * vlarb_scope_group + * vlarb_scope_across + * vlarb_scope_vlarb_high + * vlarb_scope_vlarb_low + * vlarb_scope_vlarb_high_limit + */ + + + +vlarb_scope_group: vlarb_scope_group_start string_list { + /* 'group' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +vlarb_scope_across: vlarb_scope_across_start string_list { + /* 'across' in 'vlarb-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_vlarb_scope->across_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +vlarb_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high_limit: vlarb_scope_vlarb_high_limit_start single_number { + /* 'vl-high-limit' in 'vlarb-scope' - one instance of one number */ + cl_list_iterator_t list_iterator; + uint64_t * p_tmp_num; + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_tmp_num = (uint64_t*)cl_list_obj(list_iterator); + if (p_tmp_num) + { + p_current_vlarb_scope->vl_high_limit = (uint32_t)(*p_tmp_num); + p_current_vlarb_scope->vl_high_limit_set = TRUE; + free(p_tmp_num); + } + + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +vlarb_scope_vlarb_high_limit_start: TK_VLARB_HIGH_LIMIT { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_high: vlarb_scope_vlarb_high_start num_list_with_dotdot { + /* 'vlarb-high' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_high_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_high_start: TK_VLARB_HIGH { + RESET_BUFFER; + } + ; + +vlarb_scope_vlarb_low: vlarb_scope_vlarb_low_start num_list_with_dotdot { + /* 'vlarb-low' in 'vlarb-scope' - list of pairs of numbers with ':' and ',' */ + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + cl_list_insert_tail(&p_current_vlarb_scope->vlarb_low_list,num_pair); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +vlarb_scope_vlarb_low_start: TK_VLARB_LOW { + RESET_BUFFER; + } + ; + + /* + * sl2vl_scope_entry values: + * sl2vl_scope_group + * sl2vl_scope_across + * sl2vl_scope_across_from + * sl2vl_scope_across_to + * sl2vl_scope_from + * sl2vl_scope_to + * sl2vl_scope_sl2vl_table + */ + +sl2vl_scope_group: sl2vl_scope_group_start string_list { + /* 'group' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->group_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_group_start: TK_GROUP { + RESET_BUFFER; + } + ; + +sl2vl_scope_across: sl2vl_scope_across_start string_list { + /* 'across' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,strdup(tmp_str)); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_start: TK_ACROSS { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_from: sl2vl_scope_across_from_start string_list { + /* 'across-from' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_sl2vl_scope->across_from_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_from_start: TK_ACROSS_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_across_to: sl2vl_scope_across_to_start string_list { + /* 'across-to' in 'sl2vl-scope' - any num of instances */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) { + cl_list_insert_tail(&p_current_sl2vl_scope->across_to_list,tmp_str); + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +sl2vl_scope_across_to_start: TK_ACROSS_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from: sl2vl_scope_from_start sl2vl_scope_from_list_or_asterisk { + /* 'from' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_from_start: TK_FROM { + RESET_BUFFER; + } + ; + +sl2vl_scope_to: sl2vl_scope_to_start sl2vl_scope_to_list_or_asterisk { + /* 'to' in 'sl2vl-scope' - any num of instances */ + } + ; + +sl2vl_scope_to_start: TK_TO { + RESET_BUFFER; + } + ; + +sl2vl_scope_from_list_or_asterisk: sl2vl_scope_from_asterisk + | sl2vl_scope_from_list_of_ranges + ; + +sl2vl_scope_from_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + ; + +sl2vl_scope_to_list_or_asterisk: sl2vl_scope_to_asterisk + | sl2vl_scope_to_list_of_ranges + ; + +sl2vl_scope_to_asterisk: TK_ASTERISK { + int i; + for (i = 0; i < OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + ; + +sl2vl_scope_from_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + __qos_parser_error("port number out of range 'from' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->from[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + +sl2vl_scope_to_list_of_ranges: list_of_ranges { + int i; + cl_list_iterator_t list_iterator; + uint64_t * num_pair; + uint8_t num1, num2; + + list_iterator = cl_list_head(&tmp_parser_struct.num_pair_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_pair_list) ) + { + num_pair = (uint64_t*)cl_list_obj(list_iterator); + if (num_pair) + { + if ( num_pair[0] < 0 || + num_pair[1] >= OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH ) + { + __qos_parser_error("port number out of range 'to' list"); + free(num_pair); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + return 1; + } + num1 = (uint8_t)num_pair[0]; + num2 = (uint8_t)num_pair[1]; + free(num_pair); + for (i = num1; i <= num2; i++) + p_current_sl2vl_scope->to[i] = TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.num_pair_list); + } + ; + + +sl2vl_scope_sl2vl_table: sl2vl_scope_sl2vl_table_start num_list { + /* 'sl2vl-table' - one instance of exactly + OSM_QOS_POLICY_SL2VL_TABLE_LEN numbers */ + cl_list_iterator_t list_iterator; + uint64_t num; + uint64_t * p_num; + int i = 0; + + if (p_current_sl2vl_scope->sl2vl_table_set) + { + __qos_parser_error("sl2vl-scope has more than one sl2vl-table"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + if (cl_list_count(&tmp_parser_struct.num_list) != OSM_QOS_POLICY_SL2VL_TABLE_LEN) + { + __qos_parser_error("wrong number of values in 'sl2vl-table' (should be 16)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.num_list) ) + { + p_num = (uint64_t*)cl_list_obj(list_iterator); + num = *p_num; + free(p_num); + if (num >= OSM_QOS_POLICY_MAX_VL_NUM) + { + __qos_parser_error("wrong VL value in 'sl2vl-table' (should be 0 to 15)"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + p_current_sl2vl_scope->sl2vl_table[i++] = (uint8_t)num; + list_iterator = cl_list_next(list_iterator); + } + p_current_sl2vl_scope->sl2vl_table_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +sl2vl_scope_sl2vl_table_start: TK_SL2VL_TABLE { + RESET_BUFFER; + } + ; + + /* + * qos_level_entry values: + * qos_level_name + * qos_level_use + * qos_level_sl + * qos_level_mtu_limit + * qos_level_rate_limit + * qos_level_packet_life + * qos_level_path_bits + * qos_level_pkey + */ + +qos_level_name: qos_level_name_start single_string { + /* 'name' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->name) + { + __qos_parser_error("qos-level has multiple 'name' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_name_start: TK_NAME { + RESET_BUFFER; + } + ; + +qos_level_use: qos_level_use_start single_string { + /* 'use' of 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_level->use) + { + __qos_parser_error("qos-level has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_level->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_level_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_level_sl: qos_level_sl_start single_number { + /* 'sl' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->sl_set) + { + __qos_parser_error("'qos-level' has multiple 'sl' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->sl = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->sl_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_sl_start: TK_SL { + RESET_BUFFER; + } + ; + +qos_level_mtu_limit: qos_level_mtu_limit_start single_number { + /* 'mtu-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->mtu_limit_set) + { + __qos_parser_error("'qos-level' has multiple 'mtu-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->mtu_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->mtu_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_mtu_limit_start: TK_MTU_LIMIT { + /* 'mtu-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_rate_limit: qos_level_rate_limit_start single_number { + /* 'rate-limit' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->rate_limit_set) + { + __qos_parser_error("'qos-level' has multiple 'rate-limit' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->rate_limit = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->rate_limit_set = TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_rate_limit_start: TK_RATE_LIMIT { + /* 'rate-limit' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_packet_life: qos_level_packet_life_start single_number { + /* 'packet-life' in 'qos-level' - one instance */ + cl_list_iterator_t list_iterator; + uint64_t * p_num; + + if (p_current_qos_level->pkt_life_set) + { + __qos_parser_error("'qos-level' has multiple 'packet-life' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + list_iterator = cl_list_head(&tmp_parser_struct.num_list); + p_num = (uint64_t*)cl_list_obj(list_iterator); + p_current_qos_level->pkt_life = (uint8_t)(*p_num); + free(p_num); + p_current_qos_level->pkt_life_set= TRUE; + cl_list_remove_all(&tmp_parser_struct.num_list); + } + ; + +qos_level_packet_life_start: TK_PACKET_LIFE { + /* 'packet-life' in 'qos-level' - one instance */ + RESET_BUFFER; + } + ; + +qos_level_path_bits: qos_level_path_bits_start list_of_ranges { + /* 'path-bits' in 'qos-level' - any num of instances */ + /* list of path bit ranges */ + + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->path_bits_range_len ) + { + p_current_qos_level->path_bits_range_arr = range_arr; + p_current_qos_level->path_bits_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->path_bits_range_arr, + p_current_qos_level->path_bits_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->path_bits_range_arr = new_range_arr; + p_current_qos_level->path_bits_range_len = new_range_len; + } + } + } + ; + +qos_level_path_bits_start: TK_PATH_BITS { + RESET_BUFFER; + } + ; + +qos_level_pkey: qos_level_pkey_start list_of_ranges { + /* 'pkey' in 'qos-level' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_level->pkey_range_len ) + { + p_current_qos_level->pkey_range_arr = range_arr; + p_current_qos_level->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_level->pkey_range_arr, + p_current_qos_level->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_level->pkey_range_arr = new_range_arr; + p_current_qos_level->pkey_range_len = new_range_len; + } + } + } + ; + +qos_level_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + /* + * qos_match_rule_entry values: + * qos_match_rule_use + * qos_match_rule_qos_class + * qos_match_rule_qos_level_name + * qos_match_rule_source + * qos_match_rule_destination + * qos_match_rule_service_id + * qos_match_rule_pkey + */ + + +qos_match_rule_use: qos_match_rule_use_start single_string { + /* 'use' of 'qos-match-rule' - one instance */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->use) + { + __qos_parser_error("'qos-match-rule' has multiple 'use' tags"); + cl_list_remove_all(&tmp_parser_struct.str_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->use = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_use_start: TK_USE { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_class: qos_match_rule_qos_class_start list_of_ranges { + /* 'qos-class' in 'qos-match-rule' - num of instances of list of ranges */ + /* list of class ranges (QoS Class is 12-bit value) */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->qos_class_range_len ) + { + p_current_qos_match_rule->qos_class_range_arr = range_arr; + p_current_qos_match_rule->qos_class_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->qos_class_range_arr, + p_current_qos_match_rule->qos_class_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->qos_class_range_arr = new_range_arr; + p_current_qos_match_rule->qos_class_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_qos_class_start: TK_QOS_CLASS { + RESET_BUFFER; + } + ; + +qos_match_rule_source: qos_match_rule_source_start string_list { + /* 'source' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->source_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_source_start: TK_SOURCE { + RESET_BUFFER; + } + ; + +qos_match_rule_destination: qos_match_rule_destination_start string_list { + /* 'destination' in 'qos-match-rule' - text */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + while( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + cl_list_insert_tail(&p_current_qos_match_rule->destination_list,tmp_str); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_destination_start: TK_DESTINATION { + RESET_BUFFER; + } + ; + +qos_match_rule_qos_level_name: qos_match_rule_qos_level_name_start single_string { + /* 'qos-level-name' in 'qos-match-rule' - single string */ + cl_list_iterator_t list_iterator; + char * tmp_str; + + if (p_current_qos_match_rule->qos_level_name) + { + __qos_parser_error("qos-match-rule has multiple 'qos-level-name' tags"); + cl_list_remove_all(&tmp_parser_struct.num_list); + return 1; + } + + list_iterator = cl_list_head(&tmp_parser_struct.str_list); + if ( list_iterator != cl_list_end(&tmp_parser_struct.str_list) ) + { + tmp_str = (char*)cl_list_obj(list_iterator); + if (tmp_str) + p_current_qos_match_rule->qos_level_name = tmp_str; + } + cl_list_remove_all(&tmp_parser_struct.str_list); + } + ; + +qos_match_rule_qos_level_name_start: TK_QOS_LEVEL_NAME { + RESET_BUFFER; + } + ; + +qos_match_rule_service_id: qos_match_rule_service_id_start list_of_ranges { + /* 'service-id' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->service_id_range_len ) + { + p_current_qos_match_rule->service_id_range_arr = range_arr; + p_current_qos_match_rule->service_id_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->service_id_range_arr, + p_current_qos_match_rule->service_id_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->service_id_range_arr = new_range_arr; + p_current_qos_match_rule->service_id_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_service_id_start: TK_SERVICE_ID { + RESET_BUFFER; + } + ; + +qos_match_rule_pkey: qos_match_rule_pkey_start list_of_ranges { + /* 'pkey' in 'qos-match-rule' - num of instances of list of ranges */ + if (cl_list_count(&tmp_parser_struct.num_pair_list)) + { + uint64_t ** range_arr; + unsigned range_len; + + __rangelist2rangearr( &tmp_parser_struct.num_pair_list, + &range_arr, + &range_len ); + + if ( !p_current_qos_match_rule->pkey_range_len ) + { + p_current_qos_match_rule->pkey_range_arr = range_arr; + p_current_qos_match_rule->pkey_range_len = range_len; + } + else + { + uint64_t ** new_range_arr; + unsigned new_range_len; + __merge_rangearr( p_current_qos_match_rule->pkey_range_arr, + p_current_qos_match_rule->pkey_range_len, + range_arr, + range_len, + &new_range_arr, + &new_range_len ); + p_current_qos_match_rule->pkey_range_arr = new_range_arr; + p_current_qos_match_rule->pkey_range_len = new_range_len; + } + } + } + ; + +qos_match_rule_pkey_start: TK_PKEY { + RESET_BUFFER; + } + ; + + + /* + * Common part + */ + + +single_string: single_string_elems { + cl_list_insert_tail(&tmp_parser_struct.str_list, + strdup(__parser_strip_white(tmp_parser_struct.str))); + tmp_parser_struct.str[0] = '\0'; + } + ; + +single_string_elems: single_string_element + | single_string_elems single_string_element + ; + +single_string_element: TK_TEXT { + strcat(tmp_parser_struct.str,$1); + free($1); + } + ; + + +string_list: single_string + | string_list TK_COMMA single_string + ; + + + +single_number: number + ; + +num_list: number + | num_list TK_COMMA number + ; + +number: TK_NUMBER { + uint64_t * p_num = (uint64_t*)malloc(sizeof(uint64_t)); + __parser_str2uint64(p_num,$1); + free($1); + cl_list_insert_tail(&tmp_parser_struct.num_list, p_num); + } + ; + +num_list_with_dotdot: number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dotdot TK_COMMA number_from_pair_1 TK_DOTDOT number_from_pair_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +number_from_pair_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_pair_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +list_of_ranges: num_list_with_dash + ; + +num_list_with_dash: single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA number_from_range_1 TK_DASH number_from_range_2 { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + if (tmp_parser_struct.num_pair[0] <= tmp_parser_struct.num_pair[1]) { + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + } + else { + num_pair[1] = tmp_parser_struct.num_pair[0]; + num_pair[0] = tmp_parser_struct.num_pair[1]; + } + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + | num_list_with_dash TK_COMMA single_number_from_range { + uint64_t * num_pair = (uint64_t*)malloc(sizeof(uint64_t)*2); + num_pair[0] = tmp_parser_struct.num_pair[0]; + num_pair[1] = tmp_parser_struct.num_pair[1]; + cl_list_insert_tail(&tmp_parser_struct.num_pair_list, num_pair); + } + ; + +single_number_from_range: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +number_from_range_1: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[0],$1); + free($1); + } + ; + +number_from_range_2: TK_NUMBER { + __parser_str2uint64(&tmp_parser_struct.num_pair[1],$1); + free($1); + } + ; + +%% + +/*************************************************** + ***************************************************/ + +int osm_qos_parse_policy_file(IN osm_subn_t * const p_subn) +{ + int res = 0; + struct stat statbuf; + static boolean_t first_time = TRUE; + p_qos_parser_osm_log = &p_subn->p_osm->log; + + OSM_LOG_ENTER(p_qos_parser_osm_log, osm_qos_parse_policy_file); + + osm_qos_policy_destroy(p_subn->p_qos_policy); + p_subn->p_qos_policy = NULL; + + if (!stat(p_subn->opt.qos_policy_file, &statbuf)) { + + if (strcmp(p_subn->opt.qos_policy_file,OSM_DEFAULT_QOS_POLICY_FILE)) { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse_policy_file: ERR AC01: " + "QoS policy file not found (%s)\n", + p_subn->opt.qos_policy_file); + res = 1; + } + else + osm_log(p_qos_parser_osm_log, OSM_LOG_VERBOSE, + "osm_qos_parse_policy_file: " + "QoS policy file not found (%s)\n", + p_subn->opt.qos_policy_file); + + goto Exit; + } + + __qos_parser_in = fopen (p_subn->opt.qos_policy_file, "r"); + if (!__qos_parser_in) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse_policy_file: ERR AC02: " + "Failed opening QoS policy file (%s)\n", + p_subn->opt.qos_policy_file); + res = 1; + goto Exit; + } + + if (first_time) + { + first_time = FALSE; + osm_log(p_qos_parser_osm_log, OSM_LOG_INFO, + "osm_qos_parse_policy_file: Loading QoS policy file (%s)\n", + p_subn->opt.qos_policy_file); + } + + column_num = 1; + line_num = 1; + + p_subn->p_qos_policy = osm_qos_policy_create(); + + __parser_tmp_struct_init(); + p_qos_policy = p_subn->p_qos_policy; + + res = __qos_parser_parse(); + + __parser_tmp_struct_destroy(); + + if (res != 0) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse_policy_file: ERR AC03: " + "Failed parsing QoS policy file (%s)\n", + p_subn->opt.qos_policy_file); + osm_qos_policy_destroy(p_subn->p_qos_policy); + p_subn->p_qos_policy = NULL; + res = 1; + goto Exit; + } + + if (osm_qos_policy_validate(p_subn->p_qos_policy,p_qos_parser_osm_log)) + { + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "osm_qos_parse_policy_file: ERR AC04: " + "Error(s) in QoS policy file (%s)\n", + p_subn->opt.qos_policy_file); + osm_qos_policy_destroy(p_subn->p_qos_policy); + p_subn->p_qos_policy = NULL; + res = 1; + goto Exit; + } + + Exit: + if (__qos_parser_in) + fclose(__qos_parser_in); + OSM_LOG_EXIT(p_qos_parser_osm_log); + return res; +} + +/*************************************************** + ***************************************************/ + +int __qos_parser_wrap() +{ + return(1); +} + +/*************************************************** + ***************************************************/ + +void __qos_parser_error (char *s) +{ + OSM_LOG_ENTER(p_qos_parser_osm_log, __qos_parser_error); + osm_log(p_qos_parser_osm_log, OSM_LOG_ERROR, + "__qos_parser_error: ERR AC05: " + "Syntax error (line %d:%d): %s. " + "Last text read: \"%s\"\n", + line_num, column_num, s, __parser_strip_white(__qos_parser_text)); + OSM_LOG_EXIT(p_qos_parser_osm_log); +} + +/*************************************************** + ***************************************************/ + +static char * __parser_strip_white(char * str) +{ + int i; + for (i = (strlen(str)-1); i >= 0; i--) + { + if (isspace(str[i])) + str[i] = '\0'; + else + break; + } + for (i = 0; i < strlen(str); i++) + { + if (!isspace(str[i])) + break; + } + return &(str[i]); +} + +/*************************************************** + ***************************************************/ + +static void __parser_str2uint64(uint64_t * p_val, char * str) +{ +#if __WORDSIZE == 64 + *p_val = strtoul(str, NULL, 0); +#else + *p_val = strtoull(str, NULL, 0); +#endif +} + +/*************************************************** + ***************************************************/ + +static void __parser_port_group_start() +{ + p_current_port_group = osm_qos_policy_port_group_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_port_group_end() +{ + if(!p_current_port_group->name) + { + __qos_parser_error("port-group validation failed - no port group name specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->port_groups, + p_current_port_group); + p_current_port_group = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_vlarb_scope_start() +{ + p_current_vlarb_scope = osm_qos_policy_vlarb_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_vlarb_scope_end() +{ + if ( !cl_list_count(&p_current_vlarb_scope->group_list) && + !cl_list_count(&p_current_vlarb_scope->across_list) ) + { + __qos_parser_error("vlarb-scope validation failed - no port groups specified by 'group' or by 'across'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->vlarb_tables, + p_current_vlarb_scope); + p_current_vlarb_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_sl2vl_scope_start() +{ + p_current_sl2vl_scope = osm_qos_policy_sl2vl_scope_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_sl2vl_scope_end() +{ + if (!p_current_sl2vl_scope->sl2vl_table_set) + { + __qos_parser_error("sl2vl-scope validation failed - no sl2vl table specified"); + return -1; + } + if ( !cl_list_count(&p_current_sl2vl_scope->group_list) && + !cl_list_count(&p_current_sl2vl_scope->across_to_list) && + !cl_list_count(&p_current_sl2vl_scope->across_from_list) ) + { + __qos_parser_error("sl2vl-scope validation failed - no port groups specified by 'group', 'across-to' or 'across-from'"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->sl2vl_tables, + p_current_sl2vl_scope); + p_current_sl2vl_scope = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_qos_level_start() +{ + p_current_qos_level = osm_qos_policy_qos_level_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_qos_level_end() +{ + if (!p_current_qos_level->sl_set) + { + __qos_parser_error("qos-level validation failed - no 'sl' specified"); + return -1; + } + if (!p_current_qos_level->name) + { + __qos_parser_error("qos-level validation failed - no 'name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_levels, + p_current_qos_level); + p_current_qos_level = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_match_rule_start() +{ + p_current_qos_match_rule = osm_qos_policy_match_rule_create(); +} + +/*************************************************** + ***************************************************/ + +static int __parser_match_rule_end() +{ + if (!p_current_qos_match_rule->qos_level_name) + { + __qos_parser_error("match-rule validation failed - no 'qos-level-name' specified"); + return -1; + } + + cl_list_insert_tail(&p_qos_policy->qos_match_rules, + p_current_qos_match_rule); + p_current_qos_match_rule = NULL; + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_init() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_construct(&tmp_parser_struct.str_list); + cl_list_init(&tmp_parser_struct.str_list, 10); + cl_list_construct(&tmp_parser_struct.num_list); + cl_list_init(&tmp_parser_struct.num_list, 10); + cl_list_construct(&tmp_parser_struct.num_pair_list); + cl_list_init(&tmp_parser_struct.num_pair_list, 10); +} + +/*************************************************** + ***************************************************/ + +/* + * Do NOT free objects from the temp struct. + * Either they are inserted into the parse tree data + * structure, or they are already freed when copying + * their values to the parse tree data structure. + */ +static void __parser_tmp_struct_reset() +{ + tmp_parser_struct.str[0] = '\0'; + cl_list_remove_all(&tmp_parser_struct.str_list); + cl_list_remove_all(&tmp_parser_struct.num_list); + cl_list_remove_all(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static void __parser_tmp_struct_destroy() +{ + __parser_tmp_struct_reset(); + cl_list_destroy(&tmp_parser_struct.str_list); + cl_list_destroy(&tmp_parser_struct.num_list); + cl_list_destroy(&tmp_parser_struct.num_pair_list); +} + +/*************************************************** + ***************************************************/ + +static int OSM_CDECL +__cmp_num_range( + const void * p1, + const void * p2) +{ + uint64_t * pair1 = *((uint64_t **)p1); + uint64_t * pair2 = *((uint64_t **)p2); + + if (pair1[0] < pair2[0]) + return -1; + if (pair1[0] > pair2[0]) + return 1; + + if (pair1[1] < pair2[1]) + return -1; + if (pair1[1] > pair2[1]) + return 1; + + return 0; +} + +/*************************************************** + ***************************************************/ + +static void __sort_reduce_rangearr( + uint64_t ** arr, + unsigned arr_len, + uint64_t ** * p_res_arr, + unsigned * p_res_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned last_valid_ind = 0; + unsigned valid_cnt = 0; + uint64_t ** res_arr; + boolean_t * is_valir_arr; + + *p_res_arr = NULL; + *p_res_arr_len = 0; + + qsort(arr, arr_len, sizeof(uint64_t*), __cmp_num_range); + + is_valir_arr = (boolean_t *)malloc(arr_len * sizeof(boolean_t)); + is_valir_arr[last_valid_ind] = TRUE; + valid_cnt++; + for (i = 1; i < arr_len; i++) + { + if (arr[i][0] <= arr[last_valid_ind][1]) + { + if (arr[i][1] > arr[last_valid_ind][1]) + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else if ((arr[i][0] - 1) == arr[last_valid_ind][1]) + { + arr[last_valid_ind][1] = arr[i][1]; + free(arr[i]); + arr[i] = NULL; + is_valir_arr[i] = FALSE; + } + else + { + is_valir_arr[i] = TRUE; + last_valid_ind = i; + valid_cnt++; + } + } + + res_arr = (uint64_t **)malloc(valid_cnt * sizeof(uint64_t *)); + for (i = 0; i < arr_len; i++) + { + if (is_valir_arr[i]) + res_arr[j++] = arr[i]; + } + free(arr); + + *p_res_arr = res_arr; + *p_res_arr_len = valid_cnt; +} + +/*************************************************** + ***************************************************/ + +static void __rangelist2rangearr( + cl_list_t * p_list, + uint64_t ** * p_arr, + unsigned * p_arr_len) +{ + cl_list_iterator_t list_iterator; + unsigned len = cl_list_count(p_list); + unsigned i = 0; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + list_iterator = cl_list_head(p_list); + while( list_iterator != cl_list_end(p_list) ) + { + tmp_arr[i++] = (uint64_t *)cl_list_obj(list_iterator); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(p_list); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + +static void __merge_rangearr( + uint64_t ** range_arr_1, + unsigned range_len_1, + uint64_t ** range_arr_2, + unsigned range_len_2, + uint64_t ** * p_arr, + unsigned * p_arr_len ) +{ + unsigned i = 0; + unsigned j = 0; + unsigned len = range_len_1 + range_len_2; + uint64_t ** tmp_arr; + uint64_t ** res_arr = NULL; + unsigned res_arr_len = 0; + + *p_arr = NULL; + *p_arr_len = 0; + + tmp_arr = (uint64_t **)malloc(len * sizeof(uint64_t *)); + + for (i = 0; i < range_len_1; i++) + tmp_arr[j++] = range_arr_1[i]; + for (i = 0; i < range_len_2; i++) + tmp_arr[j++] = range_arr_2[i]; + free(range_arr_1); + free(range_arr_2); + + __sort_reduce_rangearr( tmp_arr, + len, + &res_arr, + &res_arr_len ); + *p_arr = res_arr; + *p_arr_len = res_arr_len; +} + +/*************************************************** + ***************************************************/ + -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 27 16:09:50 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:09:50 +0300 Subject: [ofa-general] [PATCH 3/7 V3] osm: QoS policy C & H files Message-ID: <46D359BE.6040009@dev.mellanox.co.il> QoS policy data structures and functions Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_qos_policy.h | 189 +++++++ opensm/opensm/osm_qos_policy.c | 921 ++++++++++++++++++++++++++++++++ 2 files changed, 1110 insertions(+), 0 deletions(-) create mode 100644 opensm/include/opensm/osm_qos_policy.h create mode 100644 opensm/opensm/osm_qos_policy.c diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h new file mode 100644 index 0000000..dd15f99 --- /dev/null +++ b/opensm/include/opensm/osm_qos_policy.h @@ -0,0 +1,189 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Declaration of OSM QoS Policy data types and functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#ifndef OSM_QOS_POLICY_H +#define OSM_QOS_POLICY_H + +#include +#include +#include +#include + +#define YYSTYPE char * +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" + +/***************************************************/ + +typedef struct _osm_qos_port_group_t { + char *name; /* single string (this port group name) */ + char *use; /* single string (description) */ + cl_list_t port_name_list; /* list of port names (.../.../...) */ + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ + unsigned guid_range_len; /* num of guid ranges in the array */ + cl_list_t partition_list; /* list of partition names */ + boolean_t node_type_ca; + boolean_t node_type_switch; + boolean_t node_type_router; + boolean_t node_type_self; +} osm_qos_port_group_t; + +/***************************************************/ + +typedef struct _osm_qos_vlarb_scope_t { + cl_list_t group_list; /* list of group names (strings) */ + cl_list_t across_list; /* list of 'across' group names (strings) */ + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ + uint32_t vl_high_limit; /* single integer */ + boolean_t vl_high_limit_set; +} osm_qos_vlarb_scope_t; + +/***************************************************/ + +typedef struct _osm_qos_sl2vl_scope_t { + cl_list_t group_list; /* list of strings (port group names) */ + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; + cl_list_t across_from_list; /* list of strings (port group names) */ + cl_list_t across_to_list; /* list of strings (port group names) */ + uint8_t sl2vl_table[16]; /* array of sl2vl values */ + boolean_t sl2vl_table_set; +} osm_qos_sl2vl_scope_t; + +/***************************************************/ + +typedef struct _osm_qos_level_t { + char *use; + char *name; + uint8_t sl; + boolean_t sl_set; + uint8_t mtu_limit; + boolean_t mtu_limit_set; + uint8_t rate_limit; + boolean_t rate_limit_set; + uint8_t pkt_life; + boolean_t pkt_life_set; + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ + unsigned path_bits_range_len; /* num of bit ranges in the array */ + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_level_t; + + +/***************************************************/ + +typedef struct _osm_qos_match_rule_t { + char *use; + cl_list_t source_list; /* list of strings */ + cl_list_t source_group_list; /* list of pointers to relevant port-group */ + cl_list_t destination_list; /* list of strings */ + cl_list_t destination_group_list; /* list of pointers to relevant port-group */ + char *qos_level_name; + osm_qos_level_t *p_qos_level; + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) */ + unsigned service_id_range_len; + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ + unsigned qos_class_range_len; + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ + unsigned pkey_range_len; +} osm_qos_match_rule_t; + +/***************************************************/ + +typedef struct _osm_qos_policy_t { + cl_list_t port_groups; /* list of osm_qos_port_group_t */ + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ + cl_list_t qos_levels; /* list of osm_qos_level_t */ + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ + osm_qos_level_t *p_default_qos_level; /* default QoS level */ +} osm_qos_policy_t; + +/***************************************************/ + +osm_qos_port_group_t * osm_qos_policy_port_group_create(); +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p_port_group); + +osm_qos_vlarb_scope_t * osm_qos_policy_vlarb_scope_create(); +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p_vlarb_scope); + +osm_qos_sl2vl_scope_t * osm_qos_policy_sl2vl_scope_create(); +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p_sl2vl_scope); + +osm_qos_level_t * osm_qos_policy_qos_level_create(); +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p_qos_level); + +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey); + +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp); + +osm_qos_match_rule_t * osm_qos_policy_match_rule_create(); +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p_match_rule); + +osm_qos_policy_t * osm_qos_policy_create(); +void osm_qos_policy_destroy(osm_qos_policy_t * p_qos_policy); +int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, osm_log_t * p_log); + +void osm_qos_policy_get_qos_level_by_pr(IN const osm_qos_policy_t * p_qos_policy, + IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level); + +/***************************************************/ + +int osm_qos_parse_policy_file(IN osm_subn_t * const p_subn); + +/***************************************************/ + +#endif /* ifndef OSM_QOS_POLICY_H */ + diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c new file mode 100644 index 0000000..a5a8856 --- /dev/null +++ b/opensm/opensm/osm_qos_policy.c @@ -0,0 +1,921 @@ +/* + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * OSM QoS Policy functions. + * + * Environment: + * Linux User Mode + * + * Author: + * Yevgeny Kliteynik, Mellanox + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/*************************************************** + ***************************************************/ + +static boolean_t +__is_num_in_range_arr(uint64_t ** range_arr, + unsigned range_arr_len, uint64_t num) +{ + unsigned ind_1 = 0; + unsigned ind_2 = range_arr_len - 1; + unsigned ind_mid; + + if (!range_arr || !range_arr_len) + return FALSE; + + while (ind_1 <= ind_2) { + if (num < range_arr[ind_1][0] || num > range_arr[ind_2][1]) + return FALSE; + else if (num <= range_arr[ind_1][1] || num >= range_arr[ind_2][0]) + return TRUE; + + ind_mid = ind_1 + (ind_2 - ind_1 + 1)/2; + + if (num < range_arr[ind_mid][0]) + ind_2 = ind_mid; + else if (num > range_arr[ind_mid][1]) + ind_1 = ind_mid; + else + return TRUE; + + ind_1++; + ind_2--; + } + + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static void __free_single_element(void *p_element, void *context) +{ + if (p_element) + free(p_element); +} + +/*************************************************** + ***************************************************/ + +osm_qos_port_group_t *osm_qos_policy_port_group_create() +{ + osm_qos_port_group_t *p = + (osm_qos_port_group_t *) malloc(sizeof(osm_qos_port_group_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_port_group_t)); + + cl_list_init(&p->port_name_list, 10); + cl_list_init(&p->partition_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->name) + free(p->name); + if (p->use) + free(p->use); + + for (i = 0; i < p->guid_range_len; i++) + free(p->guid_range_arr[i]); + if (p->guid_range_arr) + free(p->guid_range_arr); + + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); + cl_list_remove_all(&p->port_name_list); + cl_list_destroy(&p->port_name_list); + + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); + cl_list_remove_all(&p->partition_list); + cl_list_destroy(&p->partition_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create() +{ + osm_qos_vlarb_scope_t *p = + (osm_qos_vlarb_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list, 10); + cl_list_init(&p->across_list, 10); + cl_list_init(&p->vlarb_high_list, 10); + cl_list_init(&p->vlarb_low_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_list); + cl_list_remove_all(&p->vlarb_high_list); + cl_list_remove_all(&p->vlarb_low_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_list); + cl_list_destroy(&p->vlarb_high_list); + cl_list_destroy(&p->vlarb_low_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create() +{ + osm_qos_sl2vl_scope_t *p = + (osm_qos_sl2vl_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); + + cl_list_init(&p->group_list, 10); + cl_list_init(&p->across_from_list, 10); + cl_list_init(&p->across_to_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) +{ + if (!p) + return; + + cl_list_apply_func(&p->group_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); + + cl_list_remove_all(&p->group_list); + cl_list_remove_all(&p->across_from_list); + cl_list_remove_all(&p->across_to_list); + + cl_list_destroy(&p->group_list); + cl_list_destroy(&p->across_from_list); + cl_list_destroy(&p->across_to_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_level_t *osm_qos_policy_qos_level_create() +{ + osm_qos_level_t *p = + (osm_qos_level_t *) malloc(sizeof(osm_qos_level_t)); + if (!p) + return NULL; + memset(p, 0, sizeof(osm_qos_level_t)); + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->use) + free(p->use); + + for (i = 0; i < p->path_bits_range_len; i++) + free(p->path_bits_range_arr[i]); + if (p->path_bits_range_arr) + free(p->path_bits_range_arr); + + free(p); +} + +/*************************************************** + ***************************************************/ + +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, + IN ib_net16_t pkey) +{ + if (!p_qos_level || !p_qos_level->pkey_range_len) + return FALSE; + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, + p_qos_level->pkey_range_len, + cl_ntoh16(pkey)); +} + +/*************************************************** + ***************************************************/ + +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp) +{ + unsigned i; + uint16_t pkey_ho = 0; + + if (!p_qos_level || !p_qos_level->pkey_range_len) + return 0; + + /* + * ToDo: This approach is not optimal. + * Think how to find shared pkey that also exists + * in QoS level in less runtime. + */ + + for (i = 0; i < p_qos_level->pkey_range_len; i++) { + for (pkey_ho = p_qos_level->pkey_range_arr[i][0]; + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; pkey_ho++) { + if (osm_physp_share_this_pkey + (p_src_physp, p_dest_physp, cl_hton16(pkey_ho))) + return cl_hton16(pkey_ho); + } + } + + return 0; +} + +/*************************************************** + ***************************************************/ + +osm_qos_match_rule_t *osm_qos_policy_match_rule_create() +{ + osm_qos_match_rule_t *p = + (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); + if (!p) + return NULL; + + memset(p, 0, sizeof(osm_qos_match_rule_t)); + + cl_list_init(&p->source_list, 10); + cl_list_init(&p->source_group_list, 10); + cl_list_init(&p->destination_list, 10); + cl_list_init(&p->destination_group_list, 10); + + return p; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) +{ + unsigned i; + + if (!p) + return; + + if (p->qos_level_name) + free(p->qos_level_name); + if (p->use) + free(p->use); + + for (i = 0; i < p->service_id_range_len; i++) + free(p->service_id_range_arr[i]); + if (p->service_id_range_arr) + free(p->service_id_range_arr); + + for (i = 0; i < p->qos_class_range_len; i++) + free(p->qos_class_range_arr[i]); + if (p->qos_class_range_arr) + free(p->qos_class_range_arr); + + cl_list_apply_func(&p->source_list, __free_single_element, NULL); + cl_list_remove_all(&p->source_list); + cl_list_destroy(&p->source_list); + + cl_list_remove_all(&p->source_group_list); + cl_list_destroy(&p->source_group_list); + + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); + cl_list_remove_all(&p->destination_list); + cl_list_destroy(&p->destination_list); + + cl_list_remove_all(&p->destination_group_list); + cl_list_destroy(&p->destination_group_list); + + free(p); +} + +/*************************************************** + ***************************************************/ + +osm_qos_policy_t * osm_qos_policy_create() +{ + osm_qos_policy_t * p_qos_policy = (osm_qos_policy_t *)malloc(sizeof(osm_qos_policy_t)); + if (!p_qos_policy) + return NULL; + + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); + + cl_list_construct(&p_qos_policy->port_groups); + cl_list_init(&p_qos_policy->port_groups, 10); + + cl_list_construct(&p_qos_policy->vlarb_tables); + cl_list_init(&p_qos_policy->vlarb_tables, 10); + + cl_list_construct(&p_qos_policy->sl2vl_tables); + cl_list_init(&p_qos_policy->sl2vl_tables, 10); + + cl_list_construct(&p_qos_policy->qos_levels); + cl_list_init(&p_qos_policy->qos_levels, 10); + + cl_list_construct(&p_qos_policy->qos_match_rules); + cl_list_init(&p_qos_policy->qos_match_rules, 10); + + return p_qos_policy; +} + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_destroy(osm_qos_policy_t * p_qos_policy) +{ + cl_list_iterator_t list_iterator; + osm_qos_port_group_t *p_port_group = NULL; + osm_qos_vlarb_scope_t *p_vlarb_scope = NULL; + osm_qos_sl2vl_scope_t *p_sl2vl_scope = NULL; + osm_qos_level_t *p_qos_level = NULL; + osm_qos_match_rule_t *p_qos_match_rule = NULL; + + if (!p_qos_policy) + return; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (p_port_group) + osm_qos_policy_port_group_destroy(p_port_group); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->port_groups); + cl_list_destroy(&p_qos_policy->port_groups); + + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); + while (list_iterator != cl_list_end(&p_qos_policy->vlarb_tables)) { + p_vlarb_scope = + (osm_qos_vlarb_scope_t *) cl_list_obj(list_iterator); + if (p_vlarb_scope) + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->vlarb_tables); + cl_list_destroy(&p_qos_policy->vlarb_tables); + + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); + while (list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables)) { + p_sl2vl_scope = + (osm_qos_sl2vl_scope_t *) cl_list_obj(list_iterator); + if (p_sl2vl_scope) + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->sl2vl_tables); + cl_list_destroy(&p_qos_policy->sl2vl_tables); + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); + if (p_qos_level) + osm_qos_policy_qos_level_destroy(p_qos_level); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_levels); + cl_list_destroy(&p_qos_policy->qos_levels); + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); + if (p_qos_match_rule) + osm_qos_policy_match_rule_destroy(p_qos_match_rule); + list_iterator = cl_list_next(list_iterator); + } + cl_list_remove_all(&p_qos_policy->qos_match_rules); + cl_list_destroy(&p_qos_policy->qos_match_rules); + + free(p_qos_policy); + + p_qos_policy = NULL; +} + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group(osm_subn_t * p_subn, + const osm_physp_t * p_physp, + osm_qos_port_group_t * p_port_group) +{ + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); + osm_prtn_t *p_prtn = NULL; + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); + uint64_t port_guid_ho = cl_ntoh64(port_guid); + uint8_t node_type = osm_node_get_type(p_node); + cl_list_iterator_t list_iterator; + char *partition_name; + + /* check whether this port's type matches any of group's types */ + + if ((node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) + || (node_type == IB_NODE_TYPE_ROUTER + && p_port_group->node_type_router)) + return TRUE; + + /* check whether this port's guid is in range of this group's guids */ + + if (__is_num_in_range_arr(p_port_group->guid_range_arr, + p_port_group->guid_range_len, port_guid_ho)) + return TRUE; + + /* check whether this port is member of this group's partitions */ + + list_iterator = cl_list_head(&p_port_group->partition_list); + while (list_iterator != cl_list_end(&p_port_group->partition_list)) { + partition_name = (char *)cl_list_obj(list_iterator); + if (partition_name && strlen(partition_name)) { + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); + if (p_prtn) { + if (osm_prtn_is_guid(p_prtn, port_guid)) + return TRUE; + } + } + list_iterator = cl_list_next(list_iterator); + } + + /* check whether this port's name matches any of group's names */ + + /* + * TODO: check port names + * + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; + */ + + return FALSE; +} /* __qos_policy_is_port_in_group() */ + +/*************************************************** + ***************************************************/ + +static boolean_t +__qos_policy_is_port_in_group_list(const osm_pr_rcv_t * p_rcv, + const osm_physp_t * p_physp, + cl_list_t * p_port_group_list) +{ + osm_qos_port_group_t *p_port_group; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(p_port_group_list); + while (list_iterator != cl_list_end(p_port_group_list)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (p_port_group) { + if (__qos_policy_is_port_in_group + (p_rcv->p_subn, p_physp, p_port_group)) + return TRUE; + } + list_iterator = cl_list_next(list_iterator); + } + return FALSE; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr( + const osm_qos_policy_t * p_qos_policy, + const osm_pr_rcv_t * p_rcv, + const ib_path_rec_t * p_pr, + const osm_physp_t * p_src_physp, + const osm_physp_t * p_dest_physp, + ib_net64_t comp_mask) +{ + osm_qos_match_rule_t *p_qos_match_rule = NULL; + cl_list_iterator_t list_iterator; + + if (!cl_list_count(&p_qos_policy->qos_match_rules)) + return NULL; + + /* Go over all QoS match rules and find the one that matches the request */ + + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); + if (!p_qos_match_rule) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + /* If a match rule has Source groups, PR request source has to be in this list */ + + if (cl_list_count(&p_qos_match_rule->source_group_list)) { + if (!__qos_policy_is_port_in_group_list(p_rcv, + p_src_physp, + &p_qos_match_rule-> + source_group_list)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has Destination groups, PR request dest. has to be in this list */ + + if (cl_list_count(&p_qos_match_rule->destination_group_list)) { + if (!__qos_policy_is_port_in_group_list(p_rcv, + p_dest_physp, + &p_qos_match_rule-> + destination_group_list)) + { + list_iterator = cl_list_next(list_iterator); + continue; + } + } + + /* If a match rule has QoS classes, PR request HAS + to have a matching QoS class to match the rule */ + + if (p_qos_match_rule->qos_class_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->qos_class_range_arr, + p_qos_match_rule->qos_class_range_len, + ib_path_rec_qos_class(p_pr))) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has Service IDs, PR request HAS + to have a matching Service ID to match the rule */ + + if (p_qos_match_rule->service_id_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->service_id_range_arr, + p_qos_match_rule->service_id_range_len, + p_pr->service_id)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* If a match rule has PKeys, PR request HAS + to have a matching PKey to match the rule */ + + if (p_qos_match_rule->pkey_range_len) { + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + if (!__is_num_in_range_arr + (p_qos_match_rule->pkey_range_arr, + p_qos_match_rule->pkey_range_len, + ib_path_rec_qos_class(p_pr))) { + list_iterator = cl_list_next(list_iterator); + continue; + } + + } + + /* if we got here, then this match-rule matched this PR request */ + break; + } + + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) + return NULL; + + return p_qos_match_rule; +} /* __qos_policy_get_match_rule_by_pr() */ + +/*************************************************** + ***************************************************/ + +static osm_qos_level_t *__qos_policy_get_qos_level_by_name(osm_qos_policy_t * p_qos_policy, + char *name) +{ + osm_qos_level_t *p_qos_level = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->qos_levels); + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); + if (!p_qos_level) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name, p_qos_level->name) == 0) + return p_qos_level; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +static osm_qos_port_group_t *__qos_policy_get_port_group_by_name(osm_qos_policy_t * p_qos_policy, + const char *const name) +{ + osm_qos_port_group_t *p_port_group = NULL; + cl_list_iterator_t list_iterator; + + list_iterator = cl_list_head(&p_qos_policy->port_groups); + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { + p_port_group = + (osm_qos_port_group_t *) cl_list_obj(list_iterator); + if (!p_port_group) + continue; + + /* names are case INsensitive */ + if (strcasecmp(name, p_port_group->name) == 0) + return p_port_group; + + list_iterator = cl_list_next(list_iterator); + } + + return NULL; +} + +/*************************************************** + ***************************************************/ + +int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, + osm_log_t *p_log) +{ + cl_list_iterator_t match_rules_list_iterator; + cl_list_iterator_t list_iterator; + osm_qos_port_group_t *p_port_group = NULL; + osm_qos_match_rule_t *p_qos_match_rule = NULL; + char *str; + unsigned i; + int res = 0; + + OSM_LOG_ENTER(p_log, osm_qos_policy_validate); + + /* set default qos level */ + + p_qos_policy->p_default_qos_level = + __qos_policy_get_qos_level_by_name(p_qos_policy, OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + if (!p_qos_policy->p_default_qos_level) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC10: " + "Default qos-level (%s) not defined.\n", + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); + res = 1; + goto Exit; + } + + /* scan all the match rules, and fill the lists of pointers to + relevant qos levels and port groups to speed up PR matching */ + + i = 1; + match_rules_list_iterator = + cl_list_head(&p_qos_policy->qos_match_rules); + while (match_rules_list_iterator != + cl_list_end(&p_qos_policy->qos_match_rules)) { + p_qos_match_rule = + (osm_qos_match_rule_t *) + cl_list_obj(match_rules_list_iterator); + CL_ASSERT(p_qos_match_rule); + + /* find the matching qos-level for each match-rule */ + + p_qos_match_rule->p_qos_level = + __qos_policy_get_qos_level_by_name(p_qos_policy, + p_qos_match_rule->qos_level_name); + + if (!p_qos_match_rule->p_qos_level) { + osm_log(p_log, OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC11: " + "qos-match-rule num %u: qos-level '%s' not found\n", + i, p_qos_match_rule->qos_level_name); + res = 1; + goto Exit; + } + + /* find the matching port-group for element of source_list */ + + if (cl_list_count(&p_qos_match_rule->source_list)) { + list_iterator = + cl_list_head(&p_qos_match_rule->source_list); + while (list_iterator != + cl_list_end(&p_qos_match_rule->source_list)) { + str = (char *)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = + __qos_policy_get_port_group_by_name(p_qos_policy, str); + if (!p_port_group) { + osm_log(p_log, + OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC12: " + "qos-match-rule num %u: source port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule-> + source_group_list, + p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* find the matching port-group for element of destination_list */ + + if (cl_list_count(&p_qos_match_rule->destination_list)) { + list_iterator = + cl_list_head(&p_qos_match_rule->destination_list); + while (list_iterator != + cl_list_end(&p_qos_match_rule-> + destination_list)) { + str = (char *)cl_list_obj(list_iterator); + CL_ASSERT(str); + + p_port_group = + __qos_policy_get_port_group_by_name(p_qos_policy,str); + if (!p_port_group) { + osm_log(p_log, + OSM_LOG_ERROR, + "osm_qos_policy_validate: ERR AC13: " + "qos-match-rule num %u: destination port-group '%s' not found\n", + i, str); + res = 1; + goto Exit; + } + + cl_list_insert_tail(&p_qos_match_rule-> + destination_group_list, + p_port_group); + + list_iterator = cl_list_next(list_iterator); + } + } + + /* done with the current match-rule */ + + match_rules_list_iterator = + cl_list_next(match_rules_list_iterator); + i++; + } + + Exit: + OSM_LOG_EXIT(p_log); + return res; +} /* osm_qos_policy_validate() */ + +/*************************************************** + ***************************************************/ + +void osm_qos_policy_get_qos_level_by_pr(IN const osm_qos_policy_t * p_qos_policy, + IN const osm_pr_rcv_t * p_rcv, + IN const ib_path_rec_t * p_pr, + IN const osm_physp_t * p_src_physp, + IN const osm_physp_t * p_dest_physp, + IN ib_net64_t comp_mask, + OUT osm_qos_level_t ** pp_qos_level) +{ + osm_qos_match_rule_t *p_qos_match_rule = NULL; + osm_qos_level_t *p_qos_level = NULL; + + OSM_LOG_ENTER(p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr); + + *pp_qos_level = NULL; + + if (!p_qos_policy) + goto Exit; + + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_qos_policy, + p_rcv, + p_pr, + p_src_physp, + p_dest_physp, + comp_mask); + + if (p_qos_match_rule) + p_qos_level = p_qos_match_rule->p_qos_level; + else + p_qos_level = p_qos_policy->p_default_qos_level; + + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "PathRecord request:" + "Src port 0x%016" PRIx64 ", " + "Dst port 0x%016" PRIx64 "\n", + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp))); + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, + "osm_qos_policy_get_qos_level_by_pr: " + "Applying QoS Level %s (%s)\n", + p_qos_level->name, + (p_qos_level->use) ? p_qos_level->use : "no description"); + + *pp_qos_level = p_qos_level; + + Exit: + OSM_LOG_EXIT(p_rcv->p_log); +} /* osm_qos_policy_get_qos_level_by_pr() */ + +/*************************************************** + ***************************************************/ -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 27 16:10:36 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:10:36 +0300 Subject: [ofa-general] [PATCH 4/7 V3] osm: QoS - rename static find_prtn_by_name() to non-static Message-ID: <46D359EC.6040904@dev.mellanox.co.il> Renaming static find_prtn_by_name() to non-static osm_prtn_find_by_name() - this function is used by QoS policy functions. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_partition.h | 29 +++++++++++++++++++++++++++++ opensm/opensm/osm_prtn.c | 4 ++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/opensm/include/opensm/osm_partition.h b/opensm/include/opensm/osm_partition.h index f00034f..e88da40 100644 --- a/opensm/include/opensm/osm_partition.h +++ b/opensm/include/opensm/osm_partition.h @@ -243,5 +243,34 @@ ib_api_status_t osm_prtn_make_partitions(IN osm_log_t * const p_log, * SEE ALSO *********/ +/****f* OpenSM: Partition/osm_prtn_find_by_name +* NAME +* osm_prtn_find_by_name +* +* DESCRIPTION +* Fides partition by name. +* +* SYNOPSIS +*/ +osm_prtn_t * +osm_prtn_find_by_name( + IN osm_subn_t *p_subn, + IN const char *name); +/* +* PARAMETERS +* p_subn +* [in] Pointer to a subnet object. +* +* name +* [in] Required partition name. +* +* RETURN VALUES +* Pointer to the partition object on success. +* +* NOTES +* +* SEE ALSO +*********/ + END_C_DECLS #endif /* _OSM_PARTITION_H_ */ diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 5dabae9..46ee429 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -265,7 +265,7 @@ static uint16_t __generate_pkey(osm_subn_t * p_subn) return 0; } -static osm_prtn_t *find_prtn_by_name(osm_subn_t * p_subn, const char *name) +osm_prtn_t *osm_prtn_find_by_name(osm_subn_t * p_subn, const char *name) { cl_map_item_t *p_next; osm_prtn_t *p; @@ -289,7 +289,7 @@ osm_prtn_t *osm_prtn_make_new(osm_log_t * p_log, osm_subn_t * p_subn, pkey &= cl_hton16((uint16_t) ~ 0x8000); if (!pkey) { - if (name && (p = find_prtn_by_name(p_subn, name))) + if (name && (p = osm_prtn_find_by_name(p_subn, name))) return p; if (!(pkey = __generate_pkey(p_subn))) return NULL; -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 27 16:10:59 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:10:59 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options Message-ID: <46D35A03.2070101@dev.mellanox.co.il> Adding QoS policy file option to OpenSM optionsm and QoS Policy field to subn object. Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ opensm/opensm/main.c | 12 +++++++++++- opensm/opensm/osm_subnet.c | 10 +++++++++- 4 files changed, 50 insertions(+), 2 deletions(-) diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h index 545779b..0a35e22 100644 --- a/opensm/include/opensm/osm_base.h +++ b/opensm/include/opensm/osm_base.h @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" #endif /***********/ + +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE +* NAME +* OSM_DEFAULT_QOS_POLICY_FILE +* +* DESCRIPTION +* Specifies the default QoS policy file name +* +* SYNOPSIS +*/ +#ifdef __WIN__ +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") +#else +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" +#endif +/***********/ + /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS * NAME * OSM_DEFAULT_SWEEP_INTERVAL_SECS diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index a7543dc..5e8b522 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -68,6 +68,7 @@ BEGIN_C_DECLS #define OSM_SUBNET_VECTOR_GROW_SIZE 1 #define OSM_SUBNET_VECTOR_CAPACITY 256 struct _osm_opensm_t; +struct _osm_qos_policy_t; /****h* OpenSM/Subnet * NAME @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { char *partition_config_file; boolean_t no_partition_enforcement; boolean_t no_qos; + char *qos_policy_file; boolean_t accum_log_file; char *console; uint16_t console_port; @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { * specified the log file will be truncated upon reaching * this limit. * +* no_qos +* Boolean that specifies whether the OpenSM QoS functionality +* should be off or on. +* +* qos_policy_file +* Name of the QoS policy file. +* * accum_log_file * If TRUE (default) - the log file will be accumulated. * If FALSE - the log file will be erased before starting current opensm run. @@ -551,6 +560,7 @@ typedef struct _osm_subn { ib_net64_t sm_port_guid; uint8_t sm_state; osm_subn_opt_t opt; + struct _osm_qos_policy_t *p_qos_policy; uint16_t max_unicast_lid_ho; uint16_t max_multicast_lid_ho; uint8_t min_ca_mtu; @@ -619,6 +629,9 @@ typedef struct _osm_subn { * opt * Subnet options structure contains site specific configuration. * +* p_qos_policy +* Subnet QoS policy structure. +* * max_unicast_lid_ho * The minimal max unicast lid reported by all switches * diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index e2541d8..599c4dc 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -269,6 +269,10 @@ void show_usage(void) " The default name is \'" OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); + printf("-Y\n" + "--qos_policy_file\n" + " This option defines the optional QoS policy file.\n" + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); printf("-N\n" "--no_part_enforce\n" " This option disables partition enforcement on switch external ports.\n\n"); @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) char *ignore_guids_file_name = NULL; uint32_t val; const char *const short_option = - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; /* In the array below, the 2nd parameter specifies the number @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) {"Pconfig", 1, NULL, 'P'}, {"no_part_enforce", 0, NULL, 'N'}, {"qos", 0, NULL, 'Q'}, + {"qos_policy_file", 1, NULL, 'Y'}, {"maxsmps", 1, NULL, 'n'}, {"console", 1, NULL, 'q'}, {"V", 0, NULL, 'V'}, @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) opt.no_qos = FALSE; break; + case 'Y': + opt.qos_policy_file = optarg; + printf(" QoS policy file \'%s\'\n", optarg); + break; + case 'y': opt.exit_on_fatal = FALSE; printf(" Staying on fatal initialization errors\n"); diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; p_opt->no_partition_enforcement = FALSE; p_opt->no_qos = TRUE; + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; p_opt->accum_log_file = TRUE; p_opt->port_profile_switch_nodes = FALSE; p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); + opts_unpack_charp("qos_policy_file", + p_key, p_val, &p_opts->qos_policy_file); + opts_unpack_boolean("accum_log_file", p_key, p_val, &p_opts->accum_log_file); @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) fprintf(opts_file, "#\n# QoS OPTIONS\n#\n" "# Disable QoS setup\n" - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); + "no_qos %s\n\n" + "# QoS policy file to be used\n" + "qos_policy_file %s\n\n", + p_opts->no_qos ? "TRUE" : "FALSE", + p_opts->qos_policy_file); subn_dump_qos_options(opts_file, "QoS default options", "qos", -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 27 16:11:28 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:11:28 +0300 Subject: [ofa-general] [PATCH 6/7 V3] osm: QoS - compiling policy file parser Message-ID: <46D35A20.6020902@dev.mellanox.co.il> Compiling QoS policy file parser. C & H files are generated from Lex & Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. Generated files can be removed by "make maintainer-clean". Signed-off-by: Yevgeny Kliteynik --- opensm/include/Makefile.am | 2 ++ opensm/opensm/Makefile.am | 16 +++++++++++++++- opensm/opensm/configure.in | 2 ++ 3 files changed, 19 insertions(+), 1 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..b83205f 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -90,6 +90,8 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ + $(srcdir)/opensm/osm_qos_policy.h \ + $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/complib/cl_thread_osd.h \ $(srcdir)/complib/cl_packon.h \ $(srcdir)/complib/cl_atomic_osd.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..9d57f93 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ osm_vl15intf.c osm_vl_arb_rcv.c \ st.c osm_perfmgr.c osm_perfmgr_db.c \ - osm_event_plugin.c osm_dump.c + osm_event_plugin.c osm_dump.c \ + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c + +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ + +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l + if OSMV_OPENIB opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 @@ -104,3 +112,9 @@ install-exec-hook: if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so + +# generate c and h files from the lex and yacc files +dist-hook: $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c + +maintainer-clean-generic: + rm -f $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in index afb20b3..a49538d 100644 --- a/opensm/opensm/configure.in +++ b/opensm/opensm/configure.in @@ -22,6 +22,8 @@ AC_PROG_INSTALL AC_PROG_LN_S AC_PROG_MAKE_SET AC_PROG_LIBTOOL +AM_PROG_LEX +AC_PROG_YACC dnl Checks for libraries -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Mon Aug 27 16:11:46 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 02:11:46 +0300 Subject: [ofa-general] [PATCH 7/7 V3] osm: QoS - reading policy file Message-ID: <46D35A32.6070003@dev.mellanox.co.il> Reading QoS policy file Signed-off-by: Yevgeny Kliteynik --- opensm/include/opensm/osm_subnet.h | 16 ++++++++-------- opensm/opensm/osm_state_mgr.c | 2 +- opensm/opensm/osm_subnet.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 15 deletions(-) diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h index 5e8b522..7e1a3e7 100644 --- a/opensm/include/opensm/osm_subnet.h +++ b/opensm/include/opensm/osm_subnet.h @@ -1118,28 +1118,28 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opt); * Subnet object, osm_subn_construct, osm_subn_destroy *********/ -/****f* OpenSM: Subnet/osm_subn_parse_conf_file +/****f* OpenSM: Subnet/osm_subn_parse_conf_files * NAME -* osm_subn_rescan_conf_file +* osm_subn_rescan_conf_files * * DESCRIPTION -* The osm_subn_rescan_conf_file function parses the configuration -* file and update selected subnet options +* The osm_subn_rescan_conf_files function parses the configuration +* files and update selected subnet options * * SYNOPSIS */ -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts); +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn); /* * PARAMETERS * -* p_opt -* [in] Pointer to the subnet options structure. +* p_subn +* [in] Pointer to the subnet structure. * * RETURN VALUES * IB_SUCCESS, IB_ERROR * * NOTES -* This uses the same file as osm_subn_parse_conf_file() +* This uses the same file as osm_subn_parse_conf_files() * *********/ diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 030d344..2a94518 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -1878,7 +1878,7 @@ void osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, p_mgr->p_subn->subnet_initialization_error = FALSE; /* rescan configuration updates */ - status = osm_subn_rescan_conf_file(&p_mgr->p_subn->opt); + status = osm_subn_rescan_conf_files(p_mgr->p_subn); if (status != IB_SUCCESS) { osm_log(p_mgr->p_log, OSM_LOG_ERROR, diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index 4162522..c847daf 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -69,6 +69,7 @@ #include #include #include +#include #if defined(PATH_MAX) #define OSM_PATH_MAX (PATH_MAX + 1) @@ -677,7 +678,7 @@ subn_dump_qos_options(FILE * file, /********************************************************************** **********************************************************************/ -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn) { char *p_cache_dir = getenv("OSM_CACHE_DIR"); char file_name[OSM_PATH_MAX]; @@ -704,28 +705,32 @@ ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) subn_parse_qos_options("qos", p_key, p_val, - &p_opts->qos_options); + &p_subn->opt.qos_options); subn_parse_qos_options("qos_ca", p_key, p_val, - &p_opts->qos_ca_options); + &p_subn->opt.qos_ca_options); subn_parse_qos_options("qos_sw0", p_key, p_val, - &p_opts->qos_sw0_options); + &p_subn->opt.qos_sw0_options); subn_parse_qos_options("qos_swe", p_key, p_val, - &p_opts->qos_swe_options); + &p_subn->opt.qos_swe_options); subn_parse_qos_options("qos_rtr", p_key, p_val, - &p_opts->qos_rtr_options); + &p_subn->opt.qos_rtr_options); } } fclose(opts_file); + /* read QoS policy config file */ + if (!p_subn->opt.no_qos) + osm_qos_parse_policy_file(p_subn); + return IB_SUCCESS; } -- 1.5.1.4 From hadi at cyberus.ca Mon Aug 27 16:23:39 2007 From: hadi at cyberus.ca (jamal) Date: Mon, 27 Aug 2007 19:23:39 -0400 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070826.190420.41652839.davem@davemloft.net> References: <46CF7B13.3020701@psc.edu> <20070826044134.eabd18cf.billfink@mindspring.com> <46D229AA.6020900@psc.edu> <20070826.190420.41652839.davem@davemloft.net> Message-ID: <1188257019.4250.55.camel@localhost> On Sun, 2007-26-08 at 19:04 -0700, David Miller wrote: > The transfer is much better behaved if we ACK every two full sized > frames we copy into the receiver, and therefore don't stretch ACK, but > at the cost of cpu utilization. The rx coalescing in theory should help by accumulating more ACKs on the rx side of the sender. But it doesnt seem to do that i.e For the 9K MTU, you are better off to turn off the coalescing if you want higher numbers. Also some of the TOE vendors (chelsio?) claim to have fixed this by reducing bursts on outgoing packets. Bill: who suggested (as per your email) the 75usec value and what was it based on measurement-wise? BTW, thanks for the finding the energy to run those tests and a very refreshing perspective. I dont mean to add more work, but i had some queries; On your earlier tests, i think that Reno showed some significant differences on the lower MTU case over BIC. I wonder if this is consistent? A side note: Although the experimentation reduces the variables (eg tying all to CPU0), it would be more exciting to see multi-cpu and multi-flow sender effect (which IMO is more real world). Last note: you need a newer netstat. > These effects are particularly pronounced on systems where the > bus bandwidth is also one of the limiting factors. Can you elucidate this a little more Dave? Did you mean memory bandwidth? cheers, jamal From sashak at voltaire.com Mon Aug 27 17:31:33 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 03:31:33 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <20070823120325.GA15042@sashak.voltaire.com> Message-ID: <20070828003133.GS18082@sashak.voltaire.com> On 12:35 Mon 27 Aug , Hal Rosenstock wrote: > > > > Thanks. > > > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > > > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > > > >>>> values */ > > > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > > > >>>> values */ > > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > > >> Right. > > > >> Since the actual VLArb setup is not implemented yet, I didn't see > > > >> this obvious thing. > > > > But it should be implemented. Right? > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > > > Then we will have two QoS managers in parallel, I don't like this too > > much. > > Isn't this needed ? What about LASH ? How is that supported ? Do you mean potential conflict between LASH and QoS in terms of SL/VLs? > I > thought there were extra options or the like to enable the higher > level QoS functions (manager) ? There is already '--qos' option, do you think it is not enough? Sasha From hal.rosenstock at gmail.com Mon Aug 27 17:37:01 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 27 Aug 2007 20:37:01 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070828003133.GS18082@sashak.voltaire.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <20070823120325.GA15042@sashak.voltaire.com> <20070828003133.GS18082@sashak.voltaire.com> Message-ID: On 8/27/07, Sasha Khapyorsky wrote: > On 12:35 Mon 27 Aug , Hal Rosenstock wrote: > > > > > > Thanks. > > > > > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > > > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > > > > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > > > > >>>> values */ > > > > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > > > > >>>> values */ > > > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > > > >> Right. > > > > >> Since the actual VLArb setup is not implemented yet, I didn't see > > > > >> this obvious thing. > > > > > But it should be implemented. Right? > > > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > > > > > Then we will have two QoS managers in parallel, I don't like this too > > > much. > > > > Isn't this needed ? What about LASH ? How is that supported ? > > Do you mean potential conflict between LASH and QoS in terms of SL/VLs? Yes, that and setting VLarb tables. > > I > > thought there were extra options or the like to enable the higher > > level QoS functions (manager) ? > > There is already '--qos' option, do you think it is not enough? Will it continue to mean what it does at OFED 1.2 ? How is the high level QoS distinguished from the existing lower level QoS in terms of starting OpenSM ? -- Hal > Sasha > From sashak at voltaire.com Mon Aug 27 18:12:54 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 04:12:54 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <20070823120325.GA15042@sashak.voltaire.com> <20070828003133.GS18082@sashak.voltaire.com> Message-ID: <20070828011254.GV18082@sashak.voltaire.com> On 20:37 Mon 27 Aug , Hal Rosenstock wrote: > On 8/27/07, Sasha Khapyorsky wrote: > > On 12:35 Mon 27 Aug , Hal Rosenstock wrote: > > > > > > > > Thanks. > > > > > > > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > > > > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > > > > > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > > > > > >>>> values */ > > > > > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > > > > > >>>> values */ > > > > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > > > > >> Right. > > > > > >> Since the actual VLArb setup is not implemented yet, I didn't see > > > > > >> this obvious thing. > > > > > > But it should be implemented. Right? > > > > > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > > > > > > > Then we will have two QoS managers in parallel, I don't like this too > > > > much. > > > > > > Isn't this needed ? What about LASH ? How is that supported ? > > > > Do you mean potential conflict between LASH and QoS in terms of SL/VLs? > > Yes, that and setting VLarb tables. Good point. It is needed to provide at least some level of QoS/LASH co-existence. > > > I > > > thought there were extra options or the like to enable the higher > > > level QoS functions (manager) ? > > > > There is already '--qos' option, do you think it is not enough? > > Will it continue to mean what it does at OFED 1.2 ? Would be nice to not change if not really necessary. > How is the high > level QoS distinguished from the existing lower level QoS in terms of > starting OpenSM ? Do you think it could be useful as separate option? I'm not sure yet, but probably yes. Sasha From hal.rosenstock at gmail.com Mon Aug 27 18:15:07 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Mon, 27 Aug 2007 21:15:07 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070828011254.GV18082@sashak.voltaire.com> References: <46C983BB.4060007@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <20070823120325.GA15042@sashak.voltaire.com> <20070828003133.GS18082@sashak.voltaire.com> <20070828011254.GV18082@sashak.voltaire.com> Message-ID: On 8/27/07, Sasha Khapyorsky wrote: > On 20:37 Mon 27 Aug , Hal Rosenstock wrote: > > On 8/27/07, Sasha Khapyorsky wrote: > > > On 12:35 Mon 27 Aug , Hal Rosenstock wrote: > > > > > > > > > > Thanks. > > > > > > > > > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > > > > > >>>> + cl_list_t across_list; /* list of 'across' group names (strings) */ > > > > > > >>>> + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit > > > > > > >>>> values */ > > > > > > >>>> + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit > > > > > > >>>> values */ > > > > > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > > > > > >> Right. > > > > > > >> Since the actual VLArb setup is not implemented yet, I didn't see > > > > > > >> this obvious thing. > > > > > > > But it should be implemented. Right? > > > > > > > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > > > > > > > > > Then we will have two QoS managers in parallel, I don't like this too > > > > > much. > > > > > > > > Isn't this needed ? What about LASH ? How is that supported ? > > > > > > Do you mean potential conflict between LASH and QoS in terms of SL/VLs? > > > > Yes, that and setting VLarb tables. > > Good point. It is needed to provide at least some level of QoS/LASH > co-existence. > > > > > I > > > > thought there were extra options or the like to enable the higher > > > > level QoS functions (manager) ? > > > > > > There is already '--qos' option, do you think it is not enough? > > > > Will it continue to mean what it does at OFED 1.2 ? > > Would be nice to not change if not really necessary. Agreed. > > How is the high > > level QoS distinguished from the existing lower level QoS in terms of > > starting OpenSM ? > > Do you think it could be useful as separate option? I'm not sure yet, > but probably yes. I think that there needs to be some way to distinguish them. I have something about this in an old email somewhere I'll try to dig out. It has a LASH use and perhaps other backward compatible uses of which we are unaware :-) -- Hal > Sasha > From kliteyn at mellanox.co.il Mon Aug 27 21:38:19 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 28 Aug 2007 07:38:19 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-28:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Thu_Aug_23_14:46:48_2007 [05cae48f68250346dc02b3583044664d6ca6474e] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From tziporet at mellanox.co.il Mon Aug 27 22:58:22 2007 From: tziporet at mellanox.co.il (Tziporet Koren) Date: Tue, 28 Aug 2007 08:58:22 +0300 Subject: [ofa-general] OFED Aug 27 meeting summary on OFED 1.3 development status Message-ID: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> OFED Aug 13 meeting summary on OFED 1.3 development status: ================================================= 1. Decided to delay the feature freeze to Sep 10. * MPI update: * Open MPI: will have some intermediate package for OFED * Mvapich: first OFED package will start with version 0.99 (that was part of 1.2.5). The package of the new release will be ready by end of Sep. * Mvapich2: new release 1.2 will be ready on time for the feature freeze. * QoS in the kernel - need update from Sean * Neteffect new RNIC driver - will be ready in 2 weeks * bonding - will be ready in 2 weeks 2. New install: * Asked all to test and review the new install since Vlad is going for 3 weeks vacation on Sep 10 * Explained the feature to build without installation is not available any more. Reasons for change were that working in the previous mode was against rpm native usage and caused many issues to other packages (like MPI). In addition it caused OFED spec files and package to be not useful for the distros. 3. bugzilla cleanup toward 1.3: * We wish to clean the bugzilla database before we start the 1.3 release phase. * Tziporet will move all open bugs that were opened for OFED 1.2 to OFED 1.3 * Older bugs will be moved to WONTFIX status * Any one that opened a bug is invited to review the bugs and update their status (most important - close fixed bugs) 4. Patches submitted to OFED : * Decided that all kernel patches must submitted for the list for review before merging them to the kernel * Regarding user space: sending patches to the list is recommended but not a must * For critical components (like libibverbs, ucma, osm) sending patches to review is required too * Reminder: Each patch must include short explanation + sign-off Tziporet Koren Software Director Mellanox Technologies mailto: tziporet at mellanox.co.il Tel +972-4-9097200, ext 380 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mst at dev.mellanox.co.il Tue Aug 28 00:12:58 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 28 Aug 2007 10:12:58 +0300 Subject: [ofa-general] Re: [ANNOUNCE] GIT 1.5.2.5 In-Reply-To: <7vr6m4mlen.fsf@gitster.siamese.dyndns.org> References: <7vr6m4mlen.fsf@gitster.siamese.dyndns.org> Message-ID: <20070828071258.GB11856@mellanox.co.il> Junio, could you please Cc general at lists.openfabrics.org on git announcements? At openfabrics.org we are using git to manage the openfabrics enterprise distribution source, so we are interested in hearing about Git releases. Thanks, MST -- MST From lrpwinnersinfo171 at aliceadsl.fr Tue Aug 28 00:16:29 2007 From: lrpwinnersinfo171 at aliceadsl.fr (=?iso-8859-1?Q?LOTTO_Rheinland-Pfalz?=) Date: Tue, 28 Aug 2007 09:16:29 +0200 Subject: [ofa-general] Win Notification-Contact Your Claims Agent. Message-ID: LOTTO Rheinland-Pfalz Ferdinand-Sauerbruch-Strasse 2 DE-56073 Koblenz, Rheinland-Pfalz, Germany. Reference Number: LRP/19-DE/9317 Batch: LRP/06/41 CONGRATULATIONS This Email is to inform you that you emerged a winner of £1,000,000 (One million Euro) on our online draws which was conducted last month. Your email address drew the lucky numbers 3-24-1-10-49-17 which consequently emerged you as one of first fifty (50) lucky winners in this category. No tickets were sold. For further Information about your Winnings,contact our Lottery Fiduciary Agent with the following address, stating clearly your reference and batch numbers. Mr.Wilson Gart Foreign Services Manager, Information and Payment Bureau. London Representative Office. phone: + 44 70457 30177 , + 44 70457 30184 Fax: 00 44 870 974 2884 Email: info at wilsongart50.vossnet.co.uk Yours Earnestly, Mr. Hans-Peter Schossler Promotions Manager(Online Promo Programme) ------------------------ ALICE C'EST ENCORE MIEUX AVEC CANAL+ LE BOUQUET ! --------------- Découvrez vite l'offre exclusive ALICEBOX et CANAL+ LE BOUQUET, en cliquant ici http://alicebox.fr Soumis à conditions. From eitan at mellanox.co.il Tue Aug 28 00:30:57 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 28 Aug 2007 10:30:57 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: References: <46C983BB.4060007@dev.mellanox.co.il><46CAD09C.4030400@dev.mellanox.co.il><20070823064648.GM1397@sashak.voltaire.com><46CD411E.1080809@dev.mellanox.co.il><20070823095615.GP1397@sashak.voltaire.com><46CD6CC1.30403@dev.mellanox.co.il> Message-ID: <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> Hi Hal, As you pointed out, coming up with a VLArb and SL2VL manager that is supporting both routing and QoS is not trivial. But even with a uniform and statically assigned tables (supported by the current options config file) different QoS classes can be provided to different clients by the new QoS policy code. I agree our OFED 1.3 is somewhat limited as it does not solve the VLArb and SL2VL setting issue. But with the time constrains of the development, review and QA this is the most we could do for 1.3 EZ Eitan Zahavi Senior Engineering Director, Software Architect Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Hal Rosenstock > Sent: Monday, August 27, 2007 7:33 PM > To: Yevgeny Kliteynik > Cc: OpenIB > Subject: Re: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy > C & H files > > On 8/23/07, Yevgeny Kliteynik wrote: > > Hi Sasha, > > > > Sasha Khapyorsky wrote: > > > On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_port_group_t_ { > > >>>> + char *name; /* single string (this port > group name) */ > > >>>> + char *use; /* single string (description) */ > > >>>> + cl_list_t port_name_list; /* list of port > names (.../.../...) */ > > >>>> + uint64_t **guid_range_arr; /* array of guid > ranges (pair of 64-bit > > >>>> guids) */ > > >>>> + unsigned guid_range_len; /* num of guid > ranges in the array */ > > >>>> + cl_list_t partition_list; /* list of > partition names */ > > >>>> + boolean_t node_type_ca; > > >>>> + boolean_t node_type_switch; > > >>>> + boolean_t node_type_router; > > >>>> + boolean_t node_type_self; > > >>>> +} osm_qos_port_group_t; > > >>> I see you are using this in "run-time", not just during > the parsing. > > >>> Instead of having all this config features you can just resolve > > >>> port guids in parse time and keep it here in cl_map() > for fast searches. > > >> By saying "config features", do you mean the four boolean flags? > > > > > > No, I mean everything except name and use. > > > > > >> It looks to me that checking the type of node is as fast as it > > >> gets, and it won't hurt to leave these booleans instead of > > >> resolving all the guids. > > > > > > It could be optimization when types are specified, which is not > > > always the case and then you are going to do linear > searches over all lists. > > > > Right, it would be linear scanning of list of partitions. > > > > > And how something like "for each guid in this group" (which is > > > needed for QoS port parameters setup) should be resolved? By > > > matching each guid in the subnet against those lists? > > > > Good point. > > > > >> Moreover, the guids here are stored in range array, > which is IMO > > >> better suited for the policy file syntax, because if a user > > >> specifies something like this "0x0-0x0FFF" in guids, it will be > > >> only one element of the array, which is efficient both > in memory and in serch time. > > > > > > And what should be there if user specifies ports in the group as: > > > guid1, guid2, guid3, etc. ? > > > > The range array is sorted and "shrinked". > > That is, if a user specifies guids as "30,1,2,3-20,15", > eventually you > > would get two elements in the array: 1-20 and 30-30. > > cl_map implemented as a binary tree, right? > > And doing binary search in this kind of array is faster > than searching > > a guid in cl_map of all the guids. > > Worst case - you will get the same performance as in case > of cl_map if > > *all* the guids are "discreet" and can't be groupped in ranges. > > > > Nevertheless, I agree that there's a problem if there are many > > partitions in the list (although I'm not sure it's a practical case) > > > > I'll work on this. > > > > >> (I probably should mention here that the efficient > search in the range > > >> array is not implemented yet, but it would be a simple > binary search - > > >> there's a "todo" comment in the search function right now) > > >> > > >>>> + > > >>>> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > > >>>> +void osm_qos_policy_port_group_destroy(); > > >>> Would be nice to have function prototypes in one place. > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_vlarb_scope_t_ { > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > >>>> + cl_list_t across_list; /* list of 'across' group > names (strings) */ > > >>>> + cl_list_t vlarb_high_list; /* list of num > pairs (n:m,...), 32-bit values > > >>>> */ > > >>>> + cl_list_t vlarb_low_list; /* list of num > pairs (n:m,...), 32-bit values > > >>>> */ > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > >> Right. > > >> Since the actual VLArb setup is not implemented yet, I > didn't see > > >> this obvious thing. > > > > > > But it should be implemented. Right? > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > Without VLArb configuration, where does this come from ? What > is the QoS support then: just SL ? > > -- Hal > > > > > >>>> + uint32_t vl_high_limit; /* single integer */ > > >>>> + boolean_t vl_high_limit_set; > > >>>> +} osm_qos_vlarb_scope_t; > > >>>> + > > >>>> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); > > >>>> +void osm_qos_policy_vlarb_scope_destroy(); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_sl2vl_scope_t_ { > > >>>> + cl_list_t group_list; /* list of strings (port > group names) */ > > >>>> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > >>>> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > >>>> + cl_list_t across_from_list; /* list of strings > (port group names) */ > > >>>> + cl_list_t across_to_list; /* list of strings > (port group names) */ > > >>>> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > > >>>> + boolean_t sl2vl_table_set; > > >>>> +} osm_qos_sl2vl_scope_t; > > >>> This will be used for sl2vl setup? Same as above - why not to > > >>> generate final port guid list just in parse time? > > >>>> + > > >>>> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); > > >>>> +void osm_qos_policy_sl2vl_scope_destroy(); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_level_t_ { > > >>>> + char *use; > > >>>> + char *name; > > >>>> + uint8_t sl; > > >>>> + boolean_t sl_set; > > >>>> + uint8_t mtu_limit; > > >>>> + boolean_t mtu_limit_set; > > >>>> + uint8_t rate_limit; > > >>>> + boolean_t rate_limit_set; > > >>>> + uint8_t pkt_life; > > >>>> + boolean_t pkt_life_set; > > >>>> + uint64_t **path_bits_range_arr; /* array of bit > ranges (real > > >>>> +values are > > >>>> 32bits) */ > > >>>> + unsigned path_bits_range_len; /* num of bit > ranges in the array */ > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > ranges (real values are > > >>>> 16bits) */ > > >>>> + unsigned pkey_range_len; > > >>>> +} osm_qos_level_t; > > >>>> + > > >>>> +osm_qos_level_t *osm_qos_policy_qos_level_create(); > > >>>> +void osm_qos_policy_qos_level_destroy(); > > >>>> + > > >>>> +boolean_t osm_qos_level_has_pkey(IN const > osm_qos_level_t * p_qos_level, > > >>>> + IN ib_net16_t pkey); > > >>>> + > > >>>> +ib_net16_t osm_qos_level_get_shared_pkey(IN const > > >>>> +osm_qos_level_t * > > >>>> p_qos_level, > > >>>> + IN const > osm_physp_t * p_src_physp, > > >>>> + IN const osm_physp_t * > > >>>> + p_dest_physp); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_match_rule_t_ { > > >>>> + char *use; > > >>>> + cl_list_t source_list; /* list of strings */ > > >>>> + cl_list_t source_group_list; /* list of pointers > to relevant port-group > > >>>> */ > > >>>> + cl_list_t destination_list; /* list of strings */ > > >>>> + cl_list_t destination_group_list; /* list of > pointers to relevant > > >>>> port-group */ > > >>> I think you should only keep port guids there (mapped > for fast searches). > > >> Same as above. > > >> I think that checking node type and then guid range array is > > >> essentially faster than checking guid map. > > > > > > _Only_ for case when the group is specified by type, > which is likely > > > will not be typical. > > > > Again, in *worst* case you will get the same performance in > searching > > range array as in searching cl_map (which is a binary tree). > > > > >> You might say, of course, that there can be many port > groups in > > >> the same match rule, but I don't see this as a practical > example. > > > > > > Of course it could (or you must disable multi groups > here, which is > > > not good idea I think and not what was presented in the RFC). > > > > Can you elaborate on this? > > Do you think that having multiple groups is not useful at all? > > > > > Also each group still have this "linear searchable" lists: > > > > > > cl_list_t port_name_list; /* list of port > names (.../.../...) */ > > > > Port names are not implemented yet :) > > There's a "todo" comment in the code. > > This is the only keyword in the parser that is not implemented - I > > mean it's parsed, but other than creating this list the > parser doesn't do anything with it. > > But It definitely won't stay as a pure list. > > > > > uint64_t **guid_range_arr; /* array of guid > ranges (pair of 64-bit > > > > This is not just a list, as I've explained before. > > > > > cl_list_t partition_list; /* list of > partition names */ > > > > I can hardly believe than there will be more than one or two > > partitions in this list. Do you think otherwise? > > > > Anyway, I can extract all the guids and prepare a map. This is > > certainly the easiest implementation. And I still think that on > > average, guid ranges and partition lists are better, but > who knows - > > perhaps someone would want to define a bunch of partitions > in a single > > port group and ruin my average... :) > > > > >>>> + char *qos_level_name; > > >>>> + osm_qos_level_t *p_qos_level; > > >>> Why do you need qos_level_name if you keep the pointer to this > > >>> qos_level struct? > > >> In policy file the match rule might appear before the > QoS levels, > > >> so matching qos level names to the actual qos levels is > done when > > >> the parsing is done. > > >> > > >>>> + uint64_t **service_id_range_arr; /* array of > SID ranges (64-bit values) > > >>>> */ > > >>>> + unsigned service_id_range_len; uint64_t > > >>>> + **qos_class_range_arr; /* array of QoS Class ranges (real > > >>>> values are 16bits) */ > > >>>> + unsigned qos_class_range_len; > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > ranges (real values are > > >>>> 16bits) */ > > >>>> + unsigned pkey_range_len; > > >>>> +} osm_qos_match_rule_t; > > >>>> + > > >>>> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > > >>>> +void osm_qos_policy_match_rule_destroy(); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +typedef struct osm_qos_policy_t_ { > > >>>> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > > >>>> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > > >>>> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > > >>>> + cl_list_t qos_levels; /* list of osm_qos_level_t */ > > >>>> + cl_list_t qos_match_rules; /* list of > osm_qos_match_rule_t */ > > >>>> + osm_qos_level_t *p_default_qos_level; /* default > QoS level */ > > >>>> +} osm_qos_policy_t; > > >>>> + > > >>>> +void osm_qos_policy_create(); > > >>>> +void osm_qos_policy_destroy(); > > >>>> +int osm_qos_policy_validate(); > > >>>> + > > >>>> +void osm_qos_policy_get_qos_level_by_pr(IN const > osm_pr_rcv_t * p_rcv, > > >>>> + IN const > ib_path_rec_t * p_pr, > > >>>> + IN const > osm_physp_t * p_src_physp, > > >>>> + IN const > osm_physp_t * p_dest_physp, > > >>>> + IN ib_net64_t comp_mask, > > >>>> + OUT osm_qos_level_t ** > > >>>> +pp_qos_level); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const > > >>>> +char > > >>>> *policy_file); > > >>>> + > > >>>> +/***************************************************/ > > >>>> + > > >>>> +#endif /* ifndef > OSM_QOS_POLICY_H */ > > >>>> diff --git a/opensm/opensm/osm_qos_policy.c > > >>>> b/opensm/opensm/osm_qos_policy.c new file mode 100644 index > > >>>> 0000000..bc2aa68 > > >>>> --- /dev/null > > >>>> +++ b/opensm/opensm/osm_qos_policy.c > > >>>> @@ -0,0 +1,901 @@ > > >>>> +/* > > >>>> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > > >>>> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. > All rights > > >>>> reserved. > > >>>> + * Copyright (c) 1996-2003 Intel Corporation. All > rights reserved. > > >>>> + * > > >>>> + * This software is available to you under a choice of one of > > >>>> + two > > >>>> + * licenses. You may choose to be licensed under the > terms of > > >>>> + the GNU > > >>>> + * General Public License (GPL) Version 2, available from the > > >>>> + file > > >>>> + * COPYING in the main directory of this source tree, or the > > >>>> + * OpenIB.org BSD license below: > > >>>> + * > > >>>> + * Redistribution and use in source and binary > forms, with or > > >>>> + * without modification, are permitted provided > that the following > > >>>> + * conditions are met: > > >>>> + * > > >>>> + * - Redistributions of source code must retain the above > > >>>> + * copyright notice, this list of conditions > and the following > > >>>> + * disclaimer. > > >>>> + * > > >>>> + * - Redistributions in binary form must > reproduce the above > > >>>> + * copyright notice, this list of conditions > and the following > > >>>> + * disclaimer in the documentation and/or > other materials > > >>>> + * provided with the distribution. > > >>>> + * > > >>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY > > >>>> + KIND, > > >>>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE > > >>>> + WARRANTIES OF > > >>>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > >>>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR > COPYRIGHT > > >>>> + HOLDERS > > >>>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > LIABILITY, WHETHER > > >>>> + IN AN > > >>>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > FROM, OUT OF > > >>>> + OR IN > > >>>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > DEALINGS IN > > >>>> + THE > > >>>> + * SOFTWARE. > > >>>> + * > > >>>> + */ > > >>>> + > > >>>> +/* > > >>>> + * Abstract: > > >>>> + * OSM QoS Policy functions. > > >>>> + * > > >>>> + * Environment: > > >>>> + * Linux User Mode > > >>>> + * > > >>>> + * Author: > > >>>> + * Yevgeny Kliteynik, Mellanox > > >>>> + */ > > >>>> + > > >>>> +#include #include > > >>>> + #include > > >>>> + > > >>>> +extern void yyerror(char *s); > > >>>> +osm_log_t *p_qos_parser_osm_log = NULL; osm_qos_policy_t > > >>>> +*p_qos_policy = NULL; > > >>> Please try to avoid globals - keep it as part of > osm_opensm_t or > > >>> osm_subn_t structures. > > >> I thought about it, but didn't want to "condaminate" the > > >> osm_opensm_t or osm_subn_t structures untill the QoS > functionality is ready. > > > > > > How globals are better in this sense? > > > > They're not :) > > The difference is that less files are modified. > > But I agree that the QoS policy should be part of > osm_opensm_t or osm_subn_t. > > > > -- Yevgeny > > > > > Sasha > > > > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From 998sherm27 at jcpportraits.com Tue Aug 28 02:54:48 2007 From: 998sherm27 at jcpportraits.com (Eugene Ware) Date: Tue, 28 Aug 2007 08:54:48 -0100 Subject: [ofa-general] Can we be friends? Message-ID: <075619694.77496502893116@jcpportraits.com> Hello! I am bored this afternoon. I am nice girl that would like to chat with you. Email me at ixdk at BestOnset.info only, because I am writing not from my personal email. If you would like to see my pictures. From sashak at voltaire.com Tue Aug 28 02:36:35 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 12:36:35 +0300 Subject: [ofa-general] Re: [PATCH] opensm/complib: remove osm_log usage from complib's API In-Reply-To: <20070826065931.GF18082@sashak.voltaire.com> References: <20070826065622.GD18082@sashak.voltaire.com> <20070826065717.GE18082@sashak.voltaire.com> <20070826065931.GF18082@sashak.voltaire.com> Message-ID: <20070828093635.GY18082@sashak.voltaire.com> On 09:59 Sun 26 Aug , Sasha Khapyorsky wrote: > > Remove pointer to osm_log from cl_event_wheel_init() parameters list. > This changes API, so complib version is bumped. > > Signed-off-by: Sasha Khapyorsky As suggested by Hal. Since there is API change I will bump complib version to 2.3.0. Will do the same with libosmcomp.ver - to unify the version numbers. Sasha > --- > opensm/complib/cl_event_wheel.c | 12 +++++------- > opensm/complib/configure.in | 2 +- > opensm/complib/libosmcomp.map | 2 +- > opensm/complib/libosmcomp.ver | 2 +- > opensm/include/complib/cl_event_wheel.h | 11 +---------- > opensm/opensm/osm_trap_rcv.c | 7 ++++++- > 6 files changed, 15 insertions(+), 21 deletions(-) > > diff --git a/opensm/complib/cl_event_wheel.c b/opensm/complib/cl_event_wheel.c > index f40a15d..f83c47e 100644 > --- a/opensm/complib/cl_event_wheel.c > +++ b/opensm/complib/cl_event_wheel.c > @@ -196,8 +196,7 @@ void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel) > } > > cl_status_t > -cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, > - IN osm_log_t * p_log) > +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel) > { > cl_status_t cl_status = CL_SUCCESS; > > @@ -218,14 +217,13 @@ cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, > > cl_status_t > cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, > - IN osm_log_t * p_log, IN cl_spinlock_t * p_external_lock) > + IN cl_spinlock_t * p_external_lock) > { > cl_status_t cl_status; > > - cl_status = cl_event_wheel_init(p_event_wheel, NULL); > - if (CL_SUCCESS != cl_status) { > + cl_status = cl_event_wheel_init(p_event_wheel); > + if (CL_SUCCESS != cl_status) > return cl_status; > - } > > p_event_wheel->p_external_lock = p_external_lock; > return cl_status; > @@ -527,7 +525,7 @@ int main() > cl_event_wheel_construct(&event_wheel); > > /* init */ > - cl_event_wheel_init(&event_wheel, NULL); > + cl_event_wheel_init(&event_wheel); > > /* Start Playing */ > cl_event_wheel_reg(&event_wheel, 1, /* key */ > diff --git a/opensm/complib/configure.in b/opensm/complib/configure.in > index 2e2bd59..6abb264 100644 > --- a/opensm/complib/configure.in > +++ b/opensm/complib/configure.in > @@ -1,7 +1,7 @@ > dnl Process this file with autoconf to produce a configure script. > > AC_PREREQ(2.57) > -AC_INIT(complib, 2.2.1, general at lists.openfabrics.org) > +AC_INIT(complib, 2.2.2, general at lists.openfabrics.org) > AC_CONFIG_SRCDIR([cl_spinlock.c]) > AC_CONFIG_AUX_DIR(config) > AM_CONFIG_HEADER(config.h) > diff --git a/opensm/complib/libosmcomp.map b/opensm/complib/libosmcomp.map > index 2a1ba63..6361554 100644 > --- a/opensm/complib/libosmcomp.map > +++ b/opensm/complib/libosmcomp.map > @@ -1,4 +1,4 @@ > -OSMCOMP_2.0 { > +OSMCOMP_2.1 { > global: > complib_init; > complib_exit; > diff --git a/opensm/complib/libosmcomp.ver b/opensm/complib/libosmcomp.ver > index fab9776..ab7e34e 100644 > --- a/opensm/complib/libosmcomp.ver > +++ b/opensm/complib/libosmcomp.ver > @@ -6,4 +6,4 @@ > # API_REV - advance on any added API > # RUNNING_REV - advance any change to the vendor files > # AGE - number of backward versions the API still supports > -LIBVERSION=2:0:0 > +LIBVERSION=2:1:0 > diff --git a/opensm/include/complib/cl_event_wheel.h b/opensm/include/complib/cl_event_wheel.h > index ab34bc5..ba466c6 100644 > --- a/opensm/include/complib/cl_event_wheel.h > +++ b/opensm/include/complib/cl_event_wheel.h > @@ -51,7 +51,6 @@ > #include > #include > #include > -#include > > #ifdef __cplusplus > # define BEGIN_C_DECLS extern "C" { > @@ -261,17 +260,13 @@ void cl_event_wheel_construct(IN cl_event_wheel_t * const p_event_wheel); > * SYNOPSIS > */ > cl_status_t > -cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, > - IN osm_log_t * p_log); > +cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel); > > /* > * PARAMETERS > * p_event_wheel > * [in] Pointer to a Event_Wheel. > * > -* p_log > -* [in] Pointer to opensm log object to be used for logging > -* > * RETURN VALUE > * CL_SUCCESS if the operation is successful. > * > @@ -291,7 +286,6 @@ cl_event_wheel_init(IN cl_event_wheel_t * const p_event_wheel, > */ > cl_status_t > cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, > - IN osm_log_t * p_log, > IN cl_spinlock_t * p_external_lock); > > /* > @@ -299,9 +293,6 @@ cl_event_wheel_init_ex(IN cl_event_wheel_t * const p_event_wheel, > * p_event_wheel > * [in] Pointer to a Event_Wheel. > * > -* p_log > -* [in] Pointer to opensm log object to be used for logging > -* > * p_external_lock > * [in] Reference to external spinlock to guard internal structures > * if the event wheel is part of a larger object protected by its own lock > diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c > index 86aac94..babc625 100644 > --- a/opensm/opensm/osm_trap_rcv.c > +++ b/opensm/opensm/osm_trap_rcv.c > @@ -203,7 +203,12 @@ osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, > p_rcv->p_resp = p_resp; > p_rcv->p_state_mgr = p_state_mgr; > > - cl_event_wheel_init(&p_rcv->trap_aging_tracker, p_log); > + if(cl_event_wheel_init(&p_rcv->trap_aging_tracker)) { > + osm_log(p_log, OSM_LOG_ERROR, > + "osm_trap_rcv_init: ERR 3800: " > + "Failed to initialize cl_event_wheel\n"); > + status = IB_NOT_DONE; > + } > > OSM_LOG_EXIT(p_rcv->p_log); > return (status); > -- > 1.5.3.rc2.38.g11308 > From 2020miller at cox.net Tue Aug 28 02:37:55 2007 From: 2020miller at cox.net (Madam Susana Kadrlik Cole) Date: Tue, 28 Aug 2007 2:37:55 -0700 Subject: [ofa-general] From Madam Susana Kadrlik Cole Here writes Madam Susan Cole, suffering from cancerous ailment. When my late husband was alive he left the sum of 5Million (Five Million Pound Sterling) which were derived from his vast estates and investment in capital market with finance Company here in Europe. Recently, my doctor told me that I have limited days to live due to the cancerous problems I am suffering from, So, I decided to contact you due to time limit. Though what bothers me most is the stroke that I have in addition to the cancer. With this hard reality that has befallen my family, and me I have decided to donate this fund to you and want you to use this gift which comes from my husbands effort to fund the upkeep of widows, widowers, orphans, destitute, the down-trodden, physically challenged children, barren-women and persons who prove to be genuinely handicapped financially. It is often said that blessed is the hand that giveth. I took this decision because I do not have any child that will inherit this money and my husband relatives are bourgeois and very wealthy persons and I do not want my husband hard earned money to be misused or invested into ill perceived ventures. I do not want a situation where this money will be used in an ungodly manner, hence the reason for taking this bold decision. I am not afraid of death hence I know where I am going. I know that I am going to be with the Almighty when I eventually pass on. The Almighty will fight my case and I shall hold my peace. I do not need any telephone communication in this regard due to my deteriorating health and because of the presence of my husband relatives around me. I do not want them to know about this development. I want you to stand as the new beneficiary to the funds. As soon as I receive your reply I shall give you the contact of the finance company. Please send all emails to my confidential emails below: Email: susancolesusancole500@hotmail.com Endeavor to send me your names, address, telephone and fax number to enable contact you with more details/all the relevant documents by email/ fax. Hope to hear from you soon. Best regards Madam Susana Kadrlik Cole Email: susancolesusancole500@hotmail.com Message-ID: <22591206.1188293875470.JavaMail.root@fed1wml24.mgt.cox.net> From vlad at lists.openfabrics.org Tue Aug 28 02:48:19 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Tue, 28 Aug 2007 02:48:19 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070828-0200 daily build status Message-ID: <20070828094819.CD512E60836@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.19 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.14 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.22 Passed on ia64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070828-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From sashak at voltaire.com Tue Aug 28 03:30:44 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 13:30:44 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> Message-ID: <20070828103044.GZ18082@sashak.voltaire.com> Hi Eitan, On 10:30 Tue 28 Aug , Eitan Zahavi wrote: > > As you pointed out, coming up with a VLArb and SL2VL manager that is > supporting both routing and QoS is not trivial. > But even with a uniform and statically assigned tables (supported by the > current options config file) different QoS classes can be provided to > different clients by the new QoS policy code. It leaves an user in the state when using QoS requires manual configuration of two different in capabilities and potentially conflicting QoS managers. > I agree our OFED 1.3 is somewhat limited as it does not solve the VLArb > and SL2VL setting issue. > But with the time constrains of the development, review and QA this is > the most we could do for 1.3 This feature was mentioned as requirement couple of times even at beginning of 1.2 days. There are extra 7 days now up to feature freeze. Also QoS is optional, and in part of VLArb and SL2VL setup the risk of affecting other OpenSM components is relatively low. OTOH it is very important to have full functional and integrated QoS. So basically I'm fine to review such patches even after feature freeze date (not in the last RC day of course :)). Sasha From sashak at voltaire.com Tue Aug 28 04:34:24 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 14:34:24 +0300 Subject: [ofa-general] [PATCH] opensm: serialize osm_state_mgr_process() Message-ID: <20070828113424.GB18082@sashak.voltaire.com> This serializes osm_state_mgr_process() - this is running only from SM thread. The rest OpenSM components will schedule osm_state_mgr_process() runs with new function osm_sm_signal(), which only updates the mask of required for processing event (signals) and wake SM thread up. This prevents MAD processor dispatchers blocking. Also "big state lock" (osm_state_mgr.state_lock) is not needed anymore and removed. Signed-off-by: Sasha Khapyorsky --- opensm/include/Makefile.am | 1 - opensm/include/opensm/osm_msgdef.h | 15 -- opensm/include/opensm/osm_node_info_rcv.h | 9 - opensm/include/opensm/osm_port_info_rcv.h | 9 - opensm/include/opensm/osm_sm.h | 37 ++++- opensm/include/opensm/osm_sm_state_mgr.h | 11 +-- opensm/include/opensm/osm_sminfo_rcv.h | 9 - opensm/include/opensm/osm_state_mgr.h | 4 - opensm/include/opensm/osm_state_mgr_ctrl.h | 224 --------------------------- opensm/include/opensm/osm_sw_info_rcv.h | 9 - opensm/include/opensm/osm_sweep_fail_ctrl.h | 16 +- opensm/include/opensm/osm_trap_rcv.h | 9 - opensm/opensm/Makefile.am | 2 +- opensm/opensm/osm_helper.c | 1 - opensm/opensm/osm_node_info_rcv.c | 12 +- opensm/opensm/osm_port_info_rcv.c | 7 +- opensm/opensm/osm_sm.c | 122 ++++++++------- opensm/opensm/osm_sm_mad_ctrl.c | 27 +--- opensm/opensm/osm_sm_state_mgr.c | 17 +-- opensm/opensm/osm_sminfo_rcv.c | 14 +- opensm/opensm/osm_state_mgr.c | 24 +--- opensm/opensm/osm_state_mgr_ctrl.c | 115 -------------- opensm/opensm/osm_sw_info_rcv.c | 7 +- opensm/opensm/osm_sweep_fail_ctrl.c | 7 +- opensm/opensm/osm_trap_rcv.c | 4 +- 25 files changed, 140 insertions(+), 572 deletions(-) delete mode 100644 opensm/include/opensm/osm_state_mgr_ctrl.h delete mode 100644 opensm/opensm/osm_state_mgr_ctrl.c diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..8b85bd2 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -87,7 +87,6 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_vl15intf.h \ $(srcdir)/opensm/osm_drop_mgr.h \ $(srcdir)/opensm/osm_port_info_rcv.h \ - $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ $(srcdir)/complib/cl_thread_osd.h \ diff --git a/opensm/include/opensm/osm_msgdef.h b/opensm/include/opensm/osm_msgdef.h index c4e1714..0dee6b9 100644 --- a/opensm/include/opensm/osm_msgdef.h +++ b/opensm/include/opensm/osm_msgdef.h @@ -130,27 +130,12 @@ BEGIN_C_DECLS * * SOURCE ***********/ -/****d* OpenSM: Dispatcher Messages/OSM_MSG_NO_SMPS_OUTSTANDING -* NAME -* OSM_MSG_NO_SMPS_OUTSTANDING -* -* DESCRIPTION -* Message indicating that there are no outstanding SMPs on the subnet. -* -* NOTES -* Sent by: osm_mad_ctrl_t -* Received by: osm_state_mgr_ctrl_t -* Delivery notice: no -* -* SOURCE -***********/ enum { OSM_MSG_NONE = 0, OSM_MSG_MAD_NODE_INFO, OSM_MSG_MAD_PORT_INFO, OSM_MSG_MAD_SWITCH_INFO, OSM_MSG_MAD_NODE_DESC, - OSM_MSG_NO_SMPS_OUTSTANDING, OSM_MSG_MAD_NODE_RECORD, OSM_MSG_MAD_PORTINFO_RECORD, OSM_MSG_MAD_SERVICE_RECORD, diff --git a/opensm/include/opensm/osm_node_info_rcv.h b/opensm/include/opensm/osm_node_info_rcv.h index 31b89ac..32e3df0 100644 --- a/opensm/include/opensm/osm_node_info_rcv.h +++ b/opensm/include/opensm/osm_node_info_rcv.h @@ -56,7 +56,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -100,7 +99,6 @@ typedef struct _osm_ni_rcv { osm_subn_t *p_subn; osm_req_t *p_gen_req; osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; cl_plock_t *p_lock; } osm_ni_rcv_t; /* @@ -114,9 +112,6 @@ typedef struct _osm_ni_rcv { * p_log * Pointer to the log object. * -* p_state_mgr -* Pointer to the State Manager object. -* * p_lock * Pointer to the serializing lock. * @@ -199,7 +194,6 @@ ib_api_status_t osm_ni_rcv_init(IN osm_ni_rcv_t * const p_ctrl, IN osm_req_t * const p_req, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock); /* * PARAMETERS @@ -215,9 +209,6 @@ ib_api_status_t osm_ni_rcv_init(IN osm_ni_rcv_t * const p_ctrl, * p_log * [in] Pointer to the log object. * -* p_state_mgr -* [in] Pointer to the State Manager object. -* * p_lock * [in] Pointer to the OpenSM serializing lock. * diff --git a/opensm/include/opensm/osm_port_info_rcv.h b/opensm/include/opensm/osm_port_info_rcv.h index 9f3ab22..c4c7d96 100644 --- a/opensm/include/opensm/osm_port_info_rcv.h +++ b/opensm/include/opensm/osm_port_info_rcv.h @@ -56,7 +56,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -100,7 +99,6 @@ typedef struct _osm_pi_rcv { osm_subn_t *p_subn; osm_req_t *p_req; osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; cl_plock_t *p_lock; } osm_pi_rcv_t; /* @@ -114,9 +112,6 @@ typedef struct _osm_pi_rcv { * p_log * Pointer to the log object. * -* p_state_mgr -* Pointer to the State Manager object. -* * p_lock * Pointer to the serializing lock. * @@ -198,7 +193,6 @@ ib_api_status_t osm_pi_rcv_init(IN osm_pi_rcv_t * const p_ctrl, IN osm_req_t * const p_req, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock); /* * PARAMETERS @@ -214,9 +208,6 @@ ib_api_status_t osm_pi_rcv_init(IN osm_pi_rcv_t * const p_ctrl, * p_log * [in] Pointer to the log object. * -* p_state_mgr -* [in] Pointer to the state manager object. -* * p_lock * [in] Pointer to the OpenSM serializing lock. * diff --git a/opensm/include/opensm/osm_sm.h b/opensm/include/opensm/osm_sm.h index 4811d2a..341ec5a 100644 --- a/opensm/include/opensm/osm_sm.h +++ b/opensm/include/opensm/osm_sm.h @@ -52,11 +52,11 @@ #include #include #include -#include #include +#include +#include #include #include -#include #include #include #include @@ -66,7 +66,6 @@ #include #include #include -#include #include #include #include @@ -83,6 +82,7 @@ #include #include #include +#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -123,9 +123,11 @@ BEGIN_C_DECLS * * SYNOPSIS */ -typedef struct _osm_sm { +typedef struct osm_sm { osm_thread_state_t thread_state; - cl_event_t signal; + unsigned signal_mask; + cl_spinlock_t signal_lock; + cl_event_t signal_event; cl_event_t subnet_up_event; cl_thread_t sweeper; osm_subn_t *p_subn; @@ -144,7 +146,6 @@ typedef struct _osm_sm { osm_nd_rcv_t nd_rcv; osm_sm_mad_ctrl_t mad_ctrl; osm_si_rcv_t si_rcv; - osm_state_mgr_ctrl_t state_mgr_ctrl; osm_lid_mgr_t lid_mgr; osm_ucast_mgr_t ucast_mgr; osm_link_mgr_t link_mgr; @@ -367,6 +368,30 @@ osm_sm_init(IN osm_sm_t * const p_sm, * SM object, osm_sm_construct, osm_sm_destroy *********/ +/****f* OpenSM: SM/osm_sm_signal +* NAME +* osm_sm_signal +* +* DESCRIPTION +* Signal event to SM +* +* SYNOPSIS +*/ +void osm_sm_signal(IN osm_sm_t * const p_sm, osm_signal_t signal); +/* +* PARAMETERS +* p_sm +* [in] Pointer to an osm_sm_t object. +* +* signal +* [in] sm signal number. +* +* NOTES +* +* SEE ALSO +* SM object +*********/ + /****f* OpenSM: SM/osm_sm_sweep * NAME * osm_sm_sweep diff --git a/opensm/include/opensm/osm_sm_state_mgr.h b/opensm/include/opensm/osm_sm_state_mgr.h index ab2aaff..b8ee174 100644 --- a/opensm/include/opensm/osm_sm_state_mgr.h +++ b/opensm/include/opensm/osm_sm_state_mgr.h @@ -104,7 +104,6 @@ typedef struct _osm_sm_state_mgr { cl_timer_t polling_timer; uint32_t retry_number; ib_net64_t master_guid; - osm_state_mgr_t *p_state_mgr; osm_subn_t *p_subn; osm_req_t *p_req; osm_log_t *p_log; @@ -126,9 +125,6 @@ typedef struct _osm_sm_state_mgr { * master_guid * Port GUID of master SM. * -* p_state_mgr -* Pointer to the state manager object. -* * p_subn * Pointer to the Subnet object for this subnet. * @@ -219,7 +215,6 @@ void osm_sm_state_mgr_destroy(IN osm_sm_state_mgr_t * const p_sm_mgr); */ ib_api_status_t osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, - IN osm_state_mgr_t * const p_state_mgr, IN osm_subn_t * const p_subn, IN osm_req_t * const p_req, IN osm_log_t * const p_log); /* @@ -227,14 +222,10 @@ osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, * p_sm_mgr * [in] Pointer to an osm_sm_state_mgr_t object to initialize. * -* -* p_state_mgr -* [in] Pointer to the State Manager object. -* * p_subn * [in] Pointer to the Subnet object for this subnet. * -* p_req +* p_req * [in] Pointer to an osm_req_t object. * * p_log diff --git a/opensm/include/opensm/osm_sminfo_rcv.h b/opensm/include/opensm/osm_sminfo_rcv.h index 1cbf2e6..9bbe7f8 100644 --- a/opensm/include/opensm/osm_sminfo_rcv.h +++ b/opensm/include/opensm/osm_sminfo_rcv.h @@ -56,7 +56,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -101,7 +100,6 @@ typedef struct _osm_sminfo_rcv { osm_stats_t *p_stats; osm_log_t *p_log; osm_resp_t *p_resp; - osm_state_mgr_t *p_state_mgr; struct _osm_sm_state_mgr *p_sm_state_mgr; cl_plock_t *p_lock; } osm_sminfo_rcv_t; @@ -119,9 +117,6 @@ typedef struct _osm_sminfo_rcv { * p_resp * Pointer to the generic MAD responder object. * -* p_state_mgr -* Pointer to the State Manager object. -* * p_sm_state_mgr * Pointer to the SM State Manager object. * @@ -207,7 +202,6 @@ ib_api_status_t osm_sminfo_rcv_init(IN osm_sminfo_rcv_t * const p_rcv, IN osm_stats_t * const p_stats, IN osm_resp_t * const p_resp, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN struct _osm_sm_state_mgr *const p_sm_state_mgr, IN cl_plock_t * const p_lock); @@ -228,9 +222,6 @@ ib_api_status_t osm_sminfo_rcv_init(IN osm_sminfo_rcv_t * const p_rcv, * p_log * [in] Pointer to the log object. * -* p_state_mgr -* [in] Pointer to the State Manager object. -* * p_sm_state_mgr * [in] Pointer to the SM State Manager object. * diff --git a/opensm/include/opensm/osm_state_mgr.h b/opensm/include/opensm/osm_state_mgr.h index 5b45a91..dada097 100644 --- a/opensm/include/opensm/osm_state_mgr.h +++ b/opensm/include/opensm/osm_state_mgr.h @@ -109,7 +109,6 @@ typedef struct _osm_state_mgr { osm_stats_t *p_stats; struct _osm_sm_state_mgr *p_sm_state_mgr; const osm_sm_mad_ctrl_t *p_mad_ctrl; - cl_spinlock_t state_lock; cl_spinlock_t idle_lock; cl_qlist_t idle_time_list; cl_plock_t *p_lock; @@ -151,9 +150,6 @@ typedef struct _osm_state_mgr { * p_mad_ctrl * Pointer to the SM's MAD Controller object. * -* state_lock -* Spinlock guarding the state and processes. -* * p_lock * lock guarding the subnet object. * diff --git a/opensm/include/opensm/osm_state_mgr_ctrl.h b/opensm/include/opensm/osm_state_mgr_ctrl.h deleted file mode 100644 index 4751335..0000000 --- a/opensm/include/opensm/osm_state_mgr_ctrl.h +++ /dev/null @@ -1,224 +0,0 @@ -/* - * Copyright (c) 2004, 2005 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. - * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. - * - * This software is available to you under a choice of one of two - * licenses. You may choose to be licensed under the terms of the GNU - * General Public License (GPL) Version 2, available from the file - * COPYING in the main directory of this source tree, or the - * OpenIB.org BSD license below: - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * - Redistributions of source code must retain the above - * copyright notice, this list of conditions and the following - * disclaimer. - * - * - Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE - * SOFTWARE. - * - */ - -/* - * Abstract: - * Declaration of osm_state_mgr_ctrl_t. - * This object represents a controller that receives the - * State indication after a subnet sweep. - * This object is part of the OpenSM family of objects. - * - * Environment: - * Linux User Mode - * - * $Revision: 1.4 $ - */ - -#ifndef _OSM_STATE_MGR_CTRL_H_ -#define _OSM_STATE_MGR_CTRL_H_ - -#include -#include -#include -#include - -#ifdef __cplusplus -# define BEGIN_C_DECLS extern "C" { -# define END_C_DECLS } -#else /* !__cplusplus */ -# define BEGIN_C_DECLS -# define END_C_DECLS -#endif /* __cplusplus */ - -BEGIN_C_DECLS -/****h* OpenSM/State Manager Controller -* NAME -* State Manager Controller -* -* DESCRIPTION -* The State Manager Controller object encapsulates the information -* needed to pass the dispatcher message from the dispatcher -* to the State Manager. -* -* The State Manager Controller object is thread safe. -* -* This object should be treated as opaque and should be -* manipulated only through the provided functions. -* -* AUTHOR -* Steve King, Intel -* -*********/ -/****s* OpenSM: State Manager Controller/osm_state_mgr_ctrl_t -* NAME -* osm_state_mgr_ctrl_t -* -* DESCRIPTION -* State Manager Controller structure. -* -* This object should be treated as opaque and should -* be manipulated only through the provided functions. -* -* SYNOPSIS -*/ -typedef struct _osm_state_mgr_ctrl { - osm_state_mgr_t *p_mgr; - osm_log_t *p_log; - cl_dispatcher_t *p_disp; - cl_disp_reg_handle_t h_disp; - -} osm_state_mgr_ctrl_t; -/* -* FIELDS -* p_mgr -* Pointer to the State Manager object. -* -* p_log -* Pointer to the log object. -* -* p_disp -* Pointer to the Dispatcher. -* -* h_disp -* Handle returned from dispatcher registration. -* -* SEE ALSO -* State Manager Controller object -*********/ - -/****f* OpenSM: State Manager Controller/osm_state_mgr_ctrl_construct -* NAME -* osm_state_mgr_ctrl_construct -* -* DESCRIPTION -* This function constructs a State Manager Controller object. -* -* SYNOPSIS -*/ -void osm_state_mgr_ctrl_construct(IN osm_state_mgr_ctrl_t * const p_ctrl); -/* -* PARAMETERS -* p_ctrl -* [in] Pointer to a State Manager Controller -* object to construct. -* -* RETURN VALUE -* This function does not return a value. -* -* NOTES -* Allows calling osm_state_mgr_ctrl_init, and osm_state_mgr_ctrl_destroy. -* -* Calling osm_state_mgr_ctrl_construct is a prerequisite to calling any -* other method except osm_state_mgr_ctrl_init. -* -* SEE ALSO -* State Manager Controller object, osm_state_mgr_ctrl_init, -* osm_state_mgr_ctrl_destroy -*********/ - -/****f* OpenSM: State Manager Controller/osm_state_mgr_ctrl_destroy -* NAME -* osm_state_mgr_ctrl_destroy -* -* DESCRIPTION -* The osm_state_mgr_ctrl_destroy function destroys the object, releasing -* all resources. -* -* SYNOPSIS -*/ -void osm_state_mgr_ctrl_destroy(IN osm_state_mgr_ctrl_t * const p_ctrl); -/* -* PARAMETERS -* p_ctrl -* [in] Pointer to the object to destroy. -* -* RETURN VALUE -* This function does not return a value. -* -* NOTES -* Performs any necessary cleanup of the specified -* State Manager Controller object. -* Further operations should not be attempted on the destroyed object. -* This function should only be called after a call to -* osm_state_mgr_ctrl_construct or osm_state_mgr_ctrl_init. -* -* SEE ALSO -* State Manager Controller object, osm_state_mgr_ctrl_construct, -* osm_state_mgr_ctrl_init -*********/ - -/****f* OpenSM: State Manager Controller/osm_state_mgr_ctrl_init -* NAME -* osm_state_mgr_ctrl_init -* -* DESCRIPTION -* The osm_state_mgr_ctrl_init function initializes a -* State Manager Controller object for use. -* -* SYNOPSIS -*/ -ib_api_status_t -osm_state_mgr_ctrl_init(IN osm_state_mgr_ctrl_t * const p_ctrl, - IN osm_state_mgr_t * const p_mgr, - IN osm_log_t * const p_log, - IN cl_dispatcher_t * const p_disp); -/* -* PARAMETERS -* p_ctrl -* [in] Pointer to an osm_state_mgr_ctrl_t object to initialize. -* -* p_mgr -* [in] Pointer to an osm_state_mgr_t object. -* -* p_log -* [in] Pointer to the log object. -* -* p_disp -* [in] Pointer to the OpenSM central Dispatcher. -* -* RETURN VALUES -* IB_SUCCESS if the State Manager Controller object -* was initialized successfully. -* -* NOTES -* Allows calling other State Manager Controller methods. -* -* SEE ALSO -* State Manager Controller object, osm_state_mgr_ctrl_construct, -* osm_state_mgr_ctrl_destroy -*********/ - -END_C_DECLS -#endif /* OSM_STATE_MGR_CTRL_H_ */ diff --git a/opensm/include/opensm/osm_sw_info_rcv.h b/opensm/include/opensm/osm_sw_info_rcv.h index 724d0d3..2e7cbfa 100644 --- a/opensm/include/opensm/osm_sw_info_rcv.h +++ b/opensm/include/opensm/osm_sw_info_rcv.h @@ -54,7 +54,6 @@ #include #include #include -#include #include #ifdef __cplusplus @@ -99,7 +98,6 @@ typedef struct _osm_si_rcv { osm_subn_t *p_subn; osm_log_t *p_log; osm_req_t *p_req; - osm_state_mgr_t *p_state_mgr; cl_plock_t *p_lock; } osm_si_rcv_t; /* @@ -113,9 +111,6 @@ typedef struct _osm_si_rcv { * p_req * Pointer to the Request object. * -* p_state_mgr -* Pointer to the State Manager object. -* * p_lock * Pointer to the serializing lock. * @@ -198,7 +193,6 @@ ib_api_status_t osm_si_rcv_init(IN osm_si_rcv_t * const p_ctrl, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, IN osm_req_t * const p_req, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock); /* * PARAMETERS @@ -214,9 +208,6 @@ ib_api_status_t osm_si_rcv_init(IN osm_si_rcv_t * const p_ctrl, * p_req * [in] Pointer to an osm_req_t object. * -* p_state_mgr -* [in] Pointer to the State Manager object. -* * p_lock * [in] Pointer to the OpenSM serializing lock. * diff --git a/opensm/include/opensm/osm_sweep_fail_ctrl.h b/opensm/include/opensm/osm_sweep_fail_ctrl.h index 579ef58..2fca6eb 100644 --- a/opensm/include/opensm/osm_sweep_fail_ctrl.h +++ b/opensm/include/opensm/osm_sweep_fail_ctrl.h @@ -52,7 +52,6 @@ #include #include #include -#include #include #ifdef __cplusplus @@ -82,6 +81,7 @@ BEGIN_C_DECLS * Steve King, Intel * *********/ +struct osm_sm; /****s* OpenSM: Sweep Fail Controller/osm_sweep_fail_ctrl_t * NAME * osm_sweep_fail_ctrl_t @@ -95,20 +95,20 @@ BEGIN_C_DECLS * SYNOPSIS */ typedef struct _osm_sweep_fail_ctrl { + struct osm_sm *sm; osm_log_t *p_log; - osm_state_mgr_t *p_state_mgr; cl_dispatcher_t *p_disp; cl_disp_reg_handle_t h_disp; } osm_sweep_fail_ctrl_t; /* * FIELDS +* sm +* Pointer to the sm object. +* * p_log * Pointer to the log object. * -* p_sate_mgr -* Pointer to the state manager object. -* * p_disp * Pointer to the Dispatcher. * @@ -194,7 +194,7 @@ void osm_sweep_fail_ctrl_destroy(IN osm_sweep_fail_ctrl_t * const p_ctrl); ib_api_status_t osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, + IN struct osm_sm * const sm, IN cl_dispatcher_t * const p_disp); /* * PARAMETERS @@ -207,8 +207,8 @@ osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, * p_log * [in] Pointer to the log object. * -* p_state_mgr -* [in] Pointer to the state manager object. +* sm +* [in] Pointer to the sm object. * * p_disp * [in] Pointer to the OpenSM central Dispatcher. diff --git a/opensm/include/opensm/osm_trap_rcv.h b/opensm/include/opensm/osm_trap_rcv.h index 393c5b1..e06f6b3 100644 --- a/opensm/include/opensm/osm_trap_rcv.h +++ b/opensm/include/opensm/osm_trap_rcv.h @@ -57,7 +57,6 @@ #include #include #include -#include #ifdef __cplusplus # define BEGIN_C_DECLS extern "C" { @@ -102,7 +101,6 @@ typedef struct _osm_trap_rcv { osm_stats_t *p_stats; osm_log_t *p_log; osm_resp_t *p_resp; - osm_state_mgr_t *p_state_mgr; cl_plock_t *p_lock; cl_event_wheel_t trap_aging_tracker; } osm_trap_rcv_t; @@ -120,9 +118,6 @@ typedef struct _osm_trap_rcv { * p_resp * Pointer to the generic MAD responder object. * -* p_state_mgr -* Pointer to the State Manager object. -* * p_lock * Pointer to the serializing lock. * @@ -210,7 +205,6 @@ ib_api_status_t osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, IN osm_stats_t * const p_stats, IN osm_resp_t * const p_resp, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock); /* * PARAMETERS @@ -229,9 +223,6 @@ ib_api_status_t osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, * p_log * [in] Pointer to the log object. * -* p_state_mgr -* [in] Pointer to the State Manager object. -* * p_lock * [in] Pointer to the OpenSM serializing lock. * diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..9411f23 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -49,7 +49,7 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_sa_sw_info_record.c osm_service.c \ osm_slvl_map_rcv.c osm_sm.c osm_sminfo_rcv.c \ osm_sm_mad_ctrl.c osm_sm_state_mgr.c osm_state_mgr.c \ - osm_state_mgr_ctrl.c osm_subnet.c \ + osm_subnet.c \ osm_sweep_fail_ctrl.c osm_sw_info_rcv.c osm_switch.c \ osm_prtn.c osm_prtn_config.c osm_qos.c osm_router.c \ osm_trap_rcv.c osm_ucast_mgr.c osm_ucast_updn.c \ diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index 04e52a8..5dd3955 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -2141,7 +2141,6 @@ static const char *const __osm_disp_msg_str[] = { "OSM_MSG_MAD_PORT_INFO", "OSM_MSG_MAD_SWITCH_INFO", "OSM_MSG_MAD_NODE_DESC", - "OSM_MSG_NO_SMPS_OUTSTANDING", "OSM_MSG_MAD_NODE_RECORD", "OSM_MSG_MAD_PORTINFO_RECORD", "OSM_MSG_MAD_SERVICE_RECORD", diff --git a/opensm/opensm/osm_node_info_rcv.c b/opensm/opensm/osm_node_info_rcv.c index 33a41fe..0f47859 100644 --- a/opensm/opensm/osm_node_info_rcv.c +++ b/opensm/opensm/osm_node_info_rcv.c @@ -65,6 +65,7 @@ #include #include #include +#include static void report_duplicated_guid(IN const osm_ni_rcv_t * const p_rcv, @@ -867,7 +868,6 @@ osm_ni_rcv_init(IN osm_ni_rcv_t * const p_rcv, IN osm_req_t * const p_req, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock) { ib_api_status_t status = IB_SUCCESS; @@ -880,7 +880,6 @@ osm_ni_rcv_init(IN osm_ni_rcv_t * const p_rcv, p_rcv->p_subn = p_subn; p_rcv->p_lock = p_lock; p_rcv->p_gen_req = p_req; - p_rcv->p_state_mgr = p_state_mgr; OSM_LOG_EXIT(p_rcv->p_log); return (status); @@ -944,13 +943,12 @@ void osm_ni_rcv_process(IN void *context, IN void *data) CL_PLOCK_RELEASE(p_rcv->p_lock); /* - * If we processed a new node - need to signal to the state_mgr that - * change detected. BUT - we cannot call the osm_state_mgr_process - * from within the lock of p_rcv->p_lock (can cause a deadlock). + * If we processed a new node - need to signal to the SM that + * change detected. */ if (process_new_flag) - osm_state_mgr_process(p_rcv->p_state_mgr, - OSM_SIGNAL_CHANGE_DETECTED); + osm_sm_signal(&p_rcv->p_subn->p_osm->sm, + OSM_SIGNAL_CHANGE_DETECTED); Exit: OSM_LOG_EXIT(p_rcv->p_log); diff --git a/opensm/opensm/osm_port_info_rcv.c b/opensm/opensm/osm_port_info_rcv.c index 114a705..fd63cd0 100644 --- a/opensm/opensm/osm_port_info_rcv.c +++ b/opensm/opensm/osm_port_info_rcv.c @@ -67,6 +67,7 @@ #include #include #include +#include /********************************************************************** **********************************************************************/ @@ -481,7 +482,6 @@ osm_pi_rcv_init(IN osm_pi_rcv_t * const p_rcv, IN osm_req_t * const p_req, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock) { ib_api_status_t status = IB_SUCCESS; @@ -493,7 +493,6 @@ osm_pi_rcv_init(IN osm_pi_rcv_t * const p_rcv, p_rcv->p_subn = p_subn; p_rcv->p_lock = p_lock; p_rcv->p_req = p_req; - p_rcv->p_state_mgr = p_state_mgr; OSM_LOG_EXIT(p_log); return (status); @@ -629,8 +628,8 @@ void osm_pi_rcv_process(IN void *context, IN void *data) "GUID 0x%" PRIx64 " port 0x%016" PRIx64 ", Commencing heavy sweep\n", cl_ntoh64(node_guid), cl_ntoh64(port_guid)); - osm_state_mgr_process(p_rcv->p_state_mgr, - OSM_SIGNAL_CHANGE_DETECTED); + osm_sm_signal(&p_rcv->p_subn->p_osm->sm, + OSM_SIGNAL_CHANGE_DETECTED); goto Exit; } diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c index 6fc9496..c052456 100644 --- a/opensm/opensm/osm_sm.c +++ b/opensm/opensm/osm_sm.c @@ -69,50 +69,51 @@ /********************************************************************** **********************************************************************/ -void __osm_sm_sweeper(IN void *p_ptr) +static void __osm_sm_sweeper(IN void *p_ptr) { ib_api_status_t status; osm_sm_t *const p_sm = (osm_sm_t *) p_ptr; + unsigned signals, i; OSM_LOG_ENTER(p_sm->p_log, __osm_sm_sweeper); - if (p_sm->thread_state == OSM_THREAD_STATE_INIT) { - p_sm->thread_state = OSM_THREAD_STATE_RUN; - } - - /* If the sweep interval was updated before - then run only if - * it is not zero. */ - while (p_sm->thread_state == OSM_THREAD_STATE_RUN && - p_sm->p_subn->opt.sweep_interval != 0) { - /* do the sweep only if we are in MASTER state */ - if (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || - p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING) - osm_state_mgr_process(&p_sm->state_mgr, - OSM_SIGNAL_SWEEP); - + while (p_sm->thread_state == OSM_THREAD_STATE_RUN) { /* * Wait on the event with a timeout. * Sweeps may be initiated "off schedule" by simply * signaling the event. */ - status = cl_event_wait_on(&p_sm->signal, + status = cl_event_wait_on(&p_sm->signal_event, + p_sm->p_subn->opt.sweep_interval ? p_sm->p_subn->opt.sweep_interval * - 1000000, TRUE); - - if (status == CL_SUCCESS) { - if (osm_log_is_active(p_sm->p_log, OSM_LOG_DEBUG)) { - osm_log(p_sm->p_log, OSM_LOG_DEBUG, - "__osm_sm_sweeper: " - "Off schedule sweep signalled\n"); - } - } else { - if (status != CL_TIMEOUT) { - osm_log(p_sm->p_log, OSM_LOG_ERROR, - "__osm_sm_sweeper: ERR 2E01: " - "Event wait failed (%s)\n", - CL_STATUS_MSG(status)); - } + 1000000 : EVENT_NO_TIMEOUT, TRUE); + + if (status == CL_SUCCESS) + osm_log(p_sm->p_log, OSM_LOG_DEBUG, + "__osm_sm_sweeper: " + "Off schedule sweep signalled\n"); + else if (status != CL_TIMEOUT) { + osm_log(p_sm->p_log, OSM_LOG_ERROR, + "__osm_sm_sweeper: ERR 2E01: " + "Event wait failed (%s)\n", + CL_STATUS_MSG(status)); + continue; } + + cl_spinlock_acquire(&p_sm->signal_lock); + signals = p_sm->signal_mask; + p_sm->signal_mask = 0; + cl_spinlock_release(&p_sm->signal_lock); + + /* do the sweep only if we are in MASTER state */ + if (status == CL_TIMEOUT && + (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || + p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING)) + p_sm->signal_mask |= 1 << OSM_SIGNAL_SWEEP; + + for (i = 0 ; signals ; signals >>= 1 , i++) + if (signals&1) + osm_state_mgr_process(&p_sm->state_mgr, i); } OSM_LOG_EXIT(p_sm->p_log); @@ -125,7 +126,8 @@ void osm_sm_construct(IN osm_sm_t * const p_sm) memset(p_sm, 0, sizeof(*p_sm)); p_sm->thread_state = OSM_THREAD_STATE_NONE; p_sm->sm_trans_id = OSM_SM_INITIAL_TID_VALUE; - cl_event_construct(&p_sm->signal); + cl_spinlock_construct(&p_sm->signal_lock); + cl_event_construct(&p_sm->signal_event); cl_event_construct(&p_sm->subnet_up_event); cl_thread_construct(&p_sm->sweeper); osm_req_construct(&p_sm->req); @@ -139,7 +141,6 @@ void osm_sm_construct(IN osm_sm_t * const p_sm) osm_ucast_mgr_construct(&p_sm->ucast_mgr); osm_link_mgr_construct(&p_sm->link_mgr); osm_state_mgr_construct(&p_sm->state_mgr); - osm_state_mgr_ctrl_construct(&p_sm->state_mgr_ctrl); osm_drop_mgr_construct(&p_sm->drop_mgr); osm_lft_rcv_construct(&p_sm->lft_rcv); osm_mft_rcv_construct(&p_sm->mft_rcv); @@ -174,7 +175,7 @@ void osm_sm_shutdown(IN osm_sm_t * const p_sm) * Destroy the thread before we tear down the other objects. */ if (signal_event) - cl_event_signal(&p_sm->signal); + cl_event_signal(&p_sm->signal_event); cl_thread_destroy(&p_sm->sweeper); @@ -196,7 +197,6 @@ void osm_sm_shutdown(IN osm_sm_t * const p_sm) cl_disp_unregister(p_sm->vla_disp_h); cl_disp_unregister(p_sm->pkey_disp_h); osm_sweep_fail_ctrl_destroy(&p_sm->sweep_fail_ctrl); - osm_state_mgr_ctrl_destroy(&p_sm->state_mgr_ctrl); OSM_LOG_EXIT(p_sm->p_log); } @@ -226,8 +226,9 @@ void osm_sm_destroy(IN osm_sm_t * const p_sm) osm_state_mgr_destroy(&p_sm->state_mgr); osm_sm_state_mgr_destroy(&p_sm->sm_state_mgr); osm_mcast_mgr_destroy(&p_sm->mcast_mgr); - cl_event_destroy(&p_sm->signal); + cl_event_destroy(&p_sm->signal_event); cl_event_destroy(&p_sm->subnet_up_event); + cl_spinlock_destroy(&p_sm->signal_lock); osm_log(p_sm->p_log, OSM_LOG_SYS, "Exiting SM\n"); /* Format Waived */ OSM_LOG_EXIT(p_sm->p_log); @@ -259,7 +260,11 @@ osm_sm_init(IN osm_sm_t * const p_sm, p_sm->p_disp = p_disp; p_sm->p_lock = p_lock; - status = cl_event_init(&p_sm->signal, FALSE); + status = cl_spinlock_init(&p_sm->signal_lock); + if (status != CL_SUCCESS) + goto Exit; + + status = cl_event_init(&p_sm->signal_event, FALSE); if (status != CL_SUCCESS) goto Exit; @@ -288,20 +293,20 @@ osm_sm_init(IN osm_sm_t * const p_sm, status = osm_ni_rcv_init(&p_sm->ni_rcv, &p_sm->req, - p_subn, p_log, &p_sm->state_mgr, p_lock); + p_subn, p_log, p_lock); if (status != IB_SUCCESS) goto Exit; status = osm_pi_rcv_init(&p_sm->pi_rcv, &p_sm->req, - p_subn, p_log, &p_sm->state_mgr, p_lock); + p_subn, p_log, p_lock); if (status != IB_SUCCESS) goto Exit; status = osm_si_rcv_init(&p_sm->si_rcv, p_sm->p_subn, p_sm->p_log, - &p_sm->req, &p_sm->state_mgr, p_sm->p_lock); + &p_sm->req, p_sm->p_lock); if (status != IB_SUCCESS) goto Exit; @@ -345,12 +350,6 @@ osm_sm_init(IN osm_sm_t * const p_sm, if (status != IB_SUCCESS) goto Exit; - status = osm_state_mgr_ctrl_init(&p_sm->state_mgr_ctrl, - &p_sm->state_mgr, - p_sm->p_log, p_sm->p_disp); - if (status != IB_SUCCESS) - goto Exit; - status = osm_drop_mgr_init(&p_sm->drop_mgr, p_sm->p_subn, p_sm->p_log, &p_sm->req, p_sm->p_lock); @@ -366,7 +365,7 @@ osm_sm_init(IN osm_sm_t * const p_sm, goto Exit; status = osm_sweep_fail_ctrl_init(&p_sm->sweep_fail_ctrl, - p_log, &p_sm->state_mgr, p_disp); + p_log, p_sm, p_disp); if (status != IB_SUCCESS) goto Exit; @@ -375,7 +374,6 @@ osm_sm_init(IN osm_sm_t * const p_sm, p_stats, &p_sm->resp, p_log, - &p_sm->state_mgr, &p_sm->sm_state_mgr, p_lock); if (status != IB_SUCCESS) goto Exit; @@ -383,13 +381,11 @@ osm_sm_init(IN osm_sm_t * const p_sm, status = osm_trap_rcv_init(&p_sm->trap_rcv, p_subn, p_stats, - &p_sm->resp, p_log, &p_sm->state_mgr, - p_lock); + &p_sm->resp, p_log, p_lock); if (status != IB_SUCCESS) goto Exit; status = osm_sm_state_mgr_init(&p_sm->sm_state_mgr, - &p_sm->state_mgr, p_sm->p_subn, &p_sm->req, p_sm->p_log); if (status != IB_SUCCESS) goto Exit; @@ -480,13 +476,11 @@ osm_sm_init(IN osm_sm_t * const p_sm, * Now that the component objects are initialized, start * the sweeper thread if the user wants sweeping. */ - if (p_sm->p_subn->opt.sweep_interval) { - p_sm->thread_state = OSM_THREAD_STATE_INIT; - status = cl_thread_init(&p_sm->sweeper, __osm_sm_sweeper, p_sm, - "opensm sweeper"); - if (status != IB_SUCCESS) - goto Exit; - } + p_sm->thread_state = OSM_THREAD_STATE_RUN; + status = cl_thread_init(&p_sm->sweeper, __osm_sm_sweeper, p_sm, + "opensm sweeper"); + if (status != IB_SUCCESS) + goto Exit; Exit: OSM_LOG_EXIT(p_log); @@ -495,10 +489,20 @@ osm_sm_init(IN osm_sm_t * const p_sm, /********************************************************************** **********************************************************************/ +void osm_sm_signal(osm_sm_t * p_sm, osm_signal_t signal) +{ + cl_spinlock_acquire(&p_sm->signal_lock); + p_sm->signal_mask |= 1 << signal; + cl_event_signal(&p_sm->signal_event); + cl_spinlock_release(&p_sm->signal_lock); +} + +/********************************************************************** + **********************************************************************/ void osm_sm_sweep(IN osm_sm_t * const p_sm) { OSM_LOG_ENTER(p_sm->p_log, osm_sm_sweep); - osm_state_mgr_process(&p_sm->state_mgr, OSM_SIGNAL_SWEEP); + osm_sm_signal(p_sm, OSM_SIGNAL_SWEEP); OSM_LOG_EXIT(p_sm->p_log); } diff --git a/opensm/opensm/osm_sm_mad_ctrl.c b/opensm/opensm/osm_sm_mad_ctrl.c index 4842f94..2ed973d 100644 --- a/opensm/opensm/osm_sm_mad_ctrl.c +++ b/opensm/opensm/osm_sm_mad_ctrl.c @@ -74,7 +74,6 @@ __osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, IN osm_madw_t * const p_madw) { uint32_t outstanding; - cl_status_t status; OSM_LOG_ENTER(p_ctrl->p_log, __osm_sm_mad_ctrl_retire_trans_mad); @@ -101,30 +100,16 @@ __osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, if (outstanding == 0) { /* The wire is clean. - Signal the state manager. + Signal the subnet manager. */ - if (osm_log_is_active(p_ctrl->p_log, OSM_LOG_DEBUG)) - osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, - "__osm_sm_mad_ctrl_retire_trans_mad: " - "Posting Dispatcher message %s\n", - osm_get_disp_msg_str - (OSM_MSG_NO_SMPS_OUTSTANDING)); - - status = cl_disp_post(p_ctrl->h_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, NULL, - NULL); + osm_log(p_ctrl->p_log, OSM_LOG_DEBUG, + "__osm_sm_mad_ctrl_retire_trans_mad: " + "signal OSM_SIGNAL_NO_PENDING_TRANSACTIONS\n"); - if (status != CL_SUCCESS) { - osm_log(p_ctrl->p_log, OSM_LOG_ERROR, - "__osm_sm_mad_ctrl_retire_trans_mad: ERR 3101: " - "Dispatcher post message failed (%s)\n", - CL_STATUS_MSG(status)); - goto Exit; - } + osm_sm_signal(&p_ctrl->p_subn->p_osm->sm, + OSM_SIGNAL_NO_PENDING_TRANSACTIONS); } - Exit: OSM_LOG_EXIT(p_ctrl->p_log); } diff --git a/opensm/opensm/osm_sm_state_mgr.c b/opensm/opensm/osm_sm_state_mgr.c index 2ff9639..185070f 100644 --- a/opensm/opensm/osm_sm_state_mgr.c +++ b/opensm/opensm/osm_sm_state_mgr.c @@ -53,7 +53,6 @@ #include #include #include -#include #include #include #include @@ -325,7 +324,7 @@ static void __osm_sm_state_mgr_polling_callback(IN void *context) osm_log(p_sm_mgr->p_log, OSM_LOG_VERBOSE, "__osm_sm_state_mgr_polling_callback: " "Signalling subnet_up_event\n"); - cl_event_signal(p_sm_mgr->p_state_mgr->p_subnet_up_event); + cl_event_signal(&p_sm_mgr->p_subn->p_osm->sm.subnet_up_event); goto Exit; } @@ -394,7 +393,6 @@ void osm_sm_state_mgr_destroy(IN osm_sm_state_mgr_t * const p_sm_mgr) **********************************************************************/ ib_api_status_t osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, - IN osm_state_mgr_t * const p_state_mgr, IN osm_subn_t * const p_subn, IN osm_req_t * const p_req, IN osm_log_t * const p_log) { @@ -411,7 +409,6 @@ osm_sm_state_mgr_init(IN osm_sm_state_mgr_t * const p_sm_mgr, p_sm_mgr->p_log = p_log; p_sm_mgr->p_req = p_req; p_sm_mgr->p_subn = p_subn; - p_sm_mgr->p_state_mgr = p_state_mgr; if (p_subn->opt.sm_inactive) { /* init the state of the SM to not active */ @@ -575,8 +572,8 @@ osm_sm_state_mgr_process(IN osm_sm_state_mgr_t * const p_sm_mgr, p_sm_mgr->p_subn->sm_state = IB_SMINFO_STATE_DISCOVERING; p_sm_mgr->p_subn->coming_out_of_standby = TRUE; - osm_state_mgr_process(p_sm_mgr->p_state_mgr, - OSM_SIGNAL_EXIT_STBY); + osm_sm_signal(&p_sm_mgr->p_subn->p_osm->sm, + OSM_SIGNAL_EXIT_STBY); break; case OSM_SM_SIGNAL_DISABLE: /* @@ -609,8 +606,8 @@ osm_sm_state_mgr_process(IN osm_sm_state_mgr_t * const p_sm_mgr, p_sm_mgr->p_subn->master_sm_base_lid = p_sm_mgr->p_subn->sm_base_lid; p_sm_mgr->p_subn->coming_out_of_standby = TRUE; - osm_state_mgr_process(p_sm_mgr->p_state_mgr, - OSM_SIGNAL_EXIT_STBY); + osm_sm_signal(&p_sm_mgr->p_subn->p_osm->sm, + OSM_SIGNAL_EXIT_STBY); break; case OSM_SM_SIGNAL_ACKNOWLEDGE: /* @@ -671,8 +668,8 @@ osm_sm_state_mgr_process(IN osm_sm_state_mgr_t * const p_sm_mgr, "Received OSM_SM_SIGNAL_HANDOVER or OSM_SM_SIGNAL_POLLING_TIMEOUT\n"); p_sm_mgr->p_polling_sm = NULL; p_sm_mgr->p_subn->force_immediate_heavy_sweep = TRUE; - osm_state_mgr_process(p_sm_mgr->p_state_mgr, - OSM_SIGNAL_SWEEP); + osm_sm_signal(&p_sm_mgr->p_subn->p_osm->sm, + OSM_SIGNAL_SWEEP); break; case OSM_SM_SIGNAL_HANDOVER_SENT: /* diff --git a/opensm/opensm/osm_sminfo_rcv.c b/opensm/opensm/osm_sminfo_rcv.c index 8d049a8..2367941 100644 --- a/opensm/opensm/osm_sminfo_rcv.c +++ b/opensm/opensm/osm_sminfo_rcv.c @@ -90,7 +90,6 @@ osm_sminfo_rcv_init(IN osm_sminfo_rcv_t * const p_rcv, IN osm_stats_t * const p_stats, IN osm_resp_t * const p_resp, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN osm_sm_state_mgr_t * const p_sm_state_mgr, IN cl_plock_t * const p_lock) { @@ -105,7 +104,6 @@ osm_sminfo_rcv_init(IN osm_sminfo_rcv_t * const p_rcv, p_rcv->p_lock = p_lock; p_rcv->p_stats = p_stats; p_rcv->p_resp = p_resp; - p_rcv->p_state_mgr = p_state_mgr; p_rcv->p_sm_state_mgr = p_sm_state_mgr; OSM_LOG_EXIT(p_rcv->p_log); @@ -387,10 +385,8 @@ __osm_sminfo_rcv_process_set_request(IN const osm_sminfo_rcv_t * const p_rcv, } /********************************************************************** - * Return a signal with which to call the osm_state_mgr_process. - * This is done since we are locked by p_rcv->p_lock in this function, - * and thus cannot call osm_state_mgr_process (that locks the state_lock). - * If return OSM_SIGNAL_NONE - do not call osm_state_mgr_process. + * Return a signal with which to call the osm_sm_signal. + * If return OSM_SIGNAL_NONE - do not call osm_sm_signal. **********************************************************************/ static osm_signal_t __osm_sminfo_rcv_process_get_sm(IN const osm_sminfo_rcv_t * const p_rcv, @@ -617,10 +613,10 @@ __osm_sminfo_rcv_process_get_response(IN const osm_sminfo_rcv_t * const p_rcv, CL_PLOCK_RELEASE(p_rcv->p_lock); /* If process_get_sm_ret_val != OSM_SIGNAL_NONE then we have to signal - * to the state_mgr with that signal. */ + * to the SM with that signal. */ if (process_get_sm_ret_val != OSM_SIGNAL_NONE) - osm_state_mgr_process(p_rcv->p_state_mgr, - process_get_sm_ret_val); + osm_sm_signal(&p_rcv->p_subn->p_osm->sm, + process_get_sm_ret_val); Exit: OSM_LOG_EXIT(p_rcv->p_log); diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c index 030d344..c7cd4e8 100644 --- a/opensm/opensm/osm_state_mgr.c +++ b/opensm/opensm/osm_state_mgr.c @@ -78,7 +78,6 @@ osm_signal_t osm_qos_setup(IN osm_opensm_t * p_osm); void osm_state_mgr_construct(IN osm_state_mgr_t * const p_mgr) { memset(p_mgr, 0, sizeof(*p_mgr)); - cl_spinlock_construct(&p_mgr->state_lock); cl_spinlock_construct(&p_mgr->idle_lock); p_mgr->state = OSM_SM_STATE_INIT; } @@ -92,7 +91,6 @@ void osm_state_mgr_destroy(IN osm_state_mgr_t * const p_mgr) OSM_LOG_ENTER(p_mgr->p_log, osm_state_mgr_destroy); /* destroy the locks */ - cl_spinlock_destroy(&p_mgr->state_lock); cl_spinlock_destroy(&p_mgr->idle_lock); OSM_LOG_EXIT(p_mgr->p_log); @@ -149,13 +147,6 @@ osm_state_mgr_init(IN osm_state_mgr_t * const p_mgr, p_mgr->p_lock = p_lock; p_mgr->p_subnet_up_event = p_subnet_up_event; - status = cl_spinlock_init(&p_mgr->state_lock); - if (status != CL_SUCCESS) { - osm_log(p_mgr->p_log, OSM_LOG_ERROR, - "osm_state_mgr_init: ERR 3301: " - "Spinlock init failed (%s)\n", CL_STATUS_MSG(status)); - } - cl_qlist_init(&p_mgr->idle_time_list); status = cl_spinlock_init(&p_mgr->idle_lock); @@ -1791,16 +1782,6 @@ void osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, if (osm_exit_flag) signal = OSM_SIGNAL_NONE; - /* - * The state lock prevents many race conditions from screwing - * up the state transition process. For example, if an function - * puts transactions on the wire, the state lock guarantees this - * loop will see the return code ("DONE PENDING") of the function - * before the "NO OUTSTANDING TRANSACTIONS" signal is asynchronously - * received. - */ - cl_spinlock_acquire(&p_mgr->state_lock); - while (signal != OSM_SIGNAL_NONE) { if (osm_log_is_active(p_mgr->p_log, OSM_LOG_DEBUG)) { osm_log(p_mgr->p_log, OSM_LOG_DEBUG, @@ -2779,8 +2760,6 @@ void osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, } - cl_spinlock_release(&p_mgr->state_lock); - OSM_LOG_EXIT(p_mgr->p_log); } @@ -2814,7 +2793,8 @@ osm_state_mgr_process_idle(IN osm_state_mgr_t * const p_mgr, cl_qlist_insert_tail(&p_mgr->idle_time_list, &p_idle_item->list_item); cl_spinlock_release(&p_mgr->idle_lock); - osm_state_mgr_process(p_mgr, OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST); + osm_sm_signal(&p_mgr->p_subn->p_osm->sm, + OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST); OSM_LOG_EXIT(p_mgr->p_log); diff --git a/opensm/opensm/osm_state_mgr_ctrl.c b/opensm/opensm/osm_state_mgr_ctrl.c deleted file mode 100644 index 53868cf..0000000 --- a/opensm/opensm/osm_state_mgr_ctrl.c +++ /dev/null @@ -1,115 +0,0 @@ -/* - * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. - * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. - * - * This software is available to you under a choice of one of two - * licenses. You may choose to be licensed under the terms of the GNU - * General Public License (GPL) Version 2, available from the file - * COPYING in the main directory of this source tree, or the - * OpenIB.org BSD license below: - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * - Redistributions of source code must retain the above - * copyright notice, this list of conditions and the following - * disclaimer. - * - * - Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS - * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN - * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE - * SOFTWARE. - * - */ - -/* - * Abstract: - * Implementation of osm_state_mgr_ctrl_t. - * This object represents the State Manager Controller object. - * This object is part of the opensm family of objects. - * - * Environment: - * Linux User Mode - * - * $Revision: 1.5 $ - */ - -#if HAVE_CONFIG_H -# include -#endif /* HAVE_CONFIG_H */ - -#include -#include -#include - -/********************************************************************** - **********************************************************************/ -void __osm_state_mgr_ctrl_disp_callback(IN void *context, IN void *p_data) -{ - /* ignore return status when invoked via the dispatcher */ - osm_state_mgr_process(((osm_state_mgr_ctrl_t *) context)->p_mgr, - (osm_signal_t) (p_data)); -} - -/********************************************************************** - **********************************************************************/ -void osm_state_mgr_ctrl_construct(IN osm_state_mgr_ctrl_t * const p_ctrl) -{ - memset(p_ctrl, 0, sizeof(*p_ctrl)); - p_ctrl->h_disp = CL_DISP_INVALID_HANDLE; -} - -/********************************************************************** - **********************************************************************/ -void osm_state_mgr_ctrl_destroy(IN osm_state_mgr_ctrl_t * const p_ctrl) -{ - CL_ASSERT(p_ctrl); - cl_disp_unregister(p_ctrl->h_disp); -} - -/********************************************************************** - **********************************************************************/ -ib_api_status_t -osm_state_mgr_ctrl_init(IN osm_state_mgr_ctrl_t * const p_ctrl, - IN osm_state_mgr_t * const p_mgr, - IN osm_log_t * const p_log, - IN cl_dispatcher_t * const p_disp) -{ - ib_api_status_t status = IB_SUCCESS; - - OSM_LOG_ENTER(p_log, osm_state_mgr_ctrl_init); - - osm_state_mgr_ctrl_construct(p_ctrl); - p_ctrl->p_log = p_log; - - p_ctrl->p_mgr = p_mgr; - p_ctrl->p_disp = p_disp; - - p_ctrl->h_disp = cl_disp_register(p_disp, - OSM_MSG_NO_SMPS_OUTSTANDING, - __osm_state_mgr_ctrl_disp_callback, - p_ctrl); - - if (p_ctrl->h_disp == CL_DISP_INVALID_HANDLE) { - osm_log(p_log, OSM_LOG_ERROR, - "osm_state_mgr_ctrl_init: ERR 3401: " - "Dispatcher registration failed\n"); - status = IB_INSUFFICIENT_RESOURCES; - goto Exit; - } - - Exit: - OSM_LOG_EXIT(p_log); - return (status); -} diff --git a/opensm/opensm/osm_sw_info_rcv.c b/opensm/opensm/osm_sw_info_rcv.c index a2feadf..5988f8c 100644 --- a/opensm/opensm/osm_sw_info_rcv.c +++ b/opensm/opensm/osm_sw_info_rcv.c @@ -59,6 +59,7 @@ #include #include #include +#include /********************************************************************** The plock must be held before calling this function. @@ -519,7 +520,6 @@ osm_si_rcv_init(IN osm_si_rcv_t * const p_rcv, IN osm_subn_t * const p_subn, IN osm_log_t * const p_log, IN osm_req_t * const p_req, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock) { ib_api_status_t status = IB_SUCCESS; @@ -531,7 +531,6 @@ osm_si_rcv_init(IN osm_si_rcv_t * const p_rcv, p_rcv->p_subn = p_subn; p_rcv->p_lock = p_lock; p_rcv->p_req = p_req; - p_rcv->p_state_mgr = p_state_mgr; OSM_LOG_EXIT(p_rcv->p_log); return (status); @@ -614,8 +613,8 @@ void osm_si_rcv_process(IN void *context, IN void *data) if (__osm_si_rcv_process_existing (p_rcv, p_node, p_madw)) { CL_PLOCK_RELEASE(p_rcv->p_lock); - osm_state_mgr_process(p_rcv->p_state_mgr, - OSM_SIGNAL_CHANGE_DETECTED); + osm_sm_signal(&p_rcv->p_subn->p_osm->sm, + OSM_SIGNAL_CHANGE_DETECTED); goto Exit; } } diff --git a/opensm/opensm/osm_sweep_fail_ctrl.c b/opensm/opensm/osm_sweep_fail_ctrl.c index 1ddb4f2..cd182c7 100644 --- a/opensm/opensm/osm_sweep_fail_ctrl.c +++ b/opensm/opensm/osm_sweep_fail_ctrl.c @@ -50,6 +50,7 @@ #include #include #include +#include /********************************************************************** **********************************************************************/ @@ -63,7 +64,7 @@ void __osm_sweep_fail_ctrl_disp_callback(IN void *context, IN void *p_data) /* Notify the state manager that we had a light sweep failure. */ - osm_state_mgr_process(p_ctrl->p_state_mgr, OSM_SIGNAL_LIGHT_SWEEP_FAIL); + osm_sm_signal(p_ctrl->sm, OSM_SIGNAL_LIGHT_SWEEP_FAIL); OSM_LOG_EXIT(p_ctrl->p_log); } @@ -89,7 +90,7 @@ void osm_sweep_fail_ctrl_destroy(IN osm_sweep_fail_ctrl_t * const p_ctrl) ib_api_status_t osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, + IN osm_sm_t * const sm, IN cl_dispatcher_t * const p_disp) { ib_api_status_t status = IB_SUCCESS; @@ -99,7 +100,7 @@ osm_sweep_fail_ctrl_init(IN osm_sweep_fail_ctrl_t * const p_ctrl, osm_sweep_fail_ctrl_construct(p_ctrl); p_ctrl->p_log = p_log; p_ctrl->p_disp = p_disp; - p_ctrl->p_state_mgr = p_state_mgr; + p_ctrl->sm = sm; p_ctrl->h_disp = cl_disp_register(p_disp, OSM_MSG_LIGHT_SWEEP_FAIL, diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c index babc625..3323a83 100644 --- a/opensm/opensm/osm_trap_rcv.c +++ b/opensm/opensm/osm_trap_rcv.c @@ -187,7 +187,6 @@ osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, IN osm_stats_t * const p_stats, IN osm_resp_t * const p_resp, IN osm_log_t * const p_log, - IN osm_state_mgr_t * const p_state_mgr, IN cl_plock_t * const p_lock) { ib_api_status_t status = IB_SUCCESS; @@ -201,7 +200,6 @@ osm_trap_rcv_init(IN osm_trap_rcv_t * const p_rcv, p_rcv->p_lock = p_lock; p_rcv->p_stats = p_stats; p_rcv->p_resp = p_resp; - p_rcv->p_state_mgr = p_state_mgr; if(cl_event_wheel_init(&p_rcv->trap_aging_tracker)) { osm_log(p_log, OSM_LOG_ERROR, @@ -665,7 +663,7 @@ __osm_trap_rcv_process_request(IN osm_trap_rcv_t * const p_rcv, p_rcv->p_subn->force_immediate_heavy_sweep = TRUE; } - osm_state_mgr_process(p_rcv->p_state_mgr, OSM_SIGNAL_SWEEP); + osm_sm_signal(&p_rcv->p_subn->p_osm->sm, OSM_SIGNAL_SWEEP); } /* If we reached here due to trap 129/130/131 - do not need to do -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Tue Aug 28 06:19:05 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:19:05 +0300 Subject: [ofa-general] Re: [PATCH 2/7 V3] osm: QoS - policy file parser Lex & Yacc In-Reply-To: <46D35992.1030804@dev.mellanox.co.il> References: <46D35992.1030804@dev.mellanox.co.il> Message-ID: <20070828131905.GC18082@sashak.voltaire.com> On 02:09 Tue 28 Aug , Yevgeny Kliteynik wrote: > QoS Policy file parser - Lex & Yacc files > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. But please clean this: ./osm_qos_parser_l.c:1762: warning: 'yyunput' defined but not used Sasha From vlad at dev.mellanox.co.il Tue Aug 28 06:32:49 2007 From: vlad at dev.mellanox.co.il (Vladimir Sokolovsky) Date: Tue, 28 Aug 2007 16:32:49 +0300 Subject: [ofa-general] [ANNOUNCE] ofed_1_3/linux-2.6.git updated to 2.6.23-rc4 Message-ID: <46D42401.6050209@dev.mellanox.co.il> FYI, git://git.openfabrics.org/ofed_1_3/linux-2.6.git I've merged in 2.6.23-rc4. Regards, Vladimir From sashak at voltaire.com Tue Aug 28 06:40:44 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:40:44 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V3] osm: QoS policy C & H files In-Reply-To: <46D359BE.6040009@dev.mellanox.co.il> References: <46D359BE.6040009@dev.mellanox.co.il> Message-ID: <20070828134044.GD18082@sashak.voltaire.com> On 02:09 Tue 28 Aug , Yevgeny Kliteynik wrote: > QoS policy data structures and functions > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. I still have some comments (below) and expect it will be addressed. Sasha > --- > opensm/include/opensm/osm_qos_policy.h | 189 +++++++ > opensm/opensm/osm_qos_policy.c | 921 ++++++++++++++++++++++++++++++++ > 2 files changed, 1110 insertions(+), 0 deletions(-) > create mode 100644 opensm/include/opensm/osm_qos_policy.h > create mode 100644 opensm/opensm/osm_qos_policy.c > > diff --git a/opensm/include/opensm/osm_qos_policy.h b/opensm/include/opensm/osm_qos_policy.h > new file mode 100644 > index 0000000..dd15f99 > --- /dev/null > +++ b/opensm/include/opensm/osm_qos_policy.h > @@ -0,0 +1,189 @@ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * Declaration of OSM QoS Policy data types and functions. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#ifndef OSM_QOS_POLICY_H > +#define OSM_QOS_POLICY_H > + > +#include > +#include > +#include > +#include > + > +#define YYSTYPE char * > +#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128 > +#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME "default" > + > +/***************************************************/ > + > +typedef struct _osm_qos_port_group_t { > + char *name; /* single string (this port group name) */ > + char *use; /* single string (description) */ > + cl_list_t port_name_list; /* list of port names (.../.../...) */ > + uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit guids) */ Instead of uint64_t ** use something like: struct range { uint64_t min, max; } > + unsigned guid_range_len; /* num of guid ranges in the array */ > + cl_list_t partition_list; /* list of partition names */ Why not pkey range here? (by name partition search is extermely slow). > + boolean_t node_type_ca; > + boolean_t node_type_switch; > + boolean_t node_type_router; > + boolean_t node_type_self; This probably could be optimized by using bitmask. Then instead of four separate checks you will start with single 'if (node_type_mask) ...'. > +} osm_qos_port_group_t; > + > +/***************************************************/ > + > +typedef struct _osm_qos_vlarb_scope_t { > + cl_list_t group_list; /* list of group names (strings) */ > + cl_list_t across_list; /* list of 'across' group names (strings) */ > + cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit values */ > + cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit values */ > + uint32_t vl_high_limit; /* single integer */ > + boolean_t vl_high_limit_set; > +} osm_qos_vlarb_scope_t; > + > +/***************************************************/ > + > +typedef struct _osm_qos_sl2vl_scope_t { > + cl_list_t group_list; /* list of strings (port group names) */ > + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > + cl_list_t across_from_list; /* list of strings (port group names) */ > + cl_list_t across_to_list; /* list of strings (port group names) */ > + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > + boolean_t sl2vl_table_set; > +} osm_qos_sl2vl_scope_t; > + > +/***************************************************/ > + > +typedef struct _osm_qos_level_t { > + char *use; > + char *name; > + uint8_t sl; > + boolean_t sl_set; > + uint8_t mtu_limit; > + boolean_t mtu_limit_set; > + uint8_t rate_limit; > + boolean_t rate_limit_set; > + uint8_t pkt_life; > + boolean_t pkt_life_set; > + uint64_t **path_bits_range_arr; /* array of bit ranges (real values are 32bits) */ > + unsigned path_bits_range_len; /* num of bit ranges in the array */ > + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ > + unsigned pkey_range_len; > +} osm_qos_level_t; > + > + > +/***************************************************/ > + > +typedef struct _osm_qos_match_rule_t { > + char *use; > + cl_list_t source_list; /* list of strings */ > + cl_list_t source_group_list; /* list of pointers to relevant port-group */ > + cl_list_t destination_list; /* list of strings */ > + cl_list_t destination_group_list; /* list of pointers to relevant port-group */ > + char *qos_level_name; > + osm_qos_level_t *p_qos_level; > + uint64_t **service_id_range_arr; /* array of SID ranges (64-bit values) */ > + unsigned service_id_range_len; > + uint64_t **qos_class_range_arr; /* array of QoS Class ranges (real values are 16bits) */ > + unsigned qos_class_range_len; > + uint64_t **pkey_range_arr; /* array of PKey ranges (real values are 16bits) */ > + unsigned pkey_range_len; > +} osm_qos_match_rule_t; > + > +/***************************************************/ > + > +typedef struct _osm_qos_policy_t { > + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > + cl_list_t qos_levels; /* list of osm_qos_level_t */ > + cl_list_t qos_match_rules; /* list of osm_qos_match_rule_t */ Here and above - where possible please use cl_qlist_t instead of cl_list_t - it is _much_ faster (I did some benchmarking when worked on up/down performance issues). > + osm_qos_level_t *p_default_qos_level; /* default QoS level */ > +} osm_qos_policy_t; > + > +/***************************************************/ > + > +osm_qos_port_group_t * osm_qos_policy_port_group_create(); > +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p_port_group); > + > +osm_qos_vlarb_scope_t * osm_qos_policy_vlarb_scope_create(); > +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p_vlarb_scope); > + > +osm_qos_sl2vl_scope_t * osm_qos_policy_sl2vl_scope_create(); > +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p_sl2vl_scope); > + > +osm_qos_level_t * osm_qos_policy_qos_level_create(); > +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p_qos_level); > + > +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > + IN ib_net16_t pkey); > + > +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp); > + > +osm_qos_match_rule_t * osm_qos_policy_match_rule_create(); > +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p_match_rule); > + > +osm_qos_policy_t * osm_qos_policy_create(); > +void osm_qos_policy_destroy(osm_qos_policy_t * p_qos_policy); > +int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, osm_log_t * p_log); > + > +void osm_qos_policy_get_qos_level_by_pr(IN const osm_qos_policy_t * p_qos_policy, > + IN const osm_pr_rcv_t * p_rcv, > + IN const ib_path_rec_t * p_pr, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp, > + IN ib_net64_t comp_mask, > + OUT osm_qos_level_t ** pp_qos_level); > + > +/***************************************************/ > + > +int osm_qos_parse_policy_file(IN osm_subn_t * const p_subn); > + > +/***************************************************/ > + > +#endif /* ifndef OSM_QOS_POLICY_H */ > + > diff --git a/opensm/opensm/osm_qos_policy.c b/opensm/opensm/osm_qos_policy.c > new file mode 100644 > index 0000000..a5a8856 > --- /dev/null > +++ b/opensm/opensm/osm_qos_policy.c > @@ -0,0 +1,921 @@ > +/* > + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > + * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. > + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + * > + */ > + > +/* > + * Abstract: > + * OSM QoS Policy functions. > + * > + * Environment: > + * Linux User Mode > + * > + * Author: > + * Yevgeny Kliteynik, Mellanox > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__is_num_in_range_arr(uint64_t ** range_arr, > + unsigned range_arr_len, uint64_t num) > +{ > + unsigned ind_1 = 0; > + unsigned ind_2 = range_arr_len - 1; > + unsigned ind_mid; > + > + if (!range_arr || !range_arr_len) > + return FALSE; > + > + while (ind_1 <= ind_2) { > + if (num < range_arr[ind_1][0] || num > range_arr[ind_2][1]) > + return FALSE; > + else if (num <= range_arr[ind_1][1] || num >= range_arr[ind_2][0]) > + return TRUE; > + > + ind_mid = ind_1 + (ind_2 - ind_1 + 1)/2; > + > + if (num < range_arr[ind_mid][0]) > + ind_2 = ind_mid; > + else if (num > range_arr[ind_mid][1]) > + ind_1 = ind_mid; > + else > + return TRUE; > + > + ind_1++; > + ind_2--; > + } > + > + return FALSE; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static void __free_single_element(void *p_element, void *context) > +{ > + if (p_element) > + free(p_element); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_port_group_t *osm_qos_policy_port_group_create() > +{ > + osm_qos_port_group_t *p = > + (osm_qos_port_group_t *) malloc(sizeof(osm_qos_port_group_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_port_group_t)); > + > + cl_list_init(&p->port_name_list, 10); > + cl_list_init(&p->partition_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_port_group_destroy(osm_qos_port_group_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->name) > + free(p->name); > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->guid_range_len; i++) > + free(p->guid_range_arr[i]); > + if (p->guid_range_arr) > + free(p->guid_range_arr); > + > + cl_list_apply_func(&p->port_name_list, __free_single_element, NULL); > + cl_list_remove_all(&p->port_name_list); > + cl_list_destroy(&p->port_name_list); > + > + cl_list_apply_func(&p->partition_list, __free_single_element, NULL); > + cl_list_remove_all(&p->partition_list); > + cl_list_destroy(&p->partition_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create() > +{ > + osm_qos_vlarb_scope_t *p = > + (osm_qos_vlarb_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); > + > + cl_list_init(&p->group_list, 10); > + cl_list_init(&p->across_list, 10); > + cl_list_init(&p->vlarb_high_list, 10); > + cl_list_init(&p->vlarb_low_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_vlarb_scope_destroy(osm_qos_vlarb_scope_t * p) > +{ > + if (!p) > + return; > + > + cl_list_apply_func(&p->group_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_list, __free_single_element, NULL); > + cl_list_apply_func(&p->vlarb_high_list, __free_single_element, NULL); > + cl_list_apply_func(&p->vlarb_low_list, __free_single_element, NULL); > + > + cl_list_remove_all(&p->group_list); > + cl_list_remove_all(&p->across_list); > + cl_list_remove_all(&p->vlarb_high_list); > + cl_list_remove_all(&p->vlarb_low_list); > + > + cl_list_destroy(&p->group_list); > + cl_list_destroy(&p->across_list); > + cl_list_destroy(&p->vlarb_high_list); > + cl_list_destroy(&p->vlarb_low_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create() > +{ > + osm_qos_sl2vl_scope_t *p = > + (osm_qos_sl2vl_scope_t *) malloc(sizeof(osm_qos_sl2vl_scope_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_vlarb_scope_t)); > + > + cl_list_init(&p->group_list, 10); > + cl_list_init(&p->across_from_list, 10); > + cl_list_init(&p->across_to_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_sl2vl_scope_destroy(osm_qos_sl2vl_scope_t * p) > +{ > + if (!p) > + return; > + > + cl_list_apply_func(&p->group_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_from_list, __free_single_element, NULL); > + cl_list_apply_func(&p->across_to_list, __free_single_element, NULL); > + > + cl_list_remove_all(&p->group_list); > + cl_list_remove_all(&p->across_from_list); > + cl_list_remove_all(&p->across_to_list); > + > + cl_list_destroy(&p->group_list); > + cl_list_destroy(&p->across_from_list); > + cl_list_destroy(&p->across_to_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_level_t *osm_qos_policy_qos_level_create() > +{ > + osm_qos_level_t *p = > + (osm_qos_level_t *) malloc(sizeof(osm_qos_level_t)); > + if (!p) > + return NULL; > + memset(p, 0, sizeof(osm_qos_level_t)); > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_qos_level_destroy(osm_qos_level_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->path_bits_range_len; i++) > + free(p->path_bits_range_arr[i]); > + if (p->path_bits_range_arr) > + free(p->path_bits_range_arr); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +boolean_t osm_qos_level_has_pkey(IN const osm_qos_level_t * p_qos_level, > + IN ib_net16_t pkey) > +{ > + if (!p_qos_level || !p_qos_level->pkey_range_len) > + return FALSE; > + return __is_num_in_range_arr(p_qos_level->pkey_range_arr, > + p_qos_level->pkey_range_len, > + cl_ntoh16(pkey)); > +} > + > +/*************************************************** > + ***************************************************/ > + > +ib_net16_t osm_qos_level_get_shared_pkey(IN const osm_qos_level_t * p_qos_level, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp) > +{ > + unsigned i; > + uint16_t pkey_ho = 0; > + > + if (!p_qos_level || !p_qos_level->pkey_range_len) > + return 0; > + > + /* > + * ToDo: This approach is not optimal. > + * Think how to find shared pkey that also exists > + * in QoS level in less runtime. > + */ When this "ToDo" will be addressed? > + > + for (i = 0; i < p_qos_level->pkey_range_len; i++) { > + for (pkey_ho = p_qos_level->pkey_range_arr[i][0]; > + pkey_ho <= p_qos_level->pkey_range_arr[i][1]; pkey_ho++) { > + if (osm_physp_share_this_pkey > + (p_src_physp, p_dest_physp, cl_hton16(pkey_ho))) > + return cl_hton16(pkey_ho); > + } > + } > + > + return 0; > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_match_rule_t *osm_qos_policy_match_rule_create() > +{ > + osm_qos_match_rule_t *p = > + (osm_qos_match_rule_t *) malloc(sizeof(osm_qos_match_rule_t)); > + if (!p) > + return NULL; > + > + memset(p, 0, sizeof(osm_qos_match_rule_t)); > + > + cl_list_init(&p->source_list, 10); > + cl_list_init(&p->source_group_list, 10); > + cl_list_init(&p->destination_list, 10); > + cl_list_init(&p->destination_group_list, 10); > + > + return p; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_match_rule_destroy(osm_qos_match_rule_t * p) > +{ > + unsigned i; > + > + if (!p) > + return; > + > + if (p->qos_level_name) > + free(p->qos_level_name); > + if (p->use) > + free(p->use); > + > + for (i = 0; i < p->service_id_range_len; i++) > + free(p->service_id_range_arr[i]); > + if (p->service_id_range_arr) > + free(p->service_id_range_arr); > + > + for (i = 0; i < p->qos_class_range_len; i++) > + free(p->qos_class_range_arr[i]); > + if (p->qos_class_range_arr) > + free(p->qos_class_range_arr); > + > + cl_list_apply_func(&p->source_list, __free_single_element, NULL); > + cl_list_remove_all(&p->source_list); > + cl_list_destroy(&p->source_list); > + > + cl_list_remove_all(&p->source_group_list); > + cl_list_destroy(&p->source_group_list); > + > + cl_list_apply_func(&p->destination_list, __free_single_element, NULL); > + cl_list_remove_all(&p->destination_list); > + cl_list_destroy(&p->destination_list); > + > + cl_list_remove_all(&p->destination_group_list); > + cl_list_destroy(&p->destination_group_list); > + > + free(p); > +} > + > +/*************************************************** > + ***************************************************/ > + > +osm_qos_policy_t * osm_qos_policy_create() > +{ > + osm_qos_policy_t * p_qos_policy = (osm_qos_policy_t *)malloc(sizeof(osm_qos_policy_t)); > + if (!p_qos_policy) > + return NULL; > + > + memset(p_qos_policy, 0, sizeof(osm_qos_policy_t)); > + > + cl_list_construct(&p_qos_policy->port_groups); > + cl_list_init(&p_qos_policy->port_groups, 10); > + > + cl_list_construct(&p_qos_policy->vlarb_tables); > + cl_list_init(&p_qos_policy->vlarb_tables, 10); > + > + cl_list_construct(&p_qos_policy->sl2vl_tables); > + cl_list_init(&p_qos_policy->sl2vl_tables, 10); > + > + cl_list_construct(&p_qos_policy->qos_levels); > + cl_list_init(&p_qos_policy->qos_levels, 10); > + > + cl_list_construct(&p_qos_policy->qos_match_rules); > + cl_list_init(&p_qos_policy->qos_match_rules, 10); > + > + return p_qos_policy; > +} > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_destroy(osm_qos_policy_t * p_qos_policy) > +{ > + cl_list_iterator_t list_iterator; > + osm_qos_port_group_t *p_port_group = NULL; > + osm_qos_vlarb_scope_t *p_vlarb_scope = NULL; > + osm_qos_sl2vl_scope_t *p_sl2vl_scope = NULL; > + osm_qos_level_t *p_qos_level = NULL; > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + > + if (!p_qos_policy) > + return; > + > + list_iterator = cl_list_head(&p_qos_policy->port_groups); > + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (p_port_group) > + osm_qos_policy_port_group_destroy(p_port_group); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->port_groups); > + cl_list_destroy(&p_qos_policy->port_groups); > + > + list_iterator = cl_list_head(&p_qos_policy->vlarb_tables); > + while (list_iterator != cl_list_end(&p_qos_policy->vlarb_tables)) { > + p_vlarb_scope = > + (osm_qos_vlarb_scope_t *) cl_list_obj(list_iterator); > + if (p_vlarb_scope) > + osm_qos_policy_vlarb_scope_destroy(p_vlarb_scope); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->vlarb_tables); > + cl_list_destroy(&p_qos_policy->vlarb_tables); > + > + list_iterator = cl_list_head(&p_qos_policy->sl2vl_tables); > + while (list_iterator != cl_list_end(&p_qos_policy->sl2vl_tables)) { > + p_sl2vl_scope = > + (osm_qos_sl2vl_scope_t *) cl_list_obj(list_iterator); > + if (p_sl2vl_scope) > + osm_qos_policy_sl2vl_scope_destroy(p_sl2vl_scope); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->sl2vl_tables); > + cl_list_destroy(&p_qos_policy->sl2vl_tables); > + > + list_iterator = cl_list_head(&p_qos_policy->qos_levels); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { > + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); > + if (p_qos_level) > + osm_qos_policy_qos_level_destroy(p_qos_level); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->qos_levels); > + cl_list_destroy(&p_qos_policy->qos_levels); > + > + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); > + if (p_qos_match_rule) > + osm_qos_policy_match_rule_destroy(p_qos_match_rule); > + list_iterator = cl_list_next(list_iterator); > + } > + cl_list_remove_all(&p_qos_policy->qos_match_rules); > + cl_list_destroy(&p_qos_policy->qos_match_rules); > + > + free(p_qos_policy); > + > + p_qos_policy = NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__qos_policy_is_port_in_group(osm_subn_t * p_subn, > + const osm_physp_t * p_physp, > + osm_qos_port_group_t * p_port_group) > +{ > + osm_node_t *p_node = osm_physp_get_node_ptr(p_physp); > + osm_prtn_t *p_prtn = NULL; > + ib_net64_t port_guid = osm_physp_get_port_guid(p_physp); > + uint64_t port_guid_ho = cl_ntoh64(port_guid); > + uint8_t node_type = osm_node_get_type(p_node); > + cl_list_iterator_t list_iterator; > + char *partition_name; > + > + /* check whether this port's type matches any of group's types */ > + > + if ((node_type == IB_NODE_TYPE_CA && p_port_group->node_type_ca) || > + (node_type == IB_NODE_TYPE_SWITCH && p_port_group->node_type_switch) > + || (node_type == IB_NODE_TYPE_ROUTER > + && p_port_group->node_type_router)) > + return TRUE; > + > + /* check whether this port's guid is in range of this group's guids */ > + > + if (__is_num_in_range_arr(p_port_group->guid_range_arr, > + p_port_group->guid_range_len, port_guid_ho)) > + return TRUE; > + > + /* check whether this port is member of this group's partitions */ > + > + list_iterator = cl_list_head(&p_port_group->partition_list); > + while (list_iterator != cl_list_end(&p_port_group->partition_list)) { > + partition_name = (char *)cl_list_obj(list_iterator); > + if (partition_name && strlen(partition_name)) { > + p_prtn = osm_prtn_find_by_name(p_subn, partition_name); > + if (p_prtn) { > + if (osm_prtn_is_guid(p_prtn, port_guid)) > + return TRUE; > + } > + } > + list_iterator = cl_list_next(list_iterator); > + } > + > + /* check whether this port's name matches any of group's names */ > + > + /* > + * TODO: check port names > + * > + * char desc[IB_NODE_DESCRIPTION_SIZE + 1]; > + * memcpy(desc, p_node->node_desc.description, IB_NODE_DESCRIPTION_SIZE); > + * desc[IB_NODE_DESCRIPTION_SIZE] = '\0'; > + */ > + > + return FALSE; > +} /* __qos_policy_is_port_in_group() */ > + > +/*************************************************** > + ***************************************************/ > + > +static boolean_t > +__qos_policy_is_port_in_group_list(const osm_pr_rcv_t * p_rcv, > + const osm_physp_t * p_physp, > + cl_list_t * p_port_group_list) > +{ > + osm_qos_port_group_t *p_port_group; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(p_port_group_list); > + while (list_iterator != cl_list_end(p_port_group_list)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (p_port_group) { > + if (__qos_policy_is_port_in_group > + (p_rcv->p_subn, p_physp, p_port_group)) > + return TRUE; > + } > + list_iterator = cl_list_next(list_iterator); > + } > + return FALSE; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_match_rule_t *__qos_policy_get_match_rule_by_pr( > + const osm_qos_policy_t * p_qos_policy, > + const osm_pr_rcv_t * p_rcv, > + const ib_path_rec_t * p_pr, > + const osm_physp_t * p_src_physp, > + const osm_physp_t * p_dest_physp, > + ib_net64_t comp_mask) > +{ > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + cl_list_iterator_t list_iterator; > + > + if (!cl_list_count(&p_qos_policy->qos_match_rules)) > + return NULL; > + > + /* Go over all QoS match rules and find the one that matches the request */ > + > + list_iterator = cl_list_head(&p_qos_policy->qos_match_rules); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) cl_list_obj(list_iterator); > + if (!p_qos_match_rule) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + /* If a match rule has Source groups, PR request source has to be in this list */ > + > + if (cl_list_count(&p_qos_match_rule->source_group_list)) { > + if (!__qos_policy_is_port_in_group_list(p_rcv, > + p_src_physp, > + &p_qos_match_rule-> > + source_group_list)) > + { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + } > + > + /* If a match rule has Destination groups, PR request dest. has to be in this list */ > + > + if (cl_list_count(&p_qos_match_rule->destination_group_list)) { > + if (!__qos_policy_is_port_in_group_list(p_rcv, > + p_dest_physp, > + &p_qos_match_rule-> > + destination_group_list)) > + { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + } > + > + /* If a match rule has QoS classes, PR request HAS > + to have a matching QoS class to match the rule */ > + > + if (p_qos_match_rule->qos_class_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_QOS_CLASS)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->qos_class_range_arr, > + p_qos_match_rule->qos_class_range_len, > + ib_path_rec_qos_class(p_pr))) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* If a match rule has Service IDs, PR request HAS > + to have a matching Service ID to match the rule */ > + > + if (p_qos_match_rule->service_id_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_SERVICEID)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->service_id_range_arr, > + p_qos_match_rule->service_id_range_len, > + p_pr->service_id)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* If a match rule has PKeys, PR request HAS > + to have a matching PKey to match the rule */ > + > + if (p_qos_match_rule->pkey_range_len) { > + if (!(comp_mask & IB_PR_COMPMASK_PKEY)) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + if (!__is_num_in_range_arr > + (p_qos_match_rule->pkey_range_arr, > + p_qos_match_rule->pkey_range_len, > + ib_path_rec_qos_class(p_pr))) { > + list_iterator = cl_list_next(list_iterator); > + continue; > + } > + > + } > + > + /* if we got here, then this match-rule matched this PR request */ > + break; > + } > + > + if (list_iterator == cl_list_end(&p_qos_policy->qos_match_rules)) > + return NULL; > + > + return p_qos_match_rule; > +} /* __qos_policy_get_match_rule_by_pr() */ > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_level_t *__qos_policy_get_qos_level_by_name(osm_qos_policy_t * p_qos_policy, > + char *name) > +{ > + osm_qos_level_t *p_qos_level = NULL; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(&p_qos_policy->qos_levels); > + while (list_iterator != cl_list_end(&p_qos_policy->qos_levels)) { > + p_qos_level = (osm_qos_level_t *) cl_list_obj(list_iterator); > + if (!p_qos_level) > + continue; > + > + /* names are case INsensitive */ > + if (strcasecmp(name, p_qos_level->name) == 0) > + return p_qos_level; > + > + list_iterator = cl_list_next(list_iterator); > + } > + > + return NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +static osm_qos_port_group_t *__qos_policy_get_port_group_by_name(osm_qos_policy_t * p_qos_policy, > + const char *const name) > +{ > + osm_qos_port_group_t *p_port_group = NULL; > + cl_list_iterator_t list_iterator; > + > + list_iterator = cl_list_head(&p_qos_policy->port_groups); > + while (list_iterator != cl_list_end(&p_qos_policy->port_groups)) { > + p_port_group = > + (osm_qos_port_group_t *) cl_list_obj(list_iterator); > + if (!p_port_group) > + continue; > + > + /* names are case INsensitive */ > + if (strcasecmp(name, p_port_group->name) == 0) > + return p_port_group; > + > + list_iterator = cl_list_next(list_iterator); > + } > + > + return NULL; > +} > + > +/*************************************************** > + ***************************************************/ > + > +int osm_qos_policy_validate(osm_qos_policy_t * p_qos_policy, > + osm_log_t *p_log) > +{ > + cl_list_iterator_t match_rules_list_iterator; > + cl_list_iterator_t list_iterator; > + osm_qos_port_group_t *p_port_group = NULL; > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + char *str; > + unsigned i; > + int res = 0; > + > + OSM_LOG_ENTER(p_log, osm_qos_policy_validate); > + > + /* set default qos level */ > + > + p_qos_policy->p_default_qos_level = > + __qos_policy_get_qos_level_by_name(p_qos_policy, OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); > + if (!p_qos_policy->p_default_qos_level) { > + osm_log(p_log, OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC10: " > + "Default qos-level (%s) not defined.\n", > + OSM_QOS_POLICY_DEFAULT_LEVEL_NAME); > + res = 1; > + goto Exit; > + } > + > + /* scan all the match rules, and fill the lists of pointers to > + relevant qos levels and port groups to speed up PR matching */ > + > + i = 1; > + match_rules_list_iterator = > + cl_list_head(&p_qos_policy->qos_match_rules); > + while (match_rules_list_iterator != > + cl_list_end(&p_qos_policy->qos_match_rules)) { > + p_qos_match_rule = > + (osm_qos_match_rule_t *) > + cl_list_obj(match_rules_list_iterator); > + CL_ASSERT(p_qos_match_rule); > + > + /* find the matching qos-level for each match-rule */ > + > + p_qos_match_rule->p_qos_level = > + __qos_policy_get_qos_level_by_name(p_qos_policy, > + p_qos_match_rule->qos_level_name); > + > + if (!p_qos_match_rule->p_qos_level) { > + osm_log(p_log, OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC11: " > + "qos-match-rule num %u: qos-level '%s' not found\n", > + i, p_qos_match_rule->qos_level_name); > + res = 1; > + goto Exit; > + } > + > + /* find the matching port-group for element of source_list */ > + > + if (cl_list_count(&p_qos_match_rule->source_list)) { > + list_iterator = > + cl_list_head(&p_qos_match_rule->source_list); > + while (list_iterator != > + cl_list_end(&p_qos_match_rule->source_list)) { > + str = (char *)cl_list_obj(list_iterator); > + CL_ASSERT(str); > + > + p_port_group = > + __qos_policy_get_port_group_by_name(p_qos_policy, str); > + if (!p_port_group) { > + osm_log(p_log, > + OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC12: " > + "qos-match-rule num %u: source port-group '%s' not found\n", > + i, str); > + res = 1; > + goto Exit; > + } > + > + cl_list_insert_tail(&p_qos_match_rule-> > + source_group_list, > + p_port_group); > + > + list_iterator = cl_list_next(list_iterator); > + } > + } > + > + /* find the matching port-group for element of destination_list */ > + > + if (cl_list_count(&p_qos_match_rule->destination_list)) { > + list_iterator = > + cl_list_head(&p_qos_match_rule->destination_list); > + while (list_iterator != > + cl_list_end(&p_qos_match_rule-> > + destination_list)) { > + str = (char *)cl_list_obj(list_iterator); > + CL_ASSERT(str); > + > + p_port_group = > + __qos_policy_get_port_group_by_name(p_qos_policy,str); > + if (!p_port_group) { > + osm_log(p_log, > + OSM_LOG_ERROR, > + "osm_qos_policy_validate: ERR AC13: " > + "qos-match-rule num %u: destination port-group '%s' not found\n", > + i, str); > + res = 1; > + goto Exit; > + } > + > + cl_list_insert_tail(&p_qos_match_rule-> > + destination_group_list, > + p_port_group); > + > + list_iterator = cl_list_next(list_iterator); > + } > + } > + > + /* done with the current match-rule */ > + > + match_rules_list_iterator = > + cl_list_next(match_rules_list_iterator); > + i++; > + } > + > + Exit: > + OSM_LOG_EXIT(p_log); > + return res; > +} /* osm_qos_policy_validate() */ > + > +/*************************************************** > + ***************************************************/ > + > +void osm_qos_policy_get_qos_level_by_pr(IN const osm_qos_policy_t * p_qos_policy, > + IN const osm_pr_rcv_t * p_rcv, > + IN const ib_path_rec_t * p_pr, > + IN const osm_physp_t * p_src_physp, > + IN const osm_physp_t * p_dest_physp, > + IN ib_net64_t comp_mask, > + OUT osm_qos_level_t ** pp_qos_level) > +{ > + osm_qos_match_rule_t *p_qos_match_rule = NULL; > + osm_qos_level_t *p_qos_level = NULL; > + > + OSM_LOG_ENTER(p_rcv->p_log, osm_qos_policy_get_qos_level_by_pr); > + > + *pp_qos_level = NULL; > + > + if (!p_qos_policy) > + goto Exit; > + > + p_qos_match_rule = __qos_policy_get_match_rule_by_pr(p_qos_policy, > + p_rcv, > + p_pr, > + p_src_physp, > + p_dest_physp, > + comp_mask); > + > + if (p_qos_match_rule) > + p_qos_level = p_qos_match_rule->p_qos_level; > + else > + p_qos_level = p_qos_policy->p_default_qos_level; > + > + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, > + "osm_qos_policy_get_qos_level_by_pr: " > + "PathRecord request:" > + "Src port 0x%016" PRIx64 ", " > + "Dst port 0x%016" PRIx64 "\n", > + cl_ntoh64(osm_physp_get_port_guid(p_src_physp)), > + cl_ntoh64(osm_physp_get_port_guid(p_dest_physp))); > + osm_log(p_rcv->p_log, OSM_LOG_DEBUG, > + "osm_qos_policy_get_qos_level_by_pr: " > + "Applying QoS Level %s (%s)\n", > + p_qos_level->name, > + (p_qos_level->use) ? p_qos_level->use : "no description"); > + > + *pp_qos_level = p_qos_level; > + > + Exit: > + OSM_LOG_EXIT(p_rcv->p_log); > +} /* osm_qos_policy_get_qos_level_by_pr() */ > + > +/*************************************************** > + ***************************************************/ > -- > 1.5.1.4 > From hal.rosenstock at gmail.com Tue Aug 28 06:45:01 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 09:45:01 -0400 Subject: [ofa-general] ibdiags and gcc Message-ID: Hi Eitan, Several of the ibdiags (ibdiagnet, ibdiagpath, ibdiagui) appear to require gcc not just in the build environment but also in the execution environment due to the following line in those scripts: xxx_path=`gcc --print-file-name ... to determine some library paths. Any reason LD_LIBRARY_PATH or something more standard can't be used for this ? -- Hal From sashak at voltaire.com Tue Aug 28 06:51:24 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:51:24 +0300 Subject: [ofa-general] Re: [PATCH 4/7 V3] osm: QoS - rename static find_prtn_by_name() to non-static In-Reply-To: <46D359EC.6040904@dev.mellanox.co.il> References: <46D359EC.6040904@dev.mellanox.co.il> Message-ID: <20070828135124.GE18082@sashak.voltaire.com> On 02:10 Tue 28 Aug , Yevgeny Kliteynik wrote: > Renaming static find_prtn_by_name() to non-static > osm_prtn_find_by_name() - this function is used by > QoS policy functions. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Tue Aug 28 06:56:57 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:56:57 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D35A03.2070101@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> Message-ID: <20070828135657.GF18082@sashak.voltaire.com> On 02:10 Tue 28 Aug , Yevgeny Kliteynik wrote: > Adding QoS policy file option to OpenSM optionsm and > QoS Policy field to subn object. > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Some comments are below. Sasha > --- > opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > opensm/opensm/main.c | 12 +++++++++++- > opensm/opensm/osm_subnet.c | 10 +++++++++- > 4 files changed, 50 insertions(+), 2 deletions(-) > > diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h > index 545779b..0a35e22 100644 > --- a/opensm/include/opensm/osm_base.h > +++ b/opensm/include/opensm/osm_base.h > @@ -224,6 +224,23 @@ BEGIN_C_DECLS > #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" > #endif > /***********/ > + > +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > +* NAME > +* OSM_DEFAULT_QOS_POLICY_FILE > +* > +* DESCRIPTION > +* Specifies the default QoS policy file name > +* > +* SYNOPSIS > +*/ > +#ifdef __WIN__ > +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") > +#else > +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" > +#endif > +/***********/ > + > /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > * NAME > * OSM_DEFAULT_SWEEP_INTERVAL_SECS > diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > index a7543dc..5e8b522 100644 > --- a/opensm/include/opensm/osm_subnet.h > +++ b/opensm/include/opensm/osm_subnet.h > @@ -68,6 +68,7 @@ BEGIN_C_DECLS > #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > #define OSM_SUBNET_VECTOR_CAPACITY 256 > struct _osm_opensm_t; > +struct _osm_qos_policy_t; > > /****h* OpenSM/Subnet > * NAME > @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > char *partition_config_file; > boolean_t no_partition_enforcement; > boolean_t no_qos; > + char *qos_policy_file; > boolean_t accum_log_file; > char *console; > uint16_t console_port; > @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > * specified the log file will be truncated upon reaching > * this limit. > * > +* no_qos > +* Boolean that specifies whether the OpenSM QoS functionality > +* should be off or on. > +* > +* qos_policy_file > +* Name of the QoS policy file. > +* > * accum_log_file > * If TRUE (default) - the log file will be accumulated. > * If FALSE - the log file will be erased before starting current opensm run. > @@ -551,6 +560,7 @@ typedef struct _osm_subn { > ib_net64_t sm_port_guid; > uint8_t sm_state; > osm_subn_opt_t opt; > + struct _osm_qos_policy_t *p_qos_policy; > uint16_t max_unicast_lid_ho; > uint16_t max_multicast_lid_ho; > uint8_t min_ca_mtu; > @@ -619,6 +629,9 @@ typedef struct _osm_subn { > * opt > * Subnet options structure contains site specific configuration. > * > +* p_qos_policy > +* Subnet QoS policy structure. > +* I guess this should be destroyed at osm_subn cleanup. > * max_unicast_lid_ho > * The minimal max unicast lid reported by all switches > * > diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > index e2541d8..599c4dc 100644 > --- a/opensm/opensm/main.c > +++ b/opensm/opensm/main.c > @@ -269,6 +269,10 @@ void show_usage(void) > " The default name is \'" > OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > + printf("-Y\n" > + "--qos_policy_file\n" > + " This option defines the optional QoS policy file.\n" > + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > printf("-N\n" > "--no_part_enforce\n" > " This option disables partition enforcement on switch external ports.\n\n"); > @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > char *ignore_guids_file_name = NULL; > uint32_t val; > const char *const short_option = > - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > /* > In the array below, the 2nd parameter specifies the number > @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > {"Pconfig", 1, NULL, 'P'}, > {"no_part_enforce", 0, NULL, 'N'}, > {"qos", 0, NULL, 'Q'}, > + {"qos_policy_file", 1, NULL, 'Y'}, > {"maxsmps", 1, NULL, 'n'}, > {"console", 1, NULL, 'q'}, > {"V", 0, NULL, 'V'}, > @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > opt.no_qos = FALSE; > break; > > + case 'Y': > + opt.qos_policy_file = optarg; > + printf(" QoS policy file \'%s\'\n", optarg); > + break; > + > case 'y': > opt.exit_on_fatal = FALSE; > printf(" Staying on fatal initialization errors\n"); > diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > index 818d73f..4162522 100644 > --- a/opensm/opensm/osm_subnet.c > +++ b/opensm/opensm/osm_subnet.c > @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > p_opt->no_partition_enforcement = FALSE; > p_opt->no_qos = TRUE; > + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > p_opt->accum_log_file = TRUE; > p_opt->port_profile_switch_nodes = FALSE; > p_opt->pfn_ui_pre_lid_assign = NULL; > @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > > opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > > + opts_unpack_charp("qos_policy_file", > + p_key, p_val, &p_opts->qos_policy_file); > + > opts_unpack_boolean("accum_log_file", > p_key, p_val, &p_opts->accum_log_file); > > @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > fprintf(opts_file, > "#\n# QoS OPTIONS\n#\n" > "# Disable QoS setup\n" > - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > + "no_qos %s\n\n" > + "# QoS policy file to be used\n" > + "qos_policy_file %s\n\n", > + p_opts->no_qos ? "TRUE" : "FALSE", > + p_opts->qos_policy_file); > > subn_dump_qos_options(opts_file, > "QoS default options", "qos", > -- > 1.5.1.4 > From sashak at voltaire.com Tue Aug 28 06:57:50 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:57:50 +0300 Subject: [ofa-general] Re: [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <46D35A20.6020902@dev.mellanox.co.il> References: <46D35A20.6020902@dev.mellanox.co.il> Message-ID: <20070828135750.GG18082@sashak.voltaire.com> On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: > Compiling QoS policy file parser. > > C & H files are generated from Lex & Yacc files and the > generated files are compiled. > Generated files are not included in git, but they do get > included in RPM as they are also created by 'make dist'. > Generated files can be removed by "make maintainer-clean". > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. As discussed I will remove lex and yacc deps. Sasha From sashak at voltaire.com Tue Aug 28 06:59:58 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 16:59:58 +0300 Subject: [ofa-general] Re: [PATCH 7/7 V3] osm: QoS - reading policy file In-Reply-To: <46D35A32.6070003@dev.mellanox.co.il> References: <46D35A32.6070003@dev.mellanox.co.il> Message-ID: <20070828135958.GH18082@sashak.voltaire.com> On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: > Reading QoS policy file > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Some comments are below. > --- > opensm/include/opensm/osm_subnet.h | 16 ++++++++-------- > opensm/opensm/osm_state_mgr.c | 2 +- > opensm/opensm/osm_subnet.c | 17 +++++++++++------ > 3 files changed, 20 insertions(+), 15 deletions(-) > > diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > index 5e8b522..7e1a3e7 100644 > --- a/opensm/include/opensm/osm_subnet.h > +++ b/opensm/include/opensm/osm_subnet.h > @@ -1118,28 +1118,28 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opt); > * Subnet object, osm_subn_construct, osm_subn_destroy > *********/ > > -/****f* OpenSM: Subnet/osm_subn_parse_conf_file > +/****f* OpenSM: Subnet/osm_subn_parse_conf_files > * NAME > -* osm_subn_rescan_conf_file > +* osm_subn_rescan_conf_files > * > * DESCRIPTION > -* The osm_subn_rescan_conf_file function parses the configuration > -* file and update selected subnet options > +* The osm_subn_rescan_conf_files function parses the configuration > +* files and update selected subnet options > * > * SYNOPSIS > */ > -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts); > +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn); > /* > * PARAMETERS > * > -* p_opt > -* [in] Pointer to the subnet options structure. > +* p_subn > +* [in] Pointer to the subnet structure. > * > * RETURN VALUES > * IB_SUCCESS, IB_ERROR > * > * NOTES > -* This uses the same file as osm_subn_parse_conf_file() > +* This uses the same file as osm_subn_parse_conf_files() > * > *********/ > > diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c > index 030d344..2a94518 100644 > --- a/opensm/opensm/osm_state_mgr.c > +++ b/opensm/opensm/osm_state_mgr.c > @@ -1878,7 +1878,7 @@ void osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, > p_mgr->p_subn->subnet_initialization_error = FALSE; > > /* rescan configuration updates */ > - status = osm_subn_rescan_conf_file(&p_mgr->p_subn->opt); > + status = osm_subn_rescan_conf_files(p_mgr->p_subn); > if (status != IB_SUCCESS) { > osm_log(p_mgr->p_log, > OSM_LOG_ERROR, > diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > index 4162522..c847daf 100644 > --- a/opensm/opensm/osm_subnet.c > +++ b/opensm/opensm/osm_subnet.c > @@ -69,6 +69,7 @@ > #include > #include > #include > +#include > > #if defined(PATH_MAX) > #define OSM_PATH_MAX (PATH_MAX + 1) > @@ -677,7 +678,7 @@ subn_dump_qos_options(FILE * file, > > /********************************************************************** > **********************************************************************/ > -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) > +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn) > { > char *p_cache_dir = getenv("OSM_CACHE_DIR"); > char file_name[OSM_PATH_MAX]; > @@ -704,28 +705,32 @@ ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) > > subn_parse_qos_options("qos", > p_key, p_val, > - &p_opts->qos_options); > + &p_subn->opt.qos_options); > > subn_parse_qos_options("qos_ca", > p_key, p_val, > - &p_opts->qos_ca_options); > + &p_subn->opt.qos_ca_options); > > subn_parse_qos_options("qos_sw0", > p_key, p_val, > - &p_opts->qos_sw0_options); > + &p_subn->opt.qos_sw0_options); > > subn_parse_qos_options("qos_swe", > p_key, p_val, > - &p_opts->qos_swe_options); > + &p_subn->opt.qos_swe_options); > > subn_parse_qos_options("qos_rtr", > p_key, p_val, > - &p_opts->qos_rtr_options); > + &p_subn->opt.qos_rtr_options); > > } > } > fclose(opts_file); > > + /* read QoS policy config file */ > + if (!p_subn->opt.no_qos) > + osm_qos_parse_policy_file(p_subn); > + Should the qos parser return value be checked and an error logged in case of failure? Sasha > return IB_SUCCESS; > } > > -- > 1.5.1.4 > From panda at cse.ohio-state.edu Tue Aug 28 07:12:01 2007 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Tue, 28 Aug 2007 10:12:01 -0400 (EDT) Subject: [ofa-general] Re: [ewg] OFED Aug 27 meeting summary on OFED 1.3 development status In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> from "Tziporet Koren" at Aug 28, 2007 08:58:22 AM Message-ID: <200708281412.l7SEC1kk006041@xi.cse.ohio-state.edu> Tziporet, > OFED Aug 13 meeting summary on OFED 1.3 development status: > > 1. Decided to delay the feature freeze to Sep 10. > * MPI update: > * Open MPI: will have some intermediate package for OFED > * Mvapich: first OFED package will start with version 0.99 (that > was part of 1.2.5). The package of the new release will be ready by end > of Sep. > * Mvapich2: new release 1.2 will be ready on time for the feature > freeze. ^^^ This should be 1.0 version (not 1.2). We already have MVAPICH2 1.0-beta available for the last 4 weeks. We are in our final testing phase. We are aiming to have the final release by the feature freeze of Sept 10th. Please correct this in your note. Thanks, DK From kliteyn at dev.mellanox.co.il Tue Aug 28 07:49:46 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 17:49:46 +0300 Subject: [ofa-general] [PATCH] osm: QoS - remove 'yyunput' compilation warning Message-ID: <46D4360A.7090602@dev.mellanox.co.il> Remove 'yyunput defined but not used' compilation warning Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_qos_parser.l | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_qos_parser.l b/opensm/opensm/osm_qos_parser.l index bc03123..0b096f8 100644 --- a/opensm/opensm/osm_qos_parser.l +++ b/opensm/opensm/osm_qos_parser.l @@ -109,6 +109,8 @@ static void reset_new_line_flags(); %} +%option nounput + PORT_GROUPS_START port\-groups PORT_GROUPS_END end\-port\-groups PORT_GROUP_START port\-group -- 1.5.1.4 From kliteyn at dev.mellanox.co.il Tue Aug 28 08:00:11 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 18:00:11 +0300 Subject: [ofa-general] Re: [PATCH 7/7 V3] osm: QoS - reading policy file In-Reply-To: <20070828135958.GH18082@sashak.voltaire.com> References: <46D35A32.6070003@dev.mellanox.co.il> <20070828135958.GH18082@sashak.voltaire.com> Message-ID: <46D4387B.3000704@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: >> Reading QoS policy file >> >> Signed-off-by: Yevgeny Kliteynik > > Applied. Thanks. > > Some comments are below. > >> --- >> opensm/include/opensm/osm_subnet.h | 16 ++++++++-------- >> opensm/opensm/osm_state_mgr.c | 2 +- >> opensm/opensm/osm_subnet.c | 17 +++++++++++------ >> 3 files changed, 20 insertions(+), 15 deletions(-) >> >> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h >> index 5e8b522..7e1a3e7 100644 >> --- a/opensm/include/opensm/osm_subnet.h >> +++ b/opensm/include/opensm/osm_subnet.h >> @@ -1118,28 +1118,28 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opt); >> * Subnet object, osm_subn_construct, osm_subn_destroy >> *********/ >> >> -/****f* OpenSM: Subnet/osm_subn_parse_conf_file >> +/****f* OpenSM: Subnet/osm_subn_parse_conf_files >> * NAME >> -* osm_subn_rescan_conf_file >> +* osm_subn_rescan_conf_files >> * >> * DESCRIPTION >> -* The osm_subn_rescan_conf_file function parses the configuration >> -* file and update selected subnet options >> +* The osm_subn_rescan_conf_files function parses the configuration >> +* files and update selected subnet options >> * >> * SYNOPSIS >> */ >> -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts); >> +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn); >> /* >> * PARAMETERS >> * >> -* p_opt >> -* [in] Pointer to the subnet options structure. >> +* p_subn >> +* [in] Pointer to the subnet structure. >> * >> * RETURN VALUES >> * IB_SUCCESS, IB_ERROR >> * >> * NOTES >> -* This uses the same file as osm_subn_parse_conf_file() >> +* This uses the same file as osm_subn_parse_conf_files() >> * >> *********/ >> >> diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c >> index 030d344..2a94518 100644 >> --- a/opensm/opensm/osm_state_mgr.c >> +++ b/opensm/opensm/osm_state_mgr.c >> @@ -1878,7 +1878,7 @@ void osm_state_mgr_process(IN osm_state_mgr_t * const p_mgr, >> p_mgr->p_subn->subnet_initialization_error = FALSE; >> >> /* rescan configuration updates */ >> - status = osm_subn_rescan_conf_file(&p_mgr->p_subn->opt); >> + status = osm_subn_rescan_conf_files(p_mgr->p_subn); >> if (status != IB_SUCCESS) { >> osm_log(p_mgr->p_log, >> OSM_LOG_ERROR, >> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c >> index 4162522..c847daf 100644 >> --- a/opensm/opensm/osm_subnet.c >> +++ b/opensm/opensm/osm_subnet.c >> @@ -69,6 +69,7 @@ >> #include >> #include >> #include >> +#include >> >> #if defined(PATH_MAX) >> #define OSM_PATH_MAX (PATH_MAX + 1) >> @@ -677,7 +678,7 @@ subn_dump_qos_options(FILE * file, >> >> /********************************************************************** >> **********************************************************************/ >> -ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) >> +ib_api_status_t osm_subn_rescan_conf_files(IN osm_subn_t * const p_subn) >> { >> char *p_cache_dir = getenv("OSM_CACHE_DIR"); >> char file_name[OSM_PATH_MAX]; >> @@ -704,28 +705,32 @@ ib_api_status_t osm_subn_rescan_conf_file(IN osm_subn_opt_t * const p_opts) >> >> subn_parse_qos_options("qos", >> p_key, p_val, >> - &p_opts->qos_options); >> + &p_subn->opt.qos_options); >> >> subn_parse_qos_options("qos_ca", >> p_key, p_val, >> - &p_opts->qos_ca_options); >> + &p_subn->opt.qos_ca_options); >> >> subn_parse_qos_options("qos_sw0", >> p_key, p_val, >> - &p_opts->qos_sw0_options); >> + &p_subn->opt.qos_sw0_options); >> >> subn_parse_qos_options("qos_swe", >> p_key, p_val, >> - &p_opts->qos_swe_options); >> + &p_subn->opt.qos_swe_options); >> >> subn_parse_qos_options("qos_rtr", >> p_key, p_val, >> - &p_opts->qos_rtr_options); >> + &p_subn->opt.qos_rtr_options); >> >> } >> } >> fclose(opts_file); >> >> + /* read QoS policy config file */ >> + if (!p_subn->opt.no_qos) >> + osm_qos_parse_policy_file(p_subn); >> + > > Should the qos parser return value be checked and an error logged in > case of failure? The parser logs the errors (whether the policy file is not found, or there is a syntax error, or any other error). If some error occurred, the p_subn->p_qos_policy will be null. Do you think I should add status checking here as well? I think it would be redundant. -- Yevgeny > Sasha > >> return IB_SUCCESS; >> } >> >> -- >> 1.5.1.4 >> > From kliteyn at dev.mellanox.co.il Tue Aug 28 08:06:23 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 18:06:23 +0300 Subject: [ofa-general] Re: [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <20070828135750.GG18082@sashak.voltaire.com> References: <46D35A20.6020902@dev.mellanox.co.il> <20070828135750.GG18082@sashak.voltaire.com> Message-ID: <46D439EF.4070200@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: >> Compiling QoS policy file parser. >> >> C & H files are generated from Lex & Yacc files and the >> generated files are compiled. >> Generated files are not included in git, but they do get >> included in RPM as they are also created by 'make dist'. >> Generated files can be removed by "make maintainer-clean". >> >> Signed-off-by: Yevgeny Kliteynik > > Applied. Thanks. As discussed I will remove lex and yacc deps. After removing these deps, Lex fails on my machine. Apparently, with these flags make uses "flex", and w/o them it uses "lex". I'll look into this, but in the mean time please put these flags back as OpenSM compilation is now broken. > Sasha > From kliteyn at dev.mellanox.co.il Tue Aug 28 08:11:35 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Tue, 28 Aug 2007 18:11:35 +0300 Subject: [ofa-general] [PATCH] osm: QoS - destroy qos_policy when destroying subnet Message-ID: <46D43B27.5040808@dev.mellanox.co.il> Destroy p_subn->p_qos_policy when destroying subnet Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/osm_subnet.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c index c847daf..58803e1 100644 --- a/opensm/opensm/osm_subnet.c +++ b/opensm/opensm/osm_subnet.c @@ -167,6 +167,8 @@ void osm_subn_destroy(IN osm_subn_t * const p_subn) cl_ptr_vector_destroy(&p_subn->port_lid_tbl); cl_map_remove_all(&(p_subn->opt.port_prof_ignore_guids)); cl_map_destroy(&(p_subn->opt.port_prof_ignore_guids)); + + osm_qos_policy_destroy(p_subn->p_qos_policy); } /********************************************************************** -- 1.5.1.4 From sashak at voltaire.com Tue Aug 28 08:17:05 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 18:17:05 +0300 Subject: [ofa-general] Re: [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <46D439EF.4070200@dev.mellanox.co.il> References: <46D35A20.6020902@dev.mellanox.co.il> <20070828135750.GG18082@sashak.voltaire.com> <46D439EF.4070200@dev.mellanox.co.il> Message-ID: <20070828151705.GO18082@sashak.voltaire.com> On 18:06 Tue 28 Aug , Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: > >> Compiling QoS policy file parser. > >> > >> C & H files are generated from Lex & Yacc files and the > >> generated files are compiled. > >> Generated files are not included in git, but they do get > >> included in RPM as they are also created by 'make dist'. > >> Generated files can be removed by "make maintainer-clean". > >> > >> Signed-off-by: Yevgeny Kliteynik > > Applied. Thanks. As discussed I will remove lex and yacc deps. > > After removing these deps, Lex fails on my machine. Yes, as well as 'yacc' - the comman line is 'bison' specific. > Apparently, with these flags make uses "flex", and w/o them it uses "lex". > I'll look into this, but in the mean time please put these flags back as > OpenSM compilation is now broken. Ok, I will put this back for now, but lex/yacc should be improved. Sasha From sashak at voltaire.com Tue Aug 28 08:23:12 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Tue, 28 Aug 2007 18:23:12 +0300 Subject: [ofa-general] Re: [PATCH 7/7 V3] osm: QoS - reading policy file In-Reply-To: <46D4387B.3000704@dev.mellanox.co.il> References: <46D35A32.6070003@dev.mellanox.co.il> <20070828135958.GH18082@sashak.voltaire.com> <46D4387B.3000704@dev.mellanox.co.il> Message-ID: <20070828152312.GP18082@sashak.voltaire.com> On 18:00 Tue 28 Aug , Yevgeny Kliteynik wrote: > >> } > >> fclose(opts_file); > >> > >> + /* read QoS policy config file */ > >> + if (!p_subn->opt.no_qos) > >> + osm_qos_parse_policy_file(p_subn); > >> + > > Should the qos parser return value be checked and an error logged in > > case of failure? > > The parser logs the errors (whether the policy file is not found, or there > is > a syntax error, or any other error). > If some error occurred, the p_subn->p_qos_policy will be null. > Do you think I should add status checking here as well? > I think it would be redundant. Ok. Sasha From Nathan.Dauchy at noaa.gov Tue Aug 28 08:35:46 2007 From: Nathan.Dauchy at noaa.gov (Nathan Dauchy) Date: Tue, 28 Aug 2007 09:35:46 -0600 Subject: [ofa-general] OFED Aug 27 meeting summary on OFED 1.3 development status In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> Message-ID: <46D440D2.3050904@noaa.gov> Quick feedback from the field... Tziporet Koren wrote: > OFED Aug 13 meeting summary on OFED 1.3 development status: > ================================================= > > 2. New install: > > o Explained the feature to build without installation is not > available any more. > Reasons for change were that working in the previous mode > was against rpm native usage and caused many issues to other > packages (like MPI). > In addition it caused OFED spec files and package to be not > useful for the distros. Perhaps I don't understand all the nuances of creating official packages for distros, but does this mean that the RPM build and install will have to be done in one step? That would make life MUCH more difficult for sites like ours that have a diskless cluster, where the nodes running OFED and where the compile is done do not have write access to the image. We must build one place and install (with chroot or 'rpm --root') in another. Separating build and install worked very well for us in the past. Thanks for your work and consideration, Nathan From hal.rosenstock at gmail.com Tue Aug 28 08:47:19 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 11:47:19 -0400 Subject: [ofa-general] OpenSM QoS build failure Message-ID: Yevgeny, I get the following failure with the latest OpenSM master: if gcc -DHAVE_CONFIG_H -I. -I. -I. -I./../include -I./../../libibcommon/include/infiniband -I./../../libibumad/include/infiniband -Wall -DOSM_VENDOR_INTF_OPENIB -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP -ggdb -D_DEBUG_ -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 -MT opensm-osm_qos_parser_y.o -MD -MP -MF ".deps/opensm-osm_qos_parser_y.Tpo" -c -o opensm-osm_qos_parser_y.o `test -f 'osm_qos_parser_y.c' || echo './'`osm_qos_parser_y.c; \ then mv -f ".deps/opensm-osm_qos_parser_y.Tpo" ".deps/opensm-osm_qos_parser_y.Po"; else rm -f ".deps/opensm-osm_qos_parser_y.Tpo"; exit 1; fi flex --prefix=__qos_parser_ -o ./osm_qos_parser_l.c ./osm_qos_parser.l flex: unknown flag '-'. For usage, try flex --help make[2]: *** [osm_qos_parser_l.c] Error 1 I have flex version 2.5.4. Seems like it doesn't like --prefix. Should this be -P ? -- Hal From hal.rosenstock at gmail.com Tue Aug 28 08:48:14 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 11:48:14 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> Message-ID: Hi Eitan, On 8/28/07, Eitan Zahavi wrote: > Hi Hal, > > As you pointed out, coming up with a VLArb and SL2VL manager that is > supporting both routing and QoS is not trivial. > But even with a uniform and statically assigned tables (supported by the > current options config file) different QoS classes can be provided to > different clients by the new QoS policy code. > > I agree our OFED 1.3 is somewhat limited as it does not solve the VLArb > and SL2VL setting issue. > But with the time constrains of the development, review and QA this is > the most we could do for 1.3 How much of the QoS policy syntax is left "dormant" by this ? How much is actually used ? Can this be detailed somewhere ? Also, there was a previous question on the list on IPoIB CM relative to QoS which I either missed the answer to or was left unanswered: How is IPoIB CM dealt with in terms of QoS ? Is it like IPoIB UD ? What documentation will be provided (e.g. man page and perhaps other doc file for this as it is quite complex) ? It would be nice to see the complete picture but it seems like this will be a set of incremental changes for which some looking back over the entirety will be needed when things settle down. -- Hal > EZ > > Eitan Zahavi > Senior Engineering Director, Software Architect > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > > Hal Rosenstock > > Sent: Monday, August 27, 2007 7:33 PM > > To: Yevgeny Kliteynik > > Cc: OpenIB > > Subject: Re: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy > > C & H files > > > > On 8/23/07, Yevgeny Kliteynik wrote: > > > Hi Sasha, > > > > > > Sasha Khapyorsky wrote: > > > > On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_port_group_t_ { > > > >>>> + char *name; /* single string (this port > > group name) */ > > > >>>> + char *use; /* single string (description) */ > > > >>>> + cl_list_t port_name_list; /* list of port > > names (.../.../...) */ > > > >>>> + uint64_t **guid_range_arr; /* array of guid > > ranges (pair of 64-bit > > > >>>> guids) */ > > > >>>> + unsigned guid_range_len; /* num of guid > > ranges in the array */ > > > >>>> + cl_list_t partition_list; /* list of > > partition names */ > > > >>>> + boolean_t node_type_ca; > > > >>>> + boolean_t node_type_switch; > > > >>>> + boolean_t node_type_router; > > > >>>> + boolean_t node_type_self; > > > >>>> +} osm_qos_port_group_t; > > > >>> I see you are using this in "run-time", not just during > > the parsing. > > > >>> Instead of having all this config features you can just resolve > > > >>> port guids in parse time and keep it here in cl_map() > > for fast searches. > > > >> By saying "config features", do you mean the four boolean flags? > > > > > > > > No, I mean everything except name and use. > > > > > > > >> It looks to me that checking the type of node is as fast as it > > > >> gets, and it won't hurt to leave these booleans instead of > > > >> resolving all the guids. > > > > > > > > It could be optimization when types are specified, which is not > > > > always the case and then you are going to do linear > > searches over all lists. > > > > > > Right, it would be linear scanning of list of partitions. > > > > > > > And how something like "for each guid in this group" (which is > > > > needed for QoS port parameters setup) should be resolved? By > > > > matching each guid in the subnet against those lists? > > > > > > Good point. > > > > > > >> Moreover, the guids here are stored in range array, > > which is IMO > > > >> better suited for the policy file syntax, because if a user > > > >> specifies something like this "0x0-0x0FFF" in guids, it will be > > > >> only one element of the array, which is efficient both > > in memory and in serch time. > > > > > > > > And what should be there if user specifies ports in the group as: > > > > guid1, guid2, guid3, etc. ? > > > > > > The range array is sorted and "shrinked". > > > That is, if a user specifies guids as "30,1,2,3-20,15", > > eventually you > > > would get two elements in the array: 1-20 and 30-30. > > > cl_map implemented as a binary tree, right? > > > And doing binary search in this kind of array is faster > > than searching > > > a guid in cl_map of all the guids. > > > Worst case - you will get the same performance as in case > > of cl_map if > > > *all* the guids are "discreet" and can't be groupped in ranges. > > > > > > Nevertheless, I agree that there's a problem if there are many > > > partitions in the list (although I'm not sure it's a practical case) > > > > > > I'll work on this. > > > > > > >> (I probably should mention here that the efficient > > search in the range > > > >> array is not implemented yet, but it would be a simple > > binary search - > > > >> there's a "todo" comment in the search function right now) > > > >> > > > >>>> + > > > >>>> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > > > >>>> +void osm_qos_policy_port_group_destroy(); > > > >>> Would be nice to have function prototypes in one place. > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_vlarb_scope_t_ { > > > >>>> + cl_list_t group_list; /* list of group names (strings) */ > > > >>>> + cl_list_t across_list; /* list of 'across' group > > names (strings) */ > > > >>>> + cl_list_t vlarb_high_list; /* list of num > > pairs (n:m,...), 32-bit values > > > >>>> */ > > > >>>> + cl_list_t vlarb_low_list; /* list of num > > pairs (n:m,...), 32-bit values > > > >>>> */ > > > >>> Why cl_list for VLArb? it should be short fixed length arrays? > > > >> Right. > > > >> Since the actual VLArb setup is not implemented yet, I > > didn't see > > > >> this obvious thing. > > > > > > > > But it should be implemented. Right? > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze in 11 days. > > > > Without VLArb configuration, where does this come from ? What > > is the QoS support then: just SL ? > > > > -- Hal > > > > > > > > >>>> + uint32_t vl_high_limit; /* single integer */ > > > >>>> + boolean_t vl_high_limit_set; > > > >>>> +} osm_qos_vlarb_scope_t; > > > >>>> + > > > >>>> +osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create(); > > > >>>> +void osm_qos_policy_vlarb_scope_destroy(); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_sl2vl_scope_t_ { > > > >>>> + cl_list_t group_list; /* list of strings (port > > group names) */ > > > >>>> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > > >>>> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > > >>>> + cl_list_t across_from_list; /* list of strings > > (port group names) */ > > > >>>> + cl_list_t across_to_list; /* list of strings > > (port group names) */ > > > >>>> + uint8_t sl2vl_table[16]; /* array of sl2vl values */ > > > >>>> + boolean_t sl2vl_table_set; > > > >>>> +} osm_qos_sl2vl_scope_t; > > > >>> This will be used for sl2vl setup? Same as above - why not to > > > >>> generate final port guid list just in parse time? > > > >>>> + > > > >>>> +osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create(); > > > >>>> +void osm_qos_policy_sl2vl_scope_destroy(); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_level_t_ { > > > >>>> + char *use; > > > >>>> + char *name; > > > >>>> + uint8_t sl; > > > >>>> + boolean_t sl_set; > > > >>>> + uint8_t mtu_limit; > > > >>>> + boolean_t mtu_limit_set; > > > >>>> + uint8_t rate_limit; > > > >>>> + boolean_t rate_limit_set; > > > >>>> + uint8_t pkt_life; > > > >>>> + boolean_t pkt_life_set; > > > >>>> + uint64_t **path_bits_range_arr; /* array of bit > > ranges (real > > > >>>> +values are > > > >>>> 32bits) */ > > > >>>> + unsigned path_bits_range_len; /* num of bit > > ranges in the array */ > > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > > ranges (real values are > > > >>>> 16bits) */ > > > >>>> + unsigned pkey_range_len; > > > >>>> +} osm_qos_level_t; > > > >>>> + > > > >>>> +osm_qos_level_t *osm_qos_policy_qos_level_create(); > > > >>>> +void osm_qos_policy_qos_level_destroy(); > > > >>>> + > > > >>>> +boolean_t osm_qos_level_has_pkey(IN const > > osm_qos_level_t * p_qos_level, > > > >>>> + IN ib_net16_t pkey); > > > >>>> + > > > >>>> +ib_net16_t osm_qos_level_get_shared_pkey(IN const > > > >>>> +osm_qos_level_t * > > > >>>> p_qos_level, > > > >>>> + IN const > > osm_physp_t * p_src_physp, > > > >>>> + IN const osm_physp_t * > > > >>>> + p_dest_physp); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_match_rule_t_ { > > > >>>> + char *use; > > > >>>> + cl_list_t source_list; /* list of strings */ > > > >>>> + cl_list_t source_group_list; /* list of pointers > > to relevant port-group > > > >>>> */ > > > >>>> + cl_list_t destination_list; /* list of strings */ > > > >>>> + cl_list_t destination_group_list; /* list of > > pointers to relevant > > > >>>> port-group */ > > > >>> I think you should only keep port guids there (mapped > > for fast searches). > > > >> Same as above. > > > >> I think that checking node type and then guid range array is > > > >> essentially faster than checking guid map. > > > > > > > > _Only_ for case when the group is specified by type, > > which is likely > > > > will not be typical. > > > > > > Again, in *worst* case you will get the same performance in > > searching > > > range array as in searching cl_map (which is a binary tree). > > > > > > >> You might say, of course, that there can be many port > > groups in > > > >> the same match rule, but I don't see this as a practical > > example. > > > > > > > > Of course it could (or you must disable multi groups > > here, which is > > > > not good idea I think and not what was presented in the RFC). > > > > > > Can you elaborate on this? > > > Do you think that having multiple groups is not useful at all? > > > > > > > Also each group still have this "linear searchable" lists: > > > > > > > > cl_list_t port_name_list; /* list of port > > names (.../.../...) */ > > > > > > Port names are not implemented yet :) > > > There's a "todo" comment in the code. > > > This is the only keyword in the parser that is not implemented - I > > > mean it's parsed, but other than creating this list the > > parser doesn't do anything with it. > > > But It definitely won't stay as a pure list. > > > > > > > uint64_t **guid_range_arr; /* array of guid > > ranges (pair of 64-bit > > > > > > This is not just a list, as I've explained before. > > > > > > > cl_list_t partition_list; /* list of > > partition names */ > > > > > > I can hardly believe than there will be more than one or two > > > partitions in this list. Do you think otherwise? > > > > > > Anyway, I can extract all the guids and prepare a map. This is > > > certainly the easiest implementation. And I still think that on > > > average, guid ranges and partition lists are better, but > > who knows - > > > perhaps someone would want to define a bunch of partitions > > in a single > > > port group and ruin my average... :) > > > > > > >>>> + char *qos_level_name; > > > >>>> + osm_qos_level_t *p_qos_level; > > > >>> Why do you need qos_level_name if you keep the pointer to this > > > >>> qos_level struct? > > > >> In policy file the match rule might appear before the > > QoS levels, > > > >> so matching qos level names to the actual qos levels is > > done when > > > >> the parsing is done. > > > >> > > > >>>> + uint64_t **service_id_range_arr; /* array of > > SID ranges (64-bit values) > > > >>>> */ > > > >>>> + unsigned service_id_range_len; uint64_t > > > >>>> + **qos_class_range_arr; /* array of QoS Class ranges (real > > > >>>> values are 16bits) */ > > > >>>> + unsigned qos_class_range_len; > > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > > ranges (real values are > > > >>>> 16bits) */ > > > >>>> + unsigned pkey_range_len; > > > >>>> +} osm_qos_match_rule_t; > > > >>>> + > > > >>>> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > > > >>>> +void osm_qos_policy_match_rule_destroy(); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +typedef struct osm_qos_policy_t_ { > > > >>>> + cl_list_t port_groups; /* list of osm_qos_port_group_t */ > > > >>>> + cl_list_t sl2vl_tables; /* list of osm_qos_sl2vl_scope_t */ > > > >>>> + cl_list_t vlarb_tables; /* list of osm_qos_vlarb_scope_t */ > > > >>>> + cl_list_t qos_levels; /* list of osm_qos_level_t */ > > > >>>> + cl_list_t qos_match_rules; /* list of > > osm_qos_match_rule_t */ > > > >>>> + osm_qos_level_t *p_default_qos_level; /* default > > QoS level */ > > > >>>> +} osm_qos_policy_t; > > > >>>> + > > > >>>> +void osm_qos_policy_create(); > > > >>>> +void osm_qos_policy_destroy(); > > > >>>> +int osm_qos_policy_validate(); > > > >>>> + > > > >>>> +void osm_qos_policy_get_qos_level_by_pr(IN const > > osm_pr_rcv_t * p_rcv, > > > >>>> + IN const > > ib_path_rec_t * p_pr, > > > >>>> + IN const > > osm_physp_t * p_src_physp, > > > >>>> + IN const > > osm_physp_t * p_dest_physp, > > > >>>> + IN ib_net64_t comp_mask, > > > >>>> + OUT osm_qos_level_t ** > > > >>>> +pp_qos_level); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +int osm_qos_parse_policy_file(IN osm_log_t * p_log, IN const > > > >>>> +char > > > >>>> *policy_file); > > > >>>> + > > > >>>> +/***************************************************/ > > > >>>> + > > > >>>> +#endif /* ifndef > > OSM_QOS_POLICY_H */ > > > >>>> diff --git a/opensm/opensm/osm_qos_policy.c > > > >>>> b/opensm/opensm/osm_qos_policy.c new file mode 100644 index > > > >>>> 0000000..bc2aa68 > > > >>>> --- /dev/null > > > >>>> +++ b/opensm/opensm/osm_qos_policy.c > > > >>>> @@ -0,0 +1,901 @@ > > > >>>> +/* > > > >>>> + * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. > > > >>>> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. > > All rights > > > >>>> reserved. > > > >>>> + * Copyright (c) 1996-2003 Intel Corporation. All > > rights reserved. > > > >>>> + * > > > >>>> + * This software is available to you under a choice of one of > > > >>>> + two > > > >>>> + * licenses. You may choose to be licensed under the > > terms of > > > >>>> + the GNU > > > >>>> + * General Public License (GPL) Version 2, available from the > > > >>>> + file > > > >>>> + * COPYING in the main directory of this source tree, or the > > > >>>> + * OpenIB.org BSD license below: > > > >>>> + * > > > >>>> + * Redistribution and use in source and binary > > forms, with or > > > >>>> + * without modification, are permitted provided > > that the following > > > >>>> + * conditions are met: > > > >>>> + * > > > >>>> + * - Redistributions of source code must retain the above > > > >>>> + * copyright notice, this list of conditions > > and the following > > > >>>> + * disclaimer. > > > >>>> + * > > > >>>> + * - Redistributions in binary form must > > reproduce the above > > > >>>> + * copyright notice, this list of conditions > > and the following > > > >>>> + * disclaimer in the documentation and/or > > other materials > > > >>>> + * provided with the distribution. > > > >>>> + * > > > >>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY > > > >>>> + KIND, > > > >>>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE > > > >>>> + WARRANTIES OF > > > >>>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > > >>>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR > > COPYRIGHT > > > >>>> + HOLDERS > > > >>>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > > LIABILITY, WHETHER > > > >>>> + IN AN > > > >>>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > > FROM, OUT OF > > > >>>> + OR IN > > > >>>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > > DEALINGS IN > > > >>>> + THE > > > >>>> + * SOFTWARE. > > > >>>> + * > > > >>>> + */ > > > >>>> + > > > >>>> +/* > > > >>>> + * Abstract: > > > >>>> + * OSM QoS Policy functions. > > > >>>> + * > > > >>>> + * Environment: > > > >>>> + * Linux User Mode > > > >>>> + * > > > >>>> + * Author: > > > >>>> + * Yevgeny Kliteynik, Mellanox > > > >>>> + */ > > > >>>> + > > > >>>> +#include #include > > > >>>> + #include > > > >>>> + > > > >>>> +extern void yyerror(char *s); > > > >>>> +osm_log_t *p_qos_parser_osm_log = NULL; osm_qos_policy_t > > > >>>> +*p_qos_policy = NULL; > > > >>> Please try to avoid globals - keep it as part of > > osm_opensm_t or > > > >>> osm_subn_t structures. > > > >> I thought about it, but didn't want to "condaminate" the > > > >> osm_opensm_t or osm_subn_t structures untill the QoS > > functionality is ready. > > > > > > > > How globals are better in this sense? > > > > > > They're not :) > > > The difference is that less files are modified. > > > But I agree that the QoS policy should be part of > > osm_opensm_t or osm_subn_t. > > > > > > -- Yevgeny > > > > > > > Sasha > > > > > > > > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit > > > http://openib.org/mailman/listinfo/openib-general > > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > From hal.rosenstock at gmail.com Tue Aug 28 09:24:42 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 12:24:42 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D35A03.2070101@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> Message-ID: On 8/27/07, Yevgeny Kliteynik wrote: > Adding QoS policy file option to OpenSM optionsm and > QoS Policy field to subn object. > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > opensm/opensm/main.c | 12 +++++++++++- > opensm/opensm/osm_subnet.c | 10 +++++++++- > 4 files changed, 50 insertions(+), 2 deletions(-) > > diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h > index 545779b..0a35e22 100644 > --- a/opensm/include/opensm/osm_base.h > +++ b/opensm/include/opensm/osm_base.h > @@ -224,6 +224,23 @@ BEGIN_C_DECLS > #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" > #endif > /***********/ > + > +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > +* NAME > +* OSM_DEFAULT_QOS_POLICY_FILE > +* > +* DESCRIPTION > +* Specifies the default QoS policy file name > +* > +* SYNOPSIS > +*/ > +#ifdef __WIN__ > +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") > +#else > +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" > +#endif > +/***********/ > + > /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > * NAME > * OSM_DEFAULT_SWEEP_INTERVAL_SECS > diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > index a7543dc..5e8b522 100644 > --- a/opensm/include/opensm/osm_subnet.h > +++ b/opensm/include/opensm/osm_subnet.h > @@ -68,6 +68,7 @@ BEGIN_C_DECLS > #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > #define OSM_SUBNET_VECTOR_CAPACITY 256 > struct _osm_opensm_t; > +struct _osm_qos_policy_t; > > /****h* OpenSM/Subnet > * NAME > @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > char *partition_config_file; > boolean_t no_partition_enforcement; > boolean_t no_qos; > + char *qos_policy_file; > boolean_t accum_log_file; > char *console; > uint16_t console_port; > @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > * specified the log file will be truncated upon reaching > * this limit. > * > +* no_qos > +* Boolean that specifies whether the OpenSM QoS functionality > +* should be off or on. > +* > +* qos_policy_file > +* Name of the QoS policy file. > +* Are these options mutually exclusive or not ? > * accum_log_file > * If TRUE (default) - the log file will be accumulated. > * If FALSE - the log file will be erased before starting current opensm run. > @@ -551,6 +560,7 @@ typedef struct _osm_subn { > ib_net64_t sm_port_guid; > uint8_t sm_state; > osm_subn_opt_t opt; > + struct _osm_qos_policy_t *p_qos_policy; > uint16_t max_unicast_lid_ho; > uint16_t max_multicast_lid_ho; > uint8_t min_ca_mtu; > @@ -619,6 +629,9 @@ typedef struct _osm_subn { > * opt > * Subnet options structure contains site specific configuration. > * > +* p_qos_policy > +* Subnet QoS policy structure. > +* > * max_unicast_lid_ho > * The minimal max unicast lid reported by all switches > * > diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > index e2541d8..599c4dc 100644 > --- a/opensm/opensm/main.c > +++ b/opensm/opensm/main.c > @@ -269,6 +269,10 @@ void show_usage(void) > " The default name is \'" > OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > + printf("-Y\n" > + "--qos_policy_file\n" > + " This option defines the optional QoS policy file.\n" > + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > printf("-N\n" > "--no_part_enforce\n" > " This option disables partition enforcement on switch external ports.\n\n"); > @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > char *ignore_guids_file_name = NULL; > uint32_t val; > const char *const short_option = > - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > /* > In the array below, the 2nd parameter specifies the number > @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > {"Pconfig", 1, NULL, 'P'}, > {"no_part_enforce", 0, NULL, 'N'}, > {"qos", 0, NULL, 'Q'}, > + {"qos_policy_file", 1, NULL, 'Y'}, > {"maxsmps", 1, NULL, 'n'}, > {"console", 1, NULL, 'q'}, > {"V", 0, NULL, 'V'}, > @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > opt.no_qos = FALSE; > break; > > + case 'Y': > + opt.qos_policy_file = optarg; > + printf(" QoS policy file \'%s\'\n", optarg); > + break; > + There should also be an update to the OpenSM man page for this. -- Hal > case 'y': > opt.exit_on_fatal = FALSE; > printf(" Staying on fatal initialization errors\n"); > diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > index 818d73f..4162522 100644 > --- a/opensm/opensm/osm_subnet.c > +++ b/opensm/opensm/osm_subnet.c > @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > p_opt->no_partition_enforcement = FALSE; > p_opt->no_qos = TRUE; > + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > p_opt->accum_log_file = TRUE; > p_opt->port_profile_switch_nodes = FALSE; > p_opt->pfn_ui_pre_lid_assign = NULL; > @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > > opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > > + opts_unpack_charp("qos_policy_file", > + p_key, p_val, &p_opts->qos_policy_file); > + > opts_unpack_boolean("accum_log_file", > p_key, p_val, &p_opts->accum_log_file); > > @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > fprintf(opts_file, > "#\n# QoS OPTIONS\n#\n" > "# Disable QoS setup\n" > - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > + "no_qos %s\n\n" > + "# QoS policy file to be used\n" > + "qos_policy_file %s\n\n", > + p_opts->no_qos ? "TRUE" : "FALSE", > + p_opts->qos_policy_file); > > subn_dump_qos_options(opts_file, > "QoS default options", "qos", > -- > 1.5.1.4 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From hal.rosenstock at gmail.com Tue Aug 28 09:27:13 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 12:27:13 -0400 Subject: [ofa-general] [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <46D35A20.6020902@dev.mellanox.co.il> References: <46D35A20.6020902@dev.mellanox.co.il> Message-ID: On 8/27/07, Yevgeny Kliteynik wrote: > Compiling QoS policy file parser. Should there be an build enable for this functionality (like --enable-qos-policy) ? I expect that some will want to not include this. By default, this should be disabled IMO. BTW, what is the "level" of this QoS support for OFED 1.3 ? Is it technology preview ? -- Hal > C & H files are generated from Lex & Yacc files and the > generated files are compiled. > Generated files are not included in git, but they do get > included in RPM as they are also created by 'make dist'. > Generated files can be removed by "make maintainer-clean". > > Signed-off-by: Yevgeny Kliteynik > --- > opensm/include/Makefile.am | 2 ++ > opensm/opensm/Makefile.am | 16 +++++++++++++++- > opensm/opensm/configure.in | 2 ++ > 3 files changed, 19 insertions(+), 1 deletions(-) > > diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > index ea62507..b83205f 100644 > --- a/opensm/include/Makefile.am > +++ b/opensm/include/Makefile.am > @@ -90,6 +90,8 @@ EXTRA_DIST = \ > $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > $(srcdir)/opensm/osm_perfmgr.h \ > $(srcdir)/opensm/osm_perfmgr_db.h \ > + $(srcdir)/opensm/osm_qos_policy.h \ > + $(srcdir)/opensm/osm_qos_parser_y.h \ > $(srcdir)/complib/cl_thread_osd.h \ > $(srcdir)/complib/cl_packon.h \ > $(srcdir)/complib/cl_atomic_osd.h \ > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index 6dfa824..9d57f93 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > osm_vl15intf.c osm_vl_arb_rcv.c \ > st.c osm_perfmgr.c osm_perfmgr_db.c \ > - osm_event_plugin.c osm_dump.c > + osm_event_plugin.c osm_dump.c \ > + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > + > +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > + > +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h > + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l > + > if OSMV_OPENIB > opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > @@ -104,3 +112,9 @@ install-exec-hook: > if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ > lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ > ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so > + > +# generate c and h files from the lex and yacc files > +dist-hook: $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c > + > +maintainer-clean-generic: > + rm -f $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c > diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in > index afb20b3..a49538d 100644 > --- a/opensm/opensm/configure.in > +++ b/opensm/opensm/configure.in > @@ -22,6 +22,8 @@ AC_PROG_INSTALL > AC_PROG_LN_S > AC_PROG_MAKE_SET > AC_PROG_LIBTOOL > +AM_PROG_LEX > +AC_PROG_YACC > > dnl Checks for libraries > > -- > 1.5.1.4 > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From mst at dev.mellanox.co.il Tue Aug 28 10:37:47 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 28 Aug 2007 20:37:47 +0300 Subject: [ofa-general] Re: [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <20070828151705.GO18082@sashak.voltaire.com> References: <46D35A20.6020902@dev.mellanox.co.il> <20070828135750.GG18082@sashak.voltaire.com> <46D439EF.4070200@dev.mellanox.co.il> <20070828151705.GO18082@sashak.voltaire.com> Message-ID: <20070828173747.GB9535@mellanox.co.il> > Quoting Sasha Khapyorsky : > Subject: Re: [PATCH 6/7 V3] osm: QoS - compiling policy file parser > > On 18:06 Tue 28 Aug , Yevgeny Kliteynik wrote: > > Sasha Khapyorsky wrote: > > > On 02:11 Tue 28 Aug , Yevgeny Kliteynik wrote: > > >> Compiling QoS policy file parser. > > >> > > >> C & H files are generated from Lex & Yacc files and the > > >> generated files are compiled. > > >> Generated files are not included in git, but they do get > > >> included in RPM as they are also created by 'make dist'. > > >> Generated files can be removed by "make maintainer-clean". > > >> > > >> Signed-off-by: Yevgeny Kliteynik > > > Applied. Thanks. As discussed I will remove lex and yacc deps. > > > > After removing these deps, Lex fails on my machine. > > Yes, as well as 'yacc' - the comman line is 'bison' specific. > > > Apparently, with these flags make uses "flex", and w/o them it uses "lex". > > I'll look into this, but in the mean time please put these flags back as > > OpenSM compilation is now broken. > > Ok, I will put this back for now, but lex/yacc should be improved. > > Sasha It might be easier to just force flex/bison usage. -- MST From hal.rosenstock at gmail.com Tue Aug 28 11:17:48 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 14:17:48 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> Message-ID: Hi Eitan, On 8/28/07, Eitan Zahavi wrote: > Hi Hal, > > > > > > As you pointed out, coming up with a VLArb and SL2VL > > manager that is > > > supporting both routing and QoS is not trivial. > > > But even with a uniform and statically assigned tables > > (supported by > > > the current options config file) different QoS classes can > > be provided > > > to different clients by the new QoS policy code. > > > > > > I agree our OFED 1.3 is somewhat limited as it does not solve the > > > VLArb and SL2VL setting issue. > > > But with the time constrains of the development, review and > > QA this is > > > the most we could do for 1.3 > > > > How much of the QoS policy syntax is left "dormant" by this ? > > How much is actually used ? Can this be detailed somewhere ? > Simply ignore the "Setup" section. > > Also, there was a previous question on the list on IPoIB CM > > relative to QoS which I either missed the answer to or was > > left unanswered: > > How is IPoIB CM dealt with in terms of QoS ? Is it like IPoIB UD ? > IPoIB CM uses a specific ServiceID. Isn't it a range of Service IDs ? > It uses CMA so it does not need any change... Doesn't it use CM rather than CMA ? -- Hal > > > > What documentation will be provided (e.g. man page and > > perhaps other doc file for this as it is quite complex) ? > > > > It would be nice to see the complete picture but it seems > > like this will be a set of incremental changes for which some > > looking back over the entirety will be needed when things settle down. > > > > -- Hal > > > > > EZ > > > > > > Eitan Zahavi > > > Senior Engineering Director, Software Architect Mellanox > > Technologies > > > LTD > > > Tel:+972-4-9097208 > > > Fax:+972-4-9593245 > > > P.O. Box 586 Yokneam 20692 ISRAEL > > > > > > > > > > > > > -----Original Message----- > > > > From: general-bounces at lists.openfabrics.org > > > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Hal > > > > Rosenstock > > > > Sent: Monday, August 27, 2007 7:33 PM > > > > To: Yevgeny Kliteynik > > > > Cc: OpenIB > > > > Subject: Re: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS > > policy C & H > > > > files > > > > > > > > On 8/23/07, Yevgeny Kliteynik wrote: > > > > > Hi Sasha, > > > > > > > > > > Sasha Khapyorsky wrote: > > > > > > On 11:11 Thu 23 Aug , Yevgeny Kliteynik wrote: > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_port_group_t_ { > > > > > >>>> + char *name; /* single string (this port > > > > group name) */ > > > > > >>>> + char *use; /* single string > > (description) */ > > > > > >>>> + cl_list_t port_name_list; /* list of port > > > > names (.../.../...) */ > > > > > >>>> + uint64_t **guid_range_arr; /* array of guid > > > > ranges (pair of 64-bit > > > > > >>>> guids) */ > > > > > >>>> + unsigned guid_range_len; /* num of guid > > > > ranges in the array */ > > > > > >>>> + cl_list_t partition_list; /* list of > > > > partition names */ > > > > > >>>> + boolean_t node_type_ca; > > > > > >>>> + boolean_t node_type_switch; > > > > > >>>> + boolean_t node_type_router; > > > > > >>>> + boolean_t node_type_self; > > > > > >>>> +} osm_qos_port_group_t; > > > > > >>> I see you are using this in "run-time", not just during > > > > the parsing. > > > > > >>> Instead of having all this config features you can just > > > > > >>> resolve port guids in parse time and keep it here > > in cl_map() > > > > for fast searches. > > > > > >> By saying "config features", do you mean the four > > boolean flags? > > > > > > > > > > > > No, I mean everything except name and use. > > > > > > > > > > > >> It looks to me that checking the type of node is as > > fast as it > > > > > >> gets, and it won't hurt to leave these booleans instead of > > > > > >> resolving all the guids. > > > > > > > > > > > > It could be optimization when types are specified, > > which is not > > > > > > always the case and then you are going to do linear > > > > searches over all lists. > > > > > > > > > > Right, it would be linear scanning of list of partitions. > > > > > > > > > > > And how something like "for each guid in this group" > > (which is > > > > > > needed for QoS port parameters setup) should be resolved? By > > > > > > matching each guid in the subnet against those lists? > > > > > > > > > > Good point. > > > > > > > > > > >> Moreover, the guids here are stored in range array, > > > > which is IMO > > > > > >> better suited for the policy file syntax, because if a user > > > > > >> specifies something like this "0x0-0x0FFF" in > > guids, it will > > > > > >> be only one element of the array, which is efficient both > > > > in memory and in serch time. > > > > > > > > > > > > And what should be there if user specifies ports in > > the group as: > > > > > > guid1, guid2, guid3, etc. ? > > > > > > > > > > The range array is sorted and "shrinked". > > > > > That is, if a user specifies guids as "30,1,2,3-20,15", > > > > eventually you > > > > > would get two elements in the array: 1-20 and 30-30. > > > > > cl_map implemented as a binary tree, right? > > > > > And doing binary search in this kind of array is faster > > > > than searching > > > > > a guid in cl_map of all the guids. > > > > > Worst case - you will get the same performance as in case > > > > of cl_map if > > > > > *all* the guids are "discreet" and can't be groupped in ranges. > > > > > > > > > > Nevertheless, I agree that there's a problem if there are many > > > > > partitions in the list (although I'm not sure it's a practical > > > > > case) > > > > > > > > > > I'll work on this. > > > > > > > > > > >> (I probably should mention here that the efficient > > > > search in the range > > > > > >> array is not implemented yet, but it would be a simple > > > > binary search - > > > > > >> there's a "todo" comment in the search function right now) > > > > > >> > > > > > >>>> + > > > > > >>>> +osm_qos_port_group_t *osm_qos_policy_port_group_create(); > > > > > >>>> +void osm_qos_policy_port_group_destroy(); > > > > > >>> Would be nice to have function prototypes in one place. > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_vlarb_scope_t_ { > > > > > >>>> + cl_list_t group_list; /* list of group names > > (strings) */ > > > > > >>>> + cl_list_t across_list; /* list of 'across' group > > > > names (strings) */ > > > > > >>>> + cl_list_t vlarb_high_list; /* list of num > > > > pairs (n:m,...), 32-bit values > > > > > >>>> */ > > > > > >>>> + cl_list_t vlarb_low_list; /* list of num > > > > pairs (n:m,...), 32-bit values > > > > > >>>> */ > > > > > >>> Why cl_list for VLArb? it should be short fixed > > length arrays? > > > > > >> Right. > > > > > >> Since the actual VLArb setup is not implemented yet, I > > > > didn't see > > > > > >> this obvious thing. > > > > > > > > > > > > But it should be implemented. Right? > > > > > > > > > > Sure, but not for OFED 1.3 - we have a feature freeze > > in 11 days. > > > > > > > > Without VLArb configuration, where does this come from ? > > What is the > > > > QoS support then: just SL ? > > > > > > > > -- Hal > > > > > > > > > > > > > > >>>> + uint32_t vl_high_limit; /* single integer */ > > > > > >>>> + boolean_t vl_high_limit_set; } osm_qos_vlarb_scope_t; > > > > > >>>> + > > > > > >>>> +osm_qos_vlarb_scope_t > > *osm_qos_policy_vlarb_scope_create(); > > > > > >>>> +void osm_qos_policy_vlarb_scope_destroy(); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_sl2vl_scope_t_ { > > > > > >>>> + cl_list_t group_list; /* list of strings (port > > > > group names) */ > > > > > >>>> + boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > > > > >>>> + boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH]; > > > > > >>>> + cl_list_t across_from_list; /* list of strings > > > > (port group names) */ > > > > > >>>> + cl_list_t across_to_list; /* list of strings > > > > (port group names) */ > > > > > >>>> + uint8_t sl2vl_table[16]; /* array of > > sl2vl values */ > > > > > >>>> + boolean_t sl2vl_table_set; } osm_qos_sl2vl_scope_t; > > > > > >>> This will be used for sl2vl setup? Same as above - > > why not to > > > > > >>> generate final port guid list just in parse time? > > > > > >>>> + > > > > > >>>> +osm_qos_sl2vl_scope_t > > *osm_qos_policy_sl2vl_scope_create(); > > > > > >>>> +void osm_qos_policy_sl2vl_scope_destroy(); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_level_t_ { > > > > > >>>> + char *use; > > > > > >>>> + char *name; > > > > > >>>> + uint8_t sl; > > > > > >>>> + boolean_t sl_set; > > > > > >>>> + uint8_t mtu_limit; > > > > > >>>> + boolean_t mtu_limit_set; > > > > > >>>> + uint8_t rate_limit; > > > > > >>>> + boolean_t rate_limit_set; > > > > > >>>> + uint8_t pkt_life; > > > > > >>>> + boolean_t pkt_life_set; > > > > > >>>> + uint64_t **path_bits_range_arr; /* array of bit > > > > ranges (real > > > > > >>>> +values are > > > > > >>>> 32bits) */ > > > > > >>>> + unsigned path_bits_range_len; /* num of bit > > > > ranges in the array */ > > > > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > > > > ranges (real values are > > > > > >>>> 16bits) */ > > > > > >>>> + unsigned pkey_range_len; > > > > > >>>> +} osm_qos_level_t; > > > > > >>>> + > > > > > >>>> +osm_qos_level_t *osm_qos_policy_qos_level_create(); > > > > > >>>> +void osm_qos_policy_qos_level_destroy(); > > > > > >>>> + > > > > > >>>> +boolean_t osm_qos_level_has_pkey(IN const > > > > osm_qos_level_t * p_qos_level, > > > > > >>>> + IN ib_net16_t pkey); > > > > > >>>> + > > > > > >>>> +ib_net16_t osm_qos_level_get_shared_pkey(IN const > > > > > >>>> +osm_qos_level_t * > > > > > >>>> p_qos_level, > > > > > >>>> + IN const > > > > osm_physp_t * p_src_physp, > > > > > >>>> + IN const osm_physp_t * > > > > > >>>> + p_dest_physp); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_match_rule_t_ { > > > > > >>>> + char *use; > > > > > >>>> + cl_list_t source_list; /* list of strings */ > > > > > >>>> + cl_list_t source_group_list; /* list of pointers > > > > to relevant port-group > > > > > >>>> */ > > > > > >>>> + cl_list_t destination_list; /* list of strings */ > > > > > >>>> + cl_list_t destination_group_list; /* list of > > > > pointers to relevant > > > > > >>>> port-group */ > > > > > >>> I think you should only keep port guids there (mapped > > > > for fast searches). > > > > > >> Same as above. > > > > > >> I think that checking node type and then guid range > > array is > > > > > >> essentially faster than checking guid map. > > > > > > > > > > > > _Only_ for case when the group is specified by type, > > > > which is likely > > > > > > will not be typical. > > > > > > > > > > Again, in *worst* case you will get the same performance in > > > > searching > > > > > range array as in searching cl_map (which is a binary tree). > > > > > > > > > > >> You might say, of course, that there can be many port > > > > groups in > > > > > >> the same match rule, but I don't see this as a practical > > > > example. > > > > > > > > > > > > Of course it could (or you must disable multi groups > > > > here, which is > > > > > > not good idea I think and not what was presented in the RFC). > > > > > > > > > > Can you elaborate on this? > > > > > Do you think that having multiple groups is not useful at all? > > > > > > > > > > > Also each group still have this "linear searchable" lists: > > > > > > > > > > > > cl_list_t port_name_list; /* list of port > > > > names (.../.../...) */ > > > > > > > > > > Port names are not implemented yet :) There's a "todo" > > comment in > > > > > the code. > > > > > This is the only keyword in the parser that is not > > implemented - I > > > > > mean it's parsed, but other than creating this list the > > > > parser doesn't do anything with it. > > > > > But It definitely won't stay as a pure list. > > > > > > > > > > > uint64_t **guid_range_arr; /* array of guid > > > > ranges (pair of 64-bit > > > > > > > > > > This is not just a list, as I've explained before. > > > > > > > > > > > cl_list_t partition_list; /* list of > > > > partition names */ > > > > > > > > > > I can hardly believe than there will be more than one or two > > > > > partitions in this list. Do you think otherwise? > > > > > > > > > > Anyway, I can extract all the guids and prepare a map. This is > > > > > certainly the easiest implementation. And I still think that on > > > > > average, guid ranges and partition lists are better, but > > > > who knows - > > > > > perhaps someone would want to define a bunch of partitions > > > > in a single > > > > > port group and ruin my average... :) > > > > > > > > > > >>>> + char *qos_level_name; > > > > > >>>> + osm_qos_level_t *p_qos_level; > > > > > >>> Why do you need qos_level_name if you keep the > > pointer to this > > > > > >>> qos_level struct? > > > > > >> In policy file the match rule might appear before the > > > > QoS levels, > > > > > >> so matching qos level names to the actual qos levels is > > > > done when > > > > > >> the parsing is done. > > > > > >> > > > > > >>>> + uint64_t **service_id_range_arr; /* array of > > > > SID ranges (64-bit values) > > > > > >>>> */ > > > > > >>>> + unsigned service_id_range_len; uint64_t > > > > > >>>> + **qos_class_range_arr; /* array of QoS Class ranges (real > > > > > >>>> values are 16bits) */ > > > > > >>>> + unsigned qos_class_range_len; > > > > > >>>> + uint64_t **pkey_range_arr; /* array of PKey > > > > ranges (real values are > > > > > >>>> 16bits) */ > > > > > >>>> + unsigned pkey_range_len; > > > > > >>>> +} osm_qos_match_rule_t; > > > > > >>>> + > > > > > >>>> +osm_qos_match_rule_t *osm_qos_policy_match_rule_create(); > > > > > >>>> +void osm_qos_policy_match_rule_destroy(); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +typedef struct osm_qos_policy_t_ { > > > > > >>>> + cl_list_t port_groups; /* list of > > osm_qos_port_group_t */ > > > > > >>>> + cl_list_t sl2vl_tables; /* list of > > osm_qos_sl2vl_scope_t > > > > > >>>> +*/ > > > > > >>>> + cl_list_t vlarb_tables; /* list of > > osm_qos_vlarb_scope_t */ > > > > > >>>> + cl_list_t qos_levels; /* list of osm_qos_level_t */ > > > > > >>>> + cl_list_t qos_match_rules; /* list of > > > > osm_qos_match_rule_t */ > > > > > >>>> + osm_qos_level_t *p_default_qos_level; /* default > > > > QoS level */ > > > > > >>>> +} osm_qos_policy_t; > > > > > >>>> + > > > > > >>>> +void osm_qos_policy_create(); void > > osm_qos_policy_destroy(); > > > > > >>>> +int osm_qos_policy_validate(); > > > > > >>>> + > > > > > >>>> +void osm_qos_policy_get_qos_level_by_pr(IN const > > > > osm_pr_rcv_t * p_rcv, > > > > > >>>> + IN const > > > > ib_path_rec_t * p_pr, > > > > > >>>> + IN const > > > > osm_physp_t * p_src_physp, > > > > > >>>> + IN const > > > > osm_physp_t * p_dest_physp, > > > > > >>>> + IN ib_net64_t comp_mask, > > > > > >>>> + OUT osm_qos_level_t ** > > > > > >>>> +pp_qos_level); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +int osm_qos_parse_policy_file(IN osm_log_t * > > p_log, IN const > > > > > >>>> +char > > > > > >>>> *policy_file); > > > > > >>>> + > > > > > >>>> +/***************************************************/ > > > > > >>>> + > > > > > >>>> +#endif /* ifndef > > > > OSM_QOS_POLICY_H */ > > > > > >>>> diff --git a/opensm/opensm/osm_qos_policy.c > > > > > >>>> b/opensm/opensm/osm_qos_policy.c new file mode 100644 index > > > > > >>>> 0000000..bc2aa68 > > > > > >>>> --- /dev/null > > > > > >>>> +++ b/opensm/opensm/osm_qos_policy.c > > > > > >>>> @@ -0,0 +1,901 @@ > > > > > >>>> +/* > > > > > >>>> + * Copyright (c) 2004-2006 Voltaire, Inc. All > > rights reserved. > > > > > >>>> + * Copyright (c) 2002-2005 Mellanox Technologies LTD. > > > > All rights > > > > > >>>> reserved. > > > > > >>>> + * Copyright (c) 1996-2003 Intel Corporation. All > > > > rights reserved. > > > > > >>>> + * > > > > > >>>> + * This software is available to you under a > > choice of one > > > > > >>>> + of two > > > > > >>>> + * licenses. You may choose to be licensed under the > > > > terms of > > > > > >>>> + the GNU > > > > > >>>> + * General Public License (GPL) Version 2, available from > > > > > >>>> + the file > > > > > >>>> + * COPYING in the main directory of this source > > tree, or the > > > > > >>>> + * OpenIB.org BSD license below: > > > > > >>>> + * > > > > > >>>> + * Redistribution and use in source and binary > > > > forms, with or > > > > > >>>> + * without modification, are permitted provided > > > > that the following > > > > > >>>> + * conditions are met: > > > > > >>>> + * > > > > > >>>> + * - Redistributions of source code must > > retain the above > > > > > >>>> + * copyright notice, this list of conditions > > > > and the following > > > > > >>>> + * disclaimer. > > > > > >>>> + * > > > > > >>>> + * - Redistributions in binary form must > > > > reproduce the above > > > > > >>>> + * copyright notice, this list of conditions > > > > and the following > > > > > >>>> + * disclaimer in the documentation and/or > > > > other materials > > > > > >>>> + * provided with the distribution. > > > > > >>>> + * > > > > > >>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT > > WARRANTY OF ANY > > > > > >>>> + KIND, > > > > > >>>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE > > > > > >>>> + WARRANTIES OF > > > > > >>>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > > > > > >>>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR > > > > COPYRIGHT > > > > > >>>> + HOLDERS > > > > > >>>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > > > > LIABILITY, WHETHER > > > > > >>>> + IN AN > > > > > >>>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > > > > FROM, OUT OF > > > > > >>>> + OR IN > > > > > >>>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > > > > DEALINGS IN > > > > > >>>> + THE > > > > > >>>> + * SOFTWARE. > > > > > >>>> + * > > > > > >>>> + */ > > > > > >>>> + > > > > > >>>> +/* > > > > > >>>> + * Abstract: > > > > > >>>> + * OSM QoS Policy functions. > > > > > >>>> + * > > > > > >>>> + * Environment: > > > > > >>>> + * Linux User Mode > > > > > >>>> + * > > > > > >>>> + * Author: > > > > > >>>> + * Yevgeny Kliteynik, Mellanox > > > > > >>>> + */ > > > > > >>>> + > > > > > >>>> +#include #include > > > > > >>>> + #include > > > > > >>>> + > > > > > >>>> + > > > > > >>>> +extern void yyerror(char *s); osm_log_t > > > > > >>>> +*p_qos_parser_osm_log = NULL; osm_qos_policy_t > > *p_qos_policy > > > > > >>>> += NULL; > > > > > >>> Please try to avoid globals - keep it as part of > > > > osm_opensm_t or > > > > > >>> osm_subn_t structures. > > > > > >> I thought about it, but didn't want to "condaminate" the > > > > > >> osm_opensm_t or osm_subn_t structures untill the QoS > > > > functionality is ready. > > > > > > > > > > > > How globals are better in this sense? > > > > > > > > > > They're not :) > > > > > The difference is that less files are modified. > > > > > But I agree that the QoS policy should be part of > > > > osm_opensm_t or osm_subn_t. > > > > > > > > > > -- Yevgeny > > > > > > > > > > > Sasha > > > > > > > > > > > > > > > > _______________________________________________ > > > > > general mailing list > > > > > general at lists.openfabrics.org > > > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > > > > > To unsubscribe, please visit > > > > > http://openib.org/mailman/listinfo/openib-general > > > > > > > > > _______________________________________________ > > > > general mailing list > > > > general at lists.openfabrics.org > > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > > > To unsubscribe, please visit > > > > http://openib.org/mailman/listinfo/openib-general > > > > > > > > > > From eitan at mellanox.co.il Tue Aug 28 11:20:02 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 28 Aug 2007 21:20:02 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: References: <46C983BB.4060007@dev.mellanox.co.il> <46CAD09C.4030400@dev.mellanox.co.il> <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9022583EE@mtlexch01.mtl.com> For more detailed info need to ask MST... > > > > Also, there was a previous question on the list on IPoIB > CM relative > > > to QoS which I either missed the answer to or was left unanswered: > > > How is IPoIB CM dealt with in terms of QoS ? Is it like IPoIB UD ? > > IPoIB CM uses a specific ServiceID. > > Isn't it a range of Service IDs ? > > > It uses CMA so it does not need any change... > > Doesn't it use CM rather than CMA ? From mst at dev.mellanox.co.il Tue Aug 28 11:23:42 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Tue, 28 Aug 2007 21:23:42 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9022583EE@mtlexch01.mtl.com> References: <20070823064648.GM1397@sashak.voltaire.com> <46CD411E.1080809@dev.mellanox.co.il> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583EE@mtlexch01.mtl.com> Message-ID: <20070828182342.GC9535@mellanox.co.il> > > > > > > Also, there was a previous question on the list on IPoIB > > > > CM relative > > > > to QoS which I either missed the answer to or was left unanswered: > > > > How is IPoIB CM dealt with in terms of QoS ? Is it like IPoIB UD ? > > > IPoIB CM uses a specific ServiceID. > > > > Isn't it a range of Service IDs ? Yes. Service ID includes the UD QPN. > > > It uses CMA so it does not need any change... > > > > Doesn't it use CM rather than CMA ? Yes. > For more detailed info need to ask MST... It's all in the spec, really. -- MST From hal.rosenstock at gmail.com Tue Aug 28 11:42:28 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Tue, 28 Aug 2007 14:42:28 -0400 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: <20070828182342.GC9535@mellanox.co.il> References: <20070823064648.GM1397@sashak.voltaire.com> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583EE@mtlexch01.mtl.com> <20070828182342.GC9535@mellanox.co.il> Message-ID: On 8/28/07, Michael S. Tsirkin wrote: > > > > > > > > Also, there was a previous question on the list on IPoIB > > > > > CM relative > > > > > to QoS which I either missed the answer to or was left unanswered: > > > > > How is IPoIB CM dealt with in terms of QoS ? Is it like IPoIB UD ? > > > > IPoIB CM uses a specific ServiceID. > > > > > > Isn't it a range of Service IDs ? > > Yes. Service ID includes the UD QPN. Given the limited predictability of the QPN, this would need to be reflected in the QoS policy when specifying ServiceIDs for IPoIB CM. > > > > It uses CMA so it does not need any change... > > > > > > Doesn't it use CM rather than CMA ? > > Yes. > > > For more detailed info need to ask MST... > > It's all in the spec, really. but I don't think that includes how it relates to QoS and specifically the QoS policy being used by OpenSM. My point was that the original defining document for OpenSM QoS policy needs to be updated as I suspect it will be a source document going forward that should be placed in the management git tree. -- Hal > -- > MST > From eitan at mellanox.co.il Tue Aug 28 11:49:42 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Tue, 28 Aug 2007 21:49:42 +0300 Subject: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy C & H files In-Reply-To: References: <20070823064648.GM1397@sashak.voltaire.com> <20070823095615.GP1397@sashak.voltaire.com> <46CD6CC1.30403@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C902257FBE@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583E9@mtlexch01.mtl.com> <6C2C79E72C305246B504CBA17B5500C9022583EE@mtlexch01.mtl.com> <20070828182342.GC9535@mellanox.co.il> Message-ID: <6C2C79E72C305246B504CBA17B5500C902258401@mtlexch01.mtl.com> Now I get you. OK we will add this info about IPoIB CM to the RFC. Eitan Zahavi > -----Original Message----- > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com] > Sent: Tuesday, August 28, 2007 9:42 PM > To: Michael S. Tsirkin > Cc: Eitan Zahavi; Yevgeny Kliteynik; OpenIB > Subject: Re: [ofa-general] Re: [PATCH 3/7 V2] osm: QoS policy > C & H files > > On 8/28/07, Michael S. Tsirkin wrote: > > > > > > > > > > Also, there was a previous question on the list on IPoIB CM > > > > > > relative to QoS which I either missed the answer to or was > > > > > > left unanswered: > > > > > > How is IPoIB CM dealt with in terms of QoS ? Is it > like IPoIB UD ? > > > > > IPoIB CM uses a specific ServiceID. > > > > > > > > Isn't it a range of Service IDs ? > > > > Yes. Service ID includes the UD QPN. > > Given the limited predictability of the QPN, this would need > to be reflected in the QoS policy when specifying ServiceIDs > for IPoIB CM. > > > > > > It uses CMA so it does not need any change... > > > > > > > > Doesn't it use CM rather than CMA ? > > > > Yes. > > > > > For more detailed info need to ask MST... > > > > It's all in the spec, really. > > but I don't think that includes how it relates to QoS and > specifically the QoS policy being used by OpenSM. My point > was that the original defining document for OpenSM QoS policy > needs to be updated as I suspect it will be a source document > going forward that should be placed in the management git tree. > > -- Hal > > > -- > > MST > > > From rdreier at cisco.com Tue Aug 28 12:38:07 2007 From: rdreier at cisco.com (Roland Dreier) Date: Tue, 28 Aug 2007 12:38:07 -0700 Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: <20070820.235804.85409183.davem@davemloft.net> (David Miller's message of "Mon, 20 Aug 2007 23:58:04 -0700 (PDT)") References: <20070817.234405.66176298.davem@davemloft.net> <20070820.235804.85409183.davem@davemloft.net> Message-ID: Sorry for the long latency, I was at the beach all last week. > > And direct data placement really does give you a factor of two at > > least, because otherwise you're stuck receiving the data in one > > buffer, looking at some of the data at least, and then figuring out > > where to copy it. And memory bandwidth is if anything becoming more > > valuable; maybe LRO + header splitting + page remapping tricks can get > > you somewhere but as NCPUS grows then it seems the TLB shootdown cost > > of page flipping is only going to get worse. > As Herbert has said already, people can code for this just like > they have to code for RDMA. No argument, you need to change the interface to take advantage of RDMA. > There is no fundamental difference from converting an application > to sendfile or similar. Yes, on the transmit side, there's not much difference from sendfile or splice, although RDMA may give a slightly nicer interface that also gives basically the equivalent of AIO. > The only thing this needs is a > "recvmsg_I_dont_care_where_the_data_is()" call. There are no alignment > issues unless you are trying to push this data directly into the > page cache. I don't understand how this gives you the same thing as direct data placement (DDP). There are many situations where the sender knows where the data has to go and if there's some way to pass that to the receiver, so that info can be used in the receive path to put the data in the right place, the receiver can save a copy. This is fundamentally the same "offload" that an FC HBA does -- the SCSI midlayer queues up commands like "read block A and put the data at address X" and "read block B and put the data at address Y" and the HBA matches tags on incoming data to put the blocks at the right addresses, even if block B is received before block A. RFC 4297 has some discussion of the various approaches, and while you might not agree with their conclusions, it is interesting reading. > Couple this with a card that makes sure that on a per-page basis, only > data for a particular flow (or group of flows) will accumulate. It seems that the NIC would also have to look into a TCP stream (and handle out of order segments etc) to find message boundaries for this to be equivalent to what an RDMA NIC does. - R. From davem at davemloft.net Tue Aug 28 13:43:17 2007 From: davem at davemloft.net (David Miller) Date: Tue, 28 Aug 2007 13:43:17 -0700 (PDT) Subject: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space. In-Reply-To: References: <20070820.235804.85409183.davem@davemloft.net> Message-ID: <20070828.134317.118522881.davem@davemloft.net> From: Roland Dreier Date: Tue, 28 Aug 2007 12:38:07 -0700 > It seems that the NIC would also have to look into a TCP stream (and > handle out of order segments etc) to find message boundaries for this > to be equivalent to what an RDMA NIC does. It would work for data that accumulates in-order, give or take a small window, just like LRO does. From sashak at voltaire.com Tue Aug 28 15:12:38 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 01:12:38 +0300 Subject: [ofa-general] Re: [PATCH] opensm: serialize osm_state_mgr_process() In-Reply-To: <20070828113424.GB18082@sashak.voltaire.com> References: <20070828113424.GB18082@sashak.voltaire.com> Message-ID: <20070828221238.GX18082@sashak.voltaire.com> On 14:34 Tue 28 Aug , Sasha Khapyorsky wrote: > [snip...] > diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c > index 6fc9496..c052456 100644 > --- a/opensm/opensm/osm_sm.c > +++ b/opensm/opensm/osm_sm.c > @@ -69,50 +69,51 @@ > > /********************************************************************** > **********************************************************************/ > -void __osm_sm_sweeper(IN void *p_ptr) > +static void __osm_sm_sweeper(IN void *p_ptr) > { > ib_api_status_t status; > osm_sm_t *const p_sm = (osm_sm_t *) p_ptr; > + unsigned signals, i; > > OSM_LOG_ENTER(p_sm->p_log, __osm_sm_sweeper); > > - if (p_sm->thread_state == OSM_THREAD_STATE_INIT) { > - p_sm->thread_state = OSM_THREAD_STATE_RUN; > - } > - > - /* If the sweep interval was updated before - then run only if > - * it is not zero. */ > - while (p_sm->thread_state == OSM_THREAD_STATE_RUN && > - p_sm->p_subn->opt.sweep_interval != 0) { > - /* do the sweep only if we are in MASTER state */ > - if (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || > - p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING) > - osm_state_mgr_process(&p_sm->state_mgr, > - OSM_SIGNAL_SWEEP); > - > + while (p_sm->thread_state == OSM_THREAD_STATE_RUN) { > /* > * Wait on the event with a timeout. > * Sweeps may be initiated "off schedule" by simply > * signaling the event. > */ > - status = cl_event_wait_on(&p_sm->signal, > + status = cl_event_wait_on(&p_sm->signal_event, > + p_sm->p_subn->opt.sweep_interval ? > p_sm->p_subn->opt.sweep_interval * > - 1000000, TRUE); > - > - if (status == CL_SUCCESS) { > - if (osm_log_is_active(p_sm->p_log, OSM_LOG_DEBUG)) { > - osm_log(p_sm->p_log, OSM_LOG_DEBUG, > - "__osm_sm_sweeper: " > - "Off schedule sweep signalled\n"); > - } > - } else { > - if (status != CL_TIMEOUT) { > - osm_log(p_sm->p_log, OSM_LOG_ERROR, > - "__osm_sm_sweeper: ERR 2E01: " > - "Event wait failed (%s)\n", > - CL_STATUS_MSG(status)); > - } > + 1000000 : EVENT_NO_TIMEOUT, TRUE); > + > + if (status == CL_SUCCESS) > + osm_log(p_sm->p_log, OSM_LOG_DEBUG, > + "__osm_sm_sweeper: " > + "Off schedule sweep signalled\n"); > + else if (status != CL_TIMEOUT) { > + osm_log(p_sm->p_log, OSM_LOG_ERROR, > + "__osm_sm_sweeper: ERR 2E01: " > + "Event wait failed (%s)\n", > + CL_STATUS_MSG(status)); > + continue; > } > + > + cl_spinlock_acquire(&p_sm->signal_lock); > + signals = p_sm->signal_mask; > + p_sm->signal_mask = 0; > + cl_spinlock_release(&p_sm->signal_lock); > + > + /* do the sweep only if we are in MASTER state */ > + if (status == CL_TIMEOUT && > + (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || > + p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING)) > + p_sm->signal_mask |= 1 << OSM_SIGNAL_SWEEP; Actually here should be just: signals != 1 << OSM_SIGNAL_SWEEP; > + > + for (i = 0 ; signals ; signals >>= 1 , i++) > + if (signals&1) > + osm_state_mgr_process(&p_sm->state_mgr, i); > } > > OSM_LOG_EXIT(p_sm->p_log); From ralph.campbell at qlogic.com Tue Aug 28 15:48:08 2007 From: ralph.campbell at qlogic.com (Ralph Campbell) Date: Tue, 28 Aug 2007 15:48:08 -0700 Subject: [ofa-general] [PATCH] IB/libipathverbs - Add recognition for new QLogic device IDs Message-ID: <1188341288.3730.180.camel@brick.pathscale.com> This patch adds an additional device ID to recognize a new HCA. This change has been committed to git://git.openfabrics.org/~ralphc/libipathverbs/.git and I would like to see this version of libipathverbs become part of OFED 1.3. Signed-off-by: Ralph Campbell diff --git a/src/ipathverbs.c b/src/ipathverbs.c index f3e2fde..293dba8 100644 --- a/src/ipathverbs.c +++ b/src/ipathverbs.c @@ -70,6 +70,10 @@ #define PCI_DEVICE_ID_INFINIPATH_6220 0x6220 #endif +#ifndef PCI_DEVICE_ID_INFINIPATH_7220 +#define PCI_DEVICE_ID_INFINIPATH_7220 0x7220 +#endif + #define HCA(v, d, t) \ { .vendor = PCI_VENDOR_ID_##v, \ .device = PCI_DEVICE_ID_INFINIPATH_##d, \ @@ -83,7 +87,8 @@ struct { HCA(PATHSCALE, SPINNERET, SPINNERET), HCA(PATHSCALE, HT, HT), HCA(PATHSCALE, PE800, PE800), - HCA(QLOGIC, 6220, 6220), + HCA(QLOGIC, 6220, 7220), + HCA(QLOGIC, 7220, 7220), }; static struct ibv_context_ops ipath_ctx_ops = { diff --git a/src/ipathverbs.h b/src/ipathverbs.h index 43eee82..803ca70 100644 --- a/src/ipathverbs.h +++ b/src/ipathverbs.h @@ -55,7 +55,7 @@ enum ipath_hca_type { IPATH_SPINNERET, IPATH_HT, IPATH_PE800, - IPATH_6220, + IPATH_7220, }; struct ipath_device { From ablewers at fujitec.com Tue Aug 28 16:01:33 2007 From: ablewers at fujitec.com (ablewers at fujitec.com) Date: Tue, 28 Aug 2007 19:01:33 -0400 Subject: [ofa-general] here comes the bell Message-ID: <000601c7e9c7$61122010$789354a7@bvy> ERMX Acquires Gladiator Challenge. EntreMetrix Inc. $0.087 Brokers read the news and are grabbing big share blocks today. Do they have the inside track on this one? Opening Bell is coming, get ready and grab ERMX. From fubar at us.ibm.com Tue Aug 28 16:26:44 2007 From: fubar at us.ibm.com (Jay Vosburgh) Date: Tue, 28 Aug 2007 16:26:44 -0700 Subject: [ofa-general] Re: [PATCH V4 8/10] net/bonding: Handlle wrong assumptions that slave is always an Ethernet device In-Reply-To: <46C9B8D9.5060508@voltaire.com> References: <46C9B474.5020202@voltaire.com> <46C9B8D9.5060508@voltaire.com> Message-ID: <416.1188343604@death> Moni Shoua wrote: >bonding sometimes uses Ethernet constants (such as MTU and address length) which >are not good when it enslaves non Ethernet devices (such as InfiniBand). > >Signed-off-by: Moni Shoua >--- > drivers/net/bonding/bond_main.c | 3 ++- > drivers/net/bonding/bond_sysfs.c | 19 +++++++++++++------ > drivers/net/bonding/bonding.h | 1 + > 3 files changed, 16 insertions(+), 7 deletions(-) > >Index: net-2.6/drivers/net/bonding/bond_main.c >=================================================================== >--- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:55:48.000000000 +0300 >+++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-20 14:29:11.911298577 +0300 >@@ -1224,7 +1224,8 @@ static int bond_compute_features(struct > struct slave *slave; > struct net_device *bond_dev = bond->dev; > unsigned long features = bond_dev->features; >- unsigned short max_hard_header_len = ETH_HLEN; >+ unsigned short max_hard_header_len = max((u16)ETH_HLEN, >+ bond_dev->hard_header_len); Since non-IB bonding masters are run through ether_setup, which sets hard_header_len to ETH_HLEN, the max() is probably unnecessary, and I think this could just be bond_dev->hard_header_len. -J --- -Jay Vosburgh, IBM Linux Technology Center, fubar at us.ibm.com From fubar at us.ibm.com Tue Aug 28 16:33:06 2007 From: fubar at us.ibm.com (Jay Vosburgh) Date: Tue, 28 Aug 2007 16:33:06 -0700 Subject: [ofa-general] Re: [PATCH V4 10/10] net/bonding: Destroy bonding master when last slave is gone In-Reply-To: <46C9BA2D.7060204@voltaire.com> References: <46C9B474.5020202@voltaire.com> <46C9BA2D.7060204@voltaire.com> Message-ID: <3403.1188343986@death> Moni Shoua wrote: >When bonding enslaves non Ethernet devices it takes pointers to functions >in the module that owns the slaves. In this case it becomes unsafe >to keep the bonding master registered after last slave was unenslaved >because we don't know if the pointers are still valid. Destroying the bond when slave_cnt is zero >ensures that these functions be used anymore. Would it not be simpler to run the bonding master through ether_setup() again when the final slave is released (to reset all of the pointers to their "ethernet" values)? I'm presuming here the pointers of questionable validity are the ones set in the bond_setup_by_slave() copied from the slave_dev->hard_header, et al. Having the bonding master disappear (but only sometimes) after the last slave is removed is a semantic change I'd rather not introduce if it's not necessary. -J --- -Jay Vosburgh, IBM Linux Technology Center, fubar at us.ibm.com From bzeltmf at spanglercandy.com Tue Aug 28 09:55:11 2007 From: bzeltmf at spanglercandy.com (will) Date: Tue, 28 Aug 2007 20:55:11 +0400 Subject: [ofa-general] Re: Message-ID: <94C6F7BFACC8862.F7CD5B1EBB@spanglercandy.com> see my craft auctions. really try best send auction. Free Credit Report Score From Experian. Learn Show StatesUS delivery Country: costs:US include full Seller this itemSign for your making projects decor using cords. topics knot index basic There are different designs -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bgimg.gif Type: image/gif Size: 4870 bytes Desc: not available URL: From sashak at voltaire.com Tue Aug 28 17:57:30 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 03:57:30 +0300 Subject: [ofa-general] Re: [PATCH] osm: QoS - remove 'yyunput' compilation warning In-Reply-To: <46D4360A.7090602@dev.mellanox.co.il> References: <46D4360A.7090602@dev.mellanox.co.il> Message-ID: <20070829005730.GD18082@sashak.voltaire.com> On 17:49 Tue 28 Aug , Yevgeny Kliteynik wrote: > Remove 'yyunput defined but not used' compilation warning > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From sashak at voltaire.com Tue Aug 28 17:57:57 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 03:57:57 +0300 Subject: [ofa-general] Re: [PATCH] osm: QoS - destroy qos_policy when destroying subnet In-Reply-To: <46D43B27.5040808@dev.mellanox.co.il> References: <46D43B27.5040808@dev.mellanox.co.il> Message-ID: <20070829005757.GE18082@sashak.voltaire.com> On 18:11 Tue 28 Aug , Yevgeny Kliteynik wrote: > Destroy p_subn->p_qos_policy when destroying subnet > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. Sasha From krkumar2 at in.ibm.com Tue Aug 28 20:23:30 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 29 Aug 2007 08:53:30 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070821.001824.105429474.davem@davemloft.net> Message-ID: Hi Dave, I am scp'ng from 192.168.1.1 to 192.168.1.2 and captured at the send side. 192.168.1.1.37201 > 192.168.1.2.ssh: P 837178092:837178596(504) ack 1976304527 win 79 192.168.1.1.37201 > 192.168.1.2.ssh: . 837178596:837181492(2896) ack 1976304527 win 79 192.168.1.1.37201 > 192.168.1.2.ssh: . 837181492:837184388(2896) ack 1976304527 win 79 192.168.1.1.37201 > 192.168.1.2.ssh: . 837184388:837188732(4344) ack 1976304527 win 79 192.168.1.1.37201 > 192.168.1.2.ssh: . 837188732:837193076(4344) ack 1976304527 win 79 192.168.1.1.37201 > 192.168.1.2.ssh: . 837193076:837194524(1448) ack 1976304527 win 79 192.168.1.2.ssh > 192.168.1.1.37201: . ack 837165060 win 3338 Data in pipeline: 837194524 - 837165060 = 29464. In most cases, I am getting 7K, 8K, 13K, and rarely close to 16K. I ran iperf with 4K, 16K & 32K (as I could do multiple threads instead of single process). Results are (for E1000 82547GI chipset, BW in KB/s): Test Org BW New BW % Size:4096 Procs:1 114612 114644 .02 Size:16394 Procs:1 114634 114644 0 Size:32768 Procs:1 114645 114643 0 And for multiple threads: Test Org BW New BW % Size:4096 Procs:8 114632 114637 0 Size:4096 Procs:16 114639 114637 0 Size:4096 Procs:64 114893 114800 -.08 Size:16394 Procs:8 114641 114642 0 Size:16394 Procs:16 114642 114643 0 Size:16394 Procs:64 114911 114781 -.11 Size:32768 Procs:8 114638 114639 0 Size:32768 Procs:16 114642 114645 0 Size:32768 Procs:64 114932 114777 -.13 I will run netperf and report CPU utilization too. Thanks, - KK David Miller wrote on 08/21/2007 12:48:24 PM: > From: Krishna Kumar2 > Date: Fri, 17 Aug 2007 11:36:03 +0530 > > > > I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will > > > run a longer one tonight). The results are (results in KB/s, and %): > > > > I ran a 8.5 hours run with no batching + another 8.5 hours run with > > batching (Buffer sizes: "32 128 512 4096 16384", Threads: "1 8 32", > > Each test run time: 3 minutes, Iterations to average: 5). TCP seems > > to get a small improvement. > > Using 16K buffer size really isn't going to keep the pipe full enough > for TSO. And realistically applications queue much more data at a > time. Also, with smaller buffer sizes can have negative effects for > the dynamic receive and send buffer growth algorithm the kernel uses, > it might consider the connection application limited for too long. > > I would really prefer to see numbers that use buffer sizes more in > line with the amount of data that is typically inflight on a 1G > connection on a local network. > > Do a tcpdump during the height of the transfer to see about what this > value is. When an ACK comes in, compare the sequence number it's > ACK'ing with the sequence number of the most recently sent frame. > The difference is approximately the pipe size at maximum congestion > window assuming a loss free local network. From kliteyn at mellanox.co.il Tue Aug 28 21:35:35 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 29 Aug 2007 07:35:35 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-29:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Tue_Aug_28_15:54:20_2007 [a3c4cb3f7a0c4b5cf764e874314fa29ae872f4b0] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From davem at davemloft.net Tue Aug 28 21:51:50 2007 From: davem at davemloft.net (David Miller) Date: Tue, 28 Aug 2007 21:51:50 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070821.001824.105429474.davem@davemloft.net> Message-ID: <20070828.215150.98552996.davem@davemloft.net> From: Krishna Kumar2 Date: Wed, 29 Aug 2007 08:53:30 +0530 > I am scp'ng from 192.168.1.1 to 192.168.1.2 and captured at the send > side. Bad choice of test, this is cpu limited since the scp has to encrypt and MAC hash all the data it sends. Use something like straight ftp or "bw_tcp" from lmbench. Using a different tool seems strange to me, why not just adjust the buffer size with command line options in the benchmark you were using in the first place? From krkumar2 at in.ibm.com Tue Aug 28 22:13:23 2007 From: krkumar2 at in.ibm.com (Krishna Kumar2) Date: Wed, 29 Aug 2007 10:43:23 +0530 Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070828.215150.98552996.davem@davemloft.net> Message-ID: netdev-owner at vger.kernel.org wrote on 08/29/2007 10:21:50 AM: > From: Krishna Kumar2 > Date: Wed, 29 Aug 2007 08:53:30 +0530 > > > I am scp'ng from 192.168.1.1 to 192.168.1.2 and captured at the send > > side. > > Bad choice of test, this is cpu limited since the scp > has to encrypt and MAC hash all the data it sends. > > Use something like straight ftp or "bw_tcp" from lmbench. OK > Using a different tool seems strange to me, why not just adjust the > buffer size with command line options in the benchmark you were using > in the first place? The reason was to run parallel copies, not for buffer limitations. Let me use the same tool for benchmark. Thanks, - KK From davem at davemloft.net Tue Aug 28 22:14:48 2007 From: davem at davemloft.net (David Miller) Date: Tue, 28 Aug 2007 22:14:48 -0700 (PDT) Subject: [ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: References: <20070828.215150.98552996.davem@davemloft.net> Message-ID: <20070828.221448.123919361.davem@davemloft.net> From: Krishna Kumar2 Date: Wed, 29 Aug 2007 10:43:23 +0530 > The reason was to run parallel copies, not for buffer limitations. Oh, I see. I'll note in passing that current lmbench-3 has some parallelization features you could play with, you might want to check it out. From kliteyn at dev.mellanox.co.il Tue Aug 28 23:42:06 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 29 Aug 2007 09:42:06 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> Message-ID: <46D5153E.6030307@dev.mellanox.co.il> Hal Rosenstock wrote: > On 8/27/07, Yevgeny Kliteynik wrote: >> Adding QoS policy file option to OpenSM optionsm and >> QoS Policy field to subn object. >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ >> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ >> opensm/opensm/main.c | 12 +++++++++++- >> opensm/opensm/osm_subnet.c | 10 +++++++++- >> 4 files changed, 50 insertions(+), 2 deletions(-) >> >> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h >> index 545779b..0a35e22 100644 >> --- a/opensm/include/opensm/osm_base.h >> +++ b/opensm/include/opensm/osm_base.h >> @@ -224,6 +224,23 @@ BEGIN_C_DECLS >> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" >> #endif >> /***********/ >> + >> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE >> +* NAME >> +* OSM_DEFAULT_QOS_POLICY_FILE >> +* >> +* DESCRIPTION >> +* Specifies the default QoS policy file name >> +* >> +* SYNOPSIS >> +*/ >> +#ifdef __WIN__ >> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") >> +#else >> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" >> +#endif >> +/***********/ >> + >> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS >> * NAME >> * OSM_DEFAULT_SWEEP_INTERVAL_SECS >> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h >> index a7543dc..5e8b522 100644 >> --- a/opensm/include/opensm/osm_subnet.h >> +++ b/opensm/include/opensm/osm_subnet.h >> @@ -68,6 +68,7 @@ BEGIN_C_DECLS >> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 >> #define OSM_SUBNET_VECTOR_CAPACITY 256 >> struct _osm_opensm_t; >> +struct _osm_qos_policy_t; >> >> /****h* OpenSM/Subnet >> * NAME >> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { >> char *partition_config_file; >> boolean_t no_partition_enforcement; >> boolean_t no_qos; >> + char *qos_policy_file; >> boolean_t accum_log_file; >> char *console; >> uint16_t console_port; >> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { >> * specified the log file will be truncated upon reaching >> * this limit. >> * >> +* no_qos >> +* Boolean that specifies whether the OpenSM QoS functionality >> +* should be off or on. >> +* >> +* qos_policy_file >> +* Name of the QoS policy file. >> +* > > Are these options mutually exclusive or not ? If QoS in OpenSM is off, the qos_policy_file will be ignored >> * accum_log_file >> * If TRUE (default) - the log file will be accumulated. >> * If FALSE - the log file will be erased before starting current opensm run. >> @@ -551,6 +560,7 @@ typedef struct _osm_subn { >> ib_net64_t sm_port_guid; >> uint8_t sm_state; >> osm_subn_opt_t opt; >> + struct _osm_qos_policy_t *p_qos_policy; >> uint16_t max_unicast_lid_ho; >> uint16_t max_multicast_lid_ho; >> uint8_t min_ca_mtu; >> @@ -619,6 +629,9 @@ typedef struct _osm_subn { >> * opt >> * Subnet options structure contains site specific configuration. >> * >> +* p_qos_policy >> +* Subnet QoS policy structure. >> +* >> * max_unicast_lid_ho >> * The minimal max unicast lid reported by all switches >> * >> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c >> index e2541d8..599c4dc 100644 >> --- a/opensm/opensm/main.c >> +++ b/opensm/opensm/main.c >> @@ -269,6 +269,10 @@ void show_usage(void) >> " The default name is \'" >> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); >> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); >> + printf("-Y\n" >> + "--qos_policy_file\n" >> + " This option defines the optional QoS policy file.\n" >> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); >> printf("-N\n" >> "--no_part_enforce\n" >> " This option disables partition enforcement on switch external ports.\n\n"); >> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) >> char *ignore_guids_file_name = NULL; >> uint32_t val; >> const char *const short_option = >> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; >> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; >> >> /* >> In the array below, the 2nd parameter specifies the number >> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) >> {"Pconfig", 1, NULL, 'P'}, >> {"no_part_enforce", 0, NULL, 'N'}, >> {"qos", 0, NULL, 'Q'}, >> + {"qos_policy_file", 1, NULL, 'Y'}, >> {"maxsmps", 1, NULL, 'n'}, >> {"console", 1, NULL, 'q'}, >> {"V", 0, NULL, 'V'}, >> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) >> opt.no_qos = FALSE; >> break; >> >> + case 'Y': >> + opt.qos_policy_file = optarg; >> + printf(" QoS policy file \'%s\'\n", optarg); >> + break; >> + > > There should also be an update to the OpenSM man page for this. That and a text file with description of policy file syntax -- Yevgeny > > -- Hal > >> case 'y': >> opt.exit_on_fatal = FALSE; >> printf(" Staying on fatal initialization errors\n"); >> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c >> index 818d73f..4162522 100644 >> --- a/opensm/opensm/osm_subnet.c >> +++ b/opensm/opensm/osm_subnet.c >> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) >> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; >> p_opt->no_partition_enforcement = FALSE; >> p_opt->no_qos = TRUE; >> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; >> p_opt->accum_log_file = TRUE; >> p_opt->port_profile_switch_nodes = FALSE; >> p_opt->pfn_ui_pre_lid_assign = NULL; >> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) >> >> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); >> >> + opts_unpack_charp("qos_policy_file", >> + p_key, p_val, &p_opts->qos_policy_file); >> + >> opts_unpack_boolean("accum_log_file", >> p_key, p_val, &p_opts->accum_log_file); >> >> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) >> fprintf(opts_file, >> "#\n# QoS OPTIONS\n#\n" >> "# Disable QoS setup\n" >> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); >> + "no_qos %s\n\n" >> + "# QoS policy file to be used\n" >> + "qos_policy_file %s\n\n", >> + p_opts->no_qos ? "TRUE" : "FALSE", >> + p_opts->qos_policy_file); >> >> subn_dump_qos_options(opts_file, >> "QoS default options", "qos", >> -- >> 1.5.1.4 >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > From vlad at lists.openfabrics.org Wed Aug 29 02:48:16 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Wed, 29 Aug 2007 02:48:16 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070829-0200 daily build status Message-ID: <20070829094817.80B31E603C4@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070829-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From hal.rosenstock at gmail.com Wed Aug 29 02:57:49 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 29 Aug 2007 05:57:49 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D5153E.6030307@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> Message-ID: On 8/29/07, Yevgeny Kliteynik wrote: > Hal Rosenstock wrote: > > On 8/27/07, Yevgeny Kliteynik wrote: > >> Adding QoS policy file option to OpenSM optionsm and > >> QoS Policy field to subn object. > >> > >> Signed-off-by: Yevgeny Kliteynik > >> --- > >> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > >> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > >> opensm/opensm/main.c | 12 +++++++++++- > >> opensm/opensm/osm_subnet.c | 10 +++++++++- > >> 4 files changed, 50 insertions(+), 2 deletions(-) > >> > >> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h > >> index 545779b..0a35e22 100644 > >> --- a/opensm/include/opensm/osm_base.h > >> +++ b/opensm/include/opensm/osm_base.h > >> @@ -224,6 +224,23 @@ BEGIN_C_DECLS > >> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" > >> #endif > >> /***********/ > >> + > >> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > >> +* NAME > >> +* OSM_DEFAULT_QOS_POLICY_FILE > >> +* > >> +* DESCRIPTION > >> +* Specifies the default QoS policy file name > >> +* > >> +* SYNOPSIS > >> +*/ > >> +#ifdef __WIN__ > >> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") > >> +#else > >> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" > >> +#endif > >> +/***********/ > >> + > >> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > >> * NAME > >> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > >> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > >> index a7543dc..5e8b522 100644 > >> --- a/opensm/include/opensm/osm_subnet.h > >> +++ b/opensm/include/opensm/osm_subnet.h > >> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > >> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > >> #define OSM_SUBNET_VECTOR_CAPACITY 256 > >> struct _osm_opensm_t; > >> +struct _osm_qos_policy_t; > >> > >> /****h* OpenSM/Subnet > >> * NAME > >> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > >> char *partition_config_file; > >> boolean_t no_partition_enforcement; > >> boolean_t no_qos; > >> + char *qos_policy_file; > >> boolean_t accum_log_file; > >> char *console; > >> uint16_t console_port; > >> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > >> * specified the log file will be truncated upon reaching > >> * this limit. > >> * > >> +* no_qos > >> +* Boolean that specifies whether the OpenSM QoS functionality > >> +* should be off or on. > >> +* > >> +* qos_policy_file > >> +* Name of the QoS policy file. > >> +* > > > > Are these options mutually exclusive or not ? > > If QoS in OpenSM is off, the qos_policy_file will be ignored But aren't there two QoSs ? Can't the old QoS be run without this ? I think that is a requirement. > >> * accum_log_file > >> * If TRUE (default) - the log file will be accumulated. > >> * If FALSE - the log file will be erased before starting current opensm run. > >> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > >> ib_net64_t sm_port_guid; > >> uint8_t sm_state; > >> osm_subn_opt_t opt; > >> + struct _osm_qos_policy_t *p_qos_policy; > >> uint16_t max_unicast_lid_ho; > >> uint16_t max_multicast_lid_ho; > >> uint8_t min_ca_mtu; > >> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > >> * opt > >> * Subnet options structure contains site specific configuration. > >> * > >> +* p_qos_policy > >> +* Subnet QoS policy structure. > >> +* > >> * max_unicast_lid_ho > >> * The minimal max unicast lid reported by all switches > >> * > >> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > >> index e2541d8..599c4dc 100644 > >> --- a/opensm/opensm/main.c > >> +++ b/opensm/opensm/main.c > >> @@ -269,6 +269,10 @@ void show_usage(void) > >> " The default name is \'" > >> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > >> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > >> + printf("-Y\n" > >> + "--qos_policy_file\n" > >> + " This option defines the optional QoS policy file.\n" > >> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > >> printf("-N\n" > >> "--no_part_enforce\n" > >> " This option disables partition enforcement on switch external ports.\n\n"); > >> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > >> char *ignore_guids_file_name = NULL; > >> uint32_t val; > >> const char *const short_option = > >> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > >> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > >> > >> /* > >> In the array below, the 2nd parameter specifies the number > >> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > >> {"Pconfig", 1, NULL, 'P'}, > >> {"no_part_enforce", 0, NULL, 'N'}, > >> {"qos", 0, NULL, 'Q'}, > >> + {"qos_policy_file", 1, NULL, 'Y'}, > >> {"maxsmps", 1, NULL, 'n'}, > >> {"console", 1, NULL, 'q'}, > >> {"V", 0, NULL, 'V'}, > >> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > >> opt.no_qos = FALSE; > >> break; > >> > >> + case 'Y': > >> + opt.qos_policy_file = optarg; > >> + printf(" QoS policy file \'%s\'\n", optarg); > >> + break; > >> + > > > > There should also be an update to the OpenSM man page for this. > > That and a text file with description of policy file syntax Good. Thanks. -- Hal > > -- Yevgeny > > > > > -- Hal > > > >> case 'y': > >> opt.exit_on_fatal = FALSE; > >> printf(" Staying on fatal initialization errors\n"); > >> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > >> index 818d73f..4162522 100644 > >> --- a/opensm/opensm/osm_subnet.c > >> +++ b/opensm/opensm/osm_subnet.c > >> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > >> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > >> p_opt->no_partition_enforcement = FALSE; > >> p_opt->no_qos = TRUE; > >> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > >> p_opt->accum_log_file = TRUE; > >> p_opt->port_profile_switch_nodes = FALSE; > >> p_opt->pfn_ui_pre_lid_assign = NULL; > >> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > >> > >> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > >> > >> + opts_unpack_charp("qos_policy_file", > >> + p_key, p_val, &p_opts->qos_policy_file); > >> + > >> opts_unpack_boolean("accum_log_file", > >> p_key, p_val, &p_opts->accum_log_file); > >> > >> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > >> fprintf(opts_file, > >> "#\n# QoS OPTIONS\n#\n" > >> "# Disable QoS setup\n" > >> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > >> + "no_qos %s\n\n" > >> + "# QoS policy file to be used\n" > >> + "qos_policy_file %s\n\n", > >> + p_opts->no_qos ? "TRUE" : "FALSE", > >> + p_opts->qos_policy_file); > >> > >> subn_dump_qos_options(opts_file, > >> "QoS default options", "qos", > >> -- > >> 1.5.1.4 > >> > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >> > > > > From kliteyn at dev.mellanox.co.il Wed Aug 29 04:45:11 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 29 Aug 2007 14:45:11 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> Message-ID: <46D55C47.6080705@dev.mellanox.co.il> Hal Rosenstock wrote: > On 8/29/07, Yevgeny Kliteynik wrote: >> Hal Rosenstock wrote: >>> On 8/27/07, Yevgeny Kliteynik wrote: >>>> Adding QoS policy file option to OpenSM optionsm and >>>> QoS Policy field to subn object. >>>> >>>> Signed-off-by: Yevgeny Kliteynik >>>> --- >>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ >>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ >>>> opensm/opensm/main.c | 12 +++++++++++- >>>> opensm/opensm/osm_subnet.c | 10 +++++++++- >>>> 4 files changed, 50 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h >>>> index 545779b..0a35e22 100644 >>>> --- a/opensm/include/opensm/osm_base.h >>>> +++ b/opensm/include/opensm/osm_base.h >>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS >>>> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" >>>> #endif >>>> /***********/ >>>> + >>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE >>>> +* NAME >>>> +* OSM_DEFAULT_QOS_POLICY_FILE >>>> +* >>>> +* DESCRIPTION >>>> +* Specifies the default QoS policy file name >>>> +* >>>> +* SYNOPSIS >>>> +*/ >>>> +#ifdef __WIN__ >>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") >>>> +#else >>>> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" >>>> +#endif >>>> +/***********/ >>>> + >>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>> * NAME >>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h >>>> index a7543dc..5e8b522 100644 >>>> --- a/opensm/include/opensm/osm_subnet.h >>>> +++ b/opensm/include/opensm/osm_subnet.h >>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS >>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 >>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 >>>> struct _osm_opensm_t; >>>> +struct _osm_qos_policy_t; >>>> >>>> /****h* OpenSM/Subnet >>>> * NAME >>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { >>>> char *partition_config_file; >>>> boolean_t no_partition_enforcement; >>>> boolean_t no_qos; >>>> + char *qos_policy_file; >>>> boolean_t accum_log_file; >>>> char *console; >>>> uint16_t console_port; >>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { >>>> * specified the log file will be truncated upon reaching >>>> * this limit. >>>> * >>>> +* no_qos >>>> +* Boolean that specifies whether the OpenSM QoS functionality >>>> +* should be off or on. >>>> +* >>>> +* qos_policy_file >>>> +* Name of the QoS policy file. >>>> +* >>> Are these options mutually exclusive or not ? >> If QoS in OpenSM is off, the qos_policy_file will be ignored > > But aren't there two QoSs ? Can't the old QoS be run without this ? I > think that is a requirement. OK, there are three ways we want this thing to work: 1. QoS is off 2. The old QoS is on but w/o policy file 3. The old QoS is on, plus reading policy file The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. Second and third options: if QoS is on, OpenSM looks for policy file in the default location or in other location that was provided by user. If the file is not found, QoS works as before. Do we want to add additional option for "enhanced" QoS? If so, we will have three QoS-ralated command line options: - option for turning the QoS on (currently -Q) - option to turn the new QoS on (some new letter - must get one quick before they all run out... :) - option for policy file location if differs from default (currently -Y) Alternatively, we can turn -Q option into levels: -Q 0: QoS is off (default) -Q 1: old QoS is on -Q 2: old QoS plus reading policy file -- Yevgeny >>>> * accum_log_file >>>> * If TRUE (default) - the log file will be accumulated. >>>> * If FALSE - the log file will be erased before starting current opensm run. >>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { >>>> ib_net64_t sm_port_guid; >>>> uint8_t sm_state; >>>> osm_subn_opt_t opt; >>>> + struct _osm_qos_policy_t *p_qos_policy; >>>> uint16_t max_unicast_lid_ho; >>>> uint16_t max_multicast_lid_ho; >>>> uint8_t min_ca_mtu; >>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { >>>> * opt >>>> * Subnet options structure contains site specific configuration. >>>> * >>>> +* p_qos_policy >>>> +* Subnet QoS policy structure. >>>> +* >>>> * max_unicast_lid_ho >>>> * The minimal max unicast lid reported by all switches >>>> * >>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c >>>> index e2541d8..599c4dc 100644 >>>> --- a/opensm/opensm/main.c >>>> +++ b/opensm/opensm/main.c >>>> @@ -269,6 +269,10 @@ void show_usage(void) >>>> " The default name is \'" >>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); >>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); >>>> + printf("-Y\n" >>>> + "--qos_policy_file\n" >>>> + " This option defines the optional QoS policy file.\n" >>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); >>>> printf("-N\n" >>>> "--no_part_enforce\n" >>>> " This option disables partition enforcement on switch external ports.\n\n"); >>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) >>>> char *ignore_guids_file_name = NULL; >>>> uint32_t val; >>>> const char *const short_option = >>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; >>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; >>>> >>>> /* >>>> In the array below, the 2nd parameter specifies the number >>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) >>>> {"Pconfig", 1, NULL, 'P'}, >>>> {"no_part_enforce", 0, NULL, 'N'}, >>>> {"qos", 0, NULL, 'Q'}, >>>> + {"qos_policy_file", 1, NULL, 'Y'}, >>>> {"maxsmps", 1, NULL, 'n'}, >>>> {"console", 1, NULL, 'q'}, >>>> {"V", 0, NULL, 'V'}, >>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) >>>> opt.no_qos = FALSE; >>>> break; >>>> >>>> + case 'Y': >>>> + opt.qos_policy_file = optarg; >>>> + printf(" QoS policy file \'%s\'\n", optarg); >>>> + break; >>>> + >>> There should also be an update to the OpenSM man page for this. >> That and a text file with description of policy file syntax > > Good. Thanks. > > -- Hal > >> -- Yevgeny >> >>> -- Hal >>> >>>> case 'y': >>>> opt.exit_on_fatal = FALSE; >>>> printf(" Staying on fatal initialization errors\n"); >>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c >>>> index 818d73f..4162522 100644 >>>> --- a/opensm/opensm/osm_subnet.c >>>> +++ b/opensm/opensm/osm_subnet.c >>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) >>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; >>>> p_opt->no_partition_enforcement = FALSE; >>>> p_opt->no_qos = TRUE; >>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; >>>> p_opt->accum_log_file = TRUE; >>>> p_opt->port_profile_switch_nodes = FALSE; >>>> p_opt->pfn_ui_pre_lid_assign = NULL; >>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) >>>> >>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); >>>> >>>> + opts_unpack_charp("qos_policy_file", >>>> + p_key, p_val, &p_opts->qos_policy_file); >>>> + >>>> opts_unpack_boolean("accum_log_file", >>>> p_key, p_val, &p_opts->accum_log_file); >>>> >>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) >>>> fprintf(opts_file, >>>> "#\n# QoS OPTIONS\n#\n" >>>> "# Disable QoS setup\n" >>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); >>>> + "no_qos %s\n\n" >>>> + "# QoS policy file to be used\n" >>>> + "qos_policy_file %s\n\n", >>>> + p_opts->no_qos ? "TRUE" : "FALSE", >>>> + p_opts->qos_policy_file); >>>> >>>> subn_dump_qos_options(opts_file, >>>> "QoS default options", "qos", >>>> -- >>>> 1.5.1.4 >>>> >>>> _______________________________________________ >>>> general mailing list >>>> general at lists.openfabrics.org >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>> >>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >>>> >> > From KatedoorknobFair at pewinternet.org Tue Aug 28 23:21:42 2007 From: KatedoorknobFair at pewinternet.org (Miranda Feliciano) Date: Wed, 29 Aug 2007 09:21:42 +0300 Subject: [ofa-general] Re: Thank you, we accepted your company application Message-ID: <797c01c7ea3a$55a72a90$0201010a@servidor> Your credit doesn't matter to us! If you have your own business and require IMMEDIATE money to spend ANY way you like or need Extra money to give your business a boost or need A low interest loan - NO STRINGS ATTACHED, here is best deal we can offer you TONIGHT (hurry, this deal will expire THIS NIGHT): $68,000+ loan Hurry, when our best deal is gone, it is gone. Simply Call Us... Don't worry about approval, your credit score will not disqualify you! Call Us Free on 866-847-0715 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hal.rosenstock at gmail.com Wed Aug 29 06:07:45 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 29 Aug 2007 09:07:45 -0400 Subject: [ofa-general] [PATCH] Opensm/Makefile.am: Change for flex version 2.5.4 Message-ID: opensm/Makefile.am: Change for flex version 2.5.4 Signed-off-by: Hal Rosenstock diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index b738cbe..abfa913 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -63,7 +63,7 @@ osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_q $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h - $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l + $(LEX) -P__qos_parser_ -o$(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l if OSMV_OPENIB opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 From sashak at voltaire.com Wed Aug 29 06:34:40 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 16:34:40 +0300 Subject: [ofa-general] Re: [PATCH] Opensm/Makefile.am: Change for flex version 2.5.4 In-Reply-To: References: Message-ID: <20070829133440.GW18082@sashak.voltaire.com> On 09:07 Wed 29 Aug , Hal Rosenstock wrote: > opensm/Makefile.am: Change for flex version 2.5.4 > > Signed-off-by: Hal Rosenstock Applied. Thanks. Sasha From anton at samba.org Wed Aug 29 06:36:22 2007 From: anton at samba.org (Anton Blanchard) Date: Wed, 29 Aug 2007 08:36:22 -0500 Subject: [ofa-general] [PATCH] Clean up some error messages in fmr_pool.c Message-ID: <20070829133622.GA5289@kryten> Hi, A number of printks in fmr_pool.c dont have newlines, eg: fmr_create failed for FMR 0<5>FS-Cache: Loaded Fix them up. Signed-off-by: Anton Blanchard --- diff --git a/drivers/infiniband/core/fmr_pool.c b/drivers/infiniband/core/fmr_pool.c index a06bcc6..d7f6452 100644 --- a/drivers/infiniband/core/fmr_pool.c +++ b/drivers/infiniband/core/fmr_pool.c @@ -152,7 +152,7 @@ static void ib_fmr_batch_release(struct ib_fmr_pool *pool) #ifdef DEBUG if (fmr->ref_count !=0) { - printk(KERN_WARNING PFX "Unmapping FMR 0x%08x with ref count %d", + printk(KERN_WARNING PFX "Unmapping FMR 0x%08x with ref count %d\n", fmr, fmr->ref_count); } #endif @@ -170,7 +170,7 @@ static void ib_fmr_batch_release(struct ib_fmr_pool *pool) ret = ib_unmap_fmr(&fmr_list); if (ret) - printk(KERN_WARNING PFX "ib_unmap_fmr returned %d", ret); + printk(KERN_WARNING PFX "ib_unmap_fmr returned %d\n", ret); spin_lock_irq(&pool->pool_lock); list_splice(&unmap_list, &pool->free_list); @@ -235,13 +235,13 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, attr = kmalloc(sizeof *attr, GFP_KERNEL); if (!attr) { - printk(KERN_WARNING PFX "couldn't allocate device attr struct"); + printk(KERN_WARNING PFX "couldn't allocate device attr struct\n"); return ERR_PTR(-ENOMEM); } ret = ib_query_device(device, attr); if (ret) { - printk(KERN_WARNING PFX "couldn't query device: %d", ret); + printk(KERN_WARNING PFX "couldn't query device: %d\n", ret); kfree(attr); return ERR_PTR(ret); } @@ -255,7 +255,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, pool = kmalloc(sizeof *pool, GFP_KERNEL); if (!pool) { - printk(KERN_WARNING PFX "couldn't allocate pool struct"); + printk(KERN_WARNING PFX "couldn't allocate pool struct\n"); return ERR_PTR(-ENOMEM); } @@ -272,7 +272,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, kmalloc(IB_FMR_HASH_SIZE * sizeof *pool->cache_bucket, GFP_KERNEL); if (!pool->cache_bucket) { - printk(KERN_WARNING PFX "Failed to allocate cache in pool"); + printk(KERN_WARNING PFX "Failed to allocate cache in pool\n"); ret = -ENOMEM; goto out_free_pool; } @@ -296,7 +296,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, "ib_fmr(%s)", device->name); if (IS_ERR(pool->thread)) { - printk(KERN_WARNING PFX "couldn't start cleanup thread"); + printk(KERN_WARNING PFX "couldn't start cleanup thread\n"); ret = PTR_ERR(pool->thread); goto out_free_pool; } @@ -314,7 +314,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, GFP_KERNEL); if (!fmr) { printk(KERN_WARNING PFX "failed to allocate fmr " - "struct for FMR %d", i); + "struct for FMR %d\n", i); goto out_fail; } @@ -326,7 +326,7 @@ struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd, fmr->fmr = ib_alloc_fmr(pd, params->access, &fmr_attr); if (IS_ERR(fmr->fmr)) { printk(KERN_WARNING PFX "fmr_create failed " - "for FMR %d", i); + "for FMR %d\n", i); kfree(fmr); goto out_fail; } @@ -381,7 +381,7 @@ void ib_destroy_fmr_pool(struct ib_fmr_pool *pool) } if (i < pool->pool_size) - printk(KERN_WARNING PFX "pool still has %d regions registered", + printk(KERN_WARNING PFX "pool still has %d regions registered\n", pool->pool_size - i); kfree(pool->cache_bucket); @@ -518,7 +518,7 @@ int ib_fmr_pool_unmap(struct ib_pool_fmr *fmr) #ifdef DEBUG if (fmr->ref_count < 0) - printk(KERN_WARNING PFX "FMR %p has ref count %d < 0", + printk(KERN_WARNING PFX "FMR %p has ref count %d < 0\n", fmr, fmr->ref_count); #endif From monisonlists at gmail.com Wed Aug 29 06:37:34 2007 From: monisonlists at gmail.com (Moni Shoua) Date: Wed, 29 Aug 2007 16:37:34 +0300 Subject: [ofa-general] Re: [PATCH V4 8/10] net/bonding: Handlle wrong assumptions that slave is always an Ethernet device In-Reply-To: <416.1188343604@death> References: <46C9B474.5020202@voltaire.com> <46C9B8D9.5060508@voltaire.com> <416.1188343604@death> Message-ID: <46D5769E.6000405@gmail.com> Jay Vosburgh wrote: > Moni Shoua wrote: > >> bonding sometimes uses Ethernet constants (such as MTU and address length) which >> are not good when it enslaves non Ethernet devices (such as InfiniBand). >> >> Signed-off-by: Moni Shoua >> --- >> drivers/net/bonding/bond_main.c | 3 ++- >> drivers/net/bonding/bond_sysfs.c | 19 +++++++++++++------ >> drivers/net/bonding/bonding.h | 1 + >> 3 files changed, 16 insertions(+), 7 deletions(-) >> >> Index: net-2.6/drivers/net/bonding/bond_main.c >> =================================================================== >> --- net-2.6.orig/drivers/net/bonding/bond_main.c 2007-08-15 10:55:48.000000000 +0300 >> +++ net-2.6/drivers/net/bonding/bond_main.c 2007-08-20 14:29:11.911298577 +0300 >> @@ -1224,7 +1224,8 @@ static int bond_compute_features(struct >> struct slave *slave; >> struct net_device *bond_dev = bond->dev; >> unsigned long features = bond_dev->features; >> - unsigned short max_hard_header_len = ETH_HLEN; >> + unsigned short max_hard_header_len = max((u16)ETH_HLEN, >> + bond_dev->hard_header_len); > > Since non-IB bonding masters are run through ether_setup, which > sets hard_header_len to ETH_HLEN, the max() is probably unnecessary, and > I think this could just be bond_dev->hard_header_len. > This is true except for the case where there are no slaves left. In that case max_hard_header_len has equals to the initialization value. bond_for_each_slave(bond, slave, i) { features = netdev_compute_features(features, slave->dev->features); if (slave->dev->hard_header_len > max_hard_header_len) max_hard_header_len = slave->dev->hard_header_len; } features |= (bond_dev->features & BOND_VLAN_FEATURES); bond_dev->features = features; bond_dev->hard_header_len = max_hard_header_len; From anton at samba.org Wed Aug 29 06:42:53 2007 From: anton at samba.org (Anton Blanchard) Date: Wed, 29 Aug 2007 08:42:53 -0500 Subject: [ofa-general] [PATCH] Remove verbose debug from ib_echa Message-ID: <20070829134253.GB5289@kryten> Hi, While debugging an ib_ehca issue I ended up with raw hcalls in dmesg: PU0003 EHCA_ERR:ehca_plpar_hcall9 opcode=168 ret=ffffffffffffffd3 arg1=103d001403000002 arg2=203d0000000000a1 arg3=8a40000000000000 arg4=5a5b4c000 arg5=0 arg6=0 arg7=0 arg8=0 arg9=0 out1=0 out2=0 out3=0 out4=2 out5=0 out6=800000000005f254 out7=0 out8=0 out9=0 This simply serves to confuse the user, they wont know that the above error really means "link down". Signed-off-by: Anton Blanchard --- diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c index fdbfebe..2dd98c7 100644 --- a/drivers/infiniband/hw/ehca/hcp_if.c +++ b/drivers/infiniband/hw/ehca/hcp_if.c @@ -132,14 +132,6 @@ static long ehca_plpar_hcall_norets(unsigned long opcode, continue; } - if (ret < H_SUCCESS) - ehca_gen_err("opcode=%lx ret=%lx" - " arg1=%lx arg2=%lx arg3=%lx arg4=%lx" - " arg5=%lx arg6=%lx arg7=%lx ", - opcode, ret, - arg1, arg2, arg3, arg4, arg5, - arg6, arg7); - ehca_gen_dbg("opcode=%lx ret=%lx", opcode, ret); return ret; @@ -188,21 +180,6 @@ static long ehca_plpar_hcall9(unsigned long opcode, continue; } - if (ret < H_SUCCESS) - ehca_gen_err("opcode=%lx ret=%lx" - " arg1=%lx arg2=%lx arg3=%lx arg4=%lx" - " arg5=%lx arg6=%lx arg7=%lx arg8=%lx" - " arg9=%lx" - " out1=%lx out2=%lx out3=%lx out4=%lx" - " out5=%lx out6=%lx out7=%lx out8=%lx" - " out9=%lx", - opcode, ret, - arg1, arg2, arg3, arg4, arg5, - arg6, arg7, arg8, arg9, - outs[0], outs[1], outs[2], outs[3], - outs[4], outs[5], outs[6], outs[7], - outs[8]); - ehca_gen_dbg("opcode=%lx ret=%lx out1=%lx out2=%lx out3=%lx " "out4=%lx out5=%lx out6=%lx out7=%lx out8=%lx " "out9=%lx", From kliteyn at mellanox.co.il Wed Aug 29 06:59:14 2007 From: kliteyn at mellanox.co.il (Yevgeny Kliteynik) Date: Wed, 29 Aug 2007 16:59:14 +0300 Subject: [ofa-general] [PATCH] Opensm/Makefile.am: Change for flex version 2.5.4 In-Reply-To: References: Message-ID: <46D57BB2.40503@mellanox.co.il> Thanks. There's still problem with yacc/bison. I'll issue a patch shortly - looks like I have to go back to moving and renaming files in Makefile, because I can't find the way to tell yacc to generate header with a specific name instead of the default one. -- Yevgeny Hal Rosenstock wrote: > opensm/Makefile.am: Change for flex version 2.5.4 > > Signed-off-by: Hal Rosenstock > > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index b738cbe..abfa913 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -63,7 +63,7 @@ osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y > $(srcdir)/../include/opensm/osm_q > $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o > $(srcdir)/osm_qos_parser_y.c > --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h > --name-prefix=__qos_parser_ > > osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l > $(srcdir)/../include/opensm/osm_qos_policy.h > - $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c > $(srcdir)/osm_qos_parser.l > + $(LEX) -P__qos_parser_ -o$(srcdir)/osm_qos_parser_l.c > $(srcdir)/osm_qos_parser.l > > if OSMV_OPENIB > opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing > -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) > -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > From monisonlists at gmail.com Wed Aug 29 07:06:21 2007 From: monisonlists at gmail.com (Moni Shoua) Date: Wed, 29 Aug 2007 17:06:21 +0300 Subject: [ofa-general] Re: [PATCH V4 10/10] net/bonding: Destroy bonding master when last slave is gone In-Reply-To: <3403.1188343986@death> References: <46C9B474.5020202@voltaire.com> <46C9BA2D.7060204@voltaire.com> <3403.1188343986@death> Message-ID: <46D57D5D.3060706@gmail.com> Jay Vosburgh wrote: > Moni Shoua wrote: > >> When bonding enslaves non Ethernet devices it takes pointers to functions >> in the module that owns the slaves. In this case it becomes unsafe >> to keep the bonding master registered after last slave was unenslaved >> because we don't know if the pointers are still valid. Destroying the bond when slave_cnt is zero >> ensures that these functions be used anymore. > > Would it not be simpler to run the bonding master through > ether_setup() again when the final slave is released (to reset all of > the pointers to their "ethernet" values)? I'm presuming here the > pointers of questionable validity are the ones set in the > bond_setup_by_slave() copied from the slave_dev->hard_header, et al. > > Having the bonding master disappear (but only sometimes) after > the last slave is removed is a semantic change I'd rather not introduce > if it's not necessary. Thanks for the comments. Having the master disappear is one way I could think of to solve the problem of leaving the bonding module with pointers to illegal addresses. The other way is to increase the usage count, with try_module_get(), of the module which owns of the slave. To do that I have to restore the field owner in structure net_device (it was removed in 2.6). Do you prefer the second approach? I wasn't sure about that. From hal.rosenstock at gmail.com Wed Aug 29 07:30:18 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 29 Aug 2007 10:30:18 -0400 Subject: [ofa-general] Re: [ewg] OFED Aug 27 meeting summary on OFED 1.3 development status In-Reply-To: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> References: <6C2C79E72C305246B504CBA17B5500C901563C50@mtlexch01.mtl.com> Message-ID: Tziporet, On 8/28/07, Tziporet Koren wrote: > > > OFED Aug 13 meeting summary on OFED 1.3 development status: > ================================================= > > 1. Decided to delay the feature freeze to Sep 10. > > MPI update: > Open MPI: will have some intermediate package for OFED > Mvapich: first OFED package will start with version 0.99 (that was part of > 1.2.5). The package of the new release will be ready by end of Sep. > Mvapich2: new release 1.2 will be ready on time for the feature freeze. > QoS in the kernel - need update from Sean > Neteffect new RNIC driver - will be ready in 2 weeks > bonding - will be ready in 2 weeks > > > 2. New install: > > Asked all to test and review the new install since Vlad is going for 3 weeks > vacation on Sep 10 > Explained the feature to build without installation is not available any > more. > Reasons for change were that working in the previous mode was against rpm > native usage and caused many issues to other packages (like MPI). > In addition it caused OFED spec files and package to be not useful for the > distros. > > > 3. bugzilla cleanup toward 1.3: > > We wish to clean the bugzilla database before we start the 1.3 release > phase. > Tziporet will move all open bugs that were opened for OFED 1.2 to OFED 1.3 > Older bugs will be moved to WONTFIX status This shouldn't be arbitrarily done as it will mean some bugs will need to be reentered as they are still relevant. Can't this be done by the developers on a discretionary basis ? -- Hal > Any one that opened a bug is invited to review the bugs and update their > status (most important - close fixed bugs) > > > 4. Patches submitted to OFED : > > Decided that all kernel patches must submitted for the list for review > before merging them to the kernel > Regarding user space: sending patches to the list is recommended but not a > must > For critical components (like libibverbs, ucma, osm) sending patches to > review is required too > Reminder: Each patch must include short explanation + sign-off > > > > > Tziporet Koren > Software Director > Mellanox Technologies > mailto: tziporet at mellanox.co.il > Tel +972-4-9097200, ext 380 > > > > > > > > > _______________________________________________ > ewg mailing list > ewg at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg > From anton at samba.org Wed Aug 29 08:28:17 2007 From: anton at samba.org (Anton Blanchard) Date: Wed, 29 Aug 2007 10:28:17 -0500 Subject: [ofa-general] [PATCH] Set props->page_size_cap in ehca Message-ID: <20070829152817.GC5289@kryten> Hi, When inserting the ib_srp module, the ehca driver spits out an error: ehca D.001.DQDXYCB-P1-C9: PU0007 EHCA_ERR:ehca_alloc_fmr unsupported fmr_attr->page_shift=9 The following code in ib_srp.c is causing it: /* * Use the smallest page size supported by the HCA, down to a * minimum of 512 bytes (which is the smallest sector that a * SCSI command will ever carry). */ srp_dev->fmr_page_shift = max(9, ffs(dev_attr->page_size_cap) - 1); Looks like we should be setting props->page_size_cap. Compile tested only, I dont have any IB attached storage. Signed-off-by: Anton Blanchard --- diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index fc19ef9..9ca2543 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -77,6 +77,7 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props) } memset(props, 0, sizeof(struct ib_device_attr)); + props->page_size_cap = EHCA_PAGESIZE; props->fw_ver = rblock->hw_ver; props->max_mr_size = rblock->max_mr_size; props->vendor_id = rblock->vendor_id >> 8; From anton at samba.org Wed Aug 29 09:05:35 2007 From: anton at samba.org (Anton Blanchard) Date: Wed, 29 Aug 2007 11:05:35 -0500 Subject: [ofa-general] [PATCH] Quieten some error messages in ehca Message-ID: <20070829160535.GA24003@kryten> Hi, ehca spits out a lot of debugging information. I had to look closely to see the "Port 1 is not active" message within all the debug: eHCA Infiniband Device Driver (Rel.: SVNEHCA_0022) eHCA scaling code enabled ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_define_sqp Port 1 is not active. ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_create_qp ehca_define_sqp() failed rc=ffffffffffffffff ib_mad: Couldn't create ib_mad QP1 ib_mad: Couldn't open ehca0 port 1 ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr unsupported fmr_attr->page_shift=9 ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr rc=ffffffffffffffea pd=c000000b4b5b2420 mr_access_flags=7 fmr_attr=c0000005afd37394 fmr_create failed for FMR 0 Remove a few debug statements so that things are clearer: eHCA Infiniband Device Driver (Rel.: SVNEHCA_0022) eHCA scaling code enabled ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_define_sqp Port 1 is not active. ib_mad: Couldn't create ib_mad QP1 ib_mad: Couldn't open ehca0 port 1 ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr unsupported fmr_attr->page_shift=9 fmr_create failed for FMR 0 Signed-off-by: Anton Blanchard --- diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c b/drivers/infiniband/hw/ehca/ehca_mrmw.c index c1b868b..57e3420 100644 --- a/drivers/infiniband/hw/ehca/ehca_mrmw.c +++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c @@ -848,10 +848,6 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd, alloc_fmr_exit1: ehca_mr_delete(e_fmr); alloc_fmr_exit0: - if (IS_ERR(ib_fmr)) - ehca_err(pd->device, "rc=%lx pd=%p mr_access_flags=%x " - "fmr_attr=%p", PTR_ERR(ib_fmr), pd, - mr_access_flags, fmr_attr); return ib_fmr; } /* end ehca_alloc_fmr() */ diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..448f6b3 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -712,8 +712,6 @@ static struct ehca_qp *internal_create_qp( if (qp_type == IB_QPT_GSI) { h_ret = ehca_define_sqp(shca, my_qp, init_attr); if (h_ret != H_SUCCESS) { - ehca_err(pd->device, "ehca_define_sqp() failed rc=%lx", - h_ret); ret = ehca2ib_return_code(h_ret); goto create_qp_exit4; } From sashak at voltaire.com Wed Aug 29 09:27:54 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 19:27:54 +0300 Subject: [ofa-general] [PATCH] opensm: timer driven periodic sweep Message-ID: <20070829162754.GE18082@sashak.voltaire.com> Periodic SM sweep (as specified by sweep_interval) is driven by cl_timer now. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_sm.h | 1 + opensm/opensm/osm_sm.c | 31 ++++++++++++++++++++++--------- 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/opensm/include/opensm/osm_sm.h b/opensm/include/opensm/osm_sm.h index 341ec5a..4c6ce27 100644 --- a/opensm/include/opensm/osm_sm.h +++ b/opensm/include/opensm/osm_sm.h @@ -129,6 +129,7 @@ typedef struct osm_sm { cl_spinlock_t signal_lock; cl_event_t signal_event; cl_event_t subnet_up_event; + cl_timer_t sweep_timer; cl_thread_t sweeper; osm_subn_t *p_subn; osm_db_t *p_db; diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c index c698492..154ac8e 100644 --- a/opensm/opensm/osm_sm.c +++ b/opensm/opensm/osm_sm.c @@ -83,9 +83,7 @@ static void __osm_sm_sweeper(IN void *p_ptr) * signaling the event. */ status = cl_event_wait_on(&p_sm->signal_event, - p_sm->p_subn->opt.sweep_interval ? - p_sm->p_subn->opt.sweep_interval * - 1000000 : EVENT_NO_TIMEOUT, TRUE); + EVENT_NO_TIMEOUT, TRUE); if (status == CL_SUCCESS) osm_log(p_sm->p_log, OSM_LOG_DEBUG, @@ -104,12 +102,6 @@ static void __osm_sm_sweeper(IN void *p_ptr) p_sm->signal_mask = 0; cl_spinlock_release(&p_sm->signal_lock); - /* do the sweep only if we are in MASTER state */ - if (status == CL_TIMEOUT && - (p_sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || - p_sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING)) - signals |= 1 << OSM_SIGNAL_SWEEP; - for (i = 0 ; signals ; signals >>= 1 , i++) if (signals&1) osm_state_mgr_process(&p_sm->state_mgr, i); @@ -118,6 +110,17 @@ static void __osm_sm_sweeper(IN void *p_ptr) OSM_LOG_EXIT(p_sm->p_log); } +static void sm_sweep(void *arg) +{ + osm_sm_t *sm = arg; + + /* do the sweep only if we are in MASTER state */ + if (sm->p_subn->sm_state == IB_SMINFO_STATE_MASTER || + sm->p_subn->sm_state == IB_SMINFO_STATE_DISCOVERING) + osm_sm_signal(sm, OSM_SIGNAL_SWEEP); + cl_timer_start(&sm->sweep_timer, sm->p_subn->opt.sweep_interval*1000); +} + /********************************************************************** **********************************************************************/ void osm_sm_construct(IN osm_sm_t * const p_sm) @@ -176,6 +179,7 @@ void osm_sm_shutdown(IN osm_sm_t * const p_sm) if (signal_event) cl_event_signal(&p_sm->signal_event); + cl_timer_stop(&p_sm->sweep_timer); cl_thread_destroy(&p_sm->sweeper); /* @@ -225,6 +229,7 @@ void osm_sm_destroy(IN osm_sm_t * const p_sm) osm_state_mgr_destroy(&p_sm->state_mgr); osm_sm_state_mgr_destroy(&p_sm->sm_state_mgr); osm_mcast_mgr_destroy(&p_sm->mcast_mgr); + cl_timer_destroy(&p_sm->sweep_timer); cl_event_destroy(&p_sm->signal_event); cl_event_destroy(&p_sm->subnet_up_event); cl_spinlock_destroy(&p_sm->signal_lock); @@ -271,6 +276,10 @@ osm_sm_init(IN osm_sm_t * const p_sm, if (status != CL_SUCCESS) goto Exit; + status = cl_timer_init(&p_sm->sweep_timer, sm_sweep, p_sm); + if (status != CL_SUCCESS) + goto Exit; + status = osm_sm_mad_ctrl_init(&p_sm->mad_ctrl, p_sm->p_subn, p_sm->p_mad_pool, @@ -481,6 +490,10 @@ osm_sm_init(IN osm_sm_t * const p_sm, if (status != IB_SUCCESS) goto Exit; + if (p_sm->p_subn->opt.sweep_interval) + cl_timer_start(&p_sm->sweep_timer, + p_sm->p_subn->opt.sweep_interval*1000); + Exit: OSM_LOG_EXIT(p_log); return (status); -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Wed Aug 29 09:30:26 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Wed, 29 Aug 2007 19:30:26 +0300 Subject: [ofa-general] [PATCH] opensm/perfmgr: PerfMgr for SM standby and inactive states In-Reply-To: <20070829162754.GE18082@sashak.voltaire.com> References: <20070829162754.GE18082@sashak.voltaire.com> Message-ID: <20070829163026.GF18082@sashak.voltaire.com> It supports PerfMgr when OpenSM is in standby or inactive states. For those cases PerfMgr will run its own fabric discovery (SMP based) yet. I think in a future this code should be merged with "regular" SM discovery. Signed-off-by: Sasha Khapyorsky --- opensm/include/opensm/osm_base.h | 3 +- opensm/include/opensm/osm_perfmgr.h | 7 +- opensm/include/opensm/osm_stats.h | 8 +- opensm/opensm/main.c | 6 +- opensm/opensm/osm_helper.c | 3 +- opensm/opensm/osm_perfmgr.c | 379 +++++++++++++++++++++++++++++------ opensm/opensm/osm_sm.c | 14 ++- opensm/opensm/osm_sm_mad_ctrl.c | 3 + 8 files changed, 350 insertions(+), 73 deletions(-) diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h index 61100e6..e635dcb 100644 --- a/opensm/include/opensm/osm_base.h +++ b/opensm/include/opensm/osm_base.h @@ -736,7 +736,8 @@ typedef enum _osm_sm_state { #define OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST 9 #define OSM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED 10 #define OSM_SIGNAL_EXIT_STBY 11 -#define OSM_SIGNAL_MAX 12 +#define OSM_SIGNAL_PERFMGR_SWEEP 12 +#define OSM_SIGNAL_MAX 13 typedef uintn_t osm_signal_t; /***********/ diff --git a/opensm/include/opensm/osm_perfmgr.h b/opensm/include/opensm/osm_perfmgr.h index 6f0971b..4bf5d28 100644 --- a/opensm/include/opensm/osm_perfmgr.h +++ b/opensm/include/opensm/osm_perfmgr.h @@ -44,7 +44,7 @@ #include #include #include -#include +#include #include #include #include @@ -110,9 +110,8 @@ typedef struct _monitored_node { * be manipulated only through the provided functions. */ typedef struct _osm_perfmgr { - osm_thread_state_t thread_state; cl_event_t sig_sweep; - cl_thread_t sweeper; + cl_timer_t sweep_timer; osm_subn_t *subn; osm_sm_t *sm; cl_plock_t *lock; @@ -223,6 +222,8 @@ void osm_perfmgr_dump_counters(osm_perfmgr_t * p_perfmgr, ib_api_status_t osm_perfmgr_bind(osm_perfmgr_t * const p_perfmgr, const ib_net64_t port_guid); +void osm_perfmgr_process(osm_perfmgr_t * pm); + /****f* OpenSM: PerfMgr/osm_perfmgr_init */ ib_api_status_t osm_perfmgr_init(osm_perfmgr_t * const perfmgr, osm_subn_t * const subn, diff --git a/opensm/include/opensm/osm_stats.h b/opensm/include/opensm/osm_stats.h index 2d2fa14..ca7fa2f 100644 --- a/opensm/include/opensm/osm_stats.h +++ b/opensm/include/opensm/osm_stats.h @@ -48,6 +48,9 @@ #ifndef _OSM_STATS_H_ #define _OSM_STATS_H_ +#ifdef ENABLE_OSM_PERF_MGR +#include +#endif #include #include @@ -90,7 +93,10 @@ typedef struct _osm_stats { atomic32_t sa_mads_outstanding; atomic32_t sa_mads_rcvd; atomic32_t sa_mads_sent; - +#ifdef ENABLE_OSM_PERF_MGR + pthread_mutex_t mutex; + pthread_cond_t cond; +#endif } osm_stats_t; /* * FIELDS diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index eab64c0..08d654e 100644 --- a/opensm/opensm/main.c +++ b/opensm/opensm/main.c @@ -1080,10 +1080,14 @@ int main(int argc, char *argv[]) sleep(exitTimeout); #endif - if (osm.mad_pool.mads_out) + if (osm.mad_pool.mads_out) { fprintf(stdout, "There are still %u MADs out. Forcing the exit of the OpenSM application...\n", osm.mad_pool.mads_out); +#ifdef ENABLE_OSM_PERF_MGR + pthread_cond_signal(&osm.stats.cond); +#endif + } Exit: osm_opensm_destroy(&osm); diff --git a/opensm/opensm/osm_helper.c b/opensm/opensm/osm_helper.c index 5dd3955..ff3da40 100644 --- a/opensm/opensm/osm_helper.c +++ b/opensm/opensm/osm_helper.c @@ -2111,7 +2111,8 @@ const char *const __osm_sm_signal_str[] = { "OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST", /* 9 */ "OSM_SIGNAL_MASTER_OR_HIGHER_SM_DETECTED", /* 10 */ "OSM_SIGNAL_EXIT_STBY", /* 11 */ - "UNKNOWN SIGNAL!!" /* 12 */ + "OSM_SIGNAL_PERFMGR_SWEEP", /* 12 */ + "UNKNOWN SIGNAL!!" /* 13 */ }; /********************************************************************** diff --git a/opensm/opensm/osm_perfmgr.c b/opensm/opensm/osm_perfmgr.c index c1d2db4..ff3cab1 100644 --- a/opensm/opensm/osm_perfmgr.c +++ b/opensm/opensm/osm_perfmgr.c @@ -59,6 +59,7 @@ #include #include #include +#include #include #include #include @@ -550,83 +551,324 @@ __osm_perfmgr_query_counters(cl_map_item_t * const p_map_item, void *context) } /********************************************************************** - * Main PerfMgr Thread. - * Loop continuously and query the performance counters. + * Discovery stuff. + * Basically this code should not be here, but merged with main OpenSM **********************************************************************/ -void __osm_perfmgr_sweeper(void *p_ptr) +static int sweep_hop_1(osm_sm_t * sm) { - osm_perfmgr_t *const pm = (osm_perfmgr_t *) p_ptr; + ib_api_status_t status = IB_SUCCESS; + osm_bind_handle_t h_bind; + osm_madw_context_t context; + osm_node_t *p_node; + osm_port_t *p_port; + osm_physp_t *p_physp; + osm_dr_path_t *p_dr_path; + osm_dr_path_t hop_1_path; + ib_net64_t port_guid; + uint8_t port_num; + uint8_t path_array[IB_SUBNET_PATH_HOPS_MAX]; + uint8_t num_ports; + osm_physp_t *p_ext_physp; + + port_guid = sm->p_subn->sm_port_guid; + + p_port = osm_get_port_by_guid(sm->p_subn, port_guid); + if (!p_port) { + osm_log(sm->p_log, OSM_LOG_ERROR, + "sweep_hop_1: ERR 4C81: No SM port object\n"); + return -1; + } - OSM_LOG_ENTER(pm->log, __osm_pm_sweeper); + p_node = p_port->p_node; + port_num = ib_node_info_get_local_port_num(&p_node->node_info); - if (pm->thread_state == OSM_THREAD_STATE_INIT) - pm->thread_state = OSM_THREAD_STATE_RUN; + osm_log(sm->p_log, OSM_LOG_DEBUG, + "sweep_hop_1: Probing hop 1 on local port %u\n", port_num); - __init_monitored_nodes(pm); + p_physp = osm_node_get_physp_ptr(p_node, port_num); + + CL_ASSERT(osm_physp_is_valid(p_physp)); + + p_dr_path = osm_physp_get_dr_path_ptr(p_physp); + h_bind = osm_dr_path_get_bind_handle(p_dr_path); + + CL_ASSERT(h_bind != OSM_BIND_INVALID_HANDLE); + + memset(path_array, 0, sizeof(path_array)); + /* the hop_1 operations depend on the type of our node. + * Currently - legal nodes that can host SM are SW and CA */ + switch (osm_node_get_type(p_node)) { + case IB_NODE_TYPE_CA: + case IB_NODE_TYPE_ROUTER: + memset(&context, 0, sizeof(context)); + context.ni_context.node_guid = osm_node_get_node_guid(p_node); + context.ni_context.port_num = port_num; + + path_array[1] = port_num; + + osm_dr_path_init(&hop_1_path, h_bind, 1, path_array); + status = osm_req_get(&sm->req, + &hop_1_path, + IB_MAD_ATTR_NODE_INFO, 0, + CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) + osm_log(sm->p_log, OSM_LOG_ERROR, + "sweep_hop_1: ERR 4C82: " + "Request for NodeInfo failed\n"); + break; - while (pm->thread_state == OSM_THREAD_STATE_RUN) { - /* do the sweep only if in MASTER state - * AND we have been activated. - * FIXME put something in here to try and reduce the load on the system - * when it is not IDLE. - if (pm->sm->state_mgr.state != OSM_SM_STATE_IDLE) + case IB_NODE_TYPE_SWITCH: + /* Need to go over all the ports of the switch, and send a node_info + * from them. This doesn't include the port 0 of the switch, which + * hosts the SM. + * Note: We'll send another switchInfo on port 0, since if no ports + * are connected, we still want to get some response, and have the + * subnet come up. */ - if (pm->subn->sm_state == IB_SMINFO_STATE_MASTER && - pm->state == PERFMGR_STATE_ENABLED) { + num_ports = osm_node_get_num_physp(p_node); + for (port_num = 0; port_num < num_ports; port_num++) { + /* go through the port only if the port is not DOWN */ + p_ext_physp = osm_node_get_physp_ptr(p_node, port_num); + if (ib_port_info_get_port_state + (&p_ext_physp->port_info) <= IB_LINK_DOWN) + continue; + + memset(&context, 0, sizeof(context)); + context.ni_context.node_guid = + osm_node_get_node_guid(p_node); + context.ni_context.port_num = port_num; + + path_array[1] = port_num; + + osm_dr_path_init(&hop_1_path, h_bind, 1, path_array); + status = osm_req_get(&sm->req, &hop_1_path, + IB_MAD_ATTR_NODE_INFO, 0, + CL_DISP_MSGID_NONE, &context); + + if (status != IB_SUCCESS) + osm_log(sm->p_log, OSM_LOG_ERROR, + "sweep_hop_1: ERR 4C82: " + "Request for NodeInfo failed\n"); + } + break; + + default: + osm_log(sm->p_log, OSM_LOG_ERROR, + "sweep_hop_1: ERR 4C83: Unknown node type %d\n", + osm_node_get_type(p_node)); + } + + return (status); +} + +static unsigned is_sm_port_down(osm_sm_t *const sm) +{ + ib_net64_t port_guid; + osm_port_t *p_port; + + port_guid = sm->p_subn->sm_port_guid; + if (port_guid == 0) + return 1; + + CL_PLOCK_ACQUIRE(sm->p_lock); + p_port = osm_get_port_by_guid(sm->p_subn, port_guid); + if (!p_port) { + CL_PLOCK_RELEASE(sm->p_lock); + osm_log(sm->p_log, OSM_LOG_ERROR, + "is_sm_port_down: ERR 4C85: " + "SM port with GUID:%016" PRIx64 " is unknown\n", + cl_ntoh64(port_guid)); + return 1; + } + CL_PLOCK_RELEASE(sm->p_lock); + + return osm_physp_get_port_state(p_port->p_physp) == IB_LINK_DOWN; +} + +static int sweep_hop_0(osm_sm_t *const sm) +{ + ib_api_status_t status; + osm_dr_path_t dr_path; + osm_bind_handle_t h_bind; + uint8_t path_array[IB_SUBNET_PATH_HOPS_MAX]; + + memset(path_array, 0, sizeof(path_array)); + + h_bind = osm_sm_mad_ctrl_get_bind_handle(&sm->mad_ctrl); + if (h_bind == OSM_BIND_INVALID_HANDLE) { + osm_log(sm->p_log, OSM_LOG_DEBUG, + "sweep_hop_0: No bound ports.\n"); + return -1; + } + + osm_dr_path_init(&dr_path, h_bind, 0, path_array); + status = osm_req_get(&sm->req, + &dr_path, IB_MAD_ATTR_NODE_INFO, 0, + CL_DISP_MSGID_NONE, NULL); + + if (status != IB_SUCCESS) + osm_log(sm->p_log, OSM_LOG_ERROR, + "sweep_hop_0: ERR 4C86: Request for NodeInfo failed\n"); + + return (status); +} + +static int wait_for_pending_transactions(osm_stats_t *stats) +{ + pthread_mutex_lock(&stats->mutex); + while (stats->qp0_mads_outstanding && !osm_exit_flag) + pthread_cond_wait(&stats->cond, &stats->mutex); + pthread_mutex_unlock(&stats->mutex); + return 0; +} + +static void reset_node_count(cl_map_item_t * const p_map_item, void *cxt) +{ + osm_node_t *p_node = (osm_node_t *) p_map_item; + p_node->discovery_count = 0; +} + +static void reset_port_count(cl_map_item_t * const p_map_item, void *cxt) +{ + osm_port_t *p_port = (osm_port_t *) p_map_item; + p_port->discovery_count = 0; +} + +static void reset_switch_count(cl_map_item_t * const p_map_item, void *cxt) +{ + osm_switch_t *p_sw = (osm_switch_t *) p_map_item; + p_sw->discovery_count = 0; + p_sw->need_update = 0; +} + +static int perfmgr_discovery(osm_opensm_t *osm) +{ + unsigned signals = osm->sm.signal_mask; + int ret; + + CL_PLOCK_ACQUIRE(&osm->lock); + cl_qmap_apply_func(&osm->subn.node_guid_tbl, reset_node_count, NULL); + cl_qmap_apply_func(&osm->subn.port_guid_tbl, reset_port_count, NULL); + cl_qmap_apply_func(&osm->subn.sw_guid_tbl, reset_switch_count, NULL); + CL_PLOCK_RELEASE(&osm->lock); + + osm->subn.in_sweep_hop_0 = TRUE; + + ret = sweep_hop_0(&osm->sm); + if (ret) + goto _exit; + + wait_for_pending_transactions(&osm->stats); + + if (is_sm_port_down(&osm->sm)) { + osm_log(&osm->log, OSM_LOG_VERBOSE, + "SM port is down\n"); + goto _drop; + } + + osm->subn.in_sweep_hop_0 = FALSE; + + ret = sweep_hop_1(&osm->sm); + if (ret) + goto _exit; + + wait_for_pending_transactions(&osm->stats); + + _drop: + osm_drop_mgr_process(&osm->sm.drop_mgr); + + _exit: + /* dirty hack: cleanup signal mask - + * this will not be needed later with both discoveries merged */ + cl_spinlock_acquire(&osm->sm.signal_lock); + osm->sm.signal_mask &= ~(OSM_SIGNAL_NO_PENDING_TRANSACTIONS| + OSM_SIGNAL_CHANGE_DETECTED); + osm->sm.signal_mask |= signals; + cl_spinlock_release(&osm->sm.signal_lock); + return ret; +} + +/********************************************************************** + * Main PerfMgr processor - query the performance counters. + **********************************************************************/ +void osm_perfmgr_process(osm_perfmgr_t *pm) +{ #if ENABLE_OSM_PERF_MGR_PROFILE - struct timeval before, after; - gettimeofday(&before, NULL); + struct timeval before, after; #endif - pm->sweep_state = PERFMGR_SWEEP_ACTIVE; - /* With the global lock held collect the node guids */ - /* FIXME we should be able to track SA notices - * and not have to sweep the node_guid_tbl each pass - */ - osm_log(pm->log, OSM_LOG_VERBOSE, - "Gathering PerfMgr stats\n"); - cl_plock_acquire(pm->lock); - cl_qmap_apply_func(&(pm->subn->node_guid_tbl), - __collect_guids, (void *)pm); - cl_plock_release(pm->lock); - /* then for each node query their counters */ - cl_qmap_apply_func(&(pm->monitored_map), - __osm_perfmgr_query_counters, - (void *)pm); + if (pm->state != PERFMGR_STATE_ENABLED) + return; + + if (pm->sm->state_mgr.state != OSM_SM_STATE_IDLE && + pm->sm->state_mgr.state != OSM_SM_STATE_STANDBY) + return; - /* Clean out any nodes found to be removed during the - * sweep - */ - __remove_marked_nodes(pm); + if (pm->sm->state_mgr.state == OSM_SM_STATE_STANDBY || + (pm->sm->state_mgr.state == OSM_SM_STATE_IDLE && + pm->subn->sm_state == IB_SMINFO_STATE_NOTACTIVE)) + perfmgr_discovery(pm->subn->p_osm); #if ENABLE_OSM_PERF_MGR_PROFILE - /* spin on outstanding queries */ - while (pm->outstanding_queries > 0) - cl_event_wait_on(&pm->sig_sweep, 1000, TRUE); - - gettimeofday(&after, NULL); - diff_time(&before, &after, &after); - osm_log(pm->log, OSM_LOG_INFO, - "PerfMgr total sweep time : %ld.%06ld s\n" - " fastest mad : %g us\n" - " slowest mad : %g us\n" - " average mad : %g us\n" - , - after.tv_sec, after.tv_usec, - perfmgr_mad_stats.fastest_us, - perfmgr_mad_stats.slowest_us, - perfmgr_mad_stats.avg_us); - perfmgr_clear_mad_stats(); + gettimeofday(&before, NULL); #endif - } + pm->sweep_state = PERFMGR_SWEEP_ACTIVE; + /* With the global lock held collect the node guids */ + /* FIXME we should be able to track SA notices + * and not have to sweep the node_guid_tbl each pass + */ + osm_log(pm->log, OSM_LOG_VERBOSE, + "Gathering PerfMgr stats\n"); + cl_plock_acquire(pm->lock); + cl_qmap_apply_func(&(pm->subn->node_guid_tbl), + __collect_guids, (void *)pm); + cl_plock_release(pm->lock); - pm->sweep_state = PERFMGR_SWEEP_SLEEP; + /* then for each node query their counters */ + cl_qmap_apply_func(&(pm->monitored_map), + __osm_perfmgr_query_counters, (void *)pm); - /* Wait for a forced sweep or period timeout. */ - cl_event_wait_on(&pm->sig_sweep, pm->sweep_time_s * 1000000, TRUE); - } + /* Clean out any nodes found to be removed during the + * sweep + */ + __remove_marked_nodes(pm); - OSM_LOG_EXIT(pm->log); +#if ENABLE_OSM_PERF_MGR_PROFILE + /* spin on outstanding queries */ + while (pm->outstanding_queries > 0) + cl_event_wait_on(&pm->sig_sweep, 1000, TRUE); + + gettimeofday(&after, NULL); + diff_time(&before, &after, &after); + osm_log(pm->log, OSM_LOG_INFO, + "PerfMgr total sweep time : %ld.%06ld s\n" + " fastest mad : %g us\n" + " slowest mad : %g us\n" + " average mad : %g us\n", + after.tv_sec, after.tv_usec, + perfmgr_mad_stats.fastest_us, + perfmgr_mad_stats.slowest_us, + perfmgr_mad_stats.avg_us); + perfmgr_clear_mad_stats(); +#endif + + pm->sweep_state = PERFMGR_SWEEP_SLEEP; +} + +/********************************************************************** + * PerfMgr timer - loop continuously and signal SM to run PerfMgr + * processor. + **********************************************************************/ +static void perfmgr_sweep(void *arg) +{ + osm_perfmgr_t * pm = arg; + + __init_monitored_nodes(pm); + + if (pm->state == PERFMGR_STATE_ENABLED) + osm_sm_signal(pm->sm, OSM_SIGNAL_PERFMGR_SWEEP); + cl_timer_start(&pm->sweep_timer, pm->sweep_time_s * 1000); } /********************************************************************** @@ -634,6 +876,7 @@ void __osm_perfmgr_sweeper(void *p_ptr) void osm_perfmgr_shutdown(osm_perfmgr_t * const pm) { OSM_LOG_ENTER(pm->log, osm_perfmgr_shutdown); + cl_timer_stop(&pm->sweep_timer); osm_perfmgr_mad_unbind(pm); OSM_LOG_EXIT(pm->log); } @@ -645,6 +888,9 @@ void osm_perfmgr_destroy(osm_perfmgr_t * const pm) OSM_LOG_ENTER(pm->log, osm_perfmgr_destroy); free(pm->event_db_dump_file); perfmgr_db_destroy(pm->db); + cl_timer_destroy(&pm->sweep_timer); + pthread_cond_destroy(&pm->subn->p_osm->stats.cond); + pthread_mutex_destroy(&pm->subn->p_osm->stats.mutex); OSM_LOG_EXIT(pm->log); } @@ -1033,6 +1279,13 @@ osm_perfmgr_init(osm_perfmgr_t * const pm, pm->max_outstanding_queries = p_opt->perfmgr_max_outstanding_queries; pm->event_plugin = event_plugin; + pthread_mutex_init(&subn->p_osm->stats.mutex, NULL); + pthread_cond_init(&subn->p_osm->stats.cond, NULL); + + status = cl_timer_init(&pm->sweep_timer, perfmgr_sweep, pm); + if (status != IB_SUCCESS) + goto Exit; + pm->db = perfmgr_db_construct(pm->log, pm->event_plugin); if (!pm->db) { pm->state = PERFMGR_STATE_NO_DB; @@ -1044,11 +1297,7 @@ osm_perfmgr_init(osm_perfmgr_t * const pm, if (pm->pc_disp_h == CL_DISP_INVALID_HANDLE) goto Exit; - pm->thread_state = OSM_THREAD_STATE_INIT; - status = cl_thread_init(&pm->sweeper, __osm_perfmgr_sweeper, pm, - "PerfMgr sweeper"); - if (status != IB_SUCCESS) - goto Exit; + cl_timer_start(&pm->sweep_timer, pm->sweep_time_s * 1000); Exit: OSM_LOG_EXIT(log); diff --git a/opensm/opensm/osm_sm.c b/opensm/opensm/osm_sm.c index 154ac8e..c13f7ab 100644 --- a/opensm/opensm/osm_sm.c +++ b/opensm/opensm/osm_sm.c @@ -63,11 +63,23 @@ #include #include #include +#include +#include #define OSM_SM_INITIAL_TID_VALUE 0x1233 /********************************************************************** **********************************************************************/ +static void osm_sm_process(osm_sm_t *sm, osm_signal_t signal) +{ +#ifdef ENABLE_OSM_PERF_MGR + if (signal == OSM_SIGNAL_PERFMGR_SWEEP) + osm_perfmgr_process(&sm->p_subn->p_osm->perfmgr); + else +#endif + osm_state_mgr_process(&sm->state_mgr, signal); +} + static void __osm_sm_sweeper(IN void *p_ptr) { ib_api_status_t status; @@ -104,7 +116,7 @@ static void __osm_sm_sweeper(IN void *p_ptr) for (i = 0 ; signals ; signals >>= 1 , i++) if (signals&1) - osm_state_mgr_process(&p_sm->state_mgr, i); + osm_sm_process(p_sm, i); } OSM_LOG_EXIT(p_sm->p_log); diff --git a/opensm/opensm/osm_sm_mad_ctrl.c b/opensm/opensm/osm_sm_mad_ctrl.c index 2ed973d..9df5f86 100644 --- a/opensm/opensm/osm_sm_mad_ctrl.c +++ b/opensm/opensm/osm_sm_mad_ctrl.c @@ -106,6 +106,9 @@ __osm_sm_mad_ctrl_retire_trans_mad(IN osm_sm_mad_ctrl_t * const p_ctrl, "__osm_sm_mad_ctrl_retire_trans_mad: " "signal OSM_SIGNAL_NO_PENDING_TRANSACTIONS\n"); +#ifdef ENABLE_OSM_PERF_MGR + pthread_cond_signal(&p_ctrl->p_stats->cond); +#endif osm_sm_signal(&p_ctrl->p_subn->p_osm->sm, OSM_SIGNAL_NO_PENDING_TRANSACTIONS); } -- 1.5.3.rc2.38.g11308 From peter.p.waskiewicz.jr at intel.com Wed Aug 29 09:43:11 2007 From: peter.p.waskiewicz.jr at intel.com (Waskiewicz Jr, Peter P) Date: Wed, 29 Aug 2007 09:43:11 -0700 Subject: [ofa-general] RE: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB In-Reply-To: <20070828.221448.123919361.davem@davemloft.net> References: <20070828.215150.98552996.davem@davemloft.net> <20070828.221448.123919361.davem@davemloft.net> Message-ID: > From: Krishna Kumar2 > Date: Wed, 29 Aug 2007 10:43:23 +0530 > > > The reason was to run parallel copies, not for buffer limitations. > > Oh, I see. > > I'll note in passing that current lmbench-3 has some > parallelization features you could play with, you might want > to check it out. I've also used iperf for parallel connections successfully, and that will allow you to mess with the buffer sizes as well along with other variables of the data streams for both TCP and UDP. Cheers, -PJ From mshefty at ichips.intel.com Wed Aug 29 10:26:04 2007 From: mshefty at ichips.intel.com (Sean Hefty) Date: Wed, 29 Aug 2007 10:26:04 -0700 Subject: [ofa-general] Re: Do you think the QoS new code will make it for 2.6.24? In-Reply-To: <46CB3697.9000206@mellanox.co.il> References: <46CB3697.9000206@mellanox.co.il> Message-ID: <46D5AC2C.10300@ichips.intel.com> I plan on submitting for 2.6.24. I will post the latest patches early next week at the latest. - Sean From anton at samba.org Wed Aug 29 10:43:01 2007 From: anton at samba.org (Anton Blanchard) Date: Wed, 29 Aug 2007 12:43:01 -0500 Subject: [ofa-general] [PATCH] Export ehca module parameters in sysfs Message-ID: <20070829174301.GD24003@kryten> Hi, At the moment the ehca module parameters are not exported in sysfs. Export them with 0444 permissions. Signed-off-by: Anton Blanchard --- diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c index 99036b6..db041df 100644 --- a/drivers/infiniband/hw/ehca/ehca_main.c +++ b/drivers/infiniband/hw/ehca/ehca_main.c @@ -65,16 +65,16 @@ int ehca_static_rate = -1; int ehca_scaling_code = 0; int ehca_mr_largepage = 0; -module_param_named(open_aqp1, ehca_open_aqp1, int, 0); -module_param_named(debug_level, ehca_debug_level, int, 0); -module_param_named(hw_level, ehca_hw_level, int, 0); -module_param_named(nr_ports, ehca_nr_ports, int, 0); -module_param_named(use_hp_mr, ehca_use_hp_mr, int, 0); -module_param_named(port_act_time, ehca_port_act_time, int, 0); -module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, 0); -module_param_named(static_rate, ehca_static_rate, int, 0); -module_param_named(scaling_code, ehca_scaling_code, int, 0); -module_param_named(mr_largepage, ehca_mr_largepage, int, 0); +module_param_named(open_aqp1, ehca_open_aqp1, int, S_IRUGO); +module_param_named(debug_level, ehca_debug_level, int, S_IRUGO); +module_param_named(hw_level, ehca_hw_level, int, S_IRUGO); +module_param_named(nr_ports, ehca_nr_ports, int, S_IRUGO); +module_param_named(use_hp_mr, ehca_use_hp_mr, int, S_IRUGO); +module_param_named(port_act_time, ehca_port_act_time, int, S_IRUGO); +module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, S_IRUGO); +module_param_named(static_rate, ehca_static_rate, int, S_IRUGO); +module_param_named(scaling_code, ehca_scaling_code, int, S_IRUGO); +module_param_named(mr_largepage, ehca_mr_largepage, int, S_IRUGO); MODULE_PARM_DESC(open_aqp1, "AQP1 on startup (0: no (default), 1: yes)"); From ssufficool at rov.sbcounty.gov Wed Aug 29 11:37:40 2007 From: ssufficool at rov.sbcounty.gov (Sufficool, Stanley) Date: Wed, 29 Aug 2007 11:37:40 -0700 Subject: [ofa-general] [PATCH] SRPT for current SCST In-Reply-To: <46CD2BC4.5020803@mellanox.com> References: <200708221128.l7MBSQ2H004759@cmf.nrl.navy.mil> <46CD2BC4.5020803@mellanox.com> Message-ID: I have updated to the most recent SRPT and applied the qp_max_atomic_res patch to hard code to "4". The WinIB gen1 initiator now sees the target without the error icon in Windows dev manager. However, WinIB still does not show the LUNs/Drives and the Target is not even logging a login attempt as it was before (Host i_port_id=...etc...). I have added debugs throughout the code to notify of "ret != 0" and all seems to be well. I will remove the max rdma patch and just swap out the target HCAs to match the initiators. For consideration, can the target be made to "auto negotiate" to the lowest capabilities of the initiator/target HCAs, and perhaps throw a message when it does? -----Original Message----- From: Vu Pham [mailto:vu at mellanox.com] Sent: Wednesday, August 22, 2007 11:40 PM To: Sufficool, Stanley Cc: chas williams - CONTRACTOR; Stanley Sufficool; general Subject: Re: [ofa-general] [PATCH] SRPT for current SCST You can get hardware max rdma atomic by doing ib_query_qp CM save the initiator's req->init_depth to its cm_id_priv->responder_resources in cm_req_handler (Win's srp initiator set it at 16) srpt call ib_cm_init_qp_attr() which will initialize qp_attr->max_dest_rd_atomic = cm_id_priv->responder_resources = 16; then srpt call ib_modify_qp() without checking/reseting the hw max rdma atomic in qp_attr; therefore, ib_modify_qp() failed You can try a quick dirty fix by applying this patch to ib_srpt.c -vu > So the client is requesting a QP with a max rdma_atomic of 15, but the > target initializes the QP with a max of 4. Is this max rdma_atomic of > 4 based on hardware capabilities (Memory vs MemFree adapters), or a > target QP initialization preset value? I'll swap adapters if needed. > > -----Original Message----- > From: chas williams - CONTRACTOR [mailto:chas at cmf.nrl.navy.mil] > Sent: Wednesday, August 22, 2007 4:28 AM > To: Sufficool, Stanley > Cc: Vu Pham; Stanley Sufficool; general > Subject: Re: [ofa-general] [PATCH] SRPT for current SCST > > In message > ,"Su > fficool, Stanley" writes: >> I'm still researching where the rdma_atomic is getting the invalid >> attribute. I will give an update if I find anything else. > > srp clients set this value as the responder_resources during login i > believe. this seems to be done in srp_get_responder_resources() on > the windows srp client. its taken from the qp attributes instead of > picking a fixed number, like 4 in the linux version of the srp client. From fubar at us.ibm.com Wed Aug 29 12:50:14 2007 From: fubar at us.ibm.com (Jay Vosburgh) Date: Wed, 29 Aug 2007 12:50:14 -0700 Subject: [ofa-general] Re: [PATCH V4 10/10] net/bonding: Destroy bonding master when last slave is gone In-Reply-To: <46D57D5D.3060706@gmail.com> References: <46C9B474.5020202@voltaire.com> <46C9BA2D.7060204@voltaire.com> <3403.1188343986@death> <46D57D5D.3060706@gmail.com> Message-ID: <16908.1188417014@death> Moni Shoua wrote: >Jay Vosburgh wrote: >> Moni Shoua wrote: >> >>> When bonding enslaves non Ethernet devices it takes pointers to functions >>> in the module that owns the slaves. In this case it becomes unsafe >>> to keep the bonding master registered after last slave was unenslaved >>> because we don't know if the pointers are still valid. Destroying the bond when slave_cnt is zero >>> ensures that these functions be used anymore. >> >> Would it not be simpler to run the bonding master through >> ether_setup() again when the final slave is released (to reset all of >> the pointers to their "ethernet" values)? I'm presuming here the >> pointers of questionable validity are the ones set in the >> bond_setup_by_slave() copied from the slave_dev->hard_header, et al. >> >> Having the bonding master disappear (but only sometimes) after >> the last slave is removed is a semantic change I'd rather not introduce >> if it's not necessary. > >Thanks for the comments. > >Having the master disappear is one way I could think of to solve the problem of leaving >the bonding module with pointers to illegal addresses. >The other way is to increase the usage count, with try_module_get(), of the module which owns of the slave. >To do that I have to restore the field owner in structure net_device (it was removed in 2.6). What I was asking above is really whether or not it's feasible to simply reset the affected pointers back to the "ethernet" values from ether_setup(). I would think this should return the bonding master back to the original state it started in before any slaves were added. Unless I'm missing something; I'm willing to believe there's some IB-specific tidbit I'm unaware of that makes this more complicated than it seems. This presumes that I'm correct in thinking that the pointers you're talking about (as being unsafe after removal of last slave) are the ones copied in your new function bond_setup_by_slave(). I don't think it's desirable to acquire a reference to the slave driver module. -J --- -Jay Vosburgh, IBM Linux Technology Center, fubar at us.ibm.com From tziporet at dev.mellanox.co.il Wed Aug 29 13:16:12 2007 From: tziporet at dev.mellanox.co.il (Tziporet Koren) Date: Wed, 29 Aug 2007 23:16:12 +0300 Subject: [ofa-general] Re: [ewg] OFED Aug 27 meeting summary on OFED 1.3 development status In-Reply-To: <200708281412.l7SEC1kk006041@xi.cse.ohio-state.edu> References: <200708281412.l7SEC1kk006041@xi.cse.ohio-state.edu> Message-ID: <46D5D40C.8090202@mellanox.co.il> Dhabaleswar Panda wrote: > Tziporet, > > >> OFED Aug 13 meeting summary on OFED 1.3 development status: >> >> 1. Decided to delay the feature freeze to Sep 10. >> * MPI update: >> * Open MPI: will have some intermediate package for OFED >> * Mvapich: first OFED package will start with version 0.99 (that >> was part of 1.2.5). The package of the new release will be ready by end >> of Sep. >> * Mvapich2: new release 1.2 will be ready on time for the feature >> freeze. >> > ^^^ This should be 1.0 version (not 1.2). > We already have MVAPICH2 1.0-beta available > for the last 4 weeks. We are in our final > testing phase. We are aiming to have the > final release by the feature freeze > of Sept 10th. > > Please correct this in your note. > > Sure (sorry for the mismatch) Tziporet From hal.rosenstock at gmail.com Wed Aug 29 14:48:38 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 29 Aug 2007 17:48:38 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D55C47.6080705@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> Message-ID: On 8/29/07, Yevgeny Kliteynik wrote: > Hal Rosenstock wrote: > > On 8/29/07, Yevgeny Kliteynik wrote: > >> Hal Rosenstock wrote: > >>> On 8/27/07, Yevgeny Kliteynik wrote: > >>>> Adding QoS policy file option to OpenSM optionsm and > >>>> QoS Policy field to subn object. > >>>> > >>>> Signed-off-by: Yevgeny Kliteynik > >>>> --- > >>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > >>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > >>>> opensm/opensm/main.c | 12 +++++++++++- > >>>> opensm/opensm/osm_subnet.c | 10 +++++++++- > >>>> 4 files changed, 50 insertions(+), 2 deletions(-) > >>>> > >>>> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h > >>>> index 545779b..0a35e22 100644 > >>>> --- a/opensm/include/opensm/osm_base.h > >>>> +++ b/opensm/include/opensm/osm_base.h > >>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS > >>>> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" > >>>> #endif > >>>> /***********/ > >>>> + > >>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > >>>> +* NAME > >>>> +* OSM_DEFAULT_QOS_POLICY_FILE > >>>> +* > >>>> +* DESCRIPTION > >>>> +* Specifies the default QoS policy file name > >>>> +* > >>>> +* SYNOPSIS > >>>> +*/ > >>>> +#ifdef __WIN__ > >>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") > >>>> +#else > >>>> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" > >>>> +#endif > >>>> +/***********/ > >>>> + > >>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>> * NAME > >>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > >>>> index a7543dc..5e8b522 100644 > >>>> --- a/opensm/include/opensm/osm_subnet.h > >>>> +++ b/opensm/include/opensm/osm_subnet.h > >>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > >>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > >>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 > >>>> struct _osm_opensm_t; > >>>> +struct _osm_qos_policy_t; > >>>> > >>>> /****h* OpenSM/Subnet > >>>> * NAME > >>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > >>>> char *partition_config_file; > >>>> boolean_t no_partition_enforcement; > >>>> boolean_t no_qos; > >>>> + char *qos_policy_file; > >>>> boolean_t accum_log_file; > >>>> char *console; > >>>> uint16_t console_port; > >>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > >>>> * specified the log file will be truncated upon reaching > >>>> * this limit. > >>>> * > >>>> +* no_qos > >>>> +* Boolean that specifies whether the OpenSM QoS functionality > >>>> +* should be off or on. > >>>> +* > >>>> +* qos_policy_file > >>>> +* Name of the QoS policy file. > >>>> +* > >>> Are these options mutually exclusive or not ? > >> If QoS in OpenSM is off, the qos_policy_file will be ignored > > > > But aren't there two QoSs ? Can't the old QoS be run without this ? I > > think that is a requirement. > > OK, there are three ways we want this thing to work: > 1. QoS is off > 2. The old QoS is on but w/o policy file > 3. The old QoS is on, plus reading policy file > > The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. > > Second and third options: if QoS is on, OpenSM looks for policy file in the default > location or in other location that was provided by user. If the file is not found, > QoS works as before. This sounds OK to me and is my first preference. > Do we want to add additional option for "enhanced" QoS? > If so, we will have three QoS-ralated command line options: > - option for turning the QoS on (currently -Q) > - option to turn the new QoS on (some new letter - must get > one quick before they all run out... :) > - option for policy file location if differs from default (currently -Y) This seems like the least preferable to me. Also, would need to deal with both on which seems to mean use new QoS. > Alternatively, we can turn -Q option into levels: > -Q 0: QoS is off (default) > -Q 1: old QoS is on > -Q 2: old QoS plus reading policy file This one also seems OK to me (second preference). Anyone else with an opinion on this ? Sasha ? Also, what about --enable-qos-policy build option for those who don't want to include this in their build ? -- Hal > > -- Yevgeny > > >>>> * accum_log_file > >>>> * If TRUE (default) - the log file will be accumulated. > >>>> * If FALSE - the log file will be erased before starting current opensm run. > >>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > >>>> ib_net64_t sm_port_guid; > >>>> uint8_t sm_state; > >>>> osm_subn_opt_t opt; > >>>> + struct _osm_qos_policy_t *p_qos_policy; > >>>> uint16_t max_unicast_lid_ho; > >>>> uint16_t max_multicast_lid_ho; > >>>> uint8_t min_ca_mtu; > >>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > >>>> * opt > >>>> * Subnet options structure contains site specific configuration. > >>>> * > >>>> +* p_qos_policy > >>>> +* Subnet QoS policy structure. > >>>> +* > >>>> * max_unicast_lid_ho > >>>> * The minimal max unicast lid reported by all switches > >>>> * > >>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > >>>> index e2541d8..599c4dc 100644 > >>>> --- a/opensm/opensm/main.c > >>>> +++ b/opensm/opensm/main.c > >>>> @@ -269,6 +269,10 @@ void show_usage(void) > >>>> " The default name is \'" > >>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > >>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > >>>> + printf("-Y\n" > >>>> + "--qos_policy_file\n" > >>>> + " This option defines the optional QoS policy file.\n" > >>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > >>>> printf("-N\n" > >>>> "--no_part_enforce\n" > >>>> " This option disables partition enforcement on switch external ports.\n\n"); > >>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > >>>> char *ignore_guids_file_name = NULL; > >>>> uint32_t val; > >>>> const char *const short_option = > >>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > >>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > >>>> > >>>> /* > >>>> In the array below, the 2nd parameter specifies the number > >>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > >>>> {"Pconfig", 1, NULL, 'P'}, > >>>> {"no_part_enforce", 0, NULL, 'N'}, > >>>> {"qos", 0, NULL, 'Q'}, > >>>> + {"qos_policy_file", 1, NULL, 'Y'}, > >>>> {"maxsmps", 1, NULL, 'n'}, > >>>> {"console", 1, NULL, 'q'}, > >>>> {"V", 0, NULL, 'V'}, > >>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > >>>> opt.no_qos = FALSE; > >>>> break; > >>>> > >>>> + case 'Y': > >>>> + opt.qos_policy_file = optarg; > >>>> + printf(" QoS policy file \'%s\'\n", optarg); > >>>> + break; > >>>> + > >>> There should also be an update to the OpenSM man page for this. > >> That and a text file with description of policy file syntax > > > > Good. Thanks. > > > > -- Hal > > > >> -- Yevgeny > >> > >>> -- Hal > >>> > >>>> case 'y': > >>>> opt.exit_on_fatal = FALSE; > >>>> printf(" Staying on fatal initialization errors\n"); > >>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > >>>> index 818d73f..4162522 100644 > >>>> --- a/opensm/opensm/osm_subnet.c > >>>> +++ b/opensm/opensm/osm_subnet.c > >>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > >>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > >>>> p_opt->no_partition_enforcement = FALSE; > >>>> p_opt->no_qos = TRUE; > >>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > >>>> p_opt->accum_log_file = TRUE; > >>>> p_opt->port_profile_switch_nodes = FALSE; > >>>> p_opt->pfn_ui_pre_lid_assign = NULL; > >>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > >>>> > >>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > >>>> > >>>> + opts_unpack_charp("qos_policy_file", > >>>> + p_key, p_val, &p_opts->qos_policy_file); > >>>> + > >>>> opts_unpack_boolean("accum_log_file", > >>>> p_key, p_val, &p_opts->accum_log_file); > >>>> > >>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > >>>> fprintf(opts_file, > >>>> "#\n# QoS OPTIONS\n#\n" > >>>> "# Disable QoS setup\n" > >>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > >>>> + "no_qos %s\n\n" > >>>> + "# QoS policy file to be used\n" > >>>> + "qos_policy_file %s\n\n", > >>>> + p_opts->no_qos ? "TRUE" : "FALSE", > >>>> + p_opts->qos_policy_file); > >>>> > >>>> subn_dump_qos_options(opts_file, > >>>> "QoS default options", "qos", > >>>> -- > >>>> 1.5.1.4 > >>>> > >>>> _______________________________________________ > >>>> general mailing list > >>>> general at lists.openfabrics.org > >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >>>> > >>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >>>> > >> > > > > From sashak at voltaire.com Wed Aug 29 16:01:20 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 30 Aug 2007 02:01:20 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> Message-ID: <20070829230120.GH7140@sashak.voltaire.com> On 17:48 Wed 29 Aug , Hal Rosenstock wrote: > > > > OK, there are three ways we want this thing to work: > > 1. QoS is off > > 2. The old QoS is on but w/o policy file > > 3. The old QoS is on, plus reading policy file > > > > The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. > > > > Second and third options: if QoS is on, OpenSM looks for policy file in the default > > location or in other location that was provided by user. If the file is not found, > > QoS works as before. > > This sounds OK to me and is my first preference. > > > Do we want to add additional option for "enhanced" QoS? > > If so, we will have three QoS-ralated command line options: > > - option for turning the QoS on (currently -Q) > > - option to turn the new QoS on (some new letter - must get > > one quick before they all run out... :) > > - option for policy file location if differs from default (currently -Y) > > This seems like the least preferable to me. Also, would need to deal > with both on which seems to mean use new QoS. > > > Alternatively, we can turn -Q option into levels: > > -Q 0: QoS is off (default) > > -Q 1: old QoS is on > > -Q 2: old QoS plus reading policy file > > This one also seems OK to me (second preference). > > Anyone else with an opinion on this ? Sasha ? I like -Q and -Y as Yevgeny proposed. Sasha > Also, what about --enable-qos-policy build option for those who don't > want to include this in their build ? > > -- Hal > > > > > -- Yevgeny > > > > >>>> * accum_log_file > > >>>> * If TRUE (default) - the log file will be accumulated. > > >>>> * If FALSE - the log file will be erased before starting current opensm run. > > >>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > > >>>> ib_net64_t sm_port_guid; > > >>>> uint8_t sm_state; > > >>>> osm_subn_opt_t opt; > > >>>> + struct _osm_qos_policy_t *p_qos_policy; > > >>>> uint16_t max_unicast_lid_ho; > > >>>> uint16_t max_multicast_lid_ho; > > >>>> uint8_t min_ca_mtu; > > >>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > > >>>> * opt > > >>>> * Subnet options structure contains site specific configuration. > > >>>> * > > >>>> +* p_qos_policy > > >>>> +* Subnet QoS policy structure. > > >>>> +* > > >>>> * max_unicast_lid_ho > > >>>> * The minimal max unicast lid reported by all switches > > >>>> * > > >>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > > >>>> index e2541d8..599c4dc 100644 > > >>>> --- a/opensm/opensm/main.c > > >>>> +++ b/opensm/opensm/main.c > > >>>> @@ -269,6 +269,10 @@ void show_usage(void) > > >>>> " The default name is \'" > > >>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > > >>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > > >>>> + printf("-Y\n" > > >>>> + "--qos_policy_file\n" > > >>>> + " This option defines the optional QoS policy file.\n" > > >>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > > >>>> printf("-N\n" > > >>>> "--no_part_enforce\n" > > >>>> " This option disables partition enforcement on switch external ports.\n\n"); > > >>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > > >>>> char *ignore_guids_file_name = NULL; > > >>>> uint32_t val; > > >>>> const char *const short_option = > > >>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > > >>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > >>>> > > >>>> /* > > >>>> In the array below, the 2nd parameter specifies the number > > >>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > > >>>> {"Pconfig", 1, NULL, 'P'}, > > >>>> {"no_part_enforce", 0, NULL, 'N'}, > > >>>> {"qos", 0, NULL, 'Q'}, > > >>>> + {"qos_policy_file", 1, NULL, 'Y'}, > > >>>> {"maxsmps", 1, NULL, 'n'}, > > >>>> {"console", 1, NULL, 'q'}, > > >>>> {"V", 0, NULL, 'V'}, > > >>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > > >>>> opt.no_qos = FALSE; > > >>>> break; > > >>>> > > >>>> + case 'Y': > > >>>> + opt.qos_policy_file = optarg; > > >>>> + printf(" QoS policy file \'%s\'\n", optarg); > > >>>> + break; > > >>>> + > > >>> There should also be an update to the OpenSM man page for this. > > >> That and a text file with description of policy file syntax > > > > > > Good. Thanks. > > > > > > -- Hal > > > > > >> -- Yevgeny > > >> > > >>> -- Hal > > >>> > > >>>> case 'y': > > >>>> opt.exit_on_fatal = FALSE; > > >>>> printf(" Staying on fatal initialization errors\n"); > > >>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > > >>>> index 818d73f..4162522 100644 > > >>>> --- a/opensm/opensm/osm_subnet.c > > >>>> +++ b/opensm/opensm/osm_subnet.c > > >>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > > >>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > > >>>> p_opt->no_partition_enforcement = FALSE; > > >>>> p_opt->no_qos = TRUE; > > >>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > > >>>> p_opt->accum_log_file = TRUE; > > >>>> p_opt->port_profile_switch_nodes = FALSE; > > >>>> p_opt->pfn_ui_pre_lid_assign = NULL; > > >>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > > >>>> > > >>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > > >>>> > > >>>> + opts_unpack_charp("qos_policy_file", > > >>>> + p_key, p_val, &p_opts->qos_policy_file); > > >>>> + > > >>>> opts_unpack_boolean("accum_log_file", > > >>>> p_key, p_val, &p_opts->accum_log_file); > > >>>> > > >>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > > >>>> fprintf(opts_file, > > >>>> "#\n# QoS OPTIONS\n#\n" > > >>>> "# Disable QoS setup\n" > > >>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > > >>>> + "no_qos %s\n\n" > > >>>> + "# QoS policy file to be used\n" > > >>>> + "qos_policy_file %s\n\n", > > >>>> + p_opts->no_qos ? "TRUE" : "FALSE", > > >>>> + p_opts->qos_policy_file); > > >>>> > > >>>> subn_dump_qos_options(opts_file, > > >>>> "QoS default options", "qos", > > >>>> -- > > >>>> 1.5.1.4 > > >>>> > > >>>> _______________________________________________ > > >>>> general mailing list > > >>>> general at lists.openfabrics.org > > >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > >>>> > > >>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > >>>> > > >> > > > > > > > From hal.rosenstock at gmail.com Wed Aug 29 16:12:27 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Wed, 29 Aug 2007 19:12:27 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070829230120.GH7140@sashak.voltaire.com> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> Message-ID: On 8/29/07, Sasha Khapyorsky wrote: > On 17:48 Wed 29 Aug , Hal Rosenstock wrote: > > > > > > OK, there are three ways we want this thing to work: > > > 1. QoS is off > > > 2. The old QoS is on but w/o policy file > > > 3. The old QoS is on, plus reading policy file > > > > > > The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. > > > > > > Second and third options: if QoS is on, OpenSM looks for policy file in the default > > > location or in other location that was provided by user. If the file is not found, > > > QoS works as before. > > > > This sounds OK to me and is my first preference. > > > > > Do we want to add additional option for "enhanced" QoS? > > > If so, we will have three QoS-ralated command line options: > > > - option for turning the QoS on (currently -Q) > > > - option to turn the new QoS on (some new letter - must get > > > one quick before they all run out... :) > > > - option for policy file location if differs from default (currently -Y) > > > > This seems like the least preferable to me. Also, would need to deal > > with both on which seems to mean use new QoS. > > > > > Alternatively, we can turn -Q option into levels: > > > -Q 0: QoS is off (default) > > > -Q 1: old QoS is on > > > -Q 2: old QoS plus reading policy file > > > > This one also seems OK to me (second preference). > > > > Anyone else with an opinion on this ? Sasha ? > > I like -Q and -Y as Yevgeny proposed. So is that the first option ? Actually, I think I like the third option best now that I think more on this. It seems a little odd to me to rely on the policy file not being present to determine which QoS to run. Seems a little cleaner this way to me. -- Hal > > Sasha > > > Also, what about --enable-qos-policy build option for those who don't > > want to include this in their build ? > > > > -- Hal > > > > > > > > -- Yevgeny > > > > > > >>>> * accum_log_file > > > >>>> * If TRUE (default) - the log file will be accumulated. > > > >>>> * If FALSE - the log file will be erased before starting current opensm run. > > > >>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > > > >>>> ib_net64_t sm_port_guid; > > > >>>> uint8_t sm_state; > > > >>>> osm_subn_opt_t opt; > > > >>>> + struct _osm_qos_policy_t *p_qos_policy; > > > >>>> uint16_t max_unicast_lid_ho; > > > >>>> uint16_t max_multicast_lid_ho; > > > >>>> uint8_t min_ca_mtu; > > > >>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > > > >>>> * opt > > > >>>> * Subnet options structure contains site specific configuration. > > > >>>> * > > > >>>> +* p_qos_policy > > > >>>> +* Subnet QoS policy structure. > > > >>>> +* > > > >>>> * max_unicast_lid_ho > > > >>>> * The minimal max unicast lid reported by all switches > > > >>>> * > > > >>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > > > >>>> index e2541d8..599c4dc 100644 > > > >>>> --- a/opensm/opensm/main.c > > > >>>> +++ b/opensm/opensm/main.c > > > >>>> @@ -269,6 +269,10 @@ void show_usage(void) > > > >>>> " The default name is \'" > > > >>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > > > >>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > > > >>>> + printf("-Y\n" > > > >>>> + "--qos_policy_file\n" > > > >>>> + " This option defines the optional QoS policy file.\n" > > > >>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > > > >>>> printf("-N\n" > > > >>>> "--no_part_enforce\n" > > > >>>> " This option disables partition enforcement on switch external ports.\n\n"); > > > >>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > > > >>>> char *ignore_guids_file_name = NULL; > > > >>>> uint32_t val; > > > >>>> const char *const short_option = > > > >>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > > > >>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > > >>>> > > > >>>> /* > > > >>>> In the array below, the 2nd parameter specifies the number > > > >>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > > > >>>> {"Pconfig", 1, NULL, 'P'}, > > > >>>> {"no_part_enforce", 0, NULL, 'N'}, > > > >>>> {"qos", 0, NULL, 'Q'}, > > > >>>> + {"qos_policy_file", 1, NULL, 'Y'}, > > > >>>> {"maxsmps", 1, NULL, 'n'}, > > > >>>> {"console", 1, NULL, 'q'}, > > > >>>> {"V", 0, NULL, 'V'}, > > > >>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > > > >>>> opt.no_qos = FALSE; > > > >>>> break; > > > >>>> > > > >>>> + case 'Y': > > > >>>> + opt.qos_policy_file = optarg; > > > >>>> + printf(" QoS policy file \'%s\'\n", optarg); > > > >>>> + break; > > > >>>> + > > > >>> There should also be an update to the OpenSM man page for this. > > > >> That and a text file with description of policy file syntax > > > > > > > > Good. Thanks. > > > > > > > > -- Hal > > > > > > > >> -- Yevgeny > > > >> > > > >>> -- Hal > > > >>> > > > >>>> case 'y': > > > >>>> opt.exit_on_fatal = FALSE; > > > >>>> printf(" Staying on fatal initialization errors\n"); > > > >>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > > > >>>> index 818d73f..4162522 100644 > > > >>>> --- a/opensm/opensm/osm_subnet.c > > > >>>> +++ b/opensm/opensm/osm_subnet.c > > > >>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > > > >>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > > > >>>> p_opt->no_partition_enforcement = FALSE; > > > >>>> p_opt->no_qos = TRUE; > > > >>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > > > >>>> p_opt->accum_log_file = TRUE; > > > >>>> p_opt->port_profile_switch_nodes = FALSE; > > > >>>> p_opt->pfn_ui_pre_lid_assign = NULL; > > > >>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > > > >>>> > > > >>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > > > >>>> > > > >>>> + opts_unpack_charp("qos_policy_file", > > > >>>> + p_key, p_val, &p_opts->qos_policy_file); > > > >>>> + > > > >>>> opts_unpack_boolean("accum_log_file", > > > >>>> p_key, p_val, &p_opts->accum_log_file); > > > >>>> > > > >>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > > > >>>> fprintf(opts_file, > > > >>>> "#\n# QoS OPTIONS\n#\n" > > > >>>> "# Disable QoS setup\n" > > > >>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > > > >>>> + "no_qos %s\n\n" > > > >>>> + "# QoS policy file to be used\n" > > > >>>> + "qos_policy_file %s\n\n", > > > >>>> + p_opts->no_qos ? "TRUE" : "FALSE", > > > >>>> + p_opts->qos_policy_file); > > > >>>> > > > >>>> subn_dump_qos_options(opts_file, > > > >>>> "QoS default options", "qos", > > > >>>> -- > > > >>>> 1.5.1.4 > > > >>>> > > > >>>> _______________________________________________ > > > >>>> general mailing list > > > >>>> general at lists.openfabrics.org > > > >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > >>>> > > > >>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > >>>> > > > >> > > > > > > > > > > > From sashak at voltaire.com Wed Aug 29 16:40:14 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 30 Aug 2007 02:40:14 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> Message-ID: <20070829234014.GI7140@sashak.voltaire.com> On 19:12 Wed 29 Aug , Hal Rosenstock wrote: > On 8/29/07, Sasha Khapyorsky wrote: > > On 17:48 Wed 29 Aug , Hal Rosenstock wrote: > > > > > > > > OK, there are three ways we want this thing to work: > > > > 1. QoS is off > > > > 2. The old QoS is on but w/o policy file > > > > 3. The old QoS is on, plus reading policy file > > > > > > > > The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. > > > > > > > > Second and third options: if QoS is on, OpenSM looks for policy file in the default > > > > location or in other location that was provided by user. If the file is not found, > > > > QoS works as before. > > > > > > This sounds OK to me and is my first preference. > > > > > > > Do we want to add additional option for "enhanced" QoS? > > > > If so, we will have three QoS-ralated command line options: > > > > - option for turning the QoS on (currently -Q) > > > > - option to turn the new QoS on (some new letter - must get > > > > one quick before they all run out... :) > > > > - option for policy file location if differs from default (currently -Y) > > > > > > This seems like the least preferable to me. Also, would need to deal > > > with both on which seems to mean use new QoS. > > > > > > > Alternatively, we can turn -Q option into levels: > > > > -Q 0: QoS is off (default) > > > > -Q 1: old QoS is on > > > > -Q 2: old QoS plus reading policy file > > > > > > This one also seems OK to me (second preference). > > > > > > Anyone else with an opinion on this ? Sasha ? > > > > I like -Q and -Y as Yevgeny proposed. > > So is that the first option ? Yes. It is simplest and provides the same functionality. > Actually, I think I like the third option best now that I think more > on this. It seems a little odd to me to rely on the policy file not > being present to determine which QoS to run. Seems a little cleaner > this way to me. We need file name option anyway, so things like '-Q 1 -Y ...' are unclear. Also it would be nice to have "universal" (not for "two QoS") user interface in order to not change it later. Sasha From kliteyn at dev.mellanox.co.il Wed Aug 29 18:19:26 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 04:19:26 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070829234014.GI7140@sashak.voltaire.com> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> Message-ID: <46D61B1E.50501@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 19:12 Wed 29 Aug , Hal Rosenstock wrote: >> On 8/29/07, Sasha Khapyorsky wrote: >>> On 17:48 Wed 29 Aug , Hal Rosenstock wrote: >>>>> OK, there are three ways we want this thing to work: >>>>> 1. QoS is off >>>>> 2. The old QoS is on but w/o policy file >>>>> 3. The old QoS is on, plus reading policy file >>>>> >>>>> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. >>>>> >>>>> Second and third options: if QoS is on, OpenSM looks for policy file in the default >>>>> location or in other location that was provided by user. If the file is not found, >>>>> QoS works as before. >>>> This sounds OK to me and is my first preference. >>>> >>>>> Do we want to add additional option for "enhanced" QoS? >>>>> If so, we will have three QoS-ralated command line options: >>>>> - option for turning the QoS on (currently -Q) >>>>> - option to turn the new QoS on (some new letter - must get >>>>> one quick before they all run out... :) >>>>> - option for policy file location if differs from default (currently -Y) >>>> This seems like the least preferable to me. Also, would need to deal >>>> with both on which seems to mean use new QoS. >>>> >>>>> Alternatively, we can turn -Q option into levels: >>>>> -Q 0: QoS is off (default) >>>>> -Q 1: old QoS is on >>>>> -Q 2: old QoS plus reading policy file >>>> This one also seems OK to me (second preference). >>>> >>>> Anyone else with an opinion on this ? Sasha ? >>> I like -Q and -Y as Yevgeny proposed. >> So is that the first option ? > > Yes. It is simplest and provides the same functionality. This is what's implemented right now >> Actually, I think I like the third option best now that I think more >> on this. It seems a little odd to me to rely on the policy file not >> being present to determine which QoS to run. Seems a little cleaner >> this way to me. > > We need file name option anyway, so things like '-Q 1 -Y ...' are > unclear. Also it would be nice to have "universal" (not for "two QoS") > user interface in order to not change it later. I would say that -Q and -Y are enough, but it poses some questions: Do we want to allow the case when a user has policy file in a default location, but he wants OpenSM to ignore this file and still have QoS on? In case of partitions file we don't have an option to ignore partition config file in a default location. And if OpenSM ignores policy file, what would it mean? Would it be the "old" QoS? And when the setup part of the new QoS will be ready, would we still want the SL2VL and VLArb tables to appear in the opts file? -- Yevgeny > Sasha > From kliteyn at dev.mellanox.co.il Wed Aug 29 18:32:03 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 04:32:03 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> Message-ID: <46D61E13.4020004@dev.mellanox.co.il> Hal Rosenstock wrote: > On 8/29/07, Yevgeny Kliteynik wrote: >> Hal Rosenstock wrote: >>> On 8/29/07, Yevgeny Kliteynik wrote: >>>> Hal Rosenstock wrote: >>>>> On 8/27/07, Yevgeny Kliteynik wrote: >>>>>> Adding QoS policy file option to OpenSM optionsm and >>>>>> QoS Policy field to subn object. >>>>>> >>>>>> Signed-off-by: Yevgeny Kliteynik >>>>>> --- >>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ >>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ >>>>>> opensm/opensm/main.c | 12 +++++++++++- >>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- >>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) >>>>>> >>>>>> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h >>>>>> index 545779b..0a35e22 100644 >>>>>> --- a/opensm/include/opensm/osm_base.h >>>>>> +++ b/opensm/include/opensm/osm_base.h >>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS >>>>>> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" >>>>>> #endif >>>>>> /***********/ >>>>>> + >>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE >>>>>> +* NAME >>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE >>>>>> +* >>>>>> +* DESCRIPTION >>>>>> +* Specifies the default QoS policy file name >>>>>> +* >>>>>> +* SYNOPSIS >>>>>> +*/ >>>>>> +#ifdef __WIN__ >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") >>>>>> +#else >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" >>>>>> +#endif >>>>>> +/***********/ >>>>>> + >>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>> * NAME >>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h >>>>>> index a7543dc..5e8b522 100644 >>>>>> --- a/opensm/include/opensm/osm_subnet.h >>>>>> +++ b/opensm/include/opensm/osm_subnet.h >>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS >>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 >>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 >>>>>> struct _osm_opensm_t; >>>>>> +struct _osm_qos_policy_t; >>>>>> >>>>>> /****h* OpenSM/Subnet >>>>>> * NAME >>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { >>>>>> char *partition_config_file; >>>>>> boolean_t no_partition_enforcement; >>>>>> boolean_t no_qos; >>>>>> + char *qos_policy_file; >>>>>> boolean_t accum_log_file; >>>>>> char *console; >>>>>> uint16_t console_port; >>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { >>>>>> * specified the log file will be truncated upon reaching >>>>>> * this limit. >>>>>> * >>>>>> +* no_qos >>>>>> +* Boolean that specifies whether the OpenSM QoS functionality >>>>>> +* should be off or on. >>>>>> +* >>>>>> +* qos_policy_file >>>>>> +* Name of the QoS policy file. >>>>>> +* >>>>> Are these options mutually exclusive or not ? >>>> If QoS in OpenSM is off, the qos_policy_file will be ignored >>> But aren't there two QoSs ? Can't the old QoS be run without this ? I >>> think that is a requirement. >> OK, there are three ways we want this thing to work: >> 1. QoS is off >> 2. The old QoS is on but w/o policy file >> 3. The old QoS is on, plus reading policy file >> >> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. >> >> Second and third options: if QoS is on, OpenSM looks for policy file in the default >> location or in other location that was provided by user. If the file is not found, >> QoS works as before. > > This sounds OK to me and is my first preference. > >> Do we want to add additional option for "enhanced" QoS? >> If so, we will have three QoS-ralated command line options: >> - option for turning the QoS on (currently -Q) >> - option to turn the new QoS on (some new letter - must get >> one quick before they all run out... :) >> - option for policy file location if differs from default (currently -Y) > > This seems like the least preferable to me. Also, would need to deal > with both on which seems to mean use new QoS. > >> Alternatively, we can turn -Q option into levels: >> -Q 0: QoS is off (default) >> -Q 1: old QoS is on >> -Q 2: old QoS plus reading policy file > > This one also seems OK to me (second preference). > > Anyone else with an opinion on this ? Sasha ? > > Also, what about --enable-qos-policy build option for those who don't > want to include this in their build ? Do you mean permanently, or just until the new QoS will be stable? Note that if someone gets ofed rpm, he gets the generated files in this rpm, so he doesn't need les or yacc, so this option would be useful only for machines that have git but don't have lex or yacc. Do you think it's a real issue? -- Yevgeny > -- Hal > >> -- Yevgeny >> >>>>>> * accum_log_file >>>>>> * If TRUE (default) - the log file will be accumulated. >>>>>> * If FALSE - the log file will be erased before starting current opensm run. >>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { >>>>>> ib_net64_t sm_port_guid; >>>>>> uint8_t sm_state; >>>>>> osm_subn_opt_t opt; >>>>>> + struct _osm_qos_policy_t *p_qos_policy; >>>>>> uint16_t max_unicast_lid_ho; >>>>>> uint16_t max_multicast_lid_ho; >>>>>> uint8_t min_ca_mtu; >>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { >>>>>> * opt >>>>>> * Subnet options structure contains site specific configuration. >>>>>> * >>>>>> +* p_qos_policy >>>>>> +* Subnet QoS policy structure. >>>>>> +* >>>>>> * max_unicast_lid_ho >>>>>> * The minimal max unicast lid reported by all switches >>>>>> * >>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c >>>>>> index e2541d8..599c4dc 100644 >>>>>> --- a/opensm/opensm/main.c >>>>>> +++ b/opensm/opensm/main.c >>>>>> @@ -269,6 +269,10 @@ void show_usage(void) >>>>>> " The default name is \'" >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); >>>>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); >>>>>> + printf("-Y\n" >>>>>> + "--qos_policy_file\n" >>>>>> + " This option defines the optional QoS policy file.\n" >>>>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); >>>>>> printf("-N\n" >>>>>> "--no_part_enforce\n" >>>>>> " This option disables partition enforcement on switch external ports.\n\n"); >>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) >>>>>> char *ignore_guids_file_name = NULL; >>>>>> uint32_t val; >>>>>> const char *const short_option = >>>>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; >>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; >>>>>> >>>>>> /* >>>>>> In the array below, the 2nd parameter specifies the number >>>>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) >>>>>> {"Pconfig", 1, NULL, 'P'}, >>>>>> {"no_part_enforce", 0, NULL, 'N'}, >>>>>> {"qos", 0, NULL, 'Q'}, >>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, >>>>>> {"maxsmps", 1, NULL, 'n'}, >>>>>> {"console", 1, NULL, 'q'}, >>>>>> {"V", 0, NULL, 'V'}, >>>>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) >>>>>> opt.no_qos = FALSE; >>>>>> break; >>>>>> >>>>>> + case 'Y': >>>>>> + opt.qos_policy_file = optarg; >>>>>> + printf(" QoS policy file \'%s\'\n", optarg); >>>>>> + break; >>>>>> + >>>>> There should also be an update to the OpenSM man page for this. >>>> That and a text file with description of policy file syntax >>> Good. Thanks. >>> >>> -- Hal >>> >>>> -- Yevgeny >>>> >>>>> -- Hal >>>>> >>>>>> case 'y': >>>>>> opt.exit_on_fatal = FALSE; >>>>>> printf(" Staying on fatal initialization errors\n"); >>>>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c >>>>>> index 818d73f..4162522 100644 >>>>>> --- a/opensm/opensm/osm_subnet.c >>>>>> +++ b/opensm/opensm/osm_subnet.c >>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) >>>>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; >>>>>> p_opt->no_partition_enforcement = FALSE; >>>>>> p_opt->no_qos = TRUE; >>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; >>>>>> p_opt->accum_log_file = TRUE; >>>>>> p_opt->port_profile_switch_nodes = FALSE; >>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; >>>>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) >>>>>> >>>>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); >>>>>> >>>>>> + opts_unpack_charp("qos_policy_file", >>>>>> + p_key, p_val, &p_opts->qos_policy_file); >>>>>> + >>>>>> opts_unpack_boolean("accum_log_file", >>>>>> p_key, p_val, &p_opts->accum_log_file); >>>>>> >>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) >>>>>> fprintf(opts_file, >>>>>> "#\n# QoS OPTIONS\n#\n" >>>>>> "# Disable QoS setup\n" >>>>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); >>>>>> + "no_qos %s\n\n" >>>>>> + "# QoS policy file to be used\n" >>>>>> + "qos_policy_file %s\n\n", >>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", >>>>>> + p_opts->qos_policy_file); >>>>>> >>>>>> subn_dump_qos_options(opts_file, >>>>>> "QoS default options", "qos", >>>>>> -- >>>>>> 1.5.1.4 >>>>>> >>>>>> _______________________________________________ >>>>>> general mailing list >>>>>> general at lists.openfabrics.org >>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>>> >>>>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >>>>>> >> > From rdreier at cisco.com Wed Aug 29 19:50:21 2007 From: rdreier at cisco.com (Roland Dreier) Date: Wed, 29 Aug 2007 19:50:21 -0700 Subject: [ofa-general] ib_umad method mask problems on big-endian 64-bit archs In-Reply-To: <20070822190519.GD1397@sashak.voltaire.com> (Sasha Khapyorsky's message of "Wed, 22 Aug 2007 22:05:19 +0300") References: <20070822190519.GD1397@sashak.voltaire.com> Message-ID: > It looks that using uint32_t for addr in set_bit() function is sufficient > fix. But for ppc64 this means that new OpenSM will break with old > kernels, probably we will need to put some ugly #ifdef in > osm_vendor_ibumad.c... Yes, that's a pain. Another possibility is to declare that the declaration of the registration request should have been long method_mask[16 / sizeof (long)]; and just add a compat_ioctl method to the ib_umad module to handle the broken case of 32-bit big endian userspace on a 64-bit kernel. However that breaks 64-bit big endian userspace that followed the old ib_user_mad.h file correctly so overall I'm leaning towards the patch I already posted. What do you think? - R. From kliteyn at mellanox.co.il Wed Aug 29 21:47:14 2007 From: kliteyn at mellanox.co.il (kliteyn at mellanox.co.il) Date: 30 Aug 2007 07:47:14 +0300 Subject: [ofa-general] nightly osm_sim report 2007-08-30:normal completion Message-ID: OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Wed_Aug_29_09:07:45_2007 [6d16fc960ab4d017844dead034859812021755bf] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=560 Fail=0 Pass: 42 Stability IS1-16.topo 42 Pkey IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: From eitan at mellanox.co.il Wed Aug 29 23:05:40 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Thu, 30 Aug 2007 09:05:40 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D61E13.4020004@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> Message-ID: <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> The standard option that is used by auto tools to have dependencies that are only used by a maintainer is: --enable-maintainer-mode We use it to hide .l/.y/.i dependencies from the Makefile such that only those who declare themselves maintainers will refresh automatically generated source files. Why do you want to invent a new one? Eitan Zahavi Senior Engineering Director, Software Architect Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Yevgeny Kliteynik > Sent: Thursday, August 30, 2007 4:32 AM > To: Hal Rosenstock > Cc: OpenIB > Subject: Re: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding > QoS policy options > > Hal Rosenstock wrote: > > On 8/29/07, Yevgeny Kliteynik wrote: > >> Hal Rosenstock wrote: > >>> On 8/29/07, Yevgeny Kliteynik wrote: > >>>> Hal Rosenstock wrote: > >>>>> On 8/27/07, Yevgeny Kliteynik > wrote: > >>>>>> Adding QoS policy file option to OpenSM optionsm and > QoS Policy > >>>>>> field to subn object. > >>>>>> > >>>>>> Signed-off-by: Yevgeny Kliteynik > >>>>>> --- > >>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > >>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > >>>>>> opensm/opensm/main.c | 12 +++++++++++- > >>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- > >>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) > >>>>>> > >>>>>> diff --git a/opensm/include/opensm/osm_base.h > >>>>>> b/opensm/include/opensm/osm_base.h > >>>>>> index 545779b..0a35e22 100644 > >>>>>> --- a/opensm/include/opensm/osm_base.h > >>>>>> +++ b/opensm/include/opensm/osm_base.h > >>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE > "/etc/ofa/opensm-partitions.conf" > >>>>>> #endif > >>>>>> /***********/ > >>>>>> + > >>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > >>>>>> +* NAME > >>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE > >>>>>> +* > >>>>>> +* DESCRIPTION > >>>>>> +* Specifies the default QoS policy file name > >>>>>> +* > >>>>>> +* SYNOPSIS > >>>>>> +*/ > >>>>>> +#ifdef __WIN__ > >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), > >>>>>> +"osm-qos-policy.conf") #else #define > OSM_DEFAULT_QOS_POLICY_FILE > >>>>>> +"/etc/ofa/opensm-qos-policy.conf" > >>>>>> +#endif > >>>>>> +/***********/ > >>>>>> + > >>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>>>> * NAME > >>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>>>> diff --git a/opensm/include/opensm/osm_subnet.h > >>>>>> b/opensm/include/opensm/osm_subnet.h > >>>>>> index a7543dc..5e8b522 100644 > >>>>>> --- a/opensm/include/opensm/osm_subnet.h > >>>>>> +++ b/opensm/include/opensm/osm_subnet.h > >>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > >>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > >>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 > >>>>>> struct _osm_opensm_t; > >>>>>> +struct _osm_qos_policy_t; > >>>>>> > >>>>>> /****h* OpenSM/Subnet > >>>>>> * NAME > >>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > >>>>>> char *partition_config_file; > >>>>>> boolean_t no_partition_enforcement; > >>>>>> boolean_t no_qos; > >>>>>> + char *qos_policy_file; > >>>>>> boolean_t accum_log_file; > >>>>>> char *console; > >>>>>> uint16_t console_port; > >>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > >>>>>> * specified the log file will be > truncated upon reaching > >>>>>> * this limit. > >>>>>> * > >>>>>> +* no_qos > >>>>>> +* Boolean that specifies whether the > OpenSM QoS functionality > >>>>>> +* should be off or on. > >>>>>> +* > >>>>>> +* qos_policy_file > >>>>>> +* Name of the QoS policy file. > >>>>>> +* > >>>>> Are these options mutually exclusive or not ? > >>>> If QoS in OpenSM is off, the qos_policy_file will be ignored > >>> But aren't there two QoSs ? Can't the old QoS be run > without this ? > >>> I think that is a requirement. > >> OK, there are three ways we want this thing to work: > >> 1. QoS is off > >> 2. The old QoS is on but w/o policy file 3. The old QoS is > on, plus > >> reading policy file > >> > >> The first option is clear: if a user doesn't turns QoS on > (-Q), QoS is off as before. > >> > >> Second and third options: if QoS is on, OpenSM looks for > policy file > >> in the default location or in other location that was provided by > >> user. If the file is not found, QoS works as before. > > > > This sounds OK to me and is my first preference. > > > >> Do we want to add additional option for "enhanced" QoS? > >> If so, we will have three QoS-ralated command line options: > >> - option for turning the QoS on (currently -Q) > >> - option to turn the new QoS on (some new letter - must get > >> one quick before they all run out... :) > >> - option for policy file location if differs from default > (currently > >> -Y) > > > > This seems like the least preferable to me. Also, would > need to deal > > with both on which seems to mean use new QoS. > > > >> Alternatively, we can turn -Q option into levels: > >> -Q 0: QoS is off (default) > >> -Q 1: old QoS is on > >> -Q 2: old QoS plus reading policy file > > > > This one also seems OK to me (second preference). > > > > Anyone else with an opinion on this ? Sasha ? > > > > Also, what about --enable-qos-policy build option for those > who don't > > want to include this in their build ? > > Do you mean permanently, or just until the new QoS will be stable? > Note that if someone gets ofed rpm, he gets the generated > files in this rpm, so he doesn't need les or yacc, so this > option would be useful only for machines that have git but > don't have lex or yacc. > Do you think it's a real issue? > > -- Yevgeny > > > -- Hal > > > >> -- Yevgeny > >> > >>>>>> * accum_log_file > >>>>>> * If TRUE (default) - the log file will > be accumulated. > >>>>>> * If FALSE - the log file will be erased > before starting current opensm run. > >>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > >>>>>> ib_net64_t sm_port_guid; > >>>>>> uint8_t sm_state; > >>>>>> osm_subn_opt_t opt; > >>>>>> + struct _osm_qos_policy_t *p_qos_policy; > >>>>>> uint16_t max_unicast_lid_ho; > >>>>>> uint16_t max_multicast_lid_ho; > >>>>>> uint8_t min_ca_mtu; > >>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > >>>>>> * opt > >>>>>> * Subnet options structure contains site > specific configuration. > >>>>>> * > >>>>>> +* p_qos_policy > >>>>>> +* Subnet QoS policy structure. > >>>>>> +* > >>>>>> * max_unicast_lid_ho > >>>>>> * The minimal max unicast lid reported > by all switches > >>>>>> * > >>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index > >>>>>> e2541d8..599c4dc 100644 > >>>>>> --- a/opensm/opensm/main.c > >>>>>> +++ b/opensm/opensm/main.c > >>>>>> @@ -269,6 +269,10 @@ void show_usage(void) > >>>>>> " The default name is \'" > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > >>>>>> printf("-Q\n" "--qos\n" " This option > enables QoS setup.\n\n"); > >>>>>> + printf("-Y\n" > >>>>>> + "--qos_policy_file\n" > >>>>>> + " This option defines the > optional QoS policy file.\n" > >>>>>> + " The default name is \'" > OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > >>>>>> printf("-N\n" > >>>>>> "--no_part_enforce\n" > >>>>>> " This option disables > partition enforcement on switch external ports.\n\n"); > >>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > >>>>>> char *ignore_guids_file_name = NULL; > >>>>>> uint32_t val; > >>>>>> const char *const short_option = > >>>>>> - > "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > >>>>>> + > >>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > >>>>>> > >>>>>> /* > >>>>>> In the array below, the 2nd parameter specifies the > >>>>>> number @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > >>>>>> {"Pconfig", 1, NULL, 'P'}, > >>>>>> {"no_part_enforce", 0, NULL, 'N'}, > >>>>>> {"qos", 0, NULL, 'Q'}, > >>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, > >>>>>> {"maxsmps", 1, NULL, 'n'}, > >>>>>> {"console", 1, NULL, 'q'}, > >>>>>> {"V", 0, NULL, 'V'}, @@ -823,6 +828,11 @@ int > >>>>>> main(int argc, char *argv[]) > >>>>>> opt.no_qos = FALSE; > >>>>>> break; > >>>>>> > >>>>>> + case 'Y': > >>>>>> + opt.qos_policy_file = optarg; > >>>>>> + printf(" QoS policy file > \'%s\'\n", optarg); > >>>>>> + break; > >>>>>> + > >>>>> There should also be an update to the OpenSM man page for this. > >>>> That and a text file with description of policy file syntax > >>> Good. Thanks. > >>> > >>> -- Hal > >>> > >>>> -- Yevgeny > >>>> > >>>>> -- Hal > >>>>> > >>>>>> case 'y': > >>>>>> opt.exit_on_fatal = FALSE; > >>>>>> printf(" Staying on fatal > initialization > >>>>>> errors\n"); diff --git a/opensm/opensm/osm_subnet.c > >>>>>> b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 > >>>>>> --- a/opensm/opensm/osm_subnet.c > >>>>>> +++ b/opensm/opensm/osm_subnet.c > >>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN > osm_subn_opt_t * const p_opt) > >>>>>> p_opt->partition_config_file = > OSM_DEFAULT_PARTITION_CONFIG_FILE; > >>>>>> p_opt->no_partition_enforcement = FALSE; > >>>>>> p_opt->no_qos = TRUE; > >>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > >>>>>> p_opt->accum_log_file = TRUE; > >>>>>> p_opt->port_profile_switch_nodes = FALSE; > >>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 > +1179,9 @@ > >>>>>> ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * > >>>>>> const p_opts) > >>>>>> > >>>>>> opts_unpack_boolean("no_qos", p_key, p_val, > >>>>>> &p_opts->no_qos); > >>>>>> > >>>>>> + opts_unpack_charp("qos_policy_file", > >>>>>> + p_key, p_val, > >>>>>> + &p_opts->qos_policy_file); > >>>>>> + > >>>>>> opts_unpack_boolean("accum_log_file", > >>>>>> p_key, p_val, > >>>>>> &p_opts->accum_log_file); > >>>>>> > >>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t > osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > >>>>>> fprintf(opts_file, > >>>>>> "#\n# QoS OPTIONS\n#\n" > >>>>>> "# Disable QoS setup\n" > >>>>>> - "no_qos %s\n\n", p_opts->no_qos ? > "TRUE" : "FALSE"); > >>>>>> + "no_qos %s\n\n" > >>>>>> + "# QoS policy file to be used\n" > >>>>>> + "qos_policy_file %s\n\n", > >>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", > >>>>>> + p_opts->qos_policy_file); > >>>>>> > >>>>>> subn_dump_qos_options(opts_file, > >>>>>> "QoS default options", "qos", > >>>>>> -- > >>>>>> 1.5.1.4 > >>>>>> > >>>>>> _______________________________________________ > >>>>>> general mailing list > >>>>>> general at lists.openfabrics.org > >>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >>>>>> > >>>>>> To unsubscribe, please visit > >>>>>> http://openib.org/mailman/listinfo/openib-general > >>>>>> > >> > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From sashak at voltaire.com Wed Aug 29 23:59:08 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 30 Aug 2007 09:59:08 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D61B1E.50501@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> Message-ID: <20070830065908.GK7140@sashak.voltaire.com> On 04:19 Thu 30 Aug , Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > On 19:12 Wed 29 Aug , Hal Rosenstock wrote: > >> On 8/29/07, Sasha Khapyorsky wrote: > >>> On 17:48 Wed 29 Aug , Hal Rosenstock wrote: > >>>>> OK, there are three ways we want this thing to work: > >>>>> 1. QoS is off > >>>>> 2. The old QoS is on but w/o policy file > >>>>> 3. The old QoS is on, plus reading policy file > >>>>> > >>>>> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is > >>>>> off as before. > >>>>> > >>>>> Second and third options: if QoS is on, OpenSM looks for policy file in > >>>>> the default > >>>>> location or in other location that was provided by user. If the file is > >>>>> not found, > >>>>> QoS works as before. > >>>> This sounds OK to me and is my first preference. > >>>> > >>>>> Do we want to add additional option for "enhanced" QoS? > >>>>> If so, we will have three QoS-ralated command line options: > >>>>> - option for turning the QoS on (currently -Q) > >>>>> - option to turn the new QoS on (some new letter - must get > >>>>> one quick before they all run out... :) > >>>>> - option for policy file location if differs from default (currently > >>>>> -Y) > >>>> This seems like the least preferable to me. Also, would need to deal > >>>> with both on which seems to mean use new QoS. > >>>> > >>>>> Alternatively, we can turn -Q option into levels: > >>>>> -Q 0: QoS is off (default) > >>>>> -Q 1: old QoS is on > >>>>> -Q 2: old QoS plus reading policy file > >>>> This one also seems OK to me (second preference). > >>>> > >>>> Anyone else with an opinion on this ? Sasha ? > >>> I like -Q and -Y as Yevgeny proposed. > >> So is that the first option ? > > Yes. It is simplest and provides the same functionality. > > This is what's implemented right now > > >> Actually, I think I like the third option best now that I think more > >> on this. It seems a little odd to me to rely on the policy file not > >> being present to determine which QoS to run. Seems a little cleaner > >> this way to me. > > We need file name option anyway, so things like '-Q 1 -Y ...' are > > unclear. Also it would be nice to have "universal" (not for "two QoS") > > user interface in order to not change it later. > > I would say that -Q and -Y are enough, but it poses some questions: > Do we want to allow the case when a user has policy file in a default > location, but he wants OpenSM to ignore this file and still have QoS on? -Y '' should be fine then. > In case of partitions file we don't have an option to ignore partition > config file in a default location. > > And if OpenSM ignores policy file, what would it mean? > Would it be the "old" QoS? Yes. Is there another useful options? > And when the setup part of the new QoS will be ready, would we still want > the SL2VL and VLArb tables to appear in the opts file? Don't think. Why we need it in two places? Sasha From sashak at voltaire.com Thu Aug 30 00:27:15 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 30 Aug 2007 10:27:15 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D61E13.4020004@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> Message-ID: <20070830072715.GL7140@sashak.voltaire.com> On 04:32 Thu 30 Aug , Yevgeny Kliteynik wrote: > > Also, what about --enable-qos-policy build option for those who don't > > want to include this in their build ? > > Do you mean permanently, or just until the new QoS will be stable? > Note that if someone gets ofed rpm, he gets the generated files in this > rpm, so he doesn't need les or yacc, so this option would be useful only > for machines that have git but don't have lex or yacc. > Do you think it's a real issue? It is not related to lex or yacc. Some users will want to run OpenSM on machines with limited resources and will like to strip it in size, when QoS is not planned for use, some 50+K could be stripped: text data bss dec hex filename 5820 21 0 5841 16d1 opensm-osm_qos.o 16722 25 96 16843 41cb opensm-osm_qos_parser_l.o 21319 28 24 21371 537b opensm-osm_qos_parser_y.o 10782 21 0 10803 2a33 opensm-osm_qos_policy.o I'm not sure what default should be, but to have such --enable/disable-qos config option would be useful. Sasha From eitan at mellanox.co.il Thu Aug 30 01:23:21 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Thu, 30 Aug 2007 11:23:21 +0300 Subject: [ofa-general] RE: ibdiags and gcc In-Reply-To: References: Message-ID: <6C2C79E72C305246B504CBA17B5500C9022AC7A3@mtlexch01.mtl.com> Done. Removed the entire lib/tls workaround. I hope we will not get stuck again on missed event during condwait. The bug was seen ~2 years ago and I hope TLS was fixed from then Eitan Zahavi > -----Original Message----- > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com] > Sent: Tuesday, August 28, 2007 4:45 PM > To: Eitan Zahavi > Cc: OpenIB > Subject: ibdiags and gcc > > Hi Eitan, > > Several of the ibdiags (ibdiagnet, ibdiagpath, ibdiagui) > appear to require gcc not just in the build environment but > also in the execution environment due to the following line > in those scripts: > > xxx_path=`gcc --print-file-name ... > > to determine some library paths. Any reason LD_LIBRARY_PATH > or something more standard can't be used for this ? > > -- Hal > From vlad at lists.openfabrics.org Thu Aug 30 02:48:38 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Thu, 30 Aug 2007 02:48:38 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070830-0200 daily build status Message-ID: <20070830094838.F35BEE608A7@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.22 Passed on powerpc with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on x86_64 with linux-2.6.21.1 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.15 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Build failed on powerpc with linux-2.6.17 Log: Log: /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070830-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- ---------------------------------------------------------------------------------- From hal.rosenstock at gmail.com Thu Aug 30 03:58:36 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 06:58:36 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070830065908.GK7140@sashak.voltaire.com> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> Message-ID: On 8/30/07, Sasha Khapyorsky wrote: > On 04:19 Thu 30 Aug , Yevgeny Kliteynik wrote: > > Sasha Khapyorsky wrote: > > > On 19:12 Wed 29 Aug , Hal Rosenstock wrote: > > >> On 8/29/07, Sasha Khapyorsky wrote: > > >>> On 17:48 Wed 29 Aug , Hal Rosenstock wrote: > > >>>>> OK, there are three ways we want this thing to work: > > >>>>> 1. QoS is off > > >>>>> 2. The old QoS is on but w/o policy file > > >>>>> 3. The old QoS is on, plus reading policy file > > >>>>> > > >>>>> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is > > >>>>> off as before. > > >>>>> > > >>>>> Second and third options: if QoS is on, OpenSM looks for policy file in > > >>>>> the default > > >>>>> location or in other location that was provided by user. If the file is > > >>>>> not found, > > >>>>> QoS works as before. > > >>>> This sounds OK to me and is my first preference. > > >>>> > > >>>>> Do we want to add additional option for "enhanced" QoS? > > >>>>> If so, we will have three QoS-ralated command line options: > > >>>>> - option for turning the QoS on (currently -Q) > > >>>>> - option to turn the new QoS on (some new letter - must get > > >>>>> one quick before they all run out... :) > > >>>>> - option for policy file location if differs from default (currently > > >>>>> -Y) > > >>>> This seems like the least preferable to me. Also, would need to deal > > >>>> with both on which seems to mean use new QoS. > > >>>> > > >>>>> Alternatively, we can turn -Q option into levels: > > >>>>> -Q 0: QoS is off (default) > > >>>>> -Q 1: old QoS is on > > >>>>> -Q 2: old QoS plus reading policy file > > >>>> This one also seems OK to me (second preference). > > >>>> > > >>>> Anyone else with an opinion on this ? Sasha ? > > >>> I like -Q and -Y as Yevgeny proposed. > > >> So is that the first option ? > > > Yes. It is simplest and provides the same functionality. > > > > This is what's implemented right now > > > > >> Actually, I think I like the third option best now that I think more > > >> on this. It seems a little odd to me to rely on the policy file not > > >> being present to determine which QoS to run. Seems a little cleaner > > >> this way to me. > > > We need file name option anyway, so things like '-Q 1 -Y ...' are > > > unclear. Also it would be nice to have "universal" (not for "two QoS") > > > user interface in order to not change it later. > > > > I would say that -Q and -Y are enough, but it poses some questions: > > Do we want to allow the case when a user has policy file in a default > > location, but he wants OpenSM to ignore this file and still have QoS on? > > -Y '' should be fine then. > > > In case of partitions file we don't have an option to ignore partition > > config file in a default location. > > > > And if OpenSM ignores policy file, what would it mean? > > Would it be the "old" QoS? > > Yes. Is there another useful options? > > > And when the setup part of the new QoS will be ready, would we still want > > the SL2VL and VLArb tables to appear in the opts file? > > Don't think. Why we need it in two places? Are you saying that the existing QoS will get these tables from the new QoS syntax (and the existing syntax eliminated) ? If so, what about backward compatibility ? Also, how straightforward is it to specify the equivalent as the existing syntax ? Will a conversion tool be supplied ? -- Hal > Sasha > From hal.rosenstock at gmail.com Thu Aug 30 04:04:26 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 07:04:26 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <46D61E13.4020004@dev.mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> Message-ID: On 8/29/07, Yevgeny Kliteynik wrote: > Hal Rosenstock wrote: > > On 8/29/07, Yevgeny Kliteynik wrote: > >> Hal Rosenstock wrote: > >>> On 8/29/07, Yevgeny Kliteynik wrote: > >>>> Hal Rosenstock wrote: > >>>>> On 8/27/07, Yevgeny Kliteynik wrote: > >>>>>> Adding QoS policy file option to OpenSM optionsm and > >>>>>> QoS Policy field to subn object. > >>>>>> > >>>>>> Signed-off-by: Yevgeny Kliteynik > >>>>>> --- > >>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > >>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > >>>>>> opensm/opensm/main.c | 12 +++++++++++- > >>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- > >>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) > >>>>>> > >>>>>> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h > >>>>>> index 545779b..0a35e22 100644 > >>>>>> --- a/opensm/include/opensm/osm_base.h > >>>>>> +++ b/opensm/include/opensm/osm_base.h > >>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS > >>>>>> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" > >>>>>> #endif > >>>>>> /***********/ > >>>>>> + > >>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > >>>>>> +* NAME > >>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE > >>>>>> +* > >>>>>> +* DESCRIPTION > >>>>>> +* Specifies the default QoS policy file name > >>>>>> +* > >>>>>> +* SYNOPSIS > >>>>>> +*/ > >>>>>> +#ifdef __WIN__ > >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") > >>>>>> +#else > >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" > >>>>>> +#endif > >>>>>> +/***********/ > >>>>>> + > >>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>>>> * NAME > >>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > >>>>>> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h > >>>>>> index a7543dc..5e8b522 100644 > >>>>>> --- a/opensm/include/opensm/osm_subnet.h > >>>>>> +++ b/opensm/include/opensm/osm_subnet.h > >>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > >>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > >>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 > >>>>>> struct _osm_opensm_t; > >>>>>> +struct _osm_qos_policy_t; > >>>>>> > >>>>>> /****h* OpenSM/Subnet > >>>>>> * NAME > >>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > >>>>>> char *partition_config_file; > >>>>>> boolean_t no_partition_enforcement; > >>>>>> boolean_t no_qos; > >>>>>> + char *qos_policy_file; > >>>>>> boolean_t accum_log_file; > >>>>>> char *console; > >>>>>> uint16_t console_port; > >>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > >>>>>> * specified the log file will be truncated upon reaching > >>>>>> * this limit. > >>>>>> * > >>>>>> +* no_qos > >>>>>> +* Boolean that specifies whether the OpenSM QoS functionality > >>>>>> +* should be off or on. > >>>>>> +* > >>>>>> +* qos_policy_file > >>>>>> +* Name of the QoS policy file. > >>>>>> +* > >>>>> Are these options mutually exclusive or not ? > >>>> If QoS in OpenSM is off, the qos_policy_file will be ignored > >>> But aren't there two QoSs ? Can't the old QoS be run without this ? I > >>> think that is a requirement. > >> OK, there are three ways we want this thing to work: > >> 1. QoS is off > >> 2. The old QoS is on but w/o policy file > >> 3. The old QoS is on, plus reading policy file > >> > >> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. > >> > >> Second and third options: if QoS is on, OpenSM looks for policy file in the default > >> location or in other location that was provided by user. If the file is not found, > >> QoS works as before. > > > > This sounds OK to me and is my first preference. > > > >> Do we want to add additional option for "enhanced" QoS? > >> If so, we will have three QoS-ralated command line options: > >> - option for turning the QoS on (currently -Q) > >> - option to turn the new QoS on (some new letter - must get > >> one quick before they all run out... :) > >> - option for policy file location if differs from default (currently -Y) > > > > This seems like the least preferable to me. Also, would need to deal > > with both on which seems to mean use new QoS. > > > >> Alternatively, we can turn -Q option into levels: > >> -Q 0: QoS is off (default) > >> -Q 1: old QoS is on > >> -Q 2: old QoS plus reading policy file > > > > This one also seems OK to me (second preference). > > > > Anyone else with an opinion on this ? Sasha ? > > > > Also, what about --enable-qos-policy build option for those who don't > > want to include this in their build ? > > Do you mean permanently, or just until the new QoS will be stable? > Note that if someone gets ofed rpm, he gets the generated files in this > rpm, so he doesn't need les or yacc, so this option would be useful only > for machines that have git but don't have lex or yacc. Didn't realize this. How are the use of flex and bison dealt with in Windows ? > Do you think it's a real issue? Aside from the ongoing issues, there's always the OpenSM size issue as Sasha mentioned which is an issue for some. This issue is larger than just this feature. -- Hal > -- Yevgeny > > > -- Hal > > > >> -- Yevgeny > >> > >>>>>> * accum_log_file > >>>>>> * If TRUE (default) - the log file will be accumulated. > >>>>>> * If FALSE - the log file will be erased before starting current opensm run. > >>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > >>>>>> ib_net64_t sm_port_guid; > >>>>>> uint8_t sm_state; > >>>>>> osm_subn_opt_t opt; > >>>>>> + struct _osm_qos_policy_t *p_qos_policy; > >>>>>> uint16_t max_unicast_lid_ho; > >>>>>> uint16_t max_multicast_lid_ho; > >>>>>> uint8_t min_ca_mtu; > >>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > >>>>>> * opt > >>>>>> * Subnet options structure contains site specific configuration. > >>>>>> * > >>>>>> +* p_qos_policy > >>>>>> +* Subnet QoS policy structure. > >>>>>> +* > >>>>>> * max_unicast_lid_ho > >>>>>> * The minimal max unicast lid reported by all switches > >>>>>> * > >>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > >>>>>> index e2541d8..599c4dc 100644 > >>>>>> --- a/opensm/opensm/main.c > >>>>>> +++ b/opensm/opensm/main.c > >>>>>> @@ -269,6 +269,10 @@ void show_usage(void) > >>>>>> " The default name is \'" > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > >>>>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); > >>>>>> + printf("-Y\n" > >>>>>> + "--qos_policy_file\n" > >>>>>> + " This option defines the optional QoS policy file.\n" > >>>>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > >>>>>> printf("-N\n" > >>>>>> "--no_part_enforce\n" > >>>>>> " This option disables partition enforcement on switch external ports.\n\n"); > >>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > >>>>>> char *ignore_guids_file_name = NULL; > >>>>>> uint32_t val; > >>>>>> const char *const short_option = > >>>>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > >>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > >>>>>> > >>>>>> /* > >>>>>> In the array below, the 2nd parameter specifies the number > >>>>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > >>>>>> {"Pconfig", 1, NULL, 'P'}, > >>>>>> {"no_part_enforce", 0, NULL, 'N'}, > >>>>>> {"qos", 0, NULL, 'Q'}, > >>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, > >>>>>> {"maxsmps", 1, NULL, 'n'}, > >>>>>> {"console", 1, NULL, 'q'}, > >>>>>> {"V", 0, NULL, 'V'}, > >>>>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) > >>>>>> opt.no_qos = FALSE; > >>>>>> break; > >>>>>> > >>>>>> + case 'Y': > >>>>>> + opt.qos_policy_file = optarg; > >>>>>> + printf(" QoS policy file \'%s\'\n", optarg); > >>>>>> + break; > >>>>>> + > >>>>> There should also be an update to the OpenSM man page for this. > >>>> That and a text file with description of policy file syntax > >>> Good. Thanks. > >>> > >>> -- Hal > >>> > >>>> -- Yevgeny > >>>> > >>>>> -- Hal > >>>>> > >>>>>> case 'y': > >>>>>> opt.exit_on_fatal = FALSE; > >>>>>> printf(" Staying on fatal initialization errors\n"); > >>>>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c > >>>>>> index 818d73f..4162522 100644 > >>>>>> --- a/opensm/opensm/osm_subnet.c > >>>>>> +++ b/opensm/opensm/osm_subnet.c > >>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) > >>>>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; > >>>>>> p_opt->no_partition_enforcement = FALSE; > >>>>>> p_opt->no_qos = TRUE; > >>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > >>>>>> p_opt->accum_log_file = TRUE; > >>>>>> p_opt->port_profile_switch_nodes = FALSE; > >>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; > >>>>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) > >>>>>> > >>>>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); > >>>>>> > >>>>>> + opts_unpack_charp("qos_policy_file", > >>>>>> + p_key, p_val, &p_opts->qos_policy_file); > >>>>>> + > >>>>>> opts_unpack_boolean("accum_log_file", > >>>>>> p_key, p_val, &p_opts->accum_log_file); > >>>>>> > >>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > >>>>>> fprintf(opts_file, > >>>>>> "#\n# QoS OPTIONS\n#\n" > >>>>>> "# Disable QoS setup\n" > >>>>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); > >>>>>> + "no_qos %s\n\n" > >>>>>> + "# QoS policy file to be used\n" > >>>>>> + "qos_policy_file %s\n\n", > >>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", > >>>>>> + p_opts->qos_policy_file); > >>>>>> > >>>>>> subn_dump_qos_options(opts_file, > >>>>>> "QoS default options", "qos", > >>>>>> -- > >>>>>> 1.5.1.4 > >>>>>> > >>>>>> _______________________________________________ > >>>>>> general mailing list > >>>>>> general at lists.openfabrics.org > >>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >>>>>> > >>>>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >>>>>> > >> > > > > From hal.rosenstock at gmail.com Thu Aug 30 04:07:11 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 07:07:11 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> Message-ID: On 8/30/07, Eitan Zahavi wrote: > The standard option that is used by auto tools to have dependencies that > are only used by a maintainer is: > --enable-maintainer-mode > > We use it to hide .l/.y/.i dependencies from the Makefile such that only > those who declare themselves maintainers will refresh automatically > generated source files. I'm not sure this is being done currently nor are all the files needed for this approach checked into the repository. -- Hal > Why do you want to invent a new one? > > Eitan Zahavi > Senior Engineering Director, Software Architect > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > > Yevgeny Kliteynik > > Sent: Thursday, August 30, 2007 4:32 AM > > To: Hal Rosenstock > > Cc: OpenIB > > Subject: Re: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding > > QoS policy options > > > > Hal Rosenstock wrote: > > > On 8/29/07, Yevgeny Kliteynik wrote: > > >> Hal Rosenstock wrote: > > >>> On 8/29/07, Yevgeny Kliteynik wrote: > > >>>> Hal Rosenstock wrote: > > >>>>> On 8/27/07, Yevgeny Kliteynik > > wrote: > > >>>>>> Adding QoS policy file option to OpenSM optionsm and > > QoS Policy > > >>>>>> field to subn object. > > >>>>>> > > >>>>>> Signed-off-by: Yevgeny Kliteynik > > >>>>>> --- > > >>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ > > >>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > > >>>>>> opensm/opensm/main.c | 12 +++++++++++- > > >>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- > > >>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) > > >>>>>> > > >>>>>> diff --git a/opensm/include/opensm/osm_base.h > > >>>>>> b/opensm/include/opensm/osm_base.h > > >>>>>> index 545779b..0a35e22 100644 > > >>>>>> --- a/opensm/include/opensm/osm_base.h > > >>>>>> +++ b/opensm/include/opensm/osm_base.h > > >>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define > > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE > > "/etc/ofa/opensm-partitions.conf" > > >>>>>> #endif > > >>>>>> /***********/ > > >>>>>> + > > >>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > > >>>>>> +* NAME > > >>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE > > >>>>>> +* > > >>>>>> +* DESCRIPTION > > >>>>>> +* Specifies the default QoS policy file name > > >>>>>> +* > > >>>>>> +* SYNOPSIS > > >>>>>> +*/ > > >>>>>> +#ifdef __WIN__ > > >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), > > >>>>>> +"osm-qos-policy.conf") #else #define > > OSM_DEFAULT_QOS_POLICY_FILE > > >>>>>> +"/etc/ofa/opensm-qos-policy.conf" > > >>>>>> +#endif > > >>>>>> +/***********/ > > >>>>>> + > > >>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > > >>>>>> * NAME > > >>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > > >>>>>> diff --git a/opensm/include/opensm/osm_subnet.h > > >>>>>> b/opensm/include/opensm/osm_subnet.h > > >>>>>> index a7543dc..5e8b522 100644 > > >>>>>> --- a/opensm/include/opensm/osm_subnet.h > > >>>>>> +++ b/opensm/include/opensm/osm_subnet.h > > >>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > > >>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > > >>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 > > >>>>>> struct _osm_opensm_t; > > >>>>>> +struct _osm_qos_policy_t; > > >>>>>> > > >>>>>> /****h* OpenSM/Subnet > > >>>>>> * NAME > > >>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > > >>>>>> char *partition_config_file; > > >>>>>> boolean_t no_partition_enforcement; > > >>>>>> boolean_t no_qos; > > >>>>>> + char *qos_policy_file; > > >>>>>> boolean_t accum_log_file; > > >>>>>> char *console; > > >>>>>> uint16_t console_port; > > >>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { > > >>>>>> * specified the log file will be > > truncated upon reaching > > >>>>>> * this limit. > > >>>>>> * > > >>>>>> +* no_qos > > >>>>>> +* Boolean that specifies whether the > > OpenSM QoS functionality > > >>>>>> +* should be off or on. > > >>>>>> +* > > >>>>>> +* qos_policy_file > > >>>>>> +* Name of the QoS policy file. > > >>>>>> +* > > >>>>> Are these options mutually exclusive or not ? > > >>>> If QoS in OpenSM is off, the qos_policy_file will be ignored > > >>> But aren't there two QoSs ? Can't the old QoS be run > > without this ? > > >>> I think that is a requirement. > > >> OK, there are three ways we want this thing to work: > > >> 1. QoS is off > > >> 2. The old QoS is on but w/o policy file 3. The old QoS is > > on, plus > > >> reading policy file > > >> > > >> The first option is clear: if a user doesn't turns QoS on > > (-Q), QoS is off as before. > > >> > > >> Second and third options: if QoS is on, OpenSM looks for > > policy file > > >> in the default location or in other location that was provided by > > >> user. If the file is not found, QoS works as before. > > > > > > This sounds OK to me and is my first preference. > > > > > >> Do we want to add additional option for "enhanced" QoS? > > >> If so, we will have three QoS-ralated command line options: > > >> - option for turning the QoS on (currently -Q) > > >> - option to turn the new QoS on (some new letter - must get > > >> one quick before they all run out... :) > > >> - option for policy file location if differs from default > > (currently > > >> -Y) > > > > > > This seems like the least preferable to me. Also, would > > need to deal > > > with both on which seems to mean use new QoS. > > > > > >> Alternatively, we can turn -Q option into levels: > > >> -Q 0: QoS is off (default) > > >> -Q 1: old QoS is on > > >> -Q 2: old QoS plus reading policy file > > > > > > This one also seems OK to me (second preference). > > > > > > Anyone else with an opinion on this ? Sasha ? > > > > > > Also, what about --enable-qos-policy build option for those > > who don't > > > want to include this in their build ? > > > > Do you mean permanently, or just until the new QoS will be stable? > > Note that if someone gets ofed rpm, he gets the generated > > files in this rpm, so he doesn't need les or yacc, so this > > option would be useful only for machines that have git but > > don't have lex or yacc. > > Do you think it's a real issue? > > > > -- Yevgeny > > > > > -- Hal > > > > > >> -- Yevgeny > > >> > > >>>>>> * accum_log_file > > >>>>>> * If TRUE (default) - the log file will > > be accumulated. > > >>>>>> * If FALSE - the log file will be erased > > before starting current opensm run. > > >>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > > >>>>>> ib_net64_t sm_port_guid; > > >>>>>> uint8_t sm_state; > > >>>>>> osm_subn_opt_t opt; > > >>>>>> + struct _osm_qos_policy_t *p_qos_policy; > > >>>>>> uint16_t max_unicast_lid_ho; > > >>>>>> uint16_t max_multicast_lid_ho; > > >>>>>> uint8_t min_ca_mtu; > > >>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > > >>>>>> * opt > > >>>>>> * Subnet options structure contains site > > specific configuration. > > >>>>>> * > > >>>>>> +* p_qos_policy > > >>>>>> +* Subnet QoS policy structure. > > >>>>>> +* > > >>>>>> * max_unicast_lid_ho > > >>>>>> * The minimal max unicast lid reported > > by all switches > > >>>>>> * > > >>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index > > >>>>>> e2541d8..599c4dc 100644 > > >>>>>> --- a/opensm/opensm/main.c > > >>>>>> +++ b/opensm/opensm/main.c > > >>>>>> @@ -269,6 +269,10 @@ void show_usage(void) > > >>>>>> " The default name is \'" > > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > > >>>>>> printf("-Q\n" "--qos\n" " This option > > enables QoS setup.\n\n"); > > >>>>>> + printf("-Y\n" > > >>>>>> + "--qos_policy_file\n" > > >>>>>> + " This option defines the > > optional QoS policy file.\n" > > >>>>>> + " The default name is \'" > > OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > > >>>>>> printf("-N\n" > > >>>>>> "--no_part_enforce\n" > > >>>>>> " This option disables > > partition enforcement on switch external ports.\n\n"); > > >>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > > >>>>>> char *ignore_guids_file_name = NULL; > > >>>>>> uint32_t val; > > >>>>>> const char *const short_option = > > >>>>>> - > > "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > > >>>>>> + > > >>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > >>>>>> > > >>>>>> /* > > >>>>>> In the array below, the 2nd parameter specifies the > > >>>>>> number @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > > >>>>>> {"Pconfig", 1, NULL, 'P'}, > > >>>>>> {"no_part_enforce", 0, NULL, 'N'}, > > >>>>>> {"qos", 0, NULL, 'Q'}, > > >>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, > > >>>>>> {"maxsmps", 1, NULL, 'n'}, > > >>>>>> {"console", 1, NULL, 'q'}, > > >>>>>> {"V", 0, NULL, 'V'}, @@ -823,6 +828,11 @@ int > > >>>>>> main(int argc, char *argv[]) > > >>>>>> opt.no_qos = FALSE; > > >>>>>> break; > > >>>>>> > > >>>>>> + case 'Y': > > >>>>>> + opt.qos_policy_file = optarg; > > >>>>>> + printf(" QoS policy file > > \'%s\'\n", optarg); > > >>>>>> + break; > > >>>>>> + > > >>>>> There should also be an update to the OpenSM man page for this. > > >>>> That and a text file with description of policy file syntax > > >>> Good. Thanks. > > >>> > > >>> -- Hal > > >>> > > >>>> -- Yevgeny > > >>>> > > >>>>> -- Hal > > >>>>> > > >>>>>> case 'y': > > >>>>>> opt.exit_on_fatal = FALSE; > > >>>>>> printf(" Staying on fatal > > initialization > > >>>>>> errors\n"); diff --git a/opensm/opensm/osm_subnet.c > > >>>>>> b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 > > >>>>>> --- a/opensm/opensm/osm_subnet.c > > >>>>>> +++ b/opensm/opensm/osm_subnet.c > > >>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN > > osm_subn_opt_t * const p_opt) > > >>>>>> p_opt->partition_config_file = > > OSM_DEFAULT_PARTITION_CONFIG_FILE; > > >>>>>> p_opt->no_partition_enforcement = FALSE; > > >>>>>> p_opt->no_qos = TRUE; > > >>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; > > >>>>>> p_opt->accum_log_file = TRUE; > > >>>>>> p_opt->port_profile_switch_nodes = FALSE; > > >>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 > > +1179,9 @@ > > >>>>>> ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * > > >>>>>> const p_opts) > > >>>>>> > > >>>>>> opts_unpack_boolean("no_qos", p_key, p_val, > > >>>>>> &p_opts->no_qos); > > >>>>>> > > >>>>>> + opts_unpack_charp("qos_policy_file", > > >>>>>> + p_key, p_val, > > >>>>>> + &p_opts->qos_policy_file); > > >>>>>> + > > >>>>>> opts_unpack_boolean("accum_log_file", > > >>>>>> p_key, p_val, > > >>>>>> &p_opts->accum_log_file); > > >>>>>> > > >>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t > > osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > > >>>>>> fprintf(opts_file, > > >>>>>> "#\n# QoS OPTIONS\n#\n" > > >>>>>> "# Disable QoS setup\n" > > >>>>>> - "no_qos %s\n\n", p_opts->no_qos ? > > "TRUE" : "FALSE"); > > >>>>>> + "no_qos %s\n\n" > > >>>>>> + "# QoS policy file to be used\n" > > >>>>>> + "qos_policy_file %s\n\n", > > >>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", > > >>>>>> + p_opts->qos_policy_file); > > >>>>>> > > >>>>>> subn_dump_qos_options(opts_file, > > >>>>>> "QoS default options", "qos", > > >>>>>> -- > > >>>>>> 1.5.1.4 > > >>>>>> > > >>>>>> _______________________________________________ > > >>>>>> general mailing list > > >>>>>> general at lists.openfabrics.org > > >>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > >>>>>> > > >>>>>> To unsubscribe, please visit > > >>>>>> http://openib.org/mailman/listinfo/openib-general > > >>>>>> > > >> > > > > > > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > From kliteyn at dev.mellanox.co.il Thu Aug 30 04:15:59 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 14:15:59 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> Message-ID: <46D6A6EF.6030509@dev.mellanox.co.il> Hal Rosenstock wrote: > On 8/29/07, Yevgeny Kliteynik wrote: >> Hal Rosenstock wrote: >>> On 8/29/07, Yevgeny Kliteynik wrote: >>>> Hal Rosenstock wrote: >>>>> On 8/29/07, Yevgeny Kliteynik wrote: >>>>>> Hal Rosenstock wrote: >>>>>>> On 8/27/07, Yevgeny Kliteynik wrote: >>>>>>>> Adding QoS policy file option to OpenSM optionsm and >>>>>>>> QoS Policy field to subn object. >>>>>>>> >>>>>>>> Signed-off-by: Yevgeny Kliteynik >>>>>>>> --- >>>>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ >>>>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ >>>>>>>> opensm/opensm/main.c | 12 +++++++++++- >>>>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- >>>>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) >>>>>>>> >>>>>>>> diff --git a/opensm/include/opensm/osm_base.h b/opensm/include/opensm/osm_base.h >>>>>>>> index 545779b..0a35e22 100644 >>>>>>>> --- a/opensm/include/opensm/osm_base.h >>>>>>>> +++ b/opensm/include/opensm/osm_base.h >>>>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS >>>>>>>> #define OSM_DEFAULT_PARTITION_CONFIG_FILE "/etc/ofa/opensm-partitions.conf" >>>>>>>> #endif >>>>>>>> /***********/ >>>>>>>> + >>>>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE >>>>>>>> +* NAME >>>>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE >>>>>>>> +* >>>>>>>> +* DESCRIPTION >>>>>>>> +* Specifies the default QoS policy file name >>>>>>>> +* >>>>>>>> +* SYNOPSIS >>>>>>>> +*/ >>>>>>>> +#ifdef __WIN__ >>>>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), "osm-qos-policy.conf") >>>>>>>> +#else >>>>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE "/etc/ofa/opensm-qos-policy.conf" >>>>>>>> +#endif >>>>>>>> +/***********/ >>>>>>>> + >>>>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>>>> * NAME >>>>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>>>> diff --git a/opensm/include/opensm/osm_subnet.h b/opensm/include/opensm/osm_subnet.h >>>>>>>> index a7543dc..5e8b522 100644 >>>>>>>> --- a/opensm/include/opensm/osm_subnet.h >>>>>>>> +++ b/opensm/include/opensm/osm_subnet.h >>>>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS >>>>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 >>>>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 >>>>>>>> struct _osm_opensm_t; >>>>>>>> +struct _osm_qos_policy_t; >>>>>>>> >>>>>>>> /****h* OpenSM/Subnet >>>>>>>> * NAME >>>>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { >>>>>>>> char *partition_config_file; >>>>>>>> boolean_t no_partition_enforcement; >>>>>>>> boolean_t no_qos; >>>>>>>> + char *qos_policy_file; >>>>>>>> boolean_t accum_log_file; >>>>>>>> char *console; >>>>>>>> uint16_t console_port; >>>>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { >>>>>>>> * specified the log file will be truncated upon reaching >>>>>>>> * this limit. >>>>>>>> * >>>>>>>> +* no_qos >>>>>>>> +* Boolean that specifies whether the OpenSM QoS functionality >>>>>>>> +* should be off or on. >>>>>>>> +* >>>>>>>> +* qos_policy_file >>>>>>>> +* Name of the QoS policy file. >>>>>>>> +* >>>>>>> Are these options mutually exclusive or not ? >>>>>> If QoS in OpenSM is off, the qos_policy_file will be ignored >>>>> But aren't there two QoSs ? Can't the old QoS be run without this ? I >>>>> think that is a requirement. >>>> OK, there are three ways we want this thing to work: >>>> 1. QoS is off >>>> 2. The old QoS is on but w/o policy file >>>> 3. The old QoS is on, plus reading policy file >>>> >>>> The first option is clear: if a user doesn't turns QoS on (-Q), QoS is off as before. >>>> >>>> Second and third options: if QoS is on, OpenSM looks for policy file in the default >>>> location or in other location that was provided by user. If the file is not found, >>>> QoS works as before. >>> This sounds OK to me and is my first preference. >>> >>>> Do we want to add additional option for "enhanced" QoS? >>>> If so, we will have three QoS-ralated command line options: >>>> - option for turning the QoS on (currently -Q) >>>> - option to turn the new QoS on (some new letter - must get >>>> one quick before they all run out... :) >>>> - option for policy file location if differs from default (currently -Y) >>> This seems like the least preferable to me. Also, would need to deal >>> with both on which seems to mean use new QoS. >>> >>>> Alternatively, we can turn -Q option into levels: >>>> -Q 0: QoS is off (default) >>>> -Q 1: old QoS is on >>>> -Q 2: old QoS plus reading policy file >>> This one also seems OK to me (second preference). >>> >>> Anyone else with an opinion on this ? Sasha ? >>> >>> Also, what about --enable-qos-policy build option for those who don't >>> want to include this in their build ? >> Do you mean permanently, or just until the new QoS will be stable? >> Note that if someone gets ofed rpm, he gets the generated files in this >> rpm, so he doesn't need les or yacc, so this option would be useful only >> for machines that have git but don't have lex or yacc. > > Didn't realize this. > > How are the use of flex and bison dealt with in Windows ? They're not. Right now trunk OpenSM is not converted to windows, so when I get back to this lin2win synchronization I'll deal with this (and other) problems. There's always a quick (and ugly) solution - adding generated files to windows sources, but I'll check the alternatives when I get back to this. -- Yevgeny >> Do you think it's a real issue? > > Aside from the ongoing issues, there's always the OpenSM size issue as > Sasha mentioned which is an issue for some. This issue is larger than > just this feature. > > -- Hal > >> -- Yevgeny >> >>> -- Hal >>> >>>> -- Yevgeny >>>> >>>>>>>> * accum_log_file >>>>>>>> * If TRUE (default) - the log file will be accumulated. >>>>>>>> * If FALSE - the log file will be erased before starting current opensm run. >>>>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { >>>>>>>> ib_net64_t sm_port_guid; >>>>>>>> uint8_t sm_state; >>>>>>>> osm_subn_opt_t opt; >>>>>>>> + struct _osm_qos_policy_t *p_qos_policy; >>>>>>>> uint16_t max_unicast_lid_ho; >>>>>>>> uint16_t max_multicast_lid_ho; >>>>>>>> uint8_t min_ca_mtu; >>>>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { >>>>>>>> * opt >>>>>>>> * Subnet options structure contains site specific configuration. >>>>>>>> * >>>>>>>> +* p_qos_policy >>>>>>>> +* Subnet QoS policy structure. >>>>>>>> +* >>>>>>>> * max_unicast_lid_ho >>>>>>>> * The minimal max unicast lid reported by all switches >>>>>>>> * >>>>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c >>>>>>>> index e2541d8..599c4dc 100644 >>>>>>>> --- a/opensm/opensm/main.c >>>>>>>> +++ b/opensm/opensm/main.c >>>>>>>> @@ -269,6 +269,10 @@ void show_usage(void) >>>>>>>> " The default name is \'" >>>>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); >>>>>>>> printf("-Q\n" "--qos\n" " This option enables QoS setup.\n\n"); >>>>>>>> + printf("-Y\n" >>>>>>>> + "--qos_policy_file\n" >>>>>>>> + " This option defines the optional QoS policy file.\n" >>>>>>>> + " The default name is \'" OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); >>>>>>>> printf("-N\n" >>>>>>>> "--no_part_enforce\n" >>>>>>>> " This option disables partition enforcement on switch external ports.\n\n"); >>>>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) >>>>>>>> char *ignore_guids_file_name = NULL; >>>>>>>> uint32_t val; >>>>>>>> const char *const short_option = >>>>>>>> - "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; >>>>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; >>>>>>>> >>>>>>>> /* >>>>>>>> In the array below, the 2nd parameter specifies the number >>>>>>>> @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) >>>>>>>> {"Pconfig", 1, NULL, 'P'}, >>>>>>>> {"no_part_enforce", 0, NULL, 'N'}, >>>>>>>> {"qos", 0, NULL, 'Q'}, >>>>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, >>>>>>>> {"maxsmps", 1, NULL, 'n'}, >>>>>>>> {"console", 1, NULL, 'q'}, >>>>>>>> {"V", 0, NULL, 'V'}, >>>>>>>> @@ -823,6 +828,11 @@ int main(int argc, char *argv[]) >>>>>>>> opt.no_qos = FALSE; >>>>>>>> break; >>>>>>>> >>>>>>>> + case 'Y': >>>>>>>> + opt.qos_policy_file = optarg; >>>>>>>> + printf(" QoS policy file \'%s\'\n", optarg); >>>>>>>> + break; >>>>>>>> + >>>>>>> There should also be an update to the OpenSM man page for this. >>>>>> That and a text file with description of policy file syntax >>>>> Good. Thanks. >>>>> >>>>> -- Hal >>>>> >>>>>> -- Yevgeny >>>>>> >>>>>>> -- Hal >>>>>>> >>>>>>>> case 'y': >>>>>>>> opt.exit_on_fatal = FALSE; >>>>>>>> printf(" Staying on fatal initialization errors\n"); >>>>>>>> diff --git a/opensm/opensm/osm_subnet.c b/opensm/opensm/osm_subnet.c >>>>>>>> index 818d73f..4162522 100644 >>>>>>>> --- a/opensm/opensm/osm_subnet.c >>>>>>>> +++ b/opensm/opensm/osm_subnet.c >>>>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * const p_opt) >>>>>>>> p_opt->partition_config_file = OSM_DEFAULT_PARTITION_CONFIG_FILE; >>>>>>>> p_opt->no_partition_enforcement = FALSE; >>>>>>>> p_opt->no_qos = TRUE; >>>>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; >>>>>>>> p_opt->accum_log_file = TRUE; >>>>>>>> p_opt->port_profile_switch_nodes = FALSE; >>>>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; >>>>>>>> @@ -1178,6 +1179,9 @@ ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * const p_opts) >>>>>>>> >>>>>>>> opts_unpack_boolean("no_qos", p_key, p_val, &p_opts->no_qos); >>>>>>>> >>>>>>>> + opts_unpack_charp("qos_policy_file", >>>>>>>> + p_key, p_val, &p_opts->qos_policy_file); >>>>>>>> + >>>>>>>> opts_unpack_boolean("accum_log_file", >>>>>>>> p_key, p_val, &p_opts->accum_log_file); >>>>>>>> >>>>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) >>>>>>>> fprintf(opts_file, >>>>>>>> "#\n# QoS OPTIONS\n#\n" >>>>>>>> "# Disable QoS setup\n" >>>>>>>> - "no_qos %s\n\n", p_opts->no_qos ? "TRUE" : "FALSE"); >>>>>>>> + "no_qos %s\n\n" >>>>>>>> + "# QoS policy file to be used\n" >>>>>>>> + "qos_policy_file %s\n\n", >>>>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", >>>>>>>> + p_opts->qos_policy_file); >>>>>>>> >>>>>>>> subn_dump_qos_options(opts_file, >>>>>>>> "QoS default options", "qos", >>>>>>>> -- >>>>>>>> 1.5.1.4 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> general mailing list >>>>>>>> general at lists.openfabrics.org >>>>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>>>>> >>>>>>>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >>>>>>>> >> > From kliteyn at dev.mellanox.co.il Thu Aug 30 04:25:52 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 14:25:52 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> Message-ID: <46D6A940.5050006@dev.mellanox.co.il> Eitan Zahavi wrote: > The standard option that is used by auto tools to have dependencies that > are only used by a maintainer is: > --enable-maintainer-mode > > We use it to hide .l/.y/.i dependencies from the Makefile such that only > those who declare themselves maintainers will refresh automatically > generated source files. We don't have generated files in the git, so if someone gets OpenSM from git repository, he *has* to generate those files when compiling OpenSM, which means that the .l/.y dependencies cannot be hidden behind maintainer mode. Another way to deal with this is to include generated files in the source tree, and then re-generating those files would be hidden behind --enable-maintainer-mode config option, and a user won't need lex/yacc at all to compile OpenSM. I suggested this option when I was working on QoS for OFED 1.2, but as I recall, the suggestion has met fierce resistance. -- Yevgeny > Why do you want to invent a new one? > > Eitan Zahavi > Senior Engineering Director, Software Architect > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > >> -----Original Message----- >> From: general-bounces at lists.openfabrics.org >> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of >> Yevgeny Kliteynik >> Sent: Thursday, August 30, 2007 4:32 AM >> To: Hal Rosenstock >> Cc: OpenIB >> Subject: Re: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding >> QoS policy options >> >> Hal Rosenstock wrote: >>> On 8/29/07, Yevgeny Kliteynik wrote: >>>> Hal Rosenstock wrote: >>>>> On 8/29/07, Yevgeny Kliteynik wrote: >>>>>> Hal Rosenstock wrote: >>>>>>> On 8/27/07, Yevgeny Kliteynik >> wrote: >>>>>>>> Adding QoS policy file option to OpenSM optionsm and >> QoS Policy >>>>>>>> field to subn object. >>>>>>>> >>>>>>>> Signed-off-by: Yevgeny Kliteynik >>>>>>>> --- >>>>>>>> opensm/include/opensm/osm_base.h | 17 +++++++++++++++++ >>>>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ >>>>>>>> opensm/opensm/main.c | 12 +++++++++++- >>>>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- >>>>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) >>>>>>>> >>>>>>>> diff --git a/opensm/include/opensm/osm_base.h >>>>>>>> b/opensm/include/opensm/osm_base.h >>>>>>>> index 545779b..0a35e22 100644 >>>>>>>> --- a/opensm/include/opensm/osm_base.h >>>>>>>> +++ b/opensm/include/opensm/osm_base.h >>>>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define >>>>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE >> "/etc/ofa/opensm-partitions.conf" >>>>>>>> #endif >>>>>>>> /***********/ >>>>>>>> + >>>>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE >>>>>>>> +* NAME >>>>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE >>>>>>>> +* >>>>>>>> +* DESCRIPTION >>>>>>>> +* Specifies the default QoS policy file name >>>>>>>> +* >>>>>>>> +* SYNOPSIS >>>>>>>> +*/ >>>>>>>> +#ifdef __WIN__ >>>>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE strcat(GetOsmCachePath(), >>>>>>>> +"osm-qos-policy.conf") #else #define >> OSM_DEFAULT_QOS_POLICY_FILE >>>>>>>> +"/etc/ofa/opensm-qos-policy.conf" >>>>>>>> +#endif >>>>>>>> +/***********/ >>>>>>>> + >>>>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>>>> * NAME >>>>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS >>>>>>>> diff --git a/opensm/include/opensm/osm_subnet.h >>>>>>>> b/opensm/include/opensm/osm_subnet.h >>>>>>>> index a7543dc..5e8b522 100644 >>>>>>>> --- a/opensm/include/opensm/osm_subnet.h >>>>>>>> +++ b/opensm/include/opensm/osm_subnet.h >>>>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS >>>>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 >>>>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 >>>>>>>> struct _osm_opensm_t; >>>>>>>> +struct _osm_qos_policy_t; >>>>>>>> >>>>>>>> /****h* OpenSM/Subnet >>>>>>>> * NAME >>>>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { >>>>>>>> char *partition_config_file; >>>>>>>> boolean_t no_partition_enforcement; >>>>>>>> boolean_t no_qos; >>>>>>>> + char *qos_policy_file; >>>>>>>> boolean_t accum_log_file; >>>>>>>> char *console; >>>>>>>> uint16_t console_port; >>>>>>>> @@ -398,6 +400,13 @@ typedef struct _osm_subn_opt { >>>>>>>> * specified the log file will be >> truncated upon reaching >>>>>>>> * this limit. >>>>>>>> * >>>>>>>> +* no_qos >>>>>>>> +* Boolean that specifies whether the >> OpenSM QoS functionality >>>>>>>> +* should be off or on. >>>>>>>> +* >>>>>>>> +* qos_policy_file >>>>>>>> +* Name of the QoS policy file. >>>>>>>> +* >>>>>>> Are these options mutually exclusive or not ? >>>>>> If QoS in OpenSM is off, the qos_policy_file will be ignored >>>>> But aren't there two QoSs ? Can't the old QoS be run >> without this ? >>>>> I think that is a requirement. >>>> OK, there are three ways we want this thing to work: >>>> 1. QoS is off >>>> 2. The old QoS is on but w/o policy file 3. The old QoS is >> on, plus >>>> reading policy file >>>> >>>> The first option is clear: if a user doesn't turns QoS on >> (-Q), QoS is off as before. >>>> Second and third options: if QoS is on, OpenSM looks for >> policy file >>>> in the default location or in other location that was provided by >>>> user. If the file is not found, QoS works as before. >>> This sounds OK to me and is my first preference. >>> >>>> Do we want to add additional option for "enhanced" QoS? >>>> If so, we will have three QoS-ralated command line options: >>>> - option for turning the QoS on (currently -Q) >>>> - option to turn the new QoS on (some new letter - must get >>>> one quick before they all run out... :) >>>> - option for policy file location if differs from default >> (currently >>>> -Y) >>> This seems like the least preferable to me. Also, would >> need to deal >>> with both on which seems to mean use new QoS. >>> >>>> Alternatively, we can turn -Q option into levels: >>>> -Q 0: QoS is off (default) >>>> -Q 1: old QoS is on >>>> -Q 2: old QoS plus reading policy file >>> This one also seems OK to me (second preference). >>> >>> Anyone else with an opinion on this ? Sasha ? >>> >>> Also, what about --enable-qos-policy build option for those >> who don't >>> want to include this in their build ? >> Do you mean permanently, or just until the new QoS will be stable? >> Note that if someone gets ofed rpm, he gets the generated >> files in this rpm, so he doesn't need les or yacc, so this >> option would be useful only for machines that have git but >> don't have lex or yacc. >> Do you think it's a real issue? >> >> -- Yevgeny >> >>> -- Hal >>> >>>> -- Yevgeny >>>> >>>>>>>> * accum_log_file >>>>>>>> * If TRUE (default) - the log file will >> be accumulated. >>>>>>>> * If FALSE - the log file will be erased >> before starting current opensm run. >>>>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { >>>>>>>> ib_net64_t sm_port_guid; >>>>>>>> uint8_t sm_state; >>>>>>>> osm_subn_opt_t opt; >>>>>>>> + struct _osm_qos_policy_t *p_qos_policy; >>>>>>>> uint16_t max_unicast_lid_ho; >>>>>>>> uint16_t max_multicast_lid_ho; >>>>>>>> uint8_t min_ca_mtu; >>>>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { >>>>>>>> * opt >>>>>>>> * Subnet options structure contains site >> specific configuration. >>>>>>>> * >>>>>>>> +* p_qos_policy >>>>>>>> +* Subnet QoS policy structure. >>>>>>>> +* >>>>>>>> * max_unicast_lid_ho >>>>>>>> * The minimal max unicast lid reported >> by all switches >>>>>>>> * >>>>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c index >>>>>>>> e2541d8..599c4dc 100644 >>>>>>>> --- a/opensm/opensm/main.c >>>>>>>> +++ b/opensm/opensm/main.c >>>>>>>> @@ -269,6 +269,10 @@ void show_usage(void) >>>>>>>> " The default name is \'" >>>>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); >>>>>>>> printf("-Q\n" "--qos\n" " This option >> enables QoS setup.\n\n"); >>>>>>>> + printf("-Y\n" >>>>>>>> + "--qos_policy_file\n" >>>>>>>> + " This option defines the >> optional QoS policy file.\n" >>>>>>>> + " The default name is \'" >> OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); >>>>>>>> printf("-N\n" >>>>>>>> "--no_part_enforce\n" >>>>>>>> " This option disables >> partition enforcement on switch external ports.\n\n"); >>>>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) >>>>>>>> char *ignore_guids_file_name = NULL; >>>>>>>> uint32_t val; >>>>>>>> const char *const short_option = >>>>>>>> - >> "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; >>>>>>>> + >>>>>>>> + "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; >>>>>>>> >>>>>>>> /* >>>>>>>> In the array below, the 2nd parameter specifies the >>>>>>>> number @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) >>>>>>>> {"Pconfig", 1, NULL, 'P'}, >>>>>>>> {"no_part_enforce", 0, NULL, 'N'}, >>>>>>>> {"qos", 0, NULL, 'Q'}, >>>>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, >>>>>>>> {"maxsmps", 1, NULL, 'n'}, >>>>>>>> {"console", 1, NULL, 'q'}, >>>>>>>> {"V", 0, NULL, 'V'}, @@ -823,6 +828,11 @@ int >>>>>>>> main(int argc, char *argv[]) >>>>>>>> opt.no_qos = FALSE; >>>>>>>> break; >>>>>>>> >>>>>>>> + case 'Y': >>>>>>>> + opt.qos_policy_file = optarg; >>>>>>>> + printf(" QoS policy file >> \'%s\'\n", optarg); >>>>>>>> + break; >>>>>>>> + >>>>>>> There should also be an update to the OpenSM man page for this. >>>>>> That and a text file with description of policy file syntax >>>>> Good. Thanks. >>>>> >>>>> -- Hal >>>>> >>>>>> -- Yevgeny >>>>>> >>>>>>> -- Hal >>>>>>> >>>>>>>> case 'y': >>>>>>>> opt.exit_on_fatal = FALSE; >>>>>>>> printf(" Staying on fatal >> initialization >>>>>>>> errors\n"); diff --git a/opensm/opensm/osm_subnet.c >>>>>>>> b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 >>>>>>>> --- a/opensm/opensm/osm_subnet.c >>>>>>>> +++ b/opensm/opensm/osm_subnet.c >>>>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN >> osm_subn_opt_t * const p_opt) >>>>>>>> p_opt->partition_config_file = >> OSM_DEFAULT_PARTITION_CONFIG_FILE; >>>>>>>> p_opt->no_partition_enforcement = FALSE; >>>>>>>> p_opt->no_qos = TRUE; >>>>>>>> + p_opt->qos_policy_file = OSM_DEFAULT_QOS_POLICY_FILE; >>>>>>>> p_opt->accum_log_file = TRUE; >>>>>>>> p_opt->port_profile_switch_nodes = FALSE; >>>>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 >> +1179,9 @@ >>>>>>>> ib_api_status_t osm_subn_parse_conf_file(IN osm_subn_opt_t * >>>>>>>> const p_opts) >>>>>>>> >>>>>>>> opts_unpack_boolean("no_qos", p_key, p_val, >>>>>>>> &p_opts->no_qos); >>>>>>>> >>>>>>>> + opts_unpack_charp("qos_policy_file", >>>>>>>> + p_key, p_val, >>>>>>>> + &p_opts->qos_policy_file); >>>>>>>> + >>>>>>>> opts_unpack_boolean("accum_log_file", >>>>>>>> p_key, p_val, >>>>>>>> &p_opts->accum_log_file); >>>>>>>> >>>>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t >> osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) >>>>>>>> fprintf(opts_file, >>>>>>>> "#\n# QoS OPTIONS\n#\n" >>>>>>>> "# Disable QoS setup\n" >>>>>>>> - "no_qos %s\n\n", p_opts->no_qos ? >> "TRUE" : "FALSE"); >>>>>>>> + "no_qos %s\n\n" >>>>>>>> + "# QoS policy file to be used\n" >>>>>>>> + "qos_policy_file %s\n\n", >>>>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", >>>>>>>> + p_opts->qos_policy_file); >>>>>>>> >>>>>>>> subn_dump_qos_options(opts_file, >>>>>>>> "QoS default options", "qos", >>>>>>>> -- >>>>>>>> 1.5.1.4 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> general mailing list >>>>>>>> general at lists.openfabrics.org >>>>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>>>>> >>>>>>>> To unsubscribe, please visit >>>>>>>> http://openib.org/mailman/listinfo/openib-general >>>>>>>> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > From kliteyn at dev.mellanox.co.il Thu Aug 30 04:31:13 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 14:31:13 +0300 Subject: [ofa-general] [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: References: <46D35A20.6020902@dev.mellanox.co.il> Message-ID: <46D6AA81.20100@dev.mellanox.co.il> Hal Rosenstock wrote: > On 8/27/07, Yevgeny Kliteynik wrote: >> Compiling QoS policy file parser. > > Should there be an build enable for this functionality (like > --enable-qos-policy) ? I expect that some will want to not include > this. By default, this should be disabled IMO. Do you mean that by default OpenSM should be configured and compiled w/o the new QoS? Why? Is it for reducing binary size? If so, is it really the most common case of using OpenSM? > BTW, what is the "level" of this QoS support for OFED 1.3 ? Is it > technology preview ? Yes, it's technology preview. -- Yevgeny > -- Hal > >> C & H files are generated from Lex & Yacc files and the >> generated files are compiled. >> Generated files are not included in git, but they do get >> included in RPM as they are also created by 'make dist'. >> Generated files can be removed by "make maintainer-clean". >> >> Signed-off-by: Yevgeny Kliteynik >> --- >> opensm/include/Makefile.am | 2 ++ >> opensm/opensm/Makefile.am | 16 +++++++++++++++- >> opensm/opensm/configure.in | 2 ++ >> 3 files changed, 19 insertions(+), 1 deletions(-) >> >> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am >> index ea62507..b83205f 100644 >> --- a/opensm/include/Makefile.am >> +++ b/opensm/include/Makefile.am >> @@ -90,6 +90,8 @@ EXTRA_DIST = \ >> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ >> $(srcdir)/opensm/osm_perfmgr.h \ >> $(srcdir)/opensm/osm_perfmgr_db.h \ >> + $(srcdir)/opensm/osm_qos_policy.h \ >> + $(srcdir)/opensm/osm_qos_parser_y.h \ >> $(srcdir)/complib/cl_thread_osd.h \ >> $(srcdir)/complib/cl_packon.h \ >> $(srcdir)/complib/cl_atomic_osd.h \ >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >> index 6dfa824..9d57f93 100644 >> --- a/opensm/opensm/Makefile.am >> +++ b/opensm/opensm/Makefile.am >> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ >> osm_vl15intf.c osm_vl_arb_rcv.c \ >> st.c osm_perfmgr.c osm_perfmgr_db.c \ >> - osm_event_plugin.c osm_dump.c >> + osm_event_plugin.c osm_dump.c \ >> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >> + >> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ >> + >> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l >> + >> if OSMV_OPENIB >> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 >> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 >> @@ -104,3 +112,9 @@ install-exec-hook: >> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ >> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ >> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so >> + >> +# generate c and h files from the lex and yacc files >> +dist-hook: $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c >> + >> +maintainer-clean-generic: >> + rm -f $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c >> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in >> index afb20b3..a49538d 100644 >> --- a/opensm/opensm/configure.in >> +++ b/opensm/opensm/configure.in >> @@ -22,6 +22,8 @@ AC_PROG_INSTALL >> AC_PROG_LN_S >> AC_PROG_MAKE_SET >> AC_PROG_LIBTOOL >> +AM_PROG_LEX >> +AC_PROG_YACC >> >> dnl Checks for libraries >> >> -- >> 1.5.1.4 >> >> _______________________________________________ >> general mailing list >> general at lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general >> > From mst at dev.mellanox.co.il Thu Aug 30 04:28:54 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 14:28:54 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> Message-ID: <20070830112854.GC1798@mellanox.co.il> > How are the use of flex and bison dealt with in Windows ? There are windows ports for these tools. Here are some links: http://www.mingw.org/MinGWiki/index.php/bison http://www.mingw.org/MinGWiki/index.php/flex -- MST From eitan at mellanox.co.il Thu Aug 30 04:30:58 2007 From: eitan at mellanox.co.il (Eitan Zahavi) Date: Thu, 30 Aug 2007 14:30:58 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <6C2C79E72C305246B504CBA17B5500C9022AC6BE@mtlexch01.mtl.com> Message-ID: <6C2C79E72C305246B504CBA17B5500C9022AC8BD@mtlexch01.mtl.com> > -----Original Message----- > From: Hal Rosenstock [mailto:hal.rosenstock at gmail.com] > Sent: Thursday, August 30, 2007 2:07 PM > To: Eitan Zahavi > Cc: Yevgeny Kliteynik; OpenIB > Subject: Re: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding > QoS policy options > > On 8/30/07, Eitan Zahavi wrote: > > The standard option that is used by auto tools to have dependencies > > that are only used by a maintainer is: > > --enable-maintainer-mode > > > > We use it to hide .l/.y/.i dependencies from the Makefile such that > > only those who declare themselves maintainers will refresh > > automatically generated source files. > > I'm not sure this is being done currently nor are all the > files needed for this approach checked into the repository. I think it is a mistake. But I will let you eat your own pudding... Eitan Zahavi > > -- Hal > > > Why do you want to invent a new one? > > > > Eitan Zahavi > > Senior Engineering Director, Software Architect Mellanox > Technologies > > LTD > > Tel:+972-4-9097208 > > Fax:+972-4-9593245 > > P.O. Box 586 Yokneam 20692 ISRAEL > > > > > > > > > -----Original Message----- > > > From: general-bounces at lists.openfabrics.org > > > [mailto:general-bounces at lists.openfabrics.org] On Behalf > Of Yevgeny > > > Kliteynik > > > Sent: Thursday, August 30, 2007 4:32 AM > > > To: Hal Rosenstock > > > Cc: OpenIB > > > Subject: Re: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS > > > policy options > > > > > > Hal Rosenstock wrote: > > > > On 8/29/07, Yevgeny Kliteynik > wrote: > > > >> Hal Rosenstock wrote: > > > >>> On 8/29/07, Yevgeny Kliteynik > wrote: > > > >>>> Hal Rosenstock wrote: > > > >>>>> On 8/27/07, Yevgeny Kliteynik > > > wrote: > > > >>>>>> Adding QoS policy file option to OpenSM optionsm and > > > QoS Policy > > > >>>>>> field to subn object. > > > >>>>>> > > > >>>>>> Signed-off-by: Yevgeny Kliteynik > > > > >>>>>> --- > > > >>>>>> opensm/include/opensm/osm_base.h | 17 > +++++++++++++++++ > > > >>>>>> opensm/include/opensm/osm_subnet.h | 13 +++++++++++++ > > > >>>>>> opensm/opensm/main.c | 12 +++++++++++- > > > >>>>>> opensm/opensm/osm_subnet.c | 10 +++++++++- > > > >>>>>> 4 files changed, 50 insertions(+), 2 deletions(-) > > > >>>>>> > > > >>>>>> diff --git a/opensm/include/opensm/osm_base.h > > > >>>>>> b/opensm/include/opensm/osm_base.h > > > >>>>>> index 545779b..0a35e22 100644 > > > >>>>>> --- a/opensm/include/opensm/osm_base.h > > > >>>>>> +++ b/opensm/include/opensm/osm_base.h > > > >>>>>> @@ -224,6 +224,23 @@ BEGIN_C_DECLS #define > > > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE > > > "/etc/ofa/opensm-partitions.conf" > > > >>>>>> #endif > > > >>>>>> /***********/ > > > >>>>>> + > > > >>>>>> +/****d* OpenSM: Base/OSM_DEFAULT_QOS_POLICY_FILE > > > >>>>>> +* NAME > > > >>>>>> +* OSM_DEFAULT_QOS_POLICY_FILE > > > >>>>>> +* > > > >>>>>> +* DESCRIPTION > > > >>>>>> +* Specifies the default QoS policy file name > > > >>>>>> +* > > > >>>>>> +* SYNOPSIS > > > >>>>>> +*/ > > > >>>>>> +#ifdef __WIN__ > > > >>>>>> +#define OSM_DEFAULT_QOS_POLICY_FILE > > > >>>>>> +strcat(GetOsmCachePath(), > > > >>>>>> +"osm-qos-policy.conf") #else #define > > > OSM_DEFAULT_QOS_POLICY_FILE > > > >>>>>> +"/etc/ofa/opensm-qos-policy.conf" > > > >>>>>> +#endif > > > >>>>>> +/***********/ > > > >>>>>> + > > > >>>>>> /****d* OpenSM: Base/OSM_DEFAULT_SWEEP_INTERVAL_SECS > > > >>>>>> * NAME > > > >>>>>> * OSM_DEFAULT_SWEEP_INTERVAL_SECS > > > >>>>>> diff --git a/opensm/include/opensm/osm_subnet.h > > > >>>>>> b/opensm/include/opensm/osm_subnet.h > > > >>>>>> index a7543dc..5e8b522 100644 > > > >>>>>> --- a/opensm/include/opensm/osm_subnet.h > > > >>>>>> +++ b/opensm/include/opensm/osm_subnet.h > > > >>>>>> @@ -68,6 +68,7 @@ BEGIN_C_DECLS > > > >>>>>> #define OSM_SUBNET_VECTOR_GROW_SIZE 1 > > > >>>>>> #define OSM_SUBNET_VECTOR_CAPACITY 256 > > > >>>>>> struct _osm_opensm_t; > > > >>>>>> +struct _osm_qos_policy_t; > > > >>>>>> > > > >>>>>> /****h* OpenSM/Subnet > > > >>>>>> * NAME > > > >>>>>> @@ -257,6 +258,7 @@ typedef struct _osm_subn_opt { > > > >>>>>> char *partition_config_file; > > > >>>>>> boolean_t no_partition_enforcement; > > > >>>>>> boolean_t no_qos; > > > >>>>>> + char *qos_policy_file; > > > >>>>>> boolean_t accum_log_file; > > > >>>>>> char *console; > > > >>>>>> uint16_t console_port; @@ -398,6 +400,13 @@ typedef > > > >>>>>> struct _osm_subn_opt { > > > >>>>>> * specified the log file will be > > > truncated upon reaching > > > >>>>>> * this limit. > > > >>>>>> * > > > >>>>>> +* no_qos > > > >>>>>> +* Boolean that specifies whether the > > > OpenSM QoS functionality > > > >>>>>> +* should be off or on. > > > >>>>>> +* > > > >>>>>> +* qos_policy_file > > > >>>>>> +* Name of the QoS policy file. > > > >>>>>> +* > > > >>>>> Are these options mutually exclusive or not ? > > > >>>> If QoS in OpenSM is off, the qos_policy_file will be ignored > > > >>> But aren't there two QoSs ? Can't the old QoS be run > > > without this ? > > > >>> I think that is a requirement. > > > >> OK, there are three ways we want this thing to work: > > > >> 1. QoS is off > > > >> 2. The old QoS is on but w/o policy file 3. The old QoS is > > > on, plus > > > >> reading policy file > > > >> > > > >> The first option is clear: if a user doesn't turns QoS on > > > (-Q), QoS is off as before. > > > >> > > > >> Second and third options: if QoS is on, OpenSM looks for > > > policy file > > > >> in the default location or in other location that was > provided by > > > >> user. If the file is not found, QoS works as before. > > > > > > > > This sounds OK to me and is my first preference. > > > > > > > >> Do we want to add additional option for "enhanced" QoS? > > > >> If so, we will have three QoS-ralated command line options: > > > >> - option for turning the QoS on (currently -Q) > > > >> - option to turn the new QoS on (some new letter - must get > > > >> one quick before they all run out... :) > > > >> - option for policy file location if differs from default > > > (currently > > > >> -Y) > > > > > > > > This seems like the least preferable to me. Also, would > > > need to deal > > > > with both on which seems to mean use new QoS. > > > > > > > >> Alternatively, we can turn -Q option into levels: > > > >> -Q 0: QoS is off (default) > > > >> -Q 1: old QoS is on > > > >> -Q 2: old QoS plus reading policy file > > > > > > > > This one also seems OK to me (second preference). > > > > > > > > Anyone else with an opinion on this ? Sasha ? > > > > > > > > Also, what about --enable-qos-policy build option for those > > > who don't > > > > want to include this in their build ? > > > > > > Do you mean permanently, or just until the new QoS will > be stable? > > > Note that if someone gets ofed rpm, he gets the generated > files in > > > this rpm, so he doesn't need les or yacc, so this option would be > > > useful only for machines that have git but don't have lex or yacc. > > > Do you think it's a real issue? > > > > > > -- Yevgeny > > > > > > > -- Hal > > > > > > > >> -- Yevgeny > > > >> > > > >>>>>> * accum_log_file > > > >>>>>> * If TRUE (default) - the log file will > > > be accumulated. > > > >>>>>> * If FALSE - the log file will be erased > > > before starting current opensm run. > > > >>>>>> @@ -551,6 +560,7 @@ typedef struct _osm_subn { > > > >>>>>> ib_net64_t sm_port_guid; > > > >>>>>> uint8_t sm_state; > > > >>>>>> osm_subn_opt_t opt; > > > >>>>>> + struct _osm_qos_policy_t *p_qos_policy; > > > >>>>>> uint16_t max_unicast_lid_ho; > > > >>>>>> uint16_t max_multicast_lid_ho; > > > >>>>>> uint8_t min_ca_mtu; > > > >>>>>> @@ -619,6 +629,9 @@ typedef struct _osm_subn { > > > >>>>>> * opt > > > >>>>>> * Subnet options structure contains site > > > specific configuration. > > > >>>>>> * > > > >>>>>> +* p_qos_policy > > > >>>>>> +* Subnet QoS policy structure. > > > >>>>>> +* > > > >>>>>> * max_unicast_lid_ho > > > >>>>>> * The minimal max unicast lid reported > > > by all switches > > > >>>>>> * > > > >>>>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c > > > >>>>>> index e2541d8..599c4dc 100644 > > > >>>>>> --- a/opensm/opensm/main.c > > > >>>>>> +++ b/opensm/opensm/main.c > > > >>>>>> @@ -269,6 +269,10 @@ void show_usage(void) > > > >>>>>> " The default name is \'" > > > >>>>>> OSM_DEFAULT_PARTITION_CONFIG_FILE "\'.\n\n"); > > > >>>>>> printf("-Q\n" "--qos\n" " This option > > > enables QoS setup.\n\n"); > > > >>>>>> + printf("-Y\n" > > > >>>>>> + "--qos_policy_file\n" > > > >>>>>> + " This option defines the > > > optional QoS policy file.\n" > > > >>>>>> + " The default name is \'" > > > OSM_DEFAULT_QOS_POLICY_FILE "\'.\n\n"); > > > >>>>>> printf("-N\n" > > > >>>>>> "--no_part_enforce\n" > > > >>>>>> " This option disables > > > partition enforcement on switch external ports.\n\n"); > > > >>>>>> @@ -580,7 +584,7 @@ int main(int argc, char *argv[]) > > > >>>>>> char *ignore_guids_file_name = NULL; > > > >>>>>> uint32_t val; > > > >>>>>> const char *const short_option = > > > >>>>>> - > > > "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:NBIQvVhorcyxp:n:q:k:C:"; > > > >>>>>> + > > > >>>>>> + > "i:f:ed:g:l:L:s:t:a:u:R:zM:U:S:P:Y:NBIQvVhorcyxp:n:q:k:C:"; > > > >>>>>> > > > >>>>>> /* > > > >>>>>> In the array below, the 2nd parameter > specifies the > > > >>>>>> number @@ -604,6 +608,7 @@ int main(int argc, char *argv[]) > > > >>>>>> {"Pconfig", 1, NULL, 'P'}, > > > >>>>>> {"no_part_enforce", 0, NULL, 'N'}, > > > >>>>>> {"qos", 0, NULL, 'Q'}, > > > >>>>>> + {"qos_policy_file", 1, NULL, 'Y'}, > > > >>>>>> {"maxsmps", 1, NULL, 'n'}, > > > >>>>>> {"console", 1, NULL, 'q'}, > > > >>>>>> {"V", 0, NULL, 'V'}, @@ -823,6 > +828,11 @@ int > > > >>>>>> main(int argc, char *argv[]) > > > >>>>>> opt.no_qos = FALSE; > > > >>>>>> break; > > > >>>>>> > > > >>>>>> + case 'Y': > > > >>>>>> + opt.qos_policy_file = optarg; > > > >>>>>> + printf(" QoS policy file > > > \'%s\'\n", optarg); > > > >>>>>> + break; > > > >>>>>> + > > > >>>>> There should also be an update to the OpenSM man > page for this. > > > >>>> That and a text file with description of policy file syntax > > > >>> Good. Thanks. > > > >>> > > > >>> -- Hal > > > >>> > > > >>>> -- Yevgeny > > > >>>> > > > >>>>> -- Hal > > > >>>>> > > > >>>>>> case 'y': > > > >>>>>> opt.exit_on_fatal = FALSE; > > > >>>>>> printf(" Staying on fatal > > > initialization > > > >>>>>> errors\n"); diff --git a/opensm/opensm/osm_subnet.c > > > >>>>>> b/opensm/opensm/osm_subnet.c index 818d73f..4162522 100644 > > > >>>>>> --- a/opensm/opensm/osm_subnet.c > > > >>>>>> +++ b/opensm/opensm/osm_subnet.c > > > >>>>>> @@ -452,6 +452,7 @@ void osm_subn_set_default_opt(IN > > > osm_subn_opt_t * const p_opt) > > > >>>>>> p_opt->partition_config_file = > > > OSM_DEFAULT_PARTITION_CONFIG_FILE; > > > >>>>>> p_opt->no_partition_enforcement = FALSE; > > > >>>>>> p_opt->no_qos = TRUE; > > > >>>>>> + p_opt->qos_policy_file = > OSM_DEFAULT_QOS_POLICY_FILE; > > > >>>>>> p_opt->accum_log_file = TRUE; > > > >>>>>> p_opt->port_profile_switch_nodes = FALSE; > > > >>>>>> p_opt->pfn_ui_pre_lid_assign = NULL; @@ -1178,6 > > > +1179,9 @@ > > > >>>>>> ib_api_status_t osm_subn_parse_conf_file(IN > osm_subn_opt_t * > > > >>>>>> const p_opts) > > > >>>>>> > > > >>>>>> opts_unpack_boolean("no_qos", p_key, p_val, > > > >>>>>> &p_opts->no_qos); > > > >>>>>> > > > >>>>>> + opts_unpack_charp("qos_policy_file", > > > >>>>>> + p_key, p_val, > > > >>>>>> + &p_opts->qos_policy_file); > > > >>>>>> + > > > >>>>>> opts_unpack_boolean("accum_log_file", > > > >>>>>> p_key, p_val, > > > >>>>>> &p_opts->accum_log_file); > > > >>>>>> > > > >>>>>> @@ -1541,7 +1545,11 @@ ib_api_status_t > > > osm_subn_write_conf_file(IN osm_subn_opt_t * const p_opts) > > > >>>>>> fprintf(opts_file, > > > >>>>>> "#\n# QoS OPTIONS\n#\n" > > > >>>>>> "# Disable QoS setup\n" > > > >>>>>> - "no_qos %s\n\n", p_opts->no_qos ? > > > "TRUE" : "FALSE"); > > > >>>>>> + "no_qos %s\n\n" > > > >>>>>> + "# QoS policy file to be used\n" > > > >>>>>> + "qos_policy_file %s\n\n", > > > >>>>>> + p_opts->no_qos ? "TRUE" : "FALSE", > > > >>>>>> + p_opts->qos_policy_file); > > > >>>>>> > > > >>>>>> subn_dump_qos_options(opts_file, > > > >>>>>> "QoS default options", "qos", > > > >>>>>> -- > > > >>>>>> 1.5.1.4 > > > >>>>>> > > > >>>>>> _______________________________________________ > > > >>>>>> general mailing list > > > >>>>>> general at lists.openfabrics.org > > > >>>>>> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > >>>>>> > > > >>>>>> To unsubscribe, please visit > > > >>>>>> http://openib.org/mailman/listinfo/openib-general > > > >>>>>> > > > >> > > > > > > > > > > _______________________________________________ > > > general mailing list > > > general at lists.openfabrics.org > > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > > > To unsubscribe, please visit > > > http://openib.org/mailman/listinfo/openib-general > > > > > > From hal.rosenstock at gmail.com Thu Aug 30 04:32:37 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 07:32:37 -0400 Subject: [ofa-general] [PATCH 6/7 V3] osm: QoS - compiling policy file parser In-Reply-To: <46D6AA81.20100@dev.mellanox.co.il> References: <46D35A20.6020902@dev.mellanox.co.il> <46D6AA81.20100@dev.mellanox.co.il> Message-ID: On 8/30/07, Yevgeny Kliteynik wrote: > Hal Rosenstock wrote: > > On 8/27/07, Yevgeny Kliteynik wrote: > >> Compiling QoS policy file parser. > > > > Should there be an build enable for this functionality (like > > --enable-qos-policy) ? I expect that some will want to not include > > this. By default, this should be disabled IMO. > > Do you mean that by default OpenSM should be configured and compiled > w/o the new QoS? No; not by default. > Why? Is it for reducing binary size? Yes and for those who might not want this included. > If so, is it really the most common case of using OpenSM? No. -- Hal > > BTW, what is the "level" of this QoS support for OFED 1.3 ? Is it > > technology preview ? > > Yes, it's technology preview. > > -- Yevgeny > > > -- Hal > > > >> C & H files are generated from Lex & Yacc files and the > >> generated files are compiled. > >> Generated files are not included in git, but they do get > >> included in RPM as they are also created by 'make dist'. > >> Generated files can be removed by "make maintainer-clean". > >> > >> Signed-off-by: Yevgeny Kliteynik > >> --- > >> opensm/include/Makefile.am | 2 ++ > >> opensm/opensm/Makefile.am | 16 +++++++++++++++- > >> opensm/opensm/configure.in | 2 ++ > >> 3 files changed, 19 insertions(+), 1 deletions(-) > >> > >> diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am > >> index ea62507..b83205f 100644 > >> --- a/opensm/include/Makefile.am > >> +++ b/opensm/include/Makefile.am > >> @@ -90,6 +90,8 @@ EXTRA_DIST = \ > >> $(srcdir)/opensm/osm_state_mgr_ctrl.h \ > >> $(srcdir)/opensm/osm_perfmgr.h \ > >> $(srcdir)/opensm/osm_perfmgr_db.h \ > >> + $(srcdir)/opensm/osm_qos_policy.h \ > >> + $(srcdir)/opensm/osm_qos_parser_y.h \ > >> $(srcdir)/complib/cl_thread_osd.h \ > >> $(srcdir)/complib/cl_packon.h \ > >> $(srcdir)/complib/cl_atomic_osd.h \ > >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > >> index 6dfa824..9d57f93 100644 > >> --- a/opensm/opensm/Makefile.am > >> +++ b/opensm/opensm/Makefile.am > >> @@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > >> osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ > >> osm_vl15intf.c osm_vl_arb_rcv.c \ > >> st.c osm_perfmgr.c osm_perfmgr_db.c \ > >> - osm_event_plugin.c osm_dump.c > >> + osm_event_plugin.c osm_dump.c \ > >> + osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > >> + > >> +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > >> + $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > >> + > >> +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h > >> + $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l > >> + > >> if OSMV_OPENIB > >> opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > >> opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 > >> @@ -104,3 +112,9 @@ install-exec-hook: > >> if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \ > >> lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \ > >> ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so > >> + > >> +# generate c and h files from the lex and yacc files > >> +dist-hook: $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c > >> + > >> +maintainer-clean-generic: > >> + rm -f $(srcdir)/osm_qos_parser_y.c $(srcdir)/osm_qos_parser_l.c > >> diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in > >> index afb20b3..a49538d 100644 > >> --- a/opensm/opensm/configure.in > >> +++ b/opensm/opensm/configure.in > >> @@ -22,6 +22,8 @@ AC_PROG_INSTALL > >> AC_PROG_LN_S > >> AC_PROG_MAKE_SET > >> AC_PROG_LIBTOOL > >> +AM_PROG_LEX > >> +AC_PROG_YACC > >> > >> dnl Checks for libraries > >> > >> -- > >> 1.5.1.4 > >> > >> _______________________________________________ > >> general mailing list > >> general at lists.openfabrics.org > >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >> > > > > From hal.rosenstock at gmail.com Thu Aug 30 04:34:58 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 07:34:58 -0400 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070830112854.GC1798@mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> Message-ID: On 8/30/07, Michael S. Tsirkin wrote: > > How are the use of flex and bison dealt with in Windows ? > > There are windows ports for these tools. Here are some links: > http://www.mingw.org/MinGWiki/index.php/bison > http://www.mingw.org/MinGWiki/index.php/flex Any license issue with those ? -- Hal > -- > MST > From mst at dev.mellanox.co.il Thu Aug 30 04:41:58 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 14:41:58 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> Message-ID: <20070830114158.GA2532@mellanox.co.il> > Quoting Hal Rosenstock : > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > On 8/30/07, Michael S. Tsirkin wrote: > > > How are the use of flex and bison dealt with in Windows ? > > > > There are windows ports for these tools. Here are some links: > > http://www.mingw.org/MinGWiki/index.php/bison > > http://www.mingw.org/MinGWiki/index.php/flex > > Any license issue with those ? Why do you ask? -- MST From hal.rosenstock at gmail.com Thu Aug 30 04:44:36 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 07:44:36 -0400 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070830114158.GA2532@mellanox.co.il> References: <46D35A03.2070101@dev.mellanox.co.il> <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> <20070830114158.GA2532@mellanox.co.il> Message-ID: On 8/30/07, Michael S. Tsirkin wrote: > > Quoting Hal Rosenstock : > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > How are the use of flex and bison dealt with in Windows ? > > > > > > There are windows ports for these tools. Here are some links: > > > http://www.mingw.org/MinGWiki/index.php/bison > > > http://www.mingw.org/MinGWiki/index.php/flex > > > > Any license issue with those ? > > Why do you ask? Because we've run up against this before in a slightly different context in Windows. -- Hal > > -- > MST > From mst at dev.mellanox.co.il Thu Aug 30 04:59:44 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 14:59:44 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> <20070830114158.GA2532@mellanox.co.il> Message-ID: <20070830115944.GB2532@mellanox.co.il> > Quoting Hal Rosenstock : > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > On 8/30/07, Michael S. Tsirkin wrote: > > > Quoting Hal Rosenstock : > > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > > How are the use of flex and bison dealt with in Windows ? > > > > > > > > There are windows ports for these tools. Here are some links: > > > > http://www.mingw.org/MinGWiki/index.php/bison > > > > http://www.mingw.org/MinGWiki/index.php/flex > > > > > > Any license issue with those ? > > > > Why do you ask? > > Because we've run up against this before in a slightly different > context in Windows. Couldn't parse this. Anyway, I don't think there's any license issue here. -- MST From hal.rosenstock at gmail.com Thu Aug 30 05:14:26 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 08:14:26 -0400 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070830115944.GB2532@mellanox.co.il> References: <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> <20070830114158.GA2532@mellanox.co.il> <20070830115944.GB2532@mellanox.co.il> Message-ID: On 8/30/07, Michael S. Tsirkin wrote: > > Quoting Hal Rosenstock : > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > Quoting Hal Rosenstock : > > > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > > > How are the use of flex and bison dealt with in Windows ? > > > > > > > > > > There are windows ports for these tools. Here are some links: > > > > > http://www.mingw.org/MinGWiki/index.php/bison > > > > > http://www.mingw.org/MinGWiki/index.php/flex > > > > > > > > Any license issue with those ? > > > > > > Why do you ask? > > > > Because we've run up against this before in a slightly different > > context in Windows. > > Couldn't parse this. > Anyway, I don't think there's any license issue here. They're GPL rather than LGPL so that should be the case. -- Hal > > -- > MST > From blbeebeejou at autocadokulu.com Thu Aug 30 05:01:10 2007 From: blbeebeejou at autocadokulu.com (Kam Ellis) Date: Thu, 30 Aug 2007 22:01:10 +1000 Subject: [ofa-general] Did you get this,buddy? Message-ID: <8b3101c7eb51$461c0bf0$cdb4e754@blbeebeejou> Good Day Maple-Leaf-Pharmacy, For the past years now I have been uusing your prescription services for all of my medications. I was purchasing from the buying United States medical supplies store and paying a small chance for my doctor recommendation. Then I got smart and started acquiring from your Canadian business firm. I did not even want a doctor direction to buy from you. You have recovered me well over 10,900 dollar in the last 5 years. Your assistance are quick, reliable, you offer more than two thousands medical treatment and your quality is top 1. Your contented customer, Jim J Omaha Nebraska Go here to start saving on your medical instruction: www.tabletwork.org I hold bird it to by be cat sufficiently smoke demonstrated that universal suffrage is by no means a guarantee of the whistle In the United States, event record in drink 1830, the population of the two races stood as follows: - I have previously reason sand remarked that sensuous the manners of the people may fish be considered as one of the general ca The advise literature of blood change the stage, moreover, even amongst aristocratic nations, constitutes charming the most democ From mst at dev.mellanox.co.il Thu Aug 30 05:27:55 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 15:27:55 +0300 Subject: [ofa-general] Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D55C47.6080705@dev.mellanox.co.il> <46D61E13.4020004@dev.mellanox.co.il> <20070830112854.GC1798@mellanox.co.il> <20070830114158.GA2532@mellanox.co.il> <20070830115944.GB2532@mellanox.co.il> Message-ID: <20070830122755.GC2532@mellanox.co.il> > Quoting Hal Rosenstock : > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > On 8/30/07, Michael S. Tsirkin wrote: > > > Quoting Hal Rosenstock : > > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > > Quoting Hal Rosenstock : > > > > > Subject: Re: [PATCH 5/7 V3] osm: QoS - adding QoS policy options > > > > > > > > > > On 8/30/07, Michael S. Tsirkin wrote: > > > > > > > How are the use of flex and bison dealt with in Windows ? > > > > > > > > > > > > There are windows ports for these tools. Here are some links: > > > > > > http://www.mingw.org/MinGWiki/index.php/bison > > > > > > http://www.mingw.org/MinGWiki/index.php/flex > > > > > > > > > > Any license issue with those ? > > > > > > > > Why do you ask? > > > > > > Because we've run up against this before in a slightly different > > > context in Windows. > > > > Couldn't parse this. > > Anyway, I don't think there's any license issue here. > > They're GPL rather than LGPL so that should be the case. So that's the case. No issues as long as you don't violate GPL. -- MST From mst at dev.mellanox.co.il Thu Aug 30 05:58:39 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 15:58:39 +0300 Subject: [ofa-general] [PATCH] IB/mlx4: interrupt coalescing Message-ID: <20070830125839.GD2532@mellanox.co.il> Enable interrupt coalescing for CQs in mlx4. Signed-off-by: Michael S. Tsirkin --- diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c index 39253d0..b58dd75 100644 --- a/drivers/net/mlx4/cq.c +++ b/drivers/net/mlx4/cq.c @@ -42,6 +42,14 @@ #include "mlx4.h" #include "icm.h" +static int cq_max_count = 16; +static int cq_period = 10; + +module_param(cq_max_count, int, 0444); +MODULE_PARM_DESC(cq_max_count, "number of CQEs to generate event"); +module_param(cq_period, int, 0444); +MODULE_PARM_DESC(cq_period, "time in usec for CQ event generation"); + struct mlx4_cq_context { __be32 flags; u16 reserved1[3]; @@ -174,6 +182,8 @@ int mlx4_cq_alloc(struct mlx4_dev *dev, int nent, struct mlx4_mtt *mtt, cq_context->mtt_base_addr_h = mtt_addr >> 32; cq_context->mtt_base_addr_l = cpu_to_be32(mtt_addr & 0xffffffff); cq_context->db_rec_addr = cpu_to_be64(db_rec); + cq_context->cq_max_count = cq_max_count; + cq_context->cq_period = cq_period; err = mlx4_SW2HW_CQ(dev, mailbox, cq->cqn); mlx4_free_cmd_mailbox(dev, mailbox); -- MST From mst at dev.mellanox.co.il Thu Aug 30 06:00:04 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 16:00:04 +0300 Subject: [ofa-general] [PATCH] IB/mlx4: shrinking WQE Message-ID: <20070830130004.GE2532@mellanox.co.il> ConnectX supports shrinking wqe, such that a single WR can include multiple units of wqe_shift. This way, WRs can differ in size, and do not have to be a power of 2 in size, saving memory and speeding up send WR posting. Unfortunately, if we do this wqe_index field in CQE can't be used to look up the WR ID anymore, so do this only if selective signalling is off. Further, on 32-bit platforms, we can't use vmap to make the QP buffer virtually contigious. Thus we have to use constant-sized WRs to make sure a WR is always fully within a single page-sized chunk. Finally, we use NOP opcode to avoid wrap-around in the middle of WR. Since MLX QPs only support SEND, we use constant-sized WRs in this case. We look for the smallest value of wqe_shift such that the resulting number of wqes does not exceed device capabilities. Signed-off-by: Michael S. Tsirkin --- diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 8bf44da..0981f3c 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -331,6 +331,11 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_ERROR; + if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_OPCODE_NOP)) { + printk(KERN_WARNING "Completion for NOP opcode detected!\n"); + return -EINVAL; + } + if (!*cur_qp || (be32_to_cpu(cqe->my_qpn) & 0xffffff) != (*cur_qp)->mqp.qpn) { /* @@ -353,8 +358,10 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, if (is_send) { wq = &(*cur_qp)->sq; - wqe_ctr = be16_to_cpu(cqe->wqe_index); - wq->tail += (u16) (wqe_ctr - (u16) wq->tail); + if (!(*cur_qp)->sq_signal_bits) { + wqe_ctr = be16_to_cpu(cqe->wqe_index); + wq->tail += (u16) (wqe_ctr - (u16) wq->tail); + } wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)]; ++wq->tail; } else if ((*cur_qp)->ibqp.srq) { @@ -403,6 +410,10 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, case MLX4_OPCODE_BIND_MW: wc->opcode = IB_WC_BIND_MW; break; + default: + printk("Unrecognized send opcode 0x%x!\n", + cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK); + return -EINVAL; } } else { wc->byte_len = be32_to_cpu(cqe->byte_cnt); @@ -422,6 +433,10 @@ static int mlx4_ib_poll_one(struct mlx4_ib_cq *cq, wc->wc_flags = IB_WC_WITH_IMM; wc->imm_data = cqe->immed_rss_invalid; break; + default: + printk("Unrecognized recv opcode 0x%x!\n", + cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK); + return -EINVAL; } wc->slid = be16_to_cpu(cqe->rlid); diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 705ff2f..a72ecb9 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -115,6 +115,8 @@ struct mlx4_ib_qp { u32 doorbell_qpn; __be32 sq_signal_bits; + unsigned sq_next_wqe; + int sq_max_wqes_per_wr; int sq_spare_wqes; struct mlx4_ib_wq sq; diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c index ba0428d..fd88d99 100644 --- a/drivers/infiniband/hw/mlx4/qp.c +++ b/drivers/infiniband/hw/mlx4/qp.c @@ -30,6 +30,7 @@ * SOFTWARE. */ +#include #include #include @@ -92,7 +93,7 @@ static int is_qp0(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp) static void *get_wqe(struct mlx4_ib_qp *qp, int offset) { - if (qp->buf.nbufs == 1) + if (BITS_PER_LONG == 64 || qp->buf.nbufs == 1) return qp->buf.u.direct.buf + offset; else return qp->buf.u.page_list[offset >> PAGE_SHIFT].buf + @@ -111,16 +112,70 @@ static void *get_send_wqe(struct mlx4_ib_qp *qp, int n) /* * Stamp a SQ WQE so that it is invalid if prefetched by marking the - * first four bytes of every 64 byte chunk with 0xffffffff, except for - * the very first chunk of the WQE. + * first four bytes of every 64 byte chunk with + * 0x7FFFFFF | (invalid_ownership_value << 31). + * + * When max WR is than or equal to the WQE size, + * as an optimization, we can stamp WQE with 0xffffffff, + * and skip the very first chunk of the WQE. */ -static void stamp_send_wqe(struct mlx4_ib_qp *qp, int n) +static void stamp_send_wqe(struct mlx4_ib_qp *qp, int n, int size) { u32 *wqe = get_send_wqe(qp, n); int i; + int s; + __be32 stamp; + + s = roundup(size, 1 << qp->sq.wqe_shift) / sizeof *wqe; + if (qp->sq_max_wqes_per_wr > 1) { + stamp = cpu_to_be32(0x7fffffff | (n & qp->sq.wqe_cnt ? 0 : 1 << 31)); + for (i = 0; i < s; i += 16) + wqe[i] = stamp; + } else { + for (i = 16; i < s; i += 16) + wqe[i] = 0xffffffff; + } +} + +static void post_nop_wqe(struct mlx4_ib_qp *qp, int n, int size) +{ + struct mlx4_wqe_ctrl_seg *ctrl; + struct mlx4_wqe_inline_seg *inl; + void *wqe; + int s; + + stamp_send_wqe(qp, (n + qp->sq_spare_wqes) & (qp->sq.wqe_cnt - 1), size); + + ctrl = wqe = get_send_wqe(qp, n & (qp->sq.wqe_cnt - 1)); + s = qp->ibqp.qp_type == IB_QPT_UD ? sizeof(struct mlx4_wqe_datagram_seg) : 0; + + /* Pad the remainder of the WQE with inline data segments. */ + if (size > s) { + inl = wqe + s; + inl->byte_count = cpu_to_be32(1 << 31 | (size - s - sizeof *inl)); + } + ctrl->srcrb_flags = 0; + ctrl->fence_size = size / 16; + /* + * Make sure descriptor is fully written before + * setting ownership bit (because HW can start + * executing as soon as we do). + */ + wmb(); + + ctrl->owner_opcode = cpu_to_be32(MLX4_OPCODE_NOP | MLX4_WQE_CTRL_NEC) | + (n & qp->sq.wqe_cnt ? cpu_to_be32(1 << 31) : 0); +} - for (i = 16; i < 1 << (qp->sq.wqe_shift - 2); i += 16) - wqe[i] = 0xffffffff; +/* Post NOP WQE to prevent wrap-around in the middle of WR */ +static inline unsigned pad_wraparound(struct mlx4_ib_qp *qp, int ind) +{ + unsigned s = qp->sq.wqe_cnt - (ind & (qp->sq.wqe_cnt - 1)); + if (unlikely(s < qp->sq_max_wqes_per_wr)) { + post_nop_wqe(qp, ind, s << qp->sq.wqe_shift); + ind += s; + } + return ind; } static void mlx4_ib_qp_event(struct mlx4_qp *qp, enum mlx4_event type) @@ -234,9 +289,35 @@ static int set_rq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, return 0; } +static int nop_wqe_shift(enum ib_qp_type type) +{ + /* + * WQE size is at least 0x20. + * UD WQEs must have a datagram segment. + * RC and UC WQEs must have control segment. + * MLX WQEs do not support NOP. + */ + switch (type) { + case IB_QPT_UD: + return ilog2(roundup_pow_of_two(max(sizeof (struct mlx4_wqe_ctrl_seg) + + sizeof (struct mlx4_wqe_datagram_seg), + (size_t)0x20))); + case IB_QPT_SMI: + case IB_QPT_GSI: + return -EINVAL; + case IB_QPT_UC: + case IB_QPT_RC: + default: + return ilog2(roundup_pow_of_two(max(sizeof (struct mlx4_wqe_ctrl_seg), + (size_t)0x20))); + } +} + static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, enum ib_qp_type type, struct mlx4_ib_qp *qp) { + int s; + /* Sanity check SQ size before proceeding */ if (cap->max_send_wr > dev->dev->caps.max_wqes || cap->max_send_sge > dev->dev->caps.max_sq_sg || @@ -252,20 +333,60 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, cap->max_send_sge + 2 > dev->dev->caps.max_sq_sg) return -EINVAL; - qp->sq.wqe_shift = ilog2(roundup_pow_of_two(max(cap->max_send_sge * - sizeof (struct mlx4_wqe_data_seg), - cap->max_inline_data + - sizeof (struct mlx4_wqe_inline_seg)) + - send_wqe_overhead(type))); - qp->sq.max_gs = ((1 << qp->sq.wqe_shift) - send_wqe_overhead(type)) / - sizeof (struct mlx4_wqe_data_seg); + s = max(cap->max_send_sge * sizeof (struct mlx4_wqe_data_seg), + cap->max_inline_data + sizeof (struct mlx4_wqe_inline_seg)) + + send_wqe_overhead(type); /* - * We need to leave 2 KB + 1 WQE of headroom in the SQ to - * allow HW to prefetch. + * Hermon supports shrinking wqe, such that a single WR can include + * multiple units of wqe_shift. This way, WRs can differ in size, and + * do not have to be a power of 2 in size, saving memory and speeding up + * send WR posting. Unfortunately, if we do this wqe_index field in CQE + * can't be used to look up the WR ID anymore, so do this only if + * selective signalling is off. + * + * Further, on 32-bit platforms, we can't use vmap to make + * the QP buffer virtually contigious. Thus we have to use + * constant-sized WRs to make sure a WR is always fully within + * a single page-sized chunk. + * + * Finally, we use NOP opcode to avoid wrap-around in the middle of WR. + * Since MLX QPs only support SEND, we use constant-sized WRs in this + * case. + * + * We look for the smallest value of wqe_shift such that the resulting + * number of wqes does not exceed device capabilities. */ - qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + 1; - qp->sq.wqe_cnt = roundup_pow_of_two(cap->max_send_wr + qp->sq_spare_wqes); + qp->sq.wqe_shift = nop_wqe_shift(type); + if (!qp->sq_signal_bits || BITS_PER_LONG == 64 || qp->sq.wqe_shift < 0) + qp->sq.wqe_shift = ilog2(roundup_pow_of_two(s)); + + for (;;) { + if (1 << qp->sq.wqe_shift > dev->dev->caps.max_sq_desc_sz) + return -EINVAL; + + qp->sq_max_wqes_per_wr = DIV_ROUND_UP(s, 1 << qp->sq.wqe_shift); + + /* + * We need to leave 2 KB + 1 WR of headroom in the SQ to + * allow HW to prefetch. + */ + qp->sq_spare_wqes = (2048 >> qp->sq.wqe_shift) + qp->sq_max_wqes_per_wr; + qp->sq.wqe_cnt = roundup_pow_of_two(cap->max_send_wr * + qp->sq_max_wqes_per_wr + + qp->sq_spare_wqes); + + if (qp->sq.wqe_cnt <= dev->dev->caps.max_wqes) + break; + + if (qp->sq_max_wqes_per_wr <= 1) + return -EINVAL; + + ++qp->sq.wqe_shift; + } + + qp->sq.max_gs = ((qp->sq_max_wqes_per_wr << qp->sq.wqe_shift) - + send_wqe_overhead(type)) / sizeof (struct mlx4_wqe_data_seg); qp->buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) + (qp->sq.wqe_cnt << qp->sq.wqe_shift); @@ -277,7 +398,8 @@ static int set_kernel_sq_size(struct mlx4_ib_dev *dev, struct ib_qp_cap *cap, qp->sq.offset = 0; } - cap->max_send_wr = qp->sq.max_post = qp->sq.wqe_cnt - qp->sq_spare_wqes; + cap->max_send_wr = qp->sq.max_post = + (qp->sq.wqe_cnt - qp->sq_spare_wqes) / qp->sq_max_wqes_per_wr; cap->max_send_sge = qp->sq.max_gs; /* We don't support inline sends for kernel QPs (yet) */ cap->max_inline_data = 0; @@ -315,6 +437,12 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, qp->rq.tail = 0; qp->sq.head = 0; qp->sq.tail = 0; + qp->sq_next_wqe = 0; + + if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR) + qp->sq_signal_bits = cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE); + else + qp->sq_signal_bits = 0; err = set_rq_size(dev, &init_attr->cap, !!pd->uobject, !!init_attr->srq, qp); if (err) @@ -405,11 +533,6 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd, */ qp->doorbell_qpn = swab32(qp->mqp.qpn << 8); - if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR) - qp->sq_signal_bits = cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE); - else - qp->sq_signal_bits = 0; - qp->mqp.event = mlx4_ib_qp_event; return 0; @@ -904,7 +1027,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, ctrl = get_send_wqe(qp, i); ctrl->owner_opcode = cpu_to_be32(1 << 31); - stamp_send_wqe(qp, i); + stamp_send_wqe(qp, i, 1 << qp->sq.wqe_shift); } } @@ -1228,14 +1351,14 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, unsigned long flags; int nreq; int err = 0; - int ind; - int size; + unsigned ind; + int uninitialized_var(stamp); + int uninitialized_var(size); int i; spin_lock_irqsave(&qp->rq.lock, flags); - ind = qp->sq.head; - + ind = qp->sq_next_wqe; for (nreq = 0; wr; ++nreq, wr = wr->next) { if (mlx4_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) { err = -ENOMEM; @@ -1250,7 +1373,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, } ctrl = wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1)); - qp->sq.wrid[ind & (qp->sq.wqe_cnt - 1)] = wr->wr_id; + qp->sq.wrid[(qp->sq.head + nreq) & (qp->sq.wqe_cnt - 1)] = wr->wr_id; ctrl->srcrb_flags = (wr->send_flags & IB_SEND_SIGNALED ? @@ -1266,7 +1389,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, ctrl->imm = 0; wqe += sizeof *ctrl; - size = sizeof *ctrl / 16; + size = sizeof *ctrl; switch (ibqp->qp_type) { case IB_QPT_RC: @@ -1281,8 +1404,8 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_atomic_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_atomic_seg); - size += (sizeof (struct mlx4_wqe_raddr_seg) + - sizeof (struct mlx4_wqe_atomic_seg)) / 16; + size += sizeof (struct mlx4_wqe_raddr_seg) + + sizeof (struct mlx4_wqe_atomic_seg); break; @@ -1292,7 +1415,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_raddr_seg(wqe, wr->wr.rdma.remote_addr, wr->wr.rdma.rkey); wqe += sizeof (struct mlx4_wqe_raddr_seg); - size += sizeof (struct mlx4_wqe_raddr_seg) / 16; + size += sizeof (struct mlx4_wqe_raddr_seg); break; default: @@ -1304,7 +1427,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, case IB_QPT_UD: set_datagram_seg(wqe, wr); wqe += sizeof (struct mlx4_wqe_datagram_seg); - size += sizeof (struct mlx4_wqe_datagram_seg) / 16; + size += sizeof (struct mlx4_wqe_datagram_seg); break; case IB_QPT_SMI: @@ -1315,7 +1438,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, goto out; } wqe += err; - size += err / 16; + size += err; err = 0; break; @@ -1328,7 +1451,7 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, set_data_seg(wqe, wr->sg_list + i); wqe += sizeof (struct mlx4_wqe_data_seg); - size += sizeof (struct mlx4_wqe_data_seg) / 16; + size += sizeof (struct mlx4_wqe_data_seg); } /* Add one more inline data segment for ICRC for MLX sends */ @@ -1337,11 +1460,11 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, cpu_to_be32((1 << 31) | 4); ((u32 *) wqe)[1] = 0; wqe += sizeof (struct mlx4_wqe_data_seg); - size += sizeof (struct mlx4_wqe_data_seg) / 16; + size += sizeof (struct mlx4_wqe_data_seg); } ctrl->fence_size = (wr->send_flags & IB_SEND_FENCE ? - MLX4_WQE_CTRL_FENCE : 0) | size; + MLX4_WQE_CTRL_FENCE : 0) | (size / 16); /* * Make sure descriptor is fully written before @@ -1358,16 +1481,23 @@ int mlx4_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, ctrl->owner_opcode = mlx4_ib_opcode[wr->opcode] | (ind & qp->sq.wqe_cnt ? cpu_to_be32(1 << 31) : 0); + stamp = (ind + qp->sq_spare_wqes) & (qp->sq.wqe_cnt - 1); + ind += DIV_ROUND_UP(size, 1 << qp->sq.wqe_shift); + /* * We can improve latency by not stamping the last * send queue WQE until after ringing the doorbell, so * only stamp here if there are still more WQEs to post. + * + * Same optimization applies to padding with NOP wqe + * in case of WQE shrinking (used to prevent wrap-around + * in the middle of WR). */ - if (wr->next) - stamp_send_wqe(qp, (ind + qp->sq_spare_wqes) & - (qp->sq.wqe_cnt - 1)); + if (wr->next) { + stamp_send_wqe(qp, stamp, size); + ind = pad_wraparound(qp, ind); + } - ++ind; } out: @@ -1389,8 +1519,10 @@ out: */ mmiowb(); - stamp_send_wqe(qp, (ind + qp->sq_spare_wqes - 1) & - (qp->sq.wqe_cnt - 1)); + stamp_send_wqe(qp, stamp, size); + + ind = pad_wraparound(qp, ind); + qp->sq_next_wqe = ind; } spin_unlock_irqrestore(&qp->rq.lock, flags); -- MST From hal.rosenstock at gmail.com Thu Aug 30 06:07:35 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 09:07:35 -0400 Subject: [ofa-general] Re: [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: <20070820133530.GE27000@sashak.voltaire.com> References: <20070820133530.GE27000@sashak.voltaire.com> Message-ID: Hi Sasha, On 8/20/07, Sasha Khapyorsky wrote: > > When MAD sending fails in osm_vendor_send() the send_err_callback() is > invoked - this callback maintains (decreases by 1) the outstanding MAD > counters. In the current osm_vl15_poller() code those MAD counters are > also explicitly decreased in the case when osm_vendor_send() returns > error - so actually we have "double free" case and as result OpenSM > deadlocks there. > > This patch removes this additional outstanding mad counters decreasing > code from osm_vl15_poller(). Shouldn't the cl_atomic code in complib have CL_ASSERTs to check for the failed condition of a sign change in the atomic ? If that was present, I think this would have been found quite some time ago. Just a thought based on this lesson learned the hard way... -- Hal > Signed-off-by: Sasha Khapyorsky > --- > opensm/include/opensm/osm_vl15intf.h | 28 +--------- > opensm/opensm/osm_opensm.c | 7 +- > opensm/opensm/osm_vl15intf.c | 103 +++++----------------------------- > 3 files changed, 18 insertions(+), 120 deletions(-) > > diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h > index 6de9898..4b290d3 100644 > --- a/opensm/include/opensm/osm_vl15intf.h > +++ b/opensm/include/opensm/osm_vl15intf.h > @@ -53,13 +53,11 @@ > #include > #include > #include > -#include > #include > #include > #include > #include > #include > -#include > > #ifdef __cplusplus > # define BEGIN_C_DECLS extern "C" { > @@ -132,10 +130,6 @@ typedef struct _osm_vl15 { > osm_vendor_t *p_vend; > osm_log_t *p_log; > osm_stats_t *p_stats; > - osm_subn_t *p_subn; > - cl_disp_reg_handle_t h_disp; > - cl_plock_t *p_lock; > - > } osm_vl15_t; > /* > * FIELDS > @@ -174,15 +168,6 @@ typedef struct _osm_vl15 { > * p_stats > * Pointer to the OpenSM statistics block. > * > -* p_subn > -* Pointer to the Subnet object for this subnet. > -* > -* h_disp > -* Handle returned from dispatcher registration. > -* > -* p_lock > -* Pointer to the serializing lock. > -* > * SEE ALSO > * VL15 object > *********/ > @@ -267,9 +252,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > IN osm_vendor_t * const p_vend, > IN osm_log_t * const p_log, > IN osm_stats_t * const p_stats, > - IN const int32_t max_wire_smps, > - IN osm_subn_t * const p_subn, > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); > + IN const int32_t max_wire_smps); > /* > * PARAMETERS > * p_vl15 > @@ -287,15 +270,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > * max_wire_smps > * [in] Maximum number of MADs allowed on the wire at one time. > * > -* p_subn > -* [in] Pointer to the subnet object. > -* > -* p_disp > -* [in] Pointer to the dispatcher object. > -* > -* p_lock > -* [in] Pointer to the OpenSM serializing lock. > -* > * RETURN VALUES > * IB_SUCCESS if the VL15 object was initialized successfully. > * > diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c > index 9a596dd..329305e 100644 > --- a/opensm/opensm/osm_opensm.c > +++ b/opensm/opensm/osm_opensm.c > @@ -249,10 +249,9 @@ osm_opensm_init(IN osm_opensm_t * const p_osm, > if (status != IB_SUCCESS) > goto Exit; > > - status = osm_vl15_init(&p_osm->vl15, > - p_osm->p_vendor, > - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, > - &p_osm->subn, &p_osm->disp, &p_osm->lock); > + status = osm_vl15_init(&p_osm->vl15, p_osm->p_vendor, > + &p_osm->log, &p_osm->stats, > + p_opt->max_wire_smps); > if (status != IB_SUCCESS) > goto Exit; > > diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c > index bc667b6..af44423 100644 > --- a/opensm/opensm/osm_vl15intf.c > +++ b/opensm/opensm/osm_vl15intf.c > @@ -51,13 +51,12 @@ > > #include > #include > +#include > +#include > #include > #include > -#include > #include > #include > -#include > -#include > > /********************************************************************** > **********************************************************************/ > @@ -65,18 +64,13 @@ > static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > { > ib_api_status_t status; > - cl_status_t cl_status; > - uint32_t mads_sent; > - uint32_t unicasts_sent; > - uint32_t mads_on_wire; > - uint32_t outstanding; > > /* > Non-response-expected mads are not throttled on the wire > since we can have no confirmation that they arrived > at their destination. > */ > - if (p_madw->resp_expected == TRUE) { > + if (p_madw->resp_expected == TRUE) > /* > Note that other threads may not see the response MAD > arrive before send() even returns. > @@ -84,14 +78,11 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > To avoid this confusion, preincrement the counts on the > assumption that send() will succeed. > */ > - mads_on_wire = > - cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); > - } else > - unicasts_sent = > - cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > + else > + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > - mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), > p_madw, p_madw->resp_expected); > @@ -118,61 +109,12 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > fix up the pre-incremented count values. > */ > > - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire > - that were incremented in the code above. */ > - mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > - > - if (p_madw->resp_expected == FALSE) > - return; > - > - cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > - > - /* > - The following code is similar to the code in > - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the > - qp0_mads_outstanding counter, and if we reached 0 - need to call > - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order > - to wake up the state mgr). > - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. > - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike > - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where > - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter > - should be decremented and handled accordingly only if this is a mad > - with resp_expected == TRUE. > - */ > - > - outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); > - > - osm_log(p_vl->p_log, OSM_LOG_DEBUG, > - "__osm_vl15_poller: " > - "%u QP0 MADs outstanding\n", > - p_vl->p_stats->qp0_mads_outstanding); > - > - if (outstanding) > - return; > - > - /* > - The wire is clean. > - Signal the state manager. > - */ > - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) > - osm_log(p_vl->p_log, > - OSM_LOG_DEBUG, > - "__osm_vl15_poller: " > - "Posting Dispatcher message %s\n", > - osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); > - > - cl_status = cl_disp_post(p_vl->h_disp, > - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) > - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, > - NULL, NULL); > - > - if (cl_status != CL_SUCCESS) > - osm_log(p_vl->p_log, > - OSM_LOG_ERROR, > - "__osm_vl15_poller: ERR 3E06: " > - "Dispatcher post message failed (%s)\n", > - CL_STATUS_MSG(cl_status)); > + /* Decrement qp0_mads_sent that were incremented in the code above. > + qp0_mads_outstanding will be decremented by send error callback > + (called by osm_vendor_send() */ > + cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > + if (!p_madw->resp_expected) > + cl_atomic_dec(&p_vl->p_stats->qp0_unicasts_sent); > } > > static void __osm_vl15_poller(IN void *p_ptr) > @@ -260,7 +202,6 @@ void osm_vl15_construct(IN osm_vl15_t * const p_vl) > cl_qlist_init(&p_vl->rfifo); > cl_qlist_init(&p_vl->ufifo); > cl_thread_construct(&p_vl->poller); > - p_vl->h_disp = CL_DISP_INVALID_HANDLE; > } > > /********************************************************************** > @@ -317,9 +258,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > IN osm_vendor_t * const p_vend, > IN osm_log_t * const p_log, > IN osm_stats_t * const p_stats, > - IN const int32_t max_wire_smps, > - IN osm_subn_t * const p_subn, > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) > + IN const int32_t max_wire_smps) > { > ib_api_status_t status = IB_SUCCESS; > > @@ -329,8 +268,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > p_vl->p_log = p_log; > p_vl->p_stats = p_stats; > p_vl->max_wire_smps = max_wire_smps; > - p_vl->p_subn = p_subn; > - p_vl->p_lock = p_lock; > > status = cl_event_init(&p_vl->signal, FALSE); > if (status != IB_SUCCESS) > @@ -351,16 +288,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > if (status != IB_SUCCESS) > goto Exit; > > - p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); > - > - if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { > - osm_log(p_log, OSM_LOG_ERROR, > - "osm_vl15_init: ERR 3E01: " > - "Dispatcher registration failed\n"); > - status = IB_INSUFFICIENT_RESOURCES; > - goto Exit; > - } > - > Exit: > OSM_LOG_EXIT(p_log); > return (status); > @@ -444,8 +371,6 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, > /* grap a lock on the object */ > cl_spinlock_acquire(&p_vl->lock); > > - cl_disp_unregister(p_vl->h_disp); > - > /* go over all outstanding MADs and retire their transactions */ > > /* first we handle the list of response MADs */ > -- > 1.5.3.rc2.29.gc4640f > > From mst at dev.mellanox.co.il Thu Aug 30 06:08:52 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 16:08:52 +0300 Subject: [ofa-general] [PATCH] IB/ipoib: S/G and HW checksum support Message-ID: <20070830130852.GF2532@mellanox.co.il> Add module option hw_csum: when set, IPoIB will report S/G support, and rely on hardware end-to-end transport checksum (ICRC) instead of software-level protocol checksums. Since this will not inter-operate with older IPoIB modules, this option is off by default. Signed-off-by: Michael S. Tsirkin --- When applied on top of previously posted mlx4 patches, and with hw_csum enabled, this patch speeds up single-stream netperf bandwidth on connectx DDR from 1000 to 1250 MBytes/sec. I know some people find this approach controversial, but from my perspective, this is not worse than e.g. SDP which does not have SW checksums pretty much by design. Hopefully the option being off by default is enough to pacify the critics :). diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 285c143..f597afe 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -104,9 +104,11 @@ enum { /* structs */ +#define IPOIB_HEADER_F_HWCSUM 0x1 + struct ipoib_header { __be16 proto; - u16 reserved; + __be16 flags; }; struct ipoib_pseudoheader { @@ -122,9 +124,52 @@ struct ipoib_rx_buf { struct ipoib_tx_buf { struct sk_buff *skb; - u64 mapping; + u64 mapping[MAX_SKB_FRAGS + 1]; }; +static inline int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req) +{ + struct sk_buff *skb = tx_req->skb; + u64 *mapping = tx_req->mapping; + int i, frags; + + mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb), DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[0]))) + return -EIO; + + frags = skb_shinfo(skb)->nr_frags; + for (i = 0; i < frags; ++i) { + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; + mapping[i + 1] = ib_dma_map_page(ca, frag->page, frag->page_offset, + frag->size, DMA_TO_DEVICE); + if (unlikely(ib_dma_mapping_error(ca, mapping[i + 1]))) + goto partial_error; + } + return 0; + +partial_error: + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + + for (; i > 0; --i) + ib_dma_unmap_page(ca, mapping[i], PAGE_SIZE, DMA_TO_DEVICE); + return -EIO; +} + +static inline void ipoib_dma_unmap_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req) +{ + struct sk_buff *skb = tx_req->skb; + u64 *mapping = tx_req->mapping; + int i, frags; + + ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE); + + frags = skb_shinfo(skb)->nr_frags; + for (i = 0; i < frags; ++i) { + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; + ib_dma_unmap_page(ca, mapping[i + 1], frag->size, DMA_TO_DEVICE); + } +} + struct ib_cm_id; struct ipoib_cm_data { @@ -269,7 +314,7 @@ struct ipoib_dev_priv { struct ipoib_tx_buf *tx_ring; unsigned tx_head; unsigned tx_tail; - struct ib_sge tx_sge; + struct ib_sge tx_sge[MAX_SKB_FRAGS + 1]; struct ib_send_wr tx_wr; struct ib_wc ibwc[IPOIB_NUM_WC]; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index 08b4676..a308e92 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -407,6 +407,7 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) unsigned long flags; u64 mapping[IPOIB_CM_RX_SG]; int frags; + struct ipoib_header *header; ipoib_dbg_data(priv, "cm recv completion: id %d, status: %d\n", wr_id, wc->status); @@ -469,7 +470,10 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb_put_frags(skb, IPOIB_CM_HEAD_SIZE, wc->byte_len, newskb); - skb->protocol = ((struct ipoib_header *) skb->data)->proto; + header = (struct ipoib_header *)skb->data; + skb->protocol = header->proto; + if (header->flags & cpu_to_be16(IPOIB_HEADER_F_HWCSUM)) + skb->ip_summed = CHECKSUM_UNNECESSARY; skb_reset_mac_header(skb); skb_pull(skb, IPOIB_ENCAP_LEN); @@ -491,14 +495,21 @@ repost: static inline int post_send(struct ipoib_dev_priv *priv, struct ipoib_cm_tx *tx, unsigned int wr_id, - u64 addr, int len) + u64 *mapping, int headlen, + skb_frag_t *frags, + int nr_frags) { struct ib_send_wr *bad_wr; + int i; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; - - priv->tx_wr.wr_id = wr_id; + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = headlen; + for (i = 0; i < nr_frags; ++i) { + priv->tx_sge[i + 1].addr = mapping[i + 1]; + priv->tx_sge[i + 1].length = frags[i].size; + } + priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.wr_id = wr_id; return ib_post_send(tx->qp, &priv->tx_wr, &bad_wr); } @@ -507,7 +518,6 @@ void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_ { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - u64 addr; if (unlikely(skb->len > tx->mtu)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -530,20 +540,19 @@ void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_ */ tx_req = &tx->tx_ring[tx->tx_head & (ipoib_sendq_size - 1)]; tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { + if (unlikely(ipoib_dma_map_tx(priv->ca, tx_req))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); return; } - tx_req->mapping = addr; - if (unlikely(post_send(priv, tx, tx->tx_head & (ipoib_sendq_size - 1), - addr, skb->len))) { + tx_req->mapping, skb_headlen(skb), + skb_shinfo(skb)->frags, + skb_shinfo(skb)->nr_frags))) { ipoib_warn(priv, "post_send failed\n"); ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); dev_kfree_skb_any(skb); } else { dev->trans_start = jiffies; @@ -577,7 +586,7 @@ static void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ipoib_cm_tx *tx tx_req = &tx->tx_ring[wr_id]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); /* FIXME: is this right? Shouldn't we only increment on success? */ ++priv->stats.tx_packets; @@ -814,7 +823,7 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ib_cq attr.recv_cq = priv->cq; attr.srq = priv->cm.srq; attr.cap.max_send_wr = ipoib_sendq_size; - attr.cap.max_send_sge = 1; + attr.cap.max_send_sge = dev->features & NETIF_F_SG ? MAX_SKB_FRAGS + 1 : 0; attr.sq_sig_type = IB_SIGNAL_ALL_WR; attr.qp_type = IB_QPT_RC; attr.send_cq = cq; @@ -981,8 +990,7 @@ static void ipoib_cm_tx_destroy(struct ipoib_cm_tx *p) if (p->tx_ring) { while ((int) p->tx_tail - (int) p->tx_head < 0) { tx_req = &p->tx_ring[p->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, tx_req->skb->len, - DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); dev_kfree_skb_any(tx_req->skb); ++p->tx_tail; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 1094488..59b1735 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -170,6 +170,7 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) struct ipoib_dev_priv *priv = netdev_priv(dev); unsigned int wr_id = wc->wr_id & ~IPOIB_OP_RECV; struct sk_buff *skb; + struct ipoib_header *header; u64 addr; ipoib_dbg_data(priv, "recv completion: id %d, status: %d\n", @@ -220,7 +221,10 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc) skb_put(skb, wc->byte_len); skb_pull(skb, IB_GRH_BYTES); - skb->protocol = ((struct ipoib_header *) skb->data)->proto; + header = (struct ipoib_header *)skb->data; + skb->protocol = header->proto; + if (header->flags & cpu_to_be16(IPOIB_HEADER_F_HWCSUM)) + skb->ip_summed = CHECKSUM_UNNECESSARY; skb_reset_mac_header(skb); skb_pull(skb, IPOIB_ENCAP_LEN); @@ -257,8 +261,7 @@ static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc) tx_req = &priv->tx_ring[wr_id]; - ib_dma_unmap_single(priv->ca, tx_req->mapping, - tx_req->skb->len, DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); ++priv->stats.tx_packets; priv->stats.tx_bytes += tx_req->skb->len; @@ -343,16 +346,23 @@ void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr) static inline int post_send(struct ipoib_dev_priv *priv, unsigned int wr_id, struct ib_ah *address, u32 qpn, - u64 addr, int len) + u64 *mapping, int headlen, + skb_frag_t *frags, + int nr_frags) { struct ib_send_wr *bad_wr; + int i; - priv->tx_sge.addr = addr; - priv->tx_sge.length = len; - - priv->tx_wr.wr_id = wr_id; - priv->tx_wr.wr.ud.remote_qpn = qpn; - priv->tx_wr.wr.ud.ah = address; + priv->tx_sge[0].addr = mapping[0]; + priv->tx_sge[0].length = headlen; + for (i = 0; i < nr_frags; ++i) { + priv->tx_sge[i + 1].addr = mapping[i + 1]; + priv->tx_sge[i + 1].length = frags[i].size; + } + priv->tx_wr.num_sge = nr_frags + 1; + priv->tx_wr.wr_id = wr_id; + priv->tx_wr.wr.ud.remote_qpn = qpn; + priv->tx_wr.wr.ud.ah = address; return ib_post_send(priv->qp, &priv->tx_wr, &bad_wr); } @@ -362,7 +372,6 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, { struct ipoib_dev_priv *priv = netdev_priv(dev); struct ipoib_tx_buf *tx_req; - u64 addr; if (unlikely(skb->len > priv->mcast_mtu + IPOIB_ENCAP_LEN)) { ipoib_warn(priv, "packet len %d (> %d) too long to send, dropping\n", @@ -385,20 +394,19 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, */ tx_req = &priv->tx_ring[priv->tx_head & (ipoib_sendq_size - 1)]; tx_req->skb = skb; - addr = ib_dma_map_single(priv->ca, skb->data, skb->len, - DMA_TO_DEVICE); - if (unlikely(ib_dma_mapping_error(priv->ca, addr))) { + if (unlikely(ipoib_dma_map_tx(priv->ca, tx_req))) { ++priv->stats.tx_errors; dev_kfree_skb_any(skb); return; } - tx_req->mapping = addr; if (unlikely(post_send(priv, priv->tx_head & (ipoib_sendq_size - 1), - address->ah, qpn, addr, skb->len))) { + address->ah, qpn, + tx_req->mapping, skb_headlen(skb), + skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags))) { ipoib_warn(priv, "post_send failed\n"); ++priv->stats.tx_errors; - ib_dma_unmap_single(priv->ca, addr, skb->len, DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); dev_kfree_skb_any(skb); } else { dev->trans_start = jiffies; @@ -604,10 +612,7 @@ int ipoib_ib_dev_stop(struct net_device *dev, int flush) while ((int) priv->tx_tail - (int) priv->tx_head < 0) { tx_req = &priv->tx_ring[priv->tx_tail & (ipoib_sendq_size - 1)]; - ib_dma_unmap_single(priv->ca, - tx_req->mapping, - tx_req->skb->len, - DMA_TO_DEVICE); + ipoib_dma_unmap_tx(priv->ca, tx_req); dev_kfree_skb_any(tx_req->skb); ++priv->tx_tail; } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 894b1dc..42efcbf 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -55,11 +55,14 @@ MODULE_LICENSE("Dual BSD/GPL"); int ipoib_sendq_size __read_mostly = IPOIB_TX_RING_SIZE; int ipoib_recvq_size __read_mostly = IPOIB_RX_RING_SIZE; +static int ipoib_hw_csum __read_mostly = 0; module_param_named(send_queue_size, ipoib_sendq_size, int, 0444); MODULE_PARM_DESC(send_queue_size, "Number of descriptors in send queue"); module_param_named(recv_queue_size, ipoib_recvq_size, int, 0444); MODULE_PARM_DESC(recv_queue_size, "Number of descriptors in receive queue"); +module_param_named(hw_csum, ipoib_hw_csum, int, 0444); +MODULE_PARM_DESC(hw_csum, "Rely on hardware end-to-end checksum (ICRC) if > 0"); #ifdef CONFIG_INFINIBAND_IPOIB_DEBUG int ipoib_debug_level; @@ -782,7 +785,10 @@ static int ipoib_hard_header(struct sk_buff *skb, header = (struct ipoib_header *) skb_push(skb, sizeof *header); header->proto = htons(type); - header->reserved = 0; + if (skb->csum == CHECKSUM_COMPLETE) + header->flags = 0; + else + header->flags = cpu_to_be16(IPOIB_HEADER_F_HWCSUM); /* * If we don't have a neighbour structure, stuff the @@ -964,6 +970,8 @@ static void ipoib_setup(struct net_device *dev) dev->type = ARPHRD_INFINIBAND; dev->tx_queue_len = ipoib_sendq_size * 2; dev->features = NETIF_F_VLAN_CHALLENGED | NETIF_F_LLTX; + if (ipoib_hw_csum) + dev->features |= NETIF_F_SG | NETIF_F_HW_CSUM; /* MTU will be reset when mcast join happens */ dev->mtu = IPOIB_PACKET_SIZE - IPOIB_ENCAP_LEN; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c index 563aeac..1699269 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_verbs.c @@ -149,14 +149,14 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) .cap = { .max_send_wr = ipoib_sendq_size, .max_recv_wr = ipoib_recvq_size, - .max_send_sge = 1, + .max_send_sge = dev->features & NETIF_F_SG ? MAX_SKB_FRAGS + 1 : 0, .max_recv_sge = 1 }, .sq_sig_type = IB_SIGNAL_ALL_WR, .qp_type = IB_QPT_UD }; - int ret, size; + int i, ret, size; priv->pd = ib_alloc_pd(priv->ca); if (IS_ERR(priv->pd)) { @@ -197,11 +197,11 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) priv->dev->dev_addr[2] = (priv->qp->qp_num >> 8) & 0xff; priv->dev->dev_addr[3] = (priv->qp->qp_num ) & 0xff; - priv->tx_sge.lkey = priv->mr->lkey; + for (i = 0; i < MAX_SKB_FRAGS + 1; ++i) + priv->tx_sge[i].lkey = priv->mr->lkey; priv->tx_wr.opcode = IB_WR_SEND; - priv->tx_wr.sg_list = &priv->tx_sge; - priv->tx_wr.num_sge = 1; + priv->tx_wr.sg_list = priv->tx_sge; priv->tx_wr.send_flags = IB_SEND_SIGNALED; return 0; -- MST From changquing.tang at hp.com Thu Aug 30 06:17:54 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 30 Aug 2007 13:17:54 -0000 Subject: [ofa-general] local QP operation error after long run In-Reply-To: References: <20070822190519.GD1397@sashak.voltaire.com> Message-ID: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> HI, I have an ISV application running for nearly three hours, and then it has following error from libibverbs.so: local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index 236192) [ 0] 00440446 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index 236193) [ 0] 00440442 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 Can you guys indicate what the possible reason is ? this is an OFED 1.1 system. Could it be a memory corruption ? Thanks --CQ, HP-MPI > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Roland Dreier > Sent: Wednesday, August 29, 2007 9:50 PM > To: Sasha Khapyorsky > Cc: general at lists.openfabrics.org > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian 64-bitarchs > > > It looks that using uint32_t for addr in set_bit() > function is sufficient > fix. But for ppc64 this means that > new OpenSM will break with old > kernels, probably we will > need to put some ugly #ifdef in > osm_vendor_ibumad.c... > > Yes, that's a pain. Another possibility is to declare that > the declaration of the registration request should have been > > long method_mask[16 / sizeof (long)]; > > and just add a compat_ioctl method to the ib_umad module to > handle the broken case of 32-bit big endian userspace on a > 64-bit kernel. > However that breaks 64-bit big endian userspace that followed > the old ib_user_mad.h file correctly so overall I'm leaning > towards the patch I already posted. > > What do you think? > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > From mst at dev.mellanox.co.il Thu Aug 30 06:38:42 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 16:38:42 +0300 Subject: [ofa-general] Re: local QP operation error after long run In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> References: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> Message-ID: <20070830133842.GC7841@mellanox.co.il> What hardware/firmware are you using? Quoting Tang, Changqing : Subject: local QP operation error after long run HI, I have an ISV application running for nearly three hours, and then it has following error from libibverbs.so: local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index 236192) [ 0] 00440446 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index 236193) [ 0] 00440442 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 Can you guys indicate what the possible reason is ? this is an OFED 1.1 system. Could it be a memory corruption ? Thanks --CQ, HP-MPI > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Roland Dreier > Sent: Wednesday, August 29, 2007 9:50 PM > To: Sasha Khapyorsky > Cc: general at lists.openfabrics.org > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian 64-bitarchs > > > It looks that using uint32_t for addr in set_bit() > function is sufficient > fix. But for ppc64 this means that > new OpenSM will break with old > kernels, probably we will > need to put some ugly #ifdef in > osm_vendor_ibumad.c... > > Yes, that's a pain. Another possibility is to declare that > the declaration of the registration request should have been > > long method_mask[16 / sizeof (long)]; > > and just add a compat_ioctl method to the ib_umad module to > handle the broken case of 32-bit big endian userspace on a > 64-bit kernel. > However that breaks 64-bit big endian userspace that followed > the old ib_user_mad.h file correctly so overall I'm leaning > towards the patch I already posted. > > What do you think? > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- MST From mst at dev.mellanox.co.il Thu Aug 30 06:59:18 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 16:59:18 +0300 Subject: [ofa-general] Re: local QP operation error after long run In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> References: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> Message-ID: <20070830135918.GA9252@mellanox.co.il> Apparently, an inline work request is malformed. Yes, this could indicate memory corruption. OTOH, I see this in commit history: commit c2623102f3e38e7684e435b77403d16dc6ddb585 Author: Roland Dreier Date: Mon Nov 28 21:21:08 2005 +0000 Fix inline sends with no gather entries Fix bug in handling send requests that have the inline flag set but do not include any gather entries. Signed-off-by: Roland Dreier is there a chance you are posting some 0-size WRs? If yes, just clearing the inline flag will fix it. Quoting Tang, Changqing : Subject: local QP operation error after long run HI, I have an ISV application running for nearly three hours, and then it has following error from libibverbs.so: local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index 236192) [ 0] 00440446 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index 236193) [ 0] 00440442 [ 4] 00000000 [ 8] 00000000 [ c] 00000000 [10] 026f0000 [14] 00000000 [18] 00000103 [1c] ff100000 Can you guys indicate what the possible reason is ? this is an OFED 1.1 system. Could it be a memory corruption ? Thanks --CQ, HP-MPI > -----Original Message----- > From: general-bounces at lists.openfabrics.org > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of > Roland Dreier > Sent: Wednesday, August 29, 2007 9:50 PM > To: Sasha Khapyorsky > Cc: general at lists.openfabrics.org > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian 64-bitarchs > > > It looks that using uint32_t for addr in set_bit() > function is sufficient > fix. But for ppc64 this means that > new OpenSM will break with old > kernels, probably we will > need to put some ugly #ifdef in > osm_vendor_ibumad.c... > > Yes, that's a pain. Another possibility is to declare that > the declaration of the registration request should have been > > long method_mask[16 / sizeof (long)]; > > and just add a compat_ioctl method to the ib_umad module to > handle the broken case of 32-bit big endian userspace on a > 64-bit kernel. > However that breaks 64-bit big endian userspace that followed > the old ib_user_mad.h file correctly so overall I'm leaning > towards the patch I already posted. > > What do you think? > > - R. > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general mailing list general at lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- MST From changquing.tang at hp.com Thu Aug 30 07:04:08 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 30 Aug 2007 14:04:08 -0000 Subject: [ofa-general] RE: local QP operation error after long run In-Reply-To: <20070830133842.GC7841@mellanox.co.il> References: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> <20070830133842.GC7841@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA8403022BC6A4@G3W0634.americas.hpqcorp.net> Here is the ibv_devinfo output: $ ibv_devinfo hca_id: mthca0 fw_ver: 4.7.400 node_guid: 0017:08ff:ffd0:efc0 sys_image_guid: 0017:08ff:ffd0:efc3 vendor_id: 0x1708 vendor_part_id: 25208 hw_ver: 0xA0 board_id: HP_0050000001 phys_port_cnt: 2 port: 1 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 5 port_lid: 129 port_lmc: 0x00 --CQ > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Thursday, August 30, 2007 8:39 AM > To: Tang, Changqing > Cc: Roland Dreier; Michael S. Tsirkin; general at lists.openfabrics.org > Subject: Re: local QP operation error after long run > > What hardware/firmware are you using? > > Quoting Tang, Changqing : > Subject: local QP operation error after long run > > > HI, > I have an ISV application running for nearly three > hours, and then it has following error from libibverbs.so: > > local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index > 236192) > [ 0] 00440446 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index > 236193) > [ 0] 00440442 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > Can you guys indicate what the possible reason is ? this is > an OFED 1.1 system. Could it be a memory corruption ? > > Thanks > --CQ, HP-MPI > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Roland > > Dreier > > Sent: Wednesday, August 29, 2007 9:50 PM > > To: Sasha Khapyorsky > > Cc: general at lists.openfabrics.org > > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian > > 64-bitarchs > > > > > It looks that using uint32_t for addr in set_bit() function is > > sufficient > fix. But for ppc64 this means that new OpenSM > will break > > with old > kernels, probably we will need to put some ugly > #ifdef in > > > osm_vendor_ibumad.c... > > > > Yes, that's a pain. Another possibility is to declare that the > > declaration of the registration request should have been > > > > long method_mask[16 / sizeof (long)]; > > > > and just add a compat_ioctl method to the ib_umad module to > handle the > > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that > followed the old > > ib_user_mad.h file correctly so overall I'm leaning towards > the patch > > I already posted. > > > > What do you think? > > > > - R. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -- > MST > From changquing.tang at hp.com Thu Aug 30 07:12:48 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 30 Aug 2007 14:12:48 -0000 Subject: [ofa-general] RE: local QP operation error after long run In-Reply-To: <20070830135918.GA9252@mellanox.co.il> References: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> <20070830135918.GA9252@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA8403022BC6DA@G3W0634.americas.hpqcorp.net> For us, the num_sge is always 1. "0-size WRs", do you mean num_sge=0, or sg.length=0 ? does the bug fixed by Roland exist in OFED 1.1, but not in OFED 1.2 ? --CQ > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Thursday, August 30, 2007 8:59 AM > To: Tang, Changqing > Cc: Roland Dreier; Michael S. Tsirkin; general at lists.openfabrics.org > Subject: Re: local QP operation error after long run > > > Apparently, an inline work request is malformed. > Yes, this could indicate memory corruption. > OTOH, I see this in commit history: > commit c2623102f3e38e7684e435b77403d16dc6ddb585 > Author: Roland Dreier > Date: Mon Nov 28 21:21:08 2005 +0000 > > Fix inline sends with no gather entries > > Fix bug in handling send requests that have the inline flag set > but do not include any gather entries. > > Signed-off-by: Roland Dreier > > is there a chance you are posting some 0-size WRs? > If yes, just clearing the inline flag will fix it. > > > > Quoting Tang, Changqing : > Subject: local QP operation error after long run > > > HI, > I have an ISV application running for nearly three > hours, and then it has following error from libibverbs.so: > > local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index > 236192) > [ 0] 00440446 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index > 236193) > [ 0] 00440442 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > Can you guys indicate what the possible reason is ? this is > an OFED 1.1 system. Could it be a memory corruption ? > > Thanks > --CQ, HP-MPI > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Roland > > Dreier > > Sent: Wednesday, August 29, 2007 9:50 PM > > To: Sasha Khapyorsky > > Cc: general at lists.openfabrics.org > > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian > > 64-bitarchs > > > > > It looks that using uint32_t for addr in set_bit() function is > > sufficient > fix. But for ppc64 this means that new OpenSM > will break > > with old > kernels, probably we will need to put some ugly > #ifdef in > > > osm_vendor_ibumad.c... > > > > Yes, that's a pain. Another possibility is to declare that the > > declaration of the registration request should have been > > > > long method_mask[16 / sizeof (long)]; > > > > and just add a compat_ioctl method to the ib_umad module to > handle the > > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that > followed the old > > ib_user_mad.h file correctly so overall I'm leaning towards > the patch > > I already posted. > > > > What do you think? > > > > - R. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -- > MST > From mst at dev.mellanox.co.il Thu Aug 30 07:22:49 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 17:22:49 +0300 Subject: [ofa-general] Re: local QP operation error after long run In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA8403022BC6DA@G3W0634.americas.hpqcorp.net> References: <20070830135918.GA9252@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA8403022BC6DA@G3W0634.americas.hpqcorp.net> Message-ID: <20070830142249.GA9552@mellanox.co.il> > Quoting Tang, Changqing : > Subject: RE: local QP operation error after long run > > > For us, the num_sge is always 1. > > "0-size WRs", do you mean num_sge=0, or sg.length=0 ? num_sge = 0 is the case that was fixed. sg.length=0 is invalid in any version of OFED. > does the bug fixed by Roland exist in OFED 1.1, but not in OFED 1.2 ? yes Quoting Tang, Changqing : Subject: RE: local QP operation error after long run For us, the num_sge is always 1. "0-size WRs", do you mean num_sge=0, or sg.length=0 ? does the bug fixed by Roland exist in OFED 1.1, but not in OFED 1.2 ? --CQ > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Thursday, August 30, 2007 8:59 AM > To: Tang, Changqing > Cc: Roland Dreier; Michael S. Tsirkin; general at lists.openfabrics.org > Subject: Re: local QP operation error after long run > > > Apparently, an inline work request is malformed. > Yes, this could indicate memory corruption. > OTOH, I see this in commit history: > commit c2623102f3e38e7684e435b77403d16dc6ddb585 > Author: Roland Dreier > Date: Mon Nov 28 21:21:08 2005 +0000 > > Fix inline sends with no gather entries > > Fix bug in handling send requests that have the inline flag set > but do not include any gather entries. > > Signed-off-by: Roland Dreier > > is there a chance you are posting some 0-size WRs? > If yes, just clearing the inline flag will fix it. > > > > Quoting Tang, Changqing : > Subject: local QP operation error after long run > > > HI, > I have an ISV application running for nearly three > hours, and then it has following error from libibverbs.so: > > local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index > 236192) > [ 0] 00440446 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index > 236193) > [ 0] 00440442 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > Can you guys indicate what the possible reason is ? this is > an OFED 1.1 system. Could it be a memory corruption ? > > Thanks > --CQ, HP-MPI > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Roland > > Dreier > > Sent: Wednesday, August 29, 2007 9:50 PM > > To: Sasha Khapyorsky > > Cc: general at lists.openfabrics.org > > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian > > 64-bitarchs > > > > > It looks that using uint32_t for addr in set_bit() function is > > sufficient > fix. But for ppc64 this means that new OpenSM > will break > > with old > kernels, probably we will need to put some ugly > #ifdef in > > > osm_vendor_ibumad.c... > > > > Yes, that's a pain. Another possibility is to declare that the > > declaration of the registration request should have been > > > > long method_mask[16 / sizeof (long)]; > > > > and just add a compat_ioctl method to the ib_umad module to > handle the > > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that > followed the old > > ib_user_mad.h file correctly so overall I'm leaning towards > the patch > > I already posted. > > > > What do you think? > > > > - R. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -- > MST > -- MST From sashak at voltaire.com Thu Aug 30 07:35:47 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Thu, 30 Aug 2007 17:35:47 +0300 Subject: [ofa-general] ib_umad method mask problems on big-endian 64-bit archs In-Reply-To: References: <20070822190519.GD1397@sashak.voltaire.com> Message-ID: <20070830143547.GM7140@sashak.voltaire.com> On 19:50 Wed 29 Aug , Roland Dreier wrote: > > > It looks that using uint32_t for addr in set_bit() function is sufficient > > fix. But for ppc64 this means that new OpenSM will break with old > > kernels, probably we will need to put some ugly #ifdef in > > osm_vendor_ibumad.c... > > Yes, that's a pain. Another possibility is to declare that the > declaration of the registration request should have been > > long method_mask[16 / sizeof (long)]; > > and just add a compat_ioctl method to the ib_umad module to handle the > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that followed the old > ib_user_mad.h file correctly so overall I'm leaning towards the patch > I already posted. > > What do you think? Under management tree only osm_vendor_ibumad.c actually uses method_mask, so switching to long in struct ib_user_mad_reg_req will break nothing here. Do we have another another user_umad users (OFA or another known, where switch could be painful)? If not, I will prefer this way instead of keeping two OpenSMs for ppc64. Sasha From changquing.tang at hp.com Thu Aug 30 07:56:09 2007 From: changquing.tang at hp.com (Tang, Changqing) Date: Thu, 30 Aug 2007 14:56:09 -0000 Subject: [ofa-general] RE: local QP operation error after long run In-Reply-To: <20070830135918.GA9252@mellanox.co.il> References: <349DCDA352EACF42A0C49FA6DCEA8403022BC5BC@G3W0634.americas.hpqcorp.net> <20070830135918.GA9252@mellanox.co.il> Message-ID: <349DCDA352EACF42A0C49FA6DCEA8403022BC7BC@G3W0634.americas.hpqcorp.net> >From our code, num_sge=1 all the time. But from error message, can you figure out num_sge is actually 0 ? and with inline flag ? --CQ > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Thursday, August 30, 2007 8:59 AM > To: Tang, Changqing > Cc: Roland Dreier; Michael S. Tsirkin; general at lists.openfabrics.org > Subject: Re: local QP operation error after long run > > > Apparently, an inline work request is malformed. > Yes, this could indicate memory corruption. > OTOH, I see this in commit history: > commit c2623102f3e38e7684e435b77403d16dc6ddb585 > Author: Roland Dreier > Date: Mon Nov 28 21:21:08 2005 +0000 > > Fix inline sends with no gather entries > > Fix bug in handling send requests that have the inline flag set > but do not include any gather entries. > > Signed-off-by: Roland Dreier > > is there a chance you are posting some 0-size WRs? > If yes, just clearing the inline flag will fix it. > > > > Quoting Tang, Changqing : > Subject: local QP operation error after long run > > > HI, > I have an ISV application running for nearly three > hours, and then it has following error from libibverbs.so: > > local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index > 236192) > [ 0] 00440446 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index > 236193) > [ 0] 00440442 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > Can you guys indicate what the possible reason is ? this is > an OFED 1.1 system. Could it be a memory corruption ? > > Thanks > --CQ, HP-MPI > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Roland > > Dreier > > Sent: Wednesday, August 29, 2007 9:50 PM > > To: Sasha Khapyorsky > > Cc: general at lists.openfabrics.org > > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian > > 64-bitarchs > > > > > It looks that using uint32_t for addr in set_bit() function is > > sufficient > fix. But for ppc64 this means that new OpenSM > will break > > with old > kernels, probably we will need to put some ugly > #ifdef in > > > osm_vendor_ibumad.c... > > > > Yes, that's a pain. Another possibility is to declare that the > > declaration of the registration request should have been > > > > long method_mask[16 / sizeof (long)]; > > > > and just add a compat_ioctl method to the ib_umad module to > handle the > > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that > followed the old > > ib_user_mad.h file correctly so overall I'm leaning towards > the patch > > I already posted. > > > > What do you think? > > > > - R. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -- > MST > From mst at dev.mellanox.co.il Thu Aug 30 08:10:59 2007 From: mst at dev.mellanox.co.il (Michael S. Tsirkin) Date: Thu, 30 Aug 2007 18:10:59 +0300 Subject: [ofa-general] Re: local QP operation error after long run In-Reply-To: <349DCDA352EACF42A0C49FA6DCEA8403022BC7BC@G3W0634.americas.hpqcorp.net> References: <20070830135918.GA9252@mellanox.co.il> <349DCDA352EACF42A0C49FA6DCEA8403022BC7BC@G3W0634.americas.hpqcorp.net> Message-ID: <20070830151059.GB9552@mellanox.co.il> It seems inline flag was set in the WR. That's all I know. Quoting Tang, Changqing : Subject: RE: local QP operation error after long run >From our code, num_sge=1 all the time. But from error message, can you figure out num_sge is actually 0 ? and with inline flag ? --CQ > -----Original Message----- > From: Michael S. Tsirkin [mailto:mst at dev.mellanox.co.il] > Sent: Thursday, August 30, 2007 8:59 AM > To: Tang, Changqing > Cc: Roland Dreier; Michael S. Tsirkin; general at lists.openfabrics.org > Subject: Re: local QP operation error after long run > > > Apparently, an inline work request is malformed. > Yes, this could indicate memory corruption. > OTOH, I see this in commit history: > commit c2623102f3e38e7684e435b77403d16dc6ddb585 > Author: Roland Dreier > Date: Mon Nov 28 21:21:08 2005 +0000 > > Fix inline sends with no gather entries > > Fix bug in handling send requests that have the inline flag set > but do not include any gather entries. > > Signed-off-by: Roland Dreier > > is there a chance you are posting some 0-size WRs? > If yes, just clearing the inline flag will fix it. > > > > Quoting Tang, Changqing : > Subject: local QP operation error after long run > > > HI, > I have an ISV application running for nearly three > hours, and then it has following error from libibverbs.so: > > local QP operation err (QPN 440446, WQE @ 00000103, CQN 10008c, index > 236192) > [ 0] 00440446 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > local QP operation err (QPN 440442, WQE @ 00000103, CQN 10008c, index > 236193) > [ 0] 00440442 > [ 4] 00000000 > [ 8] 00000000 > [ c] 00000000 > [10] 026f0000 > [14] 00000000 > [18] 00000103 > [1c] ff100000 > > Can you guys indicate what the possible reason is ? this is > an OFED 1.1 system. Could it be a memory corruption ? > > Thanks > --CQ, HP-MPI > > > > > -----Original Message----- > > From: general-bounces at lists.openfabrics.org > > [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Roland > > Dreier > > Sent: Wednesday, August 29, 2007 9:50 PM > > To: Sasha Khapyorsky > > Cc: general at lists.openfabrics.org > > Subject: Re: [ofa-general] ib_umad method mask problems on > big-endian > > 64-bitarchs > > > > > It looks that using uint32_t for addr in set_bit() function is > > sufficient > fix. But for ppc64 this means that new OpenSM > will break > > with old > kernels, probably we will need to put some ugly > #ifdef in > > > osm_vendor_ibumad.c... > > > > Yes, that's a pain. Another possibility is to declare that the > > declaration of the registration request should have been > > > > long method_mask[16 / sizeof (long)]; > > > > and just add a compat_ioctl method to the ib_umad module to > handle the > > broken case of 32-bit big endian userspace on a 64-bit kernel. > > However that breaks 64-bit big endian userspace that > followed the old > > ib_user_mad.h file correctly so overall I'm leaning towards > the patch > > I already posted. > > > > What do you think? > > > > - R. > > _______________________________________________ > > general mailing list > > general at lists.openfabrics.org > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > -- > MST > -- MST From kliteyn at dev.mellanox.co.il Thu Aug 30 08:39:47 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Thu, 30 Aug 2007 18:39:47 +0300 Subject: [ofa-general] [PATCH] osm: QoS parser - fixing yacc command Message-ID: <46D6E4C3.80201@dev.mellanox.co.il> Fixing bison command to more general yacc syntax Signed-off-by: Yevgeny Kliteynik --- opensm/opensm/Makefile.am | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index abfa913..4ab7227 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -60,7 +60,8 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h - $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ + $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm_qos_parser.y + mv -f y.tab.h $(srcdir)/../include/opensm/osm_qos_parser_y.h osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h $(LEX) -P__qos_parser_ -o$(srcdir)/osm_qos_parser_l.c $(srcdir)/osm_qos_parser.l -- 1.5.1.4 From RAISCH at de.ibm.com Thu Aug 30 08:48:06 2007 From: RAISCH at de.ibm.com (Christoph Raisch) Date: Thu, 30 Aug 2007 17:48:06 +0200 Subject: [ofa-general] [PATCH] Remove verbose debug from ib_echa In-Reply-To: <20070829134253.GB5289@kryten> Message-ID: Hi Anton, general-bounces at lists.openfabrics.org wrote on 29.08.2007 15:42:53: > > Hi, > > While debugging an ib_ehca issue I ended up with raw hcalls in dmesg: > > PU0003 EHCA_ERR:ehca_plpar_hcall9 opcode=168 ret=ffffffffffffffd3 > arg1=103d001403000002 arg2=203d0000000000a1 arg3=8a40000000000000 > arg4=5a5b4c000 arg5=0 arg6=0 arg7=0 arg8=0 arg9=0 out1=0 out2=0 > out3=0 out4=2 out5=0 out6=800000000005f254 out7=0 out8=0 out9=0 > We don't think this is the right approach to just remove these warnings, they are there for a good reason. It's the same place used by all mainline operatins which return good return codes in "normal life". Startup is the only time when this is a bit too chatty. At the moment ib_mad just breaks the complete adapter if one port is not active. I would suggest to define IB startup sequence which only uses ports that are active in ib_mad instead. Gruss / Regards Christoph Raisch From rvm at obsidianresearch.com Thu Aug 30 08:48:12 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Thu, 30 Aug 2007 09:48:12 -0600 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups Message-ID: <20070830154812.GB5617@obsidianresearch.com> Hi Sasha, This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work through IB routers. Signed-off-by: Rolf Manderscheid --- diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 46ee429..fc000b1 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, mc_rec.pkey = pkey; mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ mc_rec.scope_state = ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, From RAISCH at de.ibm.com Thu Aug 30 08:51:40 2007 From: RAISCH at de.ibm.com (Christoph Raisch) Date: Thu, 30 Aug 2007 17:51:40 +0200 Subject: [ofa-general] [PATCH] Set props->page_size_cap in ehca In-Reply-To: <20070829152817.GC5289@kryten> Message-ID: > When inserting the ib_srp module, the ehca driver spits out an error: > > ehca D.001.DQDXYCB-P1-C9: PU0007 EHCA_ERR:ehca_alloc_fmr unsupported > fmr_attr->page_shift=9 > > The following code in ib_srp.c is causing it: > > /* > * Use the smallest page size supported by the HCA, down to a > * minimum of 512 bytes (which is the smallest sector that a > * SCSI command will ever carry). > */ > srp_dev->fmr_page_shift = max(9, ffs(dev_attr->page_size_cap) - 1); > > Looks like we should be setting props->page_size_cap. > > Compile tested only, I dont have any IB attached storage. > > Signed-off-by: Anton Blanchard > --- > > diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c > b/drivers/infiniband/hw/ehca/ehca_hca.c > index fc19ef9..9ca2543 100644 > --- a/drivers/infiniband/hw/ehca/ehca_hca.c > +++ b/drivers/infiniband/hw/ehca/ehca_hca.c > @@ -77,6 +77,7 @@ int ehca_query_device(struct ib_device *ibdev, > struct ib_device_attr *props) > } > > memset(props, 0, sizeof(struct ib_device_attr)); > + props->page_size_cap = EHCA_PAGESIZE; ...yes you're on the right track, good finding! we'll have to change this even further, because ehca2 supports additional pagesizes. Patch will follow. Gruss / Regards Christoph Raisch From hal.rosenstock at gmail.com Thu Aug 30 09:01:57 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 12:01:57 -0400 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: <20070830154812.GB5617@obsidianresearch.com> References: <20070830154812.GB5617@obsidianresearch.com> Message-ID: On 8/30/07, Rolf Manderscheid wrote: > Hi Sasha, > > This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work > through IB routers. > > Signed-off-by: Rolf Manderscheid > > --- > > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > index 46ee429..fc000b1 100644 > --- a/opensm/opensm/osm_prtn.c > +++ b/opensm/opensm/osm_prtn.c > @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > mc_rec.pkey = pkey; > mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ > mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; > - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); > + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); Shouldn't this be based on whether the group is local scope or not ? -- Hal > /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ > mc_rec.scope_state = > ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > From HNGUYEN at de.ibm.com Thu Aug 30 09:03:00 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 30 Aug 2007 18:03:00 +0200 Subject: [ofa-general] Re: [PATCH] Export ehca module parameters in sysfs In-Reply-To: <20070829174301.GD24003@kryten> Message-ID: Hi Anton, Thanks for this catch. Nam Acked-by: Hoang-Nam Nguyen > > Hi, > > At the moment the ehca module parameters are not exported in sysfs. > Export them with 0444 permissions. > > Signed-off-by: Anton Blanchard > --- > > diff --git a/drivers/infiniband/hw/ehca/ehca_main.c > b/drivers/infiniband/hw/ehca/ehca_main.c > index 99036b6..db041df 100644 > --- a/drivers/infiniband/hw/ehca/ehca_main.c > +++ b/drivers/infiniband/hw/ehca/ehca_main.c > @@ -65,16 +65,16 @@ int ehca_static_rate = -1; > int ehca_scaling_code = 0; > int ehca_mr_largepage = 0; > > -module_param_named(open_aqp1, ehca_open_aqp1, int, 0); > -module_param_named(debug_level, ehca_debug_level, int, 0); > -module_param_named(hw_level, ehca_hw_level, int, 0); > -module_param_named(nr_ports, ehca_nr_ports, int, 0); > -module_param_named(use_hp_mr, ehca_use_hp_mr, int, 0); > -module_param_named(port_act_time, ehca_port_act_time, int, 0); > -module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, 0); > -module_param_named(static_rate, ehca_static_rate, int, 0); > -module_param_named(scaling_code, ehca_scaling_code, int, 0); > -module_param_named(mr_largepage, ehca_mr_largepage, int, 0); > +module_param_named(open_aqp1, ehca_open_aqp1, int, S_IRUGO); > +module_param_named(debug_level, ehca_debug_level, int, S_IRUGO); > +module_param_named(hw_level, ehca_hw_level, int, S_IRUGO); > +module_param_named(nr_ports, ehca_nr_ports, int, S_IRUGO); > +module_param_named(use_hp_mr, ehca_use_hp_mr, int, S_IRUGO); > +module_param_named(port_act_time, ehca_port_act_time, int, S_IRUGO); > +module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, S_IRUGO); > +module_param_named(static_rate, ehca_static_rate, int, S_IRUGO); > +module_param_named(scaling_code, ehca_scaling_code, int, S_IRUGO); > +module_param_named(mr_largepage, ehca_mr_largepage, int, S_IRUGO); > > MODULE_PARM_DESC(open_aqp1, > "AQP1 on startup (0: no (default), 1: yes)"); From HNGUYEN at de.ibm.com Thu Aug 30 09:04:39 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 30 Aug 2007 18:04:39 +0200 Subject: [ofa-general] Re: [PATCH] Export ehca module parameters in sysfs In-Reply-To: <20070829174301.GD24003@kryten> Message-ID: Hi Anton, Thanks for this patch. Nam Acked-by: Hoang-Nam Nguyen > Hi, > > At the moment the ehca module parameters are not exported in sysfs. > Export them with 0444 permissions. > > Signed-off-by: Anton Blanchard > --- > > diff --git a/drivers/infiniband/hw/ehca/ehca_main.c > b/drivers/infiniband/hw/ehca/ehca_main.c > index 99036b6..db041df 100644 > --- a/drivers/infiniband/hw/ehca/ehca_main.c > +++ b/drivers/infiniband/hw/ehca/ehca_main.c > @@ -65,16 +65,16 @@ int ehca_static_rate = -1; > int ehca_scaling_code = 0; > int ehca_mr_largepage = 0; > > -module_param_named(open_aqp1, ehca_open_aqp1, int, 0); > -module_param_named(debug_level, ehca_debug_level, int, 0); > -module_param_named(hw_level, ehca_hw_level, int, 0); > -module_param_named(nr_ports, ehca_nr_ports, int, 0); > -module_param_named(use_hp_mr, ehca_use_hp_mr, int, 0); > -module_param_named(port_act_time, ehca_port_act_time, int, 0); > -module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, 0); > -module_param_named(static_rate, ehca_static_rate, int, 0); > -module_param_named(scaling_code, ehca_scaling_code, int, 0); > -module_param_named(mr_largepage, ehca_mr_largepage, int, 0); > +module_param_named(open_aqp1, ehca_open_aqp1, int, S_IRUGO); > +module_param_named(debug_level, ehca_debug_level, int, S_IRUGO); > +module_param_named(hw_level, ehca_hw_level, int, S_IRUGO); > +module_param_named(nr_ports, ehca_nr_ports, int, S_IRUGO); > +module_param_named(use_hp_mr, ehca_use_hp_mr, int, S_IRUGO); > +module_param_named(port_act_time, ehca_port_act_time, int, S_IRUGO); > +module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, S_IRUGO); > +module_param_named(static_rate, ehca_static_rate, int, S_IRUGO); > +module_param_named(scaling_code, ehca_scaling_code, int, S_IRUGO); > +module_param_named(mr_largepage, ehca_mr_largepage, int, S_IRUGO); > > MODULE_PARM_DESC(open_aqp1, > "AQP1 on startup (0: no (default), 1: yes)"); From weiny2 at llnl.gov Thu Aug 30 09:24:35 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Thu, 30 Aug 2007 09:24:35 -0700 Subject: [ofa-general] [PATCH] infiniband-diags/Makefile.am: clean up extra directives Message-ID: <20070830092435.0f418956.weiny2@llnl.gov> >From 8da03fd7526e0879404425609abea7bc2da0d7f0 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 30 Aug 2007 09:22:29 -0700 Subject: [PATCH] infiniband-diags/Makefile.am: clean up extra directives Signed-off-by: Ira K. Weiny --- infiniband-diags/Makefile.am | 58 ------------------------------------------ 1 files changed, 0 insertions(+), 58 deletions(-) diff --git a/infiniband-diags/Makefile.am b/infiniband-diags/Makefile.am index fcaa29e..f6b6292 100644 --- a/infiniband-diags/Makefile.am +++ b/infiniband-diags/Makefile.am @@ -34,104 +34,46 @@ sbin_SCRIPTS = scripts/ibcheckerrs scripts/ibchecknet scripts/ibchecknode \ src_ibaddr_SOURCES = src/ibaddr.c src/ibdiag_common.c src_ibaddr_CFLAGS = -Wall $(DBGFLAGS) -# src_ibaddr_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibaddr_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibnetdiscover_SOURCES = src/ibnetdiscover.c src/grouping.c src/ibdiag_common.c src_ibnetdiscover_CFLAGS = -Wall $(DBGFLAGS) -# src_ibnetdiscover_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibnetdiscover_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibping_SOURCES = src/ibping.c src/ibdiag_common.c src_ibping_CFLAGS = -Wall $(DBGFLAGS) -# src_ibping_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibping_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibportstate_SOURCES = src/ibportstate.c src/ibdiag_common.c src_ibportstate_CFLAGS = -Wall $(DBGFLAGS) -# src_ibportstate_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibportstate_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibroute_SOURCES = src/ibroute.c src/ibdiag_common.c src_ibroute_CFLAGS = -Wall $(DBGFLAGS) -# src_ibroute_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibroute_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibstat_SOURCES = src/ibstat.c src_ibstat_CFLAGS = -Wall $(DBGFLAGS) -# src_ibstat_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la -# src_ibstat_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibsysstat_SOURCES = src/ibsysstat.c src/ibdiag_common.c src_ibsysstat_CFLAGS = -Wall $(DBGFLAGS) -# src_ibsysstat_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibsysstat_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_ibtracert_SOURCES = src/ibtracert.c src/ibdiag_common.c src_ibtracert_CFLAGS = -Wall $(DBGFLAGS) -# src_ibtracert_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_ibtracert_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_perfquery_SOURCES = src/perfquery.c src/ibdiag_common.c src_perfquery_CFLAGS = -Wall $(DBGFLAGS) -# src_perfquery_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_perfquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_sminfo_SOURCES = src/sminfo.c src/ibdiag_common.c src_sminfo_CFLAGS = -Wall $(DBGFLAGS) -# src_sminfo_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_sminfo_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_smpdump_SOURCES = src/smpdump.c src_smpdump_CFLAGS = -Wall $(DBGFLAGS) -# src_smpdump_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_smpdump_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_smpquery_SOURCES = src/smpquery.c src/ibdiag_common.c src_smpquery_CFLAGS = -Wall $(DBGFLAGS) -# src_smpquery_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -# src_smpquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_saquery_SOURCES = src/saquery.c src/ibdiag_common.c src_saquery_CFLAGS = -Wall -DOSM_VENDOR_INTF_OPENIB -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -# src_saquery_LDADD = $(libdir)/libibmad.la \ -# $(libdir)/libosmvendor.la \ -# $(libdir)/libopensm.la \ -# $(libdir)/libosmcomp.la -#src_saquery_LDADD = $(libdir)/libopensm.la \ -# $(libdir)/libosmcomp.la \ -# $(libdir)/libosmvendor.la src_saquery_LDFLAGS = -Wl,--rpath -Wl,$(libdir) src_vendstat_SOURCES = src/vendstat.c src/ibdiag_common.c src_vendstat_CFLAGS = -Wall $(DBGFLAGS) -#src_vendstat_LDADD = $(libdir)/libibcommon.la \ -# $(libdir)/libibumad.la \ -# $(libdir)/libibmad.la -#src_vendstat_LDFLAGS = -Wl,--rpath -Wl,$(libdir) #src_mcm_rereg_test_SOURCES = src/mcm_rereg_test.c #src_mcm_rereg_test_CFLAGS = -Wall $(DBGFLAGS) -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-infiniband-diags-Makefile.am-clean-up-extra-directi.patch Type: application/octet-stream Size: 4725 bytes Desc: not available URL: From rvm at obsidianresearch.com Thu Aug 30 09:44:53 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Thu, 30 Aug 2007 10:44:53 -0600 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: References: <20070830154812.GB5617@obsidianresearch.com> Message-ID: <20070830164453.GA5680@obsidianresearch.com> On Thu, Aug 30, 2007 at 12:01:57PM -0400, Hal Rosenstock wrote: > On 8/30/07, Rolf Manderscheid wrote: > > Hi Sasha, > > > > This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work > > through IB routers. > > > > Signed-off-by: Rolf Manderscheid > > > > --- > > > > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > > index 46ee429..fc000b1 100644 > > --- a/opensm/opensm/osm_prtn.c > > +++ b/opensm/opensm/osm_prtn.c > > @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > > mc_rec.pkey = pkey; > > mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ > > mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; > > - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); > > + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); > > Shouldn't this be based on whether the group is local scope or not ? If the group has local scope then the hop limit is irrelevant, packets will get dropped by the router anyway. I certainly don't object to making the hop limit conditional on scope if that's the preference. Rolf From hal.rosenstock at gmail.com Thu Aug 30 10:25:43 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Thu, 30 Aug 2007 13:25:43 -0400 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: <20070830164453.GA5680@obsidianresearch.com> References: <20070830154812.GB5617@obsidianresearch.com> <20070830164453.GA5680@obsidianresearch.com> Message-ID: On 8/30/07, Rolf Manderscheid wrote: > On Thu, Aug 30, 2007 at 12:01:57PM -0400, Hal Rosenstock wrote: > > On 8/30/07, Rolf Manderscheid wrote: > > > Hi Sasha, > > > > > > This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work > > > through IB routers. > > > > > > Signed-off-by: Rolf Manderscheid > > > > > > --- > > > > > > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > > > index 46ee429..fc000b1 100644 > > > --- a/opensm/opensm/osm_prtn.c > > > +++ b/opensm/opensm/osm_prtn.c > > > @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > > > mc_rec.pkey = pkey; > > > mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ > > > mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; > > > - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); > > > + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); > > > > Shouldn't this be based on whether the group is local scope or not ? > > If the group has local scope then the hop limit is irrelevant, packets will get > dropped by the router anyway. I certainly don't object to making the hop limit > conditional on scope if that's the preference. Looking at the PathRecord code, it only checks whether a DGID was requested or not and then sets the hop limit only being dependent on whether ROUTER_EXP was defined or not. Should we follow continue to follow this or make PathRecord responses consistent with this ? -- Hal > > Rolf > From rvm at obsidianresearch.com Thu Aug 30 10:55:42 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Thu, 30 Aug 2007 11:55:42 -0600 Subject: [ofa-general] [PATCH] opensm: set scope according to configuration for ipoib multicast groups Message-ID: <20070830175542.GA5741@obsidianresearch.com> Hi Sasha, opensm creates two multicast groups for ipoib. The group corresponding to 255.255.255.255 results in an MGID with the scope as specified by the configuration. The one MGID for 0.0.0.1 ends up with local scope irrespective of the configuration. If ipoib is configured with global scope, then its join of the latter group never succeeds. The patch below sets the scope for both groups according to the configuration. Signed-off-by: Rolf Manderscheid ---- diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 46ee429..17348bc 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -198,7 +198,6 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_mgrp_t *p_mgrp = NULL; osm_sa_t *p_sa = &p_subn->p_osm->sa; ib_api_status_t status = IB_SUCCESS; - uint8_t ts_scope; pkey = p->pkey | cl_hton16(0x8000); @@ -239,9 +238,11 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, mc_rec.mgid = osm_ts_ipoib_mgid; memcpy(&mc_rec.mgid.raw[4], &pkey, sizeof(pkey)); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ - ts_scope = ib_mgid_get_scope(&osm_ts_ipoib_mgid); /* get scope from MGID */ mc_rec.scope_state = - ib_member_set_scope_state(ts_scope, MC_FULL_MEMBER); + ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, + MC_FULL_MEMBER); + ib_mgid_set_scope(&mc_rec.mgid, scope ? scope : OSM_DEFAULT_MGRP_SCOPE); + status = osm_mcmr_rcv_find_or_create_new_mgrp(&p_sa->mcmr_rcv, comp_mask, &mc_rec, &p_mgrp); From HNGUYEN at de.ibm.com Thu Aug 30 13:03:34 2007 From: HNGUYEN at de.ibm.com (Hoang-Nam Nguyen) Date: Thu, 30 Aug 2007 22:03:34 +0200 Subject: [ofa-general] Re: [PATCH] Quieten some error messages in ehca In-Reply-To: <20070829160535.GA24003@kryten> Message-ID: Thanks Nam Acked-by: Hoang-Nam Nguyen Anton Blanchard wrote on 29.08.2007 18:05:35: > > Hi, > > ehca spits out a lot of debugging information. I had to look closely to > see the "Port 1 is not active" message within all the debug: > > eHCA Infiniband Device Driver (Rel.: SVNEHCA_0022) > eHCA scaling code enabled > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_define_sqp Port 1 is > not active. > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_create_qp > ehca_define_sqp() failed rc=ffffffffffffffff > ib_mad: Couldn't create ib_mad QP1 > ib_mad: Couldn't open ehca0 port 1 > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr unsupported > fmr_attr->page_shift=9 > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr > rc=ffffffffffffffea pd=c000000b4b5b2420 mr_access_flags=7 > fmr_attr=c0000005afd37394 > fmr_create failed for FMR 0 > > Remove a few debug statements so that things are clearer: > > eHCA Infiniband Device Driver (Rel.: SVNEHCA_0022) > eHCA scaling code enabled > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_define_sqp Port 1 is > not active. > ib_mad: Couldn't create ib_mad QP1 > ib_mad: Couldn't open ehca0 port 1 > ehca D.001.DQDXYCB-P1-C9: PU0006 EHCA_ERR:ehca_alloc_fmr unsupported > fmr_attr->page_shift=9 > fmr_create failed for FMR 0 > > Signed-off-by: Anton Blanchard > --- > > diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c > b/drivers/infiniband/hw/ehca/ehca_mrmw.c > index c1b868b..57e3420 100644 > --- a/drivers/infiniband/hw/ehca/ehca_mrmw.c > +++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c > @@ -848,10 +848,6 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd, > alloc_fmr_exit1: > ehca_mr_delete(e_fmr); > alloc_fmr_exit0: > - if (IS_ERR(ib_fmr)) > - ehca_err(pd->device, "rc=%lx pd=%p mr_access_flags=%x " > - "fmr_attr=%p", PTR_ERR(ib_fmr), pd, > - mr_access_flags, fmr_attr); > return ib_fmr; > } /* end ehca_alloc_fmr() */ > > diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c > b/drivers/infiniband/hw/ehca/ehca_qp.c > index b178cba..448f6b3 100644 > --- a/drivers/infiniband/hw/ehca/ehca_qp.c > +++ b/drivers/infiniband/hw/ehca/ehca_qp.c > @@ -712,8 +712,6 @@ static struct ehca_qp *internal_create_qp( > if (qp_type == IB_QPT_GSI) { > h_ret = ehca_define_sqp(shca, my_qp, init_attr); > if (h_ret != H_SUCCESS) { > - ehca_err(pd->device, "ehca_define_sqp() failed rc=%lx", > - h_ret); > ret = ehca2ib_return_code(h_ret); > goto create_qp_exit4; > } > From rvm at obsidianresearch.com Thu Aug 30 16:08:34 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Thu, 30 Aug 2007 17:08:34 -0600 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: References: <20070830154812.GB5617@obsidianresearch.com> <20070830164453.GA5680@obsidianresearch.com> Message-ID: <20070830230834.GA5756@obsidianresearch.com> On Thu, Aug 30, 2007 at 01:25:43PM -0400, Hal Rosenstock wrote: > On 8/30/07, Rolf Manderscheid wrote: > > On Thu, Aug 30, 2007 at 12:01:57PM -0400, Hal Rosenstock wrote: > > > On 8/30/07, Rolf Manderscheid wrote: > > > > Hi Sasha, > > > > > > > > This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work > > > > through IB routers. > > > > > > > > Signed-off-by: Rolf Manderscheid > > > > > > > > --- > > > > > > > > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > > > > index 46ee429..fc000b1 100644 > > > > --- a/opensm/opensm/osm_prtn.c > > > > +++ b/opensm/opensm/osm_prtn.c > > > > @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > > > > mc_rec.pkey = pkey; > > > > mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ > > > > mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; > > > > - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); > > > > + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); > > > > > > Shouldn't this be based on whether the group is local scope or not ? > > > > If the group has local scope then the hop limit is irrelevant, packets will get > > dropped by the router anyway. I certainly don't object to making the hop limit > > conditional on scope if that's the preference. > > Looking at the PathRecord code, it only checks whether a DGID was > requested or not and then sets the hop limit only being dependent on > whether ROUTER_EXP was defined or not. Should we follow continue to > follow this or make PathRecord responses consistent with this ? Isn't the plan to incorporate the ROUTER_EXP code eventually? I would rather not add any more #ifdefs if we can avoid it. I think your suggestion of setting the hop limit according to the scope of the multicast group seems like the better option to me ... I'll post a new patch. Are you OK with the other patch to fix the scope of the second MGID? If so, I'll base the new patch on that. Rolf From chargahagha at gmail.com Thu Aug 30 16:11:29 2007 From: chargahagha at gmail.com (Charity Organisation) Date: Thu, 30 Aug 2007 16:11:29 -0700 Subject: [ofa-general] Work with us Message-ID: Charity Organization, London United Kingdom. +44 70457 36859 Dear Sir/Madam, Top of the day to you, this is non-governmental organisation registered in the United Kingdom with a primary aim to cater for the orphanage, poor and less previlleged. We seek International financial accountants in each country globally. These Financial Accountants we seek will not be placed on any type of payment at the end of the month or year, so we seek voluntary workers who will act as our financial accountant in his/her country to receive financial donations from individuals, coporate bodies and registered organizations within his/her country. Should you be interested in working as our International Financial Accountant in your country, kindly get back to us with the information below through e-mail charityorganisationuk at yahoo.co.uk ; 1. Full Names. 2. Age. 3. Contact Address. 4. Zip Code. 5. Phone Number. 6. Occupation. Your duties shall be; i. Receiving financial donations from donators and philantropist in your country either by cash, wire transfer,cheques or in person, and remitting the donated funds to our account here or as would be instructed by the admin department of this organisation. This will not take three hours of your time every month as our donators will be instructed by this office to contact you via e-mail or phone to discuss their means of financial donations with you. Only in some cases if need be, you may be requested to meet them in person. This is very rare, as we will not want to inconvenience you with this offer. Thank you, we look forward to receiving you as our International Financial Accountant in your country and we are happy in your service to reaching out to the poor, orphanage, sick and the needy. Mr. Richard Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrcoulibalyboka at yahoo.fr Thu Aug 30 16:51:43 2007 From: mrcoulibalyboka at yahoo.fr (Mr BOKA COULIBALY) Date: Fri, 31 Aug 2007 00:51:43 +0100 Subject: [ofa-general] ATTN: Sir, Message-ID: <0c943e58103f3049ad0568ab5f79880f@www.c360uk.com> ATTN : Sir, We are representing our corporate associate a member of the contract award committee and 14 project allocation manager of the Department of Minerals and Natural Resources in Cote d'Ivoire West Africa. I am in search of an agent to assist us in the transfer of ( USD$5.5M) and subsequent investment in properties in your country.  You will be required to.(1) Assist in the transfer of the said funds(2) Advise on lucrative areas for investment(3) Assist us in purchase of properties or Profitable Business.If you decide to render your service to us in this regard, 20% of the total sum of USD$5.5M will be for you.  Thank you and God bless as I awaits in anticipation of your fullest co-operation.  Mr BOKA COULIBALYFinancierEmail: mrcoulibalyboka at yahoo.frTo join the party go to http://www.labour.org.uk/join -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvm at obsidianresearch.com Thu Aug 30 17:03:00 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Thu, 30 Aug 2007 18:03:00 -0600 Subject: [ofa-general] [PATCHv2] make ipoib broadcast scope configurable In-Reply-To: References: Message-ID: <20070831000300.GA9327@obsidianresearch.com> An ipoib subnet on an IB fabric that spans multiple IB subnets can't use link-local scope in multicast GIDs. This patch takes the scope from the link level broadcast address of the ipoib device when mapping to an IB multicast address and makes the scope in the link level broadcast address configurable. Since the mapping routines now have the broadcast address available, they can also set the pkey in the MGID. This cleans up some assignments made to the MGID after the mapping. Signed-off-by: Rolf Manderscheid --- I was probably a little overzealous in my application of the one-idea-per-patch rule. Hal pointed out that Roland already suggested a sysfs attribute or ethtool hook to make the scope configurable. This patch replaces the one entitled "use scope from ipoib device link-level broadcast address in MGIDs", I hope changing the title doesn't confuse things. There is at least one blemish: the mapping for the IPv6 all-nodes address is done rather early, so if the broadcast scope is changed, the mapping must also be changed (that's done in the code around the FIXME below). I don't see a way around that except to have ipv6 delay the join of the all-nodes group until after the link is brought up (definitely a separate patch :-) Rolf diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 9ffb998..b3d832c 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -2560,11 +2560,9 @@ static void cma_set_mgid(struct rdma_id_private *id_priv, /* IPv6 address is an SA assigned MGID. */ memcpy(mgid, &sin6->sin6_addr, sizeof *mgid); } else { - ip_ib_mc_map(sin->sin_addr.s_addr, mc_map); + ip_ib_mc_map(sin->sin_addr.s_addr, dev_addr->broadcast, mc_map); if (id_priv->id.ps == RDMA_PS_UDP) mc_map[7] = 0x01; /* Use RDMA CM signature */ - mc_map[8] = ib_addr_get_pkey(dev_addr) >> 8; - mc_map[9] = (unsigned char) ib_addr_get_pkey(dev_addr); *mgid = *(union ib_gid *) (mc_map + 4); } } diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 894b1dc..d2f5bb5 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -1017,6 +1017,43 @@ static ssize_t show_pkey(struct device *dev, } static DEVICE_ATTR(pkey, S_IRUGO, show_pkey, NULL); +static ssize_t show_bcast_scope(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct ipoib_dev_priv *priv = netdev_priv(to_net_dev(dev)); + + return sprintf(buf, "0x%x\n", priv->dev->broadcast[5] & 0xF); +} + +static ssize_t set_bcast_scope(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct ipoib_dev_priv *priv = netdev_priv(to_net_dev(dev)); + int scope; + + if (priv->dev->flags & IFF_UP) + return -EBUSY; + + if (sscanf(buf, "%i", &scope) != 1) + return -EINVAL; + + switch (scope) { + case 0x2: /* link-local */ + case 0x5: /* site-local */ + case 0x8: /* organization-local */ + case 0xE: /* global */ + break; + default: + return -EINVAL; + } + + priv->dev->broadcast[5] &= ~0xF; + priv->dev->broadcast[5] |= scope; + return count; +} +static DEVICE_ATTR(broadcast_scope, S_IWUSR | S_IRUGO, show_bcast_scope, set_bcast_scope); + static ssize_t create_child(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) @@ -1138,6 +1175,8 @@ static struct net_device *ipoib_add_port(const char *format, goto sysfs_failed; if (device_create_file(&priv->dev->dev, &dev_attr_delete_child)) goto sysfs_failed; + if (device_create_file(&priv->dev->dev, &dev_attr_broadcast_scope)) + goto sysfs_failed; return priv->dev; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index aae3670..8088afc 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -808,9 +808,11 @@ void ipoib_mcast_restart_task(struct work_struct *work) memcpy(mgid.raw, mclist->dmi_addr + 4, sizeof mgid); - /* Add in the P_Key */ - mgid.raw[4] = (priv->pkey >> 8) & 0xff; - mgid.raw[5] = priv->pkey & 0xff; + /* FIXME: ipv6 maps the all-nodes multicast group at device creation, + so the mapping can change if the broadcast_scope is changed. If + the ipv6 core can delay joining the all-nodes group until after + the link is brought up, then this can go away: */ + mgid.raw[1] = (mgid.raw[1] & ~0xF) | (priv->dev->broadcast[5] & 0xF); mcast = __ipoib_mcast_find(dev, &mgid); if (!mcast || test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) { diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h index 3ec7d07..03faeb0 100644 --- a/include/net/if_inet6.h +++ b/include/net/if_inet6.h @@ -268,18 +268,21 @@ static inline void ipv6_arcnet_mc_map(const struct in6_addr *addr, char *buf) buf[0] = 0x00; } -static inline void ipv6_ib_mc_map(struct in6_addr *addr, char *buf) +static inline void ipv6_ib_mc_map(const struct in6_addr *addr, + const unsigned char *broadcast, char *buf) { + unsigned char scope = broadcast[5] & 0xF; + buf[0] = 0; /* Reserved */ buf[1] = 0xff; /* Multicast QPN */ buf[2] = 0xff; buf[3] = 0xff; buf[4] = 0xff; - buf[5] = 0x12; /* link local scope */ + buf[5] = 0x10 | scope; /* scope from broadcast address */ buf[6] = 0x60; /* IPv6 signature */ buf[7] = 0x1b; - buf[8] = 0; /* P_Key */ - buf[9] = 0; + buf[8] = broadcast[8]; /* P_Key */ + buf[9] = broadcast[9]; memcpy(buf + 10, addr->s6_addr + 6, 10); } #endif diff --git a/include/net/ip.h b/include/net/ip.h index abf2820..a85342c 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -266,20 +266,22 @@ static inline void ip_eth_mc_map(__be32 naddr, char *buf) * Leave P_Key as 0 to be filled in by driver. */ -static inline void ip_ib_mc_map(__be32 naddr, char *buf) +static inline void ip_ib_mc_map(__be32 naddr, const unsigned char *broadcast, char *buf) { __u32 addr; + unsigned char scope = broadcast[5] & 0xF; + buf[0] = 0; /* Reserved */ buf[1] = 0xff; /* Multicast QPN */ buf[2] = 0xff; buf[3] = 0xff; addr = ntohl(naddr); buf[4] = 0xff; - buf[5] = 0x12; /* link local scope */ + buf[5] = 0x10 | scope; /* scope from broadcast address */ buf[6] = 0x40; /* IPv4 signature */ buf[7] = 0x1b; - buf[8] = 0; /* P_Key */ - buf[9] = 0; + buf[8] = broadcast[8]; /* P_Key */ + buf[9] = broadcast[9]; buf[10] = 0; buf[11] = 0; buf[12] = 0; diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c index 9ab9d53..feb643c 100644 --- a/net/ipv4/arp.c +++ b/net/ipv4/arp.c @@ -214,7 +214,7 @@ int arp_mc_map(__be32 addr, u8 *haddr, struct net_device *dev, int dir) ip_tr_mc_map(addr, haddr); return 0; case ARPHRD_INFINIBAND: - ip_ib_mc_map(addr, haddr); + ip_ib_mc_map(addr, dev->broadcast, haddr); return 0; default: if (dir) { diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index 0358e60..8275625 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -308,7 +308,7 @@ int ndisc_mc_map(struct in6_addr *addr, char *buf, struct net_device *dev, int d ipv6_arcnet_mc_map(addr, buf); return 0; case ARPHRD_INFINIBAND: - ipv6_ib_mc_map(addr, buf); + ipv6_ib_mc_map(addr, dev->broadcast, buf); return 0; default: if (dir) { From swelch at systemfabricworks.com Thu Aug 30 19:46:28 2007 From: swelch at systemfabricworks.com (swelch at systemfabricworks.com) Date: Thu, 30 Aug 2007 21:46:28 -0500 Subject: [ofa-general] [PATCH] infiniband/core: Enable loopback of DR SMP responses from userspace Message-ID: <46D78104.mailJY81GRONO@systemfabricworks.com> The local loopback of a DR SMP response is limited to those that originate at the driver specific SMA implementation as a result of an invocation of the drivers process_mad() function. This patch enables a DR SMP response originating elsewhere to be forwarded/looped back to the local management stack as well. In this case the driver specific process_mad() function does not consume or process the MAD so the original MAD is to be treated like an incoming receive and it must be manually copied to the buffer that is to be handed off the local agent. The stimulus for this change is to provide support for the forwarding of DR SMP responses to the local management stack via the user space MAD library. This will facilitate development of userspace applications utilizing the MTHCA router mode enable driver. Signed-off-by: Steve Welch --- drivers/infiniband/core/mad.c | 4 +++- drivers/infiniband/core/smi.h | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index 6f42877..9ec910b 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -701,7 +701,8 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, } /* Check to post send on QP or process locally */ - if (smi_check_local_smp(smp, device) == IB_SMI_DISCARD) + if (smi_check_local_smp(smp, device) == IB_SMI_DISCARD && + smi_check_local_resp_smp(smp, device) == IB_SMI_DISCARD) goto out; local = kmalloc(sizeof *local, GFP_ATOMIC); @@ -754,6 +755,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, if (port_priv) { mad_priv->mad.mad.mad_hdr.tid = ((struct ib_mad *)smp)->mad_hdr.tid; + memcpy(&mad_priv->mad.mad, smp, sizeof(struct ib_mad)); recv_mad_agent = find_mad_agent(port_priv, &mad_priv->mad.mad); } diff --git a/drivers/infiniband/core/smi.h b/drivers/infiniband/core/smi.h index 1cfc298..d96fc8e 100644 --- a/drivers/infiniband/core/smi.h +++ b/drivers/infiniband/core/smi.h @@ -71,4 +71,18 @@ static inline enum smi_action smi_check_local_smp(struct ib_smp *smp, (smp->hop_ptr == smp->hop_cnt + 1)) ? IB_SMI_HANDLE : IB_SMI_DISCARD); } + +/* + * Return 1 if the SMP response should be handled by the local management stack + */ +static inline enum smi_action smi_check_local_resp_smp(struct ib_smp *smp, + struct ib_device *device) +{ + /* C14-13:3 -- We're at the end of the DR segment of path */ + /* C14-13:4 -- Hop Pointer == 0 -> give to SM */ + return ((device->process_mad && + ib_get_smp_direction(smp) && + !smp->hop_ptr) ? IB_SMI_HANDLE : IB_SMI_DISCARD); +} + #endif /* __SMI_H_ */ From rdreier at cisco.com Thu Aug 30 21:23:20 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 30 Aug 2007 21:23:20 -0700 Subject: [ofa-general] Re: [PATCH] Clean up some error messages in fmr_pool.c In-Reply-To: <20070829133622.GA5289@kryten> (Anton Blanchard's message of "Wed, 29 Aug 2007 08:36:22 -0500") References: <20070829133622.GA5289@kryten> Message-ID: thanks, applied. From rdreier at cisco.com Thu Aug 30 21:27:04 2007 From: rdreier at cisco.com (Roland Dreier) Date: Thu, 30 Aug 2007 21:27:04 -0700 Subject: [ofa-general] Re: [PATCH RFC] iw_cxgb3: Support "iwarp-only" interfaces to avoid 4-tuple conflicts with the host stack. In-Reply-To: <1187905185.5547.13.camel@stevo-desktop> (Steve Wise's message of "Thu, 23 Aug 2007 16:39:45 -0500") References: <1187905185.5547.13.camel@stevo-desktop> Message-ID: > The sysadmin creates "for iwarp use only" alias interfaces of the form > "devname:iw*" where devname is the native interface name (eg eth0) for the > iwarp netdev device. The alias label can be anything starting with "iw". > The "iw" immediately after the ':' is the key used by the iwarp driver. What's wrong with my suggestion of having the iwarp driver create an "iwX" interface to go with the normal "ethX" interface? It seems simpler to me, and there's a somewhat similar precedent with how mac80211 devices create both wlan0 and wmaster0 interfaces. - R. From rajouri.jammu at gmail.com Thu Aug 30 21:54:11 2007 From: rajouri.jammu at gmail.com (Rajouri Jammu) Date: Thu, 30 Aug 2007 21:54:11 -0700 Subject: [ofa-general] Error building OFED 1.2 sources Message-ID: <3307cdf90708302154s68f103d3s75d2231a654d3dc@mail.gmail.com> Hi, I would like to make some changes to the OFED-1.2 kernel source. In order to make sure I'm doing the right thing I tried compiling the original sources (without any changes) the make is bailing out with an error (see below for the steps). I'm building on stock Centos5 kernel. Note that build.sh script worked but I would like to make changes to the sources and make sure it compiles before following the ofed_path.sh procedure. Any help would be great. This is what I did: tar -zxvf OFED-1.2.tgz rpm -ivh OFED-1.2/SRPM/ofa_kernel-1.2-0.src.rpm This created the source tar ball in /usr/src/redhat/SOURCES/ tar -zxvf /usr/src/redhat/SOURCES/ofa_kernel-1.2.tgz cd /usr/src/redhat/SOURCES/ofa_kernel-1.2 ./configure make The make immediatly bails out with the following error: modules make[1]: Entering directory `/usr/src/kernels/2.6.18-8.1.8.el5-i686' test -e include/linux/autoconf.h -a -e include/config/auto.conf || ( \ echo; \ echo " ERROR: Kernel configuration is invalid."; \ echo " include/linux/autoconf.h or include/config/auto.conf are missing."; \ echo " Run 'make oldconfig && make prepare' on kernel src to fix it."; \ echo; \ /bin/false) mkdir -p /usr/src/redhat/SOURCES/ofa_kernel-1.2/.tmp_versions Here is the full output of make Building kernel modules Kernel version: 2.6.18-8.1.8.el5 Modules directory: //lib/modules/2.6.18-8.1.8.el5/updates Kernel sources: /lib/modules/2.6.18-8.1.8.el5/build env EXTRA_CFLAGS=" -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/include -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/infiniband/include \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/infiniband/debug \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/net/cxgb3 \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/net/rds " \ make -C /lib/modules/2.6.18-8.1.8.el5/build SUBDIRS="/usr/src/redhat/SOURCES/ofa_kernel-1.2" KERNELRELEASE=2.6.18-8.1.8.el5 \ EXTRAVERSION=-8.1.8.el5 V=1 \ CONFIG_INFINIBAND= \ CONFIG_INFINIBAND_IPOIB= \ CONFIG_INFINIBAND_IPOIB_CM= \ CONFIG_INFINIBAND_SDP= \ CONFIG_INFINIBAND_SRP= \ CONFIG_INFINIBAND_USER_MAD= \ CONFIG_INFINIBAND_USER_ACCESS= \ CONFIG_INFINIBAND_ADDR_TRANS= \ CONFIG_INFINIBAND_MTHCA= \ CONFIG_INFINIBAND_IPOIB_DEBUG= \ CONFIG_INFINIBAND_ISER= \ CONFIG_SCSI_ISCSI_ATTRS= \ CONFIG_ISCSI_TCP= \ CONFIG_INFINIBAND_EHCA= \ CONFIG_INFINIBAND_EHCA_SCALING= \ CONFIG_RDS= \ CONFIG_RDS_IB= \ CONFIG_RDS_TCP= \ CONFIG_RDS_DEBUG= \ CONFIG_INFINIBAND_IPOIB_DEBUG_DATA= \ CONFIG_INFINIBAND_SDP_SEND_ZCOPY= \ CONFIG_INFINIBAND_SDP_RECV_ZCOPY= \ CONFIG_INFINIBAND_SDP_DEBUG= \ CONFIG_INFINIBAND_SDP_DEBUG_DATA= \ CONFIG_INFINIBAND_IPATH= \ CONFIG_INFINIBAND_MTHCA_DEBUG= \ CONFIG_INFINIBAND_MADEYE= \ CONFIG_INFINIBAND_VNIC= \ CONFIG_INFINIBAND_VNIC_DEBUG= \ CONFIG_INFINIBAND_VNIC_STATS= \ CONFIG_CHELSIO_T3= \ CONFIG_INFINIBAND_CXGB3= \ CONFIG_INFINIBAND_CXGB3_DEBUG= \ LINUXINCLUDE=' \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/kernel_addons/backport/2.6.18_FC6/include/ \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/include \ -I/usr/src/redhat/SOURCES/ofa_kernel-1.2/drivers/infiniband/include \ -Iinclude \ $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \ -include include/linux/autoconf.h \ -include /usr/src/redhat/SOURCES/ofa_kernel-1.2/include/linux/autoconf.h \ ' \ modules make[1]: Entering directory `/usr/src/kernels/2.6.18-8.1.8.el5-i686' test -e include/linux/autoconf.h -a -e include/config/auto.conf || ( \ echo; \ echo " ERROR: Kernel configuration is invalid."; \ echo " include/linux/autoconf.h or include/config/auto.conf are missing."; \ echo " Run 'make oldconfig && make prepare' on kernel src to fix it."; \ echo; \ /bin/false) mkdir -p /usr/src/redhat/SOURCES/ofa_kernel-1.2/.tmp_versions rm -f /usr/src/redhat/SOURCES/ofa_kernel-1.2/.tmp_versions/* make -f scripts/Makefile.build obj=/usr/src/redhat/SOURCES/ofa_kernel-1.2 Building modules, stage 2. make -rR -f /usr/src/kernels/2.6.18-8.1.8.el5-i686/scripts/Makefile.modpost scripts/mod/modpost -m -a -i /usr/src/kernels/2.6.18-8.1.8.el5-i686/Module.symvers -I /usr/src/redhat/SOURCES/ofa_kernel-1.2/Module.symvers -o /usr/src/redhat/SOURCES/ofa_kernel-1.2/Module.symvers make[1]: Leaving directory `/usr/src/kernels/2.6.18-8.1.8.el5-i686' From vlad at lists.openfabrics.org Fri Aug 31 02:47:15 2007 From: vlad at lists.openfabrics.org (Vladimir Sokolovsky) Date: Fri, 31 Aug 2007 02:47:15 -0700 (PDT) Subject: [ofa-general] ofa_1_3_kernel 20070831-0200 daily build status Message-ID: <20070831094715.7790AE60836@openfabrics.org> This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/~vlad/ofed_kernel.git git_branch: ofed_kernel Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.17 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.15 Passed on x86_64 with linux-2.6.21.1 Passed on ppc64 with linux-2.6.14 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.15 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.15 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.22 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Failed: Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.18_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.18' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- Build failed on powerpc with linux-2.6.16 Log: /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:936: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:939: error: invalid type argument of '->' /home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:940: error: invalid type argument of '->' make[4]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_3_kernel-20070831-0200_linux-2.6.16_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.16' make: *** [kernel] Error 2 ---------------------------------------------------------------------------------- From sashak at voltaire.com Fri Aug 31 03:45:55 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 13:45:55 +0300 Subject: [ofa-general] [PATCH] opensm/osm_vendor_ibumad.c: send error callback only when reponse is expected Message-ID: <20070831104554.GA11549@sashak.voltaire.com> In osm_vendor_send() when sending fails call send error callback only MADs where response is expected. So outstanding mad counters are updated properly. Pointed out and Acked-by: Hal Rosenstock Signed-off-by: Sasha Khapyorsky --- opensm/libvendor/osm_vendor_ibumad.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diff --git a/opensm/libvendor/osm_vendor_ibumad.c b/opensm/libvendor/osm_vendor_ibumad.c index 4618574..461c8bc 100644 --- a/opensm/libvendor/osm_vendor_ibumad.c +++ b/opensm/libvendor/osm_vendor_ibumad.c @@ -1120,16 +1120,18 @@ osm_vendor_send(IN osm_bind_handle_t h_bind, sent_mad_size, resp_expected ? p_vend->timeout : 0, p_vend->max_retries)) < 0) { - if (resp_expected) - get_madw(p_vend, &p_mad->trans_id); /* remove from aging table */ osm_log(p_vend->p_log, OSM_LOG_ERROR, "osm_vendor_send: ERR 5430: " "Send p_madw = %p of size %d failed %d (%m)\n", p_madw, sent_mad_size, ret); - p_madw->status = IB_ERROR; - pthread_mutex_lock(&p_vend->cb_mutex); - (*p_bind->send_err_callback) (p_bind->client_context, p_madw); /* cb frees madw */ - pthread_mutex_unlock(&p_vend->cb_mutex); + if (resp_expected) { + get_madw(p_vend, &p_mad->trans_id); /* remove from aging table */ + p_madw->status = IB_ERROR; + pthread_mutex_lock(&p_vend->cb_mutex); + (*p_bind->send_err_callback) (p_bind->client_context, p_madw); /* cb frees madw */ + pthread_mutex_unlock(&p_vend->cb_mutex); + } else + osm_mad_pool_put(p_bind->p_mad_pool, p_madw); goto Exit; } -- 1.5.3.rc2.38.g11308 From sashak at voltaire.com Fri Aug 31 03:54:35 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 13:54:35 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> Message-ID: <20070831105435.GB11549@sashak.voltaire.com> On 06:58 Thu 30 Aug , Hal Rosenstock wrote: > > > > > And when the setup part of the new QoS will be ready, would we still want > > > the SL2VL and VLArb tables to appear in the opts file? > > > > Don't think. Why we need it in two places? > > Are you saying that the existing QoS will get these tables from the > new QoS syntax (and the existing syntax eliminated) ? Yes. I think it is better than to keep two potentially conflicting configurations. > If so, what > about backward compatibility ? Also, how straightforward is it to > specify the equivalent as the existing syntax ? Will a conversion tool > be supplied ? Not sure it is needed - it is just few lines of text now. Sasha From hal.rosenstock at gmail.com Fri Aug 31 04:03:40 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 31 Aug 2007 07:03:40 -0400 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: <20070830230834.GA5756@obsidianresearch.com> References: <20070830154812.GB5617@obsidianresearch.com> <20070830164453.GA5680@obsidianresearch.com> <20070830230834.GA5756@obsidianresearch.com> Message-ID: On 8/30/07, Rolf Manderscheid wrote: > On Thu, Aug 30, 2007 at 01:25:43PM -0400, Hal Rosenstock wrote: > > On 8/30/07, Rolf Manderscheid wrote: > > > On Thu, Aug 30, 2007 at 12:01:57PM -0400, Hal Rosenstock wrote: > > > > On 8/30/07, Rolf Manderscheid wrote: > > > > > Hi Sasha, > > > > > > > > > > This patch sets the hop limit for the IPv4 broadcast groups so that broadcasts work > > > > > through IB routers. > > > > > > > > > > Signed-off-by: Rolf Manderscheid > > > > > > > > > > --- > > > > > > > > > > diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c > > > > > index 46ee429..fc000b1 100644 > > > > > --- a/opensm/opensm/osm_prtn.c > > > > > +++ b/opensm/opensm/osm_prtn.c > > > > > @@ -213,7 +213,7 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, > > > > > mc_rec.pkey = pkey; > > > > > mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ > > > > > mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; > > > > > - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); > > > > > + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0xff); > > > > > > > > Shouldn't this be based on whether the group is local scope or not ? > > > > > > If the group has local scope then the hop limit is irrelevant, packets will get > > > dropped by the router anyway. I certainly don't object to making the hop limit > > > conditional on scope if that's the preference. > > > > Looking at the PathRecord code, it only checks whether a DGID was > > requested or not and then sets the hop limit only being dependent on > > whether ROUTER_EXP was defined or not. Should we follow continue to > > follow this or make PathRecord responses consistent with this ? > > > Isn't the plan to incorporate the ROUTER_EXP code eventually? Eventually is relatively a long time IMO as it when there is an official router spec. Certainly well beyond the OFED 1.3 timeframe. >I would > rather not add any more #ifdefs if we can avoid it. I think your suggestion > of setting the hop limit according to the scope of the multicast group seems > like the better option to me ... I'll post a new patch. Are you OK with the > other patch to fix the scope of the second MGID? If so, I'll base the new > patch on that. Yes, I think that is OK as this would work for the no routers supported case as does the PathRecord code. In the longer term for both of these, I wonder whether hop limit will continue to be returned as the max or whether some actual limit would be returned. -- Hal > Rolf > From hal.rosenstock at gmail.com Fri Aug 31 04:21:46 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 31 Aug 2007 07:21:46 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070831105435.GB11549@sashak.voltaire.com> References: <46D5153E.6030307@dev.mellanox.co.il> <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> <20070831105435.GB11549@sashak.voltaire.com> Message-ID: On 8/31/07, Sasha Khapyorsky wrote: > On 06:58 Thu 30 Aug , Hal Rosenstock wrote: > > > > > > > And when the setup part of the new QoS will be ready, would we still want > > > > the SL2VL and VLArb tables to appear in the opts file? > > > > > > Don't think. Why we need it in two places? > > > > Are you saying that the existing QoS will get these tables from the > > new QoS syntax (and the existing syntax eliminated) ? > > Yes. I think it is better than to keep two potentially conflicting > configurations. > > > If so, what > > about backward compatibility ? Also, how straightforward is it to > > specify the equivalent as the existing syntax ? Will a conversion tool > > be supplied ? > > Not sure it is needed - it is just few lines of text now. The new equivalent syntax ? -- Hal > > Sasha > From sashak at voltaire.com Fri Aug 31 04:54:12 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 14:54:12 +0300 Subject: [ofa-general] Re: [PATCH] opensm: fix outstanding mad counters tracking In-Reply-To: References: <20070820133530.GE27000@sashak.voltaire.com> Message-ID: <20070831115412.GD11549@sashak.voltaire.com> On 09:07 Thu 30 Aug , Hal Rosenstock wrote: > Hi Sasha, > > On 8/20/07, Sasha Khapyorsky wrote: > > > > When MAD sending fails in osm_vendor_send() the send_err_callback() is > > invoked - this callback maintains (decreases by 1) the outstanding MAD > > counters. In the current osm_vl15_poller() code those MAD counters are > > also explicitly decreased in the case when osm_vendor_send() returns > > error - so actually we have "double free" case and as result OpenSM > > deadlocks there. > > > > This patch removes this additional outstanding mad counters decreasing > > code from osm_vl15_poller(). > > Shouldn't the cl_atomic code in complib have CL_ASSERTs to check for > the failed condition of a sign change in the atomic ? cl_atomic API deals with signed 32-bit integer, so it is legal to have negative values. Probably you want to put CL_ASSERTs in the code where outstanding counters are maintained instead of cl_atomic. > If that was > present, I think this would have been found quite some time ago. IMHO it was found fast when the bug was reliably reproduced. Anyway feel free to post patch if you think such CL_ASSERTs will be helpful for you. Sasha > Just a thought based on this lesson learned the hard way... > > -- Hal > > > Signed-off-by: Sasha Khapyorsky > > --- > > opensm/include/opensm/osm_vl15intf.h | 28 +--------- > > opensm/opensm/osm_opensm.c | 7 +- > > opensm/opensm/osm_vl15intf.c | 103 +++++----------------------------- > > 3 files changed, 18 insertions(+), 120 deletions(-) > > > > diff --git a/opensm/include/opensm/osm_vl15intf.h b/opensm/include/opensm/osm_vl15intf.h > > index 6de9898..4b290d3 100644 > > --- a/opensm/include/opensm/osm_vl15intf.h > > +++ b/opensm/include/opensm/osm_vl15intf.h > > @@ -53,13 +53,11 @@ > > #include > > #include > > #include > > -#include > > #include > > #include > > #include > > #include > > #include > > -#include > > > > #ifdef __cplusplus > > # define BEGIN_C_DECLS extern "C" { > > @@ -132,10 +130,6 @@ typedef struct _osm_vl15 { > > osm_vendor_t *p_vend; > > osm_log_t *p_log; > > osm_stats_t *p_stats; > > - osm_subn_t *p_subn; > > - cl_disp_reg_handle_t h_disp; > > - cl_plock_t *p_lock; > > - > > } osm_vl15_t; > > /* > > * FIELDS > > @@ -174,15 +168,6 @@ typedef struct _osm_vl15 { > > * p_stats > > * Pointer to the OpenSM statistics block. > > * > > -* p_subn > > -* Pointer to the Subnet object for this subnet. > > -* > > -* h_disp > > -* Handle returned from dispatcher registration. > > -* > > -* p_lock > > -* Pointer to the serializing lock. > > -* > > * SEE ALSO > > * VL15 object > > *********/ > > @@ -267,9 +252,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > > IN osm_vendor_t * const p_vend, > > IN osm_log_t * const p_log, > > IN osm_stats_t * const p_stats, > > - IN const int32_t max_wire_smps, > > - IN osm_subn_t * const p_subn, > > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock); > > + IN const int32_t max_wire_smps); > > /* > > * PARAMETERS > > * p_vl15 > > @@ -287,15 +270,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl15, > > * max_wire_smps > > * [in] Maximum number of MADs allowed on the wire at one time. > > * > > -* p_subn > > -* [in] Pointer to the subnet object. > > -* > > -* p_disp > > -* [in] Pointer to the dispatcher object. > > -* > > -* p_lock > > -* [in] Pointer to the OpenSM serializing lock. > > -* > > * RETURN VALUES > > * IB_SUCCESS if the VL15 object was initialized successfully. > > * > > diff --git a/opensm/opensm/osm_opensm.c b/opensm/opensm/osm_opensm.c > > index 9a596dd..329305e 100644 > > --- a/opensm/opensm/osm_opensm.c > > +++ b/opensm/opensm/osm_opensm.c > > @@ -249,10 +249,9 @@ osm_opensm_init(IN osm_opensm_t * const p_osm, > > if (status != IB_SUCCESS) > > goto Exit; > > > > - status = osm_vl15_init(&p_osm->vl15, > > - p_osm->p_vendor, > > - &p_osm->log, &p_osm->stats, p_opt->max_wire_smps, > > - &p_osm->subn, &p_osm->disp, &p_osm->lock); > > + status = osm_vl15_init(&p_osm->vl15, p_osm->p_vendor, > > + &p_osm->log, &p_osm->stats, > > + p_opt->max_wire_smps); > > if (status != IB_SUCCESS) > > goto Exit; > > > > diff --git a/opensm/opensm/osm_vl15intf.c b/opensm/opensm/osm_vl15intf.c > > index bc667b6..af44423 100644 > > --- a/opensm/opensm/osm_vl15intf.c > > +++ b/opensm/opensm/osm_vl15intf.c > > @@ -51,13 +51,12 @@ > > > > #include > > #include > > +#include > > +#include > > #include > > #include > > -#include > > #include > > #include > > -#include > > -#include > > > > /********************************************************************** > > **********************************************************************/ > > @@ -65,18 +64,13 @@ > > static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > { > > ib_api_status_t status; > > - cl_status_t cl_status; > > - uint32_t mads_sent; > > - uint32_t unicasts_sent; > > - uint32_t mads_on_wire; > > - uint32_t outstanding; > > > > /* > > Non-response-expected mads are not throttled on the wire > > since we can have no confirmation that they arrived > > at their destination. > > */ > > - if (p_madw->resp_expected == TRUE) { > > + if (p_madw->resp_expected == TRUE) > > /* > > Note that other threads may not see the response MAD > > arrive before send() even returns. > > @@ -84,14 +78,11 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > To avoid this confusion, preincrement the counts on the > > assumption that send() will succeed. > > */ > > - mads_on_wire = > > - cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > - CL_ASSERT(mads_on_wire <= p_vl->max_wire_smps); > > - } else > > - unicasts_sent = > > - cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > + else > > + cl_atomic_inc(&p_vl->p_stats->qp0_unicasts_sent); > > > > - mads_sent = cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > + cl_atomic_inc(&p_vl->p_stats->qp0_mads_sent); > > > > status = osm_vendor_send(osm_madw_get_bind_handle(p_madw), > > p_madw, p_madw->resp_expected); > > @@ -118,61 +109,12 @@ static void vl15_send_mad(osm_vl15_t * p_vl, osm_madw_t * p_madw) > > fix up the pre-incremented count values. > > */ > > > > - /* Decrement qp0_mads_sent and qp0_mads_outstanding_on_wire > > - that were incremented in the code above. */ > > - mads_sent = cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > > - > > - if (p_madw->resp_expected == FALSE) > > - return; > > - > > - cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding_on_wire); > > - > > - /* > > - The following code is similar to the code in > > - __osm_sm_mad_ctrl_retire_trans_mad. We need to decrement the > > - qp0_mads_outstanding counter, and if we reached 0 - need to call > > - the cl_disp_post with OSM_SIGNAL_NO_PENDING_TRANSACTION (in order > > - to wake up the state mgr). > > - There is one difference from the code in __osm_sm_mad_ctrl_retire_trans_mad. > > - This code is called for all (vl15) mads, if osm_vendor_send() failed, unlike > > - __osm_sm_mad_ctrl_retire_trans_mad which is called only on mads where > > - resp_expected == TRUE. As a result, the qp0_mads_outstanding counter > > - should be decremented and handled accordingly only if this is a mad > > - with resp_expected == TRUE. > > - */ > > - > > - outstanding = cl_atomic_dec(&p_vl->p_stats->qp0_mads_outstanding); > > - > > - osm_log(p_vl->p_log, OSM_LOG_DEBUG, > > - "__osm_vl15_poller: " > > - "%u QP0 MADs outstanding\n", > > - p_vl->p_stats->qp0_mads_outstanding); > > - > > - if (outstanding) > > - return; > > - > > - /* > > - The wire is clean. > > - Signal the state manager. > > - */ > > - if (osm_log_is_active(p_vl->p_log, OSM_LOG_DEBUG)) > > - osm_log(p_vl->p_log, > > - OSM_LOG_DEBUG, > > - "__osm_vl15_poller: " > > - "Posting Dispatcher message %s\n", > > - osm_get_disp_msg_str(OSM_MSG_NO_SMPS_OUTSTANDING)); > > - > > - cl_status = cl_disp_post(p_vl->h_disp, > > - OSM_MSG_NO_SMPS_OUTSTANDING, (void *) > > - OSM_SIGNAL_NO_PENDING_TRANSACTIONS, > > - NULL, NULL); > > - > > - if (cl_status != CL_SUCCESS) > > - osm_log(p_vl->p_log, > > - OSM_LOG_ERROR, > > - "__osm_vl15_poller: ERR 3E06: " > > - "Dispatcher post message failed (%s)\n", > > - CL_STATUS_MSG(cl_status)); > > + /* Decrement qp0_mads_sent that were incremented in the code above. > > + qp0_mads_outstanding will be decremented by send error callback > > + (called by osm_vendor_send() */ > > + cl_atomic_dec(&p_vl->p_stats->qp0_mads_sent); > > + if (!p_madw->resp_expected) > > + cl_atomic_dec(&p_vl->p_stats->qp0_unicasts_sent); > > } > > > > static void __osm_vl15_poller(IN void *p_ptr) > > @@ -260,7 +202,6 @@ void osm_vl15_construct(IN osm_vl15_t * const p_vl) > > cl_qlist_init(&p_vl->rfifo); > > cl_qlist_init(&p_vl->ufifo); > > cl_thread_construct(&p_vl->poller); > > - p_vl->h_disp = CL_DISP_INVALID_HANDLE; > > } > > > > /********************************************************************** > > @@ -317,9 +258,7 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > IN osm_vendor_t * const p_vend, > > IN osm_log_t * const p_log, > > IN osm_stats_t * const p_stats, > > - IN const int32_t max_wire_smps, > > - IN osm_subn_t * const p_subn, > > - IN cl_dispatcher_t * const p_disp, IN cl_plock_t * const p_lock) > > + IN const int32_t max_wire_smps) > > { > > ib_api_status_t status = IB_SUCCESS; > > > > @@ -329,8 +268,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > p_vl->p_log = p_log; > > p_vl->p_stats = p_stats; > > p_vl->max_wire_smps = max_wire_smps; > > - p_vl->p_subn = p_subn; > > - p_vl->p_lock = p_lock; > > > > status = cl_event_init(&p_vl->signal, FALSE); > > if (status != IB_SUCCESS) > > @@ -351,16 +288,6 @@ osm_vl15_init(IN osm_vl15_t * const p_vl, > > if (status != IB_SUCCESS) > > goto Exit; > > > > - p_vl->h_disp = cl_disp_register(p_disp, CL_DISP_MSGID_NONE, NULL, NULL); > > - > > - if (p_vl->h_disp == CL_DISP_INVALID_HANDLE) { > > - osm_log(p_log, OSM_LOG_ERROR, > > - "osm_vl15_init: ERR 3E01: " > > - "Dispatcher registration failed\n"); > > - status = IB_INSUFFICIENT_RESOURCES; > > - goto Exit; > > - } > > - > > Exit: > > OSM_LOG_EXIT(p_log); > > return (status); > > @@ -444,8 +371,6 @@ osm_vl15_shutdown(IN osm_vl15_t * const p_vl, > > /* grap a lock on the object */ > > cl_spinlock_acquire(&p_vl->lock); > > > > - cl_disp_unregister(p_vl->h_disp); > > - > > /* go over all outstanding MADs and retire their transactions */ > > > > /* first we handle the list of response MADs */ > > -- > > 1.5.3.rc2.29.gc4640f > > > > From sashak at voltaire.com Fri Aug 31 05:04:50 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 15:04:50 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> <20070831105435.GB11549@sashak.voltaire.com> Message-ID: <20070831120450.GE11549@sashak.voltaire.com> On 07:21 Fri 31 Aug , Hal Rosenstock wrote: > On 8/31/07, Sasha Khapyorsky wrote: > > On 06:58 Thu 30 Aug , Hal Rosenstock wrote: > > > > > > > > > And when the setup part of the new QoS will be ready, would we still want > > > > > the SL2VL and VLArb tables to appear in the opts file? > > > > > > > > Don't think. Why we need it in two places? > > > > > > Are you saying that the existing QoS will get these tables from the > > > new QoS syntax (and the existing syntax eliminated) ? > > > > Yes. I think it is better than to keep two potentially conflicting > > configurations. > > > > > If so, what > > > about backward compatibility ? Also, how straightforward is it to > > > specify the equivalent as the existing syntax ? Will a conversion tool > > > be supplied ? > > > > Not sure it is needed - it is just few lines of text now. > > The new equivalent syntax ? The new syntax is not equivalent, but similar in part of vlarb and sl2vl configuration, it is much more powerful and requires some extra lines. In order to have equivalent configurations vlarb and sl2vl should be specified by node/port type. Hope I'm not worng about this. Sasha From sashak at voltaire.com Fri Aug 31 05:10:13 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 15:10:13 +0300 Subject: [ofa-general] Re: [PATCH] osm: QoS parser - fixing yacc command In-Reply-To: <46D6E4C3.80201@dev.mellanox.co.il> References: <46D6E4C3.80201@dev.mellanox.co.il> Message-ID: <20070831121013.GF11549@sashak.voltaire.com> On 18:39 Thu 30 Aug , Yevgeny Kliteynik wrote: > Fixing bison command to more general yacc syntax > > Signed-off-by: Yevgeny Kliteynik Applied. Thanks. > --- > opensm/opensm/Makefile.am | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > index abfa913..4ab7227 100644 > --- a/opensm/opensm/Makefile.am > +++ b/opensm/opensm/Makefile.am > @@ -60,7 +60,8 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > > osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > - $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > + $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm_qos_parser.y > + mv -f y.tab.h $(srcdir)/../include/opensm/osm_qos_parser_y.h BTW if osm_qos_parser_y.h file is generated one and used only by generated *.c files is not it would be simpler just to lease it in current directory? Sasha From sashak at voltaire.com Fri Aug 31 05:12:12 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 15:12:12 +0300 Subject: [ofa-general] Re: [PATCH] infiniband-diags/Makefile.am: clean up extra directives In-Reply-To: <20070830092435.0f418956.weiny2@llnl.gov> References: <20070830092435.0f418956.weiny2@llnl.gov> Message-ID: <20070831121212.GG11549@sashak.voltaire.com> On 09:24 Thu 30 Aug , Ira Weiny wrote: > From 8da03fd7526e0879404425609abea7bc2da0d7f0 Mon Sep 17 00:00:00 2001 > From: Ira K. Weiny > Date: Thu, 30 Aug 2007 09:22:29 -0700 > Subject: [PATCH] infiniband-diags/Makefile.am: clean up extra directives > > > Signed-off-by: Ira K. Weiny Applied. Thanks. Sasha From hal.rosenstock at gmail.com Fri Aug 31 05:32:54 2007 From: hal.rosenstock at gmail.com (Hal Rosenstock) Date: Fri, 31 Aug 2007 08:32:54 -0400 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070831120450.GE11549@sashak.voltaire.com> References: <46D55C47.6080705@dev.mellanox.co.il> <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> <20070831105435.GB11549@sashak.voltaire.com> <20070831120450.GE11549@sashak.voltaire.com> Message-ID: On 8/31/07, Sasha Khapyorsky wrote: > On 07:21 Fri 31 Aug , Hal Rosenstock wrote: > > On 8/31/07, Sasha Khapyorsky wrote: > > > On 06:58 Thu 30 Aug , Hal Rosenstock wrote: > > > > > > > > > > > And when the setup part of the new QoS will be ready, would we still want > > > > > > the SL2VL and VLArb tables to appear in the opts file? > > > > > > > > > > Don't think. Why we need it in two places? > > > > > > > > Are you saying that the existing QoS will get these tables from the > > > > new QoS syntax (and the existing syntax eliminated) ? > > > > > > Yes. I think it is better than to keep two potentially conflicting > > > configurations. > > > > > > > If so, what > > > > about backward compatibility ? Also, how straightforward is it to > > > > specify the equivalent as the existing syntax ? Will a conversion tool > > > > be supplied ? > > > > > > Not sure it is needed - it is just few lines of text now. > > > > The new equivalent syntax ? > > The new syntax is not equivalent, but similar in part of vlarb and sl2vl > configuration, it is much more powerful and requires some extra lines. > In order to have equivalent configurations vlarb and sl2vl should be > specified by node/port type. Is there/will there be such a syntax ? -- Hal >Hope I'm not worng about this. > > Sasha > From sashak at voltaire.com Fri Aug 31 05:46:33 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 15:46:33 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: References: <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> <20070831105435.GB11549@sashak.voltaire.com> <20070831120450.GE11549@sashak.voltaire.com> Message-ID: <20070831124633.GH11549@sashak.voltaire.com> On 08:32 Fri 31 Aug , Hal Rosenstock wrote: > On 8/31/07, Sasha Khapyorsky wrote: > > On 07:21 Fri 31 Aug , Hal Rosenstock wrote: > > > On 8/31/07, Sasha Khapyorsky wrote: > > > > On 06:58 Thu 30 Aug , Hal Rosenstock wrote: > > > > > > > > > > > > > And when the setup part of the new QoS will be ready, would we still want > > > > > > > the SL2VL and VLArb tables to appear in the opts file? > > > > > > > > > > > > Don't think. Why we need it in two places? > > > > > > > > > > Are you saying that the existing QoS will get these tables from the > > > > > new QoS syntax (and the existing syntax eliminated) ? > > > > > > > > Yes. I think it is better than to keep two potentially conflicting > > > > configurations. > > > > > > > > > If so, what > > > > > about backward compatibility ? Also, how straightforward is it to > > > > > specify the equivalent as the existing syntax ? Will a conversion tool > > > > > be supplied ? > > > > > > > > Not sure it is needed - it is just few lines of text now. > > > > > > The new equivalent syntax ? > > > > The new syntax is not equivalent, but similar in part of vlarb and sl2vl > > configuration, it is much more powerful and requires some extra lines. > > In order to have equivalent configurations vlarb and sl2vl should be > > specified by node/port type. > > Is there/will there be such a syntax ? There is, as far as I understand it. Yevgeny? Sasha From sashak at voltaire.com Fri Aug 31 06:22:42 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 16:22:42 +0300 Subject: [ofa-general] Re: [PATCH] opensm: set scope according to configuration for ipoib multicast groups In-Reply-To: <20070830175542.GA5741@obsidianresearch.com> References: <20070830175542.GA5741@obsidianresearch.com> Message-ID: <20070831132242.GJ11549@sashak.voltaire.com> On 11:55 Thu 30 Aug , Rolf Manderscheid wrote: > Hi Sasha, > > opensm creates two multicast groups for ipoib. The group corresponding to > 255.255.255.255 results in an MGID with the scope as specified by the configuration. > The one MGID for 0.0.0.1 ends up with local scope irrespective of the configuration. > If ipoib is configured with global scope, then its join of the latter group never > succeeds. The patch below sets the scope for both groups according to the > configuration. > > Signed-off-by: Rolf Manderscheid Applied. Thanks. Sasha From sashak at voltaire.com Fri Aug 31 06:55:52 2007 From: sashak at voltaire.com (Sasha Khapyorsky) Date: Fri, 31 Aug 2007 16:55:52 +0300 Subject: [ofa-general] [PATCH] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: <20070830230834.GA5756@obsidianresearch.com> References: <20070830154812.GB5617@obsidianresearch.com> <20070830164453.GA5680@obsidianresearch.com> <20070830230834.GA5756@obsidianresearch.com> Message-ID: <20070831135552.GL11549@sashak.voltaire.com> Hi Rolf, On 17:08 Thu 30 Aug , Rolf Manderscheid wrote: > > Isn't the plan to incorporate the ROUTER_EXP code eventually? I would > rather not add any more #ifdefs if we can avoid it. I think your suggestion > of setting the hop limit according to the scope of the multicast group seems > like the better option to me ... I'll post a new patch. Are you OK with the > other patch to fix the scope of the second MGID? If so, I'll base the new > patch on that. It is fine with me. Sasha From fenkes at de.ibm.com Fri Aug 31 07:01:41 2007 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Fri, 31 Aug 2007 16:01:41 +0200 Subject: [ofa-general] [PATCH 0/2] IB/ehca: Fixes for rc5 Message-ID: <200708311601.42220.fenkes@de.ibm.com> These two patches fix some ehca issues that should be fixed in 2.6.23. [1/2] fixes regressions caused by the recent addition of Small QPs. [2/2] adds missing SRQ-related functionality that would have broken IPoIB CM. The patches should apply cleanly, in order, against Roland's git. Please review the changes and apply the patches for 2.6.23-rc5 if they are okay. Regards, Joachim -- Joachim Fenkes  --  eHCA Linux Driver Developer and Hardware Tamer IBM Deutschland Entwicklung GmbH  --  Dept. 3627 (I/O Firmware Dev. 2) Schoenaicher Strasse 220  --  71032 Boeblingen  --  Germany eMail: fenkes at de.ibm.com From fenkes at de.ibm.com Fri Aug 31 07:02:59 2007 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Fri, 31 Aug 2007 16:02:59 +0200 Subject: [ofa-general] [PATCH 1/2] IB/ehca: fix Small QP regressions In-Reply-To: <200708311601.42220.fenkes@de.ibm.com> References: <200708311601.42220.fenkes@de.ibm.com> Message-ID: <200708311602.59890.fenkes@de.ibm.com> From: Stefan Roscher The new Small QP code had a few bugs that would also trigger for non-Small QPs. Fix them. Signed-off-by: Joachim Fenkes --- drivers/infiniband/hw/ehca/ehca_qp.c | 10 ++++++---- drivers/infiniband/hw/ehca/ipz_pt_fn.c | 2 +- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..84d435a 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -600,10 +600,12 @@ static struct ehca_qp *internal_create_qp( if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap) && !(context && udata)) { /* no small QP support in userspace ATM */ - ehca_determine_small_queue( - &parms.squeue, max_send_sge, is_llqp); - ehca_determine_small_queue( - &parms.rqueue, max_recv_sge, is_llqp); + if (HAS_SQ(my_qp)) + ehca_determine_small_queue( + &parms.squeue, max_send_sge, is_llqp); + if (HAS_RQ(my_qp)) + ehca_determine_small_queue( + &parms.rqueue, max_recv_sge, is_llqp); parms.qp_storage = (parms.squeue.is_small || parms.rqueue.is_small); } diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.c b/drivers/infiniband/hw/ehca/ipz_pt_fn.c index a090c67..29bd476 100644 --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c @@ -172,7 +172,7 @@ static void free_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd) unsigned long bit; int free_page = 0; - bit = ((unsigned long)queue->queue_pages[0] & PAGE_MASK) + bit = ((unsigned long)queue->queue_pages[0] & ~PAGE_MASK) >> (order + 9); mutex_lock(&pd->lock); -- 1.5.2 From fenkes at de.ibm.com Fri Aug 31 07:03:37 2007 From: fenkes at de.ibm.com (Joachim Fenkes) Date: Fri, 31 Aug 2007 16:03:37 +0200 Subject: [ofa-general] [PATCH 2/2] IB/ehca: SRQ fixes to enable IPoIB CM In-Reply-To: <200708311601.42220.fenkes@de.ibm.com> References: <200708311601.42220.fenkes@de.ibm.com> Message-ID: <200708311603.38822.fenkes@de.ibm.com> a) Report max_srq > 0 if SRQ is supported b) Report "last wqe reached" event when base QP dies Signed-off-by: Joachim Fenkes --- drivers/infiniband/hw/ehca/ehca_hca.c | 10 +++++-- drivers/infiniband/hw/ehca/ehca_irq.c | 48 +++++++++++++++++++++----------- 2 files changed, 38 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index fc19ef9..cf22472 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -93,9 +93,13 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props) props->max_pd = min_t(int, rblock->max_pd, INT_MAX); props->max_ah = min_t(int, rblock->max_ah, INT_MAX); props->max_fmr = min_t(int, rblock->max_mr, INT_MAX); - props->max_srq = 0; - props->max_srq_wr = 0; - props->max_srq_sge = 0; + + if (EHCA_BMASK_GET(HCA_CAP_SRQ, shca->hca_cap)) { + props->max_srq = props->max_qp; + props->max_srq_wr = props->max_qp_wr; + props->max_srq_sge = 3; + } + props->max_pkeys = 16; props->local_ca_ack_delay = rblock->local_ca_ack_delay; diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c index ee06d8b..a925ea5 100644 --- a/drivers/infiniband/hw/ehca/ehca_irq.c +++ b/drivers/infiniband/hw/ehca/ehca_irq.c @@ -175,41 +175,55 @@ error_data1: } -static void qp_event_callback(struct ehca_shca *shca, u64 eqe, - enum ib_event_type event_type, int fatal) +static void dispatch_qp_event(struct ehca_shca *shca, struct ehca_qp *qp, + enum ib_event_type event_type) { struct ib_event event; - struct ehca_qp *qp; - u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); - - read_lock(&ehca_qp_idr_lock); - qp = idr_find(&ehca_qp_idr, token); - read_unlock(&ehca_qp_idr_lock); - - - if (!qp) - return; - - if (fatal) - ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); event.device = &shca->ib_device; + event.event = event_type; if (qp->ext_type == EQPT_SRQ) { if (!qp->ib_srq.event_handler) return; - event.event = fatal ? IB_EVENT_SRQ_ERR : event_type; event.element.srq = &qp->ib_srq; qp->ib_srq.event_handler(&event, qp->ib_srq.srq_context); } else { if (!qp->ib_qp.event_handler) return; - event.event = event_type; event.element.qp = &qp->ib_qp; qp->ib_qp.event_handler(&event, qp->ib_qp.qp_context); } +} + +static void qp_event_callback(struct ehca_shca *shca, u64 eqe, + enum ib_event_type event_type, int fatal) +{ + struct ehca_qp *qp; + u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); + + read_lock(&ehca_qp_idr_lock); + qp = idr_find(&ehca_qp_idr, token); + read_unlock(&ehca_qp_idr_lock); + + if (!qp) + return; + + if (fatal) + ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); + + dispatch_qp_event(shca, qp, fatal && qp->ext_type == EQPT_SRQ ? + IB_EVENT_SRQ_ERR : event_type); + + /* + * eHCA only processes one WQE at a time for SRQ base QPs, + * so the last WQE has been processed as soon as the QP enters + * error state. + */ + if (fatal && qp->ext_type == EQPT_SRQBASE) + dispatch_qp_event(shca, qp, IB_EVENT_QP_LAST_WQE_REACHED); return; } -- 1.5.2 From swelch at systemfabricworks.com Fri Aug 31 07:53:06 2007 From: swelch at systemfabricworks.com (swelch at systemfabricworks.com) Date: Fri, 31 Aug 2007 09:53:06 -0500 Subject: [ofa-general] [PATCH] infiniband/hw/mthca: Add optional router mode initialization Message-ID: <46D82B52.mailO2X1DVFVD@systemfabricworks.com> This patch allows for the kernel mthca driver to optionally initialize the mthca devices in router mode. Router mode is enabled at module load with the setting of the module parm "router_mode=1". This setting acts on the device as a whole. In this mode the mthca device(s) in the system will be brought up with hardware transport operations disabled and all packets that meet each ports lid/lmc as well as DR SMPS will be delivered to one of 256 router QPs (1 per TClass) along with their packet headers. Without "router_mode=1" specified, the device operation will not be altered. When in mthca router mode, the SMI/GSI QP may still be used to send packets, all ingress packets will be delivered to the consumer that creates the first 256 QP on that device. Additional QP may be created for sending. It is the responsibility of this consumer to deliver ingress SMP, and non-forwarded ingress GMP packets to the local management stack. This patch provides a API friendly method for interfacing with the driver when running in router mode. Additions to the IB Verbs API are not required to create and use the QP. A subsequent patch will provide a method for forwarding packets via the libmthca running on top of these changes. Roland, these changes certainly are not the most eloquent way to implement mthca router mode, but they are relatively non-intrusive and will provide a mechanism for expermentation of software IB routing. I suspect you may find a better solution. Signed-off-by: Steve Welch --- drivers/infiniband/hw/mthca/mthca_cmd.c | 5 +++ drivers/infiniband/hw/mthca/mthca_dev.h | 9 ++++- drivers/infiniband/hw/mthca/mthca_main.c | 47 ++++++++++++++++++++++++++ drivers/infiniband/hw/mthca/mthca_provider.c | 3 +- drivers/infiniband/hw/mthca/mthca_qp.c | 40 +++++++++++++++++---- 5 files changed, 94 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.c b/drivers/infiniband/hw/mthca/mthca_cmd.c index acc9589..d6039c7 100644 --- a/drivers/infiniband/hw/mthca/mthca_cmd.c +++ b/drivers/infiniband/hw/mthca/mthca_cmd.c @@ -1272,6 +1272,7 @@ int mthca_INIT_HCA(struct mthca_dev *dev, #define INIT_HCA_IN_SIZE 0x200 #define INIT_HCA_FLAGS1_OFFSET 0x00c +#define INIT_HCA_ROUTER_OFFSET 0x010 #define INIT_HCA_FLAGS2_OFFSET 0x014 #define INIT_HCA_QPC_OFFSET 0x020 #define INIT_HCA_QPC_BASE_OFFSET (INIT_HCA_QPC_OFFSET + 0x10) @@ -1318,6 +1319,10 @@ int mthca_INIT_HCA(struct mthca_dev *dev, if (dev->mthca_flags & MTHCA_FLAG_SINAI_OPT) MTHCA_PUT(inbox, 0x1, INIT_HCA_FLAGS1_OFFSET); + if (dev->mthca_flags & MTHCA_FLAG_ROUTER_OPT) + *(inbox + INIT_HCA_ROUTER_OFFSET / 4) = + cpu_to_be32(0x80000000 + dev->rqp_base ); + #if defined(__LITTLE_ENDIAN) *(inbox + INIT_HCA_FLAGS2_OFFSET / 4) &= ~cpu_to_be32(1 << 1); #elif defined(__BIG_ENDIAN) diff --git a/drivers/infiniband/hw/mthca/mthca_dev.h b/drivers/infiniband/hw/mthca/mthca_dev.h index 9bae3cc..5499360 100644 --- a/drivers/infiniband/hw/mthca/mthca_dev.h +++ b/drivers/infiniband/hw/mthca/mthca_dev.h @@ -66,7 +66,8 @@ enum { MTHCA_FLAG_FMR = 1 << 6, MTHCA_FLAG_MEMFREE = 1 << 7, MTHCA_FLAG_PCIE = 1 << 8, - MTHCA_FLAG_SINAI_OPT = 1 << 9 + MTHCA_FLAG_SINAI_OPT = 1 << 9, + MTHCA_FLAG_ROUTER_OPT = 1 << 10 }; enum { @@ -93,6 +94,11 @@ enum { }; enum { + MTHCA_RQP_BASE_ALIGNMENT = 256, + MTHCA_RQP_NUM_INGRESS = 256 +}; + +enum { MTHCA_EQ_CMD, MTHCA_EQ_ASYNC, MTHCA_EQ_COMP, @@ -360,6 +366,7 @@ struct mthca_dev { struct ib_ah *sm_ah[MTHCA_MAX_PORTS]; spinlock_t sm_lock; u8 rate[MTHCA_MAX_PORTS]; + int rqp_base; }; #ifdef CONFIG_INFINIBAND_MTHCA_DEBUG diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c index 76fed75..405f4b4 100644 --- a/drivers/infiniband/hw/mthca/mthca_main.c +++ b/drivers/infiniband/hw/mthca/mthca_main.c @@ -80,6 +80,10 @@ static int tune_pci = 0; module_param(tune_pci, int, 0444); MODULE_PARM_DESC(tune_pci, "increase PCI burst from the default set by BIOS if nonzero"); +static int router_mode = 0; +module_param(router_mode, int, 0444); +MODULE_PARM_DESC(router_mode, "initialize the mthca driver in router mode"); + DEFINE_MUTEX(mthca_device_mutex); #define MTHCA_DEFAULT_NUM_QP (1 << 16) @@ -292,6 +296,26 @@ static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim) return 0; } +static int mthca_init_rqp_base(struct mthca_dev *mdev) +{ + int err = 0; + int i = mdev->limits.reserved_qps + MTHCA_MAX_PORTS * 2 + 1; + + /* + * Determine the first available router QP number. The + * base must be aligned on a 256 boundary. + */ + for (mdev->rqp_base=MTHCA_RQP_BASE_ALIGNMENT; mdev->rqp_base < i; + mdev->rqp_base += MTHCA_RQP_BASE_ALIGNMENT) + ; + + if (mdev->rqp_base + MTHCA_RQP_NUM_INGRESS > mdev->limits.num_qps) { + mdev->rqp_base = 0; + err = -EINVAL; + } + return err; +} + static int mthca_init_tavor(struct mthca_dev *mdev) { u8 status; @@ -350,6 +374,15 @@ static int mthca_init_tavor(struct mthca_dev *mdev) if (err < 0) goto err_disable; + if (mdev->mthca_flags & MTHCA_FLAG_ROUTER_OPT) { + err = mthca_init_rqp_base(mdev); + if (err) { + mthca_err(mdev, "Insufficient router QP resources, " + "aborting.\n"); + goto err_disable; + } + } + err = mthca_INIT_HCA(mdev, &init_hca, &status); if (err) { mthca_err(mdev, "INIT_HCA command failed, aborting.\n"); @@ -684,6 +717,15 @@ static int mthca_init_arbel(struct mthca_dev *mdev) if (err) goto err_stop_fw; + if (mdev->mthca_flags & MTHCA_FLAG_ROUTER_OPT) { + err = mthca_init_rqp_base(mdev); + if (err) { + mthca_err(mdev, "Insufficient router QP resources, " + "aborting.\n"); + goto err_disable; + } + } + err = mthca_INIT_HCA(mdev, &init_hca, &status); if (err) { mthca_err(mdev, "INIT_HCA command failed, aborting.\n"); @@ -1104,6 +1146,11 @@ static int __mthca_init_one(struct pci_dev *pdev, int hca_type) if (ddr_hidden) mdev->mthca_flags |= MTHCA_FLAG_DDR_HIDDEN; + if (router_mode) { + printk(KERN_INFO PFX "Initializing in MTHCA router mode\n" ); + mdev->mthca_flags |= MTHCA_FLAG_ROUTER_OPT; + } + /* * Now reset the HCA before we touch the PCI capabilities or * attempt a firmware command, since a boot ROM may have left diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index 6bcde1c..5e81039 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -1309,7 +1309,8 @@ int mthca_register_device(struct mthca_dev *dev) (1ull << IB_USER_VERBS_CMD_DESTROY_QP) | (1ull << IB_USER_VERBS_CMD_ATTACH_MCAST) | (1ull << IB_USER_VERBS_CMD_DETACH_MCAST); - dev->ib_dev.node_type = RDMA_NODE_IB_CA; + dev->ib_dev.node_type = dev->mthca_flags & MTHCA_FLAG_ROUTER_OPT ? + RDMA_NODE_IB_ROUTER : RDMA_NODE_IB_CA; dev->ib_dev.phys_port_cnt = dev->limits.num_ports; dev->ib_dev.num_comp_vectors = 1; dev->ib_dev.dma_device = &dev->pdev->dev; diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c index df01b20..185bcff 100644 --- a/drivers/infiniband/hw/mthca/mthca_qp.c +++ b/drivers/infiniband/hw/mthca/mthca_qp.c @@ -1298,12 +1298,19 @@ int mthca_alloc_qp(struct mthca_dev *dev, { int err; - switch (type) { - case IB_QPT_RC: qp->transport = RC; break; - case IB_QPT_UC: qp->transport = UC; break; - case IB_QPT_UD: qp->transport = UD; break; - default: return -EINVAL; - } + /* + * In router mode no hardware transport endpoints are supported, + * always set the QP to actually use the MLX transport. + */ + if (dev->mthca_flags & MTHCA_FLAG_ROUTER_OPT) + qp->transport = MLX; + else + switch (type) { + case IB_QPT_RC: qp->transport = RC; break; + case IB_QPT_UC: qp->transport = UC; break; + case IB_QPT_UD: qp->transport = UD; break; + default: return -EINVAL; + } err = mthca_set_qp_size(dev, cap, pd, qp); if (err) @@ -1313,6 +1320,14 @@ int mthca_alloc_qp(struct mthca_dev *dev, if (qp->qpn == -1) return -ENOMEM; + /* + * Router mode QPs must use the exact QPN number range dedicated + * to this function. Take out any "top" portion inserted by the + * allocation routine. + */ + if (dev->mthca_flags & MTHCA_FLAG_ROUTER_OPT) + qp->qpn &= dev->qp_table.alloc.max - 1; + /* initialize port to zero for error-catching. */ qp->port = 0; @@ -2291,6 +2306,7 @@ int mthca_init_qp_table(struct mthca_dev *dev) int err; u8 status; int i; + int reserve_qp; spin_lock_init(&dev->qp_table.lock); @@ -2299,11 +2315,19 @@ int mthca_init_qp_table(struct mthca_dev *dev) * special QP for port 1 has to be even, so round up. */ dev->qp_table.sqp_start = (dev->limits.reserved_qps + 1) & ~1UL; + + /* + * In router mode we also reserve all of the QPs between the + * special QP and the next multiple of 256 (the router QP base). + */ + reserve_qp = dev->mthca_flags & MTHCA_FLAG_ROUTER_OPT ? + dev->rqp_base : + dev->qp_table.sqp_start + MTHCA_MAX_PORTS * 2; + err = mthca_alloc_init(&dev->qp_table.alloc, dev->limits.num_qps, (1 << 24) - 1, - dev->qp_table.sqp_start + - MTHCA_MAX_PORTS * 2); + reserve_qp); if (err) return err; From rvm at obsidianresearch.com Fri Aug 31 09:13:59 2007 From: rvm at obsidianresearch.com (Rolf Manderscheid) Date: Fri, 31 Aug 2007 10:13:59 -0600 Subject: [ofa-general] [PATCHv2] opensm: set hop limit when creating ipoib multicast groups In-Reply-To: <20070831135552.GL11549@sashak.voltaire.com> References: <20070830154812.GB5617@obsidianresearch.com> <20070830164453.GA5680@obsidianresearch.com> <20070830230834.GA5756@obsidianresearch.com> <20070831135552.GL11549@sashak.voltaire.com> Message-ID: <20070831161359.GA9728@obsidianresearch.com> Hi Sasha, This patch sets the hop limit for the multicast groups for ipoib according to the scope so that ipoib works over multiple IB subnets. Signed-off-by: Rolf Manderscheid --- diff --git a/opensm/opensm/osm_prtn.c b/opensm/opensm/osm_prtn.c index 17348bc..a23a64b 100644 --- a/opensm/opensm/osm_prtn.c +++ b/opensm/opensm/osm_prtn.c @@ -198,8 +198,12 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_mgrp_t *p_mgrp = NULL; osm_sa_t *p_sa = &p_subn->p_osm->sa; ib_api_status_t status = IB_SUCCESS; + uint8_t hop_limit; pkey = p->pkey | cl_hton16(0x8000); + if (! scope) + scope = OSM_DEFAULT_MGRP_SCOPE; + hop_limit = (scope == MC_SCOPE_LINK_LOCAL) ? 0 : IB_HOPLIMIT_MAX; memset(&mc_rec, 0, sizeof(mc_rec)); @@ -212,12 +216,10 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, mc_rec.pkey = pkey; mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */ mc_rec.pkt_life = OSM_DEFAULT_SUBNET_TIMEOUT; - mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, 0); + mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, hop_limit); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ - mc_rec.scope_state = - ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, - MC_FULL_MEMBER); - ib_mgid_set_scope(&mc_rec.mgid, scope ? scope : OSM_DEFAULT_MGRP_SCOPE); + mc_rec.scope_state = ib_member_set_scope_state(scope, MC_FULL_MEMBER); + ib_mgid_set_scope(&mc_rec.mgid, scope); /* don't update rate, mtu */ comp_mask = IB_MCR_COMPMASK_MTU | IB_MCR_COMPMASK_MTU_SEL | @@ -238,10 +240,8 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, mc_rec.mgid = osm_ts_ipoib_mgid; memcpy(&mc_rec.mgid.raw[4], &pkey, sizeof(pkey)); /* Scope in MCMemberRecord (if present) needs to be consistent with MGID */ - mc_rec.scope_state = - ib_member_set_scope_state(scope ? scope : OSM_DEFAULT_MGRP_SCOPE, - MC_FULL_MEMBER); - ib_mgid_set_scope(&mc_rec.mgid, scope ? scope : OSM_DEFAULT_MGRP_SCOPE); + mc_rec.scope_state = ib_member_set_scope_state(scope, MC_FULL_MEMBER); + ib_mgid_set_scope(&mc_rec.mgid, scope); status = osm_mcmr_rcv_find_or_create_new_mgrp(&p_sa->mcmr_rcv, comp_mask, From jaschut at sandia.gov Fri Aug 31 09:15:58 2007 From: jaschut at sandia.gov (Jim Schutt) Date: Fri, 31 Aug 2007 10:15:58 -0600 Subject: [ofa-general] [PATCH] ofed-1.2.5/mvapich2 bug fixes Message-ID: <1188576958.14461.137.camel@sale659> Hi, I've been working with mvapich2 from the OFED-1.2.5 release (I used http://www.openfabrics.org/downloads/OFED/ofed-1.2.5/OFED-1.2.5.tgz). I've found a couple bugs in that version of mvapich2, which seem to also be present in the upstream SVN at https://mvapich.cse.ohio-state.edu/svn/mpi/mvapich2/branches/0.9.8 as of revision 1480. The first is an install tool bug: DESTDIR gets prepended twice in some cases, once when calling FixInstallFile and once inside it. The second is a memory-scribbling bug: in the call-chain rdma_cm_get_hostnames()->PMI_KVS_Get()->PMIU_getval() PMIU_getval() overwrites byte PMI_vallen_max - 1 (in my test case, PMI_vallen_max had value 2048) for a character array that has length 16. I'm not sure these are the right fixes, but with these patches applied, mvapich2 from ofed-1.2.5 installs correctly and runs tests it wouldn't run without them. -- Jim -- Jim Schutt Sandia National Laboratories, Albuquerque, New Mexico USA diff -urN mvapich2-0.9.8.orig/src/mpe2/sbin/mpeinstall.in mvapich2-0.9.8/src/mpe2/sbin/mpeinstall.in --- mvapich2-0.9.8.orig/src/mpe2/sbin/mpeinstall.in 2006-04-09 11:57:00.000000000 -0600 +++ mvapich2-0.9.8/src/mpe2/sbin/mpeinstall.in 2007-08-31 10:02:10.000000000 -0600 @@ -442,10 +442,10 @@ echo "Copying MPE utility programs to $DESTDIR$bindir" CopyDirRecurP $binbuild_dir $bindir $XMODE if [ -s $binbuild_dir/mpecc -a -x $binbuild_dir/mpecc ] ; then - FixInstallFile $binbuild_dir/mpecc $DESTDIR$bindir/mpecc $XMODE + FixInstallFile $binbuild_dir/mpecc $bindir/mpecc $XMODE fi if [ -s $binbuild_dir/mpefc -a -x $binbuild_dir/mpefc ] ; then - FixInstallFile $binbuild_dir/mpefc $DESTDIR$bindir/mpefc $XMODE + FixInstallFile $binbuild_dir/mpefc $bindir/mpefc $XMODE fi fi fi @@ -457,7 +457,7 @@ CopyDirRecurP $etcbuild_dir $sysconfdir $MODE cd $etcbuild_dir && \ for file in *.conf ; do \ - FixInstallFile $file $DESTDIR$sysconfdir/$file ; \ + FixInstallFile $file $sysconfdir/$file ; \ done fi fi diff -urN mvapich2-0.9.8.orig/src/pmi/simple/simple_pmi.c mvapich2-0.9.8/src/pmi/simple/simple_pmi.c --- mvapich2-0.9.8.orig/src/pmi/simple/simple_pmi.c 2006-04-09 11:57:00.000000000 -0600 +++ mvapich2-0.9.8/src/pmi/simple/simple_pmi.c 2007-08-31 10:02:26.000000000 -0600 @@ -566,7 +566,7 @@ PMIU_getval( "rc", buf, PMIU_MAXLINE ); rc = atoi( buf ); if ( rc == 0 ) { - PMIU_getval( "value", value, PMI_vallen_max ); + PMIU_getval( "value", value, length ); return( 0 ); } else { From swelch at systemfabricworks.com Fri Aug 31 09:30:51 2007 From: swelch at systemfabricworks.com (swelch at systemfabricworks.com) Date: Fri, 31 Aug 2007 11:30:51 -0500 Subject: [ofa-general] [PATCH] userspace/libibverbs: Set ibv_device->node_type at creation Message-ID: <46D8423B.mailON11US1NX@systemfabricworks.com> Hi Roland, This patch initializes the ibv_device->node_type at creation to provide easy access by library code where node type differences are significant; specifically at context allocation. Currently this value is not initialized, this update saves the value where the current code now determines the node type. Although not limited to the mthca_router mode support, this patch is required by the upcoming libmthca router mode support patch. Signed-off-by: Steve Welch --- src/init.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/init.c b/src/init.c index a8bebd2..07ab855 100644 --- a/src/init.c +++ b/src/init.c @@ -333,7 +333,6 @@ static struct ibv_device *try_driver(struct ibv_driver *driver, { struct ibv_device *dev; char value[8]; - enum ibv_node_type node_type; dev = driver->init_func(sysfs_dev->sysfs_path, sysfs_dev->abi_ver); if (!dev) @@ -342,14 +341,14 @@ static struct ibv_device *try_driver(struct ibv_driver *driver, if (ibv_read_sysfs_file(sysfs_dev->ibdev_path, "node_type", value, sizeof value) < 0) { fprintf(stderr, PFX "Warning: no node_type attr under %s.\n", sysfs_dev->ibdev_path); - node_type = IBV_NODE_UNKNOWN; + dev->node_type = IBV_NODE_UNKNOWN; } else { - node_type = strtol(value, NULL, 10); - if (node_type < IBV_NODE_CA || node_type > IBV_NODE_RNIC) - node_type = IBV_NODE_UNKNOWN; + dev->node_type = strtol(value, NULL, 10); + if (dev->node_type < IBV_NODE_CA || dev->node_type > IBV_NODE_RNIC) + dev->node_type = IBV_NODE_UNKNOWN; } - switch (node_type) { + switch (dev->node_type) { case IBV_NODE_CA: case IBV_NODE_SWITCH: case IBV_NODE_ROUTER: From jgunthorpe at obsidianresearch.com Fri Aug 31 10:09:32 2007 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Fri, 31 Aug 2007 11:09:32 -0600 Subject: [ofa-general] [PATCH] infiniband/hw/mthca: Add optional router mode initialization In-Reply-To: <46D82B52.mailO2X1DVFVD@systemfabricworks.com> References: <46D82B52.mailO2X1DVFVD@systemfabricworks.com> Message-ID: <20070831170932.GE4472@obsidianresearch.com> On Fri, Aug 31, 2007 at 09:53:06AM -0500, swelch at systemfabricworks.com wrote: > This patch allows for the kernel mthca driver to optionally initialize the > mthca devices in router mode. Router mode is enabled at module load with > the setting of the module parm "router_mode=1". This setting > acts on the What MT firmware versions/devices is this compatible with? Thanks, Jason From rdreier at cisco.com Fri Aug 31 13:55:39 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 31 Aug 2007 13:55:39 -0700 Subject: [ofa-general] Re: [PATCH] libibverbs: increment comp_events_completed only if channel is set In-Reply-To: <200709050144.36141.hnguyen@linux.vnet.ibm.com> (Hoang-Nam Nguyen's message of "Wed, 5 Sep 2007 01:44:35 +0200") References: <200709050144.36141.hnguyen@linux.vnet.ibm.com> Message-ID: > increment counter comp_events_completed only if channel is set > this will prevent the while loop below in ibv_cmd_destroy_cq() to > hang if consumer calls ibv_ack_cq_events() without any assigned > channel Is there any valid reason a consumer would ack events if no channel is set? There's no way to get an event to ack in that case, as far as I know. Maybe the real bug is in: while (cq->comp_events_completed != resp.comp_events_reported || cq->async_events_completed != resp.async_events_reported) it might be safer to change the != to a < (actually to a subtraction to handle wraparound), just to avoid hanging if someone accidentally acks too many events. From rdreier at cisco.com Fri Aug 31 13:59:03 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 31 Aug 2007 13:59:03 -0700 Subject: [ofa-general] Re: [PATCH 2/2] IB/ehca: SRQ fixes to enable IPoIB CM In-Reply-To: <200708311603.38822.fenkes@de.ibm.com> (Joachim Fenkes's message of "Fri, 31 Aug 2007 16:03:37 +0200") References: <200708311601.42220.fenkes@de.ibm.com> <200708311603.38822.fenkes@de.ibm.com> Message-ID: thanks, applied 1 and 2. From rdreier at cisco.com Fri Aug 31 14:00:58 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 31 Aug 2007 14:00:58 -0700 Subject: [ofa-general] [GIT PULL] please pull infiniband.git for-linus branch Message-ID: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get fixes for two fairly bad ehca bugs introduced in 2.6.23: Joachim Fenkes (1): IB/ehca: SRQ fixes to enable IPoIB CM Stefan Roscher (1): IB/ehca: Fix Small QP regressions drivers/infiniband/hw/ehca/ehca_hca.c | 10 ++++-- drivers/infiniband/hw/ehca/ehca_irq.c | 48 ++++++++++++++++++++----------- drivers/infiniband/hw/ehca/ehca_qp.c | 10 ++++-- drivers/infiniband/hw/ehca/ipz_pt_fn.c | 2 +- 4 files changed, 45 insertions(+), 25 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c index fc19ef9..cf22472 100644 --- a/drivers/infiniband/hw/ehca/ehca_hca.c +++ b/drivers/infiniband/hw/ehca/ehca_hca.c @@ -93,9 +93,13 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props) props->max_pd = min_t(int, rblock->max_pd, INT_MAX); props->max_ah = min_t(int, rblock->max_ah, INT_MAX); props->max_fmr = min_t(int, rblock->max_mr, INT_MAX); - props->max_srq = 0; - props->max_srq_wr = 0; - props->max_srq_sge = 0; + + if (EHCA_BMASK_GET(HCA_CAP_SRQ, shca->hca_cap)) { + props->max_srq = props->max_qp; + props->max_srq_wr = props->max_qp_wr; + props->max_srq_sge = 3; + } + props->max_pkeys = 16; props->local_ca_ack_delay = rblock->local_ca_ack_delay; diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c index ee06d8b..a925ea5 100644 --- a/drivers/infiniband/hw/ehca/ehca_irq.c +++ b/drivers/infiniband/hw/ehca/ehca_irq.c @@ -175,41 +175,55 @@ error_data1: } -static void qp_event_callback(struct ehca_shca *shca, u64 eqe, - enum ib_event_type event_type, int fatal) +static void dispatch_qp_event(struct ehca_shca *shca, struct ehca_qp *qp, + enum ib_event_type event_type) { struct ib_event event; - struct ehca_qp *qp; - u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); - - read_lock(&ehca_qp_idr_lock); - qp = idr_find(&ehca_qp_idr, token); - read_unlock(&ehca_qp_idr_lock); - - - if (!qp) - return; - - if (fatal) - ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); event.device = &shca->ib_device; + event.event = event_type; if (qp->ext_type == EQPT_SRQ) { if (!qp->ib_srq.event_handler) return; - event.event = fatal ? IB_EVENT_SRQ_ERR : event_type; event.element.srq = &qp->ib_srq; qp->ib_srq.event_handler(&event, qp->ib_srq.srq_context); } else { if (!qp->ib_qp.event_handler) return; - event.event = event_type; event.element.qp = &qp->ib_qp; qp->ib_qp.event_handler(&event, qp->ib_qp.qp_context); } +} + +static void qp_event_callback(struct ehca_shca *shca, u64 eqe, + enum ib_event_type event_type, int fatal) +{ + struct ehca_qp *qp; + u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe); + + read_lock(&ehca_qp_idr_lock); + qp = idr_find(&ehca_qp_idr, token); + read_unlock(&ehca_qp_idr_lock); + + if (!qp) + return; + + if (fatal) + ehca_error_data(shca, qp, qp->ipz_qp_handle.handle); + + dispatch_qp_event(shca, qp, fatal && qp->ext_type == EQPT_SRQ ? + IB_EVENT_SRQ_ERR : event_type); + + /* + * eHCA only processes one WQE at a time for SRQ base QPs, + * so the last WQE has been processed as soon as the QP enters + * error state. + */ + if (fatal && qp->ext_type == EQPT_SRQBASE) + dispatch_qp_event(shca, qp, IB_EVENT_QP_LAST_WQE_REACHED); return; } diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index b178cba..84d435a 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c @@ -600,10 +600,12 @@ static struct ehca_qp *internal_create_qp( if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap) && !(context && udata)) { /* no small QP support in userspace ATM */ - ehca_determine_small_queue( - &parms.squeue, max_send_sge, is_llqp); - ehca_determine_small_queue( - &parms.rqueue, max_recv_sge, is_llqp); + if (HAS_SQ(my_qp)) + ehca_determine_small_queue( + &parms.squeue, max_send_sge, is_llqp); + if (HAS_RQ(my_qp)) + ehca_determine_small_queue( + &parms.rqueue, max_recv_sge, is_llqp); parms.qp_storage = (parms.squeue.is_small || parms.rqueue.is_small); } diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.c b/drivers/infiniband/hw/ehca/ipz_pt_fn.c index a090c67..29bd476 100644 --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c @@ -172,7 +172,7 @@ static void free_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd) unsigned long bit; int free_page = 0; - bit = ((unsigned long)queue->queue_pages[0] & PAGE_MASK) + bit = ((unsigned long)queue->queue_pages[0] & ~PAGE_MASK) >> (order + 9); mutex_lock(&pd->lock); From rdreier at cisco.com Fri Aug 31 14:20:56 2007 From: rdreier at cisco.com (Roland Dreier) Date: Fri, 31 Aug 2007 14:20:56 -0700 Subject: [ofa-general] ib_umad method mask problems on big-endian 64-bit archs In-Reply-To: <20070830143547.GM7140@sashak.voltaire.com> (Sasha Khapyorsky's message of "Thu, 30 Aug 2007 17:35:47 +0300") References: <20070822190519.GD1397@sashak.voltaire.com> <20070830143547.GM7140@sashak.voltaire.com> Message-ID: > Do we have another another user_umad users (OFA or another known, where > switch could be painful)? If not, I will prefer this way instead of > keeping two OpenSMs for ppc64. I agree. After thinking it over, the least painful thing to do to update "documentation" (ib_user_mad.h) to match what the kernel does now for 64-bit big endian systems, and fix the case of 32-bit big endian userspace on a 64-bit kernel. I'll post a patch for 2.6.24. From kliteyn at dev.mellanox.co.il Fri Aug 31 14:47:33 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sat, 01 Sep 2007 00:47:33 +0300 Subject: [ofa-general] [PATCH 5/7 V3] osm: QoS - adding QoS policy options In-Reply-To: <20070831124633.GH11549@sashak.voltaire.com> References: <20070829230120.GH7140@sashak.voltaire.com> <20070829234014.GI7140@sashak.voltaire.com> <46D61B1E.50501@dev.mellanox.co.il> <20070830065908.GK7140@sashak.voltaire.com> <20070831105435.GB11549@sashak.voltaire.com> <20070831120450.GE11549@sashak.voltaire.com> <20070831124633.GH11549@sashak.voltaire.com> Message-ID: <46D88C75.7060605@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 08:32 Fri 31 Aug , Hal Rosenstock wrote: >> On 8/31/07, Sasha Khapyorsky wrote: >>> On 07:21 Fri 31 Aug , Hal Rosenstock wrote: >>>> On 8/31/07, Sasha Khapyorsky wrote: >>>>> On 06:58 Thu 30 Aug , Hal Rosenstock wrote: >>>>>>>> And when the setup part of the new QoS will be ready, would we still want >>>>>>>> the SL2VL and VLArb tables to appear in the opts file? >>>>>>> Don't think. Why we need it in two places? >>>>>> Are you saying that the existing QoS will get these tables from the >>>>>> new QoS syntax (and the existing syntax eliminated) ? >>>>> Yes. I think it is better than to keep two potentially conflicting >>>>> configurations. >>>>> >>>>>> If so, what >>>>>> about backward compatibility ? Also, how straightforward is it to >>>>>> specify the equivalent as the existing syntax ? Will a conversion tool >>>>>> be supplied ? >>>>> Not sure it is needed - it is just few lines of text now. >>>> The new equivalent syntax ? >>> The new syntax is not equivalent, but similar in part of vlarb and sl2vl >>> configuration, it is much more powerful and requires some extra lines. >>> In order to have equivalent configurations vlarb and sl2vl should be >>> specified by node/port type. >> Is there/will there be such a syntax ? > > There is, as far as I understand it. Yevgeny? Right, There are five QoS sections in opensm.opts file: 1. QoS default options 2. QoS CA options 3. QoS Switch Port 0 options 4. QoS Switch external ports options 5. QoS Router ports options First, second, and fifth option are easily defined as port group that has ALL,CA, and ROUTER nodes respectively. Second and third are group of all SWITCHes, but it will require few extra lines in the setup section, e.g., for third option sl2vl tables should be configured "from: * to: 0" ports. -- Yevgeny > Sasha > From kliteyn at dev.mellanox.co.il Fri Aug 31 14:55:27 2007 From: kliteyn at dev.mellanox.co.il (Yevgeny Kliteynik) Date: Sat, 01 Sep 2007 00:55:27 +0300 Subject: [ofa-general] Re: [PATCH] osm: QoS parser - fixing yacc command In-Reply-To: <20070831121013.GF11549@sashak.voltaire.com> References: <46D6E4C3.80201@dev.mellanox.co.il> <20070831121013.GF11549@sashak.voltaire.com> Message-ID: <46D88E4F.4020603@dev.mellanox.co.il> Sasha Khapyorsky wrote: > On 18:39 Thu 30 Aug , Yevgeny Kliteynik wrote: >> Fixing bison command to more general yacc syntax >> >> Signed-off-by: Yevgeny Kliteynik > > Applied. Thanks. > >> --- >> opensm/opensm/Makefile.am | 3 ++- >> 1 files changed, 2 insertions(+), 1 deletions(-) >> >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am >> index abfa913..4ab7227 100644 >> --- a/opensm/opensm/Makefile.am >> +++ b/opensm/opensm/Makefile.am >> @@ -60,7 +60,8 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ >> osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c >> >> osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h >> - $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ >> + $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm_qos_parser.y >> + mv -f y.tab.h $(srcdir)/../include/opensm/osm_qos_parser_y.h > > BTW if osm_qos_parser_y.h file is generated one and used only by > generated *.c files is not it would be simpler just to lease it in > current directory? Perhaps, but then it would be the only header file in opensm source that's not located under the include/ directory. Is it the only header that isn't included by other headers in osm? -- Yevgeny > Sasha > From weiny2 at llnl.gov Fri Aug 31 15:09:23 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 31 Aug 2007 15:09:23 -0700 Subject: [ofa-general] [PATCH] opensm/opensm/Makefile.am : add osm_event_plugin.h to the installed Message-ID: <20070831150923.6b0660b3.weiny2@llnl.gov> >From 12f3c744a609916ce5c52885db63558773223062 Mon Sep 17 00:00:00 2001 From: Ira K. Weiny Date: Thu, 30 Aug 2007 14:32:29 -0700 Subject: [PATCH] opensm/opensm/Makefile.am : add osm_event_plugin.h to the installed headers to be able to compile external plugins Signed-off-by: Ira K. Weiny --- opensm/opensm/Makefile.am | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 4ab7227..e63012c 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -98,7 +98,8 @@ opensminclude_HEADERS = $(srcdir)/../include/opensm/osm_base.h \ $(srcdir)/../include/opensm/osm_madw.h \ $(srcdir)/../include/opensm/osm_mad_pool.h \ $(srcdir)/../include/opensm/osm_msgdef.h \ - $(srcdir)/../include/opensm/osm_helper.h + $(srcdir)/../include/opensm/osm_helper.h \ + $(srcdir)/../include/opensm/osm_event_plugin.h # headers are distributed as part of the include dir EXTRA_DIST = $(srcdir)/libopensm.map $(srcdir)/libopensm.ver \ -- 1.5.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-opensm-opensm-Makefile.am-add-osm_event_plugin.h-t.patch Type: application/octet-stream Size: 1078 bytes Desc: not available URL: From weiny2 at llnl.gov Fri Aug 31 15:55:29 2007 From: weiny2 at llnl.gov (Ira Weiny) Date: Fri, 31 Aug 2007 15:55:29 -0700 Subject: [ofa-general] Re: [PATCH] osm: QoS parser - fixing yacc command In-Reply-To: <46D88E4F.4020603@dev.mellanox.co.il> References: <46D6E4C3.80201@dev.mellanox.co.il> <20070831121013.GF11549@sashak.voltaire.com> <46D88E4F.4020603@dev.mellanox.co.il> Message-ID: <20070831155529.2bf8d902.weiny2@llnl.gov> We just ran into a problem with this patch applied. It seems that the output file is not y.tab.h but osm_qos_parser_y.h so should the move be: mv -f osm_qos_parser_y.h $(srcdir)/../include/opensm/osm_qos_parser_y.h ?? weiny2 at woprjr0:~/OpenIB/git-trees/management/opensm/opensm 15:51:42 > ls *.h ls: *.h: No such file or directory weiny2 at woprjr0:~/OpenIB/git-trees/management/opensm/opensm 15:52:02 > bison -d -o ./osm_qos_parser_y.c -p__qos_parser_ ./osm_qos_parser.y weiny2 at woprjr0:~/OpenIB/git-trees/management/opensm/opensm 15:52:21 > ls *.h osm_qos_parser_y.h weiny2 at woprjr0:~/OpenIB/git-trees/management/opensm/opensm 15:53:17 > bison --version bison (GNU Bison) 1.875c Written by Robert Corbett and Richard Stallman. Copyright (C) 2003 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. On Sat, 01 Sep 2007 00:55:27 +0300 Yevgeny Kliteynik wrote: > Sasha Khapyorsky wrote: > > On 18:39 Thu 30 Aug , Yevgeny Kliteynik wrote: > >> Fixing bison command to more general yacc syntax > >> > >> Signed-off-by: Yevgeny Kliteynik > > > > Applied. Thanks. > > > >> --- > >> opensm/opensm/Makefile.am | 3 ++- > >> 1 files changed, 2 insertions(+), 1 deletions(-) > >> > >> diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am > >> index abfa913..4ab7227 100644 > >> --- a/opensm/opensm/Makefile.am > >> +++ b/opensm/opensm/Makefile.am > >> @@ -60,7 +60,8 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ > >> osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c > >> > >> osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h > >> - $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o $(srcdir)/osm_qos_parser_y.c --defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h --name-prefix=__qos_parser_ > >> + $(YACC) -d -o $(srcdir)/osm_qos_parser_y.c -p__qos_parser_ $(srcdir)/osm_qos_parser.y > >> + mv -f y.tab.h $(srcdir)/../include/opensm/osm_qos_parser_y.h > > > > BTW if osm_qos_parser_y.h file is generated one and used only by > > generated *.c files is not it would be simpler just to lease it in > > current directory? > > Perhaps, but then it would be the only header file in opensm > source that's not located under the include/ directory. > Is it the only header that isn't included by other headers in osm? > > -- Yevgeny > > > Sasha > > > > _______________________________________________ > general mailing list > general at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general From swelch at systemfabricworks.com Fri Aug 31 17:07:27 2007 From: swelch at systemfabricworks.com (swelch at systemfabricworks.com) Date: Fri, 31 Aug 2007 19:07:27 -0500 Subject: [ofa-general] [PATCH] userspace/libmthca: Support forwarding of packets in router mode Message-ID: <46D8AD3F.mail1PR18GPP5@systemfabricworks.com> This patch enables the forwarding of packets (all headers and payload) from user space when the mthca device is operating in mthca router mode. Since any given device either operates in router mode or not, the library determines the mode at context allocation and overrides the default post send routines when in router mode. The decision was made to override the routines since the majority of the users will never operate in router mode and should not incur the overhead of mode checking on each post. Selection of the routines is dependent on the previous patch to the libibverbs that initializes the existing ibv_device->node_type. The post routines expect the sg list to point to the packet's LRH and end at the ICRC. These routines could be ported to the mthca kernel driver as well if a kernel module consumer is desireble; it didn't seem like a good fit at this time. Signed-off-by: Steve Welch --- src/mthca.c | 10 ++- src/mthca.h | 4 + src/qp.c | 393 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/wqe.h | 20 +++ 4 files changed, 425 insertions(+), 2 deletions(-) diff --git a/src/mthca.c b/src/mthca.c index 0f7e953..6acd78e 100644 --- a/src/mthca.c +++ b/src/mthca.c @@ -187,13 +187,19 @@ static struct ibv_context *mthca_alloc_context(struct ibv_device *ibdev, int cmd if (mthca_is_memfree(&context->ibv_ctx)) { context->ibv_ctx.ops.req_notify_cq = mthca_arbel_arm_cq; context->ibv_ctx.ops.cq_event = mthca_arbel_cq_event; - context->ibv_ctx.ops.post_send = mthca_arbel_post_send; + context->ibv_ctx.ops.post_send = + ibdev->node_type == IBV_NODE_ROUTER ? + mthca_arbel_router_post_send : + mthca_arbel_post_send; context->ibv_ctx.ops.post_recv = mthca_arbel_post_recv; context->ibv_ctx.ops.post_srq_recv = mthca_arbel_post_srq_recv; } else { context->ibv_ctx.ops.req_notify_cq = mthca_tavor_arm_cq; context->ibv_ctx.ops.cq_event = NULL; - context->ibv_ctx.ops.post_send = mthca_tavor_post_send; + context->ibv_ctx.ops.post_send = + ibdev->node_type == IBV_NODE_ROUTER ? + mthca_tavor_router_post_send : + mthca_tavor_post_send; context->ibv_ctx.ops.post_recv = mthca_tavor_post_recv; context->ibv_ctx.ops.post_srq_recv = mthca_tavor_post_srq_recv; } diff --git a/src/mthca.h b/src/mthca.h index 1f31bc3..eb264d1 100644 --- a/src/mthca.h +++ b/src/mthca.h @@ -354,10 +354,14 @@ int mthca_destroy_qp(struct ibv_qp *qp); void mthca_init_qp_indices(struct mthca_qp *qp); int mthca_tavor_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, struct ibv_send_wr **bad_wr); +int mthca_tavor_router_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, + struct ibv_send_wr **bad_wr); int mthca_tavor_post_recv(struct ibv_qp *ibqp, struct ibv_recv_wr *wr, struct ibv_recv_wr **bad_wr); int mthca_arbel_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, struct ibv_send_wr **bad_wr); +int mthca_arbel_router_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, + struct ibv_send_wr **bad_wr); int mthca_arbel_post_recv(struct ibv_qp *ibqp, struct ibv_recv_wr *wr, struct ibv_recv_wr **bad_wr); int mthca_alloc_qp_buf(struct ibv_pd *pd, struct ibv_qp_cap *cap, diff --git a/src/qp.c b/src/qp.c index 2ea9dc0..1617c08 100644 --- a/src/qp.c +++ b/src/qp.c @@ -331,6 +331,184 @@ out: return ret; } +int mthca_tavor_router_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, + struct ibv_send_wr **bad_wr) +{ + struct mthca_qp *qp = to_mqp(ibqp); + void *wqe, *prev_wqe; + int ind; + int nreq; + int ret = 0; + int size; + int size0 = 0; + int i; + struct mthca_mlx_seg *mlx; + struct mthca_lrh *lrh; + + /* + * f0 and op0 cannot be used unless nreq > 0, which means this + * function makes it through the loop at least once. So the + * code inside the if (!size0) will be executed, and f0 and + * op0 will be initialized. So any gcc warning about "may be + * used unitialized" is bogus. + */ + uint32_t f0; + uint32_t op0; + + pthread_spin_lock(&qp->sq.lock); + + ind = qp->sq.next_ind; + + for (nreq = 0; wr; ++nreq, wr = wr->next) { + if (wq_overflow(&qp->sq, nreq, to_mcq(qp->ibv_qp.send_cq))) { + ret = -1; + *bad_wr = wr; + goto out; + } + + wqe = get_send_wqe(qp, ind); + prev_wqe = qp->sq.last; + qp->sq.last = wqe; + + ((struct mthca_next_seg *) wqe)->nda_op = 0; + ((struct mthca_next_seg *) wqe)->ee_nds = 0; + ((struct mthca_next_seg *) wqe)->flags = + ((wr->send_flags & IBV_SEND_SIGNALED) ? + htonl(MTHCA_NEXT_CQ_UPDATE) : 0) | + ((wr->send_flags & IBV_SEND_SOLICITED) ? + htonl(MTHCA_NEXT_SOLICIT) : 0) | + htonl(1); + + if (wr->opcode == IBV_WR_SEND_WITH_IMM || + wr->opcode == IBV_WR_RDMA_WRITE_WITH_IMM) + ((struct mthca_next_seg *) wqe)->imm = wr->imm_data; + + wqe += sizeof (struct mthca_next_seg); + size = sizeof (struct mthca_next_seg) / 16; + + /* + * In router mode all QP appear to the user as UD, the actual + * implementation uses the MTHCA specific MLX transport. + */ + if (ibqp->qp_type != IBV_QPT_UD) { + ret = -1; + *bad_wr = wr; + goto out; + } + + /* + * First address should point to a contiguous LRH, last 4 + * bytes should point to the existing ICRC. + */ + mlx = wqe - sizeof(struct mthca_next_seg); + lrh = (struct mthca_lrh *)wr->sg_list[0].addr; + + /* CQE, do not calculate ICRC, use packet SLID */ + mlx->flags = htonl((1 << 16) | (3 << 4) | 1 << 3); + + if (MTHCA_LRH_GET_VL(lrh)==15) + mlx->flags |= htonl( 1 << 17 ); + + mlx->flags |= htonl(MTHCA_LRH_GET_SL(lrh) << 8); + mlx->flags2 = htonl((MTHCA_LRH_GET_DLID(lrh)) << 16); + + if (wr->num_sge > qp->sq.max_gs) { + ret = -1; + *bad_wr = wr; + goto out; + } + + if (wr->send_flags & IBV_SEND_INLINE) { + if (wr->num_sge) { + struct mthca_inline_seg *seg = wqe; + int s = 0; + + wqe += sizeof *seg; + for (i = 0; i < wr->num_sge; ++i) { + struct ibv_sge *sge = &wr->sg_list[i]; + + s += sge->length; + + if (s > qp->max_inline_data) { + ret = -1; + *bad_wr = wr; + goto out; + } + + memcpy(wqe,(void *)(intptr_t)sge->addr, + sge->length); + wqe += sge->length; + } + + seg->byte_count = htonl(MTHCA_INLINE_SEG | s); + size += align(s + sizeof *seg, 16) / 16; + } + } else { + struct mthca_data_seg *seg; + + for (i = 0; i < wr->num_sge; ++i) { + seg = wqe; + seg->byte_count = htonl(wr->sg_list[i].length); + seg->lkey = htonl(wr->sg_list[i].lkey); + seg->addr = htonll(wr->sg_list[i].addr); + wqe += sizeof *seg; + } + + size += wr->num_sge * (sizeof *seg / 16); + } + + qp->wrid[ind + qp->rq.max] = wr->wr_id; + + if (wr->opcode >= sizeof mthca_opcode / + sizeof mthca_opcode[0]) { + ret = -1; + *bad_wr = wr; + goto out; + } + + ((struct mthca_next_seg *) prev_wqe)->nda_op = + htonl(((ind << qp->sq.wqe_shift) + + qp->send_wqe_offset) | + mthca_opcode[wr->opcode]); + /* + * Make sure that nda_op is written before setting ee_nds. + */ + wmb(); + ((struct mthca_next_seg *) prev_wqe)->ee_nds = + htonl((size0 ? 0 : MTHCA_NEXT_DBD) | size | + ((wr->send_flags & IBV_SEND_FENCE) ? + MTHCA_NEXT_FENCE : 0)); + + if (!size0) { + size0 = size; + op0 = mthca_opcode[wr->opcode]; + f0 = wr->send_flags & IBV_SEND_FENCE ? + MTHCA_SEND_DOORBELL_FENCE : 0; + } + + ++ind; + if (ind >= qp->sq.max) + ind -= qp->sq.max; + } + +out: + if (nreq) { + uint32_t doorbell[2]; + + doorbell[0] = htonl(((qp->sq.next_ind << qp->sq.wqe_shift) + + qp->send_wqe_offset) | f0 | op0); + doorbell[1] = htonl((ibqp->qp_num << 8) | size0); + + mthca_write64(doorbell, to_mctx(ibqp->context), MTHCA_SEND_DOORBELL); + } + + qp->sq.next_ind = ind; + qp->sq.head += nreq; + + pthread_spin_unlock(&qp->sq.lock); + return ret; +} + int mthca_tavor_post_recv(struct ibv_qp *ibqp, struct ibv_recv_wr *wr, struct ibv_recv_wr **bad_wr) { @@ -707,6 +885,221 @@ out: return ret; } +int mthca_arbel_router_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, + struct ibv_send_wr **bad_wr) +{ + struct mthca_qp *qp = to_mqp(ibqp); + uint32_t doorbell[2]; + void *wqe, *prev_wqe; + int ind; + int nreq; + int ret = 0; + int size; + int size0 = 0; + int i; + struct mthca_mlx_seg *mlx; + struct mthca_lrh *lrh; + + /* + * f0 and op0 cannot be used unless nreq > 0, which means this + * function makes it through the loop at least once. So the + * code inside the if (!size0) will be executed, and f0 and + * op0 will be initialized. So any gcc warning about "may be + * used unitialized" is bogus. + */ + uint32_t f0; + uint32_t op0; + + pthread_spin_lock(&qp->sq.lock); + + /* XXX check that state is OK to post send */ + + ind = qp->sq.head & (qp->sq.max - 1); + + for (nreq = 0; wr; ++nreq, wr = wr->next) { + if (nreq == MTHCA_ARBEL_MAX_WQES_PER_SEND_DB) { + nreq = 0; + + doorbell[0] = + htonl((MTHCA_ARBEL_MAX_WQES_PER_SEND_DB << 24) | + ((qp->sq.head & 0xffff) << 8) | f0 | op0); + doorbell[1] = htonl((ibqp->qp_num << 8) | size0); + + qp->sq.head += MTHCA_ARBEL_MAX_WQES_PER_SEND_DB; + + /* + * Make sure that descriptors are written before + * doorbell record. + */ + wmb(); + *qp->sq.db = htonl(qp->sq.head & 0xffff); + + /* + * Make sure doorbell record is written before we + * write MMIO send doorbell. + */ + wmb(); + mthca_write64(doorbell, to_mctx(ibqp->context), + MTHCA_SEND_DOORBELL); + + size0 = 0; + } + + if (wq_overflow(&qp->sq, nreq, to_mcq(qp->ibv_qp.send_cq))) { + ret = -1; + *bad_wr = wr; + goto out; + } + + wqe = get_send_wqe(qp, ind); + prev_wqe = qp->sq.last; + qp->sq.last = wqe; + + ((struct mthca_next_seg *) wqe)->flags = + ((wr->send_flags & IBV_SEND_SIGNALED) ? + htonl(MTHCA_NEXT_CQ_UPDATE) : 0) | + ((wr->send_flags & IBV_SEND_SOLICITED) ? + htonl(MTHCA_NEXT_SOLICIT) : 0) | + htonl(1); + if (wr->opcode == IBV_WR_SEND_WITH_IMM || + wr->opcode == IBV_WR_RDMA_WRITE_WITH_IMM) + ((struct mthca_next_seg *) wqe)->imm = + wr->imm_data; + wqe += sizeof (struct mthca_next_seg); + size = sizeof (struct mthca_next_seg) / 16; + + /* + * In router mode all QP appear to the user as UD, the actual + * implementation uses the MTHCA specific MLX transport. + */ + if (ibqp->qp_type != IBV_QPT_UD) { + ret = -1; + *bad_wr = wr; + goto out; + } + + /* + * First address should point to a contiguous LRH, last 4 + * bytes should point to the existing ICRC. + */ + mlx = wqe - sizeof(struct mthca_next_seg); + lrh = (struct mthca_lrh *)wr->sg_list[0].addr; + + /* CQE, do not calculate ICRC, SLID from packet */ + mlx->flags = htonl((1 << 16) | (3 << 4) | 1 << 3); + + if (MTHCA_LRH_GET_VL(lrh)==15) + mlx->flags |= htonl( 1 << 17 ); + + mlx->flags |= htonl(MTHCA_LRH_GET_SL(lrh) << 8); /* SL */ + mlx->flags2 = htonl((MTHCA_LRH_GET_DLID(lrh)) << 16); + + if (wr->num_sge > qp->sq.max_gs) { + ret = -1; + *bad_wr = wr; + goto out; + } + + if (wr->send_flags & IBV_SEND_INLINE) { + if (wr->num_sge) { + struct mthca_inline_seg *seg = wqe; + int s = 0; + + wqe += sizeof *seg; + for (i = 0; i < wr->num_sge; ++i) { + struct ibv_sge *sge = &wr->sg_list[i]; + + s += sge->length; + + if (s > qp->max_inline_data) { + ret = -1; + *bad_wr = wr; + goto out; + } + + memcpy(wqe, + (void *) (uintptr_t) sge->addr, + sge->length); + wqe += sge->length; + } + + seg->byte_count = htonl(MTHCA_INLINE_SEG | s); + size += align(s + sizeof *seg, 16) / 16; + } + } else { + struct mthca_data_seg *seg; + + for (i = 0; i < wr->num_sge; ++i) { + seg = wqe; + seg->byte_count = htonl(wr->sg_list[i].length); + seg->lkey = htonl(wr->sg_list[i].lkey); + seg->addr = htonll(wr->sg_list[i].addr); + wqe += sizeof *seg; + } + + size += wr->num_sge * (sizeof *seg / 16); + } + + qp->wrid[ind + qp->rq.max] = wr->wr_id; + + if (wr->opcode >= sizeof mthca_opcode / + sizeof mthca_opcode[0]) { + ret = -1; + *bad_wr = wr; + goto out; + } + + ((struct mthca_next_seg *) prev_wqe)->nda_op = + htonl(((ind << qp->sq.wqe_shift) + + qp->send_wqe_offset) | + mthca_opcode[wr->opcode]); + wmb(); + ((struct mthca_next_seg *) prev_wqe)->ee_nds = + htonl(MTHCA_NEXT_DBD | size | + ((wr->send_flags & IBV_SEND_FENCE) ? + MTHCA_NEXT_FENCE : 0)); + + if (!size0) { + size0 = size; + op0 = mthca_opcode[wr->opcode]; + f0 = wr->send_flags & IBV_SEND_FENCE ? + MTHCA_SEND_DOORBELL_FENCE : 0; + } + + ++ind; + if (ind >= qp->sq.max) + ind -= qp->sq.max; + } + +out: + if (nreq) { + doorbell[0] = htonl((nreq << 24) | + ((qp->sq.head & 0xffff) << 8) | + f0 | op0); + doorbell[1] = htonl((ibqp->qp_num << 8) | size0); + + qp->sq.head += nreq; + + /* + * Make sure that descriptors are written before + * doorbell record. + */ + wmb(); + *qp->sq.db = htonl(qp->sq.head & 0xffff); + + /* + * Make sure doorbell record is written before we + * write MMIO send doorbell. + */ + wmb(); + mthca_write64(doorbell, to_mctx(ibqp->context), + MTHCA_SEND_DOORBELL); + } + + pthread_spin_unlock(&qp->sq.lock); + return ret; +} + int mthca_arbel_post_recv(struct ibv_qp *ibqp, struct ibv_recv_wr *wr, struct ibv_recv_wr **bad_wr) { diff --git a/src/wqe.h b/src/wqe.h index 602f512..4db7226 100644 --- a/src/wqe.h +++ b/src/wqe.h @@ -111,4 +111,24 @@ struct mthca_inline_seg { uint32_t byte_count; }; +struct mthca_mlx_seg { + uint32_t nda_op; + uint32_t nds; + uint32_t flags; /* [17]vl15, [16]slr, [14:12] static rate */ + /* [8] sl, [5:4] icrc, [3] c, [2] e */ + uint32_t flags2; /* [31:16]DLID, [15:0] vcrc */ +}; + +struct mthca_lrh { + uint32_t data[2]; +} __attribute__((packed)); + +#define MTHCA_LRH_GET_VL(x) (htonl((x)->data[0]) >> 28) +#define MTHCA_LRH_GET_LVER(x) ((htonl((x)->data[0]) >> 24) & 0x000f) +#define MTHCA_LRH_GET_SL(x) ((htonl((x)->data[0]) >> 20) & 0x000f) +#define MTHCA_LRH_GET_LNH(x) ((htonl((x)->data[0]) >> 16) & 0x0003) +#define MTHCA_LRH_GET_DLID(x) (htonl((x)->data[0]) & 0x0000FFFF) +#define MTHCA_LRH_GET_PKTLEN(x) ((htonl((x)->data[1]) >> 16) & 0x000007FF) +#define MTHCA_LRH_GET_SLID(x) (htonl((x)->data[1]) & 0x0000FFFF) + #endif /* WQE_H */ From info at vitto.net Fri Aug 31 17:03:14 2007 From: info at vitto.net (Fondazion Di Vittorio, ITALY) Date: Sat, 1 Sep 2007 08:03:14 +0800 (HKT) Subject: [ofa-general] BASIC RESEARCH. Message-ID: <56336.213.185.118.203.1188604994.squirrel@webmail.wildit.net.au> -------------- next part -------------- An HTML attachment was scrubbed... URL: